Introduction
Hi, I am Akira, the editor-in-chief of Data Without Code. In our previous tutorial, we explored connecting KNIME to Google Sheets via API to handle cloud data effortlessly. But what if your company still relies heavily on local shared folders?
Imagine this scenario: Every single day, a regional manager drops a new daily sales report (an Excel or CSV file) into a folder. At the end of the month, you have 30 individual files. You need to open every single one, clean the messy text, standardize the dates, and combine them into a master report.
Doing this manually takes hours. Even if you built a KNIME workflow for one file, clicking “run” 30 times is not true automation.
In programming, developers solve this using a “for loop.” As a non-programmer, the word “loop” might sound intimidating, but in this tutorial, I will show you how to loop through multiple files in a folder using KNIME visually and without any code.
The Concept of Looping in KNIME
A loop in KNIME consists of two special nodes: a Loop Start and a Loop End. Everything you put between these two nodes will be repeated over and over again until a specific condition is met.
For our scenario, we want KNIME to look at a folder, take the first file, process it, hold the result, go back for the second file, process it, hold the result, and finally stitch all 30 results together when it finishes.
Step 1: Get the List of Files
Before we can loop, KNIME needs a to-do list.
- Search for the List Files/Folders node and add it to your canvas.
- Double-click to configure it. Click “Browse” and point it to the folder containing your 30 daily reports (e.g.,
C:\Monthly_Sales). - Execute the node. When you view the output, you will see a simple table listing the exact file paths for every single file inside that folder. This is your to-do list.
Step 2: Start the Loop
Now we need to tell KNIME to take that list and process it row by row.
Connect the output of your List Files/Folders node to a Table Row to Variable Loop Start node. This node is pure magic. It takes the first row (the path to Day 1’s file), turns it into a hidden variable, and sends it into the loop. Once the loop finishes, it will grab the second row (Day 2’s file) and repeat.
Step 3: Read and Process the Data (Inside the Loop)
Now, add an Excel Reader (or CSV Reader) node right next to the Loop Start node. We need to tell the Reader node to use the hidden variable instead of a fixed file path.
- Connect the special “red variable port” (the top left corner) of the Loop Start node to the top left corner of the Excel Reader node.
- Double-click the Excel Reader. Look for the small button with a “v=” symbol next to the “File” browse box.
- Click it, check the “Use Variable” box, and select the variable named
Path(which came from your Loop Start node).
Now, add all your data cleaning nodes (like the Missing Value node or String to Date&Time node) directly after the Excel Reader.
Step 4: End the Loop and Combine the Data
Once your data is cleaned, you need to tell KNIME that the cycle is over and it is time to grab the next file.
Search for the Loop End node and connect the output of your last cleaning node into it. When you right-click the Loop End node and select “Execute,” KNIME will rapidly cycle through all 30 files in the background.
The Loop End node acts like a collector. Once all 30 files are processed, it automatically stacks them on top of each other (appends them) into one giant, perfectly clean master table. You can then route this table to a GroupBy node or a CSV Writer.
Conclusion: Your Next Steps
Congratulations! You just built your first programming loop using a visual interface. By mastering the List Files → Loop Start → Process → Loop End pattern, you can automate almost any repetitive desktop task. Whether you have 30 files or 3,000 files, the workflow remains exactly the same.
So, you have read the files, cleaned them, combined them, and generated the final master report. What is the very last step of your job? Probably opening Outlook or Gmail, attaching the file, and sending it to your boss.
What if KNIME could do that for you, too? Are you ready to completely eliminate your morning routine? Join me in our next Automation Hack where I will show you how to automate email sending directly from a KNIME workflow!
