Task of my package is to move FolderA csv files only to separate folder. However, the business rule is to move only those files which are matching in FolderB csv files. But I have to read the file first and get the distinct on Date Column and StoreId.
Now, I build package using two foreeach loops one for folderA and one for folderB. Now in side the foreach loops I got dataflow where the files got read and using sort I removed all the duplicates and dump into temptableA and temptableB.
My temptables column names are
Source column is actually a strFileName mapped in foreeach loop.
After loading the files, I am using execute sql task query for matching files and non matching files then using foreach loop and file system task I am moving files to separate folder.
Assuming, after completion of dataflow task, each file dumped 2 distinct records with same storeID and different source name as
View Complete Post