.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
Imran Ghani
Sohaib Khalid
Post New Web Links

Multiple Flat Files Connection manager - unexpected behavior when using wildcard

Posted By:      Posted Date: November 29, 2010    Points: 0   Category :Sql Server


I want to import multiple text files into a destination file.  This works fine when I point the connection manager directly to the 3 files that I need to import. However, these 3 files will change and sometimes there may be more or less files. Therefore I planned to use a wildcard (*). However, when I use a wildcard in the connection manager I get unexpected results - some of the records get duplicated in the output.  I have demonstrated this issue in  this short video clip. Apologies for the slight buzz / hum on the audio (you can just turn the audio off). 

Any help or advice would be appreciated. 



View Complete Post

More Related Resource Links

Loading multiple flat files files- MultipleFlatFile Connection Manager vs For each loop approach


Hi Folks

I need to load multiple  files from a folder into a package. Can someone explain the pros and cons of using the "Multiple Flat file connection manager" vs using a" For each loop" type approach (e.g http://bi-polar23.blogspot.com/2007/08/loading-multiple-excel-files-with-ssis.html).


Many thanks 



Data conversion failed --flat file connection manager


Hello All,

Can any one help me in this .

[Flat File Source [16]] Error: Data conversion failed. The data conversion for column "acct" returned status value 2 and status text "The value could not be converted because of a potential loss of data.".

[Flat File Source [16]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "output column "acct" (373)" failed because error code 0xC0209084 occurred, and the error row disposition on "output column "acct" (373)" specifies failure on error. An error occurred on the specified object of the specified component.  There may be error messages posted before this with more information about the failure.

[DTS.Pipeline] Error: SSIS Error Code DTS_E_THREADCANCELLED.  Thread "WorkThread1" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.  There may be error messages posted before this with more information on why the thread was cancelled.


can any one help me in this error :


[Lookup [465]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "component "Lookup" (465)" failed because error code 0xC020901E occurred, and the error row disposit

Flat File connection manager outputcolumnwidth 50 default

We are developing a process to import data from a CSV file that we are designing.  Although right now our CSV file has 23 columns, over time we will gradually be adding more columns (roughly 1 to 2 more columns per month).  Although the process has worked great I have run into a snag.  When we initially set up the connection manager we went in to the "Advanced" section to specifically set the outputcolumnwidth field sizes (they all defaulted to 50).  The problem is that whenever we modify this connection manager to accept the new columns that we add to the CSV file it resets all of the outputcolumnwidth values to 50 no matter how we try it.  Is there some way to make them keep the values that we set?

For example, our first field is "ID" and we initially changed it from "50" (the default) to "15".  The next month when we added two new columns, the "15" got changed back to "50" and we had to set it again.  This happens for all of the already-specified fields.

connection to SQL Server files (*.mdf) require SQL server express 2005 to function properly.


I dont have the SQL EXPRESS installed instead I have SQL Standard Edition.

 I have two SQL Server instances installed.

 1- UserLT (this is sql 2000)
2- UserLT\SQL2005 (this is SQL 2005 named instance)

But when i try to add a database to my VS website project I get the following error:

Connection to SQL Server files (*.mdf) require SQL server express 2005 to function properly. please verify the installation of the component or download from the URL: go.microsoft.com/fwlink/?linkId=4925

I went in Tools>Opetions>DataBase tools>Data Connection>Sql Server Instance Name (blank for default)

and changed the "SQLEXPRESS" to "USERLT\SQL2005".

But I still get the same error message. Any ideas how i can resolve this issue?

Word Automation - Multiple DCOM Errors/Behavior

Before I get started, I already know that Microsoft does not support and highly discourages server-based MS Office automation.  I've read a multitude of posts and smarmy "*wink*, here's a link" replies, so I'm looking for actual help here, not condescending reprimands.

That notwithstanding, here is my problem.  I have developed a simple method that will open a local Word document, modify some header information, and then save the document as a new file.  As expected, it works great as a console/windows app, but as soon as I went to ASP.NET, DCOM jacked my world up when accessing the app remotely from a different domain account (that has access to the ASP.NET app.  For reference, my method is shown below:


1    public static string ModifyAndSaveBuildDocument(string localFilePath, BuildDocumentType docType, string buildNumber, DateTime buildDate, string projectName)
2            {
3                try
4                {
5                    ProjectConfiguration config = (ProjectConfiguration)ConfigurationManager.GetSection("ProjectConfiguration");

Toolbox: Easy File Backup, Exploring Files And Folders Inside Visual Studio, Multiple Monitor Softwa


If the responsibility for creating, managing, and executing routine backups is yours, these tools will make it easier. Also see how you can browse folders and files from inside Visual Studio.

Scott Mitchell

MSDN Magazine May 2009

Going Places: How Connection Manager Connects


Marcus Perryman explains the correct use of Connection Manager when a Windows Mobile application requires network data.

Marcus Perryman

MSDN Magazine December 2008

Upload multiple files in asp.net

The article Upload multiple files in asp.net was added by habdulrauf on Friday, July 09, 2010.

Some times we need to allow user to upload as many files as he/she wants instead of fixed number of files.So here is the procedure to achieve this. Idea is taken from Joe's video.%@ Page Language="C#" AutoEventWireup="true" CodeFile

The underlying connection was closed: An unexpected error occurred on a send.


I am consuming a third party web service wsdl using my .net aps.net page. And I am getting following error message. Cannt get to resolve it. Its working on my machine but not in my development machine. I have installed the certificate also in certificates on that machine. Can somone please help.?

Error -->  The underlying connection was closed: An unexpected error occurred on a send.





Multiple Submit Connection Problem


Hello everyone,

I have an InfoPath form with two Submit Connections. One of them is a regular submit connection to a SharePoint Document Library. The other is a submit connection to the UpdateListItems web Service.

I have one button using rules that submits the forms fields to the Document Library. Some of those fields have data validation rules like entry must be four digits, and field cannot be blank.

The other submit button uses rules to submit an XML template to the UpdateListItems Template. My problem is that even though this second button does not have the first submit connection as an action, the form will still check all of the forms fields to see if they violate the validation rules before it will submit to the Web Service.

How do I tell InfoPath to ignore the validation rules when I only want to submit to the Web Service? I want the user to be able to enter data into the XML template fields and submit to the Web Service, even if some of the rest of the fields are blank. Right now if any of the other fields fail validation it prevents submittal to the web service even if the fields that failed validation are not being submitted to the web service.

Thanks for any advice that can be offered,
Bill Hurt

how to read config details from multiple config files

Hi, My web application contain 3 config files with different names, ex. firstconfig.config, secondconfig.config, thirdconfig.config.  Each config files have same Key Value pairs. Now what i want to do is, based on the condition i want to read specific info from specific config file. What i tried is, in Web.config files  <appSettings file="firstconfig.config"></appSettings>   In this way i get th info from that particular config file.  Now how do i get details from other 2 config files.   Can any one know abt this.   S. Ramkumar  Smiley

System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a rec

Hello Guys,         I am getting following exception while web service (ASMX) is returning the response. <!-- [if gte mso 10]> <mce:style> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.     <!-- [if gte mso 10]> <mce:style>                                                     InnterException (System.IO.Exception): "Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host."       I think either the Asp.net web server is closing the connection or the response is too long.   Following is my simple code.    1)   namespace JFA.IndiePay.BusinessAccess { public class VerTexProcessor { public DataSet ProcessVerTexForBatch(DataSet dsInputRequest) { DataSet dsResponse = new DataSet(); //Call vertex web service here. VertexService.Service texCalculator = new VertexService.Service(); texCalculator.Timeout = 500000;

FtpWebRequest - how to upload two files on FTP server using only one connection?

Hi I am using FtpWebRequest with option KeepAlive = true. When I upload first file I create ftp request object: FtpWebRequest result = (FtpWebRequest)FtpWebRequest.Create(""); I set KeepAlive = true and I do upload file. When I want upload second file I have to create new FtpWebRequest object using new URI FtpWebRequest result = (FtpWebRequest)FtpWebRequest.Create("");   I set KeepAlive = true and I do upload file. The problem is that I have to create new FtpWebRequest so I create new connection to FTP server (I have checked using netstat -b command that my application uses another port number for second file). How can I upload two files (or more) using only on FTP connection/session? Regards  

Excel Connection Manager Problem

I am building a package that has 2 tasks that need to use the same excel connection manager. I use a execute sql task to dynamically build a table from the headers of the excel sheet  ,then I use an excel source component to load the data from the sheet. However when I try to edit the excel source and switch the data access mode to a variable and select the variable(for the path), when I try to close the editor I get this message 'S:\myfolder'.  It is already opened exclusively by another user, or you need permission to view its data. I need this package to access the same folder each time because the files will be placed in that folder. This package also has a branch for loading a text file into sql server. It access the same folder but uses a flatfile connection manager It looks like permissions on the folder are correct

Multiple XML files into SQL SERVER Express 2005

Hello. I am familiar with classic ASP and use this with MS SQL SERVER EXPRESS. I have an SQL table and want to import multiple XML files into this on a daily basis. I currently have 3 files, transferdata.vbs which loops through the XML files. FAQschema.xml which maps XML to the SQL database and test.xml shows the xml in the test file. If I run transferdata.vbs I get the following error "Error opening the data file" line 33 char 3. Microsoft Bulkload for SQL Server". My SQL table is called EnqOrd id (int), Debitor (varchar), PurchaseDate (varchar)     transferdata.vbs set objBL = CreateObject("SQLXMLBulkLoad.SQLXMLBulkLoad") objBL.ConnectionString = "Provider=SQLOLEDB; Data Source=XXXXXX\XXXXX; Initial Catalog=XXXX; User ID=XXXXX; Password=XXXXXX" objBL.ErrorLogFile = "E:\fuelsql\Teccom\error.log" ' Here is the path to your XML files Const path = "E:\fuelsql\Teccom\XML\" Dim Text, Title, oFile Dim fso, oFolder, oFiles, wsh ' Object variables Text = "Folder " Title = "XML Files" Set wsh = WScript.CreateObject("WScript.Shell") ' Create FileSystemObject object to access the file system. Set fso = CreateObject("Scripting.FileSystemObject") ' Get Folder object. Set oFolder = fso.GetFolder(wsh.ExpandEnvironmentStrings(path)) ' Get All Files Set oFiles = oFolder.

Import Multiple XML files into Sql Server

Hi, i have nearly 1000 xml files in one folder of similar format and I have xsd for the same as well. I would like to import all the files in to sql table by either looping through all the files or any other way. Thanks

Initialize snapshot fro alternate location - multiple .bcp files for an article

I have to replicate over 50GB of data over a slow network.  I did not use the option to initialize snapshot from database backup because the replication articles contain row filters.  If I do that, I'll have to run a lot of scripts to remove the data and other unnecessary database objects on the subscriber. Instead, I created a workaround.  On the publisher, I first create the actual push subscription to the target subscriber on the publication.  This subscription is set not to initialize from snapshot.  I then created a second push subscription on the same publication, but the subscriber is a random database on the publisher server.  This second subscription is a dummy subscription set to initialize from snapshot - the purpose is to generate the necessary snapshot files.  I then reinitialized all subscriptions and generate the new snapshot files. On the subscriber, an empty database is created with the same tables as the publisher database.  I created an identical publication on this empty database, and a dummy push subscription on the target subscriber.  The subscription is reinitialized, and the snapshot files on the empty database is created.  These dummy snapshot files are then overwritten with the actual snapshot files created on the publisher, and then I synchronize the the dummy subscriptions with the actual sn
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend