.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Register
 
Win Surprise Gifts!!!
Congratulations!!!


Top 5 Contributors of the Month
david stephan
Post New Web Links

Mapping of CSV files to sql table using execute sql task

Posted By:      Posted Date: October 21, 2010    Points: 0   Category :Sql Server
 

I need to exctract data from multiple CSV files and store the data in SQL database table.

structure of data is not in same in CSV files. Number of columns and order of columns is different in CSV files. Same way, structure for all the destination database tables will not be the same. It will be according to the corresponding CSV file.

I am using execute sql task approach. I mean to use a foreach loop to loop through csv files and put an execute sql task inside foreach loop and make a BCP command dynamically to load data from csv file to sql server destination table. As using BCP we don't need to worry about structure of data.

But, I am seeing few issues:

Execute SQL task runs the sql query. Then, how can we create a common query to get the data from csv files.

                                         


View Complete Post


More Related Resource Links

execute update timedout expires , table with trigger due to recovery of database

  

I have table A in database A  , in table A i have trigger to insert record to database B table A, each update and insert of database A -table A, triiger fire and insert record in database B table A, when i try to update some times asp.net application gives error

"Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding."

and then error log says System.Data.SqlClient.SqlException: Database 'database B is being recovered. Waiting until recovery is finished.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean ret

Execute Package Task password issue

  
I have a parent and child package. Both are stored on the file system and both are password protected. My problem is that when I call the child package from the parent using execute package task, I get prompted for the child package's password. Now the PackagePassword property of the Execute Package Task is supposed to take care of that. But for some reason the password stored in the Execute Package Task is not getting passed to the child package.  I've re-entered the PackPassword property several times, but it did not help.  I even tried loading the password from a config file, but that did not make any difference.  Can anyone help?

can't execute "create table" command with Visual Web Developer express

  
I have a folder with a web site I'm developing which is connected to a database.Every time I move the folder to a different computer it's a pain when I have to move the database and change the connection string.I've thought to use for the first time a database in the app_data folder and move all the database inside it.Nevertheless, when I try to recreate all the tables in the new database for the first time with the query "create table ..." I get and error saying that "SQL CREATE TABLE" is not supported.Is there a way to manage this situation?--   Giuseppe

Receive/Delete Multiple FTP Files based on condition using SCRIPT(VB) TASK

  
I am trying to receive and delete multiple FTP files from a remote FTP server using Script task Below is the code   Dim FTPConnMgr As ConnectionManager FTPConnMgr = Dts.Connections("FTP Connection Manager") Dim FTPClientConn As FtpClientConnection = New FtpClientConnection(FTPConnMgr.AcquireConnection(Nothing)) Dim FileTimeStampNew As String = "20100913021807" Dim remoteFileNames(0) As String remoteFileNames(0) = Dts.Variables("FtpRemoteDirectory").Value & "*" & FileTimeStampNew & ".*" 'Below hardcoded FileName works good. But the problem is there are lots of file in the FTP folder that I dont want to receive 'remoteFileNames(0) = Dts.Variables("FtpRemoteDirectory").Value & "Company_alpha_Full_20100913021807.xml" Dim localPath As String = Dts.Variables("FtpLocalDirectory").Value FTPClientConn.Connect() FTPClientConn.ReceiveFiles(remoteFileNames, localPath, False, False) FTPClientConn.Close() Dts.TaskResult = ScriptResults.Success End Sub End Class If i specifically mention the RemotefileNames this works fine but, when I say *.* it executes succefully but doesn't receives the file. Please advice how to receive multiple file based on a variable BR, AWM

Execute Process task - On Error (i want to override the error message)

  
Hi, In my package I am calling one exe to copy the files from local folders to SFTP. I am using a execute process task and I am passing the parameters like (address, username, password,....). The exe and syntax are given below. SFTPCopier.exe <<machinename>> <<username>> <<password> <<local directory path>> The problem is when the exe is failing to copy the files to SFTP due to some reason, the SSIS package is outputting the entire command in the log information (including the username & password). I dont want to write the password in the log information. I want to remove that password from the log information when an error occured in the execute process task. Can some one please tell me the way to override the message? Can I put a on error handler for that execute process task and use a script task to modify the "ErrorDescription" variable? Will that works? Thanks in advance.  

execute sql task vs data flow task

  
My manager has asked me to use linked server to connect to ORACLE database rather then oledb   my source db is ORACLE and my Distination is SQL server 2005 ( in 64 bit  windows server) I am planning to write this query in an execute SQL task, INSERT  dbo.MV_MAX (PERIOD                                 ,PRODUCTGROUPID                                 ,PRODUCTID                                 ,SUBPRODUCTID                                 ,HLBER_ID                                 ,BE_ID                                 ,ENTITYID                             &n

How to get values from a multi row set in case of Execute SQL task?

  
Hi All, I'm testing the Execute SQL task, when I set the ResultSet to Full row set an try to retrieve those set into a variable, the package runs well but I' m not able to extract what is inside the record. If I try to debug the package to explore the data. All that I recieve is _ComObject as type, System._ComObject as value, I know (if I'm not wrong) that this _ComObject is the box that contains the rows though the values the question is how to extract those variables. For example, I can say how to enumerate the results sets using a ForEach container? Thank you The complexity resides in the simplicity

update statement which i can use in execute sql task.

  
I am having a table in the following manner.I am having  yearmonth till 202012 and from 201005 to 202012 I am having activeflag,tablename1,tablename2,tablename3,tablename4 as zero. This table needs to be update before runing the package.Before I load data for tablename1 for 201005 I need to update that 201005 for tablename1 to 1 and activeflag to 1 and  I need to update the 201002 tablename1 to zero.and when I load data for 201006 I need to update first that active flag =1 for 201006 and tablename1 to 1 for 201006 and then Update 201003 tablename1 to zero.i.e in tablename1 at a given time i need to see only 3 (yearmonth)=1 and update the first non zero record to zero.similary i need to update other tables also.  yearmonth                      Activeflag           tablename 1       tablename2         tablename 3   tablename4 201001                             0                 &nb

SSIS FTP Task works, but the files aren't there

  
I've created an FTP Task in my SSIS package, and when I run it in BIDS, it turns green and reports success, but when I look at the destintation folder, the file that should have been sent isn't there. I can send the same file to the same destination using the same credentials using WSFTP and it gets there. How can one begin to debug such a thing?

Retrieving users from mapping/pivot table

  
I have three tables.CustomProfileUserID (pk)DisplayNameCompanyIDTeamsTeamRecordID (Pk)TeamIDCompanyIDEmployeesInTeamsUserIDTeamIDI want to assign employees to teams via a gridview control with a templated checkbox.I have a dropdownlist control to select the TeamID.I am having problems with the select statement.The gridview needs to select employees that are not assigned to any teams as well as assigned to other teams. An employee can be assigned to many teams.What I have been trying is:cp.UserID, cp.DisplayName, et.TeamIntID, cp.CompanyID FROM            EmployeesInTeams AS et CROSS JOIN                          CustomProfile AS cp WHERE        (et.TeamIntID <> @TeamIntID) AND (NOT EXISTS                              (SELECT        TeamIntID, UserID                                FROM            EmployeesInTeams                                WHERE        (cp.UserID = UserID))) AND (cp.CompanyID = @companyid)Select cp.UserID, cp.DisplayName, et.TeamIntID, cp.C

Table mapping

  
 Hi friend, I want to know what is the use of table mappings.At what scenario we will use table mapping. Thanks  

SSIS Execute Process Task Give "Access is Denied" error but it is not related to access control.

  
Hello All - I have a very weired SSIS problem. I am using BIDS in two machines - Windows 7 Professional Edition x64 bit (development workstation) and Windows 2003 Standard Edition x32 bit SP2 (server) . The visual studio version on both the machines are Version 2.0.50727 SP2. I found that my package's ExecuteProcessTask fails with "Access is denied" error in Windows2003 but runs ok in Windows 7 dev box. I then tried to debug the package in the Windows 2003 environment itself. The package failed in debug mode too with the same message. After looking for a while I found that when I shorten the Argument length of the failing ExecuteProcessTask the task did seem to work. So in Windows2003 environment if my argument exceeds 1846 characters the ExecuteProcessTask fails with message "Access is Denied". If the argument string is less than 1846 it works ok. In Windows 7 environment there is no problem at all. It does not seem like access problem but I may be wrong. Has anyone seen this? I will appreciate any reply. Thanks, Niben  

Variables in execute sql task

  
Hi Guys, I'm working with Variables in the SSIS for the first time. I managed to get The Minimum Date (Variable StartDate) and The Maximum Date (Variable EndDate) from the Source Database Table. (Pic below the 1st Execute SQL Task | Get Start Date And End Date Parameters) Then I want to delete records from the Destination Database Table based on the StartDate and EndDate Variable carried over from the 1st Execute SQL Task. I debugged the package on the 2nd Execute SQL Task and I watched the variables carry over the date value that I expected. (please note the AS DateTime inside the Enter SQL Query Dialog Box should not be there I fixed this on my end) Once the debugger step into the 2nd Execute SQL Task everything run fine but the delete doesn't happen on the database! I appreciate any help on this.         Thank you  John

Execute SQL Task with XML resultset not return properly

  
Hello, I have a table with one column as XML data type, and it stored XML content from a XML file. In SSIS, I used execute SQL Task, and select the XML column from the table, and set resultset as XML. I then defined a variable XMLList as string, and I used this XMLList variable as resultset. When I run the package, I saw from watch window that this XMLLIst variable not returned the proper value from the table column, but with something like <ROOT><?MSSQLError HResult=\"0x80004005\" etc. Could anyone give me advice on what is wrong with my settings for resultset? Any suggestion will be appreciate!  

Implementing Transaction in SSIS Package - [Execute SQL Task] Error: Failed to acquire connection "<

  
I have a simple SSIS package with three "Execute SQL Tasks". I am using ADO.Net Connection to execute SPs on a DB server. When I execute this package It works fine. So far so good. Now, I need to implement transation on this package. And problem starts now onwards. When I try to execute package after setting TransationOption = Required for the Sequence container which contains all the tasks, I get following error. [Execute SQL Task] Error: Failed to acquire connection "NYCDB0008.Export". Connection may not be configured correctly or you may not have the right permissions on this connection. "NYCDB0008.Export" is the name of the ADO.Net connection. I have been hunting for any solution but all in vain. I have tried changing all DTC settings on the dev as well as Database server. Please respond if anyone has any solution. Thanks! Anand

SSIS Execute Process Task Problems

  
I have a command to decrypt a file that I can run from the command line and it works beautifully.  However, when I stuff it into an execute process task, it errors out every time or does nothing.   Here is the command I can run from the command line:   gpg -d --passphrase-fd 0 < c:\Dip_Fif.WUZ -o c:\Someoutputfile.in1  -r "KeyName" c:\Someinputfile.in1.pgp     I've pointed the execute process task object to the gpg.exe executable on my system and am stuffing the remainder in the arguments line.  I have also tried changing around all the timeout settings and sucess values.  I have found I can change the success value to 2 and it will show up as being green when complete, but the file doesn't decrypt.  It just in turn will throw an error on the next piece because the required file is not there.   I will probably end up writing a script to get this to work and use a script task but I really want to know why this will not work.   --Thanks--

Send multiple parameters SSIS execute process task

  

Hi,

We are using Execute Process Task in SSIS package to execute a batch file to house keep the generated flat files which are older than a week from the archive folder.

The same batch file is used to delete the some temp files generated in the process.

As the Archive folder and temp folder paths are configurable, we are trying to pass the folder paths as parameters to batch file.

I used Expressions tab of Execute process Task Editor to map the variable to Arguments property. This is working fine. but I need to pass one more variable as an Argument.

Can some one help me how to achieve this. Thanks in advance.

Regards,

Sreekanth ABS

 

 


Categories: 
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend