.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
david stephan
Gaurav Pal
Post New Web Links

row count for source and destination in ssis

Posted By:      Posted Date: October 08, 2010    Points: 0   Category :Sql Server


I need an efficient way of counting number of rows coming from source and also going to destination after passing through all the transformations.I know row count transformation is there but as it aggregates and makes the flow very slow with heavey volume, i dont want to use it.

Pls suggest me some good way of counting rows, source can be flat file/excel/access/sql server



View Complete Post

More Related Resource Links

SSIS - XML Source to SQL Server Destination. How to handle new tags in xml file?

Hi, I've created a SSIS package, which reads my XML file using XML Source, using SCD(Slowly Changing Dimension) to identify whether to do an Insert/Update and everything's fine so far. But there are chances that my xml file will have new tags added apart from what are already there. So In that case, i'm not able to figure out whether to alter the table to add new column or something like that. Need your suggestion. Thanks in advance Vijay

SSIS 2005 - XML Source no data loading to sql server database destination

Hello I am trying to load data from xml source to sql server destination, but the task executes with no data loaded to destination. i really appreciate any guidance in working and troubleshooting xml sources. Regards Harris

Data Flows in SSIS - Mapping Multiple source tables to Destination table **Newb question**

Hi I am new to SSIS and had a basic question. I have around 30+ tables in a db that needs to be migrated to a newer schema in the DB. The data flow task seems to be ideally suited for my requirement. My question is do I need to create 30+ different data flow tasks for this which will get executed one after the other or is there a better way to migrate large number of tables. Also how are referential constraints taken care of during such migration. Thanks and Regards, Ganesh Ranganathan
Ganesh Ranganathan
[Please mark the post as answer if it answers your question]

SSIS : Insert, Update, Delete records from destination table by comparing records in source table.


Requirement: Inse

rt, Update, Delete records from destination table by comparing records in source table.

Update: if records exist in both the table compare them, and update value in destination table if value is different.

Insert: if record doesn't exist in destination table, add new record in destination table.

Delete: if record exist in destination table but not in source table, delete record from destination table.

----> I have created an SSIS package that does all this task but in order to perform delete operation, I have to store all records of destination table into a temporary table.
And since it's not the most efficient way, I have created another SSIS package by using Lookup & conditional Splits
method, now it works fine with inserting and updating information but I am not sure on how to handle removing record operation!

I would really appreciate if you could help me with this.

(In order to archive inserting and deleting records, I have referred following thread!
http://sqlblog.com/blogs/andy_leonard/archive/2007/07/09/ssis-design-pattern-incremental-loads.aspx )

SSIS Object model - dynamically identifying the source and destination tables of the data flow/ tran


Hello All,

I would like to identifying the source & destination tables of the data flow or the transformation task.

I need to create a custom component which i can put into existing packages which can do some validation.

Please suggest ways i can accomplish this.




Should I use SSIS or just stored procedure to move data; both source and destination are SQL Server


Please comment the pros and cons of both methods. 


Thanks in advance.

Problem in Date Format While Exporting To Excel Destination From Flat File Source In SSIS



I have a flat file(.csv) which contains data(strings incuding date in yyyy-mm-dd format).This i am using in Flat File Source and want to export those data to an Excel Destination.Before exporting to Excel Destination i have created a Derived Column component where i am using an express to get day,month,year part and creating a date sting in dd/mm/yyyy format then conveting it to DT_DATE datatype and then exporting to Excel Component.But while exporting to Excel this date format is getting changed to mm/dd/yyyy format.

The expression i used in derived column is

(DT_DATE)(SUBSTRING(date,9,2) + "/" + SUBSTRING(date,6,2) + "/" + SUBSTRING(date,1,4))

where date is a column from Flat File Source.

Even I changed the Locale to English(United Kingdom) in Flat File Connection Manager for the above .csv file.But still it didnt work.

I searched a lot in google and failed to get any solution on this.

Can anyone help me in solving this issue ?

XML Source (Data is not loading to Destination ,Package Runs sucessful)

Hi First I run execute SQL Task SELECT TOP(10) [Message],opportunity_Id  FROM SBP FOR XML PATH ('GS_Opportunity'), ROOT('GSOpportunities') That will make 10 different file into single file then i have assigned this output to variable and it will output as XML Now I put Data flow task ,in it I put XML task and in source i have put XML Data From Variable and I select XSD file location,now XML task is showing me different source table and i have selected appropriate destination table but when i run this package it will not transform data from source to destination,Can you suggest me what is the problem i try to put data viewer it is not transforming any data  while insted of if i select direct file location instead of variable it is running good    

SSIS and XML Source

I have xsd schema. I had 2 problems that I needed to fix. 1. guid type I needed to change to string and 2. ErrorCode column that I had in xsd schema gave me duplicate error in SSIS and I renamed it. Now I am executing data flow and it runs sucessfully but 0 rows in the end of it and my XML has data in it. I do not get any error messages just this info. I am also trying to debug but it does not even open grid for debugging. What is it? Why I do not get any data back from it? Warning: 0x80047076 at Data Flow Task, SSIS.Pipeline: The output column "OrderIds" (104923) on output "ShippingOrderDetail" (18903) and component "XML Source" (1) is not subsequently used in the Data Flow task. Removing this unused output column can increase Data Flow task performance. Warning: 0x80047076 at Data Flow Task, SSIS.Pipeline: The output column "Output_Id" (104926) on output "ShippingOrderDetail" (18903) and component "XML Source" (1) is not subsequently used in the Data Flow task. Removing this unused output column can increase Data Flow task performance. Warning: 0x80047076 at Data Flow Task, SSIS.Pipeline: The output column "Copy of Guid" (105060) on output "Data Conversion Output" (53499) and component "Data Conversion" (53497) is not subsequently used in the Data Flow task. Removing this unused

SSIS XML Source with random child ID??

Hi, I am trying to load a XML document using SSIS into relational tables and pulling my hair out.  In it's simpliest form I have: <Batch>   <POSBasket>     <Header>       <SomeDetail/>     </Header>   </POSBasket>   <POSBasket>     <Header>       <SomeDetail/>     </Header>   </POSBasket> </Batch>   Now, when you have a parent child situation, you get a column in the output called xxx_Id, in my case, POSBasket_Id. This allows you to relate your header back to your root and SomeDetail back to header etc etc. Loading all of this in parrallel does not work because there is nothing unique once the load has happened more than once. i.e. You will have repeating POSBasket_Id's as well as Header_Id's. I ended up loading POSBasket into a table with an identity column. Once done, I would load the header details and do a lookup (eek) on the POSBasket table by using POSbasket_Id "where isComplete = 0" (A flag to tell me current import). Great, this works find with 1 item in my document. Add a second (2 POSBaskets) and it randomly works. (???) Add more and it hardley works. The look ups are failing with "no match found".   On using Data Views, I see that each XML source creates different POSBasket_Id's. Some matching, some not. Also seem to randomly match. My question is why? Why would there be t

SSIS And Sybase Source OLEDB connectivity issue with code page for character set

Hi I am using SSIS 2008 to connect the Sybase Server version 15.0.3 64-bit. The default character set id used by this Sybase server is 4 (i.e. roman8). (http://manuals.sybase.com/onlinebooks/group-charc/chg0300e/charsets/@Generic__BookTextView/1706;pt=266) I had installed Sybase Client for Sybase OLEDB Driver (ASEOLEDB) to integrate this in OLEDB Source Editor. As per guidance from one of the past MSDN forum discussion, I tried setting property of AlwaysUseDefaultCodePage property to "FALSE"  (http://social.msdn.microsoft.com/forums/en-US/sqlintegrationservices/thread/f5af9328-e1fb-48d7-a85d-1d08bb7cf0e5/). However, I am still experiencing error message as follows:- ********************************************************* Error at Package2 [Connection manager "xxxxxx.xxxxxxx"]: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available.  Source: "ASEOLEDB"  Hresult: 0x80004005  Description: "[00000] [Native Code: 30061] [ASEOLEDB]Could not load code page for requested charset". An OLE DB record is available.  Source: "ASEOLEDB"  Hresult: 0x80004005  Description: "[1ZZ001] [Native Code: 30016] [ASEOLEDB]Internal Error". Error at Data Flow Task 2 [OLE DB Source [1]]: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMC

Excel Data Source SSIS AcquireConnection failed 0x80004005 unspecified error

Nuances of using Excel data sources with SSIS Your Development Machine Setup Needs Jet Drivers, available along with the default Office install so if you have Excel working fine locally, this is enough. Installing the "Office Data Connectivity Components" (download file name AccessDatabaseEngine.exe) as well as having Office installed corrupts the Jet Driver stack so that SSIS in VS 2008 cannot create the object to read the XLS file. You will see the title error "Unspecified Error" and "80004005" code. Resolution: Uninstall the Jet Engine pack on your development XP machine if you have Office. It will be listed as something like "Microsoft Access Database Engine (2010)". Run Office 200x setup with the Repair option and reboot. Try configuring the task in SSIS and it should be able to read the Sheet names again and columns.  Live Server (32-bit and 64-bit) Deploying your SSIS package to a server without Office 200x means it will not have the Jet Engine drivers to read XLS, XLSX files. You must install the 32-bit Office Data Connectivity Components - Jet Drivers mentioned above found here  (http://www.microsoft.com/downloads/details.aspx?familyid=7554f536-8c28-4598-9b72-ef94e038c891&displaylang=en). There are no 64-bit drivers; therefore regardless of 32 or 64-bit live environments, this step is the same. You shoul

I need to create a script in SSIS which creates a data source that connects to an Access database.

I need to create a script in SSIS which creates a data source that connects to an Access database. The Access database file name needs to be set as a variable as it will change from month to month. I have no idea what I am doing can anybody give me some tips? Mr Shaw

SharePoint List Source and Destination BIDS adapters not working for our sharepoint instance

It was with great anticipation I installed the SharePoint List Source and Destination BIDS adapters (from CodePlex, http://sqlsrvintegrationsrv.codeplex.com/releases/view/17652). Unfortunately, the adapters are not working for our MOSS sharepoint instance.  We're a DOD (department of defense) site, so the level of security may be higher than normal and may be affecting the adapters. The error message was lengthy and I couldn't copy it and paste here.  I did screen shots and put the complete error message at http://www.christianbahnsen.com/sharepoint_source_error_message.pdf If I can get this to work I think it would be very useful.  I'd played around with KPI Lists, but one cannot create an Indicator directly from SQL Server 2008 (cf. my related thread at http://social.msdn.microsoft.com/Forums/en/sharepointbi/thread/bcc95f65-012e-4b8a-8850-b03fb0f16b0f).  I was hoping that these adapters would provide a work-around by populating sharepoint lists that could be the source for KPI Indicators. As always, thanks in advance for any assistance. Christian Bahnsen

Copying a Source DB objects into Destination DB with CDC feature...Please help

Hi,     I have a source database 'DB_Srce'. Now, my work is to create a package.If I run the package for the first time,it loads all the objects (tables/SPs/ and etc)  into the DB_Destn databse and the subsequent runs (may be schedules time),it fetches only the modified records (inset/update/delete) to the destination database. I think it involves CDC feature. I am new to SSIS.Could you please help with the steps,please?

SSIS Ado >NET Destination Identity column inserts

Hi I am trying to build an SSIS package that imports the identity values from one SQL table to another.  I have an ADO .NET destination connection as the import is for SQL Azure although Im currently testing it in SQL 2008.  The connection has RetainSameConnection set to True.  Before the import I have a SQL task that runs SET IDENTITY_INSERT  activityDisplayGroup ON.  This executes but then I get an error message at the data flow task.  [Destination - activityDisplayGroup 1 [1]] Error: An exception has occurred during data insertion, the message returned from the provider is: Explicit value must be specified for identity column in table 'activityDisplayGroup' either when IDENTITY_INSERT is set to ON or when a replication user is inserting into a NOT FOR REPLICATION identity column. In the profiler I see insert bulk "dbo"."activityDisplayGroup" ([name] VarChar(200) COLLATE SQL_Latin1_General_CP1_CI_AS) Is it at all possible to insert identies using the ADO .NET, if so what have I missed?   Thanks Dan      

SSIS 2008 - Excel Source using SQL Connection cancels as soon as package runs.

Let me preface this by saying I'm brand-new to SSIS development. I'm using 64-bit SSIS 2008 and I'm trying to read an Excel spreadsheet using OLE DB. I've already run into and corrected the Run64BitRuntime problem.  Now when I invoke the Package it cancels immediately with the following output: SSIS package "Lesson1X.dtsx" starting. Information: 0x4004300A at Data Flow Task, SSIS.Pipeline: Validation phase is beginning. SSIS package "Lesson1X.dtsx" finished: Canceled. The program '[5400] Lesson1X.dtsx: DTS' has exited with code 0 (0x0). When I edit the Excel Source component itself there are no errors and I am able to Preview the results successfully.  If I use Table or View to access the file, the package runs successfully. Any suggestions will be greatly appreciated. Thanks
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend