.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
Gaurav Pal
Post New Web Links

Data flow - Repeat of lookups in insert and update path

Posted By:      Posted Date: October 06, 2010    Points: 0   Category :Sql Server

Hi forum

I find myself repeating lookups when I have insert and update paths. In my data flows I use a Lookup component to check if the row already exists in the destination table. I insert the row in the No Match output path and update in the Match path.

I often have to do the same for the insert and update paths. I always have to do them in the insert and might have to do them for the updates. In the update path I use conditional splits to check if values have changed - if they have - then I lookup for example a supplier data warehouse key. I could place it before the lookup that check if the row exists but it would be a waste - because often the supplier did not change - some other value did.

So I end up with 2 copies of my lookups I have to maintain :(

Hope someone can help me out.


View Complete Post

More Related Resource Links

Insert/Update data to multiple tables on a single button click


Hi experts,

I need to insert/update data to 3 or more tables on a single button click.What is best method for achieving this?I am using mysql as my DB.

Thanks for your valuable reply in advance.

SQL Update and Insert: Inserting a whole data table into another table



If I create a data tabe in C# like so:

datatable t = new DataTable;

DataColum C1 = new DataColum;//ID column


//the same with column C2

Then I fill up the table with some data.

Then the goal is to either insert this data or update data in an SQLtable in SQL. If the data is inserted, the ID column in t would be empty, if updated, it would not be. So in the insertion case, the sql query needs to make sure that unique IDs are assigned during insertion, in the update case, it just has to look for the right IDs.

How to go about this?

Asking because I am only familiar with updating and inserting single lines into a table. One could loop in C# over the datatable t of course and insert or update line by line, but this would probably be slow. Is there a way to write an SQL query like this somehow:

Insert Into SQLTable Values (@t)


And Update like this:

Update SQLTable SET ... Where C1 = @t.C1

Or something like that.

I know that SQL server can create automatic update and insert queries and that you can then type things in C# like


But, this has proven to be super slow on our server, for unknown reasons. I have asked on this forum and got some answers from experts, but was not able to srot out the problem.

How can we write

How to update a variable in Data Flow component of lookup has rows that don't find matches



I'm using SSIS 2008

I have a data flow that gets data from an Excel spreadsheet. It then does numerous lookups to my database to ensure that all the entities in the spreadsheet are present in my dimension tables. For each of these lookups, I redirect rows with "no matching entries" to be written to another spreadsheet which is then sent to an email address for action.

I have a Package scoped variable that I want to update if any of the lookups have rows with no matching entries. This is used later in the package to determine if it should proceed or not. So this variable only needs to be updated once if any of the lookups fail. 


What are the options for doing this? What is the best way?


Craig Bryden - Please mark correct answers

SSIS - Data Flow Task Reads Fast But Importing the Output,Update & Deleting Extremely Slow & runs lo


Good day, I need help please.

In my source I read about 4mil records in a matter of seconds But Importing the Output to the Same Table takes about an 1 hour to update.

My Source Read is the Same Table to which i'm Importing, Updateing,Deleting

Importing: Table Lock & Check Constraints are checked

Please any ideas & suggestions will be great.

Thank You

SqlCommand.ExecuteNonQuery() returns -1 when doing Insert / Update / Delete

Sometimes you end up with a return value of -1 when using the SqlClient.SqlCommand.ExecuteNonQuery method.

Why is that?

Well, the ExecuteNonQuery method is there for statements for changing data, ie. DELETE / UPDATE /INSERT, and the returned value are the number of rows affected by that statement.

When checking the documentation we can see that there are some conditions that return -1.

For UPDATE, INSERT, and DELETE statements, the return value is the number of rows affected by the command.

When a trigger exists on a table being inserted or updated, the return value includes the number of rows affected by both the insert or update operation and the number of

rows affected by the trigger or triggers. For all other types of statements, the return value is -1. If a rollback occurs, the return value is also -1.

Dynamic Data, Domain datasource, Poco and lookups


Mr. Edward,

Excuse my question but: it is necessary to involve in our projects something more than just Dynamic Data or should we make use of all other features available in VS2010 (which will make assume taht they had been left out of the Dynamic Data framework) like: wef, poco, azure, wcf, cloud, mvc, silverllight, ajax, odata, agile, workflow, etc.?

Sorry my ignorance, I'm just transcribing names as they come to my mind and of which I don't know anything about


Carlos Porras (El Salvador)

Insert data to Excel using OLEDB



I am trying to insert data to Excel Document using OLEDB as follows:-

 sql = "Insert into [MyFirstSheet$] (" + Title1 + "," +Title2 + "," + Title3 + ") 
values('" + Value1 + "','" + Value2 + "','" + Value2 + "')";
myCommand.CommandText = sql;

 sql = "Insert into [MyFirstSheet$] (" + Title1 + "," +Title2 + "," + Title3 + ") 

values('" + Value1 + "','" + Value2 + "','" + Value2 + "')";

myCommand.CommandText = sql;


But, the Title1 ,Title2, Title3 are in the 3rd row of the Excel Document. So, the data is not inserted.

Dynamic WPF: Create Flexible UIs With Flow Documents And Data Binding


Flow documents offer enormous flexibility in arranging text layout and pagination, but they don't support data binding, so you can't dynamically change content. Here we build a component to solve that problem.

Vincent Van Den Berghe

MSDN Magazine April 2009

SQL Server 2005: Fuzzy Lookups and Groupings Provide Powerful Data Cleansing Capabilities


SQL Server 2005 offers a completely redesigned SQL Server Integration Services engine, formerly known as Data Transformation Services. It includes many new features such as built-in support for Fuzzy Lookups and Fuzzy Groupings, which help you build powerful data-cleansing solutions. This article provides an overview of fuzzy searching techniques and a dissection of the underlying fuzzy search technology implemented in SQL Server 2005.

Jay Nathan

MSDN Magazine September 2005

SQL Server CE: New Version Lets You Store and Update Data on Handheld Devices


Handheld device users need to be able to synchronize with a main data store when it's convenient and, preferably, when the back-end database server isn't busy. SQL Server 2000 Windows CE Edition allows you to build a traveling data store that can be displayed and run on a variety of devices. SQL Server CE supports a subset of the full SQL Server package, and can be used as a standalone server or in tandem with SWL Server and IIS. The architecture of SQL Server CE, along with data manipulation, synchronization, and connectivity issues, are discussed in this article. Topics such as making your data public, choosing the right type of replication, and handling errors are also covered.

Paul Yao and David Durant

MSDN Magazine June 2001

insert the data in database using gridview jquery



I want to use gridview and bind the data and after i want insert the data in database using jquery,

below the link i got for delete


but i want the sample example for insert  row from gridivew and add the database.


Insert data based on gridview user value on row.. how to



I have three column comes from database table(Table1) and one column is for entering data for the user let us consider it look like this 

ItemNo ItemName Qty  uservalue

001       A               50       20

002       A               20       5

003       B               50  

004       C               60      10

005       D               40 

006       E               90      15 


If its 300 row from database to gridview it will show 50 record with page navigation..

What i need is i need to insert data in to database table which gridview row has user value..

The output of the another table(Table2) will look like this after insert

ItemNo ItemName   uservalue

001       A                    20

002       A                      5  

review, edit and insert data using 2 diffrent database



is it possible to create a custom list in sharepoint designer to enter a vacation request and using a work flow that goes to the HR department. but when the HR department approve the request the HR can store these requested data into another database?

what i mean is, the employee fill a form in to request some vacation days using a custom list in my wss3 site. i assume that these data will be stored in the WSS_content database. now when the HR department get this filled form they have to store the data of the form in to a database but not in the WSS_Content database.

is this possible?

Insert Excel Data via File Upload into sql server database???

Hi all, I have requirement that User can Upload the Excel Sheet Data to sql server Database at once. How i do that Any Article or hint is apperciated.   Regards 

How do I add two XML tasks to a Data FLow task?

I am new to SSIS and am developing a package which takes input from an XML file and then populates an existing SQL table with these records.  However, this is a very large XML file.  I keep getting an Out of Memory exception.  (This file is 210 MB)!  So what I tried doing was separating this XML file into 8 smaller files.  Now, I can successfully load data from these smaller files one at a time.  But how can I do all of them in the same package?  I connected two XML tasks to the Data Flow task and get the error below.  I also increased my timeout and number of errors to no avail.  Advice? Error: 0xC002F304 at XML Task 1, XML Task: An error occurred with the following error message: "Exception of type 'System.OutOfMemoryException' was thrown.". Task failed: XML Task 1 Warning: 0x80019002 at zSkywardEnrollment: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors. Error: 0xC002F304 at XML Task, XML Task: An error occurred with the following error message: "Exception of type 'System.OutOfMemoryException' was thrown.". Task failed: XML Task SSIS package "

How to detect that a Data Flow Task has failed (from inside a custom componet)

Hi, in the PostExecute Method of a custom component i would like to check if the Data Flow Task has been failed. Is that possible? Also it would be great to be able to check if this custom component already got all data or has been marked as "green".  Thanks, Dennis

Custom SSIS Data Flow Component Not Showing in Toolbox or GAC

Hello - I have created a very simple data flow component for SSIS (Actually, I am following this example:  http://www.microsoft.com/downloads/details.aspx?familyid=1C2A7DD2-3EC3-4641-9407-A5A337BEA7D3&displaylang=en).  However, when I register the DLL to the GAC, I am unable to find the assembly in C:\Windows\Assembly - even though the GACUTIL says "Assembly Registered Successfully".  Furthermore, after copying the DLL to the PipelineComponents folder for SSIS (C:\Program Files (x86)\Microsoft SQL Server\100\DTS\PipelineComponents), it does not show in the "Choose Items . . ." dialog box of SSIS.   I am running SQL Server 2008 Dev edition, Visual Studio 2010 with .NET 4.0, and Windows 7 Enterprise 64-bit edition.  Any assistance/thoughts would be appreciated. Thanks!
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend