.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Register
 
Win Surprise Gifts!!!
Congratulations!!!


Top 5 Contributors of the Month
Imran Ghani
Post New Web Links

SSIS - Lookup

Posted By:      Posted Date: August 31, 2010    Points: 0   Category :Sql Server
 
Hi, I am using Lookup component in my SSIS workflow. I am comparing a string. Is there a way to ignore the case while comparing?


View Complete Post


More Related Resource Links

SSIS Lookup Transformation Issue

  
I am having a strange issue, In my data flow i have a Lookup Transformaton which will match on key columns of the fact and is followed by a condetional split that would deside if it should insert (old db destination) or go to update (oledb command) or ignore if no change. I have packages running for the last 1 year with the same logic. But in the recent packages I am experiencing a problem.  example: Key columns for join are - type_no (varchar 16) with all numeric values except one record wihh ' '(space) in it  and type_cd (decimal(18,0)) with values(0,1,2,3,4,5) It worked fine when I test the package. After couple of day running in schedule I get integrity violation and huge file with failed records which are supposed to be blocked at the condetional split as they are already in the fact. When i add a data viewer what i found is for all the llokup columns its having nulls (no match found). Workaround that is working for me for now is - I select full cash and say ok in the lookup transofrmation and again open it and set it back to no cash. Then it starts working as expected. Did anyone come accross this kind of issue? is the some standard that I have to follow to make sure this doesnot happen again  

SSIS Lookup slow

  
Ik have a SSIS package which does a lookup for a WorkID in a employee_work table. The lookup is based on date and employeeID. It then inserts the correct workID in an sick leave fact-table. The lookup table has "only" 20,000 rows, it's indexed. The fact table is about 4,000,000 rows. This is the lookup  query: select TOP(1) * from    (SELECT WerkID, werk_start_KEY, ms120_obj    FROM dbo.DimMedewerker) [refTable] WHERE [refTable].[ms120_obj] = ? and [refTable].[werk_start_KEY] <= ? ORDER BY  [refTable].[werk_start_KEY] DESC But whe I run the package it only processes about 35000 rows, the it takes a time, then it processes the next 35000 rows, etcetc. How can I make it go Faster? At the moment it takes about 1.5 hours

SSIS lookup slow

  
Hello,   Ik have a SSIS package which does a lookup for a WorkID in a employee_work table. The lookup is based on date and employeeID. It then inserts the correct workID in an sick leave fact-table. The lookup table has "only" 20,000 rows, it's indexed. The fact table is about 4,000,000 rows. This is the lookup  query: select TOP(1) * from    (SELECT WerkID, werk_start_KEY, ms120_obj    FROM dbo.DimMedewerker) [refTable] WHERE [refTable].[ms120_obj] = ? and [refTable].[werk_start_KEY] <= ? ORDER BY  [refTable].[werk_start_KEY] DESC But whe I run the package it only processes about 35000 rows, the it takes a time, then it processes the next 35000 rows, etcetc. How can I make it go Faster? At the moment it takes about 1.5 hours

SSIS lookup slow

  
Hello,   Ik have a SSIS package which does a lookup for a WorkID in a employee_work table. The lookup is based on date and employeeID. It then inserts the correct workID in an sick leave fact-table. The lookup table has "only" 20,000 rows, it's indexed. The fact table is about 4,000,000 rows. This is the lookup  query: select TOP(1) * from    (SELECT WerkID, werk_start_KEY, ms120_obj    FROM dbo.DimMedewerker) [refTable] WHERE [refTable].[ms120_obj] = ? and [refTable].[werk_start_KEY] <= ? ORDER BY  [refTable].[werk_start_KEY] DESC But whe I run the package it only processes about 35000 rows, the it takes a time, then it processes the next 35000 rows, etcetc. How can I make it go Faster? At the moment it takes about 1.5 hours

SSIS Dataflow Lookup failure using a Synonym

  

In a dataflow, I am trying to use a Lookup Transformation to join to a Synonym.  The preview button displays data, by clicking the Columns button results in "The Lookup transformation method, ReinitializeMetadata, failed."

 


SSIS 2008 - Fuzzy lookup warming index takes too much time

  

Hello All,

We are using Fuzzy lookup transformation in our SSIS package.

We have selected option to store and maintain index for first time. And for subsequent executions, we use option to use existing index. So it will not create new index each time. We are doing fuzzy lookup on single source table with single column. The source table contains more than 2.5 million rows.

So while executing the package, although its not taking time to build complete index. But still it takes considerable time and uses large memory whenever we execute it.

We have also set the WarmCache option to true. Although all data is loaded into memory and removed after execution.

Is there any other way so we can keep the data in memory, so for subsequent execution, it gets result directly from memory?

Please let me know if you have any suggestion? Its urgent.

 

Thanks in advance.



Darshan Vyas

SSIS - Lookup Percentages values change

  

Good day, i need help please.

The problem i have is that a percentage colume in my table, when i sum the total = '100.0000000'

This fine, but when i insert in SSIS - Lookup & i preview the data in Lookup task the Total = '100.072' which makes my calculations incorrect.

is there something i need to do to keep it in its original sum - but i view my decimal places in advanced properties decmal(38,12) so this should be fine.

Is in the query that i have to perhaps force it to be 100.0000, how?   

Can somebody please direct me on what i should do

Regards

 


SSIS - Lookup Task (Duplicate Row)

  

Good day,

I'm having difficulty with a lookup task.

Firstly in my Lookup my query contains a "DISTINCT" and my cols match source as only some records fail, Source also has a distinct so no Duplicate there either.

I also added on error to display error records to a dataview. Now the lookup code does exist not sure why he is failing

The Lookup transformation found duplicate key values when caching metadata in PreExecute. This error occurs in Full Cache mode only. Either remove the duplicate key values, or change the cache mode to PARTIAL or NO_CACHE.

Is the records locking each other as source & lookup read from same table with diff filter criteria to gether with (NOLOCK) specified.

Please Assist!


Adding exception table to incremental Load with SSIS Lookup task and conditional split

  

Hello,

I have built an incremental load ETL Process to load some flat files with an SSIS Lookup and Conditional Split. I only have one path in the conditional split and that is for New Records.

I have two questions:

1.       For the new records path, I have to check to see if a record exists and I don’t have a single key that is unique, therefore, I have to use a multiple keys to make the record unique.

 

Below is what I have put in the conditional transformation editor for my first output condition:

 

SSIS Lookup - Mismatched sqlcommand & sqlcommandparam Property Values

  

I had an issue recently and I'd like to solicit feedback as to whether this is an SSIS flaw:

In the lookup component, when you select the Partial or No Cache option, a property in addition to the sqlcommand property becomes available called sqlcommandparam. Originally this package was developed using Lookups pointing to a dimension on "schema_A" and we wanted to change the reference to "schema_B". Now, obviously the sqlcommandparam property became available because you can elect to use the Advanced SQL option in the component. However, I never chose to do that. So, long story short, I go into this package to change the schema reference in the Lookup's query. It changes the sqlcommand property value to be "select value from schema_B.table" but it didn't change the corresponding value in the sqlcommandparam property. That property remained "select value from schema_A.table". SSIS never threw an error to say the schema's didn't match. And the Lookup worked as designed, picking up values from the table on schema_B. Not until we finally decommisioned schema_A did I get an error at runtime that said "Table or view does not exist".

So, can anyone (hopefully Microsoft) confirm that this is in fact a flaw? I am on MSFT VS 2008 V 9.0.30729.1 SP.


Brian

Using Conditional Split data Transfer in SSIS 2008

  
This article uses the Integration Services Conditional Split Data Transformation element to filter and transfer data from a set of flat text files to SQL Server database table. The concept can be easily extended to apply to any other source or destination such as Microsoft Excel. This scenario is useful in creating denormalized database tables in a reporting and analysis situation.

lookup table query

  

we are doing our cms migration. both cms's have their own articleID/contentID, we have a lookup table that contain the article URL and its content id.  Now we want to query that table on every single click to get its respective id. 

How and where i can implement this?



using SharePoint List for SSIS package

  
I need to use a sharepoint list as data source for SSIS package, it works fine until the sharepoint site implement Kerberos only (IIS 7) authentication, anyone knows how to get around this? thanks.

BDC Entity as a Data Source for a Lookup column

  
Is it possible to create a custom lookup field type to have a BDC entity as the data source?

wss2.0 update/delete/hide lookup column that does not display any values

  

Hi All,

I have a document library that contains a Category column that is a lookup field. This is a default column that is a required field when uploading documents to the document library. The Category column is empty and I am unable to amend, hide, make it not required or delete it.

I have gone to Modify settings and columns -> clicked on the Category field to edit, but there is no option to amend the content or delete it. I am only able to amend the Column name and Description.

Since then, I have amended the column name to eg. Category1 and created a new Category field as a lookup and linked it to the correct list.

The problem I am facing now, is that I cannot hide, delete or make the Category1 (old Category) field NOT required. Either I would like to update the original field to display the correct values or alternately hide, delete or make the column not required.

Please help.


Migrate DB + files outside DB into SharePoint using SSIS and scripts

  

Hello, just wanted to ask a question about SSIS and SharePoint, any input will be greatly appreciated.... did search the forum but could not find anything....

So basically we have a SQL Server DB with the metadata ONLY, WITHOUT the related files(doc/ppt/pdf..etc) which should've been in the DB and they are on a separate web server inside folders. Each row in the DB has a key (column) which matches the related folder's name, so for row with ID #112, there's a folder named ID112 on the web server with the relevant files inside. There could be more than 8+ files for one folder.  

Now, I think it is possible but just wanted to double check, would it be possible to use SSIS and some kind of a script to get the metadata inside the SQL DB AND the related files inside those folders, combine & relate them and upload them into SharePoint as a custom list with file attachments? would they display the attachment normally (like the paperclip icon in a sharepoint list...etc) I wonder if anyone has tried this...

Any guidance, thoughts, suggestions for other solutions, inputs will be great!!! Thank you!!

 

John 


Datasheet view lookup column issue

  

When a column that has a lookup is empty in datasheet view and I attempt to update a different column I get this message: the text entered for isn't an item from the list. select an item from the list, or enter text that matches one of the listed items.

I found this hotfix, but it did not apply.

http://support.microsoft.com/kb/948952

Any ideas how to get this resolved?


Categories: 
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend