Accessing a flat file/folder via SSIS that is out of my domain

I have 10 text files which are on average of 6  GB in size. Out of those 10, 4 files average up to 10 GB in size.

We have 10 SSIS packages which loads those text files onto 2 staging tables. Each 5 of them loads 1 table.

Inside the package, its only a flat file source, data conversion for all columns from source, and write into the destination table.

And this process takes around 5 hours to complete. the size of 2 tables after the load completes is 60 M and 40 M each.

The other thing which I have observed is all the packages are scheduled through one schedule and the schedule  have 10 diff sql server jobs( for 10 packages). Does it have an impact on the processing time ,as all these jobs will be

The flat file is having 10 columns so used 9 delimitor. Few records in the Flat file is having more than 10 columns. I need to redirect in to some error table and rest valid records needs to be loaded in target. Pls help.

Hi All,

I have a package which loads data from a flat file source to an OLEDB destination, which is executed successfully and data is loaded perfectly.

But later when the data in the flat file source is modified i.e. if an extra column gets added to the text file, the package is throwing an error because it is unable to detect the extra added column.

How can i avoid this error??? I need my package to execute successfully ignoring the changes(added columns) happened in the flat file source.

Please provide me wth your suggestions and solutions....

Thanks in Advance!!

I created a SSIS to export to a flat file (from a SQL command : a stored proc).

I don't wan't my SSIS to create an empty file if there is no data.

 

How can I achieve this ?

 

Thanks,

 

Vince

Hello,

I have a corrupt Flat File and I need to write a SSIS Script Component to Replace LF to CR LF in my flat file.

Any idea how to do it?

Thanks

I have a requirement to output some large data sets as flat files.

They have specified the file format and I have no issues with all but one thing. They want the data files to end with "the standard EOF character that is used in Linux". When I look at the file I produce with SSIS in a text editor that shows the HEX values the last thing I see in the file is the newlline (0A) chactare for the last line. How can I get it to insert this EOF character they want?

I'm trying to import a flat file into a table. nothing complex I thought but I get this err and i'm baffled -

The code page on column manager_id (70) is 1252 and is required to be 65001.

What's going on here and how do I fix this?

I'm a complete SSIS beginner.  I suspect the answer to my question is out there I just don't know enough to construct the search that would find it.

I'm writing a program to generate fixed width format text files from proprietary databases for use as Flat File Sources in SSIS packages.  I know I can provide a header row containing column names.  But I have a lot more information about the file (obviously!).  I know how many columns there are, what the width of each column is, what the type of each column is, etc.

Is there some way I can record the information I have so that the SSIS user consuming my file doesn't have to input all that information through the Import and Export Wizard? 

For example, if I were generating these files for use with the BCP utility I could create a Format File to describe the structure of each field in my data file.  Is there some similar capability in SSIS or do the consumers of my data files have to input InputColumnWidth, Data Type etc. through the Advanced editor dialog box?

I am writing to a flat file destination in a 2008 SSIS package. 99.99% of it works correctly. However, I get one duplicate record in the destination file.

Here is the basic flow of the package:

 1. Read two ISO-8859-1 encoded files and encode their text to UTF8 in memory

 2. Combine the two files together in memory and load them into a lookup cache

 3. Read another source file from disk

 4. Match an ID column

Hi,

I have a flat file(.csv) which contains data(strings incuding date in yyyy-mm-dd format).This i am using in Flat File Source and want to export those data to an Excel Destination.Before exporting to Excel Destination i have created a Derived Column component where i am using an express to get day,month,year part and creating a date sting in dd/mm/yyyy format then conveting it to DT_DATE datatype and then exporting to Excel Component.But while exporting to Excel this date format is getting changed to mm/dd/yyyy format.

The expression i used in derived column is

(DT_DATE)(SUBSTRING(date,9,2) + "/" + SUBSTRING(date,6,2) + "/" + SUBSTRING(date,1,4))

where date is a column from Flat File Source.

Even I changed the Locale to English(United Kingdom) in Flat File Connection Manager for the above .csv file.But still it didnt work.

I searched a lot in google and failed to get any solution on this.

Can anyone help me in solving this issue ?

Steps I have followed are
1.I have created myres.en-GB.resx file at C:.
2.then created locale specific myres.en-GB.resources using resgen at c: only,
resgen c:\myres.en-GB.resx
3.Linked the Resource File to Assembly using AL.exe which is giving me assembly at c: only.
Al /t:lib /embed:myres.en-GB.resources /out:ClassLibrary1.dll
4.I have created one web application using vs 2008 & then build it.
5.Then placed both resource file & dll in web applications Bin\en-GB folder.
6.Anf finally written code at page load like
 CultureInfo info = new CultureInfo("en-GB");
            Thread.CurrentThread.CurrentCulture = info;
            Assembly asd = Assembly.Load("ClassLibrary1");
            ResourceManager res = new ResourceManager("myres.en-GB", asd.GetSatelliteAssembly(info));
            string sdf = res.GetString("Lab1");

but after running application I m getting following error
MissingManifestResourceException", "Could not find any resources appropriate for the specified culture or the neutral culture. Make sure "Cla

Robert Hess

MSDN Magazine May 2000

We are using a third party wsp in our solution which directly adds the web parts into the web application store rather than the web part gallery(References to them are present in IIS site's wpcatalog directory). We want to remove these web part files so that the users will not be able to add these web parts to pages directly. Or any other modification so that users will not be able to add these directly. Is there a way we can do this programmatically?

Currently I am storing my username and password (passwordFormat="SHA1") credientails in my web.config.  I would like to figure out how to access them in an xml file that I have stored in my App_Data directory rather than the web.config file because I do not want my application restarting everytime I manually add a user (small list of 5 authorized users for the CMS section).

Here is what my web.config section looks like:

<authentication mode="Forms">
      <forms name=".Administration"
            loginUrl="~/SiteAdmin/Default.aspx"
            defaultUrl="~/SiteAdmin/Administration/Default.aspx"
            protection="All"
            timeout="30"
            path="/"
            requireSSL="false"
            slidingExpiration="true"
       &n