.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
Gaurav Pal
Post New Web Links

sql insert very slow on bulk table

Posted By:      Posted Date: September 29, 2010    Points: 0   Category :ASP.Net


We are using SQL Server 2005 x86 enterprise edition. We have one table which has 13 crores / 130 million records. When we insert a record in that table it is taking around 30 seconds. Can u please help optimize it.

The table currently has 1 primary key (Clustered Index) and 4 non clustered index.

Please help.



View Complete Post

More Related Resource Links

Bulk insert of Data from one table to another table



I have one table (TABLE A) which updates with milions of records every day. Now Every day we clear the data of this TABLE A; but before clear the data from TABLE A we get all data from TABLE A and move it into another table TABLE B due to some purpose and we are using 

INSERT INTO TABLE B (col1,col2,.......colN) values (SELECT col1,col2,..........colN from TABLE A)

This approch is not giving us a best performance as we are selecting all data from TABLE A and then inserting into TABLE B in one shot and i have millions of rows in my TABLE A (sometimes 16 millions).

Can anyone suggest any other approch to do so that i can get best performance ?


JP Sharma

Bulk Insert into SQL Server Table With XML


Hey There,


Here, is the example of Bulk Insert into SQL Server Table.

From Application you have to pass a XML string to a Stored Procedure and it will insert all data into table using that XML.

Example SP.




      @strXML varchar(8000)



      Declare @intPointer int

      exec sp_xml_preparedocument @intPointer output, @strXML<

bcp or bulk insert - create table


I saw another entry on this, however it went unanswered.  I'll try again and maybe there is newer information/functionality now and hopefully it will help both of us.

I intended to use bcp as one component of an archival process.  However, during a test to recreate or insert into a new table I do not see any switches or options to tell bcp to create the table if it doesn't exist.  Since the formtat file contains the column descriptors, I would have expected bcp to provide the create table functionality (rather than having to know the orignal table format and hardcoding the create table definition into the recover or reload process).  An I overlooking something?  Is there some new functionality using XML format files that would do this?

bcp bulk insert create table invalid object name

Insert value using Table Value Functions

a real gem in Sql Server 2008. mostly people still using Stored procedure may be they shifted to SQL Server but they are not using TVF right now.

Bulk Copy From Datatable To SQL Server Table


 I have read the data of a excle file and captured the data into the dataset to a datatable, now that data is to be inserted into a SQL Server table using bulk copy option. I am using -

 public bool BulkEnterData(DataTable dt, string tblName)
        SqlBulkCopy bulk = new SqlBulkCopy(con);
        bulk.DestinationTableName = tblName;

        return true;

ERROR Getting-
A transport-level error has occurred when receiving results from the server. (provider: Shared Memory Provider, error: 0 - The pipe has been ended.) 

How to resolve the above problem or any other solution is available.


Need to add value to table on insert



I'm fairly new to .NET.  I need to add values CreatedBy, CreatedDate, ModifiedBy, ModifiedDate to the record before inserting it to the database. 

Please see code below. 

     [MetadataType (typeof(GRIPSMetaData))]
    public partial class SchoolDistrict
        internal void OnSaving(ChangeAction changeAction)
            if (changeAction == ChangeAction.Insert)
              this.CreatedBy = 5;
              this.CreatedDate = DateTime.Now;
            else if (changeAction == ChangeAction.Update)
                this.CreatedBy = 5;


Can I bulk insert to specific columns when bulk inserting a text file or do I have to bulk insert into all columns of a table? The extra column in the table is an identity integer. So I had to create a view without the identity integer to make it work. Maybe there is a more direct way?? If so what would be the syntax? I am a beginner to intermediate SQL guy.   strSQL = "BULK " & _        "INSERT CustomersUpload " & _        " FROM 'C:\MydataFile.txt' " & _        " WITH " & _        " ( " & _        " FIELDTERMINATOR = '|', " & _        " ROWTERMINATOR = '\n' " & _        " )" Thanks

How to setup db diagram to automatically insert username into table?

 I have a table setup in the database for users who are basically going to create a classified listing.  How can I setup a diagram that would easily create a table relationship with the Users table, so that it automatically inserts the users username when they submit a new ad?  If so, how?  I've tried adding a UserName row to my table and creating a relationship with the Users table "UserName" row, but it wasn't accepting the relationship when I tried to save the diagram...apparently because the UserName row in the Users table isn't a primary key?Also, one this is setup, what's the best way to provide the user a way to see a list of their ads and have the ability to edit them?

reading xml data and insert into database table

hi friends, .very urgent for me so only i send following script i create for read xml data and to import to table. i got result for reading xml and rows are added to table but the values are showing null pls clarify this problem. thanks alter procedure pizza_sales ( @xml xml ) as begin INSERT INTO pizza_xml (LOC_CODE, CUST_CODE, PRODUCT_ID, INV_TYPE, INV_NO, INV_DT, INV_QTY_IN_INV_UOM, INV_RATE_IN_BS_CURR, INV_VU_IN_INV_CURR, -- TOTAL_DISC_IN_BS_CURR, NET_INV_VU_IN_INV_CURR, NET_INV_VU_IN_BS_CURR, TOTAL_TXES_IN_BS_CURR ) select table1.column1.value('@LOC_CODE','nvarchar(75)'), table1.column1.value('@CUST_CODE','nvarchar(75)'), table1.column1.value('@PROD_CODE','nvarchar(75)'), table1.column1.value('@INV_TYPE','nvarchar(50)'), table1.column1.value('@INV_NO','nvarchar(100)'), table1.column1.value('@INV_DT','datetime'), table1.column1.value('@INV_QTY_IN_INV_UOM','numeric(28,8)'), table1.column1.value('@INV_RATE_IN_BS_CURR','numeric(28,8)'), table1.column1.value('@INV_VU_IN_INV_CURR','numeric(28,8)'), --table1.column1.value('@TOTAL_DISC_IN_BS_CURR','numeric(28,8)'), table1.column1.value('@NET_INV_VU_IN_INV_CURR','numeric(28,8)'), table1.column1.value('@NET_INV_VU_IN_BS_CURR','numeric(28,8)'), table1.column1.value('@TOTAL_TXES_IN_BS_CURR','numeric(28,8)') from @xml.nodes('pizza/pizzaxml')as table1(column1) end declare @id

create and insert into a dynamic sql table at runtime

Below I have a bunch of SQL statements that creates a table at runtime and the "pic" column is a dynamic column and the code creates the number of "pic" columns depending on the count of the uploader control.  the problem comes when I try to insert into that table and I am trying to say for every "pic" declare variable @pic.... it works well for just one file in the uploader but for multiple files i get the following error The name "pic1" is not permitted in this context. Valid expressions are constants, constant expressions, and (in some contexts) variables. Column names are not permitted.Directory.CreateDirectory(MapPath(".") & "\port\" & clientname.Text & "\")        Dim objConn As New SqlConnection("Data Source=xxxxxxxxx.hostedresource.com; Initial Catalog=mrpoteat; User ID=xxxxxxx; Password=xxxxxxxxx;")        objConn.Open()        Dim strCommandText As String = ""        For index = 1 To Attachments1.Items.Count Step 1            If String.IsNullOrEmpty(strCommandText) Then                strCommandTex

How to insert the Current date and time in to SQL Table..

Hello Members,              I have create the table as per the following ..create table company(  empname varchar(30),  empid int,  joindate smalldatetime)I tried,insert into table values('Kumar',202, ??????)I want to insert Current date and time into table Company.....Please give me the solution...Thanks.. 

pull data from table A and insert into the same table

Hi All, When I pull data from one table and try to insert into the same table, is it possible for any erroneous data loading? ex:   Insert Into TableA Select Cat,Bank, City, Act From TableA A CROSS APPLY (SELECT COUNT(ACT) AS TOT from TableA ) B   Thanks        shamen

How to insert values into my table the data is coming from inline table valued function

Hi all,   i have table oi want insert values into my table the data is coming from inline table valued function how i can use the inline table valued function to insert datainto my table i have inline function which out puts 3 collumns of data how i can insert this data into my table i have the insert statment like below insert into tablename(c1,c2,c3,c4,c5) select ('abc',....................,'xyz') how i can use the out put put of my inline function to  to insert data into my columns c2,c3,c4. please help me out Thanks & Regards Sunil Yoganna

insert into dynamic SQL Table at runtim

the follow ing is supposed to create a table and insert the columns into the table at runtime.  It also has picture column and creates the amaount of picture columns based on the amount of files in the multiple file uploaer.. it works fine with just one file in the uploader.  The problem comes when there  are two or more files in the uploader and intead of adding the second image right after the first image is added, it instead adds the first image then goes back and trys to create the table again, and i get an error saying the table already exsits.I want it to add the second image right after the first image, not go back and create the table then add the second imageDirectory.CreateDirectory(MapPath(".") & "\port\" & clientname.Text & "\")              Dim objConn As New SqlConnection("Data Source=xxx")        objConn.Open()        Dim strCommandText As String = ""        For index = 1 To Attachments1.Items.Count Step 1            If String.IsNullOrEmpty(strCommandText) Then                strCommandText = "

2 questions - Dropdownlist from SQL Table and Insert into several tables

We're trying to convert an older access app with a sql server back end (with many tables) to Sharepoint. There are entry forms, update forms, look up forms, reports, etc the 2 questions I have are: 1. How can I create a dropdownlist that gets its items from a SQL table? 2. How can I create a form that, when filled out, puts items into multiple tables? Basically, I'd like to know if any of this is possible in 2010, and does anyone have any tips on where to start?    

Errors during bulk insert

Hi, While writing a stored procedure in SQL 2008, for a bulk insert, I am getting several errors. Part of the SP is written below for your reference. Need to know where am I going wrong. Any help will be thankfully acknowledged.-Learn24x7----------CREATE PROCEDURE Sp_BulkInsert(@Directory nvarchar(256), @TblName nvarchar(50), @Collation nvarchar(50), @FieldSeparator nvarchar(50),    @RowTerminator nvarchar(50))ASBEGIN    SET NOCOUNT ON;                BULK INSERT [@TblName]    FROM ''+@Directory+'' --Error1: see below    WITH --Error2: see below     (     FIRSTROW = 2,     CODEPAGE = (SELECT CAST(@Collation AS INT)), --Error3: see below     FIELDTERMINATOR = ''+@FieldSeparator+'',     ROWTERMINATOR = ''+@RowTerminator+''     )  ENDGO----------FROM ''+@Directory+'' --Error1: An object or column name is missing or empty. For SELECT INTO statements, verify each column has a name. For other statements, look for empty alias names. Aliases defined as "" or [] are not allowed. Change the alias to a valid name.WITH --Error2: Incorrect syntax near the keyword 'with'. If this statement is a commo

Unable to prepare the SSIS bulk insert for data insertion

Hi,   I am using SQL Server Destinations in my data flow tasks. I'm running this package in the server until i encountered this error:   OnError,,,LOAD AND UPDATE Dimension Tables,,,10/24/2007 1:22:23 PM,10/24/2007 1:22:23 PM,-1071636367,0x,Unable to prepare the SSIS bulk insert for data insertion. OnError,,,Load Dimensions,,,10/24/2007 1:22:23 PM,10/24/2007 1:22:23 PM,-1071636367,0x,Unable to prepare the SSIS bulk insert for data insertion. OnError,,,Discount Reason, ISIS Condition, ISIS Defect, ISIS Repair, ISIS Section, ISIS Symptom, Job Status, Parts, Purchase SubOrder Type, Service Contract, Service Reason, Service Type, TechServiceGrp, WarrantyType, Branch, Wastage Reason,,,10/24/2007 1:22:23 PM,10/24/2007 1:22:23 PM,-1073450974,0x,SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "Dim_T_ISISDefect" (56280) failed with error code 0xC0202071. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure. What could be the reason for this? I don't usually have an error.   cherriesh
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend