.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
Gaurav Pal
Post New Web Links

How to extract large data from SQL to CSV using Asynchronous function

Posted By:      Posted Date: October 16, 2010    Points: 0   Category :ASP.Net


I have a table growth 100000row per day, I'm facing many problems  to export data from this table using ASP.net page.

I don't know how to apply this scenario:

Senerio 1:

1- run Asynchronous function to export data from SQL for specified period to CSV file (without locking the table)

2- Refresh page every 5 sec in order to track the status of extracting data

3- compress the file 

4- Stop refreshing and showing the download link in order to download the compressed file


Senerio 2:

1- run Asynchronous function to export data from SQL for specified period to CSV file (without locking the table)

3- compress the file 

4- send the compressed file by Email

thank you,

View Complete Post

More Related Resource Links

extract picture data from a string


dears,advance thanks.can anybdy help to solve my problem which is describing below.

Example- %%D1,100,0,0004[data...][0d0a]

Head - %%D1
Amount of data package -100
The data package No. - 0
Data Length -1024 (0004 (low bit in front, high bit in back) 0004= 0x0400=1024;)

data- Original Picture data -1024(maximum)
Tail -0x0d0a
Data in hex

from the above example i want to extract picture data and need to store in to the sql table.sql table columname is image and datatype is also image.after that i wanted to display this image column to an aspx page.

expecting help asap.

Data Points: Using XQuery, New Large DataTypes, and More


SQL Server 2005 introduces a lot of new features, but it also enhances the popular and oft-used Transact-SQL (T-SQL) language. Changes include the introduction of new datatypes to store large values using the MAX indicator, the integration of enhanced XML querying and data modification with XQuery, and the new XML datatype.

John Papa

MSDN Magazine March 2006

Multiprocessor Optimizations: Fine-Tuning Concurrent Access to Large Data Collections


Application performance involves more than just speed. In a Web server environment, top performance also means ensuring that the maximum numbers of users can be served concurrently. This can be accomplished through efficient use of multiprocessor machines and thread management. This article presents techniques that can solve a number of concurrency problems. One approach, using thread management, controls access to a database on a per-thread basis, which protects the integrity of the data. In the article, reusable thread classes are built and presented. The classes are then tested and their performance in a live environment is examined.

Ian Emmons

MSDN Magazine August 2001

WCF issues sending large data - "An existing connection was forcibly closed by the remote host"

Hi Guys I have posted the following on www.asp.net but thought it might be productive posting here too.I have been pulling my hair out trying to fix an issue with sending data over WCF, I have read pretty much every THREAD on this forum regarding or similar to this issue without any successful solution.  I have a simple data object which has a [DataMember] with a data type of byte[]. I use this to send binary data from my web app as follows: Asp.net MVC website --> WCF Client (Has my Service References) --> My Services (MyServices.svc) --> Business/Data Tier Everything is being executed from within VS 2008 Pro.   Now I have successfully sent data up to 2.2MB, however it fails if i try and send a file such as 2.3MB or larger i get the following exceptions: General Exception Exception rethrown at [0]:    at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)   at SoftApp.WCFClient.MyServiceReference.IMyService.Document_Save(DtoDocument dtoDocument, Int32 usercode)   at SoftApp.WCFClient.MyServiceReference.MyServiceClient.Document_Save(DtoDocument dtoDocument, Int32 usercode) in c:\dev\softapp\softapp.wcfclient\service references\myservicereference\refe

How to insert values into my table the data is coming from inline table valued function

Hi all,   i have table oi want insert values into my table the data is coming from inline table valued function how i can use the inline table valued function to insert datainto my table i have inline function which out puts 3 collumns of data how i can insert this data into my table i have the insert statment like below insert into tablename(c1,c2,c3,c4,c5) select ('abc',....................,'xyz') how i can use the out put put of my inline function to  to insert data into my columns c2,c3,c4. please help me out Thanks & Regards Sunil Yoganna

SSRS 2008 Export to PDF fails for large data with System.Exception: Parameter is not valid

Hi, We are working on Range Bar Chart using SSRS 2008. The report has huge data. It is properly displaying the data, but when we tried to export to pdf it is failing with the following exception. Exporting to excel works fine. Also if the report has small/medium data it is property exporting to PDF. It is failing only in case of huge data (we have custom page size of 28" X 14"). Of course the report has lot of expressions we used to customize the colors/text as per our requirements.   Server Error in '/Reports_SQLDEV2008' Application. Parameter is not valid. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.Exception: Parameter is not valid. Source Error: An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below. Stack Trace:   [Exception: Parameter is not valid.]   [Exception: An error occurred during rendering of the report.]   [Exception: An error occurred during rendering of the report.]    Microsoft.Reporting.WebForms.ServerReport.ServerUrlRequest(Boolean isAbortable, String url, Stream ou

data contract code generation for large/complex schema (HR-XML/OAGIS) - is there an alternative?

Hello, and thank you for reading.I am implementing a Service based on a predefined specification (HR-XML 3.0). As such, I am starting with the schema, and working my way back to code. There are a number of large Schema documents (which import yet more Schema documents) related to my implementation, provided by this specification.I am able to generate code using xsd.exe, by supplying the "main" and "supporting" xsd files as arguments. But there are several issues, and I am wondering if this is the right approach.- there are litterally hundreds of classes - the code file is half a meg in size- duplicate classes (ex. Type, Type1 - which both represent the same type)- there are classes declared as inheriting from a base class, but that base class is not generated/definedI understand that there are limitations to the types of Schema supported by svcutil.exe/xsd.exe when targeting the DataContractSerializer and even XmlSerializer. My question is two-fold:1. Are code generation "issues" fairly common when dealing with larger, modular xsd files? Has anyone had success with generating data contracts from OAGIS or HR-XML schema?2. Given the above issues, are there better approaches to this task, avoiding generating code and working with concrete objects? Does it make better sence to read and compose a SOAP message directly, while sti

extract data from WSS Database

how to extract data from WSS Database.We have TFS 2008 with WSS 3.0. By querying,programmatically?any help is appriciated?

ASP.NET Events and Pageload not firing when Datagrid containing large number of rows of data

Hi All, I have a datagrid in aspx page.Inside of datagrid i am using around 15 controls such as Button,dropdownlist and text box controls.Once the datagrid binds then the events in the aspx page not firing.Issue occured when the number of rowsgreater than 500.The number of rows less then its works fine.If anybody knows the solution please let me know."Platform i am working on ASP.NET1.1"Regards Hareesh

Can a Synchronous and Asynchronous Transformation in one data flow

Hi Guys, Can a Synchronous and Asynchronous exist in one data flow? I mean, in my data flow, i have 5 Synchronous and 1 Asynchronous Transformation in the end all in one flow. The output is not the way it was use to be because im introducing a Asynchronous Transformation at the end for 2 columns. Any suggestions? Thanks,

Wpf Charting for Large Data

Hi, Currently I am working on Charting control. Here I need to display 10000 data's using LineChart which is provided by the toolkit. But unfortunately the application gets slow because of this huge data. I tried using Dynamic Data Display which is provided by codeplex. Since it is in beta stage, is there any other way I could accomplish this without affecting the application performance. Please help me. Regards, Raaj

How do you extract data from the xml attachment of an incoming email and set it as metadata in a doc

Hi, I currently have a document library that contains InfoPath forms. As a form is loaded into the document library I need the data to be extracted from the infopath file and then set as properties/metadata of the file. The infopath forms will be sent to the SharePoint site as an email attachment. So basically, an email is sent to the SharePoint server with an InfoPath form as an email attachment. The InfoPath form needs to be extracted from the email and uploaded to the document library and at the same time updating the document library metadata with the infopath form data. This has been done using Visual Studio but can this be done using SharePoint Designer 2010 now that we have upgraded our site to SharePoint 2010? Any guidance will be greatly appreciated! Thanks Michael

Error: "Select statements included within a function cannot return data to a client"

hi all, Not sure if there is an easier way to do this but this is my first ever Function I created so far. I am getting the error: "Select statements included within a function cannot return data to a client" All I am trying to do is ...if the date is NOT passed then apply NULL as the default value and run the SELECT statement. If the date IS NOT null then run a different SELECT statement. CREATE FUNCTION [SchemaTest].[fn_Test] ( @instance as uniqueidentifier ,@date as datetime = null ) RETURNS varchar(8000) AS BEGIN If (@date is null) --@date = null ?? Begin Select * From Uds.RealtimeLogging Where instanceID = @instance End Else Begin Select * From Uds.RealtimeLogging Where instanceID = @instance and [timeStamp] > @date End Return @stepIds END Thank you!JCD

WCF Service fails with Unexpected connection closed with large amount of data

Good day all,   I'm sure this question is relatively simple, but I'm quite new to WCF (been on a Java project the last few years and it's nice to be back in MS territory even if a lot has changed!).   Anyway, here's all the details of my scenario I was hoping someone could shed some light on.   1)  This WCF service is hosted as a webservice and is being testing in the WCF Test Tool. 2)  There is a method call on the service that, while rarely used, can return a fair amount of data (over 20,000 objects) - this is a requirement of the system 3)  The object that is being serialized has the following signature/contract (property changed for example): [DataContract] public class Foo { private string prop1; [DataMember] public string Prop1 { get { return prop1; } set { prop1= value; } } private string prop2; [DataMember] public string Prop2 { get { return prop2; } set { prop2= value; } } private string prop3; [DataMember] public string Prop3 { get { return prop3; } set { prop3= value; } } private string Prop4; [DataMember] public string FaaCode { get { return prop4; } set { prop4= value; } } private string prop5; [DataMember] public string Prop5 { get { return prop5; } set { prop5= value

Storing large data in Session State

Hi, We have a scenario where we need to store large tables of data on Session in asp.net page. on a highlevel we have to store following data  1> A table (table 1) with 1000 rows and 8 to 10 columns. 2> A table(table 2) with 500 rows and 8 to 10 columns (actually user can add the data from table 1 to table 2, so as user keeps on adding from table 1 we remove the data and put it in table 2) 3> Another table(table 3) with 1000-1200 rows each having 3 columns. We are having a webfarm, so we need to store it in either SateServer of Oracle DB, please advise which one is better. What would be the performance implications of storing such huge data on the server. One more thing is at any point of time max of 70 -80 people will be accessing our website, so will this cause any performance degradation? Thanks in anticipation Harsha

Timeout issue with large data


Getting timeout error while retriving large chunk od data. it workd fine for smaller chunks.

We tried to increase the following but didnt help.. Any suggestions are appreciated.




" at System.Net.HttpWebRequest.GetResponse()\r\n at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)"


Capture large text return from Javascript function


I have a Javascript function that writes a large scale of text into a div for now, but then I need to capture the text and use it in code behind. I tried to use hidden field but it has size limit so it didn't work.  The text is shown in the page, but when I view source, there is no text in the div so I couldn't use code behind to read the text from the div even though runat is server because the text is always empty. Any suggestions? Thanks!!!

ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend