.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
david stephan
Gaurav Pal
Post New Web Links

Download http request results to memory instead of file?

Posted By:      Posted Date: April 14, 2011    Points: 0   Category :

I have a requirement to download a gazilion finance related files from serveral http sites and load to SQL Server.  I was able to use the script task and leverage the webclient object to download the requested data to files; no sweat.  Then in a file task I read in the file into the process flow (w/in the data flow) and the rest is history.  Is there a way I could bypass the "downloading to a file and re-reading it in step"?  I'd like to simply stream the request using webclient.DownloadData().  My question is how do I get the resulting bytearray into the process/data flow so that I can transform/load where ever I like?

I'm also interested in if this will help with performance.  I'm thinking is should but will this turn into a memory hog then and slow things down.  Also, If I have to use the download and read the file approach, I'm concerned about babysitting the temp file downloads.  Would I need to continuously clean up after myself for every download?

Thanks in advance,




View Complete Post

More Related Resource Links

Files won't download from ASP.NET site when the site has a custom HTTP Handler mapped to the file e


When I have a HttpHandler class in C#/ASP.NET mapped to a file extension in IIS any file with that extension fails to download/display in web browsers (it's downloaded as a 0-byte file in some browsers and nothing at all in other browsers).  After removing the application mapping for the HttpHandler in IIS so it doesn't call the IHttpHandler class in C#, the web browser downloads the file successfully.

This was tested with an IHttpHandler class in C# that has an empty ProcessRequest method. Any ideas on what the problem could be?

Attempt to download MOSS 2007 document results in server out of memory error


A user contacted me regarding a problem he was encountering. He had several .txt files in a sharepoint document library that he wanted to "move" to a linux file server.  He initially changed the doc library's view to Explorer View, then attempted to drag and drop the icons onto a non-sharepoint location. Then he got to the larger files (he has files up to 90MB) . At this point , he got the error

Server Out Of Memory

There is no memory on the server to run your program. Please contact your administrator with this problem.   at Microsoft.SharePoint.Library.SPRequestInternalClass.GetFileAsByteArray(String bstrUrl, String bstrWebRelativeUrl, Boolean bHonorLevel, Byte iLevel, OpenBinaryFlags grfob)
   at Microsoft.SharePoint.Library.SPRequest.GetFileAsByteArray(String bstrUrl, String bstrWebRelativeUrl, Boolean bHonorLevel, Byte iLevel, OpenBinaryFlags grfob)
   at Microsoft.SharePoint.SPFile.GetFileContent(SPOpenBinaryOptions openOptions)
   at Microsoft.SharePoint.SPFile.OpenBinary(SPOpenBinaryOptions openOptions)
   at Microsoft.SharePoint.SPFile.OpenBinary()
   at Microsoft.SharePoint.ApplicationPages.Download.WriteFile(String sourceUrl)
   at Micros

file upload in chunks or not buffering in memory before writing to disk?


What are the options for handling file uploads to reduce the memory footprint?  Is there a way to upload in chunks?  Is there a way to stream upload directly to disk instead of loading entire file in server memory?


I am not able to read xml file with XDocument if its Descendants has Attribute xmlns="http://abc.co


I am not able to read xml file with XDocument if its Descendants has  Attribute like:-

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
  <row ID="1" xmlns="http://www.opentravela.org/OTA/2003/05">
    <var country_name="Caribbean and Rest of the world" city_name="Kingston" flight_category="Economy Class" via="Atlanta" flight_departure="London Heathrow" airline_logo="Delta" adult_fare="Delta" child_fare="660" infant_fare="561" date1="132" date2="2010-08-15" />


I don't know what to do ?

Looking for some help..

how to know when a user finish his file download successfully ?


hi all

i am doing a page to sell files online

 the paying method is by a card or bank account and the customer is not registered to the site so i want to make sure that the customer has downloaded the file successfully after that the system completes the payment operation

so if there is any way to know do that tell me please

i use c# thanx all



Test Run: Web Application HTTP Request-Response Test Automation with JavaScript


Write a simple browser-based request-response test automation using JavaScript that's platform independent and useful when you are working in a highly dynamic environment.

James McCaffrey

MSDN Magazine January 2010

HTTP Pipelines: Securely Implement Request Processing, Filtering, and Content Redirection with HTTP


ASP.NET is a flexible and extensible framework for server-side HTTP programming. While most people think of ASP.NET in terms of pages served, there is a lower-level infrastructure sitting beneath this page model. The underlying plumbing is based on a pipeline of app, module, and handler objects. Understanding how this pipeline works is key if you want to get the most out of ASP.NET as an HTTP server platform, while making your process more efficient, and keeping your server secure. This article introduces the architecture of the pipeline and shows how you can use it to add sophisticated functionality to an ASP.NET-based app.

Tim Ewald and Keith Brown

MSDN Magazine September 2002

ISAPI Extensions: Creating a DLL to Enable HTTP-based File Uploads with IIS


The MIME-compliant content type, called multipart/form-data, makes writing HTML that uploads files almost trivial. On the server side though, ASP does not have a way to access data in the multipart/form-data format. The most flexible way to access the uploaded file is through a C++ ISAPI Extension DLL. This article describes a reusable ISAPI extension DLL that allows you to upload images and files without writing C++ code. It is coupled with a few COM components that make it readily reusable for ASP development. With .NET, this whole process is much easier, and this article shows preliminary code that uploads files using ASP.NET features.

Panos Kougiouris

MSDN Magazine October 2001

File Download Problem


Hi everyone,

I am using a download handler to handle the downloading of files from my website.  I am using a slightly modified code from the Microsoft article http://support.microsoft.com/kb/812406

Everything is working fine and downloading, even with large files, but it does not show the file size for any of the files to download. Even if theyre really small.

Here is my code for the handler

public class DownloadHandler : IHttpHandler

        public void ProcessRequest(HttpContext context)
            string path = "";
                path = context.Server.UrlDecode(context.Request.QueryString["src"]).DecryptString();
            catch(Exception ex)
            System.IO.Stream iStream = null;

            byte[] buffer = new Byte[10000];

            int length;

            long dataToRead;

            string filename = System.IO.Path.GetFileName(path);

                iStream = new System.IO.FileStream(context.Server.MapPath("~/" + path), System.IO.FileMode.Open,
                            System.IO.FileAccess.Read, System.IO.FileShare.Read);


Problem download file from SQL Server


Hello everyone. I am having an issue downloading files that I have store in SQL Server. I have no problem in a WinForm. What am I doing wrong in my code?

Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load

        Dim sch As New clsReport
        Dim fileData As New clsReport.Letter
        fileData = sch.DownloadLetter(Session("LetterID"))
        Dim ms As System.IO.MemoryStream = fileData.fil

        Dim fil As Byte() = ms.ToArray
        Dim nam As String = fileData.name
        Dim ext As String = fileData.ext
        If Not fil Is Nothing Then
                ' Response.Clear()
                Response.AppendHeader("Content-Disposition", "attachment; filename=" + nam & "." & ext)
                ' Response.AddHeader("Content-Disposition", nam & "." & ext)
                Response.AddHeader("Content-Length", fil.Length.ToString)
                Response.ContentType = "Application/octet-stream"
                Response.Buffer = True
            Catch ex As Exception
            End Try

        End If

    End Su

The file reached the maximum download limit. Check that the full text of the document can be meaning



I'm facing an issue with the indexing.

I have 1 WFE+Index server+DB server.

Index server is not installed with MS FIlter pack 1.0

When crawling, the there will be document with warning in crawl log:
The file reached the maximum download limit. Check that the full text of the document can be meaningfully crawled.

Documents that with warning are such as doc, ppt, xls, docs, ppts and many others.
However, I view into the successful crawled document, there are doucments with ext doc and ppt.

For large file index, there are MaxGrowthFactor + MaxDownLoadSize required to be added into the index server.

As my understanding is, MS Filter Pack should installed into index server(already did, correct me if i'm wrong).

I looked into the Office SharePoint Seach(CA>Services in farm), if the server is appointed to "Use this server as indexing server", then MS Filter Pack is suppose to be installed into that particular server as well.

At the bottom, there also has another option is "Use all web front end for crawling".

The question here is, IF the option "Use all web front end for crawling" is selected.
Does the WED FRONT END Server required to installed the Ms Filte

Excel file not showing in search results

We have a strange issue. I have one Excel document (or at least one that I am aware of) that will not show up in search results for content within the document. I can find the document by title just fine, but when I search for words within the document I do not see it. The document was originally a 2003 Excel document and was recently converted to 2007. Other than that there isn't really anything unusual about it (some formatting stuff, but mainly just background colors and whatnot, nothing fancy). The crawl log shows the document was crawled without error. Anyone know anything else I can check on?

Anonymous Web Site receives "The request failed with HTTP status 401: Unauthorized"


I set up a test website on my local box and was able to access sharepoint search using the querySearch web service from SP.  After I got it all working properly I transferred the code to the target website that is set up with anonymous access for all people on the fab floor.  However from this website I keep getting the 401 error.  It can't be SP because the same code worked from my local website "localhost".  Any suggestions?  we don't want to have to have the fab people log in so we can't not have the box anonymous.  I would appreciate any suggestions very much!!!

Using WSS 2007 with MOSS.


  private void GetDefaultCredentials()
      m_where = "get configuration data for credentials";
      string userid = ConfigurationManager.AppSettings[ "userid" ];
      string password = ConfigurationManager.AppSettings[ "password" ];
      string domain = ConfigurationManager.AppSettings[ "domain" ];

      m_where = "set search query credentials";
      m_query.Credentials = new NetworkCredential ( userid, password

HTTP 400 Bad Request Error when uploading to document library


I am running MOSS 3007 Standard on Windows 2003. When trying to upload files larger than about 29Mb, I am getting an HTTP 400 Bad request error. The error states "HTTP Error 400 - Request is badly formed." I am able to successfully upload a 28Mb file. Sharepoint is configured to accept files up to 75Mb.

I have done a fair amount of research on this, but none of the solutions seem to apply to our environment. The two most likely candidates do not apply, for example we are NOT running URLScan (there are a variety of solutions suggesting this could be an issue with uploads of 30Mb or more), and we are on 2003 with IIS6, so again the issue relating to 2008 and IIS7 do not apply.

The "Request is badly formed" part of the request is particularly confusing as most issues I have come across where this error is the symptom point to errors in custom code that is trying to talk to SharePoint. We have done none of our own custom coding and this error is occurring when attempting a simple upload to a document library.

Trying to drag and drop the file across when the document library is open in windows explorer also fails.

I did come across someone saying they had a similar problem which was caused by a damaged content database, however in this case, it is occuring regardless of which content database I use, so either they are all corrupt

why does WSDL file have httpS? i just need http

I built my c# web service.  Tested it.  I built my android app.  Tested it.  All is well.  I tried to get these things to talk to each other...FAIL.  I found these lines in my wsdl. <wsdl:service name="Service1"> - <wsdl:port name="Service1Soap" binding="tns:Service1Soap"> <soap:address location="" /> </wsdl:port> - <wsdl:port name="Service1Soap12" binding="tns:Service1Soap12"> <soap12:address location="" /> </wsdl:port> </wsdl:service> What decided that the path should be https?  I never told it that.  It is overriding IIS which says everything should be http.  Someone please help me before I go ballistic.  Oh and for others developing on android 2.2, it won't take unsigned certs, so you either have to configure a local certificate authority or go http. 

10 MB File Download In ASP.NET Socket Error

Hi,I received the following error message, when downloading file of 10 MB. Please let me know the solution. I already configured the httpRuntime value to <httpRuntime maxRequestLength="10250 " executionTimeout="1000". Still I am getting the error message.The description for Event ID ( 0 ) in Source (XXXXX) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: Unable to read data from the transport connection: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full. Source: mscorlib TargetSite: Void HandleReturnMessage(System.Runtime.Remoting.FileUploading.IFileUpload, System.Runtime.Remoting.FileUploading.IFileUpload) StackTrace: Server stack trace:    at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)   at System.IO.BufferedStream.Read(Byte[] array, Int32 offset, Int32 count)   at System.IO.BinaryReader.ReadBytes(Int32 count)   at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadArrayAsBytes(ParseRecord pr)   at System.Runtime.Seri
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend