.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
Gaurav Pal
Post New Web Links

How to read/write GIANT text file with no line breaks

Posted By:      Posted Date: September 20, 2010    Points: 0   Category :ASP.Net

I have to process giant XML files that are 6GB+ in size.  The problem is that there are no line breaks in the file, and the XML is improperly formatted (no root element).  Therefore I can't use the XML class to read the file, and I can't open the file in any text editor or even "Type" it in a command window because the 1 line in the file is over 6GB.

The file contains UTF8 characters.  What I want to do is first write a program just to read and display 100 characters at a time so I can look at the contents.  Next I want to be able to create a new file, write a root element, read/write the contents of the file and then write the closing root element.

Can anyone provide a code snippet of how to read and display the file contents 100 characters at a time.  I can then figure out the rest.


View Complete Post

More Related Resource Links

Deleting Last Line of Text in a text file

Hi, I needed some help on the best way to delete the last line of  text in a text file. I have a subroutine that looks like this, but it is missing some code. Could anyone advise me on this? Dim strLogFile As String = System.Web.HttpContext.Current.Server.MapPath("~/Log.txt")         Dim fs As New FileStream(strLogFile, FileMode.Open, FileAccess.ReadWrite, FileShare.None)         Dim swLog As New StreamReader(fs)         Dim array As String()         Dim value As String = swLog.ReadToEnd ''Array that holds the lines of the text file.         array = value.Split(vbCrLf)        ''code for deleting last line: . . .  fs.Close()

Read a text file on another server from a web page

How do I read/write a text file on another server from a web page. I get the error "Access to the path '//Server2/mydatafiles/test.txt' is denied". I do not get the error if I am running the browser on the server where the files exist. I think I need to set permissions on the destination server in some way.   Can you help? Thanks  

How to write a stylized XML file to a text file?

Hello All,I am using the <asp:xml> tag on front end and I am supplying the Document Content(data source) as well as the transform source(XSL file) on the back end to display the stylized XML on a web page.Now, I want to write the transformed result to a text file. I want to fetch the transformed XML without having to parse through the source HTML. Any suggestions will be greatly appreciated.Thanks.

write Custom MemberShip provdider only to read the connection string from other file

hi,    In asp.net mvc project I am using MYSQlMemberShipProvider. Now I want that instead of reading the connection string from web.config file, it will read the connection string from external file every time. So that I am implementing the cutsom mebership provider class, this class inherits the MemberShipProvider class.    But the problem is that if I inherits the MemberShipProvider class then I have to impelment all of its method in my cutsom membership provider class, But I want to use all other inbuilt methods of MemeberShip. What can i do.I only want to add the code like below: public class CustomSqlMembershipProvider :MembershipProvider    {        public override void Initialize(string name, NameValueCollection configs)        {            base.Initialize(name, configs);                Connectionstring objProducts = // redaing the connection string.   }}But on compiltaion it is giving me the error does not implement inherit abstract member. Please Suggest me any idea.        Thanks In AdanaceAayushi

Random line from a text file



Im trying to pull a random line from a text file and store it into a variable.

I found this piece of code on the web which looks good but I cant seem to get it working in vs 2010, any ideas ?

Dim MyString As String
Dim MyBuffer() As Byte
Open "C:\AFile.txt" For Binary Access Read As #1
  ReDim MyBuffer(LOF(1) - 1)
  Get 1, , MyBuffer
  MyString = StrConv(MyBuffer, vbUnicode)
  ReDim MyBuffer(0) 'Free up memory
Close #1

Many thanks in advance.


Using XMLReader and XMLWriter to read and write to a config.xml file


I have a c# program, and I have created an initialization.xml with some attributes.  One of the attributes is an output directory location that the user enters.  I want to be able to open the init file and write this directory path to the outputLocation attribute of the init xml file.

I know how to read xml files using XMLReader and create and write new XML files with XMLWriter, but not sure how to go about open an existing file, moving to a specific attribute, and writing the value to it.

I am assuming this is pretty straight forward.



How to delete a line from a text file using C#

Hello all

I am developing a little app where I need to monitor a file that is update by several machines.

I need to gather each line wrote by each machine so I need access to a specific line number and take that line, store it into a DB table and delete that line.

I really don't know how to delete the line without parse the information into a string variable and rewrite the file. This can lake the app because the file can contain up to 1000K lines... :-O...

Any suggestion will be appreciated.

I think I deleted the Internet...

Removing blank line from a text file


Hi i have a comma deliminated text file which i am using to generate a multidimensional array using the following code:

Dim DefinitiveDatabase As String(,)
    Dim strfilename As String = (Server.MapPath("~\Definitive database.txt"))
    Dim num_rows As Long
    Dim num_cols As Long
    Dim x As Integer
    Dim y As Integer
    Dim value As Integer
    Dim ConditionName As String
    Dim strlines() As String
    Dim strline() As String
    Dim tmpstream As StreamReader = File.OpenText(strfilename)

strlines = tmpstream.ReadToEnd().Split(Environment.NewLine)
        num_rows = UBound(strlines)
        strline = strlines(0).Split(",")
        num_cols = UBound(strline)
        ReDim DefinitiveDatabase(num_rows, num_cols)
        For x = 0 To num_rows
            strline = strlines(x).Split(",")
            For y = 0 To num_col

How to speed up Reading Text File line by line?


Hi all,

The following is my problem which need to be solved:


I have text file like this:

id=1, name=smith, age=22, address=London;

id=2, name=john, age=30, address=Roma;

id=3, name=Pen, age=19, weight=60, address=Paris;

id=4, name=Ajay, age=44, address=India;


using vb, I succeeded to convert it to a table format like this


id      name        age       weight      address

1      smith          22           null         London

2      john            30          null          Roma

3      pen             19           60           Paris

4      Ajay            44           null          India


I used StreamReader for that.

someone may still ask: Ok Adel so where is your problem? kindly read the following lines

In fact, my only problem is related to the time it takes if the number of rows is 70,000

Read/Write Excel file with formulas and accessing cell ranges


I need to read and write data to some specific cells in an excel file.
I want to send some values to cells from front end i.e asp.net web page and after calculation from excel the value should be send again to asp.net

Example if cell A1 = 10 Cell B1 = 20 then C1 cell should have value of A1 + B1 i.e 30 should be displayed and this cell C1 value should be fetched and displayed in ASP.NET Page or some front end

Secondly is it possible to use formula and ranges from asp.net

how to read the text from the file???


how to read the text below??


†‰€ˆ‡‰#îõ‘þüŠ ꑯõù ‚†ƒ -#ª÷‘þü “‘ ª“îù )øþ¦ùý ¤ª—ùý î‘õ•þø—¤(#•¢þ¢
ø¤÷¢ù ꑯõù îõ‘þü#^a—ú¤ö^b•¦øû÷¢ð‘ö ¤ù ¢÷©^cˆˆƒ#‚€ «.: õ¬ø¤Š ›¢øñ#……ˆí/Š…/
…€…Š}TK{^aˆˆƒ#†/„€€#}BF{#ª“îùû‘ý î‘õ•þø—¤ý#ª“îùû‘ý î‘õ•þø—¤ý -- ¥õøöû‘#^c

I use this code but not display all characters???

Serial Port Communication Read and Write using C#.net

In this simple article you will learn that how you can Open and read COM port data using System.IO.Ports Namespace.
Simply just create New Project of windows application in C#.

Import ServiceProcess APIs
using System.IO.Ports

Imports System.IO.Ports

Write Parsing functions which parse port data according to attached device

New off line application working with sdf file. Need advice


Hey Folks

I am working on a project currently where I need to get data out of a 3rd party vendor's software that uses a .sdf file I looked this up and its a SQL CE file. I was hoping to just link to with with access but thats apparently not going to happen. I need to link to the database extract data from it to a csv file for later use. I also and most important I need to get the data from it and create a report that can be printed while on a customers site that my or may not have web access so I need for this program to be on and off line. I had my webapplication that I have been working on open and was able to bring up the sdf file under the server exploere however I can't seem to get anything out of it when I try to do grid views or detail views errors out.

Can I use one of Visual Studio projects to create this sort of applicaiton if so what would be the best approach. I would love to get an access odbc driver since office is allready installed on the employee's laptop's



I am not able to read xml file with XDocument if its Descendants has Attribute xmlns="http://abc.co


I am not able to read xml file with XDocument if its Descendants has  Attribute like:-

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
  <row ID="1" xmlns="http://www.opentravela.org/OTA/2003/05">
    <var country_name="Caribbean and Rest of the world" city_name="Kingston" flight_category="Economy Class" via="Atlanta" flight_departure="London Heathrow" airline_logo="Delta" adult_fare="Delta" child_fare="660" infant_fare="561" date1="132" date2="2010-08-15" />


I don't know what to do ?

Looking for some help..

How to write xml file?


I am having a three text box. I want to write the fields given in text boxes to xml file and the xml file should be open as save dialog box for the user to download the xml file. I want without saving the xml file in the physical path of the server 
Any idea let me know?


I am having a three text box. I want to write the fields given in text boxes to xml file and the xml file should be open as save dialog box for the user to

Read Excel file from Web Application (C#)



I have a web app where user uploads a file.

File is saved in server locally and I want to read some data from it.

Currently my development  server has office 2003.

So do I need office 2007 in order to be able to  read an excel from office 2007 ?

What are other requirements  ?

I tried a test, using

Microsoft.Office.Interop.Excel.Application excelApp = new Microsoft.Office.Interop.Excel.Application();
Microsoft.Office.Interop.Excel.Workbook wrkBook; // = new Microsoft.Office.Interop.Excel.Workbook();

wrkBook = excelApp.Workbooks.Open(@"C:\www\excel_files\b.xls", ....);

But When trying to open the file I get the error:

Exception from HRESULT: 0x800A03EC

So I'm wondering what I'm missing.


How to write in .dbf file with c#


Hi all

can anybody help me to solve my problem : writing in .dbf  , as following is my code with this error :

[ ERROR [07002] [Microsoft][ODBC dBase Driver] Too few parameters. Expected 3 ]


SqlConnection cn = new SqlConnection("Data Source=(local);Initial Catalog=test;Integrated Security=True");
        SqlDataAdapter da = new SqlDataAdapter("select * from test", cn);
        DataSet ds = new DataSet();
        da.Fill(ds, "test");
        string odbccon = "Dsn=TEST.DBF;dbq=E:\\FOX;defaultdir=E:\\FOX;driverid=533;fil=dBase 5.0;maxbuffersize=2048;pagetimeout=5";

        OdbcConnection connection = new OdbcConnection(odbccon);
        OdbcCommand createCommand = connection.CreateCommand();

        createCommand.CommandText = "CREATE TABLE test (ID int, name varchar(50),last_name varchar(50))";

ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend