.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
Gaurav Pal
Post New Web Links

A large 76mb xml file best way to count the number of nodes before processing

Posted By:      Posted Date: April 10, 2011    Points: 0   Category :ASP.Net



For the general processing I am using a XMLReader due to the size of the file and technically only need a once time read per file.

But the external company who made this starts it by doing a count of the nodes to say how many inserts/updates are to be done this is done by basically looping through the xml file counting them and then they recreate the xmlreader again to finally process it.

It just seems a bit clumsy to me, understandably the size of the file means we can't just load it into memory but is there another low cost method to count the nodes without having to recreate the xmlreader each time.


Could we stream the file instead and use XMLDocument or Xdocument (giving us linq capabilities) to improve the efficience as I imagine there must be some sort of performance hit re-creating a XMLReader 2 to 3 times.



For example would this use more memory than say xmlreading and looping the whole thing through to get a count.

using (XmlReader reader = XmlReader.Create(filename, settings))
                    XPathNavigator nav = new XPathDocument(reader).CreateNavigator();
                    XPathNodeIterator xPathIt = nav.Select("//root/theNode");
                    int c = xPathIt.Count;

View Complete Post

More Related Resource Links

SSIS 2005 - Foreach loop container - Stopping the loop after processing X number of file?

I need to stop the Foreach loop container from processing more files in the folder than desired. Scenario:  If I only want 1 files processed, i need to stop the loop after it finds 1 file.  No matter what the contraints or tests or variables I set, the loop processes all the files in the folder.  This is bad because I need the value of the 1st mapped variable and not the last one it finds. I have tried counting records and setting variables used in the contraints but to no avail  Nothing seems to stop the loop. Please advise

Slow processing due to large number of threads.


Hello Friends!

I am working on a project in which I am creating around 25 threads in my application. All the threads are communicating with each other. The problem is that due to large number of threads the application is executing very slow and my CPU utilization reaches around 100%.  How can I overcome this problem? Can I create multiple programs instead of threads, if yes then how will they communicate with each other?


While processing large cubes, various file system errors: "the background thread running lazy writer



We consistently get these I/O errors when processing large OLAP databases.  After a reboot, I can usually get a database to process a few times, but then suddenly reprocessing will fail with errors similar to the one shown below, when nothing changed (no data or structure changes).  The specific file(s) it reports are always different.  Also, sometimes it crashes the msmdsrv.exe process, which seems to intermittently affect other server functions as well (killing the ability to RDP into the server, or the ability to open Event Viewer).


<error message>

File system error: The background thread running lazy writer encountered an I/O error.
Physical file: \\?\D:\OLAPData\SSAS1\SA1.166.db\SA1 Data Source View 1.238.cub\Fact Subject.231.det\Fact Subject.221.prt\271.Aspect.Source Key.fact.map.hdr.
Logical file: .
File system error: The following error occurred while writing to the file 'LazyWriter Stream':
Insufficient system resources exist to complete the requested service. .
File system error: The background thread running lazy writer encountered an I/O error.
Physical file: \\?\D:\OLAPData\SSAS1\SA1.166.db\SA1 Data

Count words in the ascx file



I have 163 .resx files. I want to count the words in these files, because my want to translate the sentences (my task is the localisation).

Now I'm opening each file in Ms Visual Studio and I count words manually. How can I count words in all files? And the second question: can I use a program what grab strings from .resx into an editor where I can translate them and save bacn into .resx files? Or must open the files one by one and edit the text fields manually?

Thanks for helping, unfortunately I'm not a programmer.

Cannot count number of checked items using jQuery

Hello,Using Microsoft built-in jquery files, when counting numbers of checked checkbox, the jquery-1.4.2.min.js cannot count the correct result ,however, the version jquery-1.3.2.min.js gives out the correct one, why?            $("#check").click(function () {                var nameSports = new Array();                var i = 0;                $('input[name=selector]:checked').each(function () {                    nameSports[i] = jQuery(this).val();                    i += 1;                });                if (loopCounter != 0) {                    alert(i);                }  

Upload large file via webservice problem in vb.net 1.1

Hi All, I am uploading large file via webservice using below my code in vb.net 1.1. Problem is vb.net 1.1 don't know Using Object loop. I would like to know any other loop similar Using loop to replace in vb.net 1.1Public Function UploadLargeFile(ByVal FileName As String, ByVal buffer As Byte(), ByVal Offset As Long) As Boolean Dim retVal As Boolean = False Try Dim FilePath As String = Path.Combine(System.AppDomain.CurrentDomain.BaseDirectory.ToString() & "\fpath\", FileName) If Offset = 0 Then File.Create(FilePath).Close() End If Using fs As New FileStream(FilePath, FileMode.Open, FileAccess.ReadWrite, FileShare.Read) fs.Seek(Offset, SeekOrigin.Begin) fs.Write(buffer, 0, buffer.Length) End Using retVal = True Catch ex As Exception Throw ex End Try Return retVal End Function  

Uploading Large File (40mb) fails... (webhttpbinding) (Azure)

I'm using javascript to upload a file to my WCF service hosted on Azure.  (40 mb file) Here is the snippets from the web.config for the service: <system.serviceModel> <bindings> <webHttpBinding> <binding transferMode="Streamed" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647" openTimeout="00:25:00" closeTimeout="00:25:00" sendTimeout="00:25:00" receiveTimeout="00:25:00" name="WebConfiguration"> </binding> </webHttpBinding> </bindings> <behaviors> <endpointBehaviors> <behavior name="RestfulEndpointBehavior"> <webHttp/> </behavior> </endpointBehaviors> <serviceBehaviors> <behavior name="CPUploadServiceBehavior"> <serviceMetadata httpGetEnabled="true" httpGetUrl=""/> <serviceDebug includeExceptionDetailInFaults="false"/> </behavior> </serviceBehaviors> </behaviors> <serviceHostingEnvironment aspNetCompatibilityEnabled="true"/> <services> <service name="CPUpload" behaviorConfiguration="CPUploadServiceBehavior"> <endpoint add

ASP.NET Events and Pageload not firing when Datagrid containing large number of rows of data

Hi All, I have a datagrid in aspx page.Inside of datagrid i am using around 15 controls such as Button,dropdownlist and text box controls.Once the datagrid binds then the events in the aspx page not firing.Issue occured when the number of rowsgreater than 500.The number of rows less then its works fine.If anybody knows the solution please let me know."Platform i am working on ASP.NET1.1"Regards Hareesh

How to read and analyze a very large log file with high efficiency and performance?

It is a very large .txt file (more than 3M), and produced everyday, the content is user's system log like below: 2007-11-01 18:20:42,983 [4520] INFO GetXXX() SERVICE START2007-11-01 18:21:42,983 [4520] WARING USER ACCESS DENIED2007-11-01 18:22:42,983 [4520] ERROR INPUT PARAMETER IS NULL CAN NOT CONVERT TO INT322007-11-01 18:23:59,968 [4520] INFO USER LOGOUT   Any idea? Thanks.

Count of number of times the document is downloaded from document library

Hi All, I am working on sharepoint 2007.  My requirement is: I need to take the count of number of times the document is downloaded from document library. I used log report which doesn't solved my purpose. Any Idea how can we acheive this ?   Regards

Insert Into SQL Server, With Current File Uploader Count

I have a Multiple File Uploader named "Attachments1" , and to get the cound I used "Attachments1.items.count"... then I have a Insert Into Command for SQL Server, that Inserts information into the table for every item in "Attachments1", you can see the below code... For Each item As AttachmentItem In Attachments1.Items             Dim objConn2 As New SqlConnection("Data Source=XXX")             objConn2.Open()             Dim objCmd2 As New SqlCommand("INSERT INTO clientport (name, event, phone, email, date1, password, path, path2, filename, number)" & "VALUES (@name, @event, @phone, @email, @date1, @password, @path, @path2, @filename, @number)", objConn2) But for the "number" Column, I would like to insert the Count for that file... For example if Attachments1 has 3 files it will insert into the database the time (which works fine)  the probem is the number column doesnt make file1 for the first file, file 2 for the second file, and file3 for the third file... Instead it puts file3 for all three rows when it Inserts into the DB.  my code is below to get the count.   Dim strcommandtext As String = ""    

Getting file count on folder basis

Hi, I am getting the count of items modified in last 30 days. But I would like to get it a folder and sub folder basis. How to achieve this. [CODE]  int coa = 0; foreach (SPListItem item in AuditList.Items) {    DateTime CurDate = DateTime.Now;    DateTime gesDate = item.File.TimeLastModified;    TimeSpan t1 = CurDate - gesDate  ;    if (t1.TotalDays <= 30)   {         coa++;    } } Console.WriteLine("count is" + coa); [/CODE] How to achieve the above . Thank you

How to find the Number of Tables Present in an XML File?

Dear All,Can any one let me know "How to find the number of Tables present in a XML Schema File"?Thanks and Regards,G.V.N.Sandeep

How to count number of characters in a textbox?

On my current form I have it validating a Canadian Postal Address (xxx xxx). The space is optional but if the user does not use a space, then once the validation passes it will insert one. How can I count the first 3 characters of the textbox, insert a space, then add the rest of the characters?Thank you

Folder's file count getting mismatched



I have a folder, which contains sub folders. For example: Doc is the root folder. Inside it has 2 sub folders named Doc1 and Doc2.

Doc1 contains 2 sub folders - fol1 and fol2.

"fol2" folder does not contain any files, where as "foc1" has a file in it. I am trying to get the count of files modified in last 30 days. Against each of the sub folders,and the corresponding parent folder i want to show the count of modified items. I am getting the sub folders count, but that count is not getting updated to the parent folder.

I am missing a small part of updation, but not sure where it is. This is the code i have used:


public static void GetOpenItems()
SPFolder folder = list.RootFolder;
int mouas = 0;

if (folder != null && folder.Name.CompareTo("Forms") != 0)
//Get Individual Folder Total
 foreach (SPFolder subFolder in folder.SubFolders)
  if (subFolder.Name.CompareTo("Forms") != 0)
  mouas = CountFilesInFolder(subFolder, System.DateTime.Now.AddDays(-30));
  subFolder.Item["Files_Count"] = mouas;

static int CountFilesInFolder(SPFolder folder, DateTime startDate)

400 Bad Request when trying to send large file to service using basichttp streaming.


Hi All,


I am trying to send the large file (approx 1.5GB) to the WCF service for upload using basic http streaming.

I have increased all the timeouts and quotas in the server and client's config files to quite large values, but still I am getting Bad Request error on the client.


Also the Bad request message is returned always in 1 minute 30 seconds after sending the request to server.


What the most unusual thing is when I use Fiddler or Charles to debug communication, everything works fine. Is it something like fiddler is pinging the IIS or service and hence its not getting timeouted or like that?




Following are the details:

Server Config:


 <serviceHostingEnvironment aspNetCompatibilityEnabled="true" />
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend