.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
david stephan
Gaurav Pal
Post New Web Links

How to read and analyze a very large log file with high efficiency and performance?

Posted By:      Posted Date: September 12, 2010    Points: 0   Category :ASP.Net
It is a very large .txt file (more than 3M), and produced everyday, the content is user's system log like below: 2007-11-01 18:20:42,983 [4520] INFO GetXXX() SERVICE START2007-11-01 18:21:42,983 [4520] WARING USER ACCESS DENIED2007-11-01 18:22:42,983 [4520] ERROR INPUT PARAMETER IS NULL CAN NOT CONVERT TO INT322007-11-01 18:23:59,968 [4520] INFO USER LOGOUT   Any idea? Thanks.

View Complete Post

More Related Resource Links

Is SQLXMLBulkLoad4Class performance competitive for large file importing


I have posted SQLXMLBulkLoad4Class question in XML and the .NET Framework forum.  No one answered.  Try my luck here.

According to http://msdn.microsoft.com/en-us/library/ms171721.aspx -- "You can insert XML data into a SQL Server database by using an INSERT statement and the OPENXML function; however, the Bulk Load utility provides better performance when you need to insert large amounts of XML data."  I thought "SQLXML Bulkload 4.0 Type Library" was a component to use for fast importing large xml file.

I have created 10 tables with auto increase primary keys and foreign keys in SQL Server database and used the following codes to shred a 390 MB file into those tables without any transformation.  It took me around an hour.  What a disappointment!  Flatting this file into an in-memory table and importing it with SqlBulkCopy or using SSIS only took me around 6 minutes. 

Did anyone experience this?  Is SQLXMLBulkLoad4Class simply just so slow or is it misused by me?




                SQLXMLBULKLOADLib.SQLXMLBulkLoad4Class objBL = new SQLXMLBULKLOADLib.SQLXMLBulkLoad4Class();

ASP.NET Best Practices for High Performance Applications

This article lists the techniques that you can use to maximize the performance of your ASP.NET applications. It provides common issues, design guidelines, and coding tips to build optimal and robust solutions.

High-Performance .NET Application Development & Architecture

It has always been a goal of project architects to plan an effective strategy from the ground up in regards to an new application. All relevant factors are taken into consideration with respect to the application, from its design and layout to a functional website infrastructure. Pre-.NET strategies and design guidelines still effective now were developed with Microsoft's DNA (Distributed interNet Application) platform. This model successfully served the purpose of architecting N(any number of)-Tier (levels) applications. In its basic sense, as in most robust, distributed applications, you'll architect 3 main layers or Tiers: presentation, business rules and data access.

High Performance ASP.NET - Speeding Up ASP.NET Pages

have been writing a series of blog posts, which I have named High Performance ASP.NET Websites Made Easy! There is no rhyme or reason to the order of these posts and certainly can be read in any order:

Developing High Performance and Scalable ASP.NET Websites
Avoid Chatty Interfaces Between the Tiers in Your ASP.NET Web Application
ASP.NET Page Profiling - Page Tracing - High Performance and Scalable ASP.NET Websites Made Easy
Web Applications: N-Tier vs. N-Layer - Benefits and Trade-Offs
Query Analyzer - Sql Server Database Indexes and Execution Plans

10 Tips for Writing High-Performance Web Applications

Writing a Web application with ASP.NET is unbelievably easy. So easy, many developers don't take the time to structure their applications for great performance. In this article, I'm going to present 10 tips for writing high-performance Web apps. I'm not limiting my comments to ASP.NET applications because they are just one subset of Web applications. This article won't be the definitive guide for performance-tuning Web applications-an entire book could easily be devoted to that. Instead, think of this as a good place to start.

I am not able to read xml file with XDocument if its Descendants has Attribute xmlns="http://abc.co


I am not able to read xml file with XDocument if its Descendants has  Attribute like:-

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
  <row ID="1" xmlns="http://www.opentravela.org/OTA/2003/05">
    <var country_name="Caribbean and Rest of the world" city_name="Kingston" flight_category="Economy Class" via="Atlanta" flight_departure="London Heathrow" airline_logo="Delta" adult_fare="Delta" child_fare="660" infant_fare="561" date1="132" date2="2010-08-15" />


I don't know what to do ?

Looking for some help..

Read Excel file from Web Application (C#)



I have a web app where user uploads a file.

File is saved in server locally and I want to read some data from it.

Currently my development  server has office 2003.

So do I need office 2007 in order to be able to  read an excel from office 2007 ?

What are other requirements  ?

I tried a test, using

Microsoft.Office.Interop.Excel.Application excelApp = new Microsoft.Office.Interop.Excel.Application();
Microsoft.Office.Interop.Excel.Workbook wrkBook; // = new Microsoft.Office.Interop.Excel.Workbook();

wrkBook = excelApp.Workbooks.Open(@"C:\www\excel_files\b.xls", ....);

But When trying to open the file I get the error:

Exception from HRESULT: 0x800A03EC

So I'm wondering what I'm missing.


Windows with C++: Exploring High-Performance Algorithms


See how you can gain efficiency in surprising ways by looking closely at your algorithms, the data they operate on, and the hardware you're designing for.

Kenny Kerr

MSDN Magazine October 2008

Winsock: Get Closer to the Wire with High-Performance Sockets in .NET


The Win32 Windows Sockets library (Winsock) provides mechanisms to improve the performance of programs that use sockets, and the Microsoft .NET Framework provides a layer over Winsock so that managed applications can communicate over sockets. To use all these layers to write a truly high-performance socket-based application requires a little background information, as Daryn Kiely explains here.

Daryn Kiely

MSDN Magazine August 2005

ASP.NET: 10 Tips for Writing High-Performance Web Applications


Writing a Web application with ASP.NET is unbelievably easy. So many developers don't take the time to structure their applications for great performance. In this article, the author presents 10 tips for writing high-performance Web apps. The discussion is not limited to ASP.NET applications because they are just one subset of Web applications.

Rob Howard

MSDN Magazine January 2005

ATL Server and Visual Studio .NET: Developing High-Performance Web Applications Gets Easier


When developing high-performance applications for the Web, developers often must choose between performance and ease of development. With ATL Server, new with Visual Studio .NET, developers get the best of both worlds. ATL Server uses a tag replacement engine written in C++, provides a simple programming model, and promotes enhanced performance and easy debugging. This article presents an overview of the ATL Server architecture, then creates a basic ATL Server project. It then goes on to explain processing SRF files, HTTP streams, forms, cookies, and header files. Managing session state is also discussed, along with file uploads and performance monitoring.

Shaun McAravey and Ben Hickman

MSDN Magazine October 2000

Moved from WinXP to Win7 - don't see choice to open file in read only or edit mode


I just recently moved from Win XP Pro SP3 to Win 7 32-bit Pro. I am logging into domain X normally. Our Sharepoint 2007 server authenticates only on domain Y. So I start Internet Explorer with "run as different user" and provide my domain Y credentials. All fine. 

In win XP, after I did this, when I click on an Excel file on Sharepoint, I would get a dialog asking if I wanted to check out and edit or open as read only. 

In win7, I don't get that choice. The file simply opens, and I get a bar at the top which allows me to check out the file if I wanted to. 

How can I restore the functionality I used to have, with the dialog for checking out the file before opening it?

Designing and Developing High-Performance SharePoint Server 2007 Publishing Sites

Learn about the Web Content Management features in SharePoint Server 2007 that enable organizations to deploy Internet-facing publishing sites, and considerations for developers and designers when designing and building high-performance publishing sites.

Read an XML file, setting uname/pw, Accept, Content-Type

Hi; To read some REST XML data we need to set the username, password, and tell it for the http request Accept: application/xml Content-Type: application/xml Is there a sample anywhere showing how to do this? thanks - daveVery funny video - Reporting as a Metaphor

How to Read values from the XAML file at Runtime

Hi , I am creating a BuildDefinition from code and Loading the Build Process template. Now,I want to read and Validate the Build Process template parameter values.When I am debuggig,I could see that these are in Xaml format. Ex: I have some value "BuildType" as "Daily" in xaml file.I want to read these values and validate them I dont know what is the bestway to read the values.Can any one please let me know how can we do it. Thanks, Srikrishna              

Read a text file on another server from a web page

How do I read/write a text file on another server from a web page. I get the error "Access to the path '//Server2/mydatafiles/test.txt' is denied". I do not get the error if I am running the browser on the server where the files exist. I think I need to set permissions on the destination server in some way.   Can you help? Thanks  

Not able to read Config File data in Windows Service

While I'm trying to read connection string which is in App.Config file I'm getting error like "Object reference not set to an instance of an object." string strconn = System.Configuration.ConfigurationManager.AppSettings["ConnectionString"].ToString(); For the windows service where I need to keep app.config file and how to integrate that config file with the windows service exe file. Thanks in advance.
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend