.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
Gaurav Pal
Post New Web Links

SQL paralllel execution performance improvement

Posted By:      Posted Date: October 04, 2010    Points: 0   Category :Sql Server


I have an SSIS package which populates almost all my dimension table.
The package populates around 20 tables..The package does not deal with much of logic..It just reads staging and writes into destination for 80 percent of those 20 tables. All tables in staging and destination are of Nvarchar and Bigint type..
My package has 3 big containers which has multiple containers inside them for each table..every small container for the table deals with truncating,inserting a row for unknown member and a data flow task which just reads staging and populates the staging.
Now the problem..we have 6 very large Junk dimensions..which we need at this point of time. ( don want to talk abt design stuff).The last big container fills one of the big dimension..e.g say sales order.which has 40 M records..we have 3 parallel processing which look up to those 40M records and another 80 M records to populate other 4 large dimension. I have observed a significant decrease in perf once it reaches ther.It just fails to swap buffers and data tranfer gets very slow and at last it fails..Those DFT includes union,derived column tranformation, split and  Lookup ( which takes almost all mem)..i have seen the memory consumption increases rapidly)..That specific container takes almost 18 GB of space for look up and we have only 40 GB of space.. Is there a way to control

View Complete Post

More Related Resource Links

SSIS flat file performance improvement


I have 10 text files which are on average of 6  GB in size. Out of those 10, 4 files average up to 10 GB in size.

We have 10 SSIS packages which loads those text files onto 2 staging tables. Each 5 of them loads 1 table.

Inside the package, its only a flat file source, data conversion for all columns from source, and write into the destination table.

And this process takes around 5 hours to complete. the size of 2 tables after the load completes is 60 M and 40 M each.


The other thing which I have observed is all the packages are scheduled through one schedule and the schedule  have 10 diff sql server jobs( for 10 packages). Does it have an impact on the processing time ,as all these jobs will be

Performance Counter SQLServer:Cache Manager:Cache Pages (Execution Contexts) = 4294967295


Performance Counter


SQLServer:Cache Manager:Cache Pages (Execution Contexts) = 4294967295


What this might mean?


It overflows, but what it might mean …

"Performance statistics" event: execution plan in "binary xml"

The SQL Trace "Performance statistics" event returns for EventSubClass 2 (Batch:Plan) the execution plan in "binary xml" format in the BinaryData column. How do I convert that data into readable XML?

Sharepoint custom code Performance improvement



We are writing some code with respect to one project that uses the SharePoint 2007, K2, .net 2008, office 2007, we are working with the outlook Add-In and Word Add-in and we are working in this project since last 2 year, so we added a lot of custom code.

As now we are completed the Functional code as well as the testing, testing team started doing the Performance testing where we are finding our system is really slow.

So the question is-

What we can do to improve the System performance, when I am writing this I do aware I am asking more of guidelines and want to learn from your experiences.

Please suggest.

Thank you so much in advance.


ASP.NET Best Practices for High Performance Applications

This article lists the techniques that you can use to maximize the performance of your ASP.NET applications. It provides common issues, design guidelines, and coding tips to build optimal and robust solutions.

Top .NET Performance Problems and how to avoid them

Every time I work with one of our .NET customers to help them with managing their application performance I come across the same problems as seen with other clients before: lots of ADO.NET queries, many hidden exceptions in core or 3rd party .NET libraries, slow 3rd party components, inefficient custom code

High-Performance .NET Application Development & Architecture

It has always been a goal of project architects to plan an effective strategy from the ground up in regards to an new application. All relevant factors are taken into consideration with respect to the application, from its design and layout to a functional website infrastructure. Pre-.NET strategies and design guidelines still effective now were developed with Microsoft's DNA (Distributed interNet Application) platform. This model successfully served the purpose of architecting N(any number of)-Tier (levels) applications. In its basic sense, as in most robust, distributed applications, you'll architect 3 main layers or Tiers: presentation, business rules and data access.

How to improve the performance of ASP.NET MVC web applications

It is an in depth analysis of a Digg-like site and how it went from serving (on a test machine) 6 req/sec to 390 req/sec.

The biggest gain, 74 req/sec to 390 req/sec happened when he introduced data caching and another 25 req/sec where gained when he introduced the compilation of LINQ queries.

High Performance ASP.NET - Speeding Up ASP.NET Pages

have been writing a series of blog posts, which I have named High Performance ASP.NET Websites Made Easy! There is no rhyme or reason to the order of these posts and certainly can be read in any order:

Developing High Performance and Scalable ASP.NET Websites
Avoid Chatty Interfaces Between the Tiers in Your ASP.NET Web Application
ASP.NET Page Profiling - Page Tracing - High Performance and Scalable ASP.NET Websites Made Easy
Web Applications: N-Tier vs. N-Layer - Benefits and Trade-Offs
Query Analyzer - Sql Server Database Indexes and Execution Plans

Improving Application Performance in .Net

Application performance has always been a concern for Web Application developers. This article contains guidelines and tips for maximizing application performance in ASP.NET.

Tips to Improve Your ASP.NET Web site performance

ASP.NET has been developed with lots of features which makes developer life easy. Start from ViewState, PostBack, Server Controls , User Controls and etc. All of them can make your web development experience fun and easy, but improper use of them can cause your site performance slowdown. Most of the controls generate overhead to the client and server. Therefore in this article, I I will list out about things you can do and don't to improve your ASP.NET website performance.

10 Tips for Writing High-Performance Web Applications

Writing a Web application with ASP.NET is unbelievably easy. So easy, many developers don't take the time to structure their applications for great performance. In this article, I'm going to present 10 tips for writing high-performance Web apps. I'm not limiting my comments to ASP.NET applications because they are just one subset of Web applications. This article won't be the definitive guide for performance-tuning Web applications-an entire book could easily be devoted to that. Instead, think of this as a good place to start.

Performance Tests: Precise Run Time Measurements with System.Diagnostics.Stopwatch

Everybody who does performance optimization stumbles sooner or later over the Stopwatch class in the System.Diagnostics namespace. And everybody has noticed that the measurements of the same function on the same computer can differ 25% -30% in run time. This article shows how single threaded test programs must be designed to get an accuracy of 0.1% - 0.2% out of the Stopwatch class. With this accuracy, algorithms can be tested and compared.

Best Coding Techniques To Improve Performance for SharePoint Applications

As more developers write custom code by using the SharePoint Object Model, they encounter common issues that can affect application performance.

The following areas reflect the main issues encountered by developers as they write custom code by using the SharePoint object model:

§ Disposing of SharePoint objects

§ Caching data and objects

§ Writing code that is scalable

LINQ to SQL Performance Considerations

As developers we have a wide array of tools at our disposal. One of the most time consuming, but most beneficial tasks we must undertake is to determine which of those tools to use for a given situation.

Performance tuning tips for database developers

Performance tuning is not easy and there aren't any silver bullets, but you can go a surprisingly long way with a few basic guidelines.

In theory, performance tuning is done by a DBA. But in practice, the DBA is not going to have time to scrutinize every change made to a stored procedure. Learning to do basic tuning might save you from reworking code late in the game.

Below is my list of the top 15 things I believe developers should do as a matter of course to tune performance when coding. These are the low hanging fruit of SQL Server performance - they are easy to do and often have a substantial impact. Doing these won't guarantee lightening fast performance, but it won't be slow either.
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend