.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
Gaurav Pal
Post New Web Links

SSAS for heavy network data to serve around 12-15 million rows per day

Posted By:      Posted Date: October 04, 2010    Points: 0   Category :Sql Server

I am a fresher with the analytics.

Presently, working on a system which tracks the data from Network giants like Bing, Google, Yahoo, Facebook etc. depending upon the profile of customers using our system.

So we our tracking data of around 5-6 million rows in a day. Apart from that, dimensions like Search Keywords and all also contain data of around 50-60 million of rows. Although, daily addition will be low in dimension tables, but it will keep on increasing as customers will increase. Tracking data is expected to grow beyond 15 million.

To provide analytics, I must have minimum of 3 months of data in a cube. To provide trends, it should increase to around 1-4 years (in our future road map).

Presently, I am suffering with fatal issues while processing a cube, like, "The operation cannot be completed because the memory quota estimate xxxx exceeds the available system memory yyyy" - while processing dimensions

Please guide with some good thought process how to manage such a huge data and provide analytics over it. As well throw some light over the approach to follow for testing cubes and data loading packages of ssis.

Please provide the links of articles/videos/blogs & name of books to discuss and decide design the analytics for data-extensive systems. Any kind of reading mater

View Complete Post

More Related Resource Links

MS SQL Server: Search All Tables, Columns & Rows For Data or Keyword Query

If you need to search your entire database for specific data, this query will come in handy.

So when a client needs a custom report or some sort of custom development using Great Plains, most of the time I will have to track down the data in the system by running this query and find the table(s) it is in.

Join 2 flat file data flows - retain unmatched rows

I have two data flows from two separate flat files. They may contain matching IDs (account number), in this case specific data from each flow should be used to create one row. When there is no match, the rows would stand on their own. At the end of the flow, I need both flows combined into one flow, with one record for each key record (account number). If I were able to use a look-up, I could easily union the no-match data flow back into the match data flow and have the desired result. I cannot use a look-up, since the source is flat files, but this is exactly the functionality I am trying to achieve. Solutions I want to avoid: staging tables, and cache transformations. Any ideas are appreciated.

OLE DB Error: Network-Related or Instance-Specific while Deploying SSAS 2008 Cube

Hi, I encountered the below listed error while attempting to deploy SSAS 2008 DB: Error 1 OLE DB error: OLE DB or ODBC error: Login timeout expired; HYT00; A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.; 08001; Named Pipes Provider: Could not open a connection to SQL Server [5]. ; 08001.  0 0  I am running a standalone MSSQL 2008 named instance on Windows Server 2008 R2. Here are the steps I have taken but yet the issue is not resolve: 1) Enable Remote connections on the server 2) Enable TCP\IP 3) SQL Browser is Running 4) Set Firewall Inbound Exceptions for SQL Browser, Default Port 1433, UDP Port 1434 5) Using Management Studio I can successfully connect to the MSSQL named instance from the app server 6) I can ping I MSSQL box from the app server 7) Read and implemented all steps in the MSDN post below: http://blogs.msdn.com/b/sql_protocols/archive/2007/05/13/sql-network-interfaces-error-26-error-locating-server-instance-specified.aspx Thanks for your help.

Getting filtered data from Role based SSAS security

Hi everyone,I've got a heap of reports that are based on various SSAS cubes. I have roles defined on these cubes that restrict data via certain dimensions. Question is, will these restrictions filter through to the report...ie, if I have a sales person restricted in the SSAS cube to only see sales against their territory (restricted in the Territory dimension), when they run the report will it filter the result based on their SSAS credentials and only show the data they have access to (even though the SSRS report has no direct filters or parameters applied)?Cheers for any help!!

Expand rows in a data grid to show details in the same page

<asp:templatecolumn headertext="Purchase Order"> <headerstyle horizontalalign="center"></headerstyle> <itemstyle horizontalalign="center"></itemstyle> <itemtemplate> <%# DataBinder.Eval(Container.DataItem, "PurchaseOrder") %> </itemtemplate> </asp:templatecolumn> <asp:templatecolumn headertext="Order"> <headerstyle horizontalalign="center"></headerstyle> <itemstyle horizontalalign="center"></itemstyle> <itemtemplate> <a href="OrderDetail.aspx?Order=<%# DataBinder.Eval(Container.DataItem, "OrderNumber")%>"> <%# DataBinder.Eval(Container.DataItem, "OrderNumber") %> </a> </itemtemplate> </asp:templatecolumn> <asp:templatecolumn headertext="Ack Date"> <headerstyle horizontalalign="center"></headerstyle> <itemstyle horizontalalign="center"></itemstyle> <itemtemplate> <%# DataBinder.Eval(Container.DataItem, "AcknowledgeDate") %> </itemtemplate> </asp:templ

Browse SSAS Cube Data From Your iPhone

Does the world want an iPhone app that allows you to connect, browse and pivot SSAS cube data?  What features would that app have?  Is there existing apps that are close?  Looking for others thoughts on this concept.  Thanks in advance!

SSAS 2008 attritube hierarchy doesn't group records and repeat rows

We are having problems with the dimension attributes in lower level hierarchies not grouping under 1 single level. Here is the hierarchy: VP    College      Department         Departmental Course            Course Level               Course Number The top 3 levels are grouping correctly without duplicate rows and they don't have compoud keys. The lowest 3 levels are not grouped correctly and they have compounds keys because the attributes are not unique by themself. The result of the hierarchy looks like this: VP Academic Affairs    Business School of Management        International Business School            Mgm101                 Lower                       Course Number 123            Mgm101                Upper                

SSAS 2008 Metadata Repository data - User Knowledge base

Hello, We have the SQL Server 2008 Analysis services instance with some databases, cubes, Dimensions, Measure groups, partitions, etc. The business users are find difficulty to search for a particular measure. I mean, it is difficult for them locate a measure from the available cubes; again finding the respective Measure Group/ Folder is not an easy task for them. Also, they need to know the source system of a particular attribute in a dimension, formula for the calculated measure and etc. So, we have planned to develop a knowledge repository, which needs to store the business metadata for the SSAS databases. The user needs to be access this information using the SharePoint site and should help them locate and understand the data. The User Interface needs to be user friendly too. I need your valuable suggestions for the design of this system. Note: I have referenced the below articles http://dwbi1.wordpress.com/2010/01/01/ssas-dmv-dynamic-management-view/ http://social.msdn.microsoft.com/Forums/en-US/sqlanalysisservices/thread/c610850f-16b4-4eaf-9b91-a2d32816f40e

SSAS C# Amomd Dimension Data

Hi, i have the following problem. I'm able to connect to the cube, get the Meassure Names, Dimension Names and Dimension Attribute Names. But how can i get the Dimension Attribute Data (MEMBERS) in a List or Array. I mean how can i acces the data of an Dimension Attribute like in MDX: Dimension -> Attribute -> Attribute Data [Dim Customer].[First Name].&[Frank] I would be great to get/recive all Data an put them in a Table(List/Array/DataTable/...). Thx for your time and your help

ASP.NET Events and Pageload not firing when Datagrid containing large number of rows of data

Hi All, I have a datagrid in aspx page.Inside of datagrid i am using around 15 controls such as Button,dropdownlist and text box controls.Once the datagrid binds then the events in the aspx page not firing.Issue occured when the number of rowsgreater than 500.The number of rows less then its works fine.If anybody knows the solution please let me know."Platform i am working on ASP.NET1.1"Regards Hareesh

data mining using nural network

<!-- [if gte mso 9]><xml> <w:WordDocument> <w:View>Normal</w:View> <w:Zoom>0</w:Zoom> <w:TrackMoves/> <w:TrackFormatting/> <w:PunctuationKerning/> <w:ValidateAgainstSchemas/> <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid> <w:IgnoreMixedContent>false</w:IgnoreMixedContent> <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText> <w:DoNotPromoteQF/> <w:LidThemeOther>EN-US</w:LidThemeOther> <w:LidThemeAsian>X-NONE</w:LidThemeAsian> <w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript> <w:Compatibility> <w:BreakWrappedTables/> <w:SnapToGridInCell/> <w:WrapTextWithPunct/> <w:UseAsianBreakRules/> <w:DontGrowAutofit/> <w:SplitPgBreakAndParaMark/> <w:DontVertAlignCellWithSp/> <w:DontBreakConstrainedForcedTables/> <w:DontVertAlignInTxbx/> <w:Word11KerningPairs/> <w:CachedColBalance/> </w:Compatibility> <w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel> <m:mathPr> <m:mathFont m:val="Cambria Math"/> <m:brkBin m:val="before"/> <m:brkBinSub m:val="--"/> <m:smallFrac m:val="off"/> <m:dispDef/> <m:lMargin m:val="0"/> <m:rMargin m:val="0"

data mining using nural network

<!-- [if gte mso 9]><xml> <w:WordDocument> <w:View>Normal</w:View> <w:Zoom>0</w:Zoom> <w:TrackMoves/> <w:TrackFormatting/> <w:PunctuationKerning/> <w:ValidateAgainstSchemas/> <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid> <w:IgnoreMixedContent>false</w:IgnoreMixedContent> <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText> <w:DoNotPromoteQF/> <w:LidThemeOther>EN-US</w:LidThemeOther> <w:LidThemeAsian>X-NONE</w:LidThemeAsian> <w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript> <w:Compatibility> <w:BreakWrappedTables/> <w:SnapToGridInCell/> <w:WrapTextWithPunct/> <w:UseAsianBreakRules/> <w:DontGrowAutofit/> <w:SplitPgBreakAndParaMark/> <w:DontVertAlignCellWithSp/> <w:DontBreakConstrainedForcedTables/> <w:DontVertAlignInTxbx/> <w:Word11KerningPairs/> <w:CachedColBalance/> </w:Compatibility> <w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel> <m:mathPr> <m:mathFont m:val="Cambria Math"/> <m:brkBin m:val="before"/> <m:brkBinSub m:val="--"/> <m:smallFrac m:val="off"/> <m:dispDef/> <m:lMargin m:val="0"/> <m:rMargin m:val="0"

I need a grouping and cumulative sum of data(TFS SSAS Report)

It is a MDX query (TFS Warehouse)     WITH MEMBER [Measures].[Date Value] AS     [Date].[Date].CurrentMember.Member_Value     MEMBER [Measures].[CalculatedRoughOrderOfMagnitude] AS Iif(     [Work Item].[Microsoft_VSTS_Common_RoughOrderOfMagnitude].CURRENTMEMBER IS [Work Item].[Microsoft_VSTS_Common_RoughOrderOfMagnitude].[All].UNKNOWNMEMBER,     2,     2^([Work Item].[Microsoft_VSTS_Common_RoughOrderOfMagnitude].CURRENTMEMBER.MEMBER_VALUE+1)     )   MEMBER [Measures].[ActiveRoughOrderOfMagnitude] AS Iif( [Work Item].[System_State].CURRENTMEMBER IS StrToMember("[Work Item].[System_State].[System_State].[" + @Active + "]") , [Measures].[CalculatedRoughOrderOfMagnitude] ,0 )   MEMBER [Measures].[ResolvedRoughOrderOfMagnitude] AS Iif( [Work Item].[System_State].CURRENTMEMBER IS StrToMember("[Work Item].[System_State].[System_State].[" + @Resolved + "]"), [Measures].[CalculatedRoughOrderOfMagnitude] ,0 )   MEMBER [Measures].[ClosedRoughOrderOfMagnitude] AS Iif( [Work Item].[System_State].CURRENTMEMBER IS StrToMember("[Work Item].[System_State].[System_State].[" + @Closed + "]"), [Measures].[CalculatedRoughOrderOfMagnitude] ,0 )           SELECT {  &nbs

Hiding rows/columns completely in SSRS tablix even when they have data

I have several columns and rows in my tablix.  I don't want to see all of them at once.  I am using a parameter that controls the visibility of the columns or rows of the tablix.  The issue is that it hides but the white space is left.  How do i take out that white space?

How to enable caching on SSRS that using SSAS cube to bind data


Hi All 

 How to set Cach refersh option for a SSRS report that using SSAS cube to bind data 

when i tried to store credential in data source getting error as show below 

"Non-Windows user credentials were supplied for a non-http connection to Analysis Services. The Microsoft SQL Server Analysis Services data extension requires Windows Integrated Security unless connecting via http or https."





SSAS Data Member Issue


Hi all,

I am using SQL 2005 - SSAS- Calculations tab to create data member. I am using two numeric columns from the fact table to create a data member, after that I use calculation properties "associated measure group' to link with the fact table. Now it is showing in perspective tab at the bottom.

But for some reason it's NOT SHOWING IN THE FACT TABLE in  browser tab where I am viewing some resutls.

Please let me know what I did wrong? Or if I missed something?




Convert excel rows wise data into column wise


public static string connectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + path + ";Extended Properties=Excel 12.0;";

               OleDbDataAdapter adapter = new OleDbDataAdapter();

            OleDbCommand selectCommand = new OleDbCommand("SELECT * FROM [Sheet1$]");

            OleDbConnection connection = new OleDbConnection(connectionString);

            selectCommand.Connection = connection;

            adapter.SelectCommand = selectCommand;

            DataSet cities = new DataSet();


ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend