.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
Gaurav Pal
Post New Web Links

DataView external datasource filter large dataset?

Posted By:      Posted Date: May 22, 2011    Points: 0   Category :C#

Basically, I have combined 3 call logs from verizon (comma-delimited), t-mobile (comma-delimited), and a internal system (SQL Server) into one table.  Built a dataview to a table that right now with test data has 300,000 rows and will have lots and lots more and all fields are varchar() in database even date and time.  Since I have 300,000 rows, I paged the data at 1000 as this would take a long time for everything to load.

So building the dataview was a challenge to get working but figured that out.  Now I am trying to build searching functions so that someone can search and filter the results on date, time, and really any of the fields.  I attempted a form webpart and setup parameter for search and it kinda works when I have a few test rows. One thing I noticed was if the data was on another page it did not include those in my filter and I think if I paged over eventually I would find it.

So my ultimate question is how would I add filtering/searching into this dataview.  Could I dynamically adjust the query for the dataview like on said date range based on dates in form web part?  Is there a better method for anything I am trying todo?

Thanks, Matt

View Complete Post

More Related Resource Links

Persist large dataset across ASP.NET pages. No database


Can anyone give me suggestions on how to implement the following requirement?

1. A online wizard (step-by step operation) processes user requests across several ASP.NET pages.  The session data is stored in memory until the user submits the request.  Once the request is submitted, a XML file is created and the data are passed to XML.

# Case Study: The parent page has a "select" button.  Click "select" will pop up a child page.  The users can select 1000+ items from a datagrid in the child page.  Once the user clicks "confirm", the selected items in the child page will be populated to the selected item gridveiw in the parent page.  The user can keep selecting more in the child page and the selected items will be appended to the gridview in the parent page.


- What's the best way to extract strings from a text file and convert to a dataset in memory and then present it in a datagrid in UI?

- What's the best way to persist and manipulate a large dataset across pages?  Session? or ViewState? Can it be accomplished by LINQ?

The app doesn't interact with a DB.  Once the request is submitted, it will convert the in memory data to a XML file.  What technology can be used to accomplished this?&nb

what is large and smal dataset

what is a large and small dataset /.. how can we say that it is large or small data set. can u xplain me clearly. and also send me some dat sets..my mail id is kalyan9029@gmail.com... thanking u .. kalyan

report builder 3.0 How do I set a dataset property filter to null

seems to be every other imaginable option ...  "NULL" is not accepted, [NULL] returns an error ... this should be obvious!

Create drop-down filter for External List

I hope this is an easy question. I will have an external list of a bunch of people.  There are several ways I want to display this data: All of them, Only the ones that have a value of 1-4 in the status field, Only the ones that have any other value in the status field, and people that have a value of 1-4 in the status field, but are at a certain location. I want to be able to put a drop-down filter above the list where people can select that. Could someone please point me in the right direction? Time is of the essence.  

Out-of-box filter stops working as soon as I "Configure Datasource"

I'm running VWD 2008 Express with ASP.NET 3.5 I realize now that 4 has much better filtering controls, but I didn't realize that when I started my project.  Anyway, when I use the standard List.aspx to display a table, the filtering works just fine (yes, it's only foreign keys and booleans, but it works).  I then modify the linq datasource so that the gridview only displays the columns that I want, and the filter function immediately stops working.  You can choose from the drop down lists, but nothing happens.  As an aside, I don't want to set scaffolding to false for those columns because I do want them to show up in the Edit.aspx page.  My understanding is that scaffolding settings would keep the field from showing up in either page. What do I need to include in either the gridview or the linqdatasource to get the filter working again?   Thanks!

SharePoint Calendar webpart with external datasource

hi all, I want to show use Sharepoint calendar webaprt and connect it to my external source where records are stored DateWise..is it possible to connect

Querying, getting the results and re-using the results on a different datasource and dataset. Is it

Hello all,  I have a dataset that is running a very simple query, pulling an OrderID field based on the TransactionID parameter (User Input).  This dataset is pointed to Database_A. SELECT OrderID FROM Orders WITH (NOLOCK) WHERE OrderID = @TransactionID   Now, this query above might return one of more records. I need to query another system (Database_B) using the results from this query. I need to store the OrderIDs that I got somewhere and check them against another system and display the results on my report. Please note - Database_B is located on a totally different server.  Can you guys think of a way of doing this? I'm using SSRS 2008, not 2008 R2.

Filter DataView by Value from Active Directory

I have a web part page with a dataview that is configured to display as a single item form.  The list that the dataview is pulling from will include many items, I just want to display the one item that is related to the current user so that I can populate some of the fields via javascript. The catch is, I need to match on a value in Active Directory other than logon name, username etc.  We are using a field in Active Directory to track Employee ID which is the unique ID that is used by our Finance/HR system.  I am able to use to the SPSWC:profilepropertyvalue control to display the Employee ID, I just cannot figure out how to pass that value to the dataview as a parameter to filter on. I don't have the option of a query string parameter because this is the first page the user will come to in this scenario.  I have tried both the Control and Form types of parameter but the value is never passed.  I currently have the SPSWC:profilepropertyvalue control in a div tag (I thought that if I gave the tag an ID I could reference it that way), but so far I've had no luck.  I have also tried using Javascript to take the value from the control within the div tag and assign it to a textbox, but I still cannot get the value to pass to the dataview. Here is the code for the SPSWC:profilepropertyvalue control: <table> <tr><td> <SP

problem in Transferring a large dataset to remote machine by using the NetTcpBinding

i am developing an WCF application, in which i have to transfer a large dataset to the remote machine!! for that im using nettcpbinding in WCF but i got this exception ' The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '10:29:59.4140000'. at my sender side.   Sender/Client side code   // Object for sending to remote client using (_dpParameters = new DPParameters()) { _dpParameters.SequenceFieldName = "Sequence"; _dpParameters.Size = 10; _dpParameters.SubjectsForFiles = "S,Y,T"; _dpParameters.SubjectNames.Equals(SubjectNames); _dpParameters.FilePath = " E:\\Source\\Phospho.xml"; // this dataset can be in GBs but for testing me using jst 6.78mb DataSet ds = new DataSet(); ds.ReadXml(_dpParameters.FilePath); _dpParameters.Usd = ds.Tables[0]; } /// This is the binding settings NetTcpBinding tcpBinding = new NetTcpBinding(); tcpBinding.ReceiveTimeout = new TimeSpan(10, 30, 00); tcpBinding.SendTimeout = new TimeSpan(10, 30, 00); tcpBinding.MaxBufferSize.Equals(67108864); tcpBinding.MaxBufferPoolSize.Equals(67108864); tcpBinding.MaxReceivedMessageSize.E

Report Builder 2.0 Filter Dataset NOT LIKE

I need a NOT LIKE filter in a dataset in ReportBuilder2.0. There is only a LIKE Operator to filter a dataset. The users who worked at the end with the ReportBuilder does not write custom code, iif or something else. Is there an easy way?

Passing a Text Value as DataView Filter Parameter

I have a dataview to which I would like to pass a text value for use as a filter.  My text value is on the same page within a DIV tag that I have given an ID.  When I view the page in the browser the text is actually rendered in a SPAN tag, within the DIV tag.  I have tried the ID for both the DIV and SPAN tags when setting up the parameter, and have tried using both a parameter type of Control and Form, but nothing I've tried is working. Any thoughts?  Thanks, Chanda

Filter External List by a range of data

I have an external list web part that I am currently using a choice filter on to filter by a certain status.  The options are yes, no, or (empty).  However, I would like to be able to add an additional option to filter where the person has the previous status set to no, but has a value in a different field greater than 90. I realize that the choice filter probably wouldn't do this, and that I could do this easily with separate list views, where I have some sort of web part that can change the list views on the fly. I would think I could use one of those list filters to do this, but when I tried to do that for my original problem, I was unable to connect the filter to my external list. I could connect the list view filter to any internal list without an issue however.  Somebody commented on my old thread that I posted as cptInsane0 that I could just make an infopath view form that has buttons that do this.  I have dabbled with infopath a little bit, but I wasn't able to get infopath to connect to my external list either.  I first had to connect to an internal one, and then for some reason I could add the external list as an alternate data source. I can't be the first person who has had this issue. Could someone point me in the right direction?

Filter and Sort on fields in a joined dataview in SharePoint Designer


I need to create a data view that joins and displays data from two lists, and to sort and group on all of the fields displayed, not just fields from the main list.

I created linked datasource from two of my lists, joined them on a common field, and first inserted fields from the main list. Then I inserted fields from the other list as a Joined Subview,  which worked just fine, but now I need to be able to sort and filter on fields in that joined list, for which there doesn't seem to be an option.

Does anybody know whether it's possible and/or any workarounds/solutions for this? Basically, Is there a way to create a joined view of two lists where all fields are displayed in separate columns, and not in a table within a cell as in a case with joined subview?

Any comments are highly appreciated.


How to set Dataset as datasource of a gridview in aspx using vb



i am using a gridview control in my aspx page. I want to set a Dataset as the datasource of the grid at programming end. But after filling the data in dataset and setting the data source property to Dataset grid is showing nothing.


If anyone can have any suggestions than please reply.


Sending Large Dataset to Webservice


Dear all,

I have one problem is that when i send large dataset containing byte array.. then webservice not respond well..

it shows error like underlying connection was closed...

I mark that when dataset size goes some large then only problem occurs otherwise it works well..

Please give me solution of how to send large dataset over web service...



Dataview filter

Hi ALL ,

I have tried the XSLT for random display.
<xsl:variable name="Rows" select="/dsQueryResponse/Rows/Row[@ID=(substring(ddwrt:FormatDateTime(ddwrt:TodayIso(),1033,':mm:ss'),6,2) mod count(//Rows/Row))+1]"/>

In the designer when I refresh the data source it displays correctly ie it rotates between all items in the list.
But when i view in the browser it picks from first 10 items in the list for display, its failed to pick above 10.

So it rotates within 10 items.

Can anyone help me out


Large Dataset Downloaded Before Rendering Report's First Page



I'm hoping to understand the behavior of SSRS a bit better in the following scenario (SSRS 2008).  A single matrix (tablix) exists on the report.  No other objects are on the report.  That matrix is populated with 1.1 million rows of data totaling about 200MB.  Why does SSRS download the entire dataset prior to rendering the report's first page or am I missing something here?

When previewing the report, the local .data file bloats to 200MB indicating all dataset rows are being returned.  The matrix does not contain any sorting, all cell "can grow" and "can shrink" properties are set to false, no expressions are used anywhere, and even the table's column data types from which the data is being pulled are appropriate.  In other words, I believe I've reduced storage and CPU needs while eliminating elements that add overhead when rendering a report.

The specific scenario here is a researcher previewing the first few pages of data then choosing to download the entire dataset.  MS documentation I've read refers to "soft page breaks" when rendering as HTML and left me with the impression that perhaps SSRS needs to know the entire height of the dataset before it'll slice the report into pages.  It's odd to me though that it needs to download 1.1 millions rows of data befor

ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend