.NET Tutorials, Forums, Interview Questions And Answers
Welcome :Guest
Sign In
Win Surprise Gifts!!!

Top 5 Contributors of the Month
Gaurav Pal
Post New Web Links

Crawl seems to be done, but content source says still crawling

Posted By:      Posted Date: May 22, 2011    Points: 0   Category :C#
I have a FULL crawl which took 8 hours to complete as the crawl log shows that it is not indexing content anymore, but the crawl status says "FUll Crawling" where it should say idle. Any ideas why this may be happening?

View Complete Post

More Related Resource Links

"Content for this URL is excluded by the server because a no-index attribute." in crawl logs


Hi All,

I am getting following error message in Crawl Logs

" Content for this URL is excluded by the server because a no-index attribute. "

Any help in this regard will be greatly appreciated.



adding a SharePoint content source using powershell

Hi I'm trying to create a content source using powershell and (I'm not a programmer) I've got this far -   [System.Reflection.Assembly]::Load("Microsoft.Office.Server.Search, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c") [System.Reflection.Assembly]::Load("Microsoft.Sharepoint, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c") [System.Reflection.Assembly]::Load("Microsoft.Office.Server, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c") $site = new-object Microsoft.SharePoint.SPSite("http://breakfast/ssp/admin") $searchContext = [Microsoft.Office.Server.Search.Administration.SearchContext]::GetContext($site)  $content = new-object Microsoft.Office.Server.Search.Administration.Content($searchContext) $Content.ContentSources; --> and now I'm stuck! I think I can use the Create method with type and name parameters but I can't work out how to translate this into powershell This is the code I'm looking at: ContentSource contentSource_com = content.ContentSources.Create(typeof(WebContentSource), "www.someurl.com"); WebContentSource webContentSource_com = (WebContentSource)contentSource_com; webContentSource_com.StartAddresses.Add(new Uri("http://www.someurl.com/index.html)); webContentSource_com.Update(); and $contentsource = $content.ContentSources.

Cannot crawl complex URL's without setting a site-wide rule to 'crawl as http content'. Help!

I have pages within a site that use a query string to provide dynamic data to the user (http://<site>/pages/example.aspx?id=1). I can get the content source to index these dynamic pages only if I create a rule which sets the root site (http://<site>/*) to 'include complex urls' and 'crawl sharepoint content as http content'. This is NOT acceptable as changing the crawling protocol from SharePoint's to HTTP will prevent any metadata from being collected on the indexed items. The managed metadata feature is a critical component to our SharePoint applications. To dispel any wondering of whether or not this is simply a configuration error on my part refer to http://social.technet.microsoft.com/Forums/en-US/sharepointsearch/thread/4ff26b26-84ab-4f5f-a14a-48ab7ec121d5 . The issue mentioned is my exact problem but the solution is unusable as I mentioned before. Keep in mind this is for an external publishing site and my search scope is being trimmed using content classes to only include documents/pages (STS_List_850 and STS_ListItem_DocumentLibrary). Creating a new web site content source and adding it to my scope presents 2 problems: duplicate content in scope and no content class defining it that I know of. What options do I have?

Can you set a crawl rule to restrict crawling at a specific depth?

Say we have a start address http://contoso.com/depth1/depth2/depth3/ and we only want to crawl from depth3 and beyond (depth4+ is fine). Is this possible to configure with a crawl rule?

SharePoint Designer 2010 External Content Types Data Source Already Exists in Business Data Connecti


I used the Secure Store to create connection in SharePoint 2010 Designer External Content Types Data Source Explorer.  Later I changed some column names in the table.  I deleted the connection from Data Source Explorer and recreated it. I right clicked on the table and selected Create All Operations. When I clicked the Finish button, I got "The system definition with the same name as this data source already exists in the Business Data Connectivity Metadata Store and it refers to a different data source.  Cannot complete operation generation.  Add a connection to this data source with a different name and try again." 

I re-created the Secure Store using a different name in SharePoint 2010 Central Administration.  I didn't find the option to choose the connection name in SharePoint 2010 Designer. After I clicked Add Connection, I entered the Database Server and the Database Name, selected Connect with Impersonated Window Identity, entered Secure Store name, and clicked OK. The connection created with the database name as before.

Search Results from one content source

Hello all,

I have created two Advanced Search pages that are designed to return the results from specific content sources. To achieve this, I have done the following to the Advanced Search box web part properties XML:
Under each <ResultType DisplayName> tag I have added 
AND ContentSource='nameofcontentsource'
to the <Query> tag. Before the "contentsource" code, there is a "IsDocument=1

In our DEV environment, this worked fine. The results being returned are only from the specific content source. In our TEST environment, I have done the same thing, but I get results from all content sources.

Can anyone help me determine what I have missed? The Core Results web part is not set up with a scope, and almost everything looks identical between the DEV & Test environment.

Crawling does not happing on a content database from set of DBs of my Web Application (MOSS)


Hi Team,

This is regarding indexing in MOSS content sources. I have an MOSS application(web site) configured as Content source for crawling. A user reported that his is not able to search on the content uploaded to his site. After performing quite a few checks and investigatations. We found below info in crawl log, we suspect that this is  crawling of all sites in Content Database(i.e., non are crawled and searchable). Atleast that is what we understand by trying search of quite a few sites residing in the databse.

Exception from HRESULT: 0x81070504 (There is no Web named "/sites/020045t".)

Also we find the site only in SiteMap table in SharePoint config Database only. We tried deletesite command with -force and -siteid switch, both were unsuccessful. Can someone tell us how to remove this orphan site/ entry in sitemap table? We are hesitent to delete the row in sitemap table since we do not impact of doing it from backend.

Thank you,


Unable to crawl the content form any of sites inside WebApplication


   I created two new web application, both the web application contain one site collection and inside contain site. In one of the web application search is working and another one Search is not working, am try to crawl the content from the website but it shows Zero item. But in other site collection it show the crawl item. Am in confused why it happed. Why am unable to crawl the content from the sites

FAST Search Connector won't crawl my Content Sources


Can anyone help me figure out why I am not able to crawl the Content Sources for the FAST Search Connector?

The error from the ULS viewer is:

Failed to connect to 1sv-sp2010.wirestone.internal:13391 Failed to initialize session with document engine: Unable to resolve Contentdistributor

I followed the install steps found at http://technet.microsoft.com/en-us/library/ff381267.aspx, including the post install validation.  FAST seems to be working in every way except the crawl.

The port number 13391 was found in Install_Info document. "Content Distributors (for GUI SSA creation):          1sv-sp2010.wirestone.internal:13391"



External Content Types + Search Service: Cannot crawl my external content type



I created an external content type by creating a new Visual Studio sharepoint project, and creating a content type (The default Entity1 content type). I created a profile page for it and everything, and when I drilled into the content type in central admin - BCS, I saw it wasn't marked as crawlable.

I saw this similar post: http://social.msdn.microsoft.com/forums/en-us/sharepoint2010general/thread/281BCEFD-59EC-41CC-B948-458A4BDA9E49

So I then created an external content type through SPD, leveraging the same code, and creating an external list and profile page. This time, when I drilled into the external content type in the BCS administration, it showed "Crawlable: Yes".

I figured at that point I was good to go, but when I went to my search service application -> Content Sources -> New Content Source and selected Line of Business Data, and selected BDC, it still says "No external data sources to choose from."

I verified also that the account for crawling has permissions for the external content type.

Are there any other things I should be looking for? From everything I read this should "just work" now :)



Content crawls fail after additional crawl component is added.


I have 2 vm sp servers in my farm connected to a fast box on dedicated hardware and noticed that the content crawls have been kind of sluggish. I added another crawl componet to my web front end and did the whole FAST cert import but when I try and run a crawl it fails and gives the top level error:  Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled.

I made sure that the search services are running under the under the same service account and that the service account has full read access to my web apps. I haven't been able to find too much documentation regarding mutliple crawl components so I figured I post out here.

Search Topology:
1 Admin Component
2 Crawl Components (APP Server / WFE Server)
1 Admin DB
1 Crawl DB

Perhaps I'm going about improving my crawl performance the wrong way, if that's the case any suggestions would be greatly appreciated.

Content Query result without link to source list

I've added a content query webpart to a page to show a multiple line text field from a list item. That works fine but the content is shown as hyperlink and if I click the content, the source item of the list is shown. I wanted to use the content query only to show the filtered content, without having the possibility to open the related item. Is that possible or is the content query webpart the wrong way to show content from different sources (lists) only as text in a page?

Sharepoint 2010 Search Results crawling content on Quick Launch


I have a custom search scope defined for a site.  When I search, it is returning results for text that it is finding on the quick launch menu.  (For example, if the listname 'Tickets' is on the Quick Launch and the user searches for 'Tickets', all of the pages in the site get returned because the word 'Tickets' is on the Quick Launch)

Is this correct?  Is there any way to override that? 

Access denied when Searc Service Application tries to crawl Sharepoint content


I have just set up a new SP2010 environtment(3 servers: WFE, App, SQL).

When I try to get my Search Service Application to crawl my main SP site and my MySite location, I get the following error in the crawl log:

"Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled."

Things I have checked:
I have ensured that the default access account has "Full Read" on the web application
-I set up crawl rules for both sources specifying a service account that has admin access to the content on those sites
-I logged in to the SP sites using the service account that the Service Application is using to crawl
-I even created a brand new search service application from scratch and got the exact same results

The only difference between this environment and my test environment, where search works just fine, is that this is the production environment and so it uses FQDN with a host header: http://portal.company.org.


Problem creating content source



We have a content source - http://Portal

This is a SharePoint site content source type

One of our customers on http://Portal/school/1234 is complaining about security trimming not working properly and we think this is because the crawl is timing out on this site

So I want to crawl just this site

I try to create a new content source but of course SharePoint won't let me because it is part of an existing content source

I can't just delete http://portal

So how do I create a content source so I can crawl just that site?

Any help would be greatly appreciated





protocol handler returns search result our of content source

I am using protocol handler to do a search from a content source. After a full crawl, a document was moved out of the Content Source, and then I do a full crawl again, now I get a warning message after crawl as "Deleted by the gatherer (This item was deleted because the crawler did not encounter it during the last full crawl)", which means the crawl is good. Then do a search, the search results still return the item out of content source. Is it a bug or as designed? How to remove this item from search results?

MySite crawling warning 'Content for this url is excluded by the server because a no-index attribut


Hi Team,

We have SharePoint farm consisting of 2 WFEs, 1 Application Server and SQL Cluster having 2 nodes.  We are are seeing many of the below warnings in the MySite crawling, more importantly we do not see many of document/s and personal blog not being picked up by the crawler and nothing is searchable. We do not have any Crawl rule on mysite url nor scope rule that points to personal site URL

Can some one help me with finding the reason/cause and solution if any one has come out of this situation.

Advance thank you.


Ramakrishna Pulipati SharePoint Consultant Bangalore, INDIA
ASP.NetWindows Application  .NET Framework  C#  VB.Net  ADO.Net  
Sql Server  SharePoint  Silverlight  Others  All   

Hall of Fame    Twitter   Terms of Service    Privacy Policy    Contact Us    Archives   Tell A Friend