We can add a crawl rule in SharePoint search to include or remove the content in a specific path while crawling the content. We have to provide the credentials to crawl the data. We can test/delete/reorder the content by using crawl rules.
To create or edit crawl rule we have to follow the steps below,
Navigate Application management in SharePoint central Administration and click on manage Service Applications link.
Select Search service application to add the crawl rule. We'll navigate to Search service application. In Search Service Administration page, click on Crawl Rules in Crawling section.
To create new crawl rule click on New Crawl Rule.
In Add Crawl Rule page, Path section enter path of the rule. We can use regular expressions in the path box. To use regular expressions select "Use regular expression syntax for matching this rule
In Crawl Configuration section, to exclude all the items in the specified path select Exclude all items in this path. Select Exclude complex URLs to exclude URLs that containing parameters use? notation. To include all items in the path to be crawled select Include all items in this path option.