2. Maximize your crawl budget.

Your crawl budget refers to the pages and resources on your site search bots will crawl.

Because crawl budget isn’t infinite, make sure you’re prioritizing your most important pages for crawling.

Here are a few tips to ensure that you’re maximizing your crawl budget:

  • Remove or canonicalize duplicate pages.
  • Fix or redirect any broken links.
  • Make sure your CSS and Javascript files are crawlable.
  • Check your crawl stats regularly and watch for sudden dips or increases.
  • Make sure any bot or page you’ve disallowed from crawling is meant to be blocked.
  • Keep your sitemap updated and submit it to the appropriate webmaster tools.
  • Prune your site of unnecessary or outdated content.
  • Watch out for dynamically generated URLs, which can make the number of pages on your site skyrocket.
en_GBEnglish