Crawling, Indexing and Ranking Explained
Search engines exist to discover and organize the internet’s content. And in order to offer the most relevant attorney results for prospective clients’ legal needs, your firm’s content must be visible to the search engines
If your attorney website cannot be found, it will be impossible to land on the search engine results page. Search engines operate in three separate categories that determine how your law firm is found.
1. Crawl: Crawling is the function that scours the Internet for content. While crawling, search engines review the code and content of the URLs they find.
2. Index: Indexing is the process of storing and organizing the content that is found during the crawling process. Once a page indexed, it becomes a viable option to be displayed as a search result relevant to the query.
3. Rank: Rankings are displayed in the order that is most relevant to the search criteria entered.
What Is Search Engine Crawling?
Search engines have a team of robots that are referred to as spiders or crawlers. They are designed to “crawl” from page to page to find updated content including websites, images, and videos, which is discovered by links.
What Is A Search Engine Index?
As the crawlers follow the links, the new attorney content is added to the Googlebot index, called Caffeine. Caffeine is a massive index database of the discovered URLs, which will be retrieved when a searcher is looking for a lawyer that practices in their area of need, whether it is personal injury or criminal law and bankruptcy or family law.
What Is A Search Engine Rank?
When prospective clients search for attorneys, the search engines will deliver results based on the query and rank the sites by their relevance to the search that was performed.
This means when someone searches “criminal defense attorney” the search engines will rank the firms that fit the criteria and rank them according to their relevance.
When your attorney content can be found and indexed by searchers, the ranking follows.
Crawling: Can Search Engines Find Your Attorney Pages?
It is important to know whether Google is crawling your website and finding all the pages you want it to.
A quick check of your website’s indexed pages will allow you to learn which return results the search giant displays.
Here’s how:
- Login to Google
- Enter “site:yourdomain.com” into the search bar
This will return the results Google has in its index for your attorney website. The results will not be exact, but it will provide a collective look at how and which indexed pages are returned in the search engine results.
You should also monitor your law firm’s Index Coverage report in the Google Search Console, which is free when you establish an account – if you haven’t already.
What If My Law Firm Is Not Showing Up Anywhere In The Search Results?
There are several reasons why your law firm may not be showing up on the search engine results page, including that your attorney website is:
- Brand new, and has not been exposed to the crawling process
- Is not linked through any external websites
- Hard for the robots to crawl effectively because of poor navigation
- Contains search engine blocking code crawler directives
- Has been penalized by Google for spam
Tell Search Engines How To Crawl Your Attorney Website
If you ran the “site:domain.com” search operator, or viewed your indexed attorney pages in the Google Search Console and found that some of your firm’s important pages are improperly indexed, there are a few ways to tell Googlebot how to better crawl your website’s content by avoiding some common navigation mistakes.
Those oversights can include:
- Mobile navigation that shows different results than desktop navigation
- Non-HTML menu items appearing in the website’s navigation, including JavaScript-enabled navigation
- Personalizing or developing unique navigation for a specific audience over others, which could appear to be cloaking
- Forgetting to link to a primary page on your website using your navigation
Proper Attorney Website Indexing:
How Do Search Engines Interpret And Store Your Law Firm’s Web Pages?
Your attorney website has been crawled, and when the information is indexed it is stored for future use. Just because your site has been discovered does not mean the information is worthy of keeping – or indexing.
So, why do website pages get removed from the index?
Several reasons, including the URL, has:
- Returned the page or site as a “not found” or “server” error
- A “noindex” meta tag added, which intentionally instructs the search engine to avoid indexing
- Been manually penalized for violating the search engine’s Webmaster Guidelines, and removed from the index
- Been blocked from crawling with the addition of a password required before visitors enter the page
What If My Attorney Website Was Previously Indexed, But Isn’t Any Longer?
There are a couple of tools that will allow you to search for pages that may have been indexed before, determine their current status, and add them to the index.
- The URL Inspection tool will show you the current status of the website’s pages
- Fetch as Google will allow you to use a “Request Indexing” feature to submit individual pages to the index
The latter will also allow you to use a tool to understand if there are any issues with how Google interprets your pages.
How Do Search Engines Work Google Webmaster Guidelines Keyword Research for Lawyers Content Creation for Lawyers Technical SEO for Lawyers Link Building for Lawyers