Direct answer

Crawling fetches a URL, indexing processes and selects it, and ranking chooses indexed material for a query; a problem at one stage is not proof of a problem at another.

What to remember
  • A URL can be discovered without being crawled, crawled without being indexed, and indexed without ranking for a target query.
  • A live test shows current technical availability, not final canonical selection or index inclusion.
  • Canonicalization groups duplicates before serving search results and is not a separate ranking trick.

01

Definition

A page cannot rank before it is indexed, and it cannot be indexed before Google can discover and process it. That sounds obvious. It still prevents an astonishing amount of wasted SEO work when applied consistently.

When someone says, “Google cannot find my page,” they may mean any of the following:

  • Google does not know that the URL exists.
  • Google knows the URL but has not fetched it.
  • Google fetched the page but could not render the useful content.
  • Google processed the page but consolidated it under another canonical URL.
  • Google did not retain the page in the index.
  • Google indexed the page but does not rank it for the expected query.
  • Google ranks it, but so low that the owner assumes it is absent.

These are different failures. They produce different evidence and require different fixes.

Short answer

  • Discovery means Google learns that a URL exists.
  • Crawling means Google requests and downloads the URL.
  • Rendering means Google processes the page like a browser, including supported JavaScript.
  • Indexing means Google analyzes the page and may store information about a canonical version in its index.
  • Ranking and serving mean Google selects indexed material for a particular search and orders the results.

Google’s own description uses three broad stages: crawling, indexing, and serving search results. Not every page proceeds through every stage, and following the technical rules does not guarantee crawling, indexing, or visibility. How Google Search works[1]

02

Mechanism

The pipeline

StageWhat happensEvidence that it happenedCommon failure
DiscoveryGoogle learns the URL existsURL appears in Search Console or logs show a crawler later requested itNo crawlable links, missing sitemap, wrong URL
CrawlingGooglebot requests the URLLast crawl date, server logs, successful fetchrobots block, login, DNS, timeout, 4xx, 5xx
RenderingGoogle processes page resources and JavaScriptRendered screenshot and HTMLMain content depends on failed scripts or blocked resources
IndexingGoogle evaluates content, metadata, duplication, and canonical choiceURL Inspection indexed datanoindex, duplicate consolidation, weak or unprocessable content
Serving/rankingGoogle selects results for a querySearch performance impressions and positionsLow relevance, weak quality, competition, query mismatch

The stages are sequential, but the evidence is not always updated simultaneously. Search Console’s indexed data reflects Google’s stored information from a crawl and indexing attempt. Its live test reflects the current page and does not reproduce every indexing decision.

Discovery is knowing the URL exists

There is no master registry containing every page on the web. Google discovers URLs through mechanisms such as:

  • crawlable links from pages it already knows;
  • sitemaps;
  • previously crawled URLs;
  • external links.

A URL can be discovered without being crawled. That is the important distinction behind “Discovered – currently not indexed.”

Good discovery signals are not merely numerous. They are consistent:

  • internal links point to the intended canonical;
  • the sitemap lists the intended canonical;
  • redirects do not send the crawler elsewhere;
  • the page is reachable through the actual information architecture;
  • links use ordinary anchor elements with usable href values.

Google says links help it discover pages and understand relevance. It can reliably crawl standard anchor links. Link best practices[6]

Crawling is fetching the URL

After discovering a URL, Google may request it with Googlebot. The site must respond.

Crawling can fail because of:

  • robots.txt;
  • authentication;
  • DNS or network failure;
  • timeouts;
  • server errors;
  • malformed or unsupported URL structures;
  • anti-bot systems;
  • redirect loops.

Google’s minimum technical requirements for an indexable page include allowing Googlebot to access it and returning HTTP 200. That is eligibility, not a promise that the page will move into the index. Google Search technical requirements[2]

The cleanest evidence for a crawl is a combination of:

  • URL Inspection’s last crawl and fetch fields;
  • server or platform logs;
  • the response Google received.

Rendering is what Google can actually see

Modern pages often depend on JavaScript. Google renders pages using a browser-like system during crawling and indexing, but rendering can still fail or produce a different result from what a human sees after clicking, scrolling, logging in, or waiting for an API request.

A URL can return 200 while the main article is absent from the rendered output.

Use URL Inspection’s live test to review:

  • screenshot;
  • tested HTML;
  • loaded resources;
  • JavaScript errors;
  • response headers.

If the source contains only a shell and the rendered result is empty, the problem is not solved merely because the server returned 200.

Indexing is analysis and selection

After crawling, Google processes the page and decides how it relates to other known pages. Google may:

  • identify the main content;
  • process metadata;
  • determine language and other signals;
  • group duplicate or strongly similar URLs;
  • select a canonical representative;
  • store information about that representative in the index.

A crawled page is not automatically indexed. Google states directly that indexing is not guaranteed. Common indexing problems include low-quality content, robots meta directives that prohibit indexing, and site designs that make the page difficult to process. How Google Search works[1]

Indexing failures and deliberate exclusions include:

  • noindex;
  • duplicate or alternate canonical;
  • error or soft-error page;
  • content Google processed but did not select;
  • a page that rendered without its meaningful content.

Canonicalization is not a separate ranking trick

Canonicalization happens during indexing. Google groups duplicate or strongly similar URLs and selects one representative.

A canonical annotation tells Google which URL you prefer. It does not force Google to agree. Redirects, sitemap inclusion, internal links, HTTPS, and canonical annotations contribute to the selection. Canonicalization[5]

This means:

  • an unindexed alternate URL may be functioning correctly;
  • the content may be indexed under another URL;
  • inspecting the wrong variant can make a healthy page look broken;
  • inconsistent internal links and sitemaps can weaken your stated preference.

Ranking begins after indexing

Ranking is query-specific. An indexed page may not appear for the phrase its owner typed because:

  • the page is not relevant enough to that query;
  • the page satisfies a different intent;
  • stronger results exist;
  • the page’s quality or evidence is weaker;
  • location, language, or device changes the result;
  • the result appears too low to be noticed.

Do not use “not indexed” as shorthand for “does not rank where I want.”

Use:

  • URL Inspection to check the exact URL’s indexed state;
  • Search Console performance data to see impressions, queries, and positions;
  • the search result itself only as contextual evidence.

03

Examples

How to identify the failing stage No Search Console knowledge and no crawl evidence

Likely stage: discovery

Check:

  • exact URL;
  • crawlable internal links;
  • sitemap;
  • redirect behavior;
  • hostname and protocol.

Known URL, no last crawl

Likely stage: discovered but not crawled

Check:

  • age of the page;
  • section-wide pattern;
  • server health;
  • duplicate URL inventory;
  • internal importance.

Crawl failed

Likely stage: access or response

Check:

  • status code;
  • robots.txt;
  • login;
  • DNS;
  • timeouts;
  • platform challenge pages.

Crawl succeeded, rendered page is empty

Likely stage: rendering

Check:

  • scripts;
  • blocked resources;
  • API calls;
  • content requiring interaction;
  • client-side errors.

Crawl succeeded, Google selected another canonical

Likely stage: canonicalization

Check:

  • duplicate versions;
  • internal-link consistency;
  • sitemap;
  • redirects;
  • declared canonical.

URL is indexed but receives no impressions

Likely stage: serving or ranking

Check:

  • query relevance;
  • content purpose;
  • competition;
  • title and snippet;
  • whether the page addresses a search demand at all.

A practical diagnosis sentence

When recording an issue, use this format:

Google **[does/does not] know the URL**, last crawled it on **[date/unknown]**, received **[response]**, reports crawling **[allowed/blocked]**, reports indexing **[allowed/prohibited]**, selected **[canonical]**, and currently classifies the URL as **[exact status]**.

That sentence is far more useful than “Google hates the page.”

04

Boundaries

Common language mistakes

“Google crawled it, so it should rank”

No. Crawling only means Google fetched the page.

“The live test says indexable, so it is indexed”

No. The live test assesses the current page against a subset of technical conditions. It does not reproduce the complete indexing decision.

“It is in the sitemap, so Google must index it”

No. A sitemap communicates preferred URLs and helps discovery. It is not an inclusion command.[7]

“The alternate URL is not indexed, so the article is missing”

Not necessarily. The content may be indexed under Google’s selected canonical.

“It does not show for my keyword, so it is not indexed”

Ranking for one query is not an index test.

05

Sources

06

References

Sources behind this record

  1. In-depth guide to how Google Search works — Google (accessed July 22, 2026)
  2. Google Search technical requirements — Google (accessed July 22, 2026)
  3. URL Inspection Tool — Google (accessed July 22, 2026)
  4. Page indexing report — Google (accessed July 22, 2026)
  5. What is canonicalization? — Google (accessed July 22, 2026)
  6. Build and submit a sitemap — Google (accessed July 22, 2026)

Corrections

Correction history

No corrections recorded.

To report an error, use the public corrections path.

Claim limit

Google’s systems evaluate pages over time and across duplicate clusters; this guide cannot predict whether a specific URL will be indexed or rank for a particular query.