Indexing
Indexing refers to the process by which a search engine takes an already discovered web page into its searchable data set, the index. Only once a page is indexed can it appear in search results at all. Without indexing, a page remains practically invisible to users and to AI systems that access search indexes.
Why indexing matters
A web page can be ever so well written: if it is not in a search engine's index, no one finds it through search. For you this means that indexing is the basic prerequisite for any form of visibility. Only the entry in the index makes a page findable, rankable, and citable. This increasingly applies to AI systems as well: many AI assistants draw on search indexes for current questions instead of answering from their training alone. If your page is not indexed, it appears neither in classic results lists nor as a source in an AI answer. Indexing is therefore not a technical detail for specialists but the entry ticket into the visibility competition.
How indexing works
The process has three steps. First comes crawling: a program, the crawler or bot, visits your page and reads out its content. Then the content is processed, that is, analyzed, evaluated, and brought into a searchable structure. Finally comes indexing, the actual admission into the database. You can steer this via signals like the robots.txt file, the noindex meta tag, and your XML sitemap, which lists all important pages for the crawler. A canonical tag helps bundle duplicates so that the right version is indexed. Important: crawling and indexing are two different things, a page can be visited and still not be admitted into the index.
Common mistakes
The classic mistake is an accidental noindex in the page head, which blocks admission into the index. Just as often, an overly strict robots.txt locks the crawler out completely so that it does not even read the content. Thin or duplicate content is often deliberately not indexed because it offers no added value. Broken redirects, wrong status codes, or a lack of internal linking also cause pages to be overlooked. You can check the status via your search engine's search console or with a location query like site:yourdomain.de. If important pages are missing there, it is almost always one of these errors, and you should fix it first before working on rankings.
Relation to AI recommendations
For AI visibility and GEO, indexing is doubly relevant. Many AI assistants and generative search systems answer current questions by querying a search index live and summarizing the pages found. Only indexed pages even come into question as a source. If your page is not admitted into the index, it also cannot be cited or recommended in an AI answer. In addition, some operators run their own AI crawlers that capture content for their systems. Indexing is thus the bridge between classic search engine optimization and modern GEO: it ensures your content is technically reachable before you take care of citability, clear structure, and convincing phrasing.
Example
Imagine a car repair shop that publishes a new advice page on the topic of winter tires. The page is online, looks good, yet after weeks it brings not a single visitor from search. The reason: during the rebuild of the website, a noindex command from the test phase was left in place. The search engine did visit the page but did not admit it into its index. Only after the shop owner removed the command and resubmitted the page for review via the search console does it appear in the results and get found.
Common questions
How long does the indexing of a new page take?
This varies greatly: from a few hours to several weeks. A well-maintained XML sitemap, internal linking, and manual submission via the search console speed up the process noticeably.
What is the difference between crawling and indexing?
Crawling is the visiting and reading of your page by a bot. Indexing is the subsequent admission of the content into the searchable database. A crawled page is not automatically indexed as well.