Guide 2 of 9 · Draft Preview

The Technical Checklist

The exact site fixes that get you crawled, indexed and cited by AI search — not just ranked on Google

Guide 2 of [series name TBD] — content draft, no design applied. Oliver builds the page from this.

The short version

Guide 1 covered why AI systems trust one business over another. This guide is the actionable follow-up: the specific, ordered list of technical things to check and fix on your own site.

The order below isn't arbitrary. It goes from "AI can't see you at all" to "AI can see you but isn't reading you optimally" to "you can't tell if any of this is working." Fix problems in that order — there's no point polishing schema markup on a page an AI crawler never reaches.

A few of these are one-time fixes: get them right once, then leave them alone. A few need attention every time you publish. The checklist below flags which is which as you go.


Step 1: Check whether your content is actually visible to a crawler that doesn't run JavaScript

(formal label: server-side rendering / crawlability)

Google's crawler can render JavaScript, eventually — it queues pages for a second rendering pass. Most AI crawlers (GPTBot, ClaudeBot, PerplexityBot) don't render JavaScript at all. They read the raw HTML that comes back from the server, before any scripts run.

If your site is built on a JavaScript framework and your key content — pricing, product details, the actual body of an article — only appears after client-side rendering, an AI crawler may see a near-empty page. This is a genuine visibility risk, not a nice-to-have to get to eventually.

How to check: view your page's raw source (not the rendered DOM in dev tools — the actual response the server sends) or disable JavaScript in your browser and reload. If your key content disappears, an AI crawler likely can't see it either.

Fix: server-side rendering or prerendering for any page whose content needs to be citable. If you've already solved this for Googlebot (many established sites have), confirm the same prerendering also serves the AI crawler user agents specifically — it's common to have fixed this for Googlebot and never checked it works for the others.

One-time fix, revisit only when you change your rendering setup.


Step 2: Get indexed on Bing, not just Google

(formal label: multi-index submission)

ChatGPT's web search and Perplexity both draw on Bing's index. Google's AI Overviews use Google's own index. A site that's fully indexed in Google but has never been submitted to Bing Webmaster Tools has a real, checkable blind spot in AI search — regardless of how well it ranks on Google.

Do this:

  • Submit your site and sitemap to Bing Webmaster Tools if you haven't already.
  • Consider IndexNow — a protocol that pushes new or updated URLs to Bing's (and Yandex's) index in near real time, rather than waiting for it to be discovered on the next crawl. Google doesn't support this protocol, so it's specifically an AI-search-lane advantage. Most useful for time-sensitive content — new pages, updated guides, anything where the gap between publishing and being indexed matters.

How to check: search site:yourdomain.com on Bing. If nothing meaningful comes back, you've found the gap.

Bing submission is a one-time setup. IndexNow, if you implement it, becomes an ongoing publishing habit — not something you tune, just something you do every time you publish.


Step 3: Confirm your robots.txt isn't blocking the AI crawlers

(formal label: crawler access control)

Most robots.txt files were written with only Googlebot in mind. If yours inadvertently blocks the AI crawlers — more common than most site owners expect — the site simply doesn't exist for that system, no matter how good the content is.

Crawler Belongs to Note
GPTBot OpenAI Powers ChatGPT's web search
PerplexityBot Perplexity Its own retrieval crawler
ClaudeBot Anthropic Anthropic runs more than one bot under this and related names — for training, live fetching, and search indexing — each controllable separately in some setups
Google-Extended Google Controls AI training data only (Gemini, Vertex AI). Blocking it does not remove you from Google's AI Overviews — those still run on regular Googlebot data. Don't confuse the two
Bingbot Microsoft Standard Bing crawling — feeds the index ChatGPT and Perplexity both draw on

Do this: fetch yourdomain.com/robots.txt and confirm none of the above are disallowed. Add explicit allow rules for the AI crawlers if they're missing, without touching your existing Googlebot/Bingbot rules — this is a low-risk, additive change.

One-time fix. Re-check after any site migration or platform change, since robots.txt often gets regenerated from a template that forgets the additions.


Step 4: Implement schema once, correctly — then stop

(formal label: structured data / JSON-LD)

Schema markup is baseline hygiene: get it right once, and it's done. It's not a lever you keep pulling for better citation — adding more schema types or re-tuning existing ones won't move the needle further once the core set is in place correctly.

Implementation order, each one building on the last:

  1. Organisation schema on the homepage — entity identity, with sameAs links to verifiable profiles (Companies House, LinkedIn) and knowsAbout for your area of expertise.
  2. Person schema for every named author — sameAs, jobTitle, knowsAbout. This is what lets an AI system distinguish a credentialed author from an anonymous byline.
  3. Article schema on all content — nested author entity, a citation array of the sources referenced, dateModified, about.
  4. FAQPage schema — start with your highest-value pages, watch how they perform for a few weeks, then roll out further. Each answer needs to be self-contained: a complete response that makes sense with no surrounding context, since AI systems extract answers independently of the rest of the page.
  5. DefinedTerm schema — for any specialist term your site defines authoritatively.
  6. Speakable schema — marks your direct-answer passages as the primary extractable content.

How to check: validate every type as you add it (Google's Rich Results Test or Schema.org's own validator both work). Don't skip validation — malformed schema is worse than none, because it can suppress rich results that were already working.

One-time implementation per schema type. Once it validates and matches the current page content, leave it — just keep it in sync when the underlying content changes (new author, updated date, etc.), which is maintenance, not optimization.


Step 5: Add an llms.txt file once, correctly — then stop

(formal label: llms.txt)

An llms.txt file is a plain-text or Markdown file at your domain root that gives AI systems a structured summary of your most important content — think of it as a signpost, not a ranking signal. No major AI provider has officially confirmed they read it, and adoption figures for it change too fast to be worth quoting here.

Treat it the same way as schema: baseline hygiene, done once. There's no version of "better llms.txt" worth iterating toward — write an accurate, current summary of your site, keep it updated when your core offering changes, and move on.

Do this: create /llms.txt at your domain root with a short, accurate summary of what the site is and links to your most important pages. Zero risk, zero interaction with your existing SEO setup.

One-time fix. Update only when what you offer materially changes — not on a schedule.


Step 6: Structure new content so a passage can be lifted on its own

(formal label: passage-level extractability)

AI systems don't read a page as a whole and decide to cite it — they identify specific passages to extract and use as source material. Whether a given passage gets picked depends on how it's written.

  • Open each section with a direct, factual statement — roughly 40 to 60 words, no "in this section we'll explore..." preamble. Give the AI something it can lift immediately.
  • Phrase headings as natural questions where that fits ("What's the difference between X and Y?") — this matches how people actually query AI systems, which is conversational rather than keyword-based.
  • Use comparison tables wherever information has a tabular shape. Tables get extracted far more readily than the same information written out in prose.
  • Attribute every statistic to its source. An unsourced number is weaker evidence than the same number with a named source attached.
  • Keep FAQ answers self-contained — a complete, standalone response, not a continuation of the paragraph above it (see Step 4).

Applies to every new piece of content going forward. Not a one-time fix — a habit for how you write, going forward.


Step 7: Use one consistent name for each thing you do, everywhere

(formal label: entity naming consistency)

If your homepage calls something "designer sunglasses," your about page calls it "luxury eyewear," and your blog calls it "premium sunglasses," a human reader understands those as interchangeable. An AI system building an entity representation from your site may not — it can end up treating them as separate concepts and fragmenting its understanding of what you actually do.

Do this: pick the core label for each product, service, or category, and use that exact term in headings, navigation, schema, and page titles across the whole site. Variation in supporting prose is fine — the primary term is what needs to stay fixed.

One-time audit, then a rule to apply to everything you publish afterward.


Step 8: State what a page doesn't cover, not just what it does

(formal label: scope disambiguation)

Explicitly stating a page's boundaries helps an AI system disambiguate it from adjacent topics — without that, the system has to infer the boundary itself, and when it infers, it sometimes gets it wrong or reduces its confidence in using the page at all.

Do this: on pages covering a narrower slice of a broader topic, add a line stating the scope and pointing to where the adjacent topic lives — for example, "this guide covers X for everyday use; for the sport-specific version, see [page]."

Applies to new content as you write it; worth retrofitting onto your highest-traffic existing pages first.


Step 9: Show when a page was last reviewed, not just when it was published

(formal label: freshness signalling)

A visible "last reviewed" date — separate from the original publish date — signals active maintenance to both readers and AI systems with live retrieval. Pair the visible date with dateModified in your Article schema so both the human-readable and machine-readable signals agree.

Do this: add a last-reviewed date to any page you intend to keep current, and actually update it when you make a substantive edit — not just when you nudge the date.

Ongoing — this only works if the date is genuinely kept current.


Step 10: Build first-party attribution capture into your lead and contact forms

(formal label: first-party attribution / server-side conversion tracking)

This isn't about distrusting Google Analytics generally — it's about a specific, known gap. ChatGPT and AI Mode visits frequently don't pass a normal browser referrer header, so tools like GA4 file those sessions under "Direct" traffic, indistinguishable from someone typing your URL in by hand. If you're relying on GA4 alone to judge whether AI search is sending you anything, you're working from an undercount.

The fix is to capture attribution data directly on the form submission itself, not just through analytics. A working pattern for this:

Fields captured on every submission:

Field What it captures
source The channel the visitor arrived through
landing_path The first page they landed on
referrer / referrer_host The raw referrer, where one exists
utm_source / utm_medium / utm_campaign / utm_term / utm_content Standard campaign parameters
gclid / fbclid / msclkid Platform click IDs, where present

Captured separately for first touch and last touch:

A returning visitor's journey often starts on one channel and converts on another — capturing only the most recent touch loses the channel that actually introduced them. Store both:

  • attr_first_referrer, attr_first_utm_*, attr_first_gclid — the visitor's original attribution, set once on their first visit and never overwritten
  • attr_last_referrer (and equivalent last-touch fields) — the most recent attribution, updated on each visit

Do this: add these fields to your form submission handler, populate them from the URL and browser state at the point of submission (not by relying on analytics after the fact), and store first-touch and last-touch separately so neither overwrites the other.

One-time build. Once it's in place, it runs itself — check periodically that new campaign parameters or lead sources you start using are still being captured correctly.


Quick reference

Step One-time or ongoing
1. Server-rendered content One-time, recheck after platform changes
2. Bing indexation One-time submission; IndexNow is a publishing habit
3. Robots.txt for AI crawlers One-time, recheck after migrations
4. Schema markup One-time per type, then maintenance only
5. llms.txt One-time, update only on material offer changes
6. Passage-level structure Ongoing — applies to everything you write
7. Naming consistency One-time audit, then an ongoing rule
8. Scope disambiguation Ongoing, retrofit high-traffic pages first
9. Last-reviewed dates Ongoing — only works if kept current
10. First-party attribution One-time build, then self-sustaining

Where to go next

This guide covers what to fix on the site you already have. Guide 3 goes a level earlier — how to build a website for SEO and AI search from the ground up, for anyone starting a new site or planning a rebuild.


Guide 2 of 9 · internal draft preview, not for search engines.