Black Friday Sale is Live: Get up to 30% OFF RankReady Pro - the WordPress AI SEO pluginBlack Friday Sale is Live: Up to 30% OFF Get Deal
RankReady

Headless WordPress and SEO: What AI Crawlers Can Read

23 Oct 2026 6 min read Updated 29 Oct 2026

A headless WordPress site keeps WordPress as the place where you write. A separate front end, built with a framework such as Next.js or Astro, sits in front of it. That can be a good architecture. It also moves a job that WordPress used to do for you, producing the HTML a crawler reads, to code you now own.

That job matters more than it used to. Google renders JavaScript, but the AI crawlers that matter for assistants may not, and a headless site that builds its pages in the browser can look empty to them. This post is about that one question, what a crawler can read from a headless WordPress site, and how to check it. It is not about the framework choices or the build.

What changes when WordPress goes headless

In a normal WordPress site, the server returns finished HTML. Your SEO plugin prints the title, the canonical and the schema into it. In a headless setup, WordPress returns data, usually through its REST API or GraphQL, and the front end turns that data into a page. Everything an SEO plugin used to place in the page now has to be placed by the front end.

JobNormal WordPressHeadless
Building the HTMLWordPress and the themeThe front end framework
Title, canonical, meta descriptionThe SEO plugin prints themThe front end has to read them from the API and print them
Structured dataA plugin prints itThe front end has to output it
robots.txt and sitemapWordPress serves themWhatever serves the public domain does, which may no longer be WordPress
RedirectsA plugin or a server ruleThe front end or its host

The plugin that used to do each of these may still hold the data. Nothing guarantees that it reaches the page.

What AI crawlers read

The finding that matters comes from a study by Vercel and MERJ, published in December 2024. It reported that none of the major AI crawlers it measured rendered JavaScript. GPTBot, ClaudeBot and PerplexityBot fetched JavaScript files, but did not run them. The practical consequence in Vercel’s write-up is to server-render the content that matters, using server-side rendering, incremental static regeneration or static generation. It says to leave client-side rendering for features that do not carry content. You can read it in the rise of the AI crawler.

Two cautions. The study is almost two years old, and crawler behaviour changes, so treat it as a reason to test, not a final answer. And Googlebot is different: Google renders JavaScript, so a client-rendered page can rank while remaining unreadable to the crawlers feeding other assistants. That gap is easy to miss because the site looks fine in a browser and in Google.

Check what a crawler receives

Browsers show you the page after the scripts have run. A crawler that does not run scripts sees only what the server sent. Fetch the page with no rendering, and look for a sentence from the body of the post.

curl -s https://yourdomain.com/your-post/ | grep -c 'a sentence from the body of your post'

A result of 1 or more means the text is in the HTML. A result of 0 means it appears only after JavaScript runs, and a crawler that does not run scripts will not see it. Do the same for the title and the schema.

curl -s https://yourdomain.com/your-post/ | grep -o '<title>[^<]*</title>'
curl -s https://yourdomain.com/your-post/ | grep -c 'application/ld+json'

The method is the same as in finding every page an AI cannot parse and the five curl commands. Run it on a handful of different templates, not only the homepage, because a single route that renders differently is enough to hide a section of the site.

What to fix, in order

ProblemWhat to look forFix
Body text missing from the HTMLThe grep above returns 0Render that route on the server or at build time
No title or canonical in the HTMLThey appear only after scripts runOutput them from the front end’s server-side layer
Schema missingNo JSON-LD in the raw HTMLInject it on the server from the data the API returns
robots.txt points at the wrong siteThe public domain serves no robots.txt, or serves WordPress’sServe the right file from the public domain, and see the robots.txt guide
Sitemap lists WordPress URLsIt lists the backend domainGenerate it for the public domain

A headless site also has a second public face to protect. If the WordPress back end is reachable on its own domain, make sure it is not indexed as a copy of the site. The file side of that is in our WordPress robots.txt guide, and the structure of the data a page should carry is in structured data in WordPress.

The ways a headless front end can render

ModeWhat a crawler receivesWhat to watch
Static generationFinished HTML built ahead of timePages are stale until the next build
Server-side renderingFinished HTML built on each requestServer load, and slow responses if the API is slow
Incremental regenerationFinished HTML that is refreshed on a schedule or on demandA page can be out of date until it refreshes
Client-side renderingA near-empty shell, with the content added by scriptsThe case that can look empty to a crawler that does not run scripts

The first three put the content in the HTML. The last does not. Vercel’s advice, quoted above, is to use the first three for anything that carries content, and to keep client-side rendering for features that do not.

The back end is a second site

A headless setup has two public faces: the front end that readers see, and the WordPress back end on its own address. If the back end is reachable, it can be crawled and indexed as a copy of the site. Check that it is blocked from indexing, that preview and draft routes are not public, and that links in the content point at the front end’s addresses and not the back end’s. A post that links to the wrong domain sends readers, and crawlers, to a place you did not mean them to go.

A test to run before launch

1. Fetch five different templates with curl and no scripts.
2. Search each for a sentence from the body.
3. Check each has one title, one canonical and one JSON-LD block.
4. Fetch robots.txt and the sitemap from the public domain.
5. Fetch the back end's address and confirm it is not indexable.

Do this before launch, and again after any front-end release. A framework upgrade, or a change to how one route is rendered, can quietly move content from the HTML into the browser.

Should you go headless for SEO?

No. Going headless does not improve search results by itself. It also adds work that a normal WordPress site does for free, because WordPress returns finished HTML and your SEO plugin writes the tags. There are good reasons to go headless, such as sharing content with an app or a team that works in a framework. Better crawling is not one of them. If you do it, the checks above are the price of keeping what you already had.

To see one way to deploy a headless front end, read Headless WordPress on Cloudflare, from your WordPress dashboard.

Where RankReady fits

RankReady’s job on a headless site is limited. In a normal setup it writes AI-facing output into the page. In a headless setup the front end has to carry it, so RankReady exposes its data, such as summaries, FAQs and ready-to-use schema, to your front end through a read-only API. That is a Pro feature, and its details are on the headless API page, not repeated here. The free AI Readiness Checker reads your public site the way a crawler does, which makes it a quick way to see what comes through.

If you fetched your most important post right now without running any JavaScript, would you still find your own first paragraph in it?

Share this
Aditya Sharma
Written by

Aditya Sharma

Host My Blog

Aditya runs Host My Blog and writes The Cited. Five years in WordPress, most of it spent on what actually makes a blog get found rather than what is supposed to. His attention now goes to how AI answers are changing that, and what a blog has to do to still be the source people land on. Everything he writes about here gets tested on the blogs Host My Blog looks after before it reaches the page.

Comments 0

Your email stays private. Be kind.
We’ll handle the technical half

Want your posts in
AI answers, without the work?

Schema, speed and structure are included on every Host My Blog plan. You write; we make sure the machines can read it.

Get tips by email