An AI crawler generally behaves like curl, not like a browser: it requests a URL, reads the raw HTML response, and moves on within a short timeout. If your pricing, features, or FAQ answers only appear after client-side JavaScript hydrates the page, that crawler sees a mostly empty shell - server-rendering and real semantic markup are what make the content structurally legible to a model in the first place.
Why it matters for AEO. Most AI crawlers fetch raw HTML with a short timeout and do not execute heavy client-side JavaScript. Slow servers, render-blocking scripts, and content that only appears after hydration are effectively invisible to them.
How to do it
- Server-render or statically generate the primary content of every page that should be quotable - do not rely on client-side JS to paint the text.
- Use real semantic tags (h1-h3, p, ul/ol, table) instead of div soup - this is what lets an LLM correctly parse structure like "steps" or "comparison rows".
- Keep Time to First Byte low (under roughly 600ms) and avoid render-blocking third-party scripts above the fold.
- Ship one clear H1 per page with a logical H2/H3 outline below it - this mirrors how AI engines chunk a page for retrieval.
How to verify. Run curl -s yourdomain.com/page | grep -c '<h1' - if it returns 0, your H1 is client-rendered and crawlers may miss it. Confirm with view-source (not DevTools Elements) that the real text is present.
Example
view-source:https://yourdomain.com/pricing
# plan names and prices should be visible in the raw HTML,
# not only after client-side JS runs.