CrawlProof
Download fix prompt

AEO Audit for x.com

Target: https://x.com/jhorhero1
Score: 25 / 100
Generated: 2026-05-21T17:13:34.301Z
Pages crawled: 1
Findings: 22 pass · 97 warn · 17 fail · 0 unknown

1. Crawl Summary

2. Data Found

Data PointFound?SourceNotes
PricingNo——
Customer logosNo——
Social proofNo——
Recent launchesNo——
Blog post activityNo——
New hiresNo—Often only on a /blog/team or LinkedIn page
Headline copyNo——
PositioningNo——
Executive teamNo——
Product/service descriptionsYesHomepageFrom meta description
Case studies or testimonialsNo——
Contact/demo/signup pathsNo——

3. Homepage Audit

  • ❌ Missing H1 No <h1> element found. LLMs use the H1 as the strongest signal of what the page is about.
  • ❌ Missing <title> The <title> tag is required for search and answer engines.
  • ❌ Content volume: 0 words Very thin content. AI models need substantive text to understand and recommend your site.
  • ⚠️ Page load time: 2.91s Acceptable — consider optimizing for faster crawl times.
  • ⚠️ Short meta description (39 chars) Aim for 50–160 chars. Restate your value prop in plain language.
  • ⚠️ Missing canonical link Add <link rel="canonical" href="https://your-domain"> to prevent dup-content confusion.
  • ⚠️ No Twitter Card tags Add twitter:card, twitter:title, twitter:description, twitter:image for richer previews in social and AI agent surfaces.
  • ❌ Heading structure: 0 (h1:0, h2:0, h3:0) Few headings make it hard for AI to understand sectioning. Use h2/h3 to label each section.
  • ❌ Internal links: 0 Few internal links. Add a nav/footer with links to your key pages so AI crawlers can discover them.
  • ✅ Homepage fetched successfully HTTP 200 · 3586 bytes · 2914ms
  • ✅ declared
  • ✅ Open Graph tags complete
  • ✅ Favicon declared

4. Content Quality

  • ❌ Snippet-ready blocks: 0 (ul:0, ol:0, table:0) Add bullet lists, numbered steps, or a comparison table. Answer engines prefer structured blocks over prose.
  • ❌ Text-to-HTML ratio: 0.0% Very low text density. AI crawlers will struggle to find substantive content.
  • ⚠️ No question-style headings found Phrase at least one heading as a user question (e.g. 'How does pricing work?') to match conversational AI queries.
  • ⚠️ No date signal found Add or article:published_time meta. AI ranking weights freshness.
  • ⚠️ No author byline found Add <meta name="author" content="Name"> or a visible byline with rel="author". Strengthens E-E-A-T signals.

5. Schema / Structured Data Audit

  • ⚠️ Organization missing Adding Organization JSON-LD helps LLMs identify your entity.
  • ⚠️ SoftwareApplication missing Adding SoftwareApplication JSON-LD helps LLMs identify your entity.
  • ⚠️ FAQPage JSON-LD missing Add an FAQPage block on pages that answer common questions — high-value for AI summaries.
  • ⚠️ Article / BlogPosting JSON-LD not found Add Article / BlogPosting where applicable so AI answer engines can resolve the entity precisely.
  • ⚠️ BreadcrumbList JSON-LD not found Add BreadcrumbList where applicable so AI answer engines can resolve the entity precisely.
  • ⚠️ Product / Offer JSON-LD not found Add Product / Offer where applicable so AI answer engines can resolve the entity precisely.
  • ⚠️ LocalBusiness JSON-LD not found Add LocalBusiness where applicable so AI answer engines can resolve the entity precisely.
  • ⚠️ Person (author / founder) JSON-LD not found Add Person (author / founder) where applicable so AI answer engines can resolve the entity precisely.
  • ⚠️ HowTo JSON-LD not found Add HowTo where applicable so AI answer engines can resolve the entity precisely.
  • ⚠️ VideoObject JSON-LD not found Add VideoObject where applicable so AI answer engines can resolve the entity precisely.
  • ✅ 1 JSON-LD block(s) found Types: WebSite
  • ✅ WebSite present

6. Links & Images

No findings.

7. Performance

  • ❌ Response time: 2914ms Slow response. Check CDN/cache headers and origin latency.
  • ✅ Page size: 4 KB Compact HTML payload — well within AI crawler limits.
  • ✅ Resource requests: 0 (scripts:0, css:0, img:0) Reasonable request count.
  • ✅ No render-blocking head scripts All head scripts use async or defer.
  • ✅ Cache-Control set (CDN: Cloudflare) Cache-Control: no-cache, private, must-revalidate, pre-check=0, post-check=0 CDN detected: Cloudflare, Fastly/t4_p

8. Security

  • ⚠️ Referrer-Policy missing Add Referrer-Policy: strict-origin-when-cross-origin for safer referrers.
  • ⚠️ Permissions-Policy missing Restrict browser features (camera, mic, geolocation) you don't use.
  • ✅ Served over HTTPS
  • ✅ No mixed content detected
  • ✅ HSTS set max-age=631138519; includeSubdomains
  • ✅ Content-Security-Policy set connect-src 'self' blob: https://fonts.googleapis.com/css https://mapsresources-pa.googleapis.com https://maps.googleapi…
  • ✅ X-Frame-Options set DENY
  • ✅ X-Content-Type-Options set nosniff

9. robots.txt and sitemap.xml Audit

  • ❌ sitemap.xml not found Add /sitemap.xml — required for reliable AI/SERP discovery.
  • ✅ robots.txt present 2514 chars
  • ✅ robots.txt references sitemap(s)

10. LLM / AI Crawler Accessibility

  • ❌ GPTBot blocked via wildcard User-agent: * is disallowed from / and no explicit rule for GPTBot overrides it.
  • ❌ ClaudeBot blocked via wildcard User-agent: * is disallowed from / and no explicit rule for ClaudeBot overrides it.
  • ❌ PerplexityBot blocked via wildcard User-agent: * is disallowed from / and no explicit rule for PerplexityBot overrides it.
  • ❌ OAI-SearchBot blocked via wildcard User-agent: * is disallowed from / and no explicit rule for OAI-SearchBot overrides it.
  • ❌ Applebot-Extended blocked via wildcard User-agent: * is disallowed from / and no explicit rule for Applebot-Extended overrides it.
  • ❌ CCBot blocked via wildcard User-agent: * is disallowed from / and no explicit rule for CCBot overrides it.
  • ⚠️ llms.txt missing Add /llms.txt — a concise, link-rich summary that helps LLMs orient on your site.
  • ⚠️ skill.md missing Add /skill.md describing what your site lets agents do — speeds up agent task routing.
  • ✅ Google-Extended has explicit rules An explicit User-agent block exists. Make sure it allows the paths you want indexed.
  • ✅ /.well-known/security.txt present Security contact published — builds trust with crawlers and security researchers.

11. Positioning Clarity

  • ⚠️ H1 missing or too short to convey value Add a clear, single-sentence H1 like 'We help X do Y.'
  • ❌ No discoverable CTA Add a clearly-labeled Contact, Demo, or Sign up link to the nav or hero.
  • ⚠️ No clear About/Team link Add an About or Team link in the nav or footer so LLMs can identify the company.
  • ⚠️ No pricing/plans link found AI summaries commonly include pricing. Add a /pricing page even if pricing is custom.
  • ⚠️ Value-prop language not detected Pages with phrases like 'we help X', 'platform for Y', 'built for Z' are easier for LLMs to summarize.

12. Missing or Hard-to-Find Information

  • ❌ 11 data point(s) could not be found from public pages · Pricing · Customer logos · Social proof · Recent launches · Blog post activity · New hires · Headline copy · Positioning · Executive team · Case studies or testimonials · Contact/demo/signup paths

  • ⚠️ Add a single, focused H1 to the homepage One <h1> per page. Write it as 'We help [audience] [do thing].' so an LLM can quote it verbatim.

  • ⚠️ Set a meaningful <title> 30–60 chars. Lead with the brand or product, then the value prop.

    <title>CrawlProof — AEO audits for AI crawlers</title>

  • ⚠️ Add more substantive homepage content AI models need 300+ words of visible body text to summarize and recommend a site. Add a value-prop paragraph, a short FAQ, and a 'how it works' section.

  • ⚠️ Allow GPTBot in robots.txt Add an explicit User-agent: GPTBot Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow ClaudeBot in robots.txt Add an explicit User-agent: ClaudeBot Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow PerplexityBot in robots.txt Add an explicit User-agent: PerplexityBot Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow OAI-SearchBot in robots.txt Add an explicit User-agent: OAI-SearchBot Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow Applebot-Extended in robots.txt Add an explicit User-agent: Applebot-Extended Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow CCBot in robots.txt Add an explicit User-agent: CCBot Allow: / block so this AI crawler can read your site.

  • ⚠️ Rewrite the homepage H1 to be self-evident Replace clever copy with literal copy. 'We help X do Y' beats 'Reimagine Y'.

  • ⚠️ Add a discoverable CTA Place 'Contact sales' or 'Start free' in the top-right of the nav. LLMs cite the visible label.

  • ⚠️ Publish a sitemap.xml Generate /sitemap.xml automatically (Next.js: app/sitemap.ts). Include every canonical URL.

  • ⚠️ Add lists or comparison tables Answer engines lift bulleted lists, numbered steps, and tables verbatim. Add at least 2 snippet-ready blocks to the homepage.

  • ⚠️ Raise your text-to-HTML ratio Strip unused inline scripts/styles and move large bundles to external files. AI crawlers struggle when most of the response is markup.

  • ⚠️ Add /llms.txt A short Markdown-flavored summary at the root. Include your H1, value prop, top 5–10 links, and pricing summary.

  • ⚠️ Reduce response time Push static HTML to a CDN edge cache. If you must server-render per-request, profile DB/template work and add Cache-Control: s-maxage=….

  • ⚠️ Make your About/Team page reachable Add a top-nav or footer link to /about or /team so LLMs can identify the entity behind the site.

  • ⚠️ Add a /pricing page Even contact-us pricing benefits from a /pricing page that LLMs can link to in answers.

  • ⚠️ Add Organization JSON-LD Include name, url, logo, sameAs (your social profiles). LLMs use this to resolve your brand entity.

  • ⚠️ Phrase a heading as a user question Use headings like 'How does pricing work?' or 'Who is this for?' — they map directly to conversational AI queries.

  • ⚠️ Publish a date signal Add <time datetime="2026-05-17"> or <meta property="article:published_time">. AI ranking heavily weights freshness.

  • ⚠️ Speed up homepage rendering AI crawlers commonly time out around 3s. Cache the HTML, ship less JS for the first paint, and pre-render the hero section server-side.

  • ⚠️ Add a meta description 50–160 chars. Repeat your core value prop in plain language; this often becomes the AI snippet.

    <meta name="description" content="CrawlProof shows you exactly how AI crawlers see your site, then tells you what to fix." />

  • ⚠️ Add a canonical link Prevents dup-content drift and tells AI crawlers which URL is authoritative.

    <link rel="canonical" href="https://yoursite.com/" />

  • ⚠️ Add Twitter Card meta tags Used by social platforms and AI agents for richer previews.

    <meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:title" content="Your Page Title" />
<meta name="twitter:description" content="50–160 char description." />
<meta name="twitter:image" content="https://yoursite.com/og-image.jpg" />

  • ⚠️ Add structured headings Use h2 for each section and h3 for sub-points. AI uses these to outline and chunk the page.

  • ⚠️ Add internal navigation links Top nav + footer with links to /pricing, /docs, /about, /contact gives AI crawlers an entry point to the rest of the site.

  • ⚠️ Add /skill.md Describe what an agent can do with your site (e.g., 'Search docs', 'Look up pricing'). Useful for agentic flows.

  • ⚠️ State your audience explicitly Use phrases like 'Built for B2B SaaS marketing teams' on the homepage and About page.

  • ⚠️ Add Product / SoftwareApplication JSON-LD On /pricing and feature pages — include offers, name, applicationCategory.

  • ⚠️ Add FAQPage JSON-LD Wrap your homepage FAQ in FAQPage JSON-LD; it routinely lifts AI answer inclusion.

  • ⚠️ Add Article / BlogPosting JSON-LD On every blog/article page, include Article JSON-LD with headline, author, datePublished, dateModified. AI engines weight these heavily for freshness and authority.

  • ⚠️ Add BreadcrumbList JSON-LD Helps AI engines understand site hierarchy and improves citation context.

  • ⚠️ Add Product / SoftwareApplication JSON-LD On /pricing and feature pages — include offers, name, applicationCategory.

  • ⚠️ Declare an author byline Add <meta name="author" content="Name"> or a visible byline with rel="author". Combine with Person JSON-LD for E-E-A-T.

  • ⚠️ Add LocalBusiness JSON-LD (if you have a physical location) Include address, geo, openingHours, telephone. Required for AI engines to surface you in 'near me' queries.

  • ⚠️ Add Person JSON-LD for authors / founders Mark up bylines and founder bios with Person schema — name, jobTitle, sameAs (their profiles). Strengthens E-E-A-T.

  • ⚠️ Add HowTo JSON-LD for step-by-step content For any 'how to' page, wrap the steps in HowTo JSON-LD. AI step-by-step answers cite these heavily.

  • ⚠️ Add VideoObject JSON-LD For embedded videos, include VideoObject with thumbnailUrl, uploadDate, duration. AI engines cite these in multimedia answers.

  • ⚠️ Set a Referrer-Policy Referrer-Policy: strict-origin-when-cross-origin is a safe default.

  • ⚠️ Set a Permissions-Policy Restrict browser features you don't use, e.g. Permissions-Policy: camera=(), microphone=(), geolocation=().

14. Priority To-Do List

  • P1 — Add a single, focused H1 to the homepage One <h1> per page. Write it as 'We help [audience] [do thing].' so an LLM can quote it verbatim.

  • P1 — Set a meaningful <title> 30–60 chars. Lead with the brand or product, then the value prop.

    ```html
<title>CrawlProof — AEO audits for AI crawlers</title>
```

  • P1 — Add more substantive homepage content AI models need 300+ words of visible body text to summarize and recommend a site. Add a value-prop paragraph, a short FAQ, and a 'how it works' section.

  • P1 — Allow GPTBot in robots.txt Add an explicit User-agent: GPTBot Allow: / block so this AI crawler can read your site.

  • P1 — Allow ClaudeBot in robots.txt Add an explicit User-agent: ClaudeBot Allow: / block so this AI crawler can read your site.

  • P1 — Allow PerplexityBot in robots.txt Add an explicit User-agent: PerplexityBot Allow: / block so this AI crawler can read your site.

  • P1 — Allow OAI-SearchBot in robots.txt Add an explicit User-agent: OAI-SearchBot Allow: / block so this AI crawler can read your site.

  • P1 — Allow Applebot-Extended in robots.txt Add an explicit User-agent: Applebot-Extended Allow: / block so this AI crawler can read your site.

  • P1 — Allow CCBot in robots.txt Add an explicit User-agent: CCBot Allow: / block so this AI crawler can read your site.

  • P1 — Rewrite the homepage H1 to be self-evident Replace clever copy with literal copy. 'We help X do Y' beats 'Reimagine Y'.

  • P1 — Add a discoverable CTA Place 'Contact sales' or 'Start free' in the top-right of the nav. LLMs cite the visible label.

  • P1 — Publish a sitemap.xml Generate /sitemap.xml automatically (Next.js: app/sitemap.ts). Include every canonical URL.

  • P2 — Add lists or comparison tables Answer engines lift bulleted lists, numbered steps, and tables verbatim. Add at least 2 snippet-ready blocks to the homepage.

  • P2 — Raise your text-to-HTML ratio Strip unused inline scripts/styles and move large bundles to external files. AI crawlers struggle when most of the response is markup.

  • P2 — Add /llms.txt A short Markdown-flavored summary at the root. Include your H1, value prop, top 5–10 links, and pricing summary.

  • P2 — Reduce response time Push static HTML to a CDN edge cache. If you must server-render per-request, profile DB/template work and add Cache-Control: s-maxage=….

  • P2 — Make your About/Team page reachable Add a top-nav or footer link to /about or /team so LLMs can identify the entity behind the site.

  • P2 — Add a /pricing page Even contact-us pricing benefits from a /pricing page that LLMs can link to in answers.

  • P2 — Add Organization JSON-LD Include name, url, logo, sameAs (your social profiles). LLMs use this to resolve your brand entity.

  • P3 — Phrase a heading as a user question Use headings like 'How does pricing work?' or 'Who is this for?' — they map directly to conversational AI queries.

Report by CrawlProof. Reusable after every major website change.