CrawlProof

Public share link

Post on XLinkedIn

AEO Audit for web-3-native-portfolio--smshakawatyt.replit.app

Target: https://web-3-native-portfolio--smshakawatyt.replit.app/
Score: 39 / 100
Generated: 2026-06-09T23:07:38.053Z
Pages crawled: 1
Findings: 29 pass · 107 warn · 9 fail · 0 unknown

1. Crawl Summary

2. Data Found

Data PointFound?SourceNotes
PricingNo——
Customer logosNo——
Social proofNo——
Recent launchesNo——
Blog post activityNo——
New hiresNo—Often only on a /blog/team or LinkedIn page
Headline copyNo——
PositioningNo——
Executive teamNo——
Product/service descriptionsYesHomepageFrom meta description
Case studies or testimonialsNo——
Contact/demo/signup pathsNo——

3. Homepage Audit

  • ❌ Missing H1 No <h1> element found. LLMs use the H1 as the strongest signal of what the page is about.
  • ❌ Content volume: 0 words Very thin content. AI models need substantive text to understand and recommend your site.
  • ⚠️ Missing canonical link Add <link rel="canonical" href="https://your-domain"> to prevent dup-content confusion.
  • ⚠️ Open Graph: missing image
  • ⚠️ Twitter Card: missing image Add twitter:card, twitter:title, twitter:description, twitter:image for richer previews in social and AI agent surfaces.
  • ❌ Heading structure: 0 (h1:0, h2:0, h3:0) Few headings make it hard for AI to understand sectioning. Use h2/h3 to label each section.
  • ❌ Internal links: 0 Few internal links. Add a nav/footer with links to your key pages so AI crawlers can discover them.
  • ✅ Homepage fetched successfully HTTP 200 · 1670 bytes · 233ms
  • ✅ Page load time: 0.23s Fast — well within AI crawler budgets.
  • ✅ declared
  • ✅ <title> present (32 chars)
  • ✅ Meta description present (95 chars)
  • ✅ Robots meta: "index, follow"
  • ✅ Favicon declared

4. Content Quality

  • ❌ Snippet-ready blocks: 0 (ul:0, ol:0, table:0) Add bullet lists, numbered steps, or a comparison table. Answer engines prefer structured blocks over prose.
  • ❌ Text-to-HTML ratio: 0.0% Very low text density. AI crawlers will struggle to find substantive content.
  • ⚠️ No question-style headings found Phrase at least one heading as a user question (e.g. 'How does pricing work?') to match conversational AI queries.
  • ⚠️ No date signal found Add or article:published_time meta. AI ranking weights freshness.
  • ⚠️ No author byline found Add <meta name="author" content="Name"> or a visible byline with rel="author". Strengthens E-E-A-T signals.

5. Schema / Structured Data Audit

  • ❌ No JSON-LD structured data found Add JSON-LD blocks (Organization, SoftwareApplication, FAQPage, BreadcrumbList) so AI answer engines can ingest your data without guessing.

6. Links & Images

No findings.

7. Performance

  • ⚠️ 1 render-blocking script(s) in Move non-critical scripts to end of or add defer/async.
  • ⚠️ No Content-Encoding compression detected Enable Brotli (preferred) or gzip compression. Reduces transfer size by ~70-80% for HTML.
  • ✅ Page size: 2 KB Compact HTML payload — well within AI crawler limits.
  • ✅ Resource requests: 4 (scripts:2, css:2, img:0) Reasonable request count.
  • ✅ Response time: 233ms Fast first response.
  • ✅ Cache-Control set Cache-Control: private

8. Security

  • ⚠️ Content-Security-Policy missing Define a CSP to limit script sources — large reduction in XSS surface.
  • ⚠️ X-Frame-Options missing Add X-Frame-Options: SAMEORIGIN (or use CSP frame-ancestors) to prevent clickjacking.
  • ⚠️ X-Content-Type-Options missing Add X-Content-Type-Options: nosniff to block MIME-type sniffing.
  • ⚠️ Referrer-Policy missing Add Referrer-Policy: strict-origin-when-cross-origin for safer referrers.
  • ⚠️ Permissions-Policy missing Restrict browser features (camera, mic, geolocation) you don't use.
  • ✅ Served over HTTPS
  • ✅ No mixed content detected
  • ✅ HSTS set max-age=63072000; includeSubDomains

9. robots.txt and sitemap.xml Audit

  • ⚠️ Sitemap is empty No entries found. Sitemap should list canonical URLs.
  • ⚠️ robots.txt does not reference a Sitemap Add Sitemap: https://yoursite.com/sitemap.xml to robots.txt.
  • ✅ robots.txt present 23 chars
  • ✅ sitemap.xml present (0 URLs)

10. LLM / AI Crawler Accessibility

  • ⚠️ GPTBot not explicitly addressed No User-agent: GPTBot block in robots.txt. We recommend explicit Allow rules so crawlers don't fall back to defaults.
  • ⚠️ ClaudeBot not explicitly addressed No User-agent: ClaudeBot block in robots.txt. We recommend explicit Allow rules so crawlers don't fall back to defaults.
  • ⚠️ PerplexityBot not explicitly addressed No User-agent: PerplexityBot block in robots.txt. We recommend explicit Allow rules so crawlers don't fall back to defaults.
  • ⚠️ Google-Extended not explicitly addressed No User-agent: Google-Extended block in robots.txt. We recommend explicit Allow rules so crawlers don't fall back to defaults.
  • ⚠️ OAI-SearchBot not explicitly addressed No User-agent: OAI-SearchBot block in robots.txt. We recommend explicit Allow rules so crawlers don't fall back to defaults.
  • ⚠️ Applebot-Extended not explicitly addressed No User-agent: Applebot-Extended block in robots.txt. We recommend explicit Allow rules so crawlers don't fall back to defaults.
  • ⚠️ CCBot not explicitly addressed No User-agent: CCBot block in robots.txt. We recommend explicit Allow rules so crawlers don't fall back to defaults.
  • ✅ llms.txt present 1664 chars
  • ✅ skill.md present
  • ✅ /.well-known/ai-plugin.json present
  • ✅ /.well-known/security.txt present Security contact published — builds trust with crawlers and security researchers.
  • ✅ /llms-full.txt present 1664 chars — full site content for LLM ingest.

11. Positioning Clarity

  • ⚠️ H1 missing or too short to convey value Add a clear, single-sentence H1 like 'We help X do Y.'
  • ❌ No discoverable CTA Add a clearly-labeled Contact, Demo, or Sign up link to the nav or hero.
  • ⚠️ No clear About/Team link Add an About or Team link in the nav or footer so LLMs can identify the company.
  • ⚠️ No pricing/plans link found AI summaries commonly include pricing. Add a /pricing page even if pricing is custom.
  • ⚠️ Value-prop language not detected Pages with phrases like 'we help X', 'platform for Y', 'built for Z' are easier for LLMs to summarize.

12. Missing or Hard-to-Find Information

  • ❌ 11 data point(s) could not be found from public pages · Pricing · Customer logos · Social proof · Recent launches · Blog post activity · New hires · Headline copy · Positioning · Executive team · Case studies or testimonials · Contact/demo/signup paths

  • ⚠️ Add a single, focused H1 to the homepage One <h1> per page. Write it as 'We help [audience] [do thing].' so an LLM can quote it verbatim.

  • ⚠️ Add more substantive homepage content AI models need 300+ words of visible body text to summarize and recommend a site. Add a value-prop paragraph, a short FAQ, and a 'how it works' section.

  • ⚠️ Rewrite the homepage H1 to be self-evident Replace clever copy with literal copy. 'We help X do Y' beats 'Reimagine Y'.

  • ⚠️ Add a discoverable CTA Place 'Contact sales' or 'Start free' in the top-right of the nav. LLMs cite the visible label.

  • ⚠️ Add JSON-LD structured data Start with Organization on the root layout and SoftwareApplication or Product on /pricing. Add FAQPage on any FAQ section.

  • ⚠️ Add lists or comparison tables Answer engines lift bulleted lists, numbered steps, and tables verbatim. Add at least 2 snippet-ready blocks to the homepage.

  • ⚠️ Raise your text-to-HTML ratio Strip unused inline scripts/styles and move large bundles to external files. AI crawlers struggle when most of the response is markup.

  • ⚠️ Create or enrich /llms.txt Follow the llmstxt.org spec:

    # Your Brand

> One-line description of your site.

## Docs

- [Getting Started](https://yoursite.com/docs/start): How to get up and running.
- [API Reference](https://yoursite.com/docs/api): Full API details.

## About

- [About us](https://yoursite.com/about): Mission and team.

    Include at least 2 section headings, 3+ linked resources, and a brief description per link. A rich llms.txt dramatically increases how often generative AI systems cite your content.

  • ⚠️ Add sameAs knowledge graph links to Organization schema Extend your Organization JSON-LD to include sameAs pointing to authoritative directories:

    {
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "Your Brand",
  "url": "https://yoursite.com",
  "sameAs": [
    "https://en.wikipedia.org/wiki/Your_Brand",
    "https://www.wikidata.org/wiki/Q12345678",
    "https://www.linkedin.com/company/your-brand",
    "https://www.crunchbase.com/organization/your-brand"
  ]
}

    These links anchor your brand as a known entity in AI knowledge graphs, making it far more likely that generative models cite you by name rather than paraphrase.

  • ⚠️ Declare your brand name in Organization JSON-LD Add "name": "Your Brand" to your Organization or SoftwareApplication schema block. AI systems match structured-data names against training data to resolve your brand as a distinct entity. Without it, mentions of your brand may not be attributed to you.

  • ⚠️ Make your About/Team page reachable Add a top-nav or footer link to /about or /team so LLMs can identify the entity behind the site.

  • ⚠️ Add a /pricing page Even contact-us pricing benefits from a /pricing page that LLMs can link to in answers.

  • ⚠️ Populate your sitemap with canonical URLs The sitemap must list <loc> entries — empty sitemaps are worse than none.

  • ⚠️ Phrase a heading as a user question Use headings like 'How does pricing work?' or 'Who is this for?' — they map directly to conversational AI queries.

  • ⚠️ Publish a date signal Add <time datetime="2026-05-17"> or <meta property="article:published_time">. AI ranking heavily weights freshness.

  • ⚠️ Add outbound links to authoritative sources Link to Wikipedia, .gov or .edu resources, peer-reviewed studies, or major news outlets when making factual claims. Generative AI systems treat pages that cite authoritative sources as more trustworthy, which raises citation likelihood.

    Examples: statistics from Statista or Census.gov, definitions from Wikipedia, research from nature.com or pubmed.ncbi.nlm.nih.gov.

  • ⚠️ Add a canonical link Prevents dup-content drift and tells AI crawlers which URL is authoritative.

    <link rel="canonical" href="https://yoursite.com/" />

  • ⚠️ Complete Open Graph tags AI bots use OG for fast disambiguation. Add all four:

    <meta property="og:title" content="Your Page Title" />
<meta property="og:description" content="50–160 char description of this page." />
<meta property="og:image" content="https://yoursite.com/og-image.jpg" />
<meta property="og:url" content="https://yoursite.com/" />
<meta property="og:type" content="website" />
<meta property="og:site_name" content="YourSite" />

  • ⚠️ Add Twitter Card meta tags Used by social platforms and AI agents for richer previews.

    <meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:title" content="Your Page Title" />
<meta name="twitter:description" content="50–160 char description." />
<meta name="twitter:image" content="https://yoursite.com/og-image.jpg" />

  • ⚠️ Add structured headings Use h2 for each section and h3 for sub-points. AI uses these to outline and chunk the page.

  • ⚠️ Add internal navigation links Top nav + footer with links to /pricing, /docs, /about, /contact gives AI crawlers an entry point to the rest of the site.

  • ⚠️ Allow GPTBot in robots.txt Add an explicit User-agent: GPTBot Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow ClaudeBot in robots.txt Add an explicit User-agent: ClaudeBot Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow PerplexityBot in robots.txt Add an explicit User-agent: PerplexityBot Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow Google-Extended in robots.txt Add an explicit User-agent: Google-Extended Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow OAI-SearchBot in robots.txt Add an explicit User-agent: OAI-SearchBot Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow Applebot-Extended in robots.txt Add an explicit User-agent: Applebot-Extended Allow: / block so this AI crawler can read your site.

  • ⚠️ Allow CCBot in robots.txt Add an explicit User-agent: CCBot Allow: / block so this AI crawler can read your site.

  • ⚠️ Eliminate render-blocking head scripts Add defer or async to any <script src="…"> in <head>, or move it to the end of <body>.

  • ⚠️ State your audience explicitly Use phrases like 'Built for B2B SaaS marketing teams' on the homepage and About page.

  • ⚠️ Reference your sitemap in robots.txt Add Sitemap: https://yoursite.com/sitemap.xml so crawlers don't have to guess.

  • ⚠️ Define a Content-Security-Policy Start with Content-Security-Policy-Report-Only to learn safe sources, then enforce. Cuts XSS blast radius.

  • ⚠️ Declare an author byline Add <meta name="author" content="Name"> or a visible byline with rel="author". Combine with Person JSON-LD for E-E-A-T.

  • ⚠️ Add X-Frame-Options X-Frame-Options: SAMEORIGIN (or CSP frame-ancestors) blocks clickjacking via iframe embeds.

  • ⚠️ Add X-Content-Type-Options X-Content-Type-Options: nosniff prevents browsers from MIME-sniffing responses.

  • ⚠️ Set a Referrer-Policy Referrer-Policy: strict-origin-when-cross-origin is a safe default.

  • ⚠️ Set a Permissions-Policy Restrict browser features you don't use, e.g. Permissions-Policy: camera=(), microphone=(), geolocation=().

14. Priority To-Do List

  • P1 — Add a single, focused H1 to the homepage One <h1> per page. Write it as 'We help [audience] [do thing].' so an LLM can quote it verbatim.

  • P1 — Add more substantive homepage content AI models need 300+ words of visible body text to summarize and recommend a site. Add a value-prop paragraph, a short FAQ, and a 'how it works' section.

  • P1 — Rewrite the homepage H1 to be self-evident Replace clever copy with literal copy. 'We help X do Y' beats 'Reimagine Y'.

  • P1 — Add a discoverable CTA Place 'Contact sales' or 'Start free' in the top-right of the nav. LLMs cite the visible label.

  • P1 — Add JSON-LD structured data Start with Organization on the root layout and SoftwareApplication or Product on /pricing. Add FAQPage on any FAQ section.

  • P2 — Add lists or comparison tables Answer engines lift bulleted lists, numbered steps, and tables verbatim. Add at least 2 snippet-ready blocks to the homepage.

  • P2 — Raise your text-to-HTML ratio Strip unused inline scripts/styles and move large bundles to external files. AI crawlers struggle when most of the response is markup.

  • P2 — Create or enrich /llms.txt Follow the llmstxt.org spec:

    ```
# Your Brand

> One-line description of your site.

## Docs

- [Getting Started](https://yoursite.com/docs/start): How to get up and running.
- [API Reference](https://yoursite.com/docs/api): Full API details.

## About

- [About us](https://yoursite.com/about): Mission and team.
```

Include at least 2 section headings, 3+ linked resources, and a brief description per link. A rich llms.txt dramatically increases how often generative AI systems cite your content.

  • P2 — Add sameAs knowledge graph links to Organization schema Extend your Organization JSON-LD to include sameAs pointing to authoritative directories:

    ```json
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "Your Brand",
  "url": "https://yoursite.com",
  "sameAs": [
    "https://en.wikipedia.org/wiki/Your_Brand",
    "https://www.wikidata.org/wiki/Q12345678",
    "https://www.linkedin.com/company/your-brand",
    "https://www.crunchbase.com/organization/your-brand"
  ]
}
```

These links anchor your brand as a known entity in AI knowledge graphs, making it far more likely that generative models cite you by name rather than paraphrase.

  • P2 — Declare your brand name in Organization JSON-LD Add "name": "Your Brand" to your Organization or SoftwareApplication schema block. AI systems match structured-data names against training data to resolve your brand as a distinct entity. Without it, mentions of your brand may not be attributed to you.

  • P2 — Make your About/Team page reachable Add a top-nav or footer link to /about or /team so LLMs can identify the entity behind the site.

  • P2 — Add a /pricing page Even contact-us pricing benefits from a /pricing page that LLMs can link to in answers.

  • P2 — Populate your sitemap with canonical URLs The sitemap must list <loc> entries — empty sitemaps are worse than none.

  • P3 — Phrase a heading as a user question Use headings like 'How does pricing work?' or 'Who is this for?' — they map directly to conversational AI queries.

  • P3 — Publish a date signal Add <time datetime="2026-05-17"> or <meta property="article:published_time">. AI ranking heavily weights freshness.

  • P3 — Add outbound links to authoritative sources Link to Wikipedia, .gov or .edu resources, peer-reviewed studies, or major news outlets when making factual claims. Generative AI systems treat pages that cite authoritative sources as more trustworthy, which raises citation likelihood.

    Examples: statistics from Statista or Census.gov, definitions from Wikipedia, research from nature.com or pubmed.ncbi.nlm.nih.gov.

  • P3 — Add a canonical link Prevents dup-content drift and tells AI crawlers which URL is authoritative.

    ```html
<link rel="canonical" href="https://yoursite.com/" />
```

  • P3 — Complete Open Graph tags AI bots use OG for fast disambiguation. Add all four:

    ```html
<meta property="og:title" content="Your Page Title" />
<meta property="og:description" content="50–160 char description of this page." />
<meta property="og:image" content="https://yoursite.com/og-image.jpg" />
<meta property="og:url" content="https://yoursite.com/" />
<meta property="og:type" content="website" />
<meta property="og:site_name" content="YourSite" />
```

  • P3 — Add Twitter Card meta tags Used by social platforms and AI agents for richer previews.

    ```html
<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:title" content="Your Page Title" />
<meta name="twitter:description" content="50–160 char description." />
<meta name="twitter:image" content="https://yoursite.com/og-image.jpg" />
```

  • P3 — Add structured headings Use h2 for each section and h3 for sub-points. AI uses these to outline and chunk the page.

Report by CrawlProof. Reusable after every major website change.

Email yourself this report

Get a PDF copy of this audit in your inbox — handy for sharing with a client, dev, or teammate.