1. Crawl Summary

pass

Fetched 1 of 2 pages successfully

Target: https://www.reddit.com/r/singularity

Evidence

{
  "pages": [
    {
      "url": "https://www.reddit.com/r/singularity",
      "bytes": 190240,
      "status": 403,
      "fetchMs": 27
    },
    {
      "url": "https://www.reddit.com/login/",
      "bytes": 8430,
      "status": 200,
      "fetchMs": 14
    }
  ],
  "origin": "https://www.reddit.com",
  "target": "https://www.reddit.com/r/singularity"
}

P5

2. Data Found

Data Point	Found?	Source	Notes
Pricing	No	—	—
Customer logos	No	—	—
Social proof	No	—	—
Recent launches	No	—	—
Blog post activity	No	—	—
New hires	No	—	Often only on a /blog/team or LinkedIn page
Headline copy	No	—	—
Positioning	No	—	—
Executive team	No	—	—
Product/service descriptions	No	—	—
Case studies or testimonials	No	—	—
Contact/demo/signup paths	No	—	—

3. Homepage Audit

fail
Homepage could not be fetched
HTTP 403
P1

4. Schema / Structured Data Audit

fail
No JSON-LD structured data found
Add JSON-LD blocks (Organization, SoftwareApplication, FAQPage, BreadcrumbList) so AI answer engines can ingest your data without guessing.
P1

5. robots.txt and sitemap.xml Audit

fail
sitemap.xml not found
Add /sitemap.xml — required for reliable AI/SERP discovery.
P1
warn
robots.txt does not reference a Sitemap
Add `Sitemap: https://yoursite.com/sitemap.xml` to robots.txt.
P3

pass

robots.txt present

538 chars

Evidence

{
  "snippet": "# Welcome to Reddit's robots.txt\n# Reddit believes in an open internet, but not the misuse of public content.\n# See https://support.reddithelp.com/hc/en-us/articles/26410290525844-Public-Content-Policy Reddit's Public Content Policy for access and use restrictions to Reddit content.\n# See https://www.reddit.com/r/reddit4researchers/ for details on how Reddit continues to support research and non-commercial use.\n# policy: https://support.reddithelp.com/hc/en-us/articles/26410290525844-Public-Content-Policy\n\nUser-agent: *\nDisallow: /\n"
}

P5

6. LLM / AI Crawler Accessibility

fail
GPTBot blocked via wildcard
User-agent: * is disallowed from / and no explicit rule for GPTBot overrides it.
P1
fail
ClaudeBot blocked via wildcard
User-agent: * is disallowed from / and no explicit rule for ClaudeBot overrides it.
P1
fail
PerplexityBot blocked via wildcard
User-agent: * is disallowed from / and no explicit rule for PerplexityBot overrides it.
P1
fail
Google-Extended blocked via wildcard
User-agent: * is disallowed from / and no explicit rule for Google-Extended overrides it.
P1
fail
OAI-SearchBot blocked via wildcard
User-agent: * is disallowed from / and no explicit rule for OAI-SearchBot overrides it.
P1
fail
Applebot-Extended blocked via wildcard
User-agent: * is disallowed from / and no explicit rule for Applebot-Extended overrides it.
P1
fail
CCBot blocked via wildcard
User-agent: * is disallowed from / and no explicit rule for CCBot overrides it.
P1
warn
llms.txt missing
Add /llms.txt — a concise, link-rich summary that helps LLMs orient on your site.
P2
warn
skill.md missing
Add /skill.md describing what your site lets agents do — speeds up agent task routing.
P3
warn
/.well-known/security.txt missing
Publish a /.well-known/security.txt with at least a Contact: line. Crawlers and security researchers expect it; AI systems use it as a trust signal.
P3

7. Positioning Clarity

warn
H1 missing or too short to convey value
Add a clear, single-sentence H1 like 'We help X do Y.'
P1
fail
No discoverable CTA
Add a clearly-labeled Contact, Demo, or Sign up link to the nav or hero.
P1
warn
No clear About/Team link
Add an About or Team link in the nav or footer so LLMs can identify the company.
P2
warn
No pricing/plans link found
AI summaries commonly include pricing. Add a /pricing page even if pricing is custom.
P2
warn
Value-prop language not detected
Pages with phrases like 'we help X', 'platform for Y', 'built for Z' are easier for LLMs to summarize.
P3

8. Missing or Hard-to-Find Information

fail

12 data point(s) could not be found from public pages

· Pricing · Customer logos · Social proof · Recent launches · Blog post activity · New hires · Headline copy · Positioning · Executive team · Product/service descriptions · Case studies or testimonials · Contact/demo/signup paths

Evidence

{
  "missing": [
    "Pricing",
    "Customer logos",
    "Social proof",
    "Recent launches",
    "Blog post activity",
    "New hires",
    "Headline copy",
    "Positioning",
    "Executive team",
    "Product/service descriptions",
    "Case studies or testimonials",
    "Contact/demo/signup paths"
  ]
}

P1

9. Recommended Fixes

warn
Allow GPTBot in robots.txt
Add an explicit User-agent: GPTBot Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "for": "aibot.GPTBot"
}
```
P1
warn
Allow ClaudeBot in robots.txt
Add an explicit User-agent: ClaudeBot Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "for": "aibot.ClaudeBot"
}
```
P1
warn
Allow PerplexityBot in robots.txt
Add an explicit User-agent: PerplexityBot Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "for": "aibot.PerplexityBot"
}
```
P1
warn
Allow Google-Extended in robots.txt
Add an explicit User-agent: Google-Extended Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "for": "aibot.Google-Extended"
}
```
P1
warn
Allow OAI-SearchBot in robots.txt
Add an explicit User-agent: OAI-SearchBot Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "for": "aibot.OAI-SearchBot"
}
```
P1
warn
Allow Applebot-Extended in robots.txt
Add an explicit User-agent: Applebot-Extended Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "for": "aibot.Applebot-Extended"
}
```
P1
warn
Allow CCBot in robots.txt
Add an explicit User-agent: CCBot Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "for": "aibot.CCBot"
}
```
P1
warn
Rewrite the homepage H1 to be self-evident
Replace clever copy with literal copy. 'We help X do Y' beats 'Reimagine Y'.
Evidence
```
{
  "for": "positioning.what"
}
```
P1
warn
Add a discoverable CTA
Place 'Contact sales' or 'Start free' in the top-right of the nav. LLMs cite the visible label.
Evidence
```
{
  "for": "positioning.cta"
}
```
P1
warn
Publish a sitemap.xml
Generate /sitemap.xml automatically (Next.js: app/sitemap.ts). Include every canonical URL.
Evidence
```
{
  "for": "sitemap.exists"
}
```
P1
warn
Add JSON-LD structured data
Start with Organization on the root layout and SoftwareApplication or Product on /pricing. Add FAQPage on any FAQ section.
Evidence
```
{
  "for": "schema.any"
}
```
P1
warn
Add /llms.txt
A short Markdown-flavored summary at the root. Include your H1, value prop, top 5–10 links, and pricing summary.
Evidence
```
{
  "for": "llms_txt"
}
```
P2
warn
Make your About/Team page reachable
Add a top-nav or footer link to /about or /team so LLMs can identify the entity behind the site.
Evidence
```
{
  "for": "positioning.who"
}
```
P2
warn
Add a /pricing page
Even contact-us pricing benefits from a /pricing page that LLMs can link to in answers.
Evidence
```
{
  "for": "positioning.pricing"
}
```
P2
warn
Add /skill.md
Describe what an agent can do with your site (e.g., 'Search docs', 'Look up pricing'). Useful for agentic flows.
Evidence
```
{
  "for": "skill_md"
}
```
P3
warn
Publish /.well-known/security.txt
A security contact builds trust with crawlers and researchers. Minimal example: ``` Contact: mailto:security@yourdomain.com Expires: 2027-01-01T00:00:00.000Z Preferred-Languages: en ```
Evidence
```
{
  "for": "security_txt"
}
```
P3
warn
State your audience explicitly
Use phrases like 'Built for B2B SaaS marketing teams' on the homepage and About page.
Evidence
```
{
  "for": "positioning.audience"
}
```
P3
warn
Reference your sitemap in robots.txt
Add `Sitemap: https://yoursite.com/sitemap.xml` so crawlers don't have to guess.
Evidence
```
{
  "for": "robots.sitemap_ref"
}
```
P3

10. Priority To-Do List

warn
[ ] Allow GPTBot in robots.txt
Add an explicit User-agent: GPTBot Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Allow ClaudeBot in robots.txt
Add an explicit User-agent: ClaudeBot Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Allow PerplexityBot in robots.txt
Add an explicit User-agent: PerplexityBot Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Allow Google-Extended in robots.txt
Add an explicit User-agent: Google-Extended Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Allow OAI-SearchBot in robots.txt
Add an explicit User-agent: OAI-SearchBot Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Allow Applebot-Extended in robots.txt
Add an explicit User-agent: Applebot-Extended Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Allow CCBot in robots.txt
Add an explicit User-agent: CCBot Allow: / block so this AI crawler can read your site.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Rewrite the homepage H1 to be self-evident
Replace clever copy with literal copy. 'We help X do Y' beats 'Reimagine Y'.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Add a discoverable CTA
Place 'Contact sales' or 'Start free' in the top-right of the nav. LLMs cite the visible label.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Publish a sitemap.xml
Generate /sitemap.xml automatically (Next.js: app/sitemap.ts). Include every canonical URL.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Add JSON-LD structured data
Start with Organization on the root layout and SoftwareApplication or Product on /pricing. Add FAQPage on any FAQ section.
Evidence
```
{
  "priority": 1
}
```
P1
warn
[ ] Add /llms.txt
A short Markdown-flavored summary at the root. Include your H1, value prop, top 5–10 links, and pricing summary.
Evidence
```
{
  "priority": 2
}
```
P2
warn
[ ] Make your About/Team page reachable
Add a top-nav or footer link to /about or /team so LLMs can identify the entity behind the site.
Evidence
```
{
  "priority": 2
}
```
P2
warn
[ ] Add a /pricing page
Even contact-us pricing benefits from a /pricing page that LLMs can link to in answers.
Evidence
```
{
  "priority": 2
}
```
P2
warn
[ ] Add /skill.md
Describe what an agent can do with your site (e.g., 'Search docs', 'Look up pricing'). Useful for agentic flows.
Evidence
```
{
  "priority": 3
}
```
P3
warn
[ ] Publish /.well-known/security.txt
A security contact builds trust with crawlers and researchers. Minimal example: ``` Contact: mailto:security@yourdomain.com Expires: 2027-01-01T00:00:00.000Z Preferred-Languages: en ```
Evidence
```
{
  "priority": 3
}
```
P3
warn
[ ] State your audience explicitly
Use phrases like 'Built for B2B SaaS marketing teams' on the homepage and About page.
Evidence
```
{
  "priority": 3
}
```
P3
warn
[ ] Reference your sitemap in robots.txt
Add `Sitemap: https://yoursite.com/sitemap.xml` so crawlers don't have to guess.
Evidence
```
{
  "priority": 3
}
```
P3

AEO Audit for reddit.com

1. Crawl Summary

2. Data Found

3. Homepage Audit

4. Schema / Structured Data Audit

5. robots.txt and sitemap.xml Audit

6. LLM / AI Crawler Accessibility

7. Positioning Clarity

8. Missing or Hard-to-Find Information

9. Recommended Fixes

10. Priority To-Do List

https://www.reddit.com/r/singularity/

1. Crawl Summary

2. Data Found

3. Homepage Audit

4. Schema / Structured Data Audit

5. robots.txt and sitemap.xml Audit

6. LLM / AI Crawler Accessibility

7. Positioning Clarity

8. Missing or Hard-to-Find Information

9. Recommended Fixes

10. Priority To-Do List

Track this site week over week

Your charts, not these.