NeuralBot — the Neural Partners bot family

One identity, one reputation. When NeuralBot fetches your site it identifies itself honestly — a real user-agent string and a link back to this page. If a NeuralBot variant shows up in your logs, this is where you find out what it wanted, why we sent it, and how to keep it out if you'd rather it didn't come back. (One request per Readiness scan uses a plain browser user-agent by design — the cloaking baseline, explained below.)

Our bots

The NeuralBot family has two members today. Each is scoped to one job — neither crawls the open web or builds a general index. New variants are documented here as they ship.

NeuralBot-Readiness

• What it does: scans a URL you submit for AI-agent readiness — structured data, meta/OG tags, robots and llms.txt discovery, cloaking, and more.
• Why: powers the free Agentic Readiness Check on our homepage.
• Cadence: on-demand only — it fires when someone runs a scan against your domain. Not a crawler: a single scan is a bounded ~13 GET requests (plus at most one courtesy retry per request if your server replies 429/503), and results are cached for one hour per host. It never runs on a schedule.
• User-agent: NeuralBot-Readiness/1.0 (+https://neuralpartners.ai/bot)
• Block it: disallow the NeuralBot-Readiness token in your robots.txt (see below).

NeuralBot-Ingest

• What it does: pulls OEM product-catalog data — product details, pricing, images, and inventory — from the storefronts, APIs, and image CDNs of manufacturers a Neural Showrooms client carries.
• Why: keeps client showroom catalogs current with the manufacturers they stock.
• Cadence: recurring, scheduled — daily catalog/inventory extraction (~06:00 UTC), a daily HTTP inventory pull, and weekly product/image pulls, plus on-demand re-extractions. Scoped to onboarded manufacturers' own sites and APIs only (operator-configured per OEM) — never open-web crawling or arbitrary domains.
• User-agent: NeuralBot-Ingest/1.0 (+https://neuralpartners.ai/bot)
• Block it: disallow the NeuralBot-Ingest token (see below). Robots enforcement for this variant is rolling out — for an immediate opt-out, email bots@neuralpartners.ai.

The cloaking baseline request

One request in a Readiness scan is sent with a standard web-browser user-agent (an ordinary Mozilla/5.0 … Chrome string) instead of the NeuralBot one. This is the single exception to “every request says NeuralBot,” and it is deliberate: to check for cloaking, we compare the page a normal browser receives against the page an AI agent (our NeuralBot-Readiness user-agent) receives. If you see one browser-identified hit alongside the NeuralBot-Readiness hits during a scan, that is it — a single read-only comparison fetch of your homepage, from the same source, counted within the request total above. We never present ourselves as another company's crawler.

How to block us

Each bot obeys its own robots.txt token. Add the token for the bot you want to block.

NeuralBot-Readiness — honored today

We check your robots.txt before we scan and refuse if it's there (if we scanned your host in the last hour, a cached result may still return until it expires):

User-agent: NeuralBot-Readiness
Disallow: /

This applies to everyone — the block stops a scan whether you trigger it or someone else does. We honor your robots.txt over anyone's request, including a claim to own the site (we can't verify that yet). If you've blocked NeuralBot-Readiness and later want to run the check on your own domain, remove the rule first. An ownership-verified override — so a verified owner can scan a still-blocked site — is on our roadmap.

NeuralBot-Ingest — enforcement rolling out

The token below is live for declaration, and robots enforcement is being added as this variant adopts the standard. Until it lands, the fastest opt-out from catalog ingestion is to email bots@neuralpartners.ai. NeuralBot-Ingest only ever fetches from manufacturers we have onboarded — it never touches arbitrary sites:

User-agent: NeuralBot-Ingest
Disallow: /

Verifiability & egress

NeuralBot is being registered for Cloudflare Verified Bots, with reverse-DNS and a published stable-egress range rolling out across the Neural Partners platform. Until that lands, you can confirm any request by emailing bots@neuralpartners.ai — a human will verify it came from us.

Report abuse or ask a question

If a NeuralBot variant is hitting your site harder than it should, ignoring your robots.txt, or you just want to verify a request came from us, email bots@neuralpartners.ai. A human reads that inbox.