DN Agentic Web Readiness Index 2027
Most websites are readable by browsers but unreliable for agents. DN introduces a 100-point system for measuring whether a site can publish discoverable capabilities, prove who controls them and support safe, auditable execution.
What Matters
Agent readiness is not the same as SEO. Search engines mainly need to crawl, interpret and rank information. An AI agent may also need to compare an offer, understand terms, authenticate, request permission, execute an action, pay and recover from failure. A website becomes agent-ready only when machine understanding is paired with bounded, auditable action.
The Web Has a Machine-Action Gap
The human web communicates through pages, menus, buttons and visual context. Agents need explicit entities, stable identifiers, declared capabilities, predictable inputs, permission boundaries and machine-readable outcomes. A beautifully designed website can be nearly unusable to an agent. An unremarkable site with structured content and a dependable API may be far more valuable in the agentic economy.
The Eight-Layer DN Agentic Web Model
| Layer | Weight | What DN measures | Failure signal |
|---|---|---|---|
| 1. Discovery and capability publication | 14 | Crawlability, canonical URLs, sitemaps, ARD discovery, authoritative-domain catalogs and federated registries | Capabilities are hidden, stale, duplicated or inconsistently addressed |
| 2. Semantic clarity | 13 | Accurate JSON-LD, explicit entities, prices, availability, dates, authorship and policies | The agent must infer decisive facts from layout or prose |
| 3. Action interfaces | 14 | Documented APIs, structured forms, MCP tools, A2A interfaces or equivalent action surfaces | Only brittle visual clicking can complete a task |
| 4. Repository instruction readiness | 10 | AGENTS.md presence, scope, nested consistency, build and test commands, tool limits, secret handling and freshness | Coding agents guess how to build, test, change or secure the software |
| 5. Permission and identity | 14 | Authentication, scopes, consent, least privilege, delegated authority and revocation | Access is all-or-nothing or authority cannot be proven |
| 6. Transaction readiness | 13 | Machine-readable totals, payment options, confirmation, receipts, refunds and idempotency | Costs change silently or repeated requests duplicate a purchase |
| 7. Reliability and recovery | 10 | Error schemas, status visibility, retry rules, cancellation, rollback and support escalation | The agent cannot distinguish pending, failed and completed actions |
| 8. Trust and verification | 12 | Domain ownership, publisher verification, cryptographic trust metadata, provenance, audit logs and human appeal | The publisher or capability cannot be verified, challenged or contained |
DN Agentic Web Readiness Audit
Check only controls that are implemented and tested. The result is a diagnostic, not a certification.
Agent-Opaque
The site is primarily human-facing and machine interpretation or execution is unreliable.
Readiness gates: Discoverable: fail · Verifiable: fail · Executable: fail · Transactable: fail
ARD verification: Not published
Priority fixes- Discovery: Publish deliberate crawl rules, canonical URLs and an accurate sitemap.
- Semantics: Add valid structured data and make decisive facts explicit.
- Actions: Create a documented, schema-validated interface for priority tasks.
How the Score Works
Scores are additive, but the headline score is not enough. The Instruction Readiness layer improves execution readiness but cannot satisfy discovery, authorization, transaction or publisher-verification gates by itself. DN reports four independent gates: Discoverable, Verifiable, Executable and Transactable. A site cannot receive transactable status unless it exposes a structured action interface, scoped authority, pre-commitment pricing and transaction evidence. An 82-point information site may be highly agent-readable without being safe for autonomous purchasing.
ARD: The Missing Discovery Layer
Agentic Resource Discovery (ARD) is an emerging approach for publishing an authoritative ai-catalog.json under an organization’s own domain. A catalog can describe MCP servers, A2A agents, OpenAPI tools, agent skills, nested catalogs, verification metadata and native connection endpoints.
ARD is not a replacement for MCP, A2A or OpenAPI. It is a discovery and verification layer that can point to those interfaces.
What Existing Standards Solve, and What They Do Not
| Mechanism | Useful contribution | Not sufficient for |
|---|---|---|
| robots.txt | Standardized crawler access rules under the Robots Exclusion Protocol | Delegated authority, transaction permission or contractual consent |
| Schema.org / JSON-LD | Explicit entities, attributes, relationships and potential actions | Proving an endpoint is secure, current or authorized for autonomous execution |
| llms.txt | A proposed convention for presenting concise, model-friendly site resources | A universal authorization or security standard; adoption and interpretation vary |
| OpenAPI and APIs | Structured inputs, outputs, authentication and predictable programmatic access | Safe delegation unless scope, confirmation and recovery are designed explicitly |
| MCP | A standardized method for AI applications to connect with tools, resources and workflows | Trusting every exposed tool or eliminating the need for least privilege and validation |
| WCAG | Accessible, operable and understandable experiences that often improve machine clarity | Machine payment, identity, task state or agent-specific governance |
DN treats llms.txt as an emerging convention, not a guaranteed ranking factor or authorization mechanism. Publishing one does not make a website agent-ready.
AGENTS.md: The Repository Execution Layer
AGENTS.md is an open Markdown format for telling coding agents how to work inside a software repository. It can declare project context, build commands, tests, code conventions, security restrictions, pull-request rules and other instructions normally explained to a new developer. Nested files can provide narrower instructions for subprojects; the nearest applicable file takes precedence, while an explicit user instruction remains higher priority.
This is a material readiness signal for organizations whose products, APIs, MCP servers or agent services are maintained by software agents. It reduces repository exploration, prevents avoidable build and test failures and makes operational restrictions explicit. It does not publish a public capability, verify a publisher, authorize an action or secure a payment.
AGENTS.md tells coding agents how to work safely in the codebase.DN Repository Instruction Readiness Test
- Root discovery: a current
AGENTS.mdexists at the repository root. - Scope clarity: nested files identify the directories and tasks they govern without unresolved conflicts.
- Executable setup: install, build, lint and test commands are complete enough to run.
- Verification: required checks and acceptance criteria are named.
- Security boundaries: secrets, production data, destructive operations and prohibited tools are addressed.
- Change discipline: code, documentation and pull-request expectations are explicit where relevant.
- Freshness: commands and paths match the current repository and are reviewed after material architecture changes.
Hard rule: the mere presence of AGENTS.md earns no automatic trust badge. DN must validate that referenced commands, scopes and restrictions are internally consistent and operational.
The Agent Discovery Protocol Stack
| Layer | Primary purpose | What it does not prove |
|---|---|---|
| ARD | Publishing, discovering and verifying capabilities | That a capability performs reliably |
| MCP Registry | Finding registered MCP servers | That every server is secure or operational |
| MCP | Connecting AI applications to tools and resources | That the publisher should be trusted |
| A2A Agent Card | Advertising an agent’s identity and capabilities | That tasks will be completed successfully |
| OpenAPI | Describing API operations and schemas | That an agent is authorized to use them |
| Agent marketplace | Commercial discovery and distribution | Independent reliability or ownership |
| Traditional directory | Human-oriented listings and comparisons | Endpoint freshness or verified execution |
The New Funnel: From Search Visibility to Agent Selection
Be discovered
Stable URLs, crawl rules, sitemaps and references allow an agent or retrieval system to locate the relevant resource.
Be understood
Structured entities, precise language, visible evidence and consistent identifiers reduce inference risk.
Be shortlisted
Price, availability, jurisdiction, reliability, policy and proof must be comparable with alternatives.
Be authorized
The agent proves who delegated the task and receives only the authority required to complete it.
Be transacted with
The site exposes a bounded action with known total cost, confirmation, receipt and repeat protection.
Be accountable
Every material action can be inspected, cancelled, disputed or escalated to a human.
DN Alpha Thesis: The Next SEO Metric Is Executability
Human traffic rewards attention. Agent traffic will increasingly reward decision confidence per unit of machine effort. The commercial winners may not be the sites producing the most content. They may be the sites that make accurate comparison and safe execution cheapest for an agent. DN calls this Machine Decision Yield: verified decision value divided by the tokens, latency, uncertainty and action risk required to obtain it.
Verified Capability Yield
This DN metric measures whether published capabilities produce verified outcomes rather than merely appearing in a catalog. Supporting measurements include discovery success rate, publisher-verification rate, connection success rate, paid execution success rate, median discovery-to-execution time, cost per successful execution, stale capability rate and human-rescue rate.
Practical Roadmap by Organization
What Would Prove This Thesis Wrong?
The index would matter less if general browser agents become so reliable that structured interfaces deliver no measurable improvement in completion, cost or safety. It would also weaken if a single closed platform intermediates nearly all agent transactions, making open-web readiness commercially irrelevant. DN will test the thesis by tracking whether higher scores correlate with lower task failure, faster completion, fewer human rescues and higher verified conversion.
Methodology and Limitations
Version: DN-AWRI v1.2, September 2026. The index evaluates public and organization-reported controls across eight weighted layers, including repository instruction evidence where an applicable software repository exists. The self-audit above is educational. A verified DN benchmark requires technical inspection, structured-data validation, catalog and endpoint tests, repeated task execution, permission review and failure-recovery testing.
Repository evidence: Instruction Readiness is assessed from applicable root and nested AGENTS.md files, the current repository structure and reproducible setup and verification commands. Public scoring records the repository and commit tested, applicable instruction paths, test date and commands excluded for safety or access reasons. DN does not execute destructive commands, expose secrets or treat prose restrictions as proof that technical controls exist.
Evidence boundary: The presence of ai-catalog.json, an ARD listing, MCP endpoint or registry entry does not prove that a capability is safe, operational or authorized. Verified readiness requires publisher validation, endpoint testing, permission review and repeated task execution. Scores should be reported with the date, tested pages, task set, user role, geography and authentication state.
Change log: Version 1.2, October 9, 2026, adds Repository Instruction Readiness as an eighth scored layer, introduces validated AGENTS.md controls and rebalances the index to preserve a 100-point total. Version 1.1 added Agentic Resource Discovery, authoritative-domain catalogs, federated registry discovery, publisher verification and paid MCP endpoint controls. Version 1.0 established the original model and maturity bands.
Primary Sources
Frequently Asked Questions
What is an agent-ready website?
An agent-ready website can be discovered and interpreted by machines and provides safe, structured ways to perform relevant actions with explicit permission, confirmation and recovery.
Is agentic web optimization the same as SEO?
No. SEO focuses primarily on search discovery and ranking. Agentic readiness also covers comparison, execution interfaces, delegated authority, payments, reliability and accountability.
Does structured data improve AI visibility?
Structured data gives machines explicit clues about entities and page meaning. It can reduce ambiguity, but no markup guarantees inclusion, ranking, citation or selection by an AI system.
Does a website need an llms.txt file?
Not necessarily. It is an emerging convention and may make selected resources easier to find, but it is not a universal requirement, ranking guarantee or permission system.
Does allowing an AI crawler authorize an AI agent to transact?
No. Crawl access governs retrieval. A transaction requires separate identity, authority, consent, scope and confirmation controls.
Does every business need an MCP server?
No. A documented API or well-structured form may be sufficient. MCP becomes useful when an organization wants compatible AI applications to discover and invoke defined tools or resources.
Does publishing an ai-catalog.json file make a website agent-ready?
No. It makes capabilities easier to publish and discover. Complete readiness also requires accurate metadata, publisher verification, secure interfaces, scoped permissions, reliable execution, payment controls and failure recovery.
What is the most important first step?
Make decisive information explicit and consistent: identity, offering, price, availability, terms, dates and contact or escalation routes. Safe action interfaces should follow a dependable information layer.
Can an information-only publisher score highly?
Yes. A publisher can be highly agent-readable and trustworthy without offering transactions. DN reports maturity type and critical gates alongside the numerical score.
How often should readiness be tested?
Quarterly for stable sites and after any major redesign, API change, authentication change, checkout update or security incident.
Does AGENTS.md replace ARD, MCP or A2A?
No. ARD and registries support external discovery, MCP connects tools and resources, and A2A Agent Cards support agent-to-agent identity and capability publication. AGENTS.md supplies repository-level working instructions to coding agents.
Does every website need an AGENTS.md file?
No. It is most relevant when a website, API, MCP server or agent product has a software repository that coding agents maintain. Information-only sites without an applicable repository should be reported as not applicable rather than automatically unsafe.
Can a repository score highly because a file merely exists?
No. DN tests clarity, scope, command validity, security guidance, conflict handling and freshness. A stale or generic file can reduce confidence.
Can the DN score certify that a website is secure?
No. The self-audit is a diagnostic. Security certification requires deeper technical testing, threat modeling and evidence beyond public website signals.
Build for the Customers That Will Never See Your Homepage
The agentic web will reward sites that are clear enough to understand, structured enough to compare and controlled enough to trust.
Explore DN Agentic Finance ResearchAbout the Author
