Building the Discovery Layer for the Agentic Web
An agent with five tools can receive five descriptions in its prompt. An enterprise agent with five thousand possible tools, skills, data interfaces, and peer agents cannot. Loading every contract into context raises cost and ambiguity, and a hard-coded allowlist goes stale as capabilities move, split, and change ownership. The problem is no longer just tool calling. It is finding the right callable resource before the model has to reason about its full schema.
Agentic Resource Discovery (ARD) defines a search-first answer. Publishers advertise resources through a small, protocol-neutral catalog. Registries crawl and index those catalogs. Clients search a standard API, verify the selected result, and then connect through the resource’s native protocol. A model receives the contract for one selected resource instead of the descriptions of the entire ecosystem.
That sequence is useful, but it is not a complete production architecture. Search relevance does not establish publisher identity. An identity does not grant a caller permission. A valid agent card does not make an action safe. Federation does not decide which external registries an enterprise should trust. Those decisions stay with the consuming system.
This paper turns the ARD v0.9 draft into an implementation model. It begins with the minimum compliant publisher and registry, then adds the trust, policy, execution, and evidence boundaries a governed agent platform requires. The concrete case is a two-plane agent platform design: a closed execution plane owns the catalog of runnable capabilities, and an operational control plane owns selection policy, approvals, and durable operational state. The platform is a design for how such systems should be built, not a deployed system, and the architecture below defines the ARD compatibility layer for it. The work is complete as a specification analysis and reference architecture, with interfaces, state transitions, controls, rollout stages, and acceptance measures. It does not claim production latency, retrieval quality, or safety results, because nothing has been measured.
When capability discovery stops fitting in a prompt
Most agent runtimes combine three concerns that become less stable as the resource set grows:
- Inventory answers what exists and where its current contract lives.
- Selection answers which resource best matches the present task.
- Authority answers whether this caller may use that resource for this effect, with these inputs, now.
A static tool array appears to answer all three, but only because the array is small and assembled by trusted application code. At ecosystem scale, inventory comes from many publishers, selection becomes a retrieval problem, and authority must stay local to the consumer. Conflating them produces one of two failures: every advertised capability is treated as executable, or every new capability requires a central application release.
ARD separates the concerns by making discovery a pre-context operation. The catalog entry is deliberately smaller than the native agent, server, or API description. The registry uses that entry to find candidates. Only after selection does the client retrieve the A2A card, MCP server card, OpenAPI description, skill, or nested catalog it needs to connect.
Implementation frame
One search contract, three independent decisions
The registry ranks resources against the task and explicit filters.
The client verifies publisher binding, artifact evidence, and freshness.
Local policy narrows the verified candidate to an allowed capability and effect.
Figure 1 · The Agentic Resource Discovery lifecycle
A canonical manifest is projected into an ARD catalog, indexed by an allowlisted registry, verified against trust and local authorization, and connected through the selected resource's native protocol.
The contract ARD adds
The current specification is version 0.9 and remains a proposal. Its catalog
schema currently uses "specVersion": "1.0"; these are different version
numbers for different artifacts. Implementations should pin both the document
revision and the machine-readable schemas they test against.
ARD adds four interoperable surfaces without replacing the resource’s native protocol:
| Surface | Required behavior | What it does not decide |
|---|---|---|
| Static catalog | publish /.well-known/ai-catalog.json over HTTPS |
runtime authorization or invocation |
| Catalog entry | identify a resource and provide exactly one url or inline data value |
whether the resource is trusted by a consumer |
| Dynamic registry | expose REST POST /search; optionally expose /explore, /agents, MCP, or A2A wrappers |
whether the highest-ranked result may execute |
| Trust manifest | carry identity, attestations, provenance, and a signature when supplied | local policy, user delegation, or source access |
Every entry has a stable logical identifier of the form
urn:air:<publisher>:<namespace>:<name>. The publisher segment is a verifiable
fully qualified domain name. The entry’s url may change when a service moves;
the URN should not. That separation gives indexes and policies a durable noun
without treating a network location as either identity or permission.
The type field is an IANA-style media type that tells a client what artifact
the url or data contains. ARD is artifact-agnostic: it can advertise A2A
agent cards, MCP server cards, OpenAPI documents, Markdown skills, datasets,
nested catalogs, and registries. The A2A and MCP card media types used by the
draft are de facto types whose formal registration is still pending, so an
intermediary should preserve them without inventing stricter validation than
the native protocol defines.
The scope boundary matters as much as the surfaces. ARD does not specify how to execute a tool, negotiate an A2A task, establish an MCP session, install a skill, obtain source credentials, or approve a state-changing action. It discovers a resource description. Native clients and local control planes own the rest.
Publish a projection, not a second source of truth
The safest publisher begins with an existing code-owned capability catalog. That catalog already holds what the runtime needs: the stable internal identifier, owner, lifecycle state, input contract, effect class, allowed tool profile, protocol adapter, and deployment location. A build step projects the discoverable subset into ARD instead of asking operators to maintain a second manifest by hand.
In the reference platform design, the execution-plane registry and coding-fleet catalog are canonical. They separate discovery metadata from runtime policy and map requests into a closed capability table. An ARD projector reads those manifests, rejects resources that are disabled or not externally describable, and emits catalog entries only for capabilities that have a real native artifact. A role description with no runnable endpoint may be useful to the control plane, but it must not be advertised as an invokable A2A agent because its name sounds agentic.
The following target catalog is illustrative. It becomes truthful only after the fleet-operations capability has a published A2A adapter and the example domains are replaced with publisher-controlled endpoints.
{
"specVersion": "1.0",
"host": {
"displayName": "Example Agent Platform",
"identifier": "did:web:platform.example.com",
"documentationUrl": "https://platform.example.com/agents"
},
"entries": [
{
"identifier": "urn:air:platform.example.com:fleet:operations",
"displayName": "Fleet Operations Agent",
"type": "application/a2a-agent-card+json",
"url": "https://agents.example.com/fleet/.well-known/agent-card.json",
"description": "Inspects the registered fleet and proposes bounded remediation.",
"tags": ["operations", "agents", "read-mostly"],
"capabilities": ["inspect_fleet", "summarize_run", "propose_remediation"],
"representativeQueries": [
"inspect the agent fleet and explain unhealthy routes",
"summarize the latest failed run without changing external state"
],
"version": "1.0.0",
"updatedAt": "2026-08-13T00:00:00Z",
"metadata": {
"effectClass": "read-mostly",
"catalogSource": "fleet-manifest-v3"
},
"trustManifest": {
"identity": "spiffe://platform.example.com/fleet/operations",
"identityType": "spiffe"
}
}
]
}
The projection follows six rules:
- Generate identifiers deterministically. A rename changes display text, not the URN. A genuinely new capability receives a new URN.
- Emit exactly one delivery form.
urlreferences an artifact;dataembeds it. Supplying both or neither fails the build. - Describe behavior, not aspiration. Capabilities and representative queries come from tested runtime behavior. The draft schema expects two to five representative queries when the field is present.
- Separate discoverability from permission. Metadata can help a consumer
filter results, but a tag such as
read-mostlyis not an authorization grant. - Version the release. Store the catalog, its normalized digest, projector version, and source-manifest revision together so a registry record can be reproduced.
- Withdraw explicitly. Disabled resources disappear from the next catalog and enter a tombstone or revocation feed used by internal consumers. A cache must not preserve an executable capability forever.
The static file is served at https://<publisher>/.well-known/ai-catalog.json
with Content-Type: application/json. The publisher guide recommends
permissive CORS for broad discovery. The production endpoint should also send
an ETag, cache policy, and a stable Last-Modified value so registries can
revalidate without transferring an unchanged catalog.
Schema and semantic conformance belong in CI, before publication:
./conformance/bin/conformance-test manifest public/.well-known/ai-catalog.json
npx ajv-cli validate \
-s spec/schemas/ai-catalog.schema.json \
-d public/.well-known/ai-catalog.json
The first command exercises the ARD project’s catalog checks. The second pins the official JSON Schema. An organization-specific test should also confirm that every advertised URL exists, its native document validates, its digest is recorded, and its URN resolves to exactly one active internal capability.
Build the registry as an index, not an authority
A minimal registry accepts configured publisher seeds, fetches catalogs,
normalizes entries, creates a search index, and implements POST /search.
Production requirements begin before ranking.
Bound the ingestion path
The crawler is a network client processing publisher-controlled input. Its seed list, redirects, DNS resolution, response size, content type, parse depth, and request budget must be bounded. Private, loopback, link-local, and metadata service addresses are rejected unless a separately isolated internal crawler is explicitly assigned to them. Redirects are resolved and checked at every hop. Nested catalogs have a depth and total-entry limit. Git, npm, and OCI ingestion remain separate optional connectors, not URLs the crawler decides to explore on its own.
Each crawl produces an observation instead of overwriting a row in place:
| Stored field | Purpose |
|---|---|
| resource URN and publisher | stable identity and ownership partition |
| source catalog URL | exact observation origin |
| catalog and artifact digest | change detection and later verification |
first seen, last seen, and publisher updatedAt |
freshness and incident analysis |
| normalized entry plus raw signed bytes | searchable fields and reproducible evidence |
| validation result and schema revision | explain why an entry was accepted or quarantined |
| active, stale, withdrawn, or revoked state | prevent silent indefinite caching |
This temporal record matters when two registries disagree. A merged search response should not silently choose whichever copy arrived last. The client can prefer a direct publisher observation over a federated copy, compare digests and versions, and reject an older or unverified binding.
Combine deterministic filters with semantic retrieval
The search request contains required natural-language query.text, optional
structured query.filter, a federation mode, and pagination. The registry
first applies tenant, visibility, lifecycle, media-type, publisher, and policy
filters. It then ranks the allowed subset using the entry’s display name,
description, tags, capabilities, and representative queries.
{
"query": {
"text": "inspect a failed fleet run and explain the likely route",
"filter": {
"type": ["application/a2a-agent-card+json"],
"publisher": ["platform.example.com"],
"metadata.effectClass": ["read-mostly"]
}
},
"federation": "none",
"pageSize": 5
}
Filters enforce declared constraints. Semantic retrieval handles the gap
between a user’s task and the publisher’s vocabulary. The registry returns
both the standard entry and a score from 0 to 100, which under the
specification means semantic relevance only. It must never be converted into
a trust badge, compliance decision, or permission threshold.
Ranking should be reproducible enough to debug. The registry records the query digest, filter set, embedding model and version, index generation, candidate set, final ordering, and upstream source. A response can explain matched capabilities or representative queries without exposing embeddings or private catalog entries.
Add federation only after local semantics are stable
ARD defines auto, referrals, and none federation modes. none searches
only the current registry. referrals returns other registry entries that the
client may choose to query. auto lets the registry query upstream registries
and merge results.
An enterprise client should begin with none. Referral following needs its own
registry allowlist, hop limit, deadline, result cap, deduplication by URN and
digest, and loop detection. Automatic federation moves those decisions into
the registry and should be enabled only when its trust zones and merge rules
are observable. The client never invents a registry location from model output.
It searches endpoints supplied by configuration or an already verified
catalog.
Turn a result into a governed connection
A search response is a candidate. Before a resource becomes callable, the consumer moves it through an explicit resolution state machine:
| State | Required evidence | Failure behavior |
|---|---|---|
| Discovered | result came from an allowed registry and matches filters | retain as non-executable metadata |
| Publisher-bound | URN publisher matches the authorized domain or delegated publisher | quarantine conflicting namespace claims |
| Verified | signature, identity, provenance, digest, and freshness satisfy policy | reject or require review; never downgrade silently |
| Resolved | URN and version map to one approved internal capability or isolated external adapter | refuse ambiguous or unknown mappings |
| Authorized | caller, tenant, effect, scopes, inputs, and approval state pass local policy | return a policy denial with evidence |
| Connected | pinned artifact validates under its native protocol | terminate on drift or contract mismatch |
The trustManifest is optional in ARD, so its absence is a policy input, not
a parser error. A public read-only resource might be permitted without a
signature. A financial or administrative agent might require a domain-bound
SPIFFE identity, a recognized attestation, fresh provenance, and a pinned
artifact digest. The consumer defines that profile; the registry’s relevance
score does not.
After authorization, the client fetches the selected artifact, validates its native schema, compares its digest with the verified observation, and creates the appropriate adapter. An A2A client sends tasks to an A2A agent. An MCP client negotiates an MCP connection. An OpenAPI client applies its own authentication and request contract. A skill loader treats code-bearing or instruction-bearing content as supply-chain input and applies installation, sandbox, and review policy before exposing it to a model.
Only the selected resource contract enters the model context. Credentials, network handles, policy rules, and approval tokens stay in application code. The model can propose a capability and arguments, but it cannot choose a new registry, bypass verification, widen tool scope, or turn a read request into a write effect.
Wiring ARD into a governed agent platform
The reference platform design has the authority separation ARD needs. The execution plane’s versioned manifests expose nodes, edges, triggers, and actions for discovery, while its code-owned catalogs remain the policy authority. The coding fleet selects roles from explicit signals. The run API maps requests into a closed capability table and rejects an unknown capability instead of letting a caller supply an arbitrary model or tool profile.
The control plane is the operational counterpart. It records run state, narrows requested tools, and owns the approval boundary for state-changing work. A model can prepare bounded action data, but code owns the action class, schema, destination, credentials, and handler. Approval persists independently of the model branch, and execution returns a durable receipt.
ARD fits between intent and that closed execution plane:
task → allowed registry → ARD search → candidate → trust and policy → stable capability → native adapter → governed run → receipt
ARD support adds six components without weakening either plane:
- Execution-plane catalog projector. Converts approved executable manifest entries into versioned ARD entries. Definition-only roles remain searchable inside the control plane but are not misrepresented as network agents.
- ARD publisher endpoint. Serves the generated catalog and native cards from a publisher-controlled domain. CI prevents a catalog release whose resource card or internal capability mapping is missing.
- Control-plane discovery client. Queries only configured registries, applies deterministic filters, and records the search evidence with the task.
- Trust and policy resolver. Verifies the publisher and artifact, then maps the URN and pinned version into a closed platform capability.
- Native protocol adapters. Translate authorized A2A, MCP, OpenAPI, or skill contracts into internal run requests. The adapter can narrow the operation but cannot add tools or privileges absent from the capability.
- Discovery receipt. Stores registry, query digest, result URN, score, trust evidence, policy decision, artifact digest, run ID, action approval, and final outcome as one correlated chain.
| Existing boundary | ARD addition | Invariant preserved |
|---|---|---|
| execution-plane manifest and capability table | generated catalog entry and native card | code still decides what can run |
| deterministic role matching | registry search as another candidate source | search does not grant authority |
| control-plane run adapter | ARD client and stable resolver | requested tools remain narrowing-only |
| prepared action and approval | selected resource recorded with the action digest | approval binds the exact discovered capability |
| status and event history | discovery and verification evidence | runtime state remains authoritative |
The first usable integration is deliberately narrow: publish one read-only fleet-operations capability, search it from a local control-plane registry client, and resolve it back into the run API. That proves the identity and policy path without giving an external discovery result a new execution mechanism. State-changing resources follow only after approval binding, revocation, and receipt tests pass.
One Google Cloud implementation
The following architecture implements the publisher, registry, control plane, and closed execution boundary on Google Cloud. ARD remains the portable contract: the catalog generator, registry database, vector index, control plane, and native adapters can be replaced independently.
Figure 2 · An Agentic Resource Discovery implementation on Google Cloud
Canonical platform manifests and approved external catalogs enter bounded publication and crawling paths. A Google Cloud registry combines normalized records with semantic search. The control plane verifies, authorizes, and pins a candidate before a closed execution plane invokes it and records evidence.
01 · Publish
02 · Ingest
03 · Index + search
04 · Discovery control plane
05 · Connect
06 · Record
Map cloud services to responsibilities
| Responsibility | Google Cloud surface | Implementation boundary |
|---|---|---|
| Publish the well-known catalog | external Application Load Balancer and Cloud Run | custom domain, managed TLS, path routing, immutable catalog response |
| Validate each release | Cloud Build or CI plus ARD conformance tooling | schema, semantic, native-card, URL, digest, and mapping checks |
| Crawl approved publishers | Cloud Scheduler, Cloud Tasks, and Cloud Run | bounded concurrency, retries, egress policy, and dead-letter handling |
| Store registry history | transactional record store plus BigQuery evidence | current records separated from append-only crawl and experiment history |
| Rank resources | Vertex AI embeddings and Vector Search | semantic retrieval over a deterministically filtered candidate set |
| Serve the ARD API | private or public Cloud Run service | required REST /search; optional /explore, /agents, and wrappers |
| Govern the internal inventory | Agent Registry as an optional projection | inventory and governance view, not assumed to be a native ARD API |
| Bind workload identity | Agent Identity, IAM, and source-native authorization | cryptographic workload identity without replacing user or source permissions |
| Enforce execution policy | control-plane services and optional gateway controls | registry allowlist, trust profiles, scope narrowing, approval, and revocation |
| Execute capabilities | execution-plane runtime on Cloud Run or another bounded runtime | only registered capability IDs and native adapters are callable |
| Correlate evidence | Cloud Logging, Trace, and BigQuery | search-to-receipt chain with cost, quality, policy, and safety measures |
The registry service can stay private for an enterprise deployment; public catalog publication does not require public search or execution endpoints.
Vertex AI Vector Search is one suitable semantic index, but it is not the registry’s record of truth. The normalized entry store holds the exact catalog data, source, version, digest, and lifecycle state, and vector results are joined back to active records and policy filters before they are returned.
Google Cloud Agent Registry provides a managed inventory and governance model
for agents, MCP servers, tools, skills, and endpoints. Its current product
documentation describes keyword, prefix, and semantic search, but it does not
document ARD’s POST /search contract or ai-catalog.json crawling as a
native interface. This architecture therefore treats Agent Registry as an
optional governed projection, and the ARD compatibility API stays a small
service until native ARD support is documented and passes the same
conformance suite.
Agent Identity can bind a cryptographic workload identity to the selected agent, while IAM, user delegation, and the source system still determine what that workload may access. ARD says which resource was selected; identity says which workload arrived; policy and the source say what it may do.
Prove the boundary before federation
The implementation is accepted in layers. Each stage leaves the previous authority boundary intact.
- Catalog-only. Generate one catalog from the canonical execution-plane manifest, serve it at the well-known path, and pass the official manifest tests.
- Local search, dry run. Ingest the catalog, implement
POST /search, and compare its results with the platform’s deterministic role matcher. Return candidates but do not create runs. - Read-only connection. Resolve one verified URN into one existing read-only capability and record the complete discovery receipt.
- Approval-gated effects. Bind a selected resource version and digest into the control plane’s prepared action so approval cannot be reused after resource drift.
- Controlled federation. Add referrals between explicitly allowed registries, then test loop, duplicate, conflict, timeout, and revocation behavior before enabling automatic federation.
The ARD repository includes a zero-dependency conformance command for manifests and registries. Both are release gates:
./conformance/bin/conformance-test manifest \
https://publisher.example.com/.well-known/ai-catalog.json
./conformance/bin/conformance-test registry \
https://registry.example.com/api
Protocol conformance alone does not establish retrieval quality or operational safety. The evaluation corpus should contain real approved tasks paired with acceptable and unacceptable capabilities, ambiguous near-neighbors, withdrawn versions, conflicting publisher claims, stale registry copies, malformed cards, unavailable endpoints, and state-changing resources that require approval. Four retrieval paths form the useful comparison:
| Path | Candidate selection | Purpose |
|---|---|---|
| A · Prompt inventory | place every resource description in model context | measure the cost and confusion of the current small-system pattern |
| B · Local matcher | use the platform’s deterministic signals | preserve an explainable in-platform baseline |
| C · ARD search | semantic search plus filters over catalog entries | measure cross-publisher discovery quality |
| D · Hybrid | ARD candidates plus control-plane policy and closed execution | measure the complete production boundary |
A stated expectation keeps the comparison falsifiable. For a single-publisher deployment whose resources already live in the closed catalog, path B should match or beat path C on precision and latency: the deterministic matcher encodes the same capability table that search can only approximate. Paths C and D earn their extra hop when resources cross publisher boundaries, vocabularies diverge, or the inventory outgrows a hand-maintained matcher. If ARD search cannot beat the local matcher after external publishers are federated, the discovery layer has not paid for its latency and operational cost.
The primary retrieval measures are top-k recall, mean reciprocal rank, incorrect publisher rate, stale-selection rate, and prompt tokens introduced after selection. Operational measures are search latency, verification latency, end-to-end cost, catalog onboarding time, crawler freshness, referral fan-out, and revocation-to-denial time. Governance measures are policy-denial precision, approvals requested, digest mismatches, unknown-capability rejections, and unauthorized invocations. The last measure has an acceptance target of zero.
Failure tests matter more than a polished happy path. A high-scoring malicious resource must remain non-executable. A valid signature from a disallowed publisher must fail policy. An approved action must fail if its selected card or capability digest changes. A withdrawn catalog entry must stop resolving even when its vector remains in an older index. A registry timeout must not cause the model to invent a URL or silently expand federation.
What this implementation establishes
ARD supplies the missing discovery plane between a task and a large, distributed capability ecosystem. Its strongest design choice is restraint: the catalog is an artifact-neutral envelope, the registry is a search service, and invocation returns to the native protocol. That makes ARD suitable as a compatibility layer around an existing governed platform rather than a replacement for its runtime and approval contracts. The stable URN is a policy key, not a credential. A search score explains task fit and nothing more. A closed execution plane is what makes open discovery governable: the control plane may discover across publishers, but the execution plane still runs only registered capabilities under narrowed tools, explicit effects, approvals, and durable receipts.
ARD remains a draft and its media types may evolve. The Google Cloud product set does not currently expose this reference architecture as one managed feature, and the reference platform is a design rather than a deployed system. That makes the first build small and testable: publish one truthful catalog projection, discover one read-only capability, verify it, resolve it into the closed runtime, and retain the evidence from query to receipt. Once that path is trustworthy, scale comes from adding publishers and federation—not from giving search results more authority.