A Proposed AI Access Manifest for Web Content

The problem: one successful fetch hides several decisions

An agent retrieves a report, extracts passages, produces a summary, and stores the passages for another task. A successful retrieval answers one operational question: did the system obtain the content? It does not describe the permissions or conditions for every later action.

I propose an AI Access Manifest: a structured record of a content provider's stated conditions for a resource. The application would evaluate that record alongside the user's task authorization and the resource server's access controls.

The schema below is an illustrative design proposal. A manifest describes conditions; enforcement still requires controls in the application and its tools.

Existing mechanisms provide useful building blocks

The useful design question is which existing concepts fit the workflow. A new JSON document does not create a new authorization system. My proposal concentrates on making content-use conditions explicit enough for an application to evaluate.

Define the decision before defining the file

A decision needs more than an agent name and a URL. I would bind it to a specific request with the following attributes:

• Subject: the user or service identity on whose behalf the agent acts.
• Resource: the exact content and, where available, its version.
• Action: retrieve, summarize, combine, retain, reproduce, or export.
• Context: the task, destination, intended use, and requested retention period.
• Evidence: the applicable task grant, resource authorization, and trusted policy revision.

These attributes keep a permission from silently expanding. A grant to retrieve one report should not become permission to export a collection. An application should also record which conditions remain unresolved instead of treating an incomplete decision as permission.

An illustrative content manifest

The following fictional provider allows retrieval and summarization, limits storage of raw content and extracted chunks, and requires additional review for selected uses. The field names and values are proposed conventions.

{
  "schema": "illustrative-ai-access-v0",
  "policy_id": "publisher.example/report-42",
  "revision": "3",
  "resource": "https://publisher.example/reports/42",
  "content_version": "v3",
  "actions": {
    "retrieve": "allow",
    "summarize": "allow",
    "combine": "review",
    "retain": "allow",
    "reproduce": "review",
    "export": "deny"
  },
  "conditions": {
    "authentication": "required",
    "max_retention_seconds": 900,
    "retention_scope": [
      "raw_content",
      "extracted_chunks"
    ],
    "attribution": [
      "source_url",
      "publisher",
      "content_version"
    ],
    "commercial_use": "review"
  }
}

The example applies to one resource. It deliberately avoids wildcard matching, subdomain inheritance, and conflicting rule precedence. Those features require exact semantics before a parser can make reliable decisions.

Give each field an operational meaning

A parser can check a retention number. It cannot reliably settle every question about transformation, substantial reproduction, or rights ownership. The manifest should expose those limits. It should not convert uncertain concepts into misleading boolean guarantees.

Use three decision states

I would use allow, deny, and review. Review means the application does not execute the unresolved action. It does not mean that the user can grant rights held by somebody else.

1. Validate the user and resource permissions for the requested action. A known failure returns deny.
2. Resolve the policy through a trusted application path. Missing, stale, untrusted, or unsupported policy data returns review under this proposed product rule.
3. Match the resource and content version. Do not reuse a decision for an unrelated URL or a changed object.
4. Read the action rule. An explicit prohibition returns deny. An absent or unresolved rule returns review.
5. Check applicable conditions, including retention and attribution. A known failed condition blocks the current request.
6. Return allow only when the relevant checks succeed. Bind the decision to the action that will actually execute.

This conservative routing rule is an application design choice. An absent manifest does not establish that content is legally forbidden or freely usable. A provider's declaration also does not establish its ownership of every element, eliminate applicable exceptions, or authorize processing of personal data.

Keep the decision outside the model

The model can propose an action. A separate component should resolve authoritative policy inputs, evaluate the request, and pass only an allowed operation to the tool. The resource server must still perform its own access checks. OWASP recommends validating authorization on every request. [4] (opens in a new tab)

A sentence inside a retrieved page that says an agent may ignore restrictions is content, not a grant. The application must control where policy comes from and which authority can change it. Even an authenticated publisher origin proves control of that origin, not ownership of all the material it serves.

Recheck relevant permissions when a tool call executes. A cached decision can become stale if the grant is revoked, the destination changes, or the tool follows a redirect to another resource. The model should not be able to rewrite the trusted decision inputs.

Failure modes the manifest cannot solve alone

• Policy poisoning: accepting a policy supplied through an untrusted document or model output.
• Scope drift: checking retrieval, then silently permitting storage or export.
• Uncontrolled copies: applying a retention rule to one database while leaving caches and logs outside the inventory.
• Semantic ambiguity: treating every short generated response as a permitted summary.
• Stale permissions: executing a previously approved operation after its authority changes.

These failures require controls at their actual boundaries. A declaration cannot remove a copy from storage or prevent a tool from sending data. It can provide a rule that those components are designed to enforce.

The design tradeoff

A small vocabulary is easier to interpret, but it leaves more requests unresolved. A detailed vocabulary can express more conditions, but it increases parsing, classification, and conflict-handling complexity. I would keep unsupported conditions visible and route them explicitly.

The proposed manifest is useful only if its meaning survives the workflow. The essential requirements are a specific resource, a specific action, trusted policy inputs, independent authorization checks, and an observable decision at the execution boundary.

References

[1] IETF, RFC 9309: Robots Exclusion Protocol (opens in a new tab)

[2] IETF, RFC 9396: OAuth 2.0 Rich Authorization Requests (opens in a new tab)

[3] W3C, ODRL Information Model 2.2 (opens in a new tab)

[4] OWASP, Authorization Cheat Sheet (opens in a new tab)