I have built more than 20 specialized agentic workflows. That work changed how I think about web access. An agent can retrieve useful evidence and produce a well-structured answer while leaving an important question unresolved: what is it permitted to do with the material it retrieved?
At Yuba, this question emerged while building a Problem Validation agent. The workflow helped assess a founder's problem statement using external evidence. I evaluated hallucinations, evidence quality, and output structure. Those checks addressed whether the report was useful and supported. They did not establish the permission boundaries for every source and operation.
The workflow that exposed the question
The Problem Validation workflow began with the founder's problem context and moved through several stages:
- Clarify and expand the problem context into questions that external evidence can address.
- Search the web for relevant sources.
- Extract evidence and rank its relevance to the problem.
- Check consistency across sources and identify gaps.
- Produce a structured report that connects findings to the original problem.
Each stage changes the material's role. A retrieved page becomes an evidence fragment. Several fragments become a comparison. The comparison becomes a report someone may retain or share. Evaluating only the final report leaves the intermediate operations outside the assessment.
During this work, I observed retrieved content that a human-facing interface limited from view, while the underlying content remained technically available. That observation made the distinction concrete. The interface communicated one experience of access. The retrieval path exposed another.
Why delivery and display can disagree
A web application can make separate decisions about delivery and display. If a response already contains text, a client-side rule may only change what the page renders. A retrieval tool may process a representation that does not apply that rendering rule. This is one possible mechanism for this class of mismatch. It is not a reconstruction of a particular website.
A diagnosis should compare the response, session identity, resource version, and retrieval path before assigning a cause. For protected content, OWASP recommends authorization checks at a trusted boundary and on every request. Interface visibility alone cannot establish that those checks occurred. [2] (opens in a new tab)
What the observation establishes
This account has a narrow evidentiary boundary. It describes a mismatch between interface visibility and technical retrieval. It does not establish that a server-side security control was defeated. It also provides no basis for a legal conclusion about a particular source. This account does not quantify how often this mismatch occurs across websites.
| Question | What the experience supports | What remains unresolved |
|---|---|---|
| Could the workflow obtain content? | Underlying content was technically retrievable. | Whether every subsequent operation was permitted. |
| Did the human interface expose the same material? | The interface limited what a person could see. | Whether that limitation represented authentication, product design, or another condition. |
| Did quality evaluation answer the permission question? | It assessed hallucinations, evidence, and report structure. | A separate account of authority, use, and retention. |
The engineering lesson is about the information available to the workflow. A successful retrieval returns content. It may leave the applicable use conditions unknown. The workflow needs a way to represent that uncertainty instead of treating the successful response as a complete authorization decision.
Separate the layers of access
I now separate four questions when reasoning about an agent's access to a source. First, can the tool obtain bytes? Second, which identity and credentials does the request use? Third, what operation does the task require? Fourth, what evidence supports permission for that operation? These questions can produce different answers.
A user may authorize an agent to research a topic. That instruction describes the user's task. It does not, by itself, specify the conditions attached to every external source. Likewise, permission to read a resource may leave storage, redistribution, or use in another context unresolved. A policy model should represent those operations separately.
Robots rules illustrate the need for precise terminology. RFC 9309 defines the Robots Exclusion Protocol and explicitly says its rules are not a form of access authorization. A robots.txt rule therefore cannot serve as the workflow's complete permission model. It answers a narrower protocol question. [1] (opens in a new tab)
Locate the unanswered decisions
The observation raises diagnostic questions at each point where the workflow changes its use of content. The following records would help establish which decisions are supported and which remain unresolved.
| Workflow boundary | Decision to represent | Useful record |
|---|---|---|
| Before retrieval | Does the request use the appropriate identity and access path? | Source identifier, request identity, and available access conditions. |
| Before transformation | Does the intended operation fit the available permission evidence? | Requested operation and applicable conditions. |
| Before retention | Can this material be stored, and for how long? | Storage purpose, expiry, and source relationship. |
| Before export | Does the destination and output fit the permitted scope? | Destination, retained provenance, and unresolved restrictions. |
An answer-quality score cannot fill this record. A report may be accurate while the authority for storing or exporting one of its sources remains unclear. That missing evidence is a separate issue from the model's confidence in the answer.
An illustrative permission mismatch
Consider a hypothetical source that explicitly permits temporary retrieval and summarization but prohibits persistent storage of the full text. A research task could fit the first two conditions and still conflict with the third if the workflow saves every retrieved page by default. An accurate summary would not reveal that storage conflict.
In this example, the required handling follows from the stated conditions. The system would need to distinguish the temporary processing input from any retained artifact. It would also need to determine what the conditions say about retaining the summary. Calling an output a summary does not answer that question automatically.
Handle incomplete permission evidence explicitly
A declaration can be missing, outdated, ambiguous, or issued by someone without authority over all the material. A machine-readable rule reduces parsing work. It cannot establish the truth of every assertion it contains. The design must record where a rule came from and which resource and operation it covers.
Unknown permission also needs an explicit state. Depending on the task and existing authority, a workflow could use an authorized alternative, narrow the requested operation, or stop the affected step for review. The essential requirement is to preserve the distinction between permission supported by evidence and an assumption made because retrieval succeeded.
My experience at Yuba established a useful design problem: report quality and access permission require separate reasoning. I can evaluate whether a report follows its evidence while also asking what authorized the operations that produced it. A dependable agent needs both accounts, and it needs to preserve uncertainty when either account is incomplete.
References
[1] RFC 9309: Robots Exclusion Protocol (opens in a new tab)