Verification, Infrastructure, and the AI-Native Organization
Over the past few years, generative AI has expanded from producing text, images, and code to carrying out tasks through tools and interaction with its environment. Improvements in reasoning and tool use have made it possible for agents to pursue a goal across multiple steps, observe what happens, and adjust their approach. As these systems take on more execution, their usefulness depends increasingly on the systems and organizations in which they work.
Software engineering makes this change particularly visible. Code generation is becoming one part of a longer process in which an agent investigates a problem, changes a system, and checks the result. Coding offers a strong starting point because compilers and tests provide feedback that machines can use. Delivering software requires extending that feedback to business requirements, dependencies, integration, and operation. Adding AI to individual stages leaves much of this work unresolved; the development process itself needs to support continuous action, verification, and correction.
That process needs infrastructure designed for agents. A Harness must preserve context and progress across long tasks. Reproducible environments must let agents run software and investigate failures. Identity and authorization must define where they can act, while execution records must make their behavior understandable and provide evidence for evaluation. These capabilities determine how much responsibility a team can safely delegate and how reliably an agent can complete it.
The organizational changes are just as consequential. As agents absorb more execution, teams need to reconsider ownership, coordination, and the role of engineering judgment. They also need ways to share tacit knowledge and develop new expertise as familiar training paths change. Building for agents therefore involves both the infrastructure that makes their work verifiable and the human organization that decides what work matters, judges its results, and remains accountable for them.
Coding offers abundant public source code, issues, and benchmarks. It also has a useful property for learning and automation: many actions produce feedback that a machine can check. A compiler reports whether a program builds. A test checks a defined behavior. In mathematics, a formal verifier can establish whether a proof follows the rules of a particular system.
AI advances quickly where feedback is available and verification can scale. Coding and mathematics benefit from this even when the problems themselves are difficult. The advantage comes from being able to distinguish a successful attempt from a failed one and use that result in the next attempt.
Enterprise work is often harder to check. Acceptance criteria are scattered across requirement systems, deployment conventions, undocumented APIs, and engineers’ experience. Obtaining feedback may require access to an internal service or a costly test environment. A patch can compile while violating a business rule. A test can pass against a substitute dependency and fail against the actual service.
Making this work accessible to agents means improving the conditions for verification. Requirements need observable acceptance criteria. Tools need interpretable results. Environments need to reproduce the conditions under which a change is expected to work. An agent’s useful autonomy depends on the feedback its environment can provide, including a clear account of what the available checks cannot establish.
Jack Reeves’s 1992 argument that code is design remains relevant here. Source code expresses decisions at a level of precision that design summaries inevitably omit. Producing that code may become cheaper, but someone still has to turn an ambiguous business request into a precise, executable behavior and establish that it meets the intended need.
Early factory electrification offers a useful analogy. Replacing a steam engine with a large electric motor could leave the transmission shafts, belts, and machine layout largely intact. Existing equipment still worked, conversion was expensive, and electrical infrastructure and standards were still developing. Changing the power source was a practical first step.
Individual motors later gave each machine more independence. Equipment could be arranged around the production process, changing material movement, assembly lines, and factory management. The productivity gains depended on reorganizing work around what the technology made possible.
Software teams face a similar choice. Adding AI to requirements, design, task decomposition, coding, and testing can improve individual stages while preserving all the old handoffs. Specification-driven development, Skills, and workflow packages can help organize that work. Their value depends on how they are used: an agent that must follow a fixed sequence and wait at every boundary still has little room to investigate unexpected results.
A failing test rarely tells us which stage to repeat. The cause might be a misunderstood requirement, a design error, incorrect data, a code defect, or a broken environment. Sending the task back one step can repeat work that was already correct and miss the actual cause.
An agent needs to obtain verification signals and use them to choose what to do next.
Figure 1. The workflow sets coordination and control boundaries. Within them, the agent uses execution results to decide whether to investigate, revise, verify, or request help.
The workflow still handles scheduling, permissions, persistent state, auditing, and high-risk checkpoints. The agent can investigate within those boundaries without having every reasoning step prescribed in advance. This division gives it room to complete the action–verification–correction loop while leaving control over consequential operations with the surrounding system.
An agent that can investigate, act, and verify gives a team more choice about how to delegate work. With local assistance, a developer directs individual steps. A cloud agent can carry a bounded assignment through to a reviewable result. A team of agents can divide a larger assignment among roles and coordinate their work.
| Collaboration model | Work delegated to agents | Human responsibility |
|---|---|---|
| Local assistance, such as OpenCode or Qoder | Suggest changes and perform developer-directed steps | Supply context, approve commands, and inspect results |
| Cloud autonomy, such as QoderWork | Fetch code, prepare an environment, implement a change, and run checks | Set the goal, review the change and its evidence, and make key approvals |
| Multi-agent team | Coordinate requirements, design, coding, testing, release, and archiving through lead and specialist roles | Design the collaboration, define role boundaries, resolve tradeoffs, and retain final accountability |
The coordination burden grows with the work being delegated. A local assistant can rely on a person to carry context between steps. A cloud agent needs durable task state and a way to return work for review. An agent team also needs explicit handoffs: what has been completed, what remains uncertain, and which agent owns the next action. Shared task state and cross-system tools become part of the collaboration.
Teams can use these models alongside one another, choosing the arrangement that fits the task. A practical rollout begins with visible ownership and bounded assignments. Connect the capabilities, handoffs, reviews, exception handling, and outcome evaluation needed for a small task, then preserve its failure lessons, business rules, and missing context for later work. Longer assignments become feasible as this support improves.
Human attention shifts toward goals, consequential decisions, and judging results. The platform must carry more of the context, coordination, and control that a developer previously supplied between individual actions.
The platform has to support this delegation across the delivery process. If writing code takes 20–30% of a team’s development effort, even a large improvement there leaves most of the work unchanged. Integration, testing, release, and operation still determine when a change reaches users. If those stages cannot absorb the additional output, generated code simply waits further down the pipeline.
Supporting an agent through delivery requires five capabilities:
Figure 2. Knowledge, code, and Skills feed a Harness whose internal capabilities share task state. The environment platform supplies isolated execution and scoped credentials. MCP and CLI access follow production guardrails or task-specific non-production policy; labels distinguish context, tool calls, provisioning, and authorization.
The knowledge base provides documents, rules, and experience through indexes, vectors, or graphs. The repository provides versioned source, commits, pull requests, and diffs. A Skill Hub packages methods as instructions, templates, workflows, plugins, and examples. Model services supply inference and may also provide embeddings, fine-tuning, and evaluation.
Knowledge, Skills, and tools have different responsibilities. Knowledge supplies facts that need to be current, trustworthy, and traceable. A Skill describes how to perform a task. Model Context Protocol (MCP) and command-line interfaces (CLIs) expose operations. A large tool catalog helps only if the agent can find the right capability, understand its effects, and use it at the right point in the task.
The Harness brings those resources into the execution loop. The environment platform manages dependencies, versions, builds, caches, images, and isolation. Identity and credential services establish who is acting and how access is granted. Guardrails apply policy, limits, checks, redaction, and audit requirements to production operations. Non-production work also remains subject to the task’s access policy and Sandbox controls.
A failed build is a useful example of how these parts fit together. It starts as a concrete engineering problem, but completing it can require all five capabilities.
An agent asked to repair a build needs to read the error, locate relevant modules, inspect source and project rules, make a change, and run verification. If a new error appears, it must decide what that result means. A dependency upgrade or public interface change may require a person’s judgment before the task can continue.
The model interprets information and proposes actions. The Agent Harness manages the continuing task: loading context, invoking tools, controlling execution, saving state, maintaining memory, recovering from interruptions, and assembling the result. Conversation history is useful, but the Harness also needs execution results and a record of what remains unresolved.
Figure 3. A build-repair task advances through investigation, changes, and checks. Checkpoints preserve progress; recovery rechecks any assumptions that may have changed during the interruption.
A build failure rarely calls for reading the entire repository at once. The agent can start with the error log and build configuration, then follow relevant symbols, dependencies, and module boundaries. Stable project rules can remain available while implementation files are loaded as needed.
More context does not necessarily improve a decision. Irrelevant material competes for attention, and old instructions can contradict the current code. The Harness needs ways to retrieve, trim, and summarize information as the investigation develops. Large tool outputs can be stored as files or artifacts, with a reference and a short summary kept in the active context.
Reusable memory also needs to be distinguished from current task state. A verified project convention may help future tasks. A hypothesis about this build is still a hypothesis. Saving both as permanent instructions can turn a temporary guess into a rule that misleads later work.
A plan should change as evidence arrives. The initial approach might be to locate the failing module, make a small correction, and rerun the build. If the result points to a dependency mismatch, the next step changes while the findings from the earlier investigation remain useful.
Save the goal, completed work, unresolved assumptions, and next actions. For a build repair, that includes modules already ruled out, the current hypothesis, modified files, and checks still to run. A conversation interruption, model change, or remote environment restart should leave enough state to resume without repeating the investigation.
A checkpoint also lets a reviewer understand where the task stands. Recovery begins from the last trustworthy checkpoint, then checks any conditions that could have changed: the branch, an external service, an environment, or an authorization. Saved task state records what was known at a particular time; it cannot guarantee that the world has stayed the same.
Source search, file operations, terminals, build systems, test platforms, and development services give the agent ways to act and observe results. Their interfaces should make inputs, outputs, and side effects clear. Reading a file, editing a local workspace, and writing to an external system have different consequences and need different controls.
Structured results make the next decision easier. An exit code, execution state, and reference to a log give the agent more to work with than an ambiguous message that something went wrong. They also make failures easier for a person to reproduce.
Completion needs evidence from checks the environment can perform. Compilation, tests, static analysis, and runtime observations establish specific facts about a change. The model’s confidence does not replace them. When a check fails, preserve the conditions, invocation parameters, and results so the next attempt can use them.
The final handoff should include the code changes, commands run, verification results, and remaining risks. A reviewer needs that evidence to assess the change; “fixed” on its own says very little.
Human intervention is useful when a requirement is ambiguous, alternatives involve a business tradeoff, an operation has consequences outside the task’s authority, or repeated attempts fail to establish correctness. An ordinary code correction may proceed through automated checks. Upgrading a foundational dependency, changing a public interface, or bypassing a quality gate can require explicit review.
An analysis of approximately 400,000 Claude Code conversations offers one example of this division: users made about 70% of planning decisions, while Claude made about 80% of execution decisions. Those proportions describe a particular product sample. The useful distinction is the responsibility behind them: people set goals, boundaries, and consequential tradeoffs; agents carry out work within those constraints. A Harness should bring people in at those decision points and preserve enough context for them to help.
Repairing a build requires a place to run it. The agent may need to fetch code, install dependencies, start services, run tests, and access a browser or an external API. These actions affect files, processes, and networks. A Sandbox gives the task a disposable workspace with explicit limits.
A container supplies part of that workspace. A usable Sandbox also needs to define the image, CPU and memory limits, lifetime, permitted commands and files, output streaming, network destinations, credential handling, recovery, and evidence retention.
The execution boundary covers the task’s resources, files, processes, and network access. These details affect both correctness and isolation. Tasks can contaminate one another’s dependencies. Runaway processes can hold resources indefinitely. Test scripts can read host files, and malicious packages can search for credentials. Two agents modifying the same workspace can produce results that neither can reproduce independently.
For the build-repair task, execution should start from a specified source revision and image. The agent uses its own branch and workspace, installs the required dependencies, and iterates through tests, static checks, and builds. At completion, the platform extracts the patch, report, and execution records before releasing the environment. Inputs, access, outputs, and lifetime can then be declared and recorded for each task.
Figure 4. OpenSandbox separates clients and public contracts, lifecycle control, alternative runtime backends, the Sandbox data plane, and network and security controls. The arrows show representative API, provisioning, execution, and enforcement relationships. Docker and Kubernetes are alternatives; optional secure runtimes strengthen isolation.
The service that creates and schedules an instance does not need to carry all of its command and file traffic. Separating those responsibilities makes four boundaries easier to define and enforce.
Lifecycle control. The control plane validates the image, entry command, resource limits, network policy, and timeout before asking Docker or Kubernetes to create an instance. It needs meaningful states such as Running, Pausing, Paused, Resuming, Terminated, and Failed. A binary “container exists” status cannot explain a failed transition or distinguish a paused task from a failed one.
Figure 5. Sandbox lifecycle states distinguish active execution, pause and recovery, failure, and termination. The diagram illustrates those relationships rather than specifying every backend transition.
A server-enforced time to live reclaims abandoned instances when a model call times out, a client disconnects, or a workflow fails. Long tasks can renew their lease explicitly. Tasks that need to preserve their execution state can use pause or snapshot support according to the backend’s guarantees.
In-instance execution. Long-running commands need continuous standard output and error streams. Background processes need status queries and incremental logs. Debugging may require an interactive terminal. A lightweight execution service can provide commands, file operations, persistent sessions, pseudo-terminals (PTYs), and resource metrics. Returning structured exit codes, logs, and execution states lets the agent decide what to do next without guessing from mixed text output.
Network and access. Unrestricted outbound access amplifies the consequences of prompt injection and compromised dependencies. Deny egress by default, then allow the repositories, dependency sources, and APIs required by the task. Each Sandbox needs its own policy; a broad cluster rule cannot express the access intended for every task.
Inbound access needs authentication too. A temporary web service or browser endpoint should be exposed through a controlled gateway. Static access tokens can support service calls, while short-lived signed URLs can support temporary sharing. In both cases, the gateway validates and strips the access credential before forwarding the request to the user process.
Credentials. Putting real tokens in environment variables exposes them to the workload. An agent can inspect its environment, read process information, or print values in a debug log. Dependencies running in the same environment can do so as well. Keeping the secret in a trusted sidecar or proxy reduces that exposure. The Sandbox sees a placeholder, and the proxy adds authentication only to requests matching an authorized domain, path, and method.
Credential state should generally stay out of snapshots. After restoration, the control plane checks current authorization and supplies fresh access. Restoring a filesystem should not silently restore a permission that has expired or is no longer appropriate.
Isolation alone cannot make a project runnable. A build may depend on a specific language runtime, database version, data baseline, initialization procedure, service configuration, and network setup. If an engineer has to reconstruct these conditions for every task, much of the work still sits outside the agent’s reach.
Figure 6a. The Harness requests an environment from the orchestrator, which uses reusable project assets to prepare a workspace and local services within one isolated task instance. The workspace uses those services; this connection does not make them consecutive execution stages. Outlined frames show shared scope.
The project runtime context layer gives those conditions an explicit definition. Project environment definitions cover the runtime, repository, initialization, and networking. Dependency adapters handle installation, health checks, queries, and service access. Data baselines describe configuration, data, and service rules.
The Harness requests a task environment from the orchestrator, which resolves versions, assembles dependencies, and creates, restores, or reclaims task instances. Definitions and baseline data can be reused while each task keeps an isolated instance. The workspace connects to its provisioned local services, giving the Harness an environment in which to modify, build, start, and test the project.
Figure 6b. Execution evidence feeds the Harness’s next decision. A snapshot that passes acceptance checks follows a separate review path before it can become a shared baseline. The red dashed arrow marks approved baseline promotion.
The runtime environment is part of the context an agent needs to execute and verify its work. Exit codes, logs, service state, database results, and network records must return to the task. A validated candidate snapshot can be promoted into a later data baseline through a controlled process. An arbitrary task’s changes should not become the next task’s shared starting point merely because they were saved.
The runtime defines where code executes and how its effects are contained. Access to a repository, service, or production system also requires a decision about the actor and the operation. For a build repair, reading source code and using test services may be appropriate, while a production release requires its own authorization. Delegating the repair does not grant authority over every system the agent can reach.
An authorization decision needs to identify the running workload, the person or team on whose behalf it acts, the requested operation, and the permitted scope and lifetime. The resource or tool enforces that decision, with operation logs and confirmation when a sensitive action requires it. When agents hand work to one another, delegated authority must remain within those limits and traceable across the handoff.
Organizational ownership and technical identity support different parts of this control. A digital employee record can give an agent a name, employee identifier, team membership, project spaces, an owner or mentor, defined duties, and review obligations. That makes its place in the organization visible. A workload identity identifies the particular running agent. Delegation records whom it represents and what authority it has been granted. Resource services evaluate that identity and authorization when deciding whether to accept a request.
Treating an agent like an intern can help clarify supervision and responsibility. It cannot supply the enforcement mechanism: an agent’s behavior comes from its model, context, tools, instructions, and runtime state. Professional ethics, organizational loyalty, and implicit common sense cannot be assumed. Governance needs machine-enforceable boundaries alongside human management. A digital employee account must not become a way to bypass permissions or obscure accountability.
Figure 7. An agent’s workload identity and delegated authority let a resource service evaluate a request. Human approval and target-specific credentials are added when required by policy or by the target system.
In this authorization flow, a request can carry a composite identity token and request proof. The resource service evaluates them against its policy at runtime. If the operation requires human approval, execution waits for that decision. If the target needs a credential in its own format, a Credential Broker exchanges the authorized request for short-lived, scoped access. The target operation proceeds only after the relevant checks allow it.
Permissions should follow the needs of the task and expire with the grant. Reviewed work can inform later delegation, but a broader assignment still needs an explicit scope, task record, audit trail, and approval requirements. Longer execution must preserve opportunities for human intervention. Organizational membership explains who is accountable; authorization determines what the agent can actually do.
Once an agent has permission to act, the team still needs to understand what happened during the task. A build may fail after several edits, a tool may return an unexpected result, or execution may stop halfway through. Service health, latency, and error rates reveal only part of that story.
Agent observability needs a trajectory: the connected record of model calls, tool calls, state changes, and outcomes across a task. For the build repair, that record should show which failure the agent investigated, what it changed, which checks it ran, and the recorded basis for stopping or continuing. Its first purpose is to make the execution reconstructable.
Figure 8a. Execution records are collected, associated with the same task, and assembled into a trajectory. The source cards identify parallel record types; the middle group shows collection capabilities. The right-hand sequence illustrates the evidence retained for a build repair. A real trajectory can also contain retries and branches.
Records from different components need a common context. A Sandbox command, a model request, and a Skill invocation should be traceable to the task they belong to. OpenTelemetry GenAI semantic conventions provide a basis for standardizing model and agent telemetry. A deployment needs to choose and version the conventions it supports so that records remain interpretable across components and over time.
Skill Runtime integration can collect much of this automatically. A shared execution wrapper or SDK emits telemetry and inherits upstream task context, linking each Skill invocation to the agent, task, and trajectory. Developers can add capabilities without rebuilding the same instrumentation each time. The resulting records support failure investigation, cost accounting, and security audits.
A complete execution record still does not establish that the task was completed correctly. Every request may have been authorized and returned successfully while the agent solved the wrong problem. That requires a separate assessment of the result against the user’s intent.
With a trajectory available, the team can examine a failure and test a proposed correction. These are two distinct activities: diagnosis explains what went wrong in an execution; evaluation checks whether a change improves behavior on defined cases.
Behavior analysis asks whether the agent achieved the user’s intent, whether its recorded decisions were supported by the evidence available at the time, whether a tool call was appropriate, and whether retrieved material was relevant. These questions require semantic analysis of observable behavior and results; they do not require access to a model’s private reasoning.
For the build repair, successful compilation establishes a useful fact. It does not show that the change preserved a business rule or that the agent investigated the failure it was asked to fix. Diagnosis connects the execution evidence to those task requirements.
A staged analysis process can keep the cost manageable. Deterministic rules screen all executions for known conditions. Model-assisted analysis helps examine ambiguous cases, with selected samples receiving deeper assessment of intent, recorded decision justifications, tool use, and retrieval relevance. Expensive analysis is concentrated on cases that simpler checks cannot resolve.
Observability and evaluation should use the same execution evidence. Link trajectories to user feedback, select useful samples, and annotate them with the expected result or behavior. The Agent Trajectory Interchange Format (ATIF) provides a common trajectory format for evaluation and training; the supported format needs to be versioned alongside the telemetry conventions.
An execution record becomes useful as an evaluation case when the team knows what it is checking. If a build repair missed a business rule, the case needs a way to distinguish a correction that respects that rule from one that merely makes the build pass. Selecting and annotating cases supplies that missing criterion.
Figure 8b. Deterministic screening identifies known conditions and routes ambiguous cases to semantic diagnosis. Selected, annotated cases provide shared inputs for current and proposed configurations. Comparing their outcomes informs an adoption decision; collecting records does not automatically change the agent.
A change to a Skill, tool, model, or Harness can now be tested against failures that have already occurred. Compare the current configuration with the proposed one on the same cases, then inspect the results and relevant tradeoffs. This gives the team a basis for deciding whether the change helps, rather than treating a plausible correction as an established improvement.
People still need to investigate failures, assess changes, and decide what to deploy. A trajectory may reveal that an undocumented business rule was missed, but making that rule usable depends on someone who understands it explaining it, checking its application, and keeping it current. The evaluation process therefore depends on the knowledge and judgment the organization continues to develop.
Much of this knowledge has been maintained through work that organizations rarely record. An engineer asks a colleague to clarify an incomplete requirement, recognizes an undocumented convention, works around an inconsistent API, or tries a change in staging because the written instructions are insufficient. These actions consume time and require judgment even when no one records them as separate tasks.
People are both a coordination bottleneck and the fallback that keeps incomplete systems working. Meetings and handoffs cost time, but the people involved also use inference, flexibility, and social knowledge to fill gaps. Removing a handoff leaves those gaps unresolved unless another mechanism supplies what is missing.
This gives platform work a cumulative effect. More agent execution exposes more failure signals. Turning those signals into explicit rules, usable context, better tools, and validated environments makes later tasks easier to complete. A team that keeps doing this can expand the scope of work its agents handle and reinvest the gains in further improvements.
Figure 9. Execution experience can build shared capability when teams preserve and use what they learn. Replacement pressure can produce the opposite effect by discouraging knowledge sharing and weakening the development of expertise.
As AI lowers the cost of working across product, design, frontend, backend, and language boundaries, teams can give individuals broader responsibility for a problem. An engineer who can learn an unfamiliar area and carry a change through to a result can avoid some of the delays created by narrowly divided roles. Important decisions still need relevant expertise and a clear owner.
A small team of roughly three to five people with one accountable lead is a useful structure for bounded work. Broader individual capability can reduce the need to divide execution among a larger group, along with the communication that division creates. Project-based teams can provide this focus without requiring repeated company reorganizations. Team size should follow the problem, its dependencies, and the risks involved.
Management work changes as well. Strategy, information gathering, resource coordination, decisions, and conflict resolution remain necessary, although systems can assist with some of them. Motivation, mentoring, hiring, departures, and culture carry responsibilities that people must continue to own. Managers also need to help engineers express intent clearly, develop a professional identity as their work changes, and address the loss of purpose that can accompany the automation of familiar tasks.
Recognition and incentives need to keep pace. Annual or semiannual reviews may respond too slowly to people taking on broader responsibilities during a rapid transition. More frequent recognition can reinforce useful changes, provided evaluation rewards the judgment, reuse, and reliable outcomes the team actually needs.
The pace of change should also reflect the business. A new initiative may prioritize speed and discovery. An established service may need to place more weight on stability and security. A department containing both needs room for different approaches.
Writing a standard operating procedure (SOP), creating a Skill, or documenting an implicit rule turns personal experience into an organizational asset. Employees may reasonably worry that explaining their work will make it easier to replace them. That concern has direct consequences for the infrastructure being built.
Someone who expects knowledge sharing to accelerate their replacement has a reason to withhold the context the Harness needs. Hidden conventions remain hidden. If people with more alternatives leave and those who stay become increasingly reluctant to take risks, the organization loses some of the initiative and judgment needed to improve its systems.
The training pipeline faces a related problem. Moving from simple code to complex implementation, system design, and strategic decisions has traditionally given newcomers repeated opportunities to develop judgment. If AI handles much of the introductory work, teams need another way for junior engineers to gain that experience. Reducing junior hiring can improve one company’s short-term costs while weakening the future supply of experienced engineers across the industry.
That risk can compound. If companies use AI primarily to shrink teams and competitors respond in kind, fewer people get the practice needed to become senior engineers and architects. Existing expertise is consumed faster than it is developed. A short-term productivity gain can leave the industry with less capacity to make the difficult decisions its automated systems still depend on.
A credible transition therefore needs concrete commitments. Explain how productivity gains will be shared and what new work they make possible. Offer real paths into architecture, cross-domain ownership, and leadership of new initiatives, with practical support for people changing roles. Be clear about which responsibilities are changing. If a company says judgment matters more than execution but still rewards output volume alone, employees will follow the metric.
People remain responsible for goals, value choices, risk decisions, and final accountability. Agents can take on frequent execution, context handling, sustained work, and reuse of established methods. Making that division useful requires a platform that produces verifiable results and a team that continues to develop the expertise needed to judge them.
Alibaba. AI Native Handbook.