AIDH: An Agent-Independent Distributed Harness for Persistent Agents

Agents usually live and die with one process on one machine. AIDH keeps no instance state between requests, so a persistent agent costs in proportion to its work rather than its existence, recovers from failed machines without losing requests and runs each execution inside its own access closure.

Abstract

Agents built on large language models are deployed as one process on one machine: the agent’s definition, accumulated context and working files live in that process, and the agent exists only while it does. A bound agent therefore costs in proportion to its existence rather than its work: n agents over a horizon h occupy Θ(nh) machine memory–time and Θ(n) machines whatever their load, and a full machine resolves an overload by killing agents. We present AIDH, an AgentIndependent Distributed Harness whose runtime keeps no instance state between requests. Because the model call is stateless, an agent’s complete state when a request completes is its definition plus its context sequence, so the commit boundary of a request is a free recovery point. AIDH serves each persistent agent through an API as an endpoint with long-term storage and no process, keeps its state as records in storage it owns, answers requests that need no tools without a sandbox and dispatches the others to nodes under capability, resource and authorization constraints and the toolkits they are predicted to use, and runs every execution in its own sandbox with a one-time credential scoped to its access closure. Decoupled execution occupies Θ(W + |Q| o) for total work W , |Q| requests and per-execution overhead o, keeps node memory bounded for any number of agents, and preserves recovery equivalence, termination and isolation. On one 16 GiB machine, a process-bound deployment of the same core holds 73 MiB per idle agent and completes none of its requests at 256 agents, while decoupled execution completes every request within bounded memory at a start-up price of 3.06 s and 3.41 core-seconds per execution under gVisor; a pre-started core cuts the latency to 0.30 s. For 100 agents under an office-hours load the bound deployment costs 3.1× as much, and an idle agent 1.15 USD a month as a process against 6 millionths of a dollar as records. On four machines, a failed machine stops its bound agents for 119.4 s, while AIDH loses no request. On the same machines, classifying each request on arrival (99.3% correctly, in 583 ms) and answering those that need no tools directly ran 32% fewer executions in sandboxes than prediction from the agents’ records alone, cut the p95 latency of such requests in a burst from 58.5 s to 15.5 s and raised the share of tool requests that found their toolkits running from 39.0% to 66.9%, while reserving sandboxes for predicted toolsets added nothing. With the real model, AIDH matches single-instance agent systems in pass rate, answers requests that need no tools as correctly as the agent core and 2.2× faster, runs 1,138 instances on one host, continues instances across devices with byte-identical context, and keeps executions inside their access closures.

This is a working paper. The full text is in preparation for publication; the abstract above reports its current results.

@techreport{guo2026aidh,
  title       = {AIDH: An Agent-Independent Distributed Harness for Persistent Agents},
  author      = {Guo, Yanming},
  institution = {AIDC Research},
  year        = {2026},
  type        = {Working paper},
  url         = {https://www.ai-dc.ai/research/aidh-distributed-harness}
}