Idle Cost Optimization for Self-Hosted Agent Fleets
Snapshotting idle agents to disk swaps compute bills for storage.

Persistent AI agent infrastructure differs from serverless functions because an agent must keep its execution state between requests, even when its compute stops. For self-hosted fleets, the architectural answer is a snapshot-and-restore scheduler that preserves idle agents on disk. Idle cost optimization therefore depends on separating persistence from continuous compute.
Why idle agents create a structurally different cost problem than traditional cloud workloads
Agent fleets consume compute in short bursts, while traditional infrastructure bills continuously for provisioned capacity. Between those bursts, agents wait for users, tool responses, or upstream processes. Token optimization and model routing can cut LLM costs, but an agent VM still incurs an infrastructure charge as it waits for its next interaction.
A web service can often change how many instances it runs as request volume changes, while a persistent agent keeps its session state even when no request is running. Persistence becomes the reason to keep the VM running. In a per-user deployment, provisioned capacity grows as more users join, even though active concurrency stays low. Most of the fleet may be idle without any individual VM being oversized for the work it eventually performs.
Why serverless functions and standard cloud remedies do not solve persistent agent idle cost
The decisive comparison between persistent AI agent infrastructure and serverless functions is whether execution state can be discarded between requests. Stateless execution works when the next invocation can start without the previous process, or when the required state lives elsewhere. Persistent agents break that assumption. An agent in the middle of a session may have files, credentials, browser state, and in-memory variables that it must keep through its next idle interval. Terminating its VM without preserving that state destroys the continuity the product requires.
Autoscaling alone therefore leaves an architectural gap. It can terminate idle instances and create replacements, but replacement is not the same as resumption. Right-sizing addresses a different variable by reducing the cost of each instance without changing how long it sits unused. Spot or preemptible instances cost less but can be interrupted, which can force an agent without state recovery to start its work again.
Warm pools address startup latency. They require a standing supply sized for expected concurrency, so underestimating demand drains the pool while overestimating it leaves paid capacity idle. The same compromise appears in a different place. These remedies remain useful where their assumptions fit, but none alone preserves an agent’s execution state while removing its idle compute allocation. That is the requirement the underlying architecture must satisfy. Maritime, an infrastructure provider of agent computers on Firecracker micro-VMs, gives each agent a permanent VM and snapshots it to disk when idle, which is one route for teams that outgrow serverless functions.
How snapshot-restore converts idle compute cost into storage cost

Snapshot-restore preserves execution state without keeping the agent resident in memory. The pattern starts by booting a microVM fully, saving its state to disk, and restoring that snapshot when execution is needed again. Think of a process paged to disk. Between sessions, the suspended agent consumes snapshot storage.
The snapshot preserves the guest kernel, open file descriptors, in-memory variables, credentials, and browser sessions. On restoration, the agent resumes from its saved execution state. The unit of persistence is the execution environment itself. As a result, the scheduler can reclaim compute without making the agent abandon its session.
A snapshot at rest costs a small fraction of the hourly cost of a running VM, so long stretches of idle time become cheap time in storage. Shared base snapshots offer another efficiency gain. With copy-on-write memory sharing, restored agents can share read-only pages instead of each immediately occupying a full independent copy of the same memory. Resident memory can therefore grow sublinearly with agent count, although active agents still need resources for their work.
What Firecracker provides that makes snapshot-restore practical at agent fleet scale
Firecracker supplies a microVM boundary suited to persistent, per-tenant execution. Each microVM has its own hardware-virtualized guest kernel, while a standard container shares the host kernel. The boundary is structural rather than a policy convention. Its one-VMM-per-microVM design maps directly to a fleet in which each agent has its own VM and kernel, separating tenant execution at the virtualization layer.
Firecracker boots in roughly 125ms with minimal per-VM memory overhead, and snapshot restoration is faster still. A restore involves process startup, memory mapping, the restoration of CPU and device state, and vsock reconnection. Adding up the time for each component puts the total around 28ms. That makes resumption plausible within an interactive latency budget rather than an operation reserved for infrequent recovery.
Memory behavior also fits the workload’s variability. Memory hotplugging lets a running guest add the memory its task needs without downtime, so every session can run without holding its peak allocation the whole time. A warm pool can still hedge latency for selected workloads. But fast restoration means fewer fully running environments need to stay ready just to avoid initialization.
AWS AgentCore V2 reached general availability on September 18, 2026, using the same prepared-snapshot principle at managed-service scale. Rather than pulling and initializing a fresh image on every create, V2 restores a prepared snapshot. This supplies evidence of the pattern’s production maturity and architectural validation for self-hosted teams.
How a snapshot-restore scheduler fits into a self-hosted fleet's architecture
The scheduler turns the VM capability into a fleet lifecycle. It detects when an agent has passed an idle threshold with no incoming requests and no active tool calls, triggers a snapshot, and then suspends or deletes the running VM. A later request reverses that path. The scheduler locates the saved artifact, restores the agent, and routes the request only after restoration. To do this, it must track each agent’s identity and where that agent’s snapshot is stored.
Networking sits on the wake path as well. A small set of pre-built network slots, with vNIC configurations and IP assignments, can let restored VMs join the network without waiting for those slots to be allocated. These slots avoid keeping entire VMs warm. Snapshot artifacts need their own lifecycle, since their count grows as agents accumulate. Retention and eviction policies can compress long-idle snapshots, move them to cheaper storage, or prune them when their agents are decommissioned.
The principal scheduling choice becomes the idle timeout. The snapshot mechanism can work without the standing inventory of running VMs and continuous refill loop that a warm pool needs, though hosts still need enough capacity for active work. The timeout can vary by workload or tenant. A narrow exception remains for workloads that constantly create brand-new template identities without prepared snapshots, where a small standing pool can provide more benefit than restoration.
Implementation — the infrastructure requirements and sequencing for a self-hosted fleet
Firecracker requires hardware virtualization through KVM on Linux. Host selection is therefore the first dependency. Shared-tenant cloud VMs typically do not expose KVM, so the host must be bare metal, a suitable dedicated host, or an environment that supports the required nested virtualization.
With the host available, the next task is baking the base image. The template snapshot should capture a clean boot with the agent runtime, dependencies, and tenant-agnostic configuration already installed, avoiding package installation during restoration. Tenant state belongs outside that shared template. Tenant-specific credentials and configuration should enter at first boot or first restore through vsock or a metadata service. This separation keeps the base reusable and supports copy-on-write sharing across agents.
The scheduler then needs the identity mapping, idle detection, snapshot creation, and request-triggered restore path described above. Its storage layer must be durable and fast enough for the product’s wake budget. Storage is a real fleet expense. Local NVMe is preferred over network block storage for latency-sensitive wakes, while the overall storage design must also account for durability and cost at rest. Thousands of suspended agents can make snapshot storage a substantial line item even when it is far cheaper than idle compute.
Once that path works, timeout tuning should reflect the product’s tolerance for resumption delay. A product with a sub-200ms restore tolerance can snapshot agents aggressively, while one that needs an immediate first token may need a longer grace period or a small warm pool for its hottest agents. Measurement should guide that choice. Instrumenting restore latency, snapshot size per agent, active versus suspended counts, and storage cost per agent exposes both the user-facing delay and the cost of preserving state. Those measurements distinguish a timeout problem from a storage or capacity problem without treating all slow wakes as the same failure.
The emerging option — speculative prewarming as a latency hedge on top of snapshot-restore
Speculative prewarming extends snapshot-restore for workloads whose latency budget cannot tolerate even a short on-demand wake. Instead of retaining every possible environment, it predicts which sandbox will be needed next and warms it before the request arrives. Prediction replaces broad reservation with selective preparation. This remains a research result moving toward production use.
The SpecBox system from Beihang University, University of Leeds, and University of Sydney, presented in August 2026, uses context-aware stochastic prefetching over a sandbox dependency graph. It mines historical agent execution traces to predict the next sandbox before the model finishes its tool call. The SpecBox paper reports a high prewarming hit rate and P99 end-to-end latency cut by up to 2.9× relative to an on-demand baseline. It also reports substantially lower peak memory consumption than permanently reserved sandbox deployments.
For most early-scale fleets, snapshot-restore alone is sufficient. Speculative prewarming suits workflows that repeat at high volume, making tool-call patterns easy to predict, and have an especially tight latency budget. Its role is a latency hedge on top of preserved state. It does not replace the scheduler’s responsibility to snapshot idle agents or manage their artifacts.
What the fleet economics look like once snapshot-restore is in place
After the transition, compute demand follows active concurrency. A bare-metal host previously limited to a smaller set of peak-sized, always-on sandboxes can support hundreds of agents when most are snapshotted and idle. Session density becomes a better measure of how the fleet runs. The host serves the active minority while the rest retain their identity and state on disk, although its running capacity still constrains simultaneous work.
Every suspended agent occupies storage, so artifact count and size continue to grow with the population of persistent agents. That residual cost is orders of magnitude cheaper than equivalent idle compute. That lower cost can make a dedicated per-user agent affordable without a VM running for each user around the clock.
For teams pricing that product, the lesson is to model unit economics around idle ratio, the fraction of time each agent spends suspended. Higher idle ratios leave more time in which storage charges can replace compute charges, while agents that stay busy all the time have less room for this saving. Active concurrency determines the compute requirement. Snapshot count and size set how much storage the fleet needs, and the restore latency distribution shows whether users get the experience the product needs to provide. Storage cost per agent per month can then feed directly into per-user pricing, with that cost separate from the expense of an always-on VM allocation.


