Firecracker Micro-VM Setup for Open-Source Agent Workloads

Firecracker isolates agent workloads, but operators must manage the entire platform around it.

· 9 min read
Cover illustration for “Firecracker Micro-VM Setup for Open-Source Agent Workloads”
Self-Hosting OAF · October 26, 2026 · 9 min read · 1,955 words

Setting up Firecracker for open-source agent workloads requires a chain of decisions about isolation, kernels, storage, networking, and lifecycle management. For persistent AI agent infrastructure versus serverless functions, the decisive criteria are state across sessions, background execution, package installation, and safe execution of generated code. Firecracker provides the VM boundary, but the surrounding platform must provide persistence, policy, and fleet economics.

Why Firecracker is the right isolation primitive for agent workloads, and what it doesn't solve on its own

Running an agent calls for different assumptions about trust than running a deployed backend. Package installs, shell commands, and model-generated programs are untrusted input. A prompt-injected agent attempting credential exfiltration is therefore a concrete threat. In a shared container, a kernel exploit can reach the host because the container shares its kernel. Firecracker puts the workload behind hardware-level KVM isolation, with sub-second cold boot and about 5 MB of memory overhead, so a host can run thousands of microVMs.

Per-thread seccomp filters restrict the virtual machine monitor, while cgroups and namespaces provide resource and process isolation. The jailer creates a chroot filesystem jail, configures cgroups and namespaces, drops privileges, and then executes Firecracker unprivileged. These controls protect the host from the execution environment. They do not decide what an agent may access.

A guest with unrestricted access to the internet, internal metadata services, or private APIs can still misuse that access. Filesystem exposure, package access, credentials, and tool approvals require separate decisions, while guest traffic requires host-level egress filtering.

What Firecracker actually exposes to the operator and what it doesn't manage for you

Firecracker exposes a RESTful API for configuring and controlling each microVM. Before boot, it accepts vCPU count, memory size, virtio block devices, TAP-backed network interfaces, rate limiters, metadata configuration, and kernel boot parameters. The host must provide accessible Linux KVM, either on bare metal or through a hypervisor exposing nested virtualization. Firecracker runs in user space above that interface. The API's control is limited to the VM.

Some resource decisions are fixed at launch. A microVM supports up to 32 vCPUs, but its vCPU count cannot change while running, so additional CPU capacity requires a new VM. Memory can grow through virtio-mem within a hotpluggable pool declared before boot. The metadata service, MMDS, provides a host-to-guest channel for configuration and per-agent credentials without embedding them in the image. Device rate limiters let the operator control resources through the same API.

The operator owns the rest of the lifecycle. That includes TAP creation, routing, firewall rules, image distribution, snapshot storage, autoscaling, observability, and incident response. There is no Docker-like daemon managing networking on the operator’s behalf. Killing a Firecracker process terminates its VM immediately, but network namespaces, sockets, chroot resources, and cgroups require explicit reclamation. Process termination is not complete resource cleanup.

Kernel selection and configuration choices that determine what agents can actually do inside the VM

The guest kernel determines which syscalls, filesystems, and networking features agents can use. Firecracker requires Linux with KVM guest support and the virtio drivers its device model expects, rather than a kernel booted through an emulated BIOS or full PC. Firecracker keeps reference kernel configurations up to date, and these are a better place to start than stripping a distribution kernel. A general-purpose configuration is a good fit for agents that run compilers, install packages, or start background processes. Over-minimization breaks otherwise ordinary developer tooling.

Module support makes that trade-off concrete. A kernel that cannot load modules has a smaller attack surface, but required filesystems or kernel-level networking features must then be compiled in or added through a rebuild. Host-side seccomp filters calls made by the Firecracker process, leaving programs inside the VM subject to guest syscall policy. Restricting those programs requires guest-side seccomp, or an explicit acceptance of their access to the guest kernel.

CPU templates require an equally early decision. Firecracker supports masked CPUID/MSR descriptors to normalize the features presented to guests. A successful restore therefore gives no guarantee that the code will run. The CPU template must be chosen before the first snapshot is baked.

VM sizing decisions for agent workloads: vCPUs, memory, and what oversubscription actually buys

Agents spend much of their time idle, so sizing every VM for sustained peak demand undermines fleet density. For coding and data analysis agents, a good place to start is 2–4 vCPUs and enough memory for the language runtime and the data in use. Workload behavior determines the final allocation. Firecracker has near bare-metal CPU performance, while virtio block and network devices keep I/O overhead low compared with software-kernel approaches.

Host CPU controls separate placement from entitlement. Cpuset cgroups pin a microVM to physical cores, reducing cache disruption and NUMA effects from migration, while cpu cgroups cap its time quota on those cores. Core pinning is the recommended default for latency-sensitive tasks. It does not replace fair sharing. Time quotas still determine how much CPU an agent receives when neighbors are active.

Memory density depends on actual working sets. Freshly booted VMs use an anonymous mmap region sized at configuration time, while restored VMs page guest memory in lazily. An 8 GB VM using 2 GB occupies 2 GB of host RAM, but Firecracker does not natively manage memory oversubscription. A 256 GB host can therefore fit more VMs than configured allocations suggest, provided active working sets fit. Block and network rate limits, including steady-state limits and burst allowances, are also necessary in multi-tenant fleets to prevent heavy I/O from degrading neighbors.

Filesystem layout and guest image design for agents that need to install packages and persist state

The rootfs must support useful work without becoming an indiscriminate collection of tools. Firecracker attaches it through virtio-blk, and it must contain a bootable Linux filesystem with the modules, runtimes, and tooling the workload needs. A useful agent image has a package manager such as apt or apk, the language runtimes it needs such as Python or Node, a build toolchain such as gcc/make, and ca-certificates for HTTPS. Browser or automation dependencies depend on the workload. Missing essentials break tooling, while unnecessary breadth increases startup work and attack surface.

Runtime package installation needs somewhere writable to go. A read-only rootfs with an overlayfs writable layer keeps the base shareable while placing changes in the overlay. Filesystem policy must specify writable directories, read-only paths, and whether the writable layer is discarded, snapshotted, or exported at exit. Access to the host filesystem should be off by default. A separate persistent data volume provides storage that survives VM teardown without making the rootfs the durable record.

Persistence also requires separating files from secrets. Session state can be retained through a VM snapshot with corresponding disk state, or through artifact export before teardown. Cross-session memory belongs on a persistent secondary volume or in an explicitly used external store. Credentials belong in neither the base image nor a reusable snapshot. They should be injected per session through MMDS, because baked credentials become shared by every VM restored from that snapshot.

How snapshot-restore changes the economics and latency of agent fleet operation

Snapshot-restore changes idle-agent cost from running compute to storage. A fully initialized guest can be frozen and restored without repeating its boot sequence, shortening the user-facing startup path to a sub-second restore. Snapshot creation happens when the agent is idle. Restore latency is the operational priority. Once the running VM is retired, the idle session requires stored state.

That state has separate memory and disk components. Firecracker snapshots capture guest memory and virtio block-device state, but not the disk data itself, which remains in a separate rootfs file. To keep execution state, the matching disk state must be saved too. This distinction also governs branching, where multiple restores begin from the same initialized state to try different approaches.

Copy-on-write layering keeps the cost of those branches down at the block-device layer. Without it, naive disk capture scales with total disk size, increasing storage and snapshot latency for large workspaces. With layering set up for this, changed disk pages can be saved without copying the whole disk. When there are few writes, the disk delta is small no matter how large the disk is configured to be. This mechanism must be built explicitly, and the earlier CPU template choice remains a prerequisite for moving restored workloads safely across mixed hardware.

Network isolation decisions: TAP interfaces, egress controls, and the host-side work Firecracker leaves to you

Networking is where operator-owned plumbing has the sharpest security consequences. Firecracker exposes a virtio-net device backed by a host TAP interface, but the operator creates that interface before boot and removes it after exit. Addressing, routing, NAT, and iptables or nftables rules are host responsibilities. A per-microVM network namespace separates traffic visibility. It does not, by itself, prevent guests from reaching the same host network.

For agent workloads, egress policy must block cloud metadata endpoints, including 169.254.169.254 and equivalents, and private address ranges the workload has no reason to reach. External access should either use an allow-list of specific domains or pass through a proxy for logging and filtering. DNS needs its own policy. Arbitrary DNS queries can provide an exfiltration path through DNS tunneling even when direct TCP egress is blocked. Hardware isolation does not neutralize an authorized outbound path.

Bandwidth controls address a different network risk. The device rate limiters described earlier prevent one agent’s bulk download from consuming the uplink shared by other VMs. Per-tenant limits support both fairness and compliance requirements. Lifecycle cleanup is equally concrete. TAP removal must be automated and checked because leaked interfaces use up kernel resources and can, over time, help exhaust the address space on long-running hosts.

Per-user and multi-tenant fleet patterns: choosing between per-user isolation and pool mode, and what each costs

Per-user isolation assigns one microVM to each user session, separating guest kernels, memory, and filesystems. It is the recommended model for public multi-tenant platforms that run user-supplied or LLM-generated code. Pool mode instead serves more users from fewer VMs through directory-level or namespace-level separation. Those users share a guest kernel. Pooling is a security downgrade.

Hardening inside a pooled VM cannot recreate a VM boundary. Per-tenant UID and PID namespace controls are necessary, but kernel-escape vulnerabilities remain relevant to tenants sharing that kernel. Pooling's acceptability depends on how untrusted the code is and the strength of the tenant-separation guarantee.

Snapshotting provides a different route to density without pooling tenants inside one kernel. Active working-set memory plus orchestration overhead then sets the practical limit, even when the memory configured for all stored sessions adds up to more. Supporting that model requires image distribution and caching, snapshot-store management, warm-pool sizing where used, and scheduling restores on demand. It also requires per-VM lifecycle tracking, TAP lifecycle management, and metrics collection. Firecracker supplies none of this orchestration, so the operator must build it or use a platform that has.

What the full setup chain looks like assembled, and where managed infrastructure fits the picture

The assembled deployment begins with a KVM-capable host and a guest kernel configured with a CPU template. The rootfs then supplies tooling and a writable-layer strategy, while MMDS provides per-session credentials. CPU and memory sizing follow with device rate limits, and TAP networking adds the egress policy. Only then does snapshot baking establish a reusable execution state. The orchestration layer ties image delivery, restore scheduling, lifecycle tracking, and cleanup into an operating service.

The difficulty is that failures are often delayed. An unsupported CPU instruction may surface after migration, an egress omission may remain unnoticed until data leaves, and abandoned host resources may accumulate across many otherwise successful sessions. Teams that build from primitives must deal with each of those failures. Managed infrastructure trades some configuration control for a tested, maintained implementation of the surrounding chain.

Sources

  1. Firecracker
Filed underSelf-Hosting OAF

More in Self-Hosting OAF