Introduction
What agent-env is, what it builds, and when to reach for it
agent-env builds the worlds agents work in, and grades what they do there. An environment is a running sandbox an agent can act on — one or more MCP servers, a website, a Postgres database — and agent-env covers the whole loop around it: defining it, deploying it, seeding it with data, driving an agent through it, and scoring the result.
The problem
Evaluating an agent on anything realistic means standing up something for it to act on, and that is where the work goes. The environment has to start from a known state, or runs are not comparable. It has to be reproducible months later, when the model changed and you need to know whether the score moved for a real reason. It has to be gradeable, by assertions or by a judge. And it has to be disposable, because you will run it a thousand times.
Most of that is plumbing, and most teams rebuild it. agent-env is that plumbing, with the pieces separated so you can replace any one of them.
How it fits together
Four nouns carry most of the weight, and Core Concepts defines them properly:
- Artifacts are immutable, versioned inputs — a Docker image, a file, a dataset. Every write allocates a new version, so a run can pin exactly what it used.
- An env is a versioned definition of a world. Deploying one produces an env instance: a live sandbox with URLs and a TTL that tears itself down.
- A task is a DAG of steps — deploy the env, load a universe into it, deploy an agent, prompt it, grade the result — threading one shared context.
- Verifiers score the run, deterministically or with a judge model, and every one of them emits the same shape.
What is pluggable
The parts most likely to be wrong for you are the parts you can swap, through a single configuration file and without forking:
| Layer | Default | Swap it for |
|---|---|---|
| Documents | SQLite on disk | MongoDB, or your own DocumentStore |
| Blobs | The local filesystem | S3, or your own ObjectStore |
| Images | A local OCI registry | ECR, GHCR, or any Docker-compatible registry |
| Compute | Docker on the host | Modal, E2B, or your own SandboxProvider |
| Models | — | Any OpenAI-compatible endpoint |
Task steps, artifact types, and environment types are equally open: register a class and agent-env resolves it the same way it resolves a built-in. The extension model covers the rules, which differ by seam in ways worth knowing before you rely on them.
Where to start
Everything runs locally by default — SQLite, the filesystem, a local registry, Docker — so nothing here needs a cloud account to try.
- Getting Started — install it and deploy something
- Core Concepts — the vocabulary the rest of the site assumes
- Your first task — deploy, prompt, and grade end to end