AgentEnv Framework
Extending

The extension model

Four extension mechanisms, and the different failure policy each one has

agent-env resolves almost every layer it uses — stores, sandbox and state providers, steps, artifact types, env types, the run dispatcher, the CLI itself — through a name rather than a hard import, and which implementation a name points at is a configuration question. You can replace any of those layers from your own package, without forking. This page covers the model those names share and the rules that differ between them; it assumes the vocabulary from Core concepts.

The mental model to hold: there are four extension mechanisms, and they deliberately fail differently. Configuration is explicit intent, so a pointer you wrote down and got wrong stops the process. Installation is weaker intent, so a plugin that merely happens to be present and fails to load is skipped while the rest of the CLI keeps working. Do not assume one policy holds everywhere: the seam you are extending decides whether your mistake stops the process or is swallowed with a warning.

The four mechanisms

MechanismHow you registerIdentity ruleFailure policyChecked when
Config impl pointersimpl = "module.path:ClassName" in a config.toml tableThe table key, or the class's own typeFail loud — ConfigError, nothing is constructedFirst time that section is built in the process, then memoized
CLI command pluginsagent_env.cli_plugins entry pointThe command's own nameWarn on stderr and skip — including a name clashEvery CLI startup
CLI root optionsagent_env.cli_root_options entry pointThe flag stringWarn and skip on a clash with a core flag; abort startup when two plugins claim the same flagEvery CLI startup
Subclass registriesA type string on the classtype, persisted inside every stored documentValueError at read — a document of an unregistered type is unreadable, not ignoredEvery deserialization

Config impl pointers

This is the dominant mechanism. A section names a class, agent-env imports it, checks it against the target interface, and constructs it:

[stores.document]
impl = "mycorp.stores:PostgresDocumentStore"

[stores.document.config]
dsn = "env:MYCORP_DSN"
pool_size = 8

The loader splits module.path:ClassName, imports it, rejects anything that is not a subclass of that seam's abstract base class, resolves the env: and secret: references in the config table, and calls cls.from_config(**config). The default from_config forwards to __init__; override it when your class has to build a client. Secret values never live in the file, only references. Which tables accept which reference form, and what every key means, is in the configuration reference — this page does not repeat it.

Four things raise ConfigError: an impl that is not in module.path:ClassName form, a module or attribute that will not import, a class that is not a subclass of the seam's ABC, and a missing impl where one is required. Unknown keys beside impl and config in a store section are only logged as a warning (a [sandbox.providers.<name>] table for a built-in is stricter and raises), and unknown top-level tables are ignored entirely — a typo'd [task_step] registers nothing and says nothing.

Resolution is lazy and per-section. A broken [stores.object] does not stop a process that never touches the object store. The provider tables are the exception worth knowing: [sandbox.providers] and [state.providers] are validated as a whole when the registry is first assembled, so one bad entry fails even if you never select it.

Registries are memoized for the process lifetime

Every registry and store section is built once per process, so editing config.toml takes effect on the next process, not the next call — see Configuration for the details and what reset_config() does not clear.

CLI command plugins

An installed package contributes one top-level command or group per agent_env.cli_plugins entry point. Entry points load in name order, after the built-ins. A plugin is skipped, with a Warning: line on stderr, when it fails to load, when it resolves to something that is not a click.Command, when its name is already taken by a core command, or when registration itself raises. Discovery failing outright skips all of them.

Core names win; plugins cannot shadow a built-in command, and they cannot graft subcommands onto a built-in group. One entry point, one top-level command.

CLI root options

An agent_env.cli_root_options entry point contributes a click.Option to the root agent-env command. It must be optional and carry expose_value=False: it is parsed before the subcommand and acts through its callback, which is the hook for selecting a config file or activating packaged configuration. Options that fail to load, are not a click.Option, or set expose_value/required are skipped with a warning, as is a flag a core option already owns.

The one hard failure in the plugin system lives here. Two installed plugins registering the same flag raise RootOptionConflictError at import of the CLI, which takes down every invocation including agent-env --help, until one of them is uninstalled. A collision never silently picks a winner.

Subclass registries

Steps, artifacts and envs are looked up by a type string, which is what the stored document carries. Custom classes reach these registries through mechanism 1 — [task_steps].impls, [artifacts].impls, [envs].impls — but the failure that bites is at read time, in a process that never saw your config. Reading a task whose step type is not registered raises ValueError: Unknown task step type: grade_essay; an env raises Unknown env type: ...; an artifact raises Unknown artifact type '...'; register it under [artifacts].impls in .agentenv/config.toml.

The document is unreadable, not merely unrecognized. Every process that reads, runs, reconnects or reaps — a second CLI, a worker, the explorer — needs the same config.toml and your module importable on its PYTHONPATH.

Identity rules

A registry name is not a label. For the two provider seams it is the reconnect identity: the string agent-env persists on the resource and later reads back to rebuild the provider that owns it. A name that does not match what the provider produces strands the resource, because nothing can find its owner at teardown. Both seams enforce the invariant rather than documenting it, but they enforce it at different moments, and the difference follows from what each one can know.

A sandbox provider's registry name must equal the .type of the Sandbox it produces. That relationship cannot be established until an object exists, so it is enforced at deploy: agent-env wraps a config-registered provider's create_sandbox, create_vm and create_container, and on a mismatch terminates the sandbox it just made and raises SandboxProviderTypeError, naming both strings and telling you which to rename. The guard is installed only on providers registered from config.toml, not on built-ins. Sandbox.type is what a later teardown or reaper resolves the provider from.

A state provider's registry name must equal its class type. That is a class attribute, readable without running anything, so it is enforced at registration: building the [state.providers] registry raises ConfigError on a mismatch, with the reason in the message — the name is the identity a store is reattached and torn down by. The same registry rejects a name that collides with the built-in local_postgres.

Built-in names are not replaceable

A [sandbox.providers.<name>] table for a built-in may carry a config table only; setting impl on a built-in raises ConfigError. A [state.providers.<name>] key may not reuse a built-in name at all. Register your provider under a new name instead — the keys each table takes are in the configuration reference.

What you can replace

SeamInterfaceModuleRegistered byContract
Document storeDocumentStoreagent_env.store.document_store[stores.document]Collections of dicts with a predicate algebra, sorts, UpdateSpec updates, and compare-and-swap where update() returning 0 means the CAS missed
Object storeObjectStoreagent_env.store.object_store[stores.object]Write-once blobs under a logical key, read back through an opaque object_url
Image storeImageStoreagent_env.store.image_store[stores.image]image_ref(repository, tag) plus auth(ref) docker-login material; the store never runs docker
Registry credentialsOciRegistryCredentialsagent_env.store.image_store[stores.image.config.credentials]mint(host) returns fresh auth, minted lazily so rotation is observed
Secret storeSecretStoreagent_env.store.secret_store[stores.secret]get(name) returns the value or None; an empty string is a value, not an absence
Sandbox providerSandboxProvideragent_env.providers.sandbox_provider[sandbox.providers.<name>]create_sandbox is the only abstract method; the Sandbox it returns carries .type equal to the registry name
State providerEnvStateProvideragent_env.providers.state.env_state_provider[state.providers.<name>]acquire a private store per run and _teardown to release it; deploy_state_context supplies the acquire context or None
Task stepTaskStepagent_env.task_step[task_steps].implsasync execute(context) returning the context, plus a to_dict/from_dict round trip
ArtifactArtifactagent_env.artifact[artifacts].implsA Pydantic model whose type field default is its identity
EnvEnvagent_env.env.env[envs].implsfrom_dict plus async deploy() returning a DeployedEnv; the base raises on both
RunnerRunneragent_env.runner.runner[runner]submit returns a handle, status returns a record or None, cancel; type is stamped on every run record
Explorer pluginExplorerPluginagent_env.explorer.plugin[explorer.plugins].implsA router mounted ahead of the core routers, so it can shadow a core catch-all
CLI commandclick.Commandyour packageagent_env.cli_plugins entry pointOne top-level command or group
CLI root optionclick.Optionyour packageagent_env.cli_root_options entry pointOptional, expose_value=False, acts through its callback

Four seams take no configuration of their own. [explorer.plugins].impls, [task_steps].impls, [artifacts].impls and [envs].impls are all flat lists of impl pointers with no per-entry config table — a step's runtime parameters come from the task document, not the TOML.

The per-seam detail lives in Storage backends, Compute and state providers, and Custom types. Keys and defaults are in the configuration reference.

A worked example

A custom step is the smallest contract and the usual first extension. Three pieces: the class, the registration, and a task document that names its type.

mycorp/steps.py
from agent_env.task_step import TaskStep, TaskStepContext


class GradeEssayStep(TaskStep):
    # The portable identity. It is written into every task document that uses this
    # step, and it is the key the registry resolves on read.
    type = "grade_essay"

    def __init__(self, id, version, rubric_id, depends_on=None, fail_task_on_error=True):
        super().__init__(id, version, depends_on=depends_on, fail_task_on_error=fail_task_on_error)
        self.rubric_id = rubric_id

    def to_dict(self) -> dict:
        # super() contributes id, type, version, fail_task_on_error and depends_on.
        return {**super().to_dict(), "rubric_id": self.rubric_id}

    @classmethod
    def from_dict(cls, data: dict) -> "GradeEssayStep":
        # _base_from_dict rebuilds the base kwargs, including depends_on edges.
        return cls(**cls._base_from_dict(data), rubric_id=data["rubric_id"])

    async def execute(self, context: TaskStepContext) -> TaskStepContext:
        from agent_env.artifact import Artifact

        rubric = Artifact.get(self.rubric_id)
        context.metadata["grade"] = {"criteria": rubric.criteria, "score": 0.9}
        return context

execute is the only abstract method. preflight() is optional and returns one message per problem detectable before a run — it must not need a sandbox or an env instance. step_param_overrides(context) is optional and lets a run override stored parameters.

.agentenv/config.toml
[task_steps]
impls = ["mycorp.steps:GradeEssayStep"]

mycorp must be importable by every process that reads this task, so install it or put it on PYTHONPATH.

essay-task.json
[
  {"id": "grade-1", "type": "grade_essay", "rubric_id": "rubric-1"}
]

That is the shape agent-env task create reads: a bare list of step dicts, each passed to the step class's constructor as keyword arguments (everything but type). The stored task document nests the same dicts under steps, and Task.from_dict rebuilds each one through from_dict. Steps round-trip through to_dict/from_dict on every save and load, so a field that does not survive that round trip silently disappears.

Get it wrong and the errors are specific. Omit the [task_steps] entry and reading the task raises ValueError: Unknown task step type: grade_essay. Forget to set type and registration raises a ConfigError saying the impl inherits the base default. Reuse a type a built-in already owns and registration names the conflict.

Proving your implementation conforms

Each store interface has an implementation-neutral conformance suite, and the built-in implementations of each kind pass the same cases — three image stores included, with the one exception noted at the end of this section. Run them against your own class: a store that does not pass them fails later, inside a run, on behaviour agent-env assumes and does not re-check. There are 27 DocumentStore cases (unique-index violations, CAS hit and miss, the AbsentOrNull and LteOrAbsent preconditions, dotted and positional-array filters), 14 ObjectStore cases, 4 ImageStore cases including a real build/push/pull round trip, and 3 SecretStore cases.

Each suite is a list of plain functions in CASES, so running them against your store is one parametrize. Only the secret cases take a store alone; the document, object and image cases take a second argument — a collection name, a key prefix, or a repository — which the parametrize passes through as case(store, coll), the way the built-in harnesses do:

tests/test_my_secret_store.py
import pytest

from mycorp.stores import MySecretStore
from tst.store import secret_conformance


@pytest.fixture
def store():
    return MySecretStore(values=dict(secret_conformance.FIXTURE))


@pytest.mark.parametrize("case", secret_conformance.CASES, ids=lambda c: c.__name__)
def test_conformance(case, store):
    case(store)

The suites are not in the wheel yet

They live in tst/, and the wheel packages only src/agent_env, so from tst.store import conformance does not resolve from a pip install. Today you can run them only from a clone of the repository. Shipping them from the installed package is a known gap.

Passing conformance is necessary, not sufficient. No conformance case covers signed_get_url, signed_put_url or signed_post; the built-in object stores cover them in their own backend-specific tests. They default to returning None, so a custom object store that does not implement them loses direct downloads: agent-env falls back to streaming the bytes through itself into the sandbox instead of handing the sandbox a URL to fetch. Of the two built-in secret stores, only the local one runs the secret cases.

On this page