Skip to content
AgentEnv Framework

Collecting Artifacts from Deployed Agents and Environments

Capture the state of deployed agents and envs as artifacts, for a judge to grade or for a later run to resume from

Built-in task steps collect artifacts from deployed agents and envs: snapshot_env, collect_artifacts and snapshot_agent_state. Other agents and envs can load what they collect, which supports snapshot and resume workflows for both. This page adds them to the task from Using Artifacts with Agents, after a prompt_agent step has the agent answer Dana and save what it sent:

task.json
{"id": "ask", "type": "prompt_agent", "agent_name": "assistant", "prompt_id": "p1",
 "prompt": "Answer Dana's email with the Q3 numbers. Save what you sent as /app/artifact/q3-summary.md.",
 "depends_on": [{"task_step_id": "skills"}]}

Snapshot an environment

A snapshot_env step reads each server's state through data/get and stores it as a new universe, with one environment artifact per server:

task.json
{"id": "snapshot", "type": "snapshot_env", "env_id": "email",
 "depends_on": [{"task_step_id": "ask"}]}

The universe is named for the env and the run, such as snapshot-email-3hpbbwj3, and the step records its id and version under metadata["env_snapshotted_universes"]["snapshot"]. It holds the inbox, and in sent the email the agent sent Sam.

Resume an environment

A load_artifact step with artifact_from_step_id loads whatever the named step produced, so it can restore the snapshot into a new instance of email:

task.json
{"id": "resume-env", "type": "deploy_env", "env_id": "email", "env_version": 1,
 "depends_on": [{"task_step_id": "snapshot"}]},
{"id": "restore", "type": "load_artifact", "env_id": "email", "env_step_id": "resume-env",
 "artifact_from_step_id": "snapshot",
 "depends_on": [{"task_step_id": "resume-env"}]}

env_step_id picks the second deployment, since the task now deploys email twice. A later task can load the same snapshot by its id and version, like any universe. A snapshot restores only what your @add_data reads, which is why EmailEnv reads sent as well as inbox.

Collect an agent's files

A collect_artifacts step reads the files an agent wrote and stores them as a new file artifact universe. The ask prompt had the agent save what it sent as q3-summary.md in /app/artifact, the directory the step reads by default:

task.json
{"id": "collect", "type": "collect_artifacts", "agent_name": "assistant",
 "artifact_paths": ["q3-summary.md"],
 "depends_on": [{"task_step_id": "ask"}]}

Grade with a judge

These steps give the collected files to a judge, a second agent that the rubrics_verifier step asks to grade the run:

task.json
{"id": "judge", "type": "deploy_agent", "a2a_agent_id": "my-judge", "agent_name": "judge",
 "depends_on": [{"task_step_id": "collect"}]},
{"id": "judge-files", "type": "load_artifact", "agent_name": "judge",
 "artifact_from_step_id": "collect",
 "depends_on": [{"task_step_id": "judge"}]},
{"id": "grade", "type": "rubrics_verifier", "agent_name": "judge", "prompt_id": "p1",
 "verifier_id": "q3-rubric",
 "criteria": [{"id": "c1", "criterion": "The email to Sam gives revenue, costs and margin for each region.", "weight": 1.0}],
 "depends_on": [{"task_step_id": "judge-files"}]}

artifact_from_step_id loads whatever collect produced, so the task does not need the new id in advance. The verifier lists the judge's files in the prompt it grades with, and Important steps covers verifiers.

Snapshot and resume an agent

A snapshot_agent_state step stores an agent's conversation under the id you give it, and a deploy_agent step with agent_snapshot_files_artifact_id starts a fresh agent from it:

task.json
{"id": "agent-state", "type": "snapshot_agent_state", "agent_name": "assistant",
 "prompt_id": "p1", "artifact_id": "assistant-state",
 "depends_on": [{"task_step_id": "ask"}]},
{"id": "resume-agent", "type": "deploy_agent", "a2a_agent_id": "my-agent",
 "agent_name": "assistant-resumed", "agent_snapshot_files_artifact_id": "assistant-state",
 "depends_on": [{"task_step_id": "agent-state"}]}

prompt_id names the prompt whose conversation the step captures. The resumed agent carries on from it, so a follow-up prompt can pick up where the first agent stopped.

Warning: Agent snapshots need the snapshot extension

Both steps reach the agent through its urn:agentenv:snapshot/v1 extension, so the agent must advertise it.

Last updated on

Ask a question · Report an issue

On this page