Collecting Artifacts from Deployed Agents and Environments
Capture the state of deployed agents and envs as artifacts, for a judge to grade or for a later run to resume from
Built-in task steps collect artifacts from deployed agents and envs:
snapshot_env, collect_artifacts and snapshot_agent_state. Other agents and envs can load what
they collect, which supports snapshot and resume workflows for both. This page adds them to the
task from Using Artifacts with Agents, after a prompt_agent step has
the agent answer Dana and save what it sent:
{"id": "ask", "type": "prompt_agent", "agent_name": "assistant", "prompt_id": "p1",
"prompt": "Answer Dana's email with the Q3 numbers. Save what you sent as /app/artifact/q3-summary.md.",
"depends_on": [{"task_step_id": "skills"}]}Snapshot an environment
A snapshot_env step reads each server's state through data/get and stores it as a new universe,
with one environment artifact per server:
{"id": "snapshot", "type": "snapshot_env", "env_id": "email",
"depends_on": [{"task_step_id": "ask"}]}The universe is named for the env and the run, such as snapshot-email-3hpbbwj3, and the step
records its id and version under metadata["env_snapshotted_universes"]["snapshot"]. It holds the
inbox, and in sent the email the agent sent Sam.
Resume an environment
A load_artifact step with artifact_from_step_id loads whatever the named step produced, so it
can restore the snapshot into a new instance of email:
{"id": "resume-env", "type": "deploy_env", "env_id": "email", "env_version": 1,
"depends_on": [{"task_step_id": "snapshot"}]},
{"id": "restore", "type": "load_artifact", "env_id": "email", "env_step_id": "resume-env",
"artifact_from_step_id": "snapshot",
"depends_on": [{"task_step_id": "resume-env"}]}env_step_id picks the second deployment, since the task now deploys email twice. A later task
can load the same snapshot by its id and version, like any universe. A snapshot restores only what
your @add_data reads, which is why EmailEnv reads sent as well as inbox.
Collect an agent's files
A collect_artifacts step reads the files an agent wrote and stores them as a new file artifact
universe. The ask prompt had the agent save what it sent as q3-summary.md in /app/artifact,
the directory the step reads by default:
{"id": "collect", "type": "collect_artifacts", "agent_name": "assistant",
"artifact_paths": ["q3-summary.md"],
"depends_on": [{"task_step_id": "ask"}]}Grade with a judge
These steps give the collected files to a judge, a second agent that the rubrics_verifier step
asks to grade the run:
{"id": "judge", "type": "deploy_agent", "a2a_agent_id": "my-judge", "agent_name": "judge",
"depends_on": [{"task_step_id": "collect"}]},
{"id": "judge-files", "type": "load_artifact", "agent_name": "judge",
"artifact_from_step_id": "collect",
"depends_on": [{"task_step_id": "judge"}]},
{"id": "grade", "type": "rubrics_verifier", "agent_name": "judge", "prompt_id": "p1",
"verifier_id": "q3-rubric",
"criteria": [{"id": "c1", "criterion": "The email to Sam gives revenue, costs and margin for each region.", "weight": 1.0}],
"depends_on": [{"task_step_id": "judge-files"}]}artifact_from_step_id loads whatever collect produced, so the task does not need the new id in
advance. The verifier lists the judge's files in the prompt it grades with, and
Important steps covers verifiers.
Snapshot and resume an agent
A snapshot_agent_state step stores an agent's conversation under the id you give it, and a
deploy_agent step with agent_snapshot_files_artifact_id starts a fresh agent from it:
{"id": "agent-state", "type": "snapshot_agent_state", "agent_name": "assistant",
"prompt_id": "p1", "artifact_id": "assistant-state",
"depends_on": [{"task_step_id": "ask"}]},
{"id": "resume-agent", "type": "deploy_agent", "a2a_agent_id": "my-agent",
"agent_name": "assistant-resumed", "agent_snapshot_files_artifact_id": "assistant-state",
"depends_on": [{"task_step_id": "agent-state"}]}prompt_id names the prompt whose conversation the step captures. The resumed agent carries on
from it, so a follow-up prompt can pick up where the first agent stopped.
Warning: Agent snapshots need the snapshot extension
Both steps reach the agent through its urn:agentenv:snapshot/v1
extension, so the agent must advertise it.
Last updated on