Skip to content
Blogs

RBAC is necessary in multi-agent RL Envs

The enterprise RL environment you built is not realistic without RBAC

Get started with AgentEnv Framework by following our docs.

At Scale AI we work towards building realistic worlds for enterprise environments. These environments are critical if we are to train agents to do meaningful work across different verticals such as accounting, finance, healthcare, and more. Simulating the real world is unsurprisingly difficult: environment tools must be functionally identical to their real counterparts, synthetic data must be indistinguishable from real data, data must not contain PII, the environment needs to be reproducible, and these are just some of the challenges that come to mind.

Enterprise RL Environment

where agents learn to do real work

What goes into a realistic enterprise RL environment. Hover or tap an ingredient to read more.

In this blog we will cover an underdiscussed dimension for building a world simulation: an agent's access control in the environment must be realistic, especially so when there are multiple agents concurrently operating in it.

1. Control the agent's tools

For most people, Role-Based Access Control (RBAC) for agents means one thing: limiting which tools they can call. This is the simplest form of RBAC in RL environments and is table stakes. To make this concrete: suppose you have an engineering startup environment containing tools from Slack, GitHub, Linear, MongoDB, Notion, Google Workspace, and a few other servers. This environment is designed to be useful for engineers, product managers, SREs, and executives; yet realistically not everyone has access to everything.

Paused

Engineering startup

8 servers · 17 tools · one gateway

  • Slack

    • post_message
    • search
  • GitHub

    • open_pr
    • merge_pr
    • search_code
  • Linear

    • create_issue
    • update_issue
    • search_issues
  • MongoDB

    • find
    • update
  • Notion

    • create_page
    • search
  • Google Workspace

    • write_doc
    • schedule_event
  • PagerDuty

    • acknowledge
    • resolve
  • Datadog

    • query_metrics

Tool callsAgentEnv-Role

  1. 08:59:58linear_search_issues okexecutive
  2. 08:59:51linear_search_issues okengineer
  3. 08:59:43slack_search okpm
  4. 08:59:36pagerduty_acknowledge oksre
Four agents share one engineering startup environment, and each is offered only its role's tools. Click an agent to see what its role is offered, or open the task DAG to see how the task sets this up.

This is trivial to set up in an AgentEnv Framework task with tool-based RBAC: deploy each agent with its role, then turn on only the tools that role needs.

Take it further, modify an agent's tool access during its task using triggers.

2. Control the agent's view

Here is where RBAC gets a bit more complicated. Your enterprise environment can contain millions of records of data, such as emails; just because your agent has access to reading emails does not mean it should be able to access every email in the organization. Like the real world, it should only be able to see its own role's emails. The same applies everywhere else. Implementing this form of RBAC in your environment requires knowing the calling agent's identity, or role, at the time of its tool call.

Paused

Enterprise environment

As Bob, role pm: 8 of 16 records

Gmail2 of 4

  • Q4 roadmap draft
  • Launch checklist
  • A record Bob can't see
  • A record Bob can't see

Slack2 of 4

  • launch
  • Bob, Priya
  • A record Bob can't see
  • A record Bob can't see

GitHub2 of 4

  • acme/weball
  • A record Bob can't see
  • acme/launch-site
  • A record Bob can't see

Google Drive2 of 4

  • Q4 roadmap
  • A record Bob can't see
  • A record Bob can't see
  • Launch spec

Tool callsAgentEnv-Role

  1. 08:59:58drive_list_docs() → 1 docsre
  2. 08:59:51drive_list_docs() → 2 docsengineer
  3. 08:59:44github_list_repos() → 2 repossales
  4. 08:59:37github_list_repos() → 2 repospm
Four agents call the same tools in one enterprise environment, and each gets back only what its person may see. Click an agent to see the environment as that person, or open the code to see how each server decides.

The AgentEnv Framework's environments hand each tool call a ToolContext containing the calling agent's role. You can use this in your environment to provide granular RBAC:

email/server.py
from agentenv_protocol import AgentEnvEnvironment, ToolContext, tool


class EmailEnv(AgentEnvEnvironment):
    ...

    @tool()
    async def search(self, query: str, context: ToolContext) -> dict:
        """Search the caller's inbox."""
        me = self.directory.get(context.role)  # "pm" -> "bob@example.com"
        mine = [e for e in self.emails if me in e["to"]]
        return {"emails": [e for e in mine if query in e["body"]]}

    @tool()
    async def send(self, to: str, body: str, context: ToolContext) -> dict:
        """Send an email as the caller."""
        me = self.directory.get(context.role)
        if me is None:
            raise PermissionError("No mailbox for this role")
        self.emails.append({"from": me, "to": [to], "body": body})
        return {"sent": True}

self.directory is the environment's record of who holds each role, so an agent deployed as pm reads and sends Bob's mail. The agent never sees context: the SDK fills it in on every call and leaves it out of the tool's input schema.

3. Control many agents

In a realistic simulation of the real world you naturally end up with a multi-agent system, with each agent acting as a different role or business function within the organization. The AgentEnv Framework lets you express such a world simply through its task DAG.

Take launch day at the engineering startup, with three of the agents from section 2. A fix for a checkout timeout is ready to go out:

  • Bob, the PM (pm), decides whether the fix ships today and says so in #launch.
  • Priya, the engineer (engineer), opens the pull request, but she can't merge it until Bob says "ship it".
  • Marcus, the SRE (sre), watches checkout latency once the fix is out and confirms in #launch that it's back to normal.

Bob and Marcus can't message each other, so anything between them goes through Priya. Once the three are done, a fourth agent judges the result: with read-only tools, it checks that the merge came after Bob's go-ahead and that Marcus confirmed checkout recovered. The task below sets all of this up and runs it; click any step to see its JSON.

task.json · 16 steps

deploy_envstartupmodify_env_tool_accesslock-allmodify_env_tool_accessallow-bobmodify_env_tool_accessallow-priyamodify_env_tool_accessallow-marcusdeploy_agentbobdeploy_agentpriyadeploy_agentmarcuspeer_agentspeersregister_env_triggerslaunch-gateprompt_agentgo-no-goprompt_agentshipprompt_agentwatchmodify_env_tool_accessallow-graderdeploy_agentgraderrubrics_verifiershipped

Run logwaiting to run

    peers · peer_agents

    Messaging

    Tells each agent who it may message: Bob and Priya, and Priya and Marcus. Bob and Marcus can’t reach each other, so anything between them goes through Priya.

    {"id": "peers","type": "peer_agents","peerings": [{"source_agent_name": "bob", "peer_agent_names": ["priya"]}, {"source_agent_name": "priya", "peer_agent_names": ["bob", "marcus"]}, {"source_agent_name": "marcus", "peer_agent_names": ["priya"]}],"depends_on": [{"task_step_id": "bob"}, {"task_step_id": "priya"}, {"task_step_id": "marcus"}]}
    A launch day for three agents and a judge, as one task. Click a step to see its JSON, or a label to see the steps that set that access.

    The DAG is where the RBAC lives: each deploy_agent gives its agent a role, modify_env_tool_access decides which tools each role is offered, and peer_agents decides which agents can message each other. A trigger changes access mid-run: here Priya gets github_merge_pr only once Bob says ship it.

    Realistic tools and data are only part of a realistic environment. Once those components are created, the important next step is setting realistic limits on what each role can call, what it can see, and whom it can talk to. Without enabling RBAC, agents operate in a simplified, sanitized environment that doesn't actually capture real-world settings. The AgentEnv Framework makes all of this possible in a few lines of JSON so you can spend your time designing interesting worlds instead of plumbing permissions. We hope you have as much fun building with it as we did!

    Try AgentEnv Framework

    It's open source, and one install away. Its hello task runs on your machine with no Docker, model or cloud account.

    pip install agentenv-framework

    Then follow a cookbook to build a complete task, from an empty folder to a graded run.

    Acknowledgements

    Written by Edgar Arakelyan, Razvan Dumitru, Alina Trinh, and Mateusz Cikowski.