Tools & Environment

Give GPT-4 Turbo a plain shell and point it at SWE-bench Lite, as a 2024 ablation did: it resolves 11.0% of the issues. Keep the model and the tasks fixed, replace the shell with a purpose-built interface, and it resolves 18.0%. That is seven points, a 64% relative gain, and not one weight changed.¹ This chapter is about that interface: the tools an agent holds, the machine it runs them on, and why both deserve as much engineering as the prompt.

The interface is a variable

The Princeton team behind SWE-agent named the thing they built an agent-computer interface, by deliberate analogy with the IDE. Humans got better at software when editors learned to jump to definitions and underline errors. Models are no different, except that their editor has to be designed for a reader that sees text, not pixels.¹

Their four design principles read like a style guide for any tool you will ever hand a model: actions should be simple and easy to understand; they should be compact and efficient; feedback should be informative but concise; and guardrails should stop errors from compounding.¹ None of these is about the model. All of them are about you.

What a good tool looks like

Anthropic's guidance on writing tools lands in the same place from the other direction.² Consolidate: one schedule_event beats separate list_users, list_events and create_event calls, and search_logs beats read_logs. Namespace by service and resource, as in asana_projects_search, so similar tools stop colliding. Return high-signal context: a Slack tool answering in a concise format used 72 tokens where the detailed one used 206. Claude Code caps tool results at 25,000 tokens and nudges the agent toward narrower queries when it hits the cap. Prefer names to UUIDs, because agents handle natural-language identifiers far better than opaque ones.²

“Tools are a new kind of software which reflects a contract between deterministic systems and non-deterministic agents.”— Ken Aizawa, Anthropic (2025)

The cheapest lever is the description. Anthropic's platform docs call detailed descriptions “by far the most important factor in tool performance” and ask for at least three to four sentences per tool.³ Compare:

// before
get_stock_price  "Gets the stock price for a ticker."
  ticker: string

// after
get_stock_price  "Returns the most recent trade price, in USD,
  for one exchange-listed ticker. Use it when the user wants a
  current quote. It returns no history, volume or fundamentals;
  call a different tool for those. Fails on unlisted symbols."
  ticker: string   // uppercase symbol, e.g. "AAPL"

The SWE-agent ablations put numbers on these principles. They are historical (2024, one model, one benchmark), but they remain the cleanest single-variable test of interface design, and the 2025 guidance above arrives at the same rules.¹ A 100-line file viewer beat both a 30-line window and the whole file, which is pagination by another name. An editor that lints every change beat one that does not, which is a guardrail. Summarized search beat iterative search, which is high-signal returns. And iterative search scored below no search at all: a badly shaped tool costs more than a missing one. Try it.

The tool-count cliff

More tools do not mean a more capable agent. Anthropic lists bloated tool sets with ambiguous decision points among the most common failure modes and recommends a minimal viable set.⁷ The measurements agree. With a large MCP tool pool dumped into the prompt, RAG-MCP found tool-selection accuracy of 13.62%; retrieving a shortlist first raised it to 43.13% and more than halved prompt tokens.⁸

A production routing study with 110 agents and 584 tools saw F1 on under-specified requests fall 16 to 23 points as the catalog grew from 10 agents to 110. Even with perfect retrieval the ceiling dropped about 10 points, a confusion gap that no shortlist can close; embedding shortlists won back 10 to 11 points.⁹ On edge devices, pruning the tool list improved function calling and cut execution time by up to 70% and power by up to 40%.¹⁰

Manus adds a constraint from the serving side: do not add or remove tools mid-task, because changing the definitions near the front of the context invalidates the KV cache. Mask, don't remove (Chapter 2 covers the cache economics).¹¹ Most hosted APIs give you no logit access. There the same idea becomes a stable tool list plus a per-turn restriction: tool_choice: none to forbid calls on a turn, or an allowed-tools list enforced by the harness. Forcing a call (any or a named tool) is not an option on the newest Claude models, which reject it with a 400. On the Claude API, changing tool_choice invalidates cached message blocks, but tool definitions and the system prompt stay cached.¹²

MCP: the standard and its bill

The Model Context Protocol uses JSON-RPC 2.0 messages. The current revision (schema 2026-07-28) makes every request stateless and self-contained, with capabilities negotiated per request. Servers offer tools, resources and prompts; clients may offer elicitation, a server-initiated request for more information from the user. Long-running asynchronous tasks live in opt-in extensions. Earlier revisions, such as 2025-06-18, ran over stateful connections and had clients also offer sampling and roots; much published MCP material still describes that design. The spec takes inspiration from the Language Server Protocol, and the payoff is the same: write an integration once and every compliant client can use it.¹³

The bill can arrive before the first question. In a harness that loads definitions eagerly, the GitHub MCP server's 93 tools take roughly 55,000 tokens before the task has touched GitHub, by one vendor's measurement (Speakeasy, March 2026).⁴ That is over a quarter of a 200K window spent on a menu. Drag the slider above to see how fast a few servers add up.

The current fix is to stop loading the menu. With deferred loading, the harness sends only tool names (or a search tool) up front and pulls a full definition in when the model asks for it. Claude Code does this by default on current models: only tool names and server instructions load at session start, and ENABLE_TOOL_SEARCH sets the behavior (false loads everything up front; auto defers only once definitions pass 10% of the window).⁵ The Claude API exposes the same mechanism as a tool search tool: mark tools defer_loading: true and the model searches the catalog and loads typically 3 to 5 tools per request. Anthropic's docs say this usually cuts a ~55k-token multi-server setup by over 85%, that selection accuracy degrades past 30 to 50 loaded tools, and that because deferred tools stay out of the prompt prefix, caching survives.¹⁴ In your own loop the generic form is retrieval over the tool catalog, the RAG-MCP result above, with the shortlist appended after the cached prefix rather than spliced into it. The cliff does not vanish; descriptions still have to be good enough to be found.

Then there is the attack surface. The spec says it plainly: “Tools represent arbitrary code execution and must be treated with appropriate caution,” and descriptions from untrusted servers are untrusted input.¹³ Invariant Labs showed why. Hidden instructions in a tool description got Cursor to read ~/.cursor/mcp.json and ~/.ssh/id_rsa and leak them. A server can change its descriptions after you approve it, and a malicious tool can shadow the behavior of a trusted one on the same client.¹⁵

Two teams converged on the same answer. Anthropic proposed presenting MCP servers to the agent as a filesystem of code modules: the agent reads only the files it needs and writes code that calls them, so intermediate results stay in the execution environment and sensitive fields can be tokenized before the model sees them. One workflow dropped from 150,000 tokens to 2,000.⁶ Cloudflare turned MCP schemas into a TypeScript API and ran the generated code in V8 isolates that start in milliseconds, use a few megabytes, and can reach nothing but the bound servers.¹⁶

“LLMs are better at writing code to call MCP, than at calling MCP directly.”— Kenton Varda & Sunil Pai, Cloudflare (2025)

Code means a sandbox

Once the agent writes and runs code, isolation stops being optional. The question is how thick the wall needs to be, and the answer comes from two numbers: how much you trust the code, and how fast the environment must start.

None of these columns is the wall that matters most. That one is the network: default-deny egress, with secrets injected by the dispatcher and never mounted in the VM. Code that cannot reach an attacker and never holds a key cannot leak one. Cloudflare's isolates show the shape: fetch() and connect() throw, and the bindings hold the credentials, so “the AI cannot possibly write code that leaks any keys.”¹⁶ Chapter 4 builds on this.

The sandbox also changes what you can permit. Claude Code pairs permission allowlists with OS-level filesystem and network restrictions so the agent can work with fewer interruptions inside a known boundary.²¹ In your own loop the equivalent is a dispatcher that runs every tool inside the sandbox and widens permissions only for what the sandbox already contains. The wall is what makes autonomy affordable.

Once there's a disk, use it

A sandbox comes with a filesystem, and a filesystem is memory the context window cannot match. Anthropic recommends holding lightweight identifiers such as paths, stored queries and links, then loading the content just in time, because recall degrades as the window fills.⁷ Manus treats the file system as unlimited, persistent, structured memory and compresses only restorably: drop the page but keep the URL, drop the document but keep the path.¹¹

Skills apply the same idea to instructions. A skill is a folder with a SKILL.md: a YAML name and description, a body, and optional files. Only the name and description sit in the system prompt at startup; the body loads when the task matches; linked files load only when needed. Anthropic compares writing one to assembling an onboarding guide for a new hire.²² The always-loaded file, CLAUDE.md, should hold only what applies to every session. The docs are blunt about the alternative: “Bloated CLAUDE.md files cause Claude to ignore your actual instructions!”²¹

Deterministic edges, universal tool

Everything so far shapes what the model is likely to do. Hooks decide what must happen. In Claude Code they are shell commands, HTTP endpoints, MCP tools, prompts or subagents that fire on lifecycle events such as PreToolUse, PostToolUse and Stop. Exit 0 proceeds. Exit 2 blocks: a PreToolUse hook stops the call, a Stop hook keeps the agent working, and a PostToolUse hook can only report, because the tool already ran. JSON output can allow, deny or ask, inject context, or rewrite a tool's input before it runs or its result after.²³ In your own loop the generic form is a pre-dispatch interceptor: plain code that sees every tool call before it runs and can refuse it. The docs draw the line cleanly: instructions in CLAUDE.md are advisory, while hooks “are deterministic and guarantee the action happens.”²¹

Copy this: a minimal, best-effort PreToolUse hook that refuses writes to .env and .envrc files (templates such as .env.example pass) and to anything under secrets/, checked against the current hooks reference.²³ Two files. First, .claude/settings.json:

{ "hooks": { "PreToolUse": [ { "matcher": "Edit|Write|NotebookEdit|Bash",
  "hooks": [ { "type": "command",
    "command": "${CLAUDE_PROJECT_DIR}/.claude/hooks/no-secrets.sh" } ] } ] } }

Then the script it points at:

#!/bin/bash
# .claude/hooks/no-secrets.sh  (chmod +x; needs jq). Best-effort string matching.
set -f; shopt -s nocasematch   # no globbing of command words; SECRETS/ == secrets/
in=$(cat)
protected() { case "$1" in
  *.env.example|*.env.sample|*.env.template) return 1 ;;   # templates are fine
  *.env|*.env.*|*.envrc|secrets/*|*/secrets/*) return 0 ;;
  *) return 1 ;; esac; }
deny() { jq -n --arg r "$1" '{hookSpecificOutput: {hookEventName: "PreToolUse",
  permissionDecision: "deny", permissionDecisionReason: ($r + ". Ask the user.")}}'; exit 0; }
if [[ $(jq -r .tool_name <<<"$in") == Bash ]]; then
  cmd=$(jq -r '.tool_input.command // empty' <<<"$in")
  case "$cmd" in *'>'*|*tee\ *|*cp\ *|*mv\ *|*'sed -i'*|*dd\ *) ;; *) exit 0 ;; esac
  for t in ${cmd//[>;|&()]/ }; do t=${t#of=}; protected "${t//[\"\']/}" && deny "Shell write may touch $t"; done
else
  f=$(jq -r '.tool_input.file_path // .tool_input.notebook_path // empty' <<<"$in")
  protected "$f" && deny "Protected path: $f"
fi
exit 0  # no decision: normal permission flow

Silent exit 0 defers to the normal permission flow; exit 2 would also block, with stderr as the reason. The Bash branch only guesses at write targets from the command string, so python -c "open('.env','w')" walks past it, the same weakness as the Bash(curl *) rule in Chapter 4. The guarantee is the sandbox: list the paths under sandbox.filesystem.denyWrite and the operating system refuses the write whatever the command looks like (Chapter 4). In another harness or your own loop, the hook is a check in the tool dispatcher that runs before the write and returns the refusal to the model as the tool result.

The most general tool remains the shell. Claude Code's docs call CLI tools the most context-efficient way to reach external services and suggest teaching an unfamiliar one with --help. The harness composes the same way: cat error.log | claude -p "explain this error" makes the agent one more filter in a pipeline.²¹ A CLI call is also reproducible by a human, who can paste the exact command and get the same result.⁴

When no CLI or API exists, computer use is the fallback. At launch in October 2024, a historical baseline, Claude 3.5 Sonnet scored 14.9% on OSWorld from screenshots alone and 22.0% with more steps, against 7.8% for the next-best system, and Anthropic called the ability imperfect.²⁴ Screenshots earn their keep as a checker even when text tools do the work: compare the rendered result against the target.²¹

Sandboxes, permission lists and hooks are where trust stops being a hope and becomes a mechanism. Chapter 4 takes that seriously: what to block, what to ask about, and how to let an agent act without letting it act on everything.