DOC LLM-HARNESS-001REF pi-mono @ eba8497LANG ASD-STE100ISSUE 2026-10-09

Tokens to
Agent Teams

How a next-token predictor becomes a coding harness that runs many agents and completes large tasks. The reference implementation is pi, an open-source harness. You can read every line of code that this document cites.

Writing standard. This document uses ASD-STE100 rules. Sentences are short. Each sentence has one idea. Verbs are in the active voice. One term has one meaning (see the terms list).

Code paths refer to packages/ in the pi-mono repository. Numbers come from the source at commit eba8497. Some values can change in later commits.

Overview

The stack

The model does one thing. It predicts the next token. All other capability comes from software around the model. This software is the harness.

The harness adds three things that the model does not have:

Memory. The model keeps no state between calls. The harness stores the conversation and sends it again on each call.
Action. The model can only write text. The harness reads special text from the model and runs real commands.
Limits management. The model can read a fixed number of tokens. The harness decides what goes into that space.

Each layer below uses only the layer under it. Read the layers in order.

FIG 1 Each layer adds one capability. The right column shows which pi package holds the code for that layer.

Terms used in this document

harness: The program that calls the model, runs tools, and stores the conversation. Here: pi.
context: All tokens that the harness sends to the model in one call.
message: One part of the context with one role: system, user, assistant, or tool result.
tool: A function that the harness can run. It has a name, a description, and a parameter schema.
tool call: A request from the model to run one tool with specified arguments. The model writes it. The harness runs it.
turn: One model response plus the execution of all tool calls in that response.
run: All turns from one user message until the agent stops.
subagent: A second agent loop that a tool starts. It has its own empty context.

L0Layer 0 · pi-ai

Next-token predictor

You know this layer. This section only names the three properties that the harness must work around.

The model is stateless. Each call starts from zero. The model knows only the tokens in that call.
The context window is finite. A model can read a fixed maximum number of tokens. Typical values are 200,000 to 1,000,000.
The output is text only. The model cannot open a file or run a test. It can only write tokens.

Post-training changes which tokens are probable. It teaches the model a format: roles, end-of-turn markers, and tool-call blocks. The harness depends on this format. Use the widget to see one assistant turn. The model writes some text, then a tool call, then an end marker.

Token sequence (the context)

Next-token distribution (top 4)

FIG 2 Sampling one assistant turn. Probabilities and token boundaries are illustrative. Real chat templates differ by vendor. The structure is the same: the model writes a tool-call block, then an end marker. The server stops generation at the end marker.

L1Layer 1 · pi-ai

Chat protocol

A conversation is one long token sequence. Special tokens divide it into messages. Each message has a role. The API server converts a list of JSON messages into this sequence. The model then continues the sequence as the assistant.

pi talks to many providers. Each provider has a different wire format. The pi-ai package converts all of them into one shape: a list of messages with roles system, user, assistant, and toolResult.

FIG 3 The context is a flat ribbon. Roles are markers inside it. The dashed box is the only part that the model generates on this call.

Each assistant message ends with a stop reason. pi defines these values in ai/src/types.ts:

stopReason	Meaning	What the harness does
`stop`	The model wrote its end-of-turn marker.	Stop the run if the message has no tool calls.
`toolUse`	The model ended its turn with one or more tool calls.	Run the tools. Start another turn.
`length`	The output hit the token limit.	Do not run any tool call. The arguments can be cut off. Return an error result for each call.
`error` / `aborted`	The request failed or the user stopped it.	End the run.

L2Layer 2 · pi-ai

Tool calls

A tool call is text with a known structure. Nothing more. The model learned in post-training to write a call block when a tool can help. The call block holds a tool name and JSON arguments.

A tool has three parts: a name, a description, and a JSON Schema for its parameters. The harness sends all tool definitions with each request. The server puts them into the prompt as text. The model reads the descriptions to decide which tool to use. The description is a prompt. Write it with care.

This is the real edit tool schema in pi. pi uses TypeBox to write JSON Schema in TypeScript:

const editSchema = Type.Object({
  path: Type.String({ description: "Path to the file to edit (relative or absolute)" }),
  edits: Type.Array(Type.Object({
    oldText: Type.String({ description: "Exact text for one targeted replacement. It must be unique
                                     in the original file and must not overlap ..." }),
    newText: Type.String({ description: "Replacement text for this targeted edit." }),
  })),
});

coding-agent/src/core/tools/edit.ts

FIG 4 The model never runs a tool. It writes a request. The harness checks the request against the schema and runs it. The result goes back into the context on the next call.

pi streams the response as events: text_start/delta/end, thinking_start/delta/end, and toolcall_start/delta/end. During toolcall_delta, pi parses the partial JSON. Thus the user interface can show the arguments while the model writes them.

ImportantIf the arguments do not agree with the schema, pi does not run the tool. It returns the validation error as the tool result. The model reads the error and writes a correct call on the next turn. This is the first example of a feedback loop. Feedback loops are the subject of L4.

L3Layer 3 · pi-agent

The agent loop

An agent is a loop around the model. The loop calls the model. If the response has tool calls, the loop runs them and adds the results to the context. Then it calls the model again. If the response has no tool calls, the loop stops.

That is the full core. In pi, it is one function, runLoop, in about 120 lines. This is a condensed form:

async function runLoop(context, config) {
  while (true) {                                   // outer loop: follow-up messages
    while (hasMoreToolCalls || pending.length) {   // inner loop: one turn per pass
      context.messages.push(...pending)          // steering messages from the user
      const msg = await streamAssistantResponse(context, config)
      if (msg.stopReason === "error" || msg.stopReason === "aborted") return

      const calls = msg.content.filter(c => c.type === "toolCall")
      const results = msg.stopReason === "length"
        ? failToolCallsFromTruncatedMessage(calls)     // never run cut-off JSON
        : await executeToolCalls(context, msg, config)
      context.messages.push(...results)
      hasMoreToolCalls = calls.length > 0

      await config.finishTurn?.(...)                   // hook: can end or continue
      pending = await config.getSteeringMessages?.()
    }
    const followUp = await config.getFollowUpMessages?.()
    if (!followUp.length) break
    pending = followUp
  }
}

agent/src/agent-loop.ts · runLoop()

Step through one real run. The task is "The parser test fails. Fix it." Look at the transcript. It grows by one message for each event. The model sees the full transcript on each call.

context.messages

FIG 5 One run with five turns. The test failure is not an error of the loop. It is information. The model uses it to choose the next action.

Control points in the loop

The loop has hooks. The coding agent (L4 to L7) uses them. You can use them in your own harness.

transformContext changes the message list before each call. Use it to remove or summarize old content.
convertToLlm converts internal message types into provider messages. The transcript can hold UI-only entries. This hook removes them.
beforeToolCall and afterToolCall wrap each tool. Use them for permission checks and audit logs.
getSteeringMessages lets the user type while the agent works. A steering message enters after the current turn.
getFollowUpMessages holds messages until the agent would stop. Then the loop starts again with them.

L4Layer 4 · pi-coding-agent

Coding tools and feedback

The loop is general. The tools make it a coding agent. pi gives the model four tools by default: read, bash, edit, and write. Optional tools are grep, find, and ls.

The set is small because bash gives access to all other programs. The compiler, the test runner, git, and package managers are all available through one tool. The model already knows how to use these programs from its training data.

Tool	Design decision in pi	Reason
`read`	Output is limited to 2,000 lines or 50 KB. The model can request an offset.	One large file must not fill the context.
`edit`	Exact text replacement. `oldText` must be unique. One call can hold many disjoint edits.	The edit fails clearly if the model has an old copy of the file. The model then reads the file again.
`write`	Writes a full file.	Use for new files. Full rewrites of large files use many output tokens.
`bash`	Runs a command. Output is truncated with the same limits.	Gives access to compilers, tests, git, and all CLI tools.

Parallel tool calls

One assistant message can hold many tool calls. pi runs them in parallel by default (executeToolCallsParallel, with Promise.all). It keeps the results in the original order. A tool can set executionMode: "sequential". Then pi runs the full batch in sequence.

Two edits to the same file must not run at the same time. file-mutation-queue.ts keeps one queue for each real file path. Operations on one file run in sequence. Operations on different files run in parallel.

Why feedback makes long tasks possible

Each step of a task has some probability of error. Without feedback, errors add up. Over many steps, the probability of a fully correct result decreases quickly.

Tool results change this. A failed test, a compiler error, or a failed edit is a signal. The model reads the signal and corrects the error on the next turn. The environment is the source of truth. The model does not have to be correct on the first attempt. It must only detect errors and correct them.

Per-step success Steps in task Retries after a detected error

No feedback (one attempt)

pⁿ

With feedback and retries

(1 − (1−p)^k+1)ⁿ

FIG 6 Illustrative model of task success. It assumes independent errors and a check that detects every error. Real checks miss some errors. Thus good tests increase agent reliability more than a better prompt does.

L5Layer 5 · pi-coding-agent

Context assembly

The model knows only what is in the context. Thus the harness must put the correct information there. It must also keep wrong or useless information out. pi builds the system prompt from named sections. Each section has an XML tag. The function is buildSystemPromptSections.

FIG 7 The order of the sections in pi's system prompt. Stable content is first. Content that changes is last. Then the transcript follows.

Progressive disclosure with skills

A skill is a folder with a SKILL.md file. The system prompt holds only the name, the description, and the path of each skill. The full instructions stay on disk. When a task needs a skill, the model reads the file with the read tool.

This costs few tokens for many skills. The model pays for the full text only when it uses the skill. Thus the description is the most important line in a skill. The model uses it to decide.

Project instructions

pi looks for AGENTS.md and CLAUDE.md files from the working directory up to the root. It puts their content in the <project_context> section. Use these files for build commands, test commands, and project rules. The model then does not have to find them again on each run.

Changes during a run

The tool set can change during a run. pi does not rewrite the first system message. It adds a new system message with toolsAdded and toolsRemoved (declareToolChanges in agent-loop.ts). The prefix of the context stays the same. The provider can reuse its cache for that prefix.

L6Layer 6 · pi-coding-agent

Context lifetime

Each turn adds tokens. Tool results are often large. A long task fills the context window. Before that occurs, the harness must remove tokens. pi does this with compaction.

The rule is in compaction.ts:

export function shouldCompact(contextTokens, contextWindow, settings) {
  return contextTokens > contextWindow - settings.reserveTokens   // reserveTokens: 16384
}
// findCutPoint() walks back from the newest message and keeps about
// keepRecentTokens (20000). An LLM call summarizes all older messages.

coding-agent/src/core/compaction/compaction.ts

Context window: 200,000 tokens

system prompt + project contextcompaction summarytranscriptwindow − reserve

Turn 0

FIG 8 Simulated growth over 60 turns. Turn sizes are illustrative. The threshold and the kept-tokens values are the pi defaults. Each compaction replaces old messages with one summary. Recent work stays verbatim.

CautionA summary loses detail. After compaction, the model can forget a file it read or a decision it made. Write important state into files (a plan, a task list, notes). Files survive compaction. The model can read them again.

Sessions are a tree

pi stores each session as a JSONL file. Each entry has an id and a parentId. Thus the entries form a tree. The active branch is the path from the root to the current entry. Only the active branch goes into the context.

Compaction adds a summary entry. It does not delete the old entries. You can go back to an earlier entry and start a new branch from it.

FIG 9 A session tree. The highlighted path is the active branch. The model sees the summary in place of the entries before it. The other branch stays in the file.

L7Layer 7 · pi-coding-agent extension

Delegation and parallel agents

One agent has one context window. A large task needs more reading than one window can hold. The solution uses the parts that you already know.

A subagent is a tool. Its execute function starts a new agent loop with an empty context. The new loop runs to its end. The tool returns only the final text of that loop. The parent agent sees one tool result. It does not see the many turns that made it.

pi does not put subagents in its core. It shows them as an extension in coding-agent/examples/extensions/subagent/. This tells you something important: delegation is not a special mechanism. It is a tool like bash.

FIG 10 One subagent tool call in parallel mode. Each worker is a separate pi process with its own context window. Each worker can use 50,000 tokens or more. The parent receives one short answer from each.

How the example extension works

Agent types are Markdown files in ~/.pi/agent/agents/ or .pi/agents/. The frontmatter sets name, description, tools, and model. The body is the system prompt.
For each task, the tool starts a new process: pi --mode json -p --no-session --model … --tools …. The process writes its agent events as JSON lines. The parent reads them to show progress.
The tool has three modes. Single runs one task. Parallel runs up to 8 tasks, with up to 4 at the same time. Chain runs tasks in sequence. Each step gets the output of the previous step in its {previous} placeholder.
A prompt template sets a workflow. implement.md runs scout → planner → worker as a chain.

---
name: scout
description: Fast codebase recon that returns compressed context for handoff to other agents
tools: read, grep, find, ls, bash
model: claude-haiku-4-5
---
You are a scout. Quickly investigate a codebase and return structured findings
that another agent can use without re-reading everything.
...

coding-agent/examples/extensions/subagent/agents/scout.md

The pool that limits parallel work is ten lines. Each worker takes the next free index until no tasks remain:

async function mapWithConcurrencyLimit(items, concurrency, fn) {
  const limit = Math.max(1, Math.min(concurrency, items.length))
  const results = new Array(items.length)
  let nextIndex = 0
  const workers = new Array(limit).fill(null).map(async () => {
    while (true) {
      const current = nextIndex++
      if (current >= items.length) return
      results[current] = await fn(items[current], current)
    }
  })
  await Promise.all(workers)
  return results
}

coding-agent/examples/extensions/subagent/index.ts · MAX_PARALLEL_TASKS = 8, MAX_CONCURRENCY = 4

FIG 11 Six tasks on four slots. Task durations are illustrative. A slot takes the next task when its current task ends. The parent waits for the last task.

Why delegation gets more work done

Context isolation. Each worker starts with a clean window. It does not carry noise from other tasks.
Compression. A scout reads 40,000 tokens and returns 1,500. The parent keeps space for decisions and integration.
Parallel time. Independent tasks run at the same time. Total time is near the longest task, not the sum.
Specialization. Each agent type can use a smaller model and fewer tools. A read-only scout cannot change files.
Composition. Chains connect agents with different jobs. The output of one is the input of the next.

A second design: durable child conversations

pi also has an experimental subagent in experimental/durable/subagent.ts. It makes a child conversation in the same store, not a new process. Two details are important:

The child removes the subagent extension from its own configuration. Thus a child cannot start more children. This stops uncontrolled recursion.
The tool is marked replay: "safe". After a crash, a new attempt finds the child that already exists. It does not start a second child.

CautionThe file mutation queue (L4) is in one process. It does not protect two subagent processes that edit the same file. Give each parallel writer different files, or give each writer its own git worktree. Use parallel mode for read-only work. Use chain mode when steps write to the same code.

NoteThe subagent does not see the parent conversation. The tool description says this: "Give it everything it needs to know". The quality of delegation depends on the task text that the parent writes. The parent model writes a prompt for another model.

→Comparison

Map to Claude Code

Claude Code uses the same layers. Its source is not open, so this table compares documented behavior only. Use pi to study the mechanism. Then use this table to find the same concept in Claude Code.

Layer	pi	Claude Code
L3 Loop	`runLoop` in pi-agent	The same loop. The Claude Agent SDK exposes it for your own agents.
L4 Tools	read, bash, edit, write (+ grep, find, ls)	Read, Bash, Edit, Write, Grep, Glob, and web tools
L4 Hooks	`beforeToolCall` / `afterToolCall`, extension events	PreToolUse / PostToolUse hooks and permission modes
L5 Project context	AGENTS.md, CLAUDE.md	CLAUDE.md files
L5 Skills	SKILL.md, description in prompt, body on demand	The same skill format and the same progressive disclosure
L6 Compaction	Automatic at window − 16,384; manual command	Automatic compaction and `/compact`
L7 Subagents	Extension: Markdown agent files, single / parallel / chain	Built-in agent tool with agent files in `.claude/agents/` (name, description, tools, model)
Working state	`todo.ts` example extension	Built-in task list tool

→Procedure

Study path in the code

Read the files in this sequence. Each file uses the files before it. Links go to the main branch.

Read coding-agent/docs/how-pi-works.md. It gives the full system in one page.
Read ai/src/types.ts. Find Message, StopReason, and the stream event types. This is L1 and L2.
Read one provider: ai/src/api/anthropic-messages.ts. See how wire events become pi events.
Read agent/src/agent-loop.ts completely. Start at runLoop. Then read executeToolCallsParallel. This is L3.
Read coding-agent/src/core/tools/: edit.ts, bash.ts, truncate.ts, file-mutation-queue.ts. This is L4.
Read coding-agent/src/core/system-prompt.ts and skills.ts. This is L5.
Read core/compaction/compaction.ts and docs/session-format.md. This is L6.
Read examples/extensions/subagent/, then src/experimental/durable/subagent.ts. This is L7.
Do a test. Install the subagent extension (its README has the commands). Run /implement on a small change. Watch the scout, planner, and worker events.