camelRun concepts: durable agents, runs and tools

View as Markdown · Docs index for coding agents

Agents are durable and keyed

An agent is a conversation that lasts. Its history, files, configuration (model, instructions, tools) and pending work live in camelRun, not in your process. Most agents are asleep at any moment and cost only storage. A message wakes one wherever camelRun has room, and it picks up where it left off. For a short tour, see durable AI agents and AI agent memory on camelRun.

You name an agent with a key, your own id for it, such as support-triage, user-123 or thread-9f2c. Keys are 1 to 80 letters, digits, _ and -.

const agent = await agents.upsert("user-123", { model, instructions, tools });

• The same key is the same agent, from any process, until you delete it with agent.delete() (DELETE /v1/agents/{id}). Only your API key can delete an agent. Its own token gets a 401, because deleting purges the agent's history and workspace.
• upsert makes the agent if there is none, and otherwise brings it to the configuration you pass. A changed model, instructions or tool list applies between turns, and the history carries over.
• Who the agent acts for (subject, context), its definition and its mounts are fixed when it is made. Changing them is a 409 that names the field. Delete the agent and upsert again to start over. An upsert may change the mounts with remount: true.
• A deleted key makes a fresh agent the next time, with a new id and token.
• Make an agent once per user, conversation or job, and reuse it. Creating agents is cheap and limited only against abuse (600 a minute; an upsert that changes nothing doesn't count). The limit to design around is busy agents: how many may run at once, 20 on free credit and more with each usage tier. Runs are limited to 240 a minute on free credit, 600 once you buy credit. See Limits.
• Over REST, the key is the Idempotency-Key header of POST /v1/agents.

Every agent also has an id (client_...), which is safe to log and store, and a token, which lets its holder run that agent and nothing else (not delete it, nor change who it acts for, its tools' sources or its limits). You rarely need the token, because upserting by key gives you the agent again. If a token leaks, POST /v1/agents/{id}/credentials/rotate gives the agent a new one. The old one stops working at once, and connections made with it close.

Agents made without a key are scratch agents that expire after a day unless you set ttlSeconds. idleTtlSeconds instead makes an agent live that long from its latest run, so an agent in use is kept and one left alone expires (say, a chat that should go once nobody has written in a week).

Agents can be grouped into workspaces, each with its own spend and usage.

Runs

A message to an agent starts a run: the model loop, with every tool call it makes, until the model answers or something stops it. An agent has one run at a time. A message sent during a run waits for it (whileRunning: "queue", the default) or joins it (whileRunning: "steer", read by the running turn after its current step). A steer is answered at once: agent.steer() returns a receipt, accepted, taken or queued.

const run = await agent.run("Summarize ticket 123", { user: "alice" });

run() has no timeout: runs can take minutes, and a run waiting on a person can wait for days. Pass an AbortSignal (Python: timeout=) to stop waiting; the run itself goes on. agent.abort() stops it: the running turn ends, and the runs queued behind it are cancelled (code cancelled). abort({ queued: "keep" }) stops the running turn only. A failed run throws a RunError carrying the run, unless you pass throwOnError: false.

Runs without an agent

Every run of an agent sees its history; long conversations are compacted automatically. When there is no agent to keep at all (classify, extract, vote, judge), agents.run({ instructions, input, output }) (REST: POST /v1/runs) takes a configuration and an input and returns the result, with no agent made. See Stateless runs.

Long conversations

An agent's history can outgrow its model's context window, so camelRun compacts it: older messages are summarized, and the model sees the summary followed by the recent messages. History keeps every message; only what the model is sent changes.

Compaction runs in the background. Once the context comes within a margin of its limit, the summary is made after the run ends, or between model requests, while the agent goes on. Only a run whose context wouldn't fit waits for a summary. Summaries are billed like model responses (kind: "compaction" in usage) and count against spend limits.

Forking

A fork is a new agent made from an existing one: the same configuration (model, instructions, definition, tools, built-ins, who it acts for, key scope, spend and run limits), a copy of its history, and a copy of its workspace files. From then on each is its own: a turn, a summary or a file written in one never shows in the other. Fork to try another direction without losing the first, to branch a conversation where a user edits an earlier message, or to start many agents from one prepared one.

const branch = await agent.fork({ key: "thread-9f2c-b" });
const earlier = await agent.fork({ atMessage: 4 }); // history through message 4

branch = await agent.fork(key="thread-9f2c-b")
earlier = await agent.fork(at_message=4)

• Where it ends. By default the copy ends with the agent's last finished turn, never inside one that is running or waiting on a person, so forking a busy agent is safe. atMessage ends it earlier: a history index keeps that message and the tool results that answer it, and a run's request id keeps that run's whole turn. A message in a turn that hasn't ended is a 409 FORK_POINT_RUNNING.
• Compaction. The copy keeps the source's summaries, so the fork's model sees what the source's would at that point. History keeps every message.
• Files. The fork's /workspace is a fork of the source's volume as it is when you fork, not as it was at atMessage. Other volumes the source mounts stay shared. Files its history refers to stay available to the fork, whatever becomes of the source.
• Not copied: schedules, channel bindings, pending inputs, runs and their events, and what the source has spent.
• Its own: subject, context, systemPromptAppend and modelHeaders can be given for the fork instead of the source's, so a fork for another conversation names that conversation.
• Keys. A fork made with a key lives until deleted, and forking again with the same key returns the same fork. Without one it's a scratch agent that lives a day, unless you set ttlSeconds.
• forkedFrom: {agentId, atMessage} on the fork (GET /v1/agents/{id}, the console) says where it came from.

A fork is an agent like any other: it counts toward your agents and storage. It also counts against the agent-create rate limit (atMessage: 0 included); to run one message without any history, use a stateless run instead. The copy itself costs only storage, and shares stored content with the source. Over REST it's POST /v1/agents/{id}/fork with {key?, name?, atMessage?, ttlSeconds?, subject?, context?, systemPromptAppend?, modelHeaders?}; in the console, Fork on the agent's page, or Fork from here under a reply. Browser tokens can't fork.

The result is the truth; events are for display

An agent's events stream every step: text as the model writes it, tool calls and their progress, and inputs asked and answered. Use them to show the agent working, not to decide what happened.

• The run's result (run(), the run.completed webhook, or GET /v1/agents/{id}/requests/{requestId}) is recorded durably and always arrives, even if your process restarts or the connection drops.
• The stream replays from memory on reconnect, as far back as it can. Where it can't (camelRun restarted, a long disconnection), the SDK gets a snapshot of the running turn instead.

Reply to your user from the run, and render the stream as progress. See Events.

Where tools run

Pick by where your code runs:

The same tool({...}) definitions work attached and served, so you can start attached and move to served without changing a tool (see Several processes, workers and deploys). Every agent also has file tools over its files, and js_exec, a sandbox where the model writes JavaScript that calls its tools. See Tools.

Processes and connections

A process that serves an agent's tools attaches to it: it holds the agent's event stream and answers its tool calls. Only one process at a time can, so a tool call never lands in the wrong place. A second process that tries gets APPLICATION_CONNECTED, unless it passes takeover: true; the one it replaces hears APPLICATION_REPLACED and goes on without serving the tools.

Any number of processes can run an agent without serving its tools: an agent upserted without local tools, or with attach: false, follows the stream read-only. That's how serverless functions and webhook handlers run agents whose tools are served elsewhere. Such a handle connects lazily: it holds the stream only while agent.stream() reads a run, so a server holding many agents holds no idle connections.

A run of an agent with attached tools while no process serves them is refused with APPLICATION_NOT_CONNECTED, after a few seconds' grace. Pass allowDisconnected: true to run anyway; calls to those tools then fail and show in toolErrors as not_connected. Schedules, channels and resumed runs are never refused, so agents that must work with nobody attached should use served tools.

People in the loop

A run can stop and wait for a person: the model asks a question (ask_user), a tool needs approval before it runs, or a tool asks for a form. The run ends with status: "input_required" and its inputs, the agent sleeps, and answering the last input resumes it.

if (run.status === "input_required") run = await run.inputs[0].answer(true, { from: "alice" });

See Human input.

Idempotency

Everything that changes something can be retried safely:

• upsert by key, and run or POST /v1/agents/{id}/prompt by idempotencyKey (the request id). The same key returns the same agent or run.
• Every other POST takes an Idempotency-Key header. A retry with the same key, path and body gets the first answer again for a day. The same key with a different body is a 409. An answer with a secret shown once (an API token, a signing secret, a browser token, a rotated agent token or a signed link) is never stored: a retry of a key that succeeded is a 409, not the secret again.
• Every tool call carries context.idempotencyKey, the same for every attempt of that call: a retry after a lost connection, or the call run again after a person answered. Key your side effects by it. (context.callId is new each attempt.)

A tool call whose answer was lost (the connection dropped, or its deadline passed) is never sent again. The model is told its outcome is unknown, so it can check before repeating anything.

Waking later

An agent can be woken by a schedule (agent.schedule({ text, inSeconds, at, everySeconds, calendar }), or the schedule built-in so the agent sets its own), by a channel (Slack, Telegram, GitHub, email, webhooks), or by your own code from a webhook. See Schedules for calendar schedules and report emails.

A repeating schedule's wake-ups are turns of the agent, so each one reads the history before it, compacted as it grows. That lets a daily agent build on what it did before.

A wake-up camelRun can't deliver (the agent is busy at its account's limit, say) is tried again 1, 5 and 15 minutes later, then given up. A refusal a retry can't change, like a spend limit (402), is given up at once. A repeating schedule then moves to its next occurrence and lists the failure (failures and lastError) until a delivery succeeds. A one-off schedule given up is removed.

Definitions

A definition is a reusable configuration: model, instructions, tool sources (MCP servers, OpenAPI specs, built-ins), limits and mounts. Make agents from one with definition: "def_..." on upsert, and roll a change out to all of them with apply: "all". See Definitions.

Several agents

Agents can work together. With the delegate built-in, an agent hands a task to a sub-agent (an agent of its own, made from a definition for the task, or an existing one) and gets its answer back as the call's result; several at once run in parallel. A sub-agent counts against its parent's limits. There are no handoffs: the conversation stays with the agent that has it. See Multi-agent.