Design an Agent Tool Router

An agent is a loop that picks a tool and calls it. This is the thing on the other end of that call. Nothing here is an algorithm. Every hard part is a safety rule that has to survive the next requirement.

The recipe

Say before you type: "A Map of names to handlers, a Map of request ids to pending work, and an append-only array. Dispatch never throws, approval is spent once, and the log is never rewritten."

Three structures carry the whole problem:

private tools = new Map<string, { handler: ToolHandler; requiresApproval: boolean }>();
private pending = new Map<string, { name: string; args: unknown }>();
private audit: AuditEntry[] = [];

Name those in the first two minutes and phase three is bookkeeping. Reach for one object and improvise, and phase two is a rewrite.

Dispatch returns failure, it does not throw

The caller is a model. It cannot catch. Every path out of call is a value:

{ ok: true, value }
{ ok: false, error: "unknown tool: search" }
{ ok: false, pending: true, requestId: "req-1" }
{ ok: false, denied: true }

A handler that throws becomes { ok: false, error } and the router keeps serving. That is the whole reason this layer exists. One flaky integration should degrade one tool call, not the agent.

Use a Map, and say why

Tool names come from config, and config contains whatever someone typed. toString is a legal tool name, and on a plain object it is already there:

const tools = {};
tools["toString"];  // a function nobody registered

A Map has no prototype chain to fall through. Object.create(null) or Object.hasOwn also work. Picking one deliberately and saying the word "prototype" out loud is a free signal, and it is one of the two or three things a reviewer actually remembers.

Two rules about the error message

thrown instanceof Error ? thrown.message : String(thrown);

Both halves are load-bearing. String(new Error("boom")) is "Error: boom", so a blanket String() corrupts the common case. thrown.message on a thrown string is undefined, so a blanket .message erases the rare one. Handlers throw strings. You will meet both.

And a handler that returns 0, false, or null succeeded. Branch on whether the tool existed, never on what it gave back.

Approval is per action, not per session

This is the phase that separates an agent people deploy from a demo. A gated tool does not run:

if (entry.requiresApproval) {
  const requestId = "req-" + this.nextId++;
  this.pending.set(requestId, { name, args });
  return { ok: false, pending: true, requestId };
}

Three decisions hide in those four lines.

Key on the request id, not the tool name. Two agents can queue two deletes on the same tool before a human looks at either. A Map keyed by name holds one of them, and approving the first request runs the second one's arguments. The blast radius of that bug is the exact set of actions you gated because they were dangerous.

A fresh id per call, even for identical calls. If the id is name + JSON.stringify(args), two identical charges collapse into one authorization. One of them is silently dropped, or worse, one approval covers both. An id is a nonce, not a hash.

Bind the id to the name and the args at issue time. approve(requestId) takes nothing else, so there is no surface to point an approval at a different call. That is the design, not a coincidence. Say it.

Spend the request before you run it

this.pending.delete(requestId);   // first
return this.invoke(entry.handler, request.args);

Reversed, or tucked inside the try after the call, and a handler that throws leaves the approval unspent. The failure mode is not theoretical: the delete succeeded, the response timed out, the handler threw, and now that approval is live again. Retrying a spent approval must be refused, not re-run:

approve("req-1")  // { ok: true, value: ... }
approve("req-1")  // { ok: false, error: "unknown request: req-1" }

Deny is the same shape with the opposite answer, and denying an id that was never issued is unknown request, not denied. A deny that reports success on garbage input is an audit log that lies.

The log is append-only

auditLog() records attempts, not successes. Unknown tools, thrown handlers, denials, all of it. The log is what someone reads at 2am after the agent did something expensive.

A gated call appears twice, once as pending and once as approved or denied:

wire  pending   1699999400000
wire  approved  1699999520000

The obvious optimization is to find the pending row and rewrite its status. Do not. The gap between those two timestamps is the only record of how long a human sat on the decision, and it is the first number anyone asks for. Rewriting in place destroys the question you were logging to answer.

Two smaller ones in the same spirit. An approved call whose handler throws logs error, not approved, because the audit says what happened and not what was authorized. And approve on a spent id resolves nothing, so it logs nothing. A replay is not an attempt on the tool.

Hand back a copy. A caller that pushes onto the array you returned has edited your evidence.

Determinism is a design choice

The optional trailing nowMs is not a test convenience you bolted on. Passing the clock in is what makes the audit reproducible in a test, and it is the same move as the sliding window in the rate limiter drill. Take the clock as an argument, default it to Date.now(), and never call the wall clock from inside a branch.

What they will ask after

The follow-ups are the interview:

• Where does pending live when the router is three processes behind a load balancer?
• Does an approval expire? What is the timeout, and is an expired request denied or something else?
• Who is allowed to approve, and how does the log record which human it was?
• A handler is idempotent. Does that change how hard you work to spend the request exactly once?
• The audit array grows forever. What ships to durable storage, and what does the retention policy have to be for this to be worth anything in an incident?

Have an answer for the last one. An audit log with no retention policy is a memory leak wearing a compliance costume, and noticing that unprompted is the difference between a pass and a strong pass.