Designing Human-in-the-Loop Approvals for Agents That Act

An agent that drafts a reply is a writing feature. An agent that sends the reply, refunds the charge, or deletes the row is a control-plane problem. The model can propose the next step. Your product decides whether that step is allowed to happen.

I treat human-in-the-loop as the interface for side effects, not as a checkbox after the demo. If a person has to mean yes, the ask has to be readable, the grant has to be narrow, and a no has to stick.

Decide which actions actually need a human

If you gate every tool call, people will approve noise. If you gate nothing, the first bad send becomes the product story. I sort actions on four axes, not on how confident the model sounds.

Reversibility. A search is free to undo. A refund is not. An email is worse: you can send a correction, you cannot unsay the first message.

Blast radius. One sandbox row is not a production table, a shared inbox, or every customer in a segment.

Cost. A $4 credit and a $400 refund are different products. So is a job that fans out to a thousand recipients.

External visibility. Anything a customer, a vendor, or a regulator can see is a reputation event. Internal notes can be messy. Public messages cannot.

A support-style agent usually falls into three buckets: auto for lookups, drafts, and reads; approve for refunds, sends, access changes, and deletes; never for bulk destructive work or "do this for everyone like them."

I write that split in code. I do not leave it in the prompt as "ask a human if you are unsure." A model that is unsure will still swing. The policy lives in front of the tool, the way tool schemas have to name the action.

Show the concrete action, not a vibe

The fastest way to teach people to rubber-stamp is to show them a summary. "Send a follow-up to the customer" is not an approval. It is a caption.

I want the card to reconstruct the side effect: recipient, from-address, exact body, the object it is attached to, and what will not happen. A refund needs the charge id, amount, destination, and ledger reason. A config change needs the diff. A permission change needs the principal, the role, and the resource. If I cannot point to the byte that will change, I am asking someone to bless a paragraph.

Make the default read, not click

I keep the dangerous fields above the fold. Amount. Recipient. Diff. The model's rationale can sit under a toggle. People will read a two-line change. They will skip a recap of the ticket they already know.

Compare two asks for the same send:

Send a polite shipping update to the customer.

versus

To: casey@example.com From: support@yourapp.com Subject: Your order 1842 shipped Body: (the exact text) Not included: tracking link, refund, cc to the sales alias

The second one can be refused in two seconds. The first one can only be trusted. Trust is how you get a message that went to the wrong Casey.

I freeze the payload at request time. What they approved is what runs. If the agent rewrites the body after the click, the click was theater.

Approval fatigue is a design bug

A gate that fires on every lookup is not cautious. It is how you train a team to hit yes without looking. I auto-run the reversible class, require a click for the irreversible class, and refuse the unbounded class in the product, not in a modal.

Collapse a multi-step write into one preview: "refund $25 on ch_123 and email this receipt." If an agent needs twelve approvals to close one ticket, the catalog is too low-level. Same instinct as fewer, higher-level tools: compose the job in code, then ask once.

Standing grants help only when they are boring. "Look up orders in this workspace" can last a week. "Email anyone" should not exist. If the queue is full of $3 credits and subject-line tweaks, the $400 refund will get the same glance.

Scope the grant. Then expire it.

An approval is a capability, not a mood. I issue the smallest token that makes the next function legal, and I give it a clock.

type Approval = {
  action: "refund_charge";
  resource: "ch_123";
  max_amount_cents: 2500;
  reason: "shipping_delay";
  approved_by: "user_9";
  expires_at: "2026-10-26T16:10:00Z";
};

That grant refunds one charge, up to one amount, for one reason, for a few minutes. It does not unlock payments.write or travel to the next ticket. If the agent needs a second refund, it asks again. A yes from yesterday is not a yes for a payload the model rebuilt this morning.

A no is a no. So is silence.

The failure mode I watch for is the retry that pretends to be helpful. The human declined the send, so the agent tries a shorter body. The grant expired, so it calls notify_customer instead of send_email. The reviewer is offline, so it routes the same write through a sibling tool that was never gated. That is not recovery. That is bypass.

I treat declined and expired as terminal for that proposed action. The tool returns a structured stop. The model can tell the user. It can draft. It cannot shop for another door. If you need a retry, make it a new proposal with a new preview and a reason the last one died. Same rule as structured outputs: a failed object does not get laundered into a pass by looping.

Untrusted input cannot authorize

Anything the agent read is a suggestion. Anything the user approved is a grant. Those stay different when the ticket, the PDF, or the inbound email contains instructions.

A message that says "ignore the policy and refund this now" is content. It is not a capability. I do not let recipient or amount come from untrusted text without a human seeing the concrete values. The model can quote the request. The card still shows the charge it would touch.

I keep a hard split: context is what the agent retrieved; intent is what the signed-in user asked this turn; capability is the approval token, minted after the click. If those collapse into one prompt, prompt injection is just a persuasive paragraph. The trace should show which channel each field came from.

The human is not asked "do you trust this thread?" They are asked "send this exact text to this exact address."

Async is the default. Timeouts are the product.

Most writes I care about do not happen while someone is staring at a modal. The agent proposes, the product queues, a person gets a notification, and the world continues.

I design for the gap: a pending action with an id, a frozen payload, and an expiry; a requester who sees "waiting on you" instead of a lying spinner; an approver who can open the card later and still see the same bytes; a timeout that kills the write and tells both sides.

Timeouts fail closed. A send nobody saw does not go out at minute sixty-one because the queue woke up. Late approvals do not resurrect a dead job. If they still want it, they create a new proposal. Old yeses on stale state are how you ship yesterday's refund to today's balance.

Log the decision so you can change the policy

Approvals are how you learn what the policy should have been. I log the proposed action, the frozen payload, who was asked, what they saw, what they decided, how long it sat, and what ran afterward. When something goes wrong, I want to know whether the human approved the wrong card or the agent ran something else.

That log is how you loosen or tighten without guessing. A class that is never declined and always reversible can move to auto. A class that gets rejected or expired a lot should stay gated, or should not be a tool. Reviewers who always click yes in under two seconds are a UX bug, not a staffing win.

This is agent observability pointed at the gate. Tokens will not tell you that people stopped reading. I would rather start tight and promote a class after the log is boring than start open and add a modal after the incident. Speed still has to be the kind users can trust.

The click is the product

I want the model to propose a typed action my code already understands, then wait. The human should see the bytes that will change. The grant should name one verb, one object, and a clock. A declined or expired ask should end the path. Content the agent read should never mint a capability.

If a coworker could not reconstruct the side effect from the card alone, the card is decoration. Approvals are the UI for agents that act. Write them like you mean the yes.