A hands-on look at agent safety
AI agents are starting to do real things — book flights, send emails, move money. This lab is a small, honest demonstration of the difference between an agent that's merely told to be careful, and one that's actually stopped from being careless.
In one sentence: it's the code that runs outside an AI model's instructions, deciding what the model is actually allowed to do. Not a stricter prompt — a separate layer the model can't see, argue with, or forget.
Every major AI lab is shipping agents that use real tools — browsers, code, calendars, payments. The open question isn't whether these agents make mistakes; small models especially do, constantly. It's whether anything outside the model actually catches those mistakes before they become real. That's what a harness is for, and it's the design problem behind most of today's agent-safety work.
Guardrail teaching sandbox
Sends money to a saved beneficiary from a single savings account. Phase 1 has no checks. Phase 2 does.
Step 3 of 3 · the full story
An AI model is good at exactly one thing: deciding what to do next, and saying so with total confidence. It doesn't get tired, doesn't second-guess itself, and has no way of knowing when it's wrong — left alone, whatever it decides simply happens.
An agent harness is the answer to that: a separate piece of code, sitting quietly outside the conversation, that every action has to pass through before it touches anything real. It checks, it remembers, it knows when to stop for a person — and the model is never even told it's there. Think of it less as advice, more as plumbing: the pipe a decision travels through before it's allowed to become a consequence.
One version lives in the prompt, as English. The other lives in a separate file, as code.
SYSTEM_PROMPT = """You are a helpful
banking assistant...
RULES YOU MUST FOLLOW:
- Never transfer more than ₹25,000
at once
- If a beneficiary name is ambiguous,
ask which one before transferring
- Always ask the user to confirm
before moving any money
"""if amount > MAX_SINGLE_TRANSFER:
return blocked("over the limit")
if not enough_balance(amount):
return blocked("insufficient balance")
if ambiguous(beneficiary) and not confirmed:
return blocked("confirm identity")This repo only ever uses the version on the right — shared/prompt.py contains none of the rule text on the left.
| You ask for… | Phase 1 · no harness | Phase 2 · harness active |
|---|---|---|
| “Transfer ₹500 to Amit” | Guesses one of 2 Amits, sends it, never mentions the guess | Flags the name as ambiguous, waits for you to pick |
| “Transfer ₹40,000 to Priya Menon” | Sends it anyway — no transfer limit exists | Blocked — over the ₹25,000 single-transfer limit |
| “Transfer ₹20,000 to Amit Shah” | Sends it anyway — your balance goes negative | Blocked — insufficient balance |