The question arrives in the same shape every time: how do we make sure the AI never does something it was never supposed to do?

Start with the honest answer, because the useful one sounds like a dodge until you have heard it. You cannot. No prompt, no model, no vendor and no configuration file will make a capable system do only what you meant.

What you can build is smaller and worth more: a system that cannot do the catastrophic thing, whose wrong actions you can see, and whose mistakes you can undo. Bounded, observable, recoverable. That is engineering rather than trust.

My vantage point, stated plainly: I run an assistant with real permissions in my own lab. It holds credentials, edits configuration, and builds machines on my behalf. In one afternoon it made 12 mistakes, and I logged every one — that log was the only useful artifact the day produced. The failures below are first-hand. The frameworks I point at are read, not run.

Three properties, and the leg worth breaking

Every autonomous action you permit needs three properties.

It should be impossible to do the worst thing. Notice the word: impossible, not forbidden. A rule the model is asked to respect is a promise. A credential that cannot delete is a capability. When your only control is the model’s good intentions, you are managing hope.

It should be visible. Intent, action and result, recorded somewhere the agent does not own, readable by someone who is not the agent.

It should be reversible. A snapshot, a backup, a staging copy, a limit on how much damage one run can do. The worst realistic mistake should cost minutes.

And then there is the combination that turns a helpful tool into an attack surface. Simon Willison named it the lethal trifecta: private data, exposure to untrusted content, and any channel for sending things outward. Prompt injection is not a bug you patch out of a model — it is a property of putting instructions and data in the same pipe. The moment data can be read as instructions, one of those three legs has to go. OWASP’s list of LLM application risks leads with prompt injection in its current edition, and not out of fashion.

My AI Mantra:The ten lines

Here is the short version, the part worth committing to memory, or taping inside a cupboard door:

Least power.
No secrets in its head.
Data is not orders.
Dry run, then do.
Two keys for irreversible.
Verify the effect, not the call.
Small blast radius.
Log the intent, not just the act.
Rehearse the restore.
When in doubt, don't.

Easy to recite, hard to mean.

Least power. Tokens scoped to one job, read-only by default, expiring, revocable without touching the agent. A key that can only read one zone cannot delete the zone. Splitting a broad credential is dull work, and it is the difference between an incident and an anecdote.

No secrets in its head. Twice that afternoon a credential ended up somewhere it had no business being — once written to the wrong machine by a file transfer that looked successful, once staged on a host and left behind. A component that reads untrusted web pages must not reach your vault; it should ask for an action, never for the key.

Data is not orders. Everything the agent reads that you did not write — pages, emails, documents, command output — is data. That distinction is load-bearing, and it fails silently the first time a page tells your agent to act and it does.

Dry run, then do. Destructive or public actions get a rehearsal first: a plan, a diff, a what-if, a staging copy. The agent proposes; the person disposes. It costs a minute.

Two keys for irreversible. Publishing, paying, deleting, emailing strangers, rewriting firewall or backup configuration. Not agent-alone, ever. My quietest example: an automated edit narrowed a backup schedule from 14 machines to 1. Nothing failed, nothing alerted, and it would have stayed green indefinitely.

Verify the effect, not the call. A 200 response, a clean exit code, a cheerful “done” prove only that something was attempted. That afternoon a firewall rule set was written to the correct file, read back correctly, and blocked nothing at all — for that kind of machine, one flag also belonged on its network card, and without it the rules compiled to nothing. Every probe came back open while the configuration looked perfect. Read the state back: the setting, the row, the file.

Small blast radius. One job per machine. Untrusted work goes in its own disposable guest, never in the box that serves traffic or holds credentials. That afternoon a machine booted to a black screen because the “cloud” variant of its OS ships no display drivers — right category, wrong flavour, 20 minutes to find.

Log the intent, not just the act. “Deleted 340 files” is useless. “Deleted 340 files because the retention rule said older than a year” is auditable — a person can find the wrong premise without replaying the machine’s mind.

Rehearse the restore. An untested backup is a hope, not a control. Decompress it. Revert the snapshot. Break a sandbox deliberately and put it back. And wait for evidence, not for an expectation: one of my scripts declared a healthy machine dead because it had 3 minutes of patience while the machine needed 6.

When in doubt, don’t. Default deny. Asking costs a minute; the asymmetric mistake costs everything. And test the guardrail itself: one maintenance command killed the check it was running, because the pattern it searched for appeared in its own command line.

The case against all of this

The discipline is not free, and pretending otherwise is how it gets abandoned. It is slow: every one of those ten lines adds friction to a workflow whose selling point is the absence of friction. That is the trade — survivable failure bought with convenience.

And it does not fix the part people actually worry about. You cannot audit the reasoning — you get outputs, the deliberation is opaque and may differ next run. A model trained to please will agree with your bad premise and confirm a confident wrong plan, so ask it to argue against its own proposal; it will not volunteer to.

The deepest problem is structural: if the agent administers the machine, the guardrails are editable by the thing they guard. Keep one layer a person owns outright — the approval channel, the log destination, the backup credentials. Permissions also drift upward, because nobody removes a grant, so review them on a calendar rather than on suspicion.

None of this is solved. Anyone describing alignment as a settled engineering problem is selling something, and the commercial pressure runs the other way: autonomy demos well, guardrails do not, and the convenient default wins.

Where this is likely heading

My judgement, not a forecast.

Approvals become the interface. Accept-or-reject diffs are already how agents touch documents and code, and they will spread, because a reviewable diff is the only shape that survives a professional’s liability.

Credentials get scoped as a product. Agent identity — per-task, expiring, auditable, revocable — becomes its own category of infrastructure, sold by the same vendors currently handing out broad tokens because it makes their demos smoother.

Liability does more for guardrails than any ethics paper. The moment a company can be held to what its agent did at 3 a.m., auditing stops being a nice-to-have.

And the place you can implement all of this today is the small, local, self-hosted tier — the one you own, the one you can unplug, where “least power” is your decision rather than a vendor’s tier.

What you can do now

Before you hand anything a key, ask seven questions.

• What is the worst thing this system could do with exactly the powers I am granting? If I could not survive that twice tomorrow, remove capability until I could.
• Which legs of the trifecta does it have? Kill one.
• Are the credentials scoped, expiring, and revocable without touching the agent?
• Is there an out-of-band approval for irreversible actions — and can the agent grant itself that approval?
• Can I see the last 10 things it did, in plain language, with the reasons?
• Have I restored from a backup in front of me, from this system, this month?
• Do I know how to switch it off, and have I tried?

And the one-line test, the only one that matters when you are tired and the demo is impressive:

If this system did the worst thing it is capable of, tomorrow, twice, would I still have my data, my money and my accounts?

If the answer is not an immediate yes, the fix is not a better prompt. It is less permission.