Thanks for joining me at ProductBC.
The Trust Continuum checklist for AI product teams
Pick one AI feature your product team owns. The checklist below shows where each of its actions sits today, and the evidence it needs before it climbs a rung.
Each rung up trades control for convenience. At every step, ask: what evidence justifies the trade?
1. Place each action on a rung
The rung belongs to the action, not the whole agent. The same assistant can draft at one rung and send at another.
0. Answer
The AI… knows only what you paste in, one conversation at a time
The human… curates every piece of context by hand
1. Assist
The AI… has standing access to your context. It drafts, finds and summarizes
The human… decides and does everything
2. Recommend
The AI… proposes specific actions
The human… approves each one. The human is in the loop
3. Act and report
The AI… acts on its own
The human… watches, and can stop or undo. The human is on the loop
4. Act Continuously within Guardrails
The AI… acts alone inside boundaries set ahead of time, and escalates the rest
The human… sets the rules up front and handles the exceptions
There's no "fully autonomous" rung. Almost nothing in business belongs there.
Rung 0 is a choice, not a failure. Some work, like legal, HR or M&A, may rightly stay at Answer. And it works in reverse: if a gate stops being true, the action drops a rung.
2. Name the gate it has to pass next
0 → 1 · Granting what it can see
☐ Accurate, complete and current. We've checked the data it will see, and we know what's missing or out of date.
☐ Permissible. We're allowed to share it with this system and its vendor, and it only shows each person what they're already allowed to see.
☐ Trusted sources. We know where each input comes from, and outside content (emails, web pages, shared docs) can't give it instructions.
1 → 2 · Granting its judgment
☐ The people approving its recommendations can actually judge them.
☐ The approvers see the source material side by side with the output.
2 → 3 · Removing approval on every action
☐ A human can notice a problem.
☐ They'll notice it in time.
☐ They have a way to stop it or undo it.
3 → 4 · Removing real-time watching
☐ The boundaries are written down.
☐ We've watched it operate and stop at those boundaries over time, under a wide enough range of conditions.
☐ Limits and guardrails. If one input is wrong, the damage stays small: it's cross-checked, or capped (like a refund limit).
3. Check what every gate needs
☐ A named owner. One person is accountable for this system at this rung.
☐ One-way or two-way door? A two-way door can be undone, like deleting a draft. A one-way door can't, like hitting send. An action you can't undo stays at Recommend until it has passed every gate below it.
☐ A threshold set in advance. We agreed what "good enough to climb" means before we looked at the results.
☐ Monitoring that stays. We still sample and review after we step back, so we can catch drift. At Alexa, we kept reviewing 10% of output after we moved to on the loop.
Why a passing eval isn't enough
An eval tells you how the system did on the cases your team chose. It can't tell you about the cases nobody thought of, or whether the people doing the grading could judge what they graded.
The question to take back to your team: what evidence would earn your AI its next rung?
Get the slides and sources
Every story in the talk comes from a public report, from the BEA's investigation of Air France 447 to the FAA's review of the 737 MAX. I'll send you the full list now, along with today's slides.