Picture the most capable intern you have ever hired. Sharp, fast, tireless. Already better with the spreadsheet than people twice their tenure, and visibly hungry to prove it. On their first morning, do you hand them the company checkbook — signing authority with no ceiling and no second look — and tell them to go commit the firm wherever they spot an opportunity?
Of course not. And the reason you don't is worth slowing down on, because it is the whole argument.
You don't withhold the checkbook because the intern is foolish, or slow, or short on talent. You withhold it because being good at the work and being allowed to act for the company are two entirely different things. The first is a matter of capability. The second is a matter of authority — and you grant authority deliberately, in bounded amounts, to someone whose name you can attach to whatever gets committed. You widen it later, as trust is earned. None of that is an insult to how good the intern is. It is simply what it means to let a person act on the company's behalf.
We understand this instinctively with people. We have somehow forgotten it with software.
The question everyone is asking — and the one they're skipping
Walk into any AI demo and listen to the question in the room. It is always the same question: can it do the task? Can it write the email, price the deal, draft the contract, approve the claim, move the money? And the answer, more and more often, is a genuinely impressive yes. The capability is real. That is not the part to be skeptical about anymore.
But capability is not the question that should keep a CFO awake. The one that should is narrower, and almost no one asks it out loud: what is this system allowed to do on its own — where does that permission stop — and whose name is on it when it is wrong?
Those are not technical questions. They are the same questions you would put to any new hire before you let them sign for anything: how much can you commit without checking with me, which decisions are yours to make and which come back to me, and who answers if it goes sideways. We ask them as a reflex about people. We skip them entirely about a system that can commit the company faster, and at greater scale, than any intern ever could.
Zillow learned the difference the expensive way
This is not a hypothetical risk. It has already produced one of the cleaner cautionary tales in recent corporate history, and the lesson in it is almost always told wrong.
Zillow built an algorithmic home-buying business. The model did exactly what it was built to do: price houses and commit real money to buy them, at scale, across the country. It was capable. Whatever you think of the bet, the system was not sitting there fumbling arithmetic — it was doing the job it was given, quickly and at volume.
What it lacked was not intelligence. It was a bounded authority to act. The model kept committing the company to purchases as the market turned underneath it, and because the permission to commit was effectively open-ended, the commitments kept landing. The bill arrived all at once: a write-down north of three hundred million dollars, the entire home-buying unit wound down, and about a quarter of the workforce let go.
Notice what no serious person called that. No one called it a story about a model that wasn't smart enough. It was a story about how much a capable system was allowed to commit with no one standing in the room to say not that much, not there, not now. The failure was not in the thinking. It was in the size of the grant.
Capability answers "can it." Authority answers "may it."
Here is the line worth drawing in permanent ink, because nearly every AI governance mistake comes from smudging it.
Capability answers a question of fact: can it do the thing. Authority answers a question of permission: may it — and on whose signature. They are different decisions, made by different people, for different reasons. Capability is something you assess, or increasingly just purchase. Authority is something you grant. And the danger lives precisely in the moment you let the first quietly stand in for the second — when "it's good enough to do this" slides, without anyone deciding it should, into "so we'll let it do this on its own."
A capable system handed unbounded authority is not a productivity gain waiting to compound. It is the brilliant intern, day one, alone with the checkbook. The competence is exactly what makes it dangerous, because competence is what tempts you to stop watching.
"The AI decided" is not a sentence anyone will accept
And a system with real authority will, eventually, get something wrong — because any actor that commits the company to anything real will eventually commit it to something wrong. That is not a knock on AI. It is true of every capable hire you have ever made. The difference is what happens next.
When it goes wrong, "the AI decided" is not a sentence a board will accept, or a regulator, or a customer who was just told something binding that the company never meant to promise. Authority always lands on a person in the end. So the questions have to be answered before the commitment, not discovered after it, and they are refreshingly concrete:
What can this system commit without a human approving it first? Which customers is it allowed to answer for, and which are off-limits? Which contracts, which dollar amounts, which decisions are inside its grant — and which are explicitly outside it, no matter how confident it sounds in the moment? And when it is wrong, whose name sits on the decision: the function leader, the CFO, the business owner — or no one in particular?
No one in particular is the dangerous answer. It is the exact mechanism by which accountability disappears into the machine. If you cannot name who authorized the system to act on its own, and say plainly where you drew the line, you have not delegated authority. You have misplaced it — and kept every last consequence. Delegation with no name attached is just an intern you forgot you handed the checkbook to.
Make the grant of authority a decision again
The move here is not to trust AI less, and it is certainly not to go shopping for a less capable system. That would be solving the wrong problem — throwing away the part that works to avoid doing the part that takes judgment.
The move is to make the grant of authority a deliberate decision again, the way it always was with people. Before a system acts on the company's behalf, somebody decides — on purpose, on the record — what it may commit without a human, where that authority stops cold, and whose name sits behind it. You start that grant narrow and widen it as the system earns trust, exactly as you would with a promising hire, rather than handing over everything on day one and trying to claw it back after the first expensive surprise.
Capable is the easy part now. You can buy capable. Bounded and accountable is the part that takes a leader's judgment, and it is the part that keeps the checkbook from walking out the door. Because trust, in the end, was never a feeling about competence. Trust is a grant of authority — with a boundary, and a name attached.
The question to take into the room
So before you ask whether your AI is good enough to do the job — and it very likely is — ask the question you would put to any new hire before you let them sign for the company.
What, exactly, is it allowed to do without you? And when it does, whose name is on the line?