Pilot or passenger

There is a moment I have learned to watch for in a training room. Someone asks the tool a question, reads the answer, and nods. Then they ask a second question, read the answer, and nod again. By the fourth or fifth exchange they have stopped reading closely. They are scrolling to the end, copying the last paragraph, and moving on.

Nothing has gone wrong yet. The answers are probably fine. But something has changed in the room, and it is the thing I care about most. The person has stopped being the pilot.

I use that word deliberately. A pilot has a destination and a set of instruments, and the instruments are extraordinary, but the pilot decides. A passenger has a seat. The aircraft goes where it goes, and the passenger finds out where that was on landing.

AI augments judgement. That is the whole of its value in serious work, and it is why I spend so much of my time on it. It does not replace judgement, and the difference is not a matter of degree. The moment a person stops exercising their own, the tool is no longer augmenting anything. It is deciding, and it is deciding on the basis of a question that was framed carelessly by someone who had already checked out.

I have watched this happen to capable people. It is not a failure of intelligence. It is what happens when something is fluent enough, often enough, that checking starts to feel like distrust. The fluency is the problem. A calculator that gave you a plausible but wrong answer one time in twenty would be thrown away. A model that does the same thing in prose is called a productivity gain.

So the leadership question is not whether your people use the tool. It is whether they are still the pilot when they do, and what you have put in place to keep them there.

Three things help, and none of them is a policy document.

The first is that people know what good looks like before they ask. If you cannot describe the right answer, you cannot recognise a wrong one, and you will accept whatever arrives. In our workshops we spend the first hour on this and no time at all on prompting, and it is the hour people tell me changed how they work.

The second is that the work stays with the person. When an output goes to a client or a board, it goes with a name on it, and that name is not the model's. This sounds obvious. It is routinely abandoned the first time a deadline is close.

The third is that leaders model it. If the people running the organisation are visibly copying the last paragraph and moving on, everyone below them will do the same, and they will be right to, because that is what the organisation has shown it rewards.

The stakes are not abstract. I was using GPT-3 in my own business six months before ChatGPT arrived, and the most expensive mistake I made in that period was not a hallucination. It was a week in which I let the tool set the agenda for a piece of analysis, because its framing was tidy and mine was not yet. The analysis was coherent, well written and answered the wrong question. I had been a passenger for a week and had not noticed the cabin door close.

I do not tell that story to argue for caution. Caution is its own way of being a passenger, and the organisations that wait find the pace is no longer theirs to set. I tell it because the alternative to being carried is not refusing to fly. It is keeping your hands on the controls, knowing your destination, and treating every fluent answer as an instrument reading rather than an instruction.

That is a habit, and habits are built in the work, by people who understand why. It is the least glamorous part of AI empowerment and, as far as I can tell, the whole of it.