Every few weeks someone shows me a dashboard. It counts prompts, or seats, or the share of staff who logged in this month, and it is climbing. The person showing me is pleased, and I understand why. A year ago the number was small and they were told to make it big. (v2)

I have stopped being pleased by it, and I want to explain why without sounding ungrateful for the effort behind it.

Almost every organisation I meet has adopted AI already. That part is done. The tools are in the building, the licences are paid for, and the people who were going to try them have tried them. What the dashboard measures is how much of that trying is happening, and it treats more as better. It is not. More usage without the conditions for using the tools well does not close the distance between adopting AI and being any better at what you do. It widens it, because the volume of work that nobody can quite rely on goes up.

I saw this first in my own business, months before ChatGPT existed, when I put GPT-3 into a process that mattered. The output was impressive on the first day and a problem by the third. Nobody could tell which parts to trust, so they checked everything, and checking everything took longer than doing it by hand. Usage was high. Value was negative. The fix was not less usage. It was deciding, in the work itself, what good looked like and who was accountable for it.

That is the pattern I have watched repeat in banks, in government departments and in consumer brands since. The conditions come first, and there are three of them.

The first is that people can use the tools well on the work that matters, not on the work that is easy to demonstrate. Writing a first draft of an email is easy to demonstrate. Reconciling two contradictory reports from two regional offices is the work that matters, and it is where the tool either earns its place or does not.

The second is that the organisation can rely on what its people produce with the tool. Reliance is not trust in the software. It is knowing which outputs have been checked, by whom, against what, and being able to say so to a regulator or a customer without hesitation.

The third is that the knowledge of what good looks like sits with the people doing the work. It cannot be bought in, because nobody outside the work has it. A framework can tell you to have a review step. It cannot tell you what a wrong answer looks like in your ledger.

None of these show up on a usage dashboard. All of them are what a leader is actually responsible for.

So what do I suggest people measure instead? Start with a single piece of work that matters and ask a plain question: is the organisation better at this than it was in January? Not faster to a first draft. Better, in the sense that the customer, the auditor or the board would notice. If the answer is yes, find out what made it so and do that again somewhere else. If the answer is no, the usage number was a vanity metric and you can stop reporting it.

This is slower than announcing a licence count, and it is less impressive in a town hall. It is also the only version of the work that compounds. Usage plateaus the moment the novelty wears off. Capability, once a team has it and knows why, does not.

I am not against the dashboard. Keep it, if it helps you see where the effort is going. Just stop treating the line going up as the point. The point was always to become better at what you do, and that is a different number, and it is yours to define.