New research from KPMG and the University of Texas found that putting a human in the loop doesn’t automatically make AI better. Sometimes it makes it worse.
The idea
Researchers had more than 500 people use the same AI agent on real business work. About half made the output better. KPMG calls them AI Amplifiers. Another group got roughly what the AI would have produced alone. And about a quarter actually made the results worse.
The surprise: the people who made AI worse weren’t less skilled or less experienced with AI. The difference was how they worked with it. Amplifiers gave it context, broke problems apart, knew what good looked like, challenged the output, and kept iterating. They weren’t handing work to AI. They were orchestrating it.
Why it matters
- Usage isn’t the right metric. Access, adoption, and training completion tell you who’s using AI, not who’s getting better results with it.
- Find your Amplifiers. Every organization has people who take these tools somewhere unexpected. Learn what they do differently and spread it.
- Human in the loop isn’t enough on its own. The human has to actively shape the work, not just approve it.
The next high performers
I think the next generation of high performers won’t just be people who are great at their jobs. It’ll be the people who combine what they know with AI and produce something neither would on its own.
So stop asking only “Are our people using AI?” Start asking “Who gets better when we give them AI, and who makes the AI better in return?”
Join the conversation: see the comments on LinkedIn.
Read the full transcript
Hey, I’m Matt.
I came across some research from KPMG and the University of Texas that really got me thinking.
They took more than 500 people and had them use the same AI agent to do actual business work.
And what happened was kind of fascinating.
About half of the people made the AI better. KPMG calls those people AI Amplifiers.
Another group basically got the same result that the AI would have produced on its own.
But there was a third group, about a quarter of the people in the study, that got worse results when using AI.
Think about that for a second.
We talk all the time about keeping a human in the loop with AI. And I’m certainly a believer in that.
But apparently just putting a human in the loop doesn’t always make the output better.
And here’s the part of the research that really got my attention.
I immediately assumed the people who made AI worse just weren’t as skilled. But that wasn’t really the case. They had solid critical-thinking skills, they understood the work they were being asked to do, and they weren’t AI novices.
The difference seemed to be in how they worked with the AI.
The Amplifiers were much more involved in shaping the work. They gave it context. They broke problems apart. They knew what good looked like. They challenged the output when it needed to be challenged. And they kept iterating with it.
They weren’t just handing work to AI.
They were orchestrating it.
And I believe there’s something really important in that distinction.
Because we’re spending a lot of time right now talking about AI adoption inside companies and asking the same old questions:
How many people have access to AI?
How many people are using it?
Did everybody take the AI training?
And, on the surface, those seem like reasonable things to measure.
But I’m starting to wonder if we’re measuring the wrong thing.
Maybe the more interesting question is: Who are the people in your company who consistently make AI better?
Because I suspect we all have these people.
There are people I work with where you give them one of these tools and somehow, a few hours later, they’ve taken it somewhere you didn’t expect. In a good way!
And there are other people who have access to exactly the same technology and don’t get anywhere near the same result.
That gap is really interesting to me.
And if I were running an organization trying to become AI-native, I wouldn’t just want to know who my biggest AI users are.
I’d want to know who my Amplifiers are.
Then I’d want to understand exactly what they’re doing differently.
Here’s the thing:
Maybe the next generation of high performers isn’t just the people who are really good at their jobs.
It’s the people who can take everything they already know about their jobs, combine that with AI, and produce something neither of them would have produced on their own.
So I believe we need to stop asking just, “Are our people using AI?”
And start asking, “Who gets better when we give them AI?”
And maybe even more importantly, “Who makes the AI better in return?”
Thanks for watching.
Leave a Reply