Talk to enough hiring leaders right now, and a pattern emerges. Companies say they’re hiring for AI fluency – rewriting job descriptions, adding AI questions to interviews, training managers to ask about prompt engineering and tool use – but keep ending up with people who can talk about AI confidently in an interview but can’t competently ship anything with it once on the job.
This is occurring across all company types, from 20-person startups to enterprises with thousands of employees. We at TestGorilla conducted a survey earlier this year of 2,000 senior hiring leaders across the US and UK, 95% of respondents said they list “AI fluency” as a hiring factor.
But eye-openingly, a staggering 59% of those same companies admit that they have already made a bad AI hire. And with over 75% of knowledge workers now using AI, your hiring processes cannot afford to fall behind.
But if you are running a startup right now, there is a quiet structural advantage hiding inside this gap, and most founders are letting it slip past them.
A startup’s speed is its advantage
Large enterprises have legacy hiring infrastructure that is genuinely difficult to retool. Roles built for processes and functions that existed before AI, contracts with agencies that take a year to negotiate, and ATS workflows that hundreds of recruiters use daily. Piloting changes to these often takes months, let alone formalising them.
Startups have none of this. A 30-person scale-up can redesign its hiring process in a week or less if the founder decides to; however, many never do. Instead, they import the worst habits of late-stage companies: hiring through networks, over-weighting resumes, anchoring on a candidate’s last employer, and rewarding the most articulate person in the room.
These habits were already weak proxies for performance before AI. But now, we’d argue they’re on life support. This is because they tilt the funnel towards confident storytellers rather than competent operators, and AI has made confident storytelling almost free.
Anyone can send a hundred tailored CVs in an afternoon with a good prompt, AI can make almost anyone look senior on paper, and candidates often walk into the interview better prepared by AI than the person interviewing them. The trade-off is that the cost of getting this wrong falls harder on startups than on a corporation.
A few years into TestGorilla, we hired a senior leader from a much bigger company. Brilliant interviews, glowing references, all the right answers to all the right questions. Six weeks in, it was clear we’d mistaken preparation for competence. We lost three months untangling it and the part that hurt most wasn’t the cost. It was that two of our best early hires quit the same quarter, because the energy in the team had shifted.
One bad hire moved three other people, all in the wrong direction.
At 10,000 people, that hire is invisible. There’s enough team around the person to course-correct while everyone keeps moving. Startups cannot afford to confuse confidence with competence. The good news is that the fixes are fast, and they’re mostly free.
Three things you can change today
Three shifts made a big difference for us, and none of them required a process overhaul.
First, change the question you ask. Stop asking which AI tools a candidate uses – that’s a vocabulary test, and anyone can pass it without ever having shipped anything. Ask instead: “Walk me through the last workflow you redesigned with AI. What changed? What broke? What did you verify before shipping?” The answers split candidates almost immediately. A practitioner answers with constraints, trade-offs, and judgment calls you can only have if you’ve done the work. A parrot lists tools.
We swapped this question in on a round of interviews last year. Two candidates for the same role, back to back. One spent four minutes listing every tool they’d ever touched. The other walked us through automating part of our own customer support triage and their own example, not a canned answer. It wasn’t perfect, there were stutters and some gaps. It was honest. We hired the second one. She’s still with us and now runs the team.
Second, build short, role-relevant AI tasks into the interview including work they can’t fake. Ask candidates to use AI live to produce an output. It doesn’t have to be perfect; what you’re watching is the prompts they reach for, why they trust (or distrust) the answer, and how they get there. If you’re worried about candidates the AI made look good give them a 30-minute problem in a controlled setting, then a short live conversation about how they solved it. Verification beats impression every time.
Third, pilot one role differently this week. You don’t need to overhaul your hiring system overnight. Pick one open role, and define what AI fluency concretely looks like in it, the observable behaviours, beyond the vocabulary.
For engineers, it’s writing evaluations that catch hallucinations. For marketers, it’s orchestrating multi-step workflows and knowing where a human has to stay in the loop. For your people team, it’s redesigning screening to remove bias rather than scale it.
Run that one role through the new process, measure the difference in who you hire, and only then roll it out. One role, one change, measure. If it works, expand it. If it doesn’t, you’ve burned one cycle, not a quarter.
If this seems daunting, we authored a deep-dive report on how to get your company ready for AI hiring.
Why this matters now
The startups that take 12 months to rebuild their hiring process around AI fluency will spend that time explaining why their headcount grew faster than their output. The ones that move now compound the advantage of every good hire from day one.
There’s one principle worth carrying into your next hard call: high execution with low judgment is not a hire; high judgment with developing execution is. Tools change every few months; judgment is what makes a hire compound. You don’t need more resumes. You need a better signal.
And as a founder, you can change yours this week.
