If you apply for a job this year, there is a reasonable chance that the first thing to read your CV will not be a person, and a growing chance that the first voice you speak to will not be one either.
Most writing about this is either a vendor promising fairness or a columnist predicting dystopia. There are now two substantial pieces of evidence, and they point in opposite directions. One is among the largest randomised experiments ever run on AI at work, and its result surprised the people who ran it. The other is an audit that found bias so lopsided it is hard to read without wincing. Both are true, and the difference between them tells you what to watch for.
The experiment: 70,000 interviews
Economists at the University of Chicago and Erasmus University Rotterdam partnered with a recruitment firm filling entry-level customer service roles in the Philippines. The scale was unusual: 70,884 applications for 48 postings.
After an initial screen, applicants were randomly assigned to one of three groups. One was interviewed by an AI voice agent. One was interviewed by a human recruiter. The third was allowed to choose. In every case, a human recruiter then evaluated the interview and made the hiring decision. The AI conducted the conversation. It did not decide.
Professional recruiters, asked beforehand, predicted the AI would do worse. It did not. Contrary to the forecasts of professional recruiters, the authors write, AI-led interviews increase job offers by 12%, job starts by 18%, and 30-day retention by 17% among all applicants.
Those are outcomes for the applicants, not just efficiency gains for the firm. People interviewed by the machine were more likely to be hired, more likely to turn up and more likely to still be there a month later. A later version of the paper adds that there was no decline in the productivity of hired workers.
Why would a machine interview better?
The researchers analysed the transcripts, and the explanation is unglamorous. AI-led interviews elicit more hiring-relevant information from applicants compared to human-led interviews.
The updated paper describes this as controlled variance: the AI’s interviews are more structured and consistent while remaining responsive to individual applicants. A human recruiter on their thirtieth call of the day skips questions, runs short of time, warms to one candidate and rushes another. The agent asks everyone the same things in the same order with the same patience.
That is a finding about human inconsistency as much as about AI capability. An AI agent is, among other things, a structured interview that never gets tired.
Recruiters’ behaviour shifted too. They scored AI-interviewed applicants higher, but placed greater weight on standardized tests in their hiring decisions. Given a more uniform interview, they leaned more on the objective measure.
The number that bothered the author
When applicants were given the choice, 78% of applicants chose the AI recruiter. The paper also found evidence that applicants with lower test scores are more likely to choose AI.
Applicants rated the interview and recruiter quality similarly in both conditions, so this was not resignation. The lead author said this was the result that most troubled him: in a high-stakes interaction with real consequences for someone’s livelihood, people leaned towards the machine.
Why they preferred it is not established. Convenience is one candidate, since an AI can interview at any hour. Fear of being judged by a person is another, and the link with lower test scores hints at it. The study cannot say.
The audit: what happens when AI does the judging
Now the other study, which tested the step where AI is most widely used already: ranking CVs.
Researchers at the University of Washington took 554 resumes and 571 job descriptions from real-world documents, across nine occupations. They swapped in 120 first names associated with white and Black men and women, changed nothing else, and asked three open-source language models to rank the CVs against the job descriptions. That produced more than three million comparisons.
The CVs were identical except for the name. The systems preferred white-associated names 85% of the time versus Black-associated names 9% of the time, and male-associated names 52% of the time versus female-associated names 11% of the time.
The intersectional result was the starkest. The systems never preferred what are perceived as Black male names to white male names. As the lead author put it, there was a really unique harm against Black men that was not visible from looking at race or gender in isolation.
Reconciling the two
These studies do not contradict each other, because the AI was doing different jobs.
In the interview experiment, the AI gathered information and a human decided. Standardisation improved the inputs to a human judgement. In the CV audit, the AI made the ranking itself, from text that included a name, and reproduced the patterns in its training data.
The lesson is about where in the pipeline the model sits. Used to ask every candidate the same questions well, AI appears to help candidates. Used to decide who is worth talking to, it can encode exactly the discrimination that hiring law exists to prevent, and do so at a scale and consistency no single biased recruiter could manage.
What is not known
Whether the interview result generalises. It was one firm, one country, one job type: entry-level customer service. Whether an AI interviewer helps or harms in a senior engineering loop, where the interview is as much about the candidate assessing the team, is untested.
What commercial screening tools actually do. The audit tested open-source models used in a way that approximates real systems. As the lead author noted, a lot of these systems are proprietary, so we are limited to analysing how they work by approximating them. Outside a New York City law, there is no regulatory, independent audit of these systems.
Whether bias persists when names are removed. Stripping names is an obvious fix, but CVs leak demographic signals through schools, societies, locations and language, and that was not the focus of the audit.
Long-run effects. Thirty-day retention is not a career. Whether people hired through AI interviews do better or worse over years is unknown.
If you are the candidate
Treat an AI interview as a structured interview. It will probably ask every question on its list and give you time to answer. Prepare concrete examples for standard competency questions, because completeness of information is what the transcript analysis found the AI extracts and what the human reviewer then reads.
Assume a human reads the output. In the experiment, recruiters made every decision. Speak to the eventual reader, not to the bot.
Give your CV substance and detail. The more a document offers beyond the name, the less weight the name itself can carry.
If you are the one hiring
Put AI where it standardises, not where it decides. The evidence supports using it to run consistent first-round conversations with a human making the call. It does not support letting a language model rank CVs with names attached.
Audit before you deploy. Run the same CVs through your tool with different names and look at what changes. If your vendor cannot show you that test, run it yourself.
Common questions
Are AI job interviews fair?
In the largest experiment to date, 70,000 applicants were randomly assigned to AI or human interviewers, and those interviewed by AI were 12% more likely to receive a job offer, with a human still making the decision. The study covered one firm and one type of entry-level role.
Do AI résumé screeners discriminate?
An audit of three language models ranking identical CVs found they favoured white-associated names 85% of the time and female-associated names only 11% of the time, and never preferred Black male-associated names over white male-associated names.
Why did AI interviewers lead to more job offers?
Transcript analysis showed AI-led interviews elicited more hiring-relevant information. The AI’s interviews were more structured and consistent than human ones while still responding to each applicant.
Do candidates prefer AI interviewers?
In the experiment, 78% of applicants chose the AI recruiter when given the option, and those with lower test scores were more likely to do so. Why they preferred it has not been established.
The bottom line
AI in hiring is neither the fairness machine vendors describe nor uniformly the threat critics fear. Asking the questions, it outperformed tired human recruiters and candidates did better for it. Making the judgement from a CV with a name on it, it reproduced racial and gender bias to a startling degree. If you are applying, prepare as you would for a rigorous structured interview. If you are hiring, let the model do the asking and keep the deciding, and the audit, in human hands.
Did this answer your question?
Thanks. That helps us decide what to fix.
Leave a comment