Insights from the State of AI Bias in Talent Acquisition 2025: What the Data Reveals

When Warden AI released their comprehensive "State of AI Bias in Talent Acquisition 2025" report, it provided something the industry has desperately needed: concrete data on how AI systems are actually performing in real-world hiring scenarios. The findings are both illuminating and important for anyone considering or currently using AI in their hiring processes - and here at Spotted Zebra, we're proud to have contributed to the piece.

The report analysed over 150 AI system audits covering more than one million test samples, combined with survey data from 50+ vendors and practitioners. This scale of research offers valuable insights into where we stand with AI bias in hiring - and where we're headed.

Key Findings: A Mixed but Encouraging Picture

The report reveals a nuanced reality that challenges some common assumptions whilst validating others. Here are the key findings:

Performance Against Fairness Standards

‍85% of audited AI systems met accepted fairness thresholds using industry-standard testing methodologies. This suggests that when properly designed and implemented, AI systems can achieve measurable fairness outcomes. However, the 15% that didn't meet these standards remind us that not all AI implementations are equal.

Comparative Performance with Human Decision-Making

‍Perhaps the most striking finding was the comparison between AI and human hiring decisions. The research found that AI systems delivered up to 39% fairer treatment for women and 45% fairer treatment for racial minority candidates compared to human-led processes. This builds on the broader impact ratios: AI systems averaged 0.94 whilst human-powered processes scored 0.67.

The Reality of Human Bias

The report contextualises AI bias concerns by examining human bias in hiring. Over 99.9% of employment discrimination claims in the past five years were related to human bias rather than AI bias. This doesn't diminish the importance of AI bias concerns and of course we must consider that GenAI has only been mainstream since November 2022 so this isn’t completely representative of the situation, but it does provide perspective on the scale of existing challenges.

Regulatory and Industry Response

‍75% of HR leaders now cite bias as a top concern when evaluating AI tools, second only to data privacy. This heightened awareness is driving vendor behaviour, with most now implementing AI governance foundations, though transparency to end-users remains limited.

What This Means for Organisations

The Importance of Vendor Selection

‍The report highlights significant variation in AI system performance -bias metrics varied by up to 40% between vendors. This underscores the critical importance of due diligence when selecting AI tools for hiring. Not all systems are created equal, and the choice of vendor directly impacts fairness outcomes.

The Need for Ongoing Monitoring

‍Whilst 85% of systems met fairness thresholds, the 15% that didn't demonstrate that initial compliance isn't enough. Continuous monitoring and auditing are essential for maintaining fair outcomes over time.

Implementation Matters

‍The research shows that how AI systems are deployed matters as much as how they're built. Proper governance, clear policies, and named ownership of AI systems all contribute to better outcomes.

‍

‍

The Development Challenge: A Critical Perspective

The variation in AI system performance highlighted in this report points to a fundamental challenge in how AI systems are being developed today. Bradney Smith, AI Lead at Spotted Zebra and contributor to the research, offers this perspective:

"Building AI systems requires deliberate, measured thinking to get right. Not too long ago, AI development was rooted in building smaller models from the ground up. Practitioners had full control of their models: they designed the architecture, curated and cleaned the datasets, and shaped entire systems by hand. But in recent years, the paradigm has shifted. Increasingly, practitioners are now building on top of large, pre-trained models from providers like OpenAI, Anthropic, Mistral, and others. As a result, it feels like the focus has moved away from foundational elements like data quality, relevancy, and bias, and more towards rapid development and productionisation.

This report highlights a crucial need: to revisit and retain the best practices of earlier development approaches. We can't afford to lose sight of AI assurance principles. These must be adapted for a new era of AI, and applied holistically across a system's entire lifecycle - not just at the end, and certainly not ignored altogether. Our toolkits must include the essentials for building safe and fair systems. Tools to: improve transparency and explainability, build guardrails, leverage domain-specific data, audit for bias, and critically, implement action plans to address issues as they arise, are fundamental components that cannot be missed.

If we're serious about building responsible AI, we need to stop treating assurance as an afterthought, and start embedding it from the very beginning."

This perspective helps explain why the report found such significant variation between vendors - the difference between those who've embedded fairness considerations from the ground up versus those treating them as an afterthought.

Our Approach at Spotted Zebra

Reading this report reinforced several principles that guide Spotted Zebra's approach to developing AI systems for hiring:

Domain-Specific Design 

We build our AI systems specifically for talent acquisition, leveraging hiring-relevant data with deep understanding of skills science and recruitment practices. This focused approach helps ensure our systems understand the context and nuances of hiring decisions.

Transparency and Explainability 

Every AI recommendation we provide includes clear reasoning that can be explained to candidates, hiring managers, or regulators. This aligns with emerging regulatory requirements whilst building trust in the process.

Continuous Auditing 

We're ISO 42001 certified and regularly test our systems for fairness across different demographic groups and maintain detailed records of performance. This ongoing monitoring helps us identify and address any issues quickly.

Human-AI Collaboration 

Our AI systems are designed to augment human decision-making, not replace it. Final hiring decisions always rest with people, whilst our systems provide structured data and insights to inform those decisions.

‍

‍

The Broader Context

Regulatory Evolution

‍The report tracks the evolving regulatory landscape, from NYC Local Law 144 to the EU AI Act. Whilst compliance is still catching up to regulation, the direction is clear: transparency and accountability in AI hiring systems will only increase.

Vendor Maturity

The research shows promising progress in vendor practices. Most now have AI governance foundations in place, though there's room for improvement in areas like end-user transparency and comprehensive bias testing - a topic that we’ve seen come to the fore with the Mobley vs. Workday case. 

Industry Adoption

‍With 75% of TA teams now using some form of AI, the question has shifted from whether to use AI to how to use it responsibly. The report provides a useful framework for thinking about this challenge.

Practical Implications

For organisations evaluating or implementing AI in hiring, the report suggests several important considerations:

Due Diligence is Critical

‍The wide variation in system performance means thorough evaluation of any AI tool is essential. Ask vendors about their fairness testing, ongoing monitoring, and transparency practices.

Start with Measurement

‍Before implementing AI, establish baselines for your current hiring processes. This helps set realistic expectations and measure improvement over time.

Plan for Governance

‍Successful AI implementation requires clear policies, defined ownership, and ongoing oversight. The organisations achieving the best outcomes have invested in these foundations.

Consider the Candidate Experience

‍The report notes that only 15% of vendors offer opt-out mechanisms for candidates. As regulations evolve, transparency about AI use and candidate rights will become increasingly important.

Looking Ahead

The report paints a picture of an industry in transition. AI adoption in hiring is accelerating, regulatory frameworks are solidifying, and vendor practices are maturing. The data suggests that when implemented thoughtfully, AI can contribute to fairer hiring outcomes.

However, the variation in system performance and the gaps in current practices highlight that we're not there yet. Success requires careful vendor selection, robust governance, and ongoing commitment to measurement and improvement.

Our Commitment

Spotted Zebra is committed to contributing positively to this evolution. We were the first Interview Intelligence platform to achieve ISO 42001 certification -the global standard for AI management systems - and continue to invest in third-party bias auditing to verify fairness practices. Our team is guided by our STRIPE framework for responsible AI deployment, continuously improving systems based on the latest research whilst working to set high standards for transparency and accountability.

The conversation about AI in hiring is complex and evolving rapidly. Reports like this one from Warden AI provide essential data points for navigating these complexities thoughtfully and responsibly.

Key Takeaways

As the industry reflects on these findings, several key points stand out:

1. Performance varies significantly between AI systems, making vendor selection crucial
2. Continuous monitoring is essential for maintaining fair outcomes
3. Transparency and governance are becoming competitive advantages, not just compliance requirements
4. The comparison with human bias provides important context for evaluating AI systems
5. Regulatory clarity is improving, helping organisations make informed decisions

The path forward requires balancing innovation with responsibility, leveraging AI's potential whilst maintaining rigorous standards for fairness and transparency. The data suggests this balance is achievable - but it requires intentional effort and ongoing commitment.

To learn more about Spotted Zebra's approach to responsible AI in hiring, check out our STRIPE framework and principles here or chat to our expert team who can help guide you on your AI journey.