An AI score without evidence is an opinion with a number on it. If an AI interviewer tells you a candidate scored 4 out of 5 on problem solving, the useful question is not whether 4 is right. It is what the candidate said that earned it.
An evidence cited AI interview scorecard answers that on the page. Every finding, strength or gap, is linked to the candidate’s own words, quoted exactly, so a hiring manager can check the claim in seconds instead of taking it on trust. This guide explains what such a scorecard contains, why model written scorecards need this safeguard, and how to audit any AI scorecard in five minutes. It ends with a template your human interviewers can use too. Sources were checked in October 2026.
What an interview scorecard is
An interview scorecard is a structured form for recording what you learned about a candidate against criteria you chose in advance. Greenhouse’s glossary describes it as a framework for everyone who interviews or assesses candidates, and lists four components: qualifications, technical skills, interpersonal skills and non required attributes that are nice to have.
The reason to use one is that structure predicts better than instinct. In their 2022 meta analysis in the Journal of Applied Psychology, Sackett and colleagues estimated an operational validity of .42 for structured interviews, the highest of the methods they compared, as summarised by SIOP. That is an average across many jobs, not a guarantee, but it explains why most hiring guides start with the scorecard.
A human scorecard has a known weakness. It is filled in from memory, after the fact, by someone who has already formed an impression. Open access HR textbooks list the usual errors: the halo effect, where one strong trait lifts every rating; the horn effect, its opposite; similar to me bias; contrast with the previous candidate instead of the job requirements; recency; and plain central tendency or leniency. A scorecard reduces these errors but does not remove them.
What changes when AI writes the scorecard
An AI interviewer can ask every candidate the same questions, record everything and draft the scorecard in minutes. That fixes consistency and speed. It also creates a new problem: the scorecard is now text written by a model, and models sometimes write things the source does not support.
How often depends on the task and the model. One recent benchmark, VeriGray, found that even state of the art models such as GPT 5 hallucinate in about 6% of sentences in summarisation tasks (Ding et al., arXiv, December 2025). That is a summarisation benchmark, not an interview scoring study, so it says nothing about the error rate of any hiring product. What it does show is that a fluent summary is not evidence.
The practical conclusion: treat an AI scorecard as a claim that needs checking, and choose tools that make the check fast.
What “evidence cited” means
In an evidence cited scorecard, each finding has four parts you can see together: the criterion, the score, the claim, and the exact words from the candidate that support it. Compare the two versions below. The candidate and answer are invented for illustration.
| Without evidence | With evidence | |
|---|---|---|
| Criterion | Database performance | Database performance |
| Score | 4 / 5 | 4 / 5 |
| Claim | “Strong understanding of query optimisation.” | Diagnoses before changing anything, and weighs the cost of a fix. |
| Evidence | None | “I’d run EXPLAIN first to see if it’s a sequential scan. If the filter is on customer_id and created_at I’d try a composite index, but I’d check the write volume first because every index slows inserts.” |
| Can a reviewer verify it? | Only by re interviewing | In ten seconds, against the transcript |
The left column might be right, but you have no way to know. The right column lets you disagree precisely. You might think the quote shows good instincts but not depth, and score it a 3. That is a conversation about a sentence, which is far easier than a standoff about a number.
A complete scorecard should also show what it could not assess. If the interview never covered failure handling, the scorecard should say “no evidence” for it, not guess.
A five minute audit for any AI scorecard
Run this on a real scorecard from any vendor, including ours, before you rely on one.
| Check | What to do | Pass looks like | Fail looks like |
|---|---|---|---|
| 1. Every finding has a quote | Scan each score and claim | A quote beside every finding | Scores with summaries only |
| 2. Quotes are verbatim | Search the transcript for three quotes | Exact matches | Paraphrases presented as quotes |
| 3. Quotes support the claim | Read the quote and ask if you would draw the same conclusion | The quote shows the behaviour claimed | Accurate quote, unrelated claim |
| 4. Gaps are visible | Look for criteria the interview did not cover | “No evidence” or low confidence marked | A confident score on an uncovered topic |
| 5. Recommendation follows the scores | Compare the overall recommendation with the dimensions | You can see how it was reached | A recommendation that contradicts its own evidence |
| 6. A human can override | Try to change the outcome | Override is easy and logged | Outcome applied automatically |
A scorecard that fails check 1 or 6 should not influence a hiring decision. If it fails check 2 or 3, test the tool further before you buy it.
Why this matters more as regulation arrives
The direction of travel is towards explainability and human oversight. India’s MeitY AI Governance Guidelines, issued in November 2025, are non binding but set out principles that include accountability and systems that are “understandable by design”. In the EU, AI used in employment is classed as high risk, and the date those obligations apply moved to 2 December 2027. India’s DPDP Rules are phased, with the main duties around 13 May 2027, so check the current dates before you rely on them.
An evidence trail will not make a tool compliant by itself, and this is not legal advice. It does give you something to show a candidate, an auditor or a colleague who asks why a decision was made.
What evidence citation does not do
- It does not prove the score is valid. An accurate quote can still be a poor predictor of job performance. Validity needs separate evidence.
- It can hide selectivity. A tool could quote the favourable lines and skip the rest. Read the whole transcript for at least a sample of candidates.
- It does not fix bad questions. If the interview asks the wrong things, the scorecard will faithfully cite the wrong answers.
- It does not replace the decision. The scorecard should recommend, and a person should decide.
A template you can use today
The same discipline works for human interviews. Copy this table into your notes tool and complete it during or straight after each interview, before any discussion with colleagues.
| Criterion | Anchor for 1 and 4 | Evidence (exact words or observed action) | Score (1–4) | Not covered |
|---|---|---|---|---|
| Example: Debugging approach | 1 = guesses; 4 = forms a hypothesis and tests it | “...” | ||
| Example: Communication | 1 = unclear; 4 = explains trade offs simply | “...” | ||
| Example: Ownership | 1 = blames others; 4 = describes fixing it in production | “...” |
Two rules make it work. Write the anchors before the interview, and do not enter a score until you have written the evidence.
How Kestrel does it
Kestrel is an AI native hiring platform for engineering teams at startups and scale ups. Its AI interview produces a scorecard with dimension scores, a confidence level and a recommended outcome (Advance, Hold or Reject). Findings cite the candidate’s own words, and each quote links to its timestamp in the transcript, which you can download as a PDF. The scorecard also lists areas with thin evidence and questions the candidate skipped.
The AI recommends and a person decides. After an interview the candidate moves to a human review stage, where a recruiter or hiring manager makes the call, and that decision is recorded in the activity log. The AI interview never rejects or hires anyone on its own.
Kestrel’s AI interviews are question based and run in text, with an optional voice mode that recruiters can enable per job. A separate Smart Canvas format covers system design. They run in the same pipeline as resume screening, MCQ and coding rounds and the applicant tracking system, so the scorecard sits next to the rest of a candidate’s record.
The audit above is a good way to evaluate us. You can book a demo, bring the checklist and run it on a real scorecard.
Frequently asked questions
What is the difference between an AI scorecard and a human scorecard?
A human scorecard is a form an interviewer fills in after a human interview. An AI scorecard is generated from an AI led interview and should cite the candidate’s own words for each finding. Both need anchored criteria, and both should end in a human decision.
Can an AI scorecard be biased?
Yes. Bias can come from the interview questions, the criteria, the model and the data it was built on. Evidence quotes make some problems easier to see, because a reviewer can check whether the cited words justify the score, but they do not remove bias. Test for it across candidate groups where you can lawfully collect the information.
Should an AI scorecard make the hiring decision?
No. Use it as a structured recommendation and evidence pack for the person who decides. Where human oversight is expected, a recommendation applied automatically is the wrong design.
How do I check that a quote is real?
Search for it in the full transcript. A trustworthy product makes this one click, and the quote should match exactly, not approximately.