InsightHireHelp Center

Interviewer calibration

See who rates higher or lower than their peers, who gives everyone the same rating, and who often differs from the panel or the final decision, so you can coach before it skews a hire.

Two interviewers can watch the same candidate and score them very differently. Interviewer calibration shows how each interviewer rates compared with their peers, so you can spot an easy grader, a hard grader, or someone whose ratings do not match how hires turn out. It is a report under Analytics, built from the scorecards your interviewers already submit.

It is part of structured interviews, so it needs that feature turned on for your organization.

Who can see it

Anyone who can view reports sees Analytics → Interviewer calibration. Organization Admins and anyone with all-jobs report access see every job. Hiring managers and others limited to certain jobs see only scorecards from the jobs they can see.

The report never shows candidate names, emails or interview titles. It shows the job, the interview kit, the date and the ratings.

Read the report

Pick a date range at the top of Analytics. You can also narrow it by department, by job, and by interview kit (the kit menu is on the report itself). Each row is one interviewer:

  • Scorecards and Training status, under the name.
  • Ratings (1 to 5): five small bars showing how many scorecards were rated 1, 2, 3, 4 and 5, with the average beside them. Point at the bars for the exact counts.
  • Vs. peers: their average rating minus the average on the same kit. A plus sign means they rate higher than the kit average. Point at it for the leniency score, which compares them with the other interviewers on the same kit (0 means in line).
  • On time: how often they submitted feedback within 24 hours of the interview ending. Drafts that are still open count as late.
  • Matches decision: how often their leaning matched the final result. A yes or strong yes counts as matching a hire, and a no or strong no counts as matching a rejection. Candidates who are still in the process are left out.
  • Off from panel: how often their rating was 1.5 or more points away from the average of the other interviewers on the same interview.

A scorecard's rating is the average of its 1 to 5 competency scores. If the scorecard has no scores, the recommendation is used instead: strong yes is 5, yes is 4, no is 2 and strong no is 1. A "no decision" with no scores is left out of the rating.

What the labels mean

A label appears only after an interviewer has 5 submitted scorecards in the window. Before that, the row says how many more are needed, because a few scorecards say very little.

LabelWhen it shows
Rates higher than peersTheir ratings are clearly above other interviewers on the same kit
Rates lower than peersTheir ratings are clearly below other interviewers on the same kit
Gives almost everyone the same ratingTheir ratings barely vary from one candidate to the next
Often disagrees with the rest of the panelIn at least 4 of 10 interviews, they were 1.5 or more points from the other interviewers
Often differs from the final hiring decisionTheir leaning matched the hire or reject result less than half the time
Often late with feedbackFewer than 6 in 10 scorecards were in within 24 hours

An interviewer with any label except Often late with feedback gets a Calibration needed note. Late feedback is a follow-up reminder, not a calibration problem. Labels are a prompt for a conversation, not a verdict: a strong referral pipeline or a tougher role can explain a lot.

See the scorecards behind a row

Select a row to open that interviewer's scorecards, newest first. For each one you see the job, kit and date, the rating, the rest-of-panel average and how far apart they were, the final decision (hired, rejected or still open) and whether the interviewer was on time. This is a good place to find the one or two interviews that moved the numbers.

Download a CSV

Select Download CSV to get one line per interviewer for the same date range and filters: counts for each rating, averages, rates and notes. Open it in a spreadsheet to share with your hiring managers.

Calibration needed on the training page

On Settings → Scheduling, under Interview limits and training, an interviewer who needs calibration gets a Calibration needed badge under their name. It looks at the last six months and links to this report. It is a good prompt for someone who is Certified but whose ratings drift, or for moving someone back to shadowing.

Tips

  • Compare people on the same kit. Ratings on different kits are not compared with each other.
  • Give new interviewers time. Until they reach 5 scorecards they are shown without labels.
  • Run the report over a longer range (for example 90 days) for steadier numbers.
  • Use the scorecard list to read why someone rated high or low before you coach them.