Welcome to the show. I’m Claire Monroe, here with Edwin Carrington. Edwin, picture this: a hiring manager has two CVs—one says “12 years of experience,” the other says “top school, great references,” and after a 30-minute conversation they go, “I’ve got a good feeling about this one.” And the uncomfortable part is… those three signals—experience, pedigree, and instinct—often predict job performance a lot worse than people think. Yes, and that gap between confidence and accuracy is where most hiring mistakes begin. For decades, industrial-organizational psychologists have studied what actually predicts performance. The classic Schmidt and Hunter research gave people a simplified hierarchy—cognitive ability high, work samples strong, structured interviews useful, and then a long list of weaker methods. The issue is, many managers remembered the ranking… but not the nuance. So the takeaway became something like, “just hire the smartest person in the room and you’re done.” Precisely. And that’s where the newer research corrected things. Sackett and his colleagues in 2022 revisited those earlier findings and showed that some of the original validity estimates were overstated due to statistical adjustments. The conclusion wasn’t “throw everything out.” It was “be more precise.” Cognitive ability still matters. Work samples still matter. Structured interviews still matter. But there is no single method that reliably solves hiring on its own. Which is kind of funny, because the update didn’t validate gut instinct. It just told people to stop acting like one tool is magic. Exactly. It lowered the volume on certainty, not the direction of the evidence. The practical hierarchy still favors structured, job-relevant methods over informal ones. A candidate demonstrating the work is more informative than a compelling conversation about the work. Then why do the weaker signals feel so convincing? Because “12 years of experience” sounds solid. A degree sounds earned. And an unstructured interview feels like you’re getting the full picture. Because those signals are easy to turn into stories. “Ten years of experience” sounds like expertise, even though it may just be repetition. Research has shown for a long time that years of experience alone are only weakly related to performance. Education helps at the entry level, certainly, but becomes a modest predictor once candidates are already in the pool. And unstructured interviews are particularly misleading because they generate confidence. The interviewer leaves thinking they understand the candidate. What they often understand is whether the interaction felt smooth. So the signal is comfort, not capability. Yes, and comfort is biased. We prefer people who feel familiar, who think in similar ways, who communicate in ways we recognize. None of that reliably answers the question, “can they do the work.” And personality tests sit somewhere in the middle, right? Because people either treat them like the answer to everything or completely dismiss them. That’s right. Broad personality traits—especially conscientiousness—can add incremental value. But they are not substitutes for skill evidence. They work best as supplements. That’s the consistent pattern in the research: weaker signals feel richer because they’re easier to talk about, while stronger predictors force you to confront the actual job. What does the work require? What decisions matter? What does good performance look like in practice? So if CVs and instinct are shaky, what actually improves prediction in a real hiring process? Not in theory—on a Monday morning with a team that’s busy. Three things consistently emerge. First, structured interviews—same questions, same criteria, same scoring logic. Second, work samples or simulations—something that resembles the job itself. If the role involves analysis, give them a case. If it involves writing, have them write. Third, job-relevant assessments—cognitive or technical, where appropriate. And importantly, you combine these methods. Hiring improves when it becomes a system, not a single bet. That word “system” matters. Because a lot of teams are still looking for the one perfect filter—the one interview, the one test, the one person who just “knows.” Yes, the idea of the all-knowing interviewer. But prediction improves when different methods cover different aspects of performance. A structured interview captures reasoning and past behavior. A work sample captures demonstrated skill. An assessment may capture learning ability or technical depth. Together, they provide a more complete picture. So the update from the newer research isn’t “everything changed.” It’s more like: the old numbers were too confident, but the direction was right. Exactly. Sackett and his colleagues refined the estimates and emphasized context—things like range restriction and variability across roles. But they did not turn informal methods into reliable predictors. The center of gravity remains the same: structure, relevance, and combination. Let me try to compress that. It’s not “find the smartest person and hope.” It’s “build a set of signals that actually reflect the job, then use them consistently.” That’s close. I would add one word: observable. You want evidence the candidate can produce something, solve something, or decide something. Observable skill is more reliable than inferred potential. Observable skill… yeah, that changes how you look at hiring immediately. And that’s where something like OAD fits in, because it shifts the focus away from who looks impressive on paper to who can actually demonstrate the capabilities the role needs. Exactly. The practical takeaway is simple, even if the execution takes discipline. Test the work, or something close to it. Don’t confuse polish with capacity. Don’t confuse tenure with mastery. And don’t let a strong interview performance substitute for missing evidence. And the side effect is it usually becomes a fairer process too. More consistency, less improvisation, fewer decisions based on “this person reminds me of me.” Yes. Better measurement tends to produce better fairness. You’re evaluating candidates against the role, not against personal preference. So if you’re hiring right now, the move is pretty clear: test for the skills that actually matter, not the ones that just sound impressive. And if you want a practical way to start, you can test OAD’s tools—like behavioral assessments—for free at o-a-d-dot-a-i. The real question is not whether someone can perform well in an interview. It’s whether they can perform well in the role. That’s where the decision should live. Thanks for listening to The Science of Leading.
Audio Coursesby Jellypod