Competency-Based Interviews for Tech Roles: How to Avoid Candidates Who Look Great on Paper but Don’t Work Out

You find a candidate with a strong CV, a polished GitHub, experience at a well-known scaleup, and a technical assessment they completed without much trouble.

You hire them.

Three months later, the problem isn’t their technical level.

They struggle with ambiguity. They need too much direction. They create friction with Product. They struggle to prioritize when things get messy.

Technically, they meet the bar.

Operationally, they don’t work.

That’s one of the biggest blind spots in tech hiring: most processes evaluate knowledge, but not enough behavior in context. And in fast-moving environments like startups, that’s often what separates a strong hire from a costly mistake.

Why “comfortable” signals aren’t enough

A typical technical hiring process reviews the candidate’s background, runs a technical assessment, discusses architecture, and makes a decision.

It feels rigorous.

But it can leave out three variables that matter far more than they initially seem:

• How they prioritize under real pressure, when something breaks in production.
• How they collaborate outside their immediate team, with Product, Data, or Business.
• How they operate without a clear playbook, when there isn’t an obvious answer.

If your interview only validates knowledge, you’re evaluating someone for an exam—not for the actual job.

Technical performance never exists in isolation. Knowing a technology matters, but so does knowing how to make decisions with incomplete information, communicate trade-offs, and defend those decisions when others disagree.

From the Job Description to the Competencies That Actually Matter

The mistake often starts before the interview.

Teams take the job description, copy generic competencies like “teamwork” or “proactivity,” and then evaluate the same things for every role.

That doesn’t tell you much.

A good JD shouldn’t only tell you which technologies someone needs to know. It should help you identify where this person is likely to be challenged in the role.

For example, if a Senior Backend Engineer role involves microservices, ownership of critical services, and close collaboration with Product, those requirements translate into observable competencies:

Designing scalable systems Debugging production issues Technical ownership Communicating with non-technical stakeholders Prioritizing under constraints

You don’t need 10 competencies.

In most interviews, four or five well-defined competencies are enough.

A simple filter is:

Is this critical to the role? Can I observe it through concrete examples? Does it help distinguish a strong candidate from an acceptable one?

If the answer isn’t yes to all three, it probably doesn’t belong on the scorecard.

Ask Questions That Produce Evidence, Not Polished Answers

Questions like “How do you work in a team?” invite rehearsed answers.

The STAR method—Situation, Task, Action, Result—works because it forces candidates to get specific: what was happening, what they were responsible for, what they actually did, and what changed afterward.

But the most important part is usually Action.

If a candidate says:

“I coordinated with the team and optimized the service.”

Keep going.

What did you actually change?

Which service did you touch?

What metrics were you looking at?

What alternatives did you consider?

Why did you choose that approach?

What happened afterward?

A strong answer shouldn’t just sound good. It should contain enough operational detail to be evaluated.

Often, the simplest follow-up questions are the most useful: ask them to walk through a specific incident, explain what hypotheses they considered, or describe how they measured the result.

Every Technical Profile Needs a Different Interview Approach

There is no universal interview question set that works equally well for every technical role.

The core competency changes depending on the profile.

Backend & Frontend

Focus on technical judgment under constraints: architecture decisions, trade-offs between speed and quality, and how candidates handle technical disagreement.

ML / AI

Look for experimental rigor and the ability to take models into production. What happens when a model performs well in a notebook but degrades in production? How do they validate data quality?

Data Engineering

Evaluate data reliability with business context. What happens when a critical metric suddenly becomes unreliable? How do they identify the problem and determine what can still be trusted?

DevOps / Cloud

Focus on operational resilience and automation judgment. How do they prioritize when inheriting fragile infrastructure? How do they balance speed, reliability, and security?

In every case, the goal isn’t to build a list of tools the candidate knows.

It’s to understand the thinking behind their decisions.

Scorecards: Turning Answers Into Comparable Evidence

Without a scorecard, interviews quickly turn into a collection of opinions.

“I liked them.”

“They seemed senior.”

“I had a good feeling.”

That’s exactly what you want to avoid.

A simple 1–5 scale for each competency, backed by concrete evidence rather than vague labels, makes candidates easier to compare and discussions much more grounded.

The difference between a useful scorecard and a useless one is often the wording.

Weak: “Shows ownership.”

Observable: “Takes responsibility for a problem without waiting for detailed instructions, proposes an execution path, and follows through until resolution.”

The second one gives interviewers something they can actually look for.

Calibration Matters as Much as the Interview

Many teams treat the calibration meeting as a quick formality.

That’s where bias can creep in.

Halo effect. Personal affinity. The seniority of the person speaking first. Selective memory of the most impressive anecdote.

A stronger calibration process is simple:

Each interviewer completes their scorecard before the discussion. Each person brings concrete evidence—not impressions. Disagreements are investigated rather than resolved by voting. Gaps are identified explicitly.

The outcome doesn’t always have to be a simple yes or no.

Sometimes the conclusion is:

“Strong technical ownership, but there’s a risk around cross-functional communication.”

Or:

“Most competencies are validated, but we need one additional interview focused on prioritization.”

That is much more useful than “I liked them, but something felt off.”

The Takeaway

Reducing hiring mistakes in technical roles doesn’t necessarily require a harder technical test or another interview round.

It requires a better sequence:

Define a small number of critical competencies → design questions that force concrete evidence → score consistently → calibrate based on evidence, not intuition.

That’s what makes an interview predictive of the actual job—not just impressive on paper.

At PeopleFirst, we build evaluation processes around this logic, adapting them to each Engineering, Data, or AI role and to the reality of the team doing the hiring.

*Carla Costantini, Global Talent Partner at PeopleFirst Agency