Reframe Interview Questions to Assess Skills, Not Stories
By: Alex Schlee
Tags: Interview, Interview Questions, Recruiting, Business

Photo by Johanna Buguet on Unsplash

A recent interview experience with a large, online retailer with notoriously prickly culture made me reflect on which interview questions work and which don’t work. Some large companies that staunchly believe they know the best way to screen for great candidates are actually missing out or hiring the wrong kind of people. To start let’s talk about my experience with this company and what was wrong with the way they formulate all their interview questions. The role I was interviewing for was a mid-level general analytics manager which was a perfect match for my skill set and previous experience on paper. I also pride myself in bringing in a wide variety of external learning from my menagerie of hobbies including programming and electrical engineering; something that I believe adds to a person’s ability to think outside the box. After passing the initial screening call with the recruiter I was scheduled to talk with the hiring manager. Dialing into the phone service I was greeted by the hiring manager; we exchanged pleasantries and jumped right into the interview…

Question 1: Tell me about a time when you took a big risk in your career. Question 2: Tell me about a time when you made a decision with no supporting data. Question 3: Tell me about a time when you made a data-backed recommendation to a stakeholder that totally changed the direction of the business.

All of these questions start with “tell me about a time when” and ask you, as the interviewee, to talk about some personal experience you had. Put another way, each of these questions asks you to dig through your memory and find one that matches the narrow set of criteria the interviewer has laid out. There is a term for this type of memory in psychology, it’s called Episodic Memory. Most cognitive psychologists will tell you there are two distinct types of long-term conscious memory: episodic and semantic. Episodic memory is used when you try to recall circumstances and situations; it is very focused on telling a story and remembering what an experience was like overall. Semantic memory is focused on remembering the outcome and the lessons learned from an experience; the details of the experience that don’t matter are not stored, only the details pertinent to the outcome¹. Another way of thinking about this is that episodic memory gives you the “what” and semantic memory gives you the “how”.

Unfortunately for myself, and countless others, a great semantic memory has come at the cost of episodic memory which makes the “tell me about a time where” questions onerous to answer. The vast majority of my memories are stored as facts and information that have been abstracted away from the “what”. Imagine for a moment that the interviewer had said to me “tell me about a time where you learned something about woodworking”. I would not easily be able to come up with an example on the spot even though I have had many experiences with woodworking throughout my life. But, if you asked me “how would you build a table with X, Y, and Z properties” I would be able to give you a lengthy explanation of drafting a design, measuring twice in the real world, creating mortise & tenon joints, using appropriate clamping with glue, and much more. At one point all of this information was learned through experience but for whatever reason I couldn’t tell you when each process was learned or what I was doing when I learned it. Knowing how to build a table is semantic memory; knowing when you learned each important part of building one is the episodic memory. If you’re evaluating someone for job placement it should be clear that you want to know how they will perform the job not whether they recall the circumstances around the last time they performed it.

There are three types of people who excel at answering “tell me about a time when” questions: people who have the narrow but relevant experience and have a great memory for the events of their life, people who are self-absorbed (“Takers” as Adam Grant would say²), or worse a person who can spin a yarn that fulfills all of your requirements without a thread of truth (a “fleecer”). Takers are the people in organizations who quietly attribute all of the successes around them to their own actions and are far more likely to have intentionally filed these moments away as “personal wins” to later be used in an interview or promotion. According to Adam Grant, Takers are the employees that actually generate the least overall productivity in a company, though by themselves they may look like top performers. Furthermore, you probably don’t want to hire a fleecer unless you are hiring a magician’s assistant or a person for an improv troupe. Can these types be differentiated during an interview? Probably not. You’re already working with sparse information which makes it easier to be duped by a believable fiction. Many companies have a “no assholes rule” following the adage made popular by Bob Sutton³, but these questions give potential Takers a key competitive advantage in the interview process.

As a key business stakeholder or business owner, wouldn’t you rather find out if a person knows how to do a job rather than just finding out how well their episodic memory works? Circling back to the original questions the interviewer asked me, here they are using the format the they could have used:

Lead-in: Imagine for a moment you’ve just been promoted to the CEO at a tortilla chip factory whose profits have slowly been declining as more competitors enter the market. The board of the company has told you they expect sales to stabilize in the long run, but it’s your job to decide what to do with the company. Question 1: The company food chemist comes to you with a great idea for a new flavor they think will revolutionize chips. Unfortunately, your company doesn’t have the equipment to produce the flavor so there would be a sizable upfront investment and might put the company in financial trouble. Would you choose to pursue the new flavor or do something else? What questions would you need to answer in order to make your decision? Question 2: What would it take for you to consider funding the new chip flavor? As the CEO, you’re able to request any research required but it will take time. Question 3: Morale at the company is low leading to a higher churn rate. The new employees that fill the positions tend to be more junior and work less efficiently. What would be your strategy for improving morale and worker productivity?

The first question focuses on overall company direction and making a big decision for the company. Does the candidate opt to minimize costs in order to maximize profit or do they opt for the bold choice of investing in the new chip flavor? By getting them to answer this question you can find out whether they are more likely to take a bold action or go with a more conservative course of action, which is what the original version of this question was seeking to answer. The added benefit of this question format is you can glimpse their thought process and may even get a creative response you hadn’t thought of yourself.

The original intent of the second question was to assess whether the person would take a leap of faith based on their gut instinct. Depending on your company culture you’d be looking for wildly different answers, but you would be able to determine how much or how little information a person requires to feel confident making a decision. This question can also give you a glimpse into the types of data collection the person typically relies on. With so many options like social media scraping, in-person focus groups, website polls, landing page tracking, etc you may hear something new that surprises you.

The original third question honestly isn’t fair. During all my years in analytics the number of people I’ve known who had opportunities for this to happen was zero. Unless you’re working at a startup, where there is minimal friction between having a radical idea and pivoting the company toward it, you’re never likely to run into this situation. Instead, the newer question focuses on a very real and common pain point which is improving morale and training new employees. If you’re hiring someone to be a people manager you want to know whether they would have a plan of action for training up junior employees. Mentoring and making sure all the junior folks feel like they are engaged and growing is essential for a positive company culture.

If you’re trying to ask questions that illuminate a candidate’s qualification more directly, try re-framing the question. Rather than asking for the candidate to recall a specific memory matching narrow criteria put them into a virtual scenario instead. You should craft a narrative and ask the candidate how they would handle various parts of the situation. Some of the best situational questions can have elements drawn from disasters in your current company or previous companies. Even if the candidate doesn’t answer the question the way you expected their problem solving approach, intuition, or external knowledge may surprise you in ways you might not have imagined; you may also find out that the candidate doesn’t have enough knowledge on the subject to craft a fitting response. In either case, there are insights to be gained if you, as the interviewer, take a bit more time to create thoughtful and immersive questions. These insights will give you a clearer picture of the candidate’s core knowledge and instincts, and they will allow you to make better hiring decisions.