L&D Detective Kit
for Solving Impact
Mysteries

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Kevin M. Yates,
The L&D Detective
Known in the global learning and development (L&D) community as “the L&D
detective,” Kevin M. Yates answers the question, “What is the impact of learning?” Hesolves measurement mysteries and investigates how learning activates performanceand business goals using facts, clues, evidence, and data.
Kevin’s career in the learning and development profession includes over 20 years ofexperience in local and global roles for facilitation, instructional design, learning
solutions design, learning technology, curriculum development, program management,
leadership development, learning operations, learning analytics, and impact analysis.
Kevin has experience working in industries including education, financial services,
insurance, news media, marketing, professional services, food services, and social
media.
His reach is worldwide through in person and virtual keynote presentations, panel
discussions, and workshops across the globe. Kevin also shares his experience andexpertise for measuring impact in industry magazines, online articles, blogs, podcasts,
and live streams.
In his own words, “My message is firmly rooted in the belief that measuring impact
is possible, necessary, and actionable. I want to inspire the global learning anddevelopment community to have a mindset for measurement. I tell the truth about
challenges for measuring impact and share methods that make it possible.”L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Table of Contents
Impact Mysteries001

Impact Standards

003

Impact Opportunity Interview

006

Case Study 1: Impact Opportunity Interview for Global Gems

012

Case Study 2: Impact Blueprint

015

Impact Design

017

Case Study 3: Impact Design for Global Gems Learning Solution

021

Case Study 4: Global Gems Learning Solution – Ready for Impact or Not?

023

Learning Impact Survey

025

Case Study 5: Learning Impact Survey for Global Gems

031

Facts, Clues, Evidence, and Data for Impact

034

Case Study 6: Impact Investigation for Global Gems

035

Impact Investigation Results

043

You, Too, Can Be an L&D Detective046

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Impact Mysteries
I am a learning and development detective, and I solve mysteries. I solve
the mystery: What is the impact of learning? I use facts, clues, evidence,and data to measure outcomes and results.

L&D Detective Guiding
Principles#1 Performance is the North Star for impact
investigation.

My confidence in the value of L&D detective work comes from my beliefs.
I believe learning fulfills its highest purpose with performance activation.
For me, L&D detective work is measuring fulfillment of purpose.

#2 Impact is intentional, specific, purposeful,and targeted.
#3 If you don’t plan for impact in the beginning,
it will be difficult to measure in the end.
#4 Find one thing about performance you can
attribute to learning and let that lead to thefacts for impact.
#5 Investigating impact may not be easy, but it
is possible.

I do not believe you can indiscriminately attribute change in performanceor achieving a business goal solely to training. There are multiple
conditions and inputs that activate performance and achieve business“My role as an L&D detective

goals. However, I do believe you can produce a compelling preponderance

is uncovering when a changeof evidence that demonstrates how learning activates performance andin performance or thedrives business goals by influencing behavior change.

achievement of a businessgoal is uniquely influenced
by learning.”

Natural ability, motivation, rewards, recognition, manager support, tools,

L&D detective work can be complicated and tricky. Discoveringand technology – to name a few – activate performance. Training andfacts, clues, evidence, and data that demonstrate impact

learning are just two contributors to a multitude of activators. My role

can be complex and challenging. I am sharing my L&D

as an L&D detective is uncovering when a change in performance or thedetective techniques to reduce complexity and make impact

achievement of a business goal is uniquely influenced by learning.

measurement mysteries easier to solve.

001

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Impact = Activation

These guiding principles give L&D detective work its purpose and meaning.

Investigating impact is as much about examining what

They are the foundation for investigating impact and solving measurement

happened as a result of training as it is about examining what

mysteries. You will see elements of these guiding principles throughout thewould not occur in its absence. Can you produce unequivocal

L&D detective kit.

evidence that something happened strictly because of
training? I do not believe you can. Can you produce evidence

What is Impact?

for learning’s influence on performance, behavior, actions, andbusiness goals? I believe you can.

L&D detective work is centered on a very specific definition of impact. Impact
is when learning activates performance and a business goal. Impact is
activation.
All learning solutions are not created with the same purpose. There aretraining or learning solutions intended to create awareness, build knowledge,
demonstrate compliance, mitigate risks, or fulfill professional development
requirements. Activating performance is not always the goal, and that’s okay.

What’s Next?

However, you must be on the same page with business partners, internal

What are the characteristics of learning solutions that haveclients, and stakeholders when you define and manage expectations forpotential for impacting performance and business goals? How

impact. A 60-minute lunch and learn or 30-minute eLearning module is

do you know if you should investigate impact, as we defined it,

not likely to have quantifiable impact. Do not try to measure impact, as weor not? The next section illustrates how to build the case for an

defined it, for these. Measure fulfillment of their purpose instead.

impact investigation with impact standards.

002

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

ImpactStandards
How do you know which learning programs and solutions are eligible for impact investigation? Which training programs require thetime, resources, and effort it takes to measure impact? Is the learning solution for which you are trying to measure impact designed with
the intent to activate performance and meet a business goal?
Impact standards build the case for measuring the impact of learning. Standards ensure that L&D detective work is focused on learning
solutions purposefully designed for impact. Standards also ensure there is opportunity for impact.

Priority

Payoff
Are there significant
investments for time,money, or both?

Does it have leadership
attention, visibility, orsponsorship?

Power

Position

Does it have support for
activating performance?

Pinpoints
Does it have measuresor key performanceindicators (KPIs)?

ImpactStandards

Is there alignment with abusiness goal or strategy?

PurposeAre there specifictargets for performanceoutcomes?

Priority
Investigate

Does it have leadership attention, visibility, or sponsorship?

Intention

Achieving business goals is the top priority of leadership and should be the priority of L&D.
When leaders champion employee development with their endorsement and messaging,
evidence for impact is necessary. When leaders support investment in employee development
for achieving a business goal, impact investigation is required.

003

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Position
Investigate

Is there alignment with a business goal or strategy?

Intention

Alignment means intentional connection. It means strategic links are present for training,learning, and the business goal. When there is agreement that learning is part of the winning
strategy, there’s eligibility for an impact investigation.

PurposeInvestigate

Are there specific targets for performance outcomes?

Intention

Performance targets are not the knowledge or understanding acquired from training. A
performance target is a behavior, action, or capability that achieves the business goal.
Training solutions with specific performance targets are eligible for impact investigation.

Pinpoints
Investigate

Does it have measures or key performance indicators (KPIs)?

Intention

It is difficult to measure impact without specific indicators to show results. Measures andkey performance indicators are evidence for impact. A learning solution must have specific,
defined measures to be eligible for an impact investigation.

Power
Investigate

Does it have support for activating performance?

Intention

A single event may activate performance. However, a learning experience with reinforcement
and continuous support is more likely to change behavior and actions. If the training solution
has on-going triggers in place to activate performance over time, it is eligible for an impact
investigation.

PayoffInvestigate

Are there significant investments for time, money, or both?

Intention

There are three ways to think about investments in training: monetary costs for designand delivery, time for design and delivery, employee hours invested in learning. The moreinvestment, the greater the expectation for return on investment. Learning projects with
substantial investments in time, money, or both are eligible for an impact investigation.

004

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Do All Standards Have to be Met to Conduct an Impact Investigation?
There is no one right answer to this question but consider this: Impact standards are clues about the extent to which a learning solution
was designed with the purpose of impacting performance and business goals. The fewer the standards met, the less likely it is that there’s
potential or opportunity for impact.
Here’s why this is important: Training and learning solutions without impact standards are difficult to measure. Before you try to measure impact,
use the standards to ensure there’s purpose and intention for impact first. You can also use impact standards to manage expectations for training
and learning solutions that are most appropriate for measuring how learning activated performance and a business goal.

What’s Next?
How do you proactively plan for impact standards? What are the steps for ensuring there are clearly defined intentions and measurable
outcomes for training? The next section demonstrates how to conduct an impact opportunity interview for proactive impact planningprior to the design and deployment of your learning solution.

005

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Impact Opportunity
Interview
Impact Opportunity Interview
The most prevalent challenge for impact investigations is not uncovering impact expectations before the learning solution is deployed.
Unfortunately, training programs are usually designed without specific targets for performance outcomes and business goals. Impact isan afterthought, and that makes measuring impact difficult.
Training is often presumed a solution before collecting information about the business opportunity or problem at hand. Learning anddevelopment teams do not have magic wands, and there are times when L&D does not have the appropriate solution. The impact
opportunity interview proactively reveals when learning has potential for impact, as well as when learning is not the solution.

“The impact opportunity interview transforms the view, theengagement, and the role of L&D from order takers to impact
makers!”Some view learning and development as a transactional fulfillment center where business functions go to place orders, “I’d like one virtual
classroom training with a side order of digital learning to go, please,” and L&D takes and delivers “the order”. The impact opportunity interview
transforms the view, the engagement, and the role of L&D from order takers to impact makers!

Core

Condition

Contribution

What is the businessgoal?

What is the opportunityor problem to solve?

Who is supporting thebusiness goal?

Community

Capability

Comparison

Whose performance isneeded?

What are the performancerequirements?

What is the performancedifference?

Causation

Calculation

What are theperformance drivers?

What are the measures
and KPIs?

CautionWhat are the risks?

006

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Core
Investigate

What is the business goal?

Intention

Identifies business focus and priority.

Insight

There is a difference between a business goal and a training goal. Accelerate growth by 20%,
reduce errors by 30%, improve customer ratings from 3 points to 5 points are examples ofbusiness goals. Design five courses for 2,500 employees by the end of the third quarter is a
training goal. The answer to this question about the business goal identifies the priority andthe measure for success for the business goal.
The best-case scenario is a business goal with a quantified target: i.e. 10% growth, 2-point
increase, or one-hour reduction. However, if the goal is simply “improve errors,” how do you
know when errors are sufficiently improved? A quantified goal defines success.

Condition
Investigate

What is the opportunity or problem to solve?

Intention

Creates perspective, purpose, and context for the business goal.

Insight

Is there opportunity for growth? Is there a need to correct a situation or change a process? Domarket conditions require a specific business response? The answer to this question about
condition frames the purpose of the business goal. There’s also opportunity to contextualize
the learning experience.

Contribution
Investigate

Who is supporting the business goal?

Intention

Reveals cross-functional partners that have ownership for achieving the business goal.

Insight

It takes a team to win. The idea that training alone will achieve a business goal is a false
assumption. The answer to this question reveals cross-functional partners who play an activerole in achieving the business goal.
Identifying cross-functional partners manages expectations for learning’s role. There isalso opportunity for removing redundancy in efforts and supporting each other’s effort.
Contribution to goal – demonstrated in the figure on the next page – is an agreed uponestimate of each partner’s input.

007

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Marketing20%
Research &Development
35%
Technology
15%

Operations
20%

Human
Resources
10%

Example of Contribution to Goal

20% contribution from marketing + 15% contribution from technology + 10% contribution from human resources + 20% contribution fromoperations + 35% contribution from research and development = 100% contribution to goal.

Community
Investigate

Whose performance is needed?

Intention

Identifies people and teams, based on role, who are needed to achieve the business goal.

Insight

Who are the people expected to help achieve the business goal? What are the dynamics anddemographics of their roles? How and where do they work? The answers to these questions ensure
clear focus on and understanding of the target audience and informs decisions for program design.

Capability
Investigate

What are the performance requirements?

Intention

Reveals specific capabilities needed to achieve the business goal.

Insight

Performance requirements are not what people need to learn, know, or understand. Performancerequirements are the behaviors, capabilities, and actions that achieve the business goal. The
answer to this question uncovers what people must do to achieve the business goal.
The interviewee may struggle with answering this question. He or she may start with, “We wantpeople to know…,” “We need people to understand…,” or “People need to learn…” Instead, try
using these prompts:• What does understanding or knowledge look like when it shows up on the job?
• If someone knows or understands, what do they do?
• What do effective behaviors and actions look like?
Performance requirements must be specific, targeted, and clearly defined.

008

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Example ofMeasuresand KPIs

Market share

Comparison
Investigate

What is the performance difference?

Intention

Uncovers the distinction between current capability and the capability
necessary to achieve the business goal.

Insight

What evidence shows that people are not performing or behaving as
needed to achieve the business goal? What is the difference between where
performance is and where it needs to be? The answers to these questions
help determine when learning is a potential solution. These answers alsobuild the foundation for measuring impact.

Employee
performance

Causation
Customersatisfaction

Operations

Investigate

What are the performance drivers?

Intention

Identifies variables that influence performance.

Insight

There are a variety of inputs that activate performance for achieving thebusiness goal. All inputs must be taken into consideration to ensure they
are activated accordingly. The answer to this question – demonstrated in
the figure below – manages expectations for where learning can influence
performance, where it cannot, and where L&D can support and reinforce
performance activators.

QualityTime

Calculation
Investigate

What are the measures and KPIs?

Intention

Reveals metrics and data that demonstrate results, movement toward
goal, and success.

Insight

What are the key performance indicators (KPIs) that measure progress toward
the goal? Are there metrics that demonstrate the connection between people
performance and business performance? Who owns the data? The answers tothese questions are the foundation for evidence and data that reveal impact
and results.

GrowthErrorsVolumeEngagement

009

Points of consideration for data include accessibility, confidentiality, andGeneral Data Protection Regulation (GDPR). You may be able to aggregate
and anonymize data to maintain privacy and confidentiality. Additionally,
consider timing and the support necessary from data owners for reporting,visualization, and analysis.

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Example of Performance Drivers

Manager Support

Accountability

Rewards & Recognition

Tools & Technology

Performance

Engagement

Natural Ability

CautionInvestigate

What are the risks?

Intention

Uncovers threats to performance for achieving the goal.

Insight

Are there potential obstacles to or inhibitors of performance? Is there something that may preventthe business from achieving the goal that has nothing to do with performance? Identifying threats
informs decisions for mitigating and addressing barriers to performance.

0010

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

The L&D Detective Says…

A concern I often hear regarding impact opportunity interviews
is: “My internal clients and business partners don’t have time forme to ask these questions. I can barely get 30 minutes of theirtime. They just want me to deliver the training solution they
believe they need. How can I have this kind of discussion withlimited time and attention?”

This question is a legitimate concern. Here are my thoughts:• Send the impact opportunity interview questions in advance.
• Ask the interviewee which questions they can answer, which ones they cannot, and direction to the people who can.
• Let them know you can take their “request for training,” but doing so minimizes opportunity for a learning solution
with measurable impact.

• If impact is not the goal and there is another purpose for the training, you have managed expectations accordingly.
• If operating at the speed of business does not allow for time to discover impact opportunity, be honest andcourageous when telling the truth about expected results and outcomes.

And remember this:• Conduct the impact opportunity interview before the training solution is designed and deployed.
• The discussion is firmly rooted in curiosity and discovery and should focus on collecting facts about theorganizational and performance requirements to achieve the business goal.

• If the discussion shifts to the learning solution, you are having the wrong discussion.
• You will more than likely need to interview more than one person to get all the answers.
• The discussion is a shift from L&D as an order fulfillment center for training requests to L&D as a consultative andtrusted advisor to the business.

What’s Next?
Results from the impact opportunity interview reveal when training is a solution and when it is not. There’s also discovery for impact opportunity.
The next section illustrates results from an impact opportunity interview with a case study.

0011

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Case Study 1: Impact
Opportunity Interview for
Global Gems
Global Gems is a luxury brand with a 100-year history in the precious stones
industry. It has stores in Africa, Asia, Australia, Europe, Latin America, andNorth America catering to wealthy customers looking for one-of-a-kindjewelry and a high-end customer experience. The business has beensuccessful and profitable over the years but is seeing a decline in sales
across all stores

Impact Opportunity Interview Results
Mustafa Nysem, director of global sales, requested training for the sales
team. He owns the business goal for growth. The L&D team conducted an
impact opportunity interview. Here are results:

Core

What is the business goal?
30% growth

Condition

What is the opportunity or problem to solve?
Sales are down 15% from prior year, and customer satisfaction scoresare declining. Post-sales feedback from customer phone interviews,
comments on the website, social media, and consumer websites
indicate customers are not having good experiences. Store interiors
are outdated, sales transactions are cumbersome, and product lines
have decreased over the years. The brand is perceived as catering toolder clientele, and salespeople show signs of apathy and ambivalence
regarding customer care.

Facilities
15%

Marketing
20%

Products
30%

Contribution

Marketing owns rebranding. Technology owns implementation oftablets for mobile point-of-sale transactions. Human resources owns
recruiting for sales roles. Products owns sourcing for new jewelry.
Facilities owns store remodeling. The goal owner and contributors
have estimated and agreed upon contribution to goal.

Technology
20%

Human
Resources
20%

Cross-Function Contribution to Goal

Who is supporting the business goal?

Community

Whose performance is needed?
Gem care specialists interact closely with customers and are theprimary source of sales growth. They are spread across the globe
working in different countries, different languages, and different cultural
norms. The average gem care specialist has over 20 years of experiencein the same store with comprehensive industry expertise in gemology.

0012

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Capability

What are the performance requirements?
There are four capabilities for customer interactions that lead to a sale.

Capability

Comparison

Performance Requirement

Product knowledge

Describe make, value, origin, and authenticity.

Intentional engagement

Actively listen to customers, and interpret
non-verbal clues.

Managed persuasion

Assess interest, evaluate doubts, addressconcerns, and drive desire.

Luxury care

Create a personalized and luxurious in-store
experience.

What is the performance difference?
Results from customer satisfaction scores show current performance:

Capability

Current*

Required

Product knowledge

55Intentional engagement

2.1

5Managed persuasion

2.9

5

Luxury care

2.9

5

*Average customer satisfaction score for all stores.

Causation

What are the performance drivers?
Gem care specialists performance influences include compensation,commission, store manager relationship and support, experience, sales
capability, motivation, and inclusion in decision-making.

Calculation

What are the measures and KPIs?
• Quarterly store sales.
• Customer satisfaction scores.
• Employee engagement scores for investment in development.
• Employee engagement scores for belief in company’s future.
• Merisol Osinga owns quarterly sales data.
• David Lenox owns customer satisfaction data.
• Barbara Ng owns employee engagement data.
• Jafari Adebayo supports data science and reporting.

CautionWhat are the risks?
• The market for luxury jewelry is shrinking, but the businessbelieves it can achieve growth with a high volume of lower-priced items
versus a low volume of higher-priced items.
• A reduction in sales commission may impact motivation to sell.
• Employee engagement scores indicate low confidence in Global
Gems future.

0013

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Impact Opportunity Decision Tree
Results from the impact opportunity interview reveal when training is a potential solution.
The impact opportunity decision tree helps inform the direction moving forward.

Is training or learning
the solution?

Yes

No

Do we already have

What is thean existing solution?

recommendation?

Yes

No

Recommend the

Do we buy, build, or

existing solution.

both?

What’s Next?
The Global Gems L&D team evaluated results from the impact opportunity interview and determined that learning can support thebusiness goal. They have prepared a summary with recommendations for the business goal owner. The next section illustrates theirfindings and their proposal for the learning solution in an impact blueprint.

0014

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Case Study 2: Impact
Blueprint
Background
Global Gems is seeing a decline in sales and has a growth goal of 30%.
The strategy for growth includes contribution from cross-functional
teams, including marketing, technology, human resources, products, andfacilities. Gem care specialists are the primary source of sales growth andPerformance Targets
Capability

Performance
Requirement

Performance Measure(Customer Satisfaction
Target Score)

Intentional
engagement

Actively listen to
customers, and interpret
non-verbal clues.

5Managedpersuasion

Assess interest, evaluate
doubts, address concerns,
and drive desire.

5

Luxury care

Create a personalized
and luxurious in-store
experience.

5

are essential for achieving business and performance goals.
Customer satisfaction scores, comments on the website, and consumer
websites show customers are not having positive in-store experiences.
There are four capabilities critical for sales success: product knowledge,
intentional engagement, managed persuasion, and luxury care. Gem
care specialists are strong in product knowledge and have opportunities
for improvement in intentional engagement, managed persuasion, andluxury care based on customer satisfaction scores.

Impact Opportunity
Findings from the impact opportunity interview reveal that L&D cancontribute to the growth goal by providing learning solutions focusedon customer experience. Learning solutions will strengthen gem

Impact Measures and KPIs
Business Performance and People Performance Measures

Measure/KPI

Measure Type

Goal

Report Timing

Quarterly sales

Business

20% growth

End of year

Customer satisfaction scores

People5 points

End of year

Employee engagement scores*

Business

80% Agree/
Strongly Agree

Beginning of nextyear

* Results for “The company invests in development that helps me succeed in myrole,” and “I believe in the company’s future.”

Data Owners and Partners
Datacare specialists’ performance in intentional engagement, managed

Owner/Partner

Quarterly sales

Marisol Ozinga

Customer satisfaction

David Lenox

product knowledge. Solutions include a mix of training and performance

Employee engagement

Maxine Ng

support. Pilot-testing in Africa is recommended before global launch.

Data science and reporting

persuasion, and luxury care and leverage their current expertise in

Piloting allows for evaluation of the solutions’ effectiveness and impact
before launch to other markets.

Jafari Adebayo

Learning Performance Measures
Measure/KPI

Goal

Business Goal Alignment

Performance impact*

Support 30% growth goal by empowering gem care specialists with

Employee-manager engagement*

90% Yes

capability that drives sales through optimal in-store customer experience.

Business goal alignment*

90% Yes

80% top 2 boxes

Frequency of use*

80% top 2 boxes

Business simulation average score

85% mastery*Post-program and 30-day follow up learning surveys

0015

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

L&D Contribution to Goal
Based on consultation and agreement with other cross-functional partners, L&D estimated contribution to goal is 15%.

Facilities
15%

Marketing
20%

L&D
15%
Technology
20%

Products
25%

Human
Resources5%

Impact Blueprint Sign-Off

Mustafa Nysem
Mustafa Nysem
Director of Global Sales

Tamara DevereauxTamara DevereauxChief Learning Officer

Learning Solutions• Virtual classroom training programs.
• Digital business simulation.
• Mobile-enabled performance support, including videos, podcasts, andinteractive PDFs.

• Leadership videos explaining the business goal and alignment with learning
solutions.

• Discussion guides for managers and employees.

What’s Next?
The Global Gems L&D team has achievedagreement to move forward with the• Coaching guides for managers.
• Learning support discussion groups.

What Do We Accomplish with the Impact Blueprint?

impact blueprint. They are aligned with the• Confirmation that L&D can contribute to the business goal.

business goal owner on performance targets,• Proactive impact planning.

outcomes, measures for impact, and timing• Alignment with the business goal and specific targets for performance.

for results. The next section illustrates impact• Measures and KPIs for impact.

design for the learning solutions.

• Agreement for impact with the business goal owner.

0016

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

ImpactDesign“Impact begins with
instructional design that
activates performance.”

How do we activate performance with learning experiences? Can we leverage and makeactionable what we discovered from the impact opportunity interview? What are thedesign implications?
Impact is intentional. There are instructional design techniques that prompt behavior,
motivate actions, and influence performance that achieves the business goal. Elementsof impact must be built into instructional design.
Impact design uses results from the impact opportunity interview to create a learning
experience purposefully designed for impact. Learning outcomes produce performanceoutcomes that achieve the business goal. Impact begins with instructional design that
activates performance.

Impact Opportunity
Interview

Interview
Results

ImpactDesignWhat are the Benefits for Impact Design?
• Impact embedded in the learning experience with instructional design.
• Content aligned with the business goal.
• Messaging and communication in support of learning’s intention and purpose for impact.
• Specific targets for performance impact.
• Measurable results for instructional design effectiveness.
0017

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved0018

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reservedCore

What is the business goal?
How will you reinforce alignment between the business goal and the learning solution?
• Create videos with leaders describing the business goal and illustrating alignmenzt with the learning solution.
• Describe learning alignment with the business goal in marketing messages and learning solution description.
• Create script for the facilitator or narrator that details the connection between learning and the business goal.

Condition

What is the opportunity or problem to solve?
How will you contextualize the situation for the business goal as part of the learning experience?
• Describe the conditions that prompted the business goal in marketing and communication messages for the learning solution.
• Use circumstances for the business goal to create similar scenarios in the learning experience.
• Create script for the facilitator or narrator that explains the circumstances prompting the business goal.

Contribution

Who is supporting the business goal?
How will you collaborate with cross-functional partners who also support the business goal?
• Identify what other contributors are doing to support the business goal.
• Examine opportunities to remove redundancies.
• Explore where learning can leverage and support other business goal contributors.

Community

Whose performance is needed?
What are the design considerations for the target audience?
• Identify local language needs.
• Create scenarios and simulations that reflect unique cultural norms, or design the learning experience with a broad
context for scalability.
• Consult with target audience representation groups for feedback on making content relevant, meaningful, and actionable.

Capability

What are the performance requirements?
How will you connect learning outcomes with performance targets?
• Use performance outcomes versus learning objectives.*• Describe performance outcomes in marketing messages and the learning solution description.
• Create script for the facilitator or narrator describing performance goals.
• Use video to illustrate behaviors and actions that achieve the business goal.
• Demonstrate how to address barriers to performance and what happens when performance is activated on the job
with real-world scenarios.
• Use role-plays and simulations to practice behaviors and build confidence.

0019

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved*Performance Outcomes versus Learning Objectives
Traditional learning objectives begin with, “You will learn…”, “You will know…”, or “You will understand…” Performance outcomes are action-based
and describe the skill or capability.

Traditional Learning Objectives

Performance Outcomes

You will understand the importance of effective
communication.

Remove communication barriers when working with people whohave a style of communicating different than yours.

You will know how to handle conflict.

Manage and resolve conflict by interpreting behavior instead of responding
emotionally.

You will learn the importance of emotional
intelligence.

Adapt your behavior when working with people who have a style ofworking that is different than yours.

L&D detective guiding principle number one is, “Performance is the North Star for impact investigation.” Impact is when learning
activates performance and the business goal. Performance outcomes are the focus for impact investigation.

Comparison

What is the performance difference?
How will you address the need to move performance from where it is to where it needs to be?
• Use video to illustrate the difference between performance that achieves the business goal and behaviors that inhibit achievement of
the business goal.
• Use role-plays and simulations to practice behaviors and build confidence.
• Demonstrate the difference in outcomes when behaviors that support the business goal are practiced and when they are not.
• Create a script for the facilitator or narrator that reinforces the critical need for applying what was learned to achieve the business goal.

Causation

What are the performance drivers?
What strategies will you use to activate performance influences within L&D’s control?
• Identify drivers where learning or performance support can impact performance outcomes.
• Create supplemental solutions to activate performance.

Calculation

What are the measures and KPIs?
How will you consider data for learning performance, employee performance, and business performance?
• Create scripts for the facilitator or narrator that explain impact metrics for employee and business performance.
• Include an illustration in the learning experience that demonstrates the connection between learning, performance outcomes, thebusiness goal, and impact measures.
• Use learning surveys to measure learning effectiveness.
• Create a script for the facilitator or narrator that encourages completion of the learning survey.
• Use business simulation scores to measure knowledge.
• Use performance support accesses to measure utilization.

CautionWhat are the risks?
How will you address threats to performance?
• Identify potential performance blockers.
• Address as needed in the learning solution.

0020

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Case Study 3: Impact Design for
Global Gems Learning Solution
Core

What is the business goal?
How will you reinforce alignment between the business goal and the learning solution?
• Create videos from Aliah Sandeep, chief executive officer, and Mustafa Nysem, director of global sales, explaining
how training will help achieve the business goal.
• Include messaging in program description that explains how learning is connected to the business goal.
• Create facilitator script that details alignment between the business goal and learning.
• Illustrate through examples and role-plays what happens when employees use behaviors and performance that
help achieve the business goal.

Condition

What is the opportunity or problem to solve?
How will you contextualize the situation for the business goal as part of the learning experience?
• Create scenarios and simulations in the learning experience that replicate current business conditions.
• Illustrate the impact of targeted performance outcomes on current business conditions.

Contribution

Who is supporting the business goal?
How will you collaborate with cross-functional partners who also support the business goal?
• Leverage mobile point-of-sale technology for digital business simulations.
• Align with marketing on new branding strategy.
• Describe cross-functional contribution to business goals in the learning experience.

Community

Whose performance is needed?
What are design considerations for the target audience?
• Pilot training program and learning solutions with gem care specialists in Africa stores.
• Localize language in learning solutions.
• Consult with gem care specialists in Africa for input on cultural considerations.
• Identify gem care specialist content partners in Africa for content development, employee perspective, andrelevance.

Capability

What are the performance requirements?
How will you connect learning outcomes with performance targets?

Performance Target

Performance Outcome

Intentional engagement

Actively listen to customers, and interpret nonverbal clues.

Managed persuasion

Assess interest, evaluate doubts, address concerns,
and drive desire.

Luxury care

Create a personalized and luxurious in-store
experience.

• Illustrate what behaviors and performance look like with videos and role-plays.
• Provide examples of what happens when behaviors and performance are utilized effectively
0021

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Comparison

What is the performance difference?
How will you address the need to move performance from where it is to where it needs to be?

• Describe consequences when performance required to achieve the business goal is not used.
• Illustrate examples of customer experience when performance required to achieve the business goal is not
used.

• Compare and contrast desired behaviors and performance with behaviors and performance that threaten thebusiness goal.

Capability

Current

Required

Intentional engagement

25Managed persuasion

3

5

Luxury care

35Causation

What are the performance drivers?
What strategies will you use to activate performance influences within L&D’s control?
• Create pre-program discussion guides for managers and employees to set expectations.
• Create post-training discussion guides for managers and employees to create accountability for applying
learning on the job.
• Create a coaching guide for store managers.
• Design post-training program performance support with simulations, videos, podcasts, and discussion
groups.

Calculation

What are the measures and KPIs?
How will you consider data for learning performance, employee performance, and business performance?
• Illustrate the employee and business performance measures targeted for impact.
• Measure learning effectiveness with learning surveys.
• Measure employee knowledge acquisition with business simulation scores.

CautionWhat are the risks?
How will you address threats to performance?
• Using videos and role-plays, include examples of success and impact on the business goal when learning isapplied to address employee engagement scores for low confidence in the future of the business.

0022

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Case Study 4: Global Gems
Learning Solution – Ready for
Impact or Not?
The Global Gems L&D team did a great job with the impact opportunity investigation, impact blueprint, and impact
design. But does the solution meet the standards for an impact investigation?

Priority

Payoff
Are there significant
investments for time,money, or both?

Does it have leadership
attention, visibility, orsponsorship?

Power

Position

Does it have support for
activating performance?

ImpactInvestigation

Pinpoints

PurposeDoes it have measuresor key performanceindicators (KPIs)?

Investigate

Is there alignment with abusiness goal or strategy?

Are there specifictargets for performanceoutcomes?

Yes

No

Evidence

Priority

Does it have leadership
attention, visibility, orsponsorship?

Leadership videos describing connection between
learning solutions and the business goal.

Position

Is it aligned with a business goal or
strategy?

Learning solutions with targets for behaviors andperformance that support 30% growth goal.

PurposeAre there specific targets for
performance outcomes?

Performance for intentional engagement,
managed persuasion, and luxury care.

Pinpoints

Does it have measures or key
performance indicators (KPIs)?

Quarterly sales, customer satisfaction scores, andemployee engagement scores.

Power

Does it have support for activating
performance?

Manager-employee discussion guides, coachingguides, videos, podcasts, business simulation, andinteractive PDFs.

Payoff

Are there significant investments
for time or money?

Costs for video production and costs for vendor
digital learning solutions.

0023

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

The L&D Detective Says…

Have you noticed that, up to this point, I have heavily focused on planning for impact?
The third L&D detective guiding principle is, “If you don’t plan for impact in the
beginning, it will be difficult to measure in the end.” This is where I see L&D teams
struggle the most. There is no plan for impact, so it is difficult to measure.
Impact is a mindset, and it is intentional. The traditional view of L&D is that thecourse or training event we create is the deliverable. However, I believe impact is thedeliverable.

“I have found the bulk of time spent investigating impact isplanning for impact.”
By sharing my L&D detective techniques for proactive impact planning and design, I hope you are better prepared tomeasure the results of your learning solutions. I have found the bulk of time spent investigating impact is planning for
impact. It is time well spent, and lays the foundation for successful impact measurement and investigation.

What’s Next?
We have seen how results of an impact opportunity interview align learning with the business goal, sets targets for
performance, and determines measures for impact. We’ve also seen how results from the interview are used for impact
design. In the next section, we will discover how a learning impact survey uncovers clues for learning effectiveness.

0024

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Learning Impact
SurveyWhat kind of clues for impact do you get from post-training surveys? Can you estimate the impact of learning with survey results?
What makes survey data actionable versus nice to have?
A learning impact survey measures learning effectiveness and predicts impact. Clues from these surveys may not necessarily
represent actual impact, but they are a good preliminary indication of expected outcomes and results. You also get clues regarding
learning effectiveness and feedback on instructional design. The clues you get from a learning impact survey help answer
questions such as:• How much of what people learned can we expect will be used on the job?
• Is the connection between performance and achieving the business goal clear?
• Are there barriers inhibiting people from using what they learned?
Unlike traditional surveys that ask whether participants liked the instructor, the facilities, or enjoyed the experience, learning
impact surveys estimate performance activation.

Learning Impact Survey Questions

Performance Impact
Ask

How much do you believe this learning experience will impact your performance?
None at all				A lot

A little					A great deal
A moderate amount
Intention

Measure belief in the learning experience’s ability to activate performance.

Investigate

Does the learning experience illustrate real-world examples of using what was learned and inspire
motivation to apply what was learned on the job?

0025

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Applied Learning
Ask

How much of what you learned do you believe you will use?
None at all				A lot

A little					A great deal
A moderate amount
Intention

Measure expected amount of activated learning.

Investigate

Does the learning experience illustrate results for activating performance on a scale of low to high use,
including success factors and how to address potential performance barriers?

Business Goal AlignmentAsk

Do you see the connection between using what you learned and achieving the business goal?
Yes
No
Not sure

Intention

Measure agreement and clarity on how activated performance helps achieve the business goal.

Investigate

Is the connection between learning and achieving the business goal described in messaging,communication, and examples throughout the learning experience?

Performance Readiness
Ask

Were there enough opportunities for interaction, activities, and practice for you to feel confident
applying what you learned?
YesNo
Not sure

Intention

Measure belief in ability to use what was learned on the job.

Investigate

Does the learning experience include real-world examples using role-play, feedback, andreinforcements that support performance readiness?

Employee-manager Engagement
Ask

Did you and your manager discuss expectations for this learning experience?
Yes
No

Intention

Measure manager-employee engagement.

Investigate

Are there resources to support discussion between managers and employees about
learning goals and expected performance outcomes?

0026

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Usefulness
Ask

How will this learning experience help your performance?
Comment:

Intention

Compare responses with intended performance outcomes. Evaluate unanticipated outcomes.

Investigate

Recurring themes from responses.

Learning Blockers
Ask

Is there anything about the learning experience that was not helpful?
Comment:

Intention

Evaluate opportunity to improve the learning experience.

Investigate

Recurring themes from responses.

0027

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

30-Day Follow-Up Learning Impact Survey Questions

Performance Impact
Ask

How much was your performance impacted by the learning experience?
None at all				A lot

A little					A great deal
A moderate amount
Intention

Measure performance impact on the job.

Investigate

What activated or blocked performance? Is there a difference between the expected impact after the
program and after 30 days? (if the performance impact survey question was included in the first survey)

Applied Learning
Ask

How much of what you learned have you used?
None at all				A lot

A little					A great deal
A moderate amount
Intention

Measure expected amount of activated learning.

Investigate

What supported or motivated performance? What acted as a barrier to learning application on the job?
Is there a difference between the expected amount of use after the program and after 30 days? (if theapplied learning survey question was included in the first survey)

Frequency of Use
Ask

How often have you used what you learned?
None at all				A lot

A little					A great deal
A moderate amount
Intention

Measure regularity of using what was learned.

Investigate

Are employees motivated to activate performance that achieves the business goal? Is there anythingthat blocks utilization of learning?

0028

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Employee-manager Engagement
Ask

Did you and your manager discuss how you will use what you learned?
YesNo

Intention

Measure manager-employee engagement.

Investigate

Is there support for setting expectations and accountability with managers and employees for using what
was learned? Do employees and managers have resources for activating performance?

Business Goal AlignmentAsk

Do you see the connection between using what you learned and achieving the business goal?
Yes
No
Not Sure

Intention

Measure agreement and clarity on how performance helps achieve the business goal.

Investigate

Is the business goal clear, realistic, and still relevant? Are there updates on progress for achieving the goal?

Use Case
Please share an example of how you have used what you learned.

Ask

Comment:

Intention

Identify success stories for activated performance.

Investigate

Recurring themes from responses.

Performance Blockers
If you have not used what you learned, please share why.

Ask

Comment:

Intention

Evaluate performance blockers and address potential risks to achieving the business goal.

Investigate

Recurring themes from responses.

0029

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

What Do Clues from Impact Learning
Surveys Tell Us?
A single data point from one survey question alone does not tell the whole story, but – in combination with other survey questions– we gain actionable clues for learning effectiveness and estimated impact. Here are some examples of stories you can tell withlearning impact survey data:• Employees who see connection between performance and achieving the business goal plan on using significant
amounts of what they learned.

• When we include role-plays and simulations in training, people have greater confidence in their ability to apply
what they learned on the job.

• There is strong agreement that training will improve performance but not significantly.
• Employees who believe the learning solution will minimally impact their performance believe the biggest barriers
to achieving goals are time and technology.

“Survey data gives good signals for impact and is not where
the story ends.”
Learning survey data is the first link in the chain of evidence for impact. Survey data gives good signals for impact and is not where
the story ends. There are more facts, clues, evidence, and data that lead to answering the question, “What is the impact of learning?”

What’s Next?
The Global Gems L&D team has decided which learning survey questions they will use. They chose brevity to maximize theopportunity for higher survey response rates with focuses on performance impact, business goal alignment, manager-employee
engagement, and frequency of use. The next section details the survey questions they chose.
0030

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Case Study 5: Learning
Impact Survey for Global
Gems
1.

How much do you believe this learning experience will impact your performance?
None at all				A lot

A little					A great deal
A moderate amountComment:2.

Did you and your manager discuss expectations for this learning experience?
Yes
No
Comment:3.

Do you see the connection between using what you learned and achieving the business goal?
Yes
No
Not sure
Comment:

4.

How will this learning experience help your performance?
Comment:

5.

Is there anything about the learning experience that was not helpful?
Comment:0031

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

30-Day Follow-Up

1.

How much was your performance impacted by the learning experience?
None at all				A lot

A little					A great deal
A moderate amountComment:2.

Did you and your manager discuss how you will use what you learned?
Yes
No
Comment:3.

How often have you used what you learned?
Never					Usually
Rarely					Always
Sometimes
Comment:

4.

Please share an example of how you have used what you learned.
Comment:5.

If you have not used what you learned, please share why.
Comment:

0032

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

The L&D Detective Says…

How can learning survey data be actionable rather than data we simply collect,evaluate, and toss in the pile? I am glad you asked. Each question in the learning
impact survey has an intended purpose and an action to investigate.
Do you have to include every question in your survey? Not necessarily. However, I
believe you should include as many questions necessary to answer your questions
regarding the effectiveness of your learning solution.

“Employees not responding to learning surveys is a complaint I hear often.”
What good is a learning survey if you cannot get a good response, right? Employees not responding to learning surveys is acomplaint I hear often. It is a major challenge for many L&D teams. Here are my suggestions for improving survey response rates:

Send request for feedback from a person
versus a system.
Set aside time for the survey during the
program.
Share survey results with attendees andactions to be taken based on feedback.
Shorten survey completion time with fewer
questions.
Send the survey link by chat with webbased/virtual programs.

What’s Next?
The best part of being an L&D detective is solving impact measurement mysteries. In the next section, our L&D detective work focuses on building
the case for impact. We will examine facts, clues, evidence, and data that answer the question, “What is the impact of learning?”
0033

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Facts, Clues, Evidence,and Data for Impact“When you follow facts, clues, evidence, and data that demonstrate
results, you discover answers to questions about impact.”
We have done a lot of work planning and preparing for impact. Results from the impact opportunity interview support proactive impact planningand builds the impact blueprint. Impact design activates performance and impact learning surveys estimate learning effectiveness. The next
phase of L&D detective work examines facts, clues, evidence, and data that demonstrate impact.
There are three evidence types that reveal impact: learning performance, people performance, and business performance. This evidence will
demonstrate how training activates employee performance and how performance activation drives the business goal. When you follow facts,
clues, evidence, and data that demonstrate results, you discover answers to questions about impact.

Learning
Performance

PeoplePerformance

Business
Performance

ImpactLearning Performance
The first link in the chain of evidence for the impact investigation is learning performance. Learning surveys, test and assessment scores, knowledge
checks, simulation scores, and results from focus groups or one-on-one interviews provide clues about learning effectiveness. These are examples
of clues and data we will use to build the case for learning’s impact.

Employee Performance
The second link in the chain of evidence for impact is employee performance. This data shows the extent to which we can attribute behaviors andactions applied on the job to learning effectiveness. Employee performance data are indicators for activated behaviors and actions.

Business Performance
The third link in the chain of evidence for impact is business performance. This is where we connect the dots between learning effectiveness,
activated performance, and business outcomes. Business performance data are where we finally reveal the answer to the question, “What is theimpact of learning?”

What’s Next?
The Global Gems L&D team is ready to examine facts, clues, evidence, and data for impact. They have results for learning performance, people
performance, and business performance. The next section shows the results and their findings.
0034

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Case Study 6: Impact
Investigation for Global
Gems
The Global Gems L&D team designed and piloted the Gem Care learning solutions in Africa stores. They are piloting programs to compare businessperformance and people performance for stores in Africa to stores in other countries in the same business quarters. They will investigate differences
between agreed-upon impact measures from the impact blueprint for the pilot stores and stores in other countries as evidence for impact.

Impact Measures & KPIs
Business Performance and People Performance Measures
Measure/KPI

Measure Type

Goal

Report Timing

Quarterly sales

Business

20% growth

End of year

Customer satisfaction
scoresPeople5 points

End of year

Employee engagementscores*Business

80% Agree/Strongly
Agree

Beginning of next year

*Results for “The company invests in development that helps me be successful in my role.”, and “I believe in the company’s future”.
Note: Pilot programs were launched in Africa at the beginning of Q2.

Learning Performance Measures
Measure/KPI
Performance impact*

Goal
80% top 2 boxes

Employee-manager engagement*

90% Yes

Business goal alignment*

90% YesFrequency of use*

80% top 2 boxes

Business simulation average score

85% mastery*Post-program and 30-day follow up learning surveys

Learning Performance: Learning Impact Survey Results
The following results are learning survey data for the virtual instructor-led training programs, immediately following the programs and 30 days after.

0035

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Performance Impact
A lot/A great deal

A moderate amount

None at all/A little

Goal = 80%76%
How much do you believe this learning

15%

experience will impact your performance?
9%

Goal = 80%

86%
How much was your performance impacted by8%

the learning experience?

6%

Observations and Actions• Results for performance impact were 4% below goal immediately after the program.
• Results for performance impact were 6% above goal 30 days after the program.
• Employees saw higher impact for learning on the job than they initially anticipated after the program.
• Survey comments indicate the business simulation employees were required to complete after the virtual classroom program reinforced
what was learned and helped build confidence.

• Action to be taken: Redesign role-plays in the virtual classroom training for greater confidence building.

Top 2 and Bottom 2 Box Scores
Global Gems used a method called top 2 box and bottom 2 box scores for performance impact and frequency of use survey results. Hereis how it works:

Strongly disagree

DisagreeNot sure

Agree

Strongly Agree

5%

10%

10%

50%

25%

• Top 2 boxes = Agree (50%) + Strongly agree (25%) = 75%• Bottom 2 boxes = Disagree (10%) + Strongly disagree (5%) = 15%0036

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Top 2 Boxes, Bottom 2 Boxes Example
15%

10%

75%

Strongly Disagree/Disagree

Not sure

Agree/Strongly Agree

Why use top 2 boxes, bottom 2 boxes?

• Better data visualization.
• Simplify results for analysis.
• Compare data points faster.

Employee - Manager Engagement
Did you and your manager discuss expectations
for this learning experience?

Goal = 90%

Goal = 90%

76%

Yes

0037

Did you and your manager discuss how you will
use what you learned?

No

92%

Yes

No

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Observations and Actions• Results for employee-manager engagement pre-program were below goal by 14%.
• Results for employee-manager engagement post-program exceeded goal by 2%.
• More employees and managers engaged after the program than they did before the program.
• Survey comments indicate that managers and employees had to go to the learning management system (LMS) to get pre-program
discussion guides and were sent the post-program discussion guides directly via email.

• Action to be taken: Send pre-program discussion guides directly to employees and managers via email.

Business Alignment
Do you see the connection between using what
you learned and achieving the business goal?

Go

=
al

90

%

91%
Yes

No

Not sure

Observations and Actions• Results for business alignment exceeded goal by 1%.
• Employees see connection between using what they learned and achieving the business goal.
• Survey comments indicate leadership videos in the program announcement, leadership videos in the training program, and messaging
in the program description reinforced alignment between learning and the business goal.

• Action to be taken: Use the same method for communicating business alignment with other programs.

Frequency of Use
Goal = 80%

6%5%Never/Rarely

89%

Sometimes

Usually/Always
0038

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Observations and Actions• Results for frequency of use exceeded goal by 9%.
• Employees believe they are using a substantial amount of what they learned.
• Survey comments indicate accountability planning between managers and employees in the post-program planning guide and checkins with managers play a significant role in application of learning on the job.

• Action to be taken: Create accountability planning guide template for other programs.

Learning Performance: Business Simulation Results

Gem Care Business Simulation
Average Score

Go

=
al

85

%

89%
Observations and Actions• Employees exceeded the mastery target average score by 4%.
• Data from the business simulation shows that 91% of employees selected correct answers on first attempt for the luxury care andintentional engagement modules compared to 80% selecting the correct response on first attempt for the managed persuasion module.

• Action to be taken: Reinforce recommendations in post-program communication and business simulation instructions for watching
the managed persuasion performance support videos before starting the simulation.

Impact Investigation Findings for Learning Performance• Post-program accountability planning reinforces employees using what they learned.
• Higher engagement between employees and managers activates performance.
• Opportunities to practice behaviors and skills builds confidence.
• Direct messaging from leadership reinforces alignment between using what was learned and achieving business goals.
• Ease of access to performance support increases utilization.

0039

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

People Performance
Gem Care customer satisfaction scores measure customers’ experience with Gem care specialists. There are three performance targets for Gem
Care training programs and learning solutions:

Capability

Performance Measure (Customer
Satisfaction Target Score)

Performance Requirement

Intentional engagement

Actively listen to customers, andinterpret non-verbal clues.

5

Assess interest, evaluate doubts,
address concerns, and drive desire.

5

Intentional Engagement

5Managed persuasion
4.2
4
Luxury care
2.9
3

2.5

Create a personalized and luxurious
in-storeexperience.
3

2.7

3

3 2.9
2.8

5

2.3
2.3
2.1 2 22 2
2
2 2
2 1.9 2The2impact investigation compares customer satisfaction scores quarter to quarter for Africa, the pilot country, to other countries.
The training
1.5
1.3
programs were piloted in Africa in the second quarter.
10

Africa*Customer Satisfaction

Asia

Australia
EuropeIntentional
Engagement
Q1

5
4.2
4
5
3
4
2
3

2.9
4

2.5

0
1

0

2.5

Africa*

Q4

3
2.3

3

2.1 2 23 3 3 2.9

Asia

Africa*

2.7
1.3

3 2.9
2.8

2.3
2 2
3.1
2.9 3

2.9 2.9 3

3

2 1.9 2
3.1
1.5 3 3 2.8

EuropeLatin America

North America

Latin America

North America

3.1
2.9 3

2.9 2.9 3

3.1 3

EuropeLatin America

2 2.1 2Australia
Q1

5

Q2

Q3

Q4

Australia
EuropeManaged Persuasion

Asia

Q1

4.5

Q2

Q3

Q4

4

4
3

2.7

2

1
2

Q3

North America

Managed Persuasion

4.5

2 2
3 2.9

Q2

Latin America

3 2.9

2

3
2.5

3

3 2.9

2.7

3

3

2.8

2 2.1 210Africa*Asia

Australia
Q1

Q2

Q3

North America

Q4
0040

2.5
2

2

2.3

2

2.1 2

2

2

2

2.3

2
L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All1.3
rights reserved10Africa*5

5
4

Customer Satisfaction

Asia

Australia
Luxury Care Europe
Q1

4.5
3.8

4

4
3.5
2.8

3

3

3.7

Q2

Q3

1.5

Latin America

2 1.9 2

North America

Q4

4.5

4

3.8

3

3

3.5 3.6 3.4

4

3.9 4

4

2.8

2

2

1.5

1
0

Africa*Asia

Australia
Q1

EuropeQ2

Q3

Latin America

North America

Q4

Impact Investigation Findings for People Performance• Customer satisfaction scores indicate pilot stores in Africa are closer to achieving the target score (5) than stores without Gem Care training
programs and learning solutions.

• Scores for intentional engagement, managed persuasion, and luxury care rose consistently quarter to quarter in Africa.
• Europe saw an increase in the score for intentional engagement in the fourth quarter but at a much lower rate compared to Africa.
• Europe saw slight increases in scores quarter to quarter for managed persuasion.
• Africa achieved the target score (5) for luxury care in the fourth quarter.
• All stores saw an increase in the fourth quarter for luxury care.
Note: Remodeling was completed in all stores in Q4 and contributed to customer in-store experience.

Business Performance
The following data shows sales results across stores. The impact investigation compares quarter to quarter sales for the pilot country, Africa, toother countries. The training programs were piloted in Africa in the second quarter.

0041

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Quarterly Sales$3,000,000$2,500,000$2,000,000$1,500,000$1,000,000$500,000$0Africa*Asia
Australia
EuropeLatin America
North America

Q1

Q2

Q3

Q4

$1,900,000$2,000,000$1,750,000$1,900,000$2,001,000$2,500,000$1,775,000$1,900,000$1,740,000$1,915,000$2,020,000$1,950,000$2,200,000$2,100,000$1,600,000$2,000,000$1,875,000$775,000$2,500,000$2,120,000$2,000,000$2,150,000$1,950,000$2,001,000*Stores where training and learning solutions for Gem Care were piloted.

Impact Investigation Findings for Business Performance• Africa exceeded the 30% growth goal.
• Africa achieved 32% growth by end of year (comparing first quarter to fourth quarter).
• Sales increased consistently from second quarter to third quarter and third quarter to fourth quarter in Africa.
• Sales in other countries remained consistent with slight increases and decreases excluding North America in the third quarter.
• All stores saw an increase in sales in the fourth quarter due to seasonal holiday sales trends.
Note: One of the stores in North America closed for remodeling after fire damage in Q3.

Tell the Whole Story
Training and learning are just two of many influences that activate performance. It is important to keep this top of mind as you examine facts, clues,
evidence, and data for impact. Why? We cannot make false assumptions that changes in behavior, performance, and actions or achieving thebusiness goal only occur as a result of training.
We cannot conduct a thorough impact investigation without considering all factors that contribute to performance and achieving the businessgoal. We need to examine all dynamics within the timeframe in which we expect to see impact. Here are some examples:• There was a product promotion that gave sales a lift.
• A new service from a competitor impacted demand for ours.
• There was a weather condition that prevented people from getting to stores.
• We had to shift supply-chain strategy unexpectedly.
• There was higher demand for a product due to unforeseen environmental factors.
None of these dynamics have anything to do with training, but they do impact results. Impact investigation is not just about examining what
happened as a result of learning. L&D detective work examines all the facts, clues, evidence, and data that reveal reasons for impact.
0042

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Impact Investigation
Results
Did training activate performance? Did activated performance help achieve the business goal? Here is a summary of the
impact investigation findings:

Pilot stores in Africa exceeded growth goal by 2%

+32%

$

Africa achieved
target customer
satisfaction score(5) for luxury carein Q4.**RESULTS

Africa achieved 32%
growth from Q1 to Q4.

*Excluding significant decrease for North
America in Q3 due to store closure from firedamage.
**Holiday season and store renovation
contributed to increase.

0043

Sales in other countries
remained consistent
with slight increases anddecreases. *Customer satisfaction
scores in Africa forintentional engagement,
managed persuasion, andluxury care increased in
Africa after training in Q2.
Engagement
between employees
and managers
activated
performance.
Businesssimulations
reinforced
confidence for using
what was learned.

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

The Global Gems L&D team recommends global deployment for the gem care specialist training programs. The recommendation is based on findings
from the impact investigation, showing:• The pilot country, Africa, exceeded the growth goal.
• Customer satisfaction scores indicate pilot stores in Africa arecloser to achieving target scores than stores without gem care specialist training solutions.

• Results from learning surveys and business simulation scoresindicate learning effectiveness for activating performance.
Results for the impact investigation are persuasive. Based on the findings, Global Gems is making a fact-based, data-informed recommendation to invest
in learning and development that measurably impacts employee performance and business goals. The most valuable impact investigations reveal
actionable insights and measurable outcomes.
We proactively planned for impact measurement with the impact blueprint, making the impact investigation easier to conduct. We knew in advance themeasures, key performance indicators, timing for reporting results, and partners with whom we’d need to work for data access, analysis, and visualization.
We had a roadmap for impact measurement with agreement from stakeholders regarding what and when we would report.
Clues, facts, evidence, and data from the investigation indicate that learning impacted both performance and the business goal. Can we claim thattraining alone drove results? No, we cannot. Have we produced a reasonable preponderance of evidence that shows learning activated performance?
Yes, we did.

What Did We Accomplish?
• Discovered opportunity for impact with the impact opportunity interview.
• Planned for impact with the impact blueprint.
• Activated performance with impact-based design.
• Used impact learning surveys to gather clues for impact.
• Investigated impact with key performance indicators.
• Revealed outcomes.
• Made fact-based, data-informed recommendations based on findings.
We solved the impact measurement mystery!

0044

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

Investigating Impact: Not Easy, but Possible
We used an abbreviated case study to demonstrate the end-to-end process for investigating impact. Here is the impact investigation timeline:

Conduct impact
opportunity interview

DiscoverGather key performanceindicators.

Build impact blueprint.

ExamineEvaluate interview results.

Plan

DesignDesign for impact.

Collect

Report impact
investigation findings.

Inspect

RevealAnalyze results.

The process is the same for a multitude of training programs – including leadership development, customer service, communication, role-specifictraining, coaching, and technical training to name a few. The purpose of L&D detective work is to discover when learning makes an impact andwhen it does not. Not all impact stories have a happy ending. Sometimes, the findings will reveal learning had no impact.

“Don’t be afraid of the ‘bad’ stories data can tell. Be afraid of notdoing anything with the results.”
Results from impact investigations that show little to no impact can be extremely valuable. There’s opportunity to improve, adjust, and reactivate
learning for desired outcomes. There’s also an opportunity to stop doing what is not working. Don’t be afraid of the “bad” stories data can tell. Be
afraid of not doing anything with the results.
Discovering facts, clues, evidence, and data that demonstrate impact can be complex and challenging. The fifth guiding principle for L&D detective
work is, “Investigating impact for learning may not be easy, but it is possible.” Hopefully, you’ve seen how L&D detective methods and techniques
reduce complexity and make impact measurement mysteries a little easier to solve.

0045

L&D Detective Kit for Solving Impact Mysteries v3 | Copyright © 2020 - All rights reserved

You, Too, Can Be an
L&D Detective
I am an L&D detective, and I solve mysteries. The mystery I solve is, “What is theimpact of learning?” The L&D detective kit shows how I investigate and solve
measurement mysteries.
I did not want to create a work deeply rooted in theory. I did not want to be overly
academic or so constrained by aspirational best practices that what I shared could
not be applied. The intention for the L&D detective kit is to be relevant, helpful,practical, and – most importantly – actionable.

“If you focus on the methods, the meaning, and themindset, I believe you will find what I have shared istransferable across industries and topics.”
The business example we used for the case study may not represent your industry,
but it does provide context. The learning solution we used as an example may not
be representative of the topics relevant to your industry. However, it illustrates theprocess. If you focus on the methods, the meaning, and the mindset, I believe you
will find what I have shared is transferable across industries and topics.
One of the most important takeaways from the L&D detective kit is the impact
opportunity interview. Where I see L&D teams struggle most and miss the greatest
opportunity is not considering impact prior to the design and deployment of alearning solution. Impact opportunity interviews consider impact in the beginning,
so it will be easier to measure in the end.
Impact is when learning activates performance and a business goal. Training andlearning fulfill the highest purpose with measurable impact. My goal was showing
you how to measure learning’s fulfillment of purpose.
The L&D detective kit is a work of passion and joy for me and was created just for you– the global learning and development community. I hope something I have shared
can be applied in your work to enhance your own impact in your organization. And I
hope you see now that you, too, can be an L&D detective.
0046

Contact andConnect
kevin@kevinmyates.com

kevinmyates.com

LinkedIn

Facebook

Twitter

Instagram

YouTube