+ ROI Report2010 World
Education Congress
ROI Impact Study
Marj Atkinson, MPI research manager andCaterina Bulgarella, PhD, senior consultant
for MeetingMetrics

Return on Investment, or ROI. Meeting Professionals International (MPI) had never calculated
personalized delegate ROI data, and it had been
several years since the World Education Congress(WEC) had been evaluated by anything other thanpost-event surveys. The time had come for a thorough event evaluation and ROI study.
September 2011 | Page 1

To that end, MPI employed MeetingMetrics for its 2010 WEC. The company’s
MyROI program provides detailed
individual evaluations for event delegates,
who received a 12-page report documenting the value of their attendance at theWEC, including a financial account basedon the ROI Institute method.

METHODOLOGYMPI used the Phillips ROI Methodology to measure the MyROI and WEC
programs. The methodology evaluates reactions, learning, application,monetary impact and ROI.

The reports also included a comparative analysis of the average ROI results ofother WEC delegates, guidance on how toanalyze ROI, a personal action planner tohelp delegates extend the benefits of WEC
and tips on how to improve ROI whenattending future education events. Of
course, MPI also had a stake in the results
to determine if programming changes
were necessary.
MyROI gathered delegate data
through a two-part questionnaire to
determine cost savings and revenue
generation as a result of WEC attendance.
MPI encouraged delegates to prepare
for the survey, and four months after the
event invited them to complete the online
questionnaire that resulted in individual
ROI reports. The organization sent thesurvey to its 2,492 WEC delegates. Of
these, 293 responded to the survey, and61 completed it entirely. MPI compiled
the results of the latter to prove delegate
ROI at the WEC and to suggest ways toimprove that ROI at future conferences.

OBJECTIVESAt the outset, MPI outlined its key
goals and takeaways for the ROI
measurement program. MPI set atarget ROI of at least 50 percent.
1. Were delegates satisfied
with the WEC program?
2. Was session content
relevant and effective?
3. How did the knowledge
from WEC benefit delegates?
4. Was WEC a worthwhile
investment for delegates?

ROI Analysis Plan

Data
Delegate
producing
more revenue
for organization as resultof exceptional
meetings/
events (i.e.
more sales,
more attendees, etc.)

Methodto Isolate
Effects

Methodto Convert
Date into$$$
Delegate
estimates

Cost Categories

Intangible Benefits

Communication Targets
for Final
ReportAttendee salaryand benefits
while attending
conference

Improved productivity

Delegates

Improved planning andschedulingManagers ofdelegates

TravelJob satisfaction, leading
to better, more productive meetings

Senior managers

Lodging
Meals
Registration
costs

Customer satisfaction
from producing more effective meetings

MPI
Sponsor(s)Other Influences/Issues

Comments

Delegates mustsee need forproviding measurement

Delegates will
identify specificimprovements
as a result of
attending WEC

Delegates
must realize
cost savings
and revenue
generation

Content andevents team

Increased job engagement/effective meeting
planning
Professional development credits
The impactof the WEC
on costs(i.e. reduced
overhead/
expenses,
project/
program costsavings)

Delegate Delegate
estimates estimates

Evaluation
costs

September 2011 | Page 2

FINDINGSHigh quality is imperative to the creationof a superior learning environment in
the classroom and beyond—attracting
enough industry players so as to facilitate
worthwhile networking opportunities,
intelligent dialogue among members andadequate occasions to learn and growthrough delegate interaction. Eventdelegates need to know what attendance
will help them reap, and MPI must provethat the WEC continues to be a stimulating, thought-provoking event, capable ofoffering high-quality education and remaining a key industry conference worth
attending every year.

DEFINITIONS

Here are the numbers:

Net Total Benefits: Net total financial
benefits generated by applying theknowledge and skills acquired at themeeting. This value is computed bysubtracting total costs related to themeeting from the total adjusted value.

Benefit Cost Ratio: The relationshipbetween meeting-related costs andmeeting-related benefits.
Return on Investment (ROI): The returnon investment made from attending ameeting. This value is computed by dividing the net total benefits by the total
costs. A 100 percent ROI means that the
net financial benefit associated with themeeting is equal to the total expenses
incurred.

$13,145

Average total adjusted value (ranged
between $4,752 and $21,538)

Total Adjusted Value: Total dollar value
generated by applying the knowledge
and skills acquired at the meeting. Thisvalue is adjusted to reflect solely the
impact of the meeting and the level ofconfidence in that value.
Total Costs: The total expenses thatwere incurred to attend the meeting.

$9,357

Average net total benefit (ranged between$986 and $17,729)

4.42

Average benefit/cost ratio (ranged between 2.00 and 6.83)

341.42%Average participant ROI (ranged between
100.39 and 582.65)

Delegates were 95% confident intheir responses

Data Collection Plan
Level

Measures

Data
Collection

Data
Sources

TimingOwner

1

SATISFACTION AND
PLANNED ACTION
A positive reaction
to WEC

Delegate reaction

4-month,follow-up
questionnaire

Delegates

4monthsResearch
Manager

2

LEARNING
Relevant session
content and speakers

Delegate
responses
to questions
about what
they learned
and how it applies to theirjobs

4-month,follow-up
questionnaire

Delegates

4monthsResearch
Manager

3

APPLICATION
Knowledge from
WEC benefits delegates on the job

Delegate
calculations

4-month,follow-up
questionnaire

Delegates

4monthsResearch
Manager

4

BUSINESS IMPACTWEC as a worthwhile investment
for delegates

Delegate
calculations

4-month,follow-up
questionnaire

Delegates

4monthsResearch
Manager5

ROI
Target ROI at least
50 percent

Comments:

EVALUATION PLANNINGMPI has traditionally used the MeetingMetrics survey system to gauge delegate
feedback. The organization sends online
surveys for the conference, the tradeshow/
hosted buyer program and individual
sessions. It also conducts intermittent discovery surveys to determine educational
content needs for future conferences. Session surveys go out daily during the show
and up to a week following. MPI sends
the post-event and hosted buyer queries
after the conference for two weeks, alongwith several reminders. The goal is toreach a statistically relevant number,which can be challenging due to confusion and survey fatigue.
For MyROI, MPI sent communications
to attendees a month after conference
outlining the survey. Three months later,delegates received the survey invitation
followed by two reminders. The questionnaire determined cost savings and revenue
generation that could then be converted
into ROI. Sixty participants responded.

Program
Objective(s)September 2011 | Page 3

DELEGATE SPEND
The 61 respondents invested $225,917 toattend the conference. This total includes
registration, travel and lodging, meals,
time out of office and other miscellaneous costs.

SATISFACTION
More than 90 percent of delegates said
they were satisfied with the overall event(93 percent from the WEC survey and95 percent from MyROI). Education
satisfaction results topped 83.3 percent
and 92 percent, respectively. Attendanceat WEC delivers educational value todelegates and measuring the ROI for attendance at level 5 will prove that value.
Overall, WEC delegates boasted a 342
percent ROI.

Delegate Costs
Average per delegate

Registration

Travel/Lodging/Meals

Time

Misc.

Total

$664.04$1,616.06$1,196.80$226.64$3,703.55$98,580.00$73,005.00$13,825.00$225,917.00

Total $40,507.00

Cost Saving Measure
Monetary Value

Isolation Estimate

Confidence in Estimates

Adjusted Value$25,002.38

55.05%

79.19%$13,667.52

Increased Revenue Measures
Monetary Value

Isolation Estimate

Confidence in Estimates

Adjusted Value$80,983.87

44.84%

72.74%$9,553.48

Registration Fees

Travel/Lodging

Time Out of the Office

Miscellaneous

Total$676.10$1,665.25$1,216.48$229.92$3,787.75

Expenses

Cost Savings ROI
Total Adjusted Value

Total Costs

Net Total Benefits

Benefit Cost Ratio

ROI$13,667.52$3,301.30$10,366.22$4.35$334.63

Increased Revenues ROI

DELEGATE REACTION
The My WEC Experience portion of thesurvey showed favorable ratings acrossnearly all participants. On average, 93
percent of respondents provided positive ratings in this area. The confidence
interval fell between 4.62 and 5.09. Thatis, the upper and lower limits of the confidence intervals suggest that the averageratings were well above 4 and as highas 5.09. On a scale from 1 to 6, where
1 indicated “Strongly Disagree” and 6
indicated “Strongly Agree,” the averagescore was 4.83.

Total Adjusted Value

Total Costs

Net Total Benefits

Benefit Cost Ratio

ROI$9,553.48$4,434.87$5,118.60$3.50$250.04

Total Adjusted Value

Total Costs

Net Total Benefits

Benefit Cost Ratio

ROI$13,145.18$3,787.74$9,357.44$4.42$341.52

Total ROIWEC DELEGATES BOASTED
A 342% ROI.

95% said the WEC met their expectations
92% were satisfied with the learning
opportunities

94% enjoyed the educational sessions
91% said the WEC was a stimulating,thought-provoking event

93% said the WEC is a key industry event
worth attending every year

September 2011 | Page 4

There was a significant drop in participation for the second part of thesurvey, when participants were asked tomonetize the benefits of their learning
experiences at the WEC. MPI tested thedifference in ratings between participants
who completed the entire survey andthose who did not. The table below provides the average ratings across these twogroups. MPI found that the only significant difference occurred when delegates
responded to the statement, “The 2010
WEC was a stimulating, thought-provoking event.” Participants who completed
the whole survey were more positive thanparticipants who did not.

WEC Experience
Survey Item

Mean Average

95% Confidence Interval
Lower Limit

95% Confidence Interval
Upper Limit

WEC met my expectations

4.95

4.84

5.05

I am satisfied with the learning opportunities at WEC

4.75

4.64

4.86

I enjoyed the educationalsessions I attended at WEC

4.73

4.62

4.84

WEC was a stimulating,thought-provoking event

4.76

4.65

4.86

WEC is a key industry event
worth attending every year

4.98

4.87

5.09

*Based on a scale of 1 to 6, where 1 indicated “Strongly Disagree” and 6 indicated “Strongly Agree”

Survey Completion Factor - Experience
Survey Item

Mean Average of Respondents Who
Completed Survey

Mean Average of Respondents
Who Didn’t Complete Survey

WEC met my expectations

5.15

4.89

I am satisfied with the learning opportunities at WEC

4.92

4.70

I enjoyed the educationalsessions I attended at WEC

4.85

4.69

WEC was a stimulating,thought-provoking event

4.97

4.70

WEC is a key industry event
worth attending every year

5.13

4.94

*Based on a scale of 1 to 6, where 1 indicated “Strongly Disagree” and 6 indicated “Strongly Agree”

WEC Education
Meanwhile, education ratings showed favorable results. On average, participants
agreed that WEC education offered significant learning opportunities (M=4.69).
And the 95 percent confidence interval
confirms that average ratings were notlower than 4.4 and as high as 4.85.

Survey Item

Mean Average

95% Confidence Interval
Lower Limit

The knowledge I gained at the WEC is highly
relevant to my job.

4.74

4.64

4.84

What I learned at the WEC enhanced my job effectiveness.

4.51

4.40

4.62

The speakers at the educational sessions that
I attended were well prepared and effective inmeeting the session’s learning objectives.

4.75

4.65

4.85

enhanced their job effectivenessThe applicability of the learning content offered at
WEC was immediate and easy to understand.

4.75

4.65

4.85

94% said the speakers were well pre-*Based on a scale of 1 to 6, where 1 indicated “Strongly Disagree” and 6 indicated “Strongly Agree”

94% said what they learned at the WEC
was highly relevant to their jobs

90% said what they learned at the WEC

pared and effective

94% said the applicability of learning

Survey Completion Factor - Education
Mean Average of
Respondents Who Completed Survey

Mean Average of
Respondents Who
Didn’t Complete
SurveyThe knowledge I gained at the WEC is highly
relevant to my job.

4.93

4.69

What I learned at the WEC enhanced my job effectiveness.

4.77

4.45

The speakers at the educational sessions that
I attended were well prepared and effective inmeeting the session’s learning objectives.

4.84

4.73

The applicability of the learning content offered at
WEC was immediate and easy to understand.

4.92

4.70

content at the WEC was immediate andeasy to understand

Survey Item

When MPI looked at differences inaverage education ratings between theparticipants who completed the survey
and participants who didn’t, it found that
respondents who completed the survey
were more likely to indicate that theknowledge they gained at the WEC was
highly relevant to their jobs and contributed to enhance their job effectiveness.

95% Confidence Interval
Upper Limit*Based on a scale of 1 to 6, where 1 indicated “Strongly Disagree” and 6 indicated “Strongly Agree”
September 2011 | Page 5

Delegates easily transferred WEC content
to their careers.

WEC On the Job

70% said they applied 10% to 50% ofSurvey Item

Mean Average

95% Confidence Interval
Lower Limit

95% Confidence Interval
Upper Limit

75% said they spent 10% to 50% of

What percent of new knowledge/skills gained at the WEC
did you apply to your job?

4.51

4.40

4.62

87% said they received adequate sup-What percent of your total
work time have you spent on
tasks that required the knowledge/skills that you gained at
the WEC?

4.75

4.65

4.85

The knowledge that I gainedat the WEC has been of great
benefit to me in my work.

4.74

4.64

4.84

WEC knowledge to their jobs

their work time on tasks that required theknowledge gained at WEC
port in their organizations when attempting to apply the knowledge gained at WEC
Note: The survey measured these items
on a scale of one to 10, where one
was equal to 10 percent and 10 to 100
percent. On average, participants agreed
with the statement, “The knowledge that
I gained at WEC has been a great benefit
to me in my work.” This was confirmedby the estimated range of values for theentire population of attendees.
Respondents applied about 45 percent of the knowledge they gained at the
WEC to their jobs. The estimate for theoverall sample was similar (between 44
percent and 46 percent). And 47 percent
of work time was spent on tasks that
required the use of that new knowledge.
For the overall delegate population, thepercentage of work time spent on tasks
that required WEC knowledge was between 46 percent and 48 percent.
Meanwhile, respondents who completed the entire survey and those who
did not differed significantly on all threemeasures.
In particular, participants whocompleted the survey were more likely
to say that the knowledge they gainedat the WEC was of great benefit to themin their work. They reported a higher
percentage of new knowledge used intheir work and spent more time on tasks
than their peers who didn’t complete thestudy. The largest difference between thetwo groups concerned the amount oftime spent at work on tasks that required
the new WEC knowledge (47 percent vs.
37 percent, respectively).
The survey also looked into the*Based on a scale of 1 to 6, where 1 indicated “Strongly Disagree” and 6 indicated “Strongly Agree”

Survey Completion Factor - On the Job
Mean Average of
Respondents Who
Completed Survey

Mean Average of
Respondents Who
Didn’t Complete
SurveyWhat percent of knowledge/skills from the WEC
did you apply to your job?

5.08

4.08

What percent of your total work time have you
spent on tasks that required the knowledge/skills
from the WEC?

4.70

3.68

Knowledge I gained at the WEC has been of great
benefit to me in my work.

4.87

4.52

Survey Item

*Based on a scale of 1 to 6, where 1 indicated “Strongly Disagree” and 6 indicated “Strongly Agree”

business impact of the WEC. The results
speak for themselves.

92% said WEC had a positive impact on
their jobs

91% agreed that their WEC experiences
helped increase their effectiveness intheir roles

93% indicated that WEC was a worth-while investment for professional development

92% reported that WEC was a worthwhile investment for their employers

The survey asked participants whatenablers helped them apply knowledge
from the WEC and if they experienced

any barriers that prevented them fromusing that knowledge. Helpful options
included personal initiative, peer support,
additional training and opportunities
to immediately use new skills. Lack oftime was the main barrier to using WEC
content on the job.

92% said that
the WEC had apositive impact
on their jobs

September 2011 | Page 6

ISOLATING THE EFFECTS:
COST SAVINGS
Some 37 of the 61 respondents said that
their cost-saving programs were affected by the new knowledge/skills theyacquired at the WEC.
• Cost-savings varied between $10 and$168,750 (adjusted value), much ofwhich can be attributable to new negotiation skills.
• The average cost savings was $13,667.
The 95 percent confidence intervalaround this value ranged between $4,752
and $21,537, suggesting that in the population of all WEC attendees the averagecost-savings was at least $4,752.
• On average, participants showed a 79
percent confidence in these answers andattributed 55 percent of the reported costsavings to knowledge and skills acquiredat the WEC.
• The confidence interval around theaverage isolation estimate for the sample
of respondents was 45 percent to 65
percent. That is, the impact of WEC content on the cost savings for the averagedelegate was not lower than 45 percent
and as high as 65 percent.

ISOLATING THE EFFECTS:REVENUES
Meanwhile, 31 respondents provided
data about some form of additional revenue driven by WEC content.
• Profit due to new knowledge and skills
acquired at the WEC varied between$150 and $168,750 (adjusted value).
These gains can potentially be thanks to
networking and/or learning opportunities.
• The average increase in profit was$9,553. The 95 percent confidence interval around this value ranged between$788 and $18,319, suggesting that in
the population of all WEC attendees, theaverage cost-savings was at least $788.
• On average, participants showed a 73
percent confidence in these answers andattributed 45 percent of the reported
profit to the knowledge/skills acquired at
the WEC.
• The average profit margin for the revenues reported was 34 percent. The 95
percent confidence interval around thisvalue ranged between 26 percent and 42

percent, indicating that on average WEC
delegates could generate a profit margin
as high as 42 percent.
• The confidence interval around theaverage isolation estimate for the sample
of respondents was 34 percent to 55 percent. That is, the impact of WEC content
on the cost savings on reported revenues
for the average WEC delegate was not
lower than 34 percent and potentially ashigh as 55 percent.

CALCULATING ROI
The survey asked participants to monetize the financial benefits of the newknowledge and skills they acquired at
the WEC. Respondents could report dataaround cost savings, revenue increase orboth. The majority of respondents provided data on only one of the measures.
The “ROI of WEC” chart on Page 8shows a sample of the data involved in
the ROI calculation for the participants
who provided monetary information
about one or two financial benefits for
the WEC.
• The average total adjusted value in thesample was $13,145. The 95 percent
confidence interval around this value
ranged between $4,752 and $21,538,
suggesting that the total financial gain
driven by the learning opportunities atthe 2010 WEC was at least $4,000 in the
population of all WEC attendees. This
gain may have been as high as $21,537.

FINANCIAL IMPACT
DEFINITIONS
Depending on whether delegatesprovide data on cost savings orrevenue increase, the delegate
report presents one or two measuretables. Each table displays the following information:
1. Measure’s Description: An explanation of the measure
2. Monetary Value: The dollar valueof the measure’s increase
3. Profit Margin: Only when reporting revenues
4. Isolation Estimate: The percentage of the measure’s increase
attributed to the application andimpact of what was learned at themeeting
5. Confidence in Estimates: This is
the confidence expressed in the estimates about the financial impactof the measure
6. Adjusted Value: The dollar valueof the measure’s increase attributable to the application of the knowledge and skills that were learnedat the meeting, after correcting forthe error level in the measure’s estimates per the level of confidence
expressed• The average net total benefit in thesample was $9,357. Based on the 95 percent confidence interval estimate for theoverall population of WEC attendees, thenet benefit may have been as low as $986
or as high as $17,729.
• The average benefit/cost ratio in thesample was 4.42. Based on the 95
percent confidence interval, the average
benefit/cost ratio in the population of all
WEC attendees ranged between 2 and6.83. That is, the average WEC delegate
benefited financially as much as seventimes more than what was invested toattend.
• The average participant ROI was
341.42 percent. Based on the 95 percent
confidence interval estimate (which ranges between 100.39 and 582.65), the average WEC attendee may have achieved an
ROI result as high as 600 percent.

September 2011 | Page 7

RECOMMENDATIONSMPI found several correlations between
the survey items measuring the effectiveness of the WEC and the total participant
ROI for the 61 respondents.
• Measures that correlated the mostincluded percentage of work time spentusing the new knowledge and skills,
percentage of new knowledge and skills
gained applied to the job and job effectiveness of the learning at the WEC.

ROI of WEC
Participant

Total Adjusted
Value

Total Costs

Net Total Benefits

Benefit Cost
Ratio

ROI

1

$14,630.00$2,636.00$11,994.00

5.55

455.01%

2$588.00$3,963.00($3,375.00)

0.15-85.16%3$540.00$4,375.00($3,835.00)

0.12-87.66%4$6,400.00$1,598.00$4,802.00

4.01

300.50%5$264.60$3,665.00($3,400.40)

0.07-92.78%31$29,250.00$2,800.00$26,450.00

10.45

944.64%32$300.00$8,400.00($8,100.00)

0,04-96.43%• Similarly, job relevancy and degreeof session enjoyment ratings were also
predictive of high participant ROI.

33

$44,725.00$5,990.00$38,735.00

7,47

646.66%34$250.00$2,075.00($1,825.00)

0.12-87.95%35$1,900.00$3,892.00($1,992.00)

0.49-51.18%• The overall event quality creates ideal
learning environments.

57$150.00$4,000.00($3,850.00)

0.04-96.25%58$3,000.00$8,000.00($5,000)

0.38-62.50%59$3,125.00$4,000.00($875.00)

0.78-21.88%60$300.00$1,700.00($1,400.00)

0.18-82.35%

61$22,400.00$2,498.00$19,902.00

8.97

796.72%Average$13,145.18$3,787.74$9,357.44

4.42

341.52%

CONCLUSION
To the extent to which the WEC continues to be a stimulating, thoughtprovoking event, capable of offeringhigh-quality education and remaining a
key industry conference worth attending
every year, WEC delegates are positioned
to reap significant financial benefits
from the application of the knowledge
and skills acquired at the MPI annual
congress.

Survey Comparisons
Correlation w/
Participant ROI

Survey Item
What percent of your total work time have you spent on tasks that required theknowledge/skills from WEC?

0.36

What percent of new knowledge/skills gained at WEC did you apply to your job?

0.35

What I learned at WEC enhanced my job effectiveness.

0.35

My WEC experience helped increase my effectiveness at work.

0.33

The knowledge that I gained at WEC is highly relevant to my job.

0.32

I enjoyed the educational sessions I attended at WEC.

0.31

The applicability of the learning content offered at WEC was immediate and easy
to understand.

0.30

The speakers at the educational sessions that I attended were well prepared andeffective in meeting the session’s learning objectives.

0.28

WEC was a stimulating, thought-provoking event.

0.27

WEC is a key industry event worth attending every year.

0.25

Overall, I am satisfied with the learning opportunities at the WEC.

0.24

September 2011 | Page 8