Michigan Judicial Institute | PO Box 30048 | Lansing, Michigan | 517-373-7171 | MJIeducation.mi.gov | ©2022

The project materials (workshop recording and workshop material) were 

developed under grant number SJI-21-P-006. The points of view expressed 

are those of the author and do not necessarily represent the official position 

of the State Justice Institute.

MATERIAL PREPARED AND PRESENTED BY:

Dr. Laura L. Bierema, EdD

University of Georgia

College of Education

WORKSHOP EVALUATION PREPARED BY:

DR. JORI N. HALL, PHD

University of Georgia

College of Education

TABLE OF CONTENTS

WORKSHOP RECORDING ......................................................................................2

BIEREMA WORKSHOP MATERIAL .........................................................................3

BIEREMA BACKWARD DESIGN TEMPLATE........................................................ 22

BIEREMA SMALL TEACHING ONLINE INVENTORY ............................................ 27

BIEREMA PRACTICE LAB SIGN UP .................................................................... 33

HALL PARTICIPANT CONVERSATION GUIDE ..................................................... 35

HALL WORKSHOP EVALUATION ........................................................................ 37

MJI eLearning Faculty Development 

Workshop 

August 31, 2021

eLearning Faculty Development Workshop Recording

https://mjieducation.mi.gov/videos/elearning-faculty-development-workshop-1

eLearning Faculty Development Workshop

This workshop offers current information on the pedagogy of online, synchronous (live) adult 

education and offer options for incorporating theory and best practices to engage the virtual learner 

into an online, live education session.
Laura L. Bierema

582 Pulaski Street

Athens, GA, 30601

Phone: 706-248-5290

E-Mail: bierema@uga.edu

This document was developed under grant number SJI-21-

P-006. The points of view expressed are those of the

author and do not necessarily represent the official

position of the State Justice Institute.

eFaculty Development for MJI 

Teaching & Facilitation

 August 31, 2021

PARTICIPANT 

GUIDE
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 2

 

. 

26 August 2021 

Dear Participant,

Welcome to the eFaculty Development for MJI Teaching and Facilitation Workshop. I look forward to engaging 

with you virtually to create explore active learning and master the technology of virtual gatherings. 

This packet contains information on the workshop purpose, outcomes, and agenda, plus supporting information 

and handouts for your use during the session. 

Please let me know if you have questions about the materials via bierema@uga.edu or 706-248-5290. 

I look forward to working with you. 

Kind regards,

Laura L. Bierema

Professor 

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 2
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 3

 

eFaculty Development for MJI Teaching and Facilitation

Workshop Purpose

The purpose of The Power of Active Learning Workshop is to model effective design and delivery of active 

pedagogy for online, synchronous (live) education and meetings that engages participants using evidence-based 

practices.

eFaculty Development for MJI Teaching and Facilitation

Workshop Outcomes

As a result of attending the full workshop, participants will: 

• Share a welcome, overview, and check-in.

• Design to facilitate active, adult learning

• Master technology for virtual learning and facilitation

• Reflect on our learning process.

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 3
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 4

 

eFaculty Development for MJI Teaching and Facilitation

Facilitation Agenda

Tuesday 31 August 2021

Time

EDT

Allotted Topic Lead Materials/Media/Notes

9:00 :15 Share: Welcome, Overview, and check-in

1. Set Context (MJI) 

2. Review Agenda (Laura)

3. Check-in (Laura)

Laura

MJI

• Handout

• Check-in Chat & Debrief

9:15 :45 Design: To Facilitate Active, Adult Learning 

1. Use the principle of “Backward Design”

2. Plan with the POP (Purpose, Outcomes, 

Process) Model

3. Foster Active Learning

Laura

Tina

• Lecturette 

• Backward design

• Pop activity

10:00 :10 Break Laura 

10:10 :40 Master: Technology For Virtual Learning And 

Facilitation

1. Poll

2. Chat

3. Share Screen

4. Show Video

5. Touch up

6. Manage Security

7. Host like a Boss

Laura Lecturette

Modeling

Video

10:50 :10 Break

11:00 :35 Build: A Toolbox For Engaging Learners Digitally

1. Enlist help!

2. Pre-game

3. Break the ice

4. Try “Small Teaching” strategies

5. Practice! 

Laura • Lecturette

• Assessment

• Application

• Poll

• Debrief

11:35 :10 Reflect: On our Learning and Process

1. Recall active learning tools

2. Share next steps

Laura • Lecturette

• Application

• At-large discussion

11:45 :00 Adjourn All

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 4
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 5

 

Quotes

Technology will never replace great teachers, but in the hands of great teachers, it’s transformational. 

-George Couros

Technology will not replace teachers, but teachers who use technology will replace those who do not. 

-Unknown

2-word Check-in

Why do 2-word Check-ins matter?

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 5
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 6

 

Backward Design

Established Outcomes Transfer

The enduring

understandings and 

learning goals of the 

lesson, unit, or course.

Learners will be able to independently use their learning to...

(Refers to how students will transfer the knowledge gained from the lesson, unit, or course 

and apply it outside of the context of the course).

Meaning

Understandings

Students will understand that...

(Refers to the big ideas and specific 

understandings learners will have 

when the complete the lesson, unit, or 

course).

Essential Questions

(Refers to the provocative questions that foster 

inquiry, understanding, and transfer of learning. 

These questions typically frame the lesson, unit, or 

course and are often revisited. If learners attain the 

established goals, they should be able to answer the 

essential question(s)).

Acquisition

Learners will know...

(Refers to the key knowledge learners 

will acquire from the lesson, unit, or 

course).

Learners will be skilled at...

(Refers to the key skills learners will acquire from 

the lesson, unit, or course).

Stage 2 – Evidence and Assessment

Evaluative Criteria Assessment Evidence

Refers to the various 

types of criteria that 

learners will be 

evaluated on.

Performance Task(s):

Refers to the authentic performance task(s) that learners will complete to demonstrate the 

desired understandings or demonstrate they have attained the goals. The performance task(s) 

are typically larger assessments that coalesce various concepts and understandings like large 

projects or papers.

Other Evidence:

Refers to other types of evidence that will show if learners have demonstrated achievement of 

the desired results. This includes quizzes, tests, homework, etc. This is also a good point to 

consider incorporating self-assessments and student reflections.

Stage 3 – Learning Plan

Summary of Key Learning Events and Instruction—The POP (Purpose, Outcomes, Process)

This stage encompasses the individual learning activities and instructional strategies that will be employed. This 

includes lectures, discussions, problem-solving sessions, etc.

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 6
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 7

 

Backward Design, Example of This Workshop

Established Outcomes Transfer

• Share a welcome, 

overview, and 

check-in.

• Design to 

facilitate active, 

adult learning

• Master technology 

for virtual learning 

and facilitation

• Reflect on our 

learning process.

Learners will be able to independently use their learning to... model effective design and 

delivery of active pedagogy for online, synchronous (live) education and meetings that 

engages participants using evidence-based practices.

Meaning

Understandings

Students will understand that...

• Pedagogy should drive the 

learning, not technology

• Active learning facilitates 

understanding, retention, and 

transfer

• Effective virtual design can be 

small and gradual

Essential Questions

1. How can I create active, meaningful 

sessions?

2. How can I help learners connect with each 

other in virtual space?

3. How can I privilege pedagogy over 

technology in this work?

4. How will I quell my fear of virtual 

facilitation? 

5. What will I do differently?

Acquisition

Learners will know...

• Design steps for virtual 

gatherings

• Principles of active, adult 

learning

• Logistics of zoom technology 

Learners will be skilled at...

• Designing active, adult learning experiences 

in virtual settings

• Mastering technology for virtual learning and 

facilitation

• Building a toolbox for engaging learners 

digitally

Stage 2 – Evidence and Assessment

Evaluative Criteria Assessment Evidence

Mastery of active, adult 

learner-friendly 

pedagogy and use of the 

virtual technology

Performance Task(s):

1. Design active, online teaching and meetings using backward design and effective 

adult learning pedagogy

2. Use Zoom effectively to facilitate virtual sessions. 

3. Apply small teaching strategies to make virtual sessions engaging and optimize 

learning. 

4. Deliver effective virtual instruction and facilitation

Other Evidence:

1. Participation in the Zoom Practice Lab

2. Teaching others ho to master virtual pedagogy and technology

Stage 3 – Learning Plan

Summary of Key Learning Events and Instruction—The POP (Purpose, Outcomes, Process), page 9, 

with examples on pages 4 and 9 of this guide.

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 7
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 8

 

Backward Design, Your Turn

Established Outcomes Transfer

•

Learners will be able to independently use their learning to...

Meaning

Understandings

Students will understand that...

Essential Questions

Acquisition

Learners will know... Learners will be skilled at...

Stage 2 – Evidence and Assessment

Evaluative Criteria Assessment Evidence

Performance Task(s):

Other Evidence:

Stage 3 – Learning Plan

Summary of Key Learning Events and Instruction—The POP (Purpose, Outcomes, Process), page 9, 

with examples on pages 4 and 9 of this guide.

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 8
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 9

 

The POP (Purpose, Outcomes, Process)

Example of a “POP” (Purpose, Outcomes, Process)* 

Training 

Title:

Train-the-Trainer Workshop Introduction

Purpose: Share and Process information related to becoming an effective trainer.

Outcomes: 1. Explain the concepts of effective training and development.

2. Reflect on your role as a trainer and articulate a teaching vision for yourself.

3. Raise concerns and hopes about becoming a trainer.



Time Allotted Learning Outcome Methods and Activities Materials and Media 

8:00 :30 Outcome: Welcome 

everyone. Set stage for 

day. Conduct icebreaker.

1. Welcome participants.

2. Facilitate introductions

3. Play the Name Game

• Agenda

• KooshTM Ball

8:30 1:00 Outcome: Explain the 

concepts of effective 

training and development.

1. Lecture on core principles.

2. Paired reflections. 

Participants will teach 

each other core points 

from lecture. 

• Workbook

• LCD 

projector

9:30 :15 Break Break • Break

9:45 1:05 Outcome: Reflect on your 

role as a trainer and 

articulate a teaching vision 

for yourself.

1. Participants individually 

draw a picture of their 

vision of themselves as 

trainers.

2. Participants will share 

pictures with table. Each 

table will share a brief 

summary of highlights.

3. Post pictures on the wall. 

• Music

• LCD 

projector

• Crayons/paper

10:50 :10 Break Break • Break

11:00 :45 Outcome: Raise concerns 

and hopes about becoming 

a trainer.

1. Dialogue about the issues. • Dialogue Ground 

Rules

• Chairs arranged 

in a circle

• KooshTM Ball

11:45 :15 Outcome: Share closing 

thoughts, comments, and 

feedback on the course 

content and facilitation. 

1. Round Robin sharing. 

Passing is okay.

12:00 :00 Adjourn Adjourn Adjourn

* Backward Design & POP Template included in participant materials for this course.

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 9
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 10

 

Learning and Meeting Activity Level Chart

Activity Level of Instructional Techniques 

High Medium Low

Group Discussion Reaction Panel Lecture

Buzz Group Screened Speech/Video Panel

Case Study Listening Group Symposium

Game Behavior Modeling Demonstration

Simulation, Application Role Playing Self-assessment

In-basket Storytelling Quizzes

Structured Experience Silence Voting

Critical Incident Observation Polling

Trial and Error Reflective Practice Reading

Jigsaw, Peer Teaching Dialogue 

Categorization 1-Minute Paper/Writing

Walking, Movement Concept Mapping

Circle some you commit to try in the next virtual session you facilitate. 

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 10

Click here for definitions of Learning and Meeting Activities.
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 11

 

Your Turn: The “POP” Template

Session 

Title: 

Purpose:

Outcomes:



Time Allotted Learning Outcome Methods and Activities Materials and 

Media 

:00 Adjourn Adjourn Adjourn

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 11
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 12

 

Abridged Small Teaching Online Strategies

Inventory of Small Online Teaching Practices 

(Based on Darby & Lang's 2019 book, Small Online Teaching)

Scan the items and identify 2 that you commit to trying, pp. 12-14. 

Designing for Learning:

Never Rarely Now 

and 

Then

Often Always

1. I use backward design—essentially planning the course with the 

end in mind by determining what learners should know when the 

class finishes. 

2. I make the purpose of class activities explicit (e.g., including the 

rationale of pedagogical decisions in module descriptions).

3. I make the purpose of class assignments explicit (e.g., what to do, 

why it is assigned, how to do it).

Guiding Through Learner Engagement:

Never Rarely Now 

and 

Then

Often Always

4. I scour class interactions for clues to whether learners understand 

the content (e.g., ask for explanations or illustrations, analyze 

learner questions, engage regularly in online discussions).

5. I provide module discussion highlights to emphasize important 

concepts from the discussions, particularly by highlighting 

exceptional posts.

Using Media and Technology Tools:

Never Rarely Now 

and 

Then

Often Always

6. I use teaching methods that make it easier, not more difficult, for 

learners to learn. 

7. I create short (3-5 minute) lecture videos (these can be short 

narrative videos webcam-style videos where you speak directly to 

the camera). 

8. I find the right tech tool for the job, instead of letting technology 

drive the curriculum (begin with the learning objective and then

select the technology). 

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 12
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 13

 

Building Community:

Never Rarely Now 

and 

Then

Often Always

9. I understand and build in the three primary aspects of building a 

learning community: 1) Cognitive presence—meaning creation 

through sustained communication), 2) Social presence—promoting 

emotional expression, open communication, and group cohesion, 

and 3) Teaching presence—good learning design and facilitation. 

10. I design and teach for cultural inclusion. 

11. I have sought continuing education in culturally responsive online 

teaching. 

Giving Feedback:

Never Rarely Now 

and 

Then

Often Always

12. I put myself in my students’ shoes and attempt to understand how 

the class is experienced from their vantage points. 

13. I give meaningful comments via media tools such as recording 

audio or video to give feedback rather than relying on text.

Fostering Student Persistence and Success:

Never Rarely Now 

and 

Then

Often Always

14. I assign a goals or learning contract to clarify expectations for how 

they should behave and can succeed in the course. 

15. I use mastery quizzes that are nonconsequential to help students 

assess their learning. 

Creating Autonomy:

Never Rarely Now 

and 

Then

Often Always

16. I encourage groups to create community agreements—basically a 

contract for how they will engage with each other. 

17. I hold students accountable for high-quality work and support them 

to create it.

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 13
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 14

 

Making Connections:

Never Rarely Now 

and 

Then

Often Always

18. I activate prior knowledge by asking what learners already know 

about a topic. This can happen through quizzes or writing activities 

that are assessed.

19. I embed concept or mind maps into the content to help learners 

visualize connections. 

20. I relate student learning to their personal experience, outside 

interests, and goals. 

21. I assign personal learning networks for students to make 

connections beyond the classroom (e.g., friends, colleagues, 

experts, or family).

22. I foster connections between course content and other unique 

things that will inspire and engage students such as work 

experiences or current events. We curate these items in a shared 

space. 

Developing as an Online Instructor:

Never Rarely Now 

and 

Then

Often Always

23. I have taken or will take an online class to see how it feels to be a 

student in an online context. 

24. I build self-efficacy for online teaching by making small changes to 

improve my teaching on an ongoing basis. 

25. I peruse resources and learning to continually develop my online 

teaching repertoire. 

26. I teach other educators what I know about effective online teaching. 

27. I make time to grow as an online educator by dedicating time and 

space to learn. 

28. I energize myself so I can energize and motivate learners. 

Bierema, L. L. (2021). Inventory of small online teaching practices. [Unpublished manuscript]. The University 

of Georgia.

2-Idea Check-out

List two key steps you will take as a result of this workshop on virtual teaching and facilitating.

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 14
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 15

 

References

Bierema, L. L. (2020c). Incorporating active learning into your educational repertoire. In M. Fedeli, & L. L. 

Bierema (Eds.). Connecting adult learning and knowledge management: Strategies for learning and 

change in higher education and organizations, pp. 27-50. New York, NY: Springer.

Darby, F., & Lang, J. M. (2019). Small teaching online: Applying learning science in online classes. John Wiley 

& Sons.

Eison, J. (2010). Using active learning instructional strategies to create excitement and enhance 

learning. Jurnal Pendidikantentang Strategi Pembelajaran Aktif (Active Learning) Books, 2(1), 1-10. 

Fullan, M. (2011). Choosing the wrong drivers for whole system reform (pp. 3-4). Melbourne: Centre for 

Strategic Education.

Swinehart, D. F. (1980). “Swinehart’s Definition,” in The Official Explanations by P. Dickson. New York: Dell 

Publishing Co.

University of Leicester. (n. d.). Active learning. Leicester Learning Institute. Retrieved from: 

https://www2.le.ac.uk/offices/lli/developing-learning-and-teaching/enhance/strategies/activelearning

Resources

Denna, M. (2020). UGA’s Zoom Skills: Meetings, Classes, and Workshops in the Digital 

World. (https://rise.articulate.com/share/zgKaEizjHLzWTaG0I477zoZaTEYJ9KoU)

Denna, M. (2020). UGA’s Tips for Supervising Remotely or https://hr.uga.edu/_resources/pdf/Tips-forsupervising-remotely.pdf

Fedeli, M., Bierema, L. L., & Taylor, E. (2020). Innovative teaching: Engaging adult learners with active 

learning. FutureLearn MOOC, https://www.futurelearn.com/courses/engaging-adult-learners-withactive-learning 

Mickel, B. How Brene Brown runs emotionally intelligent zoom meetings: It’s a sign of leadership, especially 

during challenging times. INC.: Working from Home. https://www.inc.com/betsy-mikel/how-brenebrown-runs-emotionally-intelligent-zoom-meetings.html 

Parker, P. (2020). The art of gathering: How we meet and why it matters. Riverhead Books.

Parker, P. (2020-Present). Together apart [Audio podcast]. New York Times. 

https://www.priyaparker.com/podcast

Stanford University. Teaching Effectively in Times of Disruption. 

https://docs.google.com/document/d/1ccsudB2vwZ_GJYoKlFzGbtnmftGcXwCIwxzfjkkoCU/preview#heading=h.tqx2r11qsh2q

University of Minnesota, Zoom: Teach Online Class Sessions or https://it.umn.edu/zoom-teach-online-classsessions#engaging

Using Zoom Meeting Controls: 

https://www.youtube.com/embed/ygZ96J_z4AY?rel=0&autoplay=1&cc_load_policy=1 

Woolf, N. The Two-word check-in: A powerful practice for connecting with students. Panorama Education, 

Social-Emotional Learning. https://www.panoramaed.com/blog/two-word-check-in-strategy 

Zoom Icebreakers: 55 Pro Tips to Energize your Virtual Meetings: https://www.rhythmsystems.com/blog/thebest-icebreakers-to-energize-your-virtual-meetings

Zoom. Tips & Tricks for Teachers Educating on Zoom or https://zoom.us/docs/doc/Tips and Tricks for 

Teachers Educating on Zoom.pdf

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 15
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 16

 

Biographical Information 

Dr. Laura L. Bierema

Professor, executive coach, company president, author, 

AHRD president, Lukas’ Fund co-founder, past faculty senate 

president, volunteer, avid runner sidelined by injury, exexecutive, cyclist, nonprofit board member, former dental 

assistant, disastrous waitress, rusty choral singer, failed pianist, 

aspiring potter, fumbling knitter, skilled vegetarian cook, 

bread maker, world traveler, Fulbright scholar, and a decent 

leader on her good days.

LAURA L. BIEREMA is Professor, University of Georgia (UGA), 

College of Education (COE). Her academic program is in 

Adult Learning, Leadership, and Organization Development. Previously, Dr. Bierema was Assistant Professor in 

the School of Labor and Industrial Relations at Michigan State University where she also led faculty 

development in the College of Human Medicine. Prior to her career in academia, Dr. Bierema held a variety 

of human resources management and executive positions in the automotive industry with AlliedSignal, Inc. 

(now Honeywell). From 2013-2016 Dr. Bierema was the UGA COE Associate Dean for Academic Programs.

Dr. Bierema’s research interests include workplace learning, career development, women’s 

development, organization development, executive coaching, leadership, and critical human resource 

development. Dr. Bierema holds both bachelor’s and master’s degrees from Michigan State University and a 

doctorate in adult education from UGA. She has published over 100 book chapters and articles that have 

appeared in both research and professional publications. Dr. Bierema’s books include:

• Women’s Career Development across the Lifespan

• Philosophy and Practice of Organizational Learning, Performance, and Change

• Critical Issues in Human Resource Development

• Implementing a Critical Approach to Organization Development

• Adult Learning: Linking Theory and Practice 

• An Introduction to Organization Development: An Action Research Approach, 1st and 2nd Editions

• Connecting Adult Learning and Knowledge Management: Strategies for Learning and Change 

In Organizations.

• Human Resource Development: Critical Perspectives and Practice (forthcoming)

Dr. Bierema is a Cyril O. Houle Scholar in Adult and Continuing Education and Lilly Fellow. She is the

recipient of the Richard A. Swanson Excellence in Research Award and four Academy of Human Resource 

Development’s “Cutting Edge” Awards for best conference paper. She is the 2009 recipient of the Highly 

Commended Award by the Emerald Literati Network Awards for Excellence; 2012 winner of the University of 

Georgia College of Education Russell H. Yeany, Jr. Research Award; 2012 recipient of the Sherpa Trailblazer 

of the Year Award in recognition of innovation application of the Sherpa Coaching Process; 2013 winner of 

the Academy of Human Resource Development’s Outstanding Scholar Award; 2014 winner of the Academy 

of Human Resource Development’s Book of the Year; 2015 winner of the University, Professional, and 

Continuing Education Association Phillip E. Frandson Award for Literature; 2017 recognition as a Master 

Executive Coach by Sherpa Coaching; 2018 U.S. Fulbright Research Scholar; 2018-2019 UGA Senior Teaching 

Fellow, 2019 recipient of the Ira Aaron Teaching Excellence and Collegiality Award; and 2021 Inductee into 

the Adult and Continuing Education Hall of Fame

Lest Dr. Bierema take herself too seriously, she balances her left brain by learning to knit, talking to her 

dogs, riding her bicycle around the world with her spouse Mark, tasting fine wines, collecting original art, 

cooking gourmet vegetarian meals, reading great literature and trashy novels, traveling to interesting places 

near and far, indulging in gastronomic adventures with friends, and hitting the beach whenever possible, 

preferably in Onekama, MI. 

MJI eLearning Faculty Development Workshop | August 31, 2021 

Page 16
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 10

Learning and Meeting Activity Level Chart

Activity Level of Instructional Techniques 

High Medium Low

Group Discussion

Have small groups discuss course 

material and field questions from 

learners in their audience.

Reaction Panel

A group of experts present short 

presentations and a moderator 

guides the discussion and elicits 

audience questions.

Lecture

An educational talk to an 

audience. Generally passive. 

Buzz Group

A small group of people that is 

divided from a larger group to talk 

about ideas, solutions to problems 

for a short period of time. 

Screened Speech/Video

Watch or listen to a speech or video 

an discuss. Give prompts to listen or 

watch for (e.g., what was the key 

question/point?).

Panel

A group of people who answer 

questions, give advice or opinions, 

or take part in a discussion for an 

audience.

Case Study

Learners collaborate to learn course 

concepts and develop key skills and 

competencies while focusing on real 

life problems.

Listening Group

Learners listen to each other’s work 

and discuss it.

Symposium

A collection of essays or papers on 

a particular subject by a number of 

contributors.

Game

Create or use a web-based game 

platform to review lesson. 

Behavior Modeling

Showing learners how to do 

something and guiding them through 

the process of imitating the modeled 

behavior.

Demonstration

A practical exhibition and 

explanation of how something 

works or is performed.

Simulation, Application

Learners and/or faculty perform 

specific roles for demonstration 

purposes. Include guiding principles, 

specific rules, and structured 

relationships.

Role Playing

The acting out of the part of a 

particular person or character that is 

the focus of the lesson.

Self-assessment

Evaluation of oneself or one's 

actions and attitudes related to

performance of a job or learning 

task against an objective standard.

In-basket

An assessment too to evaluate how 

well learners can make decisions 

related to content.

Storytelling

Tell stories about the topic to spark

interest, aid the flow of lectures, 

make material memorable and build 

rapport.

Quiz

A test of knowledge, especially a 

brief, informal test given to 

learners.

Structured Learning ExperienceSLE

An experiential, supervised, in-depth 

learning activity that allows learners 

to experience the concept (i.e., 

work-based learning).

Silence

Often underrated, silence can make 

space for reflection, demonstration, 

collaboration, and concentration.

Voting or Polling

A low-stakes, real-time 

engagement of earners around

course content through reflection, 

opinion assessment, or knowledge

or skill evaluation, and/or skills in 

MJI eLearning Faculty Development Workshop | August 31, 2021 

Definitions of Learning and Meeting Activities
August/September 2021 eFaculty Development Workshop PARTICIPANT GUIDE

page 11

Activity Level of Instructional Techniques 

High Medium Low

real time and with low or no 

stakes.

Critical Incident

Learners identify situations where 

they have experienced effective, 

exceptional, challenging, or 

personally meaningful outcomes. 

Observation

Learning by watching the behaviors 

and interactions of other people and 

then mimicking, critiquing, or 

analyzing them. 

Reading

Learners read silently or are 

invited to read passages aloud to 

the class.

Trial and Error

A problem solving method in which 

multiple attempts are made to reach 

a solution.

Reflective Practice

Examining one's underlying beliefs 

and assumptions about topic or 

concept. 

Video

Showing film clips to learners that 

relate to the course content. 

Jigsaw/Peer Teaching

Team based activity where each 

member/team becomes a subject 

matter expert in course material and 

teaches their subject to their team 

members./Learners teach each other 

basic/intermediate levels of course 

materials or needed skills.

Dialogue 

Communication that prioritizes 

meaning making over idea advocacy 

where participants suspend 

judgment and ask questions of 

genuine curiosity. 

Audio

Playing audible content for 

learners to listen and react to 

(e.g., speeches, poetry, music, 

literature). 

Categorization

Grouping objects or ideas according 

to criteria that describe common 

features or the relationships among 

all members of the concept. 

1-Minute Paper/Writing

Short writing task designed to focus 

learners on one term or concept 

from a lesson. Can be used for 

assessment or reflection. 

Walking, Movement

Adults learn better through 

movement. Try stretch breaks, short 

walks with a partner to discuss the 

topic and retain material. 

Concept Mapping

Visual representations of information 

in the form of charts, graphic 

organizers, tables, flowcharts, Venn 

Diagrams, timelines, or T-charts.

Think—Pair—Share

Short individual reflection or writing 

to a prompt or question; then paired 

discussion with a partner; then

sharing with a larger group.

Circle some you commit to try in the next virtual session you facilitate.

MJI eLearning Faculty Development Workshop | August 31, 2021 

Definitions of Learning and Meeting Activities
The Backward Design & POP (Purpose, Outcomes, Process): 

Examples and Templates

Created by Dr. Laura Bierema, University of Georgia, 2021

MJI eLearning Faculty Development Workshop | 2021

Page 1

Backward Design

Established 

Outcomes

Transfer

The enduring 

understandings and 

learning goals of the 

lesson, unit, or 

course.

Learners will be able to independently use their learning to...

(Refers to how students will transfer the knowledge gained from the lesson, unit, or 

course and apply it outside of the context of the course).

Meaning

Understandings

Students will understand that...

(Refers to the big ideas and specific 

understandings learners will have 

when the complete the lesson, unit, 

or course).

Essential Questions

(Refers to the provocative questions that foster 

inquiry, understanding, and transfer of learning. 

These questions typically frame the lesson, unit, 

or course and are often revisited. If learners

attain the established goals, they should be able 

to answer the essential question(s)).

Acquisition

Learners will know...

(Refers to the key knowledge 

learners will acquire from the 

lesson, unit, or course).

Learners will be skilled at...

(Refers to the key skills learners will acquire 

from the lesson, unit, or course).

Stage 2 – Evidence and Assessment

Evaluative Criteria Assessment Evidence

Refers to the various 

types of criteria that 

learners will be 

evaluated on.

Performance Task(s):

Refers to the authentic performance task(s) that learners will complete to demonstrate 

the desired understandings or demonstrate they have attained the goals. The 

performance task(s) are typically larger assessments that coalesce various concepts and 

understandings like large projects or papers.

Other Evidence:

Refers to other types of evidence that will show if learners have demonstrated 

achievement of the desired results. This includes quizzes, tests, homework, etc. This is 

also a good point to consider incorporating self-assessments and student reflections.

Stage 3 – Learning Plan

Summary of Key Learning Events and Instruction—The POP (Purpose, Outcomes, Process)

This stage encompasses the individual learning activities and instructional strategies that will be employed. 

This includes lectures, discussions, problem-solving sessions, etc.
The Backward Design & POP (Purpose, Outcomes, Process): 

Examples and Templates

Created by Dr. Laura Bierema, University of Georgia, 2021

MJI eLearning Faculty Development Workshop | 2021

Page 2

Backward Design, Example of This Workshop

Established Outcomes Transfer

• Share a 

welcome, 

overview, and 

check-in.

• Design to 

facilitate active, 

adult learning

• Master

technology for 

virtual learning 

and facilitation

• Reflect on our 

learning process.

Learners will be able to independently use their learning to... model effective design 

and delivery of active pedagogy for online, synchronous (live) education and 

meetings that engages participants using evidence-based practices.

Meaning

Understandings

Students will understand that...

• Pedagogy should drive the 

learning, not technology

• Active learning facilitates 

understanding, retention, 

and transfer

• Effective virtual design can 

be small and gradual

Essential Questions

1. How can I create active, meaningful 

sessions?

2. How can I help learners connect with each 

other in virtual space?

3. How can I privilege pedagogy over 

technology in this work?

4. How will I quell my fear of virtual 

facilitation? 

5. What will I do differently?

Acquisition

Learners will know...

• Design steps for virtual 

gatherings

• Principles of active, adult 

learning

• Logistics of zoom 

technology

Learners will be skilled at...

• Designing active, adult learning 

experiences in virtual settings

• Mastering technology for virtual learning 

and facilitation

• Building a toolbox for engaging learners 

digitally

Stage 2 – Evidence and Assessment

Evaluative Criteria Assessment Evidence

Mastery of active, adult 

learner-friendly 

pedagogy and use of the 

virtual technology

Performance Task(s):

1. Design active, online teaching and meetings using backward design 

and effective adult learning pedagogy

2. Use Zoom effectively to facilitate virtual sessions. 

3. Apply small teaching strategies to make virtual sessions engaging and 

optimize learning. 

4. Deliver effective virtual instruction and facilitation

Other Evidence:

1. Participation in the Zoom Practice Lab

2. Teaching others ho to master virtual pedagogy and technology

Stage 3 – Learning Plan

Summary of Key Learning Events and Instruction—The POP (Purpose, Outcomes, Process), page 9, 

with examples on pages 4 and 9 of this guide.
The Backward Design & POP (Purpose, Outcomes, Process): 

Examples and Templates

Created by Dr. Laura Bierema, University of Georgia, 2021

MJI eLearning Faculty Development Workshop | 2021

Page 3

 

Backward Design, Your Turn

Established 

Outcomes

Transfer

Learners will be able to independently use their learning to...

Meaning

Understandings

Students will understand that...

Essential Questions

Acquisition

Learners will know... Learners will be skilled at...

Stage 2 – Evidence and Assessment

Evaluative Criteria Assessment Evidence

Performance Task(s):

Other Evidence:

Stage 3 – Learning Plan

Summary of Key Learning Events and Instruction—The POP (Purpose, Outcomes, Process), next page
The Backward Design & POP (Purpose, Outcomes, Process): 

Examples and Templates

Created by Dr. Laura Bierema, University of Georgia, 2021

MJI eLearning Faculty Development Workshop | 2021

Page 4

 

Example of a “POP” (Purpose, Outcomes, Process)

Session 

Title:

Train-the-Trainer Workshop Introduction

Purpose: Share and Process information related to becoming an effective trainer.

Outcomes: 1. Explain the concepts of effective training and development.

2. Reflect on your role as a trainer and articulate a teaching vision for yourself.

3. Raise concerns and hopes about becoming a trainer.



Time Allotted Learning Outcome Methods and Activities Materials and 

Media 

8:00 :30 Outcome: Welcome 

everyone. Set stage for 

day. Conduct icebreaker.

1. Welcome participants.

2. Facilitate introductions

3. Play the Name Game

• Agenda

8:30 1:00 Outcome: Explain the 

concepts of effective 

training and development.

1. Lecture on core 

principles.

2. Paired reflections. 

Participants will teach 

each other core points 

from lecture. 

• Workbook

• LCD

projector

9:30 :15 Break Break • Break

9:45 1:05 Outcome: Reflect on your 

role as a trainer and 

articulate a teaching vision 

for yourself.

1. Participants individually 

draw a picture of their 

vision of themselves as 

trainers.

2. Participants will share 

pictures with table. Each 

table will share a brief 

summary of highlights.

3. Post pictures on the wall. 

• Music

• LCD

projector

• Crayons/paper

10:50 :10 Break Break • Break

11:00 :45 Outcome: Raise concerns 

and hopes about becoming 

a trainer.

1. Dialogue about the 

issues. 

• Dialogue 

Ground Rules

• Chairs arranged 

in a circle

11:45 :15 Outcome: Share closing 

thoughts, comments, and 

feedback on the course 

content and facilitation. 

1. Round Robin sharing. 

Passing is okay.

12:00 :00 Adjourn Adjourn Adjourn
The Backward Design & POP (Purpose, Outcomes, Process): 

Examples and Templates

Created by Dr. Laura Bierema, University of Georgia, 2021

MJI eLearning Faculty Development Workshop | 2021

Page 5

 “POP” (Purpose, Outcomes, Process) Template

Training 

Title:

Purpose:

Outcomes:



Time Allotted Learning Outcome Methods and Activities Materials and 

Media 

:00 Adjourn Adjourn Adjourn
1

Inventory of Small Online Teaching Practices 

(Based on Darby & Lang's 2019 book, Small Online Teaching)

Select one course to consider when responding below. Focus course:

Designing for Learning:

Never Rarely Now 

and 

Then

Often Always

1. I use backward design—essentially planning the course with the

end in mind by determining what learners should know when the

class finishes.

2. I begin working on the final assessment in Week 1 (e.g., activity to

familiarize students with the assessment, Q&A, low stake quizzes,

or student reiteration of their understanding of the assignment).

3. I make the purpose of class activities explicit (e.g., including the

rationale of pedagogical decisions in module descriptions).

4. I make the purpose of class assignments explicit (e.g., what to do,

why it is assigned, how to do it).

5. I invite students to reflect on and respond to learning objectives for

the course with activities such as connecting it to current

knowledge or rephrasing key ideas.

6. I look back and look ahead by revisiting an early activity later in the

semester (e.g., ask students to write about a principle they learned

early in the semester at the midpoint or later using 3-Takeaways—

what were 3 things I learned?, or a Think-Pair-Share activity).

7. I provide learners frequent reminders about the purpose and

content of course activities.

8. I point students back to the core objectives of the course.

9. I connect the beginning and ending by helping students see what

they are working toward on day 1 of the semester.

10. At the end of the semester, I invite learners to reflect on how far

they have come in their learning and how they will continue

learning after the course .

Guiding Through Learner Engagement:

Never Rarely Now 

and 

Then

Often Always

11. I break down complex tasks into smaller steps with checklists, miniassignments, outline submission for projects/papers for feedback,

annotated bibliographies, and published incremental deadlines.

12. I release content strategically by using the “conditional release”

(CR) function in my learning management system (e.g., requiring a

certain level of mastery on a quiz or assignment before students

can move to the next project or assignment).

13. I scour class interactions for clues to whether learners understand

the content (e.g., ask for explanations or illustrations, analyze

learner questions, engage regularly in online discussions).

14. I provide module discussion highlights to emphasize important

concepts from the discussions, particularly by highlighting

MJI eLearning Faculty Development Workshop | 2021 

Page 1
2

Guiding Through Learner Engagement:

Never Rarely Now 

and 

Then

Often Always

exceptional posts.

15. I give frequent feedback to help students know whether they are on 

track. 

Using Media and Technology Tools:

Never Rarely Now 

and 

Then

Often Always

16. I use teaching methods that make it easier, not more difficult, for

learners to learn. 

17. I provide alternative means of access to the materials and tools I 

used for online teaching (check with your institution’s disability or 

accessibility resources department). This might include captioning 

videos, providing transcripts for audio and video, or tagging 

images.

18. I create short (3-5 minute) lecture videos (these can be short 

narrative videos webcam-style videos where you speak directly to 

the camera). 

19. I spur engagement with online content such as short videos with 

brief assessments, quizzes, short writing assignments, or learner 

video or audio responses. 

20. I leverage video for spontaneous updates such as clearing up 

confusion about concepts, sharing class announcements, or giving 

feedback. 

21. I source existing video and audio media on the subject matter such 

as movie clips, podcasts, news lips, images, TEDtalks, song lyrics, 

YouTube videos, with considerations for copyright and 

accessibility. 

22. I find the right tech tool for the job, instead of letting technology 

drive the curriculum (begin with the learning objective and then

select the technology). 

Building Community: Never Rarely Now 

and 

Then

Often Always

23. I understand and build in the three primary aspects of building a 

learning community: 1) Cognitive presence—meaning creation 

through sustained communication), 2) Social presence—promoting 

emotional expression, open communication, and group cohesion, 

and 3) Teaching presence—good learning design and facilitation. 

24. I structure student interactions in the zone of proximal development

to help students learn from each other as well as me through online 

discussions, student-posted introductions/bios, meaningful online 

discussion prompts, small group discussions, and encouraging 

social interactions to support learning. 

25. I show up regularly for my students with frequent posting of video 

or text announcements, engaging in the online discussion forum, 

giving timely feedback, and letting them know when I fall behind. 

Learners feel my presence. 

MJI eLearning Faculty Development Workshop | 2021 

Page 2
3

Building Community: Never Rarely Now 

and 

Then

Often Always

26. I reveal my personality to learners by sharing my personal and 

professional interests, creating a “Meet Your Instructor” section of 

the online class with a photo and bio or video, and intentionally 

connecting with learners. 

27. I design and teach for cultural inclusion. 

28. I am mindful of differences in ethnicity, culture, sexual orientation, 

political affiliation, gender identity, faith backgrounds, and other 

aspects of human diversity among my learners. 

29. I have sought continuing education in culturally responsive online 

teaching. 

30. I monitor online discussions for culturally sensitive language and 

behavior. 

31. I take time to learn students’ names, nicknames, correct name 

pronunciation, preferred pronouns, and other aspects of student 

culture and heritage that are important to them. 

32. I convey care and support to learners by getting to know them, 

considering their needs, understanding their commitments, 

providing flexibility when needed, valuing them as individuals and 

learners, conveying empathy, following up with students who 

disappear, asking how the class or semester is going, and offering 

an “Oops Token,” for a no-questions-asked deadline extension or 

way to make up for an unexpected challenge or honest mistake.

Giving Feedback:

Never Rarely Now 

and 

Then

Often Always

33. I am timely and responsive when it comes to giving feedback to 

learners about their progress. 

34. I put myself in my students’ shoes and attempt to understand how 

the class is experienced from their vantage points. 

35. I set deadlines strategically based on when I am available for 

student queries and grading. 

36. I have real, just-in-time conversations with learners such as 

optional, synchronous sessions to review course content, answer 

questions, or discuss assignments. 

37. I schedule brief 1:1 meetings with students when feasible (this 

strategy would not work with very large classes).

38. I creatively schedule virtual office hours by rebranding them to 

“coffee breaks,” “afternoon tea,” or “consultations”; holding them 

less frequently (e.g., 4x per semester); scheduling them at useful 

times such as prior to an exam or deadline; announcing the 

schedule ahead of time, using a meeting scheduler to find the best 

time; encouraging students to submit questions in advance; and 

offering incentives to attend such as extra credit (with comparable 

opportunity for those not attending to earn it).

39. I use technology to streamline grading such as creating rubrics and 

MJI eLearning Faculty Development Workshop | 2021 

Page 3
4

Giving Feedback:

Never Rarely Now 

and 

Then

Often Always

using the rubric tool within my LMS (learning management system).

40. I give meaningful comments via media tools such as recording 

audio or video to give feedback rather than relying on text.

Fostering Student Persistence and Success:

Never Rarely Now 

and 

Then

Often Always

41. I help students commit to and achieve success with assignments 

that help students evaluate their readiness for the class and plan 

their semester. 

42. I provide a lot of structure to help my students develop and 

demonstrate mastery of foundational concepts before moving to 

more complex ones. 

43. I give students the personal touch of outreach to show I care and 

offer help. 

44. I nudge students who might need extra help or attention by 

showing struggling students I am paying attention and trying to help 

them. 

45. I assign a goals or learning contract to clarify expectations for how 

they should behave and can succeed in the course. 

46. I use mastery quizzes that are nonconsequential to help students 

assess their learning. 

47. I scaffold assignments by creating opportunities for students to 

succeed in simple, low-stakes tasks as they work toward more 

complex and higher-stakes tasks.

Creating Autonomy:

Never Rarely Now 

and 

Then

Often Always

48. I offer choice in online discussions such as multiple topics for them 

to choose from in the discussion threads. This increases learner 

agency, relevance, and capacity to create meaning.

49. I foster learner independence and autonomy. When learners have 

some control over their learning, agency to design learning 

activities, and permission to evaluate their learning, they are fully 

engaged in self-directed learning—a hallmark of adult learning. 

50. I create self-enroll scaffolded groups—this allows students to selfselect groups based on topics of interest on a first-come basis. 

Students then design a process of self-directed study. Such 

activities should include: 1) A clear rationale/purpose for the 

activity; 2) Structured community building; 3) Conflict resolution 

guidelines; and 4) Explicit grading criteria.

51. I encourage groups to create community agreements—basically a 

contract for how they will engaged with each other. 

52. I apply specifications grading that helps learners take responsibility 

for how they will be held accountable for their learning and the 

grading is all-or-nothing (if they meet the standard, they get credit, 

if they don’t they get zero). 

MJI eLearning Faculty Development Workshop | 2021 

Page 4
5

Creating Autonomy:

Never Rarely Now 

and 

Then

Often Always

53. I annotate, tweak, and co-design my syllabus. This involves

engaging students in controlling their learning and ensuring it is 

relevant to their needs. You can invite learners to annotate the 

document, share feedback, and discuss their interpretations of it.

54. I hold students accountable for high-quality work and support them 

to create it.

Making Connections:

Never Rarely Now 

and 

Then

Often Always

55. I activate prior knowledge by asking what learners already know 

about a topic. This can happen through quizzes or writing activities 

that are assessed.

56. I provide the framework for learning such as an outline or other 

organizing structure for students to complete content while reading 

text, watching a video, or moving through major sections of the 

course. 

57. I embed concept or mind maps into the content to help learners 

visualize connections. 

58. I relate student learning to their personal experience, outside 

interests, and goals. 

59. I assign personal learning networks for students to make 

connections beyond the classroom (e.g., friends, colleagues, 

experts, or family).

60. I foster connections between course content and other unique 

things that will inspire and engage students such as work 

experiences or current events. We curate these items in a shared 

space. 

Developing as an Online Instructor:

Never Rarely Now 

and 

Then

Often Always

61. I have taken or will take an online class to see how it feels to be a 

student in an online context. 

62. I seek online teaching exemplars by attending faculty professional 

development activities, observing online classes of outstanding 

teachers, visiting a Center for Teaching and Learning such as the 

University of Georgia CTL<https://ctl.uga.edu/> to identify 

exemplars of online courses and other resources, and curating

online sources of inspiration such as the Teaching Online 

Pedagogical Repository at the University of Central Florida

<https://topr.online.ucf.edu/>.

63. I build self-efficacy for online teaching by making small changes to 

improve my teaching on an ongoing basis. 

64. I certify my course design and teaching quality through rubrics 

available from Quality Matters (QM) Higher Education Rubric 

<https://www.qualitymatters.org/qa-resources/rubricstandards/higher-ed-rubric> and OSCOR (“Oscar”) from the Open 

SUNY (State of New York) Course Quality Review Rubric <

https://oscqr.suny.edu/>. 

MJI eLearning Faculty Development Workshop | 2021 

Page 5
6

Developing as an Online Instructor:

Never Rarely Now 

and 

Then

Often Always

65. I peruse resources and learning to continually develop my online 

teaching repertoire. 

66. I read books on relevant topics to improve my teaching such as 

Online Teaching at its Best: Merging Instructional Design with 

Teaching and Learning Research (Nilson & Goodson, 2018), 

Thrive Online: A New Approach for College Educators (Riggs, 

2019), The Online Teaching Survival Guide: Simple and Practical 

Pedagogical Tips (Boettcher & Conrad, 2016), Reach Everyone, 

Teach Everyone: Universal Design for Learning in Higher 

Education (Tobin & Behling, 2018), and Minds Online: Teaching 

Effectively with Technology (Millner, 2014).

67. I access other resource for ongoing development of my online 

teaching presence such as Faculty Focus 

<https://www.facultyfocus.com/>, Teaching in Higher Ed Podcast 

<https://teachinginhighered.com/episodes/>, The Teaching 

Professor Conference by Magna Publications 

<https://www.magnapubs.com/teaching-professor-conference/>, 

and the ACUE Course in Effective Teaching Practices 

<https://acue.org/?acue_courses=effective-teaching-practices> . 

68. I teach other educators what I know about effective online teaching. 

69. I make time to grow as an online educator by dedicating time and 

space to learn. 

70. I energize myself so I can energize and motivate learners. 

Bierema, L. L. (2021). Inventory of small online teaching practices. [Unpublished manuscript]. The University of Georgia.

MJI eLearning Faculty Development Workshop | 2021 

Page 6
Zoom Practice Lab Signup

The purpose of the Zoom Practice Lab is to provide a safe space for participants in the eFaculty 

Development for MJI Teaching and Facilitation Workshop to test various zoom features with a 

small peer audience and trouble shoot any issues before facilitating a workshop with a live 

audience. Dr. Laura Bierema will be your host and guide for the Zoom Practice Lab. 

Signup is on a first-come basis. You will receive a notification and Zoom link prior to the session.

Date and Time Name E-mail 

9-7-21

4:00-5:00 PM

1.

2.

3.

4.

5.

9-8-21

2:00-3:00 PM

1.

2.

3.

4.

5.

9-8-21

5:00-6:00 PM

1.

2.

3.

4.

5.

9-10-21

2:00-3:00 PM

1.

2.

3.

4.

5.

9-10-21

3:00-4:00 PM

1.

2.

3.

4.

5.

9-10-21

5:30-6:30 PM

1.

2.

3.

4.

5.

9-12-21

2:00-3:00 PM

1.

2.

3.

4.

5.

9-12-21

3:00-4:00 PM

1.

2.
Date and Time Name

E-mail 

9-12-21

3:00-4:00 PM, 

cont.

3.

4.

5.

9-12-21

5:00-6:00 PM

1.

2.

3.

4.

5.

9-14-21

3:00-4:00 PM

1.

2.

3.

4.

5.

9-14-21

4:00-5:00 PM

1.

2.

3.

4.

5.

9-14-21

5:00-6:00 PM

1.

2.

3.

4.

5.

9-15-21

2:00-3:00 PM

1.

2.

3.

4.

5.

9-15-21

3:00-4:00 PM

1.

2.

3.

4.

5.

9-15-21

4:00-5:00 PM

1.

2.

3.

4.

5.

9-15-21

5:00-6:00 PM

1.

2.

3.

4.

5.
Michigan Judicial Institute (MJI) Conversation Guide

Interview Instrument Draft 

Introduction

Hello! As you may recall, my name is Jori Hall, and I am serving as the external evaluator for 

the MJI eLearning Faculty Development project. As the evaluator, I am interested in learning 

about your experiences during the MJI eLearning Faculty Development Virtual Workshop. I will 

use what you share with me to write an evaluation report that I will provide to MJI. Given your 

court personnel position, you offer a unique perspective that will inform the report and future 

MJI Faculty Development offerings. 

Before we begin the interview, I would like to remind you that the information you share will be 

kept confidential. Your name and any other identifying information will be redacted from the

evaluation report. Feel free to skip any questions you do not want to answer. I anticipate the 

interview will take approximately 30 minutes. Though I will be asking you questions, if, at any 

time, you have questions throughout the interview, please feel free to ask. 

At this point, do you have any questions for me before we begin?

Question Probes

To begin, I would like to learn a little about you 

and your interest in the eLearning Faculty 

Development workshop. 

1. Please briefly share a bit about your 

position. 

2. Given your position, what lead to your 

decision to attend the MJI eLearning 

Faculty Development Virtual Workshop?

Now, I would like for us to take a minute to 

consider all that happened during the MJI 

eLearning Faculty Development Virtual Workshop.

3. How did what was presented in the 

workshop affect your online facilitation or 

the way you think about your online 

facilitation? 

• Was there a change in how you 

incorporate or think about virtual 

tools (chat, share screen, polls, 

video, break-out rooms, 

discussion forum)? 
Give a specific example of this change:

4. What happened? 

5. What caused it to happen? 

• Was there a change in action or 

your thinking about how to 

facilitate active, adult learning?

• Did, you participate in a Zoom 

Practice Lab?

• If so, to what extent did your 

participation in the Zoom 

Practice Lab assist with this 

change? 

6. What difference do you anticipate/hope

your participation in the MJI Faculty 

Development workshop might make over 

time?

• Might it make a difference to 

you 

• Personally?

• Professionally?

7. What, if any, challenges do you encounter 

or need to overcome in relation to 

incorporating virtual tools in your online 

facilitation?

8. What, if anything, gives you hope you will 

grow in online facilitation skills in the 

future?

Now, I would like to ask about particular workshop

elements.

The first element I’d like to discuss is the duration 

of the workshop. As you may recall, the workshop 

was 2 hours and 45 min. long. 

9. In your opinion, was this an appropriate 

length for a faculty development workshop?

Why or why not?

10. What are your thoughts about the delivery 

of the workshop?

• Thoughts about the virtual 

delivery? Organization of the 

information presented?

Preparedness of presenter?
The last element I’d like to discuss is the workshop 

format, which included large group discussions, 

small break-out room discussions, breaks, as well 

as time to reflect & debrief. 

11. From your perspective, to what extent was 

this format helpful? • To what extent did this format 

support your learning?

The next questions concern the topics covered 

during the workshop. Some key topics covered 

included the importance of a 2- word check-in, the 

principle of backward design, the purposeoutcomes-process (P.O.P) model, activity 

dashboard, polling, sharing your screen, creating 

break-out rooms, managing security, small teaching 

strategies, how to share videos, and looking good 

on Zoom.

12. Which, if any, of these topics resonated 

with you?

13. How does knowing about that topic help 

you in your current position?

• How helpful were the materials, 

handouts (small teaching online 

inventory, activity dashboard, 

participant guide, power point 

slides) provided? 

14. What, if any, recommendations do you have 

for future MJI eLearning faculty 

development workshops? 

• Seek recommendations related to 

workshop duration, delivery, 

topics covered, and any other 

workshop aspects 

15. Do you have any additional comments or 

feedback that you would like to share?

• Open-ended prompts -

understand perspectives beyond 

topics covered directly in the 

interview
Michigan Judicial Institute (MJI) 

e-Learning Faculty Development Model Project Full Report

November 2021 

Prepared for: 

Michigan Judicial Institute & 

Other State Judicial Educators

Prepared by: 

Jori N. Hall, Ph.D. 
MJI Report | November 2021 2

TABLE OF CONTENTS

Executive Summary 3

Findings 3

Recommendations 4

Introduction 5

Workshop Description 5

Evaluation Design, Questions, and Methods 5

Design 5 

Questions 6

Methods 6

Table 1. MJI eLearning Faculty Development Workshop Full Report - Evaluation Design 

Matrix 6

Observations 6

Interviews 6

Table 2. MJI Full Report - Interviewee Sample 7

End-of-workshop survey 8

Table 3. MJI Full Report - End-of-Workshop Survey Sample 8

 Data Analysis 8

Findings 9

References 13
MJI Report | November 2021 3

Executive Summary 

This evaluation report presents results from the Michigan Judicial Institute (MJI) e-Learning 

Faculty Development Project. The MJI e-Learning Faculty Development Project was conducted 

between March 2021-September 2021 by the MJI with funding from the Strategic Initiatives 

Grant (SIG) Program. The purpose of the e-Learning Faculty Development Project was to design 

a virtual workshop on active learning techniques for use by court personnel serving as faculty for 

virtual judicial education sessions. The e-Learning Faculty Development Project aimed to 

introduce and model multiple strategies for learner engagement during a virtual education 

session. The Project also sought to offer a resource that provides guidance on how to design and 

implement interactive virtual learning presentations. 

Three primary questions guided this external evaluation: 

1. How was the delivery and design of the eLearning Faculty Development for the MJI 

Teaching and Facilitation Workshop experienced by participants?

2. In what ways have participants changed how they incorporate virtual tools or think about 

their online facilitation?

3. How can the MJI improve future faculty development workshops? 

To address these questions, the evaluation design included multiple methods. First, observations 

of the e-Learning Faculty Development Workshop were conducted on both days offered: August 

31, 2021, and September 1, 2021. Second, interviews were conducted with six participants who 

indicated a willingness to participate. Interviews addressed participants’ backgrounds; whether 

and how the workshop affected their online presentations, perceptions about workshop elements 

(e.g., duration, delivery, organization, format); reflections on topics covered during the 

workshop; recommendations for future MJI e-Learning workshops; and additional comments or 

feedback. Third, findings from an online end-of-workshop survey administered by the MJI to 

nine workshop participants were reviewed. The end-of-workshop questionnaire addressed 

participants’ role; experience delivering an educational session virtually; and perceptions about 

the workshop, on a scale from 1=strongly disagree to 5=strongly agree. The end-of-workshop 

survey also included open-ended questions concerning the meaningfulness of the workshop; 

what was learned from the workshop; barriers to implementing interactive learning strategies; 

skills participants will implement when facilitating virtual meetings; and additional 

comments/feedback.

Findings 

1. Workshop Experience. All of the end-of-workshop survey respondents agreed or 

strongly agreed they experienced a high-quality workshop (9;100%). They also agreed or 

strongly agreed the workshop was informative (9; 100%) and engaging (9; 100%). 

Interviewees expanded on this finding, reporting the varied presentation styles (mix of 

large and small group activities), the pacing and variety of the content, and the Zoom 

Practice Labs, facilitated their understanding of how to structure a virtual presentation,

and enhanced their confidence in implementing virtual tools in their presentations.
MJI Report | November 2021 4

2. Application of Workshop Ideas. The majority of end-of-workshop survey respondents 

agreed or strongly agreed that they are confident in their ability to apply what I learned to 

my next presentation (8; 89%). Interviewees highlighted two skills they will apply when

designing their next virtual meeting or educational session: the principle of Backward 

Design and techniques to foster active learning. 

3. Barriers to Implementing Workshop Ideas. Survey respondents identified barriers to 

implementing ideas discussed during the workshop: time constraints, the high number of 

attendees at their virtual presentations, the significant amount of material covered, 

attendee resistance to active participation/keeping attendees engaged, anxiety about 

learning new technologies, and technology issues. Of these, respondents mentioned time 

constraints most often. 

Recommendations 

The following recommendations were offered by the interviewees:

1. Group Participants by Proficiency and/or Interest. One interviewee recommended 

organizing participants in the workshop based on their technology proficiency and/or 

virtual facilitation interests. According to this interviewee, grouping workshop 

participants by proficiency or interest—even for part of the workshop—could better cater 

to participants’ specific needs and interests. 

2. Revise Foster Active Learning Chart. While this chart was considered helpful to design 

and deliver virtual presentations, some of the activities listed on the chart were not clearly 

understood by participants. Revise the Foster Active Learning chart to include a brief 

definition/description of each activity. 

3. Reconsider Breakout Room Tasks/Organization. While the majority of interviewees 

perceived the workshop design and delivery to be well organized and engaging, a couple 

of interviewees noted drawbacks during the breakout room sessions. First, one 

interviewee perceived the prompts used to guide discussions/tasks in the breakout rooms 

to be “a little superficial.” Second, another interviewee discussed how participants were 

placed in different groups for each breakout group session; therefore, time for each 

breakout session needed to be allocated to build rapport with others, which limited 

discussion time. Given interviewees’ perceptions of breakout rooms prompts/tasks, 

expand the inclusion of participants’ issues and interests in breakout discussions/tasks. 

Build on connections made during breakout room discussions by having participants 

remain with the same small group.

4. Strongly Encourage Review of Workshop Materials. While participants were asked to 

review the materials, one interviewee noted that more emphasis on having participants 

closely examine the materials ahead of the workshop would strengthen participants’ 

understanding of the content. 
MJI Report | November 2021 5

Introduction

This evaluation report presents results from the Michigan Judicial Institute (MJI) e-Learning 

Faculty Development Workshop. The MJI e-Learning Faculty Development Project was 

conducted between March 2021-September 2021 by the MJI with funding from the Strategic 

Initiatives Grant (SIG) Program. A primary objective of the e-Learning Faculty Development 

Project was to design a virtual workshop on active learning techniques for use by court personnel 

serving as faculty for virtual judicial education sessions. Specifically, the e-Learning Faculty 

Development Workshop aimed to introduce and model multiple strategies for learner 

engagement during a virtual education session. The Project also sought to offer a resource that 

provides training on how to implement interactive virtual tools for learning and presentations. 

Workshop Description

The MJI eLearning Faculty Develop Workshops were held on August 31 and September 1, 2021. 

Each workshop lasted approximately two hours and 45 minutes. A total of 25 participants 

attended the workshop (17 participants on August 31; 8 participants on September 1). The 

workshop content covered virtual learning and facilitation tips and technologies, including but 

not limited to the two-word check-in technique, the principle of Backward Design, the Purpose, 

Outcomes, and Process (POP) Model, techniques to foster active learning, Zoom features (i.e., 

polling, creating breakout rooms, managing security), and Small Teaching strategies. 

This content was delivered via a videoconferencing platform (Zoom), and included large and 

small group discussions. During the workshop, time was allotted for reflective questions, 

application of the virtual learning and facilitation skills presented, and periodic breaks (10-15 

mins.). Packets containing information on the workshop purpose, outcomes, agenda, as well as 

materials (i.e., power points, examples, templates) were also provided to support the application 

of workshop ideas. 

After the workshops, Zoom Practice Labs were held so interested participants could practice 

various Zoom features and troubleshoot any issues before facilitating a virtual presentation.

Evaluation Design, Questions, and Methods

Design 

A mixed method design (Hall, 2013) was used for this evaluation. This approach was deemed 

appropriate to (1) assess the extent to which the eLearning Faculty Development Workshop 

met its objectives and (2) provide an independent assessment of the effectiveness or usefulness 

of the training provided. The evaluation design was approved by the MJI and all participants 

provided their informed consent. 
MJI Report | November 2021 6

Questions

Three primary questions guided this evaluation: 

1. How was the delivery and design of the eLearning Faculty Development for the MJI 

Teaching and Facilitation Workshop experienced by participants?

2. In what ways have participants changed how they incorporate virtual tools or think about 

their online facilitation?

3. How can the MJI improve future faculty development workshops? 

Methods

Data collection activities were conducted from August 31 to September 22, 2021. To address the 

evaluation questions, three data collection methods were used: observations, individual 

interviews, and a survey. Table 1 below shows the prominence of each method in addressing 

each research question. Details on each method are also provided below.

Table 1. MJI eLearning Faculty Development Workshop Full Report – Evaluation Design 

Matrix

Questions Methods

Observations Interviews Survey

(1) How was the delivery and design of the 

eLearning Faculty Development for the MJI 

Teaching and Facilitation Workshop experienced 

by participants?

X X X

(2) In what ways have participants changed how 

they incorporate virtual tools or think about their 

online facilitation?

x X X

(3) How can the MJI improve future faculty 

development workshops?

X x

Note: Large X indicates primary method and small x indicates secondary method. 

Observations

Informal observations of the e-Learning Faculty Development Workshop were conducted on 

both days offered: Tuesday, August 31, and September 1, 2021. These observations focused on 

how the participants experienced the delivery and design of the workshop. Each observation was 

conducted via Zoom for the duration of the workshop: two hours and 45 minutes. 
MJI Report | November 2021 7

Interviews

Semi-structured interviews were conducted to understand participants’ overall workshop 

experience. The interview protocol was developed in consultation with the MJI and based on the 

workshop objectives. The final interview protocol consisted of three sections: 

● Participant background: information about interviewees (e.g. their current work, their roles, 

decision to attend the workshop) 

● Experiences with the workshop: how the workshop affected their online facilitation and/or 

the way they think about online facilitation), difference the workshop will make over time 

(personally/professionally), challenges encountered when trying to incorporate virtual tools 

in online facilitation, experience relative to particular workshop elements (e.g., length, 

delivery, format, topics covered). 

● Recommendations for future MJI workshops: recommendations they have for future MJI 

eLearning faculty development workshops, and additional comments they have and areas in 

which they most want to continue learning.

To recruit participants, a combination of convenience and maximum variation sampling (Patton, 

2015) was used. At the end of each workshop, participants were invited to conduct an interview 

with the evaluator. To achieve maximum variation sampling, the MJI also followed up via email 

to recruit additional interviewees. 

A total of six semi-structured, individual interviews (Patton, 2015) were conducted. See Table 2 

for the interviewee sample characteristics (position at the MJI) and time of interview. All 

interviews were conducted via Zoom and were 20-30 minutes in length. For each interview, the 

evaluator periodically restated interviewees’ responses as a form of member checking (Creswell 

& Poth, 2018). All of the interviews were recorded via Zoom. These recordings were transcribed 

to generate written transcripts from the Zoom files. 

Table 2. MJI Full Report - Interviewee Sample

Position Interview date and time

Friend of the Court Thursday, September 16, 2021 at 9:00 a.m. 

(EST)

Court Administrator Thursday, September 16, 2021 at 10:00 a.m. 

(EST)

Court Administrator Thursday, September 16, 2021 at 1:00 p.m. 

(EST)

Probation Director Thursday, September 16, 2021 at 2:30 p.m. 

(EST)
MJI Report | November 2021 8

Judge Friday, September 17, 2021 at 8:30 a.m. 

(EST)

Judge Wednesday, September 22, 2021 at 3:00 p.m. 

(EST)

End-of-workshop survey

All workshop participants were invited to complete an end-of-workshop survey. The survey was 

administered by the MJI and addressed participants’ role; experience delivering an educational 

session virtually; and perceptions about the workshop, on a scale from 1=strongly disagree to 

5=strongly agree. 

The end-of-workshop survey also included open-ended questions concerning the meaningfulness 

of the workshop; what was learned from the workshop; barriers to implementing interactive 

learning strategies; skills participants will implement when facilitating virtual meetings; and 

additional comments/feedback. 

A total of nine workshop participants completed the survey. Table 3 provides the number and the 

MJI position of survey respondents. 

Table 3. MJI Full Report – End-of-Workshop Survey Sample

Position Number of respondents

Court administrator 2

Friend of the court 2

Judge 1

Referee 1

Probate register 1

Ethics administrator/probation supervisor/specialty 

court coordinator

2

Total Number of Respondents 9

Data Analysis
MJI Report | November 2021 9

A thematic approach was used (Braun & Clarke, 2006) to analyze the observational data, 

interview transcripts, and end-of-workshop survey open-ended responses. For this approach, the 

evaluator reviewed all responses per question and developed an initial set of codes based on 

protocol topics, followed by inductive coding based on topics raised by the participants. The 

analysis also utilized a memo technique to assist with the refinement and development of key 

findings. Using these, thematic summary statements for each evaluation question were 

developed, including illustrative quotations. 

An integrated analysis (Fetters et. al, 2013) was conducted to incorporate quantitative (survey 

findings) into the thematic summaries. This involved creating a table with survey results and 

interview themes pertaining to each evaluation question. This facilitated an examination of 

similarities and differences by method and helped to understand how the interview data 

expanded on some survey results. The dominant themes based on this analysis are presented in 

the findings section below. 

Findings

This section presents the findings organized by the evaluation questions. 

1. How was the delivery and design of the eLearning Faculty Development for the 

MJI Teaching and Facilitation Workshop participants?

All of the end-of-workshop survey respondents agreed or strongly agreed they experienced a 

high-quality workshop (9;100%). They also agreed or strongly agreed the workshop was 

informative (9; 100%) and engaging (9; 100%). Interviewees elaborated on this finding, 

reporting the delivery of the content moved at just the right pace and was generally “easy to 

follow.” The following quote from one interviewee is representative of this point: “I wasn’t 

bored. I wasn't overwhelmed, I wasn't underwhelmed. It [the workshop] kind of had a flow to it.”

Interviewees perceived the variety and depth of content covered, materials (i.e., handouts, 

templates, power points, guides, lists, etc.), periodic breaks (10-15 minutes) in between sessions, 

as well as the length of the workshop (3 hours), instrumental in making the content delivered 

understandable and the design of the workshop enjoyable.

Many interviewees noted the small group “break-out” sessions were especially beneficial to 

“connect with different people and learn different things,” “exchange ideas,” have in-depth 

conversations, and problem solve. Last, interviewees noted the instructor was well prepared and 

modeled best practices for virtual presentations while delivering the workshop content, which 

reinforced participants’ learning, interest, and engagement. 

While the majority of interviewees perceived the workshop design and delivery to be well 

organized and engaging, a couple of interviewees shared the breakout room sessions had a 

couple of drawbacks. One interviewee stated that she “would have liked a little bit more maybe 

meat or something to the breakout work.” For this interviewee, the prompts provided to guide 

discussions and tasks in the breakout rooms “seemed a little superficial.” Another interviewee 

noted how different groups of people were in each breakout session. For him, this was a 
MJI Report | November 2021 10

drawback as some of the time for the small group activity was used to get acquainted as a group. 

He put it this way: “You need time worked in to meet new people. Um and so that was the one 

drawback of it.”

Of the six interviewed, three indicated they participated in the Zoom Practice Labs offered after 

the workshops. In terms of design and delivery, these interviewees perceived the Practice Labs as 

valuable because they provided “a lot of food for thought” and “a place to go and bounce some

ideas and explore some things.” These interviewees further noted that the Practice Labs 

“provided validation for a lot of the things that I already had figured out” and allowed them to 

feel more comfortable when using Zoom for their virtual presentations. 

Overall, the varied workshop presentation styles (mix of large and small group activities), the 

pacing and variety of the workshop content, as well as the Zoom Practice Labs, facilitated 

participants’ understanding of how to structure a virtual presentation, and enhanced their 

confidence in implementing virtual tools in their presentations.

2. In what ways have participants changed how they incorporate virtual tools or 

think about their online facilitation?

When asked in what ways have you changed how you incorporate virtual tools as a result of your 

participation in the eLearning Faculty Development for the MJI Teaching and Facilitation 

Workshop, two interviewees indicated that they were able to implement strategies learned from 

the workshop into their virtual presentations. The first reported that she was able to incorporate 

the “Two-Word Check-In”—a technique to engage virtual participants immediately, which 

involves asking participants to share two words about how they are feeling either via chat or out 

loud. She further stated that before the workshop, she commonly started her virtual presentations 

by telling participants about herself. However, after participating in the workshop, she shifted her 

thinking. Now, she uses the Two-Word Check-In technique to get participants actively involved 

at the beginning of her virtual presentations. 

The second interviewee was able to implement the principle of Backward Design, which 

emphasizes establishing the outcomes of a presentation and the evaluative criteria that will be 

used to determine a successful virtual presentation or assess learners’ learning. This interviewee 

described how she used Backward Design during a webinar. Because she knew a key aspect of 

Backward Design was “presenting with the end in mind,” she sent an email ahead of her webinar 

describing the problem she wanted attendees to address. As a result, she, along with the 

attendees, came to the webinar prepared to brainstorm their ideas and resolve the issue.

Because the majority of the interviewees had not conducted a virtual presentation since time of 

the workshop; therefore, they were unable to report on how they incorporated what they learned

during the interview. However, interviewees explained the workshop provided “some really 

good food for thought on how to structure, engage, hold the attention of, [and] present 

information to people who were participating in um our programs,” which they will use to 

change the structure of their virtual presentations in the future. The end-of-workshop survey 

responses corroborate this finding as the majority of respondents agreed or strongly agreed that 

they are confident in their ability to apply what I learned to my next presentation (8; 89%)
MJI Report | November 2021 11

Interviewees commonly mentioned two strategies learned during the workshop they will 

implement when designing/delivering their next virtual meeting: (1) the principle of Backward 

Design and (2) activities designed to foster active learning. 

Multiple end-of-workshop survey respondents indicated the principle of Backward Design as one 

skill they are going to try when designing their next virtual presentation. Interviewees elaborated 

on this finding, explaining how reviewing the Purpose, Outcomes, and Process (POP) model

template provided during the workshop will support their implementation of the principle of 

Backward Design. 

Interviewees perceived the “Foster Active Learning” chart as beneficial to restructure their next 

virtual presentation because it included the engagement level (high, medium, low) of a wide 

range of instructional techniques (i.e., group discussion, quizzes, role-playing). Interviewees 

mentioned how this information will help them to determine the appropriate mix of instructional 

activities when designing their presentation. The following quote illustrates this point:

Like I said, there was a chart that talked about different engagement strategies and kind 

of grouped them, by you know, this is, you know, high-level, mid-level. I thought that 

was pretty helpful too. And I kept that because I'm going to try to use that just to kind of 

step things up...

A few interviewees discussed how the workshop increased their competency to use certain Zoom 

features, most notably how to set up a poll, enable screen share, and assign participants to a 

breakout room. According to one interviewee, learning more about the Zoom features was useful 

because it “can help improve or enhance the professionalism of the presentation.” 

It is also important to note that multiple end-of-workshop survey respondents indicated time 

constraints as a barrier to implementing the ideas discussed in the workshop. Other barriers 

mentioned by survey respondents included “the size of my typical audience coupled with the 

quantity of the material I typically need to cover,” audiences “who are often resistant to 

productive participation, hesitance and anxiety about trying something new,” keeping attendees 

engaged, and technology issues.

3. How can the MJI improve future eLearning Faculty Development Teaching and 

Facilitation workshops? 

When asked about recommendations to improve future eLearning Faculty Development for the 

MJI Teaching and Facilitation Workshops, the majority of interviewees did not have any 

recommendations. Rather, many of the interviewees made clear they perceived the workshop to 

be well organized and appreciated the opportunity to attend.

However, three interviewees did offer recommendations. The first recommendation is to 

organize participants in the workshop based on their Zoom proficiency and or technology 

interest. For this interviewee, grouping workshop participants by proficiency or interest—for 

even part of the workshop—would better cater to participants’ specific needs and interests. 
MJI Report | November 2021 12

Another interviewee stated he would have preferred more encouragement to review the 

workshop materials beforehand. He admitted that he received the materials ahead of the 

workshop and was asked to review them; however, he did not get an opportunity to do so. 

Recognizing how beneficial it would have been to look over the materials prior to the workshop, 

he recommends that the MJI push workshop participants to examine them in advance to 

strengthen their understanding of the content. 

The third recommendation concerns the “Foster Active Learning” chart. While interviewees 

viewed this chart as helpful to design their virtual presentations, some of the activities listed on 

the chart were not “self-explanatory,” according to one interviewee. Therefore, this interviewee 

suggested that the chart be revised to include a brief definition of the activities to enhance 

participants’ understanding of how each one is intended to foster active learning.
MJI Report | November 2021 13

References 

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in 

Psychology, 3(2), 77-101.

Creswell, J. W., & Poth, C. N. (2018). Qualitative inquiry & research design: choosing among 

five approaches (Fourth Edition.). SAGE Publications, Inc.

Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving Integration in Mixed Methods 

Designs—Principles and Practices. Health Services Research, 48(6 Pt 2), 2134–2156. 

https://doi.org/10.1111/1475-6773.12117

Hall, J. N. (2013). Pragmatism, Evidence, and Mixed Methods Evaluation. New Directions for 

Evaluation, 2013(138), 15–26. https://doi.org/10.1002/ev.20054

Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and 

practice (4th ed.) Sage.