1Student Learning Outcomes Assessmentin the United States:Trends and Good Practices
STACI PROVEZIS, PH.D.
NIAD-UE UNIVERSITY EVALUATION FORUM 2012
JULY 23, 2012

2Overview of NILOANILOA’s mission is to document student learning
outcomes assessment work, identify anddisseminate best practices, and support
institutions in their assessment efforts.
SURVEYS ● WEB SCANS ● CASE STUDIES ● FOCUS GROUPS ●OCCASIONAL PAPERS ● WEBSITE ● RESOURCES ● NEWSLETTER ●LISTSERV ● PRESENTATIONS ● TRANSPARENCY FRAMEWORK ●FEATURED WEBSITES ● ACCREDITATION RESOURCES ● ASSESSMENT
EVENT CALENDAR ● ASSESSMENT NEWS ● MEASURING QUALITYINVENTORY ● POLICY ANALYSIS ● ENVIRONMENTAL SCAN

www.learningoutcomesassessment.org

www.learningoutcomesassessment.org
3

4Advanced Organizers
 What is student learning outcomes assessment?

 What is accreditation in the United States?
 What is the national picture of student learning outcomes

assessment for American colleges and universities?
 What are some best practices in regards to student learningoutcomes assessment?

Definition
5Assess: (v.): to examine carefully
Assessment is the systematiccollection, review, and use ofinformation about educational
programs undertaken for thepurpose of improving student
learning and development (Palomba& Banta, 1999, p. 4)

How do we know students are learning?
6 The key questions…
 What are we trying to do and why? Or What is my program supposed to accomplish?
 How well are we doing it?
 How do we know?
 How do we use the information to improve orcelebrate successes?
 Do the improvements we make work?
Bresciani (2003)

7

Assessment Tools
8 Direct (outcomes) measures-- Evidence of whatstudents have learned orcan do Indirect (process) measures-- Evidence of effective
educational activity bystudents and institutions

Direct Assessment Methods
 Tests9
 Performance on national licensure examinations
 Standardized tests
 Locally developed tests
 Pre-and post-tests
 Essay tests blind scored across units Student work/projects
 Collections of student work (e.g. Portfolios)
 Capstone projects
 Observations of student behavior
 Internal juried review of student projects
 Project-embedded assessment Course-embedded assessment
 External review of students
 External evaluations of student performance External juried review of student projects
 Externally reviewed internshipIndirect Assessment Methods
10 Alumni, Employer, Student Surveys/Studies
 National or local surveys
 Focus groups (depending on the interview protocol, this could be usedas direct evidence) Exit Interviews with Graduates
 Graduate Follow-up Studies
 Student Success Indicators
 Percentage of students who go on to graduate school
 Retention and Transfer Studies
 Job Placement Statistics
 Faculty/Student ratios Percentage of students who study abroad Enrollment trends
 Percentage of students who graduate within five-six years
 Diversity of student body

11

Setting the Context
 Growing Interest in Student Learning Policies and Reports Accreditation

Accreditation in the United States12 Role of US Government
 No ministry of education to oversee quality
 Accreditation necessary to receive federal money

 Characteristics of accreditation
 Set of standards/criteria
 Quality assurance (accountability)/Institutional Improvement
 Peer review/self-regulation

 Process for Accreditation
 Self study Campus visit Report

13

Occasional Paper #6

Regional Accreditation andStudent Learning Outcomes
Assessment: Mapping theTerritory
Staci Provezis
This paper examines the policies and proceduresat each of the seven regional accreditation
organizations, as they relate to student learning
outcomes assessment.

www.learningoutcomeassessment.org/OccasionalPapers.htm

14What is the national picture of student learning
outcomes assessment for colleges and universities?
 Survey Report Focus Groups Accreditation Study
 Web Scan Study Occasional Papers
 Case Studies

15

Survey Report Survey asked chief

academic officers at all
accredited 2 & 4 year
colleges and universities
about their assessment
practices.
 Survey report “More Than
You Think, Less Than WeNeed” released October,2009

http://www.learningoutcomesassessment.org/NILOAsurveyresults09.htm

16Survey Guiding Questions
1. What learning outcomes are you measuring at yourinstitution?
2. How are you assessing these outcomes and using theresults?
3. What are the major factors prompting assessment at yourinstitution?
4. What do you need to further learning outcomes
assessment at your institution?17

Report Summary
1. Perhaps more assessment underway thansome acknowledge or wish to believe

Institution-Level Assessments of Learning
Outcomes for All Institutions
100%
90%
80%
70%
60%
50%
40%
30%

20%
10%0%

Program-Level Assessments of Learning Outcomesfor All Institutions
100%
90%
80%
70%
60%
50%
40%
30%

20%
10%
0%

20

Report Summary
1. Perhaps more assessment underway thansome acknowledge or wish to believe
2. More attention needed to using andreporting assessment results

Assessment driven more by accreditation andcommitment to improve than externalpressures from government or employers
Assessment Drivers for All Schools
4.00

3.50

3.00

2.50

2.00

1.50

1.00

Accreditation

ProgramInst.
Faculty/Staff National Calls
Accreditation Commitment
InteresttoImprovement

21

Governing
Board
MandateInst.
Coordinating
Associations
MandateBoard22

Accreditation
tops the list
for uses ofassessment
data.

23

Objective of Institutional Web Scans
To understand what student learning outcomes
assessment information institutions post on theirwebsites and where they post it.

Comparison of Survey with Web Studies
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

National Student Survey

Local Student Survey

2010 Web Scan

Alumni Survey

2009 National Survey
24

Other Survey

Standardized Tests

2009 Web Scan

25

Major Findings from Website Analysis
 More assessment takes place than is shown.
 Assessment information is typically found on pages

geared to internal audiences.
 Accreditation, institutional control, andparticipation in national initiatives appear toinfluence website communication on assessment.
 Institutions show results more often than you

would think they do, but you have to search for it.

26

Transparency Framework

http://www.learningoutcomeassessment.org

What is Required to Improve Learning Outcomes
Assessment for All Schools
100%
90%
80%

70%
60%
50%
40%
30%
20%
10%
0%

28

Report Summary
1. Perhaps more assessment underway thansome acknowledge or wish to believe
2. More attention needed to using andreporting assessment results
3. Involving faculty is a major challenge
4. More investment likely needed to movefrom data to improvement2010 Program-

29

Level Survey
Characteristics
 Two-staged
administration
 Usable
responses from982 programs(30% ofprograms
surveyed)

http://www.learningoutcomesassessment.org/NILOAsurveyresults11.htm

30Specialized Accreditation Matters Accredited programs were significantly

more likely to report that specialized
accreditation is of high or moderate
importance “Halo Effect” of programmatic accreditation

31

Program Learning Outcomes More than 8 of 10 programs have established anagreed-upon set of intended student learning
outcomes Overwhelming majority of programs haveestablished learning outcomes goals

Assessment Methods Used
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

Most Students

All Students
32Use Of Results
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%0%

Quite a Bit

Very Much
3370

Overall Use Index by Discipline

60504030

20
10034

Resources
35

Few resources dedicated to program levelassessment
 15% FT person 54% PT person 63% tenured faculty, 15% received one course equivalent inrelease time, and an additional 7% received more than one
course equivalent in release time.

 69% prepare an annual assessment report 60% have an assessment committee Only 6% have established budget for assessment

What Would Help?
100%
90%
80%
70%

60%
50%
40%
30%
20%
10%
0%

3637

Occasional Paper #4

Opening Doors forFaculty Involvement inAssessment
Pat Hutchings
The key to effectively using
assessment data to improve student
learning is to engage faculty inmeaningful ways in this criticalactivity. This paper discusses thechallenges inherent in andopportunities for doing so.
www.learningoutcomeassessment.org/OccasionalPapers.htm

38Key Findings
Action at the program level
Perceptions of CAOs andprograms differ
Specialized accreditation
matters a lot
Disciplinary differences
matter even more

39

Examples of Good Assessment Practice Focus assessment efforts Harness external accountability forimprovement Communicate widely Reflect on the process and results;

make meaning

40

 Empower anassessment committee Communicate openly Start small and build

Augustana College

41

 Allow variability
 Pursue facultyinterests
 Faculty development
 Use accreditation
 Improve learning

Carnegie Mellon
University

42

 Align with facultyinterests
 Make assessment

meaningful andmanageable Be intentional fromthe beginning

St. Olaf College

43

 Communicate regularly
 Review assessment

activities Support from campus

leadership Embed in campus

review systems
 Faculty development

LaGuardia
Community
CollegeMarks of Advanced
Department SLO Assessment
44

Faculty invested in development ofapproaches & identification of measures

Multiple measures
More faculty aware of markers of student success &learning (progress rates, troublesome courses)“Closing the loop”, evidence of change
Sharing results with students, communicating how
results are used

45

What’s Been Accomplished?
 Assessment Seen as Legitimate
 Goals for Learning Established A “Semi-Profession” forAssessment
 Much Better Instruments andMethods

46

What Remains to be Done?
 Authentic Faculty Ownership Assessment Still an “Add-On”

 Use of Information forImprovement is Underdeveloped Sincere Institutional Engagementwith Accreditors in Assessment

47

Discussion?
Questions?
Comments?
www.learningoutcomesassessment.org

48

Staci Provezis, Ph.D.
sprovez2@illinois.edu
www.learningoutcomesassessment.org