OECDAHELOASSESSMENT OF HIGHER EDUCATIONLEARNING OUTCOMES

HigigherOutcomesomes Education
EdAssessmenssment

Assessment of Higher Education Learning OutcomesStudeStudents
SLearningning2010-2011

2Preface
Tomorrow’s workforce is key in sustaining the wealth and development of nations and theirpeople. In this changing global economy, high quality tertiary education is essential andstudents need to obtain the right skills to ensure economic, scientific and social progress. Morestudents than ever are enrolled in higher education – a sector in which both governments andindividuals invest significant resources. The OECD’s initiative to assess the feasibility of an
Assessment of Higher Education Learning Outcomes (AHELO) will maintain a clear focus onteaching and learning and will identify a wide range of factors influencing higher education.
On average, a student’s tertiary education costs USD 67 000 (including both public and privatecosts). In some countries, such as Austria, Germany and the United States, these costs exceed
USD 100 000. How can we know that this money is spent well? Tools are needed to measure
students’ knowledge and whether they are equipped, at the end of their tertiary education, withthe skills needed for the emerging job market.

Barbara Ischinger
Director for Education© OECD 2010-2011

OECDIn 2008, discussions among OECD Education Ministers attheir meeting in Tokyo focused on the evaluation of thequality of higher education. After three consultations withinternational experts, the AHELO initiative was launched toprovide answers.
For more than 40 years, the OECD has been one of thelargest and most reliable sources of comparable statistics
as well as economic and social data.
The Directorate for Education has ample experience is thisarea with projects such as the Programme for International
Student Assessment (PISA) testing the knowledge andskills of 15 year olds and the Programme for International
Assessment of Adult Competencies (PIAAC).

© OECD 2010-2011

Assessment of Higher Education Learning Outcomes (AHELO)

3

4

AHELO at a GlanceWhat?

A test of university students comparable internationally

Why?

No international tools exist for the direct evaluation of students

Who?

The OECD, an international consortium of experts and 15 countries
from all over the world

How?

An evaluation of generic skills and discipline-specific skills(engineering and economics)

Results

Anonymous comparable data at the level of institution.
If the feasibility study is a success countries will consider a full-scaleassessment.

When?

The feasibility study: Tests administered 2011-2012
Results and final conference end 2012
© OECD 2010-2011

OECD

Assessment of Higher Education Learning Outcomes (AHELO)

What will AHELO achieve?
The project will test students nearing the end of theirbachelor degree. The tests will be created so that results
can be comparable internationally regardless of language
or cultural background. The tests will provide actual dataon the quality of learning and its relevance to the labour
market.

Lisa F. Young - Fotolia.com

The OECD will be working with a consortium of world
experts and teams in 15 participating countries to develop
and administer the tests. By the end of the feasibility study,
in 2012, we will determine if such tests can indeed be
developed and successfully administered to students.

© OECD 2010-2011

5

6

What is being tested?
Discipline-Speciﬁc Skills

The assessment will look both at the skills that students
in all fields should be acquiring (the Generic Skills) andskills that are specific to each discipline (with a focus onengineering and economics for the feasibility study).

In the case of engineering and economics, the focus of thetest will be on the capacity of students to extrapolate fromwhat they have learned and apply their competencies in
new contexts.

Generic Skills

estima - Fotolia.com

The test questions will be based on an international
adaptation of the US Collegiate Learning Assessment(developed by the Council for Aid to Education). Students
taking this test will need to use critical thinking, analytic
reasoning, problem solving and written communication toanswer several open-ended questions about a hypotheticalbut realistic situation and gather the necessary evidence
from different sources (letters, maps, memos, etc.).

© OECD 2010-2011

Assessment of Higher Education Learning Outcomes (AHELO)OECD

For the purpose of the feasibility study each of these three tests will be developed andadministered in three small groups of countries (see diagram).

Generic Skills

Economics

Engineering

Colombia (tbc), Egypt, Finland,Korea, Kuwait, Mexico, Norway,
US (Ma, Pa, Mo, Ct)

Belgium (Fl.), Egypt, Italy,
Mexico, the Netherlands,
the Russian Federation

Australia, Egypt,Japan, Sweden

Contextual Dimension
All countries

Along with each of these tests contextual questionnaires
will be administered to link the test results withinformation on universities and students. The conclusions
from the Contexual Dimension expert group on the areas
to be included in these questionnaires are available atwww.oecd.org/edu/ahelo.
© OECD 2010-2011

No one is surprised by top universities that attract A+students and turn out A+ graduates. But what aboutuniversities that accept B+ students and produce A+graduates? Which is doing the better job? If the feasibility
study is successful, at a later stage the project will also try
to determine the university’s “value added”.

7

8

Who is involved in the feasibility study?
Fifteen countries comprising a range of languages andcultures: Australia, Belgium, Colombia (tbc), Egypt,Finland, Italy, Japan, Korea, Kuwait, Mexico, theNetherlands, Norway, the Russian Federation, Sweden,
and the United States (10 institutions in each country).
International experts:– a consortium led by the Australian Council for Educational
Research (ACER)– the Council for Aid to Education (CAE)

Representatives of the higher education sector: AHELO is
an open and transparent initiative. The OECD has inviteda group of organisations with a stake or interest in higher
education to join the AHELO Stakeholders ConsultativeGroup. Members of this group include, among others,international associations of quality assurance agencies,students or universities. For a complete list of theorganisations involved please see www.oecd.org/edu/ahelo.
Sponsors: The feasibility study has been financedby the participating countries and through generous
contributions from Lumina Foundation for Education(United States), Calouste Gulbenkian Foundation(Portugal), Riksbankens Jubileumsfund (Sweden), theSpencer Foundation (United States) as well as the Higher
Education Founding Council (HEFCE – England) and theHigher Education Authority (HEA - Ireland).

© OECD 2010-2011

OECD

Timing and next steps

In the second phase (July 2011 – December 2012) these tests(and the contextual questionnaires) will be administered
to students and faculty in selected higher education
institutions in participating countries.
A final conference will be organised at the end of 2012 toreview the findings of the AHELO feasibility study: whether
an international assessment of higher education learning
outcomes is scientifically and practically possible. Basedon these conclusions, OECD member countries will decide
whether to launch a full-scale AHELO.

© OECD 2010-2011

Jetta Productions/Iconica/Getty Images

The first phase of the work (January 2010 – June 2011)
currently under way will see the development of testing
instruments for the generic and discipline-specific skillsin economics and engineering and their validation with asmall number of students.

Assessment of Higher Education Learning Outcomes (AHELO)

910

Beneﬁts from AHELO

Students

Institutional
leaders

Governments

Employers

Information
to help themdecide which
universities
best suit theirneeds andaspirations

Analysisto help
them better
understandand improvetheir own
performance

Information onwhether theconsiderable
resources
invested inhigher education
are being usedeffectivelyMoretransparency on
the capacities
of graduates
to enter andsucceed in thelabour market© OECD 2010-2011

OECD

Assessment of Higher Education Learning Outcomes (AHELO)

The Directoratefor EducationWhat is IMHEand how to join?

8 main themes

StockLib, Fotolia

The OECD Programme on Institutional Management in
Higher Education (IMHE) is a membership organisation
within the OECD Directorate for Education. It brings
together institutions of higher education as well as
government departments, agencies and other non-profit
organisations, to discuss and debate issues within higher
education management. To find out more and to submit
your online application, please visit the IMHE website at:www.oecd.org/edu/imhe/join.

More on the work of OECD on Education
please visit www.oecd.org/edu.

y© OECD 2010-201111

If you would likemore information on AHELO
please visit our website
www.oecd.org/edu/ahelo

AHELO in the news
www.oecd.org/edu/ahelo/media

or contact ahelo@oecd.org
to be added to our mailing list
and receive our newsletter

www.oecd.org

ORGANISATION FOR ECONOMIC
CO-OPERATION AND DEVELOPMENT