Report of the Panel of the external review of NVAO
September 2012

Table of contents

Summary

3

1. Background and outline of the review process

4

2. Glossary of acronyms

5

3. Introduction

6

4. Compliance with the ENQA membership criteria linked to the
relevant European Standards and Guidelines (ESG)
11
4.1 ENQA criterion 1 - Activities (ESG 3.1, 3.3)
4.2 ENQA criterion 2 / ESG 3.2: Official status
4.3 ENQA criterion 3 / ESG 3.4: Resources
4.4 ENQA criterion 4 / ESG 3.5: Mission statement
4.5 ENQA criterion 5 / ESG 3.6: Independence
4.6 ENQA Criterion 6 / ESG 3.7 External quality assurance
criteria and processes used by the members
4.7 ENQA Criterion 7 / ESG 3.8: Accountability procedures
4.8 ENQA criterion 8: Miscellaneous

5. Conclusion and recommendations
Annexes

11
29
30
32
34
36
38
41

42
43

2

Summary
Having assessed various documents and oral evidence through a site visit, the
Panel is convinced that NVAO acts in compliance with the ENQA membership
regulations and is in substantial compliance with the Standards and Guidelines
for Quality Assurance in the European Higher Education Area. The Panel
therefore recommends to the Board of ENQA that NVAO‟s Full Membership of
ENQA be confirmed for a further period of five years.
The Panel concludes that NVAO fully complies with the ENQA membership criteria
2, 3, 4, 5, 6, 7 and 8 and substantially complies with the ENQA membership
criterion 1.
A list of the recommendations of the Panel is provided in section 5.2 but, in
short, the main areas for further improvements relates to public information in
various senses and to the formulation of explicit and public criteria about how
NVAO reaches its accreditation decisions in relation to the programme
assessments in all cases.
During the site visit, interviewees mentioned several times that NVAO is in a
“transition state” as models and procedures for quality assurance are currently
being changed in relation to Flanders or have recently been changed in the
Netherlands. It is the view of the Panel that a situation of change and
development is to be expected for most quality assurance agencies. The Panel
recognises that NVAO and its partners have work ahead in order to establish the
institutional assessment approach in Flanders as well as to consolidate the new
practice in the Netherlands. The Panel encourages all the parties to learn from
the experience gathered thus far.

3

1

Background and outline of the review process

ENQA‟s regulations require all full member agencies to undergo an external
cyclical review, at least once every five years, in order to verify that they fulfil
the membership provisions. In November 2004, the General Assembly of ENQA
agreed that the third part of the Standards and Guidelines for Quality Assurance
in the European Higher Education Area (ESG) should be incorporated into the
membership provisions of its regulations. Substantial compliance with the ESG
thus became the principal criterion for Full membership of ENQA.
This is the report of the review of NVAO undertaken in June 2012 for the purpose
of determining whether the agency meets the criteria for full membership of
ENQA. The process in general, including the structure of the present report, was
run in accordance with the Guidelines for external reviews of quality assurance
agencies in the European Higher Education Area.
The present review of NVAO has been coordinated by ENQA. This is the second
external review of NVAO according to the ESG. The first review took place in
2007.
The Panel for the 2012 external review of NVAO, appointed by ENQA, was
composed as follows:






Tove Blytt Holmen, Director of the Department of Quality Assurance, NOKUT, Norway (Chair)
Signe Ploug Hansen, Director of Methodology, EVA, Denmark (Secretary)
Julian Tobias Hiller, Student at Leibniz University of Hannover, Germany
Elie Milgrom, Emeritus professor, consultant in professional development of teaching staff,
Belgium
Dr. Norma Ryan, Director of the Quality Promotion Unit, University College Cork, Ireland

The Panel was provided with ENQA‟s Terms of Reference (ToR), including a
suggested timeline (see annex A). A short presentation of the Panel members is
provided in annex B.
NVAO produced a self-evaluation report (SER) which, together with the
attachments to it, provided a substantial portion of the evidence that the Panel
used for its analysis and to reach its conclusions. The Panel conducted a site visit
to validate the statements made in the self-evaluation report, to clarify any
points at issue, and to obtain further relevant documents. For an overview of
assessed documents, see annex C. Before finalising the report, the Panel
provided an opportunity for NVAO to comment on the factual accuracy of the
draft report.
The Panel confirms that it was given access to all the documents it requested and
to all people it wished to consult with throughout the review.

4

2

Glossary of acronyms
ECA: European Consortium for Accreditation in Higher Education
ENQA: European Association for Quality Assurance in Higher Education
EQAR: European Quality Assurance Register for Higher Education
EVA: The Danish Evaluation Institute
ESG: Standards and Guidelines for Quality Assurance in the European
Higher education Area
HEI: Higher Education Institution
NOKUT: Norwegian Agency for Quality Assurance in Education
NVAO: The Accreditation Organisation of the Netherlands and Flanders
QANU: Quality Assurance Netherlands Universities
SER: Self-evaluation report
ToR: Terms of Reference
VLHORA: The Flemish Council of University Colleges
VLIR: Flemish Interuniversity Council

5

3

Introduction

i.
Purpose(s) of the review
In its SER dated April 16th 2012, NVAO describes its “main objectives” for the
review as follows:
1. Renewal of NVAO‟s full membership of ENQA and extended EQAR-registration.
2. A first external reflection on the new accreditation system in the Netherlands.
3. Contribution to NVAO‟s internal quality monitoring.
ENQA‟s ToR, dated February 2012, identifies the review of NVAO to be a type A
review. However, the ToR also stated: “In addition to … the review aims to give a
first external reflection on the new accreditation system in the Netherlands.”
When asking ENQA for clarification on its mandate, the Panel received the
following answer: “… NVAO would like to know from the Panel whether the new
system is fit for the purposes NVAO wants to achieve, in accordance with the
ESG/ENQA membership criteria and which advice could be given to improve it …”
Furthermore: “… There are no particular expectations on the Panel for n.3 as this
is not part of the purposes agreed between ENQA and NVAO. … what is meant by
“Contribute to NVAO’s internal quality monitoring” is that the overall result of the
review, all Panel’s recommendations will contribute to the agency’s quality
monitoring.”
The Panel understands its mandate was to conduct a type A review where the
focus has to be upon the current status and not upcoming developments (as per
“Guidelines for external reviews ...”), i.e. it should investigate programme
accreditation in Flanders and the new approach in the Netherlands combining
institutional audit and programme accreditation. The Panel‟s reflections around
the new system in the Netherlands will concentrate upon whether the system
supports the ESG or whether it incorporates elements that are in contradiction
with the ESG.
The Panel finds it important to point out that it has decided to rely upon the
assessments made in the first ENQA review of NVAO in 2007 in relation to those
general characteristics that have not changed between 2007 and 2012.
Being the accreditation organisation for both Netherlands and Flanders, NVAO is
the body which makes all the accreditation decisions. However, the assessments
of existing programmes on which the programme accreditation decisions of
NVAO are based are carried out by other quality assurance bodies. Some of
these are members of ENQA as a result of evaluations similar to the present one.
The Panel has noted that VLHORA, VLIR and QANU have all undergone
independent external reviews (in 2008, 2009 and 2010 respectively) and have
been reviewed specifically with regard to their alignment and compliance with the
ENQA membership criteria and thus the relevant sections of the ESG. The Panel
has decided not to re-visit these assessments or to question the evidence
presented in those assessments but to accept the decisions of the review panels
that conducted those assessments. Thus in those topics of relevance to this Panel
in its review of NVAO, the Panel took as read the compliance of these agencies

6

and accepted the credentials and expertise of these agencies in relation to the
quality assurance activities undertaken.
As regards the relation between NVAO and the quality assurance agencies the
Panel finds it important to note that the agencies are not perceived as
subcontractors to NVAO neither by NVAO, nor by the agencies themselves, nor
by the Panel. In various dictionaries a subcontractor is generally defined as an
individual or in many cases a business that signs a contract to perform a part or
all of the obligations of another‟s contract. In its decision making of existing
programmes, NVAO has to build upon quality assurance performed by other
agencies as regulated by law and the assessments undertaken by the agencies
are thus not based on a contract with NVAO. In case of assessments of new
programmes and institutional assessments NVAO performs external quality
assurance by its own panels.
ii.

The place of NVAO in the quality assurance structure of its
jurisdiction
This subsection and subsection iii are largely based on the background
information provided in the SER of NVAO. All external stakeholders interviewed
during the site visit expressed the view that the SER provides a reliable and
adequate presentation of NVAO and the systems of external quality assurance in
the Netherlands and Flanders and the Panel is thus confident that the information
provided in the SER is fully reliable.
NVAO, the Accreditation Organisation for the Netherlands and Flanders
[Nederlands-Vlaamse Accreditatieorganisatie] was formally established by the
Dutch and Flemish governments as a bi-national organisation on February 1st
2005. NVAO is thus the official, public and bi-national accreditation organisation
in the Netherlands and Flanders. NVAO covers all types of higher education
institutions (HEI) within its geographic domain. By law accreditation is the model
for quality assurance.
The primary assignment of NVAO is defined in the Treaty between the Dutch and
Flemish Education ministers of 2003, which is described as:
The primary assignment of NVAO is to accredit existing programmes in higher
education (accreditation) and to assess new programmes (initial accreditation) in
the Netherlands and Flanders. NVAO can be requested to carry out additional
tasks by the Ministers of higher education in both countries if these assignments
support or supplement NVAO’s primary assignment.
In the Netherlands, the first accreditation system was operational from 2003 to
2010, in Flanders from 2005 to 2012. So the Netherlands has already made the
transition to the new system and it is expected that Flanders will follow in 2013
for programme accreditation, while implementing institutional reviews in 2015.
Until the end of 2010, the accreditation system both in the Netherlands and
Flanders was based exclusively on (initial) programme accreditation. In
Flanders, this system is still current, at least until the end of the academic year
2012 – 2013 (what will happen later depends on pending legislation). Since
January 1st 2011, a new accreditation system based on institutional audit

7

combined with (initial) programme accreditation was introduced in the
Netherlands.
iii.

The main functions of NVAO, its current main areas of
responsibility and work, including the review methods it uses
(Initial) programme accreditation
Programme assessment and accreditation is the core of both the accreditation
system still in place in Flanders and the new system in the Netherlands.
The framework for assessments of existing programmes still in place in Flanders
comprises six themes. The six themes are subdivided into 21 standards that are
assessed on a four-point scale (excellent, good, satisfactory and unsatisfactory).
For assessing at theme level a dichotomous scale is used (satisfactory or
unsatisfactory), as is the case for the final assessment. A very similar
assessment framework is applied for accreditation of new programmes. In case
of negative decisions of existing programme accreditation a recovery period of
maximum 3 years is possible.
In the new system in place in the Netherlands the final judgement of a
programme is based on a four-point scale (Excellent, Good, Sufficient and
Insufficient); this scale is applicable both on the level of a study programme and
on the level of the separate standards of the framework. The number of
standards depends on the framework applied (cf. below)
The new system for programme accreditation in the Netherlands contains a
recovery period and an elaborated procedure for initial accreditation; if a new
programme is approved, NVAO can restrict the duration of the initial
accreditation and subject it to conditions that have to be met within one year. In
case of accreditation the recovery period is limited to two years.
The new institutional audits in the Netherlands
The new institutional audits in the Netherlands concern the assessment of an
institution's quality assurance system and ultimately lead to a judgment about
whether an institution is in control of the quality of its education programmes.
These audits comprise five standards and can have three possible outcomes:
satisfactory, conditionally satisfactory and unsatisfactory. A final conclusion that
is either 'satisfactory' or 'conditionally satisfactory' means that the assessment of
the programmes follows the framework of the so-called 'limited programme
assessment'. This framework contains only three standards.
If an institution fails the institutional audit or if it does not want to participate in
it for specific reasons (for example, the limited size of an institution), the
programmes will be assessed on the basis of the framework of the so-called
'extensive programme assessment' which contains more standards than the
framework of 'limited programme assessment' and strongly resembles the
framework for programme assessment in place in the Netherlands before 2011
and still in place in Flanders.
The role of NVAO
NVAO performs all processes related to the institutional audits in the Netherlands
and initial programme accreditation in both the Netherlands and Flanders. NVAO
8

also has the authority to make all the accreditation decisions for existing
programmes, both in Flanders and in the Netherlands. Accreditation decisions on
existing programmes are made upon quality assessments and reports from
quality assurance agencies, some of which are also ENQA-members.
The role of NVAO, the division of labour between NVAO and the quality assurance
agencies and the details of the review methods used are presented in various
sections of chapter 4.
iv.
NVAO’s engagement with the ENQA membership provisions/ ESG
In its assessment frameworks, NVAO explicitly states that its standards have
been developed in accordance with the ESG.
NVAO also has an active internationalization policy and real engagement
regarding higher education and quality assurance promoting ENQA, ECA and ESG
in many ways.
v.
Methods employed by the Panel
The Panel was appointed in May 2012 and received the Self Evaluation Report
(SER) of NVAO at the same time. The SER and its attachments were analysed by
each of the Panel members before the site visit and the Panel discussed the
individual findings during its preparatory meeting, the day before the site visit.
The site visit took place on June 26th and 27th 2012 in the NVAO offices in The
Hague. NVAO drew up the programme for the site visit in close cooperation with
the chair and secretary of the Panel. The programme included interview sessions
with 36 people, including the chairman and members of the NVAO Executive
Board and General Board, the NVAO managing director and a number of staff
members, representatives of the Advisory Council of NVAO, representatives of
umbrella organisations of higher education institutions, representatives of
student organisations, representatives of quality assurance agencies and
representatives of Dutch and Flemish ministries of higher education. The
management of NVAO informed the Panel that the external stakeholder
interviewees were selected by the organisations they represent and that the
NVAO staff interviewees were selected according to the criteria provided by the
Panel. All those interviewed had read the SER and most stakeholders had been
given the opportunity to comment on a draft version of it.
The SER contains valuable insights and statements. The Panel would, however,
have appreciated to have been provided before the site visit with more factual,
detailed evidence e.g. linking NVAO‟s standards and procedures more explicitly
with the ESG.
The SER, its attachments and added documentation constituted the frame of
reference for the interviews during the site visit, which in turn provided further
oral evidence related to the written documentation.

9

The Panel appreciates the fact that a number of relevant documents, originally in
Dutch, were provided in an English translation and also notes that the
assessment and accreditation reports (to be found on NVAO‟s web site) and
several additional documents provided during the site visit were available in
Dutch only. In order to assess the content of specific reports available only in
Dutch, these reports were read and described to the Panel by the sole Panel
member who understands the language.
After the second day of the site visit, the Panel held an internal meeting where it
agreed on the preliminary conclusions related to level of compliance of NVAO in
relation to each of the standards in part 2 and 3 of the ESG. The secretary of the
Panel then drafted the report in cooperation with the rest of the Panel. The draft
report was submitted to NVAO for factual verification on August 27th, 2012 and
with reference to ENQA standards NVAO was given two weeks to comment on
the report. The final report was submitted to ENQA on September28, 2012.
In relation to its conclusions, the Panel finds it important to note that it has
assessed NVAO‟s level of compliance with the standards and also taken into
account NVAO‟s practice in relation to some of the indicators listed in the
guidelines. It has also provided a number of recommendations. It is the ambition
of the Panel that this approach reflects the new policy for external reviews of
agencies decided by the ENQA board which came into effect on July 1st, 2011.
The policy states (among other things) that the enhancement aspect of the
reviews shall be strengthened in the second round and the agency thus be given
more recommendations for further development than in the first round of
reviews.
The panel has decided to present its understanding of the main points of the
standards and/or the guidelines in the subsections labeled “key elements” in
chapter 4.
vi.
The national (and international) context of the review
During the site visit, the interviewees mentioned several times that NVAO is in a
“transition state” as models and procedures for quality assurance are currently
being changed. It is the view of the Panel that a situation of change and
development is to be expected for most quality assurance agencies. The Panel
recognises that NVAO and its partners have work ahead in order to establish the
institutional assessment approach in Flanders as well as to consolidate the new
practice in the Netherlands. The Panel encourages all the parties to learn from
the experience gathered thus far.
***
Acknowledgement
The Panel would like to thank formally all those that engaged in the process,
including all stakeholders who were generous with their time, feedback and
insights. The hospitable and open approach by the management and staff of
NVAO was very much appreciated by the Panel.
10

4. Compliance with the ENQA membership criteria linked
to the relevant European Standards and Guidelines
(ESG)
4.1 ENQA criterion 1 - Activities (ESG 3.1, 3.3)
Agencies should undertake external quality assurance activities (at institutional
or programme level) on a regular basis. The external quality assurance of
agencies should take into account the presence and effectiveness of the external
quality assurance processes described in Part 2 of the European Standards and
Guidelines. The external quality assurance activities may involve evaluation,
review, audit, assessment, accreditation or other similar activities and should be
part of the core functions of the agency.
Key elements
Concerning the key elements “quality assurance activities”, “a regular basis” and
“core function” they are concrete and perceived in the same way by all. They are
fully met by NVAO‟s accreditation through the 6/8 year cycle and the fact that
NVAO either conducts the quality assurance exercise itself and makes the
accreditation decision (institutional audits in the “new” system in the
Netherlands, initial programme assessments in both NL and FL) or makes
decisions on the basis of the quality assessments performed by quality assurance
agencies (assessments of existing programmes in both NL and FL).
The second paragraph: “… should take into account the presence and
effectiveness of the external quality assurance processes described in Part 2 of
the ESG” refers to ESG Part 2 (8 standards), which again refers to ESG Part 1 (7
standards), each standard having a variable number of guidelines.
Evidence and analysis
NVAO‟s compliance with these standards was considered as fully met in the
ENQA review of 2007. The Panel relied upon the assessment made in this first
ENQA review of NVAO in relation to general characteristics that have not
changed. Where relevant, the Panel has also taken into account the ENQA
reviews of QANU, VLHORA and VLIR.
Reporting on the level of compliance by NVAO with the ESG Part 2 in the
following section is structured in accordance with each of the standards 2.1 to
2.8: sections 4.1.1 to 4.1.8 below contain these assessments. NVAO‟s level of
compliance with ESG 3.1 is linked to compliance with ESG 2.1 – 2.8.
Recommendations
The recommendations of the Panel are presented at the end of each main section
of chapter 4. The first set of recommendations is thus listed at the end of section
4.1

11

4.1.1 ESG 2.1 Use of internal quality assurance procedures
Standard
External quality assurance procedures should take into account the effectiveness
of the internal quality assurance processes described in Part 1 of the European
Standards and Guidelines.
Key elements
“the effectiveness of internal quality assurance processes”, “described in Part 1”.
Evidence
Assessment of the HEI‟s internal quality assurance is a major focus in the Dutch
framework for institutional audits (standards 3, 4 and 5). Internal quality
assurance processes are also incorporated as a separate theme (theme 5) in
both the Flemish (initial) accreditation frameworks and in the Dutch framework
for extensive (initial) programme accreditation.
Each of the quality aspects included in the standards contained in the Part 1 of
the ESG are included in at least one of the frameworks for assessment. The
frameworks are complementary to each other.
NVAO is responsible for all processes in relation to the new institutional audits in
the Netherlands and the assessment of new programmes in both the Netherlands
and Flanders. In both the Netherlands and in Flanders, NVAO is also responsible
for accreditation decisions on existing programmes as well as for specifying
frameworks for assessment to be applied by the quality assurance agencies and
their panels. These frameworks are formulated by NVAO in consultation with,
among others, these quality assurance agencies. In the process of assessing the
reports related to assessments of existing programmes, NVAO checks according
to the SER and interviews during the site visit whether the agencies truly adhere
to the frameworks.
The conclusion reached in the reviews of QANU, VLHORA and VLIR are that these
agencies fully comply with ESG 2.1
Analysis
Both the documentary and orally presented evidence are convincing.
Besides the frameworks for assessments of which the ones in use in the
Netherlands contain explicit references to the ESG, the Panel has been provided
with tables showing the relationship between the focus in each of the different
forms of assessments and the ESG Part(s) 1 (and 2). During the site visit, both
the executive board of NVAO and staff members stressed that the new Dutch
system for external quality assurance has been developed specifically to be
aligned with the ESG. It was confirmed that institutional audit examines the
quality assurance within each HEI as a whole.
Indications for the effectiveness of the focus on internal quality assurance of
HEIs is provided by the fact that representatives from the Dutch ministry
expressed the view that one of the main achievements of the new system for
external quality assurance in the Netherlands is that it contributes to the
improvement of the internal quality assurance within institutions. The General
12

Board expressed the view that there is an increasing emphasis on development
of a quality culture in all HEIs, and similarly the umbrella organisations of the
HEIs in the Netherlands expressed the view that the institutional audits are
useful in encouraging debate within institutions regarding the overall mission,
strategic directions etc.
With respect to Flanders, the Panel considers that the fact that VLIR and VLHORA
both have been accepted as full ENQA members is sufficient proof that they pay
due attention to Part 1 of the ESG in their own processes and thus provides
further support to the conclusion reached by the Panel.
Conclusion
NVAO fully complies with ESG 2.1.
4.1.2 ESG 2.2 Development of external quality assurance processes
Standard
The aims and objectives of quality assurance processes should be determined
before the processes themselves are developed, by all those responsible
(including higher education institutions) and should be published with a
description of the procedures to be used.
Key elements
Developing aims and objectives of quality assurance processes involving
stakeholders, “published with a description of the procedures to be used”, no
unnecessary interference with normal work (the latter is from the guidelines).
Evidence
According to the SER presented by NVAO and confirmed by those interviewed
during the site visit, both the frameworks still in use in Flanders and the new
frameworks for accreditation and institutional audit in use in the Netherlands
were designed in consultation with representatives of institutions, quality
assurance agencies and other assessment experts. Moreover, the new
frameworks used in the Netherlands were designed with reference to a thorough
evaluation of the previous system, followed by various forms of discussions with
external stakeholders and a subsequent pilot-test among a number of higher
education institutions and study programmes in both the Netherlands and
Flanders.
The frameworks were then discussed in parliament and after being decided upon
they were published online on NVAO‟s website well before the start of the
implementation of the systems in both countries. The frameworks include a
description of the procedures to be used. The frameworks for the Netherlands
also containa very short and general description of the aims and objectives of the
processes, but a similar description is not part of the current Flemish frameworks
The aims and objectives of the processes in both the Netherlands and Flanders
are described indirectly, but not specifically, in NVAO‟s new Strategic Policy
Statement, which has been published on NVAO‟s website after the Panel‟s visit.
According to both the SER and those interviewed during the site visit, an explicit
ambition of the new system, and something NVAO has committed itself to
13

achieve, is that it should lead to a 25% reduction of the administrative burden
involved in programme assessments. The idea is that this reduction is to be
achieved by the way limited programme assessment and IA complement each
other. Process oriented aspects that programmes have in common or which are
organised at a higher level, such as facilities and systems of internal quality
assurance, are (NL) or will be (FL) assessed once in the institutional audits and
not repeated at programme level anymore.
Analysis
The impression of the Panel is that all relevant stakeholders, including HEIs, have
indeed been involved in the development of the assessment frameworks. As
noted in the external review of NVAO of 2007, the evaluation of the level of
compliance in relation to this standard has to take into account the fact that
accreditation in Flanders and The Netherlands is regulated by law. This implies
that the global aims and objectives have been determined in a democratic
process by legislation and that the frameworks have been approved by the
respective ministers. In this process the impression from the SER and the
interviews during the site visit is that the frameworks that were developed by
NVAO do take into account comments made by the stakeholders.
The fact that the frameworks do not contain explicit statements of the aims and
objectives of the assessment processes surprises the Panel and is a shortcoming,
but since the aims and objectives are presented elsewhere (albeit not very
publicly) and since all relevant stakeholders have been involved in the design of
the processes, the Panel does not find this to be a critical issue in relation to the
overall assessment of NVAO‟s level of compliance with ESG 2.2 in itself.
However, NVAO has an improvement potential in presenting aims and objectives
in basic introductory parts to each of their frameworks: What kind of effects
(control and/or enhancement) are aimed for connected to each accreditation
procedure. Such an element would also facilitate the understanding of the design
of the processes (section 4.1.4)
In its SER, NVAO demonstrates that it is aware of the fact that it may take some
time before the 25% reduction of administrative burden will be achieved as the
implementation of the new frameworks implies that institutions and programmes
have to get used to the new information files and self-evaluation reports. But by
having set this ambition and committed itself to achieve it and play an active role
in monitoring whether the aim will be achieved, the Panel is convinced that NVAO
does what it can to ensure that the demands on institutions are no greater than
what is necessary for the achievement of the objectives of the new system.
Conclusion
NVAO substantially complies with ESG 2.2.
4.1.3 ESG 2.3 Criteria for decisions
Standard
Any formal decisions made as a result of an external quality assurance activity
should be based on explicit published criteria that are applied consistently.

14

Key elements
“formal decisions”, “explicit published criteria”, “applied consistently”.
By “formal decisions” the Panel understands this to refer to the accreditation
decisions that are executed by NVAO on the basis of their own assessments
(institutional audits in the Netherlands and initial programme assessments in the
Netherlands and Flanders) and assessment reports produced by quality
assurance agencies (assessments of existing programmes in the Netherlands and
Flanders).
“Explicit published criteria” are understood as
1. criteria that underpin quality as the backbone of the assessments
performed by the experts/peers
2. in the context of decision-making: criteria/procedural elements regulating
the appointment and work of experts and also explaining the process from
the presentation of an assessment report to the subsequent accreditation
decision
3. criteria for judging consistency containing indicators for deciding on any
threshold values in use and their consequences.
Evidence
The frameworks contain standards for the different types of quality assessments
and accreditations.
The frameworks contain descriptions on the composition of Panels and on
assessment processes. Regarding the use of threshold values (“scores”), the
frameworks instruct the use of three or four levels dependent on the activity
being undertaken. Chapter 8 in the assessment frameworks in the Netherlands
gives examples illustrating the use of the assessment scales for programme
assessments. Chapter 9 describes how the judgments of the Panel in relation to
individual standards must affect the overall judgment of the programme in case
of programme assessments and the overall judgment of the institution in case of
an institutional audit.
Whereas the framework for the institutional audits in the Netherlands contains a
description of the consequences of the different possible outcomes (a positive, a
conditional positive and a negative judgment) of an audit (section 2.5), the
corresponding sections in the frameworks for (initial) programme assessments in
the Netherlands only present the different possible conclusions that NVAO can
reach.
In its SER, NVAO states that it checks the quality of the assessment procedure
and the quality of the programme by means of an analysis of the assessment
report. If the report raises questions, NVAO requests additional information from
the panels. Finally NVAO (the Executive Board) makes the accreditation decision.
Concerning the process starting from the submission of an assessment report to
an accreditation decision, the assessment frameworks for Flanders contain a
description of the rules governing accreditation decisions (FL section 2.4.2). The
Panel did not find evidence of a similar procedural description in the Dutch
frameworks.

15

The SER states that NVAO has put in place several measures to ensure the
consistency of the evaluation of assessment reports produced by panels,
including:
 The use of detailed manuals for the evaluation of reports, which are
updated on a regular basis.
 Frequent staff meetings where issues regarding the evaluation of
assessment reports are discussed.
 Each application is evaluated by a policy advisor and a board member. If
the evaluation necessitates this, the NVAO asks other staff members or
board members for a second opinion. A final decision on an application is
taken by all board members in a joint session on the basis of a detailed
recommendation.
The representatives of the General Board of NVAO interviewed during the site
visit further noted that the General Board looks at consistency of decisions made
by the Executive Board.
Analysis
When NVAO underwent a similar review in 2007, the quality assurance activities
and criteria were alike for the Netherlands and Flanders. Today, NVAO states
that the regulations are still the same in Flanders as it was then. In 2007 the
assessments at this point concluded with “fully compliant”. This Panel relies upon
the conclusion made in 2007 concerning the Flemish situation since no change
has been made yet. In the present assessment the Panel focused on the new
Dutch system that was introduced in 2011.
Whereas the assessment frameworks for Flanders contain precise descriptions of
the rules that NVAO‟s formal accreditation decisions are based on, the
assessment frameworks for the Netherlands do not contain similar descriptions.
Since the assessments of existing programmes are executed by the Dutch quality
assurance agencies and thus not by NVAO, the Panel finds it critical that it is not
made clear how NVAO reaches its accreditation decisions. In other words, it was
unclear to the Panel which criteria NVAO uses, if any, for deciding whether to
follow or deviate from the conclusions reached by a quality assessment panel
and how NVAO assures consistency in this respect. NVAO did not clarify this
issue during discussions with the Panel during the site visit. It is important that
NVAO respects the assessments made by the panel experts and that NVAO
makes its considerations and ultimate accreditation decision in a manner and
based on clear evidence that is completely transparent in all cases to all the
involved parties.
In practice, however, the internal procedures/the mechanisms within NVAO
appear to promote consistency in the decision making process and the site visit
revealed no strong feelings on this matter.
Nevertheless, the Panel noted, during the site visit, a view that there is still a
possibility for improvement regarding the consistency of accreditation judgments
reached. NVAO also appears to be aware of the challenges regarding consistency
as “the points of attention” raised in relation to ESG 2.3 in its SER are both
concerned with the issue of consistent application of criteria. Furthermore, one of
the 6 issues NVAO list as those it should pay attention to in the next years in
order to improve its operation and the operation of the accreditation system in
16

the Netherlands and Flanders is: The large number of applications necessitates
permanent attention to consistency in decision making, where criteria and
mechanisms are closely connected.
Conclusion
NVAO partially complies with ESG 2.3.
4.1.4 ESG 2.4 Processes fit for purpose
Standard
All external quality assurance processes should be designed speciﬁcally to ensure
their ﬁtness to achieve the aims and objectives set for them.
Key elements
This standard is in itself wide and abstract and therefore may be subject to many
different interpretations. However, it is followed by quite extensive guidelines. In
order to establish a common understanding of quality related to this standard,
the Panel has included the guidelines as a starting point of its analysis:
“Quality assurance agencies within the EHEA undertake different external
processes for different purposes and in different ways. It is of the ﬁrst
importance that agencies should operate procedures which are ﬁt for their own
deﬁned and published purposes. Experience has shown, however, that there are
some widely-used elements of external review processes which not only help to
ensure their validity, reliability and usefulness, but also provide a basis for the
European dimension to quality assurance.
Amongst these elements the following are particularly noteworthy:
 insistence that the experts undertaking the external quality assurance
activity have appropriate skills and are competent to perform their task;
 the exercise of care in the selection of experts;
 the provision of appropriate brieﬁng or training for experts;
 the use of international experts;
 participation of students;
 ensuring that the review procedures used are sufﬁcient to provide
adequate evidence to support the ﬁndings and conclusions reached;
 the use of the self-evaluation/site visit/draft report/published
report/follow-up model of review;
 recognition of the importance of institutional improvement and
enhancement policies as a fundamental element in the assurance of
quality.”
In addition to the bullet points above, the Panel is concerned with the length of
time that elapses from presenting the assessment report until an accreditation
decision has been made. When time is unpredictable and/or running too long, it
is a challenge for the involved parties to keep up an interest in the result and a
motivation for action upon it. This is especially essential to support the aim of
quality enhancement.
Evidence
The following subsections focus exclusively on the presence and characteristics of
the elements of the processes that are listed in the guidelines to ESG 2.4.
17

Characteristics of the panels
The panels‟ required competencies are described for all activities in the various
assessment frameworks.
NVAO appoints the panels for institutional audits (NL) and for initial programme
assessment in both the Netherlands and Flanders. In terms of the assessment of
existing programmes in the Netherlands, the panels are nominated by the HEIs,
and the final composition is determined by the quality assurance agencies. Final
approval of the composition of the panels is made by NVAO.
According to the representatives of the Dutch quality assurance agencies
interviewed during the site visit, approximately 25% of the suggested panel
members are queried by NVAO. In Flanders panels for the assessment of existing
programmes are decided by the Flemish quality assurance agencies and
approved by the Recognition committee, an independent external body that
checks the independence of proposed members of review panels, before the
assessment takes place.
All of the Dutch assessments frameworks include a section describing how the
independence (both actual and perceived) of the panel members is secured and
the requirements that the panels must meet. In all cases requirements with
regards to the skills and competences of the panel members are specified. For all
types of assessment in the Netherlands it is also (among other things) a
requirement that the panels include a student and a member with international
expertise. Requirements concerning the composition of panels are described in
the Flemish accreditation decree and in the NVAO rules about quality assurance
agencies and both VLIR and VLHORA have requirements about student
participation and international expertise in their panels.
Training of panel members
Student members of the panels for initial programme assessments (both NL and
FL) and potential chairs for the institutional audits in the Netherlands are trained
by NVAO. The training for institutional audit lasts two days and includes, among
other things, role play. Panel members participating in the assessments of
existing programmes in the Netherlands and Flanders are instructed or trained
by the quality assurance agencies, but NVAO offers a “Train the trainer module”
for programme assessment.
For all types of the assessment panels in the Netherlands, those secretaries who
are not panel members are trained by NVAO. When the site visit of NVAO took
place approximately 200 secretaries had been trained. The training focuses on
the assessment frameworks, how to use them, how to guide the panel members
etc. Furthermore, secretaries participate in two reflection days per year at NVAO.
Provision of adequate evidence
The frameworks for assessments describe the review procedures in detail and
include sections stressing the importance of providing adequate evidence to
support the ﬁndings and conclusions reached.

18

Whether findings and conclusions are supported by adequate evidence is also the
main focus when NVAO assesses the reports produced by the quality assurance
agencies – and the main reason for sometimes asking for more information.
The use of self-evaluation/site visit/draft report/published report/follow-up model
of review
Self-evaluation, site visit, draft report and published report all form parts of all
the assessment processes in the Netherlands and Flanders. Follow-up is part of a
process in the assessment of existing programmes if a recovery period has been
granted. For initial programme assessments and for institutional audits, a followup review takes place only where positive decisions are made subject to
conditions.
Recognition of the importance of institutional improvement and enhancement
policies as a fundamental element in the assurance of quality
According to the SER, the new accreditation system in the the Netherlands
heralds a new approach. The SER describes the new system as challenging the
institutions to set higher targets for their programmes and their quality work in
general and thus improving quality. Furthermore, it is stated in the SER that the
new frameworks encourage the development of quality assurance and of a
quality culture within the entire institution and that the new system allows for a
more thorough discussion about systematic quality assurance and quality culture
at institutional level and the quality of content and learning in individual
programmes. Staff interviewed noted that reports are directed towards
curriculum development and improvement.
Time from submission of report until decision has been made
Interviews revealed that the time from presentation of assessment report until
decision has been made by NVAO is of varying length and, in some cases,
exceeds one year. The interviewees not employed at NVAO did not know why the
delay is so unpredictable. Some also expressed the opinion that, in some cases,
conditions within the HEI had changed substantially between the time of the site
visit and the receipt of the review report and that the decision and
recommendations were no longer truly relevant. The interview with staff
revealed that NVAO is challenged by the fact that most of the HEI‟s apply for
accreditation at the same time of the year which means that there is a great
imbalance over the year in the amount of applications that NVAO has to handle.
Analysis
The impression of the Panel from reading the SER and the assessment
frameworks is that most processes are fit for purpose. This impression is further
strengthened by the fact that none of those interviewed during the site visit
questioned or criticised the processes, except for some overly long delays in
delivery of the final report.
As the description above reveals, the processes include all elements listed in the
guidelines to ESG 2.4. Interviews during the site visit gave the impression that
the panels are considered to be competent and doing a good job. In the
interview with the Dutch quality assurance agencies, the view was raised that for
those secretaries who have already been trained by a quality assurance agency,
the NVAO training may be felt as a waste of time; the quality of the training
19

provided by NVAO was also questioned, but this view was not raised in any other
interviews.
Furthermore, the Panel has noted that in the reviews of VLIR and VLHORA the
composition of the panels for the assessment of the Flemish programmes and
the training of these panel members, as well as the assessment processes in
Flanders in general, have been positively assessed and approved.
The issue of the delay to decision-making is not specifically addressed in ESG.
The Panel considers this issue as being a part of its considerations of “fit for
purpose”, which is illustrated by the information during interviews as given
above. There is, however, no reason to believe that there are too many cases of
unpredictable and inappropriate long timelines, but in the view of the panel the
delays are still a critical issue.
Generally, the impression of the Panel is that the new system in the Netherlands
and not least the inclusion of institutional audits and the possibility for granting a
recovery period do indeed promote more focus on quality improvement and
enhancement than the previous system did. This impression is supported by the
SER which, based on an evaluation of the first institutional audits, concludes that
the assessed institutions consider the audit a valuable learning experience that
will positively affect the quality of the programmes they offer and causes staff to
reflect more intensively on internal quality assurance at an institutional level. At
the same time the Panel finds that the lack of focus on recommendations for
improvement in the decision reports in case of positively accredited programmes
in the Netherlands suggests that the focus on quality improvement and
enhancement can still be further strengthened.
Conclusion
NVAO substantially complies with ESG 2.4.

4.1.5 ESG 2.5 Reporting
Standard
Reports should be published and should be written in a style which is clear and
readily accessible to its intended readership. Any decisions, commendations or
recommendations contained in reports should be easy for a reader to ﬁnd.
Key elements
“Intended readership”, “readily accessible”
Evidence
As regards assessments of existing programmes, in Flanders, the institution
applies for accreditation of a programme with NVAO, submitting the assessment
report of the panel produced and published by the evaluating agency. In the
Netherlands, the assessment report is published after completion of the
accreditation procedure.
In both cases the reports have a summary, which NVAO can use to provide
information to the public. NVAO requires the summary to be very concise in
order to inform interested readers at a glance about the most relevant
characteristics of the programme. In addition, every assessment report contains
20

a scoring table and a separate paragraph with recommendations. The staff
interviewed during site visit stated that NVAO is the primary intended readership
for the programme assessment reports, HEIs the second one and the general
public the third one. The staff also stated that summaries have been introduced
primarily for the benefit of students and external stakeholders.
The NVAO accreditation decision reports are based on the assessment reports
and contain a summary of the findings and discussions contained in these reports
(the summary and the scoring table are integral part of the decision document),
followed by the formal decision of NVAO.
For all types of assessments, the reports and decision documents have a fixed
and structured format with the purpose of increasing readability. In 2012, NVAO
started an evaluation project to investigate how well the new reports respond to
the needs of the intended readers.
Both NVAO‟s decisions and the panel assessment reports are made public on
NVAO‟s website (www.nvao.net). The site is equipped with a search tool to find
information about a specific programme or an institution. NVAO‟s website lists all
programmes in the Netherlands and Flanders which were submitted for
accreditation.
Analysis
The reports produced and published by the Flemish quality assurance agencies
have generally been positively assessed in the external reviews of VLHORA and
VLIR. In the case of VLHORA, some criticism was raised (in 2008) about the
clarity with regard to the intended readership and the lack of a summary which
led to an assessment of VLHORA as being “only” substantially compliant; a
recommendation to include a summary in the report was made. Since the
Flemish reports - like the new Dutch reports - now have to include a summary,
but have not otherwise been affected by the introduction of the new system in
the Netherlands, the Panel rests its assessment of the Flemish reports on the
external reviews of VLIR and VLHORA, leaving aside only the issue regarding the
summary.
Regarding the reports related to Dutch programmes, NVAO states in its SER that,
with the introduction of the new system, NVAO agreed with the quality assurance
agencies to improve the readability of the programme assessment reports. NVAO
feels that the first results in the Netherlands are promising. NVAO expects that
the main findings and conclusions of a report will be easier to read for students,
employers and other involved groups.
More specifically, NVAO finds, according to its SER, that, as a result of the
introduction of limited programme assessment and extended programme
assessment, the first panel reports produced in the new format are more
informative and transparent than previous reports.
During the site visit, the relevant external stakeholders were questioned about
their view on the quality of the new Dutch reports. Both the representatives of
the Dutch Ministry of Education, the Dutch umbrella organisations, the Dutch
student organisation and the Advisory Board expressed a generally positive
appreciation of the first reports and that the reports are good and have been
21

improved with the new system, but that there is still room for improvement (or a
complete implementation of the new guidelines). The summaries were
highlighted as a new and very positive element in the reports, not least in terms
of making the reports more accessible to a wider audience including students.
The main critique raised by the representatives mentioned above was that the
intended readership of the reports is too narrow and (as a consequence) the
reports are too technical to be easily read by the general public.
In the course of the review, the Panel has looked into and assessed a number of
the reports that have been published after the introduction of the new system in
The Netherlands. The conclusion in relation to the reports following the new
institutional audits as well as the ones for the revised form of (initial) limited and
extended assessments is that they faithfully follow the relevant assessment
framework of NVAO and that the structure is more or less identical across
assessments within the same framework. All include a summary and are
structured to cover description, analysis (including relevant evidence),
conclusions and some forms of recommendations. They also contain sufﬁcient
preliminary explanation to enable a lay reader to understand the purposes of the
review, its form, and the criteria used in reaching conclusions. The criteria used
in making decisions are not explained in the NVAO accreditation decision reports
(Cf. section 4.1.3 for an assessment of this). The writing style is literary with a
few other steps to enlightening major issues or main findings.
As for the accessibility the Panel finds that, despite the fact that the search
facility of NVAO‟s web site makes it rather hard to find a specific assessment
report, the assessment reports contain what NVAO needs to be able to reach an
accreditation decision and thus fit the primary intended readership of the reports
as defined by NVAO. Despite the introduction of a summary, the Panel is,
however, not convinced about the general readability for other stakeholders and
agrees with the external stakeholders who during the site visit characterised the
reports as being too technical – and maybe also too verbose – to read for the
general public, including (prospective) students. Thus the panel also agrees with
the stakeholders that the intended readership of the reports defined by NVAO is
too narrow.
Having said this, The Panel has positively noted that NVAO has recently started
an evaluation project to investigate how well the new reports respond to the
needs of the intended readers besides NVAO itself.
Conclusion
NVAO substantially complies with ESG 2.5.
4.1.6 ESG 2.6 Follow-up procedures
Standard
Quality assurance processes which contain recommendations for action or which
require a subsequent action plan, should have a predetermined follow-up
procedure which is implemented consistently.
Key elements
“recommendations for action”, “a predetermined follow-up procedure”
22

“The objective is to ensure that areas identiﬁed for improvement are dealt with
speedily and that further enhancement is encouraged.” (Guideline)
Evidence
The programme assessment reports contain a scoring table and a chapter with
recommendations. The institutional audit reports also contain a chapter with
possible improvements/recommendations listed. When a conditional accreditation
decision is reached in relation to an institutional audit or an initial programme
assessment, a subsequent action plan is required. In case of the application of a
recovery period in relation to an assessment of an existing programme, the
institution is required to present a convincing recovery plan before the recovery
period is granted.
NVAO states, in its SER, that a formal follow-up procedure is not included as an
overall obligatory part of the accreditation system.
At the same time, the SER makes clear that a recovery period has been part of
the programme accreditation system in Flanders since 2005 and is part of the
new procedure for programme accreditation in the Netherlands. More
importantly, it is stated that in the case of the application of a recovery period,
the institution needs to present a convincing recovery plan before the recovery
period is granted and that, at the end of the recovery period, the achieved
improvements have to be assessed positively by an assessment panel before
accreditation can be granted.
Similarly, NVAO now has the authority to take a conditionally positive decision in
the procedures of initial programme accreditation and institutional audit. In the
case of a conditional decision in the Netherlands, a NVAO panel must follow-up
and assess whether the programme or the institution meets the conditions within
the set conditional period.
In the case of an unconditionally positive decision, there is no follow-up about
the extent to which the institution and/or the programme management
implement the recommendations of the assessment panel. In these cases, the
follow-up on these recommendations will be assessed in the subsequent
accreditation procedure (after 6 to 8 years).
In the 6 to 8 year period between being subject to an accreditation procedure,
the Panel learned from the site visit that more and more programmes and
institutions voluntarily engage in some forms of other reviews and publish
reports based on these.
Analysis
In its SER, NVAO states: ”An official follow-up procedure, as meant here in the
ESG guideline, is not a part of the accreditation system either, but two measures
introduce elements that provide equivalent functionality to a follow-up procedure
…”
The panel finds that follow-up procedures have been strengthened in the
Netherlands through the introduction of a recovery period similar to what has
been in place in Flanders since 2005.
23

It is now only in the case of an unconditionally positive decision that a follow-up
procedure is not applied. As ESG 2.6 states that quality assurance processes
which contain recommendations for action or which require a subsequent action
plan should have a predetermined follow-up procedure which is implemented
consistently, and as unconditionally positive accreditation decisions do not
contain recommendations or require a subsequent action plan the standards does
not apply to this part of the quality assurance processes of NVAO. Therefore the
view of the Panel is that NVAO‟s level of compliance with ESG 2.6 is adequate.
Moreover the Panel is convinced that the objective of ESG 2.6, as expressed in
the guidelines to the standard, “to ensure that areas identiﬁed for improvement
are dealt with speedily and that further enhancement is encouraged” is fulfilled.
Conclusion
NVAO fully complies with ESG 2.6.
4.1.7 ESG 2.7 Periodic reviews
Standard
External quality assurance of institutions and/or programmes should be
undertaken on a cyclical basis. The length of the cycle and the review procedures
to be used should be clearly deﬁned and published in advance.
Key elements
“Cyclical basis”, “review procedures”, “clearly defined”, “published in advance”
From the guidelines: “demands on institutions should not be greater than are
necessary”
Evidence
The accreditation period of programmes in the Netherlands is six years, in
Flanders it is eight years. After an initial accreditation, reassessment of the
programme and reaccreditation should be finalized within six years (in the
Netherlands) and within the length of the programme and two years (in
Flanders). These cycles are mandated by Dutch and Flemish legislation.
The Dutch HEI‟s are subject to an institutional audit every six years.
Analysis
The description above clearly reveals that external quality assurance in both the
Netherlands and Flanders are undertaken on a cyclical basis and that the length
of the cycle and the review procedures are clearly deﬁned and published in
advance.
Conclusion
NVAO fully complies with ESG 2.7.

24

4.1.8 ESG 2.8 System-wide analyses
Standard
Quality assurance agencies should produce from time to time summary reports
describing and analysing the general ﬁndings of their reviews, evaluations,
assessments etc.
Key elements
“from time to time”, “summary reports”, “describing and analysing general
findings”
Evidence
In the previous review in 2007, NVAO was considered only partially compliant
with ESG 2.8 and was advised to give more attention to the production of
system-wide and comparative analyses.
In the past years, NVAO completed a number of comparative analyses within
clusters or domains. The analyses are mentioned in the SER and are listed and
described in the attachments to it. An evaluation of the former accreditation
system and of the pilots undertaken in order to develop the new system are
other examples of system-wide analyses mentioned in the SER, but these have
not been included in the list in the attachments and at the time of the review it
was therefore unclear to the Panel whether they resulted in any form of reports
or what kind of results they produced.
In relation to the future, NVAO‟s new strategic policy underlines the importance
of comparative and system-wide analyses and, according to the SER, NVAO
recruited a new staff member assigned to do quantitative and qualitative
research to underpin the analyses and a number of different system-wide
analyses have been planned.
Analysis
From reading the description and analysis of the last external review of NVAO
which led to the conclusion that NVAO only partially complied with ESG 2.8 as
well as the present SER and list of comparative analysis published by NVAO since
2007, the Panel finds that NVAO has substantially improved its practice in
relation to ESG 2.8.
The Panel notes, however, that NVAO itself is relatively critical about its
achievement so far. In its SER, it states that despite its substantial effort
regarding comparative analyses, it did not have a clear policy on this issue in
recent years. It is further stated that in the new Strategic Policy Statement
(2012 - 2016) the NVAO task of system-wide and comparative analyses is
emphasized and should result in a clear agenda on this topic for the coming
years and “that NVAO has to work on implementing its new policy regarding
system-wide and comparative analyses”. Finally it concludes that one of its
points of attention is to implement a clear policy and activities regarding systemwide analyses through yearly planning.
Some of the stakeholder representatives interviewed during the site visit
expressed the view that they would like NVAO to do more system-wide analysis.
Issues such as internationalisation, student experience and comparative analysis
25

of how the quality assurance agencies in Flanders and the Netherlands work were
mentioned in this regard. This may require NVAO to adopt a more cluster-based
approach than is currently the case.
As the Panel is informed, the situation is different in Flanders and in the
Netherlands: the Flemish agencies operate on clusters of similar programmes
within all HEIs which the Panel believes facilitates system-wide comparisons,
while in the Netherlands – despite cluster assessment of university programmes
since 2003 the process is geared more towards assessments of single institutions
or single programmes.
In its assessment of NVAO‟s level of compliance with ESG 2.8, the Panel has
given the critical self-assessment contained in the SER of NVAO a high weighting.
Conclusion
NVAO substantially complies with ESG 2.8.
4.1.9 ESG 3.1 Use of external quality assurance procedures for higher
education
Standard
The external quality assurance agencies should take into account the presence
and effectiveness of the external quality assurance processes described in Part 2
of the European Standards and Guidelines.
Key elements
This standard is understood holistic in its full text, there are no key words as
such in this formulation.
Evidence and analysis
In section 4.1.1 to 4.1.8 the Panel has assessed the NVAO‟s level of compliance
with ESG 2.1 to 2.8 and thus with Part 2 of the European Standards and
Guidelines. The outcome is that NVAO is considered fully compliant with ESG 2.1,
2,6 and 2.7, substantially compliant with ESG 2.2, 2.4, 2.5 and 2.8 and partially
compliant with ESG 2.3.
Conclusion
Based on its assessment of NVAOs level of compliance with ESG 2.1 to 2.8, the
Panel concludes that NVAO substantially complies with ESG 3.1
4.1.10 ESG 3.3 Activities
Standard
Agencies should undertake external quality assurance activities (at institutional
or programme level) on a regular basis.
Key elements
This standard is understood as holistic in its full text, there are no key words as
such in this formulation.
26

Evidence
The tasks of NVAO are described in the Treaty. They are further stipulated in the
Netherlands in the Dutch Act on higher education and research and can be
summarized as the (initial) accreditation of programmes of institutions of higher
education and the assessment of these institutions (Institutional audit).
In Flanders, the Flemish Act of 4 April 2003 forms the legal basis for (initial)
accreditation in higher education and stipulates that the responsibility for (initial)
accreditation of programmes is assigned to NVAO. Since it was established in
2003, NVAO has processed nearly 5000 applications for accreditation.
NVAO can be requested to carry out additional tasks by the Ministers of higher
education in both countries if these assignments support or supplement NVAO‟s
primary assignment.
Analysis
The Panel finds that the description above as well as sections 4.1.1 to 4.1.8
clearly illustrates that NVAO undertakes external quality assurance activities on a
regular basis.
Conclusion
NVAO fully complies with ESG 3.3.
**
Conclusion ENQA criterion 1
Based on its assessment of NVAOs level of compliance with 3.1 (including ESG
2.1 to 2.8) and 3.3, the Panel concludes that NVAO substantially complies with
the ENQA membership criterion 1/ESG 3.1 and 3.3
Recommendations
With reference to the evidence and analysis provided in section 4.1.1 to 4.1.10
the Panel recommends that NVAO:









makes the link between the NVAO standards and ESG Part 1 more explicit in
the assessment frameworks;
refines the descriptions of the aims and objectives, ensures that they are
prominent in the frameworks and show how the various elements of the
frameworks contribute to the aims and objectives;
formulates explicit and public criteria about how it reaches its accreditation
decisions in relation to the programme assessments in all cases;
establishes a clear procedure on how to handle cases where the conclusions
in the assessment report are not accepted by NVAO;
strengthens the predictability of the timeframe and efficiency of its decisionmaking process;
further strengthens the focus on quality improvement and enhancement of
the HEI‟s;
clarifies a) the purpose of every kind of report, b) the readership and c) the
needs of the various kinds of readers in order to enhance the readability;
gives a high priority to: a) the identification of the interested parties for
system-wide analyses and of their needs; b) the definition of a realistic
27

schedule of system-wide analyses; and c) the production of system-wide
analyses corresponding to the needs which were identified.

28

4.2

ENQA criterion 2 / ESG 3.2: Official status

Standard
Agencies should be formally recognised by competent public authorities in the
European Higher Education Area as agencies with responsibilities for external
quality assurance and should have an established legal basis. They should
comply with any requirements of the legislative jurisdictions within which they
operate.
Key elements
This standard is understood holistically in its full text, there are no key words as
such in this formulation.
Evidence
NVAO is the official, public and bi-national accreditation organization in higher
education in the Netherlands and Flanders. Its structure and duties are described
in a bi-national Treaty (2003) and in both national legislations. NVAO reports to
the Dutch and Flemish Parliaments via their Ministers of (Higher) Education.
NVAO‟s annual report is used for accountability purposes.
Analysis
From the description in the SER and the legal documents it refers to it is evident
to the Panel that NVAO has an established legal basis and is formally recognised
as required by ESG 3.2
Although the Panel did not explicitly ask about it during the site visit, the fact
that no one raised any points which could suggest that NVAO does not comply
with requirements of the legislative jurisdictions within which it operates leads
the Panel to conclude that NVAO fully complies in this respect. The fact that all
external stakeholders expressed a general satisfaction with the work of NVAO
and that NVAO was considered fully compliant with ESG 3.2 in the 2007 review,
supports this assessment.
Conclusion
NVAO fully complies with ENQA criterion 2 / ESG 3.2.

29

4.3

ENQA criterion 3 / ESG 3.4: Resources

Standard
Agencies should have adequate and proportional resources, both human and
ﬁnancial, to enable them to organise and run their external quality assurance
process(es) in an effective and efﬁcient manner, with appropriate provision for
the development of their processes, procedures and staff.
Key elements
This standard is understood holistically in its full text, there are no explicit key
words as such in this formulation. However, the operationalization of the
standard: that is, what evidence should be presented and how it should be
analysed by the Review Panel, is more of a question.
Evidence
NVAO has an annual budget of approximately € 6 million, which is financed
jointly by the Netherlands (60%) and Flanders (40%). Each year, NVAO draws
up a budget which is then decided upon by the Committee of Ministers and both
Parliaments. In the Netherlands and Flanders, NVAO applies fixed rates by law.
The rate for an application for initial accreditation is maximum € 15,000 in the
Netherlands and € 5,000 in Flanders. For an application for accreditation the rate
is € 750 in the Netherlands and € 500 in Flanders. In the case of additional tasks
and assessment assignments abroad, NVAO charges cost-covering fees. NVAO
has agreed with both ministries in the Netherlands and Flanders that additional
tasks assigned by the ministries will be organized and budgeted on a project
basis. NVAO informs the ministries in advance about the expected costs. The
latter can decide whether to continue the project initiative or not.
Since 2005, NVAO is located in the Parkstraat in The Hague, near the city and
the Dutch parliament and the Dutch ministry of education. Several meeting
rooms for smaller and larger groups are available for the organisation of
conferences, seminars and meetings. Presentation and IT-facilities are up-todate. In November 2011, a new information system was introduced to optimize
the work-flow.
The workforce of NVAO consists of 56 people (53FTEs): four executive Board
members, one managing director, 30 policy advisors, one Dutch and one Flemish
legal advisor (together 29FTEs) and 20 other (partly supporting) staff (19 FTEs).
Four policy advisors are available for international assignments. The staff
composition includes policy, legal and communication advisors and supporting
staff (policy secretariat, finances and personnel, records department and general
services).
All academic staff hold a master‟s degree, with three members holding a PhD
degree. The acquired professional experience of NVAO‟s staff is varied. All board
members and the director have a longer career in (higher) education in board or
management positions. Most staff acquired substantial work experience in higher
education as teacher, developer, manager, researcher, inspector for higher
education, or policy advisor before being employed by NVAO. A small group of
junior staff was recruited to fulfill a kind of traineeship before being fully
employable in NVAO‟s work processes. NVAO has defined core competencies of
its staff.
30

Each member of staff participates in a performance review once a year. On this
occasion staff have the opportunity to ask for further training if they find it
relevant.
Analysis
According to its SER, NVAO has been funded sufficiently by both governments for
its primary tasks in recent years and has received appropriate additional funding
for the additional tasks that were assigned to the organization. Despite the fact
that NVAO, like all public organizations, is currently subject to budget cuts, NVAO
states that these cuts have not affected its operations up to now. As the critical
issue related to respecting deadlines does not appear to arise (primarily) from a
lack of resources (cf. section 4.1.3) the impression of the Panel from the
interviews during the site visit matches the self-assessment of NVAO.
In its SER, NVAO concludes that good facilities are available at its location and,
having visited and used these, the Panel fully agrees with that.
The SER states that, over the years, NVAO has developed a good quality of staff
with varied competences and work experiences. With reference to a recent
benchmark study of staff development among five members of ENQA, including
NVAO, it concludes that compared to similar members within the ENQA network,
NVAO staff is on par. The positive assessment of the competences of staff was
not questioned by any of those interviewed during the site visit. On the contrary,
the representatives of the General Board, for example, expressed the belief that
the quality of the staff of NVAO is one of the primary reasons for the credibility
and respect with which NVAO is held in the community.
The impression from the Panel‟s interview with NVAO staff is that they are
satisfied with the possibilities offered for further training or other forms of
competence development activities. On the job training and a mentoring system
were mentioned by staff as some of the important forms of activities, particularly
for newer members of staff.
NVAO finds, in its SER, that further development of expertise of staff on specific
themes in higher education should be improved and thus that it should invest in
further development of specific, theme-oriented, higher education expertise of its
staff. Although it may be a relevant priority for NVAO, the Panel does not in any
way find that it is needed for the purpose of being fully compliant with ESG 3.4.
Conclusion
NVAO fully complies with ENQA criterion 3 / ESG 3.4.

31

4.4 ENQA criterion 4 / ESG 3.5: Mission statement
Standard
Agencies should have clear and explicit goals and objectives for their work,
contained in a publicly available statement.
Key elements
“clear and explicit”, “publicly available” and from the Guidelines: “the division of
labour with relevant stakeholders in higher education” “documentation to
demonstrate how the statements are translated into a clear policy and a
management plan”
Evidence
In the “Strategic Policy Statement NVAO 2012 – 2016”, NVAO has defined its
mission as follows:
NVAO is the independent and authoritative accreditation organisation set up by
the Flemish and Dutch governments, whose primary goal it is to provide an
expert and objective judgement of the quality of higher education in Flanders
and the Netherlands. NVAO does this with a constructive, critical attitude,
respecting the autonomy of institutions and their primary responsibility for the
quality of their education, and with an open eye for the growing international
context. NVAO is open, clear and transparent towards society and all concerned,
especially the institutions of higher education and the students.
At the time of the review the new Strategic Policy Statement was not published
on the website of NVAO, but the mission statement was presented on the
website.
The new Strategic Policy Statement includes a section describing the context of
the work assigned to NVAO; the division of labour with relevant stakeholders in
higher education is described. NVAO states that it considers institutions to be
primarily responsible for quality assurance and quality improvement. NVAO is
required to assure that programmes meet the required standards and to
stimulate the quality debate, giving account of its procedures, disseminating
„good practices‟ and visiting institutions and their programmes.
In relation to this the mission statement lists the three main tasks of NVAO as
being:
1. Assessing and assuring the quality of Dutch and Flemish higher education.
2. Promoting the quality of higher education by promoting a culture of
quality, aimed at regular assessment and continuous quality increase.
3. Putting Dutch and Flemish sectors of higher education (institutions,
programmes) on the map and strengthening their position by means of
international cooperation.
The Strategic Policy Statement also includes sections that translate the mission
(and values and positioning) into a number of strategic goals and a strategic
agenda for NVAO for 2012 to 2016.

32

Analysis
It is evident from the description above that NVAO has clear and explicit goals
and objectives for its work. The mission statement is published on NVAO‟s
website.
A description of the cultural and historical context of the work of NVAO is
included in the separate context section. The Strategic Policy Statement clearly
demonstrates the translation of the mission statements into a clearly formulated
policy and management plan. The division of labour with relevant stakeholders in
higher education is also described in the statement. The list of the main tasks of
NVAO as well as other parts of the statement make clear that external quality
assurance processes are major activities of the agency. The assessment
frameworks confirm that NVAO employs a systematic approach to achieving its
goals and objectives.
Conclusion
NVAO fully complies with ENQA criterion 4 / ESG 3.5.

33

4.5 ENQA criterion 5 / ESG 3.6: Independence
Standard
Agencies should be independent to the extent both that they have autonomous
responsibility for their operations and that the conclusions and recommendations
made in their reports cannot be inﬂuenced by third parties such as higher
education institutions, ministries or other stakeholders.
Key elements
“Autonomy”, “independence”
The Panels finds this standard is thoroughly described in the following guidelines.
An agency will need to demonstrate its independence through measures, such
as:
•
its operational independence from higher education institutions and
governments is guaranteed in ofﬁcial documentation (e.g. instruments of
governance or legislative acts);
•
the deﬁnition and operation of its procedures and methods, the nomination
and appointment of external experts and the determination of the
outcomes of its quality assurance processes are undertaken autonomously
and independently from governments, higher education institutions, and
organs of political inﬂuence;
•
while relevant stakeholders in higher education, particularly
students/learners, are consulted in the course of quality assurance
processes, the ﬁnal outcomes of the quality assurance processes remain
the responsibility of the agency.
Evidence
NVAO was granted the status of an autonomous administrative body with legal
rights according to Dutch legislation. NVAO is accountable to the Committee of
Ministers, which approves its budget, the annual report and the annual accounts.
In accordance with the Treaty, the Committee of Ministers can only intervene in
case of serious neglect by NVAO of its (initial) accreditation task, threatening the
execution of that task. The Committee of Ministers can thus only intervene in the
functioning of NVAO, but not in NVAO‟s decision-making. From the start of the
accreditation system in the Netherlands and Flanders (2004, NL – 2005, FL)
NVAO has been fully independent regarding decision making.
Members of NVAO‟s Executive and General Board are mandated for four years by
the Committee of Ministers. They are appointed in a strictly personal capacity
and not as representatives of any organisation.
In order to guarantee the independence of board and staff members, NVAO has
ruled that members of both categories cannot participate in applications from
institutions or programmes they have been associated with in any form during
the previous five years. Members of the Board have to be completely
independent in making decisions. If there is a specific application where this
independence cannot be guaranteed, the member of the Board will withdraw
from the decision-making process for the programme concerned.
The legally binding assessment frameworks formulated by NVAO in consultation
with relevant stakeholders define the standards which the assessments have to
34

refer to, the possible judgements in relation to each of the standards and the
general conclusion that they may result in and how this conclusion should be
reached (assessment rules), requirement regarding the composition of panels,
the assessment processes and the decision-making by NVAO. The assessment
frameworks for (initial) programme assessments also include a list of the
documents that the programmes need to provide and guidance as regards the
“threshold levels” for the different possible judgement that can be reached by an
assessment panel. Finally the assessment frameworks include a chapter in which
NVAO outlines the rules laid down by implementing regulations regarding
conditional decisions and granting of improvement periods.
Analysis
The view of NVAO expressed in its SER is that the official status offers NVAO a
good formal structure to guarantee its independence and it states that its
independence is recognised by different stakeholders. The interviews with
stakeholders during the site visit confirmed this statement.
It is also evident to the Panel that NVAO has autonomous responsibility for its
operations and its operational independence from higher education institutions
and governments is guaranteed in ofﬁcial documents.
The very detailed and legally binding assessment frameworks (as well as the
continuous monitoring of the level of adherence to them) ensure that the
appointment of external experts and the determination of the outcomes of the
quality assurance processes are undertaken autonomously and independently
from governments, higher education institutions, and organs of political
inﬂuence. They also ensure that the conclusions and recommendations made in
the reports cannot be inﬂuenced by third parties such as higher education
institutions, ministries or other stakeholders and that the ﬁnal outcomes of the
quality assurance processes (the accreditation decisions) remain the sole
responsibility of NVAO.
Since the assessments of existing programmes are not carried out by NVAO, the
Panel finds that the independence of the quality assurance agencies carrying out
these types of assessments needs to be considered as part of the assessment of
the operational independence of NVAO. In this regard, the Panel has noted that
in the external assessments of both VHLORA and VLIR these agencies have been
assessed as being fully compliant and the Dutch agency QANU as being
substantially compliant with ESG 3.6 Independence and so do not raise questions
as regards the independence of the processes carried out by these agencies. The
situation is less clear regarding the other Dutch quality assurance agencies which
were not (yet) vetted by ENQA, but NVAO explicitly requires their panel members
to adhere to a Code of Conduct and to sign an independence and confidentiality
declaration.
Conclusion
NVAO fully complies with ENQA criterion 5 / ESG 3.6.

35

4.6 ENQA Criterion 6 / ESG 3.7 External quality assurance criteria
and processes used by the members
Standard
The processes, criteria and procedures used by agencies should be pre-deﬁned
and publicly available.
These processes will normally be expected to include:





a self-assessment or equivalent procedure by the subject of the quality
assurance process;
an external assessment by a group of experts, including, as appropriate, (a)
student member(s), and site visits as decided by the agency;
publication of a report, including any decisions, recommendations or other
formal outcomes;
a follow-up procedure to review actions taken by the subject of the quality
assurance process in the light of any recommendations contained in the
report.

Key elements
The standard is self-contained, with the guideline adding on the need to ensure
professional management and consistency and an appeals procedure.
Evidence
The standards (criteria), processes and procedures used by NVAO and the quality
assurance agencies in the Netherlands and Flanders are defined in the publicly
available assessment frameworks.
All types of assessments include some form of self-assessment, an external
assessment by a group of experts, including a student member, site visits and
the publication of a report including the recommendations of the panel and (in a
separate report) the decision by NVAO.
A follow-up procedures is applied in some cases, in others not. The issue of
follow-up is thoroughly described, analyses and concluded upon in section 4.1.6
and will therefore not be repeated in this section and not influence the conclusion
either.
For the programme assessments carried out by the quality assurance agencies
NVAO checks, as described in section 4.1.3, the quality of the assessment
procedure based on the information provided in the assessment report and if this
report leaves questions unanswered, NVAO requests additional investigation.
To handle appeals NVAO has set up appeals commissions for the Netherlands and
Flanders (competencies not given in the SER) that provides NVAO with advice on
which NVAO can make a decision. NVAO is pleased with this set up although it
reports further adjustments. If the decision on an internal appeal is negative, the
institution can pass the appeal on to independent ministerial body in either the
Netherlands or Flanders.
Analysis
The standards (criteria), processes and procedures used by NVAO and the quality
assurance agencies in the Netherlands and Flanders are pre-defined and publicly
36

available and the processes all include the elements expected by standard 3.7
except a follow-up procedure which is not applied in all cases.
Based on the numerous references to the assessment frameworks in the SER and
not least in many of the interviews during the site visit, the Panel is convinced
that NVAO and the quality assurance agencies pay careful attention to the
declared principles of the different forms of assessment and that NVAO does its
utmost to ensure both that the requirements and processes are managed
professionally and that the conclusions and decisions are reached in a consistent
manner. In relation to the issue of consistency the Panel wishes to point out
what was mentioned in section 4.2.8 about clusters as clustering of assessments
may also be beneficiary to consistency in assessments and decision making.
Moreover, it is clear to the panel that NVAO is aware of the fact that as it is
processing some 600 applications every year a constant attention to the
consistency of the decision making is needed.
The Panel relies upon the description of the appeals procedures. An appeals
commission is mainly there to secure the rights of an institution. The judgements
of the appeals commission may of course also tell something about the quality
assurance and professionalism of NVAO. However; the Panel is in line with ESG
and regards the appeals procedure as a part of ESG 3.7 (even if it is only given
in the guidelines and not in the standard itself).
Conclusion
NVAO fully complies with ENQA criterion 6 / ESG 3.7.

37

4.7 ENQA Criterion 7 / ESG 3.8: Accountability procedures
Standard
Agencies should have in place procedures for their own accountability.
These procedures are expected to include the following:
i.
A published policy for the assurance of the quality of the agency itself,
made available on its website;
ii.
Documentation which demonstrates that:







the agency‟s processes and results reﬂect its mission and goals of quality
assurance;
the agency has in place, and enforces, a no-conﬂict-of-interest mechanism in
the work of its external experts;
the agency has reliable mechanisms that ensure the quality of any activities
and material produced by subcontractors, if some or all of the elements in its
quality assurance procedure are subcontracted to other parties;
the agency has in place internal quality assurance procedures which include
an internal feedback mechanism (i.e. means to collect feedback from its own
staff and council/board); an internal reﬂection mechanism (i.e. means to
react to internal and external recommendations for improvement); and
an external feedback mechanism (i.e. means to collect feedback from experts
and reviewed institutions for future development) in order to inform and
underpin its own development and improvement.

iii. A mandatory cyclical external review of the agency‟s activities at least once
every ﬁve years which includes a report on its conformity with the
membership criteria of ENQA.
Key elements
This standard is understood in its full text, there are no explicit key words as
such in this very comprehensive formulation. However, the meaning of
“subcontractor” is not made clear by ENQA. In various dictionaries a
subcontractor is generally defined as an individual or in many cases a business
that signs a contract to perform a part or all of the obligations of another‟s
contract. In its decision making NVAO has to build upon quality assurance
performed by other agencies as regulated by law. As far as the Panel sees it,
ENQA‟s practise until now has not been to regards these as subcontractors,
neither do we.
Evidence
i.
A published policy for the assurance of the quality of the agency itself,
made available on its website
The “NVAO Quality Statement” of 2006 contains a section (section 4) presenting
among other items NVAOs “policy” for its own internal quality assurance and the
policy is also available on NVAO‟s website.
ii.

Documentation on specific issues

As described in section 4.1.4, all of the Dutch assessments frameworks include a
section describing how the independence (both factual and perceived) and also
how the “no-conflict of interest” of the panel members is secured. The
38

mechanisms comprise different kinds of documents that each of the panel
members and the secretary must sign. The Flemish frameworks do not contain
anything similar.
NVAO does not report that it uses subcontractors in the ordinary sense. Section
4.1 describes how NVAO ensures the quality of the activities and material
produced by the cooperating quality assurance agencies. To supplement these
descriptions, it can be mentioned here that NVAO in its SER describes that it
gives feedback to the quality assurance agencies and secretaries concerning the
quality of the reports they produced on a regular basis. At the interviews during
the site visit, the quality assurance agencies in both the Netherlands and
Flanders confirmed that NVAO provides formal and informal feedback to them.
The internal feedback and reflection mechanisms contain both written and verbal
activities. In the SER, NVAO highlights the periodic meeting with the Advisory
Council (three times a year), an extensive survey of the staff‟s satisfaction in
2011 and yearly staff meeting to discuss the realisations of previous objectives
and plans and formulate the objectives of NVAO‟s internal quality assurance.
External feedback mechanisms comprises for instance regular meetings with the
relevant ministerial representatives in the Netherlands and Flanders, the quality
assurance agencies and different written evaluation of the procedures for initial
accreditation
iii.

A mandatory cyclical external review of the agency‟s activities at least
once every ﬁve years

The documents governing the work of NVAO do not contain any requirements
concerning external reviews of NVAOs activities, but with the 2007 review and
the present 2012 review NVAO has so far been subject to an external evaluation
every five years.
Analysis
A published policy for the assurance of the quality of NVAO exists and is available
on the website of NVAO.
The assessment frameworks for the Netherlands clearly reveal that NVAO has in
place, and enforces, a no-conﬂict-of-interest mechanism in the work of the
panels. The reviews of VLHORA and VLIR leaves the Panel convinced that a noconﬂict-of-interest mechanism in the work of the panels is also in place and
enforced in the assessments in Flanders.
The quality assurance of the work of the quality assurance agencies is convincing
and so are the internal and external feedback mechanisms.
In general it is the impression from the SER and the site visit that NVAO has had
a strong focus on developing its internal quality assurance since the external
review in 2007 where critical points were raised. NVAO seems to have
strengthened internal quality assurance, specifically the level of dialogue with
stakeholders which was identified as a “weak” point in 2007, now seems to have
developed and have become a “strong” point. During the site visit, a good
39

dialogue with stakeholders was repeatedly mentioned by external stakeholders
as one of the characteristics of NVAO.
Conclusion
NVAO fully complies with ENQA criterion 7 / ESG 3.8.

40

4.8 ENQA criterion 8: Miscellaneous
i. The Agency pays careful attention to its declared principles at all times, and
ensures both that its requirements and processes are managed professionally
and that its judgements are reached in a consistent manner, even if the
judgements are formed by different groups.
ii. If the Agency makes formal quality assurance decisions, or conclusions which
have formal consequences, it should have an appeals procedure. The nature and
form of the appeals procedure should be determined in the light of the
constitution of the Agency.
iii. The Agency is willing to contribute actively to the aims of ENQA.
Key elements
“consistent manner”, ”appeals procedure”, “willing to contribute to the aims of
ENQA”
By “willing to contribute to the aims of ENQA” the Review Panel has looked upon
the following two dimensions:
1. Inwards: being a good role model by an active follow up of the
recommendations given by the review panel in the previous review in
2007
2. Outwards: actively promoting ENQA/ESG on the European Arena for
Higher Education
Evidence and analysis
Section 4.6 deals with the issues of “consistent manner” and “appeals
procedures” and the analysis and conclusions related to these issues will not be
repeated here but just form part of the frame of reference for the conclusion in
relation to NVAOs level of compliance with ENQA membership criterion 8.
The recommendations from the review in 2007 (September) was presented in
that report, chapter 1.5 Recommendations. Two years later, in a letter to the
ENQA-board, dated December 21, 2009, NVAO gives their response to the
recommendations that they consider relevant for the organisation. NVAO SER of
April 2012 contains a chapter 9 “Overview of actions taken since the previous
external review 2007”. Here NVAO deals both with recommendations regarding
NVAO and recommendations regarding the system. Without going in details, it
seems that NVAO has had a fairly good approach to this kind of follow up. The
recommendation on developing the relationship between NVAO and the quality
assessment agencies is dealt with on several occasions in the present report.
It is the Panel‟s impression from the SER as well as its general knowledge about
the international activities of NVAO, that NVAO has been actively engaged in the
European arena and in that work actively contributed to the aims of ENQA. Just
to mention: at present NVAO has an ENQA Board member, previously NVAO was
the initiator for ECA, NVAO has been hosting several international conferences
and a driving force behind international joint-projects. In short: NVAO is holding
a high international profile.
Conclusion
NVAO fully complies with ENQA criterion 8.
41

5

Conclusion and recommendations

5.1

Conclusion

In the light of the documentary and oral evidence considered by it, the Panel is
satisfied that, in the performance of its quality assessment functions, NVAO is in
compliance with the ENQA Membership Regulations and in substantial compliance
with the Standards and Guidelines for Quality Assurance in the European Higher
Education Area. The Panel therefore recommends to the Board of ENQA that
NVAO‟s Full Membership of ENQA be confirmed for a further period of five years.
The Panel concludes that NVAO fully complies with the ENQA membership criteria
2, 3, 4, 5, 6, 7 and 8 and substantially complies with the ENQA membership
criterion 1.

5.2

Recommendations

In relation to its assessment of NVAO‟s compliance with the ESG, the Panel
recommends that NVAO:
 makes the link between the NVAO standards and ESG Part 1 more explicit in
the assessment frameworks;
 refines the descriptions of the aims and objectives, ensures that they are
prominent in the frameworks and shows how the various elements of the
frameworks contribute to the aims and objectives;
 formulates explicit and public criteria about how it reaches its accreditation
decisions in relation to the programme assessments in all cases;
 establishes a clear procedure on how to handle cases where the conclusions
in the assessment report are not be accepted by NVAO;
 strengthens the predictability of the timeframe and efficiency of its decisionmaking process;
 further strengthens the focus on quality improvement and enhancement of
the HEI‟s;
 clarifies a) the purpose of every kind of report, b) the readership and c) the
needs of the various kinds of readers in order to enhance the readability;
 gives a high priority to a)the identification of the interested parties for
system-wide analyses and of their needs b) the definition of a realistic
schedule of system-wide analyses and c)the production of system-wide
analyses corresponding to the needs which were identified.

42

Annexes
A: Terms of Reference (ToR)

43

44

45

46

B: Members of the review Panel
Tove Blytt Holmen
Chair of the Panel
Director of Department of Quality Assurance, NOKUT (Norwegian Agency for
Quality Assurance in Education)
Member of Advisory Council, PKA, Poland
Member of appeals and complaints committee of ENQA
2005
2004
2002

Audit for the Irish Agency HETAC, member of Expert Panel
Institutional Accreditation and accreditation of Master in
Hydrogeology, Chairing the Expert Panel commissioned by
OAQ, Switzerland
International Comparative Evaluation of BSc programmes in
Agricultural Science. Commissioned by The Danish Evaluation
Institute. Chair of Panel.

Signe Ploug Hansen
Secretary of the Panel
Director of Methodology and coordinator of IQA (internal quality assurance) at
EVA (the Danish Evaluation Institute)
Member of the Advisory Board of the evaluation unit at Malmö Högskola,
Sweden.
2012
2007-2011
2002

External review of the National Agency for Higher Education in
Sweden. Panel member
Initiator of the ENQA Internal quality assurance group (the
IQA group) and leading member of its steering group
International Comparative Evaluation of BSc programmes in
Agricultural Science. Secretary of the panel.

Julian Tobias Hiller

Student member of the Panel
Student of Mechanical Engineering at the Leibniz University of Hannover,
Germany. Integrated Studies, now at a Masters level.
Member of the Accreditation Council of Germany
Member of the Board of the Agency for Quality Assurance Austria (AQ Austria)
Member of the Experts Pool for Quality Assurance of the European Students
Union
2012
2009

External Review of the Lithuanian Quality Assurance Agency
SKVC, conducted by ENQA
Training on the ESG by ESU and CoE in Strasbourg
47

2008 – 2010

Several Reviews (11) on Program level with several German
QA Agencies

Elie Milgrom

Member of the Panel
Emeritus professor, now consultant in professional development of teaching staff
(getting professors to know enough about learning and teaching in order to
improve the quality of higher education).
1972-2006: professor of computing science and engineering at the Université
Catholique de Louvain (Belgium)
2009-2012: member of an external review panel for VLIR: assessment of the
informatics/computer science university programs in Flanders
2011-2012: member of an external review panel for AEQES: assessment of the
informatics/computer science programs in the French-speaking part
of Belgium
2011-2012: member of an AEQES working group in charge of proposing new
criteria for quality assessment in the French-speaking part of
Belgium.
2009:
founding member of FA2L scrl (http://www.FA2L.be): a spin-off
company of Université Catholique de Louvain

Norma Ryan

Member of the Panel
Director of the Quality Promotion Unit, National University of Ireland Cork University College Cork, Ireland



Member of the Irish Universities Association Quality Committee
Member of the Irish Higher Education Quality Network
 Member of the Irish National Bologna Experts Team
2012: Representative of the EUA and Chair of the Steering Committee for the
European Quality Assurance Forum
2008: External review of Letterkenny Institute of Technology, Ireland. Panel
Member.
2010: External institutional review of Tallaght Institute of Technology, Ireland.
Panel Member.
2011: external institutional review of American College Dublin, Ireland. Panel
Member.
2011: Review of Academic Programmes, KTH, Stockholm, Sweden: external
reviewer
2011: Institutional review of St Augustine Catholic University, Tanzania. Panel
Secretary
2011: Institutional review of Weill Bugando Medical College, Mwanza, Tanzania.
Panel Secretary

48

C: Documents considered during the review
(Listed in order of importance for the assessment)
 Self-evaluation report NVAO 2012
 Self-evaluation report NVAO 2012 – Attachments
 Report of the committee for the review of the Accreditation Organization of
The Netherlands and Flanders (NVAO) (September 2007)
 The Dutch website of NVAO: http://www.nvao.net/
 The English website of NVAO: http://www.nvao.net/
 Letter from the Chairman of NVAO to the president of ENQA about The followup of NVAO on recommendations included in the external review review of
NVAO 2007 (December 21 2009)
 The NVAO Quality Statement of 2006 (handed out at site visit)
 Diagram illustrating the different forms of assessments within the new
accreditation system in the Netherlands and the relations between these
forms (handed out at site visit)
 Tables containing comparisons between the accreditation framework in
Flanders and the Netherlands and the ESG (handed out at site visit)
 Report of the External Review of the Flemish Council of University Colleges
(VLHORA) (October 2008)
 Report of the committee of the review of The VLIR Quality Assurance Unit
(May 2009)
 External review of the agency Quality Assurance of Universities in the
Netherlands (QANU) (October 2010)

49

Selfevaluation
report
NVAO 2012
April 16 th 2012

Selfevaluation
report
NVAO 2012
April 16 th 2012

Table of
Contents

Self evaluation report NVAO 2012
Introduction and aims of the external review

6

1 Summary and general appraisal

8

2 The accreditation systems in the Netherlands and Flanders

10

2.1 The ‘old system’
2.2 The evaluation of the ‘old system’
2.3 The ‘new system’

3 Analysis of Standards and Guidelines part 2
3.1 Use of internal quality assurance procedures
3.2 Development of external quality assurance processes
3.3 Criteria for decisions
3.4 Processes fit for purpose
3.5 Reporting
3.6 Follow-up procedures
3.7 Periodic reviews
3.8 System-wide analyses

4 Analysis of Standards and Guidelines part 3
4.1 Use of external quality assurance procedures for higher education
4.2 Official status
4.3 Activities
4.4 Resources
4.5 Mission statement
4.6 Independence
4.7 External quality assurance criteria and processes used by the agencies
4.8 Accountability procedures
4.9 Miscellaneous

11
11
12

14
15
16
17
18
19
20
22
22

24
25
25
25
27
28
29
29
30
33

ENQA criterion 8 I

33

ENQA criterion 8 II

33

ENQA criterion 8 III

33

5 Strengths and points of attention regarding NVAO’s operation
5.1 NVAO as organisation within the new accreditation system
5.2 NVAO’s compliance with the ESG resp. Parts 2 and 3

34
35
36

6 Introduction to the higher education systems in the Netherlands and Flanders 38
6.1 Higher education in the Netherlands
6.2 Recent developments in higher education in the Netherlands
6.3 Higher education in Flanders
6.4 Recent developments in higher education in Flanders
6.5 Overview of the Dutch and Flemish degree programme
6.6 Some key figures in higher education in the Netherlands and Flanders

39
39
39
40
41
41

Table of Contents

3

7 The external quality assurance in the Netherlands and Flanders

44
46

8 Operation of NVAO

48

8.1 Quantitative results
8.2 The assignment of NVAO
8.3 The treaty between the Netherlands and Flanders
8.4 International assignment of NVAO
8.5 Additional tasks

49
49
49
49
51

9 Overview of actions taken since the previous external review 2007

52

9.1 Follow up of recommendations regarding NVAO
9.2 Follow up of recommendations regarding the accreditation system

53
54

Attachments

4

42

7.1 Self-evaluation
7.2 Conclusion

Se lf -eval u at i o n re p or t NVAO 2012

Table of Contents

5

Introduction
and aims of
the external
review

NVAO, the Accreditation Organisation
for the Netherlands and Flanders
[Nederlands-Vlaamse Accreditatie­
organisatie] was formally established by
the Dutch and Flemish governments as a
binational organisation on 1 February
2005. The Netherlands and Flanders decided
to found a binational organisation to
express the cross-border cooperation on
external quality assurance in the spirit of
the Bologna Declaration. In this regard it is
a significant advantage that the Netherlands
and Flanders share the Dutch language.
The main objectives of the review scheduled for 2012 are:
1. Renewal of NVAO’s full membership of ENQA, by establishing
the agency’s compliance with the European Standard and
Guidelines 2009 (ESG), parts II and III, including the terms
of reference of ENQA (review A), as well as the agency’s
compliance with the equivalent ESG requirements of EQAR,
leading to the extension of NVAO’s EQAR-registration.
2. A first reflection on the new accreditation system in the
Netherlands that came into force on 1 January 2011,
which constitutes an additional request of NVAO to ENQA
and the review committee;
3. To contribute to NVAO’s internal quality monitoring with
regards to its organisation and its operation.
Accordingly, NVAO evaluates itself in this self-evaluation
report from three angles: 1. with reference to the ESG, 2. from
the point of view of introduction of the new accreditation
system in the Netherlands and, 3. with regards to its functioning organisation. This three-sided evaluation is distributed
among the chapters of the report in the following manner:
> Chapter 1 (partly), 3, 4 and 5 (partly) deal mainly with
­compliance with ESG, respectively Parts 2 and 3;
> Chapter 7 deals primarily with the new accreditation system
in the Netherlands;
> Chapter 1 and 5 describe and evaluate NVAO and its role in
the new accreditation system.
The remaining chapters 2, 8 and 9 describe the systems of
higher education in Netherlands and Flanders and summarise
NVAO’s activities, referring to the more evaluating chapters
mentioned above.
The following introductory remarks are meant to put the three
main objectives of this self-evaluation report in perspective:
1. NVAO is subject to a five-yearly external review in order to
renew its full membership of the European Association for

Quality Assurance in Higher Education (ENQA). In 2007,
NVAO successfully passed a first review of ENQA and
became full ENQA-member. The present review of 2012 is
the second of its kind. The previous review was initiated
on a binational request by the ministers of Education in
Flanders and the Netherlands. Since November 2008,
NVAO has been registered in the European Quality
­Assurance Register for Higher Education (EQAR), the
register for quality assurance agencies (QAA’s) that review
institutions or programmes in higher education on a
regular basis and substantially comply with the ESG.
2. The previous review in 2007 had as secondary objectives
the evaluation of the – then- new accreditation systems in
the Netherlands and Flanders and an assessment of their
differences. Because the new system in the Netherlands
has only been active from 2010 onwards, the present review
only entails a first reflection of the new regime. Notwithstanding the different objectives and conditions of the two
reviews, chapter 9 shortly reports the measures taken in
response to the recommendations in the 2007 review.
3. Internal quality assurance is an important issue in NVAO’s
day to day operation, which consists of processing some
600 applications every year. This task calls for constant
attention to the consistency of the decision making and
continuous improvement of processes.
NVAO sees the external review as an extra incentive for
its own quality assurance and for the improvement of its
internal and external operational processes.
NVAO is a binational institution. The differences between the
Dutch and Flemish systems in higher education and external
quality assurance are relatively small, but nevertheless
considerable in a few aspects. This self-evaluation report was
written from the perspective of the many similarities between
the systems of higher education and accreditation in both
countries. Only when necessary for a proper understanding,
it refers to the differences between these systems.
One important difference regards the timetable for the
renewal of the accreditation regimes: while the new system
has been in force in the Netherlands since 1 January 2011
and is being further optimized at the present moment, the
new accreditation system in Flanders is in its final design
stage and will be in force from the academic year 2013-2014.
In setting up its internal quality insurance system, NVAO identified its most important internal and external stakeholders
and organised regular consultations with them, individually, or
through umbrella organisations. An inventory of the results of
these consultations was drafted during the preparation of this
self-evaluation report, and is included in the document. The
NVAO also consulted various stakeholder groups on different
occasions on the present report. Eventually, the report was
approved by the General Board of the NVAO on 16 April 2012.
Introduction and aims of the external review

7

Chapter
1
Summary
and general
appraisal

On 1 February 2012 NVAO officially existed seven years.
Since its foundation NVAO has carried out the primary
­assignment as defined in the Treaty between the Dutch and
Flemish Education ministers of 2003, which is described as:
The primary assignment of NVAO is to accredit existing
programmes in higher education (accreditation) and to assess
new programmes (initial accreditation) in the Netherlands and
Flanders. NVAO can be requested to carry out additional tasks
by the Ministers of higher education in both countries if
these assignments support or supplement NVAO’s primary
assignment. Secondly NVAO is assigned to play an important
role in internationalisation.
In extension of this primary assignment NVAO formulated its
first mission and strategy in 2006 and renewed it in 2012.
Since 2005 NVAO has completed a large number of applications for (initial) accreditation in the Netherlands (NL) and
Flanders (FL), parallel to a substantial international task
in Europe and world-wide. Several additional tasks in the
Netherlands and Flanders have also been taken up. Up to
1 January 2012 NVAO decided (or in some cases advised)
on more than 4.000 applications of all kind.
The questions this review sets out to answer are:
> whether NVAO, with this substantial effort, successfully

carried out its primary assignment and;
> whether NVAO operates effectively with regard to its main

stakeholders, being both governments, the institutions in
higher education and the students in higher education in
the Netherlands and Flanders.
For this purpose NVAO’s Strategic Policy Statement
­emphasizes the importance of:
> both robust quality assurance and quality enhancement
based on expert and objective judgements;
> the further development of a quality culture in higher
­education and;
> reinforcement of the Dutch-Flemish cooperation, as well
as cooperation in the European higher education area.
What is the current position of NVAO regarding these
­objectives and challenges?
In the past years NVAO has built a strong and respected
­position in higher education in the Netherlands and Flanders
and Europe.
> NVAO plays a leading, independent and critical role in quality
assurance in higher education in the Netherlands and
­Flanders and was the initiator of the development of the
new accreditation system in the Netherlands and Flanders;

> NVAO proved to be a very flexible organisation capable of

completing a large number of additional tasks on request of
the Ministers of Education in the Netherlands and Flanders;
> NVAO communicates and cooperates with its main stakeholders in an open and transparent manner, with the aim of
contributing in the best possible way to the further development of a quality culture in higher education and thus to the
quality of higher education in the Netherlands and Flanders;
> Internationally, NVAO contributes considerably in Europe
through ENQA, ECA, EQAF and European projects, as well as
worldwide, through its involvement in INQAAHE.
Besides these achieved results there are several issues
NVAO should pay attention to in the next years to improve its
operation and the operation of the accreditation system in
the Netherlands and Flanders.
> The administrative burden of the accreditation system
should be reduced by focusing on the essence of quality
on institutional and study programme level;
> The large number of applications necessitates permanent
attention to consistency in decision making;
> Additional tasks should be planned better and formal
­deadlines should be kept more rigorously;
> NVAO should remain self-critical and self-reflective with
regards to (systems of) quality assurance and its own
­functioning, by staying first and foremost focused on
quality and its enhancement;
> The binational nature of NVAO should contribute stronger
to Dutch-Flemish cooperation in higher education;
> International projects and activities should be more
­beneficial for institutions in higher education and/or
NVAO as a whole.
According to its own assessment, NVAO fully complies with
the ESG of ENQA, even if there are several issues that require
its permanent attention. A major challenge for NVAO is the
successful implementation of the new accreditation systems
in the Netherlands and Flanders and to make these systems
converge as much as possible. A first evaluation of the new
system by the review committee is a very useful step in this
process.
Another permanent issue is the consistency of NVAO’s internal
processes. In the past years NVAO focused in its internal
quality assurance on a more effective steering of the quality of
its internal processes. The present external review by ENQA
will also be used to improve NVAO’s processes.

Summary and general appraisal

9

Chapter
2
The
accreditation
systems in
the
Netherlands
and Flanders

This self-evaluation report refers to the
years 2007-2011, a period which is
characterised by considerable changes
in the accreditation systems used in
Flanders and the Netherlands. A brief
overview of these systems is given here to
offer the reader a framework for the next
two chapters, which describe how NVAO
meets the European Standards and
Guidelines within the accreditation
systems of the Netherlands and Flanders.
2.1  ⁄ The ‘old system’
In the Netherlands, the first accreditation system was
­operational from 2003 to 2010, in Flanders from 2005 to 2012.
So the Netherlands has already made the transition to the
new system, and Flanders will follow in 2013. To avoid any
ambiguity, the current Flemish system will already be referred
to in the past tense, although it is still operational in 2012.
The previous system was layered: NVAO had to base its
­accreditation decisions on assessment reports compiled
by quality assurance agencies. In the Netherlands these
were free-market agencies, known as VBIs, which had to be
assessed annually by NVAO and when appraised positively,
were entered on a public list. Higher education institutions
were then free to use their services. In Flanders, on the other
hand, the umbrellas of universities (VLIR) and university
colleges (VLHORA) were designated by law to act as quality
assurance agencies. In Flanders programmes had to be
assessed in clusters, in the Netherlands there was no such
requirement and universities of applied sciences and private
institutions therefore chose not to do so; the universities,
however, did.
The VBIs and quality assurance agencies were authorised
to compose panels (following rough guidelines set by NVAO)
and followed the accreditation framework as defined by law.
The framework comprised six themes:
> the programme’s objectives
> curriculum
> staff
> facilities
> internal quality assurance
> results.

These six themes were subdivided into 21 standards that were
assessed on a four-point scale (satisfactory, unsatisfactory,
good and excellent). For assessing at theme level a dichotomous scale was used (satisfactory or unsatisfactory), as was
the case for the final assessment. So NVAO could not grant
any conditional accreditations. In Flanders, in the event the
final conclusion reached was ‘unsatisfactory’, a recovery
period could be granted by the Flemish Government whereas
in the Netherlands the recovery option was a theoretic
­possibility, which in practice, however, turned out not to be
a realistic one.
For new programmes, the panels were composed by NVAO
(with some exceptions in the Netherlands). Virtually the
same assessment framework was applied, with the exception,
of course, for the theme results, which was replaced with that
of ‘conditions for (financial) continuity’.

2.2  ⁄ The evaluation of the ‘old system’
In Flanders and the Netherlands a great deal of attention has
been paid to the functioning of the accreditation system. Based
in part on evaluations and discussions with stakeholders, NVAO
carried out a self-evaluation in 2008 to prepare the ground for
the design of a new system, for which the preparations started
that year. Positive elements of the previous system were:
> the accreditation system was introduced smoothly and was
fit for purpose: all programmes were assessed as planned
during the first phase of the system;
> the system has laid a quality foundation for the sectors of
higher education in Flanders and the Netherlands. To put it
bluntly: all the wrecks have been taken off the road;
> the attention for quality assurance has strongly increased;
> internationally, the system is understood and seen as a
solid one.
These positive points are offset by the following negative ones
that emerged from the evaluation:
> the fear for negative assessments (and subsequent loss of
prestige or closure of a programme) has led to programmes
and panels giving and asking (too) much information;
> the focus has been too much on procedures and processes,
and too little on the content of the programmes and the
achieved learning outcomes;
> teaching staff had difficulty seeing themselves as part of the
system;
> the large amount of available public information (assessment reports, accreditation decisions) has paradoxically led
to ‘an inaccessibility of information’;
> In the Netherlands, in the absence of an obligation to assess
in clusters, the comparative quality of the assessments has
decreased.
The accreditation systems in the Netherlands and Flanders

11

This analysis was broadly shared by and with stakeholders
and resulted in Flemish and Dutch ministers, in charge of
higher education, commissioning the design of a new
system that was to largely meet the points of criticism as
described above. The ministers put great emphasis on a
necessary reduction of the bureaucratic burden, without
any concessions being made to the international legitimacy
of the system.

2.3  ⁄ The ‘new system’
In the new system, which was introduced in the Netherlands
on 1 January 2011 and in the design that is currently being
discussed in Flanders and which is expected to make it
through the Flemish parliament before the summer,
programme accreditation remains at the heart of the system.
In both countries, however, an institutional audit is added
to the system with a view to realising an important administrative saving at programme level. The institutional audit
concerns the assessment of the institution’s quality
­assurance system and ultimately leads to a judgement on
the question whether an institution is in control of the quality
of its education. In the Netherlands, the outcome of the audits
will have immediate consequences, whereas Flanders regards
the audits, which will be carried out in 2015 and 2016, as a
baseline measurement that is obligatory for all institutions.
In the Netherlands the audit comprises five standards:
> vision of the institution regarding the desired quality of its
education;
> the policy developed and the means used to realise this
vision;
> a management information system;
> improvement capacity and measures;
> structure and culture of the organisation and its decisionmaking process.
In Flanders, the discussion about the content of the institutional audits is still ongoing, but both systems will broadly use
the same standards. The wide Flemish scope of the institutional review, ranging from the institution’s policy to internal
quality assurance by focusing on the quality of education,
can also be found in the first Dutch institutional reports.
In both systems the audits can have three possible outcomes:
satisfactory, conditionally satisfactory and unsatisfactory.
In the Netherlands, a final conclusion that is either
­‘satisfactory’ or ‘conditionally satisfactory’ means that
the assessment of the programmes follows the framework
of the so-called ‘limited programme assessment’.
This framework, which in Flanders will be applied to all
programme assessments, has only three standards:

12

S e lf -eval u at i o n re p or t NVAO 2012

> the programme’s objectives and qualifications;
> learning content and learning environment;
> assessment procedures, examination and achieved

learning outcomes.
If an institution in the Netherlands fails the institutional audit
or if it does not want to participate in it for motivating reasons
(for example, the limited size of an institution), the
programmes will be assessed on the basis of the framework
of the ‘extensive programme assessment’. This framework
strongly resembles the ‘old framework’ as described above.
In Flanders, the institutional audit will be made obligatory
for all official institutions anyway, so that the programme
assessment will always be a limited one. For non-official
institutions, the standard of institutional internal quality
assurance will be added to the programme assessment.
A number of changes have been made to the new system
in response to a number of negative experiences with the
previous one:
> NVAO is involved in the composition of the panels beforehand by means of explicit approval (the Netherlands) or
advice (Flanders);
> the Netherlands, too, now has an effective recovery period;
> the standards of the programme assessments are judged on
a four-point scale; in the Netherlands, this is also the case
for the panel’s final conclusion regarding the overall quality
of the programme. In Flanders, the discussion on this matter
is still ongoing;
> the number of documents that programmes are required
to make available has been reduced considerably;
> the reports have to contain a public-friendly summary,
which NVAO can use to provide information to the public;
> the secretaries of the programme panels will receive
training;
> in the Netherlands, assessments in clusters will become
obligatory as of 2013.
The experiences with the new system are, of course, still
limited. Chapter 7 of this SER contains a first and provisional
evaluation of the functioning of the new Dutch system.

The accreditation systems in the Netherlands and Flanders

13

Chapter
3
Analysis of
Standards
and
Guidelines
part 2

The review panel is requested to assess
on the basis of the following chapters
whether and to what extent NVAO
complies with the ESG. The assessment
concerns NVAO’s compliance regarding
the individual standards and guidelines of
ESG, as well as an overall judgement.

Part 1 of the ESG refers to the following procedures in the
institutions for higher education:
(1) Policy and procedures for quality assurance
(2) Approval, monitoring and periodic review of programmes
and awards
(3) Assessment of students
(4) Quality assurance of teaching staff
(5) Learning resources and student support
(6) Information systems
(7) Public information.

3.1  ⁄ Use of internal quality assurance
procedures

All these quality aspects are included in the NVAO frameworks
at institutional and/or programme level. Table 1 shows which
standards of the frameworks cover these respective aspects.
The frameworks LPA and IA complement each other, which is
described in chapter 2 and will be explained further in
chapter 7. As the standards of the former Dutch accreditation
framework are almost similar to those of the present EPA-NL,
and as the standards in the present Flemish framework are
still those at programme level, columns IV and V also cover the
standards used in the period 2004-2010 – the first phase5 > of
the accreditation system in the Netherlands.
A major building block of the new accreditation system is the
way LPA and IA complement each other. Process oriented
aspects that programmes have in common or are organised
at a higher level, such as facilities and systems of internal
quality assurance, are (NL) or will be (FL) assessed once in
the institutional audits and not anymore repeatedly at
programme level. This should lead to a 25% reduction of the
administrative burden involved in programme assessments.

External quality assurance procedures should take into account
the effectiveness of the internal quality assurance processes
described in Part 1 of the European Standards and Guidelines.

Facts and findings
The accreditation system in the Netherlands is a combination
of an institutional audit (IA) and programme accreditation
(PA), based on limited (LPA) or extended programme assessment (EPA). The current system in Flanders consists of
programme accreditation, based on extended programme
assessment, but a new accreditation system, containing both
IA and LPA, is under construction and will be implemented in
the academic year 2013/2014. An extended description of
(the evolution of) the Dutch and Flemish accreditation system
is included in chapter 7.
Table 1

I

II Framework IA – NL1 >

III Framework LPA - NL2 >

IV Framework EPA - NL3 >

V Framework – PA - FL4 >

Standard 4

-

13, 14 and 15

5

	Policy and procedures for quality
assurance

	Approval, monitoring and periodic review
of programmes and awards

4

2

13, 14 and 15

5

	Assessment of students

-

2

16

2, Assessment

	Quality assurance of teaching staff
	Learning resources and student support
	Information systems
	Public information

2

2

8

3, Quality of Staff

2

2

11 and 12

4

2

-

12

4

2

-

12

4

and testing

1> F
 ramework Institutional Audit (IA)– the Netherlands, attachment 11.
2> Framework Limited Assessment of Programmes (LPA) – the Netherlands,
attachment 11.
3> Framework Extended Asssessment of Programmes (EPA) – the Netherlands,
attachment 11.
4> Framework Programme Accreditation (PA) – Flanders, attachment 11.

5> T
 he initial phase spans six years in the Netherlands and eight years in
Flanders, during which all programmes will go through an assessment
under the new system.

Analysis of Standards and Guidelines part 2

15

Self-evaluation

Facts and findings

The NVAO frameworks cover all mentioned aspects of internal
quality assurance of institutions in higher education. The new
frameworks encourage the development of quality assurance
and of a quality culture within the entire institution. The IA framework focuses on the internal quality assurance (IQA) processes
at institutional level and the processes related to individual
programmes offered by the institution. The LPA framework
focuses on the quality of the content and the achieved
learning outcomes of the programme. Whereas the previous
frameworks employed a more technical approach, the new
system allows for a more thorough discussion about systematic quality assurance and quality culture at institutional level
and the quality of content and learning in individual programmes.
Most higher education institutions in the Netherlands
applied for IA in 2011 and thus will be able to benefit from
the reduction in administrative burden in LPA’s in the coming
years. Results from the first IA’s indicate that the new framework encourages the development of quality assurance at an
institutional level. However, smaller institutions are not likely
to benefit from the IA and will therefore continue to operate
under the regime of the EPA.
In Flanders, LPA and IA will be compulsory elements in the
new accreditation system for all institutions. EPA will not be
applied any longer.

Both the previous and the current frameworks were designed
in consultation with representatives of institutions and
programmes, quality assurance agencies and other assessment experts. The aims and objectives of the systems were
discussed with stakeholder groups and the frameworks were
tested before the system came into force. The description of
the drafting of the current frameworks in 2008 and 2009 in
chapter 7 outlines this process. The frameworks were
discussed in parliament and are published by the government
and NVAO. Additional information on the procedures is
provided in guidelines. The frameworks were made available
online on NVAO’s website well before the start of the systems
in both countries (www.nvao.net).

Judgement

Self-evaluation

NVAO complies with the standard.

Initially, the institutions of higher education expressed their
preference for a system of institutional accreditation only to
replace the previous programme accreditation. A proposal
to this extent was rejected by NVAO and the Dutch parliament.
­Eventually, the institutions accepted the new frameworks as
a challenging innovation and actively took part in the development of the new system by participating in the pilots. This
stakeholder involvement can be seen as a good practice.
NVAO is convinced that the new system provides an effective
compromise between firm quality assurance on programme
level through programme accreditation, on the one hand,
and reduction of the administrative burden through the
­institutional audit, on the other hand. The system allows for
the enhancement, profiling6 > , and differentiation of individual
programmes by applying for distinctive (quality) features.
The profiling is further enhanced by the introduction of the
use of a four-point scale for quality judgements on the standards of the framework (NL-FL) and in the overall conclusion.

Point(s) of attention
> The procedure for the IA consists of an overall assessment

of the procedures within an institution, combined with
several audit trails. These audit trails monitor the way
­institutional measures and regulations are perceived and
acted upon at programme level (top down) and how the
­institutional IQA system operates in case of concrete quality
problems (bottom up). The first accomplished audits show
encouraging results in this respect: institutions realise that
they can only benefit from the new system when their
internal systems are coherent and interactive at both levels.
NVAO should monitor whether this interaction is demonstrated convincingly in the IA.

3.2  ⁄ Development of external quality
assurance processes
The aims and objectives of quality assurance processes
should be determined before the processes themselves are
developed, by all those responsible (including higher education
institutions) and should be published with a description of
the procedures to be used
16

S e lf -eval u at i o n re p or t NVAO 2012

The stakeholders and NVAO refrained from drawing up overly
detailed or excessively normative or prescriptive frameworks.
The new frameworks were conceived to accommodate the
diversity of higher education programmes in the Netherlands
and Flanders, and to respect the autonomy of institutions.
The pilots of 2009 showed that stakeholders had a favourable
attitude towards the new system. The development, implementation and operation of the new accreditation system is
being monitored by resonance groups representing the important stakeholders in both Flanders and the Netherlands.

6> P
 rofiling is related to the mission, strategy and characteristics of an institution or programme. Differentiation is related to the level and quality of a
programme.

Judgement
NVAO complies with the standard.

Point(s) of attention
> It is vital that NVAO remains in close contact with the

­ takeholders, especially the representing bodies of the
s
higher education institutions, to evaluate the experiences
and to monitor whether the system functions as intended,
especially in the context of announced, new regulations by
the Dutch government. Also in Flanders this is essential
since the decree specifies two evaluations of the new
accreditation system while being implemented. This
dialogue should be open and done in full independence.

3.3  ⁄ Criteria for decisions
Any formal decisions made as a result of an external quality
assurance activity should be based on explicit published
criteria that are applied consistently.
The frameworks for EPA, IPA and IA use clearly distinguishable
standards and a four-point scale (excellent, good, sufficient,
insufficient), both for the assessment of the programme as a
whole and for the separate standards. The underlying criteria
and decision rules for each of these judgements are clearly
outlined in the frameworks and explained in guidelines
attached to the frameworks. The introduction of the recovery
period in the Netherlands gives panels more leeway to pass a
negative judgement when necessary, without causing a forced
shutdown of a programme.
In the current Flemish frameworks the four-points scale is
only applied to the sub-standards; here also the four- point
scale is explained in guidelines. The recovery period has been
part of the Flemish accreditation system since the start in 2005.
The NVAO has put in place the following measures to safeguard the consistent application of the criteria during the
assessment of applications for accreditation:
> Approval of panels by the NVAO: in the Netherlands NVAO
approves panels according to the criteria formulated in the
requirements for assessment panels7 > and the accreditation frameworks. In Flanders panels are approved by the
Recognition committee and in the near future by the
­evaluation body VLUHR8 > after a formal advise by NVAO,
7> G
 uideline ‘Requirements for the composition of panels’ of August 2011,
[Leidraad Eisen aan de panelsamenstelling, Augustus 2011].
8> Manual ‘Site visits VLIR and VLHORA’, [Handleiding onderwijsvisitaties VLIR
en VLHORA, Brussel, September 2008]. Since 2011 the evaluation bodies of
VLIR and VLHORA merged into VLUHR; the only quality assurance agency in
Flemish higher education. VLIR and VLHORA are still used separately to represent respectively the universities and the university colleges in Flanders.

using the protocol of VLUHR. A panel has to be approved
before the assessment can take place.
> Training of secretaries to assessment panels in the
­Netherlands9 > by NVAO. In Flanders the secretaries will
be trained by VLUHR in consultation with NVAO.
> A clear procedure for the review and assessment of students’
final assignments1 0 > is used by the assessment panels in
the Netherlands. A comparable procedure, included in the
mentioned protocol of VLUHR, is applied in Flanders.
> The application of domain (cluster) assessments in the entire
higher education sector in the Netherlands and Flanders.
Recently, the ministry of education has ruled that the Dutch
professional higher education sector should also adopt the
cluster approach in their assessments.
After completion of the assessment report and acceptance
by the institution,1 1 > this can be submitted by the institution
to the NVAO, together with an application for accreditation.
NVAO has put in place several measures to ensure the
consistency of the evaluation of assessment reports.
> The use of detailed manuals for the evaluation of reports,
which are updated on a regular basis.
> Frequent staff meetings where issues regarding the
­evaluation of assessment reports are discussed.
> Each application is evaluated by a policy advisor and a
responsible board member. If the evaluation necessitates
this, the NVAO asks other staff members or board members
for a second opinion. A final decision on an application is
taken by all board members in a joint session on the basis
of a detailed propsosal.
> The applicant institution may be asked to provide additional
information, to ensure that each application is fully
­documented according to the criteria outlined in the
­frameworks. Sometimes NVAO applies the instrument of
additional assessment (‘Verificatie’ in the Netherlands,
‘Aanvullende beoordeling’ in Flanders).

Self-evaluation
The new frameworks contain less standards and criteria than
the previous frameworks. In the evaluation of the accreditation system in 2008 all stakeholders mentioned the use of
9> N
 VAO trained about 185 secretaries in the Netherlands in 2011 and 2012.
After successful completion of the training a secretary is certified for one
year and his performance is monitored by NVAO based on the panel
reports he produces. VLUHR trains the secretaries in Flanders as they are
usually employees of this organisation.
1 0 > ‘NVAO guideline for the purpose of the assessment of graduation projects
during site visits’ of July 2011 [Richtlijn NVAO beoordeling afstudeerwerken
door panels tijdens het visitatieproces voor accreditatie en (indien van
toepassing) een toets nieuwe opleiding, 8 Juli 2011].
1 1 > In Flanders the assessment report is published on the website of VLUHR
before the institution applies for accreditation by NVAO.

Analysis of Standards and Guidelines part 2

17

open frameworks as a strong element. Open frameworks
enable programmes and institutions to demonstrate required
quality in their own specific way and allow for profiling and
differentiation.

> To guarantee that all accredited programmes have the

required quality;
> To assess the overall quality of a study programme on a

­four-point scale;
> To contribute to the enhancement, profiling and differentia-

It is the opinion of NVAO that the accreditation system has
improved in the second phase, especially regarding the
consistent application of criteria. The introduction of the
recovery period, the training of secretaries and clear
­procedures for the assessment of students’ assignments
have reinforced the system.
Another important improvement is the appointment of panels.
In the near future panels in the Netherlands will be appointed
and reimbursed by NVAO. The institutions will then reimburse
NVAO. NVAO considers this change as a meaningful reinforcement of the independency and authority of panels. In the
previous accreditation system the quality assurance agencies
operated on request of the institutions and this relationship
has been indicated as a weak point in the system.
The appointment of panels in Flanders is still executed by
VLUHR and the recognition committee. All stakeholders agree
on this procedure, partly because the former QAA’s from VLIR
and VLHORA are compliant with the ESG from ENQA and are
EQAR-registered. In the new accreditation framework VLUHR
will consult NVAO on the composition of panels before taking
a decision.

Judgement
NVAO complies with the standard.

Point(s) of attention
> Open frameworks contain less detailed criteria and its proper

application depends for a major part on the expertise of panels
and the way they apply the criteria. NVAO should monitor
whether panels apply the criteria in a consistent manner.
> The application of the four-point scale should be monitored
thoroughly in order to achieve consistent use and a
­well-balanced score motivation by the panels and NVAO.

3.4  ⁄ Processes fit for purpose
All external quality assurance processes should be designed
specifically to ensure their fitness to achieve the aims and
objectives set for them.

Facts and findings
The Netherlands decided on putting in place an external
quality assurance system (EQA) based on peer and expert
review and open frameworks as the best system to achieve
three main objectives of EQA:
18

S e lf -eval u at i o n re p or t NVAO 2012

tion of higher education programmes.
As mentioned, in Flanders the new accreditation system is
still under construction. Both in the current and the planned
system peer and expert review form the core of the system,
based on open standards. In Flanders the judgement on the
standards will be verified in technical annexes. It is not
decided yet whether the four-point scale will be applied on
the study programme level, next to each standard.
In the (initial) accreditation procedure roughly three steps can
be distinguished.
> The writing and submission of the self-evaluation report
(in the case of accreditation) or the information file (in the
case of initial accreditation) by the institution. The writing of
the self-evaluation report is guided by the protocols of the
QAA’s, which are based upon the NVAO frameworks.
> The composition of the assessment panel, the actual sitevisit and the assessment by the panel. A panel consists of at
least four experts including a student member. The expertise
of the panel should include knowledge of the discipline,
of the professional field, of education in the domain, of the
international dimension, and experience in reviewing or
auditing.
Student members are trained by NVAO for the procedures
for initial accreditation and institutional audit. Panel
members participating in the accreditation procedure are
instructed by the quality assurance agencies
> The third step is the decision-making process by NVAO.
If NVAO takes a negative accreditation decision, the
programme may be granted a recovery period or the
­application may be withdrawn by the institution.
The voluntary institutional audit is a new and second element
in the new accreditation system. It comprises an assessment
by an audit committee of whether the institution is ‘in control’
of the quality and the achieved learning outcomes of its
programmes. In order to answer this question the audit
committee assesses the adequacy of the institutional
strategy, its implementation through policy and the educational processes on institutional level. Institutions that pass
the institutional audit successfully are entitled to submit a
limited programme assessment (LPA). Institutions that don’t
opt for an institutional audit or have failed the audit submit an
extensive programme assessment (EPA). Flanders still has to
decide about some elements of this system, but it is clear
already that the institutional audit and LPA will be compulsory.

An important objective of the new accreditation system is the
reduction of the bureaucratic burden on institutions caused
by the (programme) assessments. The application of IA and
LPA is supposed to save time and energy in comparison with
EPA. However, in the starting phase of the new system the
combined process of institutional audit and programme
assessments might cause additional work. The NVAO has
committed itself to a reduction of 25% of the administrative
burden of programme accreditation. This was an important
condition for the stakeholders to accept the new system. For
this reason NVAO will continue to monitor the bureaucratic
loads the new accreditation system entails. It can do so by
limiting the extent of the self-evaluation report following the
decrease of the numbers of standards, by negotiating with the
QAA’s on shortening the duration of site-visits specified in
their protocols, and by making sure that the reports are clear,
self-evident and to the point, so that additional requests for
clarification or additions during the accreditation process can
be prevented or limited.

The new accreditation system operates in the Netherlands
since 1 January 2011. The first impressions and experiences
are favourable. The institutional audit causes staff to reflect
more intensively on internal quality assurance at an institutional level. The assessed institutions consider the audit a
valuable learning experience that will positively affect the
quality of the programmes they offer.
As a result of the introduction of LPA and EPA the first panel
reports prove to be more informative and transparent than
previous reports, and enable NVAO to lessen its workload
during the decision making on programme accreditations.
The current system in Flanders operates to the considerable
satisfaction of the stakeholders. VLUHR has an important role
in the system as it organises the programme assessments
and the composition of the panels.

Judgement
NVAO complies with the standard.

A new element of the accreditation system in the Netherlands
is the wider application of distinctive features.
Now institutions can apply for distinctive features at
programme and institutional level in order to enlarge the
possibilities of profiling and differentiation, which is a broadly
accepted objective in Dutch higher education. .
Up till 1 January 2012 NVAO awarded 23 distinctive features
and 19 applications for distinctive quality (the score Excellent
on a standard of the framework) in the Netherlands.
In Flanders the distinctive quality feature is part of the system
as well. Up till 1 January 2012 NVAO awarded 13 applications
for a distinctive quality feature in Flanders.
NVAO carries out the procedure for initial accreditation
without involving a QAA in the Netherlands and in Flanders,
including the evaluation of the assessment process.
­Evaluation results show that institutions and panels are
quite satisfied with the process. However, institutions
indicate that deadlines should be respected better.

Self-evaluation
The current accreditation system heralds a new approach.
The previous system focused on establishing basic quality of
programmes as required for accreditation, whereas the new
system challenges programmes and institutions to set higher
targets. New elements were introduced in Dutch law regarding
differentiation and profiling. NVAO is in favour of these
changes as they challenge institutions and programmes to
improve quality by defining a more explicit profile.

Point(s) of attention
> The implementation of the new frameworks implies that

institutions and programmes have to get used to the new
information files and self-evaluation reports. Therefore it
may take longer before the 25% reduction of administrative
burden will be achieved. All stakeholders should be aware of
that.
> During the coming years, NVAO will have to carefully monitor
the composition and the operation of panels, as well as the
operation of secretaries, as the success of system depends
for a large part on the quality on their work.
> The training of panel members. NVAO trains the chairpersons of panels of the institutional audit, but the panels
members themselves are not trained by NVAO. Quality
assurance agencies instruct panel members preceding the
review. NVAO should consider whether a training for panel
members is a realistic objective.
> In Flanders, the cooperation with the VLUHR concerning the
quality of panels and reports should be intensified, as it is
subject of a mid-term evaluation of the new accreditation
system in 2017-2018.

3.5  ⁄ Reporting
Reports should be published and should be written in a style
which is clear and readily accessible to its intended readership.
Any decisions, commendations or recommendations contained
in reports should be easy for a reader to find.

Analysis of Standards and Guidelines part 2

19

Facts and findings

Self-evaluation

The assessment reports and decision documents have a fixed
and structured format to increase readability. In the last two
years NVAO trained secretaries in NL (partly employees of the
quality assurance agencies) in report writing. Special attention has been given to the newly introduced executive
summary of the report. The new summary has to be very
concise in order to inform an interested reader at a glance
about the most relevant characteristics of the programme.
Additionally, every report contains a scoring table and a separate paragraph with recommendations. The summary and the
scoring table are integral part of the decision document.
Regarding the readability of reports NVAO started in 2012 an
evaluation project to investigate whether the new reports
respond well to the needs of the intended readers.

NVAO monitored the quality of reports during the previous
accreditation period. With the introduction of the new system
in the Netherlands and Flanders NVAO agreed with the quality
assurance agencies to improve the readability of the reports.
The first results in the Netherlands are promising. It is
expected that the main findings and conclusions of a report
will be read better by students, employers and other involved
groups.
NVAO’s website contains all accredited programmes in the
Netherlands and Flanders, but the site does not reach enough
visitors. It is the opinion of NVAO that especially (future)
students, their parents and employers can profit from
adequate information about the quality of study programmes.
This information can help future students to choose a study
programme and employers, for instance, in case of deciding to
engage themselves in a partnership with an institution of
higher education.
As it appears to be difficult to inform stakeholders adequately,
NVAO will explore other options to communicate effectively
with these groups, for instance through social media. At the
moment NVAO acts within its ability to inform especially
students and employers.

Both NVAO’s decisions and the panel assessment reports are
made public on NVAO’s website (www.nvao.net). The site is
equipped with a search tool to find information about a specific
programme or an institution. It is NVAO’s aim that relevant
information in the assessment reports and decisions should
be easily accessible for the intended readers, such as students,
their parents, the professional field and, finally, the general
public and the government. In this regard NVAO is considering
the use of social media to inform its main stakeholders more
adequately about decisions and reports. The distribution of
specific website links to target groups is an option.

Judgement
NVAO complies with the standard.

Point(s) of attention

On a regular basis NVAO gives feedback to the quality
­assurance agencies and secretaries concerning the quality of
the reports they produced. In Flanders secretaries receive
feedback from VLUHR in cooperation with NVAO. On 14 May
2012, for instance, an evaluation meeting between secretaries
of VLUHR and NVAO staff is planned regarding the quality of
reports. The new accreditation system in Flanders gives
NVAO the authority to reject a report, when it is found lacking
sufficient quality to serve as a basis for an accreditation
­decision. The new system also requires a readable summary
to be included in the report. Both options already exist in the
new accreditation system in the Netherlands.

> NVAO should monitor whether the readability of the

Additional formal information about accredited programmes
and higher education institutions is provided by the registers
on higher education in the Netherlands and Flanders:
­respectively CROHO1 2 > and HOR1 3 > .

In the previous accreditation system in the Netherlands
yes/no decisions were made unconditionally. Therefore,
the NVAO could not formally demand improvement actions.
The higher education institutions were responsible for the
follow-up on the recommendations of the assessment panel,
and the improvements were subject to assessment in the
subsequent evaluation procedure.

12 > Central register for higher education programmes in the Netherlands
[CROHO: Het Centraal Register Opleidingen Hoger onderwijs]. CROHO is
maintained by the governmental organisation DUO, [Dienst Uitvoering
Onderwijs in Groningen]. CROHO is accessible for the public (www.croho.nl).
13 > Higher Education in Flanders [HOR: Het Hogeronderwijsregister] HOR is
developed and maintained by NVAO and is accessible for the general
public (www.hogeronderwijsregister.be).

20

S e lf -eval u at i o n re p or t NVAO 2012

­ ssessment reports and NVAO’s decisions is increasingly
a
appreciated by students, employers and other involved
groups.

3.6  ⁄ Follow-up procedures
Quality assurance processes which contain recommendations
for action or which require a subsequent action plan,
should have a predetermined follow-up procedure which is
implemented consistently.

Facts and findings

Because of the lack of a recovery period in the first phase in
the Netherlands NVAO decided to introduce the instrument

specific agreements in order to create an appropriate arrangement regarding the accreditation of particular programmes.
When accreditation cannot be refused to a programme, but
serious doubts remain, NVAO can propose to the board of the
institution to deliver a report of an additional panel assessment after three years. The agreement is voluntary, but in
almost all cases institutions have agreed to this proposal in
recent years.
In Flanders the instrument of specific agreements has not
been applied, because of the more formal approach regarding
accreditation outlined in Flemish legislation and the existence
of a recovery period from the start of the system in 2005. In
the current system a recovery period is granted by the Flemish
government. In the new Flemish system NVAO will decide on
this period with a maximum of three years. In the current as
well as in the future system in Flanders the new institutional
audit and the programme accreditation decision focuses
specifically on the follow-up based on the recovery plan.
An official follow-up procedure, as meant here in the ESG
guideline, is not part of the new accreditation system either,
but two measures introduce elements that provide equivalent
functionality to a follow-up procedure in the new system in
the Netherlands:1 4 >
> The implementation of the recovery period in the procedure
of programme accreditation.
> The authority of NVAO to take a conditionally positive
­decision in the procedure of initial accreditation and
­institutional audit.
In case of the application of a recovery period the programme
has to present a convincing recovery plan before the recovery
period is granted. At the end of the recovery period the
achieved improvements have to be assessed positively by an
assessment panel before accreditation can be granted.
In case of a conditional decision in the Netherlands a NVAO
panel must assess whether the programme or the institution
meets the conditions within the set conditional period.
No follow-up procedure is applied when a programme
is accredited on the basis of a report that contains recommendations from the assessment panel. In that case, the
follow-up on these recommendations will be assessed in
the subsequent accreditation procedure (after six or eight
years [NL-FL]).
A list of recommendations of the kind mentioned above can be
included in the panel report in the new system, and provides
an additional incentive for a good follow-up by the programme
management or the institution.
14 > A recovery period has been part of the accreditation system in Flanders
since the start in 2005.

The new situation in The Netherlands and Flanders can be
illustrated as follows for each of the assessment procedures:

Formal
Institutional audit: in case of a conditional decision
> a follow-up procedure is applied
Accreditation: decision: 		
 positive: > recommendations are part of the panel report
 negative: > recovery period
> follow-up procedure is applied
> no recovery period > shutdown programme
Initial Accreditation: decision
 conditional for 1 year > follow-up procedure is applied
 unconditional
> FL: approval for ‘the length and two’15>
> NL: variable period to decide by NVAO

Informal
Specific agreements (NL)
		

Self-evaluation

During the previous accreditation system NVAO has repeatedly stated that a recovery period in the Netherlands would
improve the Dutch system. A simple yes/no-decision seems
clear and strong at first sight, but it invites anticipatory
behaviour of programme representatives and panels to avoid
negative outcomes. Therefore opponents of external quality
assurance mention ‘window dressing’ by programme
­representatives as significant disruptive behaviour during
site-visits. On the other hand advocates of the system
­emphasize the peer review approach applied in an environment of trust and quality culture as the best condition to
assess educational programmes and to contribute to its
enhancement.
NVAO is convinced that a recovery period and systematic
follow-up on the basis of a recovery plan contribute to the
flexibility of the system, to the independent and expert’s
judgement of the panels and to an open attitude of
programme representatives. Therefore NVAO welcomes the
recovery period (NL) and the possibility of conditional
­decisions regarding initial accreditation (NL) and the keeping
of the recovery period in FL.

Judgement
NVAO complies with the standard.

1 5 > In Flanders initial accreditation is given for the length of the programme
and two years (e.g. a professional bachelor programme of three years is
approved for five years).

Analysis of Standards and Guidelines part 2

21

3.7  ⁄ Periodic reviews
External quality assurance of institutions and/or programmes
should be undertaken on a cyclical basis. The length of the
cycle and the review procedures to be used should be clearly
defined and published in advance.

Facts and findings
The accreditation period of programmes in the Netherlands
is six years, in Flanders it is eight years. After an initial
­accreditation, reassessment of the programme and reaccreditation should be finalized within six years (in the Netherlands)
and within the length of the programme and two years (in
Flanders). These cycles are registered in Dutch and Flemish
legislation. Within a set period before the accreditation period
has expired, the institution should submit an assessment
report and an application for accreditation to NVAO.
The recommendation of the former ENQA committee in 2007
to equalize the accreditation cycles in the Netherlands and
Flanders has not yet been implemented. In the opinion of
NVAO a cycle of eight years is too long and that is why NVAO
discussed this point with relevant stakeholders. However,
only the governments of Flanders and the Netherlands have
the authority to change the length of the cycles.
The six-year re-assessment timeframe for accreditations is
considered to be adequate. However, the six-year re-assessment timeframe for initial accreditation in the Netherlands is
considered to be too long. In the new Dutch accreditation
system NVAO can decide on the duration of the initial accreditation, with a minimum of one and a maximum of six years.
In Flanders the duration of the initial accreditation will remain
the length of the programme and two years, while the
­accreditation period of study programmes will first remain
eight years, before being reduced to six years in order to come
in line with the institutional reviews.1 6 >

Self-evaluation
The accreditation process is undertaken on a cyclical basis,
which is clearly defined in both Dutch and Flemish legislation.
All frameworks and procedures are published in advance.
NVAO has addressed the topic of equal accreditation cycles
between the Netherlands and Flanders in the meetings of
both Ministers [Comité van Ministers] in recent years. It is
expected that the third round of accreditation in Flanders
(probably to start in 2021) will offer an accreditation cycle of
six years. The new authority of NVAO in the Netherlands to
decide on the duration of an initial accreditation is an important improvement, as is the shift of authority from the Flemish
16 > Flanders uses the term Institutional review instead of Institutional audit.

22

S e lf -eval u at i o n re p or t NVAO 2012

government to the NVAO to decide on the recovery period of a
study programme in Flanders, before this can be accredited.

Judgement
NVAO complies with the standard.

3.8  ⁄ System-wide analyses
Quality assurance agencies should produce from time to time
summary reports describing and analyzing the general findings
of their reviews, evaluations, assessments etc.

Facts and findings
In chapter 9.1 the topic of comparative analyses is discussed
in the context of the follow-up on the recommendations of the
ENQA review committee in 2007. In the past years NVAO
completed eight comparative analyses of various kinds.
Attachment 1 provides a full description of these completed
analyses, in which a distinction can be made between
comparative analyses on the level of a cluster of the same
study programmes and at the level of a domain.
In 2012 NVAO will start an international benchmarking of four
professional bachelor programmes in Switzerland, Finland,
Flanders and the Netherlands in order to compare accepted
quality indicators in these countries.
Until recently, initiatives to comparative analyses were based
on facts and findings in assessment reports or on requests
from the higher education sector or the Minister.
Cluster assessment contributes to comparative analyses.
Nearly all Dutch and Flemish programmes will be subject to
clustered assessments organised by QAA’s, since it has been
decided to reintroduce this approach in Dutch professional
education. The reports of the clustered programmes are
published in the same period accompanied by a comprehensive panel report containing general considerations regarding
the involved discipline and programmes. This report provides
an additional, more generic view on the performance of
these programmes, which is also used by NVAO for its own
comparative analyses.
Besides comparative analyses on the level of clusters or
domains, system-wide analyses on the level of the system
itself are internally planned, as part of NVAO’s internal quality
assurance (see chapter 4.8) or as part of piloting the new
accreditation system or its implementation. Both in the
­Netherlands and Flanders system-wide evaluations of the
new accreditation system are foreseen, which will be (partly)
done by NVAO.

As NVAO’s new strategic policy underlines the importance
of comparative and system-wide analyses NVAO recruited a
new staff member assigned to do quantitative and qualitative
research to underpin the analyses.

Self-evaluation
Despite the substantial effort of NVAO regarding comparative
analyses, NVAO did not have a clear policy on this theme and
time in recent years. In the new Strategic Policy Statement
of NVAO (2012 - 2016) the NVAO task of system-wide and
comparative analyses is emphasized and should result in a
clear agenda on this topic for the coming years.
Another point for improvement concerns the follow-up of the
results of system-wide or comparative analyses. The analysis
only makes sense if the results are thoroughly discussed with
representatives of the institutions and programmes involved.
Subsequently, the institutions should accept the results as
meaningful for further improvement of the programmes
involved. NVAO should take more initiatives to start these
discussions while and after completing a system-wide or
comparative analysis.
In conclusion it can be stated that NVAO completed eight
comparative analyses in the past years and is therefore
compliant with the guideline of this standard, as it is required
‘that summary reports must be produced from time to time’.
Nevertheless, NVAO has to work on implementing its new
policy regarding system-wide and comparative analyses.

Judgement
NVAO complies with the standard.

Point(s) of attention
> To implement a clear policy and activities regarding system-

wide analyses through yearly planning.

Analysis of Standards and Guidelines part 2

23

Chapter
4
Analysis of
Standards
and
Guidelines
part 3

4.1  ⁄ Use of external quality assurance
procedures for higher education

that task. The Committee of Ministers can thus only intervene
in the functioning of NVAO, but not in NVAO’s decision-making.

The external quality assurance of agencies should take into
account the presence and effectiveness of the external quality
assurance processes described in Part 2 of the European
Standards and Guidelines.

Self-evaluation

In chapter 3 it is described how NVAO takes into account all
European Standards and Guidelines of Part 2.

Judgement
NVAO complies with the standard.

4.2  ⁄ Official status
Agencies should be formally recognised by competent public
authorities in the European Higher Education Area as agencies
with responsibilities for external quality assurance and should
have an established legal basis. They should comply with any
requirements of the legislative jurisdictions within which they
operate.

Facts and findings
NVAO is the official, public and binational accreditation
organisation in higher education in the Netherlands and
­Flanders. Its structure and duties are described in the
­binational Treaty1 7 > and in both national legislations.
NVAO reports to the Dutch and Flemish Parliaments via
their Ministers of (Higher) Education. NVAO’s annual report
is used for accountability purposes.

The official status offers NVAO a good formal structure to
guarantee its independence. Its binational nature is an
unique and interesting example regarding the objective of the
Bologna treaty of 1999 to expand cross border cooperation
and to achieve one European higher education area. During
the last years lots of international delegations visited NVAO,
because of its binational nature and to learn from the
­cooperation in the field of external quality assurance between
the Netherlands and Flanders.
From the start of NVAO in 2005 it was expected that a
­binational organisation would reinforce the Dutch-Flemish
cooperation in EQAHE, which also came about. It resulted for
instance in a frequent exchange of peers in the panels
between the Netherlands and Flanders, and NVAO applies
the same criteria to decide on Dutch and Flemish applications. However, progress on the mutual recognition of degrees
between the two countries has been limited. The planned new
Treaty offers a new opportunity in this case. The making of one
unified list of (initially) accredited programmes to underpin
mutual recognition would be a step forward. Even a generic
recognition of the levels of the degrees is in sight.

Judgement
NVAO complies with the standard.

Point(s) of attention
> From its binational nature NVAO contributes to the align-

NVAO was granted the status of an autonomous administrative body with legal rights according to Dutch legislation.
Consequently, NVAO does not resort under a particular
minister and is not subject to ministerial responsibility.
NVAO has full decision-making powers concerning decisions
about (initial) accreditation.
However, NVAO is accountable to the Committee of Ministers,
which approves its budget, the annual report and the annual
accounts. In accordance with the Treaty, the Committee of
Ministers can only intervene in case of serious neglect by NVAO
of its (initial) accreditation task, threatening the execution of

17 > Treaty between the Kingdom of the Netherlands and the Flemish Community of Belgium regarding the accreditation of programmes within Dutch
and Flemish higher education, The Hague, 3 September 2003 [Verdrag
tussen het Koninkrijk der Nederlanden en de Vlaamse Gemeenschap van
België inzake de accreditatie van opleidingen binnen het Nederlandse en
Vlaamse hoger onderwijs]. Referred to in this document as the Treaty.

ment of the Dutch and Flemish systems of higher education,
including the mutual recognition of degrees. Little progress
has been made in recent years in this respect and therefore
more attention must be paid to it. A generic recognition is in
sight thanks to the attention drawn by NVAO and others.

4.3  ⁄ Activities
Agencies should undertake external quality assurance activities (at the institutional or programme level) on a regular basis.

Facts and findings
The tasks of NVAO are described in the Treaty. They are further
stipulated in the Netherlands in the Dutch Act1 8 > on higher
education and research and can be summarized as the
(initial) accreditation of programmes of institutions of
1 8 > [Wet op het hoger onderwijs en wetenschappelijk onderzoek, 1 januari
2011], changed version due to the new accreditation system.

Analysis of Standards and Guidelines part 3

25

higher education and the assessment of these institutions
(Institutional audit).
In Flanders, the Flemish Act of 4 April 20031 9 > forms the
legal basis for (initial) accreditation in higher education and
stipulates that the responsibility for (initial) accreditation of
programmes is assigned to NVAO. The number and type of
NVAO’s decisions up to 1 January 2012 are illustrated in
chapter 8.1, table 7.
Another important task of NVAO stipulated by the Treaty
concerns the international perspective on the decisions and
position of NVAO. This aspect is dealt with in chapter 8.4:
The international assignment of NVAO.
Since 30 April 2009 NVAO has also been empowered to
­validate the discipline-specific learning outcomes of
­bachelor’s and master’s programmes in Flanders.
The structures of the external quality assurance systems in
the Netherlands and Flanders will be discussed in Chapter 7.
All higher education programmes in the Netherlands and
Flanders must be accredited on a regular basis; in the
Netherlands every six years, in Flanders every eight years.
The Dutch institutions in higher education are subject to
an institutional audit every six years. In the new system in
­Flanders institutions probably will be subject to an institutional audit every six years.
NVAO is charged with several additional tasks, which are
described in detail in chapter 8.5 and attachment 2.
These additional assessment tasks enable NVAO to acquire
a better insight in quality issues in Dutch and Flemish higher
education and internationally.
An example of the organisation of NVAO’s processes of (initial)
accreditation is further described in the workflow scheme in
attachment 3.

Self-evaluation
In evaluation discussions about the accreditation process
with institutions in higher education and with quality
­assurance agencies in the Netherlands, the major comments
of both stakeholders concern the following points:
> NVAO should respect the formal deadlines better.
> additional questions and additional assessments by NVAO;
stakeholders believe that these additional activities unnecessarily increase the administrative burden in the system.
In 2011, parallel to the regular meetings with VLUHR and the
Flemish cabinet, NVAO visited in Flanders all higher education
19 > Decree on the restructuring of higher education in Flanders, 4 april 2003.

26

S e lf -eval u at i o n re p or t NVAO 2012

institutions to discuss their experiences with application
procedures for accreditation at NVAO. In general NVAO
respects deadlines in Flanders due to the strict regulations
in Flanders. Regarding additional questions and additional
assessments the Flemish higher education institutions have
the same opinion as the Dutch institutions. The results of
these evaluation meetings in both countries are processed
in NVAO’s IQA activities.
As there are approximately 600 existing programmes and
100 new programmes to be accredited in the Netherlands and
Flanders each year, the (administrative) assessment burden is
high. One of the aims of the new system is to lower this burden
and to focus on the heart of the matter: ‘does the programme
achieve what it preaches on an internationally accepted
level?´ NVAO expects that better and convincing reports will
be a positive outcome of the new assessment process and,
as a consequence, that NVAO can reduce its additional
assessment activities. This would be an improvement of the
accreditation process.
NVAO played a proactive role in the period 2008-2010 in the
evaluation and development of the new accreditation system
and, naturally, supports it strongly. NVAO is convinced that the
new accreditation system in the Netherlands challenges
­institutions to work on objectives as profiling, differentiation
and quality enhancement. The new accreditation system in
Flanders will probably incorporate the objectives differentiation and quality enhancement. Profiling is not an actual issue
in Flemish higher education.

Judgement
NVAO complies with the standard.

Point(s) of attention
> NVAO should respect deadlines better. A newly appointed

coordinator for assessments manages the workflow in the
organisational process, which should contribute to a timely
completion of applications. A careful and timely completion
of primary tasks must have priority. Additional tasks should
be part of a structured and realistic yearly plan.
> NVAO should monitor whether the aim of decreasing
numbers of additional assessments and additional questions will be achieved in the coming years. It is obvious that
from the outset NVAO should try to avoid duplicating the
work of panels.

4.4  ⁄ Resources
Agencies should have adequate and proportional resources,
both human and financial, to enable them to organize and run
their external quality assurance process(es) in an effective and
efficient manner, with appropriate provision for the development of their processes and procedures.

Facts and findings
NVAO has a (General) Board of 11 members2 0 > . The chair and
three other Board members constitute the Executive Board.
NVAO also has an Advisory Council of 14 members2 1 > .
The workforce of NVAO consists of 56 people (53,1 fte): four
executive Board members, one managing director, 30 policy
advisors, one Dutch and one Flemish legal advisor2 2 > (together
29,1 fte2 3 > ) and 20 other (partly supporting) staff (19 fte).
Four policy advisors are available for international assignments. The staff includes policy, legal and communication
advisors and supporting staff (policy secretariat, finances
and personnel, records department and general services).
Decisions are prepared by the policy and legal advisors and at
least one Executive Board member. In 2010 the Board (re)
appointed four policy advisors as coordinators for the areas
Assessments, New accreditation system, Internal quality
assurance and Internationalisation. In their role as coordinators policy advisors remain accountable to the director.
See the organisational chart (attachment 4).

Core competences of NVAO’s staff
All academic staff hold a master’s degree, three hold a PhD
degree. The executive board of NVAO consists of four
members, two from the Netherlands, two from Flanders.
The primary responsibility of the board is the accreditation
process and the accreditation decisions. A managing director
is responsible for the management process of NVAO.
The acquired professional experience of NVAO’s staff is varied.
All board members and the director have a longer career in
(higher) education in board or management positions.
Most staff acquired substantial work experience in higher
education as teacher, developer, manager, researcher,
inspector for higher education or policy advisor, before being
employed at NVAO. A small group of junior staff was recruited
20 > The members of the General Board act on personal name and don’t
­represent an organisation.
21 > The members of the Advisory Board represent stakeholder organisations.
22 > The activities of the Flemish legal advisor are outsourced by NVAO to a
Flemish legal consultancy firm.
23 > About one-third of the policy advisors is Flemish, two-third is Dutch.

to fulfil a kind of traineeship before being fully employable
in NVAO’s work processes.
Compared to similar members within the ENQA network, NVAO
staff is on par2 4 > . NVAO has built a staff group of good quality.

Finances
NVAO has an annual budget of approximately € 6 million,
which is financed jointly by the Netherlands (60%) and
­Flanders (40%). Each year NVAO draws up a budget which
is then decided upon by the Committee of Ministers and both
Parliaments. In the Netherlands and Flanders NVAO applies
fixed rates by law. The rate for an application initial accreditation is maximum € 15.000 in the Netherlands and € 5.000 in
Flanders. For an application accreditation the rate is € 750 in
the Netherlands and € 500 in Flanders. In case of additional
tasks and assessment assignments abroad NVAO charges
cost-covering fees.

Facilities
Since 2005 NVAO is located at the Parkstraat in The Hague,
near the city and the Dutch parliament and the Dutch ministry.
Good facilities are available here. Several meeting rooms
for smaller and larger groups are available to organise
­conferences, seminars and meetings. Presentation and
IT-facilities are up-to-date. In November 2011 a new information system was introduced to optimize the work-flow.2 5 >

Self-evaluation
NVAO has been funded sufficiently by both governments for
its primary tasks in recent years. However, due to the
­substantial size of additional tasks that were assigned to
NVAO, a
­ dditional funding was needed. NVAO agreed with both
­ministries in the Netherlands and Flanders that additional
tasks will be organised and budgeted on a project basis.
NVAO informs the ministries in advance about the expected
costs. The latter can decide whether to continue the project
initiative or not.
All public organisations, including NVAO are currently subject
to public cuts initiated by the governments in the Netherlands
and Flanders. These cuts don’t affect NVAO seriously up to now.
During the years NVAO developed a good quality of staff with
varied competences and work experiences. Dutch and Flemish
2 4 > In 2011 the ENQA network Internal Quality Assurance completed a first
benchmark study on Staff development among 5 members of ENQA
[QAA (UK) , ACQUIN (GE), EVA (DK), ANECA (SP), NVAO (NL)]. NVAO
­participated actively in this benchmark. The results will be presented
on the ENQA website in Spring 2012.
2 5 > NVAO uses a document management system (DMS) of RICOH.

Analysis of Standards and Guidelines part 3

27

board and staff members form a productive mixture and the
chemistry between the Dutch and the Flemish part of NVAO
is good. Flemish staff are deployed on Dutch and Flemish
applications and vice versa. Perceived culture differences
are rather stimulating and don’t cause problems. A point for
improvement is the further development of expertise of staff
on specific themes in higher education.

Judgement
NVAO complies with the standard.

Point(s) of attention
> NVAO should invest in further development of specific,

theme-oriented, higher education expertise of staff.

4.5  ⁄ Mission statement
Agencies should have clear and explicit goals and objectives
for their work, contained in a publicly available statement.

Facts and findings
In 2012 NVAO renewed its Strategic Policy Statement2 6 > for
the period 2012-2016, which includes NVAO’s new mission,
that was approved by the General Board on 19th March 2012
and is published on NVAO’s website and included in this report
as attachment 5.

Mission
NVAO has defined its new mission as follows:
NVAO is the independent and authoritative binational
­accreditation organisation set up by the Flemish and Dutch
governments, whose primary goal it is to provide an expert and
objective judgement of the quality of higher education in Flanders
and the Netherlands. NVAO does this with a constructive,
­critical attitude, respecting the autonomy of institutions and
their primary responsibility for the quality of their education,
and with an open eye for the growing international context.
NVAO is open, clear and transparent towards society and all
concerned, especially the institutions of higher education and
the students.
In its Strategic Policy Statement NVAO states that it considers
institutions primarily responsible for quality assurance and
quality improvement. NVAO is assigned to assure that
programmes meet the required standards and to stimulate
the quality debate, giving account of its procedures,
­disseminating ‘good practices’ and visiting the institutions

and programmes. NVAO respects the autonomy of institutions,
which are responsible for their internal quality assurance
and for taking measures for quality improvement.
Its mission commits NVAO to be strongly present and
­communicative in the quality debate in higher education.
In practice, NVAO’s role in stimulating debate in recent years
can be illustrated by referring to its conferences, seminars,
intensive meetings with the stakeholders and a large number
of visits each year to institutions in the Netherlands and
­Flanders. During these visits, discussions are held with the
management of the institution on, among other things, their
experiences with the accreditation system and the analysis
of a series of accreditation decisions for their programmes.
The meetings are initiated by NVAO. In addition, members of
the NVAO Executive Board or staff often give presentations at
conferences. NVAO organises its own conference every two
years either in the Netherlands or in Flanders. Furthermore,
the NVAO newsletters, published every two months, give more
information on several issues regarding accreditation.

Self-evaluation
The discussions preparing a new mission statement
­concentrated for a part on the question whether NVAO should
concentrate on its assurance assignment, its enhancement
assignment or both. These discussions were affected by the
introduction of a new accreditation system, the international
developments and recent quality incidents and their
­consequences in Dutch professional education. These events
pushed the Dutch in the direction of stressing the assurance
assignment, while the objective of the new accreditation
system and the role of NVAO initially was more oriented
towards the enhancement assignment.
The newly introduced institutional audits, focusing on
policies and the IQA system of the institution provide a good
opportunity to focus on quality enhancement, like the
planned, extended realisation of system-wide or comparative
analyses. The latter two focus on quality enhancement rather
than on quality assurance. NVAO is of the opinion that in a
well-operating accreditation system the emphasis should
be on the enhancement objective. In the final Strategic
Policy Statement this discussion was concluded.

Judgement
NVAO complies with the standard.

Point(s) of attention
> NVAO should assess regularly whether it achieves a

26 > Strategic Policy statement NVAO 2012 - 2016, 13 February 2012
[Strategische beleidsverklaring NVAO 2012-2016]

28

S e lf -eval u at i o n re p or t NVAO 2012

consistent balance between its accountability and

­ nhancement assignment, an assessment that should
e
not by dictated by ad hoc issues in higher education.

4.6  ⁄ Independence
Agencies should be independent to the extent both that they
have autonomous responsibility for their operations and that
the conclusions and recommendations made in their reports
cannot be influenced by third parties such as higher education
institutions, ministries or other stakeholders.

Facts and findings
The independence of NVAO has been described in detail in
chapter 4.2. Here it can be added that members of NVAO’s
­Executive and General Board are mandated by the Committee
of Ministers for four years. They are appointed on strictly
personal capacity and not as representatives. The other
­Advisory Council members represent different stakeholders
(see chapter 4.8).
In order to guarantee the independence of board and staff
members, NVAO has ruled that both cannot participate in
applications from institutions or programmes they have
been associated with in any form during the last five years.
Members of the Board have to be completely independent in
taking decisions. If there is a specific application where this
independence cannot be guaranteed, the member of the
Board will withdraw from the decision-making process for the
programme concerned. NVAO is responsible for recruiting its
own staff and the decisions it takes.
NVAO’s independence is recognised by different stakeholders.
NVAO is regarded as the quality controller of the quality
assurance organisations’ panels, site-visits and reports.
Two examples can illustrate this.
For instance, the Dutch parliament asked NVAO on 26 January
2012 to explain the operation and the first experiences of the
new accreditation system in order to convince the members
of parliament of the reliability of the new accreditation system
in the Netherlands.
In 2010 NVAO was invited by the Flemish Parliament to explain
its work and opinions on the new accreditation system in its
Commission of Education. In Flanders NVAO has been allowed
as member of the Advisory group (Adviesgroep) of VLOR, the
official Flemish Educational Council (Vlaamse Onderwijsraad)
to advise the government about the structure and operation
of the new accreditation system in Flanders.
These examples illustrate that NVAO is not considered as a
purely executive organisation by the involved stakeholders.
This position allows NVAO to advise independently on the
new system.

Self-evaluation
From the start of the accreditation system in the Netherlands
and Flanders (2004, NL – 2005, FL) NVAO has been completely
independent regarding decision making. This independence is
the best guarantee for a transparent and strong decision
making process.
It is not always easy to demonstrate NVAO’s independence to
all stakeholders, especially its position towards the parliament
and the institutions. The best proof of NVAO’s independence is
that it is sometimes perceived by the ministries and politicians
as too soft to the institutions, while the institutions sometimes
blame NVAO for being too severe in following the political
demand for robustness and transparency. This is partly linked
to NVAO’s deliberate choice for both accreditation and quality
enhancement (see chapter 4.5). This delicate balance can
only be maintained as a result of NVAO’s strict independence
and its policy not to mix decision-making with advice.
All recommendations for the improvement of programmes
are formulated by independent panels. NVAO sometimes
only chooses to highlight some of the recommendations.

Judgement
NVAO complies with the standard.

Point(s) of attention
> NVAO has to communicate strongly, independently and,

especially, proactively. Therefore a new communications
strategy has been set up recently.

4.7  ⁄ External quality assurance criteria
and processes used by the agencies
The processes, criteria and procedures used by agencies
should be predefined and publicly available. These processes
will normally be expected to include:
> a self-assessment or equivalent procedure by the subject of
the quality assurance process;
> an external assessment by a group of experts, including, as
appropriate, (a) student member(s), and site visits as decided
by the agency;
> publication of a report, including any decisions,
­recommendations or other formal outcomes;
> a follow-up procedure to review actions taken by the
subject of the quality assurance process in the light of
any recommendations contained in the report.
In the previous Dutch and the current Flemish system the
21 criteria or standards are grouped in 6 subjects (aims,
programme, staff, facilities, internal quality assurance (IQA)
and achieved results). These criteria were developed in coopAnalysis of Standards and Guidelines part 3

29

eration with all stakeholders and are perceived as clear and
robust, but also as burdensome for the management of
programmes because in most institutions systems for HRM,
IQA and other policies are often formulated and managed at
the institutional level. Therefore, standards for accreditation
of study programmes in the new system have been limited to
three (general aims and intended learning outcomes, the
learning environment and the achieved learning outcomes),
and an institutional audit is introduced. The standards and
underlying criteria in the new system were developed in cooperation with all stakeholders through pilots and evaluation.
The previous standards and criteria are only maintained in the
new system in the framework for EPA. The criteria within the Dutch
and Flemish systems are described more in detail in chapter 3.3.
Chapter 3.4 described the successive steps of the (initial)
accreditation procedure in the Netherlands and Flanders.
Here follows a summary of these steps, which applies both to
the previous Dutch and the current Flemish system, as well as
to the new accreditation system (NL, FL in 2013).
> The institution writes and submits the self-evaluation report
or the information file.
> A panel from a quality assurance agency or an NVAO panel
carries out a site-visit and an assessment of a programme.
> The panel prepares an advice to the Board of NVAO and
informs the institution about an outline of its conclusions
of the assessment at the end of the site-visit.
In case of programme accreditation the institution has to
submit an application to the NVAO based on the panel report.
In case of the new institutional audit or initial accreditation
the NVAO panel presents the report of its findings and judgements to the board of NVAO.
> The next step is the decision-making process by NVAO.
If NVAO takes a negative accreditation decision on an
­application for accreditation, the institution must either
stop the programme immediately or it can be granted a
recovery period. If NVAO intends to take a negative decision
on an application for initial accreditation the institution can
withdraw the application or accept the negative decision.
> The decision regarding the institutional audit in the
­Netherlands can be threefold: positive, conditionally positive
or negative. A positive decision entitles the institution to
apply for LPA for all its programmes. A conditionally positive
decision requires the institution to repair the deficits within
one year. A negative decision is in force for three years, in
which period all planned assessments of programmes have
to be executed according to the framework for an EPA. The
institutional audit in Flanders will not be concluded with a
final decision; in the first round it is a learning experience,
aiming at quality enhancement only.
30

S e lf -eval u at i o n re p or t NVAO 2012

NVAO implemented several procedures and processes
to improve the consistency of its decision making. For an
­overview, see chapter 3.3.
A formal follow-up procedure is not included in the accreditation system. However such a procedure has become common
practice, as described in chapter 3.6.

Self-evaluation
Both the previous Dutch and current Flemish criteria were
reviewed as compliant with the ESG in 2007. Programme
assessment and accreditation in the new system focus on the
content of the curricula and on the achieved learning outcomes.
Flanders and the Netherlands are at the forefront in this
respect, as this focus is not common in the European higher
education area. NVAO expects that the institutional audit will
be a strong incentive for further development of effective
internal quality assurance on an institutional level, which will
positively influence the quality of individual programmes.
The new Dutch system gives NVAO additional powers which
will improve the effectiveness of its operation. NVAO believes
that the recovery period and the power to award a conditional
initial accreditation are effective tools for improvement of the
system. NVAO’s authority to appoint (and reimburse) panels in
the Netherlands and the reintroduction of the cluster assessment approach in Dutch higher education are other important
improvements. In the new accreditation system NVAO’s
­influence on the quality of panels has improved and that is
of major importance for the quality of the system as a whole.
In Flanders NVAO’s influence on the panel composition
has improved as well. The VLUHR will only decide on the
composition of assessment panels after an NVAO advice.

Judgement
NVAO complies with the standard.

4.8  ⁄ Accountability procedures
Agencies should have in place procedures for their own
accountability.

4.8.1  ⁄ Appeal procedures
Facts and findings
Both in the Netherlands and Flanders, institutions can lodge
an appeal against (initial) accreditation decisions taken by
NVAO. Although formal appeal procedures differ in both
­countries, the principles behind them are to a large extent
similar. The main steps in the procedure in both countries are
described below.

> Prior to taking a final decision concerning an application for

(initial) accreditation, NVAO will inform the institution about
its ‘intended decision’. The institution then has two weeks to
respond. Subsequently, NVAO takes its final decision.
> An institution can lodge an internal appeal against a NVAO
decision. To handle internal appeals, NVAO has set up an
appeals commission that provides advice to NVAO on which
NVAO will take its decision.
> If the decision on an internal appeal is negative, the
­institution can lodge an external appeal against this
­decision with the Administrative Jurisdiction Department of
the Council of State [Afdeling bestuursrechtspraak van de
Raad van State] in the Netherlands. In Flanders, institutions
can lodge an external appeal with the Flemish Government
and finally with the Council of State of Belgium [Raad van
State].
Since the start of the accreditation system, NVAO has received
nine internal appeals in the Netherlands. One appeal has
been granted. In Flanders there were seven cases of internal
appeal up to now. One appeal was granted, two were rejected
and four were partly granted, but did not change NVAO’s final
decision after a new assessment by a panel.
At the time of writing, four external appeals have been lodged
against a decision of NVAO in the Netherlands. One external
appeal has been granted on procedural reasons. The others
were refused. In Flanders no external appeals have been
lodged.

Self-evaluation
The internal appeals procedures are set up satisfactorily.
In the Netherlands, they have proven to function effectively.
In Flanders, a new Procedure Regulation has been adapted in
2011 after an evaluation by the NVAO board and the appeals
commission. The adaptations concern the terms of appointment of the members of the commission, the specific
­competences of the experts in the commission and the
­decision rules concerning its advice, in which it is now clearly
stated that the appeals commission cannot change the scores
on its own, but only accept the appeal and advise to start a
new assessment on the identified problematic standards.

4.8.2  ⁄ Internal quality assurance
Facts and findings
From the outset, NVAO has developed a system for its internal
quality assurance (IQA). Initially, this system had a thematic
rather than a structural approach. Further development and
the recommendations of the review committee of 2007 led

NVAO to focus on structuring its internal quality assurance
(IQA).
Starting points of the NVAO IQA system
NVAO has developed a system for internal quality assurance
that:
> applies the criteria of the EFQM-model for an integrated
approach concerning the organisation and its working
processes;2 7 >
> facilitates frequent evaluation of the results and a
­structured plan for improvement (by following the ­
Plan-­Do-Check-Act cycle);
> is transparent, simple and non-bureaucratic through the
application of a limited number of instruments;
> provides the basis for an external (international) evaluation
by the adoption of international standards in the development of the internal quality assurance system;
stimulates involvement of all staff members, and in addition;
guarantees involvement of all stakeholders by means of an
open dialogue with these stakeholders.
Elaboration of the system
NVAO distinguishes thirteen quality areas within the internal
quality assurance system: (1) Strategy, Policy and Leadership,
(2-3)(Initial) Accreditation – including Research Master’s
Programmes, (4) International Affairs, (5) Communication,
(6) Legal Affairs, (7) Support Services, (8) Additional Tasks,
(9) Internal Quality Assurance, (10) Human Resources,
(11) General Services, (12) Finances and (13) ICT.
The system can be summarized as follows. For each quality
area, strategic objectives and targets are formulated on a
two-year basis. A renewed quality scheme2 8 > based on the
strategic goals guides the process in every area. At least every
two years, evaluation data are gathered systematically by each
quality area working group. An integral evaluation scheme2 9 >
has been developed to coordinate all NVAO’s evaluation
­activities. Stakeholders can be questioned efficiently in this
way. The evaluations are prepared by the quality coordinator
of the specific quality area. At the end of the two year
period a report of progress is produced for every quality
area (attachment 7 contains an example).
Primary processes (Accreditation, Initial Accreditation and
International Affairs) are subject to a more thorough
2 7 > EFQM (European Foundation for Quality Management) developed a
­non-prescriptive framework for organisational management systems and
designed for helping organisations in their drive towards being more
competitive. The Model is regularly reviewed and refined: the last update
was published in 2010.
2 8 > attachment 6
2 9 > attachment 8

Analysis of Standards and Guidelines part 3

31

­ valuation than other processes. The IQA group facilitates the
e
process and steers the evaluations for the whole organisation.
Every two years, after completion of the quality cycle, a quality
report is drawn up, based on an evaluation per quality area.
The next quality report will be available in 2012.
Accountability
NVAO publishes its annual report every Spring. In the annual
report NVAO presents its operational and financial results
and activities of the previous year.
Evaluation
The integral evaluation scheme structures NVAO’s IQA
­evaluation approach, and consists of written and verbal
­activities. Important NVAO evaluation structures / activities
are:
> The periodic meeting with the Advisory Council (three times
a year).
> The periodic meetings of the two- and four-party platforms
in Flanders: the Flemish cabinet and higher education
departement with NVAO (three times a year) and VLIR,
VLHORA, the Flemish cabinet, the Flemish HE department
and NVAO (three times a year).
> The periodic meetings of the Dutch ministry and NVAO
(approx. ten times a year) and the quality assurance
­agencies in the Netherlands (plenary twice a year, bilateral
once a year);
> Written evaluation of procedures initial accreditation;
> Verbal evaluation of the procedure institutional audit and
the accreditation procedures of the same institution.
> Yearly survey of the external communication;
> Yearly survey of international activities;
> Yearly staff meeting to discuss the realisations of previous
objectives and plans and formulate the new objectives
within the PDCA-cycle of NVAO’s internal quality assurance.
Substantial evaluation activities in recent years were:
> The evaluation of the accreditation system in the first phase

in Netherlands and Flanders and the pilots to prepare the
second phase;
> A visit of NVAO to all Flemish higher education institutions
in 2011 to discuss all issues regarding (initial) accreditation
processes as far as NVAO is involved. The results were
presented in the document Evaluatieve gesprekken in
Vlaanderen;3 0 >
> An extensive survey of the staff’s satisfaction in 2011.

Improvements in recent years
A number of substantial improvements in NVAO’s operation,
achieved in recent years, are listed here.
> (Initial) Accreditation
- A new and evolved accreditation system in the
­Netherlands (2011) and expected in Flanders (2013).
- Certification of about 185 external secretaries (2011
and 2012).
> Internationalisation
- The certificate distinctive feature Internationalisation,
which was developed and tested in a pilot (2010), leading
to a new framework regarding the distinctive feature
Internationalisation (2011).
- Six agreements with international partners regarding
mutual recognition of accreditation decisions (2007).
- The implementation of Crossroads (2009).
> Information Communication Technology
- A new document management (DMS – RICOH, 2011) and
a new document administration system Easydoc (2008).
> Communication
- A new NVAO website (2011), intranet (2009) and the
quality magazine Q&A (2010).
> Human Resource Management
- A Staff satisfaction survey and competence profiles for
staff members (2011).
> Strategy
- A new Strategic Policy Statement 2012-2016 (2012).
> Organisation
- The introduction of the function and rank of thematic
coordinator, regular meetings with the director (2010).

Self-evaluation
Since about three years NVAO employs a systematic
­evaluation system. All core assessment procedures are
subject to periodic evaluation. The evaluation activities are
on track and sometimes adjusted because of efficiency
reasons. NVAO intended to develop a lean and, especially,
practical IQA system. The current system is a result of this
approach. In the opinion of NVAO an IQA system has only
significant value if supported by a strong quality culture.
­Motivated, critical and proactive staff may even be more
important to achieve NVAO’s objectives. Therefore, NVAO
invests in the quality of ‘people and processes’.

Judgement
NVAO complies with the standard.

30 > Evaluation meetings in Flanders: a thematical summary of facts and considerations of NVAO, 21 October 2011, [Evaluatieve gesprekken in Vlaanderen:
thematische samenvatting van de bevindingen en overwegingen van de NVAO].

32

S e lf -eval u at i o n re p or t NVAO 2012

Point(s) of attention
> Legal procedures are time and energy consuming, as was

proved in the past years on several occasions. During an
appeal procedure accreditation of a programme is
suspended according to the law, which causes substantial
uncertainty for all involved and undermines the meaning of
accreditation.
> For NVAO (supporting) staff members the importance of
their own contribution to IQA is not always clear. Better
communication and more investment in commitment are
necessary.

4.9  ⁄ Miscellaneous
ENQA criterion 8 I
The agency pays careful attention to its declared principles at
all times, and ensures both that its requirements and
processes are managed professionally and that its judgements
and decisions are reached in consistent manner, even if the
judgements are formed by different groups.
NVAO is of the opinion that it has proven to be compliant with
this criterion in the chapters three and four.

ENQA criterion 8 II
If the agency makes formal quality assurance decisions, or
conclusions which have formal consequences, it should have
an appeals procedure. The nature and form of the appeals
procedure should be determined in the light of the constitution
of the agency.
NVAO is of the opinion that it has proven to be compliant with
this criterion in the chapter 4.8.1.

ENQA criterion 8 III
The agency is willing to contribute actively to the aims of ENQA.
NVAO is of the opinion that it has proven to be compliant with
this criterion, especially in chapter 8.4.

Analysis of Standards and Guidelines part 3

33

Chapter
5
Strengths
and points of
attention
regarding
NVAO’s
operation

In this chapter NVAO’s position in the accreditation system
in the Netherlands and Flanders is concluded by discussing
the main strengths of NVAO’s operation and the main points
of attention. This summary is based on the three angles
mentioned in the introduction:
> NVAO as organisation within the new accreditation system;
> NVAO’s compliance with the ESG resp. Parts 2 and 3 and;
> A first reflection on the new accreditation system in the
Netherlands, which is presented separately in chapter 7.

5.1  ⁄ NVAO as organisation within the
new accreditation system
Strengths
Respected
NVAO is a respected player with substantial impact on quality
and quality assurance in higher education in the Netherlands,
Flanders and in the European higher education area. As
supervisory body in higher education in the Netherlands and
Flanders NVAO holds an influential position. NVAO’s initiating
role in the evaluation and development of the accreditation
system in the period 2008-2010 is just an example of its
­critical, self-reflecting, flexible, cooperative and transparent
way of functioning, which have all contributed to the respect
for NVAO in the eyes of all stakeholders.
Its major contribution to the European Network of Quality
Assurance (ENQA) and the European Consortium of
­Accreditation (ECA) are just two examples of NVAO’s
­significant international contribution and NVAO is firmly
­positioned world-wide as a secretary of INQAAHE.

Flexible and pro-active
Since 2005 NVAO has completed a large number of additional
tasks as listed in chapter 8.5. All these tasks were assigned
to NVAO on request of the Dutch and Flemish ministers.
In completing these tasks NVAO showed not only its flexibility,
but it also rapidly augmented its expertise in the field of
quality assurance in Dutch and Flemish higher education.
The pro-active character of NVAO came to the fore during the
evaluation of the previous accreditation system and the
­preparation of the new system in the Netherlands and
­Flanders (see chapter 7).
Thanks to NVAO’s (inter)national position new developments
in quality assurance are quickly recognized, evaluated and
sometimes transferred to the accreditation systems or frameworks in the Netherlands and Flanders. The introduction of
the institutional audit and the focus on learning outcomes are
two examples of how NVAO is constantly learning from new
developments. The clear difference between intended and
achieved learning outcomes in the new accreditation frame-

work is a good example of NVAO’s pole-position in the new
developments.

Cooperation
NVAO communicates and cooperates regularly in an open
and transparent way with all stakeholders by organising or
participating in conferences, pilots and workshops in Dutch
and Flemish higher education. New developments and
­instruments are always implemented in cooperation with
the stakeholders through pilots, as has been done in the
development of the new accreditation system.

Points of attention
Administrative burden
The administrative burden of the accreditation system is an
important issue among stakeholders, especially the institutions. In the acceptance of the new system all parties agreed
on a reduction of 25% in administrative burden in programme
accreditation. Now it is up to all to monitor actively whether
this aim will be achieved. NVAO has to play a leading role here.
NVAO should also monitor whether the aim of a decrease in
additional assessments and additional questions will be
achieved in the coming years.

Consistency
Considering the large number of applications from the
­Netherlands and Flanders every year, consistency in
­assessment and its own decisions is a permanent issue for
NVAO and the panels. In this respect NVAO will continue to
support the quality assurance agencies, the (NVAO) panels
and to monitor its internal processes. A new element in the
accreditation system in the Netherlands and, possibly, in
Flanders too, is the judgement of the programme on a
­four-point scale, which makes consistency even more
­important. The new evaluation matrix and the new internal
organisational structure of NVAO are supposed to be of help
in its strive towards consistency.

Additional tasks and time limits
Over the years NVAO has completed a large number of
­additional tasks, which have sometimes caused workflow
problems. As a result time limits of regular applications
could not be respected more than once. In alignment with
the Dutch and Flemish ministry NVAO must set priorities to
better manage its primary tasks and additional assignments.

Added value of binational nature
Because of its binational nature NVAO tries to contribute
significantly to Dutch – Flemish cooperation in higher
­education. But an important objective from the start in 2003,
Strengths and points of attention regarding NVAO’s operation

35

the mutual recognition of degrees, has not been achieved so
far. This is disappointing, although mutual recognition is up
to both governments and not to NVAO. NVAO has to look for
other opportunities to strengthen the cooperation in higher
education between the two countries in the near future.

5.2  ⁄ NVAO’s compliance with the ESG
resp. Parts 2 and 3
NVAO is convinced it complies with the European Standards
and Guidelines, Parts two and three, from ENQA. NVAO leaves
it to the review panel to judge on NVAO’s degree of compliance
with each ESG and in the final conclusion.
Regarding two standards of the ESG NVAO would like to add
the following considerations.

ESG 2.6: Follow-up procedures
The accreditation system in the Netherlands and Flanders
does not include a formal follow-up procedure. NVAO is
­confident it fully complies with what it believes is the core of
the guideline in this respect – the transparent evaluation of
the implementation of recommendations made by review
panels, also in the case of positive accreditation decisions. In
case of negative or conditional positive decisions of NVAO the
follow-up ­procedure is clear, as is outlined in chapter 3.6. In
case of an unconditional ­positive decision it is the responsibility of the institution and/or the programme to implement
the recommendations of the panel. In the following assessment and accreditation procedure the panel will assess
whether the recommendations are implemented adequately.
­Accreditation decisions are taken periodically. In this kind
of system a formal approach regarding the implementation
of recommendations of accredited programmes cannot
operate well. For reasons of trust this should be left to the
accredited programme.
The new accreditation system in the Netherlands offers the
possibility of conditionally satisfactory in the procedures
for institutional audit and initial accreditation. In addition,
NVAO decides on the length of the recovery period, on the
basis of a recovery plan that guarantees the follow-up of
recommendations.

ESG 2.8: System wide analyses
Up to now NVAO completed several comparative analyses
within a cluster or domain, as is described in chapter 3.8
and therefore NVAO complies strictly speaking with this
standard. This does not preclude that NVAO has to work
on implementing its new policy regarding system-wide and
comparative analyses.

36

S e lf -eval u at i o n re p or t NVAO 2012

A good step is the system-wide evaluation in the Netherlands
and Flanders of the operation of the accreditation system in
the second phase.

Overall conclusion
NVAO is of the opinion that it is compliant with the Standards
and Guidelines of ENQA, while it is aware of points of
­improvement regarding its work processes.

Strengths and points of attention regarding NVAO’s operation

37

Chapter
6
Introduction
to the higher
education
systems in
the
Netherlands
and Flanders

This chapter gives an outline of the higher education systems
in the Netherlands and Flanders. First a description will be
given of the types of institutions and programmes, and
secondly some key figures on higher education will be quoted.

In the near future a new degree will be introduced:
the Associate Degree as a short cycle part of the bachelor’s
programme with professional orientation (expected in 2013).
Currently Associate Degree programmes are offered as pilots.

6.1  ⁄ Higher education in the Netherlands

6.2  ⁄ Recent developments in higher
education in the Netherlands

Two types of institutions offer higher education programmes:
universities of applied sciences (41)31> [hogescholen] and universities (14), which both are publicly funded. They can offer
programmes with an academic or a professional orientation.
However, academic programmes are mostly offered by universities
and professional programmes by universities of applied sciences.
Besides these regular public institutions, there are recognised
private institutions in higher education [Rechtspersonen voor
hoger onderwijs]. Recognised private institutions do not
receive public funding. However, after having completed
the special procedure Legal body higher education, these
­institutions are allowed to apply for and to offer accredited
bachelor’s and master’s programmes. Currently about 60
recognised private institutions offer accredited higher
­education programmes in the Netherlands.
Students who wish to enter higher education generally need
a degree issued by one of the following types of secondary
education:
1. VWO (pre-academic education), the highest level of
secondary education, required for being admitted to
universities;
2. HAVO (higher general secondary education) and
3. MBO-4 (the highest level of middle professional education):
both give access to enter higher professional education.
In the Netherlands, higher education consists of three cycles:
bachelor, master and PhD level. Bachelor’s and master’s
programmes can be offered by universities of applied sciences,
universities and recognised institutions. Only universities are
allowed by law to award PhD degrees.
Table 2

Orientation / level

Bachelor Master

Research Doctorate
Master

Programmes with an

wo-

wo-

wo-

academic orientation

bachelor

master

master

Programmes with a

hbo-

hbo-

professional orientation

bachelor

master

PhD

In 2011, the Dutch Government issued a new strategic
­statement for higher education3 2 > , which emphasizes the
need for a more ambitious study climate in higher education,
with a focus on excellence. The new strategy imposed a
number of requirements on institutes in higher education.
Universities need to reinforce the relation between research
and education and develop a sharply distinguished profile.
Universities of applied sciences are required to focus on the
development of research and entrepreneurial skills in the
programmes, on upgrading the achieved learning outcomes
of the professional bachelor’s programmes and a master’s
degree for all educational staff in 2020 (already 80% in 2016).
A National Qualification Framework for the entire educational
sector (NLQF) was finalized in the Netherlands in 2011.
This followed the development and acceptance of the
­European Quality Framework in 2008. As a result of the
Bologna agreement, the National Qualification Framework
for higher education (NQF HE) was approved in 2009. It is
compatible with the Qualification framework of the European
Higher Education Area (QF EHEA). NVAO monitors the NQF
higher education framework in the Netherlands.

6.3  ⁄ Higher education in Flanders
In Flanders, higher education is provided by institutions
­registered in HOR. Only these institutions can offer bachelor’s
and master’s programmes. There are two categories:
> Statutory registered institutions. These were already
­recognised by the Flemish or Belgian government before the
introduction of the bachelor’s and master’s degree system
in 2003. All these institutions receive public funding for
education and research. There are four different types of
statutory registered institutions: universities (6), university
colleges (21), postgraduate institutions and protestant
faculties.
> Registered institutions. Since 2004 some new providers
have successfully completed a procedure for registration
and, consequently, were recognised as higher education
institutions by the Flemish government.

32>	[Kwaliteit in verscheidenheid, Strategische agenda hoger onderwijs,­
31 > Universities of professional education of significant size present
­themselves abroad as Universities of Applied Sciences. Dutch
­professional higher education counts 41 such institutions.

onderzoek en wetenschap, Ministerie van OCW, Den Haag, augustus 2011].

Introduction to the higher education systems in the Netherlands and Flanders

39

Universities have the authority to award academicallyoriented bachelor’s and master’s degrees and doctoral
degrees. University colleges have the authority to award
professionally-oriented bachelor’s degrees and in association3 3 > with a university academically-oriented bachelor’s
and master’s degrees. The Flemish government intends to
incorporate these programmes within universities. The
Flemish system does not offer a professional master’s degree.

Access to higher education
In Flanders, the following secondary school diplomas or
­certificates give access to higher education:
> secondary school leaving diploma
> certificate of short higher education courses with full
­curriculum or;
> a diploma or certificate recognised as equivalent under a law,
decree, European directive or other international agreement.
Higher education institutions may adopt special admission
requirements to admit persons who cannot meet the general
admission requirement. These individual admission decisions
must be based on: the individual education level of the
student, humanitarian grounds; medical, psychological or
social grounds, assessed by the higher education institution.
Higher education in Flanders consists of three cycles:
­bachelor’s, master’s and PhD level.
Table 3

Orientation / level

Bachelor

Master

Doctorate

Academically

academische

master

PhD

oriented Programmes

gerichte bachelor­

master na

opleiding

master

Professionally-

professioneel

oriented Programmes

gerichte bachelor­
opleiding
bachelor na bachelor

A new degree has been introduced and will come into effect
in the near future, which is linked to the short cycle
programme HBO5. Like the Associate degree programme in
the Netherlands, HBO5 is a short cycle programme on level
5 of the European Qualification Framework.

33 > An association is a legal body in which the cooperation between a
­university and one or more university colleges is officially established.

40

S e lf -eval u at i o n re p or t NVAO 2012

6.4  ⁄ Recent developments in higher
education in Flanders
Recent developments in Flemish higher education are partly
directed by the policy statement 2009-2014 of the Minister
of Education, Youth, Equal opportunities and Brussels3 4 > .
From this statement several pending issues and new
­initiatives in Flemish higher education can be derived.
> A major topic in Flemish higher education for years is the
‘academisation’ 3 5 > of academic programmes in university
colleges. In this regard it can be mentioned that the
academic programmes in arts will be offered by specific
Schools of Arts within the university colleges. The legislation
allowing these changes should be ready in 2012.
> An ongoing debate in Flanders is the extension of academic
master’s programmes with a duration of one year to a length
of two years. Recently (November 2011) the universities and
university colleges offered their proposals to the Flemish
Minister. A final decision remains to be taken in April 2012.
The institutions have to finance the development of these
programmes for the longer duration themselves, because
the government will not offer extra funding.
> A related topic is the discussion about the reduction of the programme Medical training to six years (now seven) in order to be
equivalent to similar programmes in other European countries.
> A recent and significant development in Europe is the growing
importance of short cycle higher education. In Flanders the
HBO5 programmes can be provided in the future by university
colleges or regional centres for adult education. These
programmes are considered to lower thresholds for enrolment in higher education. NVAO supported the preparation of
HBO5 by organising a pilot in 2010 to assess potential HBO5
programmes. As a result of the pilot a report of learning
effects36> was produced and discussed with all stakeholders.
> To increase participation in higher education new
­arrangements for disadvantaged groups will be created.
> A last objective is related to Internationalisation. In 2016
at least 15% of the graduates of bachelor’s and master’s
programmes should have completed a learning experience
abroad of at least three months. Either a practical training
in the professional field or a couple of courses or a part of a
programme at a foreign institution for higher education.

3 4 > Policy Statement 2009-2014 of the Flemish minister of Education,
November 2009, [Beleidsnota 2009 - 2014, Onderwijs, Samen grenzen
verleggen voor elk talent, Pascal Smet, Vlaams minister van Onderwijs,
Jeugd, Gelijke Kansen en Brussel].
3 5 > Academisation is the development of certain programmes from university
colleges to academic programmes within an association, which is a legal
body in which the cooperation between a university and one or more
university colleges is officially established.
3 6 > Pilot report Learning effects HBO5, NVAO, December 2010,
­[Leereffectenrapport proefprojecten HBO5, NVAO].

National Qualification Framework and Domain
Specific Learning Outcomes
The National Qualification framework of higher education
(NQF HE) was approved by the Flemish government on 18 July
2008. Related to the Flemish NQF HE are the domain specific
learning outcomes, which are developed by all suppliers of the
same kind of programme and which must be validated by
NVAO for the levels 6 and 7 of NQF. Level 5 is validated by
AKOV3 7 > . NVAO was an observer in the process of making the
manual on how to write domain specific learning outcomes.
As a result, NVAO produced a procedure concerning validating
such learning outcomes of new and existing programmes on
levels 6 and 7. Validated learning outcomes are automatically
recognised by the Flemish NQF and replace the prepared
domain specific frameworks by assessment committees.

6.6  ⁄ Some key figures in higher education in the Netherlands and Flanders
From 2005 to 2010 enrolment in higher education in the
­Netherlands grew from about 560.000 to 635.000 students,
in Flanders enrolment grew from about 160.000 to 195.000
students in this period.
In 2005 35% of the labour force between 27 and 34 years
old in the Netherlands held a higher education degree, in
Flanders this percentage was higher at 40%. Both countries
performed better than average in Europe. Short programmes
in higher education in both countries are not included in
the percentages. These percentages are expected to grow
further4 0 > . Table 5 shows several figures at a glance.
Table 5 ⁄ Overview of (rounded) key figures of higher education in the

The National Qualification Framework (Het Vlaams Kwalificatie­
raamwerk) for the entire education sector in Flanders was
launched on 30 November 2009. An important step, because
application of the NQF makes it possible to compare all individual, acquired qualifications and competences in educational
programmes, jobs and other societal activities. The NQF is an
important instrument in lifelong learning. NVAO has been in­volved
in the development of the NQF and was a member of the Steering
Group of the procedure that connected the NQF to the EQF.

Netherlands and Flanders (2009-2010)

6.5  ⁄ Overview of the Dutch and Flemish
degree programme

Recognised private institu-

The Dutch and Flemish types of programmes described in the
preceding paragraphs are shown in table 4.

Other statutory registered

Variables / Countries / Total

The Nether-

Flanders

Total

lands
Inhabitants

16.750.000

Universities

14

6.250.000 23.000.000
6

20

41

21

62

60

7

3500

1400

4900

195.000

830.000

Universities of applied
sciences (NL) and University
colleges (FL)
tions (NL) and Registered
institutions (FL)
institutions (FL)
Programmes

5

Table 4 ⁄ Matrix of the degree programmes in higher education of the

Number of students

635.000

Netherlands and Flanders

Professional education

400.000

90.000

490.000

Academic education

235.000

105.000

340.000

Level - EQF

Associate

Orientation

degree / HBO5 (5)3 8 >

Bachelor (6)

Master (7)

Male vs. Female students

48 vs. 52% 46 vs. 54%

Attachment 9 contains these figures more in detail.

Academic
orientation (the
60 or 120 or 180

Sources

-

180 ECTS

or 2403 9 > ECTS

120 ECTS

240 ECTS

60 to 90 ECTS

Decreet van 4 April 2003 betreffende de herstructurering van
het hoger onderwijs; Flemish education in figures, 2009-2010,
Flemish authorities Education and Training policy area;
www.hbo-raad.nl; www.vsnu.nl; www.vlir.be; www.vlhora.be;
www.nvao.net; Jaarboek Onderwijs in cijfers CBS, 2009,
2010 en 2011

Netherlands
and Flanders)
Professional
orientation (the
Netherlands)

180 ECTS

Professional

Advanced: 60

orientation
(Flanders)

90 or 120 ECTS

ECTS

37 > AKOV [Agentschap Kwaliteit Onderwijs en Vorming] is the new agency
within the department of education to deal with (recognition of)
­qualifications and quality from EQF level 1 to 5.

3 8 > Will be formalised in the Netherlands and Flanders in the next years
3 9 > Just some medical programmes have a study load of 180 or 240 EC.
In the near future 90 ECTS will be possible in FL.
4 0 > OESO, Education at a glance, 2007.

Introduction to the higher education systems in the Netherlands and Flanders

41

Chapter
7
The external
quality
assurance in
the
Netherlands
and Flanders

The current accreditation system in the Netherlands and
Flanders is described and evaluated below.
Until the end of 2010 the accreditation system in the
­Netherlands was based on programme accreditation, peer
review and the opportunity for programmes to apply for
distinctive (quality) features. In Flanders this system is still
current until the end of the academic year 2012-2013.
From 1 January 2011 a new accreditation system came into
force in the Netherlands. It is expected that a roughly similar
system will be implemented in Flanders in the academic year
2013-2014.
At the core of both the previous and the new system is the
assessment and accreditation of programmes. Institutions
prepare a self-evaluation report and quality assurance
­agencies assess the existing programmes of higher education
institutions on the basis of the NVAO accreditation framework
and procedures. Most agencies apply their own procedural
handbook or protocol for the assessment procedure. In Flanders,
the ­institution applies for accreditation of a programme with
NVAO, submitting the assessment report of the panel
produced and published by the evaluating agency. In the
­Netherlands the assessment report is published after
completion of the accreditation procedure.
NVAO checks the quality of the assessment procedure and the
quality of the programme according to the assessment report.
If this report leaves questions unanswered, NVAO requests
additional investigation. Finally NVAO takes an accreditation
decision.
The initial accreditation procedure is carried out entirely by
NVAO, appointing the panels that assess a new programme on
the basis of the framework for initial accreditation and submit
their report to NVAO.
Before developing a new system in the Netherlands and
­Flanders NVAO decided to thoroughly evaluate the previous
accreditation system in both countries. After having discussed
the experiences on accreditation with the external stakeholders in 2007, NVAO proposed a draft model for the new
system, combining institutional audits and programme
accreditation. The draft model was discussed during an
­international seminar in January 2008 attended by partner
accreditation organisations in Europe that already gained
experience with a mixed accreditation model of institutional
audit and programme assessment. Subsequently NVAO
discussed the draft of the new system with external stakeholders in Spring 2008 and started a pilot to test the draft in
Autumn 2008. The pilot consisted of an institutional audit and
a limited assessment of a programme (focusing on the quality
of content and the achieved learning outcomes) of the same

institution. Nine institutions participated in this pilot, six from
the Netherlands and three from Flanders. The pilot was an
important learning experience for the institutions involved
and for NVAO. The results were presented and discussed on
special conferences for all stakeholders organised by NVAO
in March-April 2009. Afterwards NVAO presented a definite
proposal to the ministers of Education in the Netherlands
and Flanders.
Based on the experiences gained during the pilot and the
support of stakeholders a new system of external quality
assurance was introduced in Dutch higher education on
1 January 2011 (one year later than intended). In Flanders
the new system will start in the academic year of study
2013-2014.
Besides the use of less but more open standards, another
novelty of the new system is the institutional audit. In the
Netherlands an audit committee assesses whether the
­institution is ‘in control’ of the quality and the achieved
learning outcomes of its programmes. In order to answer this
question the audit committee assesses the adequacy of the
strategy, policy and educational processes in the institution.
During the institutional audit at least two audit trails are
executed. Institutions that pass an institutional audit
successfully or conditionally with an improvement assignment are entitled to submit limited programme assessment
(LPA). In the case of a conditional decision, the institution has
to comply with a limited set of conditions within three years. If
a negative decision has be taken, all institution’s programmes
that were assessed in the limited mode must undergo an
additional, extended assessment within one year. Institutions
that do not opt for an institutional audit can only submit
application based on extensive programme assessment (EPA).
Flanders still has to decide about some elements of this
system, but it seems clear already that the institutional
review and LPA will be compulsory, although in the first round
without formal consequences for the result of the institutional
review. This first round is meant as an important learning
exercise and will be evaluated mid-term (2017-18) and at
the end (2021) in order to give time to work out the next
accreditation system.
The new accreditation system offers more opportunities for
institutions and programmes to acquire recognition for higher
quality or a specific profile. In the Netherlands, the final
judgement is based on a four-point scale (Excellent, Good,
Sufficient and Insufficient); this scale is applicable both on
the level of a study programme and on the level of the
­separate standards of the framework. The new accreditation
The external quality assurance in the Netherlands and Flanders

43

system contains a recovery period and an elaborated
­procedure for initial accreditation in the Netherlands; if a
new programme is approved, NVAO can restrict the duration
of the initial accreditation and subject it to conditions that
have to be met within a fixed period.
The Flemish government still has to decide on the application
of the four-point scale on the level of the study programme.
An approved new programme is accredited for the length of
the programme and two years (e.g. the duration of a professional bachelor programme is three years, so the period of
approval is five years). And as mentioned earlier, a recovery
period is already part of the Flemish accreditation system.

7.1  ⁄ Self-evaluation
The old and new accreditation systems both in the
­Netherlands and Flanders can be summarized in the
table below.
Table 6

Item

Old system

New system NL

New system FL

Accreditation unit

Study programme

Study programme

Study programme

Quality assurance

Study programme

Institution

Institution

# Standards

6 with 21 underlying aspects

LPA : 3 (open)

Study programme : 3

Clustered assessment
Assessment organisation

EPA : 16

(4)4 1 > (open)

Institution : 5

Institution : 5

No clustering (NL prof)

Clustered assessment

Clustered assessment

Clustering (NL ac. & FL)

of programmes

of programmes

Initial : NVAO

Initial : NVAO Study programme :

Initial : NVAO

Study programme : QAA

NVAO orders QAA

Study programme: VLUHR4 2 >

Institution : NVAO

or EQAR/foreign QAA
Institution : NVAO

Panel

Completely QAA-controlled

NVAO-controlled

QAA-controlled with NVAO advice

Site visit

2 days

1 day+prep. meeting

day+prep. meeting

Report

public

public

public

Possibility (not FL) to disapprove

Possibility to disapprove

Possibility to disapprove

Recovery period

(not NL) Recovery period

Recovery period

Recovery period

Doubt

Further information Executive

Further information Conditional

Further information

arrangement (NL) Hearing (FL)

accreditation

Not official

After recovery period After

Follow-up

After recovery period

conditional decision
System-wide analysis

Only as additional and
voluntary task

41 > Four standards for only registered institutions adding internal QA.
42 > VLUHR [Vlaamse Universitaire en Hogescholenraad]. VLUHR is a merger
organisation of the former quality assurance organisations of VLIR and
VLHORA, which both are EQAR registered.

44

S e lf -eval u at i o n re p or t NVAO 2012

Together with inspection

Evaluation system-wide analyses
in decree

The previous accreditation system is generally accepted
in both countries. The quite open frameworks are widely
appreciated. These frameworks provide assessment panels
with the required room to apply the peer review to its full
extent. The system is built on programme accreditation,
which is to the satisfaction of most stakeholders, especially
the government and students.
On the other hand, the previous system shows some
­weaknesses as well. In the opinion of the institutions the
administrative burden of programme accreditation is too
high. For each programme common institutional aspects,
like facilities and internal quality assurance, which are often
managed at a higher level in the institution, had to be
assessed separately. Such assessments were considered
excessively formal and, as a consequence, educational staff
felt less involved in the assessments.
NVAO never intended the frameworks and procedures to
oblige programmes to produce a lot of documentation for
the assessments. To the contrary, NVAO encourages the
programmes to deliver concise self-evaluation reports
supported by a limited number of appendices. Programme
management often feels uncomfortable with a concise
­self-evaluation and compensate their uncertainty by
producing heaps of documents for the panels.
Another weakness is the position of quality assurance
­agencies in the Netherlands. These agencies are private
organisations that are commissioned by the institutions.
This position has raised questions about the required
­independency of the agencies and their panels.
A complicating feature was the lack of a recovery period in the
Netherlands under the previous system, which put the panels
under extra pressure to formulate a positive assessment.
The consequences of a negative assessment – cessation of
public funding and no admission of new students - are very
severe and can be fatal for a programme. As a consequence,
some panels tended to write less convincing reports with a
positive conclusion when assessing weak programmes.
In some cases NVAO had to carry out additional assessments
to verify the panel assessment and the quality of the
programme. Sometimes the panel report was rejected. It is
obvious that additional assessments (or additional questions)
are considered as administrative burden by the institutions.
But to guarantee quality assurance NVAO had to apply these
steps occasionally.

In the new accreditation frameworks, the institutional audit
will focus on aspects that are defined at institutional level and
includes organisational processes on programme level.
Programme assessments will focus on content and results
through the achieved learning outcomes. This combination
should lead to a reduction in administrative burden.
­Institutional policy and processes within programmes are
not repeatedly assessed in this model. The reintroduction of
the cluster approach in the professional higher education
sector in the Netherlands contributes to the consistency of
the assessment of individual programmes. The introduction
of a recovery period improves the objective judgement of the
panels. The independent status and the quality of the panels
has to improve as well, since the criteria are stricter, with a
stronger focus on international and educational experience
of the panelists. Moreover, in the near future the panels in the
Netherlands will be commissioned by NVAO instead of by the
institutions. The first indications of the effects of the new
Dutch system are positive. NVAO already received a number
of very well-written and well-argued reports and some with
a negative judgement, which might be a first indication that
panels pass stricter judgements, without the burden of a
possible shutdown of the programme.
As mentioned, the previous system is still running in Flanders.
New elements in the Dutch system are already part of the
system in Flanders, such as a recovery period and the cluster
approach in both sectors of professional and academic
education. In the past the quality assurance agencies in
­Flanders were part of the umbrella organisations of the
universities (VLIR) and the universities of applied sciences
(VLHORA). But in 2011 these agencies merged into VLUHR
(Vlaamse Universiteiten en Hogescholenraad) partly in order
to perform their external quality assurance activities separated from the umbrella organizations VLIR and VLHORA.
This separation should guarantee that the external quality
assurance system operates independently from the umbrella
organisations VLIR and VLHORA. Nevertheless in the new
Flemish accreditation system institutions will also be able to
work with foreign QAA’s as long as these are EQAR-registered
or recognized by the NVAO.
Procedures used by VLUHR are based upon the NVAO
­accreditation framework and procedures. The independence
of the assessment committees is guaranteed by the
­Recognition committee [Erkenningscommissie] of the Flemish
government. As this will be changed in the new accreditation
system VLUHR will only decide on the composition of the
assessment panel after an NVAO advice. VLUHR plays an
important role in the composition of panels. As this
­organisation is leading in the programme assessment
The external quality assurance in the Netherlands and Flanders

45

process, the added value of NVAO in the accreditation process
is not immediately clear for all institutions and this obliges
NVAO to communicate clearly about its role in Flemish higher
education.
NVAO organises periodic conferences and visits to improve
the involvement and consent of the Flemish institutions. Both
instruments were applied once again in autumn 2011 as NVAO
organised its Flemish conference in November in Brussels
and visited all institutions to evaluate the experiences of
the institutions with NVAO. Both activities produced useful
information to improve the operation of NVAO in Flanders
and are explicitly included in NVAO’s new strategic policy.

7.2  ⁄ Conclusion
NVAO has positive expectations about the results of the
new accreditation system in the Netherlands and Flanders.
Lessons are learned from the first accreditation cycle in both
countries and are translated into an improved system that
responds well to the needs and the current achieved level
of internal quality assurance in the Dutch and Flemish
­institutions. The systems offer an adequate challenge for
the institutions to work further on internal quality assurance
on institutional and programme level.
The systems in the Netherlands and in Flanders will be quite
equivalent, with some clear differences, but after all and in
practice more similar than in the first cycle. As a result of
the distrust that followed recent quality incidents in Dutch
professional higher education, the initial openness of the
new Dutch system was restricted by additional requirements.
In Flanders the involvement of the higher education department in the Bologna transparency agenda resulted in an effort
to restrict the openness of the standards with underlying
specific criteria and proofs. The talks to come to a consensus
on the Flemish framework opened up the standards again.
The first audit reports from the Netherlands demonstrate that
these audits have a considerably wider scope than just the
technicalities of the institutional internal quality assurance
system. The audits start from the strategic policy of the
­institution, which is precisely the focus of the Flemish audits.
Next to the recovery period it will be possible to reject an
assessment report both in the Netherlands and Flanders.
Remaining issues in both countries are the reduction of
administrative burden in the system and the consistency
and fairness of the panels.
Even before the new Dutch system was introduced, the
­Netherlands and Flanders have learned from each other for
the better. In the Netherlands the test of the macro-efficiency
46

S e lf -eval u at i o n re p or t NVAO 2012

of a new programme has been placed before the assessment
of initial accreditation, following the example of the order
of these procedures in Flanders. The other way around,
­Flanders has learned from the Netherlands to introduce open
standards and to move the authority to impose a recovery
period from the government to the NVAO. The differences that
will remain in the new accreditation system are the result
of the different degrees of trust in the quality of higher
­education and of cultural differences. Yet, the ultimate aims
of the systems are similar, and the differences provide interesting insights and deserve being investigated in order to
learn from each other.

The external quality assurance in the Netherlands and Flanders

47

Chapter
8
Operation of
NVAO

8.1  ⁄ Quantitative results
The quantitative results of the accreditation process in the
Netherlands and Flanders until 1 January 2012, are presented
in table 7.
Table 7

Accreditation results

The Netherlands

Flanders

3982

864

- Regular

3385

772

- Research Masters

233

Number of applications
(Initial) accredited programmes

Negative decision
- Regular programmes

21

- Research Masters

7

38

Panel report disapproved
- Regular programmes

17

Application withdrawn
- Regular programmes

274
45

Total - Not accredited

364

NVAO is established as a binational accreditation organisation by treaty in 2003 by the Dutch and Flemish Education
ministers. The Treaty assigns the tasks of NVAO, its form of
administration and its supervision.
At the time of writing (2011) adjustment of the Treaty to recent
developments is prepared. Since two years the ministers of
Education from the Netherlands and Flanders discuss the
possibility of incorporating the mutual recognition of
diplomas in the Treaty. A long expected and natural step would
be finalized in that case.
Another reason to adjust the Treaty is the recent constitutional merge of the Caribbean islands Bonaire, Eustatius
and Saba with the Netherlands. Consequently, NVAO will be
authorized to act on these islands in the near future.

8.4  ⁄ International assignment of NVAO

- Research Masters

- Research Masters

8.3  ⁄ The treaty between the
­Netherlands and Flanders

54

92

As mentioned, the first phase of the accreditation system in
Flanders runs till the end of the academic year of study
2012-2013.
Apart from the activities concerning accreditation, NVAO
received more than 400 applications for additional tasks.
These tasks are specified in attachment 2.
Results regarding the internationalisation task are presented
in chapter 8.4.

8.2  ⁄ The assignment of NVAO
The primary assignment of NVAO is to accredit existing
programmes in higher education (accreditation) and to assess
new programmes (initial accreditation) in the Netherlands and
Flanders. NVAO can be requested to carry out additional tasks
by the Ministers of higher education in both countries if these
assignments support or supplement NVAO’s primary assignment.4 3 > Secondly NVAO is assigned to play an important role
in internationalisation (see chapter 8.4).

From the outset in 2003, NVAO was assigned the task to play
an important role in internationalisation. In order to fulfil
this international role adequately, NVAO formulated five
­long-term objectives that outline its international policy.
These objectives are:
> Playing an active membership role in international networks
of accreditation and quality assurance organisations
(INQAAHE, ENQA, ECA).
> Facilitating the profiling of the Dutch and Flemish
­accreditation and higher education systems so as to
strengthen the international position of the Dutch and
Flemish higher education institutions.
> Cooperating with other accreditation organisations in order
to achieve mutual recognition of accreditation decisions.
> Contributing to the creation of a European Qualifications
Area where the competent authorities automatically
­recognise degrees from accredited programmes and
­institutions.
> Pro-actively following up on international developments
in quality assurance and higher education.
The department Internationalisation of NVAO works
­continuously on achieving these objectives. Operational
targets and activities relating to each objective are formulated
in an action plan.

43 > The Treaty, page 3.

Operation of NVAO

49

Below the activities of the department are described shortly.

1. Active membership of international networks
in accreditation and quality assurance
NVAO is an active member of three international networks
of accreditation and quality assurance organisations:
> International Network for Quality Assurance Agencies in
Higher Education (INQAAHE)
> European Association for Quality Assurance in Higher
Education (ENQA)
> European Consortium for Accreditation in higher education
(ECA)

International Network for Quality Assurance Agencies
in Higher Education (INQAAHE)
INQAAHE is a worldwide organisation for quality assurance
agencies (www.INQAAHE.org). NVAO coordinates the INQAAHE
Secretariat and the NVAO Vice-Chairman is as Secretary also
Board member of INQAAHE. In the autumn
of 2011 NVAO has organised a INQAAHE seminar jointly with
ENQA. Almost 120 participants from all continents were
present at this three-day seminar in Brussels.

European Association for Quality Assurance in higher
education (ENQA)

2. Facilitating the international profiling of the
Dutch and Flemish accreditation and higher
education systems
NVAO representatives took part in numerous international
conferences and workshops in which they provide active
contributions. The Dutch-Flemish accreditation system and
the binational cooperation clearly set an example for other
countries, considering the large number of international
­delegations that visited NVAO in the recent years. In 2010 and
2011 NVAO welcomed delegations from 31 countries from all
over the world (Armenia, Aruba, China, Curaçao, Cyprus,
Denmark, East-Timor, Ethiopia, Finland, France, Germany,
Ghana, India, Indonesia, Japan, Kirgizstan, Lithuania, Norway,
Peru, Poland, Russia, Saudi-Arabia, Slovakia, South Korea,
Suriname, Sweden, Switzerland, Tadzhikistan, Taiwan,
­Trinidad and Tobago).
NVAO participated in the international project on learning
outcomes CoRe2 and has presented its views and approached
at international seminars and in the ECA Working Group 4.
In 2009 NVAO prepared a framework for a distinctive feature
internationalisation, which was tested in 2010 in a pilot in
which 12 institutions and 21 programmes in the Netherlands
and Flanders participated.

Members of ENQA are quality assurance agencies
from ­countries that are party to the Bologna process
(www.enqa.net). The NVAO Chairman is also a ENQA Board
member. NVAO participates in the annual General Assembly
and is as a rule present at ENQA Workshops. Several NVAO
policy advisers have followed the ENQA reviewers training and
presented at ENQA workshops or seminars. From 2007 to
2011 NVAO was represented in the steering group of the
Internal Quality Assurance network of ENQA. The yearly
seminar of this network was organised by NVAO in The Hague
in 2009.

3. Mutual recognition of accreditation decisions

European Consortium for Accreditation in higher
education (ECA)

In the period 2005-2006, organisations responsible for
­recognition of foreign qualifications and the accreditation
organisations from the Netherlands and Flanders, Norway,
Austria, Poland and Switzerland signed the Joint Declaration
concerning the automatic recognition of qualifications.
The declaration is based on mutual recognition of
­accreditation decisions and on the implementation of
compatible national qualification frameworks in order to
achieve the automatic recognition of qualifications.

The ECA network currently consists of sixteen accreditation
organisations from ten European countries
(www.ecaconsortium.net). NVAO coordinates the ECA
­Secretariat since the start in 2003. The NVAO Chairman is
also Vice-Chair of ECA. NVAO participates in all ECA events,
including the annual plenary ECA Workshop, and plays an
active role in the four working groups of ECA, chairing two of
them. The winter seminars of ECA were organised by NVAO
in the past years.
The ECA projects TEAM II (2008-2010), JOQAR (since 2010) and
E-Train (since October 2010) are coordinated by NVAO.

50

S e lf -eval u at i o n re p or t NVAO 2012

Mutual recognition of accreditation decisions is one of NVAO’s
main objectives. Mutual recognition of accreditation decisions
facilitates the recognition of degrees and international
mobility.
NVAO signed six bilateral agreements on mutual recognition
of accreditation decisions with agencies in other European
countries and one multilateral agreement with five other
European partners on the mutual recognition of accreditation
of joint programmes (December 2010).

4. Automatic recognition of qualifications

5. P
 ro-actively following up on international
developments in quality assurance and higher
education
NVAO maintains excellent contacts with the appropriate
government officials in the Dutch and Flemish ministries and
with the appropriate members of the international network
organisations.

Other additional tasks of NVAO are the assessment of domain
specific learning outcomes in Flanders and the monitoring
of NLQF, both are described in chapter 6. In attachment 2
the additional tasks of NVAO are described more in detail.

Other international activities that the Dutch government
assigned to NVAO were the assessment of programmes
offered on the former Netherlands Antilles and Aruba.
NVAO also participates in international projects in order to
follow up international developments in specific fields. In
2010 and 2011 NVAO supervised university colleges and their
umbrella organisations in Croatia to built an accreditation
system. NVAO will supervise a similar project in Armenia in
2012.
Finally NVAO coordinated the external review of ENQA of
the accreditation organisation HSV in Sweden in 2012.

8.5  ⁄ Additional tasks
Apart from its primary responsibilities (i.e. (initial)
­accreditation), NVAO has been charged with several additional
tasks over the years. These tasks have to be approved by the
Committee of Ministers or the respective departments of
Education and should be compatible with NVAO’s mission.
For these tasks, extra funding is made available. In NVAO’s
Strategic Policy Statement it is set down that new tasks
should have an obvious connection with quality assessment
of programmes.
NVAO’s most important additional tasks are the assessments
of applications for:
> Associate degree programmes (up till now 300 applications
over the past years);
> Open system (5 applications);
> Room for Talent (11);
> Recognition of private higher education institutions (15);
> Broadened programmes (30);
> Training Schools (83);
> Educational Minors (10);
> Academic Teacher training primary education (6);
> Management of the official Register of Higher Education
of Flanders (hogeronderwijsregister);
> Academisation (140);
> HBO5 (5);
> Extension of academic master’s programmes (29).

Operation of NVAO

51

Chapter
9
Overview of
actions taken
since the
previous
external
review 2007

9.1  ⁄ Follow up of recommendations
regarding NVAO
In the assessment report of the external review in 2007 the
committee suggested several recommendations regarding
the operation of NVAO. In this chapter the follow-up by NVAO
on these recommendations will be discussed shortly.
The review committee formulated the following
­recommendations.

Recommendation on ESG 2.8

in the Netherlands. An international comparative analysis
on the quality and level of three professionally oriented
programmes in the Netherlands and Flanders together with
two foreign countries or regions is planned.
Another kind of comparative analysis concerns the
­professional master programmes in the Netherlands in 2007.
At that time NVAO had assessed these programmes for three
years. As professional master programmes are a new type
of programme in Dutch higher education since 2004, an evaluation was useful to share first experiences and good practices.

NVAO is advised to give more attention to the production of
system-wide and comparative analyses – also with regard to
the functioning of the accreditation procedures. More attention
for these analyses will be beneficial for the ‘information
­function’ of accreditation, as expressed by umbrella
­organizations of institutes and by student organizations.

The evaluation of the former system and of the pilots in
order to develop the next system are good examples of
system-wide analyses of the system.
In attachment 1 the completed comparative and ­­
system-wide analyses are described in more detail.

During the last few years NVAO completed a substantial
number of comparative analyses. In several cases the
­analyses were ordered by the executive board of NVAO.
A well-executed analysis can support and validate the
­decision-making of NVAO in case of single applications,
and it provides also a solid basis to start a discussion with a
cluster of programmes about quality assurance and quality
­enhancement. In other recent cases NVAO was asked by (one
of) the stakeholders to present a comparative analysis of a
specific programme cluster and a discipline or domain.
The comparative analyses of the Teacher training (cluster)
and the Arts programmes (discipline or domain) are good
examples of such requests. In both cases the Dutch Minister
issued the request.

To strengthen its accountability procedures regarding the
­relation with stakeholders and to focus in its internal quality
assurance system and to include relevant stakeholders in
the general board that are not represented yet.

A short overview of the main comparative analyses in the last
few years includes:
> Academic programmes Linguistics in NL (2007);
> Academic programmes Sociology in NL (2007);
> Professional bachelor programmes Industrial Engineering
in NL (2009);
> Academic programmes Educational Sciences in NL and FL
(2009);
> Professional bachelor programmes Teacher training primary
education in NL (2009);
> Professional bachelor and master programmes Fine &
Performing Arts (2010) in NL and
> Academic research master programmes in NL (2012).
Several comparative analyses are still in progress, for
instance the professional bachelor programmes Human
Resource Management, Communication and Social Work

Recommendation on ESG 3.8

Since February 2010 a former president of the European
Students Organisation is a member of the General Board of
the NVAO. A new member of the professional field is present
as well. Members of the General Board however do not
­represent a sector or field, but are nominated purely on
­individual merits and operate purely in their personal name.
The Advisory Council of NVAO includes students since 2007.
Representatives of the student unions ISO and LSVB (in the
Netherlands) and VVS (in Flanders) have joined the Advisory
Council for some years now.
Recently, in 2011, a resonance group was established to
­evaluate the introduction and the performance of the new
system in the Netherlands. Representatives of institutions,
umbrella organisations and students are the main members
of this board.
In Flanders a four-party platform has been introduced.
­Representatives of the Flemish cabinet, the higher education
department of the ministry, VLIR and VLHORA (the umbrella
organisations of the institutions) and NVAO meet on a regular
basis to discuss matters of quality assurance and the
­preparation of the new accreditation system in Flanders.
Representatives of VLHUR, being the new Flemish QAA
merger of the former quality departments of VLIR and
VLHORA, and VVS are invited when appropriate.
Overview of actions taken since the previous external review 2007

53

NVAO focused in structuring its Internal Quality Assurance
system since the last review in 2007. In chapter 4.8 the main
characteristics, strengths and weaknesses of the current
operational system are evaluated.

9.2  ⁄ Follow up of recommendations
regarding the accreditation system
The review committee of 2007 recommended on several
issues regarding the higher education system and the
­accreditation system in the Netherlands and Flanders.
As the accreditation system in the Netherlands and Flanders
as such is not subject of this review the follow-up on the
recommendations are only described here in relation to a
first reflection on the new accreditation system (see also
chapter 7) and more specifically, in attachment 10.

All in all, the conclusion of the self-evaluation of the new
system in chapter 7 can be repeated. Over the years, and
surely in the new accreditation system, the Netherlands and
Flanders have come closer together, if not in system and
­regulations, then in practice. Even before the new system was
introduced, the two countries have learned from each other
for the better. The order of testing the macro-efficiency of a
new programme before initial accreditation is a good example.
The difference that keep existing in the new accreditation
system are results of the different degrees of trust and of
cultural differences. Such differences provide interesting
insights and deserve being investigated in order to learn
from each other. The fact that the first round of institutional
reviews in Flanders will not have legal consequences and
will be ­evaluated can provide input for improvements to the
Dutch system.

The Committee recommended further harmonization of
­regulations and procedures between the two countries with
regard to the categories in the table below. As the table makes
clear on almost all recommendations progress is being made.
The new accreditation system complies with a number of
­recommendations.
Table 8

2007 recommendation

New system NL

New system FL

Legal protection of titles

Recommendations of minister to be decided

Only with accreditation and strict use of Arts and

Will be checked in external QA

Sciences only with academic programmes

Recovery period

Recovery period

Macro-efficiency check for programmes by

Macro-efficiency check for programmes by

official institutions

official institutions

Length of accreditation cycles

6 years

8 years followed by 6 years

Clustered assessments on binational scale

Clustered assessments on national scale

Clustered assessments on national scale

Time limits & sanctions

Time limits (6 months) with fines unless warning Flexible time limits (4 months) without fines

Relationship NVAO-QAAs

NVAO instead of institution orders QAA

Sanctions in the case of a negative
accreditation. The Flemish system (of a
statutory repair period) should also be
implemented in the Dutch system
Position of macro-efficiency check

NVAO communicates and supports QAA on
quality (protocol)

Future concept of self-accrediting institutions Institutional audit possible first step

54

S e lf -eval u at i o n re p or t NVAO 2012

Institutional audit possible first step

Overview of actions taken since the previous external review 2007

55

Colophon
Self-evaluation report NVAO 2012
© April 16 th 2012
Editors: NVAO/Hendrikx Van der Spek
Lay-out: Smidswater
Offset: Platform P

NVAO
Accreditation Organisation of the Netherlands and Flanders
Parkstraat 28 / 2514 JK Den Haag
P.O. Box 85498 / 2508 CD The Hague
The Netherlands
T +31 (0)70 312 23 00
F +31 (0)70 312 23 01
info@nvao.net
www.nvao.net

56

S e lf -eval u at i o n re p or t NVAO 2012

NVAO
Accreditation Organisation of the Netherlands and Flanders
Parkstraat 28 / 2514 JK Den Haag
P.O. Box 85498 / 2508 CD The Hague
The Netherlands
T +31 (0)70 312 23 00
F +31 (0)70 312 23 01
E info@nvao.net
www.nvao.net

Attachments

Selfevaluation
report
NVAO 2012
April 16 th 2012

NVAO
Accreditation Organisation of the Netherlands and Flanders
Parkstraat 28 / 2514 JK Den Haag
P.O. Box 85498 / 2508 CD The Hague
The Netherlands
T +31 (0)70 312 23 00
F +31 (0)70 312 23 01
E info@nvao.net
www.nvao.net

List of Attachments

1.

Comparative and system-wide analyses

2.

Additional tasks of NVAO

3.

Workflow scheme Initial Accreditation

4.

Organisational chart NVAO

5.

Strategic Policy Statement NVAO 2012 – 2016

6.

IQA Quality scheme, targets per domain

7.

IQA Report of progress per domain

8.

IQA Evaluation scheme

9.

Key figures higher education in the Netherlands and Flanders

10.

Follow-up of the recommendations of the review committee 2007
regarding the accreditation systems in the Netherlands and Flanders

11.

Accreditation Frameworks in the Netherlands and Flanders

12.

Sources

13.

Abbreviations

Attachment 1 Comparative and system-wide analyses
Analysis Academic programmes Linguistics (2007)1
In 2007 the executive board of NVAO decided to start a pilot regarding the evaluation of the assessment and
accreditation of the academic programmes Language Sciences in the Netherlands, in order to contribute to the
enhancement objective of the accreditation system. The analysis focuses on strengths and weaknesses that
programmes in this cluster have in common and of the approach and execution of the assessment by the panel.
The report was discussed in the meeting of the NVAO board in September 2007 and was used internally as a try
–out to discuss what NVAO’s policy and practice regarding comparative and system-wide analyses should be.

Analysis Academic programmes Sociology (November 2007)2
The executive board decided to start another meta-analysis of academic programmes in 2007. The cluster
Sociology in the Netherlands was analyzed, first because of the low quantitative study results of the programmes,
but also because of a lack of ambition on the side of the institutions. It was felt that this lack of ambition reflected
both student and teacher attitudes and was not influencing quality of graduates in a positive way. The report was
published in November 2007 and the results were discussed with the deans of the faculties of all involved
universities during a meeting in Utrecht on 27 November 2007.

Conference professional master programmes and seminar (2007)
On 21 June 2007 NVAO organised a conference Professional master programmes. At that time NVAO had
assessed these programmes for three years. As Professional master programmes are a new type of programme
in Dutch higher education since 2004 a evaluation was useful to share first experiences and good practice. During
the conference four approved applications by NVAO from different disciplines were presented as examples of
good practice, namely examples of programmes from the sectors Health, Arts, Education and Natural Sciences.
Furthermore the importance of professional master programmes for Dutch higher education and the professional
field was emphasized by the umbrella organisation of the universities of applies sciences [The HBO-raad]. The
conference was concluded by a forum discussion between four experts, contributing from different perspectives,
and the audience of about 80 people. NAVO presented all presentations and the results of the conference on its
website.

Analysis Professional bachelor programmes Industrial Engineering (2009)3
In 2009 NVAO started an internal, comparative analysis of the professional programme cluster Industrial
Engineering. The main objective was to apply a check regarding the internal consistency in decision making as a
large number of programmes were involved. A second objective was to compare the quality of the programmes
on some specific aspects as coherence, study results and assessment. The report was discussed in the meeting
of the executive board of NVAO on 1 September 2009. The result was not shared with the involved institutions as
the objective was mainly NVAO internal oriented to support NVAO’s decision making process.

Analysis Academic programmes Educational Sciences (2009)4
In 2009 NVAO completed the comparison of the academic programmes Educational Sciences in the Netherlands
and Flanders. This investigation was initiated, because of the concern about the achieved learning outcomes, the
lack of complexity and study load of the curriculum in the programmes Pedagogical Sciences [Pedagogische
Wetenschappen] in the Netherlands. It turned out that in general the Flemish programmes are more intense than
the programmes in the Netherlands. Especially the achieved study load and the achieved learning outcomes are
at a higher level. The results of this study were used by NVAO for its internal process to get a better picture of the
quality of these programmes in the Netherlands and Flanders.

Analysis professional bachelor programmes Teacher training (primary education, 2009)5
The programmes Teacher training for primary education have been subject of public debate in the Netherlands in
2009. Social and political concern raised about the expertise of teachers in primary education, especially
1

[Meta-analyse cluster Taalwetenschappen, September 2007]

2

[Meta-analyse cluster Sociologie, November 2007]

3

[Analyse Technische Bedrijfskunde, September 2009]

4

[Breedteanalyse Agogische en Pedagogische Wetenschappen Vlaanderen en Pedagogische Wetenschappen en

Onderwijskunde Nederland, December 2009]
5

Systeembrede analyse hbo-bachelor Opleiding tot leraar basisonderwijs, 14 Oktober 2009

regarding their elementary skills related to calculating and Dutch language. As a result, on request of the Minister,
NVAO started a meta-analysis on the Teacher programmes in primary education after completion of the
accreditation procedure in these clusters. NVAO presented the results to the Minister, the Parliament and the
involved stakeholders in higher education in 2009 and delivered through this report to the public discussion about
the quality of these programmes.

Sector-wide analysis "Analysis of Fine & Performing Arts Assessment Reports" (2010) 6
The request from the Ministry of Education, Culture and Science (OCW) to the Netherlands and Flanders
Accreditation Organisation (NVAO) to describe the developments in Fine & Performing Arts programmes
(professional bachelor and master) resulted in an extensive and in-depth analysis of 52 assessment reports in the
disciplines of Fine Arts, Design, Dance, Theatre and Music.
This sector-wide "Analysis of Fine & Performing Arts Assessment Reports" produced a positive picture of the
quality of these arts programmes. This picture reflects the thoroughness of the programmes and is underscored
by several strengths including a profession-oriented approach, individual artistic development, technical skills in
the field, up-to-date input and a wide range of expertise on the part of the teachers, differentiation in the
programmes offered and stringent selection for admission. In addition, several points for attention regarding the
theory courses, the research component, the preparation for “entrepreneurship” and the formal and structural
involvement of alumni and the professional field, indicate issues that could raise the quality of these arts
programmes with additional attention and effort. These matters require a result-oriented approach.
On 16 September 2010 NVAO organised a seminar for representatives of the Arts programmes to share the main
facts, findings and conclusions and to stimulate the follow up of the recommendations.
Later on, the report provided a good reference to the committee Dijkgraaf, which formulated a new sector plan for
the Arts sector (2011), commissioned by the umbrella organization of the Dutch universities of applied sciences
[HBO-Raad].

Review Research Master 2011
In 2010 NVAO started the review project Research Master. The Research Master is a separate category of
academic master’s programmes focusing on the role of academic researcher. The length of these master’s
programmes is two years (120 EC). Part of the review is an International Thesis Assessment Study that was
completed in Spring 2011 and presented during a NVAO seminar on 24 March 2011. A panel of experts assessed
74 master thesis, 37 of graduates of research master programmes and 37 thesis from graduates of regular
master programmes in the same discipline. Within the same discipline all thesis of research master graduates
were assessed with a higher note. The assessors were unaware of the origin of the thesis, belonging to research
master or the regular group and unaware of the result of the thesis, achieved by the student. In the opinion of the
assessment panel the research master theses are in general ‘good or excellent’ and can meet international top
standards.
In the investigation participated the programmes Cognitive and Clinical Neurosciences (Universiteit Maastricht);
Economics & Business Administration (Universiteit van Tilburg); Communication Science (Universiteit van
Amsterdam) en Media Studies (Universiteit van Amsterdam).
The results were presented during the seminar Excellent profiling in The Hague on 24 March 2011 attended by
about 125 representatives from higher education institutions. Later a publication was completed (Peaks In Sight).

6

[Analyse visitatierapporten Kunstvakopleidingen, mei 2010]. Available in Dutch on the NVAO website

Attachment 2 Additional tasks NVAO
Associate degrees
A substantial additional task of NVAO since 2006 concerns the assessment of Dutch applications for Associate
degree (Short cycle) programmes. More than 300 applications of these ‘pilot’ programmes were assessed by
NVAO at the request of the State Secretary of Education in the Netherlands in eight rounds of execution in the
period 2006 – 2012. These programmes can be provided by universities of applied sciences or recognised
institutions which offer accredited bachelor’s programmes in the same field of study. Associate degree
programmes are short programmes in the professional bachelor’s programme that is usually completed within two
years. After completion of these programmes graduates can enter the labour market or continue their studies in
the bachelor’s programme of the university of applied sciences. It is expected that Associate degree programmes
will be definitely legalized in the Dutch act in 2013. The graduates of the running Associate degree pilot
programmes receive already the Associate degree, as has been regulated by law, and which situates at level 5 of
the European Qualifications Framework (EQF).

‘Open system’
A second extra task in the last years concerned the initial accreditation of programmes within the Dutch
experiment ‘Open System’ [Open Bestel]. These programmes are offered by either a recognised (private)
university of applied sciences or a recognised (private) institution for higher education, which under the terms of
this experiment can receive public funding. The objective of the pilot is to investigate the (dis)advantages of a
level playing field for funded and recognised (private) higher education institutions. The results of the experiment
are not available yet.

‘Room for Talent’
Another additional task concerns the assessment of the Dutch experiment ‘Room for Talent’ [Ruim baan voor
talent]. Under the terms of this experiment, programmes are allowed to raise tuition fees and enrol students on
the basis of selection if programmes offer ‘recognised and evident added value’. Another part of this experiment
was that it aimed to enhance the development of honours’ programmes. To assess ‘evident added value’ in
programmes with selection of students and higher tuition fees and for the assessment of honours’ programmes,
NVAO developed assessment frameworks and set up an evaluation committee. The evaluation report of the
experiment was published in 20077. Partly due to the results of the experiment a new law concerning Room for
Talent will came into effect on 8 July 2011. The secretary of state asked NVAO to develop a framework for the
assessment of the distinctive feature Small and intensive education. A concept was discussed with stakeholders
in August 2011.

Recognition of private higher education institutions
The Dutch State Secretary of Education, Culture and Science stipulated the policy regulation concerning
recognition of private higher education institutions and requested NVAO to play a role in the recognition
procedure. NVAO is involved in this procedure since 2004 and assessed 15 applications. In September 2010 the
procedure was renewed by the ministry. For this purpose, NVAO developed a new protocol8, including the criteria
that programmes have to meet to be eligible for recognition. Recognition means that in the Netherlands a higher
education institution – with the exception of the right to public funding – obtains the same rights as a publicly
funded institution. If an institution is neither publicly funded, nor recognised and applies for accreditation of its
programmes with NVAO, the institution will have to apply for recognition first.

Protocol broadened programmes
The Dutch minister for Education, Culture and Science determines whether an institution’s intention to combine
two or more programmes already incorporated in the Central Register for Programmes in Higher Education
(CROHO) leads to a broadened new programme. In this case, NVAO carries out a marginal assessment and

7

Evaluation report Sminia Comittee, [Evaluatie van de Commissie Sminia in het kader van Ruim Baan voor Talent, 25 mei

2007]
8

Procedure Initial Accreditation regarding new Legal bodies in higher education [Protocol verzwaarde toets nieuwe opleiding,

augustus 2010]

9

provides advice to the minister. For this purpose, the ‘Protocol Broadened Programmes’ has been developed.
Since 2004 NVAO assessed 30 applications for broadening programmes.

Higher Education Register
Another NVAO task concerns the online development and database administration of the Higher Education
Register in Flanders (HOR, www.highereducation.be). This website lists all the recognised bachelor and master’s
programmes in Flanders.

Assessment Teacher Training Schools
The concept ‘school based teacher education’ has made a rapid progress in the Netherlands. ‘Education together’
is an important innovation with a complex implementation.
A Teacher Training School (“Opleidingsschool”) is a partnership between a school for primary, secondary or
vocational education and one or more teacher training institutes. In 2009 the Minister of Education asked NVAO
to assess the Teacher Training Schools. NVAO accepted this assignment and assessed 83 Teacher Training
Schools. For this purpose NVAO developed an evaluation framework and composed an assessment panel that
advised NVAO about the achieved learning outcomes of the graduates and about the partnership between the
involved schools and the teacher training institutes.
All Teacher Training Schools have been visited by two teams of experts: one team advised the NVAO about the
achieved learning outcomes, another team advised the NVAO about the partnership between the Training School
and the Teacher programme(s).
Based on the assessment results, NVAO advised the Minister about the quality of the Teacher Training Schools.
Three Teacher Training Schools were assessed as excellent, 24 as good, 30 as sufficient and 26 as insufficient.
An additional result of the assessment was that success factors for Teacher Training Schools could be indicated,
as the importance of real partnership, controlled growth of Teacher Training Schools, personnel policy, internal
quality assurance and an inspiring learning environment.

Assessment of Academic Teacher training primary education
Academic Teacher training primary education is a special programme for students, attending an academic
bachelor programme educational sciences, with interest in primary education. These students can follow at the
same time the professional bachelor programme Teacher training primary education. This dual programme lasts
four years (240 EC) and was introduced by the minister to enlarge the number of teachers in primary education
and to increase the quality of graduates Teacher primary education. NVAO was asked by the minister to advise
him on the quality of these dual programmes. In the years 2010 and 2011 NVAO assessed six programmes.

Assessment of Educational Minors
Educational minors offer the opportunity to students of academic, discipline oriented bachelor programmes to
obtain for a teaching qualification. In 2009 the Minister asked NVAO to advise regarding the quality of plans for
Educational minors. NVAO developed criteria and a limited framework and advised on 10 applications.
Research in February 2011 showed that the running Educational minors do meet the expectations of the
graduates and the schools, where they start to work.

Applications Extension of academic master’s programmes
Academic master programmes in the Netherland last mostly one of two years (60 or 120 ECTS). Programmes of
60 ECTS can apply for extension to 120 ECTS, when they are of the opinion that the intended learning outcomes
cannot be achieved by a curriculum of 60 ECTS. It is up to an expert panel and NVAO to formulate an advice and
to the minister to decide. Up to now NVAO assessed 29 applications of this kind.

Academisation
In Flanders the former programmes of second cycle from the university colleges have to be transformed to
academic bachelor’s and master’s programmes. These programmes are to develop to full academic programmes,
but are professionally oriented as well. This reform has been running step by step since 2004. Special attention is
spent to the development of academic research skills of students. In 2013 all involved programmes should meet
the same requirements as the academic programmes of universities. NVAO incorporated the transition in the
Accreditation framework and developed a manual Academisation and a manual Academisation Arts programmes.
9

[Protocol ter uitvoering van de artikelen 7.1 en 7.2 van de Beleidsregel doelmatigheid hoger onderwijs 2009 (Stcrt 2009, 115),

inhoudende de uitgangspunten van de NVAO met betrekking tot de samenvoeging (verbreding) van opleidingen, de
zogenoemde planningsneutrale conversie]

In 2010 and 2011 NVAO decided on applications of 12 institutions for the accreditation of 140 academic
bachelor’s and master’s programmes in several disciplines, in which it was assessed to what extent the
Academisation process has been implemented and whether it can be assumed that the process will be finished in
the academic year 2012-2013. For each cluster of study programmes NVAO has been making an internal
comparative analyses of the scores, textual appreciations and recommendations by the review panels in order to
decide consistently on the possible conclusions (positive, negative, additional information, hearing or additional
assessment). NVAO will decide on another substantial number of applications regarding academisation in 2012.

HBO5
Flanders will introduce a new level of (Short cycle) education on level 5 of the Flemish Qualification framework:
HBO5. It is a professional oriented programme, which does not offer a bachelor’s or master’s degree, but a
graduate diploma.
To support the implementation of HBO5 programmes in Flanders NVAO developed a framework and organized a
pilot to generate and disseminate learning experiences. Five institutions participated and their experiences were
presented during a seminar10 in December 2010. The results of the pilots are described in a publication11, which
was handed to the Flemish Minister of Education, Youth, Equal chances and Brussels as well as to an audience
of 270 people.

10

Seminar HBO5, Learning from pilots, 17 december 2010, [Studiedag HBO5 startklaar – Leren uit proefprojecten]

11

Pilot report Learning effects, December 2010, [Leereffectenrapport – proefprojecten HBO5]

Attachment 3 Workflow scheme - an example: Initial Accreditation NVAO
1

Registration of a new postal item (Archive)

2

Preparation of a new application file (Archive)

3

Application to be completed (adding information of the programme) (Archive)

4

To allocate the application in the organisation (Managing director)

5

Adding templates to application en sending confirmation of receipt and invoice (Secretariat policy advisors)

6

Checking the application for completeness (First responsible policy advisor)

7

Proposal panel composition (First responsible policy advisor)

8

Approval panel composition (First responsible Execute board member)

9

Adding panel composition to agenda meeting of the Executive board (Secretary meeting Executive board)

10

Report meeting Executive board (Secretary meeting Executive board)

11A

Informing institution about the panel composition, informing panel, including the supply of the application file, planning
site visit (Secretariat policy advisors)

11B
12

Analysing information file, organisation of the prepatory meeting panel and site visit (First responsible policy advisor)
Preparation of the intended decision and belonging documents for the meeting of the Executive board (First responsible
policy advisor)

13
14

Approval intended decision (First responsible board member and lawyer)
Adding the intended decision to agenda of the meeting of the Executive board (Secretary of the meeting of the Executive
board)

15

Report meeting of the Executive board (Secretary of the meeting of the Executive board)

16

Sending intened decision to the higher education institution (Secretariat policy advisors)

17A

Adding the application to the agenda of the General board (Secretary of the meeting of the General board)

18A

Report meeting of the General board (Secretary of the meeting of the General board)

17B

Preparation definite decision (Secretariat policy advisors)

18B

Control of all documents (First responsible policy advisor)

19

Sending definite decision (Secretariat policy advisors)

20

Publication definite decision and assessment report on NVAO website (department Communication)

21

Completion of the application file and all documents (First responsible policy advisor)

22

Control file by archive en provisional closure (Archive)

23

Definite closure or appeal (Archive)

Attachment 4 Organisational chart NVAO

Attachment 5 Strategic Policy Statement NVAO 2012 - 2016

STRATEGY NVAO 2012 – 2016
1. INTRODUCTION
Now that NVAO has celebrated its first lustrum, the Committee of Ministers has evaluated and extended the
NVAO treaty, and Flanders and the Netherlands are entering or have already entered the second phase of the
accreditation system, NVAO wants to reassess its strategic goals for the next five years. This is not without
reason: the social contexts for higher education in the Netherlands and Flanders have changed considerably the
last few years and European developments are also moving forward. Higher education institutions have to adapt
to these changing contexts, with students increasingly demanding higher standards and society critically
appraising the achievements of institutions and students. This requires an active approach and responsive
attitude of NVAO so that it can contribute to achieving optimal quality for higher education in Flanders and the
Netherlands. For good education is the most important 'raw material' for both regions and NVAO may be
expected to make a significant contribution to good education through its activities. The point of departure for this
is that the responsibility for the quality of education lies first and foremost with the institutions. There can be no
other way in a mature system of higher education. However, primary responsibility is not a carte blanche:
education is too important and too expensive to be not monitored continuously. External quality control assures
that the responsibility of the institutions is filled in appropriately.
This Strategy Memorandum first continues with an overview of the most important external developments in
higher education in the Netherlands, Flanders and Europe. Next, NVAO will present its mission, values and
positioning. These will be translated into a number of strategic goals, after which the relationship with
stakeholders, the internal organisation and the working methods of NVAO will be discussed. All this will result in
the NVAO's 'strategic agenda' for 2012 - 2016.
2. CONTEXTS
2.1 Society
Due to the importance of higher education for Flanders and the Netherlands, society wants to and should be able
to trust the continuing good quality of higher education. Public interest in the achievements of higher education
institutions has undeniably increased. Abuses in a small number of institutions, failing administrative procedures
and complaints from students and lecturers have led to consternation and general indignation. As a consequence,
doubts have arisen about the institutions' educational achievements and there is a call for more stringent
supervision to make sure that each diploma has been obtained legitimately and that its holder has the expertise,
skills and attitude expected of a recent graduate. Although this concern is understandable and these incidents
indeed call for specific measures, the increased media attention for higher education appears to turn each
incident into a structural shortcoming, as a result of which the institutions and their supervisors are subjected to
an ever-increasing pressure.
In Flanders, the public debate is dominated by recent developments, such as flexibilisation in higher education,
academisation of colleges of higher education and their integration in universities as well as the consequences for
the organisation and the quality of higher education as a whole. The institutions' difficult financial positions play a
large part in this debate.
2.2 Students
By far the majority of students realise it is a privilege to take a degree programme at a university or college of
higher education. This position comes with rights and obligations. In return for a substantial study commitment,
students may expect to be offered high-quality education that is well organised. This is all the more cogent as the
governments increasingly place the responsibility for and the costs of longer study duration and second-degree
programmes with the students. For this requires the institutions to take even greater care of the content and
process side of education. Moreover, institutions are expected to help students make appropriate study choices
by providing good information and showing them at the earliest possible stage if they are suited for a certain
study. This involves intake interviews, orientation tests, a (binding) study advice and at least good student career
counselling.

2.3 Ambitions
Flanders and the Netherlands both realise that their economic growth and social-cultural power are very much
driven by knowledge. The term 'knowledge economy' appears in every policy memorandum and strategy view.
Moreover, both regions lack modesty - and quite rightly so - and show considerable ambition. The Netherlands
has even expressed the ambition to rank with the top five knowledge-based economies in the world. Flanders,
too, pursues an ambitious agenda in VIA. These aspirations require a large effort of the institutions, which are
expected to compete with leading institutions in the United States, Great Britain and other western and emerging
economies. For this reason, the institutions' achievements in education and research are closely followed.
2.4 Diversity
The belief that students cannot be regarded as one uniform and homogeneous group is taking firmer root all the
time. The attention for differences in ability, social backgrounds and learning styles has increased strongly and
leads to the conclusion that also in education more diversity is wanted and needed. The last few years the
institutions have positioned themselves accordingly: this shows for instance in the creation - and flourishing - of
the (liberal arts) colleges, the excellent master's track and honours programmes, preparatory programmes, track
and study counselling, new broad programmes, but also associate degree and HBO5 programmes, and a variety
of deficiency courses. All this has led to a dramatic rise in the diversity of programmes offered.
Moreover, this diversity is reinforced by the governments that also want the institutions to try and have different
profiles from each other. Especially the Dutch government has turned this into an important issue acting on the
so-called Veerman report.
2.5 Rankings
In the global 'market' for higher education and in the battle to attract top talent, the prestige of the institutions
plays an ever-growing role. This prestige is largely determined by rankings. No matter how strongly one may
object methodologically and sometimes also morally to these rankings that are solely based on research results,
they do indeed play an important role and the institutions - and also the governments to a certain extent - have no
choice but to take them into account. Universities, therefore, will have to carefully balance their attention between
research and education, between investments in research and education. In doing so, it is not improbable that
research will prevail. The colleges of higher education have managed to escape these rankings so far, but the first
signs of wanting to make profiles and differences in quality transparent are already visible.
2.6 Implementation of Bologna
The policy implementations of the Bologna Declaration in 1999 have been formally completed in many European
countries. An extraordinary amount of work has been done, but it is clear that formal completion does not equal
material implementation in all cases. This is compounded by the fact that implementation did not everywhere take
place simultaneously. Great national differences remain within the European Higher Education Area. The Bologna
Process is continuing, however, working towards its final goal of realising one European area for higher education
and research. After the introduction of the Bachelor Master structure, the implementation of the ECTS system and
the Diploma Supplement as well as an independent external assessment system for the quality of education,
attention has now shifted to bringing about national qualifications frameworks within a European context and the
application of learning outcomes systems. In Europe, Flanders and the Netherlands lead the way in both areas.
The last few years, there have been discussions about the most desirable system for external quality assessment
in a considerable number of European countries. Although the direction of change is not unequivocal, a
noticeable trend seems to emerge towards combinations of institution and programme assessments. The
Netherlands and Flanders fit seamlessly into this trend and in a number of areas they have even set the trend for
developments in other European countries.
Special attention is needed for the evolutions of the European Standards and Guidelines (ESG) and the European
Quality Agency Register (EQAR), two important outcomes of the Bologna Process. The ESGs put initial
responsibility for standards on the institutions and they will be updated in the coming years. The other evolution is
the careful expansion of the work area of EQAR registered agencies through national legislation.
2.7 The EU
In addition to 'Bologna', which is a non-legally binding agreement, the EU is increasingly taking initiatives in the
area of higher education. In doing so, one cannot escape the impression that the EU wishes to take over the
Bologna Process and is taking steps that will impact the national control of higher education. This applies in
particular to the manner in which the EU promotes quality labels, aimed at the exercise of professions. This also
applies to the increased attention and promotion of directives that set conditions for exercising regulated
professions and have an effect on the content of education. And it also applies, possibly and to a lesser extent, to

heavily sponsored EU initiatives to come to its own European classification (U-map) and ranking system (UMultirank). This European ranking system continues to struggle with the difficulty of formulating sound indicators
to measure the quality of education.
Of course, the European activities are stimulated by the increased competition between the two large power
blocks. A small number of Anglo-Saxon institutions of higher education still dominate the world's top research
universities while Asian knowledge institutions are strongly on the rise. In relative terms, Europe has a much
higher number of educational institutions of more than considerable quality, but if obviously feels challenged to
take part in this competition and to take initiatives to also create more top institutions, within or across national
borders.
3. MISSION, VALUES AND POSITIONING
Based on the Accreditation Treaty, the tasks and powers as laid down in laws, structural decrees and other
general regulations as well as the contexts outlined in section 2, NVAO's mission is as follows:
"NVAO is an independent and authoritative binational accreditation organisation set up by the Flemish and Dutch
governments, whose primary goal it is to provide an expert and objective judgement of the quality of higher
education in Flanders and the Netherlands. NVAO does this with a constructive, critical attitude, respecting the
autonomy of institutions and their primary responsibility for the quality of their education, and with an open eye for
the growing international context. NVAO is open, clear and transparent towards society and all concerned,
especially the institutions of higher education and the students."
This mission translates into three main NVAO tasks:
1.
2.
3.

assessing and assuring the quality of Dutch and Flemish higher education;
promoting the quality of higher education by promoting a culture of quality, aimed at regular assessment and
continuous quality increase;
putting Dutch and Flemish sectors of higher education (institutions, programmes) on the map and
strengthening their position by means of international cooperation.

NVAO carries out these tasks based on the following composition of values:

independence

clarity

respect for (the autonomy of the institutions and their primary responsibility for the quality of their education).
Mission, tasks and values form the basis for the way in which NVAO will organise its activities in the coming
years. In doing so, NVAO also chooses a clear positioning with regard to and for its stakeholders. First and
foremost, this positioning will be marked by the necessary independence. For NVAO is neither the 'extended arm
of the government' nor is it part of the sector of higher education.
Its judgements have to be respected and trusted, which means that institutions have to feel treated fairly, that the
government has to be able to trust the outcomes of accreditation surveys, that students and society have to be
able to trust the value of the accreditation judgements and that other parties (for instance foreign institutions and
students) know that accredited programmes deliver what they promise. The necessary trust in the quality of the
accreditation process and its outcomes have to lead to authority: authority for NVAO, authority for the assessment
panels and organisations, but also authority for the institutions and programmes that have been assessed and
accredited.
The positioning chosen by NVAO requires an active approach. Active when supplying information, active when
pointing out shortcomings and areas for improvement, active also when taking position on issues regarding the
quality of higher education. This active approach applies to both the national and international arenas. For NVAO
explicitly chooses to put the position of Dutch and Flemish higher education on the international map. With this
also comes the obligation for NVAO itself to constantly look for the best possible ways to fulfil its tasks. In the
same way education is and should be constantly evolving, the accreditation system has to be adapted regularly to
the requirements of the time, to circumstances and to good developments elsewhere. This requires permanent
reflection on one's own functioning and the functioning of the system.

4.

STRATEGIC IMPLEMENTATIONS

4.1 Assessment and assurance

Clear and independent assessment
It is essential and also an absolute minimum that the quality of education provided by the institutions and
programmes is able to meet the (internationally accepted) standards of generic quality. By now institutions and
programmes have had the opportunity to gain ample experience with the accreditation system. Their own
professionalism and responsibility ought to tell them where to place the pass mark that constitutes a 'pass' or a
'fail' and it is up to them to deliver good results. The most important evidence for this can be found in the achieved
learning outcomes in final reports, portfolios and exams. By performing regular benchmarking through self
evaluations and by calling in (inter)national experts during assessment procedures, it has to become clear that the
achieved learning outcomes compare well internationally. It is with full conviction that NVAO chooses an
assessment system that is supported by peers. In the new accreditation system even more demands are made
on their qualities and they are either appointed (the Netherlands) or their appointment is recommended (Flanders)
by NVAO. The quality of the peers and therefore the panels is decisive for the functioning of an assessment and
accreditation system. The term 'peers' does not only refer to subject and disciplinary expertises, but also to
expertises in the professional field, education and auditing as well as to international and student-related ones.
Due to (the introduction of) a recovery period in the Dutch and Flemish systems, panels are now able to award a
fail to standards when required with greater conviction: for in many cases programmes can be given more time to
make the necessary improvements and thus raise their quality levels. This requires a completely independent
approach of panel members, because the trust placed in the accreditation system stands or falls with the
independence and clarity of the panel's judgements.
The independence and clarity with which NVAO and the panels deliver their judgements are in principle based on
trust. Trust in lecturers, students, board and management. For the point of departure can be no other than that
each person concerned, in whatever role, makes the effort to deliver the best possible education. But even with
respect and trust, the panel may come to a negative judgement. The motivation for this judgement - positive or
negative - and the manner in which the recommendations for improvement are formulated, are crucial for an
improvement incentive to the programmes.

Explicit attention for purpose of quality improvement in reports
An independent assessment by authoritative peers does not only guarantee a good judgement, but it also
provides opportunities to formulate recommendations and advice. Judging and advising are two different activities
from essentially different perspectives and they ought to be separated carefully in the decision process and in
reports. One of the lessons learned from the first phase of the accreditation system is that this distinction was not
sufficiently made and was not deemed sufficiently important. As a result, the NVAO's panel judgements were
sometimes regarded as less solid and recommendations were considered points of criticism. In the new systems
in Flanders and the Netherlands and in the way panels and NVAO operate, the true distinction between judging
and recommending has to be made apparent, thus strengthening the purpose of quality improvement of the
system.

Increasing the accessibility and information content of decisions and reports
NVAO is aware of the fact that the sheer volume of the accreditation decisions and the assessment reports from
the first phase of the system have rendered them insufficiently accessible to serve as a source of information for a
wider public. This is regrettable, for the reports contain a wealth of information. In the new system, guidelines
have been added concerning the public character of the reports (a summary of the judgement that is easily
accessible), the obligatory enclosures with factual information (such as contact hours, staff-student ratio, returns)
and a differentiated judgement on the level of standards and the total. This range of informative improvements
should lead to more information for those concerned (in the first place the students) and those interested in higher
education. This does require a meaningful and unequivocal defining of data, such as, for instance, success rates.
NVAO will take initiatives to this end.

Making differences in quality and profiling transparent
When increasing the informative value of judgements, decisions and reports, it is also fitting to make differences
in quality and profiling between institutions and programmes transparent. This has the strong backing of in
particular the Dutch government, arising mostly from the need to make clear to students and the labour market
what (institutions and) programmes stand for. A great help in all this is cluster assessment, which already exists in
Flanders and is obligatory in the Netherlands as of 2012. Assessing on a four-point scale makes it possible to

map out the differences in quality even more clearly. Not only a differentiated judgement but profile features, too,
can help to have students make appropriate choices and to have employers choose from the correct pool of
graduates. The profiles of the programmes can be expressed in the content objectives and the education process,
and can be supported further by applying for distinctive (quality) features. Although programmes have not been
very keen to do this so far, NVAO initiatives to arrive at a distinction between the areas of 'internationalisation' and
'entrepreneurship' have been greeted with enthusiasm by Dutch higher education at least. Besides, making the
differences in profiling and quality transparent may encourage institutions and programmes to position themselves
clearly. This, too, can contribute to the content development of and differentiation between fields of study.
4.2 Promoting a culture of quality

Making good use of institutional audits
Important for the continuous improvement of quality is the attention for the creation of a culture of quality in
educational institutions. For accreditation provides a random picture and it is not good if the attention of the
programmes is only aimed at achieving the threshold for accreditation every number of years. A culture of quality,
by contrast, is an organisational culture that is aimed at quality and its continuous improvement. For this reason,
in the new accreditation system much attention is given to assessing the quality assurance system at institutional
level. The question then is whether an institution is in control of the quality of its education, in the sense that the
board of an institution has a clear picture of its education results, receives signals of inadequate functioning, is
able and willing to take measures and pays attention to improvements.
Vital for the creation and existence of a culture of quality is having an attitude that is critical and development
driven with regard to one's own achievements: this goes for lecturers among themselves, this goes for students,
management, the professional field and alumni. Collectively they all ought to guarantee the existence and
improvement of the quality of a programme. Stimulating this critically constructive attitude is a prerequisite for the
creation of a culture of quality. In this area, too, NVAO will act encouragingly, especially by spreading good
practices.

An active approach in promoting the quality of education
NVAO wants to contribute to intensifying discussions about the quality of education. There are various tools for
this. Dutch and Flemish legislators agree that NVAO should devote more time and attention to making analyses
that cross programme boundaries. These can be analyses of clusters of programmes in a particular domain, but
also thematic reports dealing with issues that politics and society find relevant. Of course, the aim of these
analyses is to come to conclusions and to subsequently base recommendations for improvement on these.
Moreover, NVAO will gain an insight into good practice, when the judgement of 'excellent' is awarded, for
example, to standards and programmes. The same goes for distinctive (quality) features. NVAO will focus on
these through conferences, debates, seminars and publications. Its role here is to facilitate and inspire the sector
of higher education, offering institutions and programmes the opportunity to assume responsibility to come to
quality improvement.
4.3 International perspective
As a bi-national organisation, NVAO has paid much attention to its international position from when it was
founded. Both governments have encouraged this international profiling and it is also in keeping with NVAO's binational character. ECA, the European Consortium for Accreditation, was founded as early as 2003; NVAO is in
charge of the secretariat of INQAAHE; through its board members, NVAO has a presence in the boards of ENQA
and EQAR; collaboration agreements have been signed and projects have been carried out with numerous other
countries.
Obviously, NVAO is not internationally active for its own sake. Of course, its international activities have to serve
goals that will be of use to Flemish and Dutch institutions of higher education. However, it is not always easy to
make that direct use immediately obvious. But in the same way the Netherlands and Flanders wish to profile
themselves as prominent knowledge economies, Flemish and Dutch institutions of higher education choose to
have prominent international profiles, NVAO, too, will have to profile itself internationally, in the vanguard of
quality assurance organisations.

Good positioning of Flemish and Dutch higher education abroad
On balance, Flemish and Dutch higher education is of good quality. This is also recognised abroad. That quality
has to be underlined by the quality of the assessment systems used. This makes it necessary for NVAO to
expound the quality of the accreditation system, especially by drawing attention to the unambiguous manner in

which it judges the level aspired to by the institutions and that achieved by their students. The Netherlands and
Flanders are ahead when it comes to assessing achieved learning outcomes. This practice is a prerequisite for
thorough international comparisons, but it will be a long time yet before other countries and quality assurance
organisations will follow their lead. The binary character of Dutch and Flemish higher education fairly often
requires an explanation in countries with a unitary system. Especially the Dutch 'professional master' is not known
as a separate degree in many countries. Therefore, NVAO will take initiatives to underpin and promote
international acceptance of this degree.

Intensifying cooperation with reliable partners
It is beyond doubt that institutions benefit from a system of accreditation that is considered solid and reliable by
foreign institutions and quality assurance organisations. It increases trust in its quality. However, the broadness
and strictness of the Flemish and Dutch system of accreditation, which is remarkable in an international
perspective, occasionally lead to self-critical comments about our higher education, while foreign programmes
and systems are a lot less critical with regard to certain aspects, such as for instance 'achieved level', which they
may not even assess! So now and then it appears that a difference is arising between domestic criticism and
foreign appreciation. This does not mean to say that the standard of quality should be lowered, but it does mean
that NVAO and the institutions have to communicate well and choose their partnerships carefully.

Developing a set of instruments for promoting internationalisation
Choosing carefully plays an important role, for instance, when it comes to the applaudable foreign mobility of
students. Whereas during the first years of mobility promotion, academic tourism was by no means accidental
when choosing foreign destinations, now quality requirements are set by institutions, programmes and students.
This is why NVAO chooses to work with preferred partners (especially in Central and North-West Europe), but
also looks for strong partners elsewhere in Europe by means of distinctive features, in particular in the field of
internationalisation. In this way, NVAO also hopes to make an indirect contribution to one of the objectives on the
Leuven agenda, namely promoting student and teacher mobility.

Active contribution to comparability between programmes
Student mobility can also be promoted as more and better data are known about the content of the programmes
that are offered in the thousands of European institutions of higher education. Qrossroads continues to be an
excellent instrument for this. But the possibilities to compare between programmes require more effort, which is
why active involvement in determining and interpreting national qualifications frameworks is of great importance.
In this area, too, the Netherlands and Flanders are leading the way in Europe. The same goes for the use of
learning outcomes, for which Flanders has developed detailed procedures, which have been tested in pilots the
last few years. In the Netherlands, too, learning outcomes will be translated into more specifically domain related
ones. It may be a consideration to give learning outcomes a more prominent place in assessment and
accreditation reports.

Increasing activities outside Europe
Of course, international cooperation cannot be limited to Europe. More and more institutions and programmes are
active in Asia and the US, and this also requires intercontinental relations and agreements from NVAO. For the
Netherlands and Flanders have major ambitions in higher education. External quality control ought to be in line
with these ambitions and to promote and assure them. The NVAO's international activities and collaborations
need to also serve this purpose.
Relations have been established and sometimes formalised with organisations in Japan and China, while
cooperation with American accreditation organisations takes place on an ad-hoc basis. Especially the elimination
of degree mills requires close cooperation and a pattern of close relations.

Enhancing the added value of Flemish-Dutch cooperation
NVAO wishes to clearly emphasise and enhance the added value of the Flemish-Dutch cooperation. This added
value is primarily evident in the detached manner with which NVAO can make its judgements due to its bi-national
nature. This manifests itself in the way NVAO operates, with both Flemish and Dutch eyes looking at how
programmes are judged. The same bi-nationality is reflected in the composition of the panels, the more so as
soon as the Netherlands, too, will have introduced cluster assessments. But more is possible, not only in the
cooperation across boundaries between institutions and programmes, tuning of domain-related learning
outcomes, analysing education results in domains and disciplines, but also by simply exchanging information and
good practices. In all these areas NVAO wants to organise more activities and thus strengthen Flemish-Dutch
cooperation. However, it is with great disappointment that NVAO concludes that the existence of close Flemish

and Dutch cooperation in the accreditation system has not yet led to unequivocal steps towards mutual DutchFlemish recognition of diplomas. With the accreditation system in mind this is hard to understand and not easy to
explain either. For in both regions assessment of the bachelor's and master's levels takes place based on the
Dublin descriptors and the generic and domain-specific learning outcomes that are based on these. The NVAO
pronouncements ensure that bachelor's and master's levels are achieved. This should be the basis of mutual
recognition rather than comparing the content and/or length of programmes. In the light of the intended creation of
a European Higher Education Area, the Netherlands and Flanders should at least see to it that this issue is sorted
binationally.
5.

RELATIONS WITH STAKEHOLDERS

NVAO works in a comprehensive field. Not only is it confronted with two systems of higher education, but also
with different political cultures and structures that give rise to different courses of action and positions. This means
that the stakeholders in both systems have to be dealt with in a careful but sometimes different manner. Good
communication is essential for this.
5.1 Institutions
Of course, the institutions of higher education are very important stakeholders. In its approach of the institutions,
NVAO chooses for critical fairness: a critical, objective and respectful approach. This attitude is a result of the way
NVAO sees its role, which has as its first main task the assessment and assurance of the level of Dutch and
Flemish education. The institutions need to be well aware of this role: NVAO is neither there to 'settle scores' nor
to cover up, but to assess if at least a generic level of quality is provided - and preferably more than that.
The recommendations of the panels and the broader, more reflexive analyses by NVAO offer good directions to
come to quality improvement at the level of an individual programme, institution, domain or theme.
5.2 Politics
The position of politics, as a derivative of society, as a stakeholder has changed considerably the last few years.
On the one hand, this has to do with the aspirations of the governments to strengthen their knowledge
economies, in which higher education plays a crucial role. On the other hand, politics has also become the selfappointed spokesperson of generally felt concern about noted shortcomings in quality. This leads to calls for clear
pronouncements on quality, for identifying problems and closing leaks in supervision. Therefore it is necessary
that NVAO, and consequently also the institutions, provide unequivocal information about those critical success
factors that are considered decisive for the current quality and for the improvement capacity of institutions and
programmes. The situation in the Netherlands, for that matter, is complex because both NVAO and the
Inspectorate of Education have a role in the assessment and supervision of the quality of education. Although
NVAO in principle is only concerned with assessing the content and results of education, and the Inspectorate's
tasks have the application of rules and legislation as their departure point, an overlap between the activities of
both organisations is almost inevitable. For this reason, the Inspectorate and NVAO have signed a covenant in
which agreements have been made about the way in which the two organisations cooperate in order to come to
an assessment system that functions to the best of its ability and also keeps the costs down for the institutions.
5.3 Society
The last few years NVAO has been relatively reticent in contributing facts or opinions to social debates. By
consciously choosing to call itself an 'authoritative' organisation in its mission and by putting more emphasis on
promoting quality improvement, NVAO will also adopt a more public stance. This political stance is especially
aimed at quality improvement in higher education and strengthening a culture of quality.
This comes with one caveat: more supervision and more rules do not automatically lead to better quality. NVAO
also considers it its duty to point to the consequences of changes to the system and a new, greater need for
information. The issue of 'reducing the burden of accreditation' is firmly on the agenda and the issue of a possible
reduction in bureaucracy remains topical under all circumstances.
5.4 Students
Much of the NVAO's activities has to benefit students. It would be good if, before starting their studies, more
students would act like well-informed consumers, making rational considerations regarding the future institution
and programme of their choice. NVAO is going to actively contribute to this, not only by actively engaging
students in all types of assessment panels, but also by taking care of good accessibility of information on the
quality of the programmes. The latter increasingly plays a role in view of the growing call for realising a distinction

between institutions and programmes. By also involving the institutions' students as well as possible in assessing
the quality of education, they will evolve from education consumers into education participants.
5.5 Evaluation organisations
NVAO duly takes into account the functioning and quality of evaluation organisations. In Flanders the position of
these organisations is defined by law, in the Netherlands they will have to qualify to fulfil an active role in the
assessment and accreditation system. The quality of the system is strongly dependent on the work of the panels
and evaluation organisations. Open communication, good feedback and high-quality participation from all those
concerned will therefore have to give the system a solid foundation.
5.6 Professional field and labour market
The labour market, too, is increasingly interested in data on quality and profiling. This interest can be seen at the
'back' of the system, from graduates, but certainly also at the 'front' when setting the intended learning outcomes
or qualifications that students have to meet. By publishing clear reports and organising thematic meetings, NVAO
can contribute to complying with requests for data on quality and profiling.
5.7 International colleague organisations
NVAO also considers its foreign colleague organisations as stakeholders. They should not only understand how
the Flemish-Dutch accreditation system functions, but they should also be able to judge if the outcomes of the
assessments are sufficiently reliable. Moreover, NVAO has to communicate and explain changes and
developments in the Dutch-Flemish system. In international trust, two 'external judgements' play an important
role: the ENQA assessment in 2012 to determine if NVAO meets the European Standards and Guidelines
resulting in full membership of ENQA, and the subsequent assessment to determine if NVAO continues to be
registered with EQAR.
At the moment there are various forms of structural meetings with stakeholders:
 the binational advisory council: with representatives from institutions, students and the professional field;
 meetings with the Committee of Ministers, which take place twice a year;
 the Flemish sounding board meeting with umbrella organisations, evaluation organisations and the
government;
 bilateral contacts with (umbrella) organisations, ministries, student organisations, employer organisations.
In order to inform its stakeholders and other interested parties, NVAO yearly publishes an annual report as well as
an account of income and expenditure, complete with a positive auditor's report. There are also other
publications, a Newsletter, and NVAO has an up-to-date website with a search engine for finding topical
information on programmes and institutions.
6.

INTERNAL ORGANISATION

It is with a certain satisfaction that NVAO looks back on its achievements of the past 6 to 8 years: as agreed all
programmes have now been assessed once (with the exception of a number of Flemish clusters that are still to be
accredited in 2012). This means that some 4,000 existing programmes and over 500 applications for new
programmes have been assessed.
These activities have been complemented with all kinds of extra tasks, such as assessing Associate Degree
programmes in the Netherlands and a HBO5 pilot in Flanders, from programmes like 'training in the school' and
educational minors in the Netherlands to domain specific learning results in Flanders, as well as executing all
kinds of international projects. This has put considerable pressure on the organisation, both on its staff and
finance. Yet the NVAO's regular budget has hardly risen the last few years, which is thanks to the great efficiency
and effectiveness of and within the organisation. With the second phase of the accreditation system, the
organisation is entering a new phase, but the internal professionalisation and the organisation of its business
processes seem to be well prepared for this.
It is becoming clear though that there will be a greater need for the analytical and reflexive abilities of the NVAO
as well as its internal quality assurance system. The former is necessary to come to good analyses and
recommendations that rise above programme level, the latter especially to implement the desire for differentiation
as expressed in judgements in a well-motivated and consistent manner: after all, the institutions and programmes
stand to benefit greatly from this.

This means that also within NVAO much attention needs to be paid to the continuous development of its own
culture of quality, with as its most important elements:
 permanent sharpness and a critical attitude
 collegial intervision and procedural purity
 consistency in judgements
 an active and reflexive attitude
 quick and institution-oriented processing of applications
 continuous professionalisation of people and work methods
 clear and reliable internal and external communication
 stimulating and directional management
 a slim, but efficient, internal quality assurance on all processes
In this, the NVAO's General Board will act using its common sense and helicopter view as non-executive fellow
director, sounding board and facilitator of general procedures. In order to support the introduction of the new
system, NVAO has set up a sounding board group for the Netherlands, which will become operational as soon as
the first reports on the institutional audits and the new programme assessments have been completed.
In addition, NVAO is considering the creation of an international advisory council, which can apply its foreign
expertise especially to detect possible problematic issues in the new systems in Flanders and the Netherlands
and also to establish a link with developments abroad. This could be seen as an extra legitimisation as well as the
start of thinking about the third phase of the system.
For NVAO it has become clear these past few months that the enthusiasm with which new tasks and possibilities
have been received, cannot be allowed to continue without restraint. Increasingly the question will be asked how
new demands and developments will be able to contribute to the quality increase and transparency of the Dutch
and Flemish higher education as well as our own operations, both internally and externally. This requires more
focused choices, which will have to be in line with the strategic course as described in this memorandum.
STRATEGIC AGENDA 2012 – 2016
From the above mission, positioning and strategic elaborations, NVAO derives the following goals it wants to
(continue to) realise in the coming years.
Assessment and assurance
 clear and independent assessment of programme quality and of quality assurance at institutional level
 explicit attention for purpose of quality improvement of assessments reports
 increasing information accessibility of decisions and reports, including meaningful and unequivocal defining of
core data
 making transparent differences in quality and profiling between programmes and institutions
 implementation and further development of new accreditation systems in the Netherlands and Flanders
Promoting culture of quality
 good use of institutional audits
 active approach in promoting quality of education by means of:
o
encouraging institutions and programmes to act on recommendations in assessment reports
o
broader analyses of and reflections on system and domain level and theme analyses
o
meetings and seminars, aimed at opportunities for quality improvement in higher education and
possibilities to realise these
International perspective
 good positioning of the quality of Flemish and Dutch higher education abroad
 intensive cooperation with reliable and internationally leading partners
 development of instruments for international cooperation of institutions and programmes
 active contribution to increase comparability of programmes
 increase activities outside Europe
 (continue) to look for the best possible forms of external quality assessment
 increasing and clarifying the added value of Flemish-Dutch cooperation
 contributing to the mutual and international recognition of diplomas

Clear positioning and relations with stakeholders
 positioning of NVAO based on mission and core values
 making explicit the balance between independent, clear and - if necessary - critical judgements on the one
hand and promoting and facilitating the content of continuous quality improvement on the other hand
 external communication and a more public stance to realise the desired positioning and stakeholder relations,
including the use of social media
 executing the covenant with the Inspectorate of Education
 continue with transparent reporting, in terms of both finance and content
 good relations with the Flemish and Dutch evaluation organisations
Internal organisation NVAO
 adapting the office organisation to new/changing activities
 further development of a top-notch NVAO quality culture, including renewed international certification
 continuous professionalisation of all staff members
 frequent testing of internal and external work on strategic objectives through slim but efficient internal quality
assurance
 making use of the strategic objectives as criteria for taking on additional tasks and (international) projects
 linking internal and external communication with this strategic policy plan
FINALLY
For NVAO, the 2012 - 2016 strategy means a different positioning. Whereas in the past few years the emphasis
has been on executing primary tasks and assessing whether or not standards of basis quality have been met, the
emphasis in the coming years will be placed on two axes: on the assessment of future-oriented quality assurance
within the institutions and predominantly past programme results on the one hand and the promotion of a culture
of quality and quality improvement on the other hand. This positioning calls for authoritative assessments and
authoritative advice. In doing so, NVAO will claim a place for itself in the public debate, aimed at providing sound
information on the one hand and thorough analyses and recommendations on the other.
This new positioning is not formulated for the greater good and glory of NVAO. No, it fits the aspirations of the
Flemish and Dutch governments, of the Flemish and Dutch institutions of higher education and those of their
students, as well as those of the labour markets in both regions in order to offer the best possible higher
education to a great diversity of students in a fast changing labour market. To all these aspirations and objectives,
external quality assurance ought to contribute!

Attachment 6 Internal Quality Assurance - Quality scheme per domain - an example

Domain12 : Policy, Strategy and Leadership
Nr

Target13

I.
I.1

Internationalisation
New policy plan
Internationalisation
Improving
Communication
internally & externally

I.2

Nr

Target

II.
II.1

Cooperation NL - FL
Associate degree
programmes

II.2

HBO5-programmes

II.3

Automatic mutual

12

Controller:
L.Bollaert
Realisation19
Chair/Board

Working group
Internationalisation
Project coordinators
Internationalisation

Board

Actions for
improvement14

Indicator15

Resources16

Deadline17

Development new policy
plan Internationalisation
Drawing newsletters

Approval policy plan
Internationalisation
Produced newsletters

Time and
discussions
webstek

End 2010

Drawing newsflashes
internally

Reports on intranet per
project

intranet

3x/a year (2
in 2010)
Every project

Extending involvement in
Interationalisation

Internal employee in x
projects

projectresources

2010
2011

Actions for
improvement

Indicator

Resources

Deadline

Owner

Realisation
Chair/Board

Embedding Associate
degree programmes in
the accreditation cycle
Assessment HBO5opleidingen

Timeline
Number of applications

Subsidization

End 2010?

HP & RB?

RD

Pilots completed
Protocols
Number of applications
DFG

FL grant
100.000
VL budget
FL

End 2010
End 2010
Juni 2011
2010

JB

LB

GL & LB

Board

Involved in progress and

Domain or quality area is a defined part of the organizational process that is subject to internal quality assurance
Targets are the elaborations of the strategic and operational objectives of NVAO in the current period
14
Actions for improvement are concrete planned action that follows from the targets
15
Indicators show when the target is achieved.
16
Resources: like budget, time, information or required meetings
17
Deadline is the final date of achievement
18
Owner is the person or group responsible for the result
19
Realisation: the finale responsibility is lodged with the board, one of its member or the managing director
13

Period :
2010-2011
Owner18

LB

recognition degrees
NL & VL

II.4

More visibility of
NVAO

preparation Decision
Flemish Government
(DFG)
Involved in progress and
preparation list of
equivalence
New communication plan
To approve

Treaty & list of
equivalence

FL

2011

Board

AB

Communication plan

Own resources &
Sponsoring

Juni 2010

KD & GL & staff
communication

Board

End 2010

3 publication Q&A?

Nr

Target

III

New Accreditation
System (NAS)
NAS NL

III.1

Direct contact with
institutions / programmes
and quality Assurance
agencies
Actions for
improvement

Preparing NAS NL
Implementation NAS NL

III.2
III.3
Nr

NAS FL
Academisation FL
Target

IV.
III.4

General strategic
goals
Strategic plan

IV.1

System wide analysis

IV.2

(Initial) accreditation

Preparing NAS VL
Assessment programmes
Actions for
improvement

5 visites
3 meetings
4 Seminars / workshops

Own &
subsidization &
sponsoring

2010-04-12
2011

Board
Policy advisors

Board

Indicator

Resources

Deadline

Owner

Realisation
Chair/Board

Execution preparation
plan
# institutional audit
# programme
accreditation
Preliminary draft Act
# decisions
Indicator

own

2010

New coordinator

Board

NL subsidization

2011

New coordinator Policy
advisors

Board

own
FL subsidization
Resources

2010
2010
Deadline

GL & LB
Policy advisors
Owner

Board
Board
Realisation
Chair/Board

New strategic plan

own

End 2010

Board

Board

Approval new strategic
plan
Preparation of a
structural, internal
process
Implementation

Proces description

Own

End 2010

SWIFT (Information
system)

Board

1ste new analysis

ICT

Spring 2011

Board

Preparation relevant

indicators

Subsidized

End 2010?

HB & working groep
ICT
HP

RD

IV.3

Evaluation among
stakeholders

Nr

Target

V.

Internal operation
core processes & IQA

V.1

Internal operation

V.2

ICT

V.3

Quality & Strategic
Statement

indicators
Extending systematic
evaluation among
stakeholders and follow
up procedure
Implementation
Actions for
improvement
Approval new
organisational chart and
operation
Approval new
organisational chart and
operation
Further dvelopment ICTarchitecture
New Quality & Strategic
Statement

Questionnaires

operation
ICT
Meetings

End 2010?

HP

LB

2011

HP

Board

Deadline

Owner

quality criteria
Discussion evaluation
results
Indicator

Subsidized
operation
Resources

Approval

Subsidized
operation

2010

LK

Realisation
Chair/Board
Board

Approval

Subsidized
operation

2010

LK

Board

Approval contract
award
Approval

Subsidized
operation
Subsidized
operation

End 2010

GL & RD & HB &
working groep ICT
LB & HP & working
groep IQA

Board

End 2010

Board

Attachment 7 Internal Quality Assurance - Report of progress per domain - an example (to be completed in Spring 2012)

Report of progress BELEID, STRATEGIE en LEIDERSCHAP
Quality area
year
First responsible
Strategy, Policy and
2010-2011
LB in consultation with the
Leadership
Executive Board
Global strategic
Specific aims
Results
objectives (P)
(P)
(D)
1a Approved by the Executive
1. Internationalisation 1a. New policy plan
board on 25January 2011
Internationalisation
1b. Improving Communication
internally & externally

1b. -Newsletters, including
Internationalisation, were
published and sent to NL and FL
HE institutions.
-On NVAO intranet activities of
Internationalisation are reported
on a regular basis.
-NVAO has improved its website
in English
-Involvement of more NVAO policy
advisors in International projects
(e.g. Croatia, Armenia, ENQA
workshops and seminars)

Document number in the Quality Manual
To be determined
Documents
(C)
1 1a. ‘Actieplan
Internationalisering
2011’
1b. -NVAO newsletters

-NVAO intranet, pages
Internationalisation
-NVAO website in
English

Discussion
(C)

Goals for improvement
& Plans 2012-2013 (A)
1a.

1b

Global strategic
objectives (P)
2.
Cooperation NL - FL

Specific aims
(P)
2a. Associate degree
programmes

Results
(D)
2a. Assessment of 70 applications
in 2010.

Documents
(C)
2a. List of NVAO
judgments in 2010.

2b. HBO5-programmes

2b. A pilot project completed with
5 programmes (17 December
2010)
Involved in progress and
preparation Decision Flemish
Government (DFG)
Involved in progress and
preparation list of equivalence
Approved Communication plan by
the Executive Board, 2 June 2010

2b. Report Learning
effects, NVAO,
December 2010
2c

2b.

2d. Communication plan,
2 June 2010

2d.

Results
(D)
3a. Implementation of the new
accreditation system on 1 January
2011
3b. Planned implementation of the
new accreditation system per
September 2013
3c. NVAO assessed XX
applications regarding
Academisation in 2010 and 2011

Documents
(C)
3a. Approved new
Frameworks (2010)

2c. Automatic mutual
recognition degrees NL & VL.

2d. More visibility of NVAO

Global strategic
objectives (P)
3.
New Accreditation
System (NAS)

Specific aims
(P)
3a NAS NL

3b. NAS FL

3c. Academisation FL

Discussion
(C)

Goals for improvement &
Plans 2012-2013 (A)
2a.

2c.

Discussion
(C)

Goals for improvement &
Plans 2012-2013 (A)
3a.

3b Current planning
scheme, Spring 2012

3a+b.

3c. List of NVAO
judgments in 2010 and
2011.

3c.

Global strategic
objectives (P)
4.
General strategic
goals

Global strategic
objectives (P)
5.
Internal operation
core processes &
IQA

Specific aims
(P)
4a. Strategic plan

Results
(D)
4a. Approved by the General
Board on 19 February 2012

Documents
(C)
4a. NVAO strategic plan,
19th February 2012

4b. System wide analysis

4b. -NVAO completed eight SWA
in recent years.

4b. Eight reports

4b.

4c. (Initial) accreditation

Preparation relevant indicators

4c. XX

4c.

4d. Evaluation among
stakeholders

4d. Evaluation scheme approved
by the Executive board in April
2011. This scheme is applied
currently.

4d. Evaluation scheme,
April 2011

5a. Internal operation

5b. ICT

5c. Quality & Strategic
Statement

5a. A new organizational chart is
approved and implemented in
February 2011.
5b. The new information system
SWIT is in operation since
January 2012
5c. Approved by the General
Board on 19 February 2012

Documents
(C)
5a. Organisational chart,
February 2011
5b. XX

5c. NVAO strategic plan,
19th February 2012

Discussion
(C)

Discussion
(C)

Goals for improvement &
Plans 2012-2013 (A)
4a.

Goals for improvement &
Plans 2012-2013 (A)
5a..

5b.

Attachment 8 Internal Quality Assurance - Evaluation scheme - April 2011
Process

Aim

Target group

Executed by

Instrument

Frequency and time

Responsible for
task

Institutional audit

First feedback,
improving approach
and process
institutional audit

Level in the institution: Board

Board and
Procescoordinator

Meeting: together with the
evaluation of (initial)
accreditation

After the completion of the
institutional audit. Other
important topics can be
discussed as well.

Board and
procescoordinator

Institutional audit

First feedback,
improving approach
and process
institutional audit
and new
accreditation
system

Resonance group

Board

Meeting Resonance group

Twice a year

Board and chair
Resonance group:
Paul Rullmann

Accreditation

Improving the
content and process
of accreditation

Level in the institution:
Faculty management

Procescoordinator

Questionnaire (in
consultation with the
NVAO Board)

Maximum once a year
(and not to apply for
institutions that apply for
an institutional audit in the
same year). A sample is
used here

Coördinator IQA HP
Communication IJvH

4, 8, 9, 10,
13, 15, 16

Accreditation

Reinforcement
contact Flemish
institutions
Reinforcement
image NVAO
Improvement
content and process
accreditation

Level in the institution:
Managing director Education and
Quality Assurance coordinator
(Flanders)

1.Team (FL/NL) policy
advisors NvS & FW,
possibly supervised by a
board member

1.Meetings structured by
prepared evaluation
questions

Spring 2011

Portfolio holder (LB)

4, 8, 9, 10,
13, 15, 16

2.Team policy advisors
and board members

2.Information meetings in
Flanders

Improvement
content and process
initial accreditation

Level in the institution:
Faculty management or
programme coordinator

Policy advisor /
procescoordinator

Questionnaire (in
consultation with the
NVAO Board)

Coördinator IQA HP
Communication IJvH

4, 8, 9, 10,
13, 15, 16

Quality of the panel
members, student
included

Policy advisor / procescoordinator

Policy advisor /
procescoordinator

List with attention points is
added to the memorandum
of the application that is
offered to the board when
a decision has to be taken

After the completion of the
process of initial
accreditation (and not to
apply for institutions that
apply for an institutional
audit in the same year).
After every procedure
initial accreditation

Coördinator IQA HP

4, 8, 9, 10,
13, 15, 16

Initial accreditation

1

Related SG
ENQA and
ECA

Process

Aim

Target group

Executed by

Instrument

Frequency and time

Responsible for
task

Related SG
ENQA and
ECA

(Initial) accreditation
or institutional audit

Quality external
secretaries

External secretaries

procescoordinator

After every procedure
(initial) accreditation or
institutional audit

Coördinator IQA HP

4, 8, 9, 10, 13,
15, 16

Communication

Image research

Level in the institution:
Faculty management, Head
department, Quality Assurance
coordinators on different levels

Staff Communication

Every 5 years

Coördinator
communication IJvH

8, 9, 10

Communication

Improvement internal
communication and
commitment to
NVAO

All staf of NVAO

Staff Communication

List with attention points is
added to the memorandum
of the application that is
offered to the board when a
decision has to be taken
and a evaluation form has
to be filled in
Questionnaire (in
consultation with the NVAO
Board)
Is part of the meeting
between the institutional
board and NVAO board
Evaluation employee
satisfaction (in consultation
with the NVAO Board)

Every 2 years

Staff management and 8, 9, 10
support IJvH

Internationalisation

Improvement
international
processes and
activities

National en international structural
partners

Policy advisors
Internationalisation

Questionnaire (in
consultation with the NVAO
Board)

Once a year

Coördinator
Internationalisation
MF

Internationalisation

Feedback on the
objectives,
improvement
processes and
activities regarding
the certificate
Internationalisation

Level in the institution:
Programme director and coördinator
internationalisation

Policy advisors
Internationalisation

Meeting

After completion of every
application for a certificate
internationalisation

Coördinator
11
Internationalisation MF

Research masters

Operation
committees Royal
Academy of
Sciences, students
included
Quality training

Chairs committees Royal Academy
of Sciences

Portfolio holder and
responsible policy advisor

Meeting with the chairs of
the committee based on
prepared questions

Two years meeting with the
chairs of the committees
(after the completion of
every application round)

Portfolio holder KD

Questionnaire

After every training

Coördinator IQA HP

Training secretaries

2

Secretaries that attended the
training

trainer

KD en FM

11

4, 8, 9, 10, 13,
15, 16

Attachment 9 Key figures higher education the Netherlands and Flanders
Table I: Overview of key figures of higher education in the Netherlands and Flanders (2009-2010)
The Netherlands
Flanders
Total
Variables / Countries / Total
Inhabitants
16.750.000
6.250.000
23.000.000
Universities
14
6
20
Universities of applied sciences (NL) and
41
21
62
University colleges (FL)
Recognised private institutions (NL) and
60
3
Registered institutions (FL)
Other statutory registered institutions (FL)
4
Programmes
3500
1400
4900
Fulltime students
635.000
195.000
830.000

Table II: Total enrolment higher education in the Netherlands
Enrolment 2009-2010 (x 1000)
Male
Professional Education
193,5
Academic Education
114.3
Total
304,3

Female
209.6
119,0
329,7

Total
403,1
233,3
634,0

Table III: Evolution of enrolment in higher professional education in the Netherlands
Evolution of Enrolment 2009-2010 (x 1000)
Male
Female
2005 – 2006
170,8
186,0
2006 – 2007
175,1
191,6
2007 – 2008
178,5
195,8
2008 – 2009
184,2
199,5
2009 – 2010
193,5
209,6

Total
356,8
366,7
374,4
383,7
403,1

Table IV: Evolution of enrolment in academic education in the Netherlands
Evolution of Enrolment 2009-2010 (x 1000)
Male
Female
2005 – 2006
103,2
102,7
2006 – 2007
103,4
105,2
2007 – 2008
104,6
108,2
2008 – 2009
108,0
112,5
2009 – 2010
114,3
119,0

Total
205,9
208,6
212,7
220,5
233,3

Table V: Total enrolment higher education in Flanders
Enrolment 2009-2010 (x 1000)
Male
Professional Education
37,6
Academic Education
50,6
Total
88,2

Female
52,2
52,7
104,9

Total
89,8
103,3
193,1

Table VI: Evolution of enrolment in higher professional education in Flanders
Evolution of Enrolment 2009-2010 (x 1000)
Male
Female
2005 – 2006
46,9
55,5
2006 – 2007
46,8
55,7
2007 – 2008
47,8
56,4
2008 – 2009
50,5
59,3
2009 – 2010
53,7
62,9

Total
102,4
102,5
104,2
109,8
116,6

Table VII: Evolution of enrolment in academic education in Flanders
Evolution of Enrolment 2009-2010 (x 1000)
Male
2005 – 2006
26,5
2006 – 2007
27,3
2007 – 2008
28,7
2008 – 2009
32,3
2009 – 2010
34,6

Female
32,7
33,5
35,6
39,4
42,0

Total
59,2
60,8
64,3
71,7
76,6

Sources:
1.
[Structuur decreet, 4 April 2003];
2.
Flemish education in figures, 2009-2010, Flemish authorities Education and Training policy area;
3.
[Jaarboek Onderwijs in cijfers CBS, 2009, 2010 en 2011];
4.
www.vsnu.nl
5.
www.vlir.be;
6.
www.vlhora.be;
7.
www.nvao.net.

Attachment 10 Follow-up of the recommendations of the review committee 2007

regarding the accreditation systems in the Netherlands and Flanders
The review committee of 2007 recommended to several issues regarding the higher education system and the
accreditation system in the Netherlands and Flanders. The recommendations concern the following topics:
The Committee recommend further harmonization of regulations and procedures between the two countries with
regard to:
Legal protection of titles
‐
In Flanders it is forbidden by law to provide bachelor’s or master’s programmes or degrees in case of
non-accredited programmes. The degrees of academic programmes have the addition Arts or Science.
The degrees of professional bachelor’s programmes have no addition.
‐
In the Netherlands the Veerman committee recommended on the introduction of clear titles for all
bachelor’s and master’s programmes in academic and professional higher education. The
recommendations were adopted by the Dutch Minister in his strategic statement (2011). The institution
chooses an addition to the title that fits to the profile of the programme. The programme will be
accountable on this matter during the accreditation procedure at NVAO. When this proposal is finalized
into law, the additions Arts and Sciences will be no longer exclusive for academic programmes.
Professional programmes that opt for the addition Arts or Sciences must prove the appropriateness of
these additions during the accreditation procedure.
‐
The Dutch minister is looking for legal sanctions against institutions that offer non-accredited bachelor’s and master’s programmes.
Sanctions in the case of a negative accreditation. The Flemish system (of a statutory repair period) should also be
implemented in the Dutch system
‐
Since the introduction of the new accreditation system in the Netherlands on 1 January 2011 a recovery
period of two years has been included in the system. NVAO is authorized to grant this period in case of a
negative assessment report (regarding accreditation), that can improve sufficiently in two years.
The position of the macro-efficiency check
‐
In 2009 the Dutch Minister decided to reverse the procedure macro-efficiency check and the procedure
initial accreditation. From then on applications for new programmes aiming at funding have to pass the
procedure Macro-efficiency check first (at CDHO20) and secondly, in case of a positive decision from the
Minister, these programme can apply for the procedure initial accreditation at NVAO.
The length of the accreditation cycles
‐
The length the accreditation cycles in the Netherlands and in Flanders differ since the start of the DutchFlemish cooperation in higher education in 2003; six years in the Netherlands, eight years in Flanders.
NVAO has addressed this topic in the meetings of both Ministers [Comité van Ministers] in the recent
years. It is foreseen that the third round of accreditation in Flanders (from 2021 on) will offer an
accreditation cycle of six years.
The scale of assessments. The outcome of the system would benefit if there were clustered assessments on a binational scale
‐
Clustered assessments are current practice in Flanders and in the Netherlands regarding the
assessment of academic programmes. The assessment of professional programmes in the Netherlands
lost the cluster approach in the first phase of the accreditation system. The government prefers,
supported by NAVO, to reintroduce the cluster approach in professional higher education. Initiatives
have been taken meanwhile and it has been decided that the cluster assessment will be reintroduced.
Cluster assessments on a bi-national scale are not planned in the near future. The main objection of a
bi-national scale approach is logistical; the number of involved programmes would be mostly too high,
which causes a very long turnaround of the assessment process.

20

CDHO [Commissie Doelmatigheid Hoger Onderwijs] is an independent advisory body of the Dutch Minister of Education,

Culture en Sciences.

Different time limits for the processing of applications and differing the sanctions (in case NVAO does not adhere
to the time limit)
‐
In the recent years the time limits regarding the treatment of applications did not change in the
Netherlands and Flanders. In the Netherlands the time limit is six months for an application Initial
Accreditation, in Flanders it is four months. In the Netherlands the time limit is three months for an
application accreditation, in Flanders it is four months. In the past years these time limits did not cause a
lot problems, but it must be stated that NVAO could not always respect these time limits, mainly due to
the very high workload as a consequence of several additional tasks. Since 1st October 2009 in the
Netherlands the Act Penalty [Wet Dwangsom] is applicable, which obliges every government
organisation to complete applications in time. Up till now NVAO has not been punished in the cases that
deadlines of applications have not been respected.
In Flanders the programmes have to apply for accreditation at least two months after the public
assessment report and six months before the end of the previous accreditation.
The relationship between NVAO and the quality assessment agencies. There is an issue of ‘low trust’ that needs
to be adjusted
‐
The relation between the quality assurance agencies and NVAO is influenced by the design of the
accreditation system in the Netherlands. The assessment agencies operate on a private basis and are
commissioned by the institutions. Furthermore, the system in the first phase lacked a recovery period,
which sometimes caused anticipatory behaviour of the assessment panels; it seemed that negative
jugdements were avoided to prevent the programme from serious continuity problems, because a
negative decision would mean a shutdown of the programme. As mentioned, in the new system in the
Netherlands a recovery period has been introduced, which is supported by all stakeholders. Moreover, in
the new accreditation system in NL, the panels will be appointed by NVAO and act on behalf of and
reimbursed by NVAO. The institutions will reimburse NVAO.
‐
In Flanders the four party platform (of VIR, VLHORA, the Flemish cabinet and NVAO) aims at reinforcing
the relationship between NVAO and its primary stakeholders. For instance, NVAO will advice VLUHR on
the composition of the panels.
The further development of the system. The committee suggest looking into the concept of self-accrediting
institutions.
‐
The design of the new accreditation system in the Netherlands might provide a first step towards self
accrediting institutions, but that will be decided in about five years. The institutional audit is a new
element of the system that offers opportunities to learn regarding the concept of institutional
accreditation. However, the actual opinion among the stakeholders includes a preference for programme
assessment as a necessary element of the accreditation system. Especially students are strong
proponents of programme assessment.
‐
The first concepts of the new Flemish accreditation system introduce the institutional review. The
provisional outlines of the system in the third phase offer the possibility of self accrediting institutions.

Attachment 11 Accreditation Frameworks in the Netherlands and Flanders

Assessment frameworks
for the higher education
accreditation system

22 November 2011

Contents
1

Structure of the system

6

2

Institutional quality assurance assessment
2.1
Set-up
2.2
Assessment framework for institutional quality
assurance assessments
2.3
Composition of the audit panel
2.4
Assessment process
2.4.1
Administrative consultation
2.4.2
Accreditation portrait
2.4.3
Critical reflection
2.4.4
Site visit
2.4.5
Assessment procedure within the audit panel
2.4.6
Advisory report
2.5
NVAO decision-making
2.6
Required documents
2.6.1
Basic data concerning the institution
2.6.2
Required appendices to the critical reflection
2.6.3
Documents for inspection during the visits

8
8

3

4

Page 3

Limited programme assessment
3.1
Set-up
3.2
Assessment framework for limited programme
assessments
3.3
Composition of the assessment panel
3.4
Assessment process
3.4.1
Critical reflection
3.4.2
Site visit
3.4.3
Assessment procedure within the assessment panel
3.4.4
Assessment report
3.5
NVAO decision-making
3.6
Required documents
3.6.1
Basic data concerning the programme
3.6.2
Required appendices to the critical reflection
3.6.3
Documents made available during the visit
Extensive programme assessment
4.1
Set-up
4.2
Assessment framework for extensive programme
assessments
4.3
Composition of the assessment panel
4.4
Assessment process
4.4.1
Critical reflection
4.4.2
Site visit
4.4.3
Assessment procedure within the assessment panel
4.4.4
Assessment report
4.5
NVAO decision-making
4.6
Required documents

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

9
11
11
11
12
12
13
14
14
15
16
16
16
16
17
17
18
19
20
20
21
22
22
22
23
23
23
24
25
25
26
29
31
31
31
32
32
32
33

4.6.1
4.6.2
4.6.3
5

6

Page 4

Basic data concerning the programme
Required appendices to the critical reflection
Documents made available during the visit

33
33
34

Limited initial accreditation
5.1
Set-up
5.2
Assessment framework for limited initial
accreditations
5.3
Composition of the assessment panel
5.4
Assessment process
5.4.1
Information dossier
5.4.2
Site visit
5.4.3
Assessment procedure within the assessment panel
5.4.4
Advisory report
5.5
NVAO decision-making
5.6
Required documents
5.6.1
Basic data concerning the programme
5.6.2
Required appendices to the information dossier
5.6.3
Documents made available during the visit
(The list of material studied will be incorporated into the
advisory report.)

35
35
36
37
39
39
39
39
40
40
40
41
41
41
41

Extensive initial accreditation
6.1
Set-up
6.2
Assessment framework for extensive initial
accreditations
6.3
Composition of the assessment panel
6.4
Assessment process
6.4.1
Information dossier
6.4.2
Site visit
6.4.3
Assessment procedure within the assessment panel
6.4.4
Advisory report
6.5
NVAO decision-making
6.6
Required documents
6.6.1
Basic data concerning the programme
6.6.2
Required appendices to the information dossier
6.6.3
Documents to be made available during the visit

43
46
47
47
48
48
48
49
49
49
50
50

7

Distinctive features
7.1
Background
7.2
Criteria for distinctive features

51
51
51

8

Assessment scales for programme assessments
8.1
Unsatisfactory
8.2
Satisfactory
8.3
Good
8.4
Excellent

53
53
53
54
54

9

Assessment rules
9.1
Programme assessments

55
55

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

42
42

9.2
10

11

Page 5

Institutional quality assurance assessments

55

Accreditation Decision under the Higher Education and
Research Act
10.1
Conditional initial accreditation and institutional
quality assurance assessment
10.2
Improvement period for accreditation

56
56

Appeals

58

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

56

1 Structure of the system
Accreditation takes place at the programme level. Thus, the accreditation system continues
to focus on the quality of individual programmes. In addition, institutions may request NVAO
to conduct a so-called institutional quality assurance assessment. Should such a thorough
audit reveal that an institution’s quality assurance is in such good order that the quality of
the programmes is systematically improved, wherever necessary, NVAO will then place that
institution in a different accreditation regime. The accreditation methods practised under this
regime differ from those implemented for programmes without a positive institutional quality
assurance assessment. Under this regime, an assessment panel of independent experts
assesses each programme on a limited number of standards pertaining to the essence of
educational quality. On the basis of this assessment, NVAO decides whether or not to
accredit that programme. This leaves the teaching staff free to devote their attention and
energy to expert suggestions for improvement relating to the core of their teaching, rather
than spending time on pre-conditional aspects that are better dealt with at the institutional
level, as the trustworthiness of the institution regarding those themes has already been
demonstrated at the institutional level. This is a system in which:
a. institutional quality assurance assessments bolster an institution-wide internal quality
culture;
b. programme accreditations focus on the essence of the education provided: (improving)
substantive quality;
c. a proper balance is achieved between assessing programmes on the one hand and
quality improvement on the other.
The accreditation system comprises six assessment frameworks:
1.
an institutional-level framework to be used for “institutional quality assurance
1
assessments” , the so-called institutional quality assurance assessment;
2.
a programme-level framework with “limited assessment criteria for the accreditation of
institutions whose institutional quality assurance assessment produced a positive
2
result , the so-called limited programme assessment;
3
3.
a programme-level framework with “extensive assessment criteria for accreditations” ,
the so-called extensive programme assessment (required if an institutional quality
assurance assessment turns out negative and for institutions that have not applied for
an institutional quality assurance assessment);
4.
a programme-level framework with “limited assessment criteria for the initial
accreditation of new programmes provided by institutions whose institutional quality
assurance assessment produced a positive result”4, the so-called limited initial
accreditation;
5.
a programme-level framework with “extensive assessment criteria for the initial
5
accreditation of new programmes” , the so-called extensive initial accreditation
(required if an institutional quality assurance assessment turns out negative and for
institutions that have not applied for an institutional quality assurance assessment);
6.
an assessment framework to determine whether an institution or a programme has any
6
distinctive features.
1 Dutch Higher Education and Research Act (WHW), Articles 5a.13a - 13e.
2 Dutch Higher Education and Research Act (WHW), Article 5a.13f.
3 Dutch Higher Education and Research Act (WHW), Article 5a. 8.
4 Dutch Higher Education and Research Act (WHW), Article 5a.13g.
5 Dutch Higher Education and Research Act (WHW), Article 5a. 10a.
6 Dutch Higher Education and Research Act (WHW), Article 5a.10.

Page 6

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

The five chapters below present the first five assessment frameworks. The following is
successively outlined for each of the assessment frameworks: its set-up, the framework
itself, the composition of the audit panel or the assessment panel, the elements of the
assessment process, the decisions to be taken by NVAO and the minimum documentation
required. Chapter 7 outlines the background and the criteria that apply to the award of a
distinctive feature. Chapter 8 defines the assessment scales that apply to (extensive and
limited) programme assessments and presents examples for the operationalisation of said
scales. Chapter 9 encompasses the assessment rules.
The document at hand concludes by outlining the appeal procedures.
Justification
The frameworks have been formulated on the basis of the European guidelines for the
internal quality assurance of higher education institutions. These guidelines are presented in
Chapter 2 of the Standards and Guidelines for Quality Assurance in the European Higher
Education Area (‘European Standards and Guidelines’; ESG) of the European network for
quality assurance agencies, the European Association for Quality Assurance (ENQA). In
2005, this document was adopted by the education ministers during their meeting in Bergen.
NVAO has established that the requirements with regard to institutional quality assurance
assessment, set by the Minister of Education, Culture and Science (OCW) in his
7
memorandum Focus op quality [Focus on Quality], are in excellent alignment with the ESG.
NVAO has translated and rearranged the ESG in order to shift the focus from quality
assurance to quality enhancement.

7

Dutch Lower Chamber, meeting year 2007–2008, 31 288 and 28 879, no. 21. Cf. Het Hoogste Goed, Strategische agenda voor het
hoger onderwijs-, onderzoeks- en wetenschapsbeleid [The Greatest Good, strategic agenda for higher education, research and science
policy], Parliamentary Document 2007-2008, 31288, no. 1.

Page 7

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

2 Institutional quality assurance assessment

2.1

Set-up
The object of the institutional quality assurance assessment is to determine whether the
board of an institution has implemented an effective quality assurance system, based on its
vision of the quality of the education provided, which enables it to guarantee the quality of
the programmes offered. Institutional quality assurance assessments are not expressly
aimed at assessing the quality of individual programmes.
In essence, institutional quality assurance assessments revolve around five coherent
questions:
1.
What is the vision of the institution with regards to the quality of the education it
provides?
2.
How does the institution intend to realise this vision?
3.
How does the institution gauge the extent to which the vision is realised?
4.
How does the institution work on improvement?
5.
Who is responsible for what?
These five questions have been translated into five standards. Regarding each of these five
standards, the audit panel gives a weighted and substantiated judgement on a three-point
scale: meets, does not meet or partially meets the standard. The audit panel subsequently
gives a substantiated final conclusion on the question of whether an institution is in control
with regard to the quality of its programmes. This judgement is also given on a three-point
scale: positive, negative or conditionally positive.

Page 8

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

2.2

Standard 1:

Assessment framework for institutional quality assurance assessments
Vision of the quality of the education provided
The institution has a broadly supported vision of the quality of its education and the
development of a quality culture.

Explanation:

This vision pertains to the institution’s ambition regarding the quality of its education and its
requirements regarding the quality of its programmes.
For the purpose of developing a quality culture, the board of the institution encourages the
programmes to monitor their quality and implement improvements wherever required. An
active role by all those involved in the education provided is vitally important to this end.

Judgement:

Meets, does not meet or partially meets the standard (weighted and substantiated).

Standard 2:

Policy
The institution pursues an adequate policy in order to realise its vision of the quality of its
education. This comprises at least: policies in the field of education, staff, facilities,
accessibility and feasibility for students with a functional disability, embedding of research in
the education provided, as well as the interrelation between education and the
(international) professional field and discipline.

Explanation:

The policy fields to be assessed are not limited to those stated in the standard but depend
on the institution’s vision of the quality of its education. Adequate policy presupposes
concrete objectives ensuing from said vision and allocation of sufficient resources to
implement said policy.
Anchoring research in the education provided is important because all higher education
institutions have to engage in research to some extent, even if they do not conduct research
themselves and only wish to inform students of new scientific developments in the domain
of the programme in which they are enrolled. This standard expressly does not involve an
assessment of the research itself.

Judgement:

Meets, does not meet or partially meets the standard (weighted and substantiated).

Standard 3:

Output
The institution has insight into the extent to which its vision of the quality of its education is
realised. It gauges and evaluates the quality of its programmes on a regular basis, among
students, staff, alumni and representatives of the professional field.

Explanation:

The institution has management information with regard to the implementation of policy and
the output of its programmes. It also has an adequate system of internal evaluations and
external assessments. The evaluation and gauging activities have been set up efficiently
and provide the board of the institution with aggregated information. Institution-wide
uniformity in the evaluation and gauging activities is not required.

Judgement:

Meets, does not meet or partially meets the standard (weighted and substantiated).

Page 9

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

Standard 4:

Improvement policy
The institution can demonstrate that it systematically improves the quality of its programmes
wherever required.

Explanation:

The institution pursues an active improvement policy based on its insight into the output
achieved. This contributes to the quality culture within the institution.

Judgement:

Meets, does not meet or partially meets the standard (weighted and substantiated).

Standard 5:

Organisation and decision-making structure
The institution has an effective organisation and decision-making structure with regard to
the quality of its programmes, which clearly defines the tasks, authorities and
responsibilities and which encompasses the participation of students and staff.

Explanation:

The organisation and decision-making structure enables the institution to realise its vision
(standard 1), its policy (standard 2), the output (standard 3) and its improvement policy
(standard 4) in a coherent fashion.
The commitment of staff and students is demonstrated by the manner in which they are
consulted and the consideration of their recommendations in the programmes. If laid down
by law, the assessment of this standard also covers the terms of reference and the
positioning of examining boards and programme committees.

Judgement:

Meets, does not meet or partially meets the standard (weighted and substantiated).

General judgement
Based on its vision of the quality of the education provided, the board of the institution has
implemented an effective quality assurance system, which enables it to guarantee the
quality of the programmes offered.
Judgement:

Positive, negative or conditionally positive (weighted and substantiated).
If the audit panel pronounces the judgement of conditionally positive, it will explicitly state
the relevant conditions.

Page 10

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

2.3

Composition of the audit panel
NVAO convenes and appoints an audit panel to conduct the institutional quality assurance
assessment. The institution to be assessed is entitled to lodge substantiated objections to
the composition of the audit panel.
Audit panels must meet the following requirements:
1.
the panel is composed of at least four members, including one student;
2.
the panel commands administrative, educational and audit expertise, is acquainted
with developments in the higher education sector at home and abroad, and is
authoritative;
3.
one of the members with administrative expertise will act as chair;
4.
the panel is independent (its members have had no ties with the institution to be
assessed over at least the past five years).
The audit panel is counselled by an NVAO process co-ordinator and supported by a
secretary. The secretary and the process co-ordinator are also independent of the institution
in question. The secretary and the process co-ordinator do not sit on the panel.
Prior to the first visit, all panel members and the secretary certify to not maintaining any
connections or ties with the institution in question, of either a personal or a professional
nature, which could affect an independent judgement in either a positive or a negative
sense, and to not having had such connections or ties with the institution during the past five
years.
In addition to the factual independence, as expressed above in the nature of the relationship
and the number of years, it is essential for any prospective panel member or secretary to
feel independent. In some cases, an independence of more than five years may not provide
sufficient guarantee for an independent position; a prospective panel member or secretary
could still experience too strong a relationship with the institution or, for example, be
involved too closely with an institution or programme because of family ties. In such cases,
the prospective panel member or secretary cannot sit on the panel. Panel membership
requires a professional attitude. To that end, NVAO has formulated a code of conduct for
panel members and secretaries. Panel members and secretaries will sign the code of
conduct beforehand; after the assessment process, they will sign a declaration drafted by
NVAO that the assessment has been carried out independently.
Stakeholders such as panel members, staff or students may report to NVAO any matters
arising during the assessment process that could affect the independence of the
assessment.

2.4
2.4.1

Page 11

Assessment process
Administrative consultation
NVAO wishes to customise its operations and take account of the diversity in organisational
formats, the desired degree of internationalisation of the audit and the specific nature of an
institution. For that reason, the assessment process starts off with administrative
consultations between the institution and NVAO. These consultations focus on the
institution’s organisational structure with regard to the education it provides, the possible
(international) composition of the audit panel, the language in which the audit must be

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

conducted, the relevant time frame and the material available in the institution for the
purpose of the institutional quality assurance assessment. If so desired, an institution may
use the institutional quality assurance assessment to assess the quality assurance of socalled non-degree programmes.
2.4.2

Accreditation portrait
Based on the administrative consultations with the institution, NVAO starts off by drawing up
an “accreditation portrait”. The accreditation portrait is based on the accreditation decisions
NVAO has taken with regard to new and existing programmes during the initial stage of the
accreditation system. Any side letters and other information available to NVAO are also
factored in. The accreditation portrait is handed over to the audit panel. It is submitted to the
institution in question beforehand and the institution is given an opportunity to respond.

2.4.3

Critical reflection
The institution draws up a critical reflection. In essence, the critical reflection answers the
question of how the institution demonstrates that it is in control of the quality of the
programmes offered. The critical reflection follows the standards outlined for the institutional
quality assurance assessment framework, whereby the institution’s strengths and
weaknesses are described by reference to notable examples. The critical reflection is a selfcontained document that can be read separately.
The critical reflection contains a number of basic data on the institution and its programmes.
These enable the audit panel to gain a global picture of the institution. (The required basic
data are listed in paragraph 2.6.)
When considering the standards, the institution itself is expected to set a course for its
vision and policies, whereupon it is up to the audit panel to assess to what extent the
institution manages to achieve its ambitions. This means, for example, that an audit panel
must check whether an institution whose vision indicates that it wishes to focus on
internationalisation of – or in – its programmes not only develops policy and makes
resources available to that end but also evaluates and wherever necessary adjusts said
policy.
A similar reasoning applies to aspects such as teaching strategies, prior experiential
learning, input from the professional field etcetera. If these have been incorporated in the
vision and policy regarding the programmes, they should also be covered by the evaluation,
the information gathering, an assessment of the quality achieved and measures for
improvement, if any.
As for staff and facilities, the institutional quality assurance assessment involves assessing
the policy and procedures in place with regard to staff and the facilities rather than their
programme-specific realisation. Consequently, the policy and procedures must be specified
in the critical reflection. The implementation of policy is considered in the institutional quality
assurance assessment but programme-specific substantive matters such as, for example,
teachers’ subject expertise are only considered in limited programme assessments.
In its assessment, the audit panel limits itself to the institution’s policy regarding the
effectiveness of the quality assurance system in place for the programmes. The panel does
not judge the quality of the programmes.

Page 12

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

For the purpose of describing its output, an institution can include relevant public
information, for example from the Keuzegids [Guide to Higher Education Courses] or
Studiekeuze123 [www.studychoice.nl].
The critical reflection comprises a maximum of 50 pages; it has very few appendices and
any appendices are limited in size. (The required appendices are listed in paragraph 2.6.2.)
2.4.4

Site visit
The site visit for the purpose of the institutional quality assurance assessment comprises
two components and takes a total of at least two to, in principle, five days. The panel may
extend its visit if prompted by the circumstances. As a rule, the audit panel starts off by
visiting the institution for a day, followed by a second visit after two to four weeks. The
institution and the panel may decide to divert from this set-up by mutual agreement.
First visit: exploration
Prior to the first visit, the audit panel has studied the institution’s critical reflection and the
accreditation portrait. Prior to its first visit, the panel discusses the questions it intends to put
to the discussion partners. During the preliminary meeting, the audit panel also discusses a
number of documents underpinning the critical reflection.
The first visit has an exploratory nature. The audit panel gains insight into the ins and outs
of the institution, the specific points for attention of the board of the institution and
satisfaction among students, teaching staff and other stakeholders. It identifies the topics to
be investigated in more detail.
During the first visit, the audit panel will, in any case, meet with the following discussion
partners:

the board of the institution (and, if the institution so desires: a representative from the
supervisory board);

the managers responsible for education;

quality assurance experts and other relevant staff;

teachers from representative bodies;

students from representative bodies;

if relevant: representatives from the professional field.
The schedule for the visit is drawn up by the NVAO process co-ordinator in consultation with
the chair of the audit panel and the contact person of the institution. The panel determines
the structure and organisation of the visit. It decides at his own discretion which teachers
and which students it would like to see and which documents it would like to examine. In
principle, the following pre-conditions are observed:

The meetings take 45 – 60 minutes.

In principle, the delegations of the institution comprise no more than six people.

In between the meetings, the audit panel takes time to deliberate.
In addition, the panel will set aside time for open consultations. The institution and the panel
will make these open consultations widely known, both prior to and during the visit.
At the end of the first visit, the chair of the audit panel provides brief feedback information to
the institution. This feedback presents the panel’s first impressions of the quality assurance
in place in the institution. In addition, it indicates the audit trails to be conducted. Audit trails
are studies pertaining to the implementation of policy and/or the management of problems,

Page 13

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

in which the audit panel follows the trail from the institutional level to the implementation
level or vice versa. The audit panel decides which topics to consider in the audit trails and
which individuals it would like to see to that end. The institution may point out well-founded
other options to the audit panel or request the panel to conduct an additional audit trail.
In order to minimise the workload for the institution, the audit panel gives specific
instructions regarding the documents to be studied for the audit trails and the required
discussion partners.
The institution prepares the second visit in consultation with the NVAO process co-ordinator.
The panel ultimately decides on the structure and the organisation of the visit.
Second visit: in-depth study
During the second visit to the institution, a further discussion takes place between the audit
panel and representatives of the institution regarding points for attention emerging from the
meetings and the documents studied during the first visit. This discussion enables the audit
panel to ascertain whether its initial impressions were correct. Again, the delegations of the
institution comprise a maximum of, in principle, six persons.
The audit panel needs to demonstrate how it has ascertained whether an institution’s quality
assurance system works. This is where the audit trails come in. There are vertical and
horizontal audit trails. It should be noted in this regard that institutional quality assurance
assessments expressly do not involve assessing programmes but rather assessing the
functioning of the quality assurance system in relation to the programmes.
In a vertical audit trail, the panel examines to what degree an element of the vision referred
to in standard 1 is actually put into practice in two or more programmes. All standards in the
framework are considered in succession. For example, the panel may examine whether the
intended international exchange of students is actually effected.
Horizontal audit trails focus on the realisation of a single standard (in other words: a
component of the quality assurance system) in a number of programmes. For example, the
functioning of programme committees or the monitoring of measures for improvement.
At the end of the second visit, the chair of the audit panel provides brief feedback to the
institution regarding the general judgement and the underpinning considerations.
2.4.5

Assessment procedure within the audit panel
The audit panel presents a judgement regarding all the standards contained in the
assessment framework. This judgement is based on an appraisal of the positive and critical
elements in the panel’s findings. Options for the judgement are: meets, does not meet or
partially meets the standard. Subsequently, the panel formulates a general, weighted and
substantiated judgement on the question of whether an institution is in control of the quality
of the programmes it offers. That judgement is also given on a three-point scale: positive,
negative or conditionally positive.

2.4.6

Advisory report
The audit panel secretary draws up an advisory report comprising 20 to 30 pages. The main
content of the report is made up of the panel’s judgements regarding the standards. It is
important for the audit panel to include underpinnings based on the institution’s critical
reflection, the meetings with representatives of the institution and the underlying data from
the documents studied. For that reason, the advisory report will include notable and
representative examples.
The advisory report is preceded by a general consideration of the judgement regarding the
institution’s quality assurance comprising a maximum of two pages. Any measures for

Page 14

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

improvement will be presented in a separate paragraph of the advisory report. In addition,
the report contains a score table with the judgements emerging from the institutional quality
assurance assessment, information on the dates of the site visits, the names of the
discussion partners, a number of basic data concerning the institution (see paragraph 2.6),
an overview of the material studied and the declarations of independence signed by the
panel members and the secretary. In its report, the panel gives an account of the manner in
which it has organised its visit and how it has arrived at its choice of discussion partners and
documents.
At the end of the second visit, the contents of the advisory report are discussed and
tentatively laid down by the audit panel members.
NVAO forwards the advisory report to the board of the institution once all panel members
have approved its contents. The institution is given a term of two weeks to respond to any
factual inaccuracies in the report, whereupon the chair of the audit panel endorses the
report after all panel members have approved its contents. The report is signed by the chair
and the secretary of the panel and submitted to NVAO for decision-making. If NVAO finds
that a report raises questions or if an institution so desires, NVAO may invite the institution
and/or the audit panel for further consultations.

2.5

NVAO decision-making
A “positive” judgement results in approval by NVAO for a term of six years. This means that
the programme assessments may be conducted in accordance with the limited programme
assessment framework.
A “conditionally positive” judgement results in approval by NVAO under resolutive
8
conditions for a term of one year. During that year, the programme assessments my be
conducted in accordance with the limited programme assessment framework. A positive
accreditation decision or a positive initial accreditation decision based on the limited
framework will subsequently be valid for one year. When given the “conditionally positive”
judgement, the institution must acquire a positive judgement within a year, whereby NVAO
ascertains whether the institution meanwhile meets the conditions set. If the institution fails
to apply for an additional judgement or does not meet the conditions, the positive judgement
expires.
If within that year the institution, in the opinion of NVAO, manages to satisfy the conditions
set on the basis of the institutional quality assurance assessment, the validity of a positive
accreditation decision or positive initial accreditation decision will be extended to six years.
Satisfaction of the conditions set will be assessed by an audit panel commissioned by
NVAO. The additional assessment will basically be carried out in accordance with the
procedure for regular institutional quality assurance assessments. The audit panel will focus
on the conditions set earlier.9

8

Article 5a,13d, paragraph 6, Dutch Higher Education and Research Act; Accreditation Decree of the Dutch Higher Education and
Research Act.

9

Page 15

Article 5a,13d, paragraph 6 in conjunction with Article 5a,13e of the Dutch Higher Education and Research Act

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

A “negative” judgement results in the withholding of approval for a minimum period of three
years. Any programme assessments will be carried out in accordance with the extensive
programme assessments regime. Programmes that have been accredited during the
introduction regime based on a limited assessment or have passed initial accreditation must
undergo additional assessment if approval is withheld following an institutional quality
assurance assessment. The institution must apply to NVAO for such additional assessment.
A decision regarding such applications will be taken within three months.

2.6

Required documents
During the assessment process, the institution will provide the audit panel with a limited
number of documents. NVAO assumes that these are existing documents, available within
the institution, rather than documents prepared especially for the institutional quality
assurance assessment. The documents serve as a substantiation and if need be as
verification. Other material is only required when explicitly requested by the panel or if the
institution wishes to demonstrate a particular distinctive feature.

2.6.1

Basic data concerning the institution
(The basic data is incorporated into the critical reflection, the advisory report and the NVAO
decision.)
1.
2.
3.
4.

2.6.2

Required appendices to the critical reflection
(The list with appendices studied will be included in the advisory report.)
1.
2.

2.6.3

Mission and/or view regarding the education provided and, if available, the institution’s
latest strategic policy plan;
Organisation chart.

Documents for inspection during the visits
(The list with material studied will be included in the assessment report.)
1.
2.
3.
4.
5.
6.
7.
8.

Page 16

Name of the institution;
Status of the institution (publicly funded or legal body providing higher education);
Location(s);
Overview with all programmes, enrolment figures and staff numbers.

Education policy plan or similar document(s);
Policy plan regarding research in relation to the programmes offered or similar
document(s);
Staff (policy) plan or similar document(s);
Facilities plan or similar document(s);
Policy plan regarding the accessibility and feasibility for students with a functional
disability;
Quality assurance plan;
Summary and analysis of recent evaluation results and relevant management
information;
Institution’s annual report on “appeals against examinations”.

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

3 Limited programme assessment

3.1

Set-up
The framework for limited assessments of existing programmes is used for institutions that
have obtained a positive judgement following an institutional quality assurance assessment.
The assessment is based on a discussion with peers regarding the content and quality of
the programme. It focuses on three questions:
1.
2.
3.

What is the programme aiming for?
How is the programme realising this aim?
Is the programme achieving its objectives?

These three questions have been translated into three standards. Regarding each of these
three standards, an assessment panel gives a substantiated judgement on a four-point
scale: unsatisfactory, satisfactory, good or excellent. The panel subsequently gives a
substantiated final conclusion regarding the overall quality of the programme, on the same
four-point scale.

Page 17

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

3.2

Standard 1:

Assessment framework for limited programme assessments
Intended learning outcomes
The intended learning outcomes of the programme have been concretised with regard to
content, level and orientation; they meet international requirements.

Explanation:

As for level and orientation (bachelor’s or master’s; professional or academic), the intended
learning outcomes fit into the Dutch qualifications framework. In addition, they tie in with the
international perspective of the requirements currently set by the professional field and the
discipline with regard to the contents of the programme.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 2:

Teaching-learning environment
The curriculum, staff and programme-specific services and facilities enable the incoming
students to achieve the intended learning outcomes.

Explanation:

The contents and structure of the curriculum enable the students admitted to achieve the
intended learning outcomes. The quality of the staff and of the programme-specific services
and facilities is essential to that end. Curriculum, staff, services and facilities constitute a
coherent teaching-learning environment for the students.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 3:

Assessment and achieved learning outcomes
The programme has an adequate assessment system in place and demonstrates that the
intended learning outcomes are achieved.

Explanation:

The level achieved is demonstrated by interim and final tests, final projects and the
performance of graduates in actual practice or in post-graduate programmes. The tests and
assessments are valid, reliable and transparent to the students.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

General conclusion
The quality of the programme is
Judgement:

Page 18

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

The assessment is based on the following definitions. These definitions relate to both the
scores obtained for the individual standards and the overall scores awarded to the
programme.
Generic quality
The quality that can reasonably be expected in an international perspective from a higher
education bachelor’s or master’s programme.
Unsatisfactory
The programme does not meet the current generic quality standards and shows serious
shortcomings in several areas.
Satisfactory
The programme meets the current generic quality standards and shows an acceptable level
across its entire spectrum.
Good
The programme systematically surpasses the current generic quality standards across its
entire spectrum.
Excellent
The programme systematically well surpasses the current generic quality standards across
its entire spectrum and is regarded as an (inter)national example.
Chapter 8 presents examples for the operationalisation of these assessments.
3.3

10

Composition of the assessment panel

It is imperative that assessment panels are composed in a manner allowing meaningful
discussions among peers, in which the panel remains sufficiently independent. The
institution convenes the panel, appoints a secretary and subsequently presents the panel to
NVAO for approval. To that end the institution provides data on the expertise and
independence of the panel members and the secretary, in a manner stipulated by NVAO.
The institution may also commission an external quality assessment agency to convene a
panel; in such cases the panel must also be presented to NVAO for approval.
The panel secretary has completed NVAO training leading to certification. Every year,
NVAO publishes a list of NVAO certified secretaries.
Assessment panels must meet the following requirements.
1.

The panel is composed of a minimum of four members, among whom at least two
authoritative domain experts11 and a student.

2.

Overall, the panel commands the following expertise:
a. expertise regarding developments in the discipline,
b. international expertise,

10

This paragraph is explained in detail in the guideline titled Eisen aan de panelsamenstelling [Requirements for the composition of
panels]. This guideline contains detailed requirements to be met by panels. It also contains a submission procedure, a form to be filled out
by the institution and a code of conduct for panel members.

11

Domain expertise is understood to mean specialist expertise, international expertise or professional expertise.

Page 19

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

c.
d.

e.
f.

practical expertise in the professional field relevant to the programme (if
applicable),
experience in teaching and educational development at the relevant
programme level and expertise regarding the teaching format(s) used in
the programme12,
student-related expertise,
assessment or audit expertise.

3.

The panel is independent (its members have not had any ties with the institution
providing the programme for at least the past five years).

4.

The panel is assisted by an independent, external secretary trained and certified by
NVAO. The secretary does not sit on the panel.

Prior to the visit, all panel members and the secretary certify to not maintaining any
connections or ties with the institution in question, either as a private individual or as a
researcher / teacher, professional or adviser, which could affect an independent judgement
of the quality of the programme in either a positive or a negative sense, and to not having
had such connections or ties with the institution during the past five years.
In addition to the factual independence, as expressed above in the nature of the relationship
and the number of years, it is essential for any panel member or secretary to feel
independent.
Panel members and secretaries will sign a declaration of independence and confidentiality
prior to the assessment process. In this declaration, they attest to having taken note of the
code of conduct. Following the assessment process, the chair and secretary sign the
assessment report once all panel members have read and approved the report. The report
includes a declaration that the assessment has been carried out independently.
Stakeholders such as panel members, staff or students may report to NVAO any matters
arising during the assessment process that could affect the independence of the
assessment or pertain to other complaints regarding the panels or secretaries.

3.4
3.4.1

Assessment process
Critical reflection
For the purpose of the assessment by the assessment panel, the programme presents a
critical reflection of the programme. The critical reflection should follow the standards
outlined for the limited programme assessment framework and describes the programme’s
strengths and weaknesses. In its critical reflection, the programme outlines how it checks
student and staff satisfaction and reports on the results. Underpinning documents are made
available for the panel to inspect. In addition, the report indicates which measures for
improvement have been taken following the previous assessment. The critical reflection is a
self-contained document that can be read separately.

12

This includes, for example, distance learning, work-related courses, flexible education, skill-oriented education or education aimed at
excellent students.

Page 20

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

The assessment framework for limited programme assessments is structured in a manner
allowing programmes ample scope to emphasise their unique character. The programme
may use that scope in the critical reflection. The critical reflection is the pre-eminent tool to
allow teachers and peers to comment on the contents of the programme. Therefore it must
be a document in which teachers and students recognise the programme.
In addition, the assessment framework offers opportunities to discuss the ambitions of the
programme during the site visit, rather than focus on the results obtained in the past. What
choices will the programme make for the future, what direction will it take? In order to
conduct such discussions, the assessment panel is expected to be able to reflect on the
programme’s plans for the future, together with the programme’s representatives.
It is imperative that any overlap with assessments within the context of the institutional
quality assurance assessment is avoided when drawing up the critical reflection and during
the assessment procedure. Should any reference to institutional policy or, for example,
departmental policy be necessary, programme assessments strictly focus on the fitness for
purpose of the policy pursued regarding the programme in question. This does not include
pre-conditional matters, such as the structure of quality assurance or the institution’s staff
policy; these are considered in institutional quality assurance assessments.
The critical reflection comprises a maximum of 25 pages, excluding appendices.
3.4.2

Site visit
In principle, the required site visit for the purpose of a limited programme assessment takes
one day. In the event of a collective assessment of comparable programmes within a single
institution, the duration may be reduced proportionally.
Prior to the visit, the assessment panel has studied a number of final projects in order to
gain insight into the exit level attained in the programme. To that end, a selection is made
from a comprehensive overview drawn up by the programme. The final projects, the
relevant assessment criteria and the requirements are forwarded to the panel members
prior to the visit, or the panel members examine the documents on site prior to the visit.
Prior to the visit, the panel members form a preliminary opinion about the programme and
draw up questions for their site visit. The panel factors the outcomes of the institutional
quality assurance assessment into its judgement.
During the site visit, the assessment panel will, in any case, meet with the programme
management, members of the examining board and the programme committee, teachers,
students, alumni and wherever relevant representatives of the professional field. In addition,
the panel examines the material made available by the programme. The panel determines
the exact scope of the discussions, the possible clustering of discussion participants and the
further organisation of the visit. The panel decides at its own discretion which teachers and
students it would like to see and which documents it would like to examine. In principle, the
programme delegations comprise no more than six persons. The panel will set aside time
for open consultations. The programme and the panel will make these open consultations
widely known, both prior to and during the visit. In addition, the panel may visit lectures or
other teaching-learning situations, such in consultation with the programme.

Page 21

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

At the end of the site visit, the chair of the assessment panel provides brief feedback
information to the programme regarding the general judgement and the underlying
considerations.
3.4.3

Assessment procedure within the assessment panel
The assessment panel presents its judgement regarding all the standards incorporated in
the assessment framework. This judgement is substantiated by an appraisal of the positive
and critical elements from the panel’s findings. The judgement may be: unsatisfactory,
satisfactory, good or excellent. The panel subsequently formulates a general, weighted and
substantiated judgement regarding the quality of the programme. This judgement is also
given on a four-point scale, ranging from unsatisfactory to excellent.

3.4.4

Assessment report
The assessment panel secretary draws up an assessment report comprising some 20
pages. The main content of the report features the panel’s judgements regarding the
standards. It is important for the audit panel to include underpinnings based on the
programme’s critical reflection, the meetings with representatives of the programme and the
underlying data from the documents made available. The report will include significant and
representative examples. In its report, the panel gives an account of the manner in which it
has organised its visit and how it has arrived at its choice of discussion partners and
documents.
The assessment report is preceded by a summary judgement regarding the quality of the
programme comprising a maximum of two pages. Any measures for improvement will be
presented in a separate paragraph. In addition, the report contains a score table with the
panel judgements, information on the date(s) of the site visit, the names of the discussion
partners, basic data concerning the programme (see paragraph 3.6), an overview of the
material studied and the declarations of independence signed by the panel members and
the secretary.
The assessment panel secretary forwards the advisory report to the board of the institution
once all panel members have approved its contents. The institution is given the opportunity
to respond to any factual inaccuracies in the report, whereupon the panel chair endorses the
report after all panel members have taken note of and approved its contents. The report is
signed by the chair and the secretary of the panel.

3.5

NVAO decision-making
The board of the institution applies to NVAO for accreditation based on the assessment
report. NVAO may decide to accredit the programme, not accredit it or grant an
improvement period. The Accreditation Decree of the Dutch Higher Education and Research
Act stipulates how, on what grounds and under what circumstances NVAO may grant an
improvement period.
Because of the limited nature of the assessment, NVAO exercises more reticence in
reviewing the assessment report than it does with extensive programme assessments. The
positive assessment of the institution inspires sufficient confidence that the quality
assurance regarding the quality of the education provided by the institution is effectively
guaranteed. In addition, its prior approval of the assessment panels and the fact that these

Page 22

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

panels’ secretaries have been trained and certified provides sufficient context and certainty
to resort to a more reticent review of the assessment report. The review focuses on the
completeness and validity of the assessment report.

3.6

Required documents
During the assessment process, the programme provides the assessment panel with a
limited number of documents. NVAO assumes that these are existing documents, available
within the institution, rather than documents prepared especially for the programme
assessment. The documents serve as a substantiation and if need be as verification. Other
material is only required when explicitly requested by the panel or if the programme wishes
to demonstrate a particular distinctive feature.

3.6.1

Basic data concerning the programme
(The basic data is incorporated into the critical reflection, the assessment report and the
NVAO decision.)
Administrative data regarding the programme
1.
Nomenclature of the programme in CROHO [central register of higher education
programmes];
2.
Orientation and level of the programme;
3.
Number of credits;
4.
Specialisations;
5.
Location(s);
6.
Mode (s) of study;
7.
CROHO registration number.
Administrative data regarding the institution
1.
Name of the institution;
2.
Status of the institution (publicly funded or legal body providing higher education);
3.
Outcome of the institutional quality assurance assessment.
Quantitative data regarding the programme
1.
Data on intake, transfers and graduates pertaining to – if possible – the last six
cohorts;
2.
Teacher -student ratio achieved;
3.
Average amount of face-to-face instruction per stage of the study programme (a stage
can be expressed in, for example, regular years of study, the work placement and the
graduation period).

3.6.2

Required appendices to the critical reflection
(The list of appendices studied will be incorporated into the assessment report.)
1.
2.
3.

4.

Page 23

Subject-specific reference framework and the learning outcomes of the programme;
Overview of the curriculum in diagram form;
Outline description of the curriculum components, stating learning outcomes,
attainment targets, teaching method(s), assessment method, literature
(mandatory/recommended), teacher and credits;
Teaching and examination regulations;

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

(Items 2 to 4 are usually reflected in a study guide, in which case this can be annexed to the
report.)
5.
6.
7.
8.
3.6.3

Documents made available during the visit
(The list of material studied will be incorporated into the assessment report.)
1.
2.

3.
4.
5.
6.

Page 24

Overview of allocated staff with names, positions, scope of appointment, level and
expertise;
List of the last 25 final projects or the final projects of the past two years (or portfolios /
projects demonstrating the exit levels attained by the students);
Overview of the contacts maintained with the professional field (if relevant);
Report on the institutional quality assurance assessment.

Reports on consultations in relevant committees / bodies;
Test questions with corresponding assessment criteria and requirements (answer
models) and a representative selection of actual tests administered (such as
presentations, work placements, portfolio assessments) and assessments;
Representative selection of final projects, selected by the panel, of the past two years
with corresponding assessment criteria and requirements;
Reference books and other learning materials;
Summary and analysis of recent evaluation results and relevant management
information;
Documentation regarding teacher and student satisfaction.

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

4 Extensive programme assessment

4.1

Set-up
The framework for extensive assessments of existing programmes is used for institutions
that have failed to obtain a positive judgement following an institutional quality assurance
assessment. The assessment is based on a discussion with peers regarding the content
and quality of the programme. It focuses on six questions:
1.
2.
3.
4.
5.
6.

What is the programme aiming for?
With what curriculum?
With what staff?
With what services and facilities?
How does the programme intend to safeguard quality?
Is the programme achieving its objectives?

These six questions have been translated into six themes and 16 standards. Regarding
each of these standards, an assessment panel gives a substantiated judgement on a fourpoint scale: unsatisfactory, satisfactory, good or excellent. The panel subsequently gives a
substantiated final conclusion regarding the overall quality of the programme, on the same
four-point scale.

Page 25

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

4.2

Standard 1:

Assessment framework for extensive programme assessments
Intended learning outcomes
The intended learning outcomes of the programme have been concretised with regard to
content, level and orientation; they meet international requirements.

Explanation:

As for level and orientation (bachelor’s or master’s; professional or academic), the intended
learning outcomes fit into the Dutch qualifications framework. In addition, they tie in with the
international perspective of the requirements currently set by the professional field and the
discipline with regard to the contents of the programme.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 2:

Curriculum
The orientation of the curriculum assures the development of skills in the field of scientific
research and/or the professional practice.

Explanation:

The curriculum has demonstrable links with current developments in the professional field
and the discipline.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 3:

The contents of the curriculum enable students to achieve the intended learning outcomes.

Explanation:

The learning outcomes have been adequately translated into attainment targets for
(components of) the curriculum. Students follow a study curriculum which is coherent in
terms of content.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 4:

The structure of the curriculum encourages study and enables students to achieve the
intended learning outcomes.

Explanation:

The teaching concept is in line with the intended learning outcomes and the teaching
formats tie in with the teaching concept.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 5:

The curriculum ties in with the qualifications of the incoming students.

Explanation:

The admission requirements are realistic with a view to the intended learning outcomes.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Page 26

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

Standard 6:

The curriculum is feasible.

Explanation:

Factors pertaining to the curriculum and hindering students’ progress are removed as far as
possible. In addition, students with functional disabilities receive additional career tutoring.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 7:

The programme meets statutory requirements regarding the scope and duration of the
curriculum.

Explanation:

Scope and duration:
Bachelor’s programmes (professional orientation): 240 credits;
Bachelor’s programmes (academic orientation): in principle, a minimum of 180 credits;
Master’s programmes (professional orientation): in principle, a minimum of 60 credits;
Master’s programmes (academic orientation): in principle, a minimum of 60 credits,
depending on the programme.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 8:

Staff
The programme has an effective staff policy in place.

Explanation:

The staff policy provides for the qualifications, training, assessment and size of the staff
required for the realisation of the curriculum.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 9:

The staff is qualified for the realisation of the curriculum in terms of content, educational
expertise and organisation.

Explanation:

The factual expertise available among the staff ties in with the requirements set for
professional or academic higher education programmes.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 10:

The size of the staff is sufficient for the realisation of the curriculum.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 11:

Services and facilities
The accommodation and the facilities (infrastructure) are sufficient for the realisation of the
curriculum.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 12:

Tutoring and student information provision bolster students’ progress and tie in with the
needs of students.

Page 27

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 13:

Quality assurance
The programme is evaluated on a regular basis, partly on the basis of assessable targets.

Explanation:

The programme ensures the quality of the intended learning outcomes, the curriculum, the
staff, the services and facilities, the assessments and the learning outcomes achieved
through regular evaluations. The programme also collects management information
regarding the success rates and the staff-student ratio.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 14:

The outcomes of these evaluations constitute the basis for demonstrable measures for
improvement that contribute to the realisation of the targets.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 15:

Programme committees, examining boards, staff, students, alumni and the relevant
professional field of the programme are actively involved in the programme’s internal quality
assurance.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Standard 16:

Assessment and learning outcomes achieved
The programme has an adequate assessment system in place and demonstrates that the
intended learning outcomes are achieved.

Explanation:

The level achieved is demonstrated by interim and final tests, final projects and the
performance of graduates in actual practice or in subsequent programmes. The tests and
assessments are valid, reliable and transparent to the students.

Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).

Page 28

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

General conclusion
The quality of the programme is
Judgement:

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated).
The assessment is based on the following definitions. These definitions relate to both the
scores obtained for the individual standards and the overall scores awarded to the
programme.
Generic quality
The quality that can reasonably be expected in an international perspective from a higher
education bachelor’s or master’s programme.
Unsatisfactory
The programme does not satisfy the current generic quality standards and shows serious
shortcomings in several areas.
Satisfactory
The programme satisfies the current generic quality standards and shows an acceptable
level across its entire spectrum.
Good
The programme systematically surpasses the current generic quality standards across its
entire spectrum.
Excellent
The programme systematically well surpasses the current generic quality standards across
its entire spectrum and is regarded as an (inter)national example.
Chapter 8 presents examples for the operationalisation of these assessments.

4.3

13

Composition of the assessment panel

It is imperative that assessment panels are composed in a manner allowing meaningful
discussions among peers, in which the panel remains sufficiently independent. The
institution convenes the panel, appoints a secretary and subsequently presents the panel to
NVAO for approval. To that end the institution provides data on the expertise and
independence of the panel members and the secretary, in a manner stipulated by NVAO.
The institution may also commission an external quality assessment agency to convene a
panel; in such cases the panel must also be presented to NVAO for approval.
The panel secretary has completed NVAO training leading to certification. Every year,
NVAO publishes a list of NVAO certified secretaries.
Assessment panels must meet the following requirements.
13

This paragraph is explained in detail in the guideline titled Eisen aan de panelsamenstelling [Requirements for the composition of
panels]. This guideline contains detailed requirements to be met by panels. It also contains a submission procedure, a form to be filled out
by the institution and a code of conduct for panel members.

Page 29

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

1.

The panel is composed of a minimum of four members, among whom at least two
authoritative domain experts14 and a student.

2.

Overall, the panel commands the following expertise:
a. expertise regarding developments in the discipline,
b. international expertise,
c. practical expertise in the professional field relevant to the programme (if
applicable),
d. experience in teaching and educational development at the relevant
programme level and expertise regarding the teaching format(s) used in
the programme15,
e. student-related expertise,
f. assessment or audit expertise.

3.

The panel is independent (its members have not had any ties with the institution
providing the programme for at least the past five years).

4.

The panel is assisted by an independent, external secretary trained and certified by
NVAO. The secretary does not sit on the panel.

Prior to the visit, all panel members and the secretary certify to not maintaining any
connections or ties with the institution in question, either as a private individual or as a
researcher / teacher, professional or adviser, which could affect an independent judgement
of the quality of the programme in either a positive or a negative sense, and to not having
had such connections or ties with the institution during the past five years.
In addition to the factual independence, as expressed above in the nature of the relation and
the number of years, it is essential for any panel member or secretary to feel independent.
In some cases, an independence of more than five years may not provide sufficient
guarantee for an independent position; a prospective panel member or secretary could still
experience too strong a relationship with the institution or be involved too closely with an
institution or programme, for example because of family ties. In such cases, the prospective
panel member or secretary cannot sit on the panel. Panel membership requires a
professional attitude. To that end, NVAO has formulated a code of conduct for panel
members and secretaries. This code of conduct comprises elements pertaining to the
independence, confidentiality and attitude of the panel members and the secretary during
the assessment process.
Panel members and secretaries will sign a declaration of independence and confidentiality
prior to the assessment process. In this declaration, they attest to having taken note of the
code of conduct. Following the assessment process, the chair and secretary sign the
assessment report once all panel members have read and approved the report. The report
includes a declaration that the assessment has been carried out independently.

14

Domain expertise is understood to mean specialist expertise, international expertise or professional expertise.

15

This includes, for example, distance learning, work-related courses, flexible education, skill-oriented education or education aimed at
excellent students.

Page 30

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

Stakeholders such as panel members, staff or students may report to NVAO any matters
arising during the assessment process that could affect the independence of the
assessment or pertain to other complaints regarding the panels or secretaries.
4.4
4.4.1

Assessment process
Critical reflection
For the purpose of the assessment by the assessment panel, the programme presents a
critical reflection of the programme. The critical reflection follows the standards outlined for
the limited programme assessment framework and describes the programme’s strengths
and weaknesses. In its critical reflection, the programme outlines how it checks student and
staff satisfaction and reports on the results. Underpinning documents are made available for
the panel to inspect. In addition, the report indicates which measures for improvement have
been taken following the previous assessment. The critical reflection is a self-contained
document that can be read separately.
The critical reflection is the pre-eminent tool to allow teachers and peers to comment on the
contents of the programme. Therefore, it must be a document in which teachers and
students recognise the programme.
In addition, the assessment framework offers opportunities to discuss the ambitions of the
programme during the site visit, rather than focus on the results obtained in the past. What
choices will the programme make for the future, what direction will it take? In order to
conduct such discussions, the assessment panel is expected to be able to reflect on the
programme’s plans for the future, together with the programme’s representatives.
The critical reflection comprises a maximum of 40 pages, excluding appendices.

4.4.2

Site visit
The required site visit for the purpose of an extensive programme assessment takes about
two days. In the event of a collective assessment of comparable programmes within a single
institution, the duration may be reduced proportionally.
Prior to the visit, the assessment panel has studied a number of final projects in order to
gain insight into the exit level attained in the programme. To that end, a selection is made
from a comprehensive overview drawn up by the programme. The final projects, the
relevant assessment criteria and the requirements are forwarded to the panel members
prior to the visit, or the panel members examine the documents on site prior to the visit.
Prior to the visit, the panel members form a preliminary opinion about the programme and
draw up questions for their site visit.
During the site visit, the assessment panel will, in any case, meet with the programme
management, members of the examining board and the programme committee, teachers,
students, alumni and wherever relevant, representatives of the professional field. In
addition, the panel examines the material made available by the programme. The panel
determines the exact scope of the discussions, the possible clustering of discussion
participants and the further organisation of the visit. The panel decides at its own discretion
which teachers and students it would like to see and which documents it would like to
examine. In principle, the programme delegations comprise no more than six persons. The
panel will set aside time for open consultations. The programme and the panel will make

Page 31

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

these open consultations widely known, both prior to and during the visit. In addition, the
panel may visit lectures or other teaching-learning situations, such in consultation with the
programme.
At the end of the site visit, the chair of the assessment panel provides brief feedback
information to the programme regarding the general judgement and the underlying
considerations.
4.4.3

Assessment procedure within the assessment panel
The assessment panel presents its judgement regarding all the standards incorporated in
the assessment framework. This judgement is substantiated by an appraisal of the positive
and critical elements from the panel’s findings. The judgement may be: unsatisfactory,
satisfactory, good or excellent. The panel subsequently formulates a general, weighted and
substantiated judgement regarding the quality of the programme. This judgement is also
given on a four-point scale, ranging from unsatisfactory to excellent.

4.4.4

Assessment report
The assessment panel secretary draws up an assessment report comprising some 30
pages. The main content of the report features the panel’s judgements regarding the
standards. It is important for the audit panel to include underpinnings based on the
programme’s critical reflection, the meetings with representatives of the programme and the
underlying data from the documents made available. The report will include significant and
representative examples. In its report, the panel gives an account of the manner in which it
has organised its visit and how it has arrived at its choice of discussion partners and
documents.
The assessment report is preceded by a summary judgement regarding the quality of the
programme, comprising a maximum of two pages. Any measures for improvement will be
presented in a separate paragraph. In addition, the report contains a score table with the
panel judgements, information on the date(s) of the site visit, the names of the discussion
partners, basic data concerning the programme (see paragraph 4.6), an overview of the
material studied and the declarations of independence signed by the panel members and
the secretary.
The assessment panel secretary forwards the advisory report to the board of the institution
once all panel members have approved its contents. The institution is given the opportunity
to respond to any factual inaccuracies in the report, whereupon the panel chair endorses the
report after all panel members have taken note of and approved its contents. The report is
signed by the chair and the secretary of the panel.

4.5

NVAO decision-making
The board of the institution applies to NVAO for accreditation based on the assessment
report. NVAO may decide to accredit the programme, not accredit it or grant an
improvement period. The Accreditation Decree of the Dutch Higher Education and Research
Act stipulates how, on what grounds and under what circumstances NVAO may grant an
improvement period.

Page 32

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

4.6

Required documents
During the assessment process, the programme provides the assessment panel with a
limited number of documents. NVAO assumes that these are existing documents, available
within the institution, rather than documents prepared especially for the programme
assessment. The documents serve as a substantiation and if need be as verification. Other
material is only required when explicitly requested by the panel or if the programme wishes
to demonstrate a particular distinctive feature.

4.6.1

Basic data concerning the programme
(The basic data is incorporated into the critical reflection, the assessment report and the
NVAO decision.)
Administrative data regarding the programme
1. Nomenclature of the programme in CROHO [central register of higher education
programmes];
2. Orientation and level of the programme;
3. Number of credits;
4. Specialisations;
5. Location(s);
6. Mode (s) of study;
7. CROHO registration number.
Administrative data regarding the institution
1. Name of the institution;
2. Status of the institution (publicly funded or legal body providing higher education);
Quantitative data regarding the programme
1. Data on intake, transfers and graduates pertaining to – if possible – the last six cohorts;
2. Teacher -student ratio achieved;
3. Average amount of face-to-face instruction per stage of the study programme (a stage
can be expressed in, for example, regular years of study, the work placement and the
graduation period).

4.6.2

Required appendices to the critical reflection
(The list of appendices studied will be incorporated into the assessment report.)
1.
2.
3.

Subject-specific reference framework and the learning outcomes of the programme;
Overview of the curriculum in diagram form;
Outline description of the curriculum components, stating learning outcomes,
attainment targets, teaching method(s), assessment method, literature
(mandatory/recommended), teacher and credits;
4. Teaching and examination regulations;
(Items 2 to 4 are usually reflected in a study guide, in which case this can be annexed to the
report.)
5.
6.

Page 33

Overview of allocated staff with names, positions, scope of appointment, level and
expertise;
List of the last 25 final projects or the final projects of the past two years (or portfolios /
projects demonstrating the exit levels attained by the students);

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

7.
4.6.3

Overview of the contacts maintained with the professional field (if relevant).

Documents made available during the visit
(The list of material studied will be incorporated into the assessment report.)
1.
2.

Education policy plan or similar document(s);
Policy plan regarding research in relation to the programmes offered or similar
document(s);
3. Staff (policy) plan or similar document(s);
4. Services and facilities plan or similar document(s);
5. Quality assurance plan;
6. Policy plan regarding the accessibility and feasibility of the programme for students with
a functional disability;
7. Summary and analysis of recent evaluation results and relevant management
information;
8. Documentation regarding student and staff satisfaction;
9. Reports on consultations in relevant committees / bodies;
10. Test questions with corresponding assessment criteria and requirements (answer
models) and a representative selection of actual tests administered (such as
presentations, work placements, portfolio assessments) and assessments;
11. Representative selection of final projects, selected by the panel, of the past two years
with corresponding assessment criteria and requirements;
12. Reference books and other learning materials.

Page 34

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

5 Limited initial accreditation

5.1

Set-up
The framework for limited assessments of new programmes is used for institutions that have
obtained a positive judgement following an institutional quality assurance assessment. The
assessment is based on a discussion with peers regarding the content and quality of the
programme. It focuses on four questions:
1.
2.
3.
4.

What is the programme aiming for?
How does the programme intend to achieve its objectives?
How does the programme intend to assess its performance?
Does the programme have sufficient financial resources?

These four questions have been translated into four standards. Regarding each of these
standards, an assessment panel gives a substantiated judgement on a two-point scale:
unsatisfactory or satisfactory. The panel subsequently gives a substantiated final conclusion
regarding the quality of the programme, on the same two-point scale.

Page 35

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

5.2

Standard 1:

Assessment framework for limited initial accreditations
Intended learning outcomes
The intended learning outcomes of the programme have been concretised with regard to
content, level and orientation; they meet international requirements.

Explanation:

As for level and orientation (bachelor’s or master’s; professional or academic), the intended
learning outcomes fit into the Dutch qualifications framework. In addition, they tie in with the
international perspective of the requirements currently set by the professional field and the
discipline with regard to the contents of the programme.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 2:

Teaching-learning environment
The curriculum, staff and programme-specific services and facilities enable incoming
students to achieve the intended learning outcomes.

Explanation:

The contents and structure of the curriculum enable the students admitted to achieve the
intended learning outcomes. The quality of the staff and of the programme-specific services
and facilities is essential to that end. Curriculum, staff, services and facilities constitute a
coherent teaching-learning environment for the students.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 3:

Assessment
The programme has an adequate assessment system in place.

Explanation:

The level achieved is demonstrated by interim and final tests. The tests and assessments
are valid, reliable and transparent to the students.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 4:

Graduation guarantee and financial provisions
The institution guarantees students that they can complete the entire curriculum and makes
sufficient financial provisions available.

Explanation:

The graduation guarantee spans a reasonable period of time that is related to the length of
the studies.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Page 36

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

General conclusion
The quality of the programme is
Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

The assessment is based on the following definitions. These definitions relate to both the
scores obtained for the individual standards and the overall scores awarded to the
programme.
Generic quality
The quality that can reasonably be expected in an international perspective from a higher
education bachelor’s or master’s programme.
Unsatisfactory
The programme does not satisfy the generic quality standards.
Satisfactory
The programme satisfies the generic quality standards.
For programmes that are not entirely new and programmes that are being converted, the
learning outcomes achieved will be factored into the judgement.

5.3

16

Composition of the assessment panel

NVAO convenes and appoints the assessment panel that will conduct the initial
accreditation. The programme to be assessed is entitled to lodge substantiated objections to
the composition of the assessment panel.
It is imperative that assessment panels are composed in a manner allowing meaningful
discussions among peers, in which the panel remains sufficiently independent.
The panel secretary has completed NVAO training leading to certification. Every year,
NVAO publishes a list of NVAO certified secretaries.
Assessment panels must meet the following requirements.
1.

The panel is composed of a minimum of four members, among whom at least two
authoritative domain experts17 and a student.

2.

Overall, the panel commands the following expertise:
a. expertise regarding developments in the discipline,
b. international expertise,
c. practical expertise in the professional field relevant to the programme (if
applicable),

16

This paragraph is explained in detail in the guideline titled Eisen aan de panelsamenstelling [Requirements for the composition of
panels]. This guideline contains detailed requirements to be met by panels. It also contains a submission procedure, a form to be filled out
by the institution and a code of conduct for panel members.

17

Domain expertise is understood to mean specialist expertise, international expertise or professional expertise.

Page 37

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

d.

e.
f.

experience in teaching and educational development at the relevant
programme level and expertise regarding the teaching format(s) used in
the programme18,
student-related expertise,
assessment or audit expertise.

3.

The panel is independent (its members have not had any ties with the institution
providing the programme for at least the past five years).

4.

The panel is assisted by an independent, external secretary trained and certified by
NVAO. The secretary does not sit on the panel.

The assessment panel is counselled by an NVAO process co-ordinator and supported by a
secretary. The secretary and the process co-ordinator are also independent of the institution
in question. The secretary and the process co-ordinator do not sit on the panel.
Prior to the visit, all panel members and the secretary certify to not maintaining any
connections or ties with the institution in question, either as a private individual or as a
researcher / teacher, professional or adviser, which could affect an independent judgement
of the quality of the programme in either a positive or a negative sense, and to not having
had such connections or ties with the institution during the past five years.
In addition to the factual independence, as expressed above in the nature of the relation and
the number of years, it is essential for any panel member or secretary to feel independent.
In some cases, an independence of more than five years may not provide sufficient
guarantee for an independent position; a prospective panel member or secretary could still
experience too strong a relationship with the institution or be involved too closely with an
institution or programme, for example because of family ties. In such cases, the prospective
panel member or secretary cannot sit on the panel. Panel membership requires a
professional attitude. To that end, NVAO has formulated a code of conduct for panel
members and secretaries. This code of conduct comprises elements pertaining to the
independence, confidentiality and attitude of the panel members and the secretary during
the assessment process.
Panel members and secretaries will sign a declaration of independence and confidentiality
prior to the assessment process. In this declaration, they attest to having taken note of the
code of conduct. Following the assessment process, the chair and secretary sign the
assessment report once all panel members have read and approved the report. The report
includes a declaration that the assessment has been carried out independently.
Stakeholders such as panel members, staff or students may report to NVAO any matters
arising during the assessment process that could affect the independence of the
assessment.

18

This includes, for example, distance learning, work-related courses, flexible education, skill-oriented education or education aimed at
excellent students.

Page 38

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

5.4
5.4.1

Assessment process
Information dossier
For the purpose of the assessment by the assessment panel, the programme presents an
information dossier regarding the programme. The information dossier should follow the
standards outlined for the limited initial accreditation framework. It positions the programme
in relation to existing (and new) programmes at home and abroad. The critical reflection is a
self-contained document that can be read separately.
The framework for limited initial accreditations is structured in a manner allowing
programmes ample scope to emphasise their unique character. The programme may use
that scope in its information dossier. The information dossier is the pre-eminent tool to allow
teachers and peers to comment on the contents of the programme. It must be a document
reflecting the commitment of the stakeholders.
It is imperative that any overlap with assessments within the context of the institutional
quality assurance assessment is avoided when drawing up the information dossier and
during the assessment procedure. Should any reference to institutional policy or, for
example, departmental policy be necessary, programme assessments strictly focus on the
fitness for purpose of the policy pursued regarding the programme in question. This does
not include pre-conditional matters, such as the structure of quality assurance or the
institution’s staff policy; these are considered in institutional quality assurance assessments.
The information dossier comprises a maximum of 20 pages, excluding appendices.

5.4.2

Site visit
In principle, the required site visit for the purpose of limited initial accreditations takes one
day. Prior to the visit, the panel members will have formed a preliminary opinion about the
programme and drawn up questions for their site visit. The panel factors the outcomes of
the institutional quality assurance assessment into its judgement.
During the site visit, the assessment panel meets with the (prospective) programme
management, the (prospective) members of the examining board and the programme
committee, (prospective) teachers and, wherever relevant, representatives of the
professional field. In addition, the panel examines the material made available by the
programme. The panel determines the exact scope of the discussions, the possible
clustering of discussion participants and the further organisation of the visit. The panel
decides at its own discretion which teachers and students it would like to see and which
documents it would like to examine. In principle, the programme delegations comprise no
more than six persons.
At the end of the site visit, the chair of the assessment panel provides brief feedback
information to the programme regarding the general judgement and the underlying
considerations.

5.4.3

Page 39

Assessment procedure within the assessment panel
The assessment panel presents its judgement regarding all the standards incorporated in
the assessment framework. This judgement is substantiated by an appraisal of the positive
and critical elements from the panel’s findings. The judgement may be: unsatisfactory or
satisfactory. The panel subsequently formulates a general, weighted and substantiated

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

judgement regarding the quality of the programme. This judgement is given on the same
two-point scale (unsatisfactory or satisfactory).
5.4.4

Advisory report
The assessment panel secretary draws up an advisory report comprising some 20 pages.
The main content of the report is made up of the panel’s judgements regarding the
standards, including underpinnings based on the programme’s information dossier, the
meetings with representatives of the programme and the underlying data from the
documents studied. The report will include significant and representative examples. In the
report, the panel gives an account of the manner in which it has organised its visit and how
it has arrived at its choice of discussion partners and documents.
The assessment report is preceded by a summary judgement regarding the quality of the
programme comprising a maximum of two pages. Any measures for improvement will be
presented in a separate paragraph. In addition, the report contains a score table with the
panel judgements, information on the date(s) of the site visit, the names of the discussion
partners, basic data concerning the programme (see paragraph 5.6), an overview of the
material studied and the declarations of independence signed by the panel members and
the secretary.
NVAO forwards the advisory report to the board of the institution once all panel members
have approved its contents. The institution is given a term of two weeks to respond to any
factual inaccuracies in the report, whereupon the panel chair endorses the report after all
panel members have taken note of and approved its contents. The report is signed by the
chair and the secretary of the panel and submitted to NVAO for decision-making. If NVAO
finds that a report raises questions or if an institution so desires, NVAO may invite the
programme and/or the assessment panel for further consultations.

5.5

NVAO decision-making
Basically, NVAO can take two decisions: a positive initial accreditation decision for a period
of six years, or a negative initial accreditation decision.
In special cases, NVAO may attach conditions to its decision. In that case, the programme
must apply for additional assessment within a year, whereupon NVAO ascertains whether
the programme meanwhile meets the conditions set. If the programme fails to apply for an
additional assessment or does not meet the conditions, the positive decision expires.
Satisfaction of the conditions set will be assessed by an assessment panel commissioned
by NVAO. The additional assessment will basically be carried out in accordance with the
procedure for regular limited initial accreditations. The assessment panel will focus on the
programme’s shortcomings identified earlier.

5.6

Required documents
During the assessment process, the programme provides the assessment panel with a
limited number of documents. NVAO assumes that these are existing documents, available
within the programme or the institution, rather than documents prepared especially for the
programme assessment. The documents serve as a substantiation and if need be as

Page 40

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

verification. Other material is only required when explicitly requested by the panel or if the
programme wishes to demonstrate a particular distinctive feature.
5.6.1

Basic data concerning the programme
(The basic data is incorporated into the information dossier, the advisory report and the
NVAO decision.)
Administrative data regarding the programme
1. Nomenclature of the programme;
2. Orientation and level of the programme;
3. Number of credits;
4. Specialisations;
5. Location(s);
6. Mode (s) of study;
Administrative data regarding the institution
1.
Name of the institution;
2.
Status of the institution (publicly funded or legal body providing higher education);
3.
Outcome of the institutional quality assurance assessment.
Quantitative data regarding the programme
1.
Intended teacher-student ratio;
2.
Intended amount of face-to-face instruction per stage of the study programme (a stage
can be expressed in, for example, regular years of study, the work placement and the
graduation period).

5.6.2

Required appendices to the information dossier
(The list of appendices studied will be incorporated into the advisory report.)
1.
2.
3.

4.
5.
6.
7.
8.
5.6.3

Documents made available during the visit
(The list of material studied will be incorporated into the advisory report.)
1.
2.

Page 41

Subject-specific reference framework and the learning outcomes of the programme;
Overview of the curriculum in diagram form;
Outline description of the curriculum components for the first year, stating learning
outcomes, attainment targets, teaching method(s), assessment method, literature
(mandatory/recommended), teacher and credits;
Teaching and examination regulations;
Overview of allocated staff with names, positions, scope of appointment, level and
expertise;
If so required, the macro-efficiency decision;
Overview of the contacts maintained with the professional field (if relevant);
Report on the institutional quality assurance assessment.

Reports on consultations in relevant committees / bodies;
Reference books and other learning materials.

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

6 Extensive initial accreditation

6.1

Set-up
The framework for extensive assessments of new programmes (extensive initial
accreditation framework) is used for institutions that have failed to obtain a positive
judgement following an institutional quality assurance assessment. The assessment is
based on a discussion with peers regarding the content and quality of the programme. It
focuses on seven questions:
1.
2.
3.
4.
5.
6.
7.

What is the programme aiming for?
With what curriculum?
With what staff?
With what services and facilities?
How does the programme intend to safeguard quality?
How does the programme intend to assess its performance?
Does the programme have sufficient financial resources?

These seven questions have been translated into seven themes and 16 standards.
Regarding each of these standards, an assessment panel gives a substantiated judgement
on a two-point scale: unsatisfactory or satisfactory. The panel subsequently gives a
substantiated final conclusion regarding the overall quality of the programme, on the same
two-point scale.

Page 42

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

6.2

Standard 1:

Assessment framework for extensive initial accreditations
Intended learning outcomes
The intended learning outcomes of the programme have been concretised with regard to
content, level and orientation; they meet international requirements.

Explanation:

As for level and orientation (bachelor’s or master’s; professional or academic), the intended
learning outcomes fit into the Dutch qualifications framework. In addition, they tie in with the
international perspective of the requirements currently set by the professional field and the
discipline with regard to the contents of the programme.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 2:

Curriculum
The orientation of the curriculum assures the development of skills in the field of scientific
research and/or the professional practice.

Explanation:

The curriculum has demonstrable links with current developments in the professional field
and the discipline.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 3:

The contents of the curriculum enable students to achieve the intended learning outcomes.

Explanation:

The learning outcomes have been adequately translated into attainment targets for
(components of) the curriculum. Students follow a study curriculum which is coherent in
terms of content.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 4:

The structure of the curriculum encourages study and enables students to achieve the
intended learning outcomes.

Explanation:

The teaching concept is in line with the intended learning outcomes and the teaching
formats tie in with the teaching concept.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 5:

The curriculum ties in with the qualifications of the incoming students.

Explanation:

The admission requirements are realistic with a view to the intended learning outcomes.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Page 43

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

Standard 6:

The curriculum is feasible.

Explanation:

Factors pertaining to the curriculum and hindering students’ progress are removed as far as
possible. In addition, students with functional disabilities receive additional career tutoring.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 7:

The programme meets statutory requirements regarding the scope and duration of the
curriculum.

Explanation:

Scope and duration:
Bachelor’s programmes (professional orientation): 240 credits;
Bachelor’s programmes (academic orientation): in principle, a minimum of 180 credits;
Master’s programmes (professional orientation): in principle, a minimum of 60 credits;
Master’s programmes (academic orientation): in principle, a minimum of 60 credits,
depending on the programme.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 8:

Staff
The programme has an effective staff policy in place.

Explanation:

The staff policy provides for the qualifications, training, assessment and size of the staff
required for the realisation of the curriculum.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 9:

The staff is qualified for the realisation of the curriculum in terms of content, educational
expertise and organisation.

Explanation:

The factual expertise available among the staff ties in with the requirements set for
professional or academic higher education programmes.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 10:

The size of the staff is sufficient for the realisation of the curriculum.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 11:

Services and facilities
The accommodation and the facilities (infrastructure) are sufficient for the realisation of the
curriculum.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 12:

Tutoring and student information provision bolster students’ progress and tie in with the
needs of students.

Page 44

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 13:

Quality assurance
The programme is evaluated on a regular basis, partly on the basis of assessable targets.

Explanation:

The programme ensures the quality of the intended learning outcomes, the curriculum, the
staff, the services and facilities, the assessments and the learning outcomes achieved
through regular evaluations. The programme also collects management information
regarding the success rates and the staff-student ratio.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 14:

Programme committees, examining boards, staff, students, alumni and the relevant
professional field of the programme are actively involved in the programme’s internal quality
assurance.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 15:

Assessment
The programme has an adequate assessment system in place.

Explanation:

The level achieved is demonstrated by interim and final tests. The tests and assessments
are valid, reliable and transparent to the students.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

Standard 16:

Graduation guarantee and financial provisions
The institution guarantees students that they can complete the entire curriculum and makes
sufficient financial provisions available.

Explanation:

The graduation guarantee spans a reasonable period of time that is related to the length of
the studies.

Judgement:

Unsatisfactory, satisfactory (weighted and substantiated).

General conclusion
The quality of the programme is
Judgement:

Page 45

Unsatisfactory, satisfactory (weighted and substantiated).

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

The assessment is based on the following definitions. These definitions relate to both the
scores obtained for the individual standards and the overall scores awarded to the
programme.
Generic quality
The quality that can reasonably be expected in an international perspective from a higher
education bachelor’s or master’s programme.
Unsatisfactory
The programme does not satisfy the generic quality standards.
Satisfactory
The programme satisfies the generic quality standards.
For programmes that are not entirely new and programmes that are being converted, the
learning outcomes achieved will be factored into the judgement.

6.3

19

Composition of the assessment panel

NVAO convenes and appoints the assessment panel that will conduct the initial
accreditation. The programme to be assessed is entitled to lodge substantiated objections to
the composition of the assessment panel.
It is imperative that assessment panels are composed in a manner allowing meaningful
discussions among peers, in which the panel remains sufficiently independent.
The panel secretary has completed NVAO training leading to certification. Every year,
NVAO publishes a list of NVAO certified secretaries.
Assessment panels must meet the following requirements.
1.

The panel is composed of a minimum of four members, among whom at least two
authoritative domain experts20 and a student.

2.

Overall, the panel commands the following expertise:
a. expertise regarding developments in the discipline,
b. international expertise,
c. practical expertise in the professional field relevant to the programme (if
applicable),
d. experience in teaching and educational development at the relevant
programme level and expertise regarding the teaching format(s) used in
the programme21,
e. student-related expertise,
f. assessment or audit expertise.

19

This paragraph is explained in detail in the guideline titled Eisen aan de panelsamenstelling [Requirements for the composition of
panels]. This guideline contains detailed requirements to be met by panels. It also contains a submission procedure, a form to be filled out
by the institution and a code of conduct for panel members.

20

Domain expertise is understood to mean specialist expertise, international expertise or professional expertise.

21

This includes, for example, distance learning, work-related courses, flexible education, skill-oriented education or education aimed at
excellent students.

Page 46

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

3.

The panel is independent (its members have not had any ties with the institution
providing the programme for at least the past five years).

4.

The panel is assisted by an independent, external secretary trained and certified by
NVAO. The secretary does not sit on the panel.

The assessment panel is counselled by an NVAO process co-ordinator and supported by a
secretary. The secretary and the process co-ordinator are also independent of the institution
in question. The secretary and the process co-ordinator do not sit on the panel.
Prior to the visit, all panel members and the secretary certify to not maintaining any
connections or ties with the institution in question, either as a private individual or as a
researcher / teacher, professional or adviser, which could affect an independent judgement
of the quality of the programme in either a positive or a negative sense, and to not having
had such connections or ties with the institution during the past five years.
In addition to the factual independence, as expressed above in the nature of the relation and
the number of years, it is essential for any panel member or secretary to feel independent.
In some cases, an independence of more than five years may not provide sufficient
guarantee for an independent position; a prospective panel member or secretary could still
experience too strong a relationship with the institution or be involved too closely with an
institution or programme, for example because of family ties. In such cases, the prospective
panel member or secretary cannot sit on the panel. Panel membership requires a
professional attitude. To that end, NVAO has formulated a code of conduct for panel
members and secretaries. This code of conduct comprises elements pertaining to the
independence, confidentiality and attitude of the panel members and the secretary during
the assessment process.
Panel members and secretaries will sign a declaration of independence and confidentiality
prior to the assessment process. In this declaration, they attest to having taken note of the
code of conduct. Following the assessment process, the chair and secretary sign the
assessment report once all panel members have read and approved the report. The report
includes a declaration that the assessment has been carried out independently.
Stakeholders such as panel members, staff or students may report to NVAO any matters
arising during the assessment process that could affect the independence of the
assessment.

6.4
6.4.1

Assessment process
Information dossier
For the purpose of the assessment by the assessment panel, the programme presents an
information dossier regarding the programme. The information dossier should follow the
standards outlined for the extensive initial accreditation framework. It positions the
programme in relation to existing (and new) programmes at home and abroad. The critical
reflection is a self-contained document that can be read separately.
The information dossier is the pre-eminent tool to allow teachers and peers to comment on
the contents of the programme. It must be a document reflecting the commitment of the
stakeholders.

Page 47

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

In addition, the assessment framework offers opportunities to discuss the ambitions of the
programme during the site visit. What choices will the programme make for the future, what
direction will it take? In order to conduct such discussions, the assessment panel is
expected to be able to reflect on the programme’s plans for the future, together with the
programme’s representatives.
The information dossier comprises a maximum of 35 pages, excluding appendices.
6.4.2

Site visit
In principle, the required site visit for the purpose of extensive initial accreditations takes one
day. Prior to the visit, the panel members will have formed a preliminary opinion about the
programme and drawn up questions for their site visit.
During the site visit, the assessment panel meets with the (prospective) programme
management, the (prospective) members of the examining board and the programme
committee, (prospective) teachers and, wherever relevant, representatives of the
professional field. In addition, the panel examines the material made available by the
programme. The panel determines the exact scope of the discussions, the possible
clustering of discussion participants and the further organisation of the visit. The panel
decides at its own discretion which teachers and students it would like to see and which
documents it would like to examine. In principle, the programme delegations comprise no
more than six persons.
At the end of the site visit, the chair of the assessment panel provides brief feedback
information to the programme regarding the general judgement and the underlying
considerations.

6.4.3

Assessment procedure within the assessment panel
The assessment panel presents its judgement regarding all the standards incorporated in
the assessment framework. This judgement is substantiated by an appraisal of the positive
and critical elements from the panel’s findings. The judgement may be: unsatisfactory or
satisfactory. The panel subsequently formulates a general, weighted and substantiated
judgement regarding the quality of the programme. This judgement is given on the same
two-point scale (unsatisfactory or satisfactory).

6.4.4

Advisory report
The assessment panel secretary draws up an advisory report comprising 20 to 30 pages.
The main content of the report is made up of the panel’s judgements regarding the
standards, including underpinnings based on the programme’s information dossier, the
meetings with representatives of the programme and the underlying data from the
documents studied. The report will include significant and representative examples. In the
report, the panel gives an account of the manner in which it has organised its visit and how
it has arrived at its choice of discussion partners and documents.
The assessment report is preceded by a summary judgement regarding the quality of the
programme comprising a maximum of two pages. Any measures for improvement will be
presented in a separate paragraph. In addition, the report contains a score table with the
panel judgements, information on the date(s) of the site visit, the names of the discussion
partners, basic data concerning the programme (see paragraph 6.6), an overview of the

Page 48

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

material studied and the declarations of independence signed by the panel members and
the secretary.
NVAO forwards the advisory report to the board of the institution once all panel members
have approved its contents. The institution is given a term of two weeks to respond to any
factual inaccuracies in the report, whereupon the panel chair endorses the report after all
panel members have taken note of and approved its contents. The report is signed by the
chair and the secretary of the panel and submitted to NVAO for decision-making. If NVAO
finds that a report raises questions or if an institution so desires, NVAO may invite the
programme and/or the assessment panel for further consultations.

6.5

NVAO decision-making
Basically, NVAO can take two decisions: a positive initial accreditation decision for a period
of six years, or a negative initial accreditation decision.
In special cases, NVAO may attach conditions to its decision. In that case, the programme
must apply for additional assessment within a year, whereupon NVAO ascertains whether
the programme meanwhile meets the conditions set. If the programme fails to apply for an
additional assessment or does not meet the conditions, the positive decision expires.
Satisfaction of the conditions set will be assessed by an assessment panel commissioned
by NVAO. The additional assessment will basically be carried out in accordance with the
procedure for regular limited initial accreditations. The assessment panel will focus on the
programme’s shortcomings identified earlier.

6.6

Required documents
During the assessment process, the programme provides the assessment panel with a
limited number of documents. NVAO assumes that these are existing documents, available
within the programme or the institution, rather than documents prepared especially for the
programme assessment. The documents serve as a substantiation and if need be as
verification. Other material is only required when explicitly requested by the panel or if the
programme wishes to demonstrate a particular distinctive feature.

6.6.1

Basic data concerning the programme
(The basic data is incorporated into the information dossier, the advisory report and the
NVAO decision.)
Administrative data regarding the programme
1. Nomenclature of the programme;
2. Orientation and level of the programme;
3. Number of credits;
4. Specialisations;
5. Location(s);
6. Mode (s) of study;

Page 49

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

Administrative data regarding the institution
1.
Name of the institution;
2.
Status of the institution (publicly funded or legal body providing higher education).
Quantitative data regarding the programme
1.
Intended teacher-student ratio;
2.
Intended amount of face-to-face instruction per stage of the study programme (a stage
can be expressed in, for example, regular years of study, the work placement and the
graduation period).
6.6.2

1.
2.
3.

4.
5.
6.
7.
6.6.3

Required appendices to the information dossier
(The list of appendices studied will be incorporated into the advisory report.)
Subject-specific reference framework and the learning outcomes of the programme;
Overview of the curriculum in diagram form;
Outline description of the curriculum components for the first year, stating learning
outcomes, attainment targets, teaching method(s), assessment method, literature
(mandatory/recommended), teacher and credits;
Teaching and examination regulations;
Overview of allocated staff with names, positions, scope of appointment, level and
expertise;
If so required, the macro-efficiency decision;
Overview of the contacts maintained with the professional field (if relevant).
Documents to be made available during the visit
(The list of documents studied will be incorporated into the advisory report.)
1.
2.
3.
4.
5.
6.
7.
8.

Page 50

Education policy plan or similar document(s);
Policy plan regarding research in relation to the programmes offered or similar
document(s);
Staff (policy) plan or similar document(s);
Services and facilities plan or similar document(s);
Policy plan regarding the accessibility and feasibility for students with a functional
disability;
Quality assurance plan;
Reports on consultations in relevant committees / bodies;
Reference books and other learning materials.

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

7 Distinctive features

7.1

Background
The distinctive features have been incorporated into the accreditation system because they
can contribute to the national and international profiling of higher education programmes. A
distinctive feature enables institutions to draw attention to aspects that are not directly
related to programme levels but involve, for example, the orientation of a programme (such
as research master’s programmes), objectives such as sustainability or the residential
nature.
Distinctive features are assessed on the basis of the following principles:
1.

2.
3.
4.
5.

6.
7.
7.2

Criterion 1:

The audit panel or assessment panel assesses a distinctive feature by reference to the
relevant framework in combination with the criteria set out below. The panel ascertains
whether the institution or programme profiling the distinctive feature fulfils its promise.
To that end, the panel determines, in consultation with the institution or programme,
what standards it will focus on during the assessment.
The required comparison with other relevant institutions or programmes is performed
by the institution or programme itself.
The composition of the assessing panel is geared to the assessment of the distinctive
feature.
The point of departure is that an institution or programme may apply for assessment of
a distinctive feature at any time. However, its accreditation period may not exceed the
final date of the original application.
A distinctive feature must meet the following criteria.

Criteria for distinctive features
Distinguishing nature
The distinctive feature distinguishes the institution or programme from other relevant
institutions or programmes in the Dutch higher education sector.

Explanation:

The institution or programme demonstrates that the distinctive feature has a distinguishing
but not necessarily unique nature vis-à-vis relevant institutions or programmes in the Dutch
higher education sector.

Judgement:

Meets, does not meet or partially meets the standard (weighted and substantiated).

Page 51

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

Criterion 2:

Concretisation
The impact of the distinctive feature on the quality of the education provided has been
operationalised on the basis of the relevant standards in the appropriate assessment
framework.

Explanation:

The assessing panel indicates which standard(s) it regards as relevant to the realisation of
the feature and why. The judgement must demonstrate the operationalisation of the
distinctive feature for the relevant standard(s). If a distinctive feature spans several
standards in the framework in question, the judgement should provide a concrete and
complete assessment of the feature for all standards concerned.

Judgement:

Meets, does not meet or partially meets the standard (weighted and substantiated).

General conclusion
The distinctive feature is
Judgement:

Page 52

Granted, not granted (weighted and substantiated).

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

8 Assessment scales for programme assessments
Each judgement is illustrated with a number of examples to assist in its operationalisation.
Examples that apply exclusively to extensive programme assessments are marked “EPA”.

8.1

Unsatisfactory
The programme does not meet the current generic quality standards and shows serious
shortcomings in several areas.
This judgement could be operationalised as follows:

The level and/or orientation of the learning outcomes do not fit within the (inter)national
qualification frameworks and have not been concretised into subject- or programmespecific performance levels.

The aggregate of curriculum, staff, services and facilities does not constitute an
environment conducive to learning.

The programme lacks a programme-wide, transparent and coherent assessment
policy.

The intended learning outcomes are not being achieved.

Quality assurance in the programme is not pursued in a systematic manner, which
translates into a lack of improvement policy (EPA).

8.2

Satisfactory
The programme meets the current generic quality standards and demonstrates an
acceptable level across its entire spectrum.
This judgement could be operationalised as follows:

The level and/or orientation of the learning outcomes fit within the (inter)national
qualification frameworks and have been concretised into subject- or programmespecific performance levels.

The aggregate of curriculum, staff, services and facilities constitutes an environment
conducive to learning which enables students to achieve the learning outcomes.

The programme has developed a programme-wide, transparent and coherent
assessment policy, which, however, is not yet pursued by all parties involved.

The intended learning outcomes are achieved.

Quality assurance in the programme is pursued in a systematic manner, which
translates into a consistent improvement policy (EPA).

Page 53

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

8.3

Good
The programme systematically surpasses the current generic quality standard across its
entire spectrum.
This judgement could be operationalised as follows:

The level and/or orientation of the learning outcomes fit within the (inter)national
qualification frameworks and have been concretised into subject- or programmespecific performance levels. These are given a specific interpretation based on the
programme’s explicit views.

The aggregate of curriculum, staff, services and facilities constitutes a challenging
learning environment.

The programme has developed a programme-wide, transparent and coherent
assessment policy, which is pursued by all parties involved.

The learning outcomes achieved translate into products that are systematically above
average.

Quality assurance in the programme is pursued in a systematic manner, which
translates into a consistent improvement policy that is reflected in a growing quality
culture (EPA).

8.4

Excellent
The programme systematically well surpasses the current generic quality standards across
its entire spectrum and is regarded as an (inter)national example.
This judgement could be operationalised as follows:

The level and/or orientation of the learning outcomes fit within the (inter)national
qualification frameworks and have been concretised into subject- or programmespecific performance levels. These are given a specific interpretation based on the
programme’s explicit and unique views. The programme serves as an example both
nationally and internationally.

The aggregate of curriculum, staff, services and facilities constitutes an innovative,
original learning environment.

The learning outcomes achieved are of excellent quality and translate into awards and
(inter)national publications.

Quality assurance in the programme is pursued in a systematic manner, which
translates into a consistent improvement policy and a strong ability for self-reflection.
This is reflected in a robust quality culture (EPA).

Page 54

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

9 Assessment rules

9.1

Programme assessments
For programmes offering various modes of study (for example, full-time, part-time and workbased learning), the assessment must demonstrate that the generic quality of each mode of
study is assured, based on the standards in the relevant assessment framework, in order to
arrive at a positive final conclusion regarding the programme.
Programmes that are offered at various locations under a single CROHO registration only
qualify for accreditation if the assessment shows that each location meets the generic
quality standards stated in the relevant framework.
Limited programme assessments

The final conclusion regarding a programme will always be “unsatisfactory” if standard
3 is judged “unsatisfactory”. In case of an unsatisfactory score on standard 1, NVAO
cannot grant an improvement period.

The final conclusion regarding a programme can only be “good” if at least two
standards are judged “good”; one of these must be standard 3.

The final conclusion regarding a programme can only be “excellent” if at least two
standards are judged “excellent”; one of these must be standard 3.
Extensive programme assessments

The final conclusion regarding a programme will always be “unsatisfactory” if standard
1 or standard 16 is judged “unsatisfactory”. In case of an unsatisfactory score on
standard 1, NVAO cannot grant an improvement period.

The final conclusion regarding a programme can only be “good” if at least standards 1,
3, 6, 9,13, 14, 15 and 16 are judged “good”.

The final conclusion regarding a programme can only be “excellent” if standards 1, 3,
6, 9, 13, 14, 15 and 16 are judged “excellent”.
Limited initial accreditations
The final conclusion regarding a programme will always be “unsatisfactory” if standards 1 or
3 are judged “unsatisfactory”. In case of an unsatisfactory score on standards 1 or 3, NVAO
cannot grant a conditional initial accreditation.
Extensive initial accreditations
The final conclusion regarding a programme will always be “unsatisfactory” if standards 1 or
15 are judged “unsatisfactory”. In case of an unsatisfactory score on standards 1 or 15,
NVAO cannot grant a conditional initial accreditation.

9.2

Institutional quality assurance assessments
The final conclusion following institutional quality assurance assessments will always be
“negative” if standards 1 or 4 are judged “does not meet the standard”. In case of an
unsatisfactory score on standards 1 or 4, NVAO cannot grant a conditional pass on the
institutional quality assurance assessment.

Page 55

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

10 Accreditation Decision under the Higher Education and
Research Act
In this chapter, NVAO outlines the rules laid down by implementing regulations [Dutch:
AMVB] regarding conditional decisions and the granting of improvement periods. The Dutch
Higher Education and Research Act [WHW] stipulates that implementing regulations be
formulated to specify the conditions under which and the situations in which improvement
periods may be granted in the accreditation of programmes (Article 5a.12a, first paragraph),
the conditional initial accreditation of programmes (Article 5a.11, fourth paragraph) and
conditional institutional quality assurance assessments (Article 5a.13d, sixth paragraph). In
this document, this implementing regulation is referred to as: Accreditation Decision under
the Higher Education and Research Act.
10.1Conditional initial accreditation and institutional quality assurance assessment
NVAO may attach conditions to an initial accreditation or institutional quality assurance
assessment if, on the basis of the advice submitted by the panel of experts, it arrives at the
conclusion that certain quality aspects are unsatisfactory but can reasonably be remedied
within a timeframe of one year.
With regard to initial accreditations, this pertains to both extensive and limited assessments
(Article 5a.10a, second paragraph, and Article 5a.13g, first paragraph). In a conditional
initial accreditation of institutional quality assurance assessment, the conditions in question
relate to the efforts expected from the board of the institution to improve the quality aspects
that are assessed as unsatisfactory as well as the manner in which these efforts must be
expended, the manner in which and the timeframe within which the board of the institution
must ultimately report on these efforts to NVAO and the communication by the board of the
institution to the students and other stakeholders regarding the conditions set.
The timeframe to be observed for reporting must logically follow the timeframe allowed to
implement the improvements. A timeframe shorter than one year may be set if, in the
opinion of NVAO, the improvements may be realised sooner. Communication is important
because students must be informed to the full when selecting a study programme. This
information is also relevant to others, such as employers with whom the institution maintains
a special relationship and who employ many graduates.
An initial accreditation application must be denied if the standards of ‘Intended learning
outcomes’ or ‘Testing’ are judged unsatisfactory. An application for an institutional quality
assurance assessment must be denied if the standards of ‘View of the quality of the
education provided’ or ‘Improvement policy’ are judged unsatisfactory. In those cases, a
conditional initial accreditation or institutional quality assurance assessment cannot be
granted.
10.2Improvement period for accreditation
If NVAO assesses an application for the renewal of an existing accreditation or accreditation
following an initial accreditation decision and determines that the programme does not meet
all the required quality aspects, it may decide to renew the existing accreditation or initial
accreditation and grant a so-called ‘improvement period’. This pertains to both extensive

Page 56

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

and limited accreditation assessments (Article 5a.8, second paragraph and Article 5a.13f,
first paragraph).
An improvement period may only be granted if, in the opinion of NVAO, the deficiencies may
reasonably be remedied within a timeframe of no more than two years.
The assessment report submitted by the assessment panel is essential in this respect.
However, if the standard of ‘Intended learning outcomes’ is judged unsatisfactory, an
improvement period cannot be granted and the application for accreditation must be denied.
This is because a programme’s ambitions level must be at least up to par. Generic quality is
not guaranteed in programmes whose intended exit level is sub-standard; thus, they lack a
critical quality culture and vision, the basis for good-quality higher education of world-class
standards. In such cases, granting an improvement period is uncalled for.
NVAO may set conditions when granting an improvement period. In terms of content and
function, these conditions correspond to the conditions that may be attached to initial
accreditations and institutional quality assurance assessments. The difference from
conditional initial accreditations and institutional quality assurance assessments, however, is
that the board of the institution is required to submit a new application to NVAO no later than
six months before the end of the improvement period, viz. an application for a decision to
determine whether the programme meets the accreditation framework as yet (Article
5a.12a, fourth and fifth paragraphs of the Act).
Similar to the points for improvement in initial accreditations, an improvement period may be
shorter than two years if, in the opinion of NVAO, improvement may be achieved within a
shorter space of time. The above timeframe for submitting applications is, however, based
on the assumption that extensions of the validity by a period of less than one year would be
improbable. In addition, an unsatisfactory score on the ‘Testing and learning outcomes
achieved’ standard warrants a maximum improvement period of one year, i.e., any
improvements on this standard must be feasible within a year, otherwise renewal of the
existing accreditation or initial accreditation cannot be granted. The assessment panel
reviews the manner in which the institution has remedied the deficiencies identified by
NVAO and determines whether the programme meanwhile scores satisfactorily on all the
statutory quality aspects.
By analogy with Article 5a. 2, second paragraph of the Act, the assessment panel that
reviews the improvement must be approved by NVAO, as does the assessment panel that
originally assessed the programme. The assessment panel that reviews the improvement
comprises, as a minimum, two domain experts from the panel that originally assessed the
programme.

Page 57

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

11 Appeals
Before making a decision regarding an institutional quality assurance assessment, limited
programme assessment, extensive programme assessment, limited initial accreditation or
extensive initial accreditation, NVAO allows the board of the institution a term of two weeks
to present its views concerning the intended decision. These two weeks fall within the
statutory time frame of six months (for institutional quality assurance assessments, limited
initial accreditations and extensive initial accreditations) or three months (for limited
programme assessments and extensive programme assessments) within which NVAO is
required to make its decision.
Once ratified, the decision is immediately forwarded to the board of the institution. At the
same time, NVAO publishes its decision by placing it on its web site.
NVAO decisions are open to appeal.
Stakeholders may lodge an internal appeal with NVAO. The time frame for lodging internal
appeals is six weeks. The processing of the appeal involves a hearing. NVAO makes its
decision within twelve weeks after receiving the appeal. A decision after appeal may be
postponed for no more than six weeks. Such postponement is communicated in writing.
NVAO decisions after appeal are open to external appeals with the Administrative
Jurisdiction Department of the Council of State. The time frame for lodging external appeals
is six weeks. In principle, the Department gives its verdict six weeks after the session. This
term can be extended by a maximum of six weeks.
Pending the internal or external appeal procedure, the Chair of the Administrative
Jurisdiction Department of the Council of State may be requested to make provisional
arrangements if urgency, due to the interests involved, so requires.

Page 58

NVAO | Assessment frameworks for the higher education accreditation system | 22 November 2011

Accreditation framework for
existing higher education
programmes in Flanders
1 September 2009

Parkstraat 28 | 2514 JK | Postbus 85498 | 2508 CD Den Haag
P.O. Box 85498 | 2508 CD The Hague | The Netherlands
T +31 (0)70 312 2300 | F +31 (0)70 312 2301
info@nvao.net | www.nvao.net

Contents
The structure of the accreditation framework

1
2

3

Accreditation of programmes based on a published external assessment
2.1
Assessment framework
5
2.1.1 Aims and objectives
2.1.2 Curriculum
2.1.3 Staff commitment
2.1.4 Services
2.1.5 Internal Quality Assurance system
2.1.6 Results
2.2
Distinctive quality features
11
2.3
Assessment rules
12
2.3.1 Assessment of the standards
2.3.2 Assessment of the programme
2.4
Assessment rules for accreditation
13
2.4.1 Evaluation of external assessments
2.4.2 Assessment rules for accreditation

3

Accreditation of programmes based on accreditation abroad

5
5
7
9
9
9
10

12
12
13
14

15

4
Transitional arrangement for the assessment and accreditation of converted
academic programmes at university colleges, with regard to the inter-relatedness of
education and research (2005-2013 period)
18
4.1
Introduction
18
4.2
Related themes and standards of the accreditation framework 19
4.3
Assessment of the transitional nature
19
4.4
Scope of accreditation criteria in the transition period
20
Explanatory notes

22

Explanatory notes per chapter

29

Explanatory notes to the system of accreditation by virtue of law

34

Appendix 1. Interpretation of accreditation criteria in transition period for programmes
with an academic orientation that ensue from the conversion of university college
programmes (pertaining to embedment in research)
35

Page 2

NVAO | September 2009

1 The structure of the accreditation framework
Under the Accreditation Treaty, NVAO is charged with the (initial) accreditation of existing
bachelor’s and master’s programmes in the Netherlands and the Flemish Community. The
present accreditation framework1 refers to applications for accreditation submitted by the
boards of higher education institutions in the Flemish Community.
In the accreditation process, NVAO has two methods to assess whether the programme
sufficiently meets the generic quality standards (clause 58 of the Flemish Higher Education
Act):
1. NVAO either grants accreditation based on a published external assessment report;
2. or NVAO grants accreditation based on an “accreditation abroad” already obtained by the
programme, if – at the discretion of NVAO – this accreditation abroad was granted using a
comparable methodological approach.
Chapter 2 describes the framework for the assessment of programmes in higher education
based on an external assessment report.
To assess whether a higher education programme can be granted accreditation, the
programme is assessed according to six themes (see chapter 2.1). These themes are:
1. Aims and objectives
2. Curriculum
3. Staff
4. Services
5. Internal quality assurance
6. Results
These themes are assessed according to standards and corresponding criteria, with a
differentiation as to level (bachelor’s or master’s) and orientation (professional or academic) of
the programme.
A higher education institution may also request a quality assessment agency to carry out an
assessment of distinctive quality features of a programme. Such features can then be included
in the assessment report. The assessment of distinctive quality features does not affect the
outcome of an accreditation procedure (see Chapter 2.2).
To assess a programme, an assessment scale is used and assessment rules are applied that
are based on a judgement of the underlying standards (see Chapter 2.3).
The final accreditation decision depends on the Accreditation Organisation’s validation of the
external assessment. To this end, criteria have been adopted for the evaluation of the
procedure and the assessment report drawn up by the quality assessment agency (see Chapter
2
2.4).

Chapter 3 describes the criteria according to which NVAO can declare an international
accreditation equivalent. These criteria relate to the equivalence of the methodological

1
Clause 10.1 of the Accreditation Treaty gives NVAO the authority to develop an “assessment framework”. Based on this
framework, NVAO assesses whether a programme sufficiently meets the generic quality standards. Assessment
frameworks must be ratified by the Flemish government.
2
Information on the procedure followed with respect to accreditation applications can be found in the Guidelines for
Accreditations in the Flemish Community, drawn up by NVAO.

Page 3

NVAO | September 2009

approach used by the accreditation organisation abroad. If equivalence is established, NVAO
can accredit the programme involved on that basis.
Chapter 4 describes the transitional arrangement (period 2005 – 2013) for the assessment and
accreditation of converted academic programmes provided by university colleges, concerning
the inter-relatedness between education and research.
The explanatory part outlines:
- the clauses of the accreditation framework;
- accreditations granted by virtue of law during the transitional period following the introduction
of the accreditation system.
Appendix 1 outlines the rules to be followed during the transitional period regarding the criteria
in the accreditation framework that are relevant to the embedment in research of academic
bachelor’s and master’s programmes ensuing from the conversion of two-cycle undergraduate
programmes at university colleges.

Page 4

NVAO | September 2009

2 Accreditation of programmes based on a published external
assessment
2.1 Assessment framework
2.1.1 Aims and objectives
Standards
Level and
orientation:
bachelor’s
programmes
(professional
orientation)

Criteria
The aims and objectives of the programme are focused on having the students
master:
– general competences such as the capacity for logical thought and reasoning,
the ability to acquire and process information, the ability for critical reflection
and project-based work, creativity, the ability to perform simple supervision
tasks, the ability to communicate information, ideas, problems and solutions to
both specialists as well as laymen and a positive attitude towards life-long
learning;
– general professional competences such as the ability to work together as part
of a team, a solution-oriented attitude in the sense of being able to define and
analyse independently complex problematic situations in professional practice,
and the ability to develop and apply effective strategies to resolve them and to
develop a sense of social responsibility in connection with the professional
practice;
– specific professional competences at the level of a newly-qualified
professional.

Level and
orientation:
bachelor’s
programmes
(academic
orientation)

The aims and objectives of the programme are focused on having the students
master:

Page 5

– general competences such as the capacity for logical thought and reasoning,
the ability to acquire and process information, the capacity for critical
reflection, creativity, being able to perform simple management tasks, the
ability to communicate information, ideas, problems and solutions both to
specialists and laymen and a positive attitude towards life-long learning;
– general academic competences such as a research attitude, knowledge of
research methodologies and techniques and the ability to apply them
adequately, the ability to collect relevant data in order to form an opinion about
social, academic, scientific and ethical issues, an appreciation of the
uncertainty, ambiguity and limits of knowledge and the ability to initiate
problem-driven research;
– an understanding of basic academic, discipline-related knowledge inherent to
a certain domain of the sciences or the arts, systematic understanding of the
key elements of a discipline which includes acquiring coherent and detailed
knowledge that is inspired partly by the most recent developments in the
discipline and an understanding of the structure of the discipline and its interrelatedness with other disciplines.

NVAO | September 2009

Level and
orientation: master´s
programmes

The aims and objectives of the programme are focused on having the students
master:
– general competences at an advanced level such as the ability to reason and
act in an academic manner, the ability to handle complex problems, the ability
to reflect on one’s own thoughts and work and the ability to convert this
reflection into the development of more effective solutions, the ability to
communicate one’s own research and solutions to peers and laymen and the
ability to develop an opinion in an uncertain context;
– general academic competences at an advanced level such as the ability to
apply research methods and techniques, the ability to design research, the
ability to apply paradigms in the domain of the sciences or the arts and the
ability to indicate the limits of paradigms, the ability to be original and creative
with a view to continuously expanding knowledge and insight and the ability to
collaborate in a multi-disciplinary environment;
– advanced understanding of and insight into scientific, discipline-specific
knowledge inherent to a certain domain of the sciences or the arts, insight into
the most recent knowledge in the subject/discipline or parts thereof, the ability
to follow and interpret the direction in which theory formation is developing, the
ability to make an original contribution towards the body of knowledge of one or
several parts of the subject/discipline and display specific competences
characteristic to the subject/discipline such as designing, researching,
analysing and diagnosing;
– the competences needed for either independent research or the independent
practice of the arts at the level of a newly-qualified researcher or artist or the
general and specific professional competences needed for independent
application of academic or artistic knowledge at the level of a newly-qualified
professional.

Subject-/disciplinespecific
3
requirements

The aims and objectives of the programme (expressed in the learning outcomes
of the students) correspond with the requirements set by professional colleagues,
both nationally and internationally, and the relevant professional field for a
programme in the domain concerned (subject/discipline and/or professional
practice or practice of the arts). In the case of regulated professions, the
requirements correspond with the regulation or legislation concerned.
The learning outcomes of professional bachelor’s programmes have been
authenticated by the relevant professional field.
The learning outcomes of academic bachelor and master’s programmes stem
from requirements set by the academic and/or artistic discipline, international
academic practice and, for programmes to which this applies, the practice in the
relevant professional field.

3

The subject-/discipline-specific requirements operationalise the generic quality standards. In terms of hierarchy, they are subordinate to these
standards.

Page 6

NVAO | September 2009

2.1.2 Curriculum
Standards
Requirements for
professional /
academic
orientation

Criteria
The proposed curriculum corresponds with the following criteria for a professional or
academic orientation:
Professional orientation (bachelor’s programmes):
– Students develop their knowledge through the study of professional literature, the
study of material derived from professional practice or practice of the arts and
through interaction with the professional field, practice of the arts and/or (applied)
research;
– The curriculum has verifiable links with current developments in the professional
field or the discipline;
– The curriculum ensures the development of professional or artistic competences
and has verifiable links with current professional practice.
Academic orientation (bachelor’s and master’s programmes):
– Students develop their knowledge through the interaction between education and
research (including research in the arts) within relevant disciplines;
– The curriculum corresponds with current developments in the relevant academic
or scientific discipline(s) through verifiable links with current academic or scientific
theories;
– The curriculum ensures the development of competences in the field of research
and/or the development and practice of the arts;
– Where appropriate, the curriculum has verifiable links with the current relevant
professional practice.

Correspondence
between the aims /
objectives and the
curriculum

The curriculum is an adequate realisation of the intended learning outcomes of the
programme with regard to the level, orientation and subject-/discipline-specific
requirements.
The intended learning outcomes are properly reflected in the educational goals of
the curriculum or parts thereof.
The contents of the curriculum enable students to achieve the intended learning
outcomes.

Consistency of the
curriculum

Students follow a curriculum that is coherent in terms of content.

Workload

The actual duration of the programme is assessed and corresponds with the
statutory standards.
The intended learning outcomes are attainable because factors relating to the
curriculum that could hamper study progress are eliminated wherever possible.

Page 7

NVAO | September 2009

Admission
requirements

The structure and contents of the curriculum are in line with the qualifications of the
4
incoming students :
Bachelor’s programmes:
• Secondary school leaving certificate, Diploma van het Hoger Onderwijs van het
korte type met volledig leerplan, Diploma van het Hoger Onderwijs voor Sociale
Promotie or a diploma or certificate that is recognised as equivalent in accordance
with a law, decree, European directive or other international agreement;
• Predefined conditions set by the institution for individuals who do not meet the
above mentioned requirements.
Advanced bachelor’s programmes:
• A bachelor’s degree, with a qualification or qualifications specified in more detail by
the management of the institution, possibly supplemented with either an assessment
concerning the fitness or capacity of the individual or a preparatory programme.
Master’s programmes:
• A bachelor’s degree, with a qualification or qualifications specified in more detail by
the management of the institution, possibly supplemented with an individualised
curriculum, a preparatory programme or a bridging programme.
Advanced master’s programme:
• A master’s degree, with a qualification or qualifications specified in more detail by
the management of the institution, possibly supplemented with either an assessment
concerning the fitness or capacity of the individual or a preparatory programme.

Credits

The curriculum meets the legal requirements regarding the association of credits:
- Bachelor’s degree: at least 180 credits
- Advanced bachelor’s degree: at least 60 credits
- Master’s degree: at least 60 credits
- Advanced master’s degree: at least 60 credits

Coherence of
structure and
contents

The educational concept is in line with the aims and objectives. The study methods
correspond with this educational concept.

Learning
assessment

By means of evaluations, tests and examinations, the students are assessed in an
adequate manner which is insightful to them to determine whether they have
achieved the intended learning outcomes of the programme or parts thereof.

Master’s thesis

The master’s programme is concluded with the master’s thesis whereby the student
demonstrates the ability for analytic and synthetic reasoning, independent problem
solving at an academic level or artistic creation. The work reflects the general criticalreflective attitude or the research attitude of the student.
The master’s thesis corresponds to at least a fifth of the total number of credits with a
minimum of 15 and a maximum of 30 credits.

The admission requirements are laid down in the Flemish Higher Education Act. With effect from 2005/06, the
admission requirements pertaining to higher education have been laid down in the Act on Flexible Pathways in Higher
Education. In terms of content, these admission requirements are identical to those listed here.

4

Page 8

NVAO | September 2009

2.1.3 Staff commitment
Standards
Requirements for
professional /
academic
orientation

Criteria
The programme meets the following criteria for the deployment of staff for
a programme with a professional or an academic orientation.
Professional orientation:
– Teaching is principally provided by staff that link the programme to the
professional practice or practice of the arts.
Academic orientation:
– Teaching is principally provided by researchers who contribute to the
development of the subject/discipline (including research in the arts).
– In addition, and where appropriate, sufficient staff will be deployed with
knowledge of and insight in the professional field or practice of the arts
concerned.

Quantity of staff

Sufficient staff are deployed to realise the intended quality of the
programme.

Quality of staff

The staff deployed are sufficiently qualified to ensure that the aims and
objectives of the programme, in terms of content, didactics and
organisation, are achieved.

2.1.4 Services
Standards
Facilities

Criteria
Housing and facilities are sufficient for the realisation of the curriculum.

Tutoring

Tutoring and the provision of information to students are adequate in view
of study progress.
Tutoring and information provision are geared to students’ needs.

2.1.5 Internal Quality Assurance system
Standards
Evaluation of
results

Criteria
The curriculum is periodically evaluated in the light of verifiable objectives and
other measures.

Measures for
improvement

The outcomes of the evaluation form the basis for verifiable measures for
improvement that contribute to the achievement of the objectives.

Involvement of
staff, students,
alumni and the
professional field

Staff, students, alumni and the relevant professional field are actively involved in
the internal quality assurance system.

Page 9

NVAO | September 2009

2.1.6 Results
Standards
Achieved learning
outcomes

Criteria
The achieved learning outcomes correspond with the aims and objectives
regarding level, orientation and subject-/discipline-specific requirements.

Success rate

Target figures regarding success rate have been formulated on the basis
of a comparison with relevant other programmes.
The programme’s success rate is in line with these target figures.

Page 10

NVAO | September 2009

2.2 Distinctive quality features
The aim of accreditation is to establish whether a programme complies with generic quality
standards. To this end, the programme is assessed according to the assessment framework
referred to in Chapter 2. In order to emphasise any distinctive quality features, an institution
may additionally request the Accreditation Organisation to assess distinctive quality features of
a programme. This can lead to an annotation in the accreditation report that the programme
does indeed possess this distinctive quality feature. The assessment of a distinctive quality
feature has no influence on the accreditation decision of the Accreditation Organisation.
Distinctive quality features must fulfil the following criteria.

Standards
Criteria
Differentiation and The feature contributes in a meaningful way to differentiation and profiling in
profiling
higher education.
Quality

The feature contributes to the quality of the programme.

Specification

The effects of the feature for the quality of the programme itself (enrolment,
curriculum, teaching and learning process, output, services and staff quality) have
been operationalised.

Page 11

NVAO | September 2009

2.3 Assessment rules
2.3.1 Assessment of the standards
The assessment of the programme is performed by an external assessment panel (panel of
experts) co-ordinated by a quality assessment agency that lays down the assessment protocol.
The assessment panel judges the standards in the assessment framework according to the
following assessment scale:
– excellent
– good
– satisfactory
– unsatisfactory.
2.3.2 Assessment of the programme
For a positive final conclusion regarding the programme, each theme must be judged as
satisfactory. The assessment panels judge the themes in the assessment framework according
to a two-point scale: satisfactory or unsatisfactory. The assessment of a theme in the
assessment framework is based on the assessments of the separate standards of that theme.
The assessment panel has to demonstrate clearly how the assessment of the different
standards led to its final conclusion concerning that theme. In other words, the panel has to
clarify how – given the criteria in this accreditation framework and the reference framework
employed – it has arrived at its assessment of a theme on the basis of the analyses of the
underlying standards.
In its final conclusion regarding the quality of the programme, the assessment panel needs to
indicate how its conclusions are based on facts, its analysis of the evidence and its assessment
of the programme on the basis of this accreditation framework and the reference framework
employed. In its final conclusion regarding the programme, the assessment panel indicates
whether, based on the standards in the assessment framework, sufficient generic quality
standards are in place to warrant a positive final conclusion regarding the programme.
With respect to programmes comprising various modes of study, as referred to in Article 59 ter
of the Flemish Higher Education Act, a final conclusion can only be positive if the assessment
shows that sufficient generic quality standards are in place for each mode of study.

Page 12

NVAO | September 2009

2.4 Assessment rules for accreditation
2.4.1 Evaluation of external assessments
The quality assessment agency publishes the report regarding the quality of a programme as
adopted by the assessment panel. The report also outlines the procedure followed in the
external assessment.
The Accreditation Organisation verifies whether the external assessment has been conducted
in a regular and conclusive manner.
A regular external assessment publication is structured in accordance with the format outlined
in the quality assurance protocol. It comprises an assessment based on the generic quality
standards and the assessment criteria listed in this accreditation framework.
An assessment report is conclusive if it provides insight into the quality of the composition of the
assessment panel and permits a thorough investigation into the presence of sufficient generic
quality standards.
In this manner, a number of procedural and substantive requirements are verified.
With regard to the procedural requirements, the Accreditation Organisation verifies whether:
– 1. the assessment panel followed the assessment protocol laid down by the quality
assessment agency;
– 2. the reference framework used by the assessment panel contains the themes and
standards of the accreditation framework for existing programmes formulated by the
Accreditation Organisation, including a subject-/discipline-specific interpretation;
– 3. the assessment panel adequately applied the assessment rules described in this
accreditation framework;
– 4. the procedure followed conforms to the assessment framework for the procedures used by
the VLIR [Flemish Interuniversity Council] and the VLHORA [Council of Flemish University
Colleges] as quality assessment agencies or in the regulations for the recognition of other
quality assessment agencies.
With regard to the substantive requirements, the Accreditation Organisation verifies whether:
– 1. the quality judgement of the assessment panel is, in part, based on a comparison with
similar programmes and, if possible, with international standards applicable to programmes in
the domain concerned;
– 2. the report adopted by the assessment panel and published by the quality assessment
agency contains sufficient evidence to verify whether or not the programme fulfils the standards
of the assessment framework (Chapter 2) and whether sufficient generic quality standards are
in place. The report includes at least the six themes listed in this accreditation framework,
paying attention to all the standards in each theme. Each standard is assessed on a four-point
scale; the themes are assessed based on the underlying assessments of the standards. The
assessments are substantiated with evidence and analyses as thoroughly as possible. The
report is finalised with a summarised conclusion on the programme;
– 3. the report also provides insight into the quality of the assessment panel (the panel of
experts) that assessed the programme. Information is provided about the size and composition
of the assessment panel with reference to the knowledge and experience contributed by each
member, as well as his or her independence, expertise and authority.

Page 13

NVAO | September 2009

2.4.2 Assessment rules for accreditation
If the Accreditation Organisation determines that the external assessment panel took great care
in formulating its judgement5, it addresses the question whether the programme has sufficient
generic quality standards in place.
In making this decision, the Accreditation Organisation applies the same assessment rules as
the assessment agency.
The essence of these assessment rules has been laid down in the Regulation for DecisionMaking Procedures concerning (Initial) Accreditation of Programmes in the Flemish Community
[In Dutch: “Reglement tot bepaling van bestuursbeginselen die van toepassing zijn bij de
besluitvorming inzake accreditatie en toets nieuwe opleiding ten aanzien van opleidingen in de
Vlaamse Gemeenschap”]. Clauses 7 and 9 of this regulation read:
“Clause 7. An (initial) accreditation decision is positive if the overall mark for each theme
described in the (initial) accreditation framework is satisfactory.
If, at the discretion of NVAO, standards that are rated “unsatisfactory” are balanced out by
strengths demonstrated within other standards of that same theme, the theme will be rated as
“satisfactory”.
[…]
Clause 9. If a programme comprises various modes of study, as referred to in Article 59 ter of
the Flemish Higher Education Act, a theme can only be rated “satisfactory” if such rating is
awarded regarding that theme for each separate mode of study.”

5
If need be, following depletion of the options for remediation referred to in clauses 58bis and 59bis of the Flemish
Higher Education Act.

Page 14

NVAO | September 2009

3 Accreditation of programmes based on accreditation abroad
This chapter operationalises the implementation of Clause 60 sexies of the Flemish Higher
Education Act. This clause stipulates that in the assessment of accreditation applications for
programmes that have already been granted an international accreditation, the Accreditation
Organisation must verify whether such international accreditation has been granted in
accordance with a methodological approach that compares with the accreditations granted on
the basis of an external review.

Criteria for equivalence
General
The equivalence of a concrete accreditation granted by a foreign accreditation body is
assessed by criteria for equivalence whose contents can be clustered around five themes:
– - the foreign accreditation decision is based on a positive assessment of the quality of the
programme concerned;
– - the foreign accreditation decision is of a recent enough date;
– - the foreign accreditation decision is based on a public external review;
– - the foreign accreditation organisation has an adequate organisational structure;
– - the foreign accreditation organisation applies valid quality standards.
In principle, all criteria for equivalence per theme should be met before the Accreditation
Organisation can grant an accreditation.
Theme 1
The foreign accreditation decision is based on a positive assessment of the quality of the
programme concerned.
The foreign accreditation decision or the preceding review should demonstrate that the quality
of the programme concerned was positively assessed. The international accreditation may
pertain to other programmes as well, as long as there is a separate positive assessment that
refers specifically to the programme concerned.
Theme 2
The foreign accreditation decision is of a recent enough date.
The foreign accreditation decision must be of a recent enough date to allow a reasonable
judgement to be made as to the equivalence. For this reason, the period of time between the
date on which the accreditation to be recognised as equivalent was granted and the date on
which the accreditation application was submitted to NVAO must not exceed more than 90
calendar days.
Theme 3
The foreign accreditation decision is based on a public external review
The foreign accreditation decision must be based on a public external review of the programme
concerned, of the discipline under which the programme resorts or of the institution offering the
programme. The external review should be laid down in a public report.
The external review must be carried out according to previously communicated quality
guidelines. These guidelines must have been issued by an organisation that functions
independently from higher education institutions, such as an international association or
organisation, the legislative or executive powers of the country or state concerned, the foreign
accreditation organisation itself, an assessment agency recognised by the said accreditation
organisation or an assessment agency that has signed an administrative agreement with the
said accreditation organisation in which its independent functioning is stipulated.

Page 15

NVAO | September 2009

The external review must be carried out by an independent panel of experts consisting of peers
and one or more students, unless no candidacy for a student expert was fulfilled and/or student
involvement in external reviews concerning the programme, the discipline or the institution is
organised in a different way.
Among the experts in the panel, at least the following expertise must have been represented:
– subject-/discipline-specific expertise as to the subject/discipline concerned and the
professional practice of that discipline. Insofar as relevant, this expertise must be supplemented
by expertise regarding the international development of that discipline;
– educational expertise and expertise in the field of didactics;
– assessment expertise.
Theme 4
The foreign accreditation organisation has an adequately functioning organisational structure.
The international accreditation must have been granted by an autonomous accreditation
organisation. In this context, “autonomous” means that the political authority of the country or
state concerned and the institution concerned cannot decisively influence the decision-making
process of the accreditation organisation. The fact that an appeal can be lodged against
decisions taken by the accreditation organisation does not, as such, imply that the accreditation
organisation does not function autonomously. However, a precondition is that an appeal can
only lead to adjournment or annulment of the disputed accreditation decision. Lodging an
appeal must never lead to another authority taking a positive or a negative accreditation
decision. If that were the case, the final accreditation decision would be taken by another
(public) body and not by the accreditation organisation, which is a prerequisite for autonomy.
The international accreditation must have been granted by an organisation that operates a
functioning and continuous accreditation practice according to a set of distinctive working rules.
The accreditation must not have been granted on an ad-hoc basis. For that reason, the
accreditation of programmes must be the primary objective (or one of the primary objectives) of
the foreign accreditation organisation.
As a guarantee for its continuity, the foreign accreditation organisation must have been
recognised by a governmental authority. This recognition may be focused on the academic
and/or professional orientation.
The foreign accreditation organisation must keep in close touch with its stakeholders in higher
education in the country or state concerned, so as to be up to date where developments in
higher education are concerned.
Finally, the Accreditation Organisation verifies whether the foreign accreditation organisation
has an apparently adequate internal quality assurance system in place and operates in
accordance with such a system.

Page 16

NVAO | September 2009

Theme 5
The foreign accreditation organisation applies valid quality standards.
The foreign accreditation organisation must have assessed the same themes and standards
that are assessed in the Flemish accreditation framework. This means that the foreign
accreditation organisation must be able to demonstrate that it has assessed the following:
– the coherence and relevance of the intended learning outcomes of the specific programme or
group of programmes or the manner in which this coherence and relevance is guaranteed for
the programmes offered at the institution;
– the coherence of the curriculum of the specific programme or group of programmes or the
manner in which the curriculum is structured within the institution;
– the quality of the staff deployed for the specific programme or group of programmes or for the
programmes as a whole offered at the institution;
– the quality of the services and facilities for that specific programme or group of programmes
or for the programmes as a whole offered at the institution;
– the realisation of an internal quality assurance system as regards the specific programme or
group of programmes or the programmes as a whole offered at the institution;
– the quality of the achieved learning outcomes of the specific programme or group of
programmes or of the programmes as a whole offered at the institution.
The foreign accreditation organisation must have based its quality assessment on the Dublin
descriptors regarding the quality of higher education or another coherent set of criteria that
does not differ substantially from the Dublin descriptors.
The international accreditation should give a clear indication as to the professional or academic
orientation of the programme.
In its assessment, the foreign accreditation organisation should have demonstrably verified the
extent to which the specific programme or group of programmes or the programmes as a whole
offered at the institution meet broadly accepted and well-documented subject-/disciplinespecific requirements.

Page 17

NVAO | September 2009

4 Transitional arrangement for the assessment and
accreditation of converted academic programmes at
university colleges, with regard to the inter-relatedness of
education and research (2005-2013 period)
4.1 Introduction
An essential characteristic of a bachelor or master’s programme with an academic orientation is
that the programme reflects a sufficient degree of:
- academic support and
- inter-relatedness between education and research.
6
This hallmark of its academic nature, comprising the two aforementioned aspects , is
hereinafter referred to as “embedding education in research” or “embedment in research”.
Various criteria, standards and themes of the assessment framework contained in the
accreditation framework present the requirement for programmes with an academic orientation
of reflecting such characteristics.
Two-cycle university college programmes that have been converted to programmes with an
academic orientation will not be sufficiently embedded in research right from the introduction of
the converted programme. Undoubtedly, conversion to programmes with an academic
orientation is a difficult task for many of these programmes, especially with regard to the
aspects mentioned above. These programmes will be embedded in research in the years
ahead. The term “embedding in research” denotes the process by which the educational
content of a programme is gradually embedded in research until the programme meets all the
requirements set for programmes with an academic orientation. This embedding process
should be finalised by the end of the academic year 2012/13. In accreditation procedures based
on external assessment reports published after 1 January 2005, the academic nature of the
programmes under consideration is assessed under a transitional arrangement. In particular,
the requirements for embedment in research will be lowered for these programmes in the
period up to 2013. As regards all the other standards and themes in the accreditation
framework, these programmes must meet all the requirements of the accreditation framework
with effect from 2005, just like other programmes with an academic orientation.
This transitional arrangement has been laid down as a supplement to Clause 124 § 9 in the
Flemish Higher Education Act:
Clause 124 § 9 The Accreditation Organisation assesses the presence of sufficient generic
quality standards in programmes with an academic orientation provided by university colleges,
as referred to in § 1 and §4 and in Clause 125 ter, taking into consideration the transitional
nature of the academic support and the intertwining of education and research in the
programmes, provided the university college has submitted the accreditation dossier before the
end of the 2012/13 academic year.
This chapter aims to clarify the transitional arrangement with regard to the assessment
elements in the accreditation framework. To that end, we will first explore which themes of the
accreditation framework comprise standards or criteria that are directly related to a
programme’s embedding in research. Subsequently, we discuss how the criteria involved are
assessed under the transitional arrangement outlined above.

6
The first aspect primarily pertains to the academic “environment” of the programme, i.e., mainly the manner in which
teaching staff are actively involved in research. The second aspect relates to the manner in which students are brought into
contact with research.

Page 18

NVAO | September 2009

This transitional arrangement applies to programmes with an academic orientation provided by
university colleges and resulting from the conversion of two-cycle programmes, for which no
assessment reports have been published by the end of 2004. It also applies to programmes
with an academic orientation provided by university colleges and resulting from the conversion
of postgraduate programmes or the conversion of those departments of adult education centres
that have been taken over by university colleges. This transitional arrangement does not apply
to the programmes with an academic orientation resulting from the conversion of two-cycle
university college programmes whose assessment report has been published before the end of
2004. These programmes may be granted transitional accreditation until 1 October 2012 or 1
October 2013, depending on whether the assessment report was published in 2003 or 2004.
For these programmes, a progress test will be organised by the end of 2007 to assess the state
of affairs with regard to their embedding in research. This progress test will be conducted under
the responsibility of the Recognition Commission. In order to be granted accreditation, these
programmes must meet all of the requirements in the accreditation framework by 2012 or 2013,
respectively.
4.2 Related themes and standards of the accreditation framework
A study of all the themes, standards and criteria in the accreditation framework shows that the
following themes are directly related to embedment in research: Aims and objectives,
Curriculum, Staff commitment, Services. The following standards are relevant in this regard:
– Aims and objectives
– Level and orientation, bachelor’s programmes (academic orientation)
– Level and orientation, master’s programmes
– Subject-/discipline-specific requirements
– Curriculum:
– Requirements for professional and academic orientation
– Relationship between aims and objectives / curriculum content
– Master’s thesis
– Staff commitment
– Academic orientation
– Services
– Facilities (research-related infrastructure relevant to teaching)
Appendix 1 presents all the themes, criteria and standards that pertain to programmes with an
academic orientation. The criteria or sub-criteria printed in italics denote those that are relevant
with regard to the assessment of the transitional nature of the programme’s embedding in
research.
4.3 Assessment of the transitional nature
The essence of the transitional assessment is that the requirements for criteria related to
embedment in research are lowered during the transitional period (2005-2013). A general
principle in the transitional arrangement is that the programme must make a reasonable case,
on the basis of its progress at the time of assessment and its plans with regard to the further
embedding of its curriculum in research, which by 2013 will fully meet all the requirements in
the accreditation framework.
The procedure implemented for the accreditation of these programmes in the period from 2005
to 2013 corresponds to the general procedure for the accreditation of existing programmes, as
outlined in the draft accreditation framework for existing programmes. Along with the plans for
the further embedment in research, the conversion dossier (including the additional information
requested by the Recognition Commission) constitutes a source of information for the
assessment panels in this respect. These panels are responsible for assessing:

Page 19

NVAO | September 2009

– whether the progress made in embedding the programme in research, up until the time of the
external assessment, corresponds to the intentions specified in the conversion dossier and the
plans for further embedment in research; and
– whether the progress made and the further plans make a reasonable case that the
programme will fully meet the requirements for embedment in research laid down in the
accreditation framework by 2013 at the latest.
However, with effect from October 2013, only the actual progress made will be taken into
consideration in the accreditation of these programmes. No account will be taken of potential
progress by way of compensating for any failure in building up sufficient capacity.
7

The theme of “Aims and objectives” warrants a specific approach in this respect. A conversion
to programmes with an academic orientation is obviously intended to have the programmes
meet the criteria the accreditation framework sets for the aims and objectives of programmes
with an academic orientation. On the other hand, most programmes will not yet be able to fully
realise this academic nature. For that reason, feasible aims and objectives will be drawn up
during the transition period, which will gradually evolve into the aims and objectives for
programmes with an academic orientation. This means that in the assessments, a distinction
will be made between:
– long-term aims and objectives: the aims and objectives of the programme (with an academic
orientation) with regard to competencies to be achieved following completion of the embedment
process;
– short-term aims and objectives: the aims and objectives of the programme with regard to
competencies sought to be achieved during the transition period. When assessing the
programme, the assessment panels must verify whether the programme is achieving these
short-term aims and objectives and whether it is likely to achieve its long-term aims and
objectives by 2013 at the latest.
In the years ahead, the conversion process will gradually be accomplished. Bachelor’s
programmes with an academic orientation will be introduced in the 2004/05 or 2005/06
academic years, the subsequent master’s programmes will start in the 2007/08 or 2008/09
academic years. This must be taken into account in the assessment. For the accreditation of
the university college programmes involved this means in concrete terms:
– that the initial years will mainly focus on descriptions of the aims and objectives, the
curriculum and the intentions with regard to staff commitment and services/facilities related to
embedment in research,
– that gradually more attention will be focused on the progress made with regard to
embedment in research, for the bachelor’s programmes from the 2005/06 or 2006/07 academic
years onward and for the master’s programmes from the 2008/09 or 2009/10 academic years
onward.
4.4 Scope of accreditation criteria in the transition period
The supplement to the Higher Education Act stipulates that university college bachelor’s and
master’s programmes (academic orientation) to which the transitional arrangement applies will
be granted accreditation for four years when they apply for accreditation under the transitional
arrangement before the end of the 2008/09 academic year. Such programmes applying for
accreditation after this date will be granted accreditation for six years. For that reason, a
distinction is made between the period from 2005 to 2008 and the period from 2008 to 2012 in
the application of the accreditation criteria. This defines the progress programmes must achieve
in each period in order to make a reasonable case that they will fully meet the requirements set
down in the accreditation framework by 2013. This means that these requirements involve

7
The term “aims and objectives” in the Flemish Higher Education Act and the accreditation framework indicates the
competences the programme seeks to impart on its students.

Page 20

NVAO | September 2009

benchmarks rather than strict criteria. Similarly, the point of time indicated should be regarded
as a reference date rather than a fixed fact.
The table in Appendix 1 presents an overview of the transitional scope of the criteria from the
draft accreditation framework that are relevant to the embedment in research of bachelor’s and
master’s programmes (academic orientation) resulting from the conversion of two-cycle
undergraduate programmes provided by university colleges. In the transition period up to 2013,
the requirements to be met under these criteria will be lowered for the programmes involved.
No transitional assessment will be implemented for the other criteria, standards and themes
that are not related to embedment in research.
The table is structured as follows.
– Column 1 describes all the themes, standards and criteria from the draft accreditation
framework for existing programmes relating to programmes with an academic orientation; the
elements that pertain to embedment in research are printed in italics.
– Column 2 describes the related scope of the transitional assessment of the criteria or subcriteria pertaining to embedment in research with a view to accreditation of such programmes,
when the assessment is conducted prior to 2008.
– Column 3 describes the same for programmes whose assessment is conducted in 2008 or
beyond.
– By way of explanation for quality assessment agencies and the Recognition Commission,
column 4 indicates what information may be used to determine whether the programme meets
the criteria with regard to embedment in research. This information pertains to both actual
achievements and plans. The conversion dossier, which specifies the state of affairs regarding
the programme’s embedment in research at the time the dossier is compiled and the intentions
with regard to the programme’s embedment in research, is used as a source of information in
this respect, along with the plans for continued embedment in research. Other essential
information documents are the additional policy plans regarding the programme’s embedment
in research, covering the period up to 2013.
The conversion dossier also serves as a reference for the progress test that will be conducted
under the responsibility of the Recognition Commission. The progress test is foreseen for the
end of 2007, which constitutes additional grounds for opting for 2008 as the pivot point in the
transitional arrangement.

Page 21

NVAO | September 2009

Explanatory notes
General
Points of departure
Accreditation is defined as “the formal recognition of a programme based on a decision by an
independent organisation, stating that the programme fulfils predefined minimum requirements
regarding quality and level”. Such recognition depends on the satisfactory fulfilment of generic
quality standards. By granting a formal quality mark, accreditation is the culmination of the
assessment of the quality of programmes. The mark is granted by an Accreditation
Organisation after validating an external assessment report drafted by an assessment panel coordinated by a quality assessment agency. The subject of accreditation is the programme. The
initiative for the application for accreditation lies with the institution.
By signing the Bologna Declaration, all European countries concerned decided to implement an
educational model of two cycles in the pursuit of a European Higher Education Area. In line with
this, many countries implemented accreditation systems in order to determine the level –
bachelor’s or master’s – and quality of their programmes. This also bolsters the international
mobility of students.
The Accreditation Organisation for Flanders was established through a treaty between the
Flemish and the Dutch governments. This treaty defines the tasks, composition and
competence of the Accreditation Organisation. The treaty also regulates how the accreditation
framework to be set down by the Accreditation Organisation acquires legal force.
The Accreditation Organisation decides separately on the accreditation of bachelor’s and
master’s programmes. This does not alter the fact that related bachelor’s and master’s
programmes can be assessed jointly. However, the assessment panel is obliged to give an
overall mark – satisfactory or unsatisfactory – for each programme (i.e., separately for
bachelor’s and subsequent master’s programmes).
Accreditation framework
The Flemish Higher Education Act and the Accreditation Framework for existing programmes
dated 14 February 2003, formulated by the Accreditation Organisation of the Netherlands
(NAO) were taken as the point of departure for drafting this accreditation framework. The views
of the organisations consulted prompted us to, on the one hand, endeavour to achieve the best
possible agreement between the Dutch framework and the Flemish framework, while on the
other deviating from the Dutch framework when necessitated by stipulations in the Flemish
Higher Education Act or differences between Flemish and Dutch higher education. Various
aspects of these Explanatory Notes explicitly refer to the regulations of the Flemish higher
education system. Similarly, the themes, standards and criteria are explained in the context of
the Flemish situation.
Accreditation of existing programmes is based on an assessment of six themes:
1. Aims and objectives
2. Curriculum
3. Staff commitment
4. Services
5. Internal quality assurance
6. Results

Page 22

NVAO | September 2009

These themes are subdivided into standards and criteria, differentiated according to level
(bachelor´s or master´s) and orientation (professional or academic) of the programme.
The table below shows how the themes and standards from the accreditation framework
correspond to the generic quality standards laid down in the Flemish Higher Education Act
(Clause 58).
Generic quality standards laid down in
the Flemish Higher Education Act
Educational contents
– Nature and level of the curriculum
– Consistency of the curriculum
– Workload
– Correspondence between intended
learning outcomes and content

Educational process
– Harmony between design and contents
– Student counselling
– Insightful evaluation and testing

Material facilities, quality of staff,
organisation and internal quality assurance
Methods for self-evaluation

Accreditation framework for existing
programmes in Flanders
Aims and objectives
– Level and orientation: bachelor’s degree,
professional higher education
– Level and orientation: bachelor’s degree,
academic higher education
– Level and orientation: master’s degree
– Subject-/discipline-specific requirements
Curriculum:
– Requirements for professional / academic
orientation
– Correspondence between the aims and
objectives and the curriculum
– Consistency of the curriculum
– Workload
– Admission requirements
– Credits
Curriculum
– Harmony between design and contents
– Learning assessment
– Master’s thesis
Services
– tutoring
Staff commitment
– Requirements for professional / academic
orientation
– Quantity of staff
– Quality of staff
Services
– Facilities
Internal quality assurance
– Evaluation of results
– Measures for improvement
– Involvement of staff, students, alumni and
the professional field
Assessment by the assessment panel of the
quality of the self-evaluation (including the
methods used)

Page 23

NVAO | September 2009

Wherever possible, the subdivision of the accreditation framework, the choice of themes,
standards and criteria and the level of abstraction applied have been geared to the Dutch
accreditation framework.
Subject-/discipline-specific framework of reference
An assessment of the programme cannot be based solely on general criteria pertaining to aims
and objectives, curriculum, facilities, staff commitment and internal quality assurance. The
panel must expressly examine whether the programme fulfils the requirements for this specific
programme and the field of study concerned as set by professional colleagues at home and
abroad as well as by the professional practice or practice of the arts. In cases of regulated
professions, account must also be taken of the regulations or rules concerned. The assessment
panel must therefore be able to assess both the subject-/discipline-specific quality as well as
the general quality of the programme. This entails specific requirements with regard to the
composition of the assessment panel.
In addition, the assessment panel must have a subject-/discipline-specific reference framework
in the light of which the programme is assessed. For this purpose, the quality assessment
agency draws up a procedure in the assessment protocol. For academically-oriented
programmes, connection can be sought with national and international developments in the
subject/discipline. The Flemish Higher Education Act stipulates that professionally-oriented
bachelor’s programmes must fulfil “profession-specific competences” and that academic
programmes must fulfil “specific profession-oriented competences required for independent use
of knowledge”. These competences must enable academic graduates to either conduct
independent research / practice the arts independently or deploy academic / scientific / artistic
knowledge independently at the level of a newly-qualified professional. These provisions
constitute the statutory basis for subject-/discipline-specific reference frameworks. Without such
frameworks, a quality assessment agency is unable to assess whether a programme fulfils the
requirements demanded of it by the field of study and/or the relevant professional practice. The
quality assessment agency may adopt any existing professional profiles and curriculum profiles,
provided these profiles are supported in the sector and the professional field and tie in with
relevant developments in the professional field and in the field of study. The quality assessment
agency will have to indicate the procedure for drafting a subject-/discipline-specific reference
framework. If need be, reference frameworks drawn up for earlier assessments or reference
frameworks provided by the programmes may be used, insofar as these are sufficiently current
and carry sufficient support within the subject/discipline and relevant national and international
professional practice. Ultimately, the assessment panel is responsible for establishing a
subject-/discipline-specific framework.

Page 24

NVAO | September 2009

Embedding programmes with an academic orientation in research
An essential characteristic of programmes with an academic orientation is that the programme
is supported by research and that teaching and research are intertwined. After all, teaching in
academically-oriented programmes should be founded on research. In various places in the
accreditation framework, requirements have been formulated regarding the link with research.
These requirements refer to:
– the aims and objectives of the programmes (in accordance with the provisions in Clause 58
of the Flemish Higher Education Act);
– the curriculum (requirements regarding academic orientation, such as interaction between
research and teaching, tying in with academic developments and current scientific theories,
guaranteeing competences in the area of research and the master’s thesis);
– the staff (teaching is predominantly undertaken by researchers who contribute to the
development of their subject/discipline);
– the material facilities concerning the research-related infrastructure relevant to education.
These requirements apply to all programmes with an academic orientation. Validation of these
requirements should be undertaken in the light of the research activities of the staff (including
research projects, doctoral research and scientific output). The interpretation of these
requirements will depend on the type of programme (bachelor’s, master’s or advanced master’s
degree) and the phase in the course. As a rule, embedding the programme in research will be
more pronounced in the master’s programmes than in the bachelor’s programmes. Research
competences will be introduced particularly in master’s programmes and will be most
pronounced in the master’s thesis.
As possible aims and objectives of a master’s programme, the Flemish Higher Education Act
mentions on the one hand imparting the necessary academic competences that will enable the
graduate to conduct independent research or to practice the arts independently at the level of a
newly-qualified researcher or artist, and on the other, imparting general and specific professionoriented competences necessary for the graduate to use academic or artistic knowledge
independently at the level of a newly-qualified professional.
Depending on the subject-/discipline-specific nature of the programme, the requirements for the
programmes will be specified in more detail. Thus, for instance, in the case of the master’s
programmes that are specifically aimed at training in research (such as “research master’s
programmes”), the subject-/discipline-specific requirements should run parallel with the
international standards for such programmes. In addition, in assessing the aims and objectives
of these programmes, more weight is given to the acquiring of competences necessary for
conducting independent research at the level of a newly-qualified researcher and stricter
requirements are set for embedding the programmes in research. This means that stricter
requirements will be imposed regarding various aspects, including the research capacity of the
staff and their experience in training newly-qualified researchers, a wide research orientation in
the curriculum, the academic or scientific interpretation of the master’s thesis and the
connection with current academic or scientific developments.
External quality assessment and accreditation
The point of departure for the accreditation process is that it is a continuation of the external
quality assurance. External quality assurance is particularly aimed at improving quality, while
accreditation is aimed at assessing whether the requirements for generic quality have been
fulfilled, as described in Chapter 2 of this accreditation framework.
VLIR [Flemish Interuniversity Council] and VLHORA [Council of Flemish University Colleges]
are responsible for co-ordinating the external assessment of the statutory registered institutions
for higher education. All bachelor’s and master’s programmes organised by these institutions
must participate in the assessments co-ordinated by VLIR and VLHORA. VLHORA co-ordinates
the joint external assessment of professionally-oriented bachelor’s programmes. VLIR coordinates the joint external assessment of programmes that are provided by the universities and

Page 25

NVAO | September 2009

by the statutory registered institutions that are neither a university nor a university college.
Together VLIR and VHLORA co-ordinate the joint external assessment of the academicallyoriented bachelor’s and master’s programmes that are provided by university colleges within
the framework of an association, and of the academic programmes jointly organised by a
university and university college.
With effect from the external assessments that are conducted on the basis of self-evaluation
reports completed before 1 September 2010, VLUHR [Council of Flemish Universities and
University Colleges] will take over the role of quality assessment agency from VLIR and
VHLORA.
The Flemish Higher Education Act prescribes clustered external assessment of similar
programmes at all universities or university colleges and other statutory registered institutions
within the framework of the external quality assessment by VLIR/VHLORA/VLUHR.
Assessment panels carry out the external assessment of the programmes or clusters of
programmes. The quality assessment agencies publish the result of the assessment of the
programmes in a report adopted and made public by the assessment panel.
The following procedure for accreditation is incorporated in the Flemish Higher Education Act.
The Accreditation Organisation decides to grant accreditation, if based on the report of an
external assessment it is of the opinion that the programme fulfils the generic quality standards.
The Accreditation Organisation assesses the report of the quality assessment agency. The
report of the quality assessment agency contains a summarised conclusion on whether the
programme fulfils the requirements for generic quality, as described in Chapter 2 of this
accreditation framework.
External assessments for the purpose of accreditation can be performed by VLIR, VLHORA
and at a later date, VLUHR or another quality assessment agency recognised by the
Accreditation Organisation. Alongside VLIR, VLHORA and VLUHR, only quality assessment
agencies recognised by the Accreditation Organisation may perform external assessments for
the purpose of accreditation.
For external assessments relating to external quality assurance that are performed under the
responsibility of VLIR, VLHORA or VLUHR and used as external assessments for the purpose
of accreditation, the report and the procedure followed must fulfil the requirements set by the
Accreditation Organisation.
It is of the utmost importance that the implementation of the accreditation system does not lead
to corrosion of the improvement function of the external quality assurance, which may be
regarded as an important achievement of the former assessment system. Ensuring the purpose
of quality improvement is the responsibility of VLIR, VLHORA or, as the case may be, VLUHR,
as part of their statutory task of co-ordinating external quality assurance.

Page 26

NVAO | September 2009

Scope
Accreditation relates to programmes of statutory registered institutions and programmes of nonstatutory registered institutions.
Accreditation is a precondition for registering a programme on the Flemish Higher Education
Register. Registration on the Higher Education Register is a precondition for a programme to
award recognised bachelor´s and master’s degrees. For the period 2003—2006, the Flemish
Higher Education Act, in anticipation of a new funding system, provides university colleges and
universities with individually established (“frozen”) budgets. Under the Flemish Higher
Education Act, only programmes that are registered on the Higher Education Register will be
eligible for funding. It is to be expected that in the future funding system, the funding of
programmes provided by statutory recognised institutions will depend on their having attained
accreditation.
Under the Flemish Higher Education Act, other institutions may be registered, provide
recognised bachelor´s and master’s programmes and award recognised bachelor´s and
master’s degrees. A precondition is that the programmes concerned are accredited and
registered on the Higher Education Register. Programmes that are provided by these registered
institutions do not have to comply with all the provisions of the Flemish Higher Education Act,
for instance, the language regulation. These non-statutory registered institutions are not funded
by the government.
Embedding programmes in research
Within the framework of embedding programmes in research, two-cycle university college
programmes may be converted to academically-oriented bachelor’s and master’s programmes
in association with a university. By the end of the period of accreditation by virtue of law (the
end of the academic year 2012/13), all academic programmes provided by universities and
university colleges must fulfil the same accreditation requirements. These requirements are
described in Chapter 2.1 of this accreditation framework.
A number of two-cycle university college programmes may not be able to shortly fulfil the
requirements laid down in this assessment framework regarding the criteria pertaining to
embedding the programme in research. Moreover, within the associations, embedding these
programmes in research will still take a number of years. For this reason, during the transitional
period up to 2013, these programmes may request accreditation under a transitional
arrangement. This transitional arrangement is described in Chapter 4 and applies to
programmes whose first assessment report was published after 31 December 2004. These
programmes must apply for accreditation within one year after the publication of this report, at a
time when the embedment process is still in progress. The time of accreditation in the transition
period will strongly affect the extent to which the embedment process has already been
realised. In the period up to 2013, assessment of the embedding in research of programmes
with an academic orientation provided by university colleges that are making use of the
transitional arrangement will be based on the extent to which both the criteria from the
accreditation framework as well as the intentions for embedment of the programmes have been
realised. This transitional assessment also applies to programmes with an academic orientation
provided by university colleges that ensue from the conversion of advanced programmes or the
conversion of programmes that were taken over from adult education centres.
In the self-evaluation document, the programme must indicate to what extent embedment in
research has been realised and what steps it is taking to fully meet the criteria from the
accreditation framework at the end of the transitional period. This is reviewed by the
assessment panel. For programmes that are accredited under the transitional arrangement, the
accreditation term is reduced to four years provided the institution submits its application for
accreditation prior to the end of the academic year 2008/09 and to six years when the institution
submits its application for accreditation between 1 October 2009 and the end of the academic
year 2012/13. Programmes that do not wish to appeal to the transitional assessment of
academic/scientific support and the intertwining of education and research must state this
explicitly in their self-evaluation.

Page 27

NVAO | September 2009

Higher Education in the Arts
Flanders has a limited number of one-cycle art programmes that are being converted into
professionally-oriented bachelor’s programmes. In addition, there are two-cycle programmes in
the field of the audio-visual and visual arts, music and performing arts. These programmes are
being converted into academically-oriented programmes; they must be provided within the
framework of an association with a university and must fulfil the requirements that are
demanded of bachelor’s and master’s programmes with academic orientation, as formulated in
Chapter 2 of this accreditation framework. This means, among other things, that the programme
must be intertwined with research and that a substantial number of the teachers must be active
in research in the arts.
Advanced bachelor’s and master’s programmes
The admission requirements for advanced bachelor’s and master’s programmes differ from
those that are in place for regular bachelor’s and master’s programmes. Apart from that, these
programmes must fulfil the same quality requirements as other bachelor’s and master’s
programmes.

Page 28

NVAO | September 2009

Explanatory notes per chapter
Chapter 1: structure of the accreditation framework
The accreditation framework comprises criteria concerning contents along with criteria for the
procedure followed and the report by the quality assessment agency. The criteria for assessing
distinctive quality features of a programme are different in status from the other criteria because
the conclusion about distinctive quality features does not influence the accreditation.

Chapter 2: accreditation of programmes based on a published external assessment
report
Assessment framework
The assessment framework contains the criteria that the Accreditation Organisation uses to
examine whether the programme fulfils the requirements for generic quality.
– Re 2.1. Aims and objectives of the programme
The level and orientation of the aims and objectives of the programme are examined in the light
of the learning outcomes for professionally-oriented bachelor’s programmes, academicallyoriented bachelor’s programmes and master’s programmes as contained in Clause 58 of the
Flemish Higher Education Act.
In addition, the aims and objectives are assessed in the light of subject-/discipline-specific
requirements that are set by the relevant discipline concerned and/or by the professional
practice or practice of the arts. For professionally-oriented bachelor’s programmes, the learning
outcomes must be assessed with the relevant professional field in the broadest sense, which
may include the relevant social sector.
– Re 2.2. Curriculum
The programme’s intended general and subject-/discipline-specific learning outcomes (the
competences to be attained) must be specified explicitly in the curriculum. The proposed
contents and design of the curriculum (including the attainment targets of the programme) must
ensure that incoming students can achieve the intended learning outcomes within the set time.
The aims and objectives of the programme are assessed through its contents. Professionallyoriented bachelor’s programmes must accord to relevant developments in professional practice;
academically-oriented bachelor’s and master’s programmes must accord to developments in
the relevant research disciplines. Essential aspects in the assessment of academically-oriented
programmes are the embedment of the programme in research and the development of
competences in the area of research (including research in the arts). Academically-oriented
master’s programmes may aim at imparting competences for independent research or
independent practice of the arts at the level of a newly-qualified researcher or artist, and/or at
imparting profession-oriented competences for independent use of knowledge at the level of a
newly-qualified professional. In the latter case, such programmes will have an additional
professional orientation. Therefore, the requirements for bachelor’s and master’s programmes
with an academic orientation include that where applicable the programme must have
demonstrable ties with the actual practice of the relevant profession.
The structure of the curriculum is also subject to assessment. The focus is on the
correspondence between the aims and objectives and the contents of the curriculum, the
correspondence between the various components of the curriculum, the workload, the
alignment with the needs of incoming students, the coherence of structure and contents,
learning assessments (including test formats and the information provided to students about the
Page 29

NVAO | September 2009

procedure and the forms of evaluation) and the organisation of the curriculum. With regard to
the workload, the correspondence between actual and budgeted study time is also assessed.
The standardisation applied in this framework must be seen in the light of the current statutory
provisions. The standards included in the framework correspond to the questions posed in the
current assessments by VLHORA and VLIR. They focus on assessing whether the curriculum
contributes to the attainment of the aims and objectives. To this end, formulations are used that
provide sufficient room for current developments concerning the standards.
The quality of the curriculum must also be apparent from its alignment with the qualifications of
incoming students. In this respect the statutory and regulatory admission requirements are
important. In addition to fulfilling the formal admission requirements, the programme will have to
ensure that its curriculum is effectively geared to the average level of incoming students. In this
respect, the quality of preparatory / bridging programmes and the admissions policy will also be
assessed. With regard to curricula that are assessed as a model trajectory for achieving the
aims and objectives of the programme, application of the Flemish Act on Flexible Pathways in
Higher Education allows individualised trajectories for students, flexibility in learning
environments, curricula and educational organisation, and the recognition of prior learning
(competences and qualifications achieved elsewhere). When the occasion arises, these forms
of flexibilisation, careers guidance and supervision regarding study progress are also taken into
account in assessing the themes, standards and criteria involved.
– Re 2.3. Staff commitment
Regarding the criteria for the deployment of staff, it is important that the expertise of the staff
8
employed in the programme is sufficiently in line with the level (bachelor’s or master’s) and the
orientation (academic or professional) of the programme.
This means that in the case of programmes with a professional orientation, sufficient staff must
be able to draw correspondences with professional practice. In the case of programmes with an
academic orientation, a significant proportion of the staff deployed for the programme must be
active in research and in addition, wherever relevant, sufficient staff must possess experience,
knowledge and insight into the profession or artistic practice concerned and/or possess
international expertise.
The first standard refers to the requirements for staff given the orientation of the programme
(professional or academic). The third standard (quality of staff) refers to the necessity of the
presence of a range of expertise depending on the aims and objectives of the programme and
the educational organisation, and to the corresponding staff development policy.
The Flemish Higher Education Act does not explicitly mention the standard of quantity of staff
9
under generic quality standards. It is included in the accreditation framework because the
presence of sufficient staff is an obvious precondition for generic quality.
– Re 2.4. Services
The services must be sufficient to realise the curriculum. Services include facilities, tutoring and
the provision of information to students. Tutoring must be provided throughout the length of the
course. The provision of information to students and tutoring must be tailored to the
requirements of the students. Future students must also be informed adequately about
expectations regarding their competences.
The facilities include multimedia libraries, laboratories, equipment, study space or subjectspecific databases. The nature and level of these facilities differ, depending on the character of
the programme. During assessment, it is verified whether the facilities correspond with the
generally prevalent level of quality for such facilities.
8
This not only includes staff associated with the institution providing the programme but, as the occasion arises, any
persons contributing to the programme regardless of their position.
9
Dutch legislation also fails to explicitly mention this standard.

Page 30

NVAO | September 2009

– Ad 2.5. Internal quality assurance
The institution must demonstrate that quality is structurally and permanently monitored and
improved. The quality of its quality assurance offers the best guarantee that the programme is
actively engaged in identifying weaknesses and taking measures to improve them. Thus, quality
assurance is the best guarantee for maintaining and improving the quality established at
accreditation, throughout the period for which the programme has been accredited.
Involvement, particularly of staff and students but also of alumni and the professional field, is a
fundamental feature of internal quality assurance. This involvement should be evident from the
method of consultation with all parties, their role in decision-making and how their
recommendations are incorporated in the programme.
– Ad 2.6. Results
The ultimate question is whether the intended aims and objectives have been realised. This is
determined by comparing the achieved learning outcomes with the intended learning outcomes
for level, orientation and subject-/discipline-specific features. Comparing the achieved learning
outcomes with the intended learning outcomes also comprises assessing whether the achieved
learning outcomes of the graduates of the programme have sufficient social relevance, as
mentioned in Clause 58 of the Flemish Higher Education Act. After all, the social relevance of
the programme is expressed in the aims and objectives of the programme (in particular in the
subject-/discipline-specific requirements). The success rates must also be acceptable. The
stipulation contained in the Act (satisfactory success rates) has been operationalised in the
standard for success rate by incorporating in the criteria that target figures must be formulated
in comparison with relevant other programmes. When formulating the target figures, the
requirements to be demanded of incoming students can also be taken into account, as well as
the open access to higher education for most programmes and the programme’s own profiling
on distinctive features. The programme must indicate the success rate achieved in its selfevaluation in relation to the target figures quoted and the success rate of other relevant
programmes.

Page 31

NVAO | September 2009

Distinctive quality features
Besides accreditation, which is aimed at establishing the presence of sufficient generic quality
standards, the Flemish Higher Education Act (Clause 59) also offers the possibility of
commenting on distinctive features of the programme in the accreditation report. In this
accreditation framework, the following criteria are included for that purpose:
– the distinctive feature must make a significant contribution to the differentiation and profiling
within Flemish higher education, i.e., the distinctive feature must lead to greater diversity in the
design of programmes;
– the distinctive feature must contribute to the overall quality of the programme, i.e., the
programme concerned would not be as good without that specific feature (or without all the
elements that make up this feature);
– the effects of the distinctive feature on the quality of education must be made operational,
i.e., it must be clear what the consequences of the feature are for the aims and objectives,
contents and design of the curriculum and services.
Prior to the assessment, the institution should apply for assessment of the distinctive feature by
the quality assessment agency. The institution has to provide the relevant information about the
standards of the distinctive feature on the basis of which the quality assessment agency can
arrive at a conclusion.
Examples of distinctive features are a pronounced regional or international orientation, a
specific bond with research, a distinctive educational concept or an extremely effective
connection to the profession.
Assessment rules
The assessment panel gives a summarised assessment of the quality of the programme with a
positive or negative overall conclusion concerning the presence of sufficient generic quality
standards. The Accreditation Organisation has formulated assessment rules that describe the
method by which the assessment panel gradually arrives at this final conclusion. For a positive
final conclusion the programme must be rated “satisfactory” on all themes from the
accreditation framework. The assessment panel must provide insight into its considerations at
all levels – standards, themes and final conclusions – so that it will be clear how the final
conclusion was reached and how the different standards were weighed.
The standards are assessed on the basis of a four-point scale: unsatisfactory, satisfactory,
good and excellent. The descriptors “unsatisfactory” and “satisfactory” are the primary
benchmark for the assessment. The rating “good” shows that the quality surpasses the generic
quality. “Excellent” was included in the assessment scale as the rating that exposes “best
practices”, which can have an exemplary function for other programmes.
Within a theme, individual standards can be rated as “unsatisfactory”. However, at the theme
level, a programme must always earn the mark “satisfactory”. If certain standards are rated
“unsatisfactory” but the overall theme is rated “satisfactory”, the assessment panel has to
provide insight into its considerations in order to enable the Accreditation Organisation to follow
and evaluate its decision.
Some programmes comprise various modes of study, such as specialisations, different
locations, languages,…. Accreditation, however, takes place at the level of the entire
programme rather than that of the individual modes of study. When a programme is rated as
unsatisfactory because of a lack of quality in a particular mode of study, the institution
concerned may exclude that mode of study from the accreditation application. Excluding a
mode of study from the application for accreditation implies that it can no longer be offered once
the programme has been accredited.

Page 32

NVAO | September 2009

Assessment rules for accreditation
In its report, the quality assessment agency accounts for the procedure followed during external
assessment. The Accreditation Organisation assesses the procedure on the basis of the report
by the assessment panel and the assessment protocol of the quality assessment agency.
Independence, expertise and authority of the assessment panel convened by the quality
assessment agency are the essential factors. The expertise comprises knowledge of one’s own
discipline and the professional field concerned as well as expertise in the field of quality
assessment and expertise of educational issues relating to higher education programmes. The
Accreditation Organisation also assesses the procedure as regards the contribution of students,
both during the assessment (has the panel interviewed students enrolled in the programme?)
and in the composition of the assessment panel.
The assessment panel has to substantiate its conclusions on the basis of the reference
framework used (which also includes the subject-/discipline-specific requirements) and its
analysis of the facts. A comparison of the programme with other programmes, preferably
international, must be part of the procedure.
If the Accreditation Organisation concludes that the external assessment has been conducted
in a regular and conclusive manner, it takes a final decision on whether the programme offers
sufficient quality guarantees.
In this respect, the Accreditation Organisation may, of course, endorse the external
assessment. In this case, the external assessment constitutes the grounds for the accreditation
decision.
However, the Accreditation Organisation may also partly or completely disagree with the
external assessment report. In that case, the accreditation decision must be based on the
assessment rules in place for assessment panels (see Chapter 4). The Accreditation
Organisation shall then explicitly give grounds for its decision in the accreditation report.

Page 33

NVAO | September 2009

Explanatory notes to the system of accreditation by virtue of
law
For the purpose of introducing the accreditation system, the possibility of accreditation by virtue
of law has been laid down by decree. The duration of accreditation by virtue of law depends on
the publication date of the last assessment report. Programmes whose assessment report was
published between 1 January 1998 and 31 December 2004 are deemed to have been
accredited up until the end of the academic year beginning in the eighth calendar year following
the year in which the assessment report was published. Programmes whose external
assessment was published after 1 January 2005 are deemed to have been accredited up until
the end of the academic year beginning in the year following that in which the assessment
report was published. In any case, all accreditations by virtue of law will expire by the end of the
2012/13 academic year.
Postgraduate programmes converted into bachelor’s or master’s programmes will be accredited
by virtue of law for a period of four years, with the exception of postgraduate programmes that
have been converted into initial bachelor’s or master’s programmes through amalgamation with
an undergraduate programme. In the latter case, the arrangement for converted initial
bachelor’s or master’s programmes applies. The period of four years for accreditation by virtue
of law also applies to bachelor’s and master’s programmes ensuing from the conversion of
adult education centre departments taken over by university colleges.
The Flemish government may adapt the duration of the accreditation by virtue of law or the
timeframes for application for converted programmes originating from (components of) several
original programmes or in order to allow for the joint assessment of programmes.
For programmes whose assessment report was published after 1 January 2005, the external
assessment must be based on the criteria laid down in this accreditation framework.
For the new bachelor’s and master’s programmes, external assessment for the purpose of
accreditation will only be used to a limited extent in the years ahead. The assessment reports
on which the accreditation decisions are based will mainly reflect the external assessment of
the quality of existing programmes, followed by that of the bachelor’s and master’s programmes
under construction and the “old” programmes that are being phased out. The converted
bachelor’s and master’s programmes will not be reviewed until the end of the transition period.
This procedure has been set up in order to avoid having to conduct an excessive number of
assessments at the same time, i.e., once the bachelor’s and master’s programmes have been
fully developed.

Page 34

NVAO | September 2009

Appendix 1.
Interpretation of accreditation criteria in transition period for programmes with an academic
orientation that ensue from the conversion of university college programmes (pertaining to
embedment in research)
Themes, standards, criteria

Specific assessment of designated
elements until 2008

Accreditation framework for existing higher education programmes in Flanders

Pagina 35 NVAO |
| Februari 2005 |

Specific assessment of designated
elements after 2008

Specific information

Theme: 1. Aims and objectives
Standard:
1.1 . Level and orientation: bachelor’s programmes,
professional orientation
Criteria:
The aims and objectives of the programme are focused on having
the students master:
 general competences such as the capacity for logical
thought and reasoning, the ability to acquire and process
information, the ability for critical reflection and projectbased work, creativity, the ability to perform simple
supervision tasks, the ability to communicate information,
ideas, problems and solutions to both specialists as well as
laymen, and a positive attitude towards life-long learning;
 general professional competences such as the ability to
work together as part of a team, a solution-oriented attitude
in the sense of being able to define and analyse
independently complex problematic situations in
professional practice, and the ability to develop and apply
effective strategies to resolve them, and to develop a sense
of social responsibility in connection with the professional
practice;
 specific professional competences at the level of a newlyqualified professional.

Page 36

NVAO | September 2009

not applicable in this issue

not applicable in this issue

Themes, standards, criteria
Standard:
1.2. Level and orientation: bachelor’s programmes, academic
orientation
Criteria:
The aims and objectives of the programme are focused on having
the students master:
 general competences such as the capacity for logical
thought and reasoning, the ability to acquire and process
information, the capacity for critical reflection, creativity,
being able to perform simple management tasks, the ability
to communicate information, ideas, problems and solutions
both to specialists and laymen and a positive attitude
towards life-long learning;
 general academic competences such as a research attitude,
knowledge of research methodologies and techniques and
the ability to apply them adequately, the ability to collect
relevant data in order to form an opinion about social,
academic, scientific and ethical issues, an appreciation of
the uncertainty, ambiguity and limits of knowledge, and the
ability to initiate problem-driven research;
 an understanding of basic academic, discipline-related
knowledge inherent to a certain domain of the sciences or
the arts, systematic understanding of the key elements of a
discipline which includes acquiring coherent and detailed
knowledge that is inspired partly by the most recent
developments in the discipline, and an understanding of the
structure of the discipline and its inter-relatedness with other
disciplines.

Page 37

NVAO | September 2009

Specific assessment of designated
elements until 2008

From 2004/05 or 2005/06:
With regard to academic/scientific support and
intertwining of education and research, the
long-term aims and objectives of the
programme relating to the intended
competences of graduates correspond to
general aims and objectives of bachelor’s
programmes with an academic orientation
outlined in the draft accreditation framework
for existing programmes, including with
respect to:
expertise and application of research
methodologies and techniques
research attitude
formation of opinions
skills related to the academic/scientific
discipline
The short-term aims and objectives must
evolve in a progressive manner in order to
make a reasonable case that the long-term
aims and objectives will be achieved by 2013.

Specific assessment of designated
elements after 2008

From 2008/09
As in the preceding column

Specific information

Description of short-term and longterm aims and objectives of the
programme

Themes, standards, criteria
Standard:
1.3. Level and orientation, master’s programmes
Criteria:
The aims and objectives of the programme are focused on having
the students master:
 general competences at an advanced level such as the ability
to reason and act in an academic manner, the ability to handle
complex problems, the ability to reflect on one’s own thoughts
and work, and the ability to convert this reflection into the
development of more effective solutions, the ability to
communicate one’s own research and solutions to peers and
laymen, and the ability to develop an opinion in an uncertain
context;
 general academic competences at an advanced level such as
the ability to apply research methods and techniques, the
ability to design research, the ability to apply paradigms in the
domain of the sciences or the arts and the ability to indicate the
limits of paradigms, the ability to be original and creative with a
view to continuously expanding knowledge and insight, and the
ability to collaborate in a multi-disciplinary environment;
 advanced understanding of and insight into scientific,
discipline-specific knowledge inherent to a certain domain of
the sciences or the arts, insight into the most recent knowledge
in the subject/discipline or parts thereof, the ability to follow
and interpret the direction in which theory formation is
developing, the ability to make an original contribution towards
the body of knowledge of one or several parts of the
subject/discipline, and display specific competences
characteristic to the subject/discipline such as designing,
researching, analysing and diagnosing;
 the competences needed for either independent research or
the independent practice of the arts at the level of a newlyqualified researcher or artist, or the general and specific
professional competences needed for independent application
of academic or artistic knowledge at the level of a newlyqualified professional.

Page 38

NVAO | September 2009

Specific assessment of designated
elements until 2008

From 2004/05 or 2005/06:
The long-term aims and objectives of the
programme relating to the intended
competences of graduates have at least been
outlined and correspond to the general aims
and objectives for master’s programmes in the
draft accreditation framework for existing
programmes, including with respect to:
use of research methodologies and
techniques
design of research
use of paradigms
formation of opinions
skills related to the academic/scientific
discipline
The short-term aims and objectives must
evolve in a progressive manner in order to
make a reasonable case that the long-term
aims and objectives will be achieved by 2013.

Specific assessment of designated
elements after 2008

From 2007/08 or 2008/09:
The long-term aims and objectives of the
programme relating to the intended
competences of graduates have been
elaborated in full and correspond to the
general aims and objectives for master’s
programmes in the draft accreditation
framework for existing programmes.

The short-term aims and objectives must
evolve in a progressive manner in order to
make a reasonable case that the long-term
aims and objectives will be achieved by 2013.

Specific information

Description of short-term and longterm aims and objectives of the
programme

Themes, standards, criteria

Specific assessment of designated
elements until 2008

Specific assessment of designated
elements after 2008

Specific information

Standard:
1.4. Subject-/discipline-specific requirements
Criteria:
 The aims and objectives of the programme (expressed in the
learning outcomes of the students) correspond with the
requirements set by professional colleagues, both nationally
and internationally, and the relevant professional field for a
programme in the domain concerned (subject/discipline and/or
professional practice or practice of the arts). In the case of
regulated professions, the requirements correspond with the
regulation or legislation concerned.
 The learning outcomes of bachelor and master’s programmes
with an academic orientation stem from requirements set by
the academic and/or artistic discipline, international academic
practice and, for programmes to which this applies, the
practice in the relevant professional field.

From 2004/05 or 2005/06 (bachelor’s
programmes, academic orientation) and
2007/08 or 2008/09 (master’s programmes)
respectively:
The long-term aims and objectives of the
programme relating to the embedment in
research correspond to the requirements set
by national and international peers.
The short-term aims and objectives must
evolve in a progressive manner in order to
make a reasonable case that the long-term
aims and objectives will be achieved by 2013
at the latest.

Page 39

NVAO | September 2009

As in the preceding column

The short-term aims and objectives must
evolve in a progressive manner in order to
make a reasonable case that the long-term
aims and objectives will be achieved by 2013
at the latest.

Description of short-term and longterm aims and objectives of the
programme

Themes, standards, criteria

Specific assessment of designated
elements until 2008

Specific assessment of designated
elements after 2008

Specific information

Theme: 2. Curriculum
Standard:
2.1. Requirements for professional and academic orientation
Criteria:
Bachelor’s programmes with academic orientation and master’s
programmes:
 Students develop their knowledge through the interaction
between education and research (including research in the
arts) within relevant disciplines;
 The curriculum corresponds with current developments in the
relevant academic or scientific discipline(s) through verifiable
links with current academic or scientific theories;
 The curriculum ensures the development of competences in
the field of research and/or the development and practice of
the arts;
 Where appropriate, the curriculum has verifiable links with the
current relevant professional practice.

From 2004/05 or 2005/06 (bachelor’s
programmes, academic orientation) and no
later than 2007/08 or 2008/09 (master’s
programmes) respectively:
The long-term description of the curriculum
shows sufficient:
a. interaction with research (including
submitting research themes, feedback
regarding research issues and results in
education, involving students in
research)
b. connection with current
academic/scientific theories from the
disciplines involved
c. development of skills relating to
research

See up to 2008

Short-term and long-term description
of the curriculum

In addition:
From 2008/09 or 2009/10 (master’s
programmes):
The curriculum described is increasingly being
realised – the degree of realisation of the
curriculum makes it likely that by 2013 the
curriculum will fully comply with the
requirements laid down in the accreditation
framework with respect to the intertwining with
research.
The curriculum must evolve in a progressive
manner in order to fulfil the requirements by
2013.

The curriculum must evolve in a progressive
manner in order to fulfil the requirements by
2013.
Standard:
2.2. Correspondence between the aims / objectives and the
curriculum
Criteria:
 The curriculum is an adequate realisation of the intended
learning outcomes of the programme with regard to the level,
orientation and subject-/discipline-specific requirements.
 The intended learning outcomes are properly reflected in the
educational goals of the curriculum or parts thereof.
 The contents of the curriculum enable students to achieve the
intended learning outcomes.

Page 40

NVAO | September 2009

From 2004/05 or 2005/06 (bachelor’s
programmes, academic orientation) (and no
later than from 2007/08 or 2008/09 (master’s
programmes, academic orientation)):
the long-term curriculum described is
increasingly being realised;
the short-term realisation of the curriculum
makes it likely that by 2013 the curriculum will
fully comply with the requirements laid down
in the accreditation framework with respect to
the intertwining with research.

As above

Short-term and long-term description
of the curriculum

Themes, standards, criteria
Standard:
2.3. Consistency of the curriculum
Criteria:
 Students follow a curriculum that is coherent in terms of
content.
Standard:
2.4. Workload
Criteria:
 The actual duration of the programme is assessed and
corresponds with the statutory standards.
 The intended learning outcomes are attainable because factors
relating to the curriculum that could hamper study progress are
eliminated wherever possible.
Standard:
2.5. Admission requirements
Criteria:
The structure and contents of the curriculum are in line with the
qualifications of the incoming students:
Bachelor’s programmes:
 Secondary school leaving certificate, Diploma van het Hoger
Onderwijs van het korte type met volledig leerplan, Diploma
van het Hoger Onderwijs voor Sociale Promotie or a diploma
or certificate that is recognised as equivalent in accordance
with a law, decree, European directive or other international
agreement ;
 Predefined conditions set by the institution for individuals who
do not meet the above-mentioned requirements.
Master’s programmes:
 A bachelor’s degree, with a qualification or qualifications
specified in more detail by the management of the institution,
possibly supplemented with an individualised curriculum, a
preparatory programme or a bridging programme.
Advanced master’s programme:
 A master’s degree, with a qualification or qualifications
specified in more detail by the management of the institution,
possibly supplemented with either an assessment concerning
the fitness or capacity of the individual, or a preparatory
programme.

Page 41

NVAO | September 2009

Specific assessment of designated
elements until 2008

Specific assessment of designated
elements after 2008

Specific information

Themes, standards, criteria

Specific assessment of designated
elements until 2008

Specific assessment of designated
elements after 2008

Specific information

The long-term draft of the master’s thesis is in
line with the statutory provisions, including
with respect to the substantial
academic/scientific foundation.

(Depending on the first implementation of the
master’s programmes, from 2008/09 or
2009/10):

Short-term and long-term description
of the curriculum

Standard:
2.6. Credits
Criteria:
The curriculum meets the legal requirements regarding the
association of credits:
 Bachelor’s degree: at least 180 credits
 Master’s degree: at least 60 credits
 Advanced master’s degree: at least 60 credits
Standard:
2.7. Coherence of structure and contents
Criteria:
 The educational concept is in line with the aims and objectives.
 The study methods correspond with this educational concept.
Standard:
2.8. Learning assessment
Criteria:
 By means of evaluations, tests and examinations, the students
are assessed in an adequate manner which is insightful to
them to determine whether they have achieved the intended
learning outcomes of the programme or parts thereof.
Standard:
2.9. Master’s thesis:
Criteria:
 The master’s programme is concluded with the master’s thesis
whereby the student demonstrates the ability for analytic and
synthetic reasoning, independent problem solving at an
academic level or artistic creation. The work reflects the
general critical-reflective attitude or the research attitude of the
student.
 The master’s thesis corresponds to at least a fifth of the total
number of credits with a minimum of 15 and a maximum of 30
credits.

The short-term draft must evolve in a
progressive manner in order to make a
reasonable case that the requirements will be
fulfilled by 2013 at the latest.

The long-term draft of the master’s thesis is in
line with the statutory provisions, including
with respect to the substantial
academic/scientific foundation.
The short-term draft and its realisation must
evolve in a progressive manner in order to
fulfil the requirements by 2013 at the latest.

Page 42

NVAO | September 2009

Themes, standards, criteria

Specific assessment of designated
elements until 2008

Specific assessment of designated
elements after 2008

Specific information

It is likely that by 2013 a significant proportion
of the teaching will be provided by staff that
are actively engaged in research and thus
contribute to the development of their
discipline, based on the actual achievement
and the intentions with respect to:
a. staff policy and recruitment policy
b. the research tasks and the qualifications
of the workforce (including the extent to
which a substantial research task has
been incorporated into vacancies for new
staff, increase in number of PhD
students, increase in number of PhDs,
increase in research staff vis-à-vis overall
staff)
c. deployment of active researchers, if any,
from partners within and outside the
Association in education
d. the scope of the research in which
programme staff participates

As in the preceding column.
The initial achievement is already manifest.

Staff policy:
recruitment policy
measures for boosting the research
qualifications of staff
Description of workforce
research workload of staff (number of FTEs and
ratio vis-à-vis total workload), with breakdown
by new staff members and doctoral mandates
multi-year plans
Description of research policy (structures,
organisation, strategy)

Theme: 3. Staff commitment
Standard:
3.1. Requirements for professional / academic
orientation
Criteria:
The programme meets the following criteria for
the deployment of staff for a programme with a
professional or an academic orientation.
 Teaching is principally provided by researchers
who contribute to the development of the
subject/discipline (including research in the
arts).
 In addition, and where appropriate, sufficient
staff will be deployed with knowledge of and
insight in the professional field or practice of
the arts concerned.

Research projects in which programme staff
participate (including doctoral mandates)
Qualifications of staff (in FTEs):
doctorate
research experience
current research activities
Deployment of academic staff from partners:
within Association
outside Association

Standard:
3.2. Quantity of staff
Criteria:
 Sufficient staff are deployed to realise the
intended quality of the programme.
Standard:
3.3. Quality of staff
Criteria:
 The staff deployed are sufficiently qualified to
ensure that the aims and objectives of the
programme, in terms of content, didactics and
organisation, are achieved.

Page 43

NVAO | September 2009

Themes, standards, criteria

Specific assessment of designated
elements until 2008

Specific assessment of designated
elements after 2008

Specific information

Theme: 4. Services
Standard:
4.1. Facilities
Criteria:
 Housing and facilities are sufficient for the realisation of the
curriculum.

Standard:
4.2. Tutoring
Criteria:
 Tutoring and the provision of information to students are
adequate in view of study progress.
 Tutoring and information provision are geared to students’
needs.

Page 44

NVAO | September 2009

Based on the actual achievement and the
intentions, it is likely that the quality and
quantity of the research facilities
(infrastructure, equipment, academic/scientific
databases, etc.) that are essential to
guarantee the embedment of education in
research will be sufficient by 2013 at the
latest.

As in the preceding column

Research facilities (achievement and
planning in institution and through
collaboration within and outside
Association), related to embedding
education in research
Use of funds earmarked for the
embedment of programmes in
research and other financial
resources for research and
embedding education in research

Themes, standards, criteria

Theme: 5. Internal quality assurance
Standard:
5.1. Evaluation of results
Criteria:
 The curriculum is periodically evaluated in the light of verifiable
objectives and other measures.
Standard:
5.2. Measures for improvement
Criteria:
 The outcomes of the evaluation form the basis for verifiable
measures for improvement that contribute to the achievement
of the objectives.
Standard:
5.3. Involvement of staff, students, alumni and the
professional field
Criteria:
 Staff, students, alumni and the relevant professional field are
actively involved in the internal quality assurance system.

Page 45

NVAO | September 2009

Specific assessment of designated
elements until 2008

Specific assessment of designated
elements after 2008

Specific information

Themes, standards, criteria

Theme: 6. Results
Standard:
6.1. Achieved learning outcomes
Criteria:
 The achieved learning outcomes correspond with the aims and
objectives regarding level, orientation and subject-/disciplinespecific requirements.
Standard:
6.2. Success rate
Criteria:
 Target figures regarding success rate have been formulated on
the basis of a comparison with relevant other programmes.
 The programme’s success rate is in line with these target
figures.

Page 46

NVAO | September 2009

Specific assessment of designated
elements until 2008

Specific assessment of designated
elements after 2008

Specific information

Assessment framework
for the initial accreditation of
higher education programmes
in Flanders
1 September 2009

Parkstraat 28 | Postbus 556 | 2501 CN Den Haag
P.O. Box 556 | 2501 CN The Hague | The Netherlands
T +31 (0)70 312 2300 | F +31 (0)70 312 2301
info@nvao.net | www.nvao.net

Contents

Page 2

1

Structure of the assessment framework

3

2

Evaluation framework
2.1
Aims and objectives of the programme
2.2
Curriculum
2.3
Staff commitment
2.4
Facilities
2.5
Internal quality assurance
2.6
Conditions for continuity

4
4
5
7
7
8
8

3

Assessment rules

9

4

Initial accreditation procedure

10

Explanatory notes
General

11
11

Explanatory notes, chapter by chapter

17

NVAO | September 2009

1

Structure of the assessment framework
The assessment framework for the initial accreditation of higher education programmes
comprises:
– a framework for the evaluation of themes, standards and criteria
– assessment rules
– a description of the initial accreditation procedure, including the criteria for the evaluation
of the assessment and the initial accreditation report.
The decision regarding the accreditation of new programmes is founded on an assessment
1
based on six themes.
These themes are:
– aims and objectives of the programme
– curriculum
– staff commitment
– services and facilities
– internal quality assurance
– conditions for continuity.
The above themes are reviewed on the basis of standards and the corresponding criteria
(see chapter 2).
Assessment rules have been laid down for initial accreditations (see chapter 3).
The Accreditation Organisation bases its judgement regarding the new programme on an
assessment carried out or commissioned by the accreditation organisation. This
assessment will result in an initial accreditation report. Criteria have been drawn up
regarding the quality of the initial accreditation report (see chapter 4 on the Initial
2
accreditation procedure).
The method of working and procedure to be followed in initial accreditations have been laid
down by the accreditation organisation. The organisation has opted for a differentiated
approach. Although all new programmes have to meet the same criteria, the nature of the
assessment to be carried out may vary depending on the degree of newness of the
programme in the institution involved and in the Flemish higher education sector, and
3
depending on the extent to which the programme is already offered.

1
Article 58 of the Flemish Higher Education Act uses the concept of
“generic quality standards” in this respect. The explanatory notes to this
assessment framework clarify the relationship between the generic
quality standards from the Higher Education Act and the themes and
standards in the assessment framework.
2
The assessment framework briefly touches upon the basics of several
relevant provisions in the Higher Education Act. Please refer to the
Higher Education Act for the full wording of these provisions.
3
The procedure for processing applications is outlined in the Guidelines
for Initial Accreditations in the Flemish Community, drawn up by NVAO.

Page 3

NVAO | September 2009

2

Evaluation framework

2.1

Aims and objectives of the programme
Standards
Level and
orientation:
bachelor’s,
professional higher
education

Criteria
The programme aims and objectives are aimed at leading the students to:

master general competencies such as thinking and reasoning skills,
retrieving and processing information, critical reflection and the ability to
work in a project context, creativity, the ability to perform simple
management duties, the ability to communicate information, ideas,
problems and solutions both to specialists and laymen and an attitude
conducive to lifelong learning;


master general professional competencies such as the ability to work in a
team, the ability to work in a solution-oriented fashion in the sense of
autonomously defining and analysing complex problem situations in
professional practice and being able to develop and apply meaningful
solution strategies and an awareness of the social responsibility that
ensues from professional practice;



master profession-specific competencies at the level of a newly qualified
professional.

The programme aims and objectives are aimed at leading the students to:
Level and

master general competencies such as thinking and reasoning skills,
orientation:
retrieving and processing information, critical reflection, creativity, the
bachelor’s,
ability to perform simple management duties, the ability to communicate
academic education
information, ideas, problems and solutions to both specialists and laymen
and an attitude conducive to lifelong learning;

Page 4

NVAO | September 2009



master general academic competencies such as an investigative attitude,
knowledge of and the ability to use research methods and techniques in an
adequate fashion, the ability to collect the data relevant to the formation of
a judgement on social, academic and ethical issues, an appreciation of the
uncertainty, ambiguity and boundaries of knowledge and the skills required
for a problem-based initiation of research;



gain an understanding of the academic basic knowledge specific to a
particular domain of science or the arts, a systematic knowledge of the
core elements of a discipline, including the acquisition of coherent and
detailed knowledge in part inspired by the latest developments in the
discipline and an understanding of the structure of the field of study and its
relationship with other fields of study.

Level and
orientation: master’s

Domain-specific
requirements

The programme aims and objectives are aimed at leading the students to:

master general competencies at an advanced level, such as the ability to
think and act in an academic fashion, the ability to deal with complex
problems, the ability to reflect on their own thinking and working and
being able to translate that reflection into the development of more
adequate solutions, the ability to communicate their own research and
problem solutions to peers and laymen and the ability to form a
judgement in an uncertain context;


master general academic competencies at an advanced level, such as
the ability to use research methods and techniques, the ability to design
research, the ability to use paradigms in the domain of science or the arts
and being able to identify the boundaries of paradigms, the ability to
demonstrate originality and creativity with a view to the continuous
expansion of knowledge and insights and the ability to work in concert in
a multi-disciplinary environment;



an advanced understanding of and insight into the academic knowledge
specific to a particular domain of science or the arts, insight into the latest
knowledge of the field of study or components thereof, the ability to follow
and interpret developments in the formation of theories, the ability to
make an original contribution to the knowledge relevant to one or more
components of the field of study and the possession of specific skills
related to the field of study, such as designing, researching, analysing
and diagnosing;



either master the competencies required to independently conduct
research or independently practise art at the level of a newly-qualified
researcher or artist; or master the general and specific professional
competencies required to independently apply academic or artistic
knowledge at the level of a newly-qualified professional.

The aims and objectives of the programme (expressed in learning outcomes)
correspond to the requirements set for a programme in the relevant domain
(field of study / discipline and/or professional practice or art practice) by
(foreign) peers and the relevant professional field. For regulated professions,
such aims and objectives are in keeping with the relevant rules and
regulations.
The learning outcomes of bachelor’s programmes with a professional
orientation have been reviewed by the relevant professional field.
For bachelor’s programmes and master’s programmes with an academic
orientation, the learning outcomes have been derived from the requirements
set by the academic and/or artistic discipline, the international world of
science and, when appropriate, actual practice in the relevant professional
field.

2.2

Curriculum
Standards
Requirements
regarding

Page 5

NVAO | September 2009

Criteria
The curriculum is geared to the following criteria regarding a professional or
academic orientation:

professional and
academic orientation

Bachelor’s programmes (professional orientation):

Students develop knowledge through specialised literature, learning
materials derived from the professional or arts practice and interaction
with professional practice, arts practice and/or (applied) research;


The curriculum has demonstrable links with current developments in the
field of study / discipline;



The curriculum safeguards the development of professional or artistic
skills and has demonstrable links with current professional practice;

Bachelor’s programmes and master’s programmes (academic orientation):

Students develop knowledge through the interaction between education
and research (including research in the arts) within relevant disciplines;


The curriculum ties in with developments in the relevant academic
discipline(s) by demonstrable links with current academic theories;



The curriculum safeguards the development of skills in the field of
research and/or the development and practice of the arts;



When appropriate, the curriculum has demonstrable links with the current
practice of relevant professions.

Correspondence
between aims and
objectives and
curriculum

The curriculum, the teaching concept, the teaching methods and the test
formats reflect the learning outcomes to be attained by the students.

Cohesion in
curriculum
Duration

The contents of the curriculum are coherent.

Admission
requirements

The form and content of the curriculum are geared to the qualifications of
4
incoming students.

The learning outcomes to be attained have demonstrably been translated into
learning aims and objectives for (components of) the curriculum.

The course duration is in keeping with the statutory standards.

Bachelor’s programmes:

secondary education qualifications, certificate of short higher education
courses with full curriculum, certificate of higher education programmes
aimed at social promotion or a diploma or certificate recognised as
equivalent under a law, decree, European directive or other international
agreement;

conditions determined by the board of the institution for applicants that do
not meet the above conditions.
Advanced bachelor’s programmes:

bachelor’s degree, with (a) particular qualification(s) determined by the
board of the institution, possibly supplemented by a study into the
candidate’s aptitude or competence or a preparatory curriculum.

4
The admission requirements outlined are those laid down
in the Higher Education Act. With effect from 2005-2006,
the requirements for admission to higher education have
been regulated by the Flexibilisation Act. In terms of
content, these admission requirements are identical to the
ones listed here.

Page 6

NVAO | September 2009

Master’s programmes:

bachelor’s degree, with (a) particular qualification(s) determined by the
board of the institution and, when appropriate, supplemented by an
individualised curriculum, a preparatory curriculum or bridging curriculum.
Advanced master’s programmes:

master’s degree, with (a) particular qualification(s) determined by the
board of the institution, possibly supplemented by a study into the
candidate’s aptitude or competence or a preparatory curriculum.

2.3

Number of credits

The programme meets formal requirements regarding the number of credits:
 Bachelor’s programmes: a minimum of 180 credits
 Advanced bachelor’s programmes: a minimum of 60 credits
 Master’s programmes: a minimum of 60 credits
 Advanced master’s programmes: a minimum of 60 credits

Master’s thesis

Master’s programmes are rounded off with a master’s thesis. This carries at
least one fifth of the total number of credits, with a minimum of 15 and a
maximum of 30 credits.

Staff commitment
Standards
Requirements
regarding
professional /
academic orientation

Criteria
The programme is in keeping with the following criteria for the staff
commitment for programmes with a professional or academic orientation:
Programmes with a professional orientation:

A major part of the curriculum will be taught by staff who establish a link
between the programme and professional or arts practice.
Programmes with an academic orientation:

A major part of the curriculum will be taught by staff who contribute to the
development of the field of study (including research in the arts).


Quantity of staff

In addition, when appropriate, the programme will have sufficient staff
with knowledge of and insight into relevant professional or arts practice.

The capacity made available is sufficient to launch the new programme.
The capacity made available is sufficient to continue the new programme.

Quality of staff

2.4

Page 7

The staff that will be asked to implement the programme are qualified for the
substantive, educational and organisational realisation of the curriculum.

Facilities
Standards
Material facilities

Criteria
The proposed accommodation and material facilities are sufficient to realise
the curriculum.

Tutoring

The staff capacity arranged for tutoring and information provision to students
is sufficient to safeguard their progress.

NVAO | September 2009

2.5

Internal quality assurance
Standards
Criteria
Systematic approach An internal quality assurance system is in place, involving measures for
improvement taken on the basis of testable targets and periodical evaluations.
Staff, students, alumni and the relevant professional field will be actively
Involving staff,
students, alumni and involved in the programme’s internal quality assurance.
the professional field

2.6

Page 8

Conditions for continuity
Standards
Graduation
guarantee

Criteria
The institution provides its students with the guarantee that they can complete
the curriculum in its entirety.

Investments

The investments envisaged are sufficient to establish the programme
(including the facilities).

Financial facilities

The financial facilities are sufficient to be able to offer the full course of the
programme.

NVAO | September 2009

3

Assessment rules
The proposal for a new programme is assessed by the accreditation organisation. This
assessment determines whether the new programme provides potentially sufficient generic
quality standards. Each of the standards is evaluated with either a satisfactory or an
unsatisfactory result. For a positive assessment, each theme in the assessment framework
must score a satisfactory. The individual theme scores are determined by weighing the
conclusions regarding the various standards in the theme under consideration.
Transparency is provided with respect to the manner in which the assessment of the various
standards has resulted in the summarising conclusion on a theme, in other words, how –
given the criteria in this assessment framework – the organisation arrived at its conclusion
for each theme on the basis of the analysis of each standard.
In its final conclusion regarding the new programme, the accreditation organisation will
indicate how this conclusion is based on the facts, the analysis of these facts and the
assessment of the programme based on this assessment framework.
The essence of these assessment rules has been laid down in the regulations pertaining to
“the determination of administrative principles that bear on the decision-making regarding
the accreditation and initial accreditation of programmes provided in the Flemish
Community”. Articles 7 and 8 of these regulations read:
“Art. 7. An accreditation decision or initial accreditation report, as the case may be, is
positive if all summarising conclusions on the themes listed in the Accreditation Framework
or the Assessment Framework, as the case may be, are satisfactory. If, at the discretion of
NVAO, standards judged as “unsatisfactory” are compensated by strengths within other
standards under that same theme, a theme will be judged as “satisfactory”.
Art. 8. In the event of programmes comprising various specialisations and/or locations, a
theme can only score a “satisfactory” if each of the various specialisations and/or locations
is judged as “satisfactory”.

Page 9

NVAO | September 2009

4

Initial accreditation procedure
1. The points of departure for the assessment are the Flemish Higher Education Act, the
above regulations and the themes, standards, criteria and assessment rules laid down in
this assessment framework.
2. For each new programme, the institution draws up documents that portray the proposed
new programme and demonstrate that the programme meets the statutory requirements.
In addition, the institution will provide the following information:
– a description of the programme on the basis of the themes and standards of the
assessment framework;
– a financial overview stating the expenditure relating to the realisation of the
programme;
– a description of the required staff, stating numbers and qualifications.
3. The institution submits an application for initial accreditation to the accreditation
organisation, enclosing the information listed under 2. The format and content of the
application dossier must conform to the regulations laid down by the accreditation
5
organisation. For programmes provided by statutory registered institutions , this
application can only be submitted after a positive judgement by the Recognition
Committee regarding the macro efficiency of the programme or a positive judgement to
that effect by the Flemish government; a judgement of the Flemish government is
deemed positive if not communicated within thirty calendar days. The application must be
submitted within a term of fifteen days after receipt of the judgement referred to above or
after expiry of the period of time within which the Flemish government is to pass its
judgement.
The application must state the name of the institution and, if appropriate, the association it
belongs to. In addition, the application must indicate:
– the name of the programme;
– the field of study or the (components of) fields of study;
– whether the programme already exists within the Flemish higher education system or
– whether this is a programme that is new to the higher education system in Flanders;
– whether the proposed programme trains students for a bachelor’s degree with a
professional orientation, a bachelor’s degree with an academic orientation, a master’s
degree, an advanced bachelor’s degree or an advanced master’s degree;
– which specialisations the new programme comprises;
– whether the programme is to be provided at one or more locations.
4. On the basis of the data listed under 3 and the initial accreditation procedure it has laid
down, the accreditation organisation decides the scope of the assessment, the procedure
to be followed and the expertise to be commanded by the designated external experts.
5. The actual assessment is conducted by an external panel of experts commissioned by
the accreditation organisation. They will lay down their findings in an advisory report, in
accordance with the regulations pertaining to the initial accreditation methodology.

5
Article 7 of the Flemish Higher Education Act defines
statutory registered institutions as: the university colleges,
the universities, the institutions providing post-graduate
programmes and the recognised faculties for Protestant
theology.

Page 10

NVAO | September 2009

6. In their assessment, the panel experts determine whether the plans and documents meet
the criteria laid down in this assessment framework. The assessment results in a
summarising conclusion regarding the new programme, which is motivated in an initial
accreditation report.
7. On the basis of the advisory report, the accreditation organisation draws up its initial
accreditation report, which contains a summarising conclusion. This initial accreditation
report must meet the following criteria:
a. The quality score awarded in the assessment is – insofar as relevant – also based on a
comparison with other, similar programmes and internationally accepted criteria for
programmes in the domain under consideration.
b. The initial accreditation report indicates whether or not the new programme potentially
provides sufficient generic quality standards. The report must discuss at least the six
themes listed in this assessment framework, devoting attention to all the standards
under each theme. For each standard, a satisfactory or unsatisfactory valuation is
awarded, which serves as the basis for the conclusion given for each theme. The
conclusions are underpinned by facts and analyses wherever possible. The report is
finalised with a summarising conclusion regarding the new programme.
c. The initial accreditation report describes the procedure followed in the assessment. It
explicitly explains:
– the methods used;
– the sources of information used;
– the reference framework used in the assessment.
The accreditation organisation draws up its initial accreditation report and summarising
conclusion on the basis of the advisory report, once it has ascertained that the advisory
report meets the three criteria referred to above and that the methods, sources of
information and reference framework used are reliable.
8. The accreditation organisation will make a decision within four months after receipt of the
application. The decision is laid down in an initial accreditation report. Before the
timeframe of four months expires and before finalising the initial accreditation report, the
accreditation organisation allows the board of the institution a term of ten days to
formulate appeals and comments regarding the draft initial accreditation report. If the
accreditation organisation deems that the generic quality standards provided for the new
programme are potentially sufficient, the assessment results in a positive initial
accreditation report.
The accreditation organisation forwards its initial accreditation report to the institution and
the Flemish minister responsible for education.

Explanatory notes
General
New programmes
A new programme is a programme that does not ensue from an accepted transformation of
the existing programmes on offer and that is not registered to the institution involved on the
Higher Education Register. Two types of new programme can be distinguished:
programmes that are entirely new to the Flemish higher education system and that are not

Page 11

NVAO | September 2009

yet listed on the Higher Education Register and programmes that are new to the institution
involved but nonetheless registered on the Higher Education Register.
The Flemish government decides on the recognition of new programmes. To that end, the
programmes must meet the conditions stated in Articles 61 and 62 of the Higher Education
Act, such as passing the initial accreditation by the accreditation organisation and a positive
judgement from the Recognition Committee or the Flemish government with respect to the
macro efficiency of the programme. The initial accreditation determines whether the generic
quality standards are potentially represented to a sufficient degree in the new programme.
A newly recognised programme is deemed to have been accredited up to and including the
end of the second academic year in which the first cohort of students completed the full
study load laid down for the new programme.
Subject to assessment by the accreditation organisation
One of the tasks of the accreditation organisation is to assess whether applications for new
programmes demonstrate sufficient potential for generic quality standards. This assessment
also answers the question of whether the programme meets the necessary conditions for
launching a new programme and whether the institution is capable of providing the new
programme in a sustainable manner.
A positive initial accreditation decision from the accreditation organisation is one of the
conditions to be met in order to be able to offer a new programme. With a view to the
commensurable legal effects – the authorisation to offer a recognised programme – the
themes, standards and criteria observed in initial accreditations must in essence correspond
to those used in the accreditation of existing programmes. At the standards and criteria
level, however, the accreditation of existing programmes differs from initial accreditations.
Unlike the assessment of existing programmes, the assessment of new programmes
involves an assessment based on a plan. The question is whether the proposal is
underpinned by clear aims and objectives that tie in with the various requirements set for
bachelor’s programmes with a professional orientation, those for bachelor’s and master’s
programmes with an academic orientation and the subject-specific requirements. In
addition, the plan must comprise a description of the curriculum and the facilities that
justifies the expectation that the aims and objectives will be achieved. In order to be able to
assess these themes, high requirements are set for the quality of the information to be
provided by the institutions (burden of proof principle). The accreditation organisation calls
in external experts for the assessment.
A key point of departure in the assessment of new programmes is that the scope of the
external assessment may depend on the extent to which a proposed new programme differs
from already existing programmes, on the degree of newness of the programme to the
institution or to Flanders, and on the elements that are already in place (already existing
programme components, staff, infrastructure, financial resources) for the realisation of the
new programme. Applications that differ substantially from existing programmes in terms of
content will be subjected to a more extensive assessment than applications for programmes
that are already offered elsewhere in the Flemish higher education system. Thus, the
assessments may differ as regards depth of investigation. For each application, the actual
set-up of the assessment procedure is determined on the basis of the procedure regulations
laid down.
The assessment pertains to the expected quality of the programme.
If a new programme involves various specialisations or locations, the assessment must
demonstrate that the generic quality is safeguarded for the entire programme with all its
specialisations and at all its locations.

Page 12

NVAO | September 2009

Transparency
The accreditation organisation regards it as its duty to contribute to the transparency of the
programmes on offer. This means that it will ascertain whether each proposed new
programme is sailing under true colours, i.e., whether the aims and objectives and the
curriculum are in keeping with the proposed nomenclature. For proposed new programmes
whose nomenclature is already listed on the Higher Education Register, it will ascertain
whether the aims and objectives and the curriculum sufficiently correspond to existing
programmes registered on the Higher Education Register under that same nomenclature, in
order to justify the nomenclature under consideration for that programme. With regard to
new programmes that are not yet offered elsewhere in Flanders under that nomenclature
and must therefore be regarded as truly new in terms of content, the institution will have to
satisfy the accreditation organisation that the proposed programme differs substantially from
existing programmes and consequently must not be regarded as a variation of such
programmes.
Assessment framework
This assessment framework is based on the Flemish Higher Education Act of 4 April 2003
and the Assessment Framework for New Programmes established by the Dutch
Accreditation Organisation (NAO) on 14 February 2003. We aimed for maximum
harmonisation between the Dutch and the Flemish frameworks. We deviated from the Dutch
framework whenever the Flemish Higher Education Act or differences between the Flemish
and Dutch higher education systems so necessitated. For various aspects, these
explanatory notes explicitly highlight the connection with the regulations governing higher
education in Flanders and explain the themes, standards and criteria within the context of
the Flemish situation.
The assessment of new programmes is based on evaluation of the following themes:
– aims and objectives of the programme;
– curriculum;
– staff commitment;
– facilities;
– internal quality assurance;
– conditions for continuity.
These themes have been subdivided into standards. The table below indicates how the
themes and standards of the assessment framework tie in with the generic quality standards
from the Flemish Higher Education Act (Article 58).
Generic quality standards,
Higher Education Act

Page 13

NVAO | September 2009

Assessment framework for new programmes in
Flanders

Study contents:
– nature and educational level
– cohesion in curriculum
– workload
– relationship between aims and
objectives and content

Aims and objectives:
– level and orientation, bachelor’s degree
professional higher education
– level and orientation, bachelor’s degree
academic education
– level and orientation master’s degree
– domain-specific requirements
Curriculum:
– requirements for professional and academic
orientations
– relationship between aims and objectives and
content
– cohesion in curriculum
– workload
– admission requirements
– number of credits

Teaching and learning process:
– harmonisation of format and content
– tutoring
transparent assessment and testing

Curriculum:
– relationship between aims and objectives and
curriculum
– master’s thesis
Services and facilities:
– tutoring

Educational outcomes:
– social relevance
learning outcomes
– success rates

(related to macro efficiency check)

Material facilities, quality of staff,
organisation and internal quality assurance

Staff commitment:
– requirements for professional/academic orientations
– quantity of staff
– quality of staff
Services and facilities:
– material facilities
Conditions for continuity:
– graduation guarantee
– investments
– financial facilities
Internal quality assurance:
– systematic approach
– involvement of staff, students, alumni and
professional field

Self-evaluation methods

n.a.

Several elements of the generic quality standards – transparent assessment and testing,
educational outcomes, self-evaluation methods, harmonisation of format and content – have

Page 14

NVAO | September 2009

not or only partially been included in the criteria for this assessment framework because
these elements do not bear an ex ante assessment or are fairly irrelevant for the
assessment of plans. The additional theme of conditions for continuity has been added in
order to ensure that the new programme can actually be offered.
As far as possible, the structure of the assessment framework, the choice of themes,
standards and criteria and the level of abstraction observed are in keeping with the Dutch
assessment framework.
The point of departure is that the institution, within its authorisation to provide education,
submits an application for a new programme as a bachelor’s programme with a professional
orientation, a bachelor’s programme with an academic orientation, a master’s programme,
an advanced bachelor’s programme or an advanced master’s programme. For that reason,
the programme must be assessed as such.
An assessment of a new programme is aimed at reviewing, on the basis of the themes,
standards and criteria of the assessment framework, whether the accreditation organisation
is justified in concluding that the programme has sufficient generic quality standards in
place. To that end, it may review the plans and finalised curricula but also existing elements
that will be used in the new programme (staff, material resources, courses or components of
existing programmes that will be incorporated into the new programme…). This does not
mean, however, that the quality of achievements or results ensuing from these existing
elements will be judged by reference to the existing programme(s), but rather that they will
be taken into consideration from the point of view of quality guarantees for the new
programme. The presence of these quality guarantees is always assessed on the basis of
the assessment framework for new programmes, not the assessment framework for existing
programmes, even if certain components of the proposed new programme already exist.
To some extent, the assessment of new programmes imposes different requirements on the
expertise of those conducting the assessment than an assessment of existing programmes.
Particular differences relate to the ex ante assessment of the quality of the programme, the
quality of the staff to be committed and the business aspects.

Domain-specific requirements

6

A programme assessment cannot be based solely on general criteria pertaining to aims and
objectives, curriculum, services and facilities, staff commitment and internal quality
assurance. In its description, the institution must take account of the requirements set for
this specific programme by (foreign) peers and professional or art practice. With regard to
regulated professions, the relevant rules and regulations must also be taken into account.
Embedding academic programmes in research
Essential characteristics of programmes with an academic orientation are the fact that these
programmes are inseparable from research and the fact that teaching and research are
intertwined. Teaching in programmes with an academic orientation must be founded on
research. Various paragraphs of the assessment framework comprise requirements relating
to the inseparability of research and teaching. These requirements pertain to:
– the aims and objectives of the programmes (in accordance with the stipulations contained
in Article 58 of the Higher Education Act);

6

The domain-specific requirements operationalise the
generic quality standards. They are subordinate to these
standards in terms of hierarchy.

Page 15

NVAO | September 2009

– the curriculum (requirements regarding academic orientation, such as interaction between
education and research, keeping abreast of academic developments and current
academic theories, guaranteeing skills in the field of research and the master’s thesis);
– staff commitment (teaching is principally provided by researchers who contribute to the
development of their field of study).
These requirements pertain to all programmes with an academic orientation. Their
assessment must also be based on the research activities of the staff (such as research
projects, doctoral research and academic output). The actual interpretation of the
requirements will vary according to the nature of the programme (bachelor’s, master’s or
advanced master’s) and their position in the course of the programme. As a rule, master’s
programmes will tend to be more firmly embedded in research than bachelor’s programmes
and the introduction of research skills will feature primarily in master’s programmes, most
prominently in the master’s thesis. Depending on the domain-specific nature of the
programme, the requirements to be met by the programmes will be specified in more detail
(cf. the explanatory note to Chapter 2, the curriculum).
Scope
A positive result in the initial accreditation is one of the conditions for new programmes,
whether provided by statutory registered institutions or by non-statutory registered
institutions, to be recognised and included on the Higher Education Register. Registration of
the programme on the Higher Education Register is a requirement for the authority to award
recognised bachelor’s and master’s degrees. Under the Higher Education Act, the possibility
is created for other institutions to be registered, offer recognised bachelor’s and master’s
programmes and award the protected bachelor’s and master’s degrees. New programmes
that are offered by such registered institutions must also pass the initial accreditation.
Advanced bachelor´s and advanced master´s programmes
The entrance requirements for advanced bachelor’s and advanced master’s programmes
differ from those that apply for regular bachelor’s and master’s programmes. Otherwise,
these programmes must meet the same quality requirements as other bachelor’s and
master’s programmes.

Page 16

NVAO | September 2009

Explanatory notes, chapter by chapter
Chapter 1: Structure of the assessment framework
The assessment framework comprises substantive criteria and criteria for the procedures to
be followed with regard to the assessment and the initial accreditation report. Unlike the
assessment of existing programmes, the assessment of new programmes does not offer the
additional opportunity to review distinctive features. Our point of departure is that such
features must first have demonstrated their worth in actual practice before they can be
judged.
Chapter 2: Evaluation framework
The accreditation organisation reviews the assessment of the new programme on the basis
of the criteria laid down in the assessment framework. These criteria refer to the statutory
generic quality standards. The object of the assessment is to ascertain whether potentially
sufficient generic quality standards are in place.
– Re 2.1: Aims and objectives of the programme
The aims and objectives of the programme refer to the statutory competencies students
must acquire in the intended programmes.
The aims and objectives of the programme must be geared to the domain-specific quality
requirements and general criteria regarding the level and orientation of bachelor’s
programmes with a professional orientation and bachelor’s / master’s programmes with an
academic orientation. In addition, the aims and objectives of the programme must reflect the
intention to keep abreast of recent developments in the discipline and, as the occasion
arises, professional practice.
– Re 2.2: Curriculum
The aims and objectives pursued must be convincingly concretised in the curriculum of the
programme. In addition, the proposed contents and format of that curriculum (including the
attainment targets) must enable qualified entering students to acquire the envisaged
competencies within the stipulated timeframe. A review of the translation of the aims and
objectives into the curriculum requires a substantive review of the study package and the
educational set-up, given the specific choice for bachelor’s and master’s programmes with
either a professional or an academic orientation and the domain in question. Compared to
bachelor’s programmes, a master’s programme will be characterised by a more in-depth
and/or broader approach. Programmes with a professional orientation will tend to focus on
the introduction of knowledge and experience from current professional practice, whereas
embedding in research and developing research skills constitute essential aspects of
programmes with an academic orientation. Some academic programmes, however, have an
additional professional orientation. For that reason, the requirements for academic
bachelor’s and master’s programmes also stipulate that the curricula of eligible programmes
must have demonstrable ties with current practice in relevant professions.
In order to be able to review the programmes in this context, those who conduct the
assessment will usually have to command relevant domain-specific expertise.
Depending on the domain-specific contents of the master’s programmes, higher
requirements will be set for certain standards. For example, for master’s programmes with a
particular focus on researcher training, the domain-specific requirements will need to tie in
with international standards for such programmes; in the review of the aims and objectives
of these programmes, the emphasis will be placed on mastering the competencies required
for independently conducting research at a newly-qualified researcher’s level; and the

Page 17

NVAO | September 2009

embedding of these programmes in research will have to meet more stringent requirements.
This means that higher requirements will be set for, among other things, the research
capacity of the staff and their experience in training newly-qualified researchers, a broad
research orientation in the curriculum, the academic interpretation of the master’s thesis and
the connection with current academic developments.
As applied academic research is also one of the tasks of university colleges, (applied)
research has been included in the options for knowledge development for bachelor’s
programmes with a professional orientation.
A plan for a new programme will need to explain in clear terms how the new programme will
be organised. Attention must be paid to the cohesion between the aims and objectives and
the contents of the curriculum, the cohesion within the curriculum, the workload and the
manner in which the programme is tailored to the enrolling students. The plan must sketch a
realistic picture of the practice of the intended programme. The wording of the assessment
framework, on the basis of which the curriculum is reviewed, leaves sufficient room for
current developments with regard to the standards referred to above.
The quality of the curriculum will also have to be demonstrated by the manner in which the
curriculum is geared to the qualifications of enrolling students. Important considerations in
this respect are the statutory requirements for bachelor’s and master’s programmes, as
outlined in the assessment framework.
Whereas the programme curriculum is reviewed as a model course for the realisation of its
aims and objectives, on the students’ side the introduction of the Flexibilisation of Higher
Education Act in Flanders has created opportunities for individualised training routes,
flexible curricula, learning environments, educational organisation and the recognition of
prior learning. As the occasion arises, these forms of flexibilisation, students’ careers
guidance and their study progress counselling will be taken into consideration in the review
of the themes, standards and criteria involved.
– Ad 2.3: Staff commitment
The commitment of staff is an important condition for quality and decisive for the distinction
between professional and academic orientations. A significant part of programmes with a
professional orientation will be taught by staff who establish a connection between the
programme and professional or arts practice. Programmes with an academic orientation, on
the other hand, will largely be taught by researchers who contribute to the development of
the discipline (including research in the arts). In addition, the appropriate academic
programmes will have sufficient staff with knowledge of and insight into the relevant
professional or arts practice.
The institution must indicate the number of staff to be committed to the new programme, the
availability of such staff and the quality of the intended staff. The review of the intended staff
commitment to the programme weighs heavily, as the quality of the new programme will be
directly determined by the available staff. A separate consideration is whether this
commitment suffices for launching the programme – which will require an additional effort –
and for its continuation on a regular basis. The assessment will also have to review whether
the staff designated by the institution as deployable will indeed be available for the new
programme.

Page 18

NVAO | September 2009

The Higher Education Act does not explicitly mention the standard of staff quantity among
7
the generic quality standards. This standard has been included in the assessment
framework because the presence of sufficient staff is an evident condition for generic
quality.
– Re 2.4: Services and facilities
The criteria that apply to the services and facilities are, in principle, identical to those
observed in the accreditation of existing programmes. The services and facilities must be
sufficient to realise the curriculum. The institution must make a reasonable case that the
services and facilities concerned will be available (in time).
The services and facilities will likely include, for example, media centres, laboratories,
educational equipment, studio space or subject-specific data bases. The nature and level of
these services and facilities differ depending on the nature of the programme. The
assessment will check whether the services and facilities for the new programmes are in
accordance with the generally received level of quality for such services and facilities.
– Re 2.5: Internal quality assurance
The institution must demonstrate that it intends to pursue an acceptable level of quality from
the outset of the new programme and will subsequently continuously seek to monitor and
improve that quality. These intentions are more likely if the institution as a whole has in
place an effective system of internal quality assurance that covers all its programmes.
– Re 2.6: Conditions for continuity
One of the objects assessed with regard to new programmes is the uncertainties involved in
a starting situation, in particular the question of whether the administrative and staff
conditions warrant the sustainable realisation of the intended substantive quality. These
conditions pertain to both the programme as such and the institution providing it. It must be
likely that the institution can raise the required initial investment and continue to run the
programme, in the longer term as well, at least for the number of years required to build up
a full curriculum and the corresponding population of enrolled students. In principle, this
period of time equals the envisaged duration of the curriculum, depending on the nature of
the programme (bachelor’s programmes with a professional orientation, bachelor’s and
master’s programmes with an academic orientation, advanced bachelor’s programmes,
advanced master’s programmes). As a rule, this will be demonstrated by a business plan for
the new programme. The financial strength of the institution as a whole is demonstrated by
its annual accounts.
It is advisable that an application for a new programme is accompanied by a plan that
shows which programmes the institution intends to offer for the medium range. In this plan,
which will usually form part of a long-range plan or a strategic memorandum, the institution
can make a reasonable case that:
– the intended new programme is an expression of the institution’s strategic policy and the
ensuing priorities in the programmes offered;
– it will be capable, in terms of business economics, of providing the total range of existing
and intended programmes in a permanent manner.
The above plan can also serve as source of information for assessing whether the intended
investments and the envisaged operation of the new programme are realistic in the context
of the overall plans and expenditure of the institution. Submitting such a plan is not
mandatory on the part of the institution but can support the assessment process.
7

The Dutch legislation does not explicitly mention this

standard either.

Page 19

NVAO | September 2009

To the accreditation organisation, the data concerned is of importance in the longer term as
well. In case of a subsequent application for accreditation, the accreditation organisation will
ascertain or have ascertained whether the institution has fulfilled the conditions for continuity
and acted consistently in line with the intentions stated in its application for recognition.
Chapter 3: Assessment rules
The assessment results in a summarising review of the potential quality of the new
programme, with a positive or negative overall conclusion regarding the presence of
sufficient generic quality standards. A positive final conclusion requires a satisfactory score
on all themes of the assessment framework. For each theme, the conclusion is based on a
review of the various standards contained in that theme. The initial accreditation report gives
a transparent overview of the considerations at all levels – standards, themes and final
conclusion – which provides clarity regarding the underpinning of the final conclusion and
the weighing-up of the various standards.
Programmes may comprise more than one specialisation. If these specialisations set
different requirements with regard to a number of quality aspects – such as cohesion,
tutoring, staff commitment and tailoring to entering students – the review must demonstrate
that sufficient generic quality standards are potentially in place for each specialisation. The
assessment, however, concerns the programme as a whole. A similar assessment rule
applies to new programmes that are provided at more than one location.
Chapter 4: Initial accreditation procedure
In its assessment of new programmes, the accreditation organisation ties in with the
requirements and timeframes laid down in the Flemish Higher Education Act and the
regulations regarding the methodology for initial accreditations. An essential feature of this
method of working is the initiating role of the institution. It determines the nature of the
programme (bachelor’s programme with a professional orientation, bachelor´s programme
with an academic orientation, advanced bachelor´s programme, master´s programme,
advanced master´s programme). With regard to programmes provided by statutory
registered institutions, applications may only be submitted following a positive conclusion by
the Recognition Committee or an (explicit or implicit) positive conclusion by the Flemish
government regarding their macro-efficiency. Submitting an application to the accreditation
organisation sets the assessment process in motion. Based on the application, the
accreditation organisation determines how new this programme is to the Flemish higher
education system and consequently how extensively it needs to be assessed.
Subsequently, the accreditation organisation commissions external experts to review the
application and draw up an advisory report.
On the basis of the advisory report, the accreditation organisation then draws up an initial
accreditation report with its summarising conclusion. Guiding elements in the formulation of
the report are the benchmarks outlined in this chapter. The conclusions in the initial
accreditation report are substantiated on the basis of facts established, which are analysed
and tested against a reference framework that ties in with the assessment framework for
new programmes.
The initial accreditation report also explains the methods, sources of information and
reference framework used for the assessment. With regard to programmes new to the
Flemish education system, it indicates how the domain-specific requirements have been
mapped out. For existing programmes, it indicates which programmes have been used as a
reference.
The initial accreditation report gives an account of the procedure followed.
Before finalising the initial accreditation report, the accreditation organisation allows the
board of the institution a term of ten days to present its view regarding the draft initial
accreditation report. The initial accreditation report is positive if, in the opinion of the

Page 20

NVAO | September 2009

accreditation organisation, the application gives reasonable cause to assume that the new
programme will pass the test regarding the presence of sufficient generic quality standards.
The accreditation organisation must explicitly indicate whether or not the initial accreditation
is positive.
Following its finalisation, the accreditation organisation submits the initial accreditation
report to the board of the institution and the Flemish minister responsible for higher
education.

Page 21

NVAO | September 2009

Attachment 12 Sources
Accreditatiekader bestaande opleidingen hoger onderwijs Vlaanderen, NVAO, Den Haag, 1 september 2009.
(Vlaanderen)
Algemene wet bestuursrecht, 4 juni 1992 (Nederland)
Bestuursreglement NVAO, NVAO, Den Haag, 3 september 2003.
Beleidsnota 2009 - 2014, Onderwijs, Samen grenzen verleggen voor elk talent, Pascal Smet, Vlaams minister van
Onderwijs, Jeugd, Gelijke Kansen en Brussel, november 2009 (Vlaanderen)
Beoordelingskaders accreditatiestelsel hoger onderwijs, 22 november 2011. (Nederland)
Beoordelingskaders accreditatiestelsel hoger onderwijs, Instellingstoets kwaliteitszorg, 22 november 2011.
(Nederland)
Beoordelingskaders accreditatiestelsel hoger onderwijs, bijzonder kenmerk, 22 november 2011. (Nederland)
Beoordelingskader Onderzoeksmasters, 22 november 2011. (Nederland)
Beoordelingskader Pilot Bijzonder Kenmerk Ondernemen, 22 november 2011. (Nederland)
Bijzonder kenmerk ‘Kleinschalig en intensief onderwijs’, 4 november 2011. (Nederland)
Code of Good Practice, European Consortium for Accreditation in higher education, Zürich, 3 december 2004.
Communicatieplan 2010 - 2011, NVAO, Den Haag, 2 juni 2010. (Nederland)
Communicatieplan Vlaanderen, 16 november 2010 (Vlaanderen)
Decreet betreffende de herstructurering van het hoger onderwijs in Vlaanderen, 4 april 2003 en latere versies.
Eisen aan de Panelsamenstelling ten behoeve van het accreditatiestelsel, 24 augustus 2011. (Nederland)
Evaluatieve gesprekken in Vlaanderen: thematische samenvatting van de bevindingen en overwegingen van de
NVAO, 21 oktober 2011. (Vlaanderen)
Frameworks for the Assessment of Internationalisation, 14 November 2011.
Handleiding accreditatie Vlaamse Gemeenschap, 1 september 2009. (Vlaanderen)
Handleiding Uitschrijven Domeinspecifieke Leerresultatenkaders, proefversie voor de pilot, zonder datum.
(Vlaanderen)
Handleiding erkenning evaluatieorganen, 22 februari 2005. (Vlaanderen)
Handleiding toets nieuwe opleiding Vlaamse Gemeenschap, 1 september 2009. (Vlaanderen)
Handleiding onderwijsvisitaties VLIR en VLHORA, Brussel, September 2008
Handreiking TNO Panel: samenstelling, taakstelling en werkwijze, 22 augustus 2011.
Handreiking aanvraag Toets Nieuwe Opleiding Nederland, 21 oktober 2011. (Nederland)
Jaarboek onderwijs in cijfers 2009, 2010 en 2011, Centraal Bureau voor de Statistiek, Voorburg / Heerlen,
2009/2010/2011.
Kaders HBO5, 9 februari 2010. (Vlaanderen)
Kwaliteit in verscheidenheid, Strategische agenda hoger onderwijs, onderzoek en wetenschap, Ministerie van
OCW, Den Haag, augustus 2011. (Nederland)
Leereffectenrapport proefprojecten HBO5, NVAO, December 2010. (Vlaanderen)
Leidraad beoordelingen Nederlandse opleidingen door AACSB (aangepast), 24 januari 2012 (Nederland)

Leidraad NVAO ten behoeve van de accreditatie van academiserende opleidingen, 30 oktober 2009.
(Vlaanderen)
NVAO Gedragscode, part of "Leidraad eisen panelsamenstelling", 24 augustus 2011. (Nederland)
Protocol toetsing Associate degreeprogramma door de NVAO, ronde 5A (voor hogescholen), 2012 (Nederland)
Protocol toetsing Associate degreeprogramma door de NVAO, ronde 5B (voor bve-instellingen en hogescholen),
2012. (Nederland)
Protocol voor Nederlandse Aanvragen Accreditatie leidend tot een Joint degree, 7 juni 2010, versie februari 2011.
(Nederland)
Protocol voor Nederlandse aanvragen Toets Nieuwe Opleiding leidend tot een Joint degree, 7 juni 2010, versie
februari 2011. (Nederland)
Protocol ter uitvoering van de artikelen 7.1 en 7.2 van de Beleidsregel doelmatigheid hoger onderwijs 2009 (Stcrt
2009, 115), inhoudende de uitgangspunten van de NVAO met betrekking tot de samenvoeging (verbreding) van
opleidingen, de zogenoemde planningsneutrale conversie. (Nederland)
Protocol cursusduur masters, 8 oktober 2003. (Nederland)
Protocol erkenning als 'rechtspersoon voor hoger onderwijs', augustus 2010. (Nederland)
Protocol verlengde masters Beoordeling van de verlenging van de cursusduur van enkele specifieke
masteropleidingen in het WO, 23 april 2003. (Nederland)
Richtlijn NVAO beoordeling afstudeerwerken door panels tijdens het visitatieproces voor accreditatie en (indien
van toepassing) een toets nieuwe opleiding, 8 juli 2011 (Nederland)
Taken en competenties gecertificeerde secretarissen en coördinatoren NVAO, 17 augustus 2010 (Nederland)
Toelichting op accreditatie- en toetsingskader bij beoordeling van postinitiële wo-masteropleidingen in Nederland,
7 september 2010. (Nederland)
Toetsingskader nieuwe opleidingen hoger onderwijs Vlaanderen, 1 september 2009. (Vlaanderen)
Toetsingskader opleidingsschool, 3 maart 2009. (Nederland)
Sjabloon aanvraagdossier Accreditatie op basis van een internationaal Accreditatierapport, zonder datum.
(Vlaanderen)
Sjabloon aanvraagdossier ambtshalve geregistreerde instellingen macrodoelmatigheidstoets Toets nieuwe
opleidingen, zonder datum. (Vlaanderen)
Sjabloon aanvraagdossier geregistreerde en nog niet geregistreerde instellingen, zonder datum. (Vlaanderen)
Sjabloon aanvraagdossier Toets nieuwe opleidingen niet-Nederlandstalige opleidingen, zonder datum.
(Vlaanderen)
Regeling betreffende de omschrijving van sommige werkingsaspecten van de Nederlands-Vlaamse
accreditatieorganisatie, NVAO, Den Haag, 14 maart 2005.
Regeling bezwaarschriftenprocedure Awb, NVAO, Den Haag.
Reglement van de NVAO van 19 februari 2005 tot bepaling van bestuursbeginselen die van toepassing zijn bij de
besluitvorming inzake accreditatie en toets nieuwe opleiding ten aanzien van hogeronderwijsopleidingen in de
Vlaamse Gemeenschap, 19 februari 2005. (Vlaanderen)
Reglement tot bepaling van methodologische regelen voor de uitvoering van de Toets nieuwe opleiding ten
aanzien van hogeronderwijsopleidingen in de Vlaamse Gemeenschap, 19 februari 2005. (Vlaanderen)
Reglement tot bepaling van methodologische regelen voor de uitvoering van de Toets nieuwe opleiding ten
aanzien van hogeronderwijsopleidingen in de Vlaamse gemeenschap waarvan het opleidingsprogramma geheel
of gedeeltelijk wordt aangeboden buiten de Vlaamse Gemeenschap, 11 april 2006. (Vlaanderen)

Reglement van de Nederlands-Vlaamse Accreditatieorganisatie betreffende de validatie van de gezamenlijke
domeinspecifieke leerresultaten van hogeronderwijsopleidingen in de Vlaamse Gemeenschap, 1 februari 2011.
(Vlaanderen)
Reglement betreffende vormvereisten voor aanvragen tot uitvoering van de accreditatie of de Toets nieuwe
opleiding ten aanzien van hogeronderwijsopleidingen in de Vlaamse Gemeenschap, 1 september 2009.
(Vlaanderen)
Samenwerkingsprotocol NVAO – Inspectie voor het hoger onderwijs, 30 januari 2008. (Nederland)
Standpunt NVAO ten aanzien de Masterproef in de kunstopleidingen Vlaanderen, November 2009. (Vlaanderen)
Standards and Guidelines for Quality Assurance in the European Higher Education Area, European Association
for Quality Assurance in Higher Education, Helsinki, 2005.
Strategie NVAO 2012 – 2016, Den Haag, 13 februari 2012.
Verdrag tussen het Koninkrijk der Nederlanden en de Vlaamse Gemeenschap van België inzake de accreditatie
van opleidingen binnen het Nederlandse en Vlaamse hoger onderwijs, Den Haag, 3 september 2003.
Wet op het hoger onderwijs en wetenschappelijk onderzoek, 1 Januari 2011
http://www.ecaconsortium.net)
http://www.enqa.net).
http://www.hbo-raad.nl
http://www.hogeronderwijsregister.be
http://www.inqaahe.org
http://www.minocw.nl
http://www.vsnu.nl
http://www.croho.nl
http://www.efqm.org/en

Attachment 13 List of Abbreviations
AKOV

Agentschap Kwaliteit Onderwijs en Vorming

ANECA

Agencia Nacional de Evaluación de la Calidad y Acreditación

AQUIN

Akkreditierungs-, Certifizierungs- und Qualitätssicherungs-Institut für alle Studiengänge

Awb

Algemene wet bestuursrecht

CROHO

Centraal Register Opleidingen Hoger Onderwijs

DLR

Domeinspecifieke Leerresultaten

DUO

Dienst Uitvoering Onderwijs

ECA

European Consortium for Accreditation in higher education

EFQM

European Framework of Quality Management

ENQA

European Association for Quality Assurance in Higher Education

E-Train

The project European Training of Quality Assurance experts

EQAR

European Quality Assurance Register for Higher Education

EQF

European Qualifications Framework

EVA

Danish Evaluation Institute

HAVO

Hoger Algemeen Voortgezet Onderwijs

hbo

hoger beroepsonderwijs

HBO-raad

Vereniging van hogescholen (Nederland)

HBO5

Hoger beroepsonderwijs op niveau 5 van de Vlaamse kwalificatiestructuur

HOR

Hogeronderwijsregister

IQA

Internal Qulity Assurance

JOQAR

Joint programmes: Quality Assurance and Recognition of degrees awarded.

INQAAHE

International Network for Quality Assurance Agencies in Higher Education

MBO

Middelbaar beroepsonderwijs

NLQF

Netherlands Qualificatons Framework

NQF

National Qualifications Framework

NVAO

Nederlands-Vlaamse Accreditatieorganisatie

OCW

Ministerie van Onderwijs, Cultuur en Wetenschappen (Nederland)

OECD

Organisation for Economic Co-operation and Development

OZM

onderzoeksmasters

PhD

Doctor of Philosophy

QAA

Quality Assurance Agencies

QAA

Quality Assurance Agency for Higher Education in the United Kingdom

SERV

Sociaal-Economische Raad van Vlaanderen

TEAM II

Transparent European Accreditation decisions & Mutual recognition agreements II

TNO

Toets nieuwe opleiding

UAS

Universities of Applied Sciences

VLHORA

Vlaamse Hogescholenraad

VLIR

Vlaamse Interuniversitaire Raad

VLOR

Vlaamse Onderwijsraad

VLUHR

Vlaamse Universitaire en Hogescholenraad

VSNU

Vereniging van Samenwerkende Nederlandse Universiteiten

VWO

Voorbereidend Wetenschappelijk Onderwijs

wo

wetenschappelijk onderwijs

WOT

Wet op het onderwijstoezicht