2020
ENQA AGENCY REVIEW: 
NATIONAL ENTITY FOR ACCREDITATION AND QUALITY 
ASSURANCE IN HIGHER EDUCATION (NEAQA)	
DORIS HERRMANN, DURDICA DRAGOJEVIC, PEDRO TEIXEIRA, IGNAS GAIZIUNAS
20 FEBRUARY 2020 
ENQA 
AGENCY 
REVIEW

1/70 
 ContentsEXECUTIVE SUMMARY
 ...................................................................................................................... 3 INTRODUCTION
 ................................................................................................................................ 5 BACKGROUND OF THE REVIEW AND OUTLINE OF THE REVIEW PROCESS
 ............................... 5 
BACKGROUND OF THE REVIEW .............................................................................................................. 5 
MAIN FINDINGS OF THE 2017 REVIEW ................................................................................................... 5 
REVIEW PROCESS ................................................................................................................................ 6 HIGHER EDUCATION AND QUALITY ASSURANCE SYSTEM OF THE AGENCY
 ........................... 7 
HIGHER EDUCATION SYSTEM ................................................................................................................. 7 
QUALITY ASSURANCE ........................................................................................................................... 8 NATIONAL ENTITY FOR ACCREDITATION AND QUALITY ASSURANCE IN HIGHER EDUCATION
 ...................................................................................................................................... 9 
NEAQA’S ORGANISATION/STRUCTURE .................................................................................................. 9 
NEAQA’S FUNCTIONS, ACTIVITIES, PROCEDURES ................................................................................... 10 
NEAQA’S FUNDING .......................................................................................................................... 14 FINDINGS: COMPLIANCE OF NEAQA WITH THE STANDARDS AND GUIDELINES FOR QUALITY ASSURANCE IN THE EUROPEAN HIGHER EDUCATION AREA (ESG)
 .............. 15 ESGPART 3:QUALITY ASSURANCE AGENCIES
 ........................................................................ 15 
ESG 3.1 ACTIVITIES, POLICY, AND PROCESSES FOR QUALITY ASSURANCE ................................................... 15 
ESG 3.2 OFFICIAL STATUS .................................................................................................................. 18 
ESG 3.3 INDEPENDENCE .................................................................................................................... 19 
ESG 3.4 THEMATIC ANALYSIS ............................................................................................................. 22 
ESG 3.5 RESOURCES ......................................................................................................................... 23 
ESG 3.6 INTERNAL QUALITY ASSURANCE AND PROFESSIONAL CONDUCT ................................................... 25 
ESG 3.7 CYCLICAL EXTERNAL REVIEW OF AGENCIES ................................................................................ 27 ESGPART 2:EXTERNAL QUALITYASSURANCE
 ........................................................................ 28 
ESG 2.1 CONSIDERATION OF INTERNAL QUALITY ASSURANCE .................................................................. 28 
ESG 2.2 DESIGNING METHODOLOGIES FIT FOR PURPOSE ........................................................................ 32 
ESG 2.3 IMPLEMENTING PROCESSES .................................................................................................... 35 
ESG 2.4 PEER-REVIEW EXPERTS .......................................................................................................... 39 
ESG 2.5 CRITERIA FOR OUTCOMES ...................................................................................................... 43 
ESG 2.6 REPORTING ......................................................................................................................... 46 
ESG 2.7 COMPLAINTS AND APPEALS.................................................................................................... 48 ADDITIONAL OBSERVATIONS
 ..................................................................................................... 52 
TIMING OF THE REVIEW ................................................................................................................ 52

2/70 
 
SUPPORT TO INSTITUTIONS .......................................................................................................... 52 
MOVING TOWARDS GREATER INSTITUTIONAL RESPONSIBILITY .................................................. 52 CONCLUSION
 ................................................................................................................................. 53 SUMMARY OF COMMENDATIONS
 ............................................................................................. 53 OVERVIEW OF JUDGEMENTS AND RECOMMENDATIONS
 .................................................... 53 SUGGESTIONS FOR FURTHER DEVELOPMENT
 ........................................................................ 55 ANNEXES
 ............................................................................................................................................ 57 ANNEX 2:TERMS OF REFERENCE OF THE REVIEW
 ................................................................ 62 ANNEX 3:GLOSSARY
 ................................................................................................................... 69 ANNEX 4.DOCUMENTS TO SUPPORT THE REVIEW
 ............................................................... 70 
DOCUMENTS PROVIDED BY NEAQA .................................................................................................... 70 
OTHER SOURCES USED BY THE REVIEW PANEL ........................................................................................ 70

3/70 
 EXECUTIVE SUMMARY
This report analyses the compliance of the National Entity for Accreditation and Quality Assurance in 
Higher Education (NEAQA) with the Standards and Guidelines for Quality Assurance in the European 
Higher Education Area (ESG). It is based on an ENQA-coordinated external review conducted between 
April to December 2019 with a site visit to Belgrade, Serbia, between 9 and 11 October 2019. NEAQA 
was established as an independent institution in 2018, and it is considered to be a legal successor of 
the Commission for Quality Assurance (CAQA). CAQA was established in 2005, and had been a full 
member of ENQA since 2013 and listed in EQAR since 2014. CAQA continues to exist as a professional 
body of NEAQA. This is thus considered to be the third review of NEAQA. The agency is applying for 
renewal of its ENQA membership and EQAR registration. The last review took place in 2017, and 
resulted in CAQA gaining the status of a member under the review with ENQA. This review is thus 
taking place 2 years later, to potentially enable NEAQA to continue being an ENQA member. It is the 
opinion of the panel that the review was probably premature, as the system saw a major change with 
the new 2017 Law on Higher Education and NEAQA started functioning less than two years ago. In 
comparison to CAQA as its legal predecessor, NEAQA has changed the role of CAQA to act as a 
decision-making body. NEAQA also introduced independent peer panels who write published reports 
and conduct site visits, which was both done by CAQA members in the past. However, this was the 
major change in the procedures, while the standards used have not changed substantially, and a 
number of recommendations from the 2017 report remain to be implemented, which was also 
reflected in this panel’s assessments of compliance.  
 
NEAQA conducts initial and periodic accreditation reviews of higher education institutions and 
programmes to ensure compliance with minimum requirements, and institutional-level audits aimed 
at quality enhancement but also with a possibility to end in a formal removal of accreditation. All 
processes are mandatory for institutions, with their frequency predetermined by Law (and relatively 
frequent, with 7-year accreditation cycles and audit mid-cycle). NEAQA received initial funding from 
the state budget, but it is supposed to be funded only from accreditation fees paid by higher education 
institutions in the future. NEAQA structure and functioning are largely predefined by the Law. Its 
decision-making body is a seventeen-member CAQA which is composed exclusively by academics, 
divided in sub-commissions when working, according to their various fields. Operatively the agency is 
overseen by the Management Board, which also includes students and labour-market representatives, 
and run by the Director and the General Secretary, who heads the small Secretariat of nine full-time 
staff. Even though not formally part of NEAQA, the National Council for Higher Education, a 
government body composed of academics, labour-market representatives and students, has an 
important role in NEAQA functioning. It adopts NEAQA regulations and standards, it creates the lists 
of experts for NEAQA panels, it suggests CAQA members to NEAQA Management Board, and it 
functions as an appeals committee which has the legal possibility to change CAQA decisions as a last 
resort. 
The Law on Higher Education provides NEAQA with a clear legal basis for its regular external quality 
assurance activities and for the outcomes of its processes to be recognised by its stakeholders. 
NEAQA’s work is appreciated by the stakeholders, and they all put much emphasis on the need for 
such an institution in the Serbian higher education system which is characterised by a low level of trust 
in newly established institutions, but also charges of complacency with the old ones - the quality of 
which is not questioned. In comparison to its predecessor, NEAQA’s independence was strengthened 
by the establishment as an independent institution and change in the procedure of CAQA’s 
appointment. While the role of National Council, as a government body, can be seen as a threat to

4/70 
 
NEAQA independence, the panel agreed on seeing its role as part of a system of checks and balances 
which should ensure that there is no single body making all the decisions. NEAQA independence as an 
institution also means that it now has full-time staff which should be able to devote themselves to 
developmental work, produce thematic analyses, participate in international exchanges and projects, 
etc. However, again probably due to review timing, none of this has as yet taken place, and there 
remains the need to develop this, as well as overall NEAQA internal quality assurance. NEAQA’s 
funding model as based solely on fees remains to be tested. While there is no question that NEAQA 
currently has sufficient funds, it is not clear if this will remain so in the next year, when government 
seed funding is spent and the number of reviews is increased.  
 
NEAQA’s external quality assurance processes and decisions are based on published standards and 
procedures which are detailed and cover Part 1 ESG to a substantial degree. There is evidence that 
the consistency of the assessments has improved with the changes in the procedure. The same is true 
of the developmental orientation of the reports, which are now published. However, NEAQA has 
implemented very few procedures in line with the new methodology, and no audits among them, 
which means that the evidence is limited. Much work remains to be done on understanding and 
implementing the ESG, and as noted by the 2017 report, improving the effectiveness of HEI’s internal 
quality assurance and encouraging institutions to take the primary responsibility for quality. NEAQA 
had little time to engage in substantial discussions with stakeholders on the fitness for purpose of its 
processes and how its procedures and criteria can be changed. The change should aim to: alleviate 
the institutional burden, make procedures more flexible and thus better adapted to different types of 
institutions, focus on quality development rather than quantitative indicators, and increase the 
robustness of the procedures so that they are sufficiently challenging for the institutions. While in 
comparison with its predecessor NEAQA did improve the way it implements processes, the way it 
works with peer-review experts, and the way it writes and publishes the reports, much still needs to 
be done to further increase the quality of the agency’s work. The steps that the panel recommends 
always revolve around devoting more time and providing mechanisms for deeper reflection with the 
stakeholders and self-reflection. In other aspects, such as introducing a complaints body, NEAQA did 
little even though it was required to do so by the 2017 report.   
 
The review panel has found NEAQA to be fully compliant with ESG 3.2 and 3.7; substantially compliant 
with ESG 3.1, 3.3, 3.5, 2.4, 2.5 and 2.6; and partially compliant with ESG 3.4, 3.6, 2.1, 2.2, 2.3 and 2.7. 
It has made recommendations under most of the ESG and a number of suggestions for further 
development. Despite the ‘partially compliant’ judgments under six ESG, the panel concludes that 
NEAQA is, overall, in substantial compliance with the ESG. The reason for this is the level of visible 
improvement that did take place since 2017, and the need to understand NEAQA in the context of 
Serbian HE which is in a large degree determined by the Law and characterised by a relatively low level 
of institutional autonomy. The panel hopes that NEAQA will understand the positive assessment as an 
encouragement to continue working on implementing the recommendations of the 2017 review and 
this report. In this context, and again due to the timing of this review, a follow-up report will be 
important, and it would be well if all types of procedures would be implemented with reports available 
by the time it is drafted.

5/70 
 INTRODUCTION
This report analyses the compliance of the National Entity for Accreditation and Quality Assurance in 
Higher Education (NEAQA) (Nacionalno telo za akreditaciju i proveru kvaliteta u visokom obrazovanju, 
NAT) with the Standards and Guidelines for Quality Assurance in the European Higher Education Area 
(ESG). It is based on an external review conducted in a six-month period from April to December 2019. BACKGROUND OF THE REVIEW AND OUTLINE OF THE REVIEW PROCESS
BACKGROUND OF THE REVIEW 
ENQA’s regulations require all member agencies to undergo an external cyclical review, at least once 
every five years, in order to verify that they act in substantial compliance with the ESG as adopted at 
the Yerevan ministerial conference of the Bologna Process in 2015. Commission for Accreditation and 
Quality Assurance (CAQA) has undergone its second ENQA review in 2017, which resulted in CAQA 
gaining the status of a “member under review”, which was inherited by NEAQA as CAQA’s legal 
successor. Due to the legal and organisational changes that took place in the meantime, the current 
review is organised as full review and thus, NEAQA’s third review. The panel is thus expected to 
provide clear evidence of results in all areas and to acknowledge progress from the previous review. 
The panel has adopted a developmental approach, as the Guidelines for ENQA Agency Reviews aim at 
constant enhancement of the agencies. 
MAIN FINDINGS OF THE 2017 REVIEW 
The panel of experts that reviewed CAQA, the legal predecessor of NEAQA, in 2017 with the report 
published in 2018, established that in the performance of its functions, CAQA was overall in substantial 
compliance with the ESG. The panel found CAQA to be partially compliant with seven of the ESG but 
wished to emphasise that in four cases (standards 2.2, 2.3, 2.4 and 2.6) this resulted mainly from one 
issue in the overall design of the EQA system in Serbia. The issue was that CAQA members acted as 
the core group of peer-review experts conducting EQA processes, with only limited involvement of 
external experts, and CAQA was the decision-making body in these processes. Additionally, CAQA was 
conceived more as a committee of experts than a fully-fledged QA agency, which had implications for 
how effectively it could perform its planning and management functions and this had also to a smaller 
extent influenced the panel’s judgments under ESG 3.1, 3.5 and 3.6. Overall the panel’s assessment 
was as follows:  
- fully compliant with ESGs 3.2 (Official status) and ESG 3.7 (Cyclical external review of agencies); 
- substantially compliant with ESGs 3.3 (Independence), 3.4 (Thematic analysis), 3.5 (Resources), 2.1 
(Consideration of internal quality assurance) and 2.5 (Criteria for outcomes);  
- partially complaint, as noted above, with ESGs 3.1 (Activities, policy and processes for quality 
assurance), 3.6 (Internal quality assurance and professional conduct), 2.2 (Designing methodologies 
fit for purpose), 2.3 (Implementing processes), 2.4 (Peer-review experts), 2.6 (Reporting) and 2.7 
(Complaints and appeals).  
The recommendations made by the panel of the 2017 review are noted under each standard. The 
panel also commended CAQA for commitment to its mission and doing its job in a way which is 
genuinely appreciated by its stakeholders despite tough accreditation decisions it had made (ESG 3.1), 
sustained and successful efforts to safeguard its independence within the constraints placed by the 
national legislation (ESG 3.3), addressing very comprehensively student involvement in internal quality 
assurance as part of its standards for audits (ESG 2.1) and, finally, publishing the Guide to accredited 
HEIs and programmes as a useful source of information for prospective students (ESG 2.6).

6/70 
 
Finally, it should be noted that the panel visited CAQA at the time when the current, 2017 Law on 
Higher Education was already adopted, as well as the 2017 standards for QA, however they have not 
been implemented at the time of the review and were thus not considered.  
REVIEW PROCESS 
The 2019 external review of NEAQA was conducted in line with the process described in the Guidelines 
for ENQA Agency Reviews and in accordance with the timeline set out in the Terms of Reference. The 
panel for the external review of NEAQA was appointed by ENQA and composed of the following 
members: 
• 
Doris Herrmann, Managing Director, Agency for Quality Assurance through Accreditation of 
Study Programmes, Germany, Chair, quality assurance professional (ENQA nominee) 
• 
Durdica Dragojevic, Head of the Department for Higher Education Quality, Croatian Ministry 
of Science and Education, Croatia, Secretary, quality assurance professional (ENQA nominee) 
• 
Pedro Teixeira, Associate Professor, Faculty of Economics, University of Porto; Director, 
Centre for Research in Higher Education Policies (Cipes), Portugal, Panel member, academic 
(EUA nominee) 
• 
Ignas Gaižiūnas, Student in Physics, study programme Theoretical Physics and Astrophysics, 
Vilnius University, Lithuania, Panel member, student (ESU nominee) 
Goran Dakovic from ENQA acted as the review coordinator, and also participated in the site visit to 
the agency.  
Self-assessment report 
In April 2019 the team for preparation of the self-assessment report (SAR) was appointed by the 
NEAQA Director, whose members were nominated by the Managing Board. The team consisted of the 
Director, Secretary General, four members of CAQA, three Managing Board members of which two 
are labour market representatives, and three students’ representatives, which is an important 
improvement to the previous report in the creation of which no stakeholders have participated. The 
first draft version of the report, made on the basis of the previous CAQA and ENQA reports, was made 
by NEAQA staff and sent to the SAR team for further improvements. Like the previous CAQA report, it 
includes analysis made based on the answers of higher education institutions and peer-reviewers to 
the questions regarding the establishments of NEAQA and new system, which were collected during 
meetings and trainings conducted by CAQA members. The survey focused on the expectations from 
the new system rather than providing comments on the SAR. Finally, in May SAR was adopted by the 
Board and sent to ENQA on 31st May 2019.  
 
Site visit 
The site visit programme was prepared in cooperation with the NEAQA liaison person. On the day 
preceding the visit the panel had an internal preparatory meeting. The visit took place between 9 and 
11 October 2019. The panel interviewed all key stakeholders, including the SAR working group, 
NEAQA’s members and staff, academic experts, students and employers involved in NEAQA processes, 
and representatives of higher education institutions and their conferences, students’ conferences and 
the national authorities. The visit was well organised. The meetings were conducted efficiently, 
despite the interpreting provided in most meetings. At the end of the visit, the panel had an internal 
meeting to agree on conclusions from the review and a debriefing for NEAQA on the main findings. 
For details, see the site visit agenda in Annex 1.

7/70 
 HIGHER EDUCATION AND QUALITY ASSURANCE SYSTEM OF THE AGENCY 
 
HIGHER EDUCATION SYSTEM 
Serbian higher education is governed by the 2017 Law on Higher Education (LoHE); the Law is 
complemented by the 2018 Law on the Qualifications Framework and at the time of the site visit, a 
Law on Dual Education with relevance to higher education was coming into force. Higher education 
covers programmes at three levels, with each type of programme having its own category in the 
qualifications framework: (1) first-cycle programmes which include basic academic programmes (180-
240 ECTS) and professional programmes (180 ECTS), leading to the degree of Bachelor, Bachelor with 
Honours (240 ECTS) or Bachelor (Appl.) respectively; (2) second-cycle programmes, including master 
academic (60-120 ECTS) and professional (120 ECTS) programmes, leading to the degree of Master or 
Master (Appl.) respectively; and specialist academic and professional programmes, both bearing 60 
ECTS and leading to a Specialist degree / diploma; and (3) third-cycle programmes / studies leading to 
a PhD degree, with 180 ECTS (with at least 300 ECTS earned earlier) and a dissertation as the final part 
of the programme. Master’s degree programmes in medical fields are offered as long-cycle 
programmes of 360 ECTS. LoHE defines a specific grading system with grades from 5 (fail) to 10 
(excellent).  
According to LoHE, a university has to have accredited academic study programmes in at least 3 
scientific/artistic fields at all 3 levels. University constituents – faculties and arts academies – can be 
separate legal persons if they have 3 accredited programs or more, and are considered separately 
within institutional accreditations. There are however also universities without other legal persons as 
constituents. Additionally, while normally constituents deliver programs and the university acts as a 
central administration, it can also independently deliver programs and some do. There is a total of 18 
accredited universities in Serbia, 9 owned by the state and 9 private, and they have a total of 124 
accredited faculties – 86 state and 38 private. They are represented by the Conference of Universities 
(CONUS).  
Academic studies at first two levels can also be delivered by colleges (higher schools) of academic 
studies, of which there are five. Applied courses are delivered by 61 colleges of applied studies, of 
which 70% are state-owned. LoHE established ‘academies of applied studies’ and the process is 
ongoing during which the existing state-owned colleges will be merged into such academies. These 
institutions are represented by the Conference of Academies of Applied Studies and Colleges of 
Academic Studies (CAASS).  
Table 1: types of HEIs and the differences in assessment (source: NEAQA)  
Type of institution 
Assessment in the accreditation process 
University 
- 
It has to have all levels of studies within 
minimum of three fields and three research 
areas defined in the Article 37 of LoHE;  
- 
it has to have an accreditation for scientific 
research organisation by the National 
Council for Scientific and Technological 
Development – NCSTD. 
Faculty  
- 
it has to be part of the university; 
- 
if it is a legal person it has to have 
accreditation 
for 
scientific 
research 
organisation by NCSTD.

8/70 
 
Higher school of academic studies  
- 
in case of accreditation for master studies it 
has to have an accreditation for scientific 
research organisation. 
Academy of applied studies 
- 
standards for its accreditation are under 
development; 
- 
recently formed academies are consisted of 
state higher schools of applied studies. 
Higher school of applied studies 
- 
special attention in accreditation is on its 
connections to the labour market. 
 
There is about 240 000 students in Serbia, 80% of which study at state-owned universities. The 
students are organised in the Student Conference of Universities (SCONUS) and Student Conference 
of Academies of Applied Studies and Colleges of Academic Studies (SCAASS). Regarding the number 
of students, the system is very uneven as the University of Belgrade has 95 000 students and 50 000 
study at Novi Sad University, so only two universities make up more than a half of the system.   
QUALITY ASSURANCE 
The first Serbian Commission for Accreditation was set up in 2002 and tasked with evaluating newly 
established private HEIs and developing an accreditation methodology; in 2005, a new Law on Higher 
Education introduced the principles of the Bologna reform to Serbia and established Commission for 
Accreditation and Quality Assurance, CAQA, as a body of the National Council for Higher Education 
(hereinafter: NCHE; the National Council) with the administrative support provided by the Ministry 
of Education and NCHE serving as an appeal body. For more than a decade, CAQA functioned in a 
manner that was in principle unchanged, performing programme and institutional accreditations in 
five-year cycles and audits (referred to as external quality assurance or external quality control in the 
law) in between the accreditations, together with initial programme and institutional accreditations 
for newly-established HEIs. CAQA was composed by 15 academics proposed by CONUS and appointed 
by NCHE, to whom in 2015 two CAASS representatives were added. The reviews were done by 
anonymous academic experts selected by NCHE and supplemented by site visits by sub-commissions 
composed of CAQA members joined by student and employer representatives selected by the student 
organisations and the Chamber of Commerce. The sub-commission prepared an overall report, 
integrating experts’ assessments and proposing to CAQA a final decision. The Ministry of Education, 
Science and Technological Development (MoESTD) issued and revoked operating licences, without an 
option to change CAQA decisions. NCHE and CAQA periodically changed the standards in use, always 
on the basis of standards as defined by the law, and the last such change took place in 2017. By that 
time, CAQA has completed two accreditation cycles and one audit cycle, and was starting the third 
cycle of accreditations.  
HEIs have since 2005 been required by law to set up an internal body responsible for IQA and carry 
out a self-evaluation at least every three years. Such a body is composed of teaching and non-teaching 
staff and students and is responsible for self-evaluation. Self-evaluation obligatorily takes into account 
findings from student course evaluations. As of 2019, a new job description was included by MoESTD 
in the catalogue of positions at HEIs to enable employment of quality managers.  
In October 2017 new LoHE established new body for accreditation– National Entity for Accreditation 
and Quality Assurance in Higher Education (NEAQA)– and CAQA was put in technical mandate in 
February 2018. All rights and obligations, documentation, accreditation requests, equipment and 
archives were transferred to NEAQA that applied for the ENQA review in 2018.

9/70 
 
NCHE has kept its quality assurance role also in the new law: it suggests the list of reviewers to be 
nominated to the CAQA pool, and CAQA members to the NEAQA Board; it adopts EQA standards on 
the suggestion of NEAQA and IQA standards on the suggestion of the two HEI conferences, and serves 
as the appeals body of NEAQA. NCHE has 17 members appointed by the Government, observing the 
gender balance: 6 academics at the proposal of CONUS and 2 at the proposal of CAASS; 7 
experts/academics proposed by MoESTD, and 2 representatives of the Chamber of Commerce. It is 
also stipulated by LoHE that NCHE invites 2 representatives of the 2 student organisations to 
participate in its work on issues relevant to them.  NATIONAL ENTITY FOR ACCREDITATION AND QUALITY ASSURANCE IN HIGHER EDUCATION
NEAQA was established on 9 February 2018 by the decision of the Government of the Republic of 
Serbia. The decision states that NEAQA is an independent legal entity established for the purpose of 
performing accreditation and quality assessment of HEIs and their units, study programmes and 
quality assurance in HE. The legal provisions also formulated the appointing of the acting director; the 
Board was formed in May 2018 and elected the director (the same person appointed as the acting 
director) in July. NEAQA is the successor of CAQA established in accordance with the LoHE from 2005. 
CAQA itself was transformed from a body of NCHE to the expert body of the newly established NEAQA, 
and new members were appointed by the NEAQA Management Board (the Board) upon the proposal 
of NCHE, meeting in the new composition for the first time in August 2018. NEAQA took over all 
accreditation requests not completed according to the old law, more than 300 of them submitted 
before December 2018, and the new CAQA (i.e. NEAQA) is completing them in line with the old 
procedures.  
NEAQA’S ORGANISATION/STRUCTURE 
According to the LoHE, NEAQA is composed of managing body, the executive body, the professional 
body and professional services which perform the administrative-technical tasks.  
The management body of NEAQA is the Managing Board (hereinafter: the Board) consisting of seven 
members, appointed by the Government every four years. The Government appoints members of the 
Board upon the proposal of the following institutions: one member by CONUS from full professors of 
the university, one member by CAASS from professors of applied studies, two members by the Serbian 
Chamber of Commerce, and three members by the Ministry. Its members are elected for a period of 
four years, with the possibility of another re-election. The student representative is nominated by 
SCONUS as an associate member of the Board, based on the decision of the Board.  
The Managing Board: elects and dismisses the director of NEAQA, and oversees their work; elects and 
dismisses the members of CAQA on suggestion by NCHE; adopts the annual programme of work and 
the financial plan, upon the consent of the Government; adopts the Statute upon prior consent of the 
Government; adopts general acts and coda; determines the amount of accreditation fee, with the 
consent of the Government.  
The Director is elected by the Board based on a public competition, among full professors of the 
university who have experience in managing and assuring quality in higher education for a period of 
five years, with the possibility of re-election. The Director represents NEAQA, manages and organises 
its work, operations and staff, implements the decisions of the Managing Board, appoints reviewers 
from the lists defined by NCHE, at the proposal of CAQA, and passes all CAQA decisions.  
While it is still completing procedures inherited from the previous CAQA, the new CAQA’s tasks are to 
propose review panels and accreditation standards and procedures, and serve as the expert and

10/70 
 
decision-making body of NEAQA. The members of CAQA are elected every five years through an open 
call by the Managing Board, at the proposal of the National Council, respecting the gender equality 
and representation of the educational-scientific, i.e. educational-artistic domains. CAQA elects the 
President and Vice-President from among its members and establishes sub-commissions to perform 
its tasks, e.g. prepare decisions etc. In its work, CAQA cooperates with more than 800 external experts.  
The Secretariat is the internal organizational unit which performs administrative and professional 
support activities of NEAQA, managed by the Secretary. The Secretary coordinates and manages the 
work of the administrative and professional service, takes care of the preparation of the materials for 
the sessions of the bodies of NEAQA, monitors their work and formulation of decisions, coordinates 
their work in accordance to the instructions of the Director, executes decisions of the bodies that are 
in the competence of the administrative and professional service and acts upon the decisions of the 
Director, takes care of the protection of the assets of NEAQA, keeps the seals of NEAQA, etc. In 
addition to the Secretary, the Secretariat currently employs 2 more legal experts and 5 more staff 
members working directly on QA procedures. While in the pilot procedures the Secretary acted as a 
coordinator at all site visits, it is planned that all of them act as site visit coordinators, accompanying 
and briefing panels, and most already have this experience from working for CAQA. NEAQA employs 
additional 2 support staff, and more support staff, e.g. for finances and IT, are hired through 
temporary contracts.   
Figure 1: NEAQA organigram (source: SAR) 
 
 NEAQA’S FUNCTIONS, ACTIVITIES, PROCEDURES 
According to LoHE, NEAQA performs the following procedures:  
1. Initial accreditation of higher education institutions (HEIs)  
2. Initial accreditation of programmes (also applied to existing programmes when they are 
changed more than 20%) 
3. Regular, periodic accreditation of higher education institutions (HEIs)  
4. Regular, period accreditation of HEI programmes 
5. ’Externa quality control of HEIs’ - audit of HEIs’ internal quality assurance (IQA) systems.

11/70 
 
The timeline of the procedures, defined by LoHE, is shown on the figure below.  
Figure 2: the timeline of various NEAQA procedures 
 
Thus, the same basic procedures have been kept from the previous period, with procedural changes 
described in the table below.  
Table 2: Comparison of NEAQA old and new procedures (source: NEAQA) 
 
The agency’s old QAA procedures 
The 
agency’s 
new 
QAA 
procedures 
Cycle  
Period accreditation every 5 years, audit 
mid-cycle  
Period accreditation every 7 
years, audit mid-cycle  
Launching 
the 
procedure 
The accreditation request submitted to 
CAQA 
The 
accreditation 
request 
submitted to NEAQA which 
issues 
the 
certificate 
on 
completeness 
of 
the 
documentation 
Finding experts 
CAQA appoints two reviewers 
CAQA proposes members of the 
expert panel and NEAQA director 
appoints the panel 
Reporting 
Two anonymous experts check the 
documentation and give the evaluations 
Peer review panel checks the 
documentation and goes to a site 
visit in order to complete a 
report and propose a decision

12/70 
 
Decision-making 
process 
Two reports of the reviewers  are 
delivered to a sub-commission, which 
performs site-visit and makes a report 
with the decision proposal to CAQA 
Report is sent to CAQA’s  sub-
commission which forwards it to 
CAQA  
Issuing the decision 
CAQA makes decision and issues a 
decision and the formal certificate  
(“uverenje”  or “rešenje”) 
CAQA  makes and issues the 
decision and NEAQA issues the 
formal certificate (“uverenje”  or 
“rešenje”) 
 
As noted, NEAQA has inherited more than 300 procedures to be completed according to the old 
model. According to the SAR, 246 were completed by May 2019. The remaining ‘old’ procedures are 
to be completed no later than 2020 according to the information from the site visit. In 2018 NEAQA 
completed 6 institutional and 37 programme periodic accreditations and 3 initial accreditations, and 
in 2019 it completed 12 institutional and 107 programme periodic accreditations and 4 initial 
accreditations.  
Table 3: The number of old and new procedures implemented by NEAQA with outcomes (source: SAR) 
For year 2018  
 
Accreditation of HEIs  
6 positive  
Accreditation of study programmes;  
37  
(35 positive 
and 2 
negative)  
Initial accreditation of HEI and study programmes;  
3  
(1 positive 
and 2 
negative)  
External quality control – audit.  
0  
 For year 2019  
 
Old procedure  
 
Accreditation of HEIs  
12  
(positive 11 
and 1 
negative)  
Accreditation of study programmes;  
107  
(92 positive 
and 15 
negative)  
Initial accreditation of HEI and study programmes;  
4  
(1 positive 
and 3 
negative)  
External quality control – audit.  
0

13/70 
 
New procedure by 8 July 2019  
 
Accreditation of HEIs  
0  
Accreditation of study programmes;  
5  
Initial accreditation of HEI and study programmes;  
0  
External quality control – audit.  
0  
 
New procedural documents were adopted by the National Council and officially published by March 
2019. As before, there are separate regulations for programme and institutional periodic 
accreditation, audit and self-evaluation of HEIs (which serves as a basis for audit). The key change is 
that CAQA members no longer participate in site visits, but make decisions solely on the basis of the 
report of the panel whose members are no longer anonymous, and include a student and a labour 
market representative. The procedure is the same in all cases: a sub-commission of CAQA proposes 
the composition of a Peer-review Panel for accreditation of HEI/programme/initial accreditation/audit 
consisting of five members (three academics, one student and one labour market representative) to 
CAQA which submits its final proposal to the NEAQA Director. The Director further appoints a peer-
review panel with a president, and a Coordinator from the administrative and professional services of 
NEAQA. The decision on the appointment of a peer-review panel for accreditation is published on the 
NEAQA website. The panel drafts the preliminary report, and then goes to the mandatory site visit, 
after which the report is completed and sent to the HEI which has 15 days to correct factual errors. 
The corrected report is submitted to CAQA which can then ask the panel for clarifications.  
Based on the regulations, separate forms, templates and tables are available for different types of 
HEIs (one for universities and academies of applied studies, another for faculties and colleges) and 
programmes (one for scientific and one for artistic PhDs, and one for first and second cycle programs), 
and are supplemented by templates to be used by the panels. At the time of the site visit, a new 
separate set of standards was being prepared specifically for the academies of applied studies. SAR 
reported that according to the new procedure 5 accreditations of study programmes were completed, 
and by the site visit 5 HEIs had one or more programme accredited, and 2 of those also received 
institutional accreditation.  
 
Institutions are established through initial program and institutional accreditation. A new HEI needs 
to fulfil of a number of conditions defined by the LoHE and NCHE regulations regarding the number 
and area of study programmes, infrastructure, employed staff etc. The founder then prepares the 
documentation defined by NEAQA regulations, and after the accreditation procedure a decision on 
initial accreditation or refusal of the request is brought by CAQA. A positive decision on initial 
accreditation of the HEI is sent to the Ministry for issuing an operating licence to the HEI that is valid 
for only one year. After one year, HEI has to undergo regular, periodic institutional and program 
accreditation procedure. Even though each program accreditation is legally a separate procedure, led 
individually from the panel appointment to the final decision, these often - when possible due to 
different deadlines of QA activities - take place through one site visit, and panel members are in that 
case shared among programmes. If successfully accredited, after 2 years the HEI will report to CAQA 
on the progress in implementing the recommendations. After 3 years, HEI will perform a self-
evaluation and submit it to CAQA as a basis for audit which will be done 4 years after the last 
accreditation – if that one did not result in an earlier audit deadline, which is an option foreseen by 
the regulations. Audit will only have a follow-up after no more than 6 months if shortcomings are

14/70 
 
identified – but if these are not corrected, accreditation may be revoked to a programme or the whole 
HEI. After successfully completing an audit, new periodic accreditation follows in 3 years.  
A HEI can at any time submit a new programme to undergo initial programme accreditation. It is also 
obliged by LoHE to submit an existing programme to CAQA for evaluation if it is changed more than 
20%.  However, if new programs have been launched in the meantime, these will not necessarily fit 
the schedule, and it is possible that HEI will have programme reviews within the 7-year periods in 
which it undergoes its institutional accreditation.  
HEI will continue to send all new programmes to initial programme accreditation and submit any 
programs in which more than 20% is changed to a partial programme accreditation procedure. The 
new programme will undergo periodic accreditation after 7 years, not necessarily fitting with the 
timeline of periodic accreditation of the institution and its other programmes.  
NEAQA’S FUNDING 
Until the establishment of NEAQA, CAQA’s financial and administrative work was performed by the 
MoESTD. The financial account of CAQA was attached to the Ministry and its revenues and 
expenditure was attached to the budget of the Ministry. As an independent legal person NEAQA has 
its own account. The Ministry still provides it with premises in a large government building free of 
charge, and has provided seed funding of about 150 000 euro to establish offices, acquire equipment 
etc., but does not plan to fund it further. NEAQA is meant to be funded solely through fees which were 
thus increased to about 15 000 euro for initial institutional accreditation, 5000 euro for periodic 
institutional accreditation or audit and 2700 euro for programme accreditation, with small variations 
in price for different types of institutions and programmes. As it started being fully operative only in 
the second half of the year, NEAQA was left with a surplus approximately equal to government funding 
in 2018, and plans to use this to sustain itself in 2019 according to the published financial plan.

15/70 
 FINDINGS: COMPLIANCE OF NEAQAWITH THE STANDARDS AND GUIDELINES FOR QUALITY ASSURANCE IN THE EUROPEAN HIGHER EDUCATION AREA (ESG)ESGPART 3:QUALITY ASSURANCE AGENCIES
ESG 3.1 ACTIVITIES, POLICY, AND PROCESSES FOR QUALITY ASSURANCE 
Standard:  
Agencies should undertake external quality assurance activities as defined in Part 2 of the ESG on a 
regular basis. They should have clear and explicit goals and objectives that are part of their publicly 
available mission statement. These should translate into the daily work of the agency. Agencies 
should ensure the involvement of stakeholders in their governance and work. 
2017 review recommendation: The panel recommends that CAQA (1) revise its mission so that it 
clearly defines the range of its responsibilities, principles underlying its work and the nature of its 
interaction with stakeholders; (2) put in place mechanisms for effective forward planning and 
reviewing progress towards its objectives; and (3) take action, insofar as it is possible within its remit, 
to ensure that students and employers have their representatives in CAQA governance. 
The panel also suggested that CAQA could establish an advisory body involving international experts. 
Evidence 
The 2017 LoHE established the National Accreditation Body “for the purpose of performing the 
accreditation tasks, the assessment of quality of higher education institutions and the units therein, 
evaluation of study programmes and assurance of quality in higher education” (Article 14.) The LoHE 
further defines the external QA activities of NEAQA as to perform external evaluation of quality of 
higher education institutions, accreditation and initial accreditation. This is repeated in the February 
2018 Government Decision on the establishment of the National Entity for Accreditation and Quality 
Assurance in Higher Education, and served as the basis for the NEAQA 2019 Strategy which defines 
NEAQA mission as follows: “NEAQA’s mission is to, in cooperation with the key stakeholders, maintain 
and enhance the quality of higher education in Serbia in accordance with the international standards 
and thereby increase its competitiveness.” The Strategy also defines its vision “to become the main 
driving force for QA development in the Western Balkans and significant partner in EHEA” and values 
of “independence, accountability, transparency, professionalism, efficiency, cooperation, accessibility 
and commitment”, while SAR (p. 15) lists as core values “independence, integrity, competence, 
transparency and openness.”  
The basic relationship with stakeholders is defined by LoHE, which established NCHE, the National 
Council, as the body ensuring the development and enhancement of QA of HE in Serbia, and the Board 
and CAQA as NEAQA bodies that include stakeholder representatives. In addition to academics as HEI 
representatives, LoHE also lists Chamber of Commerce representatives as NCHE and the Board 
members. Students are not defined as members of either body by LoHE, but the Law does prescribe 
that two students with an average grade of eight or more, appointed by the two student conferences, 
will be invited to NCHE to “take part in the issues significant for them”. As confirmed during the site 
visit, the Board members also decided to invite a student representative to participate as an associate 
member even though this is not expected by the LoHE nor the NEAQA Statute. CAQA, on the other

16/70 
 
hand, is defined by LoHE as composed exclusively of academics. While both CAQA and the Board are 
supposed to be appointed with a view towards gender balance, there are only 2 women in the Board 
out of 7 members.  
In addition to the representation in the NEAQA bodies, stakeholders are involved through surveys (the 
most recent survey among professional HEIs and NEAQA experts is cited in SAR; see ESG 3.6 on the 
timing of the surveys) and meetings, as confirmed by HEI representatives during the site visit. The 
LoHE does not require students or employers to be involved as reviewers, however this is prescribed 
by the 2019 regulations on standards and procedures which have the legal status of bylaws.   
The 2019-2022 Strategy of NEAQA is primarily focused on NEAQA’s compliance with the ESG, and its 
short-term objectives are all devoted to “offsetting shortcomings” on ESG standards, to be 
implemented according to the Action Plan mostly in 2019 with only few planned for 2020 or later. 
Regarding ESG 3.1, the Action Plan included workshops with stakeholders aimed towards defining the 
new mission, development of a plan to monitor its implementation with external experts, and special 
workshops for newly-involved student and employer reviewers to evaluate and enhance their 
participation. By the time of the site visit only four common trainings for experts took place. As a public 
institution, NEAQA is obliged by LoHE to report to Government once a year on its activities and 
achievement of its objectives.  
As noted, NEAQA is in a period of transition, and as CAQA was not legally able to finish procedures or 
start new ones, it produced a backlog of more than three hundred accreditations NEAQA has had to 
deal with. By the time of the site visit it managed to complete most, and CAQA members were 
confident that the backlog would be cleared no later than the 1st half of 2020 if counting on delays 
connected to the difficulties in involving foreign experts. Five pilot accreditations were completed 
according to the new procedures at this point, 2 of which in addition to program accreditations also 
included institutional accreditation. NEAQA does not engage in other types of work, such as 
consultancies.   
Analysis  
 The Strategy Action Plan provided for workshops for effective forward planning and reviewing 
progress, however there was no evidence that these took place, and overall the action plan lacked 
clear, qualitative indicators to monitor progress (see also ESG 3.6). This is not surprising when 
considering that, as noted, NEAQA is a newly established body, and had to focus on developing and 
implementing new procedures and preparing for this review. It should also be added that considering 
the Action Plan provided, the Strategy only refers to the 2019-20 period.  
As recommended by the 2017 review, NEAQA has revised its mission statement to put focus on 
stakeholders, the regulations were amended to include employers and students in review panels, and 
a student member was included in the work of the Management Board. However, there seems to be 
some confusion over the goals of NEAQA’s work. The 2017 panel identified as CAQA’s mission and the 
sole objective of accreditation procedures “to assure adequate conditions (officially recognised 
qualifications held by staff, space and facilities) for higher education” (p. 17) and recommended CAQA 
to focus more on developing quality through the audit procedures. The 2019 SAR emphasises 
reorientation towards developing the quality culture, and the panel was informed at the site visit that 
NEAQA wished to make a break from the former CAQA’s image of a controller. There is however no 
evidence that such a change took place. Namely, while changing important aspects of the 
methodology (as discussed under other standards), NEAQA essentially kept the standards and 
conditions existing from before its establishment, for both accreditations and audits (as discussed in

17/70 
 
more detail under ESG 2.1), thus keeping the national minimal conditions prescribed by LoHE as the 
key standards in accreditation. Furthermore, no audit procedures have been implemented since 2017. 
It is thus not possible to confirm that the mission of “enhancing quality in accordance with 
international standards” is currently translated to the daily work of the agency. The Strategy 
recognizes that there is room for improvement, noting that in addition to formal participation and 
occasional surveys, more quality-oriented work needs to be introduced, including smaller workshops. 
At the same time, several stakeholders at the site visit noted that the purpose of NEAQA was precisely 
to perform better control than its predecessor which had been overtly lenient. In the view of the panel, 
it is acceptable that the level of trust in the HE system is such that a modicum of control is needed, 
and it is possible that HEIs which have proven a certain level of maturity of their internal quality 
assurance system do not need as much control as those that have just been established. The issue is 
not that NEAQA should give up all control, but that there is currently a wide gap between the 
proclaimed target of moving from a control-based to enhancement-based approach, and the reality 
on the ground.  
The Strategy notes that the LoHE should be changed to assure that the governing bodies, notably 
CAQA, also include employer and student representatives, and that an independent appeals body is 
established. However, the desired legal changes would still put great emphasis on academic 
excellence of CAQA members. The proposed changes foresee that the employers could be involved 
only if they had a scientific master (an advanced postgraduate degree) and only doctoral students 
would be able to participate. The changes also do not propose that a student or an employer would 
be a member of the appeals committee. These proposed legal changes were emphasised by the Board 
members during the site visit, however, the competent Ministry has not given any guarantees that 
they would be taken into account or implemented. The focus on the (uncertain) legal changes could 
also be one of the reasons the Board did not focus on the changes they could themselves build in the 
Statute and the procedures of appointing CAQA members to assure student (and employer) 
participation.  
Students seem to be well aware of the importance of accreditation due to the socially recognised issue 
of fraudulent degrees and possibly also thanks to the list of accredited programmes CAQA has been 
publishing for several years now. The student representatives and reviewers are well aware of the 
power and the importance of their roles. However, students have only recently been involved as 
stakeholders and they report that there is still little awareness of the role and meaning of quality 
assurance among their peers. While this may be improved through the growing number of 
accreditations involving students, this should not be taken for granted. NEAQA would do well to 
implement its plan to organise more student workshops, and work harder and more effectively in 
making its work known and recognized. Student organisations have the potential to significantly aid 
this work through cooperation with their peers from other parts of Europe as well as their involvement 
in IQA at their HEIs.  
Starting from the 2017 review report, it is possible to follow how during the past decade the various 
groups of stakeholders have been organised and recognized as stakeholders in the Serbian HE QA 
system – starting with universities and their students, followed by professional HEIs and their 
students, and finally employers. The Chamber of Commerce, as stated during the site visit, managed 
to participate in legal changes and include its members in NCHE and the Board. However, the timing 
of appearance of the stakeholder groups is still reflected in the composition of the NEAQA bodies, and 
is also connected to the issues of trust in the newly established HEIs mentioned at the beginning. The 
majority of the Board, NCHE and CAQA academic members come from old public HEIs, and there is no 
SCAAS representative in the Board. It is true this presents a good approximation of the system (more

18/70 
 
than 2/3 of students study at such HEIs), that several stakeholders have mentioned that the status 
and prestige of such HEIs help support public trust in NEAQA and the whole system, and that there 
was no complaint of this by the stakeholders during the site visit. Still, the governance of the agency 
should reflect the diversity of stakeholders and of the HE system and avoid being dominated by a small 
number of institutions, regardless of their prestige and importance. Rather than lowering the quality 
standards, this could help in raising the quality of the whole system, and adapting the instruments and 
procedures to the needs of various institutional profiles. This is especially important in a binary system 
like Serbian which includes institutions as varied as comprehensive research universities, small 
universities oriented towards the local labour market, and small private institutions. The robustness 
of procedures needs to be emphasised as a guarantee of trust, and this could be aided not only by 
more comprehensive student involvement in both EQA and IQA, but also by much stronger 
involvement of foreign experts and, indeed, an establishment of an international advisory board. Such 
a board could be especially helpful in the transfer from quality control to quality culture as its 
members could not only participate in the discussion of NEAQA internal stakeholders but also provide 
support to international cooperation in training and developing HEI’s internal QA systems.    
Panel recommendations 
The agency should, in cooperation with the stakeholders, develop a new strategy which would enable 
the implementation of its mission of enhancing quality of Serbian HE in line with the international 
standards while keeping a level of control that they find is needed in their system.  
Even within the present legal constraints, the agency should strive towards involving all stakeholders, 
and particularly students, as full members in all of its bodies. The agency should use its international 
connections to also more actively encourage the participation of international experts in its bodies.  
The governance of the agency should reflect the diversity of stakeholders and of the HE system and 
avoid being dominated by a small number of institutions, regardless of their prestige and importance. 
Panel suggestions for further improvement 
NEAQA would do well to implement its plan to organise more student workshops, and work harder 
and more effectively in making its work known and recognized. Student organisations have the 
potential to significantly aid this work through cooperation with their peers from other parts of Europe 
as well as their involvement in IQA at their HEIs. 
Panel conclusion: substantially compliant 
ESG 3.2 OFFICIAL STATUS  
Standard: 
Agencies should have an established legal basis and should be formally recognised as quality 
assurance agencies by competent public authorities.  
Evidence 
The 2017 LoHE establishes NEAQA “for the purpose of performing the accreditation tasks, the 
assessment of quality of higher education institutions and the units therein, evaluation of study 
programmes and assurance of quality in higher education” as a legal entity (Article 14) to issue 
decisions in audits and periodic accreditations, and provide reports as a basis for licensing in initial 
accreditations. It then establishes the Accreditation Commission (CAQA) as the NEAQA professional 
body (Article 20) to “conduct the procedure of accreditation of higher education institutions and the

19/70 
 
study programmes, as well as the procedure of external evaluation of quality of higher education 
institutions,” propose review panels (from the list compiled by the National Council) to the NEAQA 
director, and decide on the outcomes of the accreditation and audit procedures. LoHE stipulates the 
accreditation and audit procedures as obligatory for awarding Serbian HE qualifications. It does not 
allow for other bodies to perform these procedures and it does not contain provisions on the 
possibility of NEAQA operating in other HE systems.  
Analysis  
NEAQA was formed by LoHE as a legal entity, and CAQA as its professional body, which is a clear 
improvement to the previous system when CAQA functioned as a body of the National Council which 
is itself a body of the Government. There is no doubt that NEAQA has clear legal basis for its 
operations, and the site visit confirmed that the competent Ministry and all other stakeholders 
support its legal status as the only body responsible for EQA procedures in the country.  
Panel conclusion: fully compliant 
ESG 3.3 INDEPENDENCE 
Standard: 
Agencies should be independent and act autonomously. They should have full responsibility for their 
operations and the outcomes of those operations without third party influence.  
2017 review recommendation: CAQA should be provided with its own bank account to reduce its 
dependence on the Ministry of Education in administrative terms which has significant impact on 
CAQA’s planning and management. For the recommendation on the appeals procedure, see ESG 2.7. 
Evidence 
As already noted and as recommended by the previous review, LoHE turned CAQA into a professional 
body of NEAQA which is a legal person, self-funded and with its own bank account. The LoHE, the 
standards adopted by the National Council nor the NEAQA Statute do not explicitly proscribe 
independence of NEAQA apart from establishing it as an independent legal person. LoHE establishes 
a clear division and sharing of responsibilities between the National Council, NEAQA and its bodies –
CAQA and the Management Board, which can be repeated here, as follows.  
- 
Appointment of both the Management Board and the National Council is done by the 
Government. Four out of seven of the Management Board members are selected by 
stakeholders (two by HEI conferences, two by Chamber of Commerce) and three are selected 
by the Ministry; the Board then decided to also include a student representative even though 
the documents do not proscribe so. Regarding the National Council, out of its seventeen 
members, eight are selected by HEI conferences, two by the Chamber of Commerce, and 
seven by the Ministry, while it is specified that they have to be well-recognized academics or 
artists. LoHE proscribes that the selection of all of the National Council appointees is based on 
a public call and lists of all applicants need to be published prior to the selection; in making 
the selection, the HEI conferences need to take into account the size of the HEIs from which 
the applicants are coming, to make their selection statistically representative.  
- 
Selection and appointment of the decision-making body, CAQA, is the task of the Management 
Board, which, as confirmed at the site visit, chooses from the longlist provided by the National 
Council on the basis of a public call, and can also ask for the longlist to be amended. The 
Management Board also selects, on the basis of the public call, the NEAQA director, who signs 
all official decisions, including those on accreditation.

20/70 
 
- 
Selection of experts is done by CAQA, which is able to choose from the list kept by the National 
Council and suggestions done by the Chamber of Commerce and student conferences. 
- 
LoHE specifies that the National Council, CAQA, Management Board and NEAQA cannot 
employ or have same people as members, that their staff and members cannot serve as 
experts; and that neither of them can include people performing political functions.  
- 
The minimal standards set by law are then further developed and combined with the ESG into 
the procedural standards suggested by CAQA and adopted by the National Council. The 
NEAQA Statute, passed by the Management Board, further specifies that the standards will 
be developed by CAQA and adopted by the Board before submission to the Council. At the 
site visit it was confirmed that the current standards were done in cooperation between CAQA 
and the National Council, with National Council commenting primarily the clarity of the 
standards.  
- 
In accreditation and audit procedures, the decisions are suggested by expert panels; on the 
basis of this, CAQA issues formal decisions which are signed by the Director. The name in 
Serbian and the status of the formal decisions are aligned with the Administrative Procedures 
Act, and vary from one procedure to another, however the difference is not substantial. The 
decisions are submitted to the Ministry which transforms them into final, legally binding 
decisions.  
- 
CAQA independently selects its president and passes its rules of procedure, which determine 
the method of decision making – according to the current version, they decide by majority 
vote on the basis of a suggestion of the sub-commission of its experts for the discipline in 
which a HEI or a programme is placed. Depending on the discipline, the sub-commissions have 
two to five members.   
- 
There was some confusion among the agency stakeholders regarding the difference between 
complaints and appeals; finally, it was established that these are basically defined by the 
Administrative Procedures Act which proscribes that complaints are submitted to the director 
during the procedure, and appeals on formal decisions need to be submitted to a second-
instance body. LoHE specifies the National Council as the second-instance body for NEAQA 
decisions which can upon the first appeal, return the decision to NEAQA to repeat the 
procedure, and upon second appeal, overrule the decision if it deems necessary (more on this 
under ESG 2.7).   
The Management Board adopts the Code of Ethics which applies to all NEAQA staff and experts, 
and prohibits people currently employed at a HEI or cooperating with a HEI to participate in its 
accreditation procedures.  
Both the SAR and the NEAQA Strategy emphasise independence as the core value to be upheld, 
and commitment to independence of the outcomes was also emphasised during the site visit by 
all stakeholders. Additionally, NEAQA has emphasised in the SAR that even though it is satisfied 
by the provisions of LoHE, it would like to have the ability to adopt its own procedures and solve 
its own appeals without the involvement of the National Council. At the site visit it was confirmed 
that even though in practice the National Council limited itself to asking clarifications both during 
the adoption of procedures and when discussing the appeals, NEAQA is actively proposing that 
the Ministry changes this legal provision (the Ministry does not have a formal plan to do so at this 
point).  
Analysis

21/70 
 
Compared to the analysis in the last ENQA Review Report from 2018 significant steps have been taken 
to strengthen the independence of NEAQA on all levels, e.g. concerning the organisation, procedures, 
outcome and finances.  
Regarding the organisational independence of NEAQA, while there is no written evidence or oral 
evidence that challenges on the independence of the agency exist, the panel does agree that there is 
room for improvement in the legislation and that some clearer structures and regulations would be 
needed to create mutual trust, transparency and comparability on the European level. What causes 
concern in the opinion of the panel is, first, the fact that the ministry appoints seven NCHE members 
and NCHE nominates, through the Management Board, all seventeen members of CAQA. From the 
perspective of the stakeholders in Serbia the structure described above guarantees that there are 
checks and balances in the system– especially between the two expert bodies, National Council and 
CAQA – which prevent any single body from adopting an autocratic approach. However, from the 
perspective of the panel, such a structure cannot fully assure that no government influence is taken 
on the work of CAQA and its panels. Financial independence should also be discussed here, as NEAQA 
is supposed to be fully funded from the fees it charges from HEIs (as discussed in more detail under 
ESG 3.5), however it is not clear at this point, according to the information from the site visit, if it will 
be able to sustain itself in this view, or it would be forced to ask for additional funding from the 
Government.  
Furthermore, while the LoHE is focused on preventing anyone involved with politics to participate in 
the bodies in the system, it assumes that most members will be staff and students of Serbian HEIs, 
that the majority will come from universities, and even contains provisions that the size of HEIs should 
be taken into account. Such a system results in the fact that the vast majority of all members of NEAQA 
bodies come from the largest public universities (for example, 4 out of 8 Board members come from 
the University of Belgrade). Again, this is seen by the stakeholders as a guarantee of trust, as the 
largest public universities are widely considered to be the best HEIs in the country, while at the same 
time there is little trust in smaller and private HEIs. CAQA Rules of Procedure and NEAQA Code of 
Ethics also prevent experts from evaluating or deciding on HEIs they are connected with, and states 
that they are personally responsible for the decisions and recommendations they make. However, the 
panel finds that such a system of proportional representation does allow for undue influence of the 
largest public universities. It should also be noted that most activities require active participation of 
the director, who does seem very committed to the work of the agency, but still remains employed as 
a full-time professor at the University of Belgrade, working for the agency as a contractor. There is no 
evidence that this has an impact on the agency procedures. Still, such a practice should be changed 
and adapted to the common international practice in which the director, as the key staff member, is 
fully employed by the agency, rather than one of the institutions it is in charge of. This is especially 
important as it is the role of the director to sing all NEAQA decisions, and thus participate in 
establishing the formal outcomes of its procedures.  
A specific area of concern is that in the ENQA review in 2012 as well as in 2017 it was stated that the 
National Council should not be in charge for establishing CAQA and be the appeal body to CAQA’s 
decisions and that it would be better to establish a separate appeals body. This has not taken place so 
far, and the appeal procedure continues to differ from the practice of other European agencies and, 
consequently, damages the independence and integrity of the core body of NEAQA.  
Regarding the operational independence, again, it is the National Council that adopts the procedures 
and compiles the list from which NEAQA chooses academic experts. Again, arguments can be made 
that this provides for a well-balanced system, with the National Council as a parallel, independent 
expert body to CAQA. Still, operationally it means that NEAQA always needs to prove the need to

22/70 
 
change its standards and methods to the Council, and that it cannot choose experts ad-hoc, which 
could limit its potential to engage more foreign experts.  
All of the above reflects on the independence of formal outcomes, and in this regard it is especially 
worrying that the appeals procedure enables the National Council to ultimately overrule NEAQA’s 
decision.  
In conclusion, it does seem that members of the Management Board as well as CAQA are devoted to 
their role of building up the new agency NEAQA. Nevertheless, especially the independence of these 
two core bodies of the agency, as well as the director, needs to be supported. On one hand it would 
be beneficial not only to have a public call for the application of the members of CAQA but to develop 
selection criteria for them which are published in advance. For both bodies it is also true that the few 
largest HEIs are more actively involved in the agency than others. But for the future success of the 
accreditation system and its acceptance it will be crucial that the diversity of the Serbian system is also 
represented in the agency. It has to be an ongoing effort of NEAQA to strengthen the independence 
of the agency vis-à-vis the different stakeholders, namely the government and the major HEIs. 
Panel recommendations 
In addition to involving more experts from outside the system (see ESG 3.1), the independence of the 
agency from HEIs as well as the government can be strengthened by the following steps, which need 
to be taken in cooperation with all stakeholders in the system.  
(1) To uphold the operational independence of the agency, define the criteria for membership in the 
bodies of the system which would focus on a balance of personal experiences and motivation, in 
addition to the proportional representation of the system and formal criteria defined by LoHE.  
(2) To uphold the organisational independence of the agency as well as the independence of formal 
outcomes, ensure that the Director is a full-time position with no contractual obligations towards HEIs 
within the system.  
(3) To uphold the organisational independence of the agency as well as the independence of formal 
outcomes, a clearer appeals procedure should be defined (see ESG 2.7).  
Panel conclusion: Substantially compliant 
ESG 3.4 THEMATIC ANALYSIS 
Standard:  
Agencies should regularly publish reports that describe and analyse the general findings of their 
external quality assurance activities.  
2017 review recommendation: CAQA should produce regularly thematic analyses focusing on quality 
and internal quality assurance, in addition to those already available and planned which address 
quantitative aspects. 
Evidence 
While CAQA produced a number of thematic analyses which were considered in the 2017 review, 
NEAQA has not had the time to produce any, as also acknowledged in the SAR (p.30) and during the 
site visit. This reflects back to the timing of the review and the fact that few procedures have been 
implemented by NEAQA. The Strategy foresees the establishment of a special Research Department 
that would produce them, and the attached Action Plan contains a “strategic area” devoted to the

23/70 
 
production of thematic analyses. The objectives within include “creating a plan for thematic analyses” 
and “analysing the procedures and the decision-making,” to be done by the end of 2019. In addition 
to the meta-analyses of NEAQA’s work, it can be concluded from the Strategy that the topics of 
interest will be IQA, quality culture and excellence in HE. The stakeholders at the site visit did not 
mention more specific plans to produce analyses, but did mention that they plan to do quantitative 
analyses of e.g. outcomes per fields. Upon the request of the panel during the site visit, NEAQA 
submitted a short list of the analyses it plans to produce.  
It was also noted during the site visit that the agency plans to establish a specific department to deal 
with projects and research; these functions should serve self-reflection in conducting evaluation 
procedures. 
Analysis  
The 2017 panel was satisfied with the number of thematic analyses CAQA was publishing, however, 
they recommended that they focus more on the qualitative aspects and emphasise good practices and 
areas of improvement at the evaluated HEIs. The discussions at the 2019 site visit left this panel with 
the impression that the stakeholders do not see the full potential of thematic analyses. While there is 
no doubt that the agency plans to produce them, as also shown by the table above and indicated in 
the Strategy, the plans remain vague and there are no guarantees that the 2017 recommendation will 
be implemented.  
It should be emphasised that in addition to helping the agency and other stakeholders to adopt a more 
reflexive attitude regarding accreditation and QA and the potential and complexities of those 
activities, the thematic analyses could also have the potential to fight the often-mentioned 
complacency in the HE system by encouraging self-reflection and promoting innovations. By 
identifying trends and promoting good and innovative practices in areas such as pedagogical 
innovation, teaching and assessment practices, attention to employability, curriculum patterns and 
changes, etc.; rather than focusing on compliance and excellence, they could spread good practices 
throughout the system and thus help develop the quality culture.  
Panel recommendations 
The panel recommends NEAQA not only to start regularly producing thematic analyses, but to use 
them as a tool for self-reflection (see ESG 3.6) and as a tool to promote good and innovative practices 
that can be spread throughout system.  
Panel suggestions for further improvement  
It would also be helpful to reflect what kind of information is needed by the HE system, and which 
information can be provided on a solid basis by an agency. Once the concept of thematic analyses is 
developed, a project plan should be put in place on how to put it in practice. For this it is not crucial 
to establish a specific department, but rather clearly and transparently establish responsibilities for 
producing thematic analyses within the existing structure. 
Panel conclusion: partially compliant 
ESG 3.5 RESOURCES 
Standard:  
Agencies should have adequate and appropriate resources, both human and financial, to carry out 
their work.

24/70 
 
2017 review recommendation: CAQA should enhance its resources planning and management to 
ensure that it makes best possible use of the resources available. See also the related 
recommendation about a bank account under ESG 3.3. 
Evidence 
Due to the fact that NEAQA was reorganized in 2018 the resources, finances and staff changed 
compared to the previous review. The agency now has a separate bank account, and the Managing 
Board adopts the annual programme of work and the financial plan upon the consent of the 
Government. The panel was able to check the Financial Report for the period between July 2018 and 
December 2018 and the Financial Plan for 2019. Major expenses refer to the allowances for the Board 
and CAQA members as well as expert fees, implementation of the IT infrastructure and the 
development of a new IT system, which was also emphasised in the Strategy and by stakeholders 
during the site visit. As the system is still under development, the panel was not able to study it – but 
its main purpose seems to be to enable electronic submission of the detailed information and tables 
used in accreditations. The offices of NEAQA are now in another building, but still provided by the 
Government without costs.  
 
NEAQA is financed from the income generated by fees for accreditation and audit charged from the 
HEIs. Because the fee for programme accreditation was raised, there were complaints from large 
public universities which normally have a large number of programmes, and this was a topic at the 
site visit. The Ministry provided seed funding last year, but does not have plans to provide further 
funding. Instead, NEAQA plans to use internationally-funded projects to supplement what it earns 
from fees. At the site visit, the crucial stakeholders agreed that currently there are only short-time 
plans and that there can be no certainty that NEAQA would be able to break even if relying solely on 
funding from fees, as there is still little information on the full costs of its work.  
NEAQA managed to take over some of the staff members from CAQA which can transfer some 
experiences with the old system, and to gain new, competent staff members with an international 
background. It now relies on nine full-time staff, seven of which work full-time with evaluations 
directly, while the other 2 provide technical support. The agency also hires external support for 
finances and IT. There are plans in place to acquire additional offices in the building and thus create 
conditions to employ more staff – the management mentioned the possibility of employing up to five 
more people.  
Analysis  
The agency managed to become financially independent from MoESTD and there seems to be no 
reason for concern in the starting phase of the new agency, as the resources seem to be sufficient for 
the procedures which took place in 2018/19. However, the real costs of the new procedures have not 
been established yet, as only 5 were completed by the time of the site visit. Additionally, it was 
mentioned at the site visit that because the price of programme accreditation was increased, HEIs 
might be trying to combine several programmes into one programme with modules to decrease 
accreditation costs. Thus the long-term financial sustainability will be an issue for the management of 
NEAQA and it is one of their core tasks to develop a more predictable financial model which allows 
long-term-planning. Moreover, priorities have to be set in spending the finances on priorities which 
are in line with the Strategy. While the panel understands that both financial planning and reporting 
are done in line with the regulations, in the future it would probably be useful to develop more 
detailed plans and reports which would enable both the agency and the public to understand full costs 
of each type of the procedure, to differentiate between fees to various bodies, experts and staff, etc.

25/70 
 
This is of special relevance as the agency would probably do well to streamline the fees it charges from 
HEIs by, namely by making visits more efficient (e.g., with clusters of programs in the same field). 
It is crucial for the success of an agency to have a sufficient number of qualified staff. The number of 
staff was sufficient for the limited number of procedures and those the panel met were motivated 
and committed. Considering that the number of site visits will increase significantly in 2020, the need 
for more staff members with experiences in internal and external QA is obvious. Additional staff would 
enable NEAQA to implement a number of necessary activities, for example:  offer more trainings for 
experts on external QA and for HEIs on internal and external QA; develop projects and thematic 
analyses; devote more time to reflection on the existing procedures, etc., which would all be beneficial 
for the whole accreditation system in Serbia.  
Additionally, while the staff seem very enthusiastic, the system cannot run on enthusiasm for a long 
time, and long-term methods of keeping staff motivated and qualified need to be considered. In this 
regard, one of the core tasks of NEAQA should be to develop a concept for staff development and to 
allow them to take part in trainings which are relevant for QA agency staff on national and 
international level, including trainings and international exchange on QA methods as well as generic 
skills (e.g. communication trainings, project management). Regarding this, it has to be emphasised 
that these are of equal relevance for existing as for new staff members. Funds for staff development 
have to be provided in the next years to assure that there is a continuous process of upgrading staff 
competences. It is possible that this would require additional state funding.  
Panel commendations 
The panel wishes to commend the agency for hiring enthusiastic and competent staff.  
Panel recommendations 
The agency should establish full costs of the procedures, in order to streamline them if possible and 
thus reduce the overall costs to HEIs, but also to establish long-term financial plans which would 
enable it to acquire timely additional support from the state budget if necessary.   
The agency should establish a human resources development plan which would provide not only the 
criteria and plans for employing new staff, but also training and development activities for both 
existing and new staff.  
Panel suggestions for further improvement 
Strengthen the international exchange of staff.  
Panel conclusion: substantially compliant 
ESG 3.6 INTERNAL QUALITY ASSURANCE AND PROFESSIONAL CONDUCT 
Standard:  
Agencies should have in place processes for internal quality assurance related to defining, assuring 
and enhancing the quality and integrity of their activities. 
2017 review recommendation: CAQA should (1) devise an action plan for the development of its 
internal quality system; and (2) put in place formal mechanisms for gathering external feedback after 
each accreditation review and audit and internal feedback on a regular basis, and for following up on 
internal and external feedback collected. 
Evidence

26/70 
 
The Code of Ethics defines rules of conduct, and cases considered, and sanctions for, a breach of the 
Code. It also provides for the establishment of an ad-hoc Ethics Committee to deal with a breach. It 
explicitly prevents nepotism, undisclosed financial gain from participating in the procedures, and 
defines evaluating or deciding on an evaluation of a HEI one is currently working with as a conflict of 
interest.  
 
According to the SAR, the previous CAQA has collected external feedback through surveys among HEIs 
(2011, 2015 and 2017), students (2011 and 2015) and academic experts (2017) and NEAQA plans to 
continue with this practice. Feedback on new procedures was collected from HEIs and reviewers 
participating at workshops and the analysis of the response was included in the SAR. Questionnaires 
were made for the purpose of collecting the opinions of stakeholders regarding the changes of 
regulations and efficiency of NEAQA, and a version of the questionnaire to be used in the future was 
shown to the panel. The questionnaire focuses on the panels’ and the HEI’s satisfaction with the 
changes in the procedure, and the version for panel members has additional items for evaluating the 
work of the other panel members. However, according to the information from the site visit this 
questionnaire was not sent to the HEIs nor the panels participating in the panel procedures, and there 
is no evidence that the agency will use this systematically as a regular practice in the future.  
 
The SAR also includes a SWOT analysis, and the Strategy provides further self-reflection of the agency. 
The SAR (p. 32) states that throughout the pilot projects the members of panels were asked to give 
their opinions and comments regarding the new procedure, with insights of labour-market 
representatives considered particularly important as they were the last group to be included in the 
procedures. The panel established that this was done informally, between the site visit meetings and 
according to some reports, even during the site visit meetings of the panel and HEI representatives. 
One of NEAQA’s actions planned to be conducted in the near future is the implementation of ISO 
standards: 9000, 20000 and 27000 in order to improve the quality of its activities and define principles 
underlying its work based on the PDCA cycle.  
 
Analysis  
Even though this was a 2017 recommendation, and even though we were told that there were plans 
to do so, the panel did not find sufficient evidence that the agency will regularly and systematically 
collect feedback on its procedures, i.e. send online questionnaires to both experts and HEIs once the 
procedure takes place. Even for the pilot procedures, the very purpose of which should be to test an 
approach in order to improve it, the feedback was collected only informally and during the site visits. 
Collecting feedback from everyone involved after implementing a procedure is an important segment 
of what should become a structured system of collection regular feedback in which it is important to 
combine different tools and use triangulation. At the moment there is a lack of systematic collection 
and analysis of feedback provided from these various participants in the process, and the majority of 
feedback is collected informally.  
 
A QA agency should be a learning organization, benefiting from the experience and insights from the 
various stakeholders, namely staff, reviewers and institutions. While an informal IQA system can be 
expected in young, small institutions, and while some input will always be coming through informal 
channels of communication, NEAQA might already be too large an organisation to collect feedback 
only in this manner. In addition to regularly sending out questionnaires and analysing the qualitative 
information they provide, the agency should triangulate by e.g. creating opportunities for ex-post 
evaluation discussions about its processes and how to improve them, such as internal commissions, 
especially given that it is in its initial steps and performing pilot reviews that are supposed to be 
evaluated and help to improve and adjust subsequent reviews. Emphasis should always be put on 
content and self-reflection, instead of e.g. grading the experience and producing charts. As this is a 
crucial aspect of the quality culture, and again as already noted by the 2017 panel, NEAQA would do

27/70 
 
well to give an example of promoting critical discussions and in this way turning informal practices of 
collecting feedback into a formal method of improvement. The information collected in this way 
should be communicated to the public in an appropriate form (e.g. aggregated results), together with 
the steps the NEAQA plans to take on the basis of the feedback. Such information can also be used as 
a basis for a thematic analysis. Communication to the public of this form is especially important in the 
current situation marked with a low level of public trust in the educational system.  
 
While it is of course necessary to adopt the PDCA cycle and introduce regular feedback collection and 
self-reflection, it might be questioned if full ISO certification would be beneficial considering the 
currently available human and financial resources. The process orientation of ISO is beneficial for the 
management of an agency but the effort necessary to introduce ISO in such a small organisation is 
high and NEAQA is in the panel’s view lacking staff to implement it. This might be an appropriate tool 
for a more mature organisation. There is a number of methods the agency can use to assure its own 
quality that are formal and structured, but not as resource-intensive like ISO certification, as noted.  
 
Panel recommendations 
NEAQA should introduce formal mechanisms for collecting feedback from experts and institutions 
after each procedure, collect more qualitative additional feedback, e.g. through focus groups, 
interviews etc. to reflect on the procedures, especially the pilot procedures and establish similar 
mechanisms to regularly collect feedback from internal stakeholders – staff and CAQA members. The 
feedback should be collected and analysed in a manner that enables the stakeholders to comment on 
what they find relevant, and that can be used by NEAQA to continuously improve its work and the 
framework in which it operates, and report this to the public.  
Panel suggestions for further improvement 
The agency can introduce annual workshops with staff (and possibly also separate workshops with 
CAQA and the Board) to give them the opportunity to reflect on its work and plan future steps.  
Panel conclusion: partially compliant 
ESG 3.7 CYCLICAL EXTERNAL REVIEW OF AGENCIES 
Standard:  
Agencies should undergo an external review at least once every five years in order to demonstrate 
their compliance with the ESG.  
Evidence 
Even though the regulations do not specifically require NEAQA to undergo external reviews, this is the 
agency’s third ENQA-coordinated review. The stakeholders at the site visit discussed ENQA 
membership and EQAR listing as important for the whole HE system and a basis of trust in the system.  
Analysis  
The panel confirms the commitment of the whole system to support further evaluations of the agency 
against ESG as an important aspect of belonging to the European Higher Education Area.  
Panel conclusion: fully compliant

28/70 
 ESGPART 2:EXTERNAL QUALITY ASSURANCE
ESG 2.1 CONSIDERATION OF INTERNAL QUALITY ASSURANCE 
Standard:  
External quality assurance should address the effectiveness of the internal quality assurance 
processes described in Part 1 of the ESG. 
2017 review recommendation: CAQA should (1) amend slightly its audit standards so that they 
embrace all aspects of ESG 1.1, 1.3, 1.4 and 1.7, and have a discussion with academic experts, students 
and employers participating in processes to arrive at a common understanding of how student-
centred learning and recognition should be addressed; (2) refocus audits more towards the 
effectiveness of internal quality assurance; and (3) provide greater support for HEIs to take the primary 
responsibility for quality. 
Evidence 
Minimal standards for accreditation are set by LoHE: the number and type of programs necessary to 
become a certain type of HEI; that at least 70% of classes need to be taught by full professors at 
academic programs (50% for arts programmes), and at least 50% by people with PhDs at professional 
programs; that a HEI needs to employ at least 20 full-time professors; division of ECTS for each type 
of programme; HEI governance; necessary resources, etc. These are then further developed by the 
National Council, in cooperation with CAQA, and listed in regulations (separate regulations exist for 
institutional and programme accreditation, for initial accreditation of both, for audit and for HEI self-
evaluation that precedes the audit) and then further detailed in lists of standards, published for each 
type of procedure on the NEAQA website.   
In the discussion of this ESG standard, SAR briefly repeated the analysis of the 2017 panel and included 
the table on the relation of standards to the ESG Part 1 from the 2017 panel report (slightly adapted 
to changes in the numbering of standards), copied below.    
Table 4: analysis of ESG part 1 and NEAQA standards (source: 2017 CAQA review report, with 
adaptations) 
Process   
Initial HEI accreditation 
and programmes  
Periodic HEI accreditation  
Initial 
and 
periodic 
programme accreditation*  
Self-evaluation as a basis for 
audit  
ESG 1.1 
Policy for QA 
Standards  
1 
 
(HEI’s objectives 
and 
tasks); 
12 
(IQA 
mechanisms) 
Standards  
1  
(Goals  
and 
objectives); 11 (IQA mechanisms) 
Standard 11 (Quality control) 
Standards 1 (QA strategy); 3 (QA 
system); Standard 2 (QA standards 
and procedures) and 8 (with regard 
to discrimination) 
ESG 1.2 
Design 
and 
approval 
of 
programmes 
Standards  
3 
(Programmes/Studies); 4 
(Research) 
Standards  
4 
(Programmes/Studies);  
5 
(Research) 
Standards 
1 
(programme 
structure); 5 (curriculum); 6 
(quality, 
modernity 
and 
international compatibility); 
15 (distance learning) 
Standards 
4 
(quality 
of 
programmes); 6 (quality of 
research/artistic/professional 
activities); Standard 3 with regard 
to 
student 
involvement 
in 
programme design. 
ESG 1.3 
SCL, teaching 
and 
assessment 
Standards  
3 
(programme/studies);  
8 
(students) 
Standards  
4 
(programme/studies);  
8 
(students) 
Standards 
4 
(Graduate 
competences); 5 (curriculum); 
8 (student assessment and 
progression) 
Standards 
4 
(quality 
of 
programmes); 5 (quality of the 
teaching process); 8 (quality of 
students)

29/70 
 
ESG 
1.4 
Student 
admission, 
progression, 
recognition 
and 
certification 
Standard 8 (students) 
Standard 8 (students) 
Standards 
7 
(student 
admission); 8 (student 
assessment and progression) 
Standard 8 (quality of students) 
ESG 1.5 
Teaching staff 
Standards 4 (research); 5 
(quality of teaching staff); 
6  
(requirements  
for 
teaching staff) 
Standards 5 (research and artistic 
activities); 6 (teaching staff) 
Standard 9 (teaching staff) 
Standards  
6  
(Quality  
of 
research/artistic/professional 
activities); 7 (Quality of teaching 
staff) 
ESG 1.6 
Learning 
resources and 
student 
support 
Standards 7 (Non-teaching 
staff); 9 (Facilities); 10 
(Library, resources and IT); 
11 (Funding); 
Standards 7 (Non-teaching staff); 
9 
(Facilities); 
10 
(Library, 
resources and IT); 12 (Funding 
sources); 
Standard 10 (organizational 
and material resources) 
Standards 9 (Quality of library 
resources and IT facilities); 10 
(Quality of HEI management and 
non-teaching support); 11 (Quality 
of facilities); 12 (funding) 
ESG 1.7  
Information 
management  
Standard 2 (Organization 
of HEI) 
Standards 
2 
(Planning 
& 
monitoring); 3 (organization & 
administration) 
Standard 11 (Quality control) 
Standards 3 (QA system); 10 
(Quality of HEI management and 
non-teaching support);Standards 1 
(QA strategy); 2 (QA standards and 
procedures) 
and 
Standard 
14 
(Systematic surveillance and 
periodic quality control) 
ESG 1.8 Public 
information  
Available after positive 
decision. NEAQA plans to 
publish short summaries 
of CAQA 2018 decision on 
the website in the future. 
Standard 13 (Transparency) 
2  
(Programme  
purpose); 
Standard 9 (Teaching staff); 12 
(Transparency: 
3rd 
cycle 
programmes) 
Standards 1 (QA strategy); 2 (QA 
standards and procedures); 4 
(Quality of programmes); 7 (Quality 
of teaching staff); Standard 5 
(Quality of the teaching processes) 
ESG 1.9  
On-going 
monitoring 
and 
period 
review 
of 
programmes  
Standard  
12  
(IQA 
mechanisms) 
Standard 11 (IQA mechanisms); 
Standards  
4 
(Programmes/Studies)  
and  
5 
(Research and artistic activities) 
Standard 11 (Quality control) 
Standards 1 (QA strategy); 2 (QA 
standards and procedures); 3 (QA 
system); Standards 4 (quality of 
programmes); 5 (quality of the 
teaching process);13 (role of the 
students in self-assessment) 
ESG 1.10  
Cyclical 
external 
quality 
assurance  
Periodic accreditation after 
one year required by 
LoHE 
Required by LoHE on every seven 
years 
Required by LoHE on every 
seven years 
Required by LoHE on every three 
years 
 
A comparison between the NEAQA standards and CAQA 2017 standards which preceded them with 
the standards evaluated in the 2017 report, reveals only minor changes were made, and that the 
standards have remained fundamentally unchanged since 2006 (when, according to the 2017 report, 
p. 28, they were first created). Thus, NEAQA still has separate standards for initial and periodic 
institutional accreditation, accreditation of first and second cycle programs, accreditation of third 
cycle programs in arts and in other disciplines, audit and self-evaluation of HEIs. ‘Audit standards’ are 
actually rules of procedure and instructions to the panel on how to consider self-evaluation standards 
during audit (and thus not included in the table above).  
The institutional accreditation standards focus on minimal conditions to perform studies, primarily 
resources (teachers, support staff, space, laboratories, library) and the content and existence of 
various governance mechanisms and documents required by LoHE. The standards in initial 
institutional accreditation are similar, but with lower requirements in terms of numbers. NEAQA staff

30/70 
 
noted at the site visit that they have tried to improve them by focusing on more qualitative aspects – 
e.g., checking teacher’s CVs in the context of the teaching they do rather than just counting them. In 
comparison to the CAQA 2017 standards, the current standards contain additional criteria in the 
chapter on students related to the social dimension, primarily the inclusion of students with 
disabilities and from minority groups. The standards for first and second cycle programmes do not 
differ much from those for the third cycle, nor there are significant varieties for different types of third 
cycle programmes. They all  look at the resources available to the program in addition to its content. 
Changes from 2017 include the addition of the criteria for: ‘interdisciplinary, multidisciplinary and 
transdisciplinary programs’ (to the list of separate criteria for every research area and arts); joint 
programmes; programmes in foreign languages; programmes leading to regulated professions in 
health; and resources and adaptations for students with disabilities. Audit standards were changed so 
that they no longer require the panel to evaluate space and equipment, non-teaching staff, 
governance and information to the public, which are still covered both by self-evaluation and 
accreditation criteria. As a supplement to standards, NEAQA publishes tables that HEIs need to fill in 
and guidelines for submitting SARs and panel reports. These ask for very comprehensive evidence, 
including administrative documents such as employment records, property documentation etc.  
SAR (p. 35) states that it plans to implement the 2017 recommendations and the Strategy includes 
plans to revise the standards in cooperation with stakeholders, as recommended by the 2017 panel, 
at the end of 2019 and 2020. It also states that, as expected by the 2017 report, audit will become 
more frequent and thus the focus will be on developing quality culture instead of control, and that 
trainings and other developmental steps will take place. However, no audit procedures have been 
implemented since 2017, as the third cycle of accreditations has just started and audits are meant to 
be implemented mid-cycle; the same is true of the planned analyses and trainings.  
As already noted above, during the site visit the panel discussed the manner in which the standards 
take into account issues of teaching and learning such as retention and SCL; most stakeholders did not 
consider these issues to be crucial in quality assurance. Only few HEI representatives and members of 
NEAQA management noted that QA should increasingly start focusing on student outcomes, 
progression, and teaching and learning.  
Analysis  
IQA at Serbian HEIs was primarily described at the site visit as a data-collecting exercise done once a 
year or once in two years. This is done by ad-hoc committees and groups, and only starting from this 
year HEIs would be able to employ specialised administrative staff to deal with QA. The difference 
between ‘low’ and ‘high’ quality HEIs in the system, it follows from on-site discussions, is in the level 
of compliance to the legal standards, e.g. number of teachers, publications etc.   
As can be seen from the table 4, it is possible to establish that all ESG Part 1 standards are 'covered' 
by various NEAQA criteria. The 2017 panel conclusion (p. 31) that self-evaluation (audit) standards 
cover the ESG Part 1 comprehensively, while the programme and HEI standards focus solely on details 
(e.g. regarding student-centred learning, the focus is on assessment), thus still stands. It follows from 
the 2017 report that already in 2012 the panel recommended CAQA to focus on quality culture and 
enhancement rather than controlling the minimal legal conditions, and thus enable HEIs to 
demonstrate their improvements and the way in which they implement the first part of the ESG. The 
same thing was again recommended in 2017, and this goal was supported by the SAR and the site visit. 
However, discussions with stakeholders have shown that there is little understanding in the system 
overall of what this would mean. Even though already the third cycle of checking the compliance with 
minimal conditions is under way, there is no trust in the system that these have been met by all HEIs.

31/70 
 
At the site visit, several stakeholders mentioned that a number of HEIs are run as for-profit businesses 
or even diploma mills. They listed various examples of cheating and playing the system such HEIs 
would recur to. Accordingly, it was noted that the increased fee for initial accreditation is meant to 
deter new attempts at establishing such HEIs. Most stakeholders seemed to agree that the way to 
solve these issues is to perform even more stringent controls and introduce novel control tools such 
as surprise visits to HEIs, within the limits posed by the institutional independence and the 
administrative law. Even though the HEIs complain that they have to submit very comprehensive 
administrative evidence to prove things such as owning their property, actually employing their 
teachers etc., they consider this necessary. The tables and guidelines attached to the standards further 
illustrate the low level of trust in the system. To give one example, the guidelines require that the 
number of students is calculated as the number of students admitted to the first-year times the 
number of years in the program – not even trying to ask for the HEI’s own records and the actual 
number of students. An understanding prevails among the stakeholders that any flexibility in the 
interpretation of standards and the method of application would lead to overall lowering of standards; 
and that focus on enhancement and support would mean that HEIs would no longer be required to 
meet minimal, quantitative requirements. At the same time, even though the standards seem 
demanding at the first glance, closer scrutiny reveals that most include exceptions. Additionally, in the 
old system, if these were not met the HEI normally got an opportunity to meet them in a period of 
time. The stakeholders at the site visit mentioned a number of times that this would no longer be the 
case and that NEAQA would remove the shady providers from the system which its predecessor did 
not. However, its predecessor had also been praised as the institution that purged the system, and 
there is still too little evidence to comment on this further.  
The strategic documents of the agency emphasise the responsibility of the HEI for the quality 
assurance of teaching and learning which is in line with the ESG philosophy. However, in practice it 
seems that it is up to the agency to prove that a HEI is not compliant with the standards, rather than 
giving the HEI an opportunity to show that it is. The system thus still manoeuvres between quality 
control as its original starting point and the aim in the view of most stakeholders, and an 
enhancement-oriented approach as the philosophy behind the ESG. Due to problems in the HE system 
in Serbia both approaches are reasonable, and, in the view of the panel, these are not conflicting 
approaches. The panel acknowledges that quality control is needed up to a certain extent, however, 
the panel finds that a system that focusses solely on the analysis of inputs and resources does not 
seem likely to achieve the stated objective of quality development. HEIs have obviously found ways 
to demonstrate compliance, and always checking and re-checking the same numbers and documents 
will not produce novel results. Instead, the focus should be on the robustness of both internal and 
external QA procedures, the effectiveness of HEI’s IQA, and the outcomes reached by students. 
NEAQA needs to define in which areas quality control is needed and integrate these aspects into an 
approach which targets quality enhancement. It will be one of the core tasks of the agency to start 
requiring HEIs to demonstrate their own improvement according to their own aims, and reflect on 
their needs and objectives in cooperation with stakeholders, rather than simply filling in pre-defined 
tables. The panel understands that such a reorientation would demand a cultural change and a gradual 
learning process among all stakeholders. The detailed 2017 panel’s recommendations to include 
various aspects of the ESG in the standards, such as recognition procedures or student-centred 
learning, could have easily been implemented even in the existing model through simple addition of 
criteria, but were not. Of course, this can be explained by the short period the agency had to 
implement the numerous procedural and organisational changes, as noted in the introduction. 
However, considering that the reorientation was first recommended in 2012, and that both the 2017 
and 2019 SAR state that this would be done without providing any concrete steps, it is also possible

32/70 
 
to argue that the agency has so far failed to take any steps to change this. At the same time, the 
conditions have changed, the agency is now independent and has competent full-time staff, and thus 
prerequisites exist for the change to take place, and the Strategy indicates that steps will be taken in 
the direction recommended by the panel.  
Panel recommendations:  
NEAQA internal stakeholders need to reflect on the 2017 recommendations and start a broad 
discussion with all stakeholders on how to implement them and start the reorientation of the whole 
system towards the development of the quality culture which presupposes institutional responsibility 
for implementing the first part of the ESG in its entirety, rather than a more lenient system of external 
control.  
Panel suggestions for further improvement:  
In addition to the recommended changes in the content of the criteria and standards, NEAQA can start 
experimenting with a less control-oriented approach by making several standards broader, removing 
the requirement to comment detailed criteria and fill in tables, and leaving just broad questions HEIs 
can respond to as they see fit and in line with their own internal practices. This is one way in which 
the agency can move the onus of proof to the HEIs, and enable them to demonstrate improvement as 
they see fit even without changing the law or fully revamping the standards.  
Panel conclusion: partially compliant 
 
ESG 2.2 DESIGNING METHODOLOGIES FIT FOR PURPOSE 
Standard:  
External quality assurance should be defined and designed specifically to ensure its fitness to achieve 
the aims and objectives set for it, while taking into account relevant regulations. Stakeholders should 
be involved in its design and continuous improvement.  
2017 review recommendation: CAQA should (1) devise, in cooperation with the national authorities, 
arrangements where its members do not combine the role of a body taking decisions with the role of 
key external experts, and where they focus on the former role in line with the spirit of the ESG; (2) 
place a stronger emphasis on quality improvement in its processes; and (3) amend its internal 
regulations to ensure full clarity and consistency. For a related recommendation about expert 
involvement, see ESG 2.4. 
Evidence 
The SAR repeats the finding of the 2017 report that the LoHE defines the purpose of accreditation as 
checking compliance with minimum standards and conditions laid down in LoHE (types and minimum 
number of programmes for the individual types of HEIs; number, qualifications, mode of employment 
and workload for teachers, etc.) while audits are intended to assess whether HEIs comply with their 
QA obligations, and are geared towards quality enhancement (p. 31). However, the wording of the 
2017 LoHE (which was not considered by the 2017 panel as it was adopted mid-evaluation) is different 
and states that both procedures are aimed towards checking compliance. Furthermore, the term 
‘audit’ is never used and the literal translation of the Serbian term would be ‘external 
control/evaluation of HEI quality.’ However, while the law envisages both procedures to be aimed 
towards quality control, in practice they also include enhancement-oriented elements. Both require

33/70 
 
the HEI to have a functional IQA system, for example, and to regularly improve the quality of their 
programmes.    
The difference in the aims of the two types of procedures is however visible from the standards 
themselves, which look at the same areas but take a different approach, as noted under ESG 2.1. While 
accreditation standards focus on the existing state of affairs, such as documents and resources 
available, in relation to pre-determined minimal conditions which are often quantitative, audits take 
a developmental, process-oriented approach. To give an example, in accreditation the criteria ask if 
the HEI’s teachers have no more than 12 classes per week, that no less than 70% of all classes are 
taught by teachers employed by the HEI, and that at least 50% of teachers have doctoral degrees. In 
audit, the criteria refer to institutional HR policies and the ways in which it monitors the quality of 
teachers’ work. No audits have been implemented since 2017, when the first cycle of audits was 
completed. Thus no audit reports or SARs according to the new audit procedure were available to the 
panel as evidence, and the SARs and reports from the pilot accreditations were available only in 
Serbian.  
The key change in the procedures implemented by the new LoHE and new procedures was to solve 
the systemic issue commented on by the 2017 panel and introduce independent expert panels, leaving 
CAQA only the decision-making. The regulations for all of NEAQA’s procedures now, without variation, 
prescribe that a five-member panel is formed. In all procedures the panel is composed by three 
academics from the list kept by the National Council, one student nominated by one of the two 
student conferences, and one representative of the labour market nominated by the Chamber of 
Commerce. Site visits have been introduced to all procedures, as well as the practice of publishing all 
the reports and follow-up documents (even though the publication of the reports on the NEAQA 
website only took place during the site visit, and SAR expected only report summaries to be published). 
The procedures are analysed in detail under ESG 2.3.  
The stakeholders were involved in the development of the new standards and procedures through 
their representatives in NEAQA bodies and the National Council, which adopted the standards in 2019. 
However, apart from ad-hoc surveys and thematic analyses done by the previous CAQA, and the 
activities related to the 2017 ENQA evaluation, there was no systematic stakeholder feedback to work 
with. As discussed under ESG 2.1, the changes to the standards were in any case minimal. And also as 
discussed under 2.1, both in the SAR and during the site visit NEAQA internal stakeholders emphasised 
the need to focus on quality culture rather than quality control, however the external stakeholders 
emphasised the need to better implement quality control. 
The system requires HEIs to undergo some type of external evaluation preceded by a SAR every 4 
years, with follow-ups and possibly also additional program accreditations; in addition to all this, the 
Ministry can always request a site visit or check in cases of concerns. The panel was of the impression 
that the burden on the HEIs is very high, and discussed this during the site visit. HEIs considered this 
to be unavoidable and expressed their satisfaction with the system, especially with the new procedure 
with an independent panel. The only complaints regarded the increased price of periodic program 
accreditations and the need to submit comprehensive documentation as evidence. 
Similarly, the panel found the criteria and the supporting documents confusing and not very user-
friendly, and, just like the 2017 panel, required explanations on when the programme accreditation 
takes place as separate from the institutional accreditation. It was explained at the site visit that this 
happens when a programme was initially accredited mid-cycle, and thus its periodic accreditation 
cannot fit the schedule of other accreditations done at the institutions. However, the stakeholders 
expressed satisfaction with the standards and their clarity.

34/70 
 
Regarding the possibility to implement the European Approach to QA of Joint Programmes, LoHE does 
not explicitly allow or forbid it. NEAQA has no procedural documents to implement it at this time, but 
the staff have informed the panel at the site visit that there is a joint programme in Serbia accredited 
according to the European Approach by a foreign EQAR-registered agency. NEAQA staff have 
commented that they planned to develop a procedure through which the decision made by the foreign 
agency would be accepted, thus also enabling NEAQA to adopt other decisions made by other agencies 
on the basis of the European Approach. However, at the time of the site visit no formal plans have 
been adopted to do so.   
Analysis  
This standard presents significant challenges for many agencies and this one is no exception to that, 
especially given the diversity of institutions it faces. On the one hand, the agency aims to ensure that 
all programs and institutions fulfil certain minimum standards and that implies applying similar 
standards to all of them. On the other hand, this may hinder the diversity in the system and leave all 
programs and institutions to conform to the same standards, regardless of their profile, mission, and 
specificities. The agency has tried to accommodate some diversity, especially regarding some 
specificities in the types of programs (University vs Applied Sciences; Artistic vs. Scientific). 
Nonetheless, it should devote more careful attention to this, namely regarding the way the standards 
are applied and to what extent its panels are not too homogeneous and too much influenced by 
certain institutional profiles and values. Moreover, certain types of institutions, notably the privately-
owned, may require an attention to aspects of ownership and governance, which may be specific and 
relevant to an adequate understanding of their decision-making process and actual engagement with 
QA.  
Even though only a few stakeholders emphasised this, the panel believes that the agency should take 
into account the financial strain put on institutions by fees of the high number of evaluation 
procedures. Currently existing system will require significant financial resources from HEIs and this 
was confirmed in the meetings with representatives of HEI conferences. This system does create a lot 
of workload and institutions would benefit from integration of the different activities. The panel is 
especially concerned that the number of procedures could lead to evaluation fatigue, and thus a 
superficial approach to evaluations which is already a risk in the system. At the same time, the panel 
understands that a number of stakeholders find such an involved approach necessary in this incipient 
phase of internal quality assurance systems, but again in the future the system can be adapted in a 
way in which not all HEIs need to demonstrate compliance and progress according to the same 
standardised schedule, but this can be made dependent on the robustness of their internal quality 
assurance procedures. And as NEAQA is well aware, this is not necessarily a proxy of HEIs ranking or 
history.  
Regarding the design of the procedures, while discussed under ESG 2.3, there is no doubt that in this 
regard the 2017 recommendation was taken into account and implemented. NEAQA has implemented 
major changes compared to the old accreditation procedure. The core process: SAR by HEI, site visit 
by an independent panel of experts (including a labour market representative and a student), decision 
by an independent body of the agency (CAQA) and follow-up is now in line with the ESG. There are 
some national specifics, e.g. that the preliminary report, drafted by the panel of experts, is written 
before the site visit takes place or that the panel is nominated by the director of the agency. CAQA 
takes a yes or no decision and gives no conditions but can give recommendations in all procedures, 
which is again a change from the previous system. Only in the old procedures which NEAQA is now 
completing conditions can be given through an ‘opinion’ which can be given for a period of up to 6 
months, and is then followed by a decision based on the improvement demonstrated.

35/70 
 
However, the standards are very detailed and lead to very specific recommendations which leave little 
room for independent institutional action, as they are told e.g. to add or remove a specific course with 
a specific number of ECTS, etc. One consequence of the focus on the compliance with detailed criteria 
is the seeming equalisation of all of them. To give an example, in an opinion that was made available 
to the panel in English, an institution was given 30 days to correct deficiencies. A number of 
deficiencies referred to failure to submit documents such as copies of teachers’ degrees, journal 
articles and rent contracts. A few however are substantial: the same program is sometimes referred 
to as ‘professional’ and sometimes as ‘academic’, and a clarification is necessary; it is unclear why the 
institution has two programs in the same field, at the same level and similarly titled; the institutions 
has not defined its goals or mission; the allocation of ECTS varies between documents; there is no 
assessment criteria for the students in most courses; students are not included in the IQA system. It 
is the opinion of the panel that placing all of these on the same list of deficiencies to be corrected 
within 30 days encourages HEIs to make superficial changes to the documentation, rather than going 
through a real process of change and development. HEIs are not given a real chance to demonstrate 
improvement in such a system. However, this opinion was issued in the ‘old’ procedure. In the new 
procedures, only summaries of reports were available in English and the whole reports in Serbian only. 
A far as this is sufficient evidence, the recommendations were mostly more general, and it remains to 
be seen how these will be taken up by HEIs and how their fulfilment will be assessed in future 
procedures.  
Regarding the last recommendation of the 2017 panel, to improve the clarity of the documents, the 
discrepancy of the panel’s impression and that of the stakeholders is obvious. Namely, this panel also 
found the documents confusing and took some time to understand which document refers to which 
procedures, and to understand their content and mutual differences and similarities. This is most likely 
due to the fact that all stakeholders are used to the standards as they have not been changed much 
for more than a decade. More on clarity of the procedures under 2.3.  
Panel recommendations:  
The agency should consider the ways to adapt its procedures to different types of institutions it 
evaluates and their varying contexts.   
In addition to and while implementing the recommendations under ESG 2.1, 3.4 and 3.6 which are all 
also relevant for this standard, NEAQA should establish, in collecting feedback from the HEIs, which 
issues are strategic and require long-term institutional action, as opposed to details which can be 
corrected quickly. This should be emphasised in guidelines to panels and taken into account by CAQA 
when giving recommendations, to avoid an overtly prescriptive approach and enable change and 
development.   
Panel suggestions for further improvement 
The agency could actively monitor the workload it puts on HEIs through external evaluations and 
follow-up procedures and be prepared to adjust the methodology to ease up the burden if needed. 
Panel conclusion: partially compliant 
ESG 2.3 IMPLEMENTING PROCESSES  
Standard:  
External quality assurance processes should be reliable, useful, pre-defined, implemented 
consistently and published. They include:

36/70 
 
- 
a self-assessment or equivalent 
- 
an external assessment normally including a site visit 
- 
a report resulting from the external assessment 
- 
a consistent follow-up 
 
2017 review recommendation: CAQA should (1) revise its procedures to include a site visit as part of 
each periodic programme review; and (2) devise a way for HEIs to report on progress in the 
implementation of its recommendations as part of existing or new arrangements; (3) consider 
developing guidelines on scoring for academic experts to ensure greater consistency in their approach 
to assessment. See also the related recommendations about the involvement of CAQA members and 
experts under ESG 2.2. and 2.4. 
Evidence 
While already the 2017 standards started introducing the site visit with students and labour market 
experts, changes in the LoHE were necessary to remove the anonymity of external experts as a 
condition and thus enable CAQA to appoint independent panels to participate in site visits, as is now 
defined by the new standards adopted in 2019. The procedure is now the same for all types of 
accreditations and the audit, and is depicted by the chart from the SAR below.  
Figure 3: accreditation timeline (source: SAR, p. 64)  
 
Upon request of the panel, NEAQA has also produced the table below.

37/70 
 
Table 5: comparison of the old (2017) and new (2019) procedure 
 
2017 procedure implemented by 
CAQA 
2019 
procedure 
implemented by NEAQA 
Launching 
the 
procedure 
The accreditation request submitted to 
CAQA staff 
The 
accreditation 
request 
submitted to NEAQA staff which 
issues 
the 
certificate 
on 
completeness 
of 
the 
documentation 
Finding experts 
CAQA appoints two reviewers 
CAQA proposes members of 
Peer review panel and director 
of NEAQA appoints it 
Experts decision 
Two experts check the documentation 
and give the evaluations 
Peer review panel checks the 
documentation and goes to a 
site visit in order to complete a 
report and makes a decision 
proposal 
Sub-commission of 
CAQA 
Two reports of the reviewers  are 
delivered to a sub-commission, which 
performs site-visit and makes a report 
with the decision proposal to CAQA 
Report is sent to CAQA’s sub-
commission which forwards it to 
CAQA 2018 
Making decision 
CAQA adopts the decision and issues a 
formal certificate  
CAQA adopts the decision and 
NEAQA 
issues 
a 
formal 
certificate  
 
As is noted in the SAR, only the self-analysis which is prepared during the 3rd year of the accreditation 
cycle in preparation for the audit should be considered a true analysis, as it provides HEIs with an 
opportunity to describe their aims, goals and procedures. While HEIs need to submit comprehensive 
documentation prior to each accreditation, this documentation is not analytical but rather amounts 
to describing compliance with detailed criteria and collecting evidence and data.  
While every programme and institutional accreditation is administratively considered as a separate 
file, led by a separate expert panel, in practice very often the panels evaluating programmes of the 
same institutions overlap in members and only one site visit is organised, normally lasting between 6 
and 8 hours. Among the new pilot procedures, there was an institution at which 7 programmes and 
the whole institutions were evaluated during one such site visit. As is already noted in the 2017 report, 
it is not fully clear if the programme and institutional evaluations would take place at the same time. 
At the site visit it was established that this indeed is the preferred method, however sometimes the 
programmes have varying accreditation deadlines and in such cases this is not possible.  
One unusual aspect according to the panel is that during the meeting with students at some point the 
rest of the panel leave the room, leaving only the student member to talk to the students. . The 
explanation provided during the site visit by the agency representatives and the experts was that the 
students feel safer talking to their peers.  
LoHE provides for a possibility to issue an ‘opinion’ – a type of a conditional decision – in the audit 
procedure, for a period of up to 6 months, after which a yes/no decision must be made. The SAR notes 
that opinion will no longer be issued for the new procedures, but, as no audits were implemented 
according to the new procedures, there is no evidence available to show if that would also be the case 
for audits. The opinion was already discussed under 2.2, and provides recommendations for

38/70 
 
improvement. LoHE does not include the possibility to issue an opinion in accreditation procedures, 
and it is not regulated in detailed by the NEAQA regulations, nor discussed as a possibility in the SAR. 
An opinion issued in a programme accreditation procedure was submitted to the panel, as apparently 
in the transition phase it was kept in old cases as a replacement for the ‘act of warning’ that used to 
be issued by CAQA in line with the old law.  
The reports from the new, pilot accreditations were available to the panel in Serbian, with summaries 
in English. They all contain general recommendations for institutional action, with a note that only the 
recommendations referring to standards assessed with a low grade (6 or 5) are mandatory for the 
institution to implement.  
Similarly, follow-up is not described in the regulations; instead of a mid-cycle follow-up, an audit exists 
as a separate procedure. SAR mentions the possibility of introducing a separate follow-up report as 
obligatory after two years, and recommending that an audit is implemented earlier at newly 
established HEIs. No follow-up or audit reports from the new cycle are available yet, and no 
procedures apart from periodic program and institutional accreditations have been implemented.   
Analysis  
As NEAQA also notes in the SAR (p. 37), the purpose of evaluations should not be a binary result, as 
there are always areas for improvement, especially in a system that has limited experience and 
awareness of QA. Thus, if the purpose is enhancement, then the agency should be particularly 
committed to follow-up and fostering continuous improvement, and implementation can be a major 
factor in this respect. While introducing a regular follow-up report to all procedures would mean that 
HEIs produce some kind of a report every second year, considering the overall stakeholder support to 
frequent monitoring, this might be a reasonable demand if it is as administratively light as possible. 
The expectation is also that after two cycles all HEIs meet the minimal accreditation criteria, and is 
thus not to be expected that they would receive positive decisions with many mandatory 
recommendations the fulfilment of which would need to be evidenced with comprehensive 
documentation. One aspect that it was not fully clear is what happens when major structural issues 
are identified in a program or in an institution. What can the agency do? How can it lead the institution 
to address them? While the panel did receive samples of reports which did identify such issues, as 
there are no follow-up or audit reports available, this will remain an open question.  
Though some meetings can be common and even though it is fairly standard to cluster similar 
programs in a single site visit, the panel finds the current site visits too short. Interviews normally last 
for half an hour which does not seem sufficient to get more into more than few issues. It is true that 
this is partly offset by the practice of drafting reports before the site visit, which might help panels 
focus. It is also true that conducting site visit is a novelty for most experts, as only CAQA members 
were conducting them before, and thus the opportunity to discuss anything at all is vastly appreciated.  
Additionally, the agency should consider carefully how its expert panels will react to issues which were 
raised during the part of the meeting with students which is held by the student panel member alone. 
It is true that this practice is understandable as it provides a safe space for students. However, because 
this meeting is not attended by the whole panel, the integrity of the review could be at risk. The 
separation of the panel could translate into a trust issue between a student expert and other experts 
when controversial information is received during the individual meeting with students.   
Panel recommendations:

39/70 
 
The agency should deepen the level of assessment, in which longer site visits would be of help. This 
would enable longer meetings which could thus serve as a forum for enhancement-oriented 
discussions in addition to being a source for evidence checking.  
The agency should develop mechanisms of clustering similar programmes in the same institution to 
assure greater congruence and efficiency of evaluations, rather than organising this ad-hoc and 
without an impact on the costs of the evaluation.  
It should be decided in a broad discussion with stakeholders if audit is meant to serve as a follow-up 
to accreditation or a separate follow-up procedure is necessary, and if audit itself should have a clear 
follow-up also in cases when the opinion is not issued.  
Panel suggestions for further improvement 
The agency could try discussing if the students would support involving the whole panel in the whole 
of the student meetings during the site visit, and if the feedback is positive, try piloting such 
procedures.  
Panel conclusion: partially compliant 
 
ESG 2.4 PEER-REVIEW EXPERTS 
Standard:  
External quality assurance should be carried out by groups of external experts that include (a) 
student member(s). 
 
2017 review recommendation: CAQA should (1) ensure the involvement of external experts as playing 
a central role in EQA processes, in line with the ESG; (2) ensure the involvement of students and labour 
market representatives in all periodic programme accreditation reviews; (3) provide space for 
students to contribute to its EQA processes beyond a range of aspects considered to be strictly student 
matters; (4) ensure much wider involvement of international experts, not only in periodic programme 
accreditation reviews but also in audits; (5) provide regular training, and feedback on reports, to 
academic experts. See also the related recommendation about CAQA’s role in EQA processes under 
ESG 2.2 and guidelines for experts under ESG 2.3. 
Evidence 
It has already been noted that the most crucial change taking place with the 2017 LoHE was that the 
experts are no longer anonymous and that independent peer-review panels are formed in every 
procedure. The panels are appointed by CAQA – with NEAQA director signing the decision, which is 
then published. HEIs do not get a specific opportunity to respond to the panel composition while in 
principle the Administrative Law enables them to file a complaint. CAQA is not able to independently 
find experts, but has to choose among the lists provided to them, as follows.  
 
The academic experts apply to a public call published by the National Council. As confirmed at the site 
visit, even though the regulations would allow for the National Council to do this independently it 
forwards the applications received to CAQA to remove those not meeting the formal criteria and then 
publishes as long a longlist as possible. This list currently contains around 1000 names according to

40/70 
 
the information from the site visit. The list contains the following data: name and surname of the 
reviewer; state, place and institution at which he acquired the highest level of education; current 
teaching-scientific qualification and the year and institution in which he was elected for the teaching-
scientific position; and educational-scientific, i.e. educational-artistic field and domain within which 
he was chosen for the teaching-scientific position. LoHE proscribes that a person elected, designated 
or appointed to an office in a state authority, a body of the autonomous province or local self-
government, a body of a political party or to the post of the executive body of a higher education 
institution, as well as the person being a member of the National Council, the Accreditation 
Commission, i.e. employed at NEAQA may not be found on the list of reviewers. 
 
Due to the lack of international experts, the call for their applications is open until the end of 2019. In 
the accreditation process of PhD programmes one of the academics is always an international 
expert/reviewer. At the moment in the pool these experts are mostly those from the region and 
Serbian-speaking academics living abroad. The SAR states that the reasons for current pool 
composition is in two facts: (1) NEAQA wishes to foster regional cooperation keeping in mind 
numerous similarities of HE systems (e.g. countries of ex -Yugoslavia) which should encourage 
cooperation and best practice exchange; (2) the language barrier which asks for translation of the 
documents in English language, which induces additional costs wither for HEIs or NEAQA.   
 
According to the SAR, NEAQA has decided to internationalise its pool. The first step was taken in the 
second call for reviewers when in the application form the applicants were asked to assess their 
English language skills. This will also allow NEAQA to recommend its experts to other QA agencies and 
provide them with additional experience and education which will be valuable for the domestic 
external assessments. Furthermore, in order to overcome the language barrier CAQA plans to ask from 
the applicants in one of the coming new accreditations, to submit the accreditation documentation 
for PhD programmes in English language. These steps are aimed to expand the quality processes and, 
even though LoHE regulates this only for PhD programmes, there is no barrier for use of international 
experts in the future in other quality assurance processes.   
 
Students are nominated by the two student conferences while previously they were nominated by 
HEIs. The representatives of the student conferences explained at the site visit that they invited 
applications of their members through their local organisations at each HEI – student parliaments -  
and left it to them to make a selection. As this was the first time they were doing it and they had a 
relatively short deadline, they decided to use grade average as criteria. They do not consider this to 
be the best criterion however and plan to develop better criteria in the future. The other stakeholders 
at the site visit specifically expressed their satisfaction with the student reviewers, and students 
themselves confirmed that they felt as equal panel members and, after the first review, confident with 
their acquaintance with the procedure. Lists of students and labour market representatives are 
available on NEAQA website. The SAR states that the assessment of their knowledge of English 
language will be conducted in the future.   
 
The labour market representatives are nominated by the Chamber of Commerce and other 
professional associations. It is mentioned in the SAR that extra effort should be invested in their 
animation and willingness to participate because this new aspect of QA induces some challenges:

41/70 
 
labour market representatives should invest their time to learn about HE system and QA standards 
since the whole system is not well known to them; adequate financial compensation for the 
opportunity costs, etc. At the site visit it was explained that due to the wish to engage only with the 
most prominent professionals, numerous cancellations took place as these are very busy people.  
 
In order to prevent conflict of interest NEAQA adopted Code of Ethics and Regulations on Peer review 
experts which regulate the behaviour in QA processes. As confirmed at the site visit, all three groups 
are informed of the existence of these regulations and are obliged to sign the statement on the 
absence of conflict of interest. However, it happened in one case that an expert was invited to assess 
his previous employer.  
 
As noted in the SAR, CAQA graded the work of experts and ranked them on the basis of the quality of 
their reports. The panel learned at the site visit that low grades were given to experts that were overtly 
lenient in their assessments. There was limited explanation of how the process of grading of experts 
works and what are its actual consequences. It seems that this is kept as an informal practice as 
discussed at the site visit, as there are no formal surveys to assess the experts’ work, while NEAQA 
takes that feedback into consideration when forming future assessment committees. However, it was 
confirmed at the site visit that CAQA did not stop hiring experts with low grades, but decided to give 
them one more chance. It was also noted during the site visit that because the new procedures no 
longer include individual assessments, the experts would probably no longer be graded. Instead, there 
are plans to collect feedback on each expert from other peers in the panel.  
 
CAQA has created guidelines and report templates for reviewers. These were available to the panel in 
Serbian, and it was noted during the site visit that while these were useful, they do not provide 
instructions on grades. During April and May 2019 CAQA has organised trainings for reviewers in three 
regional centres: Nis, Novi Sad and Belgrade. On these trainings the total of 448 reviewers was present 
of which 408 academics, 39 students and 1 labour market representative. Beside these general 
trainings, CAQA organises briefings for peer-review panels before each assessment and site visit, as 
the panels meet to write the draft report. The international experts have confirmed to be included in 
this meeting via videoconferencing and emails. The expert panels were also accompanied by NEAQA 
staff when visiting HEIs during the pilot procedures. All the experts present at the site visit confirmed 
that they were always supported by NEAQA secretariat during the entire evaluation process. 
 
The panel discussed the trainings during the site visit. While the students showed interest in discussing 
various aspects of the ESG, the student-centred learning, learning outcomes, etc., the peers from the 
academia and labour market representatives did not express the wish for more trainings.   
 
Analysis  
The significant involvement of students and professionals is in the view of the panel rightly regarded 
by the stakeholders as very positive and relevant, and the panel additionally believes that overall the 
involvement of full peer panels is the major improvement that took place since 2017 review. NEAQA 
itself is well aware of the most challenges that remain, as noted in the SAR and during the site visit.

42/70 
 
The use of international reviewers has been very much limited by language issues, leading to a 
dominance of experts from neighbouring countries that, in several cases, may have been trained in 
Serbia or have strong links to the system. Internationalization is very important (given the small size 
of the system) and hinders an actual international perspective and therefore the selection of 
international experts should be broadened, especially with PhD programs and more research-
intensive institutions. Agency should also consider expanding geography of its foreign expert pool as 
a measure to achieve its goal of internationalization.  
 
Introducing international experts is also an important mechanisms of dealing with conflict of interest. 
There was also a comment during the site visit that the concept could also be redefined. Currently 
NEAQA considers only current connections to present a potential conflict, which is acceptable, 
however experts could also be allowed to report a conflict of interest also if it is only perceived – e.g., 
if they do not wish to evaluate a former employer. Generally speaking, the panel is satisfied that 
NEAQA takes this aspect seriously and ensures that the reviewers are not partial in their evaluations.  
 
The training seems rather limited, as the trainings only took a day and included almost a hundred 
people per training. The time is short and the number of experts trained in each session is too large 
to prevent significant learning and interaction. In the discussion with the representatives of the 
experts trained by NEAQA the panel got the impression that not all experts are aware of the 
complexity of QA and do not distinguish between the experiences gained in their academic career and 
the aspects with have to be checked due to the requests of external QA. Some found the training 
unnecessary, which is less surprising considering that it focused on showing the standards, which 
could be easily read by the experts in their own time. Training for experts needs to be provided 
regarding grading the results of the checking of NEAQA criteria, as well as in report writing, together 
with practicing interaction on site. The agency should aim for deeper training that could not only 
familiarize the experts with basic procedures, but also train them in reviewing and QA, provide time 
for commenting reports, role-playing etc. As also noted by student representatives, general EHEA 
issues should also be part of the training, such as learning outcomes, ECTS, approaches to ESG, etc.  
Having a PhD and a lot of experience does not mean that one will automatically avoid speechifying, 
have good understanding of concepts like student centred learning, etc. The presence of a coordinator 
from the agency on site is crucial for the same reason. Trainings are better organised as smaller 
workshops with e.g. examples of bad reports, unacceptable behaviour on site and potential mistakes 
and pitfalls, than ex-cathedra lectures on standards which the experts can anyway study on their own. 
Rather than insisting on mass lectures of this type, it would be better to focus on small, interactive 
training sessions, possibly starting by including only e.g. students and panel presidents.  
 
Finally, students are usually expected to work on the standards which refer to the students during a 
review. However, they can contribute to every standard in the review and should be encouraged to 
do so. 
 
Panel recommendations 
The training of reviewers should be more regular and focus on gaining actual competences, preferably 
through working in much smaller groups. These competences include the skills necessary to

43/70 
 
successfully lead interviews and organise group work, as well as understanding of key EHEA concepts 
such as student-centred learning, ECTS and learning outcomes, etc.   
NEAQA should develop a process which secures that an expert is not asked to assess their previous 
employers, e.g. by checking short CVs of the experts before inviting them.   
Panel suggestions for further improvement 
To save time and funds, NEAQA can consider creating online presentations, videos or briefing 
documents covering the content the experts can study on their own, such as the legal documents 
including evaluation standards, but also key EHEA concepts. This could help them prepare for the 
training, and those unable to participate in trainings could use this to prepare for the first meeting in 
the evaluation.   
The agency would do well to more actively implement its plans to internationalise the reviewers’ pool.  
Panel conclusion: substantially compliant 
 
ESG 2.5 CRITERIA FOR OUTCOMES 
Standard:  
Any outcomes or judgements made as the result of external quality assurance should be based on 
explicit and published criteria that are applied consistently, irrespective of whether the process leads 
to a formal decision. 
 
2005 review recommendation: CAQA should define more explicitly in its internal regulations the 
benchmarks for its decisions (an ‘act of warning’ as opposed to refusal of accreditation, and approval 
as opposed to follow-up n audits) which are consistently used in practice. 
Evidence 
As explained in the SAR, all of NEAQA’s processes end with a formal outcome, and accreditation 
outcomes are binding for the Ministry which uses them to issue or revoke a licence. Initial 
accreditation reviews lead to a positive or negative decision by CAQA, based on which NEAQA issues 
an opinion to the Ministry. Upon completion of a periodic accreditation review, NEAQA may grant 
accreditation and issue an accreditation certificate, or refuse accreditation. An ‘act of warning’ 
(regarded as an intermediate or temporary outcome) is no longer provided in LoHE for accreditations, 
but it is used for the accreditation requests submitted before the enactment of new Law. In the case 
of refusal, Ministry revokes the licence or amends it to exclude the non-accredited programme. An 
audit ends with CAQA adopting a report which confirms that a HEI fulfils its QA obligations with 
recommendations for further improvement of which HEI needs to inform CAQA after two years, or 
initiating a follow-up process through the opinion discussed under ESG 2.2, with a six-month period 
for HEI to improve.   
Minimal standards for accreditation are set by LoHE: the number and type of programs necessary to 
become a certain type of HEI; that at least 70% of classes need to be taught by full professors at 
academic programs (50% for arts programmes), and at least 50% by people with PhDs at professional 
programs; that a HEI needs to employ at least 20 full-time professors; division of ECTS for each type 
of programme; HEI governance; necessary resources, etc. The standards are then further developed

44/70 
 
by the National Council, in cooperation with CAQA. The National Council had produced separate 
regulations containing standards for initial accreditation, programme and institutional periodic 
accreditation, audit and self-evaluation of HEIs (which serves as a basis for audit). At the time of the 
site visit, a new separate set of standards was being prepared specifically for the academies of applied 
studies to enable evaluation of the proposed dual education models. Based on these regulations, 
separate forms, templates and tables are available for different types of HEIs (one for universities and 
academies of applied studies, another for faculties and colleges) and programmes (one for scientific 
and one for artistic PhDs, and one for first and second cycle programs), and are supplemented by 
templates to be used by the panels and HEIs. The templates and the forms provide further instructions 
on how to apply the standards – e.g. how to calculate indicators on student numbers and publications 
in various disciplines. All of the documents are published on NEAQA’s website both in Serbian and 
English.  
Regarding the decision-making, at the moment CAQA operates in two manners. One is applied to the 
“old cases” and the decision-making processes for accreditation requests for institutions, programmes 
and initial accreditation received before December 2018 are conducted as follows: decisions are 
proposed by the CAQA sub-commission whose members are directly involved in a given review / audit. 
CAQA takes decisions in its meetings by a simple majority vote, with at least two-thirds of the 
membership required to attend; a member from a HEI undergoing the review / audit concerned does 
not vote. CAQA takes decisions based on compliance with the standards defined for its EQA processes, 
as assessed in its final reports where CAQA integrates findings from its own analysis of documentation, 
academic experts’ reports and site visit reports. The experts were anonymous and they did individual 
reports based on reviewing documents, and graded each standard using the 5-10 scale used in the 
Serbian HE system for student assessments, without participating in the site visit. During the site visit, 
performed by the CAQA sub-commission to which a student and a labour market representative were 
added later, the key standards for accreditation were those covering curriculum, staff, space and 
facilities. An ‘act of warning’ or ‘opinion’ is issued where shortcomings can be eliminated in a short 
time (one week to six months, according to the procedure, e.g. one course to be replaced; mission 
and vision not clear), and otherwise (e.g. significant gaps in curricula, inadequate qualifications of 
staff) accreditation is refused. No minimum criteria or benchmarks are defined for audits, but there 
are detailed reports with as many recommendations as possible, and follow-up is initiated if a 
significant shortcoming is identified under any standard. In August 2018 the reappointed CAQA 
inherited 383 pending requests. At the moment 254 were processed of which: 135 got the 
accreditation, 77 acts of warning/opinion, 34 were rejected and 8 waived the request. Deeper analysis 
of these decisions has not been performed by CAQA, and this was explained in the SAR by time 
constraints.  
The standards have not been changed substantially with the new procedure (see ESG 2.1 on this), but 
a number of procedural changes took place. The expert panel composed of 2 or 3 academics, a student 
and a labour market representative meets before visiting the institution to produce a draft report, and 
then finalises after the site visit. They agree on a single set of grades and a decision and submit a draft 
report to NEAQA which forwards to the HEI for a 15-day factual check. Comments are then forwarded 
to the Panel which takes them into consideration and prepares the final assessment report which is 
submitted to CAQA which can ask for clarifications, additional arguments etc., but does not participate 
in the panel’s decision. A sub-commission is formed which analyses the report and, based on their 
conclusions, CAQA passes the decision. The SAR notes that NEAQA considers the new procedure to be 
improved through trainings of experts who now receive guidelines on decision-making. No decisions 
were yet made in the new procedures when SAR was being written. Before the site visit several new 
procedures were completed, and the participants commented on the site visit that the decision-

45/70 
 
making is now more objective as the whole panel needs to reach common grades and agree on the 
final decision, while previously each expert provided their own set of grades. The experts the panel 
met on the site visit confirmed that, as expected by the procedure, CAQA members required additional 
comments and explanations in some cases but did not try to change the reports. Experts’ work is 
considered to be completed only after CAQA has made its decision based on their report. As any 
decisions in the end need to be signed by the NEAQA director, this was mentioned at the site visit as 
an additional control mechanism.  
Analysis  
As noted under 2.1, the criteria for outcomes have not meaningfully changed since the 2017 review, 
however, the grading method was improved. Now the whole panel reaches a common grade, while 
previously each done their own grading individually. While grades are still crucial for decision making 
and the overall outcome, there still seems to be no interpretation on what particular grade for a 
standard means besides a general expectation of a correlation between the grade and the evidence 
and recommendations provided. At the site visit the discussions seem to point to a general subjectivity 
in assessment which cannot be avoided. This is a very important challenge, especially as the number 
of programs and institutions being reviewed will grow significantly in the near future. As a result, the 
agency does not have in place robust and explicit mechanisms to ensure a consistent and fair 
application of the criteria. There are various possibilities to introduce these (critical readers of 
preliminary reports, committee of experienced reviewers, a scrutiny group of experts who check and 
the reports without writing them or making decisions themselves, systematic comparison of different 
standards in the same field, a database of decisions and precedents, etc.) and it is up to the agency to 
devise the one that finds more suitable, as this is a critical issue in building credibility upon its decisions 
and that these are regarded as transparent, well-grounded, and fair. 
Numerical grading is a common practice in many countries, even though it can be self-serving, time 
consuming, and leading to an illusion of objectivity. However, in principle there are no obstacles to 
using it but it does require clear descriptions of when a grade should be given, to avoid subjectivity in 
the assessment. Even this does not necessarily remove all bias from grading, and does leave the issue 
of essentially a subjective scale being treated as objective and e.g. calculations of averages being 
incorrectly used for decision-making.  
All of this is especially important as a few stakeholders met at the site visit commented the credibility 
of the previous CAQA, and most stressed the importance of NEAQA being seen as an objective body 
with robust decision-making procedures. As NEAQA is just starting a new cycle of reviews, this is a 
good moment to introduce them.  
Panel recommendations 
NEAQA needs to implement the 2017 recommendation to provide benchmarks for its decisions and 
not base them solely on numerical grades. 
In addition to improved trainings (see ESG 2.4) NEAQA should also implement one or more of the 
numerous practices existing to improve consistency in its decision-making – such as critical readers of 
preliminary reports, committee of experienced reviewers, a scrutiny group of experts who check the 
reports, systematic comparison of different standards in the same field, a database of decisions and 
precedents, etc.).  
Panel suggestions for further improvement

46/70 
 
When CAQA discusses a HEI or a programme a CAQA member is connected with, in addition to 
abstaining from voting, it would be well that the member leaves the room so that other CAQA 
members can discuss freely.  
Panel conclusion: substantially compliant 
 
ESG 2.6 REPORTING 
Standard:  
Full reports by the experts should be published, clear and accessible to the academic community, 
external partners and other interested individuals. If the agency takes any formal decision based on 
the reports, the decision should be published together with the report. 
 
2017 review recommendation: CAQA should (1) devise a way for ensuring, even within the current 
legal constraints, a more substantial contribution from external experts, including academic experts, 
students and labour market representatives, to its final reports; (2) devise, in collaboration with the 
Ministry of Education, an arrangement for CAQA to publish its initial accreditation opinions / reports 
on newly accredited institutions and their programmes; (3) define more clearly the rules for the 
publication of reports in its internal regulations; and (4) consider providing drafts of its reports to HEIs 
for a factual accuracy check. See also the related recommendation about involvement of external 
experts under ESG 2.2. 
Evidence 
As discussed in the 2017 report (p. 45) and repeated in the SAR (p. 46), according to the old procedures 
three types of reports were produced as part of CAQA’s processes: (1) individual academic experts’ 
reports based on the analysis of documentation from HEIs; (2) site visit reports (where a visit is 
undertaken) drafted by CAQA members involved, with comments on drafts from students and 
employer representatives integrated; and (3) final reports, drafted by the CAQA sub-commissions 
concerned, which incorporate findings from the other two reports which provide the basis for CAQA’s 
decisions/opinions. Where there was a difference of opinion between experts in their preliminary 
reports, CAQA members verified this during a site visit and rely on their findings. Only samples and 
summaries of these reports were published, as the old regulations explicitly required that reviewers 
remained anonymous and that reports were not published.  
According to the new procedure, a final report is written by a peer-review panel, which includes three 
academics, a student and a labour market representative respectively. For all QA processes the 
templates were developed by CAQA and they are available on NEAQA website. They contain general 
information on the QA process, HEI, panel, analysis of standards and their grades, summary, 
recommendations and signatures of panel members. The draft of a report is sent to the HEI for a 
factual accuracy check and if it has any comments on it the panel has to take them into consideration 
before making the final report, which is sent to the appropriate sub-commission of CAQA. Beside the 
report, panel is also obliged to write a short summary in Serbian and English. There are no longer any 
provisions in the Law on the report publication.  While the plan in the SAR was to publish only 
summaries, during the site visit full reports from new procedures were published on the NEAQA 
website (in the rubric Accreditation Outcomes, available in the Accreditation section) and assurances 
were given that this practice would continue. It was also confirmed that other documents arising from 
the procedure –such as the opinions which serve as a temporary conditional decision – would also be

47/70 
 
published. The documents are available in Serbian, and summaries are available at the English version 
of the website. The agency also publishes a list of accredited institutions and programmes as a booklet 
for students and the public. The panel however had no opportunity to see a SAR from the new 
procedure in any language, or a new report in English which makes it difficult to give a feedback on 
this.  
From the discussions at the site visit it could be established that each panel member assumes 
responsibility for a couple of standards in the report, and then the president coordinates them and 
submits the final version to NEAQA. The responsibilities are divided on the basis of experience, and 
there are standards which are typically ‘covered’ by students or labour market representatives. 
NEAQA staff do not participate in writing reports as it is the role of the panel president to coordinate 
this. During the site visit there were comments how it could only be expected that the reports vary in 
style and quality as they are written by different people.  
Analysis  
The panel is satisfied that the reports are currently published, and it is easy to locate them on the 
NEAQA website. However, currently there is no evidence that also decisions and opinions are 
published, as there were none by the site visit (even though a number of procedures was completed, 
it was directly before the site visit and there was thus still no time to adopt decisions). Publishing 
reports and decisions is also not   part of the regulations and until it is, there will be no guarantees 
that all reports are indeed published.  
Bearing in mind that the whole reports and decisions are only available in Serbian and not the whole 
panel was able to read them, in addition to the fact that few new procedures have been implemented, 
the following can be noted. The reports represent a satisfactory analysis of the standards, with 
detailed arguments and evidence. A comparison of new and old reports reveals that new contain more 
general recommendations, rather than brief instructions on how to meet a certain minimal criterion. 
The decisions are only available for the old cases, however they are short and informative, providing 
the summary of the review, and in the case of institutions they list all the programmes and 
accreditation decisions for those.   
It was shown at the site visit, and is also obvious from the 2017 report, that the public is aware of the 
importance of accreditation and that students check the status of their HEI. However, as the 
publication of the reports is a very recent practice, it is not possible to discuss their dissemination. 
Also considering that NEAQA is a newly established body, dissemination is an important issue at this 
stage.  
Panel recommendations 
It is necessary to uphold the practice of publishing all full reports, and include this in the regulations, 
at least the NEAQA statute as other regulations can only be changed by other bodies.  
The responsibility for the dissemination of results should be awarded to both the agency and to the 
institutions, which should ensure that prospective and current students, employers, and other 
stakeholders have easy access to the results of accreditation processes. Furthermore, the agency 
should find creative ways to make those stakeholders aware of its work and its impact, as this is critical 
to build trust upon the HE system and to show the relevance of its work in that respect.  
Panel suggestions for further improvement

48/70 
 
The agency is aware of possible unevenness of the reports as an issue and would do well to collect 
good practice examples in reporting and disseminate them through its trainings. 
Panel conclusion: substantially compliant 
 
ESG 2.7 COMPLAINTS AND APPEALS 
Standard:  
Complaints and appeals processes should be clearly defined as part of the design of external quality 
assurance processes and communicated to the institutions.  
2017 review recommendation: (1) CAQA should put in place a procedure for HEIs to file a complaint 
as it is defined under ESG 2.7; and (2) a separate appeals body should be established within CAQA (or 
within a new quality assurance body to be set up by the newly enacted Law on Higher Education). 
Evidence 
LoHE regulates the appeal procedure stating that the appellate body is the National Council. In the 
case of refusal of accreditation, the HEI concerned may file an appeal to the Council through NEAQA. 
The same is true of a negative decision in the audit procedure, according to LoHE. The appeals 
procedure is laid down in the Rules of Conduct of NCHE in making decision upon the appeal in the 
process of accreditation and operation of the appellate commission, published on the Council website 
and available to the panel in Serbian. According to these rules when NEAQA receives an appeal and if 
it finds the appeal justified, it may change its original decision, with or without an additional review. 
Otherwise, an appeal is considered by the Council within 90 days. The Council appoints an appellate 
commission among the experts from its list who have not otherwise participate in the procedure, 
whose task is to analyse the appeal and the documentation and propose a decision within 30 days 
from the day of appointment. The Council then takes a decision on the basis of this by a majority vote. 
It may (1) reject the appeal if the procedure was properly implemented (or procedural faults had no 
impact on the decision) and its decision is justified and taken in compliance with the relevant 
regulations; (2) override the decision, in whole or in part, and instruct CAQA to reconsider its decision 
and conduct another review if key evidence is incomplete or incorrect, or the relevant regulations 
were not taken into account, or the decision is unclear or not adequately substantiated; (3) override 
CAQA’s decision and take its own decision based on the evidence collected if the evidence was 
incorrectly assessed or led to an erroneous conclusion, or the relevant regulations were incorrectly 
applied. The third option is only possible if CAQA, after the Council has returned the decision for 
reconsideration, again rejects the request, that is, in the second appeal. According to the LoHE if the 
Council refuses the appeal, HEI can initiate an administrative dispute in the court. Table 6 provides a 
description of the appeal procedure in accreditation, however the same procedure also applies to 
negative decisions in audits.  
 
Table 6: the appeal procedure (source: NEAQA) 
Phases of appellate procedure 
Procedures for accreditation of HEIs/study 
programmes 
Rejection decision and possibility to file an 
appeal 
The higher education institution may file an 
appeal against the decision of the National 
Accreditation Body rejecting the request for 
accreditation, within 15 days from the date of

49/70 
 
receipt of the decision, to the National Council 
and through the National Accreditation Body. 
Role of NEAQA in the appellate procedure 
If CAQA finds that reasons stated in appeal are 
grounded it could change its decision. If CAQA 
does not change the decision it forwards it to 
NCHE. 
Decision of the appellate body   
NCHE can:                                                                      
1. Dismiss the appeal in cases when it is 
inadmissible, untimely or filed by an 
unauthorised person; 
2. reject the appeal when it finds that the 
accreditation procedure is conducted in the 
correct way and that the decision on rejection is 
made in accordance with the LoHE and 
regulations; 
3. to annul the first instance decision and with 
adequate explanation return to NEAQA when it 
finds omissions in the process of accreditation. 
NEAQA’s decision upon the annulment of its 
decision 
NEAQA is obliged, upon the receiving of the 
decision of NCHE, to make a decision in 
accordance with the legal reasoning of the 
NCHE.  
NCHE’s decision after NEAQA’s repeated 
rejection of accreditation  
 
In case that NEAQA rejects again accreditation 
and HEI files again a complaint, NCHE will make 
a decision on its own, within the 30 days upon 
the receiving of appeal. 
 
Some stakeholders at the site visit commented on the option to overrule CAQA’s decision as a fault in 
the procedure which needs to be amended and as a key motivation to suggest a new law to the 
Ministry, as is also noted in NEAQA Strategy. Others emphasised that this was used only once in 
practice and that the National Council would normally uphold CAQA’s decisions and try to avoid such 
a move.  
 
Table 7: Number of appeals submitted to NCHE in 2018/19 (source: NEAQA) 
Appeals 
Number 
Not yet reviewed by NCHE 
13 
Returned to NEAQA to repeat the review process 
5 
 
The SAR did not comment on the complaints procedure, and there was significant confusion expressed 
at the site visit as regards differences between complaints and appeals. It was finally established by a 
National Council member that the complaints procedure is defined by the Administrative Law, and 
that according to it HEI can at any point in the process file a complaint to the NEAQA director or 
Management Board, as is the case with any other public institution. NEAQA has no specific regulations 
on this.  
 
Analysis

50/70 
 
This is a recognized weakness by the agency and needs to be addressed effectively. As both complaints 
and appeals are regulated by the Administrative Law, it is beyond the scope of this report to comment 
on the possible legal challenges in changing this practice. Despite the legal constraints, however, a 
fair, independent, and effective mechanism of complaint and of appeal is a crucial piece of the system 
and needs to be in place to nurture the trust of institutions. The option the Council has to overrule 
CAQA’s decision is a major threat to the independence of formal outcomes of NEAQA’s procedures. It 
is even more worrying that the Council itself has created no detailed guidelines for itself on when and 
how it should make such a decision. The argument that this happens very rarely should not be used in 
this context, because a decision to remove the accreditation from a programme or an institution is a 
serious one and should not be subject to change without an established procedure of checking 
evidence and decision-making. In the perspective of the panel, a complaints procedure should focus 
on the procedural aspects of the evaluation and not its outcome. Thus, the complaint can never result 
in a changed decision, but at most the decision to annul the original one and repeat the evaluation; 
otherwise the agency’s independence is threatened.  
The panel also finds that rather than having a separate second-instance body established by the 
Government (as is the case with the National Council), it is a more common practice that the agency 
establishes its own body which would still function independently. This is the only way in which the 
panel would be able to check if the agency is fully responsible and independent in deciding on the 
formal outcomes of its procedures. It would also enable the agency to involve people with sufficient 
expertise and experience as well as students. However, it is not explicitly required by the ESG that 
there is an independent appeals body that is established by the agency itself. The Council should 
abstain from overruling CAQA’s decisions. It would be even better if a legislation could be changed in 
a manner that would provide NEAQA with a standing appeals committee. Such a committee should 
be independent from the decision-making body and not be influenced by the institutions. The 
committee should never override the decisions of the agency. And it should have clear rules of 
procedure. 
Regarding complaints, apart from comments on the factual accuracy of the reports, it seems that the 
agency and its stakeholders have only during the site visit become aware the possibility to file 
complaints on any step in the procedure. This might not be surprising concerning that it arises from 
the Administrative Law and applies to public legal persons, which NEAQA has become only recently. 
The lack of a complaints procedure can be amended without changing the legislation.  
Panel recommendations 
It is necessary to ensure that an independent body decides on appeals in line with an established 
procedure of decision-making. Such a procedure should not have changing the decision as an 
outcome, and the agency and its stakeholders should try to ensure this to the degree in which this is 
possible without changing the law. Such an independent body should always include not only different 
stakeholder perspectives but also legal and QA expertise and experience.  
The panel recommends NEAQA to develop a complaints procedure adapting the Administrative Law. 
Panel conclusion: partially compliant

51/70

52/70 
 ADDITIONAL OBSERVATIONS
TIMING OF THE REVIEW  
It is necessary to note that even though CAQA, of which NEAQA is the legal successor, has gone 
through two ENQA reviews, NEAQA itself is a body formally established in February 2018, less than 2 
years before the time of this review. In the short 2-year period between the last and current review, 
NEAQA was established together with its Board in line with the 2017 LoHE, and CAQA was reorganised. 
In addition to compliance with the changed legislation, the major challenge in the reorganisation of 
the institution and the connected system has been compliance with the ESG, as emphasised in the 
NEAQA 2019-22 Strategy. Many important steps in the right direction have thus been taken in a very 
short time, and the panel would like to emphasise the improvement in a number of aspects of NEAQA 
work and the visible commitment of the staff and stakeholders to continued development. However, 
the timing also resulted in the fact that on several standards evidence was lacking, and it is the overall 
impression of the panel that the compliance assessment could have been more positive if the review 
had not taken place so soon. At the same time, the change NEAQA is trying to achieve is much broader 
and more difficult than the organisation of its structure and processes, and the NEAQA 2019-22 
Strategy shows that NEAQA is aware of this. A lack of trust is present in the system and the society 
which moulds the attitudes and behaviour in QA. While NEAQA emphasises throughout the SAR the 
need to focus on quality culture and, as phrased by the Strategy, leave the idea of conducting an 
“inspection” behind, many of the stakeholders still hold that its main mission is to check HEI resources 
and further purge the HE system from untrustworthy providers. A cultural change thus needs to take 
place if the HE and QA system are to be reoriented in line with the expectations of the 2015 ESG, while 
at the same time keeping the level of control over HEIs that is required in a system characterized by 
low trust. It is the hope of the panel that this report will help NEAQA, its staff and members of its 
bodies, to receive further support in working towards this aim. 
SUPPORT TO INSTITUTIONS  
As is expected by its Strategy, NEAQA would do well to support HEI training as well as their 
involvement in international exchange in the area of internal quality assurance and enhancement of 
teaching and learning using available funding through the Erasmus+ programme and other forms of 
cooperation with other HEIs and agencies. This would be particularly timely as Serbian HEIs will now 
start employing quality managers. Further involvement of international experts in NEAQA’s work will 
also aid the institutional learning process, as will participation in international projects and workshops.  
MOVING TOWARDS GREATER INSTITUTIONAL RESPONSIBILITY  
Providing more general criteria – instead of requiring HEIs to list each and every detail of their work – 
could remove the need to do a patchwork between ESG and the national standards, as noted under 
ESG 3.6. Ideally, this would lead to a situation in which NEAQA is able mid-term to truly operationalise 
the ESG in the Serbian context, rather than just trying to patch the citations of ESG with the LoHE 
standards. This is one way in which NEAQA would be able to implement its vision of becoming a leader 
in QA in the region.

53/70 
 CONCLUSIONSUMMARY OF COMMENDATIONS
ESG 3.5: The panel wishes to commend agency for hiring enthusiastic and competent staff.  OVERVIEW OF JUDGEMENTS AND RECOMMENDATIONS
In light of the documentary and oral evidence considered by it, the review panel is satisfied that, in 
the performance of its functions, NEAQA is in compliance with the ESG.  
ESG 3.1: substantially compliant 
Recommendations: Even within the present legal constraints, the agency should strive towards 
involving all stakeholders, and particularly students, as full members in all of its bodies. The agency 
should use its international connections to also more actively encourage the participation of 
international experts in its bodies.  
The governance of the agency should reflect the diversity of stakeholders and of the HE system and 
avoid being dominated by a small number of institutions, regardless of their prestige and importance. 
The agency should, in cooperation with the stakeholders, develop a new strategy which would enable 
the implementation of its mission of enhancing quality of Serbian HE in line with the international 
standards while keeping a level of control that they find is needed in their system.  
ESG 3.2: fully compliant 
ESG 3.3: substantially compliant 
Recommendations: In addition to involving more experts from outside the system (see ESG 3.1), the 
independence of the agency from HEIs as well as the government can be strengthened by the 
following steps, which need to be taken in cooperation with all stakeholders in the system.  
(1) To uphold the operational independence of the agency, define the criteria for membership in the 
bodies of the system which would focus on a balance of their personal experiences and motivation, in 
addition to the proportional representation of the system and formal criteria defined by LoHE.  
(2) To uphold the organisational independence of the agency as well as the independence of formal 
outcomes, ensure that the Director is a full-time position with no contractual obligations towards HEIs 
within the system.  
(3) To uphold the organisational independence of the agency as well as the independence of formal 
outcomes, work with the National Council to define a clearer appeals procedure (see ESG 2.7).  
ESG 3.4: partially compliant 
Recommendations: The panel recommends NEAQA not only to start regularly producing thematic 
analyses, but to use them as a tool for self-reflection (see ESG 3.6) and as a tool to promote good and 
innovative practices that can be spread throughout system.  
ESG 3.5: substantially compliant 
Recommendations: The agency should establish full costs of the procedures, in order to streamline 
them if possible and thus reduce the overall costs to HEIs, but also to establish long-term financial 
plans which would enable it to acquire timely additional support from the state budget if necessary.

54/70 
 
The agency should establish a human resources development plan which would provide not only the 
criteria and plans for employing new staff, but also training and development activities for both 
existing and new staff.  
ESG 3.6: partially compliant 
Recommendations: NEAQA should introduce formal mechanisms for collecting feedback from experts 
and institutions after each procedure, collect more qualitative additional feedback, e.g. through focus 
groups, interviews etc. to reflect on the procedures, especially the pilot procedures and establish 
similar mechanisms to regularly collect feedback from internal stakeholders – staff and CAQA 
members. The feedback should be collected and analysed in a manner that enables the stakeholders 
to comment on what they find relevant, and that can be used by NEAQA to continuously improve its 
work and the framework in which it operates, and report this to the public.  
ESG 3.7: fully compliant 
ESG 2.1: partially compliant 
Recommendations: NEAQA internal stakeholders need to reflect on the 2017 recommendations and 
start a broad discussion with all stakeholders on how to implement them and start the reorientation 
of the whole system towards the development of the quality culture which presupposes institutional 
responsibility for implementing the first part of the ESG in its entirety, rather than a more lenient 
system of external control.  
ESG 2.2: partially compliant 
Recommendations: The agency should consider the ways in which it could adapt its procedures to 
different types of institutions it evaluates and their varying contexts.   
In addition to and while implementing the recommendations under ESG 2.1, 3.4 and 3.6 which are all 
also relevant for this standard, NEAQA would do well to establish, in collecting feedback from the HEIs, 
which issues are strategic and require long-term institutional action, as opposed to details which can 
be corrected quickly. This should be emphasised in guidelines to panels and taken into account by 
CAQA when giving recommendations, to avoid an overtly prescriptive approach and enable change 
and development. 
ESG 2.3: partially compliant 
Recommendations: The agency should try to deepen the level of assessment, in which longer site visits 
would be of help. This would enable longer meetings which could thus serve as a forum for 
enhancement-oriented discussions in addition to being a source for evidence checking.  
The agency should develop mechanisms of clustering similar programmes in the same institution to 
assure greater congruence and efficiency of evaluations, rather than organising this ad-hoc and 
without an impact on the costs of the evaluation.  
It should be decided in a broad discussion with stakeholders if audit is meant to serve as a follow-up 
to accreditation or a separate follow-up procedure is necessary, and if audit itself should have a clear 
follow-up also in cases when the opinion is not issued.  
ESG 2.4: substantially compliant 
Recommendations:

55/70 
 
The training of reviewers should be more regular and focus on gaining actual competences, preferably 
through working in much smaller groups. These competences include the skills necessary to 
successfully lead interviews and organise group work, as well as understanding of key EHEA concepts 
such as student-centred learning, ECTS and learning outcomes, etc.   
NEAQA should develop a process which secures that an expert is not asked to assess their previous 
employers, e.g. by checking short CVs of the experts before inviting them.   
ESG 2.5: substantially compliant 
Recommendations: NEAQA needs to implement the 2017 recommendation to provide benchmarks 
for its decisions and not base them solely on numerical grades. 
In addition to improved trainings (see ESG 2.4) NEAQA should also implement one or more of the 
numerous practices existing to improve consistency in its decision-making – such as critical readers of 
preliminary reports, committee of experienced reviewers, a scrutiny group of experts who check the 
reports, systematic comparison of different standards in the same field, a database of decisions and 
precedents, etc.).  
ESG 2.6: substantially compliant 
Recommendations: It is necessary to uphold the practice of publishing all full reports, and include this 
in the regulations.  
It is necessary to uphold the practice of publishing all full reports, and include this in the regulations, 
at least the NEAQA statute as other regulations can only be changed by other bodies.  
The responsibility for the dissemination of results should be awarded to both the agency and to the 
institutions, which should ensure that prospective and current students, employers, and other 
stakeholders have easy access to the results of accreditation processes. Furthermore, the agency 
should find creative ways to make those stakeholders aware of its work and its impact, as this is critical 
to build trust upon the HE system and to show the relevance of its work in that respect.  
ESG 2.7: partially compliant 
Recommendations:  
It is necessary to ensure that an independent body decides on appeals in line with an established 
procedure of decision-making. Such a procedure should not have changing the decision as an 
outcome, and the agency and its stakeholders should try to ensure this to the degree in which this is 
possible without changing the law. Such an independent body should always include not only different 
stakeholder perspectives but also legal and QA expertise and experience.  
The panel recommends NEAQA to develop a complaints procedure adapting the Administrative Law. SUGGESTIONS FOR FURTHER DEVELOPMENT
It is the hope of the panel that this report will be taken as further encouragement to take the role of 
the agency and QA even more seriously, as well as the level of implementation of ESG in the Serbian 
system. The report has provided a number of suggestions for further development as well as 
recommendations elaborated in the report text. These can be summarised as follows:  
- 
QA can become more flexible in adapting to different types of institutions, and more relevant 
by focusing on outputs and evidence of quality culture rather than inputs. After the legal

56/70 
 
changes and with the existing review experience, Serbian HE should be mature for such a 
change in the approach.   
- 
This needs to be combined with improved robustness and consistency in applying standards 
in all NEAQA procedures, for which issues emphasised by the ESG Part 3 standards, especially 
independence and internal QA, are crucial.  
- 
The opportunity for a changed approach comes at a time when the Serbian HEIs are receiving 
additional support for QA through the opportunity to employ quality managers. This 
opportunity can be used well by employing people who could act as change managers, 
participate in international cooperation focused on teaching and learning and implementing 
ESG Part 1, and they can also be supported through NEAQA analyses and reports.

57/[total pages] 
 ANNEXES
 
 
9.10.2019 
TIMING  
TOPIC  
PERSONS FOR INTERVIEW  
ISSUES TO BE DISCUSSED  
09:00 - 10:00  
Arrival to the agency, meeting with the Director  
 Prof Jelena Kočović 
Interpreter  
 
  
 
 
 
 
10:00-10:15  
Discussion among panel members  
 
 
10:15-11:00 
Meeting with the Working Group responsible for the 
self-assessment report (including the Director) 
 Prof Jelena Kočović 
Prof Ana Šijački 
Ana Jakovljevic 
Aca Jovic 
Interpreter  
 
 Preparation for the external review 
11:00-11:10  
Discussion among panel members  
 
 
11:10-12:00 
Meeting with the Management Board  
Prof Sima Avramović, president of the 
Managing Board 
Prof Slobodan Unkovic, vice president 
Prof Milorad Milovančević, CONUS 
representative 
Prof Vera MIlošević, CAASS representative 
Miroljub Aleksić, Labour market 
representative 
Borijan Soković, SCONUS representative 
Interpreter 
Strategic goals and planning; communication policy (ESG 3.1); Status and 
independence (ESG 3.2 and 3.3); Research capacity and thematic analyses (ESG 
3.4); Funding and human resources (ESG 3.5); appointment of the Director and 
CAQA; Internal QA and professional conduct (ESG 3.6)   
12:00-12:15 
Discussion among panel members 
 
 
12:15-13:00 
Meeting with students involved in external quality 
assurance processes  
Jelena Novakovic 
Dino Martinic 
Marija Arsic 
External quality assurance processes and methodologies, incl. selection and 
training of students, role of students in processes, and student perspective (ESG 
2); Independence (ESG 3.3); professional conduct and feedback collection (ESG 
3.6)

58/70 
 
13:00-14:00 
Lunch (discussion among panel members)  
 
 
14:00-15:00 
Meeting with staff (QA and International 
Department)  
  
Ana Jakovljevic 
Aleksandar Jović 
Branko Pavlović  
Marin Milojević 
Tanja Ristić 
Working conditions; development opportunities; independence; staff 
involvement in internal quality assurance and thematic analyses (ESG  
3.3, 3.4, 3.5 and 3.6)    
15:00-15:15 
Discussion among panel members  
  
  
15:15-16:15 
Meeting with CAQA members  
Prof Ana Sijacki 
Prof Tanja Bugarski 
Prof Vesna Colić 
Prof Ilija Kovačević 
Prof Aleksandra Janković 
Prof Darko Plecas 
Interpreter  
 External quality assurance processes and methodologies, criteria for decisions 
(ESG 2.1- 2.6) 
16:15-16:30 
Discussion among panel members  
  
  
16:30-17:15  
Meeting with the employers’ organisation (Serbian 
Chamber of Commerce) and employer 
representatives involved in external quality  
assurance processes  
Miroljub Aleksic, vice president of the 
Serbian Chamber of Commerce 
Representative  of the Serbian Chamber of 
Commerce 
Dragan Živković 
Rade Jeftović 
Interpeter  
NEAQA governance, and employers’ role in external quality assurance (ESG 3.1); 
selection and training of employers for NEAQA processes (ESG 2.4);  
employability and related issues in NEAQA processes (ESG 2); Independence of 
NEAQA and employers involved in its processes (ESG 3.3) and professional 
conduct (ESG 3.6).    
17:15  
Wrap-up meeting among panel members   
  
 
10.10.2019  
09:00-10:00 
Meeting with experts involved in external quality 
assurance processes (2 per each type of procedure) 
Prof. Goran Petković, Unviersity of 
Belgrade 
Prof. Sandra Fišer Šobot, Unviersity of Novi 
Sad 
Prof. Ljubomir Lazić, Metropolitan 
University  
Goran Belojević, University of Belgrade 
Danijela Pecarski,  High Medical School for 
Applied Studies 
Interperter  
External quality assurance processes and methodologies; selection and training 
of experts; reporting; criteria for decisions (ESG 2.1-2.6);  
Independence (ESG 3.3); professional conduct and feedback collection (ESG 3.6)  
  
10:00-10:15 
Discussion among panel members

59/70 
 
10:15-11:00 
Meeting with HEIs: Conference of Universities, 
Conference of Academies of Applied Studies and 
Colleges of Academic Studies 
 Prof Ivanka Popovic 
Prof Svetlana Karić 
New legal framework; appointment and establishment of new agency; HEIs’ 
perspective on NEAQA governance and independence, incl. relations between 
NEAQA, CAQA, NCHE and MoES (ESG 3.1 and 3.3); Involvement in designing 
NEAQA methodologies (ESG 2.2); NEAQA accountability (ESG 3.6);  
NEAQA thematic analyses (ESG 3.4) 
11:00-11:15 
Discussion among panel members 
 
 
11:15-12:15 
Meeting with representatives of the Students  
Conferences of Universities (SCONUS) and Higher 
Schools (SCOHS)   
 Milan Savic, SCONUS 
Igor Knežević, SCONUS 
Milutin Markovic, SCAAS 
Nikola Vajagić, SCAAS 
Marko Despotović, SCAAS 
Marija TasmanovićAnita Maljavec 
 
NEAQA governance, and students’ role in external quality assurance (ESG 3.1); 
Involvement in designing NEAQA methodologies (ESG 2.2); Selection and training 
of students for NEAQA processes (ESG 2.4); student perspective in NEAQA 
processes (ESG 2); Independence of NEAQA and students involved in its 
processes (ESG 3.3) and professional conduct (ESG 3.6).    
12:15-12:30  
Discussion among panel members  
  
  
  
12:30-13:15 
Meeting with the National Council for Higher 
Education (NCHE)  
 Prof. Bela Balint, president of NCHE 
Prof. Marijana Dukić Mijatović, vice 
president  of NCHE 
Prof. Gordana Kocić, secretary of NCHE 
Prof. Hasan Hanić, member of NCHE 
Complaints and appeals (ESG 2.7) Designing external quality assurance 
methodologies processes (ESG 2.2 and 2.3); Criteria for decisions, and appeals 
(ESG 2.5 and 2.7); thematic analyses (ESG 3.4) 
13:15-14:15 
Lunch (discussion among the panel members) 
 
 
14:15-15:00 
Skype meeting with international experts involved in 
external quality assurance processes  
Milijana Novovic Buric, Montenegro 
Zlatan Car, Croatia 
 
External quality assurance processes and methodologies; selection and training 
of experts (esp. ESG 2.3 & 2.4); Independence (ESG 3.3); professional conduct 
and feedback collection (ESG 3.6)  
  
15:00-15:15  
Discussion among panel members  
  
  
  
15:15-16:15 
Meeting with HEIs which are part of the pilot 
accreditations (including QA staff from HEIs)  
Milo Tomašević, dean University of 
Belgrade School of Electrical Engineering 
Bojan Milisavljević, vice-dean University of 
Belgrade Faculty of Law 
Prof. Mladen Veinović, Singidunum 
University 
External quality assurance processes and methodologies; experts; criteria for 
decisions; reporting; and appeals (ESG 2); independence and accountability, incl. 
external feedback collection (ESG 3.3 and 3.6) Internal quality assurance policy, 
feedback collection and follow-up mechanisms (ESG 3.6)

60/70 
 
Representative of the University of 
Belgrade Faculty of Organisational Science 
Representative of the University of 
Kragujevac 
Representative of the University of Nis 
Faculty of Electrical Engineering 
Vesna Spasojević-Brkić 
Interpreter 
16:15-16:25  
Discussion among panel members 
  
  
  
16:25-17:010  
Meeting with the Ministry of Education, Science and 
Technological Development  
Mladen Šarčević, minister 
Prof Ana Langović, state secretary 
Prof. Viktor Nedović, ministry assistant for 
European integration 
Overall strategic issues behind the new law and its implementation so far; 
appointment of the Managing Board; independence (ESG 3.3) and funding (ESG 
3.5); thematic analyses (ESG 3.4), cyclical reviews (3.7)  
17:10 
Wrap-up meeting among panel members   
 
 
 
 
 
 
 
11.10.2019  
09:00-9:45   
Meeting among panel members to agree issues to be 
clarified  
  
  
  
09:45-10:45   
Meeting with the CAQA Chairman and NEAQA 
Director (or the Self-Assessment Report Working 
Group) to clarify any pending issues  
 Prof Ana ŠIjacki 
Prof Jelena Kočović 
Interperter  
Outstanding issues  
  
10:45-12:15   
Private meeting among panel members to agree on 
the main findings   
  
  
  
12:15-12:45   
Final meeting with to inform about preliminary 
findings  (max. 12 participants, as chosen by the 
Agency) 
 Prof Sima Avramović 
Prof Slobodan Unković 
Prof Jelena Kočović 
Preliminary findings of the review

61/70 
 
Prof Ana Šijački 
Prof Biljana Jovanovic Gavrilovic 
Prof. Vesna Djukić 
interpeter 
Ana Jakovljevic 
Aca Jović 
12:45  
Lunch and departure of the panel members

62/[total pages] 
 ANNEX 2:TERMS OF REFERENCE OF THE REVIEW
 
External review of the National Entity for Accreditation and Quality Assurance in Higher Education 
(NEAQA) by the European Association for Quality Assurance in Higher Education (ENQA) 
Annex I: TERMS OF REFERENCE  
May 2019 
1. Background and context 
 
The National Entity for Accreditation and Quality Assurance in Higher Education (NEAQA) is an 
independent agency in Serbia, established in 2018, for the purpose of quality enhancement of higher 
education institutions and study programs. According to the previous Law on Education the 
Commission for Accreditation and Quality Assurance (CAQA) was founded in 2006 as the central body 
for accreditation and quality assurance in Serbia until the new Law of Education of 2017. NEAQA 
provides the legal, institutional, substantial and functional continuity with the previous institution.  As 
a new independent legal entity NEAQA is consisted of Managing Board, Director, Commission for 
Accreditation and Quality Assurance (CAQA), and Professional and Administrative Service. 
 
By virtue of the Law of Higher Education NEAQA is responsible for performing the accreditation tasks, 
the assessment of quality of higher education institutions and the units therein, evaluation of study 
programs and assurance of quality in higher education. In realizing those goals CAQA is an expert / 
professional body of NEAQA who decides on the requests for accreditation, conducts the accreditation 
procedure and the procedure of external quality evaluation upon opinion of five peer review experts. 
The Director of the NEAQA appoints peer -review experts upon the proposal by CAQA. 
 
Working plan of NЕАQА is focused on promotion of higher education in Serbia through ensuring its 
compliance with the internationally recognized accreditation and quality assurance standards. The 
planned activities will be performed in the process of accreditation and quality assurance in 
accordance with the quality principles in the European Higher Education Area (EHEA), the European 
Standards and Guidelines (ESG) and the regulations in the Republic of Serbia. 
 
In the process of preparation for the new cycle of accreditation, new procedures were adopted and 
published on 28 February 2019 in accordance with the requirements of ESG and the Law on higher 
education of the Republic of Serbia. The new procedure changed the accreditation system regarding 
the role of the peer reviewers, as suggested by ENQA and EQAR report about CAQA/NEAQA. The new 
CAQA had to review around 350 requests for accreditation which were left from the previous CAQA. 
During this process CAQA has noted numerous and various shortcomings, informed the academic 
community about these findings and established cooperation with it in order to overcome the

63/70 
 
shortcomings. CAQA informed the National Council for Higher Education1 (NCHE) about the shortage 
of international peer-reviewers necessary for the review of Ph.D. study programmes and NCHE decided 
to keep the call for experts open constantly. Moreover, the members of NCHE and CAQA took the 
obligation to spread the information about the call among the international academic community. The 
call is published on NCHE website in two languages: Serbian and English. 
 
NEAQA has been ENQA member under review since 22 February 2018 (the predecessor organization 
CAQA - Commission for Accreditation and Quality Assurance) and is now applying for ENQA 
membership. The review will also be used for the agency’s application for registration on the European 
Quality Assurance Register for Higher Education (EQAR). 
 
2. Purpose and scope of the evaluation 
This review will evaluate the way in which and to what extent NEAQA fulfils the requirements of the 
Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG). 
Consequently, the review will provide information to the ENQA Board to aid its consideration of 
whether membership of NEAQA should be confirmed and to EQAR to support NEAQA application to 
the register. 
 
The review panel is not expected, however, to make any judgements as regards granting membership. 
 
2.1 Activities of NEAQA within the scope of the ESG 
In order for NEAQA to apply for ENQA membership and for registration in EQAR, this review will analyse 
all NEAQA activities that are within the scope of the ESG, i.e. reviews, audits, evaluations or 
accreditation of higher education institutions or programmes that relate to teaching and learning (and 
their relevant links to research and innovation). This is regardless of whether these activities are carried 
out within or outside the EHEA, and whether they are obligatory or voluntary. 
 
The following activities of NEAQA have to be addressed in the external review: 
- 
Accreditation on Higher Education Institutions 
- 
Accreditation of Study Programmes 
- 
External quality evaluation 
- 
Initial accreditation of HEIs and study programmes 
 
3. The review process 
The process is designed in line with the Guidelines for ENQA Agency Reviews and the requirements of 
the EQAR Procedures for Applications. 
 
 
1 In charge for the creation of the list of peer reviewers

64/70 
 
The evaluation procedure consists of the following steps: 
- 
Formulation of the Terms of Reference for the review; 
- 
Nomination and appointment of the review panel; 
- 
Self-assessment by NEAQA including the preparation and publication of a self-assessment 
report; 
- 
A site visit by the review panel to NEAQA; 
- 
Preparation and completion of the final evaluation report by the review panel;  
- 
Scrutiny of the final evaluation report by the ENQA Review Committee;  
- 
Analysis of the scrutiny by the ENQA Board and their decision regarding ENQA membership;  
- 
Follow-up of the panel’s and/or ENQA Board’s recommendations by the agency, including a 
voluntary progress visit. 
 
3.1 Nomination and appointment of the review team members 
The review panel consists of four members: one or two quality assurance experts (at least one of which 
is currently employed by an ENQA member agency), an academic employed by a higher education 
institution, a student member, and eventually a labour market representative (if requested). One of 
the members will serve as the Chair of the review panel, and another member as a review Secretary. 
For ENQA Agency Reviews at least one of the reviewers is an ENQA nominee (most often the QA 
professional[s]). At least one of the reviewers is appointed from the nominees of either the European 
University Association (EUA) or the European Association of Institutions in Higher Education 
(EURASHE), and the student member is always selected from among the ESU-nominated reviewers. If 
requested, the labour market representative may come from the Business Europe nominees or from 
ENQA. An additional panel member may be included in the panel at the request of the agency under 
review. In this case an additional fee to cover the reviewer’s fee and travel expenses is applied. 
 
In addition to the four members, the panel will be supported by the ENQA Secretariat review 
coordinator who will monitor the integrity of the process and ensure that ENQA’s requirements are 
met throughout the process. The ENQA staff member will not be the Secretary of the review and will 
not participate in the discussions during the site visit interviews. 
 
Current members of the ENQA Board are not eligible to serve as reviewers. 
 
ENQA will provide NEAQA with the list of suggested experts with their respective curriculum vitae to 
establish that there are no known conflicts of interest. The experts will have to sign a non-conflict of 
interest statement as regards NEAQA review. 
 
3.2 Self-assessment by NEAQA, including the preparation of a self-assessment report 
NEAQA is responsible for the execution and organisation of its own self-assessment process and shall 
take into account the following guidance: 
- 
Self-assessment is organised as a project with a clearly defined schedule and includes all 
relevant internal and external stakeholders; 
- 
The self-assessment report is broken down by the topics of the evaluation and is expected to 
contain, among others: a brief description of the national HE and QA system; background 
description of the current situation of the Agency; an analysis and appraisal of the current

65/70 
 
situation; proposals for improvement and measures already planned; a SWOT analysis; each 
criterion (ESG part II and III) addressed individually. All agency’s QA activities (whether within 
their national jurisdiction or outside of it, and whether obligatory or voluntary) will be 
described and their compliance with the ESG analysed.  
- 
The report is well-structured, concise and comprehensively prepared. It clearly demonstrates 
the extent to which NEAQA fulfils its tasks of external quality assurance and meets the ESG 
and thus the requirements of ENQA membership. 
- 
The self-assessment report is submitted to the ENQA Secretariat who has 4 weeks to pre-
scrutinise it before forwarding the report to the panel of experts. The purpose of the pre-
scrutiny is to ensure that the self-assessment report is satisfactory for the consideration of the 
panel. The Secretariat will not judge the content of information itself but whether the 
necessary information, as stated in the Guidelines for ENQA Agency Reviews, is present. For 
the second and subsequent reviews, the agency is expected to enlist the recommendations 
provided in the previous review and to outline actions taken to meet these recommendations. 
In case the self-assessment report does not contain the necessary information and fails to 
respect the requested form and content, the ENQA Secretariat reserves the right to reject the 
report and ask for a revised version within two weeks. In such cases, an additional fee of 1000 
EUR will be charged to the agency.  
- 
The report is submitted to the review panel a minimum of six weeks prior to the site visit. 
 
3.3 A site visit by the review panel 
The review panel will draw up a draft proposal of the schedule for the site visit to be submitted to the 
agency at least two months before the planned dates of the visit. The schedule includes an indicative 
timetable of the meetings and other exercises to be undertaken by the review panel during the site 
visit, the duration of which is usually 2,5 days. The approved schedule shall be given to NEAQA at least 
one month before the site visit, in order to properly organise the requested interviews. 
The review panel will be assisted by NEAQA in arriving in Belgrade, Serbia. 
The site visit will close with a final de-briefing meeting outlining the panel’s overall impressions but not 
its judgement on the ESG compliance of the agency, or the granting or reconfirmation of ENQA 
membership. 
 
3.4 Preparation and completion of the final evaluation report 
On the basis of the review panel’s findings, the review Secretary will draft the report in consultation 
with the review panel. The report will take into account the purpose and scope of the evaluation as 
defined under articles 2 and 2.1. It will also provide a clear rationale for its findings with regards to 
each ESG. A draft will be first submitted to the ENQA review coordinator who will check the report for 
consistency, clarity and language and it will be then submitted to NEAQA within 10 weeks of the site 
visit for comment on factual accuracy. If NEAQA chooses to provide a statement in reference to the 
draft report it will be submitted to the Chair of the review panel within two weeks after the receipt of 
the draft report. Thereafter the review panel will take into account the statement by NEAQA, will 
finalise the document and submit it to ENQA.

66/70 
 
The report is to be finalised within three months of the site visit and will not exceed 40 pages in length. 
When preparing the report, the review panel should also bear in mind the EQAR Policy on the Use and 
Interpretation of the ESG, so as to ensure that the report will contain sufficient information for the 
Register Committee for application to EQAR. 
For the purpose of applying for ENQA membership, NEAQA is also requested to provide a letter 
addressed to the ENQA Board outlining its motivation to apply for membership and the ways in which 
NEAQA expects to contribute to the work and objectives of ENQA during its membership. This letter 
will be taken into consideration by the ENQA Board, together with the final evaluation report, when 
deciding on the agency’s membership. 
 
4. Follow-up process and publication of the report 
NEAQA will consider the expert panel’s report and will publish it on its website once the ENQA Board 
has made its decision. The report will also be published on the ENQA website, regardless of the review 
outcome and decision by the ENQA Board. NEAQA commits to preparing a follow-up plan in which it 
addresses the recommendations of the review panel and to submitting a follow-up report to the ENQA 
Board within the timeframe indicated in the Board’s decision on membership. The follow-up report 
will be published on the ENQA website, in addition to the full review report and the Board’s decision. 
 
The follow-up report will be complemented by a small-scale progress visit to the agency performed by 
two members of the original panel (whenever possible). This visit will be used to discuss issues, based 
on the ESG, considered as of particular importance or challenge to NEAQA. Its purpose is entirely 
developmental and has no impact on the judgement of membership and/or judgement of compliance 
of the agency with the ESG. Should the agency not wish to take advantage of this opportunity, it may 
opt out by informing the ENQA Review Coordinator about this. 
 
5. Use of the report 
ENQA shall retain ownership of the report. The intellectual property of all works created by the expert 
panel in connection with the review contract, including specifically any written reports, shall be vested 
in ENQA. 
 
The review report is used by the Board of ENQA for the purpose of reaching a conclusion on whether 
NEAQA is in compliance with the ESG and can be thus admitted/reconfirmed as a member of ENQA. 
The report will also be used for registration on EQAR and is designed to serve these two purposes. 
However, the review report is to be considered final only after being approved by the ENQA Board. 
Once submitted to NEAQA and ENQA and until it is approved by the Board the report may not be used 
or relied upon by NEAQA, the panel, or any third party and may not be disclosed without the prior 
written consent of ENQA. NEAQA may use the report at its discretion only after the Board has approved 
of the report. The approval of the report is independent of the decision on membership.

67/70 
 
The Chair of the panel shall remain available to respond to questions of clarification or further 
information from the EQAR Register Committee provided that the ENQA Secretariat is copied in all 
such requests. 
 
6. Budget 
NEAQA shall pay the following review related fees:  
Fee of the Chair 
4,500 EUR 
Fee of the Secretary 
4,500 EUR 
Fee of the 2 other panel members 
4,000 EUR (2,000 EUR each) 
Fee of 2 panel members for progress visit 
1,000 EUR (500 EUR each) 
Administrative overhead for ENQA Secretariat 
7,000 EUR 
Experts Training fund 
1,400 EUR 
Approximate travel and subsistence expenses  
6,000 EUR 
Travel and subsistence expenses progress visit 
1,600 EUR 
 
This gives a total indicative cost of 30,000.00 EUR VAT excl. for a review team of 4 members. In the 
case that the allowance for travel and subsistence expenses is exceeded, NEAQA will cover any 
additional costs after the completion of the review. However, the ENQA Secretariat will endeavour to 
keep the travel and subsistence expenses in the limits of the planned budget, and will refund the 
difference to NEAQA if the travel and subsistence expenses go under budget.   
 
It is understood that the fee of the progress visit is included in the overall cost of the review and will 
not be reimbursed in case the agency does not wish to benefit from it. 
 
In the event of a second site visit required by the Board and aiming at completing the assessment of 
compliance, and should the agency accept a second visit, an additional fee of 500 EUR per expert, as 
well as travel and subsistence costs are recoverable from the agency. 
 
7. Indicative schedule of the review 
Agreement on terms of reference 
March 2019 
Appointment of review panel members 
Late March/beginning of April 2019 
Self-assessment completed 
31 May 2019 
Pre-screening of SAR by ENQA coordinator 
June 2019 
Preparation of site visit schedule and indicative timetable 
July 2019 
Briefing of review panel members 
August 2019 
Review panel site visit 
Late September/ beginning of October 
2019 
Draft of evaluation report and submitting it to ENQA coordinator 
for pre-screening 
Mid-November 2019

68/70 
 
Draft of evaluation report to NEAQA 
End-November 2019 
Statement of NEAQA to review panel (if necessary) 
December 2019 
Submission of final report to ENQA 
Early-January 2020 
Consideration of the report by ENQA Board 
February 2020 
Publication of the report 
March 2020

69/70 
 ANNEX 3:GLOSSARY
 
CAASS 
Conference of Academies of Applied Studies and Colleges of Academic Studies 
CAQA 
Commission for Accreditation and Quality Assurance  
CONUS 
Conference of Universities  
ENQA 
European Association for Quality Assurance in Higher Education 
ESG 
Standards and Guidelines for Quality Assurance in the European Higher Education Area, 
2015 
HE 
higher education 
HEI 
higher education institution 
LoHE 
Law on Higher Education 
 
Ministry of Education, Science and Technological Development 
NCHE 
National Council for Higher Education 
NEAQA 
National Entity for Accreditation and Quality Assurance in Higher Education 
QA 
quality assurance 
SAR 
self-assessment report 
SCAASS 
Student Conference of Academies of Applied Studies and Colleges of Academic Studies 
SCONUS Student Conference of Universities 
ENQA 
European Association for Quality Assurance in Higher Education 
ESG 
Standards and Guidelines for Quality Assurance in the European Higher Education Area, 
2015 
HE 
higher education 
HEI 
higher education institution 
QA 
quality assurance 
SAR 
self-assessment report

70/70 
 ANNEX4.DOCUMENTS TO SUPPORT THE REVIEW
DOCUMENTS PROVIDED BY NEAQA 
• 
2018 Financial Report and 2019 Financial Plan 
• 
2019 SAR 
• 
CAQA Rules of Procedure, 2019 
• 
Decision on the Establishment of NEAQA, 2018 
• 
Draft questionnaire for review panels and reviewed HEIs (in Serbian) 
• 
Law on Higher Education 
• 
List of reviewers and breakdown according to gender 
• 
NEAQA 2019-2022 Strategy 
• 
NEAQA Code of Ethics, 2018 
• 
NEAQA Statute, 2018 
• 
Regulations on Standards and Procedures for Accreditation of Higher Education Institutions, 
2019 
• 
Regulations on Standards and Procedures for Accreditation of Study Programmes, 2019 
• 
Regulations on Standards for Initial Accreditation of Higher Education Institutions and Study 
Programmes, 2019 
• 
Rulebook on Standards and Procedure for External Quality Control of Higher Education 
Institutions (Audit), 2019  
• 
Rulebook on Standards and Procedure for Self-Evaluation of Higher Education Institutions, 
2019  
• 
Sample of a site visit protocol 
• 
Sample of reports from new and old procedures – except audit -  in Serbian, with summaries 
in English 
• 
Summary of Accreditation Outcomes, 2018 (in Serbian) 
 
OTHER SOURCES USED BY THE REVIEW PANEL  
• 
CAQA 2017 SAR with Annexes and Review Report  
• 
NCHE Appeals Procedure, 2018 (in Serbian) 
• 
NCHE Rules of Procedure, 2019 (in Serbian)

THIS REPORT presents findings of the ENQA Agency Review of the National Entity for Accreditation and Quality 
Assurance in Higher Education (NEAQA), undertaken in 2019.
2020 ENQA AGENCY REVIEW