名古屋高等教育研究 第３号（2003）

Quality Assessment in English Universities
Keith J. MORGAN＜Abstract＞
The procedures used for assessment of quality in teaching andinstitutional management in English universities are described. Overthe past 15 years a rapid learning process has occurred, shared by
the universities and the assessment agency. The agency has identified a non-intuitive framework necessary for adequate quality assurance; the universities have accepted this framework and recognised
the institutional advantages it provides. The results confirm a generally high level for the quality of teaching and academic standards andrapidly improving standards of institutional management.
This is now to lead in 2003 to replacement of the previous agency-centred external review process by an internal university-based
structure. The new procedure identifies university responsibility foracademic standards and effective institutional management. External
scrutiny will in future focus on the adequacy of procedures and practice within universities to sustain delivery of high standards.
An analysis of the impact of quality assessment on the universities suggests that the original initiative by the universities to introduce quality assurance and their subsequent positive response to its
development have been influential in reasserting the role of universities in determining academic standards.

1. Introduction
Quality assessment is a well-established feature of higher education.

Academic discussions regularly turn to appraisal of the relative

Ex-vice-chancellor, University of Newcastle (Australia)

83excellences of institutions, departments and colleagues.

Yet even thesubjective views expressed in such discussions increasingly refer toquantified performance indicators.

More importantly, external interest

in its quality has increased as the significance, the scale and the expenseof higher education has grown.

Considerations of public accountability

for the financial support it requires and the significance attached to its
work are persuasive arguments to justify demands for explicit anddetailed assessment of its quality.
In the UK, quality assessment has been institutionalised in the
Quality Assurance Agency1.

This was established in 1997 as a

successor to the Higher Education Quality Council (HEQC) and thearrangements for academic audits introduced in the late 1980’s by theuniversities themselves to monitor quality across the system.

QAA,

funded jointly by the universities and by government through theHigher Education Funding Councils (HEFC), has a wider mission2.
“..to promote public confidence that the quality of provision andstandards of awards in higher education are being safeguarded andenhanced.”
The procedures adopted to fulfil this mission were programmes ofdetailed external reviews of academic departments (subject areas) andinstitutional practices.

British universities have lengthy experience of

external assessment by external examiners at undergraduate andpostgraduate levels and by the arrangements for accreditation of degree
programmes by the professional institutions (e.g. engineering, medicine,law, education….) where graduation satisfies the initial professional
requirements.

Even so, the procedures adopted for detailed scrutiny

by QAA provided new challenges to the universities.

The results

have been generally accepted as valuable in identifying appropriate
criteria and demonstrating achievement of high quality across thesystem.

Although this satisfied the requirements of QAA’s mission,

the universities were deeply concerned about the burdens it imposed.
In particular, it became evident that any proposal to repeat and extend
84

Quality Assessment in English Universities

the reviews in their existing forms would be regarded as incapable ofachieving any benefit commensurate with the cost.

Given the generalevidence of existing good quality performance across the system, it was
not difficult for a decision to be accepted that the review process should
be reformulated to reduce its burdens.

A modified scheme, without

detailed subject reviews, and emphasising each university’s responsibilities for achieving high quality work is to be introduced in 2003.
Previously, QAA had identified two components in its total
programme of Academic Reviews:
reviews .
3

Subject reviews and Institutional

In Sections 2 and 3 there are outline descriptions of the

CVCP - HEQC - QAA schemes that operated in these two areas overthe period 1989 – 2001:

these reviews have now ended.

Section 4

describes the features of the modified scheme and arrangements to be
introduced in 2003;

and Section 5 provides some commentary on theimpact of quality assessment on the British university system.

2. Subject Reviews, 1993 –2001
A complete cycle of reviews across all the British universities,
covering all 42 subject areas was completed over the eight-year period
ending in 2001.

Initially, as Teaching Quality Assessment (TQA) thiswas the responsibility of HEQC (1993-6); subsequently QAA assumed
contractual responsibility.

Over this period there were evolutionarychanges to the procedures, mainly in the emphasis placed on some of
the components.
unchanged.
2.1

Generally though the essential structure remained

It consisted of four main elements4.

It was a peer-review exercise.

Small teams of specialists inthe subject area visited each university to conduct the reviews.

Usuallyin each team there were 3 or 4 specialists together with a chairman.
The reviewers were usually academics from other universities but
specialists from industry, commerce and the professions were recruited
for appropriate areas.

852.2 The review sought to assess performance within a subject area
for all taught courses, both postgraduate and undergraduate.

Under

the QAA, the assessment became codified under 6 headings.

Each ofthem contained a number of components, identified in the form ofquestions to be answered.

(i) Curriculum Design, Content and Organisation
How appropriate are the design, organisation and content of thecurriculum in relation to the intended learning outcomes and experiences?
outcomes?

Are the learning opportunities appropriate to the intended
What evidence is there that the curriculum reflects good

research, scholarship and practice, and, where relevant, current professional experience?

(ii) Teaching, Learning and Assessment.
Does the pattern adopted for teaching, learning and assessment
correspond to the stated aims and objectives of the department andthe university?

Is the programme appropriate in terms of the knowl-edge, understanding, analytical and subject-specific skills that are
expected?

What evidence exists to show that the teaching achieves

its objectives?

How effective are the assessment processes inpromoting learning and identifying achievement?

(iii) Student Progression and Achievement
Do the numbers of applicants and the qualifications for admission
match the aims of the department?

Are the achieved rates of progres-sion and completion satisfactory?

What evidence is there that thequalifications awarded indicate an appropriate level of student achievement?

(iv) Student Support and Guidance
Is there an overall strategy to support and guide students?

Are

the arrangements in this strategy understood by staff and students?
Are the arrangements for tutorials well matched to the needs of the86Quality Assessment in English Universitiesstudents, the curriculum and the assessment processes?

Are arrange-ments for student welfare and for career guidance clear and well understood?

(v) Learning Resources
What arrangements exist for access to appropriate resources forlearning?

Are library resources available and accessible to meet therequirements of the curriculum?

Is IT and other equipment available

at levels and of sufficient quality to match the needs of teaching andlearning?

Is accommodation for teaching, learning and social activity

available and appropriate to the needs?

(vi) Quality Management and Enhancement
How effective are the internal arrangements for monitoring andevaluating achievement of the aims and objectives of the subject area?
How have these arrangements made improvements possible?

How

effective are the processes of self-evaluation and continuing improvement of standards?
2.3 The department would have prepared a document offering its
own self-assessment.

This would include a statement of its aims andobjectives and evidence to demonstrate the quality of its educational
provision over the six aspects identified in section (see 2.2).

The aims

were expected to express the department’s broad educational purposes
in providing its programmes of study, essentially answering the question“Why do we seek to provide education?”

The objectives would identify

the results and experiences that constituted successful completion of
the courses: in effect, the objectives would aim to answer the questions“What are the intended outcomes, how are these achieved, and how
do we know that they are achieved?”
Limitations were imposed on the length of the description of theaims (250 words) and objectives (500-1000 words).

The self-assessment

document was required to contain much additional factual material,
relating to student and staff numbers for the university and the depart87

ment, entrance standards, applications, degree results, graduate careers.
Including the answers to the questions posed in relation to the qualityof the six aspects of educational provision identified in section 2.2 its
total length was expected to be no more than about 20 pages.
2.4

The quality of educational provision in the subject area wasassessed in the context of the criteria defined by the department’s
stated aims and objectives.

The quality review did not seek to invoke

a preferred or recommended set of aims and objectives;

acceptanceand recognition of diversity was implicit in the review.

The abilityof the programmes and the provisions identified by the department tomatch the criteria it had defined for itself provided the basis for thereview team’s judgement.
The review process extended over a lengthy period.

Timing ofa review into a specified subject would be agreed with a university atleast 12 months in advance.

This triggered preparations by the univer-sity and the QAA in order to provide an appropriate programme forthe review team.

The review culminated in a visit to the department

lasting about 4 days.

Some six months before the visit, the depart-

ment was expected to submit documentation of its self-assessment (see
2.3).

Within the department, extensive additional documentation would

be assembled.

This would include material related to general univer-sity practice and much specific departmental information.

The infor-

mation would include, course handbooks and descriptions, lecture notes,
student work and project reports, student feedback, reports fromexternal examiners, internal reviews, professional and statutory bodies,and from employers.

Statistical data relating to applications, enrol-

ment and progression of students, examination results, graduations andemployment would be collated.

Detailed information about academic

staff, training and development programmes and their wider researchand service activities would be included.

All this together withaccounts of the university’s operating principles and practices would
either be incorporated in the documents submitted to QAA for thereview team or held for their use during the visit. It became customary88Quality Assessment in English Universities

for at least one senior member of the department together with experienced staff in the central university administration to devote most of
one year to co-ordinating these preparations.
The visit of the review team provided an opportunity for them totest assertions in the department’s self-assessment and to clarify any
areas of obscurity in the documentary submissions.

The programme

for the visit would always include time for discussions with staff andstudents and allow the subject specialists to observe the range ofteaching provided.
At the end of the visit the review team would present its findings.
In the initial years, under HEQC, the results of the TQA were expressed
either as “Excellent”, “Satisfactory” or “Unsatisfactory”.

Under QAA,

the subject review produced a more detailed “Graded Profile”.

For

this, each of the six aspects of provision (listed in 2.2) was assigned anumerical grade 1-4 : level 1 was unacceptable; level 2 corresponded
to attainment of at least an acceptable standard.

However, accumu-

lation of three or more aspects graded at 2 would require the university and the department to prepare a plan for immediate improvement;
any assessment at grade 1 would cause QAA to undertake a further
full review within 12 months.
The final meeting with members of the department and the university enabled the chairman to indicate the strengths and weaknesses
identified by the review team.

This “summative judgement” included

comment on the achievements in each of the six aspects of provision
as well as other matters of substance.

Subsequently these comments

were developed into written report that included details of the gradedprofile.

This was presented to the university and published by QAA.

These reports – for all colleges and universities and for all 42 subject
areas – are available on the QAA web site.

3. Institutional Reviews 1989-2001
The universities themselves first established provision of a systemwide scheme of external quality review of British universities.

In 1989
89

the Committee of Vice-Chancellors and Principals (CVCP, now
Universities UK (UUK)) established its Academic Audit unit to examine
and report on the procedures used by individual universities to ensure
the quality and standards of their academic programmes.

Following

incorporation of the polytechnics as universities, this work was takenover by HEQC (1992) and subsequently transferred to QAA in 1997.
Its original designation was academic audit, extended to academic
quality audit (HEQC) and to continuation audit (QAA) and institutional
review (QAA);

it is to re-emerge in 2003 as institutional audit.

Persistence of the term “audit” does not imply either a passive processor the arithmetical methods of financial audits.

It does though combine

features of both classical and financial implications by indicating relianceon material presented by a university and on seeking documentary
evidence to substantiate claims.
The original CVCP academic audit sought to establish whether
institutional procedures constituted a satisfactory guarantee of educational standards and quality of awards.

In the context of institutional

aims and objectives it identified four specific areas for enquiry.
- arrangements for approval and review of awards and degree
schemes- availability of effective teaching facilities- quality of academic staff- procedures to establish that academic programmes operated
satisfactorily
HEQC added two further areas for enquiry:- assessment processes- procedures for establishing and monitoring standards of awards
In principle, QAA followed the path identified by CVCP and HEQC
but broadened the scope of the review process by directing it to cover
the effectiveness of all the relevant institutional arrangements5 .
90

Quality Assessment in English Universities

Explicitly it posed questions: “How do you know that you are able toguarantee standards and quality?” and “Can you convince us that yourarrangements are sufficient, valid and reliable?”

In directing those

who performed the reviews they listed four key questions that needed
answers from the institution in each of the areas of review:- Why are you doing it this way?
- Why is this the best way to do it?
- How do you know it works?
- How can it be improved?
By using these questions, QAA was able to create a structure for
its Continuation Reviews and Institutional Reviews similar to that for
the Subject Reviews.
3.1

Its structure consisted of 4 main components.

It was a peer-review exercise with a review team similar insize and composition to that used for subject review.

Expertise ininstitutional matters replaced expertise in subject areas.
3.2 Review teams concentrated on four main areas of scrutiny.
(i)Institutional strategy for management of the educational
provision.

(ii) Academic standards of the awards.
(iii) Learning infrastructure.
(iv) Internal and external communications.
In essence, the evidence submitted to the review by the university in each of these areas would be subjected to the tests of the 4
key questions.
3.3

Universities prepared a substantial document for the review,

identified as an “Analytical Account”.

It was expected that this would

indicate and evaluate the way in which the university exercised its
91responsibility for ensuring the standard of each award granted in its
name was maintained;

and how it ensured that the quality of educa-tion it provided enabled students to attain these standards.

An

Analytical Account, usually some 30-40 pages long, would describe
briefly the key features of the university’s processes relating to thefour main areas of scrutiny (see 3.2), provide a self-assessment of thestrengths and limitations of these processes, and describe arrangements
being made to enhance its existing arrangements.

It would be expectedthat a university would comment on any developments subsequent toprevious audits, particularly if these had identified areas of concern.
An essential component of the Analytical Account would be an
analysis of the effectiveness of its policies and procedures, identifying
documentary evidence to support its conclusions.

Where this evidence

was not included in the Analytical Account, it would be necessary forthe university to make it available to the review team.
3.4

The fitness of a university’s provision for institutional quality

assurance was judged in relation to its stated aims and objectives.

Aswith subject reviews, no attempt was made to impose absolute
standards: institutions were expected to be diverse.

However, famil-

iarity with standards for awards across the whole system of universities implied that review teams would comment if a university failed toensure an appropriate level and maintain adequate scrutiny across thefull range of its provision.
The actual process of an institutional review followed a similar
sequence to that for a subject review.

The team would expect toreceive the “Analytical Account” about 3 months before its visit to theuniversity.

This would allow its members opportunity to inform theuniversity of additional material they would expect to find available atthe time of the visit.

The university would collect material similar tothat required by a subject review but covering all academic courses
in the university together with statistical data for student applications,enrolments, progression, and graduation.

The visit would last normally

one week, with members of the team separately or jointly meeting with92

Quality Assessment in English Universities

academic, administrative and other staff and with students.
The final part of the visit would be devoted to drafting a report.
The review team would have started with an assumption that explicit
arrangements for monitoring its academic standards and quality would
exist.

The purpose of the review was to confirm that these arrange-ments were appropriate, reliable and robust.

Usually a report would

be structured to reflect the evidence in regard to each of the four main
themes of enquiry.

The review team would expect that the evidence

allowed them to comprehend the university’s current practices in eachof them and to identify those matters justifying commendation andthose requiring further consideration.

The latter would be charac-

terised as those where it might be desirable to modify current practice,those where some action might be advisable, and those where it wasnecessary that urgent reform be implemented.

Where it could beconcluded that no matters required urgent remedy and few changes
seemed advisable, the review judgement would be that the institutionalarrangements carried overall confidence. Evidence of a small numberof matters requiring immediate remedy or a larger number that it
would be advisable to modify would carry an overall judgement oflimited confidence.

Where more serious concerns were evident, thejudgement would be one of no confidence:

this was not a common

occurrence.
A final report following each visit to a university was published
by QAA: these reports are available both in printed form and on the
QAA web site.

Summarising the overall judgements, QAA wrote6:“(A)fter visits to almost all the higher education institutions in
the UK, (it) might be (stated) that all now have a battery of procedures intended to ensure quality; that many have some procedures that do assure particular aspects of quality; but that
only a small number have well developed and reliable means offully assuring their quality……There is an increasing recognition within higher education that academic standards need to
be both better understood and articulated, in theory and prac93

tice……..But it will cost time and money to achieve (this), andwill profoundly alter the way in which higher education functions and is structured.”

4. Institutional Audits 2003It had become apparent on completion of the cycle of subject
reviews in 2001 that changes in the arrangements for quality assessment had become essential.

In its operating plans, QAA had sched-

uled extensive modification of its procedures at this stage; the universities indicated that the burdens imposed, particularly by subject
reviews, were seen to be quite disproportionate to any benefits.
Following joint discussions with UUK, QAA and other bodies, theprincipal government agency, HEFCE, undertook a major consultative
exercise in 20017.

Over the comparatively short period of 4 months,

general agreement was reached about a new system of Institutional
Audits that would operate with “a lighter touch”8.

The new arrange-ments were formally announced in March 2002 for implementation in
20039.
While Institutional Audits are to retain much of the format andstructure of subject and institutional reviews, their intent and emphases
are substantially changed10.

Primarily this is indicated by the emphasis

now given to the universities rather than the regulatory agency forwhat they do educationally, academically and through exercise of theirformal powers of awarding degrees.

Recognition of the universities

as the responsible entities is not new – it appeared regularly in QAA
documents – but it is now given far greater prominence.
Three significant contributions to this change in emphasis can be
identified.

First, QAA, despite the caution of its comments written

earlier in 2001 and quoted at the end of Section 3, was able to saylater in 200111:“….these proposals will capitalise on the considerable
progress made by higher education institutions over the past94

Quality Assessment in English Universities

decadeinsecuring

theiracademic

qualityandstandards.”…….”the audits and assessments over the past
decade have confirmed that there are no major systemic problems with quality and standards across higher education in theUK.

There is no need to subject the academic community torepeated detailed external review at the micro level.”
Second is completion of a set of QAA basic documents, providing
fixed points of reference for quality assessment across the whole system.
There are three key documents: the Code of Practice for Assurance ofAcademic Quality12, the Framework for Higher Education Qualifications13
and Subject Benchmark Statements14.

The Code of Practice contains

10 sections covering a wide range of institutional management fromrecruitment and admission of students, programme review, and assessment to careers education, to student appeals.

The Code identifies

system-wide expectations for the precepts that should be embodied inmanagement in all universities.

The Framework is designed to ensure

a consistent basis for the standards used in designating university
degrees and other awards.

Subject Benchmarks have been prepared

for a total of 42 subject areas, effectively covering almost all academic
subject areas.

They are compiled by subject specialists and present

general expectations about the standards for the award of degrees andthe attributes that graduates should be able to demonstrate.

These

three documents are deliberately non-prescriptive in that they do notseek to impose prescriptions but they do provide a structure within
which the academic standards and quality of universities and theircourses of study can be assessed.
Third is acceptance by the HEFCE and the universities of agreed
lists of information that will be provided by all institutions15.

The lists

are extensive, covering quantitative and qualitative material that is
expected to be available in all universities, and part of which should
available publicly (Appendix 1).

Much of the quantitative data isalready collected by universities both for their own purposes and forsubmission to HEFCE and publication by HESA; similarly much of the95

qualitative information already constitutes part of a university’s regularmanagement data base.

There may though have been considerable

variation between universities in the way information has beenpresented.

It appears that a standardised presentation will be encour-

aged for the publicly available information and a more uniform pattern
for all data may well develop across the whole system. This set of
information will be the focus of scrutiny during the institutional audit,
and in the context of the three key QAA documents, will constitutethe basis for establishing the level of confidence that the university canbe accorded.
The structure of the institutional audit follows the pattern established for quality assessment reviews16.
4.1

It remains a peer-review exercise.

expected to consist of 3-7 members.

The audit teams are

They will be drawn from anextensive list of experienced university and professional staff, commonly
having been nominated to QAA by their own institutions.

They will

have attended special induction and training programmes arranged onbehalf of QAA.
4.2 The audit will concentrate on three main areas.

(i)

Conformity of procedures and practices to the three basic
documents (Code of Practice, Framework for Academic
Awards, Subject Benchmarks)(ii) Accuracy, completeness and reliability of the publicly available
information(iii) Examples of internal quality assessment at programme level 17
4.3 The main submission from the university is now to be known
as the Self-Evaluation Document (SED).

This will be a substantial

document, perhaps some 30-40 pages long.

It is intended to provide

a university with opportunity to describe and evaluate the ways inwhich it provides programmes and awards of appropriate standards
96Quality Assessment in English Universities

and quality, and show how it ensures that its published information isaccurate and reliable.

The university will seek to present evidence

in its SED to justify confidence in all aspects of this work.

To achieve

this it will need to indicate how it knows that its internal review
processes are effective; and when advice has been given previously
following institutional and subject reviews, show that it has responded
adequately.
Where the audit team has indicated that it will pursue discipline
audit trails, additional, shorter SED’s (3000 words) will be needed fromeach of the identified subject areas.

These will take the form of much

shortened versions of the self-assessments prepared previously for
subject reviews.
It is also expected that the students of the university will make awritten submission.

This might comment on the same topics as theuniversity’s SED but with particular emphasis on information available
to students, about the academic performance expected of them, and of
their experience of the instruction, examination and assessment
processes.

It is noted that the student submission should be relevant,concise, adopt an institution-wide perspective and be balanced between
description and analysis.
4.4

The audit will focus on the efficiency and effectiveness of auniversity’s arrangements for internal scrutiny of quality and standards
in the context of its published information and the relationship to thethree key documents – the Code, the Framework, and the Subject
Benchmarks.
The
sive.

“lighter” schedule for an institutional audit remains exten-

Initial planning will have started 10-12 months before the visitof the audit team.

A preliminary planning meeting with the univer-sity will have been arranged by QAA about 9 months before the visit
and this will be followed by decisions on the number of discipline audit
trails and thematic enquiries to be included.

The university, thestudents and those departments involved in audit trails will submit their
SED’s about 5 months before the visit and this will be followed by a97

briefing meeting to identify any subsequent clarification needed.

The

visit itself will last one week and will be devoted principally to discussions of the documentary submissions.

At the end of the visit theaudit team will determine its decisions.
The decisions take the form of judgements on the levels of confidence that can be accorded to the university on its practices and procedures and on the reliability that can be placed in its published information.

The judgements will usually be accompanied by recommen-

dations for consideration or implementation by the university.
levels of confidence are identified.

Three

Broad confidence indicates that theaudit confirms the existence and effective use of rigorous mechanisms
for management of quality and standards; these will include full use of
independent external examiners in assessment and of independent
external members in internal reviews.

A judgement of limited confi-

dence implies either detectable weakness in the management or the
implementation of existing procedures or in the reliability of the
published information.

It may well be that the weakness lies in the

failure of the university to be aware of a weakness or alternatively inits failure to rectify it. A decision of limited confidence will be followed
by a formal request that the university indicates within 3 months theremedial action it will take.

A decision of no confidence indicatesserious and fundamental failure at institutional and/or departmental
level to provide proper control of standards and quality; or that the
published information is unreliable and may be misleading.

Such ajudgement requires immediate remedial action by the university.
Where there has been a judgement of “no confidence”, an immediate
plan would be required and be followed by institutional action, regularprogress reports and a return visit by QAA after 18 months.

If doubts

about confidence remain, QAA might decide to undertake a further
institutional audit.
The audit team will also identify aspects of the university’s arrangements which merit commendation as examples of good practice; andmake recommendations where some improvements may be indicated.
Recommendations will be identified by priority as desirable, advisable98Quality Assessment in English Universities

or essential.
The audit is completed by publishing a report.

In the report, as

well as presenting the evidence for their judgements and recommendations, the audit team will be able to comment on any other mattersrelating to the university’s internal quality assurance regime or thequality and standards of its awards.

They will also report on thefindings of the disciplinary audit trails pursued.

It is expected thatthe report will be published no later than 5-6 months after the visit
has been concluded.

5. The impact of quality assessment
Replacement of the existing schemes of quality assessment with
the new institutional audits constitutes a major change in policy.

Thechange is far greater than that anticipated by QAA, which had beenplanning merely to modify its existing arrangements.
force for change came from the universities.

The driving

Their experience of

subject reviews led to the view that they had become grossly extravagant, with the benefits for institutions being perceived to be insubstantial in terms of the time and money expended.

Indeed, in theyear 2000-2001, QAA conducted 384 subject reviews:of the 2,300

grades, 96% were at level 4 (65%) or level 3 (31%) and all subject areas
reviewed in universities and colleges of higher education were rated“quality approved”18.
Yet the change in universities has been equally remarkable.

The

importance departments and institutions now assign to teaching andthe quality of its delivery has increased hugely.

Over the 12-year

period of audits and reviews, their value has been recognised to theextent that their methods are now incorporated within universities own
procedures.

This was not so in 1989 when the older universities

started their scheme of academic audit.

That initiative was largely

intended to demonstrate that universities did exercise responsibility inassessing the quality of their work.

Implicit in that scheme was confi-

dence that academic standards were assured by the long-standing99system of external examiners.

While satisfaction can properly be takenfrom the wisdom of the universities in providing the first audits in
1989, and that the new institutional audits in 2003 will attach similar
importance to external examiners and assessors, the system itself has
undergone fundamental change.

For convenience, the components ofchange can be grouped under three headings.
5.1 Massification
The period of quality assessment in the universities has coincidedwith a period of growth.

The two aspects are not unrelated.

Growthin many organisations is commonly accompanied by concern aboutmaintenance of quality; when growth is linked to increased diversity,
the concern is magnified.
ically since 1989.

The university system has expanded dramat-

There have been two phases: incorporation of thepolytechnics as full universities, effectively doubling the number of institutions and students; and rapid growth across the whole system,completing its transformation into a massified system with the age
participation rate increased from less than 20% to over 30%.
Broadening of the base for enrolment and of the range of degree courses
corresponded to social and political aspirations but simultaneously fed
concerns about academic standards and quality.

For the new univer-sities there were concerns about low entrance requirements, weak
programmes in professional courses, and new degree schemes in nontraditional subjects; for the old universities there were equal concerns
about failure to respond to the needs of non-traditional students, of rigid
procedures and of out-dated programmes.

Moreover it was evident

that differences in degree schemes reflected a great diversity in aims
and objectives across the whole university system.

As expressed bygovernment, there were doubts - about the ability of the universities
to meet the needs of students and of employers - and questions - about
the value they provided for the public money invested in them.
5.2 Role of the Regulator
Concerns about the universities in the early 1990’s were100

Quality Assessment in English Universities

summarised in the questions “What is it that the universities do, andhow do we know that they do it well?”Answers provided by theuniversities own initiative of “academic audits” were insufficient toconvince a sceptical government:
protecting self-interest.

self-regulation was regarded as

Creation of HEQC and subsequently QAA

accompanied new legislation that imposed statutory requirements on
the universities to meet standards both for their financial management
and for their academic work.
The initial task of HEQC and QAA was to establish methods forassessing the quality of work in universities.

Their success in achieving

this constitutes an important achievement.

A major difficulty, diver-sity of procedures, aims and objectives across the now wide range ofuniversity institutions, would have undoubtedly defeated any self-regulatory system attempted by the universities themselves.

Initial reluc-tance to submit to the intrusion of subject reviews was modified, firstby the reassuring approval accorded by the reports (in most cases),
and then by recognition that the reviews provided institutionally useful
information.

An intra-institutional learning process allowed all univer-sity departments to share in the benefits.

Consequently proceduresthat were found to be satisfactory by QAA reviewers were rapidly
adopted, so limiting the additional benefits from actual reviews.

It isarguable that this rapid learning experience could have allowed anearlier change in the QAA review process.

However, seeking a change

before the cycle of reviews was completed might well have appeared
premature to the external clients.

In the event, by allowing theevidence to accumulate, even government was satisfied that its statutory requirements permitted it to accept the universities’ rejection ofa continuing programme of subject reviews.
In effect by 2001 QAA had provided an answer to the question“Do we know that they do it well?”

It was also able to offer someanswers to the accompanying question “What is it that they do?”
These answers are in the form of the 3 key documents, the “Code ofPractice for the Assurance of Academic Quality and Standards”, “TheFramework for Higher Education Qualifications” and the “Subject
101

Benchmark Statements”.

Together the three documents define theboundaries within which universities’ aims and objectives are to befound.

The documents do not attempt to prescribe arrangements orcontent for university work;

rather they present a spectrum of

attributes, which, by common consent, will cover the programmes, andpractices of universities.

Even so, by themselves,the documents

might be construed as a further erosion of the academic autonomy of
the universities.

That this would be a misapprehension is clearly

indicated by the emphasis now given by QAA to recognition of the
responsibility of each university for its academic standards.

Acceptanceof these documents by its clients defines the new role for QAA in its
work of quality assurance.
The principal group of clients must be the universities themselves.
Here the job of QAA as regulator becomes essentially to “keep theuniversities honest”.

This will entail ensuring that each university’s

practices are coherent and consistent with the 3 key documents andthat its public statements are accurate and reliable.

In both of these,

QAA is acting as proxy for the public, who even with user-friendly
university web sites are unlikely to seek detailed analyses.

Twoconventionally important public groups of clients are prospective
students and prospective employers.

There is little evidence thateither of these groups use university performance or quality data rather
than accessibility, achievement and status as bases for enrolment or
employment.

Conversely, students attending a university may wellbenefit from freer access to the data both for academic and career
purposes.
More important are the requirements of government as a client.
It was at the insistence of the Department for Education and Skills(DfES) that the requirements for quantitative information (Appendix 1)
were included in the performance criteria.

The statistical data, alreadypublished by HESA, cover student ages, gender, ethnicity, disabilities,
socio-economic background, admission, progression, qualifications andemployment.

The universities are seen by DfES as principal agentsin their programmes of social engineering.
102

The data provide perfor-

Quality Assessment in English Universities

mance indicators indicating the levels of provision and support forminority ethnic groups, students with disadvantage, disability, or socioeconomic deprivation.

With government priorities identified in all theseareas and a stated objective of widening access to an age-participation
rate of 50%, this may provide a new dimension for quality performancein the universities.
5.3 Impact on the Universities
The pragmatic and sensible decision of the CVCP to start an
academic audit in 1989 has been rewarded now in the arrangements
for the new institutional audits across the system.

The universities’

original initiative enabled them to exercise a powerful influence informulating the agenda for HEQC and QAA.

Although the benefitsof this appeared less evident in the latter years of the subject reviews,
recognition of the relative responsibilities of the universities and theregulator are now reaffirmed.
Transfer of detailed scrutiny to the institutions is in accord withchanges in their managerial practices. A combination of increased sizeand complexity in conjunction with regulations and the requirementsof accountability have enforced change.

The extent of this is indicatedby the array of basic management data that is accepted as necessaryfor quality control in a modern university.

The catalogue of internal

documentation (Appendix 1) does not constitute a new burden but
rather reflects the accretion of existing practice.
While rejecting the burdens and the mechanics of the QAA subject
reviews, universities and departments have acknowledged the benefits
Criticism that high scores achieved in the subject reviews merelyreflected successful review techniques and submissions is not valid.
Even superficial observation reveals fundamental changes in attitudes
to teaching and departmental performance across the university system.
The matrix, of aspects of teaching and learning provision and the key
questions used in the QAA subject reviews constitutes a robust structure, previously lacking in institutional internal reviews.

Undoubtedly

it will continue to be used as a checklist for assessors in internal103

reviews.
The roles of external examiners and assessors are identified as
crucial in maintaining these improvements through internal review.
Emphasis is placed on the independence of external advice; but it also
carries an implication of common standards and achievement across thesystem.

The many professional bodies that accredit the courses inuniversity departments have similarly shared interest in common
standards and achievement. A continuing test of the diversified system
will be how well its standards can satisfy the narrower demands ofprofessional accreditation.

A number of the professional bodies –notably those for lawyers and the health professions – were reluctant
to see the end of subject reviews as they provided acceptable substitutes for their own appraisals.

However, as the professional bodies

have contributed to formulation of the Subject Benchmarks, it is likelythat they will become reconciled to a return to the pre-1990’s patternof accreditation.
Application of common standards across the system in no way
precludes diversity, as is already established.

The increasing numbersof non-traditional students and courses will continue to encourage
academic diversity.

It is though not always appreciated that theconcept of diversity must include retain some aspects of the older elite
system.

Many employers – and notably government through the civil

service – actively seek graduates from traditional programmes.
Inevitably, some universities are better able to prepare students forsuch employment.

Over time, more employers will learn how best to

utilize the special advantages available by recruiting graduates of nontraditional programmes.

Equally, universities will increasingly discoverways to inculcate traditional strengths through non-traditional
programmes.

Even so, while quality-of-output rather than value-added-

to-input is both a key performance indicator and a requirement of themarket, universities will compete for students with high abilities andattainment.

Conflict is already evident when this pursuit of ability isseen to be in opposition to official policies of socio-economic engineering.
Conventionally- and crudely- a dividing line is seen is this regard
104Quality Assessment in English Universities

between the “old” and the “new” universities.

Criticism of those olduniversities whose performance indicators for admission and enrolment
fail to respond to official policies of social equity accompanies commendations for academic excellence.
The impact of quality assessment on the British universities forms
only part of the changes effected by the series of reforms culminatingin the 1991 Higher and Further Education Act.

The Act empowers

government to obtain assurance of the standards and quality of thework of the universities.

This now forms part of the contract between

each university and government through which HEFC provides its
annual subsidy.

Introduction of tuition fees paid by students identi-

fies them as clients and leads to the central –and increasingly significant – role they play in quality assurance.

Conversely the changed

status of academic staff from “members of the university” to“employees” might be expected to reduce their collegial commitment.
This may be counterbalanced by encouragement to enhance theirprofessional skills and status as university teachers through formal qualification and registration.
Overall, the effects of the quality assurance exercises have provided
a system that is demonstrably accountable and managerially effective.
The quality of academic work in British universities is evidently highboth in regard to teaching and, through the separate exercise, in
research.

It should be a cause of satisfaction that the huge expan-

sion of the system has been accompanied by improvement in theassessed performance.

Yet within the universities it is apparent thatthe system is deeply stressed.

Inevitably failure to match increased

productivity with funding through the 1990’s expansion shows a cumulative effect.

Perhaps the biggest challenge to sustaining high standards

will be a failure to accept that they entail high – or at least highercosts.

105

Appendix 1
A. Information expected to be available in all universities
1. General statements- Mission statement and corporate plan
- Statement of the University’s quality assurance policies andprocesses- Statement of the University’s learning and teaching strategy andreviews of progress
2. Student admission, progression and completion- Student entry qualifications- Students at entry classified by age, gender, ethnicity, socioeconomic background, disability, geographical origin- Student progression and retention by year of each course
Student completions and qualifications- First destinations of graduates
3. Information on procedures for assuring academic quality andstandards
a. Information on programme approval, monitoring and review.
- Programme specifications- Roles and responsibilities of committees involved in internal
programme approval and review- Results of current approvals and annual monitoring and review
processes- Internal reports of major programme reviews and reviews by
departments or faculties- Accreditation reports from professional, statutory or regulatory
bodies106

Quality Assessment in English Universities

b. Information of assessment procedures- Assessment strategies, processes and procedures- Range and nature of prescribed student work
- External examiners reports and actions taken
Reports of reviews of assessment methodsc. Information covering student views on university services and facilities.
- Academic and tutorial guidance, support and supervision- Equipment, space and facilities for teaching and learning- Perceptions of teaching quality and the range of teaching
methods- Assessment arrangements- Pastoral support
d. Information available for internal reviews of quality and standards.
- Effectiveness of teaching and learning in relation to programmeaims and curriculum content- Range of teaching methods used
- Availability of specialist equipment, resources and materials to
support teaching and learning- Professional development for staff including peer observationand mentoring programmes- Use of external benchmarking and other comparators- Involvement of external assessors in reviews and actions taken
following their comments
B. Information for publication by all universities
1. Quantitative Data
- All data in A2 except classification of students by age, gender,
ethnicity, socio-economic

background, disability, and geographical107

origin2. Qualitative Data
- Summaries of external examiners reports and a commentary at
institutional level on their findings- Feedback from recent graduates, obtained from national survey
data- Feedback from current students obtained by internal survey- Summary statement of university’s learning and teaching strategy- Summary of the results from internal departmental reviews- Summaries of the university’s links with employers and the impact
made by these links on teaching programmes
Notes of Appendix 1

1.

Most of the general statements (A1) and much of the quantitative
data (A2, B1) is already supplied to HEFCE or the Higher Education
Statistics Agency (HESA) and is accessible through their publications.

2.

An extended national First Destination Survey is proposed tosupply the additional information at institutional and departmental
levels that is envisaged in section B2.

3.

A standard form is proposed for internal surveys for the feedback
information from current students (B2).

4.

Information (Section B) will be published through the Higher
Education and Research Opportunities(HERO) web site, which

provides links to each university’s home page.

108

Quality Assessment in English Universities

Appendix 2
Glossary of Abbreviations Used
CVCP

Committee of Vice-Chancellors and Principals (now
UUK)

DfES

Department for Education and Skills
http://www.dfes.gov.uk/index.htm

HEFC

Higher Education Funding Council

HEFCE

Higher Education Funding Council for England

HEQC

Higher Education Quality Council

HERO

Higher Education and Research Opportunities

http://www.hefce.ac.uk/

http://www.hero.ac.uk/
HESA

Higher Education Statistics Agency
http://www.hesa.co.uk/

QAA

Quality Assurance Agency for Higher Education

SED

Self-Evaluation Document

TQA

Teaching Quality Assessment

UUK

Universities UK (successor to CVCP)

http://www.qaa.ac.uk/

Notes1）QAA (2002) Quality assurance in UK higher education-a brief guide.
2）QAA (2002) The Quality Assurance Agency for Higher Education: An

Introduction.
3）QAA (1999) Handbook for academic review. Assessments of research
quality have been conducted separately by HEFCE.
An analysis of theresults and the impact of the research assessment exercises on English
universities are given in Morgan (2002).
4）QAA (1999) QAA Subject review handbook (2000-2001).
5）QAA (2001) Notes for the guidance of audit teams.
6）Ditto.
7）HEFCE(2001) Quality assurance in higher education, Proposals for consul-

109tation, HEFCE, QAA, UUK, Standing Conference of Principals.
8）QAA (2001) Report on responses to HEFCE Consultative document 01/45.
9）QAA (2002) New review system for quality assurance in English higher

education, press release.
10）QAA (2002) QAA external review process for higher education in
England: Operational description.
11）QAA (2001) Higher Quality 9-Reaping the benefits.
12）QAA, Code of Practice for the Assurance of Academic Quality andStandards in Higher Education, sections 1-10, 1999-2001.
13）QAA (2001) The framework for higher education qualifications in England,
Wales and Northern Ireland.
14）QAA, Subject Benchmark Statements, parts 1- 42, 2000- 2002.
15）HEFCE(2002) Information on quality and standards in higher education.
16）QAA (2002) Handbook for institutional audit: England – Draft for
Consultation, 0205/2002.
17）These may be in the form of both discipline audit trails and thematic
enquiries across the whole institution. It is indicated that discipline audit
trails might be expected to cover about 10% of a university’s teaching
programmes.
18）QAA (2002) Annual Report and financial summary.

110