APPLICATIONS VOLUME

AWARDS VOLUME

Volume of research grant applications submitted to external funding
bodiesVolume of research awards granted and available to be spent

Applications Volume calculates the number and price, or amount appliedfor, of research grant applications.

Awards volume calculates the number and value of research awards fromexternal funding bodies.

Applications Volume addresses new applications only. It avoids doublecounting of the same applications by excluding prior submissions in a multistage application process such as outlines and expressions of interest.

Awards Volume considers aggregated values of awards over the award
lifetime. In other words, it considers the total value awarded at the time ofaward and not the value (to be) spent in any particular time period.
This metric includes subsequent financial
amendments to awards, including supplements andreductions, and funding from industry. It does notinclude non-financial amendments such as no-costextensions (i.e. zero-cost extensions).

-+

SUCCESS RATEACADEMIC-INDUSTRY LEVERAGE

Proportion of research grant applications that have been successful

Private investment leverage from public sponsorship

The Success Rate calculated by this method will change over time:• Success Rate by count is calculated according to whether submitted
applications have been awarded or rejected, or whether a decision ispending.
• Success Rate by value is calculated according to the proportion of thetotal requested price associated with awarded or rejected applications orwhether a decision is pending.

Academic-Industry Leverage calculates the total income for publicly
sponsored research projects that are performed in collaboration with at least
one other non-academic organization, and the percentage of funds from
non-academic collaborators that this is used to leverage.

Price used should be the most up-to-date available. Awards that are nottied to an application do not count towards Success Rate.
The year of success is the date of submission to the funder.

A publicly sponsored research project is one which is funded by grant-in-aid
from a Government or other public body. The collaboration should include
material contribution, whether cash or “in kind”, from at least one external
non-academic collaborator.

BUSINESS CONSULTANCY ACTIVITIESRESEARCH STUDENT FUNDING

Volume of business engagements

Sources of funding of tuition fees for research students

Business Consultancy Activities calculates the number and value of businessengagements.

Research Student Funding calculates the number and proportion ofresearch students whose tuition fees are funded by each category of fundertypes.

All consultancy activities where there is income to the institution should beconsidered, regardless of the contract-type of the staff involved. The staff
may be academic staff, or not on academic contracts, such as senior
university managers or administrative/support staff.

This metric is influenced by the composition of the student body, forexample the mix of international and national students, and the range of
disciplines.
This metric is not concerned with the amount offunding received from each category of funder types.
Head count, not FTE, is used as the basis for counting
research students during any reporting period.

INCOME VOLUME

MARKET SHARE

Volume of research expenditure

Percentage of sector total research income per institution

Income Volume calculates the value of awarded budget derived fromresearch awards from external funding bodies that has been spent.

Market Share calculates the percentage of total research income across thesector related to a given institution.

Any research that could not be classified as “Research” under
HESA’s accounting conventions for the finance record (Financial
Reporting Standard 1021) will not be considered in Income Volume.

Any research that could not be classified as “Research” under HESA’s
accounting conventions for the finance record (Financial ReportingStandard 102) will not be considered in Market Share.

£/$
1

https://www.hesa.ac.uk/collection/c15031/coverage#conventions

1

https://www.hesa.ac.uk/collection/c15031/coverage#conventions

CONTRACT RESEARCH VOLUMERESEARCH STUDENT TO ACADEMIC STAFF RATIO

Total value of contract research

Ratio of research students to academic staff involved in researchContract research income is that received from an industrial or private
external body, which is neither a university nor an academic organization,from commissioning a particular piece of research with specific terms.

Research Student to Academic Staff Ratio calculates the average number of
research students per researcher.

Contract research volume excludes:• Research income from external funding bodies, which iscovered in Awards Volume.
• Income associated with research projects that are
publicly sponsored, and that are performed incollaboration with at least one other non-academicorganization, which is covered in Academic-Industry
Leverage.

• This metric is concerned with researchers who are eligible to supervise
research students. This metric is not concerned with undergraduate
students.
• The situation of the particular institution will
determine whether a high or low value of this ratio isdesired, and should be known when using this metric forbenchmarking to enable a proper interpretation.

SCHOLARLY OUTPUT

CITATION COUNT

Productivity based on any type of scholarly output

Total citations received by institution’s scholarly output

Scholarly Output counts the number of institutional outputs of any type.

Citation Count sums the citations received to date by institutional outputs.

Example outputs of the following types are included in Scholarly Output:• Digital or visual media•Journal publications• Exhibitions•Book series• Internet publications•Stand-alone books• Performances• Artefacts• Reports• Compositions• Software• Designs• Devices and Products

It is likely that citation data will not be available for all elements that
constitute an institution’s Scholarly Output.

Outputs of the following types are excluded:• Patents (counted in Intellectual Property Volume)• Theses

A partial reflection of an institution’s activity is still
valuable in providing an evidence-based support for
decision making through benchmarking, since thislimitation is likely to affect all comparators equally.

CITATION PER OUTPUT

h-index

Average citations received by an item of scholarly output

h-index per discipline

Citations per Output calculates the average citations received to date byeach output that is part of a particular set.

The h-index is calculated, as defined by Professor Jorge Hirsch1, for
institutional disciplines. To quote from this paper that defines the h-index interms of researchers: “A scientist has index h if h of his or her Np papers
have at least h citations each and the other (Np – h) papers have ≤h citations
each”.

It is likely that citation data will not be available for all elements that
constitute an institution’s Scholarly Output.

A partial reflection of an institution’s activity is still
valuable in providing an evidence-based support for
decision making through benchmarking, since thislimitation is likely to affect all comparators equally.

A group of papers has an h-index of 17 if 17 of these
papers have each received at least 17 citations, and 18 ofthese papers have not each received at least 18 citations.
The h-index is not calculated at institutional level.
1

Hirsch, J. E. (2005). “An index to quantify an individual’s scientific research output” Proc. Natl
Acad. Sci. 201 (46): 16569–16572.

FIELD WEIGHTED CITATION IMPACTOUTPUTS IN TOP PERCENTILES

Actual citation count relative to the expected world citation countOutputs that have reached a particular citation threshold in the data
universe

Field-Weighted Citation Impact is the ratio of the citations actually received
by the denominator’s output, and the average number of citations received
by all other similar publications.

The citation thresholds that represent the top 1%, 5%, 10% and 25% of
outputs in the data universe being used are established. The absolutecounts, or percentage of total counts, of outputs that lie within eachthreshold is calculated.

A Field-Weighted Citation Impact of:• More than 1.00 means that the output is more cited than
expected according to the global average.

This metrics depends on being able to divide outputs in the data
universe into 100 percentiles. Early in the current calendar year, it
is unlikely that enough citations will have been received byoutputs to enable this, especially at more granular denominatorssuch as disciplines.

• Less than 1 means that the output is cited less than
expected according to the global average.

This metric will only be calculable some months into the current
year, and we suggest from 1 July.

• Exactly 1.00 means that the output performs just as expected for the global
average.

PUBLICATIONS IN TOP JOURNAL PERCENTILESCOLLABORATION

Outputs that have been published in serials with a particular average
citation threshold in the data universe

Volume and proportion of nationally and internationally coauthored scholarly outputs

The average citation thresholds that represent the top 1%, 5%, 10% and
25% of serials in the data universe being used are established. The absolutecounts, or percentage of total counts, of outputs that lie within eachthreshold is calculated.

Collaboration calculates the number and percentage of outputs that havenational or international co-authorship:• An output has national co-authorship if it has an affiliation that does notbelong to the parent institution but is within the parent institution’s
country.
• An output has international co-authorship if it has an affiliation that does
not belong to the parent institution and is outside the parent institution’s
country.

This metric depends on being able to divide the serials indexed by the data
universe into 100 percentiles. Any journal metric that enables this can be
employed in this metric, and this metric could have multiple versions dependingon which journal metric is used to create percentiles from the data universe.
Percentile boundaries are calculated independently for eachpublication year, not overall for the entire data universe, andan output is compared to the percentile boundaries for its
publication year.

• An output that has both national and international coauthorships will be classified as international, to avoid double
counting.
• Countries are defined as in the ISO classification1.
1

http://www.iso.org/iso/ics6-en.pdf

COLLABORATION PUBLICATION SHARECOLLABORATION IMPACT

Proportion of institutional output associated with a particularcollaboration partner

Citation impact of nationally and internationally co-authoredscholarly outputs

Any collaboration partner of interest to the institution can be considered, forexample:Collaboration impact calculates the average citations received by the sets of
output that have national or international co-authorship:• An output has national co-authorship if it has an affiliation that does notbelong to the parent institution but is within the parent institution’s
country.
• An output has international co-authorship if it has an affiliation that does
not belong to the parent institution and is outside the parent institution’s
country.
• An output that has both national and international coauthorships will be classified as international, to avoid double
counting.
• Countries are defined as in the ISO classification1.

• A country.
• A set of countries related by social, political or economic considerations suchas the BRIC countries: Brazil, Russia, India and China.
• Another institution.
• A group of institutions, such as the Ivy League in the US or a set of peers thatan institution has chosen.
Single author papers are excluded.

1

http://www.iso.org/iso/ics6-en.pdf

COLLABORATION FIELD-WEIGHTED
CITATION IMPACT
Field-weighted citation impact of institutional output associated witha particular collaboration partner
Any collaboration partner of interest to the institution can be considered,for example:• A country.
• A set of countries related by social, political or economic considerations
such as the BRIC countries: Brazil, Russia, India and China.
• Another institution.
• A group of institutions, such as the Ivy League in the US or a set of peersthat an institution has chosen.
A Field-Weighted Citation Impact of exactly 1.00 means that theoutput performs as expected, above 1 means it is more cited than
expected and below 1 means it is cited less than expected,
according to the global average.

ACADEMIC-CORPORATE COLLABORATIONIMPACT
Citation impact of scholarly outputs co-authored by researchers
from both academic and corporate affiliations
Academic-Corporate Collaboration Impact calculates the average citations
received by the outputs that have been co-authored by researchers fromboth academic and corporate, or industrial, affiliations.
The assignment or not of an academic-corporate collaborative output
applies only to the institutional outputs.
The count of citations is independent of the collaboration status of theciting output.

ACADEMIC-CORPORATE COLLABORATION
Volume and proportion of scholarly outputs co-authored byresearchers from both academic and corporate affiliations
Academic-Corporate Collaboration calculates the number and percentage of
outputs that have been co-authored by researchers from both academic andcorporate, or industrial, affiliations.
It is likely that affiliation and affiliation-type data will not be available for all
elements that constitute an institution’s Scholarly Output.
A partial reflection of an institution’s activity is still valuable
in providing an evidence-based support for decision making
through benchmarking, since this limitation is likely to affectall comparators equally.

ALTMETRICS – SCHOLARLY ACTIVITY
Number of times an institution’s output has been posted in online
tools typically used by academic scholars
This metric answers the question of how often an institution’s output has
been downloaded, bookmarked or captured in online tools with theintention to come back to use the output at a later date.
Example online tools include:• Mendeley• Figshare• SSRN• Academia.edu• ResearchGate

ALTMETRICS – SCHOLARLY COMMENTARY
Number of times an institution’s output has been commented on in
online tools typically used by academic scholars

ALTMETRICS – SOCIAL ACTVITY
Number of times an institution’s output has stimulated social media
postsThis metric answers the question of how often an institution’s output has
been mentioned in online tools typically used by scholars to discuss, engage
or review research outputs.

This metric answers the questions of how often an institution’s output has
stimulated attention or buzz in social networking services or how well an
institutions researchers are promoting their work through social activity.

Example online tools include science blogs, video posts such as those on
YouTube, Stack Exchange and Wikipedia.

Example services include:• Facebook• Twitter• LinkedIn• delicio.us

ALTMETRICS – MASS MEDIA

PUBLIC ENGAGEMENT

Number of times an institutions output has been referred to in mass
media outlets

Attendees at public events

This metric answers the questions of how often an institution’s
output has been mentioned in press clippings or news websites, suchas The Guardian newspaper.

Public Engagement calculates the number of attendees at public events.
It answers the question of what an institution’s wider social and cultural impact
on their region and nation is.
They include knowledge, facility, and cultural awareness events, regardless ofwhether the events were chargeable or free.
Open days, student union activity and commercial conferences are excluded.

ACADEMIC RECOGNITIONINTELLECTUAL PROPERTY VOLUMEElected fellowships of national academy of sciences

Volume of patents and licenses

Academic Recognition calculates the number of elected fellowships to thenational academy of sciences.

Intellectual Property includes copyrights, trademarks, design rights, trade
secrets and patents for the protection of inventions, and licenses granted to
private companies allowing them to exploit an institutional invention that is
protected by a patent.

Elected fellowships are counted at the university at which the academic isbased, and move with the academic. No historic assignment to previous career
universities is made.
The number of elected fellowships is counted, not the number of researchers.
Only awards that are open internationally should be counted in this recipe.
The following are not counted:• Prizes.
• Paid memberships.

The number of patents granted includes all individual patents, and any
individual national patents.
The number of licenses granted includes those granted from licenceagreements, assignments, exercised option agreements, licences to spinouts and income-generating Material Transfer Agreements.

INTELLECTUAL PROPERTY INCOMESUSTAINABLE SPIN-OFFS

Revenue from patents and licenses

Number of sustainable spin-offs

It answers the question of how much commercial return an institution isderiving from its interactions with a range of external partners.
Intellectual Property Income is that received by the institution fromupfront fees, milestone fees, royalties, and reimbursement of patent costs.
Income from design interactions and licensing is also included.

A spin-off is a company that has been set up to exploit intellectual property thatoriginated from within the institution.
The types of spin-off counted in this recipe are those for which the definition is
specific and not open to interpretation, and where the institutional data quality
upon which the metric is based are relatively high.

Spin-offs where the definitions are less specific and the data quality isconsequently relatively low are excluded.
Sustainable spin-offs are active companies that havesurvived for at least 3 years

SPIN-OFF RELATED FINANCESTIME TO AWARD OF DOCTORAL DEGREE

Financial benefits derived from active spin-offs

Time taken to be awarded a doctoral qualification as a proportion of
expected course length

A spin-off is a company that has been set up to exploit intellectual property
that originated from within the institution.
The types of spin-off counted in this recipe are those for which the definitionis specific and not open to interpretation, and where the institutional data
quality upon which the metric is based are relatively high.
Spin-offs where the definitions are less specific and the data quality isconsequently relatively low are excluded.

Active spin-offs are those which are currently active, regardlessof the number of years that they have existed.
External investment includes all investment from external
partners, with the exception of third-stream funds.

DESTINATION OF RESEARCH STUDENT LEAVERS
Research students continuing in an academia following successful
completion of course
Destination of Research Student Leavers calculates the proportion ofresearch students who continued to work in the university sector,
compared to moving into other career paths.
This metric is concerned only with successful research students.

• A Time to Award of Doctoral Degree ratio of 1 means that the expected
time was taken to have the degree awarded; above 1 means longer than
expected; and below 1 means shorter than expected.
• Where there are both part-time and full-time students following a
programme of study, the expected length of study should be the normal
length applicable for the mode of study (part-time or full-time) of thestudent in question.