BIBLIOMETRIC REPORT

Bibliometric study of the Swiss Federal Institute for
Forest, Snow and landscape Research WSL (20092020/2021)
August 22, 2022

Bibliometric study of the Swiss Federal Institute for
Forest, Snow and landscape Research WSL (20092020/2021)

Report for the ETH Board
Nicolas Leclaire
Häldeliweg 158092 Zurich, Switzerland
E-mailnicolas.leclaire@ethrat.ch
Webpage https://www.ethrat.ch/en

CWTS
Ed Noyons, Clara Calero, Rodrigo Costas, Jeroen van Honk
CWTS B.V.
P.O. Box 905
2300 AX Leiden, The Netherlands
Tel+31 71 527 5806
E-mailinfo@cwts.nl
Webpage http://www.cwtsbv.nl/

General parameters of the bibliometric reportParameters
Database:

VersionClassification system::

Publication window
Citation window
Counting Method:::

Self-citations
Top indicators::

Web of Science (Articles, Reviews and Proceedings
papers in the SCIE, SSCI, AHCI, and CPCI)
2152 (CWTS)
Publication-level classification system (about 4000
fields, referred to as research areas)
2009–2020
Maximum 4 years (and until 2021)
Fractional counting at the level of organisation forcitation impact measurement
ExcludedTop 10%Contents
List of Indicators

6

Definitions, abbreviations and acronyms

8

1 Introduction

9

2 Data collection and methodology
2.1 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Summary of method . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.1 Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.2 Additional indicators . . . . . . . . . . . . . . . . . . . . . . . .
2.2.3 Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11
11
11
11
14
18

3 Results
3.1 Overall output and impact . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.1 Overall performance . . . . . . . . . . . . . . . . . . . . . . . .
3.1.2 Trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Research focus in context . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.1 Research profile . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.2 Female author contribution across subjects . . . . . . . . . .
3.2.3 Interdisciplinary research across subjects . . . . . . . . . . . .
3.3 Collaboration and partners . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 Collaboration profile . . . . . . . . . . . . . . . . . . . . . . . .
3.3.2 Collaboration within the ETH Domain . . . . . . . . . . . . .
3.3.3 Collaboration outside the ETH Domain . . . . . . . . . . . . .
3.4 Research accessibility . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.1 OA publishing and impact . . . . . . . . . . . . . . . . . . . . .
3.4.2 OA publishing and impact by subject . . . . . . . . . . . . . .
3.5 Impact and knowledge use . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.1 Impact and knowledge use at country level . . . . . . . . . .
3.5.2 Impact by citing institution . . . . . . . . . . . . . . . . . . . .

20
21
21
24
28
28
29
30
32
32
34
36
39
39
42
44
44
46

Annexes49

A WSL’s author gender statistics

50

B Publication level classification
B.1 Journal scope (including multi-disciplinary journals) . . . . . . . . . .
B.2 Granularity of the WoS subject categories . . . . . . . . . . . . . . . .
B.3 Multiple assignment of journals to categories . . . . . . . . . . . . . .
B.4 The CWTS publication-based classification scheme . . . . . . . . . .

51
51
51
51
52

4

www.cwtsbv.nl

CONTENTS
C Citation data and analysis
C.1 Database coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.2 Database Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.3 Citation Window, Counting Method and Field Normalisation . . . .

54
54
54
55

D Interdisciplinary research57

www.cwtsbv.nl

5

List of indicators
Avg Reads Average number of reads per DOI. A read is defined by saving a publication in a Mendeley user account.
IntCov Internal coverage. Estimated Web of Science coverage of a set of publications. A description of the calculation is provided in Annex C.1.
IntDisc Measure of interdisciplinary research, defined by the proportion of references in a publication assigned to other fields. Fields are defined by journal
categories. In addition, the cognitive distance of fields to each other is also
considered (more info at Section 2.2 (p. 16) and Annex D).
MCS Mean citation score. The average number of citations received by a publication (TCS/P[full]).
MNCS The mean normalised citation score. This represents average citation scoreper publication, normalised by research area and publication year. Researchareas are defined by a detailed publication classification system of CWTS,
consisting of about 4000 areas. The average MNCS in the entire databaseis 1. Scores higher than 1 reflect a citation-based impact that is higher than
the world average.
MNJS The mean normalised journal score. This represents the normalised average
citation impact of journals. The MNJS is an average score for all publicationsin the same journals in which an institution published. The normalisation isbased on the same principles as the MNCS. The average MNJS in the entire
database is 1. Scores higher than 1 reflect a journal citation impact that ishigher than the world average.
P[full] The number of publications, full counting. Each publication is counted infull (i.e. as 1).
P[fract] The number of publications, fractionally counted. The fraction is determined based on the number of co-authoring organisations.
P[OA] Number of publications, full counting, in Open Access(OA). In addition, we
provide the number for the different kinds of OA: Gold, Hybrid, and Green.
A publication is tagged by one type only. Gold and Hybrid overrule Green.
Information is based on Unpaywall data (July 2021).
PP[OA] The proportion of publications in Gold, Hybrid or Green OA, while publications without a DOI are discarded (OA unknown).
PP[collab] Proportion of publications, full counting, involving collaboration (atleast two institutions co-authoring).
6

www.cwtsbv.nl

PP[int collab] Proportion of publications, full counting, involving international collaboration (co-authorship of organisations from more than one country).
PP[industry] Proportion of publications, full counting, involving industry (co-authorshipwith companies).
PP[uncited] Proportion of publications, full counting, that are not cited.
PP[self cits] The average number of author-self citations per publication. A selfcitation is defined as any of the authors of a cited publication is the same as
any of the authors of the citing publication.
P[top10%] The number of publications, counted in full belonging to the top 10%of their research area. The area is determined on the basis of a detailed
publication classification system of CWTS, consisting of about 4000 areas(See Annex B).
PP[top10%] The proportion of publications (P[fract]) belonging to the top 10% mostcited of their area and in the same year. The areas are determined using
a detailed publication-level classification system , consisting of about 4000
areas. The PP[top10%] in the entire database is 10%. A score above 10%
represents impact that is higher than the world average.
PA[F inst] Share of female authors of an institution within a publication.
PA[F pubs] Share of female authors within a publication (institution plus coauthors).
A[M inst] Number of male authors of an institution.
A[FM inst] Number of authors of an institution for which we could define gendermale or female.
RPA[F] Proportion of female authors compared to the total of authors for which
gender (male or female) was defined (more info at Section 2.2).
TCS The total citation score. This represents the total number of citations accumulated within the citation window, excluding author self-citations.
For more details about the normalised citation indicators, please refer to Waltman
et al. (2012). More information about the mentioned publication-level classification
is in Annex B.

www.cwtsbv.nl

7

Definitions, abbreviations and acronyms
CWTS Centre for Science and Technology Studies, Leiden University
A&HCI Arts & Humanities Science Citation Index
SCIE Science Citation Index Expanded
SSCI Social Science Citation Index
CPCI Conference Proceedings Citation Index
DOI Digital Object Identifier (a permanent ID for publications)
JSC Journal Subject Category
OA Open AccessResearch area A set of publications on a certain topic, identified by the Leiden
Algorithm (Annex B)
Subject A set of publications in journals belonging to a (subject) category
WoS Web of Science8

www.cwtsbv.nl

Introduction

1

Introduction
The ETH Domain consists of two Federal Institutes of Technology, ETH Zurich and
EPFL, and four research institutes PSI, WSL, Empa and Eawag. Together, theyplay a vital role in the Swiss science system for education, research and transfer
of knowledge and technology.
The ETH Board commissions an intermediate evaluation every four years. The mostrecent one took place in 2019. The bibliometric study was executed in 2018. The
evaluation is a moment for the Swiss Federal Council, the ETH Board, as well as
staff and management of ETH Domain to find out where ETH Domain stands visa-vis the ambitions and measures formulated in the strategic planning document.
Moreover, the intermediate evaluation should lead to recommendations relating tothese ambitions and measures.
Bibliometric studies can provide evidence related to ambitions and measures aspart of a self-assessment report. Although we consider that meeting the standardsof objectivity for determining the impact of scientific research is important, we believe that decision-making towards the goal of evaluating the quality of institute’s
research ought to be multi-dimensional rather than overwhelmingly quantitative.
Bibliometric measures provide objective evidence about production, collaborationand impact but only for the research that has been published in (international)
journals and proceedings. Therefore, we strongly recommend that quantitative evaluations are complemented with qualitative information (for example the mission andthe research goals of a department) and expert assessments.
This report includes the bibliometric analysis of the scientific output of WSL, covering the period 2009-2020, including citations up to 2021. The studies are basedon a quantitative analysis of scientific publications in journals and proceedings
processed for the Web of Science (WoS) versions of the Science Citation Index
and associated citation indices: the Science Citation Index (SCI), the Social Science Citation Index (SSCI), the Arts & Humanities Citation Index (A&HCI) and theConference Proceedings Citation Index (CPCI).
Although most of the methodology is similar to the study performed four years ago
for WSL, the results may sometimes differ substantially, due to the fact that in thecurrent report conference proceedings papers are included and fully integrated, butthat depends on the role conferences play for an institution if this is actually thecase. Moreover, new indicators were introduced: RPA[F], IntDisc, P[OA], PP[OA],
and Avg Reads.
We introduce each result in brief, while more detailed information about data andmethod is provided in Section 2 and Annex C) of this report.
In Section 3 the results of our analysis and interpretations are reported. These
results are discussed in 5 parts:
www.cwtsbv.nl

9

Introduction

1. Section 3.1: Overall output and impact
2. Section 3.2: Research focus in context
3. Section 3.3: Collaboration and partners
4. Section 3.4: Research accessibility
5. Section 3.5: Impact and knowledge use.
In the annexes, we provide more detailed scores for some indicators, more detailed information about specific approaches, as well as information about CWTS
infrastructural elements involved in the analyses.

10

www.cwtsbv.nl

Data collection and methodology

2

Data collection and methodology

2.1 Data collection
WSL provided CWTS with a list of publications from its own repository. CWTS
used these data to match the publication records with the records in its database(matched results). Simultaneously, CWTS collected WSL’s publication data fromits database using the author affiliations in publications. Both data sets werecompared to each other.
After WSL and CWTS compared, checked and corrected these two sets, the final
dataset was prepared for the bibliometric analysis.
Additionally, for the Mendeley readership analysis WSL provided CWTS with any
DOI registered in its repository.

2.2 Summary of method
In this section, we discuss the methods underlying the bibliometric analysis developed. We discuss the basic principles of our indicators and the context in which
they can (or should not) be used. Additional and more detailed information about
methods and data can be found in the annexes.

2.2.1

Indicators

In bibliometric analyses regarding research performance, we usually discern two
types of indicators: size-dependent and size-independent, taking into account thedifferent size of institutions under investigation. Larger institutions, for instance, will
be involved in more publications than smaller ones. Subsequently, this will affectthe absolute number of top 10% publications, as well as all other size-dependent
indicators. In Figure 1 we visualise the correlation between the two indicators forthe 6 ETH institutions.
www.cwtsbv.nl

11

Data collection and methodology

16,
000
15,
000
14,
000
13,
000
12,
000
11,
000

P[
t
op10%]

10,
000
9,
000
8,
000
7,
000
6,
000
5,
000
4,
000
3,
000
2,
000
1,
000
0
0

5,
000

10,
000

15,
000

20,
000

25,
000

30,
000

35,
000

40,
000

45,
000

50,
000

55,
000

60,
000

65,
000

70,
000

75,
000

P[
f
ul
l
]

Figure 1: P[full]vs.P[top10%]for 6 ETH institutions

Proportion indicators (e.g., PP[collab], PP[int collab], PP[industry], PP[OA], PP[top10%])and average indicators (MNCS, MNJS) are size-independent, while others used inthis study (e.g., P[full], P[fract], TCS) are size-dependent. In the report we will
primarily discuss the results using the size-independent indicators to account forthe size differences of the organisations. Moreover, the results for size-independent
indicators can, in most cases, be related to the world average.
Output indicators
Size-dependent
The total number of publications in which researchers from an institution wereinvolved (P[full]) is the basic output measure. In addition, we provide the indicator P[fract] which assesses an institution’s contribution to the output P[full]. Eachindividual publication is divided by the number of organisations co-authoring, regardless of the number of organisations involved. If authors have two affiliations
and mention both, both affiliations are counted as fractions. P[fract] is the sum ofthese fractions of publications in which an institution was involved.
Size-independent
Proportion indicators characterise sets of publications regardless of the number andare therefore size-independent. They are often used to characterise output. For12

www.cwtsbv.nl

Data collection and methodology
instance, PP[collab] indicates the proportion of output with at least two different
organisations involved. PP[int collab] indicates the proportion of output involving
international collaboration. In this report, a publication is tagged as an international
collaboration if at least one of the co-authoring organisations is based outside of
Switzerland. Furthermore, two other proportion indicators are used: PP[industry],
representing the proportion of P[full] co-authored with a company and PP[OA], theproportion of P[full] published in Open Access (OA).
For OA publications, we discern different types: OA Gold, OA Hybrid and OA
Green. The definition of the types used in this report are:• Gold: The publisher makes all articles and related content available for free
immediately on the journal’s website.
• Hybrid: Publication freely available under an open license in a paid-accessjournal.
• Green: Published in toll-access journals, self-archived by authors (in repositories or researchers’ websites), independently from publication by a publisher.
OA publications are counted only as one type at the same time. If a paper is bothGreen and Gold, it is counted as Gold. Bronze OA publications are free to read
only on the publisher page without a license. As such, they were disregarded as
OA. These were identified as Closed Access publications.
Impact indicators
Size-dependent
The scientific impact of an institution’s output is measured by citations. We provide
the total number of citations received (TCS) in the period of maximum 4 years after
publication, up to 2021. For more recent years the citation window is shorter than
4 years. We exclude author self-citations. Another size-dependent indicator of
impact is P[top10%], i.e. the absolute number of publications belonging to the top
10% most cited publications (in their area and from the same year).
It should be noted that all citation-based indicators (including TCS) are calculatedusing a limited and fixed time-window. The total amount of citations for early
publications may therefore be higher than processed for this report.
Size-independent
The MNCS is the indicator to measure citation impact after normalising by researcharea and publication year. The research area to which a publication belongs isdefined by a publication-level classification (for details, see Annex B). In this classification each publication is uniquely assigned to a research area. Areas are definedwww.cwtsbv.nl

13

Data collection and methodology

by their citation environment (cited and citing publications). This classification ismore fine-grained and is considered more accurate than a journal classification(Ruiz-Castillo and Waltman, 2015). In a journal classification all publications fromone journal are in the same class. Similar journals are in the same class andjournals may belong to more than one class. We use this journal classification
to characterise an institution’s output in its research profiles but not to normalise
impact. The journal classification is less fine-grained and as such easier to relate
to the main subjects addressed.
In addition, we provide the proportion of publications in the top 10% most cited publications (within their research area, i.e. class, and in the same year, PP[top10%]).
This indicator correlates strongly with the MNCS but is not sensitive to outliers.
The MNCS can sometimes be biased by one paper being cited many times. The
PP[top10%] is not influenced by this one paper, as it is ‘just’ one of the top 10%
or not. An MNCS that is relatively much higher than the PP[top10%] points toa highly skewed distribution of impact across publications. In other words, a few
publications receive a huge number of citations, compared to the other publications.
Finally, we also use an indicator measuring the impact of journals, the Mean
Normalised Journal Score (MNJS). This indicator assesses the impact in term ofcitations of the journals (aggregated), in which the institution has published, using
the same normalisation as we use for measuring the impact (MNCS). As such, the
MNJS does not measure the (average) impact of an institution’s publications, butrather the impact of the journals in which its researchers publish.
2.2.2

Additional indicators

In this study we introduce indicators that relate to the context of the publishedresearch. We will discuss them in brief in the next subsections.
Worldwide growth of research fields
An indicator to position an institution’s research activities in the context of what
happens at a larger scale is the [Field growth]. We use the science landscape (seeAnnex B) to reflect what happens worldwide, by calculating a growth indicator foreach area (the [Area Growth]).
The [Field growth] relates the output of an institution to these area growth values([Area Growth]) as follows. First, we calculate for each of the 4000 research areasin the science landscape, the share output of the most recent two years (20192020) as compared to the total in 2009-2020 (the period under study). This shareof output in the most recent years is normalised by a reference value, which is theresult of the number of recent years (2) and the number of years of the total period
considered (12): 0.17. Areas in which the share of output in the recent years is14

www.cwtsbv.nl

Data collection and methodology

higher than 0.17, have a [Area Growth] above 1, a positive growth.
Any value above 1 means a positive growth, while values below 1 indicate a negative
growth. In Figure 2, we plotted the [Area Growth] in the landscape of all science,by color-coding. Green areas show a positive growth (>1) in the most recent
two years, while red areas show a negative growth (<1). The size of a circle
proportionally reflects the number of ETH Domain publications published in 20092020 worldwide, ranging from 1 up to 1,400.

Rel
at
i
veAr
ea.
.
0.
00

2.
00

Figure 2: Landscape of all science, color-coded by [Area Growth][Field growth]
We use the [Area Growth] to characterise the fields in which WSL researchers
are active. Thus we contribute to the answer to the question: is WSL’s research
positioned in fields with an increasing interest worldwide or not?
The [Field growth] is the average of [Area Growth] values of the areas in which
an institution’s publications can be found. Consider the output of an institution
X, with 100 publications. These 100 publications may be in 20 different areas.
Depending on the [Area Growth] values of these areas, these 100 publications
relate to 20 different [Area Growth] scores. The average [Area Growth] values of
the 100 publications, then indicates the estimated growth of fields in which X is
active: the [Field growth] of institution X.
www.cwtsbv.nl

15

Data collection and methodology
Interdisciplinary research
We introduce a measure related to the interdisciplinary character of the publishedresearch. Being more or less interdisciplinary is defined by the knowledge base(the prior art that is being cited) of the published research. The content of cited
publications is defined by the journal subject categories.
If a publication cites research from one (and most likely its own) subject category
only, it is defined as mono-disciplinary (measure close to 0). If a publication cites
research from different subjects, we consider it as interdisciplinary. If the subjects
are cognitively at a long distance from each other, the measure of interdisciplinarity
is even higher, with a maximum of 1.
The cognitive distance between subject categories is determined by the density of
the citation traffic between them. If a publication (A) cites output in subject X and Y,
while X and Y are remote from each other (little citation traffic between them), it isconsidered more interdisciplinary than publication B, which cites publications fromY and Z, which are cognitively closely related (i.e., in subject categories frequently
citing each other).
For each publication we calculate an interdisciplinary value and for sets of publications we then calculate their average (IntDisc), which is a value between 0 and
1, where 0 indicates mono-disciplinary and 1 means maximum interdisciplinarity.
In summary, interdisciplinarity is:
1. Defined by cited references in a publication;
2. On the basis of the variety of journal categories of cited publications;
3. Considering cognitive distance between these categories;
4. While this distance between categories is based on mutual citation traffic.
The above leads to the definition of interdisciplinarity we use in this report:
The interdisciplinarity indicator (IntDisc) relates to the diversity of
research supporting the current research.
In order to be able to interpret the IntDisc measure in a broader context, we
calculated a reference value (Ref Intdisc), which is the IntDisc for the journal
category at large in 2020. In this way interdisciplinarity can be assessed withineach journal subject category by relating it to the world average. We integrated
both scores (IntDisc and Ref Intdisc) in profiles, where interdisciplinarity is included.
More info can be found in Annex D.
16

www.cwtsbv.nl

Data collection and methodology
Share of female authors
We also introduce an indicator related to gender diversity of research staff. We
calculated the probability of an author name to be male or female, by looking at
the first name. If first names (or nicknames) point to a gender within a specificcountry, the gender is set using the following four-step procedure (also described
at CWTS Leiden Ranking):
1. Author disambiguation. Using an author disambiguation algorithm developed
by CWTS (Caron and van Eck, 2014), authorships are linked to authors. If
there is sufficient evidence to assume that different publications have beenauthored by the same individual, the algorithm links the corresponding authorships to the same author.
2. Author-country linking. Each author is linked to one or more countries.
If the country of the author’s first publication is the same as the country
occurring most often in the author’s publications, the author is linked to thiscountry. Otherwise, the author is linked to all countries occurring in his or
her publications.
3. Retrieval of gender statistics. For each author, gender statistics are collectedfrom three sources: Gender API, Genderize.io , and Gender Guesser. Gender
statistics are obtained based on the first name of an author and the countries
to which the author is linked.
4. Gender assignment. For each author, a gender (male or female) is assigned
if Gender API is able to determine the gender with a reported accuracy ofat least 90%. If Gender API does not recognize the first name of an author,
Gender Guesser and Genderize.io are used. If none of these sources are able
to determine the gender of an author with sufficient accuracy, the genderis considered unknown. For authors from Russia and a number of othercountries, the last name is also used to determine the gender of the author.
Using the above procedure, the gender can be determined for about 70% of all
authorships of major universities. For the remaining authorships, the genderis unknown.
For each publication, we counted the number of female authors at the level of theinstitution (A[F inst]) as well as at the level of the entire publication (A[F pubs]).
In addition we counted those for male authors. We disregarded authors for which
the gender cannot be defined or is ambiguous. The total amount of authors which
we defined female or male is indicated by A[FM inst] and A[FM pubs].
Hence, for each publication in which WSL authors were involved, there is a share offemale WSL authors (PA[F inst]), and a share of female authors for the publication
at large (PA[F pubs]). The latter is used as a benchmark for the former. RPA[F]www.cwtsbv.nl

17

Data collection and methodology

indicates the WSL share, normalised by the share of the benchmark. A value higher
than 1 for an institution X, indicates a higher proportion of female authors at X
than for its community at large (X plus co-authoring partners).

2.2.3

ProfilesIn the report we use two types of profiles:
1. A research profile in which we look at performance of an institution on thelevel of journal categories; and2. A collaboration profile in which we look at performance of an institute ofthree collaboration types of publications.
In a research profile, we breakdown the WSL output into Journal Subject Categories(JSC) to add content to the general statistics. It gives a general impression of all
the broad subjects in which WSL is involved. We include categories that cover at
least 1% of the total output (P[full]).
For collaboration profiles, we classify all publications by their author affiliation information. The different types of collaboration are: (1) Single institution, in which
only the institution under study is involved, (2) National collaboration for publications with co-authors from at least two different institutions from the same country,
and (3) International collaboration for publications co-authored by institutions fromat least two countries.

OutputBy breaking down the output over categories, we provide a broad overview ofactivities and focus, by subject. In each profile we include both P[full] and P[fract],
i.e. the number of publications in which an institution was involved (P[full]) andthe number of publications normalised by the number of institutions involved asco-author (P[fract]). Moreover, if a publication is in a journal that belongs to two
categories, it is assigned 0.5 to each category. In addition, we include an estimatedgrowth factor for each subject [Field growth]. This growth factor is calculated on
the basis of developments of research areas (see Section 2.2.2). A [Field growth]
above 1 means a growth of output worldwide in the most recent two years.
By breaking down an institution’s output over collaboration types, we provide insight into the publication strategy, as well as the integration of an institution into
the national or international research community. Large shares of international
collaboration output (P[full] and P[fract]) point to a strong international network.
18

www.cwtsbv.nl

Data collection and methodology
ImpactIn both types of profiles, the impact of individual publications is measured in thesame way as for the entire institution (PP[top10%], MNCS and MNJS) and broken
down over subjects and collaboration types. In the research profile, we rank subject
categories on the basis of P[full] (using full counting). In this way we depict an
institution’s main focus by the number of publications in which its researchers are
involved, while the impact is measured by the proportion to which it contributes,
hence consistent with the overall impact measurement.
Research profiles in other contexts
We also used the breakdown over subject categories to provide more detailed
information on the context in which research is executed and published. The mainindicators we provide by subject are:
• RPA[F]: the share of Female authors relative to a benchmark• P[OA], PP[OA]: the number and share of publications in OA• IntDisc: the measure to which research is interdisciplinary• PP[collab]: the proportion of output involving collaboration• PP[int collab]: the proportion of output involving international collaborationwww.cwtsbv.nl

19

Results

3

Results

20

www.cwtsbv.nl

Results

3.1 Overall output and impact
Main findings
The overall output of WSL amounts up to 4,936 publications, with theoverall number of publications increasing over time. WSL exhibits anoverall high citation impact, with field-normalised impact above theinternational reference values (MNCS values always above 1.37 andPP[top10%] above 14%). WSL’s publications are predominantly performed in collaboration (91%), with a predominant role of international
collaboration (74%), and about 5% involving collaboration with industry. The scientific production of WSL is mostly published Open Access(61%), showing an increasing pattern over time towards more openness.
WSL contributes somehow to research areas of all the 5 main disciplines of the science landscape, however WSL has very stronger focus
on topics related to Life & Earth Sciences.
3.1.1

Overall performance

In Table 1 the overall bibliometric statistics for WSL are presented. Overall WSL
has produced a total of 4,936 publications, of which 4,808 are journal papers and
128 are proceeding papers. The overall internal coverage (IntCov) is 0.77, meaningthat about 77% of WSL cited references are themselves also covered in the Web of
Science database, implying that the topics researched by WSL can be considered as
being well covered by the database chosen (i.e. Web of Science) for this bibliometricstudy.
WSL publications have received a total of 61,350 citations (excluding self-citations- which roughly represent 29% of all citations). The vast majority of citations
are concentrated around journal papers, with a mean citation impact (MCS) of
12.74. The mean overall citation impact of the proceeding papers is much lower(MCS=0.63) which can be explained by the shorter nature of proceeding papers,
making them less prone to receive citations, which is also supported by the rather
high percentage of uncited proceeding papers (PP[uncited]=67%).
When it comes to field-normalised citation impact, the MNCS value of WSL israther high with a value of 1.42, meaning that WSL field-normalised impact is 42%
higher than it would be expected by its international expected baseline. Proceeding
papers have a particularly high normalised impact (MNCS=1.71), indicating that
although this document type is not especially prone to accrue citations, WSL isstill having a high citation impact in its set of proceeding papers.
When analysing the production of highly cited outputs, WSL has produced 1,020 top
10% highly cited publications (P[top10%]=997 of journal papers and P[top10%]=23
of proceeding papers), meaning that WSL has produced about 16% of its contribuwww.cwtsbv.nl

21

Results

tions with high impact (PP[top10%]=16%).
More than 60% of WSL publications have some form of Open Access (PP[OA]=61%).
Proceeding papers are slightly more often published in OA as compared to journal
papers, with 63% of proceeding papers with some form of OA version.
WSL publications are mostly performed in collaboration, with about 91% of its
outputs with some degree of institutional collaboration (PP[collab]=91%), and 74%
of all WSL publications involving co-authors from more than one country (PP[intcollab]=74%). In the case of collaboration with industry (indicator PP[industry]),
about 5% of all WSL publications are performed in co-authorship with industrial
partners. Proceeding papers, although with an overall lower presence of institutional collaboration (PP[collab]=70% in contrast with 91% of journal papers) as wellas international collaboration (PP[int collab]=45% vs. 75% of journal papers), exhibit a higher presence of collaboration with industrial partners (PP[industry]=12%
vs. 4% in journal papers). This may suggest a potential role of proceeding papers
at WSL as conveyors of more local and industry-oriented research.
Finally, WSL’s publications’ level of interdisciplinarity is captured by the indicator
IntDisc(0.41). Compared to the overall value of the ETH Domain (IntDisc=0.35), it
can be argued that WSL has a higher degree of interdisciplinary than the domain
at large. In Section 3.2 we will discuss the IntDisc values in more detail.
Most of the bibliometric results in Table 1 are provided by document type (proceedings and journals). Readership and author gender statistics are presented at theoverall level only. Readership results are based on provided DOIs which were notclassified by these types, while author gender could be defined in journal papers
only. The results for these indicators are in their proper section (Section 3.2 and3.5).
Overall, 24% of the WSL authors is female (1,952 vs 6,100 male, PA[F inst]: 0.24),
which is high but 10% below the benchmark (all co-authors in the WSL output,
PA[F pubs]: 0.27). The share of female author for the ETH Domain is 20%. The
average number of reads (Avg Reads) is 7.56, while the Avg Reads for ETH Domain
is 5.09.

22

www.cwtsbv.nl

Results

Table 1: Overall bibliometric performance statistics WSL
Indicator

Journals

Proceedings

Overall

128
72
0.56
0.41
24
63%

4,936
1,670
0.77
0.41
2,927
61%

91%
4%
75%

70%
12%
45%

91%
5%
74%

61,270
12.74
997
16%
1.41
1.30
29%
6%

80
0.63
23
17%
1.71
1.17
26%
67%

61,350
12.43
1,020
16%
1.42
1.30
29%
8%

OutputP[full]P[fract]
Int Cov
InterDiscP OA [Gold, Hybrid, Green]
PP [OA]

4,808
1,598
0.78
0.41
2,903
61%
Collaboration

PP[collab]
PP[industry]
PP[int collab]
Citedness
TCS
MCS
P[top10%]
PP[top10%]
MNCS
MNJS
PP[self cits]
PP[uncited]

Author genderA[F inst]
A[M inst]
PA[F inst]
PA[F pubs]
RPA[F]

1,952
6,100
0.24
0.27
0.90
Readership

N reads
N pubs read
Avg Reads

20,882
2,763
7.56

The landscape in Figure 3 is a two-dimensional representation of all science (covered by WoS) with an overlay of the output by WSL researchers in the different
research areas. In Annex B we provide a more detailed description of the landscape and the way it is created. The size of a circle reflects the relative number ofpublications in which WSL researchers were involved. The colors in the landscapepoint to 5 main disciplines we use to support the interpretation of the landscape.
Figure 3 captures the topical distribution of WSL publications across all the rewww.cwtsbv.nl

23

Results

search areas of the publication-level classification system of CWTS. As can be seen
WSL has contributed to areas of all the 5 main disciplines of the classification system, although there is a pronounced concentration of publications in the areas ofLife & Earth Sciences, with only very marginal and sparse contributions to the otherfour disciplines. Via this link you can open a web-based version of the landscapein your browser. By opening the menu on the left, you can change the perspective
to any of the six ETH institutions.

Soc
i
alSc
i&HumanBiomed&Heal
t
hSc
i
Physi
c
alSc
i&Engi
n
Li
f
e&Ear
t
hSc
i
Mat
hs&ComputSciFigure 3: Distribution of WSL’s output across landscape of science (interactive
version via this link)

3.1.2

TrendsTable 2 below presents the trend analysis of WSL by overlapping four-year periodof the indicators previously considered. Figure 4 captures the trend evolution of the
Journal papers of WSL, while Figure 5 captures the trend of proceeding papers.
In general, a sustained increasing trend in the number of journal papers publishedby WSL is observable in Figure 4, with WSL almost doubling its output of journal
papers between the period 2009-2012 and 2017-2020. Proceeding papers however
exhibit some sort of decreasing trend from the first period of 2009-2012 until thelast period of 2017-2020 (Figure 5), although it is important to remark the rather
low number of this type of publication, which makes it more prone to temporalfluctuations.
24

www.cwtsbv.nl

Results

In addition to the number of publications, WSL also exhibits patterns of increase
in indicators such as IntCov, suggesting an increasing focus on research covered in
Web of Science. The growth in the indicator IntDisc indicates an increase in theinterdisciplinarity of the research of the institute. The proportion of OA publications(PP[OA]) has also substantially increased from 49% in the period 2009-2012 toabout 70% in the most recent period 2017-2020.
The overall impact of the institute as measured by the TCS indicator shows asustained increase from the initial period 2009-2012 up to the period 2016-2019.
There is a slight decline in the overall TCS impact of WSL in the more recent period(2017-2020). This decline could be partly attributed to the time lag indexing of
publications and citations in Web of Science.
The share of female authors at WSL (RPA[F]) fluctuates and remains below thebenchmark over time. Readership is not included in the trend analyses due tomissing proper publication year information in DOIs.

2011-20142012-20152013-20162014-20172015-20182016-20192017-2020

P[full]P[fract]
Int Cov
InterDiscP [OA]PP [OA]
PP[collab]
PP[industry]
PP[int collab]
TCS
MCS
P[top10%]
PP[top10%]
MNCS
MNJS
PP[self cits]
PP[uncited]
RPA[F]

2010-2013

Indicator

2009-2012

Table 2: Trends of WSL’s bibliometric performance

1,137
501
0.73
0.39
521
49%
83%
4%
61%
12,179
10.71
202
14%
1.37
1.19
25%
12%
0.95

1,242
509
0.76
0.39
606
51%
86%
4%
65%
15,954
12.85
238
15%
1.42
1.26
26%
7%
0.95

1,310
491
0.77
0.39
650
52%
90%
4%
69%
16,769
12.80
265
15%
1.40
1.31
27%
7%
0.93

1,457
516
0.78
0.40
762
54%
91%
4%
71%
19,852
13.63
312
16%
1.39
1.28
28%
6%
0.90

1,603
531
0.78
0.41
890
57%
92%
5%
75%
22,779
14.21
355
16%
1.41
1.30
29%
5%
0.88

1,743
550
0.78
0.41
1,011
60%
93%
5%
77%
22,761
13.06
380
17%
1.40
1.29
30%
5%
0.86

1,907
589
0.78
0.42
1,201
64%
93%
5%
78%
27,710
14.53
410
17%
1.42
1.34
31%
4%
0.87

2,062
609
0.79
0.43
1,352
67%
94%
5%
80%
28,635
13.89
440
17%
1.45
1.38
31%
4%
0.90

2,196
639
0.79
0.43
1,516
70%
94%
4%
81%
26,392
12.02
463
16%
1.47
1.38
31%
7%
0.92

In terms of field-normalised impact (i.e., PP[top10%] and MNCS; see Figures 6 and7) there is a quite stable pattern of very high field-normalised citation impact of
WSL in its journal papers during most of the period, with slight increases bothin the values of PP[top10%] and MNCS. The overall high impact of WSL journal
papers during the entire period, with MNCS values always higher than 1.30 (andoccasionally with values higher than 1.40), and PP[top10%] values increasing from
15% in the period 2009-2012 to 16% in the period of 2017-2020, can be remarked.
www.cwtsbv.nl

25

Results

In the case of proceeding papers (see Figure 7), there is an overall increase in
the field-normalised impact of WSL, particularly from the period 2009-2012 till
the period 2017-2020, in which the MNCS and PP[top10%] values have nearly
doubled. Some fluctuations are observed in the intermediate periods (e.g. between
2010-2013 and 2016-2019), although these can be attributed to the the very low
numbers of proceeding papers published by WSL (see Figure 5).

2,
200
2,
000
1,
800
1,
600

P[
f
ul
l
]

1,
400
1,
200
1,
000
800
600
400

20172020

20162019

20152018

20142017

20132016

20122015

20112014

20092012

020102013

200

Sour
c
et
ype
J
our
nalPaperFigure 4: WSL’s journal output trend (P[full]) by overlapping 4-years’ period

65
60
55
50
45
P[
f
ul
l
]

40
35
30
25
20
15
10

20172020

20162019

20152018

20142017

20132016

20122015

20112014

20102013

20092012

5

Sour
c
et
ype
Pr
oc
eedi
ngsPaperFigure 5: WSL’s proceedings output trend (P[full]) by overlapping 4-years’ period26

www.cwtsbv.nl

Results

1.
50
16%

1.
40
1.
30

14%

1.
20
1.
10

MNCS

0.
90

10%

0.
80
8%

0.
70
0.
60

PP[
t
op10%]

12%

1.
00

6%

0.
50
0.
40

4%

0.
30
0.
20

2%
20172020

20162019

20152018

20142017

20132016

20122015

20112014

20102013

20092012

0.
10

0%

I
ndi
c
at
or
s
MNCS
PP[
t
op10%]

Figure 6: WSL’s journal impact trend (MNCS and PP[top10%]) by overlapping 4years’ period

2.
80
2.
60

30%
28%

2.
40

26%
24%

2.
20

22%

2.
00

MNCS

18%

1.
60

16%

1.
40

14%

1.
20

12%

1.
00

PP[
t
op10%]

20%

1.
80

10%

20172020

20162019

20152018

20142017

2%
20132016

4%0.
2020122015

6%

0.
40

20112014

0.
6020102013

8%

20092012

0.
80

I
ndi
c
at
or
s
MNCS
PP[
t
op10%]

Figure 7: WSL’s proceedings impact trend (MNCS and PP[top10%]) by overlapping
4-years’ periodwww.cwtsbv.nl

27

Results

3.2 Research focus in context
Main findings
The most important categories for WSL in terms of the output are
Ecology; Forestry; Geosciences, Multidisciplinary; Environmental Sciences; Plant Sciences; Multidisciplinary Sciences; Geography, Physical and Meteorology & Atmospheric Sciences. The impact of WSL’s
publications in these categories is high. These categories show alsoworldwide growth during the last two years. Furthermore, in thesemain subject categories the share of female authors lower than thebenchmark. Finally, they show equal or less interdisciplinarity values compared to the benchmark, except for Plant Sciences; Multidisciplinary Sciences and Meteorology & Atmospheric Sciences, whereinterdisciplinarity is above the benchmark.
3.2.1

Research profile

In this section we break down the output of WSL into Journal Subject Categories(JSC) to add context to the general statistics. We call this a research profile. It gives
a general impression of broad subjects in which WSL’s researchers are involved.
The list of categories in the profile is limited to those that represent at least 1% of
WSL’s total output.
In each profile we include both P[full] and P[fract], i.e. the number of publications
in which WSL was involved (P[full]) and the number of publications normalised by
the number of organisations involved. Note that in such profiles, if a publication isin a journal that belongs to two subject categories, it is assigned half (0.5) to eachcategory. The profile (Figure 8) also shows MNCS, MNJS (second column) andPP[top10%] (third column) per category, to measure impact.
It is important to keep in mind that the indicators displayed in the research profiles
are distributed into journal subject categories (since these are well know and recognized discipline categories), while their normalisation has been performed basedon the CWTS field categorisation (as these are more fine-tuned, see Annex B).
In addition, we include a growth indicator in Figure 8 for each category: [Field
growth] (second column). This value indicates the estimated growth worldwide ofa subject category. A [Field growth] above 1 means a positive growth of output
worldwide in the most recent two years.
Figure 8 shows that the main Subject Categories in terms of share of the total output
are: Ecology; Forestry; Geosciences, Multidisciplinary; Environmental Sciences;
Plant Sciences; Multidisciplinary Sciences; Geography, Physical and Meteorology& Atmospheric Sciences. These Subject Categories have at least 5% of WSL’s total
output. The impact of these main subjects of activity goes from slightly high, with28

www.cwtsbv.nl

Results
PP[top10%] values between 13% and 14% for Forestry; Environmental Sciences andMeteorology & Atmospheric Sciences, to very high, with PP[top10%] values above
20% for Multidisciplinary Sciences and Ecology.
At the lower end of the profile we discern a couple of categories with very highimpact values. It should be noted, however, that in these categories the number ofpublications is very low.
Finally, the [Field growth] indicator shows that each subject present in Figure 8
grow during the last two years, especially Environmental Studies.
P[
f
r
ac
t
]

MNCS

Subj
ec
tCat
egor
y0 200 400 600 800
Ec
ol
ogy

0.
00 1.
00

2.
00

13%

20%

12%

For
est
r
y
Geosc
i
enc
es,Mul
t
i
di
sc
i
pl
i
nar
y

13%

11%

15%

9%

Envi
r
onment
alSc
i
enc
es
Pl
antSc
i
enc
es

14%

7%

Mul
t
i
di
sc
i
pl
i
nar
ySc
i
enc
es

6%

Geogr
aphy,Physi
c
al

5%

19%
26%
17%

5%

Met
eor
ol
ogy&Atmospher
i
cSc
i
enc
es

14%

Wat
erResour
c
es

3%

19%

Bi
odi
ver
si
t
yConser
vat
i
on

3%

19%

Soi
lSc
i
enc
e

2%

Envi
r
onment
alSt
udi
es

2%

Evol
ut
i
onar
yBi
ol
ogy

2%

Remot
eSensi
ng

2%

11%
16%
26%
14%

Myc
ol
ogy 1%

0 200 400 600 8000.
00 0.
50
P[
f
ul
l
]

1.
00 1.
500.
00 1.
00

Fi
el
dgr
owt
h

32%

10%

1.
00

1.
00

9%

Mi
c
r
obi
ol
ogy 1%

2.
00

MNJ
S

0% 10% 20% 30%
PP[t
op10%]

P[
f
ul
l
]
P[
f
r
ac
t
]
Fi
el
dgr
owt
h
MNCS
MNJ
S
PP[
t
op10%]

Figure 8: WSL’s research profile (output, impact across subject categories)

3.2.2

Female author contribution across subjects

In Figure 9, we present the same Journal Subject Categories as in Figure 8 andadded information related to author gender diversity (RPA[F], third column). WSL’s
authors are tagged as male or female using the first name or nickname as it appearson the publication. PA[F inst] indicates WSL’s share of female authors identified for
publications (second column). Subsequently, this share is compared with the shareof female authors in the publication at large (including all co-authors, PA[F pubs]).
The ratio of female authors within WSL and the share within the publication atlarge is RPA[F] and visualised in the third column with 1 as a point of reference. A
value above 1 means a higher share of WSL female authors than for all institutionsin the same set of publications. For instance, if a publication has 10 authors, of
www.cwtsbv.nl

29

Results

which 3 are female, the PA[F pubs] (reference value) is 0.33. If WSL is represented
by 4 authors, 2 of which are female, the PA[F inst] is 0.5. The RPA[F] would thenbe 0.5/0.33: 1.52.
A more detailed description of the approach is in Section 2.2. Underlying statistics
for WSL as large can be found in Annex A.
Focusing on the indicator RPA[F], Figure 9 shows that for most subjects the share of
WSL’s female authors is below the benchmark. Among these we find Geosciences,
Multidisciplinary; Geography, Physical and Remote Sensing.
PA[
Fi
nst
]
0.
10 0.
20 0.
30 0.
40 0.
50

Subj
ec
tCat
egor
y
Ec
ol
ogy

0.
27

Geosc
i
enc
es,Mul
t
i
di
sc
i
pl
i
nar
y

0.
87

0.
16

0.
76
0.
24

Envi
r
onment
alSc
i
enc
es
Pl
antSc
i
enc
es

0.
91

0.
29

0.
94

0.
23

Mul
t
i
di
sc
i
pl
i
nar
ySc
i
enc
es
Geogr
aphy,Physi
c
al

0.
87

0.
19

0.
73

0.
22

Met
eor
ol
ogy&Atmospher
i
cSc
i
enc
es
Wat
erResour
c
es

1.
01

0.
17

0.
85
0.
26

Bi
odi
ver
si
t
yConser
vat
i
on
Soi
lSc
i
enc
e

0.
92

0.
21

0.
80

Envi
r
onment
alSt
udi
es

0.
31

Evol
ut
i
onar
yBi
ol
ogy

0.
31

0.
96
1.
00

0.
12

Remot
eSensi
ng

0.
78

Myc
ol
ogy

1.
36

1.
00

0.
52
0.
28

Mi
c
r
obi
ol
ogy
0

0.
90

0.
22

For
est
r
y

200

400

600

0.
10 0.
20 0.
30 0.
40 0.
50

P[
f
ul
l
]

PA[
Fpubs]

0.
90

0.
00

0.
50

1.
00

1.
50

RPA[
F]

PA[
Fi
nst
]
PA[
Fpubs]
P[
f
ul
l
]
RPA[
F]

Figure 9: WSL’s share of female authors across subject categories

3.2.3

Interdisciplinary research across subjects

Figure 10 represents interdisciplinarity of WSL’s research output. It uses the same
subject categories as in Figure 8 and relies on the publications’ references (i.e.
other publications cited by the publication of interest). For a more detailed explanation of our definition of interdisciplinary research, see Section 2.2 and Annex
D. If a publication cites publications from different subject categories, it is moreinterdisciplinary than if it cites publications from the same category. In addition,
we use a cognitive distance measure to value the diversity of fields being cited.
If a paper cites publications from fields that are not closely related (e.g., medical
sciences and mathematics) it is more interdisciplinary than if it cites publications
from different medical fields. The benchmark we introduce for this indicator is the30

www.cwtsbv.nl

Results

IntDisc for a subject category at large in 2020.
As Table 1 showed in Section 3.1 the overall value of IntDisc=0.41 for WSL indicates a relatively moderate degree of interdisciplinarity, since WSL research tends
to rely on a relatively diverse set of cognitively distant disciplines.However, from acomparative perspective, the degree of interdisciplinarity of WSL is higher than the
average value of ETH Domain (IntDisc=0.35), therefore WSL exhibits a more interdisciplinary research profile in the context of ETH Domain. At the level of subject
categories, Environmental Sciences; Remote Sensing; Multidisciplinary Sciences;Environmental Sciences and Meteorology & Atmospheric Sciences are the ones
with the highest interdisciplinarity values for WSL.
Figure 10 also shows the overall value of IntDisc per subject categories (greycolor). This value is used as the benchmark for the interdisciplinarity values for
WSL (green color). In general, most of the categories with a significant output, show
equal or lower IntDisc values compared to the benchmark. Only Plant Sciences
and Multidisciplinary Sciences are the ones with higher interdisciplinarity value
compared to the benchmark.
I
nt
er
Di
sc
0.
00 0.
10

Subj
ec
tCat
egor
y

For
est
r
y

0.
40

578

Envi
r
onment
alSc
i
enc
es

0.
39
0.
42

421

0.
44

347

Pl
antSc
i
enc
es
Mul
t
i
di
sc
i
pl
i
nar
ySc
i
enc
es

0.
38

299

0.
45

266

Geogr
aphy,Physi
c
al
Met
eor
ol
ogy&Atmospher
i
cSc
i
enc
es

0.
41

228

Wat
erResour
c
es
Bi
odi
ver
si
t
yConser
vat
i
on

0.
44

166

0.
40

157

0.
40

Soi
lSc
i
enc
e

115

Envi
r
onment
alSt
udi
es

110

0.
43
0.
48

79

0.
38

75

Myc
ol
ogy

52

Mi
c
r
obi
ol
ogy

50

0

0.
50

0.
40

525

Geosc
i
enc
es,Mul
t
i
di
sc
i
pl
i
nar
y

Remot
eSensi
ng

0.
30

664

Ec
ol
ogy

Evol
ut
i
onar
yBi
ol
ogy

0.
20

100

0.
46
0.
35
0.
42

200

300

400

P[
f
ul
l
]

500

600

700 0.
00 0.
10

0.
20

0.
30

0.
40

0.
50

Catavg

P[
f
ul
l
]
I
nt
er
Di
sc
CatavgFigure 10: WSL’s interdisciplinarity across subject categories

www.cwtsbv.nl

31

Results

3.3 Collaboration and partners
Main findings

For WSL, both the proportion of publications done in collaboration andinternational collaboration have gone up over the analyzed time period,
while industry collaboration remained relatively stable. International
collaboration takes up the largest part of the output, though it is somewhat less dominant looking at P[fract]. International collaboration also
consistently has the highest impact over all three impact indicators.
Regarding collaboration with other ETH institutions, WSL has boththe highest output and impact together with ETH Zurich. On a country
level, WSL collaborates most with German affiliations, though this isalmost tied with Switzerland.

3.3.1

Collaboration profile

This section includes a trend analysis for the collaboration indicators as well as acollaboration profile.
The trend analysis in Table 3 breaks WSL’s output and collaboration indicators
down over time, using overlapping four-year publication windows.
In the collaboration profile in Figure 11, we break down WSL’s output and impact
by collaboration type, distinguishing between ’no collaboration’ (single author or
all authors affiliated with WSL), national collaboration (all authors having a Swiss
affiliation from different institutions) and international collaboration.

2011-20142012-20152013-20162014-20172015-20182016-20192017-2020

P[full]PP[collab]
PP[int collab]
PP[industry]

2010-2013

Indicator

2009-2012

Table 3: WSL’s trend collaboration statistics

1,137
83%
61%
4%

1,242
86%
65%
4%

1,310
90%
69%
4%

1,457
91%
71%
4%

1,603
92%
75%
5%

1,743
93%
77%
5%

1,907
93%
78%
5%

2,062
94%
80%
5%

2,196
94%
81%
4%

In Table 3, we see a clear upward trend for both PP[collab] and PP[int collab].
Particularly in the first two four-year windows of the trend table, the proportion ofcollaborations went up quickly. For PP[industry] we see more stability, with thenumber fluctuating only between 4 and 5%.
32

www.cwtsbv.nl

Results
P[
f
r
ac
t
]
0

Si
ngl
ei
nst
i
t
ut
i
on

Nat
i
onal

2,
000

4,
000

I
nt
er
Di
sc

MNCS

0.
100.
200.
300.
40

0.
50 1.
00 1.
50

451
9%

815
17%

3,
670
74%

1.
28

0.
41

1.
34

0.
42

12%

16%

1.
53

17%

0

2,
000
P[
f
ul
l
]

4,
000

0.
100.
200.
300.
40

0.
50 1.
00 1.
50

Catavg

MNJ
S

10%

1.
00

I
nt
er
nat
i
onal

0.
40

0% 5%

10% 15% 20%

PP[t
op10%]

P[
f
ul
l
]
P[
f
r
ac
t
]
I
nt
er
Di
sc
Catavg
MNCS
MNJ
S
PP[
t
op10%]

Figure 11: Collaboration profile (output, impact) of WSL

Immediately notable in Figure 11 is the dominance of international collaboration
when it comes to full-counting output, taking up roughly 74% of WSL’s total. For
full-counting, national has the second-highest output, but this perspective changes
for fractional-counting, where single institution overtakes the second place. In thisfractional-counting perspective, international collaboration only takes up roughly
52% of total WSL’s output.
The green bars indicate the interdisciplinarity (IntDisc) measures for the different
collaboration types (for more information on how this is calculated, please refer toAnnex D). The difference for IntDisc between the categories is small, with international leading national by one decimal point (0.42 vs. 0.41), which in turn leads
single institution by one decimal point (0.40).
For all three impact indicators in the last two columns, international collaboration
performs best, though the difference is much smaller for PP[top10%] than for both
MNCS and MNJS. This could suggest some influence of outliers on the MNCS,
since the PP[top10%] indicators is less susceptible to that.
Another observation is that MNCS values are higher than MNJS values for all
three categories, implying that publications with WSL’s involvement outperform theaverage of the journals in which they were published.
www.cwtsbv.nl

33

Results

PP[
i
ntc
ol
l
ab]
0% 20% 40% 60% 80% 100%

Subj
ec
tCat
egor
y
Ec
ol
ogy

81%

3%

For
est
r
y

78%

3%

Geosc
i
enc
es,Mul
t
i
di
sc
i
pl
i
nar
y

66%

5%

Envi
r
onment
alSc
i
enc
es

78%

Pl
antSc
i
enc
es

76%

Mul
t
i
di
sc
i
pl
i
nar
ySc
i
enc
es

6%

90%

Geogr
aphy,Physi
c
al

69%

Met
eor
ol
ogy&Atmospher
i
cSc
i
enc
es

5%
7%

72%

Wat
erResour
c
es

62%

6%

Bi
odi
ver
si
t
yConser
vat
i
on

5%

80%

Soi
lSc
i
enc
e

75%

Envi
r
onment
alSt
udi
es

1%
3%

70%

Evol
ut
i
onar
yBi
ol
ogy

82%

Remot
eSensi
ng

4%
24%

64%

Myc
ol
ogy

79%

Mi
c
r
obi
ol
ogy
0

6%
2%

0%
4%

79%

200

400

600

0% 20% 40% 60% 80% 100%

P[
f
ul
l
]

PP[
c
ol
l
ab]

0% 5%

10% 15% 20% 25%
PP[
i
ndust
r
y]

P[
f
ul
l
]
PP[
c
ol
l
ab]
PP[
i
ntc
ol
l
ab]
PP[
i
ndust
r
y]

Figure 12: WSL’s output and collaboration types across subject categories
In Figure 12, the collaboration indicators PP[collab], PP[int collab] and PP[industry]
are calculated by Web of Science subject category for WSL publications.
We find the highest output in the categories Ecology, Forestry and Geosciences,
Multidisciplinary, with 664, 578 and 525 publications respectively.
For PP[collab], we can observe that all categories are very collaborative, withthe lowest (Environmental Studies) still on 84% and the highest (Multidisciplinary
Sciences) as high as 97%. Values for PP[int collab] are slightly more dispersed,
ranging from 62% for Water Resources to 90% for Multidisciplinary Sciences.
Finally, values for PP[industry] are relatively low if we compare to values for other
ETH Domain institutions. One category clearly stands out: Remote Sensing (24%).
Below that, Mycology actually sports no industry collaborations at all.
3.3.2

Collaboration within the ETH Domain

Table 4: Co-authorship and impact within the ETH Domain

34

Indicator

ETH Zurich

EPFL

PSI

WSL

Empa

Eawag

P[full]
MNCS

1,107
1.76

390
1.46

125
1.47

4,936
1.42

20
1.06

65
1.60

www.cwtsbv.nl

Results

Table 4 shows WSL’s output and impact (highlighted column), as well as the numberof co-publications and impact of WSL with other ETH institutions.
We can see that ETH Zurich is the most frequent collaborator with over 1,000
publications co-authored. The number of collaborations with Empa and Eawag,
meanwhile, is almost negligible. In terms of impact, ETH Zurich also performs best,with an MNCS of 1.76.

www.cwtsbv.nl

35

Results
3.3.3

Collaboration outside the ETH Domain

This section seeks to delve deeper into WSL’s collaboration partners outside of theETH Domain, categorising them first by country and then by institution. Tables
5 and 6 highlight the top collaborators in terms of output. For the results atcountry level, we used full counting. The output numbers reflect the number andshare of output in which countries were involved. For the analysis of co-authoring
institutions (Table 6), we used fractional counting. The output numbers indicate thecontribution of partnership compared to the total.
The map in Figure 13 highlights countries with more intensive collaboration, withthe darkness or intensity of the red indicating the relative level of co-authorship.
In this section we exclude collaborations within the ETH Domain. However, if apublication involves a ETH Domain member and also an external member, it isincluded.

Country-level

Table 5: Top 12 countries co-authoring with WSL researchers, excluding ETH
Domain internal co-authorship. P[full] and % to WSL’s total
Country
Switzerland
Germany
United States
FranceUnited Kingdom
Italy
Spain
Netherlands
Czech Republic
Austria
SwedenCanadaCo-pubs% to total

1,270
1,228
1,019
777
712
514
445
408
386
379
366
355

26%
25%
21%
16%
14%
10%
9%
8%
8%
8%
7%
7%

In Table 5, Switzerland (here represented by non-ETH Domain collaborations)comes out on top, followed closely by Germany and the United States. This top
three is common among institutions.
36

www.cwtsbv.nl

www.cwtsbv.nl

0%

Per
ct
oTot
al

15%

©2022Mapbox©OpenSt
r
eet
Map

Coaut
hor
i
ngc
ount
r
i
es(
c
ol
ori
ndi
c
at
esshar
et
oWSL'
st
ot
al
:4,
936)

Results

Figure 13: Map of countries co-authoring with WSL

37

Results
Institutions

Table 6: Top 20 collaborating institutions of WSL, excluding ETH Domain internalco-authorship (fractional output and impact)
Inst
University of Zurich
University of Bern
Chinese Academy of Sciences
University of Lausanne
University of Freiburg
Vrije Universiteit Amsterdam
University of NeuchâtelTechnical University of Munich
University of BaselCentre National de la Recherche Scientifique
Wageningen University & Research
University of Natural Resources and Life Sciences, Vienna
Agroscope
Georg-August-Universität Göttingen
Czech Academy of Sciences
Johannes Gutenberg University Mainz
University of Arizona
University of Colorado, Boulder
Czech University of Life Sciences Prague
Université Grenoble Alpes - EPE

CountryCo-pubs

MNCS

CH
CH
CN
CH
DE
NL
CH
DE
CH
FR
NL
AT
CH
DE
CZ
DE
US
US
CZ
FR

93
76
56
45
40
39
31
28
25
25
25
24
23
22
22
19
17
16
16
16

1.70
1.91
1.06
1.48
2.24
1.87
1.91
2.01
1.82
2.16
2.12
1.83
1.73
2.55
1.76
1.81
4.17
1.69
1.62
1.96

In Table 6, we can observe that three out of the top five collaborating institutions
are Swiss. Regarding impact, institutions that do well in collaboration with WSL
include the University of Freiburg (MNCS of 2.24), the Georg-August-Universität
Göttingen (MNCS of 2.55), and the University of Arizona (4.17). It should be noted,
though, that with these low publication counts, the MNCS indicator becomes more
susceptible to outliers, so these indicators should be considered carefully.

38

www.cwtsbv.nl

Results

3.4 Research accessibility
Main findings
WSL’s research is published increasingly in Open Access. The number(and share) of all three types of OA publications grows steadily during
the period 2009 up to 2020, especially Gold OA. Also the number of
top 10% most cited publications of all three type grows steadily, while
the number of Closed Access top 10% publications drops in the mostrecent years. The impact of OA publications is similar to the impactof Closed Access publications. In the more recent years, the impact of
OA publications is somewhat higher.
3.4.1

OA publishing and impact

In this section we discuss the accessibility of WSL’s research output. For publications with a DOI we could define whether it was published Open Access (OA) or
not based on Unpaywall data (version July 2021). Therefore, the below statistics
only include publications for which we could define OA or not. In addition, we
could also determine the type of OA (Gold, Hybrid or Green). The trend analyses
allow us to monitor the evolution of WSL regarding OA publishing.
Using OA information we assess the overall accessibility of WSL’s OA output aswell as its citation-based impact, by benchmarking it to non-OA output.
Table 7: WSL’s Open Access (OA) performance statistics by type, excluding publications for which no OA info available
Indicator
P[full]P[top10%]
PP[top10%]
PP[int collab]

OA Gold

OA Hybrid

OA Green

Closed Access

Total

1,099
187
14%
74%

424
129
19%
84%

1,404
354
18%
77%

1,843
338
16%
72%

4,770
1,009
16%
75%

In Table 7, we provide an overview of main performance statistics for three types
of OA (Gold, Hybrid and Green) together with their overall performance. P[full]reflects the total number of publications, P[top10%] the number belonging to the
top 10% most cited (within its year and field). PP[top10%] assesses the impact ofeach type, while PP[int collab] reflects the share of output involving international
collaboration.
Looking at the entire period (2009-2020), we see a preference for Green OA publications (P[full]). The impact is particularly high for Hybrid and Green OA publications
www.cwtsbv.nl

39

Results(PP[top10%]). The share of output involving international collaboration is the highest for Hybrid OA output (PP[int collab]: 84%). Both PP[top10%] and PP[int collab]
are higher for all types of OA publications, compared to Closed Access publications.

2009-20122010-20132011-20142012-20152013-20162014-20172015-20182016-20192017-2020

Table 8: WSL’s performance statistics trend, Closed vs. Open Access publications

ClosedP[full]P[top10%]
PP[top10%]
PP[int collab]

532
104
16%
63%

588
115
16%
65%

609
125
16%
67%

645
134
17%
69%

663
133
17%
72%

682
138
18%
76%

663
119
17%
76%

668
117
17%
78%

648
102
15%
79%

Open

P[full]P[top10%]
PP[top10%]
PP[int collab]

521
95
15%
63%

606
121
16%
67%

650
136
15%
71%

762
173
15%
74%

890
217
16%
77%

1,011
236
16%
78%

1,201
288
17%
79%

1,352
319
17%
81%

1,516
359
17%
82%

Indicator

In Table 8, we provide trend results for the same indicators as in Table 7, comparing
OA publications with non-OA (Closed Access) publications. These results only
include publications for which OA information was available (included in Unpaywall,
have a DOI). In Figures 14 and 15, P[full] and P[top10%] are depicted by OA type.
The results in Table 8, show a tripling of the volume of OA publications (from 521
up to 1516). For P[top10%] we see an even stronger increase. Normalised by
the total number of output per year (PP[top10%]) the impact increases from 15%
up to 17%. The impact of Closed Access publications remains at the same level
throughout (around 16%).
From the collaboration perspective, we see that the proportion of output involving international collaboration increased for both types of output, albeit that it issomewhat higher for OA output (PP[int collab]).
40

www.cwtsbv.nl

Results
Per
i
od
700
650
600
550
500

P[
f
ul
l
]

450
400
350
300
250
200
150
100
50
0
20092012

20102013

20112014

20122015

20132016

20142017

20152018

20162019

20172020

OAt
ype
Cl
osed
OAGol
d
OAGr
een
OAHybrid
OAunknown

Figure 14: WSL’s output trend by Open Access (OA) typeThe steep growth of Gold OA publications (Figure 14) shows the increased interestof WSL researchers to publish OA. Together with Hybrid these are the two rapidly
growing types. The Green OA type was already there but has been surpassed byGold OA in the most recent years. The number of Closed Access publications isslowly decreasing.
Per
i
od
160
150
140
130
120

P[
t
op10%]

110
100
90
80
70
60
50
40
30
20
10
0
20092012

20102013

20112014

20122015

20132016

20142017

20152018

20162019

20172020

OAt
ype
Cl
osed
OAGol
d
OAGr
een
OAHybrid
OAunknown

Figure 15: WSL’s trend of top 10% publications by Open Access (OA) type

Looking at the P[top10%] of OA publications (Figure 15), we see a strong sustained
growth of all OA types and a drop in the Closed Access publications in the mostrecent couple of years. This points to the importance of OA publications in the
profile of WSL.

www.cwtsbv.nl

41

Results
3.4.2

OA publishing and impact by subject

In this section we present WSL’s performance statistics by journal subject category.
In Figure 16, we visualise the share of OA publications, related to the overall
output (for which access information was available). The bars in the second columnof the diagram represent the ratio of the sum of OA publications to the sum of all
publications. The light blue bar in the profile in the first column represents thetotal number of publications. The list of subject categories is limited to those thatcover at least 1% of the total output of WSL.
In Figure 17, the second column visualises the impact of both Closed and Open
access publications by PP[top10%] by subject.

POA[
Gol
d,Hybr
i
d,Gr
een]
Subj
ec
tCat
egor
y0

100

200

300

Ec
ol
ogy

500

600

700

394

60%

274

For
est
r
y

48%

Geosc
i
enc
es,Mul
t
i
di
sc
i
pl
i
nar
y

409

82%

222

Envi
r
onment
alSc
i
enc
es
Pl
antSc
i
enc
es

54%

189

56%
283

Mul
t
i
di
sc
i
pl
i
nar
ySc
i
enc
es
Geogr
aphy,Physi
c
al

95%

161

62%

172

Met
eor
ol
ogy&Atmospher
i
cSc
i
enc
es
Wat
erResour
c
es

81%

111

69%

81

Bi
odi
ver
si
t
yConser
vat
i
on
Soi
lSc
i
enc
e

400

51%

48

Envi
r
onment
alSt
udi
es

60

Evol
ut
i
onar
yBi
ol
ogy

57

Remot
eSensi
ng

43

42%
55%
72%
67%

Myc
ol
ogy 24
Mi
c
r
obi
ol
ogy
0

46%

37

100

73%

200

300

400

P[
f
ul
l
]

500

600

700 0%

20%

40%

60%

80%

100%

PP[
OA]

P[
f
ul
l
]
POA[
Gol
d,Hybr
i
d,Gr
een]
PP[
OA]

Figure 16: WSL’s output and share of OA publications across subject categories

In the above profile, the share of OA publications (PP[OA]) in Multidisciplinary
Sciences journals (typically PLOS One and Scientific Reports) stands out. 95% ispublished OA. Other categories to be mentioned here are Geosciences, Multidisciplinary; Multidisciplinary Sciences and Meteorology & Atmospheric Sciences with
80% or more published OA. It is worth mentioning that all categories cover morethan 40% of OA publications and 9 out of 16 more than 60%.
42www.cwtsbv.nl

Results
POA[
Gold,Hybr
i
d,Gr
een]
100

200

300

400

500

PP[
t
op10% OA]
600

0% 10%

20%

10%

Subj
ec
tCat
egor
y0
Ec
ol
ogy
For
est
r
y

30%

50%

60%

70%

50%

60%

70%

11%

Geosc
i
enc
es,Mul
t
i
di
sc
i
pl
i
nar
y

15%

Envi
r
onment
alSc
i
enc
es

15%

Pl
antSc
i
enc
es

20%

Mul
t
i
di
sc
i
pl
i
nar
ySc
i
enc
es

25%

Geogr
aphy,Physi
c
al

15%

Met
eor
ol
ogy&Atmospher
i
cSc
i
enc
es

11%

Wat
erResour
c
es

17%

Bi
odi
ver
si
t
yConser
vat
i
on

17%

Soi
lSc
i
enc
e

8%

Envi
r
onment
alSt
udi
es

20%

Evol
ut
i
onar
yBi
ol
ogy

24%

Remot
eSensi
ng

17%

Myc
ol
ogy

8%

Mi
c
r
obi
ol
ogy
0

40%

21%

35%

100

200

300

400

P[
f
ul
l
]

500

600

0% 10%

20%

30%

40%

PP[
t
op10% Cl
osed]

POA[
Gol
d,Hybr
i
d,Gr
een]
PP[
t
op10% Cl
osed]
PP[
t
op10% OA]
P[
f
ul
l
]

Figure 17: WSL’s impact distribution (PP[top10%]) of Open and Closed output
across subject categories

For WSL, we found that OA publications in almost all categories have a high impact(PP[top10%]). Only in a few cases the impact exceeds the impact of Closed Accesspublications in the same category, though. Only in the case of Microbiology, theimpact is substantially higher for OA publications.
Overall, we see that WSL stimulates OA publishing from the strong growth of all
OA types of output. The impact of WSL’s output is high, but there is not yet sucha clear impact advantage for OA publications as often seen for other institutions.

www.cwtsbv.nl

43

Results

3.5 Impact and knowledge use
Main findings

WSL’s research is read and cited from all over the world. The citationbased impact is primarily determined by institutions located in Europe,
Asia and the United States. The readership analysis also shows significant impact of WSL’s research in countries that are not well represented in WoS as these countries (e.g., Brazil, Mexico, South Africa
and Argentina).

In this section, we discuss the actors (countries, institutions) that define the impact
and use of WSL’s research. We estimate the impact and use by analysing (1) the
publications citing WSL’s publications and (2) the country of people reading its
publications.
The analysis of publications citing WSL’s output shows the most prominent countries
and institutions. Thus we provide an overview of the geographical distribution of
WSL’s impact and more specifically the institutions that use WSL’s research.
The readers are analysed using Mendeley data, in which a ’read’ is defined by aperson (i.e., Mendeley user) saving a publication. The results should be interpretedwith that disclaimer in mind. The user information includes the country of origin(if available). In this report, we will present the countries and compare these to
the ones citing WSL’s output. Including readership in this study does not show abroader (e.g., societal) impact of WSL research but merely catches the (potential)
scientific impact beyond the WoS data.

3.5.1

Impact and knowledge use at country level

The citation-based impact is defined by publications citing WSL’s output. In these
citing publications, we use the affiliations of authors to measure their contribution to
the impact of WSL’s research. Table 9 shows the 20 most prominent countries citing
WSL’s research output. In the table we include the number of WSL publications
being cited, the number of citations they receive and the average number of citations
per publication. The top 20 is defined by the number of citations received (N cits).
This list is obviously dominated by countries with many publications in WoS, andwe cannot deny their significant role in determining the citation-based impact. By
considering the top countries and subsequently looking at the average number ofcitations given, we normalise to some extent the results.
44

www.cwtsbv.nl

Results

Table 9: WSL given citations by country (top 20 most given citations)
Country
United States
China
Germany
United Kingdom
FranceSwitzerland
CanadaSpain
Italy
Australia
SwedenNetherlands
Austria
Czech Republic
Finland
PolandBrazil
NorwayBelgiumJapan

N pubs

N citsAvg cits

3,072
2,401
2,604
2,186
2,109
2,328
1,850
1,807
1,930
1,365
1,229
1,227
1,334
1,130
1,013
980
808
918
953
875

13,247
7,823
7,039
5,640
4,575
4,248
3,938
3,734
3,454
3,108
2,133
2,011
1,709
1,558
1,528
1,443
1,430
1,352
1,341
1,306

4.31
3.26
2.70
2.58
2.17
1.82
2.13
2.07
1.79
2.28
1.74
1.64
1.28
1.38
1.51
1.47
1.77
1.47
1.41
1.49

In Table 9, we see the dominance of the United States, China and Germany defining
WSL’s impact. Not only by absolute numbers of citations but also by the averages,
these countries attribute great value to WSL’s research. US researchers cite on
average a WSL publication 4.31 times, Chinese researchers 3.26 and German researchers 2.7. Next in line are researchers and other European countries, Canada,
Australia, Brazil, and Japan with between 1.38 (Czech Republic) and 2.58 (UK)citations per publication on average.
In Table 10, we introduce a different perspective on the impact WSL’s researchhas. By looking at the number of reads by Mendeley users from different countries,
we get a better view on the geographical distribution beyond the perimeter of
the academic debate (as defined by citations). We realise that this distribution
is defined primarily by the authors citing WSL’s output but we hope to broaden
the view on the impact somewhat. The List in Table 10 shows the top 20 mostprominent countries ’reading’ WSL’s publications. The list order is defined by the
number of reads (second column: N reads). In the table the first column shows thenumber of publications being read (N pubs). The third column shows the average
number per read publication (Avg Reads). We consider the countries that end up
in the readership list (Table 10) but not in the citing countries list (Table 9) as theones showing the impact beyond the WoS.
www.cwtsbv.nl

45

Results

Table 10: WSL readership by country (top 20, by most reads)
Country
United States
Germany
Switzerland
BrazilSpain
United Kingdom
CanadaFranceItaly
MexicoPortugal
Czech Republic
Japan
Australia
South Africa
Austria
Argentina
SwedenBelgium
NorwayN pubs

N reads

Avg Reads

1,259
887
880
611
764
699
606
601
586
449
316
307
300
266
260
259
220
234
241
221

3,491
1,605
1,551
1,497
1,247
1,169
958
952
827
578
448
416
371
365
350
323
312
308
289
286

2.77
1.81
1.76
2.45
1.63
1.67
1.58
1.58
1.41
1.29
1.42
1.36
1.24
1.37
1.35
1.25
1.42
1.32
1.20
1.29

From the reader perspective we see some interesting differences, comparing themto Table 9. First of all, the absence of China which is an artefact of the data
being used. Chinese researchers academics do not use Mendeley to manage theirliterature (Fairclough and Thelwall, 2015; Zahedi and Costas, 2020). In addition,
we see a much more prominent position of Brazil in this list, in absolute numbers
but also on average. In this list of top 20 countries, Brazil is second with 2.45
reads per publication after the US (2.77). Other non-European countries includedin Table 10 and not in Table 9 are: Mexico, South Africa, and Argentina. Most ofthese countries have less visibility in WoS but show a significant interest in the
research published by WSL through readership.

3.5.2

Impact by citing institution

In Table 11, we list the top 20 most prominent citing institutions of WSL’s publications. This list provides more insight in the actual research actors being impacted
by WSL. As the list is based on the number of citations given (N citing pubs, second
column), it will be biased towards large institutions (with many publications). We
normalise these large numbers by including the number of publications being cited(N cited pubs, first column), which leads to the average in the third column (Avg
46

www.cwtsbv.nl

Resultscits).
Table 11: WSL’s top 20 most citing institutions (by number of given citations)
Institution

Country

N cited pubs

N citing pubs

Avg cits

CHINESE ACAD SCICNRSSWISS FED RES INST WSLUNIV CHINESE ACAD SCIETH ZURICH
CSIC SPAIN
INRA
UNIV HELSINKI
SWED UNIV AGR SCI UPPSALA
ACAD SCI CZECH REPUBLIC
WAGENINGEN UNIV
UNIV BERNGA UNIV GOTTINGEN
UNIV MONTPELLIER
U.S. GEOLOGICAL SURVEY
RUSSIAN ACAD SCIIRD INST RECHERCHE DEV
KOBENHAVNS UNIV
MAX PLANCK SOCIETY
UNIV GRENOBLE ALPES EPE

CN
FR
CH
CN
CH
ES
FR
FI
SE

1,516
1,362
1,373
770
1,028
818
695
574
566

3,486
2,410
1,574
1,345
1,181
1,097
926
741
722

2.30
1.77
1.15
1.75
1.15
1.34
1.33
1.29
1.28

CZ
NL
CH
DE
FR
US
RU
FR
DK
DE
FR

571
565
536
548
495
447
453
531
483
449
570

697
654
651
649
642
632
630
627
620
616
585

1.22
1.16
1.21
1.18
1.30
1.41
1.39
1.18
1.28
1.37
1.03

Table 11 too is dominated by the largest research institutions in the world withmany WoS publications and located in the countries in Table 9, the Chinese
Academy of Science and CNRS being mega-institutions. WSL is the third institution contributing to its impact. We mention the Danish University of Copenhagen
which cites WSL’s publications 1.28, while Denmark is not included in Table 9.

www.cwtsbv.nl

47

REFERENCES

References
Emile Caron and Nees Jan van Eck. Large scale author name disambiguation using
rule-based scoring and clustering. In 19th International Conference on Scienceand Technology Indicators, volume 19, pages 79–86, Leiden, September 2014.
Ruth Fairclough and Mike Thelwall. National research impact indicators from
Mendeley readers. Journal of Informetrics, 9(4):845–859, October 2015. ISSN
1751-1577. doi: 10.1016/j.joi.2015.08.003. URL https://www.sciencedirect.
com/science/article/pii/S1751157715300596.
Javier Ruiz-Castillo and Ludo Waltman. Field-normalized citation impact indicators using algorithmically constructed classification systems of science. Journalof Informetrics, 9(1):102–117, January 2015. ISSN 17511577. doi: 10.1016/
j.joi.2014.11.010. URL https://linkinghub.elsevier.com/retrieve/pii/
S1751157714001126.
Ludo Waltman and Nees Jan van Eck. A new methodology for constructing apublication-level classification system of science. Journal of the American Society for Information Science and Technology, 63(12):2378–2392, December 2012.
ISSN 15322882. doi: 10.1002/asi.22748. URL http://doi.wiley.com/10.
1002/asi.22748.
Ludo Waltman, Clara Calero-Medina, Joost Kosten, Ed C. M. Noyons, Robert J. W.
Tijssen, Nees Jan van Eck, Thed N. van Leeuwen, Anthony F. J. van Raan, Martijn S. Visser, and Paul Wouters. The Leiden ranking 2011/2012: Data collection,
indicators, and interpretation. Journal of the American Society for Information
Science and Technology, 63(12):2419–2432, December 2012. ISSN 1532-2890.
doi: 10.1002/asi.22708. URL https://onlinelibrary.wiley.com/doi/abs/
10.1002/asi.22708.
Zohreh Zahedi and Rodrigo Costas. Do Online Readerships Offer Useful Assessment Tools? Discussion Around the Practical Applications of Mendeley
Readership for Scholarly Assessment. Scholarly Assessment Reports, 2(1):14,
November 2020. ISSN 2689-5870. doi: 10.29024/sar.20. URL http://www.
scholarlyassessmentreports.org/articles/10.29024/sar.20/. Number: 1 Publisher: Levy Library Press.

48

www.cwtsbv.nl

REFERENCES

Annexeswww.cwtsbv.nl

49

WSL’s author gender statistics

A WSL’s author gender statistics
Table 12: WSL: Underlying gender diversity statistics
Indicator

Value

A[F inst]
PA[F inst]
A[FM inst]
A[F pubs]
PA[F pubs]
A[FM pubs]
RPA[F]

1,952
0.24
8,052
8,311
0.27
30,9180.90

The indicators presented in this table are described in Section 2.2, p. 17.

50

www.cwtsbv.nl

Publication level classification

B Publication level classification
The CWTS citation database is a bibliometric version of Web of Science (WoS). Oneof the special features of this database is the publication-based classification. This
classification is an alternative to the WoS journal classification, the WoS subject
categories. The reason to have this publication-based classification is the problems
we encounter using the journal classification for particular purposes. We discern
the following as the most prominent ones.

B.1 Journal scope (including multi-disciplinary journals)
A journal classification introduces sets of journals to represents a class, in thiscase a subject category. This implies that journals have a similar scope. They
do not need to be comparable with regard to volume (number of articles per year)
but they should represent a similar specialisation. This is not the case, of course.
Journals represent a very broad spectrum. There are very specialist journals (e.g.,
Scientometrics) and very general ones (e.g., Nature or Science but also British
Medical Journal). The classification scheme can therefore not be very specialised.
In WoS, a subject category Multi-disciplinary hosts the very general ones so that abibliometric analysis of, for instance, the Social Sciences or Nanotechnology, using
this classification, will not take papers in Nature into consideration.

B.2 Granularity of the WoS subject categories
The WoS journal classification scheme contains 255 elements. As such it is a stablesystem. In many cases however, it appears that these 255 subject categories are
insufficient to be used for proper field analyses. The problem is that the granularityof the system looks somewhat arbitrary. ‘Biochemistry & Molecular Biology’ on the
one hand and ‘Ornithology’ on the other, for instance, represent rather different
aggregates of research. This is illustrated by the number of journals in each ofthem. Where the ‘Biochemistry & Molecular Biology’ category contains almost
500 journals, ‘Ornithology’ has only 27. We acknowledge that there is no perfectgranularity, but we argue that in the WoS subject categories the differences arereally too big. A classification based on more objective grounds does not solve thisproblem but is at least transparent.

B.3 Multiple assignment of journals to categories
In journal classifications from multi-disciplinary databases, journals are assigned
to more than one category. Journals often have broader scopes than the categories
allow. Also here there are large differences between categories. In the example weused before, ‘Biochemistry & Molecular Biology,’ journals are on average assigned
to almost 2 categories. This means that (on average) each journal in this category is
also assigned to one other category. For the more specialist category of ‘Ornitholwww.cwtsbv.nl

51

Publication level classificationogy’, the average is 1. This means that in this category all journals are assigned
to this category only. If publications in journals with a multiple assignment would
always cover the categories at stake, this should not necessarily be a problem.
However, it mostly means that such journals structurally contain publications fromthe different categories. Therefore, publications may be assigned to two categories
although they belong to just one of them.

B.4 The CWTS publication-based classification scheme
CWTS has developed an advanced alternative for the Web of Science journal classification. It counters three major issues:
1. Journal scope (including multi-disciplinary journals)
2. Granularity of the WoS subject categories
3. Multiple assignment of journals to categories
The CWTS publication-based classification is developed as described in Waltman
and van Eck (2012). Since the first version there have been yearly updates of thesystem. The main characteristics of the classification are as follows.
Publication to publication citation clustering
Clusters of publications are created on the basis of citations from one publication toanother. Tens of millions of publications have been processed. The clusters containpublications from multiple years (2000–2020). Each publication is assigned to one
cluster only at each level. A cluster is considered, and in many cases validated as,
representative for disciplines, research areas, fields or sub-fields. For each cluster,
we can calculate growth indices pointing at changing research focus over time.
Multi-level clustering
The classification scheme has at present three different levels. The clusters arehierarchically organised. Currently we discern the following levels.
1. A top level of 25 clusters (fields)
2. A second level of around 800 clusters (sub-fields)
3. A third level of more than 4,000 clusters (research areas or micro-fields)
A common way of visualising the landscape of science by the publication clusters is
a 2-dimensional map. In such a landscape (see Figure 18), we position publication
clusters in relation to each other on the basis of citation traffic. The denser the traffic
between two clusters, the closer they are. The two dimensions do not represent
anything. The only thing that matters is the distance. Furthermore, the size of a52

www.cwtsbv.nl

Publication level classification

cluster represents the relative volume (number of publications included), while thecolor coding adds a main clustering labeled by main disciplines.

Mai
ndi
sc
i
pl
i
ne
Soc
i
alSc
i&HumanBiomed&Heal
t
hSc
i
Physi
c
alSc
i&Engi
n
Li
f
e&Ear
t
hSc
i
Mat
hs&ComputSciFigure 18: Landscape of all science (around 30 million WoS publications). Circles
represent (over 4,000) publication clusters. Position is defined by citation traffic
between clusters. Size indicates relative volume. Color reflects 5 main disciplines

www.cwtsbv.nl

53

Citation data and analysis

C Citation data and analysis
In this annex we provide more detail about the methodology developed at CWTS
and applied in this study.

C.1 Database coverage
In a bibliometric study, we base the analyses on publication data. To relate counting
and measuring to standards, we depend on international bibliographic databases,
such as Web of Science, Scopus, Dimensions. We realise that by using suchdatabases, we may be missing relevant scientific outputs and achievements. In order
to assess how much the database does cover we calculate the Internal Coverage(IntCov) indicator. This indicator is the ratio of cited references covered by the
database, to the total number of cited references. If a publications contains 10
references, five of which are also in the database, the IntCov of this publication
is 0.5. For a set of publications the IntCov is defined by the average IntCov per
publication. If the IntCov of an institution’s output in WoS is 0.8, we estimate thecoverage of WoS for this institution at 0.8 (80%).

C.2 Database Structure
At CWTS, we calculate bibliometric indicators based on an in-house version of the
Web of Science (WoS) online database, which will be referred to as the CI-system.
The WoS is a bibliographic database that covers publications of about 12,000
journals and each of these journals is assigned to one or more Journal Subject
Categories (JSC). Each publication in the CI-system has a document type. The mostfrequently occurring document types are ‘articles’, ‘reviews’, ‘proceeding papers’,‘corrections’, ‘editorial material’, ‘letters’, ‘meeting abstracts’ and ‘news items’. In
this report, we only consider document types ‘articles’, ‘reviews’ and ’proceedings
papers’. In limiting the analysis to these three types of publications, we consider
that these documents reflect most of the original scientific output in a field.
The CI-system is an improved and enhanced version of the WoS database versionsof the Science Citation Index (SCI), Social Science Citation Index (SSCI), and Arts
& Humanities Citation Index (A&HCI). The CI-system implements a publicationbased field classification which clusters publications into research areas based
solely on citation relations (Waltman and van Eck, 2012) (more detail in Annex
B). One important advantage of this publication-level classification system is thatit allows for a taxonomy of science that is more detailed and better matches thecurrent structure of scientific research. This not only reduces classification bias
but is also essential for calculating field-normalised indicators (Ruiz-Castillo andWaltman, 2015).
Moreover, in this study we include citation data up to 2021. Please note thatpublications require at least one full year to receive citations in order to make54

www.cwtsbv.nl

Citation data and analysis

robust calculations of citation impact indicators. For this reason, we will workwith publications up to and including 2020, counting citations up to and including
2021. For each publication (and its benchmark publications), we consider 4 yearsof citations since the year of publication. For a publication from 2010, we countcitations in the years 2010-2014.

C.3 Citation Window, Counting Method and Field Normalisation
Citation window
Several indicators are available for measuring the average scientific impact of the
publications of a research unit, e,g. and institution. These indicators are all basedon the idea of counting the number of times the publications of a unit have been
cited. Citations can be counted using either a fixed-length citation window ora variable-length citation window. In the case of a fixed-length citation window,
only citations received within a fixed time period (e.g. four years fixed window)
are counted. The main advantage of a fixed-length citation window is that it ispossible to meaningfully analyse the trend patterns of the non-normalised impact
indicators, setting the same criteria for all publications included. A variable-length
window, on the other hand, uses all the citations that are available in the database
until a fixed point in time, which not only yields higher citation counts (dependingon the window length), but also more robust impact measurements. When using
a variable-length citation window, impact indicators such as the average impact(MCS) and the total impact score (TCS) may systematically present a decreasing
pattern.
In this study, we use a fixed-length window of 4 year (if available) for the overall
period of the analysis (2009-2020). The most recent year for receiving citations is
2021.
Self-citations
In the calculation of advanced citation impact indicators, we disregard self-citations.
A citation is considered a self-citation if the cited publication and the citing publication have at least one author (i.e. last name and initials) in common. The main
reason for excluding self-citations is that they often have a different purpose fromordinary citations. Specifically, self-citations may indicate how different publications of a researcher build on one another, or they may serve as a mechanism forself-promotion rather than for indicating relevant related work. Self-promotion canin turn be used to manipulate the impact of a publication in terms of the numberof citations received. Excluding self-citations from the analysis effectively reduces
the sensitivity of impact indicators to potential manipulation. In doing so, impact
indicators can be interpreted as the impact of researchers’ work on other membersof the scientific community rather than on his or her own work.
www.cwtsbv.nl

55

Citation data and analysis
Field Normalisation
There can be quite large differences in citation practices in different scientific fields.
Field normalisation is about correcting for differences in citation practices between
different scientific fields. The goal of field normalisation is to develop citation-based
indicators that allow for valid between-field comparisons.
In this report, we will use our in-house publication-based classification system ofscience to define the scientific fields that are used in this normalisation process.
This system has three major advantages compared to the conventional journal-basedclassification systems of science: Web of Science Journal Subject Categories:• Proper granularity in terms of fields.
• Fields are defined at the level of publications citing each other, not on allocating complete journals to field(s) where inaccuracies are introduced.
• Publications from journals like Nature, Science, PLoS ONE (multidisciplinary
journals) are allocated to the field they actually belong to and not to theartificial journal field ‘Multidisciplinary Sciences’.
The reasons to use this publication-based classification are furthered explained in
Annex B.
Counting method
Counting methods are about the way in which co-authored publications are handled.
For instance, if a publication is co-authored by researchers from two countries,
should the publication be counted as a full publication for each country or should
it be counted as half a publication for each of them? In this study, we use both fulland fractional counting. Full counting means that if a publication is co-authoredby multiple organisations, that publication counts multiple times, once for everyorganisation, regardless of the weight of their contribution. In this report, we use
mainly the full counted publications for output and fractionalised (by number of
institutions involved) for impact measures.

56

www.cwtsbv.nl

Interdisciplinary research

D Interdisciplinary research
While there are different understandings of interdisciplinarity, the definition thathas gained more consensus is the one provided by the US National Academy of
Sciences (2005) that states:“Interdisciplinary research (IDR) is a mode of research by teams or individuals that integrates information, data, techniques, tools, perspectives,
concepts, and/or theories from two or more disciplines or bodies of specialised knowledge to advance fundamental understanding or to solve
problems whose solutions are beyond the scope of a single disciplineor field of research practice."https://www.nap.edu/read/11153/chapter/4
There are two key elements in this definition we consider as basic notions to articulate our proposal: the concept of integration and the idea of combining knowledge
from two or more disciplines.
We characterise interdisciplinarity at the level of each individual publication, by
analysing the disciplines cited by the publication. This approach will allow us
to consider the citations to distinct disciplines by the same citing publication as aproxy of the integration of knowledge from different disciplines. For this analysis we
consider the Web of Science Journal Subject Categories as disciplines. We analyse
the degree or extent of integration through the concept of diversity. Diversity
is based on three concepts: variety, balance and disparity. We operationaliseinterdisciplinarity using Rao-Stirling diversity, an indicator which captures thethree inter-related concepts of diversity, and is computed as follows:X∆=pi pj dijij

(i ̸= j)Where pi is the proportion of cited references in the subject categoryi, pj is the proportion of cited references in the subject category j, anddij is the cognitive distance between the subject categories i and j
In this formula, disparity refers to the cognitive distance existing between two scientific disciplines (or subject categories, in our case). In order to compute thedisparity measure, we will create a similarity matrix Sij for the WoS subject categories based on the of citation flows between them. This will be then transformed
into a Salton’s cosine similarity matrix in the citing dimension. In this transformed
matrix, the Sij represents the similarity between each pair of WoS categories, thus
the cognitive distance (d) between two subject categories can be computed as d =1- Sij.
www.cwtsbv.nl

57

Interdisciplinary research

The indicators of interdisciplinarity will allow us to identify an institution’s subject
categories of a prepresenting the most interdisciplinary research.
We apply the state of the art in analysing interdisciplinarity using bibliometrictechniques. However, current approaches to characterise interdisciplinary researchfrom a bibliometric perspective remain contentious. Like any other methodology
suggested so far to measure and characterise interdisciplinarity based on scientificpublications, our approach is not free of limitations and therefore results of theseanalyses need to be interpreted with caution.

58

www.cwtsbv.nl