EA

Proportion of Open Access
Peer-Reviewed Papers at theEuropean and World Levels—2004-2011
August 2013

Eric Archambault, Didier Amyot, Philippe Deschamps,
Aurore Nicol, Lise Rebout & Guillaume Roberge

Proportion of Open Access
Peer-Reviewed Papers at theEuropean and World Levels—2004-2011
August 2013

Produced for theEuropean Commission DG Research & Innovation
RTD-B6-PP-2011-2: Study to develop a set of indicators to measure open access
The views expressed in this report are those of the authors anddo not necessarily represent the views of the European Commission

by Science-Metrix Inc.
Brussels | Montreal | Washington
1335 Mont-Royal E., Montréal
Québec, Canada, H2J 1Y6
1.514.495.6505
info@science-metrix.com
www.science-metrix.com
Cover image: iStockphotoProportion of OA papers—2004-2011

Executive Summary
This report re-assesses the Open Access (OA) availability of scholarly publications during the2004 to 2011 period, for 22 fields of knowledge, as well as for the European Research Area
countries, Brazil, Canada, Japan, and the US. Using a strategy to increase the number of freearticles retrieved (that is, which aims to increasing recall), led to close to a doubling of the
proportion of OA estimated by teams lead by Björk1 and by Harnad2. The present report showsthat the tipping point for OA (more than 50% of the papers available for free) has been reached inseveral countries, including Brazil, Switzerland, the Netherlands, the US, as well as in biomedical
research, biology, and mathematics and statistics.
Pilot study

This study comprised an important pilot phase which involved the retrieval of a set of 20,000
randomly selected records corresponding to papers published in 2008 from the Elsevier Scopus
database. This set of 20,000 records was then provided to the Harnad team in Montreal for ablind analysis using a protocol they had developed for previous studies. This test suggested thatthe proportion of total OA was as high as 32%, compared to the 22% Harnad’s team had obtainedin Thomson Reuters’ Web of Science (WoS). Extensive tests performed on a subset of 500 records
extracted randomly from the 20,000 records set suggested that 48% of the literature published in2008 was available for free in December 2012.
We inferred that OA availability had likely passed the tipping point in December 2012 for articles
published in 2008 and that the majority of peer-reviewed/scholarly papers published in journals
in that year were available for free in one form or another to end-users.
These carefully determined results diverge widely from the measures previously published.
Harnad’s team measurement of an OA availability rate of only 22% compared to the 48% rate
obtained here may be explained by Scopus’ broader coverage of the scientific literature comparedto WoS and by Google Scholar’s imperfect recall. These results also diverge from the measureobtained by Björk’s team, who used the Scopus database and suggested that only 20% of thearticles published in 2008 were available for free. This discrepancy may be explained by the timerequired for embargoed articles to appear online and by differences in the methodologicalapproach applied by Björk et al, who sought to measure the share of OA copies available to theaverage researcher based on Google searches, and excluding papers available in spite of
publishers' policies to the contrary. By contrast, our team aimed to measure the share of OA
copies available anywhere on the web, regardless of the status of the papers.
The final stage of the pilot study involved drawing a new random sample of 20,000 records from
Scopus, and adjusting the sample to include at least 100 records from the smaller fields in termsof number of articles (Philosophy & Theology, Visual & Performing Arts, General Arts,
Humanities & Social Sciences—GAHSS, Built Environment & Design). Our research projectrequires precisely determining the proportion of OA papers by estimating the number of OA

1 Björk, B. C., Welling, P., Laakso, M., Majlender, P., Hedlund, T., & Gudnason, G. (2010). Open Access To
The Scientific Journal Literature: Situation 2009. PLoS ONE, 5(6). doi: 10.1371/journal.pone.0011273.
2 Gargouri, Y., Larivière, V. Gingras, Y. and Harnad, S. (2012). Green and Gold Open Access Percentages

and Growth, by Field. In Archambault, É, Gingras, Y. and Larivière. V. (2012). Proceedings of 17th
International Conference on Science and Technology Indicators, Montréal: Science-Metrix and OST

August 2013
Produced for theEuropean Commission DG Research & InnovationiProportion of OA papers—2004-2011

peer-reviewed papers (the numerator) and dividing this by the number of peer-reviewed articles(the denominator) for 22 fields, and for the total literature. Since there is currently no extensive
database of scientific publications, the Ulrich periodical database at the journal level, in
conjunction with Scopus at the article level, was used here to provide an estimate of thedenominator. Although imperfect, Ulrich remains the most extensive, authoritative and probably
the least biased source of data on academic peer-reviewed journals. A sensitivity analysis revealed
that the distribution of records in Scopus is only slightly outside the boundaries of three models
used to estimate the denominator for 18 out of 22 fields.
Large scale study

The subsequent phase used a relatively large-scale measurement of OA availability based on asample of 320,000 randomly drawn papers for the Scopus database—that is, 40,000 records per
publication year between 2004 and 2011.
The same sample of 500 articles used in the pilot study was used for the characterisation of the
OA harvester used to measure availability of the 320k sample. A slight variation was observed in
the availability of articles in this sample measured in December 2012 (47.6%) and April 2013(44.8%). It is noteworthy that 249 articles were available for free at one time or another between
December 2012 and April 2013, just a hair under 50%. These results suggest that there areimportant transient aspects that need to be taken into consideration while measuring OA
availability. These results also show that the harvesting engine has very good retrieval precision(98%) and fairly good recall (86.6%), resulting in fairly robust measures of OA availability.
At the whole database level, there is an exponential growth of gold OA papers indexed in Scopus.
The growth rate is 24% per year (obtained through exponential regression curve fitting), whichmeans that the number of gold papers doubles every 2.9 years.
The availability of gold OA in a random sample of 320,000 papers, not surprisingly, closely
follows the population-level statistics. Green and hybrid OA availability grows in the distant pastand recedes in the more recent past. This is due, at least in part, to editors having embargo
periods on many of the papers in their journals which are sometimes available initially only
through subscription and are subsequently being made available for free.
The measurement of overall OA is based on the addition of gold OA and of hybrid and green OA.
According to this measure, 38% (the statistical margin of error is ± 0.5 percentage points) of the2004 peer-reviewed journal articles indexed in Scopus papers are currently available for free. This
proportion reached 44% (± 0.5) in 2011. The growth rate is very low, that is, only 1.9% per year.
This low growth rate over time likely reflects the translation of the OA availability curve for backyears. An adjusted OA availability curve can be computed by applying a conversion factor that
accounts for the precision and recall of the instrument (this calibration is based on the analysisof the 500 records sample). This estimation suggests that the tipping point of OA availability wasreached in 2011.
Free availability of a majority of articles has been reached in general science & technology, inbiomedical research, biology, and mathematics & statistics. The fields where OA availability ismost limited are the social sciences and humanities and in the more applied sciences,engineering, and technology. The lowest prevalence of OA availability is in visual and performing
arts (13%) and communication & textual studies.
A growth index was computed by dividing the percentage of OA availability in 2008-2011 by thatobserved in 2004-2007. Overall, between the two periods, there has been an 8% increase in OA
availability (slightly more than 3 percentage point). The fields with the fastest growth between
August 2013
Produced for theEuropean Commission DG Research & Innovationii

Proportion of OA papers—2004-2011

the most recent four years and the preceding four years are chemistry, general science &technology, public health and health services, clinical medicine, agriculture, fisheries & forestry,and enabling & strategic technologies.
All the fields derive an OA citation advantage. Interestingly, many of the fields where the OA
proportion is low have a sizeable citation advantage, such as in philosophy and theology (54%more cited), general arts humanities and social sciences, communication and textual studies,engineering, and visual & performing arts. What is particularly interesting here is that thecitation advantage is derived almost exclusively from the green and hybrid portion, as gold OA is
associated with a citation disadvantage on average for all fields except for physics & astronomy.
The statistics on gold journals require careful interpretation. First, many gold journals areyounger and smaller, and these factors have an adverse effect on the citation rate and hence on
measured ARC values. Authors frequently prefer reading and citing established journals, and it istherefore a challenge to start a journal from scratch, and to have authors submit high quality
articles. It takes time to build a reputation and to attract established authors. Importantly
though, gold journals might provide an avenue for less mainstream, more revolutionary science.
If so, the signature would be a much greater level of variation between the more highly cited
papers and the baseline (no citation).
An examination of OA availability was performed for EU28, EFTA (European Free Trade
Association), Accession countries, ERA (European Research Area), and for four additionalcountries, namely, Brazil, Canada, Japan, and the US. For the period 2008-2011 considered as awhole, eight of the EU28 (30%) have reached the tipping point. If the statistical precision andrecall of the harvesting instrument are taken into account, 20 out of 27 countries (74%) would
have tipped. Calibrating for precision and recall, the proportion of ERA countries having morethan 50% of papers in OA is 74%, that is, the same as for the EU28 overall.
In countries outside the ERA, it is noteworthy that the US has passed the tipping point andCanada is approaching. Even more salient is the proportion of 63% observed in Brazil. This is no
doubt due to the important contribution of Scielo, which plays a key role in the Southernhemisphere in making scientific knowledge more widely available.
State of Open Access scholarly publications

Between 2004 and 2011, the average annual rate of increase of OA availability was relatively
limited, with a compound growth rate of 2% per year. In addition to having year-on-year growth,
there is an upward translation of the whole availability curve over time. This is due to anincreasing number of authors making their manuscripts available for the current year but also
for previous years.
There are also transient effects that have to be considered when measuring OA availability,
including temporary promotional OA offered by publishers and variations in websites’
availability. All in all, more than 50% of the papers could be found for free in
November/December 2012 (pilot phase of this study) and in March/April 2013 (1st fullmeasurement stage) but somewhat less so at either time period. This shows that measuringphenomena on the Internet requires particular attention to detail and constant questioning on
the meaning of the results.
Green OA appears to be moving slowly, whereas gold and hybrid OA (such as pay-per-article for
OA release) appear to be driving in the fast lane. This impression will require further
investigation. Efforts should be made to characterise these changes, and to distinguish what

August 2013
Produced for theEuropean Commission DG Research & Innovationiii

Proportion of OA papers—2004-2011

percentage of growth comes from green self-archiving and what comes from other forms ofhybrid OA.
The fact that the open access tipping point has likely been reached is an important finding for
the whole publishing industry. This industry is likely to be undergoing revolutionary change, andat a pace much faster than anticipated, in large part because previous measures of OA availability
proved to be misleading. This means that aggressive publishers are likely to gain much in theredesigned landscape, whereas those attached to the old ways are likely to suffer and to lose
market share. An important question is whether the switch to a more atomistic, fine-grained
market with millions of researchers as buyers will reduce, augment or leave unchanged thenegotiating power of publishers.

August 2013
Produced for theEuropean Commission DG Research & Innovationiv

Proportion of OA papers—2004-2011

Contents
Executive Summary .....................................................................................................i
Contents.................................................................................................................. v
Tables .................................................................................................................... v
Figures ................................................................................................................... v
1
Introduction ...................................................................................................... 1
2
Methods .......................................................................................................... 3
2.1
2.2
2.3

3

National and regional policies, incentives and legislation ..................................................... 3
Strategy to measure the proportion of gold, green and hybrid OA in a large
sample .................................................................................................................................... 8
Key OA metrology concepts .................................................................................................. 8

Results ......................................................................................................... 10
3.1
3.2
3.3
3.4
3.5
3.6

Quality of the estimates ........................................................................................................ 10Gold OA as a proportion of scientific papers ....................................................................... 11Gold, and Green and Hybrid OA as a proportion of scientific papers ................................ 12
Availability of OA papers by field ......................................................................................... 13
Citation advantage and disadvantage of OA papers .......................................................... 15
Availability of OA papers in European and selected countries ........................................... 17

4
Discussion ..................................................................................................... 20
References ............................................................................................................ 22

TablesTable I
Table II
Table III
Table IV
Table V
Table VI
Table VII

Availability of OA in a sample of 500 Scopus records, 2008..................................................... 4
Sensitivity analysis of three models compared with Scopus, 2008 ........................................... 7
OA availability in April 2013 of a sample of 500 articles published in 2008 ............................. 11
Proportion of OA per field, 4-year non-weighted sampling, 2008-2011.................................. 14
Number of papers indexed in Scopus available in OA, 2008-2011......................................... 15
Rebased scientific impact (ARC) of OA publications, 2008-2011 ........................................... 16
Proportion of OA per country, 4-year non-weighted sampling, 2008-2011 ............................ 18

FiguresFigure 1
Figure 2
Figure 3
Figure 4

Accuracy and statistical precision ............................................................................................ 8
Number of papers from gold journals in Scopus, 1996-2011 ................................................. 12
Per cent of papers from gold journals in Scopus, 1996-2011 ................................................ 12
Per cent of freely available peer-reviewed papers, 2004-2011 ............................................... 13

August 2013
Produced for theEuropean Commission DG Research & Innovationv

Proportion of OA papers—2004-2011

1 Introduction
Since the 1990s, interest in the academic community for Open Access (OA) publications has been
increasing steadily, especially following the introduction of the arXiv e-print archives (arXiv.org).
Several articles appeared to promote self-archiving in the interest of making scientific knowledge
freely available to all. In parallel, an emerging movement aimed to measure and monitor OA
availability and impact. Quite early on, proponents of OA used these measures to promote freeavailability and it is not always easy to distinguish what papers with OA as a subject areattempting to do, i.e. advocate or measure OA. The present paper is all about metrology, notadvocacy.
The initial interest in the use of bibliometric methods, focused on accessing the so-called citation
advantage of OA as opposed to subscription-based journals (Antelman, 2004; Harnad & Brody,
2004; Craig, 2007). The literature of the time recognised a clear citation advantage to papers
available in OA as opposed to papers diffused solely through subscription-based journals. Strong
advocacy by authors such as Harnad (2003, 2008, 2012) suggested that benefits would ensue
from so-called green OA, that is, research papers self-archived by their authors in various types ofrepositories. Unsurprisingly, in this context, librarians and information scientists noted that they
had a new mission, which meant setting up and curating OA repositories (Proser, 2003; Bailey,2005; Chan, Kwok, & Yip, 2005; Chan, Devakos & Mircea, 2005 Repanovici, 2012).
A part of the OA literature has discussed how authors, researchers (Pelizzari, 2004; Swan &Brown, 2004; Dubini, Galimberti & Micheli, 2010) and publishers (Morris, 2003; Regazzi, 2004)
would react to this new paradigm. Evidently, business and economic models were discussed(Bilder, 2003; Kurek, Geurts & Roosendaal, 2006; Houghton, 2010; Lakshmi Poorna, Mymoon &Hariharan, 2012), but there was also interest in what models academia and libraries would follow(Rowland et al., 2004; Swan et al., 2005; Hu, Zhang & Chen, 2010).
As OA continued to make inroads, a growing number of papers examined the state ofdevelopment of OA in specific countries (Nyambi & Maynard, 2012; Sawant, 2012; WoutersenWindhouwer, 2012; Miguel et al., 2013) and in specific fields of research (Abad-Garcí et al., 2010;
Gentil-Beccot, Mele, & Brook, 2010; Charles, & Booth, 2011; Henderson, 2013). In this context, it
was not surprising to find papers that addressed the general question of OA availability as aproportion of the scientific literature, and the proportion of OA papers available in different
fields of science (Björk et al. 2010; Gargouri et al., 2012).
This paper re-assesses OA availability during the 2004-2011 period by carefully tuning harvesting
methods in order to increase recall. The current version of the harvesting engine developed by
Science-Metrix searches on specific sites including Scielo, PubMed Central and the websites ofscientific peer-reviewed journals publishers, uses a locally hosted version of large-scale specialised
repositories such as arXiv and CiteSeerX,3 and systematically harvests metadata frominstitutional repositories listed in the Registry of Open Access Repositories (ROAR) and the
Directory of Open Access Repositories (OpenDOAR).
The approach used here leads to a measurement of OA availability which is close to a doubling of
the proportion of OA estimated by Björk et al. and by Gargouri et al. The present paper shows

3 The authors would like to thank Lee Giles and Douglas Jordan at Penn State for giving access to
CiteSeerX data.

August 2013
Produced for theEuropean Commission DG Research & Innovation1

Proportion of OA papers—2004-2011that the tipping point for OA (more than 50% of the papers available for free) has been reached inseveral countries, including Brazil, Switzerland, Netherlands, the US, and in biomedical research,
biology, and mathematics and statistics. Data are presented for 22 fields of knowledge, as well as
for the European Research Area countries, Brazil, Canada, Japan, and the US. Before entering
into the methodological details associated with the measurement of OA, it is important toproduce operational definitions of OA, green OA, gold OA, and hybrid OA.
Types of OA scientific literature: Peter Suber suggests that ‘[o]pen-access (OA) literature isdigital, online, free of charge, and free of most copyright and licensing restrictions.’4 A colloquial
definition of OA would be ‘OA, whether Green or Gold, is about giving people free access to peerreviewed research journal articles.’5 The following operational definitions of gold, green andhybrid OA will be used in the present study.

Gold OA refers to journals that use a funding model that does not charge readers or their
institutions for access, and makes all contents available without embargo period.
Green OA generally refers to authors’ self-archiving [of papers accepted in academic journals
following a successful peer-review process].
Hybrid OA is an increasingly important trend in scientific publishing by which authors pay
for their papers to be available in OA in an otherwise not OA journal—‘[h]ybrid open accessjournals provide Gold OA only for those individual articles for which their authors (or theirauthor’s institution or funder) pay an OA publishing fee.’ There are other cases such as therelease of subscription-based journal articles after an embargo period, this type of OA
articles could also be called delayed OA. There are cases where editors make articles available
for free for limited period of time for promotional purpose but then retract them. This is infact time-limited OA and presents specific measurement problems.

4 http://www.earlham.edu/~peters/fos/overview.htm.
5 http://scholarlykitchen.sspnet.org/2011/09/07/oa-rhetoric-economics-and-the-definition-of-research/.

August 2013
Produced for theEuropean Commission DG Research & Innovation2

Proportion of OA papers—2004-2011

2 Methods
This study comprised an important pilot phase (Section 2.1) followed by a phase of relatively
large scale measurement (Section 2.2). Key metrology concepts used in this report are presented
in Section 2.3.

2.1

National and regional policies, incentives and legislation

The pilot phase comprised four stages. A first stage involved the development of a manual
retrieval process where we sought to retrieve 20,000 randomly selected papers form the Elsevier
Scopus database. The retrieval was made in a manner reminiscent of that used by Björk et al.
(2010). Although we later discovered that the sample had some randomness imperfection, it was
sufficient for the initial experimental phase. Importantly, this approach was abandoned after
three months as it appeared to be prohibitively expensive and extremely slow, and as we noticed
that our approach contained a methodological flaw which limited the proportion of papers weretrieved from Google Scholar. Importantly also, Google Scholar routinely blacklisted ourmanually operated retrieval instrument, thus curtailing our measurement efforts and showing
the limits of relying on that source of data for large scale measurement as typically performed in
bibliometric studies.
A second stage started with 20,000 records extracted from Scopus for the year 2008 being
provided to the Stevan Harnad team in Montreal. This test suggested that the proportion of total
OA was as high 32%, compared to the 22% Harnad’s team obtained in Thomson Reuters’ Web of
Science (WoS) (Gargouri et al., 2012). As Björk et al. found a score of 20% using Scopus andGoogle as a search engine, it appeared necessary to perform an in-depth inquiry on a smaller
sample to determine whether these new scores were erroneous. After all, this suggested that OA
availability was about 50% higher than previous measures suggested.
In the third stage of the pilot study, some 500 of the 20,000 records set were then extracted
randomly and extensive tests were performed. The records were all searched manually in Google
Scholar, Google, and Microsoft Academics. Records that could be downloaded for free and thatcame from any of these sources were considered OA, and the carefully verified sample called a‘ground truth.’ Importantly, the ‘ground truth’ can be considered a floor value as none of thesearch engines used can be considered to have perfect recall, that is, the capacity to retrieve all
relevant results.
These tests led to the following observations: Google Scholar and Google have substantial
overlap, but each search engine has a somewhat distinct set of positive results (Table I). Microsoft
Academics does not add much to the combined results of Google and Google Scholar.
Importantly also, the results obtained suggest that the accuracy of the data collection method,
and the coverage of the database, are more important than a large sample size (statisticalprecision).
Extensive testing was done with the subsample of 500 records. The results for the Harnad’s teamrobot are as is and contain a few false positives, so the real positive score is actually lower. The
Scholar, Google and Ground Truth results were meticulously validated by hand and the
documents downloaded, and as such, they can be considered as being highly accurate. The
Ground Truth comprises the combined validated results from Google and Google Scholar in
addition to one result from Microsoft Academics. Results from Microsoft Academics are notshown, as only the negative results from Scholar and Google were tested to examine whether thisadded any substantial results to the previous ones. Please note that these results were obtained inAugust 2013
Produced for theEuropean Commission DG Research & Innovation3

Proportion of OA papers—2004-2011

December 2012. Our tests showed that some of the documents freely available at that timeceased to be free later. This is certainly one difficulty in the measurement of OA, the Internet isvery organic and changes constantly.

Table I

Availability of OA in a sample of 500 Scopus records, 2008

Result
FALSE
TRUE
Total
% OA

UQAM (Gargouri-Harnad)
350
150
500
30%

Source:Scholar
293
207
500
41%

Google
290
210
500
42%

Ground Truth
262
238
500
48%

Computed by Science-Metrix

This analysis suggested that 48% of the literature published in 2008 was available for free in
December 2012. Despite their high level of sophistication, neither Google nor Google Scholar
can be expected to crawl the Web perfectly or to have a search engine so robust that it
systematically presents all the relevant records in the first page of results (to which we limited ouranalysis), and hence cannot be expected to have a 100% recall, especially for academic articles(Arlitsch & O'Brien, 2012). Consequently, we inferred that OA availability had likely passed thetipping point in December 2012 for 2008 articles and that the majority of peerreviewed/scholarly papers published in journals in that year were available for free in one form or
another to end-users.
An important question is why these carefully determined results are diverging so much from themeasures previously published, including those published by Harnad’s team itself (Gargouri etal., 2012). Our initial tentative explanation was that this difference was likely due to the use of
Scopus by our team, as opposed to the Web of Science as Harnad’s team had done before. This
explanation could account to an increase of 10 percentage points, that is, from 22% availability
using WoS to 32% using Scopus with the same harvesting engine used by the Harnad team. Basedon his answers and comments made in a conference,6 Harnad appears to prefer measuring OA
availability only in the most highly cited portion of the scientific literature. This can be doneusing WoS as compared to Scopus which has somewhat more extensive coverage. This isdifferent from the objective of the present team which aims to estimate the proportion of OA
availability for all peer- or editorially-reviewed scholarly journals. We feel this objective isimportant as emerging OA journals will frequently have meagre citation scores, and are thus
excluded from the WoS which concentrates on highly cited journals, but that they should benevertheless be taken into account to warrant a more comprehensive understanding of theevolution of scientific publication. However, it is also obvious that the harvester used by
Harnad’s team has quite an imperfect recall as it caught only 30% of OA articles using Google
Scholar, whereas we retrieved 41% by hand with the same search engine. Combining the use of
WoS instead of Scopus, and taking into consideration the imperfect recall of Google Scholar, andthe imperfect recall of Harnad's team robot, goes a long way in explaining why Harnad’s teammeasured an OA availability rate of only 22% compared to the 48% rate obtained here.
Another important divergence is with the measure obtained by Björk’s team, who used the
Scopus database. They suggested that only 20% of the articles published in 2008 were available
for free. This is half the figure obtained in our own tests in Google as we were able to retrieve 42%

6 Science and Technology Indicators (STI) Conference, Montreal 2012.

August 2013
Produced for theEuropean Commission DG Research & Innovation4

Proportion of OA papers—2004-2011of the articles for free. At least three aspects may have played a role: 1) it takes a certain time for
embargoed articles to make their way to the web, and authors may not always rapidly self-archiveor otherwise post their papers on sites such as Research Gate, and as our measures were madethree years later (Autumn 2012 versus Autumn 2009), it is very likely that more papers had bythen appeared for free; 2) Björk et al aimed to answer “what share of OA copies would the average
researcher find” whereas our team aims to answer “what share of OA copies are available
somewhere on the web”, and hence we did not act as average researchers but as researchersdetermined to find all articles that Google had indexed, however hard it may be to retrieve themusing Google, and other search engines; 3) Björk et al. removed articles which could accidentally
be available for free “clearly against the site policy”. We made no such attempt as our researchquestion seeks to find what is available for free at a given time.
As discussed later in the paper, there is an important transient aspect to OA availability.
Unknown to our team at the time the sample was manually searched, Springer had made several
articles available for free during Autumn 2012 but subsequently many of them were withdrawn
from free circulation. However, and despite this, six months later the proportion of OA papers
found in Google was very similar as new free articles had appeared by then.
The final, fourth stage of the pilot study involved drawing a new random sample of 20,000
records from Scopus. The aim here was twofold; first, to use this sample to calibrate a new OA
harvesting engine purposefully developed for this project taking into account what was learnedin the previous stages of the pilot study, and second, develop a technique to estimate the overall
proportion of OA availability in Scopus using the Ulrich periodical database as a calibration aid.
This sample was restricted to papers published in 2008, and the results were restricted to originalcontributions to knowledge (e.g. document type similar to article, notes and reviews). Records
where the journal name or the record type contained a conference were excluded as well as thosefor which the field was unknown. The eligible record set from 2008, comprising somewhat morethan 1.36 million records in Scopus, was ‘tossed’ five times using a pseudo-random method(using the newid() command in SQL Server). Then, a subset of 100,000 records was selected,
placed in a subset, and tossed again. These 100,000 records were then imported into Excel, wherea straightforward analysis of the distribution of the records by field was performed. This analysis
showed that a subsample of 20,000 records would keep few records in three of the smaller fields(Philosophy & Theology, Visual & Performing Arts, and General Arts, Humanities & Social
Sciences—GAHSS). For these fields, a random sample of 100 records was selected, and for the
Built Environment & Design field, the 101 records that were part of the 100,000 records were all
selected. As the objective was to produce a record set of 20,000, a subsequent selection was donefor 19,599 records. These were selected by tossing the 100,000 a few more times using the rand()command in Excel, then proceeding to the selection of the required number of records.
An original aspect of the present project is our willingness to assess the proportion of OA
availability in all scholarly articles published in peer- and editorially-reviewed journals. There iscurrently no such extensive database of scientific publications. The Web of Science traditionally
aimed to cover only the most highly cited and authoritative journals and while competition from
Elsevier’s Scopus has had the effect of pushing Thomson Reuters to augment the coverage of the
WoS, it is far from being an extensive database of all peer-reviewed journals, as we would have
liked for this project. Scopus has a more extensive coverage, and adds journals which typically are
somewhat less frequently cited. However, it still leaves out several smaller journals, and it ishighly likely that those excluded will frequently be those published in a non-English language. It
is remarkable that despite the fact that R&D spending at the world level approaches US $ 2
trillion per year (authors' own estimate using UNESCO statistics on R&D), no one can tell how
August 2013
Produced for theEuropean Commission DG Research & Innovation5

Proportion of OA papers—2004-2011

many scholarly peer-reviewed journals there are currently. This makes the task of measuring the
proportion of OA availability in that set a complex exercise as the ‘denominator’, that is, the total
number of peer-reviewed articles published per year, is currently unknown.
Our research project requires precisely determining the proportion of OA papers by estimating
the number of OA peer-reviewed papers (the numerator) and dividing this by the number of peerreviewed articles (the denominator) for 22 fields, and for the total literature. The Ulrich
periodical database was used here to provide an estimate of the denominator, that is, thepopulation of peer-reviewed scholarly journals. The strengths and weaknesses of Ulrich data are
well known: for example, some journals that should be classified as peer-reviewed are not (andthe reverse is also true). A good example of this is the OA journal Activités, which mentions that
‘Texts that have been submitted to Activités (www.activites.org/) will be assessed by two referees(called upon in view of the article). Each will give his or her opinion on the text.’7 Despite this,and a detailed description that clearly suggests scholarly content and the presence in papers ofreferences to scholarly work, Ulrich has not classified this journal as refereed. Although several
journals are likely to be classified ‘Academic/Scholarly’ in Ulrich and might be considered ascontributing to science, this category cannot be included en masse as it comprises a substantial
amount of material published in universities that has little scientific content. This is the case, forexample, with the ‘The Hilltop’, classified by Ulrich as Academic/Scholarly, and claiming to be
the ‘The Student Voice of Howard University’ (see http://www.thehilltoponline.com/).
Consequently, the selection was restricted to Ulrich listed journals considered refereed/peerreviewed AND Academic/Scholarly.
Although imperfect, Ulrich remains the most extensive, authoritative and probably the least
biased source of data on academic peer-reviewed journals. The reason Ulrich was used is because
it was felt that publishers of article-level database publishers such as Elsevier (publisher of Scopus)
and Thomson Reuters (WoS) were faced with choices having important commercial andprofitability impacts. When selecting journals to be included for an article-level database such as
Scopus, deciding whether to include a journal has a direct impact on production costs and partly
because of this, database publishers tend to have a bias towards larger journals (which help
achieve economies of scale) and larger publishers (which help lower transaction costs andeconomies of scale and several similarly formatted journals are provided).
Whether a journal is small or large in terms of number of articles has substantially fewer
consequences when it is included in a journal title database, where journal size can be expected tohave comparatively little impact on cost (some differences remain as it is likely easier to findinformation about the larger journals even when building journal-level databases such as Ulrich).
Importantly, one can only assume there are still important economies of scale involved withcovering large editors: even for inclusion in Ulrich, it is less costly to input data from one
publisher with one thousand journals, and covering one journal each for 2000 publishers. Thismeans that smaller, more marginal journals and editors are more likely to be excluded from
Ulrich, and these are perhaps more likely to be non-English journals and editors. Hence, Ulrich isimperfect but it still more likely to be more broadly balanced than Scopus and the WoS.
The technique used to estimate the proportion of each scientific field in the overall total involved
the following three steps: 1) journals in Ulrich were matched to those contained in Scopus; 2)journals that intersected were given the field that was already contained in the classification of
7 http://www.activites.org/resources/activites.eng.book.pdf

August 2013
Produced for theEuropean Commission DG Research & Innovation

6

Proportion of OA papers—2004-2011

Scopus journals used by Science-Metrix. For those that did not intersect, the Ulrich classification
was compared with that used in our classification, and a matching table was used to attributeone of 22 fields to each of the journals; and 3) the number of articles per field was counted in theintersecting set, while the number of articles in the Ulrich set with no Scopus counterparts wasdetermined by projecting the average number of articles for the 50% journals in Scopus with thefewest articles per journal. The reason for using the average number of articles for the 50%
smaller journals is that experience has revealed that databases such as Scopus and the WoS tend
to prioritize the largest journals. For instance, the WoS covers about 12,000 journals, and Scopus
about 18,000. Despite a 50% increase in journal coverage, Scopus only has about 20% more
articles.
A sensitivity analysis was performed to see the effect of calculating the average for the 75%, 50%,
and 25% smallest journals (ranked by decreasing number of articles), and the results were broadly
similar. This analysis reveals that the distribution of records in Scopus is inside the boundaries ofthese models for 18 out of 22 fields, the four fields where Scopus’ proportion is outside are green
shaded. In relative terms, the largest discrepancies between the Scopus sample are in General
Science & Technology, Communication & Textual Studies, Visual & Performing Arts, Philosophy& Theology, and the Social Sciences. Except for the General Science & Technology field, all these
are underestimated in Scopus compared with the modelled projection (the 50% model).
Importantly though, these are all relatively small fields in terms of peer-reviewed papers and, inthe end, the correction of total OA availability based on this approach produced the same overall
result as the substantially simpler random sample drawn from Scopus (in the pilot phase, both
the random Scopus score and the Ulrich-calibrated score using the purposefully developed OA
harvester yielded an availability score of 42%).

Table II

Sensitivity analysis of three models compared with Scopus, 2008

Discipline

25% smallest

50% smallest

75% smallest

Agriculture, Fisheries & Forestry

3.75%

3.84%

3.90%

Scopus3.74%

Biology

4.81%

4.96%

5.15%

4.29%

Biomedical Research

8.54%

8.31%

8.09%

8.00%

Built Environment & Design

0.49%

0.50%

0.50%

0.55%

Chemistry

8.72%

8.40%

8.11%

8.29%

Clinical Medicine

26.12%

25.89%

25.78%

27.62%

Communication & Textual Studies

0.98%

1.13%

1.24%

0.63%

Earth & Environmental Sciences

3.03%

3.00%

3.00%

3.02%

Economics & Business

2.94%

3.07%

3.13%

2.47%

Enabling & Strategic Technologies

6.54%

6.39%

6.34%

7.62%

Engineering

5.80%

5.78%

5.84%

6.75%

General Arts, Humanities & Social Sciences

0.14%

0.13%

0.12%

0.12%

General Science & Technology

0.80%

0.81%

0.83%

1.45%

Historical Studies

1.03%

1.13%

1.16%

0.91%

Information & Communication Technologies

2.93%

2.92%

2.95%

3.38%

Mathematics & Statistics

3.04%

3.10%

3.13%

2.71%

Philosophy & Theology

0.69%

0.78%

0.82%

0.45%

Physics & Astronomy

9.91%

9.58%

9.36%

9.99%

Psychology & Cognitive Sciences

2.22%

2.20%

2.18%

1.94%

Public Health & Health Services

3.01%

2.96%

2.91%

2.95%

Social Sciences

4.36%

4.96%

5.26%

3.02%

Visual & Performing Arts

0.15%

0.18%

0.21%

0.10%

Source: Computed by Science-Metrix using Ulrich (Serials Solutions) and the Scopus (Elsevier) databases.

August 2013
Produced for theEuropean Commission DG Research & Innovation

7

Proportion of OA papers—2004-2011

2.2

Strategy to measure the proportion of gold, green and hybrid OA in alarge sample

Somewhat distinct strategies were used to calculate the occurrence of gold OA and total OA. For
gold articles, an estimate of the proportion of papers was made from the random sample bymatching the journals that were known to be gold the year a paper was published. These journals
were obtained from the Directory of Open Access Journals (DOAJ) and the list of OA journals in
PubMed Central. This was done by matching journals’ ISSN, E-ISSN and names from Scopus to
the relevant records in the sample (the matching had close to 100% precision, but recall may havebeen imperfect, hence the figures presented here can be considered a floor, rather than a ceiling).
Statistics were computed for gold on both the Scopus sample and for the overall population ofarticles present in the Scopus databases.
Except for the details on the gold journals measurement methods, all the aforementioned
methodological details and results were for the pilot phase. The subsequent measurement stage
used a relatively large-scale measurement of OA availability based on a sample of 320,000
randomly papers drawn for the Scopus database—that is, 40,000 records per publication year
between 2004 and 2011.8 Additional articles were drawn for smaller fields but these were onlyused to compute statistics at the field level, the overall proportion relied on the quasi-random
selection process of 320,000 papers. Before presenting the results of the study, important
metrology concepts are first presented.

2.3

Key OA metrology concepts

This paper presents results using two important metrology concepts: (1) accuracy, which reflect
the quality of the instruments used and the care taken in making measurements; (2) precision,
which reflect the use of repeated measures, sampling and statistical analysis (see Figure 1)—thelater concept will be called statistical precision for reasons that will become obvious.

Figure 1

Accuracy and statistical precision

Source:Adapted from http://en.wikipedia.org/wiki/Accuracy_and_precision

8 With hindsight, the use of a constant number of papers per year (40,000) was not a judicious decision as

it does not reflect annual variations in the number of papers. A forthcoming update to this study will
sample a large number of papers over the whole period and, in the presence of the law of large numbers,
one will be able to assume that this sample will naturally reflect the rate of change observed in the
database. This will make it easier to compute accurate statistics and lower error margins.

August 2013
Produced for theEuropean Commission DG Research & Innovation8

Proportion of OA papers—2004-2011

Statistical precision can be examined with the margin of error (ME). For a proportion (p) where
the population is finite and known, (N) is not systematically much larger than the sample size(n), and in which the values are discrete (for example, papers), given a critical score Z (which will
be set at 0.95 in the study), ME is calculated as follows:

1

0.5
1

What complicates the use of these definitions is the need to examine accuracy with two moreconcepts used in information retrieval: recall and precision (hence the need to call the previous
concept ‘statistical precision’; the second precision-related concept will be referred to as ‘retrieval
precision’). Recall is the proportion of relevant records that are retrieved, while retrieval precision
is the proportion of retrieved records that are relevant. If an instrument retrieves 25 records ofwhich only 20 are relevant, and fails to retrieve 30 additional relevant records, its retrieval
precision is 20/25 = 80% while its recall is 25/50 = 50%. Precision is related to Type I errors (false
positives), and recall to type II errors (false negatives). Thus, a high recall means that an
instrument returned most of the relevant results, while high retrieval precision means that it
retrieved more relevant results than irrelevant ones.

August 2013
Produced for theEuropean Commission DG Research & Innovation

9

Proportion of OA papers—2004-2011

3 Results
This section examines the quality of the estimates obtained in this study (Section 3.1) followed
by a presentation of the proportion of gold OA in Scopus (Section 3.2). The proportion of OA
papers published between 2004 and 2011 as measured in April 2013 is presented in Section 3.3
followed by a description of OA availability by field of knowledge production (Section 3.4). The
OA citation advantage or disadvantage is briefly examined in Section 3.5 while Section 3.6
provides a brief geographical examination of OA availability.

3.1

Quality of the estimates

The results from a perfect classifier would solely comprise a mix of true positives and true
negatives. A true positive (tp) in the present case is a paper known to be available in OA which isfound by the harvesting instrument developed in the current project. A true negative (tn) is anarticle which is not available for free and is not found by the instrument. However, such an
instrument rarely is perfect and there are usually false positives (fp), that is, articles not available
for free but wrongly assigned to that category, and false negatives (fn), that is, articles which areavailable for free but are not found by the instrument. These concepts can be used to characterise
how good measurement is with a number of indicators.
Retrieval accuracy provides an overall estimate of the instrument capacity to yield true results asa percentage of records, or more formally, the “closeness of agreement between a measured
quantity value and a true quantity value of a measurand”.9 It is given by:
Accuracy =Retrieval precision, also called positive predictive value, provides an estimation of how frequently
the instrument finds correctly positive results and is given by:
Precision =Specificity, or true negative rate, is the capacity of the instrument to correctly identify negative
results:
Specificity =Finally, recall, also called true positive rate or sensitivity, is the capacity to correctly identify alarge proportion of the positive records:
Recall =The same sample of 500 articles used in the pilot study was also used for the characterisation of
the OA harvester used to measure availability of the 320k sample. Whereas 238 articles wereavailable for free in December 2012 (47.6%), some 224 articles could be found in April 2013(44.8%). It is also noteworthy that 249 articles were available for free at one time or anotherbetween December 2012 and April 2013, just a hair under 50%. Between December 2012 andApril 2013, there were 11 new records that appeared in OA, but 25 disappeared, a large part of

9 BIPM. 2012. International vocabulary of metrology – Basic and general concepts and associated terms (VIM). 3rdedition. JCGM 200:2012.

August 2013
Produced for theEuropean Commission DG Research & Innovation

10

Proportion of OA papers—2004-2011

them (13 out of 25) were actually Springer articles which were available for free during thelimited time of a promotion. Most of the others were on websites that disappeared, or no longer
appeared on the website where they were originally found. These results suggest that there areimportant transient aspects that need to be taken into consideration while measuring OA
availability. Free articles sometimes come and go and it is therefore important to present
information not only on the year of publication of articles, but also when the measure wasactually made. This also makes replication of results a challenging undertaking.

Table IIIOA availability in April 2013 of a sample of 500 articles published in 2008

Type of results

ArticlesQuality Tests

Score

True positive (tp)194

Retrieval accuracy

93.2%

True negative (tn)272

Retrieval precision

98.0%

False positive (fp)4

Specificity

98.6%

False negative (fn)30

Recall

86.6%

These results show that the harvesting engine has very good retrieval precision (98%) and fairly
good recall (86.6%). Our design goal is to maximize retrieval precision at the expense of recall,and progressively improve recall while maintaining or improving precision. We therefore
consider the harvesting engine’s results as consistent with these goals and sufficiently good topresent fairly robust measures of OA availability. Please note that even the method used here is afloor of OA availability as neither Google nor Google Scholar, which were used here to determinethe ‘ground truth’, can be expected to have perfect recall themselves.

3.2

Gold OA as a proportion of scientific papers

Publications in gold OA journals are examined first at the whole databases level and willsubsequently be examined within the sample in greater details. Figure 2 and Figure 3 present
current trends on the evolution of gold OA articles availability in Scopus from 1996 to 2011.
Figure 2 shows there is an exponential growth of gold OA papers indexed in Scopus. The growthrate is 24% per year (obtained through exponential regression curve fitting), which means thatthe number of gold papers doubles every 2.9 years. There are currently about 940,000 papers
indexed from gold journals in Scopus for the years 1996 to 2011.
The percentage of gold peer-reviewed articles available in Scopus for 1996 was only 0.9% but grewto 11.5% for 2011, the overall compound annual growth rate (CAGR) being 18.7% (Figure 3).
Importantly, an availability score for a given year does not mean that such a proportion ofarticles was initially available at the time. For example, though there are now 0.9% of 1996 papers
from Scopus which are available in gold journals, it might have been lower at the time. The
reason is that journals sometimes convert to gold status and make their whole back year
collection freely available along the way. This creates an upward translation of the availability
curve.

August 2013
Produced for theEuropean Commission DG Research & Innovation11

Proportion of OA papers—2004-2011% of papers available in gold OA

20%

15%
y = 1E-151e0.1718x
R² = 0.9902
10%

5%

0%
1994

1996

1998

2000

2002

2004

2006

2008

Figure 2

Number of papers from gold journals in Scopus, 1996-2011

Source:Computed by Science-Metrix using DOAJ, PubMedCentral, Scopus.

2010

2012

2010

2012

Number of papers available in gold OA

250,000

200,000

y = 3E-186e0.2184x
R² = 0.9984

150,000

100,000

50,000

0
1994

1996

1998

2000

2002

2004

2006

2008

Figure 3

Per cent of papers from gold journals in Scopus, 1996-2011

Source:Computed by Science-Metrix using DOAJ, PubMedCentral, Scopus.

3.3

Gold, and Green and Hybrid OA as a proportion of scientific papers

The results from the harvesting of the random sample of 320,000 papers are presented in Figure
4. Starting from the bottom, one can find the availability of gold OA in this sample and, notsurprisingly, the results closely follow the population-level statistics presented in Figure 3. As
stated earlier, the green and hybrid OA represent a heterogeneous mix of papers comprising selfarchived version of papers published in subscription-based journals, papers made available forfree by editors, or because authors paid upfront fees to provide free access to their papers. Papers
can also be found on aggregator websites such as CiteSeerX, ResearchGate and others. In fact, thehybrid and green OA category comprises a complex continuum of ways to offer papers for free.
As one can see, green and hybrid OA availability grows in the distant past and recedes in the more
recent past. This is at least in part due to editors having embargo periods on many of the papers
August 2013
Produced for theEuropean Commission DG Research & Innovation

12

Proportion of OA papers—2004-2011

in their journals which are sometimes available initially only through subscription and aresubsequently being made available for free.
The estimate of overall OA proportion is based on the sum of gold OA and of hybrid and green
OA. According to this measure, there are 38% (the statistical margin of error is ± 0.5 percentage
points) of the 2004 peer-reviewed journal articles indexed in Scopus papers which are currently
available for free. This proportion reached 44% (± 0.5) for papers published in 2011. The growthrate is very low, that is, only 1.9% per year. This low growth rate over time likely reflects that acore phenomenon to consider is the translation of the OA availability curve for back years. Onceagain, this translation is due to the fact that when authors and editors decide to embrace OA,
they often make both recent and older publications available for free, thus creating an upward
translation in OA availability.
An adjusted OA availability curve can be computed by applying a conversion factor that accounts
for the precision and recall of the instrument (this calibration is based on the analysis of the 500
records sample). This estimation suggests that the tipping point of OA availability was reached in
2011. Though it is an estimate, one can be relatively confident that this is more a floor estimate
than a ceiling as there is currently no gold standard for retrieving every freely available peerreviewed paper (even by the search engines used to establish the ground truth). Moreover, one
may suspect that Scopus (and WoS also) may have a bias towards non-gold journals, and having
a perfectly non-biased database as a starting point would likely yield slightly higher scores.
60%
55%% of papers available in OA

50%
45%
40%
35%
30%
Adjusted OA

25%

Total OA

20%

Green & Hybrid

15%

Gold

10%
5%
0%
2003

2004

2005

2006

2007

2008

2009

2010

Figure 4

Per cent of freely available peer-reviewed papers, 2004-2011

Source:Computed by Science-Metrix using DOAJ, PubMedCentral, and Scopus.

3.4

2011

2012

Availability of OA papers by field

Table IV presents data on the proportion of articles per field which are available in green andhybrid forms, in gold journals, and in OA overall. Considering that these data can be considered
as floor values, there is little doubt that free availability of a majority of articles has been reachedin general science & technology, in biomedical research, biology, and mathematics & statistics.
The general S&T category comprises prestigious journals such as PNAS, Science, Nature, but also
gold journals such as PLoS ONE, Scientific Reports, European Journal of Scientific Research, andthe Journal of Applied Sciences. The large availability of biomedical papers is owed at least in part
August 2013
Produced for theEuropean Commission DG Research & Innovation

13

Proportion of OA papers—2004-2011

to the effort by the US National Institute of Health’s OA policy and the presence of the
PubMedCentral website which serves as a central platform for the diffusion of scientific evidencein biomedical research and clinical medicine.

Table IV

Proportion of OA per field, 4-year non-weighted sampling, 2008-2011

Field

PapersAgriculture, Fisheries & Forestry
Biology
Biomedical Research
Built Environment & Design
Chemistry
Clinical Medicine
Communication & Textual Studies
Earth & Environmental Sciences
Economics & Business
Enabling & Strategic Technologies
Engineering
General Arts, Humanities & Social Sciences*
General Science & Technology
Historical Studies
Information & Communication Technologies
Mathematics & Statistics
Philosophy & Theology
Physics & Astronomy
Psychology & Cognitive Sciences
Public Health & Health Services
Social Sciences
Visual & Performing Arts*
Total*

6,142
7,031
12,571
998
13,39942,806
1,108
4,700
3,950
12,714
11,264
8,220
2,760
1,590
5,620
4,318
732
15,028
3,027
4,688
5,170
6,572
160,000

Notes:

Source:Green & Hybrid%
Papers1,655
27 ± 1
2,749
39 ± 1
6,377
51 ± 0.9237
24 ± 32,642
20 ± 0.7
15,47936 ± 0.5168
15 ± 2
1,771
38 ± 1
1,583
40 ± 22,567
20 ± 0.7
2,366
21 ± 0.8
2,250
27 ± 0.2
720
26 ± 2374
24 ± 2
1,993
35 ± 1
2,091
48 ± 1170
23 ± 35,824
39 ± 0.8
1,204
40 ± 21,640
35 ± 11,374
27 ± 1609
9 ± 0.1
53,07233 ± 0.2

Gold%Papers1,033 17 ± 0.9
1,224 17 ± 0.9
1,346 11 ± 0.5323± 11,275 10 ± 0.5
4,658 11 ± 0.3
63
6± 1
339
7 ± 0.7191
5 ± 0.7
1,061
8 ± 0.5269
2 ± 0.3137
2 ± 0.0
1,036 38 ± 2
124
8± 1
584 10 ± 0.8
310
7 ± 0.838
5± 2
698
5 ± 0.3172
6 ± 0.8
589 13 ± 0.9489
9 ± 0.8231
4 ± 0.1
15,538 10 ± 0.1

OA
Papers
2,688
3,973
7,723
269
3,917
20,137
231
2,110
1,774
3,628
2,635
2,387
1,756
498
2,577
2,401
208
6,522
1,376
2,229
1,863
840
68,610%44 ± 157 ± 161 ± 0.8
27 ± 329 ± 0.847 ± 0.5
21 ± 245 ± 145 ± 229 ± 0.8
23 ± 0.8
29 ± 0.2
64 ± 231 ± 246 ± 156 ± 128 ± 343 ± 0.845 ± 248 ± 136 ± 113 ± 0.1
43 ± 0.2

OA Growth
Trend
Index
1.13
1.08
1.04
0.99
1.39
1.17
1.06
0.95
1.02
1.11
1.03
1.08
1.28
0.99
1.01
0.95
1.03
0.98
1.10
1.21
1.00
1.00
1.08

*In order to decrease the margin of error for the fields of General Arts, Humanities & Social Sciences,and Visual & Performing Arts, the Scopus population of papers were used instead of a sample.
However, this does not affect the error margin for the total as this was based on sample data only.
Computed by Science-Metrix using DOAJ, PubMedCentral, and Scopus.

A growth index was computed by dividing the percentage of OA availability in 2008-2011 by thatobserved in 2004-2007. Overall, between the two periods, there has been an 8% increase in OA
availability (slightly more than 3 percentage point). The fields with the fastest growth between
the most recent four years and the preceding four years are chemistry, general science &technology, public health and health services, clinical medicine, agriculture, fisheries & forestry,and enabling & strategic technologies. Here, one can suspect once again that the NIH policy is atplay (in public health and clinical medicine). Another important observation is that some of themore applied sciences, where OA was not all that prevalent in 2004, appear to be catching up.
Also noteworthy is the observation that the cradle of OA availability is now standing still: there isno growth in physics & astronomy. Finally, growth is slightly negative in mathematics &statistics, earth & environmental sciences, built environment, and historical studies.
Gold OA availability is greatest in general S&T (38% of the papers in Scopus) and lowest in
general social sciences and humanities (2%) and is also very low in engineering (3%). Other fields
with high availability in gold journals include biology as well as agriculture, fisheries & forestry(17%).
The fields where OA availability is most limited are within the social sciences and humanities andin the more applied sciences, engineering, and technology. The lowest prevalence of OA
availability is in visual and performing arts (13%) and communication & textual studies. It is alsoquite low (less than 30% availability) in engineering, built environment & design, enabling &strategic technologies, and chemistry.

August 2013
Produced for theEuropean Commission DG Research & Innovation

14

Proportion of OA papers—2004-2011

The statistics for the 2008–2011 period in Table IV were computed by grouping all the
publications for these four years, thus providing a good approximation of the overall trend.
However, because sample size remained stable at 40,000 publications per year, this does notreflect the annual variations in the number of papers indexed in Scopus. There is usually an
increase in scientific production at the world level in most fields. In Table V, data has been reweighted to reflect the annual rate of growth of papers in Scopus and in the various fields. By
and large, the proportion of OA remains unchanged except for only a few fields, where one ormore proportions changed upwards or downwards by one percentage point.
These weighted data show that the absolute number of papers in OA form is rising rapidly as theobserved increase in OA proportion noted in Table IV combined by the growth of the number ofpublications produces a strong overall increase. For example, the growth of the OA proportion inagriculture, fisheries & forestry was 1.13, but the number of papers grew at 1.49 (49% growth in
the number of OA papers indexed in Scopus in 2008–2011 compared to the 2004–2007 period).
Overall, out of the 5.8 million scientific papers from peer-reviewed journals indexed in Scopus
during the 2008–2011 period, 2.5 million were available for free in April 2013. A very large
number of papers are available free in clinical medicine, biomedical research, and physics andastronomy. This is partly because of the policy of the US National Institute of Health thatmandated the use of the PubMed Central repository for supported research and because of thearXiv e-print archive mirror, which has been largely adopted by researchers in the field of physics.

Table V

Number of papers indexed in Scopus available in OA, 2008-2011

Field
Agriculture, Fisheries & Forestry
Biology
Biomedical Research
Built Environment & Design
Chemistry
Clinical Medicine
Communication & Textual Studies
Earth & Environmental Sciences
Economics & Business
Enabling & Strategic Technologies
Engineering
General Arts, Humanities & Social Sciences**
General Science & Technology
Historical Studies
Information & Communication Technologies
Mathematics & Statistics
Philosophy & Theology
Physics & Astronomy
Psychology & Cognitive Sciences
Public Health & Health Services
Social Sciences
Visual & Performing Arts**
Total**

Notes:

SamplePapers*6,142
7,031
12,571
998
13,39942,806
1,108
4,700
3,950
12,714
11,264
8,220
2,760
1,590
5,620
4,318
732
15,028
3,027
4,688
5,170
6,572
160,000

ScopusPapers216,797
252,150
459,911
34,230481,575
1,564,310
42,863
172,130
150,081461,531404,900
8,220
101,464
57,584200,487
157,207
27,056552,749
113,841
173,475
191,674
6,572
5,830,804

Green & Hybrid%
Sco. Papers
58,363 27 ±2
98,461 39 ±2
232,867 51 ±2
8,147 24 ±5
94,527 20 ±1
565,524 36 ± 0.9
6,439 15 ±464,813 38 ±3
60,173 40 ±3
93,056 20 ±1
85,001 21 ±1
2,250 27
26,239 26 ±3
13,485 23 ±4
71,230 36 ±3
75,813 48 ±3
6,385 24 ±6
213,949 39 ±2
45,310 40 ±4
60,680 35 ±3
51,011 27 ±2609
91,930,853 33 ± 0.5

Gold%Sco. Papers
36,644 17 ±44,137 18 ±
49,745 11 ±1,110
3±
46,446 10 ±
171,539 11 ±2,518
6±
12,391
7±7,242
5±38,645
8±
9,714
2±
137
2
38,662 38 ±
4,481
8±
20,781 10 ±11,3557±1,426
5±25,925
5±6,585
6±21,893 13 ±18,1059±231
4570,947 10 ±

2
2
1
2
1
0.6
3
1
1
1.0
0.6
3
3
2
2
3
0.7
2
2
2
0.3

OA
OA Growth (papers)%
Sco. Papers
Trend
Index
95,007 44 ±2
1.49
142,598 57 ±2
1.39
282,613 61 ±2
1.19
9,257 27 ±6
1.26
140,973 29 ±2
1.69
737,063 47 ± 0.9
1.36
8,957 21 ±5
1.58
77,204 45 ±3
1.16
67,415 45 ±
3
1.50
131,701 29 ±2
1.60
94,716 23 ±2
1.38
2,387 29
1.54
64,901 64 ±4
2.21
17,966 31 ±5
1.34
92,011 46 ±3
1.26
87,168 55 ±3
1.23
7,811 29 ±7
1.52
239,874 43 ±
2
1.15
51,895 46 ±
4
1.40
82,573 48 ±3
1.59
69,116 36 ±3
1.46
840 131.51
2,501,800 43 ± 0.5
1.36

Source:* The number of sampled papers is for the whole four-year period (unweighted) whereas the number of
papers in Scopus, percentage of OA, and errors use yearly weighted data that reflect growth in Scopus.
**In order to decrease the margin of error for the fields of General Arts, Humanities & Social Sciences,and Visual & Performing Arts, the Scopus population of papers were used instead of a sample.
However, this does not affect the error margin for the total as this was based on sample data only.
Computed by Science-Metrix using DOAJ, PubMedCentral, and Scopus.

3.5

Citation advantage and disadvantage of OA papers

A question that has animated OA advocates has been the so-called OA citation advantage.
Evidence on this question is examined in Table VI using the Average of Relative Citation (ARC), a

August 2013
Produced for theEuropean Commission DG Research & Innovation

15

Proportion of OA papers—2004-2011

measure that reflects citation rates and is normalised to account for differences among scientificspecialities in the propensity to use references and receive citations. These data present therelative citation rate of OA publications overall, Gold OA and hybrid OA forms relative to all
publications in each field. All the fields have been set back to 1.0 to allow the calculation of acitation advantage/disadvantage, this baseline comprises all the papers in a field for the given
time period. A score above 1 denotes that papers are more cited than in the field overall, while ascore below 1 means that these publications are less frequently cited. For instance, papers inAgriculture, Fisheries & Forestry receive roughly the same level of citation (1.06) in OA overallthan they do usually.
All the fields derive an OA citation advantage (rebased ARC > 1). Paradoxically, many of the fields
where the OA proportion is low have a sizeable citation advantage such as in philosophy andtheology (1.54), general arts humanities and social sciences, communication and textual studies,engineering, and visual & performing arts.

Table VI

Rebased scientific impact (ARC) of OA publications, 2008-2011

Field
Agriculture, Fisheries & Forestry
Biology
Biomedical Research
Built Environment & Design
Chemistry
Clinical Medicine
Communication & Textual Studies
Earth & Environmental Sciences
Economics & Business
Enabling & Strategic Technologies
Engineering
General Arts, Humanities & Social Sciences
General Science & Technology
Historical Studies
Information & Communication Technologies
Mathematics & Statistics
Philosophy & Theology
Physics & Astronomy
Psychology & Cognitive Sciences
Public Health & Health Services
Social Sciences
Visual & Performing Arts
Total
Source:All Publications
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00

Green & Hybrid
1.38
1.41
1.25
1.33
1.38
1.56
1.66
1.30
1.32
1.43
1.55
1.53
2.57
1.54
1.37
1.22
1.56
1.36
1.37
1.36
1.52
1.93
1.41

Gold
0.52
0.48
0.76
n.c.
0.36
0.54
0.88
0.82
0.22
0.75
0.55
0.10
0.54
0.51
0.89
0.71
n.c.
1.01
0.69
0.72
0.55
0.11
0.60

OA
1.06
1.15
1.17
1.23
1.09
1.34
1.46
1.22
1.20
1.25
1.46
1.46
1.54
1.29
1.27
1.16
1.54
1.32
1.29
1.19
1.26
1.40
1.24

Computed by Science-Metrix using DOAJ, PubMedCentral, and Scopus.

What is particularly interesting here is that the citation advantage is derived almost exclusively
from the green and hybrid portion, as gold OA is associated with a citation disadvantage on
average for all fields except for physics & astronomy. In physics & astronomy, information &communication technologies, and in communication & textual studies, there are close to nodifferences between gold and regular journals in terms of average citation rates. Currently, there
is a marked disadvantage for publishing in gold journals in GAHSS, visual & performing arts,
economics & business and in chemistry. Interestingly, visual & performing arts has one of thehighest advantages derived from the use of green & hybrid OA, yet it is the field with the least
prevalent use of OA.

August 2013
Produced for theEuropean Commission DG Research & Innovation

16

Proportion of OA papers—2004-2011

These data reveal that a whole new area in the sociology of science may open up to explain
scientists’ publication behaviour in the various fields of knowledge production. There are
extremely marked communication differences which make inquisitive studies of the kind that
Derek de Solla Price initiated in the 1960s very worthwhile again. Considering there are a fairly
large number of studies on specific scientific fields currently, there are comparatively little
systemic studies of change in contemporary science. This is partly because the sociology andphilosophy of science have distanced themselves from, and now largely snob measurement
studies such as the present one. The development of OA provides a natural experiment that
could revive interest for inquiring into the sociology of science publication, should there be aninterest among scientists, science policy practitioners, and scientific journals editors.
The statistics on gold journals require careful interpretation. First, many gold journals areyounger and smaller, and these factors have an adverse effect on the citation rate and hence on
measured ARC values. Authors frequently prefer reading and citing more established journals,
and it is therefore a challenge to start a journal from scratch, and to have authors submit highquality articles. It takes time to build a reputation and to attract established authors. It is also
possible that gold journals might provide an avenue for less mainstream, more revolutionary
science. If so, the signature would be a much greater level of variation between the more highly
cited papers and the baseline (no citation). Also, the ARC is not scale-invariant, and larger
journals have an advantage as this measure is not corrected sufficiently for journal size (namely,
it is not a scale-independent measure). So it might not always be the gold nature of journals thatlowers their ‘citedness’; instead several structural aspects might be at play. Even so, the gold
journal industry is young, and it is still difficult to separate the wheat from the chaff. In thisrespect, it might be useful for authors to examine Beall’s List of ‘potential, possible, or probablepredatory scholarly open-access publishers’ to lower one’s risk of spending money on journals
that do not espouse scientific publishing best practices.10

3.6

Availability of OA papers in European and selected countries

An examination of OA availability was performed for EU28, EFTA (European Free Trade
Association), Accession countries, ERA (European Research Area), and four additional countries,
namely, Brazil, Canada, Japan, and the US.11 Please note that these data are not weighted toreflect growth in Scopus. If weighted data were used, some countries would see the proportion of
OA go up or down by one percentage point. Countries with a very small sample size (Latvia,
Luxembourg, Malta, Liechtenstein, and Macedonia) would show greater variations, but theresults presented here for these countries should be interpreted with great care in any case,because of the small sample size and ensuing large error margin.
Fractional counting is used in this table. In fractional counting, if two authors are from separate
countries, each country is given half a publication. In contrast, full paper counting would haveascribed one paper to each country. One advantage of fractional counting is that one can add thecalculated output of all countries’ output in a table and obtain a total of 100%.

10 http://scholarlyoa.com/publishers/.
11 See Annex I for statistics on OA taking into account yearly growth of the number of publications in the

Scopus database.

August 2013
Produced for theEuropean Commission DG Research & Innovation17

Proportion of OA papers—2004-2011

The EU28, EFTA, and ERA all have roughly the same proportion of OA articles as that observedat the world level (43% for 2008-2011, a figure which is not recalibrated for precision and recall),though there are noticeable differences among countries (Table VII). For the period 2008-2011
considered as a whole, eight of the EU28 (30%) have reached the tipping point, namely (bydecreasing percentage of OA): the Netherlands, Portugal, Lithuania, Estonia, Denmark, Malta,
Ireland, and Belgium. If the statistical precision and recall of the harvesting instrument are taken
into account, 20 out of 27 countries (74%) would have tipped towards a majority of papers
published in 2008-2011 being made available for free in April 2013.

Table VIIGroup

Proportion of OA per country, 4-year non-weighted sampling, 2008-2011

Country

EU28

Austria
Belgium
Bulgaria
Croatia
CyprusCzech Republic
Denmark
Estonia
FinlandFranceGermany
GreeceHungary
IrelandItaly
LatviaLithuania
Luxembourg
Malta
Netherlands
PolandPortugal
Romania
Slovakia
Slovenia
Spain
SwedenUnited Kingdom
Total EU28
EFTA
Iceland
Liechtenstein
NorwaySwitzerland
Total EFTA
Candidate Turkey
Macedonia
Total Candidate
IsraelTotal ERA
OthersUnited States
Japan
CanadaBrazilWorld

Source:Papers in
Sample
1,349
2,088
293
403
88
1,252
1,392
161
1,178
7,959
10,531
1,336
709
833
6,094
71
228
37
23
3,759
2,474
1,047
734
368
358
5,461
2,301
11,781
53,622
85
6
1,159
2,642
3,830
2,873
39
3,303
1,376
59,852
41,740
9,703
6,676
4,224
160,000

Green & Hybrid%
Papers545 40 ± 3
939 45 ± 2
91 31 ± 5
147 36 ± 535 40 ± 11
411 33 ± 3
633 45 ± 3
63 39 ± 8
488 41 ± 3
3,205 40 ± 1
4,026 38 ± 0.9
452 34 ± 3
279 39 ± 4
358 43 ± 32,389 39 ± 1
23 32 ± 11
88 39 ± 6
13 35 ± 17
8 35 ± 21
1,936 52 ± 2
707 29 ± 2
479 46 ± 3
289 39 ± 4
127 35 ± 5
110 31 ± 5
2,074 38 ± 1
922 40 ± 2
5,100 43 ± 0.9
20,204 37 ± 0.4
39 46 ± 11
1 17 ± 38
494 43 ± 3
1,214 46 ± 2
1,705 45 ± 2
657 23 ± 2
15 38 ± 16
672 23 ± 2
640 47 ± 3
22,085 37 ± 0.4
20,894 50 ± 0.5
3,264 34 ± 0.9
2,885 43 ± 1
876 21 ± 1
53,072 33 ± 0.2

Gold%Papers
105
8± 1
126
6± 1
21
7± 3
95 24 ± 433± 4
130 10 ± 2
103
7± 1
23 14 ± 6
80
7± 1
472
6 ± 0.5673
6 ± 0.5
116
9± 2
49
7± 2
75
9± 2
421
7 ± 0.6
8 11 ± 8
35 15 ± 525± 9
4 17 ± 17263
7 ± 0.8
326 13 ± 1
97
9± 2
75 10 ± 2
41 11 ± 3
50 14 ± 4
604 11 ± 0.8
181
8± 1
728
6 ± 0.4
4,192
8 ± 0.2
3
4± 4
110
208
319
598
11
608
92
5,009
2,535
804
411
1,799
15,5389± 2
8± 1
8 ± 0.921 ± 128 ± 1521 ± 17± 1
8 ± 0.2
6 ± 0.2
8 ± 0.5
6 ± 0.643 ± 1
10 ± 0.1

OA%
Papers650 48 ± 3
1,065 51 ± 2
112 38 ± 6
242 60 ± 5
38 43 ± 11
541 43 ± 3
736 53 ± 3
86 53 ± 8
568 48 ± 3
3,677 46 ± 1
4,699 45 ± 0.9
568 43 ± 3
328 46 ± 4
433 52 ± 3
2,810 46 ± 1
31 44 ± 12
123 54 ± 7
15 41 ± 17
12 52 ± 22
2,199 58 ± 2
1,033 42 ± 2
576 55 ± 3
364 50 ± 4
168 46 ± 5
160 45 ± 52,678 49 ± 1
1,103 48 ± 2
5,828 49 ± 0.9
24,396 45 ± 0.4
42 49 ± 11
1 17 ± 38
604 52 ± 3
1,422 54 ± 2
2,024 53 ± 2
1,255 44 ± 2
26 67 ± 16
1,280 44 ± 2
732 53 ± 3
27,094 45 ± 0.4
23,429 56 ± 0.5
4,068 42 ± 1
3,296 49 ± 1
2,675 63 ± 1
68,610 43 ± 0.2

Computed by Science-Metrix using DOAJ, PubMedCentral, and Scopus.

Leaving aside smaller countries with only few papers in the sample (less than 100 papers), thecountries with the lowest rate of OA adoption are Bulgaria, Poland, Greece, the Czech Republic,
and Germany. In EFTA countries, both Norway and Switzerland have clearly tipped towards
August 2013
Produced for theEuropean Commission DG Research & Innovation

18

Proportion of OA papers—2004-2011

having a majority of papers in OA, and by counting Israel, there are 14 ERA countries (40%)
which had a majority of their 2008-2011 papers being freely available by 2013. Calibrating for
retrieval precision and recall, the proportion of ERA countries having more than 50% of papers in
OA is 74%, that is, the same as for the EU28.
In countries outside the ERA, it is noteworthy that the US has passed the tipping point andCanada is approaching that point. Even more salient is the proportion of 63% observed in Brazil.
This is no doubt due to the important contribution of Scielo, which plays a key role in theSouthern hemisphere in making scientific knowledge more widely available.
Publishing in gold journals is much more frequently encountered in the eastern part of Europe.
There is a clear pattern that can be observed as the percentage of papers in gold journals isgreater in Macedonia, Croatia, Turkey, Malta, Lithuania, Estonia, Slovenia, Poland, Latvia,
Slovakia, Spain, the Czech Republic and Romania. One interesting hypothesis is that researchers
in these countries may use gold journals because they more frequently allow publishing inlanguages other than English. Should that be the case, this may also contribute to explaining thelower citation scores received by papers in gold journals as the readership for “vernacularlanguages”, as Eugene Garfield (1998) would put it, is lower and the size of the potentialreference pool is consequently also smaller. There is therefore a potentially fertile ground forstudying the social and linguistic aspects of science by examining where and why gold open
access journals are appearing and who actually makes use of them.

August 2013
Produced for theEuropean Commission DG Research & Innovation

19

Proportion of OA papers—2004-2011

4 Discussion
If one makes a straight reading of the data presented here, between 2004 and 2011, the averageannual rate of increase of OA availability was relatively limited, with a compound growth rate of
2% per year. Importantly though, examining the growth curve is misleading. Though the data
does not support this hypothesis, there seems to be an important translation movement in OA
availability. This means that in addition to having year-on-year growth, there is an upward
movement of the whole availability curve over time. This is due to an increasing number ofauthors making their manuscript available for the current year but also for previous years. The
same is true for journal publishers who may decide to opt for an OA model and make back files
freely available as well. Finally, when one looks at the availability curve, it can be seen that theavailability falls somewhat for the most recent few years. This is likely due to the presence ofembargo for some of the subscription based journals which make all articles freely available to all
after a few years—a practice which may be called “delayed OA”.
There are also transient effects that have to be considered when measuring OA availability. Many
articles that were available for free in December 2012 were no longer available for free in April
2013. This is in part due to a promotion Springer was running in late 2012 by making several
subscription-journal papers available for free, and later making them available for a fee again.
Some articles disappeared from websites, and some websites were not responding when visited in
April thus reducing further OA availability. Conversely, many new papers then became available
for free. All in all, more than 50% of the papers could be found for free in November/Decemberst2012 (pilot phase of this study) and in March/April 2013 (1 full measurement stage) but
somewhat less so at either time period. This shows that measuring phenomena on the Internet
requires particular attention to detail and constant questioning on the meaning of the results—
one has to ask whether these results are permanent or transient.
Other authors have presented results suggesting that OA availability was only half as high as
measured here, a reminder that it is important to be reflective and self-critical in all bibliometric
exercises. Whereas other authors have measured what appeared in specific databases, or what
specific search engines were able to do, the more important question is that of overall freeavailability of peer-reviewed, academic-level journal papers available. Measuring how well Google
Scholar fares at identifying a part of this literature is certainly an interesting exercise in itself, but
it does not inform on the state of affairs of the scientific publishing industry, and what
researchers who are reasonably tenacious will find for free.
Suggesting that the tipping point has been reached in open access certainly has important
implications for academia, for university librarians, and perhaps even more so for the scientific,
technical and medical publishing industry.
Much has been said about the cost of publishing in gold and hybrid OA being disadvantageous
relative to self-archiving, but one has to place this in perspective. The cost of academic papers in
the US is over $100,000—which is calculated by dividing the higher education expenditures on
R&D (HERD) by the number of papers published by academia. Adding to or included in thisamount, a $2,000 OA publication fee only accounts for a few percentage points of a typicalresearch project budget, especially in the natural and health sciences.
If the toll plaza moves from the end of the scientific publication process to the beginning, one
category of workers is likely to be highly affected: the university and research centre librarian.
Librarians have been highly affected already by the shift from paper to digital media, and losing
the financial responsibility of spending the large sums paid in journal subscriptions will certainly
August 2013
Produced for theEuropean Commission DG Research & Innovation20

Proportion of OA papers—2004-2011

create another large dent in their traditional sphere of responsibilities. If the toll plaza is moved
before the publication process and subscriptions are obsolete, it means that researchers will gain
control over financial matters associated with scientific publishing, and that librarians will lose
control over sizeable part of their budgets. The market power will shift tremendously from thetens of thousands of buyers that publisher sales staff have nurtured in the past to the millions ofresearchers that will now make the atomistic decision of how best to spend their publicationbudget.
The impression gained from carrying out this study and developing our OA harvester is that thetoll plaza is being moved to the beginning of the publishing process, away from the back-end of
the process, and thus away from librarians and closer to researchers. Despite what several authors
thought, and argued for, green OA only appears to be moving slowly, whereas gold and hybrid
OA (such as pay-per-article for OA release) appear to be driving in the fast lane. This impression
will require further verifications. If confirmed, it would mean that the industry is increasingly
placing the “toll before” as opposed to placing the “toll after”. Efforts need to be made tocharacterise these changes, and to distinguish what percentage of growth comes from green selfarchiving and what comes from other forms of hybrid OA.
Using a wider lens, the fact that the tipping point has likely been reached in open access is an
important finding for the whole publishing industry. This industry is likely to be undergoing
revolutionary change, and at a pace much faster than anticipated, in large part because previous
measures of OA availability proved to be misleading—having missed more than half of what wasavailable for a number of reasons. This means that aggressive OA publishers are likely to gainmuch in the redesigned landscape, whereas those attached to the old ways are more likely tosuffer and to lose market share. An important economic question is whether the switch to a more
atomistic, fine-grained market with millions of researchers as buyers will reduce, augment orleave unchanged the negotiating power of publishers.

August 2013
Produced for theEuropean Commission DG Research & Innovation21

Proportion of OA papers—2004-2011

References
Abad-García, M. F., Melero, R., Abadal, E., & González-Teruel, A. (2010). Self-archiving of
biomedical papers in open access repositories. Autoarchivo de artículos biomédicos enrepositorios de acceso abierto, 50(7), 431-440. doi: 10.1371/journal.pbio.0040157.
Antelman, K. (2004). Do open-access articles have a greater citation impact? College & Research
Libraries, 65(5), 372–382.
Arlitsch, K., & O'Brien, P. S. (2012). Invisible institutional repositories: Addressing the low
indexing ratios of IRs in Google Scholar. Library Hi Tech, 30(1), 60-81. doi:
10.1108/07378831211213210.
Bailey Jr, C. W. (2005). The role of reference librarians in institutional repositories. Reference
Services Review, 33(3), 259-267.
Bilder, G. (2003). Ingenta's economic and technical models for providing institutional OA
archives. Information Services and Use, 23(2-3), 111-112.
Björk, B. C., Welling, P., Laakso, M., Majlender, P., Hedlund, T., & Gudnason, G. (2010). Open
Access To The Scientific Journal Literature: Situation 2009. PLoS ONE, 5(6). doi:
10.1371/journal.pone.0011273.
Carbone, P. (2007). Consortium negotiations with publishers - Past and Future. LIBER Quarterly,
17(2).
Chan, D. L. H., Kwok, C. S. Y., & Yip, S. K. F. (2005). Changing roles of reference librarians: Thecase of the HKUST Institutional Repository. Reference Services Review, 33(3), 268-282. doi:
10.1108/00907320510611302
Chan, L., Devakos, R., & Mircea, G. (2005). Workshop 2: Implementing and filling institutional
repositories introduction, Leuven-Heverlee.
Charles, L., & Booth, H. A. (2011). An Overview of Open Access in the Fields of Business andManagement. Journal of Business and Finance Librarianship, 16(2), 108-124. doi:
10.1080/08963568.2011.554786
Craig, I. D., Plume, A. M., McVeigh, M. E., Pringle, J., & Amin, M. (2007). Do open access articles
have greater citation impact? A critical review of the literature. Journal of Informetrics, 1(3),239-248. doi: 10.1016/j.joi.2007.04.001.
Dubini, P., Galimberti, P., & Micheli, M. R. (2010). Authors publication strategies in scholarly
publishing. In ELPUB 2010 International Conference on Electronic Publishing, Helsinki(Iceland), 16-18 June 2010.
Garfield, E. (1998). The Use of Journal Impact Factors and Citations Analysis for Evaluation of
Science. Presented at Cell Separation, Hematology and Journal Citation Analysis Mini
Symposium in tribute to Arne Bøyum Rikshospitalet, Oslo, April 17, 1998. Retrieved
from http://www.garfield.library.upenn.edu/papers/eval_of_science_oslo.html.
Gargouri, Y., Larivière, V. Gingras, Y. and Harnad, S. (2012). Green and Gold Open Access
Percentages and Growth, by Field. In Archambault, É, Gingras, Y. and Larivière. V. (2012).
Proceedings of 17th International Conference on Science and Technology Indicators, Montréal:
Science-Metrix and OST.
Gentil-Beccot, A., Mele, S., & Brooks, T. C. (2010). Citing and reading behaviours in high-energy
physics. Scientometrics, 84(2), 345-355. doi: 10.1007/s11192-009-0111-1
August 2013
Produced for theEuropean Commission DG Research & Innovation

22

Proportion of OA papers—2004-2011

Harnad, S. (2003). The research-impact cycle. Information Services and Use, 23(2-3), 139-142.
Harnad, S. (2008). Waking OA's "slumbering giant": The university's mandate to mandate open
access. New Review of Information Networking, 14(1), 51-68. doi:
10.1080/13614570903001322.
Harnad, S. (2012). Open access: A green light for archiving. Nature, 487(7407), 302. doi:
10.1038/487302b
Harnad, S., & Brody, T. (2004). Comparing the impact of open access (OA) vs. non-OA articles inthe same journals. D-Lib Magazine, 10(6).
Henderson, I. (2013). Open-Access and Institutional Repositories in Fire Literature. Fire
Technology, 49(1), 155-161. doi: 10.1007/s10694-010-0198-1
Houghton, J. W. (2010). Economic implications of alternative publishing models: Self-archiving
and repositories. LIBER Quarterly, 19(3-4), 275-292.
Hu, C., Zhang, Y., & Chen, G. (2010). Exploring a New Model for Preprint Server: A Case Study ofCSPO. Journal of Academic Librarianship, 36(3), 257-262. doi: 10.1016/j.acalib.2010.03.010
Kurek, K., Geurts, P. A. Th M., & Roosendaal, H. E. (2006). The split between availability andselection: Business models for scientific information, and the scientific process?
Information Services and Use, 26(4), 271-282.
Lakshmi Poorna, R., Mymoon, M., & Hariharan, A. (2012). A study of select open access journals
and their business models listed in DOAJ in the fields of civil and structural engineering.
Journal of Structural Engineering (India), 39(4), 458-468. doi: 10.1371/
journal.pone.0020961.
Larivière, V., Archambault, É., Gingras, Y. & Vignola-Gagné, É. (2006). The Place of Serials in
Referencing Practices: Comparing Natural Sciences and Engineering with Social Sciencesand Humanities, Journal of the American Society for Information Science and Technology, 57(8),997-1004. doi: 10.1002/asi.20349.
Miguel, S., Bongiovani, P. C., Gómez, N. D., & Bueno-de-la-Fuente, G. (2013). Prospect forDevelopment of Open Access in Argentina. Journal of Academic Librarianship, 39(1), 1-2.
doi: 10.1016/j.acalib.2012.10.002
Morris, S. (2003). Open Publishing: How publishers are reacting. Information Services and Use, 23(23), 99-101.
Nyambi, E., & Maynard, S. (2012). An investigation of institutional repositories in state
universities in Zimbabwe. Information Development, 28(1), 55-67. doi:
10.1177/0266666911425264
Pelizzari, E. (2004). Academic authors and open archives: A survey in the social science field. Libri,
54(2), 113-122.
Prosser, D. (2003). Institutional repositories and Open access: The future of scholarly
communication. Information Services and Use, 23(2-3), 167-170.
Regazzi, John. (2004). The Shifting Sands of Open Access Publishing, a Publisher's View. Serials
Review, 30(4), 275-280. doi: http://dx.doi.org/10.1016/j.serrev.2004.09.010
Repanovici, A. (2012). Professional profile of digital repository manager. Library Hi Tech News,
29(10), 13-20. doi: 10.1108/07419051211294473

August 2013
Produced for theEuropean Commission DG Research & Innovation

23

Proportion of OA papers—2004-2011

Rowland, F. et al. (2004). Delivery, management and access model for e-prints and open accessjournals. Serials Review, 30(4), 298-303. doi: 10.1016/j.serrev.2004.09.006
Sawant, S. (2012). Past and Present Scenario of Open Access Movement in India. Journal of
Academic Librarianship. doi: 10.1016/j.acalib.2012.11.007
Swan, A. et al. (2005). Developing a model for e-prints and open access journal content in UK
further and higher education. Learned Publishing, 18(1), 25-40. doi:
10.1087/0953151052801479
Swan, A., & Brown, S. (2004). Authors and open access publishing. Learned Publishing, 17(3), 219224. doi: 10.1087/095315104323159649
Woutersen-Windhouwer, S. (2012). The Future of Open Access Publishing in the Netherlands:
Constant Dripping Wears Away the Stone. Journal of Academic Librarianship. doi:
10.1016/j.acalib.2012.11.015

August 2013
Produced for theEuropean Commission DG Research & Innovation

24