# Galvanising the Open Access Community: A Study on the Impact of Plan S

Pablo de Castro, Ulrich Herb, Laura Rothfritz, W. Benedikt Schmal, Joachim Schöpfel

► To cite this version:

Pablo de Castro, Ulrich Herb, Laura Rothfritz, W. Benedikt Schmal, Joachim Schöpfel. Galvanising the Open Access Community: A Study on the Impact of Plan S. cOAlition S. 2024. hal-04737864

HAL Id: hal-04737864

https://lilloa.hal.science/hal-04737864v1

Submitted on 15 Oct 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
GALVANISING THE OPEN
ACCESS COMMUNITY

A STUDY ON THE IMPACT OF PLAN S

Pablo de Castro ID, Ulrich Herb ID, Laura Rothfritz ID,
W Benedikt Schmal ID, Joachim Schöpfel ID

SCIDECODE SCIENCE CONSULTING

15 October 2024

<figure><img src="image_2.png" /></figure>

scidecode science consulting

De Castro, Herb, Rothfritz GbR
# A STUDY ON THE IMPACT OF PLAN S

Report commissioned by:

cOAlition S

<figure><img src="image_3.png" /></figure>

Funded by the European Union

Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or cOAlition S. Neither the European Union nor the granting authority can be held responsible for them.

Contact:

Pablo de Castro

scidecode science consulting
info@scidecode.com

Report dated: 15 October 2024

<figure><img src="image_4.png" /></figure>

This work is licensed under a Creative Commons CC BY 4.0 Underpinning dataset licensed under CC 0

## Photo credits

Figure 2 Default rights retention clause kindly provided by Jisc

Available on Zenodo at:

https://doi.org/10.5281/zenodo.13738479
# CONTENTS

## EXECUTIVE SUMMARY ................................................................................................................ 5

INTRODUCTION .................................................................................................................................................. 5

STUDY RESULTS .................................................................................................................................................. 6

WHAT PLAN S GOT RIGHT (AND NOT SO RIGHT) .................................................................................. 7

RECOMMENDATIONS........................................................................................................................................ 8

Recommendations for cOAlition S .................................................................................................................. 8

Recommendations for other stakeholders ........................................................................................................ 8

## 1. METHODOLOGY.................................................................................................................. 9

1.1. QUANTITATIVE ANALYSIS – COUNTERFACTUAL IMPACT EVALUATION OF PLAN S .................................................. 9

1.1.1. The open data requirement .................................................................................................................. 9

1.1.2. Quantitative analysis: data collection ........................................................................................................ 10

1.2. QUALITATIVE ANALYSIS ........................................................................................................................................ 10

## 2. INTRODUCTION.................................................................................................................. 12

2.1. A BIT OF HISTORY: THE RUN-UP TO PLAN S .................................................................................. 12

2.2. PLAN S, ITS PRINCIPLES AND ITS POLICIES (2018-2023) ........................................................................ 14

2.3. THE ROLE OF FUNDERS WITHIN THE SCHOLARLY COMMUNICATIONS LANDSCAPE........................................ 16

2.4. THE IMPACT OF PLAN S ON A SPECIFIC NATIONAL OPEN ACCESS FRAMEWORK ........................................ 19

## 3. COUNTERFACTUAL IMPACT EVALUATION OF PLAN S .......................................................... 21

3.1. CIE OVERVIEW .................................................................................................................................................. 21

3.1.1. The difference-in-differences method ........................................................................................................ 22

3.1.2. Establishing the timeframe for the analysis........................................................................................................ 22

3.1.3. Technical assumptions........................................................................................................................................ 23

3.1.4. Treatment, control and comparison groups........................................................................................................ 23

3.1.5. Rationale for funder selection into the treatment and control groups................................................................ 25

3.2. LIMITATIONS OF THE CIE STUDY.................................................................................................................. 26

3.3. RESEARCH DESIGN.................................................................................................................................................. 28

3.3.1. Data collection and grouping .................................................................................................................. 28

3.3.2. Data preprocessing........................................................................................................................................ 30

3.4. CIE RESULTS .................................................................................................................................................. 32

3.4.1. Pooled Comparisons........................................................................................................................................ 32

3.4.2. Pairwise Comparisons ........................................................................................................................................ 34

3.4.3. Changes across Open Access models........................................................................................................ 38

3.5. CONCLUSIONS .................................................................................................................................................. 39
<table><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>4. QUALITATIVE ANALYSIS: SOME PERSPECTIVES</td><td>42</td></tr><tr><td>4.1. TRANSFORMATIVE AGREEMENTS</td><td>42</td></tr><tr><td>4.2. RIGHTS RETENTION STRATEGY</td><td>47</td></tr><tr><td>4.3. FULLY GOLD OPEN ACCESS</td><td>51</td></tr><tr><td>4.3.1. APC-based Gold Open Access</td><td>51</td></tr><tr><td>4.3.2. Diamond Open Access</td><td>53</td></tr><tr><td>4.4. PARALLEL AREAS OF PRACTICE: PREPRINTS. LONG-FORM PUBLICATIONS</td><td>58</td></tr><tr><td>4.4.1. Preprints</td><td>58</td></tr><tr><td>4.4.2. Long-form publications</td><td>59</td></tr><tr><td>4.5. THE IMPACT OF PLAN S BEYOND ITS STRICT AREA OF INFLUENCE: OPEN ACCESS AND THE GLOBAL SOUTH</td><td>63</td></tr><tr><td>4.6. A MODEST PROPOSAL FOR RESEARCHERS</td><td>67</td></tr><tr><td>4.7. PUBLISHERS AND PLAN S</td><td>70</td></tr><tr><td>4.7.1. APC-based Gold OA and fully Open Access publishers</td><td>71</td></tr><tr><td>4.7.2. Hybrid Open Access publishers</td><td>72</td></tr><tr><td>4.7.3. Publishers and rights retention strategies</td><td>74</td></tr><tr><td>4.7.4. A word on book publishers</td><td>74</td></tr><tr><td>5. WHAT PLAN S GOT RIGHT – AND NOT SO RIGHT</td><td>76</td></tr><tr><td>5.1. WHAT PLAN S AND COALITION S GOT RIGHT</td><td>76</td></tr><tr><td>5.2. WHAT PLAN S AND COALITION S DID NOT GET SO RIGHT</td><td>77</td></tr><tr><td>6. RECOMMENDATIONS</td><td>79</td></tr><tr><td>7. ANNEXES</td><td>83</td></tr><tr><td>ANNEX 1: OPEN BIBLIOGRAPHIC DATA SOURCES</td><td>83</td></tr><tr><td>ANNEX 2: LIST OF INTERVIEWEES AND OTHER ACKNOWLEDGEMENTS</td><td>85</td></tr><tr><td>ANNEX 3: PLAN S OPEN ACCESS POLICY COMPLIANCE</td><td>87</td></tr><tr><td>ANNEX 4: GLOSSARY</td><td>91</td></tr></tbody></table>
# EXECUTIVE SUMMARY

## Introduction

Plan S is an initiative for Open Access publishing that was launched in September 2018. The plan is supported by cOAlition S, an international consortium of research funding and performing organisations.

Implemented in 2021 by a number of cOAlition S members, Plan S encompasses **three different routes** to achieve full and immediate Open Access: **fully Open Access journals** (either involving the payment of Open Access publishing fees or otherwise), **transformative arrangements** (covering transformative agreements and journals) and **rights retention policies** for immediate Green Open Access. cOAlition S have stated no specific preference for any of these routes, but did instead emphasise their wish not to limit the choice of publishing venues by researchers, and the key role the **10 principles of Plan S** played in the design of these routes.

Some 5 years after the publication of Plan S, cOAlition S (via the European Science Foundation) contracted scidecode science consulting to undertake a study to assess the impact Plan S has had on the global scholarly communication ecosystem and on facilitating research to be published Open Access. The present report is the outcome of that study.

## Aims of the study

The Invitation to Tender published by cOAlition S specified that the study must address the following questions:

1. Assess the impact Plan S has had on the scholarly communication ecosystem and facilitating research to be published Open Access. This should be considered from both a global perspective – including regions where cOAlition S has no or little footprint – and from the perspective of policy compliance amongst researchers funded by cOAlition S members.

2. Provide an analysis of the likely progress towards full and immediate Open Access had Plan S (and cOAlition S) NOT existed and estimate what contribution Plan S (and cOAlition S) has made to the changing scholarly communication ecosystem, using appropriate techniques, such as counterfactual impact evaluation and/or contribution analysis, or similar.

3. Review the effect of transformative arrangements, as well as the option of providing immediate Open Access to subscription content via open repositories, on achieving a transition to full and immediate Open Access.

4. Identify those areas where cOAlition S has been effective and ineffective in meeting its objectives, based on published literature, interviews with relevant stakeholders, and other approaches as deemed appropriate, and provide a reasoned analysis of these findings.

5. Looking to the future, provide recommendations on how to build on the achievements of cOAlition S.

Plan S has been the subject of much criticism since it was first announced. Most critics highlighted the perceived dangers of its **playing into the hands of commercial publishers** by promoting the so-called transformative arrangements and APCs. On the other hand, it has also been considered as a bold policy that **helped move the OA process ahead** after years of stagnation. This report looks at the impact that Plan S has had to inform this debate.
# Study results

The analysis of the impact of a relatively recent policy intervention like Plan S is best achieved via a **mix of a quantitative and a qualitative assessment**.

This study has conducted a quantitative analysis in the form of a **counterfactual impact evaluation (CIE)** aimed at identifying how the scholarly communications landscape might have evolved if Plan S had never existed. The methodology involved a comparison between 'treatment group funders', (i.e. cOAlition S funders) and 'control group funders' (i.e. those funders whose OA policies are not aligned with Plan S).

The results of this CIE, shown in chapter 3, are mixed. While most treatment group funders **have seen sizeable increases in the rates of Open Access for their funded publications**, the 'pooled comparison' (i.e. group vs group instead of individual funder vs individual funder) shows that cOAlition S funders **haven't seen a statistically significant growth when compared** to the baseline control group funders.

We believe that this is due to two key reasons. First, it is **too early** to assess the impact of Plan S-aligned Open Access policies **on a quantitative basis alone** when these only started to be adopted early in 2021, often for funding calls issued from 1 January 2021 onwards (annex 3 provides a detailed analysis of the implications of these late and diverse kick-off dates). Second, control group funders **started from a much lower position** in terms of Open Access rates. This makes them likely to experience a proportionally quicker rate of growth than those funders with already very high Open Access rates.

There is however an Open Access model where the results show a **marked increase** among cOAlition S funders, and this is **Hybrid Open Access**. This outcome is not surprising given the ever-larger number of transformative agreements that consortia in many countries (including those represented within cOAlition S) are negotiating with a wide range of publishers.

The **qualitative analysis** of Plan S is then a **key complement** to the CIE findings.

A concise assessment of the impact of Plan S from the findings of this qualitative analysis is provided in the title of the report: **Plan S has galvanised the Open Access community** and opened new avenues for achieving full and immediate Open Access.

While it's hard to pin down a specific achievement that can be exclusively attributed to Plan S, the report is full of references for ongoing initiatives for which Plan S has been a source of inspiration, even a driver.

This qualitative analysis is based on the views of interviewees from all relevant stakeholders in the domain – institutional Open Access advocates, librarians, researchers, consortia, funders and publishers – supplemented with desk research. The list of formal interviewees is provided in annex 2, together with a number of funders who agreed to engage in more informal conversations and a list of events where valuable discussions were held.

Those who participated in formal interviews have generally **praised Plan S**. Institutional representatives value the way cOAlition S have managed to raise the profile of **Open Access**, especially amongst the University management, which in turn has led to the negotiating teams becoming far more diverse and influential. The **inclusion of top management figures at institutions** in the public-sector teams in charge of the negotiations with publishers and the increasing ability of institutional Open Access support teams **to suggest researchers where to publish** have been some of the benefits of the funders' push for immediate Open Access highlighted by interviewees.

Library consortia representatives were also appreciative of the way Plan S **has drawn publishers to the negotiating table**. This group were also supportive of other aspects of Plan S – most notably rights retention – and are working to include these elements into evolved versions of Read & Publish agreements.
Publishers generally value the **harmonisation in funder policies** introduced by Plan S, although there are mixed feelings on the duration of the transition towards fully Open Access business models (section 4.7 in the report is specifically devoted to publishers and Plan S).

The qualitative analysis of the impact of Plan S also looks at the various policy instruments in the Plan S toolbox. Of note is the **potentially game-changing effect of rights retention strategies**, and the fact that institutions have built on the Plan S rights retention policy and expanded it into their Institutional Rights Retention Policies.

**Transformative agreements (TAs) are perceived to be valuable** – for different reasons – for both institutions and publishers, but they also have **numerous downsides**. cOAlition S have announced they will cease to support them beyond 2024, but these instruments are **unlikely to go away** in the short and medium term. The report suggests defining clear red lines for TAs and **not hesitating to drop them** when these are overstepped.

The contribution of cOAlition S to the current momentum around **Diamond Open Access** is another positive outcome of Plan S whilst a significant – if not universal – **transition in research publishing business models** is also starting to occur that very much aligns with the spirit and principles of Plan S.

This transition also encompasses domains such as **preprints** and **long-form publications** that are either not part of Plan S (in the case of the former) or are only starting to be jointly addressed by cOAlition S funders. A specific chapter in the report is devoted to these 'parallel developments' in which Plan S is also having an impact.

Finally, it is worth noting that although cOAlition S is perceived to have promoted the Gold Open Access publishing model, it has also helped to raise awareness of the inequities of publishing models that rely on article-based charges. This is evident through its support for a **pricing framework** designed to foster global equity in scholarly publishing and in the establishment of a “Beyond Article-based charges” working group.

## What Plan S got right (and not so right)

As outlined above, one of the key tasks of the study was to identify areas where cOAlition S has been effective and ineffective in meeting its objectives.

On the positive side, we conclude that cOAlition S was effective in identifying the right levers to pull, most notably by working as a consortium, by trying to work with publishers and by adopting a highly flexible approach – as evidenced by the multiple strategies for delivering full and immediate OA.

Moreover, the attempts to address issues beyond the geographic remit of cOAlition S – especially those that impact the Global South – and to adopt a listening approach (and being prepared to tweak Plan S) are also seen as effective strategies.

Turning to where cOAlition S has been less effective, a number of areas were identified. These include the fact that researchers are largely unaware of this initiative and a sense that Plan S is too top-down and too abstract: the link between Plan S policies and their national-level context remains tenuous.

We also conclude that cOAlition S spread itself too thinly, as seen by the ambition to simultaneously solve all three challenges associated with OA (accessibility, affordability and equity). Finally, we note the failure to secure support from additional funders, especially from Asia and Latin America/Caribbean. This inevitably impacts the credibility of the discourse around equitable publishing.
# Recommendations

In total we make 15 recommendations; eleven addressed at cOAlition S and four focused on other stakeholders, summarised below.

## Recommendations for cOAlition S

1. Keep cOAlition S running beyond 2025.

2. Support initiatives to reform research assessment

3. Support innovative and equitable publishing models and venues

4. Establish a narrative that connects the pre-and post-31 Dec 2024 stages of Plan S.

5. Explore the feasibility of expanding the reach of cOAlition S into new geographical areas

6. Keep working as closely as possible with national and regional consortia

7. Step up the communications-related effort of cOAlition S

8. Collaborate with Open Access monitors

9. Promote international collaboration around rights retention policies

10. Introduce a 'responsible publishing' section in the project reporting requirements

11. Re-run the study on the impact of Plan S in 5 or 10 years

## Recommendations for other stakeholders

12. Identify and disseminate best practices in reallocating research literature budgets by libraries

13. Promote a more distributed Open Access support network at institutions that involves researcher representatives

14. Devise mechanisms to increase the value of transformative agreements.

15. Devise and implement mechanisms to secure ownership of new publicly-owned or -supported ventures in the scholarly communications landscape

Further details regarding these recommendations can be found in chapter 6.

In conclusion, we believe that whilst funders may well be the most powerful actors within the multi-stakeholder scholarly communications environment in terms of exerting change, any meaningful transition towards more sustainable, affordable and equitable publishing **will require the collaboration of all actors** involved in this ecosystem.

Terms in blue-color italics on first occurrence in each section below are included in the glossary in annex 4 for a detailed explanation.
The assessment of the impact of Plan S presented in this report has been conducted on the basis of a quantitative analysis - following a counterfactual impact evaluation approach - and a complementary qualitative analysis. The methodologies applied for both exercises are explained in this section.

## 1.1. Quantitative analysis – Counterfactual Impact Evaluation of Plan S

### 1.1.1. The open data requirement

The cOAlition S Invitation to Tender (ITT) published in July 2023 stated the requirement that all data underpinning the quantitative analysis should be made openly available under a CC BY licence [1]. This requirement led the scidecode team to choose an approach based on open bibliographic databases. These include the likes of OpenAlex, Unpaywall and Crossref among other openly available data sources. Other options such as Scopus or the Web of Science were also considered but were quickly discarded in view of the negotiations that would be required and the potential costs for making the datasets openly available.

The project to assess the impact of *Plan S* overlapped with the publication of the Barcelona Declaration on Open Research Information (DORI) [2] and with a string of announcements from organisations indicating that they were switching from closed to open data sources. Notably, these included the Sorbonne University in Paris [3] and the release of the Open Edition of the CWTS Leiden Ranking [4], a section of which measures the uptake of Open Access publications at institutions worldwide.

This move towards more transparency in the research information domain is a welcome development and is expected to lead to a quick improvement in the quality of data sources. Several funders interviewed for this study have for instance expressed their intention to directly share their data with open bibliographic databases. However, basing a quantitative analysis exclusively on data sources that may still see significant increases in quality poses a certain risk<fnref n="1" />. Section 3.2. devoted to the limitations of the study within the chapter on the *Counterfactual Impact Evaluation (CIE)* of Plan S provides more detail on how these potential risks have been addressed.

As a means to test the quality of these open bibliographic data sources, a preliminary analysis was conducted before any data were collected for the CIE. This preliminary test involved the comparison of datasets for funded publications by a specific research funder taken from open and commercial data sources. The research funder chosen for this test was the Fundação Oswaldo Cruz (Fiocruz) in Brazil and samples of Fiocruz-funded publications in the period 2021-2022 were taken from different sources and compared to each other.

<footnote n="1">¹ The team behind the CWTS Leiden ranking have stated their intention to keep running the previous ranking based on WoS data alongside their new, “still of experimental nature” Open Edition of the ranking for some time [4].</footnote>
The process and its results are thoroughly documented in the scidecode blogpost "Choosing a data sample provider for our study on the impact of Plan S" [5]. The results show that the quality of the data on a sample deliberately chosen for its 'difficulty' (not a cOAlition S funder, not in the Global North and not having English as its 'native' language) was sufficiently satisfactory to validate the approach taken. A post was subsequently released by cOAlition S stating their support for the move to open bibliographic data sources [6].

### 1.1.2. Quantitative analysis: data collection

The data collection process for the CIE involved the selection of a number of funders for each of the three groups that would be analysed, namely the *treatment group*, the *control group* and the comparison group. These groups are thoroughly discussed in chapter 3.

Sets of funded publications were then obtained for each of these funders for the period 2015-2023. The actual data was provided by the external partner OA.Works<fnref n="2" />, leveraging only open bibliographic data sources. The data was pre-processed and cleaned, broken into two subsets for pre- and post-Plan S<fnref n="3" /> time-windows (2015-2020 and 2021-2023 respectively) and fed into the *difference-in-differences* routine that allowed the CIE to be conducted. The detailed methodology for this work is provided in chapter 3.

## 1.2. Qualitative analysis

Cognisant of the fact that it is too early to fully assess the impact of Plan S on a *quantitative* basis alone, the study also undertook a qualitative analysis, via the six routes outlined below:

* A literature review was conducted to collect bibliographic references to Plan S.

* A significant amount of desk research has been conducted including the analysis of the references identified as part of the literature review. This has also involved the participation in a number of Open Access activities and informal conversations with professionals in the domain.

* A wide range of Open Access experts from all relevant stakeholders in the scholarly communications domain were approached and interviewed on the topics addressed in this report. These 1-hr interviews were conducted over Zoom between October 2023 and May 2024 and subsequently analysed. The list of 35 interviewees and their affiliations is available in [annex 2](#).

* A number of informal conversations were held with research funders to gather more information on how their adoption of Plan S-aligned policies had influenced their workflows around Open Access implementation and monitoring. Annex 2 also includes a list of the funders with whom these conversations were held.

* Members of the scidecode team took part in a number of events specifically addressing various aspects of the ongoing evolution in the Open Access landscape. This were held by various organisations and provided an opportunity to have discussions on topics like the financial implications of the OA transition or the multiple Open Access monitoring initiatives currently underway. A list of these events is also provided in annex 2.

<footnote n="2">² OA.Works, https://oa.works/</footnote>

<footnote n="3">³ Pre- and post-Plan S is shorthand for before and after the adoption of Plan S-aligned Open Access policies by cOAlition S funders at the start of 2021</footnote>
* An early presentation of the findings of the report was delivered during the cOAlition S Expert Group meeting held June 11-12, 2024. This presentation was followed by a breakout session during which some of the findings were discussed with the cOAlition S experts in the room and online.

All data underpinning the quantitative analysis of the impact of Plan S is openly available under a CC BY licence in Zenodo at https://doi.org/10.5281/zenodo.12523229. This is a companion record to the one that contains the report. Additionally, a list of the data and code files than have been used for the CIE is provided in annex 1.

## REFERENCES

[1] cOAlition S (2019). "A study to assess the impact of Plan S on the global scholarly communication ecosystem: an Invitation to Tender (ITT)". https://www.coalition-s.org/a-study-to-assess-the-impact-of-plan-s-on-the-global-scholarly-communication-ecosystem-invitation-to-tender/

[2] Barcelona Declaration on Open Research Information (2024). https://barcelona-declaration.org/

[3] Andrew Jack (2023). Sorbonne's embrace of free research platform shakes up academic publishing. Financial Times. https://www.ft.com/content/89098b25-78af-4539-ba24-c770cf9ec7c3

[4] Ludo Waltman at el (2024). Introducing the Leiden Ranking Open Edition. Leiden Madtrics. https://www.leidenmadtrics.nl/articles/introducing-the-leiden-ranking-open-edition

[5] Pablo de Castro (2024). Choosing a data sample provider for our study on the impact of Plan S. scidecode blog. https://scidecode.com/2024/01/22/choosing-a-data-sample-provider-for-our-study-on-the-impact-of-plan-s/

[6] Nora Papp-Le Roy (2024). Are open bibliometric data sources better than proprietary ones? cOAlition S blog. https://www.coalition-s.org/blog/are-open-bibliometric-data-sources-better-than-proprietary-ones/
### 2.1. A BIT OF HISTORY: THE RUN-UP TO PLAN S

A number of milestones have paved the way that led to the emergence of *Plan S*. One of the earliest ones was the release of the so-called *Finch report* in 2012. This report included the recommendations of the Working Group on Expanding Access to Published Research Findings (also known as the Finch Committee) convened in 2011 and chaired by Dame Janet Finch in the United Kingdom. The report is credited with having opened the door to paid-for *Gold Open Access* to the detriment of the until then prevalent *Green Open Access*. Widespread Open Access publishing fee (or *Article Processing Charge*, APC) payments for articles in *hybrid journals* raised the issue of *double-dipping*, which in turn led to the emergence of offsetting agreements with publishers whereby these would compensate institutions for the double payments they were making.

The institutional block grant system was introduced in response to the Finch report by funders in the UK. Both the Research Councils UK, later UK Research and Innovation (UKRI), and the Wellcome Trust (both members of cOAlition S) introduced these mechanisms for supporting the Gold Open Access publishing costs via funding transfers to institutions. These would report back annually to the funders on the expenditure and the funded publications that had been supported. These still active funder block grant mechanisms also provided the means to cover the costs for the *Read & Publish agreements* when these started emerging in 2015. There had been previous attempts to couple the read and the publish costs in the UK, but the Springer Compact was the first proper Read & Publish agreement, at the time expected (by the publisher) to "have fixed the issue of Open Access once and for all".

In parallel to this, the OA2020 initiative was launched at the 12th Berlin Open Access conference in 2015. This was a plan to flip the whole research publishing landscape from a subscription to a fully Open Access model. Its proponents argued there was already enough funding in the system for this transition to happen at no additional cost [1].

The move towards full immediate Open Access was accelerating. The Amsterdam Call for Action on Open Science was issued in 2016 under Dutch Presidency of the European Council. Jisc released its Principles for Offset Agreements also in 2016. The European Commission ran its 2015-2016 Gold Open Access Pilot to cover APCs for FP7 post-grant publications. Various countries started joining the Springer Compact at a national level via their consortia and the DFG-funded ESAC project saw the opportunity to establish a degree of international coordination [2] where national consortia could exchange information and learn how other countries were implementing the same agreement.

The OA2020 initiative and the Max Planck Digital Library (MPDL) continued to support the Berlin Open Access Conferences where the successive anniversaries of the 2003 Berlin Declaration were celebrated. This created momentum for a worldwide collaboration across countries and national consortia to explore the strategy and the mechanisms to flip the whole landscape to immediate Open Access. The CERN-led SCOAP3 initiative provided a blueprint for the plans for a large-scale Open Access transition. SCOAP3 had already figured out a way to flip a number of very relevant journals in the discipline of High Energy Physics (HEP) via a worldwide consortium that is still operational – and growing. SCOAP3 hasn't been replicated for any other set of disciplines, partly due to the fiendishly complex nature of the agreement [3]. However, the discussions and the international collaboration around the OA2020 initiative led to a gradual, uneven evolution in the Open Access landscape that was happening before Plan S was released.
A more detailed account of the run-up to Plan S – including the emergence of coAlition S as an alliance of funders – can be found in the literature [4]. This short summary simply highlights how the release of Plan S conceptually followed in the steps of previous initiatives.

The reason why this recent Open Access history is relevant in a study examining the impact of Plan S is that it's extremely difficult for any counterfactual impact evaluation of Plan S to filter out this background effect of a transition that was already happening. The authors of this study are well aware that it may not be possible to completely filter out this background effect and this is why the study contains an equally important complementary qualitative analysis on top of the quantitative one.

## REFERENCES

[1] Schimmer, R., Geschuhn, K. K., & Vogler, A. (2015). Disrupting the subscription journals' business model for the necessary large-scale transformation to open access. https://doi.org/10.17617/1.3

[2] Kai Geschuhn, Graham Stone (2017). "It's the workflows, stupid! What is required to make 'offsetting' work for the open access transition". UKSG Insights 30(3): 103-114. https://doi.org/10.1629/uksg.391
[Part of the special issue/UKSG Insights collection "From Finch to Plan S: and you may ask yourself, well how did I get here?" launched in Mar 2019 *after* the release of Plan S]

[3] Richard Van Noorden (2012). "Open-access deal for particle physics". Nature News 489, 486.
https://doi.org/10.1038/489486a

[4] Robert-Jan Smits, Rachael Pells (2022). Plan S for Shock: Science. Shock. Solution. Speed. London: Ubiquity Press.
https://doi.org/10.5334/bcq
## 2.2. PLAN S, ITS PRINCIPLES AND ITS POLICIES (2018-2023)

Plan S as announced in September 2018 is a strategy to achieve full, immediate Open Access backed by a consortium of research funders and performing organisations called cOAlition S. Frontrunner organisations within cOAlition S implemented Plan S-aligned Open Access policies of their own in 2021. Plan S is based on 10 principles for scholarly publishing [1] and encompasses a combination of complementary strategies, a number of which decidedly support the regaining of control on scholarly communications by Academia. At its launch, Plan S allowed three routes to compliance for authors funded by the 12 signatory organisations that originally formed cOAlition S:

* Fully Open Access journals or platforms with or without publishing fees – so-called Gold or Diamond Open Access

* Subscription venues, by depositing the full-text accepted manuscript (AAM) in an open access repository – the so-called Green Open Access or repository route, supported by the Rights Retention Strategy

* Transformative agreements and journals – as a means to support Hybrid Open Access in a way that promotes a transition towards fully Open Access business models. cOAlition S has announced that these instruments will no longer be financially supported from 2025 [2].

One key objective in the design of these complementary routes was not to restrict the ability of researchers to submit their manuscripts to their publishing venue of choice. Instruments like the Journal Checker Tool (JCT) were promptly developed after the release of Plan S [3]. This online tool allows authors to easily tell the routes available to them to comply with the Plan S principles for a specific combination of a publishing venue, funding agency and institutional affiliation.

Research funders have for a long time played a key role in the implementation of Open Access, as they sit at the crossroads of researchers, research-performing organisations and publishers. The funders' ability to include Open Access and Open Science requirements in the grants they award their funded researchers makes them ideally placed to foster an Open Access culture. Funders like the European Commission (EC) or the then Higher Education Funding Council for England (HEFCE), now Research England, were critical in the gradual embedding of Open Access and Open Science policies into their grant agreements.

The EC Open Access policy provided a blueprint for national Open Access policies in Member States and beyond, as evidenced by the PASTEUR4OA project work [4]. For its part, HEFCE played a pivotal role in coupling research assessment to an affordable Open Access implementation via the deposit of accepted manuscripts as part of the requirements for the Research Excellence Framework (REF) national-level assessment exercise in the United Kingdom. This coupling of Open Access policies and research assessment by HEFCE was one of the first instances for a mechanism that is currently being conceptually revisited by more recent initiatives to reform research assessment. This policy intervention at a funder level<fnref n="4" /> led universities to implement strategies to collect all accepted manuscript produced at the institutions, a practice that resulted in percentages of Open Access above 90% for a good number of them [5].

<footnote n="4">⁴ The REF2014 was followed by the REF2021 Open Access policy, which essentially maintained the same guidance to require manuscript deposit as soon as possible upon acceptance. At the time of writing, the REF2029 OA policy is undergoing public consultation and could see some changes.</footnote>
However, these early funder policies tolerated publisher-defined *embargoes*, thus limiting their utility of making their funded research Open Access. Plan S can then be seen as an attempt at incorporating the findings of the Finch report making emphasis on the value of immediate Gold OA while simultaneously addressing the issues raised by previous developments – such as delayed Green Open Access and the long-standing concerns around *double-dipping* emerging from Hybrid Open Access.

It's worth noting that no Open Access business model is specifically advocated for in the ten principles. Due to the inclusion of transformative agreements and journals in the routes to compliance listed above, Plan S was quickly identified by some in the scholarly communications community as an attempt to move towards a fully Gold Open Access landscape with APC payments and transformative agreements and journals as its commercial publisher-friendly flagship policies. But the plans to support embargo-free Green OA via rights retention and Diamond OA were there from the start and were quickly laid out in detail following the feedback received from the scholarly communications community to the consultation run upon the announcement of Plan S [6].

All of this is relevant when trying to assess the impact of Plan S because many of its policies are very recent. Not only was the 5th anniversary of the release of Plan S commemorated shortly after this study was commissioned [7], but the adoption of the first Plan S-aligned Open Access policies by cOAlition S funders only happened in 2021 [8]. This means that it may well be too early to quantitatively assess the impact of Plan S by looking into the evolution in the distribution of funded publications across Open Access models since Plan S was released.

## REFERENCES

[1] The Plan S Principles, https://www.coalition-s.org/plan_s_principles/

[2] cOAlition S (2023). "cOAlition S confirms the end of its financial support for Open Access publishing under transformative arrangements after 2024". https://www.coalition-s.org/coalition-s-confirms-the-end-of-its-financial-support-for-open-access-publishing-under-transformative-arrangements-after-2024/

[3] cOAlition S (2020). "cOAlition S releases the Journal Checker Tool, a search engine that checks Plan S compliance", https://www.coalition-s.org/coalition-s-releases-the-journal-checker-tool/

[4] András Holl, Erika Bilicsi, David Ball, Alma Swan (2016). "PASTEUR4OA Briefing Paper: Open Access mandate support", https://openaccess.mtak.hu/wp-content/uploads/bp_open_access_mandate_support.pdf

[5] Pablo de Castro, Anna Rovira, Antti Rousi (2022). "Successful implementation of Open Access strategies at Universities of Science & Technology". CESAER White Paper, https://doi.org/0.5281/zenodo.6410867

[6] cOAlition S (2019). "Rationale for the Revisions Made to the Plan S Principles and Implementation Guidance". https://www.coalition-s.org/rationale-for-the-revisions/. One of the drivers for these revisions was the response to the release of Plan S that the Confederation of Open Access Repositories (COAR) posted on their website at https://www.coar-repositories.org/files/COAR-response-to-Plan-S-Sept-2018.pdf (file now withdrawn, but the response stated that Plan S "overlooks the main role of the global network of over 3000 repositories around the world").

[7] cOAlition S (2023). "5 Years of Plan S: webinar". cOAlition S blog, https://www.coalition-s.org/5-years-of-plan-s-webinar/

[8] cOAlition S (s.a). "Implementation Roadmap of cOAlition S Organisations", https://www.coalition-s.org/plan-s-funders-implementation/
## 2.3. THE ROLE OF FUNDERS WITHIN THE SCHOLARLY COMMUNICATIONS LANDSCAPE

The key role *research funders* play as enablers for Open Access policies has previously been discussed in the literature [1], and there is widespread historical praise for OA policies like the one introduced by the then Higher Education Funding Council for England (HEFCE) in support of the UK's Research Excellence Framework (REF) national assessment exercise [2]. Moreover, there were calls for the international alignment of funder OA policies since as early as 2013 [3] and the EU-funded FP7 PASTEUR4OA project (2014-2016) [4] provided a conceptual framework for operationalising such alignment. The emergence of *cOAlition S* as an alliance of funders aiming to streamline the way towards full immediate Open Access is thus completely aligned with the historical trend that puts funders at the core of the goal to make Open Access the default in scholarly communications.

This role largely stems from the relationship that funders have with their funded researchers and with their influence over the way their funded research results are published. This gives funders a leverage over publishers that no single institution could ever achieve on its own<fnref n="5" />. The drive caused by Plan S for publishers to get closer to consortia in order to realise some of the routes towards full immediate OA suggested in *Plan S* is evident proof of such influence.

However, scholarly publishing is rarely the main area of activity for research funders<fnref n="6" />. This is rather the issuing of research funding calls and the assessment of project proposals received in response to these, together with the admin management of the grants they award. While a number of funders in various countries have supported Open Access for their funded publications for a long time, there are far more for whom Open Access is not a key priority even today. This turns the formation of a group of like-minded funders like cOAlition S into a landmark that funders elsewhere may be in a position to replicate, especially on a national level where the logistics become simpler.

cOAlition S brings together a wide range of research funders – national vs philanthropic funders or charities, international ones like the EC or the WHO, general-purpose vs discipline-specific ones like the INFN or the numerous biomedical ones. There is even the odd cOAlition S member who is not strictly a research funder but rather a research-performing organisation who also funds research. The implications of these variations become for instance apparent when examining the impact national elections have on the ability of national public funders to commit to sustained policies and fixed time schedules, but It's this diversity on top of the geographical one which gives cOAlition S its strength, as it's able to cover ground from different perspectives.

However, the actual implementation of the Open Access policies agreed by cOAlition S members will typically not be carried out by the funders themselves, but by the institutions at which their funded researchers work and by the publishers to whom they submit their manuscripts. In the former case – i.e. institutions as conveyors of the funders' OA policies – this is a reasonable distribution of tasks. First, because institutions are where the research activity and the publishing of its results takes place. Besides that, institutional researchers are typically funded by many different agencies whose policies can be applied in a single layer of Open Access support at the institution.

<footnote n="5">⁵ The positive reaction from specific publishers unaware of the rights retention route to embargo-free Green Open Access when funded authors pointed out to them that this route was actually required by their research funder is testament to this leverage. No institution on its own would have been able to persuade a publisher to distinctly include such route on their website.</footnote>

<footnote n="6">⁶ Arcadia is worth a mention here as a one-of-its-kind funder for whom scholarly communications is indeed on the top-priority list. It's not a member of cOAlition S and there are hence not many direct references to their work in this report, but numerous developments celebrated in it have seen a significant input from them.</footnote>
The transferring to institutions of the responsibility for the practical implementation of the Open Access policies designed by the funders makes the workflows significantly more complex. These may also be subject to adaptations suited to the national policy frameworks, which creates differences across countries. An example for such an adapted application of the Plan S guidance to the specific national policy framework in the United Kingdom is described as an example in the following section. This issue sits at the core of the current fragmentation of the Open Access landscape along national boundaries.

Despite the solid communication channels between institutions and funders, the extensive implementation network that funders must manage is challenging, especially with limited staff available in a complex, non-core domain. These challenges lead to the establishing of a highly distributed multiple-actor scholarly communications landscape where the coordination needs multiply as does the risk of misalignment across each of the involved stakeholder's objectives. Broadly speaking, these stakeholders are:

* **Research funders.** Where the Open Access policy emerges from, ideally aligned with the national policy framework in each country. Funders will typically disseminate their policy to their funded researchers, but these are not experts in the ever more complex nuances of Open Access and will often not completely grasp the implication of what is written in the grant agreements.

* **Institutions** (usually universities and research centres). Where funded researchers work and publish. It's the institutional Open Access teams who tend to do the heavy lifting on the practical implementation of the OA policies issued by funders in terms of both dissemination and on-the-ground application (payments, reporting etc).

* **Consortia.** Where the framework of applied policy instruments arises that will allow the implementation of OA policies, particularly with regard to agreements with publishers. So far this area of activity has heavily focused on the negotiation of **transformative agreements (TAs)** with publishers, but there is a growing trend for consortia to support the diversification of research publishing models. This would enable alternative business models such as **Diamond Open Access** or **Subscribe to Open** to flourish. Consortia are hence additional key actors in the Open Access landscape with whom the funders need to engage so that the instruments consortia offer to their members (the institutions) are in line with the funders' policy objectives. Many cOAlition S funders are represented in the decision-making committees at consortia, but the way this happens varies across countries<fnref n="7" />.

<figure><img src="image_5.png" /></figure>

It's this complexity in the design of the mechanisms for the OA policy implementation that drives the fragmentation along national borders. Consortia tend to serve their members/ customers the research-performing organisations and despite the existence of international coordination bodies like ICOLC, they understandably tend to focus on their national-level priorities. This defeats – only to some extent and by no means completely – the purpose of the funder coordination on Open Access policies at an international level. Some of

> "I think you have the people that actually create the policies, the consortia, and then you have the libraries, and within the libraries, you have the research support offices where they advise the authors. And I think the three of them, they seem to have a different interpretation of what they need, and how they want to get there"

(Publisher)

<footnote n="7">⁷ There are cases where a single actor plays several of these roles at the same time: such is for instance the case of the cOAlition S funder FCT (Portuguese Foundation for Science and Technology), who are simultaneously the national research funder and the organisation, through its digital services unit FCCN (www.fccn.pt), in charge of negotiating with publishers (i.e. they also play the role of a consortium and subsequently maintain the country's b-on database of agreements with publishers at https://www.b-on.pt/).</footnote>
the recommendations provided at the end of this report try to suggest possible actions to address this risk of fragmentation.

It's worth mentioning in a section devoted to the role of funders the trend whereby these organisations are increasingly becoming involved in publishing. The launch of the F1000-based Open Research platforms by the likes of the Wellcome Trust, the European Commission, the Gates Foundation and the Irish Health Research Board a few years ago is proof of this additional role some funders are adopting on top of their policy issuing activity.

The current level of uptake for these platforms is only moderately successful and raises questions on whether their role within a wider move towards a more sustainable and equitable publishing ecosystem could be enhanced. Funded researchers could be persuaded to more consistently use these platforms going forward via a combination of the appropriate recognition mechanisms and of best practice example dissemination.

Also worth mentioning are some structural limitations on the role of funders as enforcers for their Open Access policies. Section 2.4 below on the impact of Plan S on a specific national Open Access framework highlights the limitations in the funder-institution rapport. But there are also limitations in the funder-researcher balance. A July 2020 workshop held by a research funder in London gathered a good number of academics in the Social Sciences and Humanities funded by that research funding organisation. The purpose of the session was to present to them the forthcoming Plan S-aligned Open Access policy the funder was planning to adopt from the start of the following year. When the discussion reached the item on Open Research platforms, many academics in the room reacted to the suggestion to submit their manuscripts to such platforms with overt threats of telling their students to submit their project proposals elsewhere if the funder persisted in trying to enforce the “absurd” concept of Open Publishing platforms to replace “community-building” (commercial) journals in their disciplines.

No funding application was apparently withdrawn – or grant returned – as a consequence of this exchange. But Open Research platforms haven’t (yet) realised their full potential either. This report – and the qualitative analysis in chapter 4 in particular – is written from a pragmatic position bearing these regular interactions with researchers in mind.

## REFERENCES

[1] Pablo de Castro (2017). The quest for funding information. euroCRIS blog. https://eurocris.org/blog/quest-funding-information

[2] Sierra Williams (2014). HEFCE announces Open Access policy for the next REF in the UK: Why this Open Access policy will be a game-changer. LSE blog. https://blogs.lse.ac.uk/impactofsocialsciences/2014/04/01/hefce-open-access-ref-gamechanger/

[3] Alma Swan (2013). As the European Commission paves the way for open access, a consistent policy environment is needed across the EU. LSE blog. https://blogs.lse.ac.uk/impactofsocialsciences/2013/10/22/swan-eu-open-access-policy/

[4] Open Access Policy Alignment STrategies for European Union Research (PASTEUR4OA) FP7 project, https://cordis.europa.eu/project/id/611742/fr

[5] Wellcome Trust – Funded people and projects, https://wellcome.org/grant-funding/funded-people-and-projects

[6] CORDIS - EU research results, https://cordis.europa.eu/

[7] FWF Research Radar, https://www.fwf.ac.at/en/discover/research-radar

[8] Research Information – Industry Trends (2024). African research funders in global spotlight. https://www.researchinformation.info/news/african-research-funders-global-spotlight

[9] Rick Anderson (2023). Where Did the Open Access Movement Go Wrong?: An Interview with Richard Poynder. The Scholarly Kitchen. https://scholarlykitchen.sspnet.org/2023/12/07/where-did-the-open-access-movement-go-wrong-an-interview-with-richard-poynder/
## 2.4. THE IMPACT OF PLAN S ON A SPECIFIC NATIONAL OPEN ACCESS FRAMEWORK

This section presents a case study to illustrate the complexity embedded in the multiple-stakeholder scholarly communications landscape described above. It does so by examining the way *Plan S* policies have influenced the Open Access implementation workflows applied by institutions in the United Kingdom. This is aimed to show both how top-down policies may be adapted to a specific national policy framework and how this may lead to a fragmentation in the Open Access implementation workflows across countries.

As mentioned in section 2.2 above, the Open Access policy framework in the UK has largely been driven by the REF (Research Excellence Framework) *Green Open Access* policy and its requirement for full-text *accepted manuscripts* to be deposited as soon as possible upon acceptance. This policy, tightly coupled to research assessment, has traditionally provided a sustainable and affordable mechanism for ensuring Open Access to accepted manuscripts, albeit after long *embargo periods*. With its decisive stance on immediate Open Access, Plan S has significantly impacted these workflows for the better, but not massively changed them.

The main initial contribution of Plan S to tackle the issue of delayed Open Access were the so-called *transformative arrangements* – including both transformative or *Read & Publish agreements* and *transformative journals*. Read & Publish agreements have seen a very significant uptake in the UK – with the UK consortium, the Jisc, playing a key role in this development – and have in fact consistently increased the levels of immediate Open Access. But these agreements have not stopped Green Open Access: full-text accepted manuscripts are still being collected in institutional systems even when the *Versions of Record (VoRs)* are published *Gold OA* via a transformative agreement or an *APC* payment.

In fact, Read & Publish agreements have strengthened the Green Open Access workflows. The effect of the author-pays model in driving researchers to rely ever more consistently on their institutional Open Access support services is well documented in the literature [1]. If institutions demand a copy of the *AAM* to be provided as a prerequisite for funding an APC or for approving Gold Open Access for a paper under a Read & Publish agreement, researchers will be happy to oblige. This reinforces a Green Open Access culture totally in line with the REF requirements and the national policy framework.

Previous studies on the impact of transformative agreements have reported that immediate, hybrid Open Access facilitated by R&P agreements is happening at the expense of Green OA while not increasing the aggregate OA levels [2]. This finding is confirmed by the quantitative analysis of Plan S described in chapter 3 below, but Green OA nevertheless remains at the basis of the Open Access implementation effort.

When *VoRs* are made openly available in the journal, AAMs are not visible from the outside and are “not needed” anymore – at least for the purpose of Open Access; its internal role remains very important within institutions.

By tackling delayed Open Access via a combination of strategies – APC-based Gold OA, R&P agreements and embargo-free Green Open Access – Plan S has *in practice* reinforced the Green Open Access workflows at institutions in the UK. AAMs may seem superfluous when the VoR is openly available, but the only way to implement rights retention policies is to systematically collect these AAMs. This is another reason for the extraordinary uptake of *Institutional Rights Retention Policies* in the UK examined in section 4.2, where Plan S is credited as a major driver for change. For many years institutions have promoted a scholarly communications culture whereby researchers have become used to the fact that AAMs need to be provided. Nowadays this is standard practice for researchers and Open Access support services and *no top-down policy is going to change it*, be it a new REF policy or Plan S itself.
The three-layer Open Access implementation strategy at UK institutions is shown as a pyramid in the figure below. This reflects how Open Access is being implemented *in practice* and provides a case study for the impact of Plan S – given that the two top layers are its direct consequence. This is relevant as an illustration of the implications of a multiple-stakeholder Open Access landscape on the role of funders and on the national-level coordination needs.

<figure><img src="image_6.png" /><figcaption>**Figure 1.** The Open Access pyramid as an illustration for the national Open Access framework in the UK</figcaption></figure>

One of the main issues around this construct is that it's UK-specific: there are many countries and regions where the Green Open Access layer at the base is simply missing. The reasons for this are multiple – the absence of the appropriate policies, the lack of awareness among researchers, the lack of sufficiently trained staff at institutions to implement this route. The absence of this Green Open Access layer does not mean that progress cannot be achieved on the basis of the intermediate layer alone, but it will be limited. The risk of fragmentation along national borders mentioned in the previous section is a direct consequence of these diverse national policy frameworks.

Maintaining a healthy Green Open Access layer at the bottom – even if its apparently “not needed” – allows institutions the possibility to dispose of transformative agreements altogether if they become financially unsustainable. The choice recently made by the Japanese Ministry of Education, Culture, Sports, Science and Technology (MEXT) to strengthen this Green Open Access layer [3] is fully aligned with this architecture, regardless of whether or not Read & Publish agreements with publishers are added on top in a Japanese context.

Strong differences will inevitably arise in the way this model is applied across countries and geographies, and even in the same country there can be changes of course too [5]. Hence the key relevance of an international body like cOAlition S that can oversee the way Plan S is adopted in practice under different national policy frameworks.

## REFERENCES

[1] Pablo de Castro, Gwen Franck (2019). "Funding APCs from the research funder's seat: Findings from the EC FP7 Post-Grant Open Access Pilot". Profesional de la información 28(4). https://doi.org/10.3145/epi.2019.jul.13

[2] Kira Brayman, Amy Devenney, Helen Dobson, Mafalda Marques, Anna Vernon (2024). "A review of transitional agreements in the UK". Jisc report. https://doi.org/10.5281/zenodo.10787391

[3] Dalmeet Singh Chawla (2024). "Japan's push to make all research open access is taking shape". Nature News. https://doi.org/10.1038/d41586-024-01493-8

[4] cOAlition S (2019). "OA2020 and cOAlition S Launch Joint Statement". https://www.coalition-s.org/oa2020-and-coalition-s/

[5] Rosalyn Bass (2024). University of Kent's response to the REF 2029 Open Access Policy consultation. University of Kent Information Services blog. https://blogs.kent.ac.uk/isnews/university-of-kents-response-to-the-ref-2029-open-access-policy-consultation/
# 3. COUNTERFACTUAL IMPACT EVALUATION OF PLAN S

The Counterfactual Impact Evaluation (CIE) that informs the quantitative analysis of the impact of Plan S is presented in this chapter. The methodology and limitations of the study are described and the results are summarised. The two main results are:

1. The rate of Open Access growth measured for funders included in the treatment group (i.e. those with Plan S-aligned OA policies) has been slower than the rates for funders in the two counterfactuals (i.e. the control and the comparison groups)

2. Hybrid OA has grown more significantly among cOAlition S funders than among funders in the control group.

The Invitation to Tender for this study that cOAlition S released in July 2023 [1] made emphasis on the need for a quantitative analysis of the impact of *Plan S*:

Provide an analysis of the likely progress towards full and immediate open access had Plan S (and cOAlition S) NOT existed and estimate what contribution Plan S (and cOAlition S) has made to the changing scholarly communication ecosystem, using appropriate techniques, such as a counterfactual impact evaluation and/or contribution analysis, or similar.

This study subsequently includes a *counterfactual impact evaluation (CIE)* of Plan S. This CIE relies on an econometric statistical approach to establish the impact that Plan S has so far had on the worldwide scholarly communications landscape in the five years since it was unveiled in September 2018.

This chapter is structured in two broad sections. The first one provides the rationale for and the design of the mechanism to quantitatively assess the impact of Plan S, along with some of the inevitable methodological limitations. The second part is devoted to the explanation and analysis of the results of this evaluation.

## 3.1. CIE OVERVIEW

To quantitatively measure the impact of Plan S, a difference-in-differences analysis is conducted on the rates of Open Access growth for subsets of publications funded by different groups of funders. A 'treatment group' composed of cOAlition S funders provides the evidence of the impact of Plan S when compared against a 'control group' of funders with barely any Plan S footprint. This approach poses some methodological limitations which may affect the reliability of the CIE results.

The attempt at quantitatively assessing the impact of Plan S as introduced by cOAlition S in September 2018 is based on the application of a Counterfactual Impact Evaluation to Plan S. This aims to quantitatively ascertain what the evolution of the scholarly communications landscape would have been if Plan S had never been released.
The most evident way to try and measure this would be to explore the extent to which the number and percentage of Open Access publications – as per the accepted definition that includes the publication under an open licence, preferably a *Creative Commons* one [2] – may have changed *worldwide* in the period during which Plan S has been active (2018-2023). However, this is seen as impractical: widening the scope too much would both minimise the measurable impact of the initiative and exponentially increase the demands on data treatment and analysis.

A more effective approach in both senses is to base the analysis on *funded publications by cOAlition S funders*. This will not only increase the likelihood that Plan S-aligned Open Access policies issued by such funders have had an impact on the publications stemming from their funded projects, but it will also allow a reliable comparison between members and non-members of cOAlition S when it comes to the rates of Open Access achieved by different research funders for their funded publications.

Based on the assumption that it may be possible to measure a statistically significant influence of Plan S on the rates of Open Access publications, a first step will examine the difference in the *aggregated* rates of Open Access in the period before and after the release of Plan S. Given that Plan S includes various policy instruments addressing various routes to achieve immediate Open Access, a second step of the analysis will drill down into the different *Open Access flavours* to ascertain where the impact of Plan S has been at its highest thus far.

### 3.1.1. The difference-in-differences method

The underlying research design to elicit any effect is the so-called *difference-in-differences (DID)* method [3]. This is an econometric strategy to replicate the methodology for a scientific experiment – such as clinical trial – into the social sciences, with similar *treatment* and *control groups*. The treatment in this case is Plan S as a policy instrument to achieve full, immediate Open Access. DID-based approaches are frequently applied to measure the impact of specific policy interventions, especially in the realm of labour- and health-related policies [4].

The analysis of the impact of Plan S is not a controlled laboratory experiment, but rather what is known as a 'natural' experiment. Based on the definition of the adequate treatment and control groups, the likelihood of a funded publication being Open Access will be compared prior to the introduction of Plan S and after its release. This leads to the differences in prior-post as well as in treatment-control, which culminates in the difference-in-differences framework.

The purpose of this methodology is to filter out effects that would have happened anyway – for instance there was a growing number of so-called transformative agreements before the arrival of Plan S that would have had an impact even if Plan S had not been introduced. To achieve this filtering out, a *control group* needs to be identified and analysed that was *unaffected by Plan S*. This control group shall capture general developments and changes in patterns in Open Access publishing and funding unrelated to Plan S. The *treatment group* on the other hand should be composed of subjects where *the impact of the treatment policy may have been the largest*, i.e. in this case cOAlition S funders who have issued a Plan S-aligned Open Access policy. The composition of the different groups of funders that will be used for this CIE and the rationale for the funder selection are outlined in section “control and treatment groups” below.

### 3.1.2. Establishing the timeframe for the analysis

For the temporal comparison (i.e. pre- vs post-Plan S release), **1 January 2021** is established as the **treatment date** instead of 2018, which was the year in which Plan S was actually announced. This is because 1 Jan 2021 marks the date when most Plan S-aligned Open Access policies were introduced by cOAlition S funders [5].
A 9-year time horizon is selected for this analysis: 6 years before the 1 Jan 2021 treatment date, i.e., **2015-2020** as the 'pre-Plan S' span and **2021-2023** as the 3-year 'post-Plan S' period. For the most recent year, 2023, data up to December has been collected. Where Plan S-aligned Open Access policies were introduced *after* 1 Jan 2021 by specific cOAlition S funders included in the treatment group, the date of the release of such policies becomes the cut-off date (or treatment date) applied for these specific funders. This is the case for both the Howard Hughes Medical Institute (HHMI) in the United States and the UK Research and Innovation (UKRI) in the United Kingdom, with Plan S-aligned OA policy adoption dates of 1 Jan 2022 and 1 Apr 2022 respectively.

**Note:** The 1 Jan 2021 'treatment date' is far from being fully accurate (see section 3.2. Limitations of the CIE study for more information). On the one hand, Plan S-aligned Open Access policies often apply to manuscripts submitted on or after the date when the policies come into force. This means that it will typically take several months for the first manuscripts to be published that fall under the new OA policy (in the case of the British university represented in the affiliations for the authors of this study, the first UKRI-funded article that carried the 2-line rights retention statement as per the funder's policy was released on 1 September 2022, i.e. 5 months after the policy kicked-in). On the other hand, the application guidance for Plan S-aligned OA policies for many funders states "All new grants awarded after 1 Jan 2021" or "All new funding calls opened after 1 Jan 2021". This shortens even more the post-Plan S period under analysis and introduces further inaccuracies in the quantitative analysis of its impact. **Annex 3** includes an analysis of the kick-off dates for Plan S-aligned Open Access policies issued by cOAlition S funders.

### 3.1.3. Technical assumptions

A couple of statistical assumptions must be met for a DID analysis to be technically solid. First, there must be a sufficient impact of the treatment, in this case Plan S. While in previous work of this nature [6] the analysis was based on the whole set of research publications produced in a given country over a period of time, in this case – in order to increase the chances of detecting an effect – the scope will be narrowed to publications that acknowledge funding from specific research funders. The main reason for this narrower scope is that it is unlikely that the level of awareness of Plan S among unfunded scholars has been very high, as opposed to funded researchers, research librarians, and other members of relevant stakeholders. On top of this, grant agreements often include Open Access clauses that may be directly inspired or caused by Plan S.

A second core element of the CIE is the stable unit treatment variable assumption (SUTVA). This assumption states that the potential outcome of a particular unit or observation is not affected by the treatment status of another. In this case this would mean that the likelihood of a paper being published Open Access is not affected by other papers where Plan S has had an impact on their own likelihood of being published under an Open Access licence. Due to group dynamics and best practice examples, this might not necessarily be the case here.

### 3.1.4. Treatment, control and comparison groups

From a practical viewpoint, a key element to ensure a successful CIE exercise is the selection of the countries and research funders that will be included in the **treatment** and **control** groups. Following the clinical trial analogy, the control group should ideally capture scenarios where Plan S has had a very low to non-existent footprint, which will allow the modelling of the "if Plan S hadn't existed" hypothesis. On the other hand, the treatment group will target countries and funders where the impact of Plan S is expected to have been at its maximum. This is best represented by funders represented in cOAlition S and the countries where they are based.
**Treatment Group** (composed of “Plan S-intensive funders”):

Funders that were part of Plan S since its introduction

Austrian Science Fund (FWF, Austria)

Howard Hughes Medical Institute (HHMI, US)

National Science Center (NCN, Poland)

Dutch Research Council (NWO, Netherlands)

UK Research and Innovation (UKRI, United Kingdom)

**Control Group** (composed of funders with no – or barely any – Plan S footprint)

National Funders that are not cOAlition S members and whose OA policies do not align with Plan S

Agencia Nacional de Investigación y Desarrollo (ANID, Chile*)

Ministry of Business, Innovation and Employment (MBIE, New Zealand)

National Natural Science Foundation of China (NSFC)

* Given that the ANID was only legally established in 2018 [7] and started operating as of 1 January 2020, the snapshot for their funded publications in previous years also includes CONICYT or "Comisión Nacional de Investigación Científica y Tecnológica" which was the previous name of the national Chilean research funder.

On top of these canonical control and treatment groups, a third group is defined (referred hereafter as the **comparison group**) that could yield statistically interesting results, namely those research funders (and countries) that, while technically not being part of cOAlition S, sit very close from a policy perspective to cOAlition S funders and countries and may subsequently have experienced some impact of Plan S. This impact would mainly arise from the fact that an Open Access policy alignment across research funders [8] makes life much easier for their funded researchers. On top of that, cOAlition S funders like the European Commission, the Gates Foundation or the Wellcome Trust are international in scope and therefore their policies have wider effects beyond the countries where they are issued.

The hypothesis to be tested is that this third group of so-called “non-Plan S-intensive” funders and countries could be expected to have experienced some impact of Plan S but not as high as the one measured for cOAlition S funders and countries (the treatment group). The quantitative results for this third group (the comparison group) could subsequently be expected to be sitting midway between the other two. This is an interesting hypothesis to test for the purpose of confirming the statistical soundness of the method that will be applied throughout.

**Comparison Group** (composed of “non-Plan S-intensive funders”):

National or Regional Funders that while technically not part of cOAlition S, have OA policies well aligned with Plan S

Fonds Wetenschappelijk Onderzoek - Vlaanderen (FWO, Flanders, Belgium)

German Research Foundation/Deutsche Forschungsgemeinschaft (DFG)

National Cancer Institute (NCI, United States)
### 3.1.5. Rationale for funder selection into the treatment and control groups

The main criteria for the selection of the funders to be represented in the treatment and control groups has been to maximise the possible impact of Plan S while simultaneously providing a diverse snapshot from a geographic perspective. Thus, the treatment group includes coAlition S funders from outside Europe besides the various European ones representing different regions within the continent. As for the control group, the basis for choosing countries where the impact of Plan S is deemed to be negligible is again mostly geographical: while the complementary qualitative analysis carried out suggests awareness of Plan S all across the world, it is highly unlikely that publications funded by research funders with little exposure to Plan S may have seen any significant effect of this policy.

Further considerations taken into account in the funder allocation to the different groups have also include funder size and funder discipline. As explained in the “limitations of the study” section below, it's impossible to completely filter out unintended effects arising from comparing apples to pears, but it has been assumed that a sufficiently diverse choice of funders may reduce the intrinsic inaccuracies.

The same approach towards diversity has been followed when selecting the funders to be part of the comparison group. These also include both European and non-European funders with varying sizes and disciplinary focus.
## 3.2. LIMITATIONS OF THE CIE STUDY

Despite the careful design of the CIE, a number of limitations need to be highlighted that potentially affect the ability of this methodology to provide conclusive evidence for the impact of Plan S. These limitations have been addressed in the design of the study, but they're bound to have some influence. A qualitative analysis of the impact of Plan S provides a necessary complement to any quantitative assessment.

The CIE overview has already mentioned a number of simplifications built into the design of the DID method used to quantitatively assess the impact of Plan S. It is subsequently necessary to examine the study limitations in some detail, as some of these specifically affect the outcome of the CIE.

1. Some of the policies included in the Plan S toolbox are arguably too recent for a statistically sound effect to be measured. Plan S proposes a combination of complementary strategies to pursue full immediate Open Access. Some of these strategies – such as APC-based *Gold OA* and *Diamond OA* – have been around for quite some time, whereas other ones like *Read & Publish agreements* and particularly *rights retention policies* are far more recent. As a result of this, it is very difficult to quantitatively identify the impact of some of these strategies as part of an analysis conducted at a time when Plan S has just turned 5 years old. Moreover, most of the Plan S-aligned Open Access policies issued by cOAlition S funders were not released until January 2021, which makes the timeframe for the evaluation of their impact even shorter, especially when considering that for many funders the policy applied to new grants awarded after 2021 of funding calls made after this date [5].

2. There is a risk in assigning any changes on Open Access trends to Plan S that **may not be strictly attributable to it**. A strong trend is going on in parallel to Plan S towards increasing the level of *Hybrid Open Access* via the coordinated efforts of worldwide university consortia to advance with the signing of transformative agreements. These agreements, collected in the ESAC Registry [9], often overlap with Plan S and are likely to have had a simultaneous effect. The way this is addressed is by focusing on publications arising from projects funded by cOAlition S funders, but even when applying this approach some background effect of initiatives like OA2020 is likely to be misattributed. On the other hand, OA2020 may be seen as an additional mechanism to further the impact of Plan S, as shown by the joint statement issued by both initiatives in 2019 [10].

3. **Possible data quality issues** with regard to data samples taken from open sources. The post published in the scidecode blog in January 2024 [11] compared the quality of the data samples provided by commercial vs openly available bibliometric databases for a set of funded publications in 2021-22 by a specific research funding agency (Fiocruz in Brazil). The results of this limited-scope analysis showed that the number of publications captured by open databases (OpenAlex + Crossref + Unpaywall + others) was higher than – and frequently also of higher quality than – those in the sample provided by a commercial literature database.
<table><caption>**Table 1.** Funded publications by funding body and year. Colour codes are blue for treatment group, red for control group and green for comparison group funders (source: https://doi.org/10.5281/zenodo.12523229).</caption><thead><tr><th></th><th>2015</th><th>2016</th><th>2017</th><th>2018</th><th>2019</th><th>2020</th><th>2021</th><th>2022</th><th>2023</th><th>Total</th></tr></thead><tbody><tr><td>FWF (Austria)</td><td>1.745</td><td>2.329</td><td>2.520</td><td>2.772</td><td>2.978</td><td>3.225</td><td>3.361</td><td>2.948</td><td>2.759</td><td>24.637</td></tr><tr><td>HHMI (United States)</td><td>714</td><td>925</td><td>1.124</td><td>1.089</td><td>1.193</td><td>1.191</td><td>1.173</td><td>891</td><td>754</td><td>9.054</td></tr><tr><td>NCN (Poland)</td><td>2.229</td><td>3.177</td><td>3.864</td><td>4.950</td><td>5.476</td><td>5.876</td><td>6.655</td><td>5.857</td><td>5.654</td><td>43.738</td></tr><tr><td>NWO (Netherlands)</td><td>1.386</td><td>2.177</td><td>2.690</td><td>3.378</td><td>3.934</td><td>4.256</td><td>4.444</td><td>3.951</td><td>3.733</td><td>29.949</td></tr><tr><td>UKRI (United Kingdom)</td><td>10.658</td><td>14.721</td><td>17.231</td><td>20.018</td><td>21.681</td><td>23.635</td><td>25.083</td><td>21.488</td><td>19.590</td><td>174.105</td></tr><tr><td>ANID/CONICYT (Chile)</td><td>1.003</td><td>1.477</td><td>1.958</td><td>2.264</td><td>2.510</td><td>3.039</td><td>3.434</td><td>3.094</td><td>2.882</td><td>21.661</td></tr><tr><td>MBIE (New Zealand)</td><td>225</td><td>335</td><td>361</td><td>387</td><td>467</td><td>489</td><td>636</td><td>570</td><td>538</td><td>4.008</td></tr><tr><td>NSFC (China)</td><td>90.508</td><td>124.170</td><td>152.544</td><td>202.064</td><td>246.243</td><td>268.058</td><td>277.593</td><td>310.029</td><td>322.996</td><td>1.994.205</td></tr><tr><td>DFG (Germany)</td><td>11.929</td><td>15.360</td><td>17.757</td><td>20.755</td><td>23.038</td><td>25.573</td><td>28.277</td><td>25.054</td><td>23.909</td><td>191.652</td></tr><tr><td>FWO (Belgium)</td><td>1.166</td><td>1.491</td><td>1.914</td><td>2.304</td><td>2.635</td><td>3.161</td><td>3.536</td><td>3.034</td><td>3.074</td><td>22.315</td></tr><tr><td>NCI (United States)</td><td>3.143</td><td>4.935</td><td>5.640</td><td>6.706</td><td>7.695</td><td>8.234</td><td>8.902</td><td>7.760</td><td>7.080</td><td>60.095</td></tr><tr><td>Total</td><td>124.706</td><td>171.097</td><td>207.603</td><td>266.687</td><td>317.850</td><td>346.737</td><td>363.094</td><td>384.676</td><td>392.969</td><td>2.575.419</td></tr></tbody></table>

As an example for these possible data quality issues, two tables are provided alongside each other on this page. Table 1 above shows the annual number of publications by funder in the 2015-2023 samples used as a basis for this CIE. Table 2 below shows the number of funded publications collected in Europe PMC in that same period for funders included in the control group (both the Austrian FWF and the Dutch NWO are members of Europe PMC). A comparison reveals significant discrepancies between both snapshots: not only the figures for the funders included in both snapshots (FWF, NWO and MRC as a proxy for the UKRI) are different, but the number of funded publications is also far more stable year-on-year on the Europe PMC snapshot.

<table><caption>**Table 2.** Number of annual funded publications by Europe PMC research funder (selection)
Source: Europe PubMed Central, figures as of 30 January 2024</caption><thead><tr><th></th><th>2015</th><th>2016</th><th>2017</th><th>2018</th><th>2019</th><th>2020</th><th>2021</th><th>2022</th><th>2023</th></tr></thead><tbody><tr><td>FWF</td><td>1,984</td><td>1,975</td><td>2,105</td><td>2,191</td><td>2,267</td><td>2,464</td><td>2,505</td><td>2,229</td><td>1,696</td></tr><tr><td>NWO</td><td>1,672</td><td>2,021</td><td>2,310</td><td>2,707</td><td>2,920</td><td>3,173</td><td>3,381</td><td>2,840</td><td>2,475</td></tr><tr><td>MRC<br>(proxy for UKRI)</td><td>11,952</td><td>11,931</td><td>12,161</td><td>12,542</td><td>12,663</td><td>13,953</td><td>14,837</td><td>12,670</td><td>5,920</td></tr></tbody></table>

This is to say that there is no fully accurate sample of funded publications on which to base the whole quantitative analysis. This is likely to introduce some imprecision in the CIE results and is again one of the reasons why a complementary qualitative analysis has been conducted.

1. The three groups of funders defined for the CIE are not clear-cut. The funders included in the control group have been carefully selected in a way that ensures they have had no – or barely any – Plan S footprint, but this is very difficult to assess. In fact, given the levels of researcher mobility and the ‘contagion effects’ in Open Access trends, it’s safe to assume that there will have been a slight impact of Plan S on these geographically remote outputs too. This effect will be “filtered out” via the comparison with treatment group funders where the impact of Plan S will have been considerably larger, but it still introduces a certain imprecision in the analysis.

2. Other factors affecting the reliability of sample analysis. These are mostly arising from the fact that all funders in any of the three groups are being treated equally, i.e. without introducing any correction for research discipline, geographical location or funder size. These corrections cannot be accommodated as there is no clear pattern for the effects of these variables. As explained in the rationale for the selection of funders for the different CIE groups above, the way these potential imprecisions have been addressed is via a deliberate attempt at diversity in the choice of funder size, disciplinary focus and geographic location.
The impact of the geographical location of the funders is particularly relevant: countries where ambitious OA policies have been in place for quite some time are expected to see a well-established trend to publish OA *before* the arrival of Plan S, meaning that its measured effect would as a result be smaller than otherwise. Finally, significant differences in size across funders in the same group also introduce inaccuracies. **Table 1** above clearly shows that one specific funder in each of the groups is significantly larger than the other ones: in the treatment group, the number of annual UKRI-funded publications outweighs the other four funders combined, while the Chinese NSFC and the German DFG are also much larger than their counterparts in the control and the comparison groups. As a consequence of this, the behaviour of the group as a whole in the analysis will inevitably be biased towards the largest funder in it.

## 3.3. RESEARCH DESIGN

*A snapshot of the data is provided on which the CIE is performed, together with a description of its processing and of the technical application of the difference-in-differences (DID) method. On the face of the figures for funder publications per funder, one of the caveats is the high levels of Open Access already achieved by cOAlition S funders before the arrival of Plan S.*

### 3.3.1. Data collection and grouping

The snapshot of the funded publications data for all funders in the three groups (treatment, control and comparison) described in the CIE overview above was provided by OA Works in December 2023 and January 2024. The data – shown in **table 1** above – covers the 2015-2023 period under analysis. The data was aggregated, cleaned and pre-processed into the csv files that are provided as the companion datasets to this report.

The *counterfactual impact evaluation (CIE)* shall use these figures as a basis to investigate the effect of Plan S on the uptake of open access among publications that have been supported by grants awarded by a range of cOAlition S funders, namely the Austrian Science Fund (FWF), the Howard Hughes Medical Institute (HHMI) in the United States, The National Science Centre (NCN) in Poland, the Dutch Research Council (NWO) in the Netherlands, and the UK Research and Innovation (UKRI) in the United Kingdom. The full code used for the analysis and all the datasets that underpin it are openly available in Zenodo at [https://doi.org/10.5281/zenodo.12523229](https://doi.org/10.5281/zenodo.12523229).

A first analysis of the data on funded publications breaks down the figures per funder into different subgroups, pre- and post-Plan S and then classed by *Open Access flavour*. These detailed figures shown on **table 3** below provide a first insight into what the impact of Plan S may have been. It's worth bearing in mind at this point that the time-windows used to analyse the pre- and post-Plan S stages are different in their extension: the pre-Plan S sample covers 6 years (2015 to 2020) while the post-Plan S time-window runs from 2021 to 2023. While the analysis is based on average values and is thus independent of the length of the pre- and post-Plan S time-windows, a quick examination of the figures in table 3 could lead to the wrong conclusions – it might for instance feel as if Plan S had caused an abrupt drop in the number of annual publications.

The funders in the three different groups have been alphabetically sorted by funder group on **table 1** above and 3 below. In order to simplify the identification, the character and composition of the three groups of funders is reiterated here, since funders in each of these groups belong together for the purposes of the analysis.

**Treatment group:** cOAlition S funders where Plan S influence is expected to have been at its highest. These are: FWF (Austria), HHMI (US), NCN (Poland), NWO (Netherlands), UKRI (UK).
**Control group:** Funders with barely any Plan S impact. ANID/CONICYT (Chile), MBIE (New Zealand), NSFC (China).

**Comparison group:** Non-cOAlition S funders expected to have experience some Plan S impact. These include the DFG (Germany), the FWO (Flanders/Belgium) and the NCI (United States).

The National Science Foundation in China (NSFC) is by far the largest funder in the whole dataset. Within the control group, the NSFC substantially drives all results. For the treatment group, the UK Research & Innovation (UKRI), whose Plan S-aligned OA policy came into force only in April 2022, has the largest influence on the results in terms of covered publications. The largest funder in the comparison group is the German Research Foundation or Deutsche Forschungsgemeinschaft (DFG). All three groups include one funder that has funded substantially more publications than the other members of the group.

<table><caption>Table 3. Funded publications by OA flavour and funder pre- (2015-2020) and post-Plan S (2021-2023). Funder colour codes are blue for treatment group, red for control and green for comparison group</caption><thead><tr><th rowspan="2">OA Flavor</th><th colspan="2">PRE Plan S</th><th colspan="2">POST Plan S</th><th rowspan="2">Total</th><th colspan="2">PRE Plan S</th><th colspan="2">POST Plan S</th><th rowspan="2">Total</th></tr><tr><th>Number</th><th>Share</th><th>Number</th><th>Share</th><th>Number</th><th>Share</th><th>Number</th><th>Share</th></tr></thead><tbody><tr><td colspan="10" style="text-align: center; font-weight: bold;">FWF</td></tr><tr><td>bronze</td><td>931</td><td>6.0%</td><td>167</td><td>1.8%</td><td>1.098</td><td>1.540</td><td>20.8%</td><td>46</td><td>2.8%</td><td>1.586</td></tr><tr><td>gold</td><td>3.420</td><td>22.0%</td><td>2.923</td><td>32.2%</td><td>6.343</td><td>2.374</td><td>32.0%</td><td>637</td><td>38.7%</td><td>3.011</td></tr><tr><td>green</td><td>3.020</td><td>19.4%</td><td>1.424</td><td>15.7%</td><td>4.444</td><td>1.161</td><td>15.7%</td><td>162</td><td>9.8%</td><td>1.323</td></tr><tr><td>hybrid</td><td>6.132</td><td>39.4%</td><td>3.786</td><td>41.8%</td><td>9.918</td><td>1.754</td><td>23.7%</td><td>559</td><td>34.0%</td><td>2.313</td></tr><tr><td>closed</td><td>2.066</td><td>13.3%</td><td>768</td><td>8.5%</td><td>2.834</td><td>580</td><td>7.8%</td><td>241</td><td>14.7%</td><td>821</td></tr><tr><td>Total</td><td>15.569</td><td></td><td>9.068</td><td></td><td>24.637</td><td>7.409</td><td></td><td>1.645</td><td></td><td>9.054</td></tr><tr><td colspan="10" style="text-align: center; font-weight: bold;">NCN</td></tr><tr><td>bronze</td><td>944</td><td>3.7%</td><td>256</td><td>1.4%</td><td>1.200</td><td>1.070</td><td>6.0%</td><td>210</td><td>1.7%</td><td>1.280</td></tr><tr><td>gold</td><td>5.365</td><td>21.0%</td><td>6.623</td><td>36.5%</td><td>11.988</td><td>3.628</td><td>20.4%</td><td>3.186</td><td>26.3%</td><td>6.814</td></tr><tr><td>green</td><td>3.479</td><td>13.6%</td><td>2.027</td><td>11.2%</td><td>5.506</td><td>4.524</td><td>25.4%</td><td>2.156</td><td>17.8%</td><td>6.680</td></tr><tr><td>hybrid</td><td>4.976</td><td>19.5%</td><td>4.724</td><td>26.0%</td><td>9.700</td><td>6.823</td><td>38.3%</td><td>5.691</td><td>46.9%</td><td>12.514</td></tr><tr><td>closed</td><td>10.808</td><td>42.3%</td><td>4.536</td><td>25.0%</td><td>15.344</td><td>1.776</td><td>10.0%</td><td>885</td><td>7.3%</td><td>2.661</td></tr><tr><td>Total</td><td>25.572</td><td></td><td>18.166</td><td></td><td>43.738</td><td>17.821</td><td></td><td>12.128</td><td></td><td>29.949</td></tr><tr><td colspan="10" style="text-align: center; font-weight: bold;">UKRI</td></tr><tr><td>bronze</td><td>7.091</td><td>5.1%</td><td>451</td><td>1.3%</td><td>7.542</td><td>773</td><td>6.3%</td><td>271</td><td>2.9%</td><td>1.044</td></tr><tr><td>gold</td><td>31.146</td><td>22.3%</td><td>10.267</td><td>29.8%</td><td>41.413</td><td>2.553</td><td>20.8%</td><td>3.613</td><td>38.4%</td><td>6.166</td></tr><tr><td>green</td><td>40.619</td><td>29.1%</td><td>5.033</td><td>14.6%</td><td>45.652</td><td>2.195</td><td>17.9%</td><td>1.165</td><td>12.4%</td><td>3.360</td></tr><tr><td>hybrid</td><td>51.127</td><td>36.6%</td><td>15.178</td><td>44.1%</td><td>66.305</td><td>1.237</td><td>10.1%</td><td>880</td><td>9.4%</td><td>2.117</td></tr><tr><td>closed</td><td>9.726</td><td>7.0%</td><td>3.467</td><td>10.1%</td><td>13.193</td><td>5.493</td><td>44.8%</td><td>3.481</td><td>37.0%</td><td>8.974</td></tr><tr><td>Total</td><td>139.709</td><td></td><td>34.396</td><td></td><td>174.105</td><td>12.251</td><td></td><td>9.410</td><td></td><td>21.661</td></tr><tr><td colspan="10" style="text-align: center; font-weight: bold;">MBIE</td></tr><tr><td>bronze</td><td>121</td><td>5.3%</td><td>35</td><td>2.0%</td><td>156</td><td>37.075</td><td>3.4%</td><td>21.193</td><td>2.3%</td><td>58.268</td></tr><tr><td>gold</td><td>468</td><td>20.7%</td><td>513</td><td>29.4%</td><td>981</td><td>193.976</td><td>17.9%</td><td>241.686</td><td>26.5%</td><td>435.662</td></tr><tr><td>green</td><td>183</td><td>8.1%</td><td>123</td><td>7.1%</td><td>306</td><td>53.978</td><td>5.0%</td><td>41.886</td><td>4.6%</td><td>95.864</td></tr><tr><td>hybrid</td><td>246</td><td>10.9%</td><td>362</td><td>20.8%</td><td>608</td><td>42.556</td><td>3.9%</td><td>29.189</td><td>3.2%</td><td>71.745</td></tr><tr><td>closed</td><td>1.246</td><td>55.0%</td><td>711</td><td>40.8%</td><td>1.957</td><td>756.002</td><td>69.8%</td><td>576.664</td><td>63.3%</td><td>1.332.666</td></tr><tr><td>Total</td><td>2.264</td><td></td><td>1.744</td><td></td><td>4.008</td><td>1.083.587</td><td></td><td>910.618</td><td></td><td>1.994.205</td></tr><tr><td colspan="10" style="text-align: center; font-weight: bold;">DFG</td></tr><tr><td>bronze</td><td>8.104</td><td>7.1%</td><td>1.742</td><td>2.3%</td><td>9.846</td><td>965</td><td>7.6%</td><td>365</td><td>3.8%</td><td>1.330</td></tr><tr><td>gold</td><td>23.369</td><td>20.4%</td><td>24.633</td><td>31.9%</td><td>48.002</td><td>2.540</td><td>20.0%</td><td>2.783</td><td>28.9%</td><td>5.323</td></tr><tr><td>green</td><td>20.794</td><td>18.2%</td><td>11.133</td><td>14.4%</td><td>31.927</td><td>4.542</td><td>35.8%</td><td>2.468</td><td>25.6%</td><td>7.010</td></tr><tr><td>hybrid</td><td>20.096</td><td>17.6%</td><td>23.067</td><td>29.9%</td><td>43.163</td><td>1.559</td><td>12.3%</td><td>1.382</td><td>14.3%</td><td>2.941</td></tr><tr><td>closed</td><td>42.049</td><td>36.8%</td><td>16.665</td><td>21.6%</td><td>58.714</td><td>3.065</td><td>24.2%</td><td>2.646</td><td>27.4%</td><td>5.711</td></tr><tr><td>Total</td><td>114.412</td><td></td><td>77.240</td><td></td><td>191.652</td><td>12.671</td><td></td><td>9.644</td><td></td><td>22.315</td></tr><tr><td colspan="10" style="text-align: center; font-weight: bold;">NCI</td></tr><tr><td>bronze</td><td>6.705</td><td>18.4%</td><td>2.120</td><td>8.9%</td><td>8.825</td><td>65.319</td><td>4.5%</td><td>26.856</td><td>2.4%</td><td>92.175</td></tr><tr><td>gold</td><td>7.974</td><td>21.9%</td><td>8.024</td><td>33.8%</td><td>15.998</td><td>276.813</td><td>18.9%</td><td>304.888</td><td>27.5%</td><td>581.701</td></tr><tr><td>green</td><td>14.031</td><td>38.6%</td><td>5.264</td><td>22.2%</td><td>19.295</td><td>148.526</td><td>10.1%</td><td>72.841</td><td>6.6%</td><td>221.367</td></tr><tr><td>hybrid</td><td>4.867</td><td>13.4%</td><td>3.585</td><td>15.1%</td><td>8.452</td><td>141.373</td><td>9.6%</td><td>88.403</td><td>8.0%</td><td>229.776</td></tr><tr><td>closed</td><td>2.776</td><td>7.6%</td><td>4.749</td><td>20.0%</td><td>7.525</td><td>835.587</td><td>56.9%</td><td>614.813</td><td>55.5%</td><td>1.450.400</td></tr><tr><td>Total</td><td>36.353</td><td></td><td>23.742</td><td></td><td>60.095</td><td>1.467.618</td><td></td><td>1.107.801</td><td></td><td>2.575.419</td></tr></tbody></table>
### 3.3.2. Data preprocessing

Due to the high quality of the data delivered by OA.works, only slight adjustments were necessary to conduct the statistical analysis. To run the analysis and do the preprocessing, version 15.1 of the statistical software STATA was used. 236 duplicates in the data were removed. Furthermore, the date variable was reformatted, and additional binary indicator variables were constructed as explained below. To visualise the regression results, the coefficients and confidence intervals were exported in order to use ggplot2 in R for plotting the results.

Difference-in-differences methodology: technical description

To elicit the effect of Plan S, a simple difference-in-differences model is computed that compares the change in the probability of a paper being published under an Open Access licence among the “treated” cOAlition S funders compared to those publications funded by funding agencies not part of this alliance. The formula looks as follows:

$$ \pi(\text{OA}) = 1_T + 1_{\text{treat}} + 1_T * 1_{\text{treat}} + \theta_i + \varepsilon $$

On the left-hand side of the equation $\pi(\text{OA})$ is the outcome variable, i.e., whether a paper has been published Open Access. Two separate specifications are used. First, an indicator that turns 1 for any type of Open Access, i.e., Green, Hybrid, Gold, or Diamond OA. In the second narrower specification, we investigate solely Gold Open Access publications. $1_T$ is a binary indicator that captures whether a paper has been published before ($1_T=0$) or after Plan S was released ($1_T=1$). $1_{\text{treat}}$ is a binary indicator that turns 1 if a paper has been funded by an agency that is part of cOAlition S and remains zero otherwise. $1_T * 1_{\text{treat}}$ is the interaction term that, by design, only turns 1 if both factors are equal to 1. $\theta_i$ represents the funder fixed effect. It shall capture time-invariant funder-specific characteristics that may confound the analysis. It is designed as a categorical variable, which means that one reference category is omitted to avoid perfect collinearity. $\theta_i$, by construction, only applies to the pooled comparisons with several funders per group and, obviously, not to the pairwise comparisons. In that case, $\theta_i$ replaces $1_{\text{treat}}$ as the latter would be simply a linear combination of the aggregation of the vector $\theta_i$. Lastly, $\varepsilon$ is the idiosyncratic error term. The standard errors are clustered on the journal level as the observations are likely to be correlated among journals. This specification implicitly also captures publishers.

Overall, the research design described above measures changes relative to the counterfactual, i.e. if the treatment had not happened. Hence, every effect we measure is relative to the change in either the control or comparison group. Furthermore, it decouples the relative from the nominal change. For example, if the Open Access share grew by 8 percentage points in the treatment group but by 10 percentage points in the counterfactual comparison, we would have a treatment effect of -2 percentage points even though there would be a positive nominal change. The counterfactual comparison always aims to tell what could have happened (in this case: how Open Access would have evolved had Plan S not existed). It also emphasises why the careful choice of a suitable counterfactual is so important. With regard to the present analysis, several doubts have to be raised in that domain. As one can draw from **table 3** above, most Plan S funders already see high levels of OA in the research outputs they fund (even pre-Plan S, the rates of closed access publications for all funders in the treatment group except for the Polish NCN are rarely above 10%). Put differently, given the high uptake of Open Access already before the arrival of Plan S, it is quite difficult to push the uptake further, potentially more difficult than for those funders in the two counterfactual groups.
One way to allow the widest possible range of findings to arise is to interpret the present analysis as 'Plan S member funders relative to a basket of counterfactual funders'. This is specifically done in the 'pairwise comparisons' below in which a 'treated' Plan S funder is compared against a set of 'non-treated' funders, either from the control or the comparison group.
## 3.4. CIE RESULTS

The CIE results are divided into pooled and pairwise comparisons, i.e. comparisons across whole groups and across individual funders. Aggregated levels of OA are examined as well as individual OA flavours. The quantitative impact of Plan S is higher on the hybrid Open Access category and the lower the original OA rates were for publications funded by a specific research funder.

The results of the quantitative assessment of the impact of Plan S are grouped in three different sections:

* A comparison of aggregate figures per funder group for Open Access publications pre- and post-Plan S (called *pooled comparisons*)

* Pairwise comparisons between individual funders in the treatment group and every funder in both counterfactuals (control and comparison groups)

* Changes across *Open Access flavours*, in an attempt to tell where the impact of Plan S has been at its highest and whether the growth on a specific OA flavour has happened as a 'net win' (i.e. by diminishing the rates of closed access) or rather at the expense of some other OA flavour.

### 3.4.1. Pooled Comparisons

#### 3.4.1.1. All Open Access types combined

In the so-called 'pooled comparisons,' we look at the effect per group, i.e., treatment vs control and treatment vs comparison. Four panels are provided comparing the possible causal effects of the introduction of Plan S on the likelihood of papers funded by cOAlition S funders to be published in either any type of Open Access or a specific Open Access flavour relative to restricted access. For this analysis of all Open Access flavours together, we compare all Open Access publications against the whole set of publications, including closed access ones.

<figure><img src="image_7.png" /></figure>

The figure above shows the results for the aggregation of all types of Open Access (gold, hybrid, green, and bronze OA). The effects shown are relative to both the control and the comparison group.

The results show that funding agencies participating in Plan S see a less pronounced change in the likelihood of their supported publications being published under any type of Open Access. This is both the case when compared against the control group funders and the comparison group ones.
This effect is likely to be due to the already substantial adoption rate of Open Access among publications funded by cOAlition S funders before Plan S became active. Put differently, with Open Access rates for cOAlition S funders being regularly above 80% in the “pre-Plan S phase”, it is difficult if not impossible to increase the uptake of Open Access quicker than for subsets of publications whose pre-Plan S Open Access rates were much lower.

### 3.4.1.2. Pooled comparisons by Open Access type

In a second step, the different Open Access models are specifically examined. For this analysis, all Gold OA publications (upper left panel below), Hybrid OA publications (upper right panel) and Green OA publications (lower panel) are respectively compared against the whole set of publications, including the closed access ones. This is done for treatment group funders as a whole vs control group funders and comparison group funders.

<figure><img src="image_8.png" /></figure>

On the *Gold Open Access* snapshot (i.e. for publications in fully OA titles) shown on the upper left panel, no change is observed with regard to the control group but a weaker performance (on the 95% significance level) is identified relative to the comparison group. Expressed simply, this suggests that the rates of Gold OA publications funded by comparison group funders (DFG, FWO, NCI) have grown quicker than those funded by cOAlition S funders. Individual funder comparisons in the ‘pairwise comparisons’ section below shed some light on whether these differences may be attributed to the performance by specific funders in either of the groups.
On the **hybrid OA** panel (upper right), Plan S funders show a significant positive change relative to the control group but a negative effect relative to the comparison group (consisting of the DFG, the FWO, and the NCI). With the same caveats mentioned in previous paragraphs, this means that hybrid OA has grown more significantly among cOAlition S funders than among funders in the control group. However, the rate of growth in hybrid OA is larger for funders in the comparison group, who are also running transformative agreements that increase the levels of hybrid OA for their funded publications.

As explained above, the German DFG largely drives the behaviour of the comparison group and given the widespread availability of transformative agreements with major publishers reached within the DEAL project in Germany [12], this contributes to explaining this result.

With respect to *Green Open Access*, both counterfactual groups had a better development relative to the cOAlition S funders included in this study. It should be noted that a Gold or Hybrid Open Access status “overwrites” any previous Green OA classification in data sources like OpenAlex. This means that even if existing copies of accepted manuscript were available in a repository, the publication would still be labeled solely as Gold (or Hybrid) Open Access. The decrease in Green OA with regard to the control group is likely to be linked to this phenomenon.

### 3.4.2. Pairwise Comparisons

In this section of the counterfactual impact evaluation, we look at differences between funder pairs. In other words, one-to-one comparisons are separately conducted between every funder from the treatment group (cOAlition S members) and each funder from both counterfactuals, the control group and the comparison group. We do this for several outcome variables:

* All OA types

* Only Gold OA

* Only Hybrid OA

* Only Green OA

Same as for the pooled comparisons above, we first compare all OA types together, then individual OA types against all funded publications. Two panels are provided for each category, showing the 1:1 treatment vs control group funder and the 1:1 treatment vs comparison group funder benchmark.

#### 3.4.2.1. All OA flavours together

<figure><img src="image_9.png" /></figure>
In a first step, we look at the likelihood of all Open Access types jointly. Each colour represents one cOAlition S funder that is compared to each funder of either the control or the comparison group, which is listed on the horizontal axis. The vertical axis depicts the effect size and the 95 and 99% confidence intervals in which the point estimates are located. The figure above shows the results relative to the control group funders: a positive effect for global rates of OA is only evident for the Polish funding agency NCN. This is also the only cOAlition S funder in this study that had a significant share of closed access publications before Plan S became active.

Second, we run the aggregated OA regression also for the comparison group. On the figure below we observe that it's mainly the German funder DFG that drives the small but significantly negative pooled effect identified in the previous section for pooled comparisons. In other words, DFG-funded publications have a much higher growth in Open Access rates, causing the comparison between cOAlition S funders and the DFG to become severely negative. In contrast, several treatment group funders (FWF, NCN, NWO and the UKRI) display notable positive effects relative to the other two comparison group funders FWO in Flanders and the NCI in the United States.

<figure><img src="image_10.png" /></figure>

### 3.4.2.2. Gold OA

<figure><img src="image_11.png" /></figure>

The Gold OA-only analysis shows again that only the Polish NCN has positive effects relative to all three control group funders. On the other hand, HHMI and UKRI have weaker changes than the control group funders, presumably as a result of higher Gold OA rates pre-Plan S. For the Austrian FWF and the Dutch NWO, the findings are mixed.
When conducting pairwise comparisons for Gold OA against comparison group funders, the figure below shows again the DFG pushing the overall effect downwards. In contrast, most cOAlition S funders outperform the US National Cancer Institute (NCI) and the Flemish funder FWO. Thus, the uptake of ‘pure’ gold open access was stronger among the cOAlition S-funded publications relative to publications funded by the mentioned agencies in the comparison group.

<figure><img src="image_12.png" /></figure>

### 3.4.2.3. Hybrid OA

Both the table showing the pre- and post-Plan S figures (table 3) and the pooled comparisons above showed that the highest impact of Plan S has been on the likelihood of funded papers being published Hybrid Open Access. This is also reflected on the pairwise comparisons below. The comparison of treatment vs control group funders shows generalised positive effects except for the HHMI, whose wide confidence intervals reveal a lack of statistical power. Furthermore, null effects arise for the comparison of all treatment group funders save the FWF against the MBIE funder in New Zealand (the FWF-MBIE comparison shows a negative effect). Put differently, the cOAlition S funders FWF, NCN, NWO, and UKRI outperformed the control group funders ANID and NSFC for Hybrid OA.

<figure><img src="image_13.png" /></figure>

The same analysis against the comparison group funders shows a similar pattern as above. The HHMI shows equally large error bars as before. Notably, the German funder DFG outperformed all cOAlition S funders in the treatment group in terms of shifting their funded publications to Hybrid Open Access. As already mentioned in the analysis of the results of the pooled comparisons for Hybrid OA, this is likely to be driven by the transformative DEAL agreements negotiated between the Alliance of German
research institutions and a number of large publishers [12]. On the other hand, there are positive effects for treatment group funders NCN, NWO, and UKRI relative to the FWO and the NCI.

<figure><img src="image_14.png" /></figure>

### 3.4.2.4. Green OA

The last Open Access model we individually investigated was Green Open Access. On the 1:1 treatment vs control group funder comparison we find a mix of null and negative effects, with the significant exception of the NCN when compared to ANID/CONICYT. Regarding the control group funders MBIE (New Zealand) and NSFC (China), we see mostly negative and occasionally null effects. Put differently, from a Green OA-only perspective, cOAlition S funders underperformed with regard to those in the control group.

<figure><img src="image_15.png" /></figure>

When using the comparison group as alternative counterfactual, a quite diverse picture is also on display. Relative to the German funder DFG, the effect is either null (for FWF, HHMI and NCN) or negative (for the NWO and the UKRI). However, when compared to the US National Cancer Institute (NCI), the effect among treatment group funders is largely positive, with the sole exception of UKRI. Thus, there would seem to be a trend to shift away from Green OA relative to the counterfactual funders, but the situation is quite diverse.

It's worth again noting here that the negative effect on Green Open Access when measuring the NWO and the UKRI against the DFG happens in a context (shown in **table 3**) where both cOAlition S funders have the highest rates of Green OA pre-Plan S among all 11 funders under analysis. This again highlights
the influence of the initial conditions on the outcome of the assessment – the higher the pre-Plan S uptake for a given OA model, the higher the probability that the analysis will result in negative effects against funders with lower pre-Plan S uptakes.

<figure><img src="image_16.png" /></figure>

### 3.4.3. Changes across Open Access models

In the last section of this counterfactual analysis, we look at changes between the various Open Access models. The previous analysis was based on comparing the different OA models against all funded publications open or closed. In this step, all closed access publications are excluded from the regressions, and we compare one specific Open Access model (gold, hybrid, green) against the combination of the others. All funders in the various groups (treatment, control and comparison) are jointly processed, i.e. we are again examining pooled comparisons.

<figure><img src="image_17.png" /></figure>

The plot above shows that on the basis of OA publications only, there is no significant change in the relative Open Access likelihoods by OA model between funders in the treatment and the comparison groups. For the control group, however, a positive effect is evident for Hybrid Open Access, as well as a negative one for Green Open Access. This means that Open Access publications supported by cOAlition S funders have a higher tendency to appear in hybrid journals (potentially due to the impact of transformative agreements) and are less likely to be classed as Green OA.
This effect, which has also been documented in previous works [13], suggests a stronger focus on
immediate commercial publishing relative to the control group.

**3.5. CONCLUSIONS**

The quantitative analysis of the impact of Plan S as measured by this CIE does not provide conclusive evidence on its positive effect in terms of globally increasing the Open Access availability of funded publications by cOAlition S funders. On the contrary, the CIE figures suggest that the rate of Open Access growth measured for funders included in the treatment group (i.e. those with Plan S-aligned OA policies) has been somewhat slower than the rates for funders in the two counterfactuals (i.e. the control and the comparison groups). As explained in the “limitations of the study” section above, several reasons may explain this apparent lack of a generalised quantitative effect, the main one being that it’s still too early to adequately assess the impact of Plan S on a quantitative basis alone.

The fact that the complementary qualitative assessment of Plan S conducted within this study via interviews with a wide range of stakeholders has often yielded very positive evaluations of its impact in terms of – among others – firmly placing the topic of Open Access on the policymakers’ radar and attracting publishers to the negotiating table provides a counterbalance to the results of the quantitative analysis.

Some of the CIE outcomes could be termed “negative results”. These are summarised in the first few bullet items below. On top of these there are however certain aspects where the CIE results deliver insights worth highlighting:

1. Global Open Access rates for cOAlition S funders **have not increased as significantly** as they have for their counterfactuals in either the control or the comparison groups. This doesn't mean there hasn't been any progress – all five cOAlition S funders included in the treatment group have seen increases in their Gold and Hybrid OA shares following the release of Plan S – but that the comparison against non-cOAlition S funders with often much more modest initial values in terms of rates of Open Access publications has yielded lower growth rates for cOAlition S funders.

2. The **one clear exception to this trend is the Polish National Science Centre (NCN)**. The CIE results for the NCN show a distinct impact of Plan S on its rates of funded publications published Open Access, both as a whole and broken down by OA flavours, when compared to funders in the control and the comparison groups. Considering that the rate of closed NCN-funded publications pre-Plan S was 42.3% and has come down to 25%, this suggests that Plan S has had a much clearer impact – from a quantitative perspective alone – the lower the Open Access rates were for specific funders before its introduction.

3. When examining the CIE results for the different Open Access models, the most statistically significant effect of Plan S seems to have been the **increase in the rates of Hybrid OA**. The CIE analysis focused on Hybrid OA under ‘pooled comparisons’ in section 3.4.1.2 above is the only one where the treatment group funders outperform the control group ones. This is likely to be the consequence of transformative arrangements, whose adoption often overlaps with the arrival of Plan S. A parallel influence in this regard is exerted by the OA2020 initiative and it’s not possible to ascertain which fraction of the increase in the rates of Hybrid OA may be due to Plan S vs OA2020, especially when both initiatives have been working together [10], but Plan S has clearly had an influence.

4. The outcome of the quantitative analysis is severely affected by the rate of Open Access availability of publications funded by cOAlition S funders **before Plan S was released**. Because most funders who have joined cOAlition S already had high or very high level of Open Access compliance, any analysis based on the comparison between the growth of Open Access rates between cOAlition S
funders and their counterfactuals (be it from the 'control group' with barely any or no Plan S footprint or from the 'comparison group') will mask the progress towards immediate Open Access among the former. This is because it's much harder to raise Open Access rates from 85 to 90% than it is to raise these from for instance 45 to 50%. As a result, the impact of Plan S that can be measured by applying this methodology is inevitably underestimated. The quantitative assessment of the impact on other immediate Open Access routes that are either specific to Plan S (rights retention) or strongly supported by cOAlition S (Diamond OA) [14] would provide an additional level of nuance to the results.

5. Diamond Open Access has not been specifically monitored within this CIE for two main reasons. First, the Diamond OA rates among cOAlition S-funded publications are very low at present. On top of that, there isn't yet a consolidated strategy to measure such rates, which are also extremely difficult to capture with a sufficient level of accuracy. However, some early attempts at capturing those Diamond OA rates are starting to emerge [15] and, while it's still too early to assess the impact that the Diamond OA support strategy in the Plan S toolbox may have going forward, any impact in this area could at least partially be ascribed to Plan S.

6. It should also be very interesting to monitor the uptake of **rights retention policies** and their impact on cOAlition S-funded publications as a subset of Green OA. Same as for Diamond OA, it's still too early to capture – or even to adequately measure [16] – a rate of immediate Green OA that remains very low at present, but with so many mainstream research funders supporting rights retention policies and the onset of institutional, even national policies, this area is one to watch in future years. It is then fair to expect that a re-running in a few years' time of exactly the same analysis conducted for this CIE of Plan S would yield much clearer impact patterns for this policy initiative.

## REFERENCES

[1] cOAlition S (2019). "A study to assess the impact of Plan S on the global scholarly communication ecosystem: an Invitation to Tender (ITT)". https://www.coalition-s.org/a-study-to-assess-the-impact-of-plan-s-on-the-global-scholarly-communication-ecosystem-invitation-to-tender/

[2] cOAlition S (2018). Plan S Principles. https://www.coalition-s.org/plan_s_principles/

[3] Wikipedia. Difference in differences, https://en.wikipedia.org/wiki/Difference_in_difference

[4] See for instance: Lenhart, Otto and Chakraborty, Kalyan (2024) Income assistance programs and population health: the dual impact of minimum wages and the earned income tax credit. Economics Letters, 234. 111508.
https://doi.org/10.1016/j.econlet.2023.111508

[5] Implementation Roadmap of cOAlition S Organisations, https://www.coalition-s.org/plan-s-funders-implementation/

[6] Justus Haucap, Nima Moshgbar, W. Benedikt Schmal (2021). The impact of the German 'DEAL' on competition in the academic publishing market. Managerial and Decision Economics. https://doi.org/10.1002/mde.3493

[7] Wikipedia. Agencia Nacional de Investigación y Desarrollo de Chile. https://es.wikipedia.org/wiki/Agencia_Nacional_de_Investigaci\%C3\%B3n_y_Desarrollo_de_Chile

[8] EU-funded PASTEUR40A FP7 project (Open Access Policy Alignment STrategies for European Union Research) at https://cordis.europa.eu/project/id/611742

[9] ESAC Transformative Agreement Registry, https://esac-initiative.org/about/transformative-agreements/agreement-registry/

[10] cOAlition S (2019). OA2020 and cOAlition S Launch Joint Statement. https://www.coalition-s.org/oa2020-and-coalition-s/

[11] Pablo de Castro (2024). Choosing a data sample provider for our study on the impact of Plan S. scidecode blog. https://scidecode.com/2024/01/22/choosing-a-data-sample-provider-for-our-study-on-the-impact-of-plan-s/

[12] Michael Mills (2020). Project DEAL — Germany is leading the way in the transition to open access publishing. J Appl Clin Med Phys. 21(5): 4–5. https://doi.org/10.1002/acm2.12890

[13] Kira Brayman, Amy Devenney, Helen Dobson, Mafalda Marques, Anna Vernon (2024). "A review of transitional agreements in the UK". Jisc report. https://doi.org/10.5281/zenodo.10787391
[14] coalition S (2023). "Global Summit on Diamond Open Access concludes with pledges to advance Diamond OA initiatives globally". https://www.coalition-s.org/global-summit-on-diamond-open-access-concludes-with-pledges-to-advance-diamond-oa-initiatives-globally/

[15] The all-time oa.report dashboard produced (in beta) for the Bill and Melinda Gates Foundation (BMGF) is able to capture the rate of Diamond OA articles among the BMGF-funded publications. It currently amounts to a meagre **1.2% of the total** (but is higher than the number of closed articles). At the time of writing, this feature is not yet shown on the publicly available BMGF dashboard at https://oa.report/bill-and-melinda-gates-foundation/?start=1980-01-01&end=2024-05-23

[16] Pablo de Castro (2024). "Improving CRIS features to support new Open Access implementation workflows at institutions". 16th International Conference on Current Research Information Systems (CRIS 2024). Procedia Computer Science (in press).
## 4. QUALITATIVE ANALYSIS: SOME PERSPECTIVES

In order to address the qualitative assessment of Plan S and its impact, this study has engaged with top experts among the wide range of stakeholders involved in the scholarly communications landscape. This qualitative analysis complements and enriches the CIE conducted in the previous chapter. These interviews, conversations and other engagement activities (see annex 2 for more detail on all these) have resulted in a whole set of perspectives and insights that are summarised in this chapter. A significant amount of desk research and the wide experience of the authors of the study in the Open Access landscape and how it has evolved and keeps evolving in different countries have also contributed to this section of the report.

The structure for this qualitative analysis of the impact of Plan S follows the various policy instruments that sit at the core of Plan S – such as *transformative arrangements, rights retention policies* and *fully Gold Open Access*, the latter one including *Diamond Open Access*. A section is also devoted to parallel areas of Open Access practice such as *preprints* and *long-form publications* that are relevant to its implementation and impact (even if preprints are not strictly part of Plan S). More general topics such as Open Access and the Global South and Publishers and Plan S are addressed in additional sections. This chapter also includes a proposal for a more balanced distribution of Open Access costs within institutions that may allow available library budgets to gradually be shifted and reinvested into more affordable, sustainable and equitable research publishing models.

### 4.1. TRANSFORMATIVE AGREEMENTS

Read and Publish agreements are widely perceived to be the main instrument in the Plan S toolbox. These mechanisms to support the transition to a full and immediate Open Access landscape offer undeniable advantages for institutions and publishers, but also face strong criticism. This study has found that these agreements have been key in bringing publishers to the negotiating table with institutions and their consortia, whose negotiating teams have become more diverse and influential. TAs have also enhanced the level of international collaboration across consortia in different countries. These trends were already happening before the arrival of Plan S, but have been significantly reinforced.

*Transformative agreements (TAs) and *transformative journals (TJs)* are one of the routes to achieve full, immediate Open Access that Plan S proposed upon its release in September 2018 grouped under the term "transformative arrangements". The 'transformative' term in these names indicates that these instruments are expected to usher a transformation in scholarly publishing business models towards fully Open Access options. Given how unlikely that transformation seems at the time of writing, many prefer the term *Read & Publish agreements* to that of TAs. This section is focused on the analysis of Read & Publish agreements, although some space will also be devoted to transformative journals. For simplicity purposes, the TA term will be used throughout.

cOAlition S never prioritised TAs over the other two routes to immediate Open Access defined by Plan S – fully Gold Open Access and rights retention policies. However, TAs were widely seen and as the most
disruptive mechanism within the Plan S toolbox upon its release, partly because APC payments for fully
Open Access journals were already standard practice and the immediate Green OA route via rights
retention took some time to be unequivocally defined. Plan S-promoted TAs are well aligned with the
developments that were already taking place in the OA landscape before Plan S arrived. The OA2020
initiative had been promoting them for some time – including their registration into the ESAC directory
– as an instrument to flip the scholarly publishing landscape to a fully Open Access business model.
Having the funders joining this attempt was a significant and welcome move [1]. As the quantitative
analysis in the previous chapter has shown, TAs are also the individual strategy in the Plan S toolbox
that has had the largest impact thus far in boosting the levels of Hybrid Open Access.

The way these agreements are being addressed varies strongly across countries: in some of them – usually in Central and Northern Europe – TAs are widespread and working well as a mechanism to increase the levels of immediate Open Access. There are few signs however that the promised transitions to a fully Open Access publishing landscape may happen anytime soon. Additional issues raised by these instruments such as the high costs, the publishing market concentration and the entrenchment of large commercial publishers in it are creating divisions in the Open Access landscape. Countries like Germany – whose main national funder is not a member of cOAlition S – or Austria, the Netherlands and the United Kingdom have seen a wealth of TAs offering unlimited OA. In Sweden however, the Bibsam Consortium and the University Rectors are already advocating for a move away from these instruments [2].

The review of transitional agreements in the UK published by the Jisc earlier this year [3] found out that at the current pace it would take 70 years for the very large commercial publishers to complete the transition to a fully Open Access publishing model that this strategy aims to secure.

At the same time, both researchers and many
institutional Open Access practitioners speak
positively on the role TAs can play to deliver full
and immediate Open Access. The former – who
are obviously not paying for them – tend to
consider these as the most effective and smooth
way for their publications to be published Gold
Open Access by default. In the course of a hybrid
workshop on “Financial implications of the
transition towards a fully Open Access landscape”
organized by the CESAER Association of
Universities of Science and Technology in April
2024 [4], one researcher advocated for EU-level
transformative agreements as a mechanism to
level the playing field across European countries.

TAs have also bridged the gap between ‘funded’ and ‘unfunded’ researchers and departments – meaning that the discretionary Open Access funding eligibility that previously applied just to academics involved in projects funded by those research funders (often cOAlition S members) that made available specific funding for OA publishing has now been extended to all scholars and departments. This has significantly widened the embracing of Open Access across disciplines, particularly in the Humanities and Social Sciences who were as a rule not eligible for APC funding.

<figure><img src="image_18.png" /></figure>

"We're talking about transformative agreements like they're one thing but it's a broad spectrum. On the one hand we have cost neutral transformative agreements – we see an eight percent reduction in our overall total cost of ownership but we get the subscriptions and we get hybrid open access thrown in for free. That type of TA I think we know that's going to be around for the long term. It's not a simple question of will TAs be here in the future but what type of transformative agreement"

(Institutional Open Access lead)
Institutional Open Access teams for their part see an evident improvement in both financial and technical workflows. On the economic front, moving away from a model where hundreds of individual APC payments would need to be made to a single annual invoice is a clear plus.

Technically, TAs also result in a significant streamlining of the workflows in several ways:

* Much smoother *processing of Gold OA requests*, both for researchers and for institutions. The introduction of publisher dashboards has simplified the process of having research works published Open Access and TAs have more generally empowered libraries in their communications with authors;

* Institutions are notified *upon manuscript acceptance* or, depending on the publisher, upon the author's request for Gold OA. This is particularly useful for institutional Open Access support services to be able to assist researchers in making sure the correct funding acknowledgements and data availability statements are on the paper in accordance with the policies issued by research funders;

* Notifications received by institutions usually contain a wide range of very useful metadata, occasionally even the full-text accepted manuscripts;

* An increasing number of authors, especially ECRs, are requesting advice from Open Access support teams on *where to publish* to be able to secure Gold Open Access at no cost for them. Right now, the guidance largely involves directing them to the suitable hybrid journal covered by a TA, but the trust relationship built between researchers and their Open Access support teams could lead to opportunities for the latter to eventually guide the submissions to more sustainable publishing options.

Were it not for the vendor lock-in side-effect and the unsustainable costs associated with running so many TAs in parallel, these advantages would make institutional Open Access support services as supportive of these mechanisms as researchers themselves tend to be.

Transformative agreements also raise issues around geographic equity. Interviewees in countries where research is not that well-funded (not just in the Global South but also in Southern Europe) tend to operate in an environment of limited OA agreements with a much smaller number of publishers. As a result of these issues, TA are widely seen as instruments for entrenching inequality. Still, as the agreements established by the Consorcio Colombia prove [5], it is often more affordable to negotiate a TA with a publisher than to pay for the myriad APCs that would be required to make those very same articles in hybrid journals Gold Open Access.

The streamlining of the Open Access implementation workflows and the visibilisation of payments that TAs make possible is even more acutely perceived in Global South countries. Researchers in these countries are often paying APCs from their own pocket to have these expenses offset by the bonuses their institutions offer for publications in top journals. These APC payments in the wild subsequently leave little or no trace in institutional systems because research libraries are not empowered to undertake such support tasks. TAs bring a degree of centralisation to these workflows that make these transactions much more accountable – in a trend likely to be extended to APC payments in fully Open Access journals.

Some of these aspects are addressed in section 4.5 devoted to the impact of Plan S beyond its strict sphere of influence. One finding from this study in this regard is that the worldwide publishing landscape is too complex and too different across countries and regions for a single approach to deliver for everyone, but that there is value in TAs so long as not all eggs are placed in the same basket [6].
Moreover, although coAlition S announced in January 2023 that it would cease to provide financial support for transformative arrangements by the end of 2024 [7], this mechanism is now far more widespread than just in the countries with funder representation in coAlition S. Interviewees outside the coAlition S countries regularly state that they are closely following the signals produced by this group of funders, but in many countries TAs are being directly funded by universities because they are seen as a good service to researchers and to research more generally. This means that the funding structures for TAs are likely to remain in place no matter what the research funders may say. Interestingly, this potentially leads to a fragmentation along national borders whereby each country chooses its own path – its own mix of models – depending on the balance of power across the various stakeholders involved in the task of charting the way forward.

As shown when comparing the views on TAs from different European countries above, this fragmentation is arguably already happening. It could also risk triggering a competition for research impact across countries that could lead to their picking Open Access models that may boost it regardless of the concerns about cost and sustainability. Hence the suggestion in the recommendations section to identify the appropriate mechanisms to assess the value of specific Read & Publish agreements and to drop those which don't meet the requirements.

An interesting pattern has emerged in the numerous interviews that this project has conducted with institutional Open Access experts whereby the closer the interviewee sits to researchers in their daily activity, the more favourable their views on TAs tend to be. It's however university consortia who have been decisively empowered by the onset of these agreements, in a trend that again predates the release of Plan S. Both the intranational communication channels between institutions and their consortia and the international communication channels across consortia in different countries have seen a remarkable boost in recent years [8]. The impact of Plan S in this regard has been very significant due to the influence of the funders in the scholarly communications landscape. When asked what they thought the main impact of Plan S had been, representatives of university consortia interviewed for this study invariably said, “Plan S has brought publishers to the negotiating table”. This has in turn been retro fed by the impact of Plan S reported by institutions in “placing the topic of Open Access high on the agenda of the University management”, leading to the negotiating teams at consortia becoming far more diverse and influential.

The potential that the consolidation of these workflows offers going forward for assessing the value of specific TAs according to criteria defined by the institutions themselves in collaboration with the consortia cannot be overstated. One specific aspect where this is already happening is the embedding into TAs other strategies to achieve immediate OA (notably immediate Green OA via rights retention clauses, see the section devoted to rights retention below). There are strong differences in the negotiating power of university consortia across countries, but the international coordination channels should allow the gradual spreading of this practice.

<figure><img src="image_19.png" /></figure>

"I don't think by any means [that the Open Access battle is over and the battle is lost]. These things happen a lot slower than people think. And the absolute behemoth that is university consortia procurement processes is only just beginning to wake up. I think that in terms of the shift to being able to define what we need on Academia's terms, there is still everything to play for"

(Open Science Expert)
A note on transformative journals

The complementary route to securing immediate (Hybrid) Open Access that transformative journals
offer has received less attention from institutional Open Access support services than TAs. This is both
because the list of titles is much shorter and because the TJ model still involves the payment of an APC
within an institutional framework that saw “no hybrid” policies being widely adopted before Plan S
arrived. As opposed to this, the more publications secure “free” Gold Open Access<fnref n="8" /> via an (unlimited)
Read & Publish agreement, the more justified the investment becomes that the institution has made in
it. However, in cases where TAs have been dropped [9], transformative journals have become the sole
mechanism for institutions to be able to offer Gold Open Access (in fact Hybrid OA) to publications
funded by cOAlition S funders. Given that transformative journals will also cease to be supported at the
end of 2024, immediate Green Open Access via rights retention seems set to become the default
approach whenever TAs are dropped beyond the end of 2024.

REFERENCES

[1] cOAlition S (2019). "OA2020 and cOAlition S Launch Joint Statement". https://www.coalition-s.org/oa2020-and-coalition-s/

[2] Association of Swedish Higher Education Institutions. "Beyond transformative agreements" Working Group (2023). "Charting Sweden's path beyond transformative agreements – analysis and proposals for strategic direction". https://suhf.se/app/uploads/2023/10/Charting-Swedens-path-beyond-transformative-agreements-%E2%80%93-analysis-and-proposals-for-strategic-direction-October-2023.pdf

[3] Kira Brayman, Amy Devenney, Helen Dobson, Mafalda Marques, Anna Vernon (2024). "A review of transitional agreements in the UK". Jisc report. https://doi.org/10.5281/zenodo.10787391

[4] CESAER (2024). Workshop on the financial implications of the transition to Open Access (Aalto University Helsinki and online, 17 April 2024). The by-invitation-only workshop was held within the Spring meeting of the CESAER Task Force Openness of Science and Technology (TFOST), https://www.cesaer.org/task-forces/task-force/?id=11825

[5] Consorcio Colombia (2024). Acuerdos transformativos. https://www.consortiocolombia.co/acuerdos-transformativos/

[6] W Benedikt Schmal (2024). "The 'Must Stock' Challenge in Academic Publishing: Pricing Implications of Transformative Agreements". ArXiv preprint. https://doi.org/10.48550/arXiv.2403.03597

[7] cOAlition S (2023). “cOAlition S confirms the end of its financial support for Open Access publishing under transformative arrangements after 2024”. cOAlition S blog, https://www.coalition-s.org/coalition-s-confirms-the-end-of-its-financial-support-for-open-access-publishing-under-transformative-arrangements-after-2024/

[8] International Coalition of Library Consortia (ICOLC). Participating Consortia, https://icolc.net/participating-consortia

[9] FinELib "Agreement between FinELib and ACS not renewed", https://finelib.fi/agreement-between-finelib-and-acs-not-renewed/

<footnote n="8">⁸ The brand of Open Access offered by Read & Publish agreements was originally only Hybrid OA, but the sort of tweaks to TAs suggested in this study have resulted in an increasing coverage for fully Gold Open Access titles too – with the subsequent lightening in the payment workflows that will often no longer require APCs. The price tag for the agreement is of course higher when it includes fully OA journals.</footnote>
## 4.2. RIGHTS RETENTION STRATEGY

Rights retention policies, introduced by cOAlition S to enable immediate Green Open Access to their funded outputs, have a considerable potential and have been very well received by the institutional Open Access community. Some institutions have in fact built on the Plan S rights retention policy and expanded it into their Institutional Rights Retention Policies. This section examines these developments together with the nascent efforts to adopt these policies more widely.

When Plan S was first announced in September 2018, repositories and *Green Open Access* were the least developed part of the roadmap to achieve full, immediate Open Access. This was however promptly addressed with the introduction of the *rights retention strategy (RRS)* as an additional mechanism in the Plan S toolbox [1]. From the start of 2021, Plan S-aligned Open Access policies issued by cOAlition S funders included this rights retention strategy as an alternative route to compliance when Gold OA was not feasible.

In line with the first Plan S principle, rights retention involves the retention of copyright by authors of research outputs by means of an open licence such as the *Creative Commons licence* or equivalent, allowing immediate sharing and reuse. This allows the *accepted author manuscripts (AAMs)* to be made openly available via institutional repositories with no embargo period and an open licence (typically a CC BY). cOAlition S funders provided grantees with a standard statement to be added to their submissions – the rights retention statement – that would allow authors of publications subject to the Plan S-aligned OA policy to notify publishers upon submission that the AAM will be made immediately Open Access via the Green OA route.

Given that this mechanism was first introduced by the frontrunner cOAlition S funders in 2021 and applied to manuscripts submitted on or after the date the policy was implemented (see annex 3), it's too early to quantitatively assess the impact of this specific Plan S tool.

The key development around the Plan S-fostered Rights Retention Strategy is the expansion from funder-mandated rights retention policies to *Institutional Rights Retention Policies (IRRPs)*. This is an increasingly frequent move by institutions subject to cOAlition S funder policies supporting rights retention to expand their application of rights retention to all their research outputs. These IRRPs were pioneered by the University of Edinburgh with their Research Publications & Copyright Policy (2021) which entered into force as of 1 January 2022 [2]. This is one year after the Wellcome Trust had introduced rights retention into their Plan S-aligned OA policy and four months *before* the UK Research and Innovation (UKRI), the largest public research funder in the United Kingdom, followed suit. Over thirty universities in the UK

have already developed their own IRRP in the two and a half years since the University of Edinburgh made its move [3], most of them research-intensive HEIs, many of them in partnership with other institutions in their region. The extraordinary rate of adoption of Plan S-aligned IRRPs in the UK is the

<figure><img src="image_20.png" /></figure>

> "What Coalition S has done for us is to clearly articulate the direction of movement that research funders want to take. And this has allowed us to piggyback on that locally at our institution and say "this is what the funders are doing. This is what they're expecting. This is where we're going. The institution needs to align with the research funders. We need to have a rights retention strategy in place". And that argument allowed us to bring the senior management team on board. And so that was key. If I had to pick one of those four instruments in the Plan S toolbox [transformative arrangements, APC-based Gold, Diamond OA or rights retention], it'd be rights retention in Green OA"

*(Open Science Expert)*
direct consequence of a permissive copyright legislation and of the extensive work previously done around the UK Scholarly Communications Licence (UKSCL) [4], whose rationale was very much aligned with rights retention but was strongly contested at the time. It took Plan S and cOAlition S funders to back a similar approach for the UKSCL effort to revive and for IRRPs to start quickly spreading across universities in the UK.

A strong cross-institutional coordination activity is also taking place in this area, with the University of Edinburgh's advocates having conducted an extensive dissemination effort both at home and abroad [5]. Besides organising regular webinars for institutions to explain their paths to and results of their rights retention policies, the Jisc national consortium in the UK have recently launched a Task and Finish Group devoted to IRRPs [6].

Institutions in most of the countries with a cOAlition S funder may already apply rights retention and deposit embargo-free AAMs in their repositories since that funder adopted a Plan S-aligned Open Access policy. And many of them are doing so [7]. They could also replicate the move to widely adopt institutional rights retention policies by starting with the limited application of rights retention enabled for the Plan S-aligned Open Access policy issued by the national funder. Because the European Commission also has a Plan S-aligned Open Access policy, this would at the very least allow rights retention to be applied to EU-funded publications regardless of how flexible the national copyright framework may be in the corresponding author's country.

The widespread adoption of institutional rights retention policies provide university consortia with additional power in the negotiations with publishers [8]: not only can the consortia include a rights retention clause into a given Read & Publish agreement (the default wording of such a clause promoted by the Jisc in the UK is shown in the figure below), but the widespread availability of embargo-free full-text accepted manuscripts is a strong card in their hands during negotiations.

<figure><img src="image_21.png" /><figcaption>**Figure 2.** Rights retention addressed at the Dec 2023 “Insight into research negotiations” Jisc workshop</figcaption></figure>

Some initiatives currently exploring how to best address the challenges posed by the application of rights retention from a copyright perspective are:

* The National Open Research Forum (NORF)-funded 'Secondary rights, Copyright, Open access, Institutional policies, and Rights retention' (SCOIR) project in Ireland (2023-2025) [9]
* The Knowledge Rights 21 (KR21) programme supported by the Arcadia Foundation, in partnership with IFLA, LIBER and SPARC Europe, whose aim is to "bring about changes in legislation and practice across Europe that will strengthen the right of all to knowledge" [10]

A recommendation arising from the analysis of the interviews with experts is for the Open Access community to consider the organisation of an international rights retention conference that allows institutions from different countries to explain how they are addressing both the legal challenges and the implementation of these IRRPs. A specific strand in a big library conference such as LIBER's could also be a suitable way to address this need for international coordination<fnref n="9" />.

Some topics should specifically be on the radar for such an international discussion:

1. **Green Open Access workflows.** For these rights retention policies to be technically feasible, there needs to be a strong Green Open Access culture at the institution.

This means that workflows must be in place to enable a widespread collection of full-text accepted manuscripts before such policies can be enacted (see an example in [section 2.4](#)). This is already the case in most countries represented in cOAlition S but it's not happening worldwide.

2. **IRRP monitoring.** A discussion needs to take place on the monitoring of the embargo-free publications arising from the adoption of IRRPs. Early attempts at monitoring the uptake of rights retention have systematically relied on the identification of the rights retention statement wording in openly available manuscripts via text mining [12]. This generic attempt at capturing rights retention instances overestimates the number of cases, given that many researchers, unsure of the actual funder requirements regarding this policy mechanism, include the rights retention statement on any publication of theirs that carries the sort of funding acknowledgements that might call for such wording. As a result, the majority of the outputs listed in the github reference above were actually published Gold OA, either via APC payments for fully OA titles or via transformative agreements. However, institutional monitoring workflows based on institutional systems like CRIS may offer a way to more accurately ascertain the uptake of rights retention policies [13].

3. **Adoption of rights retention policies without piggybacking on Plan S.** The process for adopting IRRPs described in this section involves the expansion to all publications of a previous rights retention policy issued by a cOAlition S funder that applies to a specific institution. However, it is also possible for institutions - like the University of Harvard did in 2008 - to *directly* adopt IRRPs without building on top of any previous cOAlition S funder policy (this would mainly apply in countries with no representation in cOAlition S). Recent guidance by EIFL [14] suggests this mechanism could already be in application in countries where this international organisation works. In addition, some countries whose cOAlition S funder has not included rights retention in their Plan S-aligned Open Access policy are trying to enable this embargo-free Green Open Access route by pushing for updated secondary publishing rights legislation.

<figure><img src="image_22.png" /></figure>

"I don't think publishers want confrontation so we're being a bit antagonistic, but then we're also saying we're still going to continue paying so there's this kind of unwritten agreement – we're saying we'll continue our customer relationship with you but we're going to also have our rights retention"

(Institutional Open Access lead)

<footnote n="9">⁹ The June 3-6 Open Repositories 2024 Conference in Gothenburg inspiringly included the June 5th "Rights Retention and Repositories: Accelerating Global Progress" panel discussion, but the approach was largely repository-focused and somewhat UK-centric.</footnote>
4. An exploration of Secondary Publishing Rights (SPRs) copyright legislation. As opposed to the permissive copyright legislation in the UK mentioned above, the current national copyright legislation in other countries makes the adoption of rights retention policies for immediate Green OA difficult or even unfeasible. In these cases, Secondary Publishing Rights may provide an alternative way forward to internationally harmonise the approach to national copyright legislation [15].

REFERENCES

[1] cOAlition S (2020). "Plan S: Rights Retention Strategy" [factsheet]. https://www.coalition-s.org/wp-content/uploads/2020/07/RightsRetentionStrategy.pdf

[2] University of Edinburgh (2021). "Research Publications & Copyright Policy (2021)". https://www.ed.ac.uk/sites/default/files/atoms/files/research_publications Copyright policy 2021 approved.pdf

[3] Stephen Englen (2024). List of UK universities that have passed an institutional rights retention policy. Github. https://github.com/sje30/rrs/blob/main/ukinstitutions.csv

[4] Julie Baldwin, Stephen Pinfield (2018). "The UK Scholarly Communication Licence: Attempting to cut through the Gordian Knot of the complexities of funder mandates, publisher embargoes and researcher caution in achieving open access". Publications 6(3): 31. https://eprints.whiterose.ac.uk/133383/

[5] Dominic Tate, Per Pippin Aspaas (2022). "The Rights Retention Policy of Edinburgh University". Open Science Talk, (43). https://doi.org/10.7557/19.6859

[6] Jisc (2024). "Institutional rights retention policies task and finish group". https://www.jisc.ac.uk/get-involved/institutional-rights-retention-policies-task-and-finish-group

[7] Knowledge Rights 21 (2024). University Policies for Rights Retention and Secondary Publishing Rights in the Netherlands. New story. https://www.knowledgerights21.org/news-story/university-policies-for-rights-retention-and-secondary-publishing-rights-in-the-netherlands/

[8] Samuel Moore (2023). "The Politics of Rights Retention". Publications 11(2): 28. https://doi.org/10.3390/publications11020028

[9] SCOIR project, https://scoir.moodlecloud.com/

[10] Knowledge Rights 21. Key objectives. https://www.knowledgerights21.org/key-objectives/

[11] Susanna Nykiri (2023). "RRS discussed at FUN's [Finnish University Library Network] webinar in August". FUN blog. https://yliopistokirjastot.fi/en/rrs-discussed-at-funs-webinar-in-august/

[12] Ross Mounce (2021). Research outputs including rights retention language. Github. https://github.com/rossmounce/rrs-language-including-outputs/blob/main/RRS-outputs-asof-2021-09-01.csv

[13] Theo Andrew (2023). "Rights retention in action (impact of the UOE Research Publications Policy after 12 months)". Edinburgh Open Research Conference 2023 presentation recording, https://media.ed.ac.uk/media/Rights+retention+in+action+%28impact+of+the+UOE+Research+Publications+Policy+after+12+months%29++Theo+Andrew/1_9v7pcolv

[14] EIFL (2024). Rights retention and secondary publishing rights: an EIFL guide for libraries. https://eifl.net/resources/rights-retention-and-secondary-publishing-rights-eifl-guide-libraries

[15] Knowledge Rights 21 (2022). A Position Statement from Knowledge Rights 21 on Secondary Publishing Rights. https://www.knowledgerights21.org/wp-content/uploads/2022/10/Secondary-Publishing-Rights-Position-Paper.pdf
## 4.3. FULLY GOLD OPEN ACCESS

This section deals with the two distinct business models – *Article Processing Charge (APC)-based Gold OA* and APC-free *Diamond Open Access* – covered by the fully Gold Open Access denomination<fnref n="10" />. Given that the considerations for both avenues to fully Gold OA are very different in nature, these business models are addressed in independent subsections below.

### 4.3.1. APC-based Gold Open Access

*The paid-for Gold Open Access business model has experienced the largest worldwide growth among all Open Access types in recent years. While this is in principle good news for the purpose of accessibility, there are very serious downsides to this consolidation. The positive and negative sides of this trend are examined in this section.*

The number of funded publications by *OA type* and funder pre- (2015-2020) and post-Plan S (2021-2023) featured in **table 3** in chapter 3 on the CIE above shows the increase in the rate of Gold OA publications. It is also telling to see that this post-Plan S increase does not take place just for cOAlition S funders but *also* for the funders in the other groups, including the control group of funders with little or no Plan S footprint.

This reveals the degree of consolidation this business model has experienced over recent years. This development suggests that Open Access has become a regular aspiration by researchers, and not just because their funders are requiring it from them. Academics interviewed for this study recognise that Open Access enables their research findings to be accessible to as wide a readership as possible. This trend applies beyond the realm of funded researchers. Early Career Researchers (ECRs) in particular are especially keen to have their first publications published Open Access and regularly check with their institutional Open Access support teams before submission that their manuscripts can be published (Gold) Open Access. This is an area where *Read & Publish agreements* have made a big difference, as the model allows them to submit to hybrid journals and publish Gold OA at no cost for them.

These *Gold OA* figures in **table 3** are also due to the fact that the range of fully Open Access titles available to researchers has grown significantly over the years. When the Finch report was published in 2012 there were serious gaps in the availability of Open Access titles that seemed to pose a challenge to the practical implementation of the recommendations. Thanks to the efforts of many publishers and learned societies, the landscape has dramatically evolved and there are now choices available to publish Gold Open Access in practically every discipline.

Megajournals like *PLoS ONE* and *Scientific Reports* are very much part of this consolidated Gold Open Access infrastructure. While megajournals are part of the progress around accessibility, they are also part of the problem around affordability.

There are three significant issues arising from the expansion of this Gold Open Access model [1]. The first one is the unsustainable costs associated with it becoming the prevalent model. The second is its impact on quality and the third relates to the sheer volume of publications.

The business model behind fully Open Access titles is evolving and it may be that in a few years the *Article Processing Charge (APC)* model will not be as predominant as it is now [2]. In the meantime,

<footnote n="10">¹⁰ In the context of this section, Gold Open Access is understood to mean fully Gold OA or pure Gold OA, i.e. it does not include Hybrid Open Access. While this is explicitly mentioned in the section title, the term Gold OA will regularly be used in the body of the section meaning fully Gold OA.</footnote>
institutions and funders are struggling to cope with the budgets associated with a widespread APC-based research publishing model.

This pay-to-publish model also creates severe divides between those who can afford their Open Access publishing costs to be covered (usually funded by institutions or by research funders) and those who can't. Leaving to one side the big issue around affordability for researchers in the Global South, pay-to-publish models exacerbate the divide between funded and unfunded researchers ("funded" in this context meaning involved in projects funded by research funding organisations who will award them an additional budget to cover the costs of Open Access publishing). This funded vs unfunded researcher divide moreover translates into a further divide between funded and unfunded *disciplines*.

This is again an area where Read & Publish agreements have meant an improvement – if not a particularly affordable one for all organisations – as they remove this eligibility barrier among researchers, departments and disciplines that has traditionally hit the humanities and social sciences the hardest.

For researchers based in the Global South, these considerations become critical. The default (if unstated) mechanism for Open Access implementation is that each institution covers "its own papers", meaning those whose submitting corresponding authors are affiliated with the institution. This leaves authors in the Global South completely exposed and unable to face the costs unless they relinquish the corresponding authorship. The increasingly frequent phenomenon of "APC tourism" has been discussed in the interviews with Open Access experts, particularly from consortia. This phenomenon involves the selection of the corresponding author for a publication on the basis of their ability to cover the Open Access publishing costs instead of on strictly scientific grounds.

<figure><img src="image_23.png" /></figure>

"We agree with the need to make full and immediate Open Access a reality but recognise that in the context of Latin America and Chile in particular, making this a reality would mostly mean paying APCs to publishers. We are adopting the Green OA as a non-profit open access initiative, without paying APCs"

(Funder)

On the quality front, there are widespread concerns about *predatory publishing* among academics. In addition, there are problems with a business model that encourages volume-based publishing (often on the basis of special issues) as a way of maximising revenues and profits. One possible way to tackle this problem is for funders to determine that guest contributions to special issues aren't eligible for Open Access funding [3] and should instead have their APCs waived. That said, it would be a difficult statement to enforce, as it's institutions who would be tasked with identifying the problematic works in time to warn researchers about the non-eligibility (i.e. *before* manuscript submission).

The reflection has also frequently emerged in the interviews with Open Access experts at institutions that researchers need to take a much more active and better-informed stance on Open Access. It cannot just be a question of achieving the maximum possible visibility at any cost just because someone else will pick up the tab. This may belong in the area of research integrity, but librarians are not generally equipped to discuss research integrity with researchers. What librarians, institutions and funders *can do* is to identify best practices in research publishing among academics and to support (via CoARA and other initiatives [4]) better mechanisms to acknowledge and reward these best practices.

Funders within coAlition S are ideally placed to define what such best practices should look like, particularly in a context of “responsible publishing”. One possible way would be to require a narrative section in the project reporting devoted to (the Principal Investigator) explaining how the project team have approached the general guidance on publishing their results responsibly. This would not just mean
requiring Open Access, but Open Access in the appropriate fashion and venues. Given the current levels of unawareness among researchers, it's likely that institutional Open Access support services would end up drafting these narrative statements on their behalf, but this would provide such services with an invaluable tool for advocating for alternative publishing venues.

Raising awareness of the costs associated with their Open Access publishing choices is the main rationale for the “modest proposal for researchers” in section 4.6 below. While a reformed research assessment framework develops, a mechanism should be implemented to bring home the issue of publishing costs. While it’s a proposal in the sense that the study suggests a more public discussion about it, the move described in 4.6. is already happening – albeit in a very stealthy way – at some European universities and is under consideration in a few additional ones.

### 4.3.2. Diamond Open Access

Diamond Open Access is perhaps the scholarly communications area that has received the largest attention in these first five years of Plan S. This short period of time has seen significant progress in strengthening this business model, with decisive cOAlition S backing. However, key challenges remain in place, not least how to win researchers' hearts and minds with regard to this model so that its uptake will grow and consolidate.

Although Diamond Open Access is technically a subset of Gold OA, none the main issues around Gold OA explored in the previous subsection – unsustainable costs, impact on quality and sheer volume of publications – apply to Diamond OA. This is why Diamond OA has been independently addressed in its own subsection.

In an interview with a Southern European national research funder conducted for this study, they stated they had been strongly supporting the Diamond Open Access model since 2005 with the establishing of their national node for SciELO [5]. They also welcomed the opportunity to set their national Diamond Capacity Centre (DCC) in line with the developments in other European countries under the umbrella of the EU-funded DIAMAS and Craft-OA projects [6]. In fact, SciELO recently celebrated its 25th anniversary in Brazil [7] and remains a much-admired scholarly publishing initiative among Open Access advocates in the Global North.

Relevant challenges remain though: not every Open Access expert interviewed for this study fully agreed with the subsidising of the free-to-read, free-to-publish model by research funders in the SciELO countries. Same as with the launch of Open Research platforms by research funders, some views raise doubts on the advisability of a move by the very actors that fund research into the publishing domain. Moreover, the percentage of Diamond OA outputs among publications funded by the various research funders analysed within this study above remains very low.

Still, the Diamond OA model has unique advantages and is currently receiving a great deal of attention following the celebration of the Global Diamond OA summit hosted by Redalyc in Toluca (Mexico) in October 2023 [8]. Diamond OA has been firmly on the cOAlition S radar since the very start of the initiative [9] and very early on cOAlition S commissioned and funded – together with Science Europe – a study on the global Diamond OA landscape aimed at exploring the challenges faced and the

<figure><img src="image_24.png" /></figure>

> "I've done a big chart which shows where our money goes and what it goes on: it goes from completely closed on one side thru Read & Publish agreements and then at the other end you have Diamond and the repository. The equity increases as you go down but actually at the most equitable end you're spending the least amount of money"

*(Library Director)*
opportunities posed by the available infrastructure in this area [10]. EU-funded projects such as DIAMAS, Craft-OA and PALOMERA are operating in this space and trying to foster an international conversation on the most suitable ways to design and support (particularly from an economic perspective, but also technically) the infrastructure that will allow this model to thrive. This has in turn resulted in a plethora of national Diamond Open Access frameworks and initiatives having recently been established in various European countries [11] [12] [13].

The emergence of collective funding frameworks like the Lyrasis Open Access Community Investment Program (OACIP) [14] to implement accountable mechanism to ensure the sustainability of specific Diamond OA journals provides further evidence for the very dynamic environment presently surrounding the Diamond OA model. The role of university consortia like Lyrasis and the Jisc [15] is in fact critical to make room for other Open Access business models beyond *subscriptions* and *Read & Publish agreements*.

Two recommendations arise from an examination of the wide range of initiatives currently taking place around Diamond OA:

1. The infrastructure needs to be consolidated by the same university consortia that are dealing with the other business models, ideally devising criteria that allow good Diamond OA journals currently running on a shoestring to apply for funding under some kind of agreed framework.

2. Institutions need to showcase to their academics the Diamond OA initiatives they may be supporting – either via crowdfunding or by directly supporting their own portal of institutional journals or other open infrastructure – on an equal footing with the online space devoted to disseminating other business models like transformative agreements.

There are inspiring examples also in this latter area, providing a range of institutional best practices around showcasing the support for open infrastructures and services (including publishing venues). Specific sections on the websites for the KU Leuven Library [16] and the University of York Library [17] highlight the wide range of services they support. As more institutions gradually join the move to support these open infrastructures and services and openly display the information on their online spaces, it will be possible to gradually come up with an international service portfolio that new libraries wanting to diversify their investments in scholarly communications may be able to consider.

<figure><img src="image_25.png" /></figure>

> "The diamond model is a strength of Latin America that has been around for many years. Last year SciELO turned 25 years old. The diamond model is a way to make scientific knowledge a common good. The SciELO Network has worked and could be replicated"

*(Funder)*

This growing visibility is relevant because one of the main challenges the Diamond OA business model currently faces is the low awareness among researchers and their reluctance to submit their manuscripts to these Diamond OA titles. There are many different factors that may contribute to explain this reluctance<fnref n="11" />, and those related to low or non-existent *Journal Impact Factors* are already being addressed by initiatives like DORA or the Coalition for Advancing Research Assessment (CoARA) [18].

It is however unrealistic to blame the researchers' reluctance to submit to Diamond OA titles solely on their being led to do so by an unforgiving research assessment framework. An examination of the composition of the editorial boards for consolidated commercial journals (including those following the

<footnote n="11">¹¹ In some disciplines, especially in STEM fields, there aren't enough Diamond OA titles for researchers to consider a possible submission</footnote>
paid-for Gold Open Access model) shows unequivocal differences with the average Diamond OA title.
The influence on Early Career Researchers' choices of publishing venue exerted by their seniors'
representation in editorial boards is a non-negligible and potentially concerning one.

There is also a marked divide between STM and SSH disciplines with regard to the potential success of the Diamond OA model. Initiatives that manage to attract journals to their Diamond OA model that were previously sitting behind a paywall with commercial publishers [19] are almost exclusively operating in the Social Sciences and Humanities. It is difficult as a result to assess the general potential of a business model without accounting for these stark disciplinary differences.

At the same time, there are networks of Diamond OA titles that collaborate across disciplines [20]. The mapping of the Diamond OA landscape is getting better as a result of the work done by the multiple projects and initiatives working in the area<fnref n="12" />, so this networking is bound to improve going forward.

The analysis of the volumes of publication and the ruthless competition among various concurrent publishing avenues suggest that it will take a significant amount of time for any kind of ‘transition’ towards a Diamond OA business model to occur. Likewise for this model to gain some traction among publications funded by cOAlition S funders. In the meantime, it is worth investing in its development and exploring the best avenues to consolidate the model both academically and economically.

This is why the “modest proposal” is raised in **section 4.6** to promote the sharing of the costs for publishing *between the library and the departments or schools*. This is to ensure that libraries are able to ‘liberate’ part of their budget currently tied into these commercial options so that they can be devoted to the necessary consolidation of complementary business models such as Diamond OA or **Subscribe to Open**. This is proposed as a mechanism to gradually move the funding where it’s consumed while the discussions on research assessment reform come to fruition. In the meantime, the researchers’ wish to publish in their preferred journals could be granted – but allowing the cost implications to sit much closer to home.

One of the technical challenges posed by the Diamond OA model is the nascent attempts at its accurate monitoring. This is currently being examined by the EU-funded projects working in the consolidation of this publishing business model, but at the time of writing it remains very difficult to monitor the publishing activity in Diamond OA titles within a specific institution - or within the publications funded by any given research funder. In the data science era where detailed information is being collected on the uptake of the various Open Access business models, this is a problem.

This issue is mainly due to the very scattered nature of these Diamond OA titles, but there are additional, more strategic factors. Read & Publish agreements are much more closely monitored because of the very significant economic investment institutions are devoting to this other model. If such economic driver is not there, or not yet, it's very difficult to make the case for a careful monitoring of Diamond Open Access adoption within the institutional publishing patterns.

The wide variety of research disciplines and venues means this monitoring will always be challenging, but strategies will gradually emerge to make it possible so that the progress in the consolidation of the model can be quantitatively measured. The economic incentive mentioned above is also expected to gain prominence as institutions and libraries gradually shift their strategic investment towards these alternative publishing models.

<footnote n="12">¹² Figure 11 (p.83) on Bianca Kramer's "Study on scientific publishing in Europe: Development, diversity, and transparency of costs" for the European Commission at https://doi.org/10.2777/89349 shows a remarkable uptake for Diamond OA and non-DOAJ-listed fully OA in specific European countries (Croatia and the Baltic Republics). This kind of mapping may provide valuable insights into the drivers for a successful Diamond OA implementation.</footnote>
Open Research platforms – of which there are plenty of examples at present – provide an interesting case study regarding the researchers’ reluctance to submit their manuscripts to alternative publishing venues. Open Research Europe and other similar F1000-based platforms currently lack the sort of solid analysis for their uptake<fnref n="13" /> that was for instance provided back in 2020 for the Wellcome Open Research platform [21] or more recently for the Gates Open Research platform [22]. Specifically for the ORE, it would be critical to see the distribution across countries and disciplines, ideally together with the growth in the number of publications in each of these categories over time. This is addressed in the recommendations section.

**Overlay journals** constitute another interesting subset of Diamond Open Access titles sitting conceptually next to *preprints*. Recent analysis of the overlay journal landscape [23] show a distinct disciplinary mix closer to the STEM domain and a potential for consolidation even if their share of the publishing market remains very small. The strong representation of French partners in the DIAMAS and Craft-OA EU-funded projects represents some guarantee for progress, since it's in France where this model has seen a particularly high uptake via the development of the Episciences publishing platform for overlay journals on top of the HAL national repository [24].

Commercial publishers occasionally host Diamond OA titles too alongside their paid-for ones (be it APC-based Gold OA, Hybrid OA or subscription journals) by offering smaller publishers and societies the possibility of relying on their well-established technical systems and

marketing practices to boost the visibility and impact of these titles. This could offer some opportunities for collaboration, but also raises the potential risk to the Diamond OA model of being ‘captured’ by commercial actors that may switch their business model going forward [25]. This is another area that the consolidating Diamond OA community is addressing – see principle #2 on “ownership and governance” on the list of 10 principles of Diamond OA publishing released by cOAlition S [26].

<figure><img src="image_26.png" /></figure>

"Do we want commercial journals to transition to Open Access or do we actually want Open Access for research findings and do we want to decouple that from the publishing process? These are two different questions. At the moment, scholarly communication, Open Access and publishing are all coupled together in the journal. By launching their Open Research platform, Wellcome have demonstrated that you can have scholarly communication and then publishing as a potential next step. The funders are funding the research and the communication of that research. Should they also be funding the publication of the communication of that research?"

(Library Director)

## REFERENCES

[1] John Frank, Rosemary Foster, Claudia Pagliari (2023). Open access publishing – noble intention, flawed reality. Social Science & Medicine 317: 115592. https://doi.org/10.1016/j.socscimed.2022.115592

[2] Roheena Anand, Robert Kiley, Anna Vernon (2023). Beyond article-based charges: working group established. cOAlition S blog. https://www.coalition-s.org/beyond-article-based-charges-working-group-established/

[3] Swiss National Science Foundation (2024). "The SNSF is no longer funding Open Access articles in special issues". https://www.snf.ch/en/g2ICvujLDm9ZAU8d/news/the-snsf-is-no-longer-funding-open-access-articles-in-special-issues

[4] GraspOS: next Generation Research Assessment to Promote Open Science, https://www.graspos.eu/

<footnote n="13">¹³ The fact that the operation of these Open Research platforms is outsourced to external service providers, moreover to service providers owned by one of the ‘big five’ commercial publishers, poses issues with regard to the quality of the analysis of the success or otherwise experienced by these Open Research platforms. The quality of the analysis significantly increases when some internal manager for the platform *at the funder* coauthors the piece, but this rarely happens.</footnote>
[5] SciELO 25 Anos | Entrevista com João Mendes Moreira. SciELO Youtube channel (2023).
https://www.youtube.com/watch?v=fqM37UBydqA

[6] Pierre Mounier, Johan Rooryck (2023). "Towards a federated global community of Diamond Open Access". The Diamond Papers. https://thd.hypotheses.org/296

[7] Jan Velterop (Ed, 2023). We so loved Open Access. https://doi.org/10.21452/abec.2023.isbn.978-65-993452-6-5

[8] II Diamond Open Access Conference. October 25-26. Toluca (Mexico). https://globaldiamantoa.org/diamond-open-access-conference/en/home-2/

[9] cOAlition S (2023). Diamond Open Access. https://www.coalition-s.org/diamond-open-access/

[10] Jeroen Bosman, Jan Erik Frantsvåg, Bianca Kramer, Pierre-Carl Langlais, Vanessa Proudman (2021). OA Diamond Journals Study. Part 1: Findings. Zenodo. https://doi.org/10.5281/zenodo.4558704

[11] Germany: DFG initiative "Diamond Open Access stärken", https://www.dfg.de/de/aktuelles/neuigkeiten-themen/info-wissenschaft/2024/ifw-24-02

[12] Netherlands: Project Diamond Open Access in the Netherlands (13 Mar 2024),
https://www.openaccess.nl/en/events/project-diamond-open-access-in-the-netherlands

[13] Ireland: Publish OA Ireland, https://publishoa-ie.moodlecloud.com/

[14] Lyrasis Open Access Community Investment Program (OACIP), https://www.lyrasis.org/content/Pages/oacip.aspx

[15] Helen Dobson (2022). "A new approach to supporting scholarly communications: announcing the open access community framework (OACF)". Jisc blog. https://www.jisc.ac.uk/blog/a-new-approach-to-supporting-scholarly-communications-announcing-the-open-access-community-framework-oacf

[16] Open Science services supported by KU Leuven Libraries, https://www.kuleuven.be/open-science/what-is-open-science/scholarly-publishing-and-open-access/open-access-kuleuven/OS-services

[17] University of York Library – Supporting the wider open research ecosystem,
https://www.york.ac.uk/library/about/policies/open-research-ecosystem/

[18] Coalition for Advancing Research Assessment (CoARA), https://coara.eu/

[19] Paula Clemente Vega (2024). The Open Library of Humanities launches 7 new journals. OLH blog.
https://www.openlibhums.org/news/700/

[20] Free Journal Network, https://freejournals.org/

[21] Robert Kiley, Michael Markle (2020). "Wellcome Open Research: a summary of year 3". Wellcome Open Research blog.
https://blog.wellcomeopenresearch.org/2020/01/27/wellcome-open-research-a-summary-of-year-3/

[22] Collette McKenny (2023). Six years of supporting Gates-funded open research. Gates Open Research blog.
https://blog.gatesopenresearch.org/2023/03/28/six-years-of-supporting-gates-funded-open-research/

[23] Antti M Rousi, Mikael Laakso (2024). "Overlay journals: A study of the current landscape". Journal of Librarianship and Information Science, 56(1), 15-28. https://doi.org/10.1177/09610006221125208

[24] Raphaël Tournoy (2023). Episciences overlay journals: a bridge between scientific publications, open repositories, data and software repositories. The 18th Munin Conference on Scholarly Publishing, University of Tromsø, Nov 2023, Tromsø, Norway. https://doi.org/10.7557/5.7147

[25] Emerald Publishing (2018). Emerald acquires Archnet-IJAR: International Journal of Architectural Research. Press release. https://www.emeraldgrouppublishing.com/news-and-press-releases/emerald-acquires-archnet-ijar-international-journal-architectural-research. This was a successful Diamond OA title run on a shoestring whose editor-in-chief decided to move to a commercial publisher. The publisher immediately put it behind a paywall and switched it to a hybrid OA model.

[26] Johan Rooryck (2023). Principles of Diamond Open Access Publishing: a draft proposal. cOAlition S blog.
https://www.coalition-s.org/blog/principles-of-diamond-open-access-publishing-a-draft-proposal/
## 4.4. PARALLEL AREAS OF PRACTICE: PREPRINTS. LONG-FORM PUBLICATIONS

### 4.4.1. Preprints

Preprints do not specifically feature in the Plan S toolbox given that these are non-peer reviewed outputs, but an ever-growing number of researchers are choosing to follow this instrument for openly sharing their results. The number of preprint servers and related venues has also significantly grown in the past years. This has driven specific funders to explore the possibility of turning preprints into an alternative route to comply with their Open Access policies.

Given that *preprints* are not specifically part of Plan S, this model was only fleetingly addressed during the interviews that provided the basis for the qualitative analysis. While the arXiv flagship has been around for over 30 years now [1], preprint servers have seen a major boost since the Covid-19 pandemic and the range of disciplines they cover has multiplied over the years. There aren't many published analyses of the number of preprints posted post-pandemic by researchers at a given institution or in a specific preprint server, but the available evidence shows a significant increase on all counts [2].

This gradual adoption of the preprint culture in many disciplines, including the life sciences, has no doubt been one of the factors behind the decision by research funders like the Wellcome Trust at the start of 2024¹⁴ and more recently the Gates Foundation [3] to accept preprints as a means to comply with their Open Access policy (the costs involved in Open Access publishing are clearly another one of these factors). Both legacy and fully Open Access publishers like Springer Nature, JMIR or MDPI have for their part been taking their own positions in the domain [4] and are also promoting the posting of preprints among their authors.

These two large philanthropic funders – Wellcome and the BMGF – are coAlition S members, so their move hints at an increasing relevance of preprints in the discussions held by the alliance on how to best achieve full, immediate Open Access. The need to implement some systematised peer-review mechanism for preprints remains the most evident challenge arising from this approach, but there is a certain symmetry with the “Towards Responsible Publishing” strategy that coAlition S released for consultation in October 2023 (which falls beyond the scope of this study). Several among the interviewees for this qualitative analysis – both among library directors and funders – have pointed out that the key challenge in this area going forward should be to gradually identify and implement mechanisms for these peer-review workflows to be provided as a service by third-party service providers.

<footnote n="14">¹⁴ The January 2024 Wellcome Trust policy with regard to preprints states that “if using a preprint to ensure compliance, researchers must post it to a preprint server indexed by Europe PMC before final publication of the paper, https://europepmc.org/Preprints#preprint-servers”. The Europe PMC page lists 32 preprint servers at the time of writing (July 2024).</footnote>
4.4.2. Long-form publications

In its statement on Open Access for academic books released in September 2021, the cOAlition S
recognizes that academic book publishing is very different from journal publishing and
acknowledges that standards and funding models may need more time to develop. However, the
role of cOAlition S funders is seen as critical to bring about these funding models and standards
beyond the five recommendations provided in the statement.

Open Access for monographs and book chapters was mentioned in principle #7 when Plan S was
originally released in 2018. This principle stated that “the timeline to achieve Open Access for
monographs and book chapters will be longer and requires a separate and due process”. As it had been
announced, a specific statement on Open Access for academic books was issued in September 2021 [5]
committing cOAlition S to make progress “as soon as possible” while acknowledging the complexities
in this area. This statement also highlighted the coverage of long-form publications within the Open
Access policies issued by some of its members and offered five recommendations to cOAlition S funders.

Same as for preprints, this parallel area of activity of Open Access for *long-form publications* has not
received too much attention in the interviews, especially with so many other Open Access routes and
strategies to explore. There is however a widespread and sustained effort underway to promote Open
Access for books.

As stated in principle #7, the Open Access movement for books and book chapters is lagging somewhat with regard to the far more consolidated OA for journal articles: the EU-funded PALOMERA project (Policy ALignment of Open access Monographs in the European Research Area, 2023-2024) addresses the same policy alignment domain that the FP7 PASTEUR4OA project promoted for OA in the period 2014-2016.

This lagging is not necessarily bad news, as this is a significantly more challenging endeavour to address
than Open Access for journal articles and conference papers. Moreover, the road towards Open Access
for journal articles has offered a number of lessons that may be put to good use when deciding the way
forward for long-form publications.

One area where this application of lessons learnt is clearly happening is the diversity in the range of
routes under consideration for Open Access books to gradually become more widespread. Because
well-established initiatives like OAPEN<fnref n="15" /> and the Directory of Open Access Books (DOAB) are now
working alongside EU-funded projects like CRAFT-OA, DIAMAS, and PALOMERA and other initiatives like
Copim, there is a much deeper awareness of the role that alternative publishing venues will need to play
in the process. It's also clear that *Book Processing Charges (BPCs)* cannot be the sole – not even the
main – route to Open Access for long-form publications.

As a result, the Open Access landscape for books is largely one of experimentation at present. However, the role of cOAlition S funders in exploring avenues to gradually make this happen and in collecting data that underpins the policymaking going forward remains of paramount importance.

The annex to the 2021 cOAlition S statement on Open Access for academic books lists a number of
cOAlition S funders who have already issued their Open Access policies for long-form publications. Two
of these are briefly examined here for their relevance:

<footnote n="15">¹⁵ The OAPEN OA Books Toolkit at https://oabooks-toolkit.org/ provides a valuable collection of resources for the promotion of Open Access for books, including a set of “author success stories” where book authors from different disciplinary and geographical backgrounds explain how Open Access publication has benefitted their work.</footnote>
1. The Austrian Science Fund (FWF) OA policy for books is particularly well explained on their website, which includes a reference to a €22,000 funding cap for Book Processing Charges [6]. The workflows for the implementation of the policy are common to other cOAlition S funders on the list in proposing the Gold and the Green Open Access routes for compliance. Also similar is the fact that the funding workflows are largely run by the funder vis-à-vis the researchers and projects. The key best practice that the FWF have regularly followed is the deposit of open datasets for the payments made on an annual basis in specific records in the Zenodo repository. These are Excel spreadsheets with different tabs for APCs and BPCs [7]. It's hard to understate the relevance of having this information available for the purpose of mapping the available options for authors and – critically – the associated costs<fnref n="16" />.

2. The UK Research and Innovation (UKRI) policy for long-form publications<fnref n="17" /> that came into force on 1 January 2024 has become the pioneering approach in terms of involving institutional Open Access teams in the policy implementation workflows. It is this policy that has mostly been discussed with interviewees. The main topics addressed in these conversations are highlighted below, providing an overview of the challenges faced by this specific initiative to fund Open Access for long-form publications as well as some best practice elements that other cOAlition S funders might wish to consider when issuing their own policies.

Specific challenges raised by Open Access for long-form publications

* **Vanity publishing.** The issue raised in previous sections of the acute trend for authors of journal articles to select their publishing venue on the basis of prestige is even more pronounced in the choice of publishing venue for books. There may be a rapidly increasing variety of homes for Open Access books to be published (more on this in the section on Plan S and publishers), but authors tend to go for the proven quality of the well-established brands, which happen to be the most expensive ones. Addressing this issue will require the intervention of institutions in the workflow (which the UKRI has now made possible) and the open availability of the data on payments to specific publishers from the funders' budgets.

* **Disciplinary bias.** The vast majority of books arise from disciplines in the Social Sciences and Humanities (SSH), which have traditionally had a more cautious approach to Open Access. This is relevant for a number of aspects, among them the choice of Creative Commons licences – while CC BY is widely accepted as the default in the natural sciences, this is hardly the case in the SSH, which leads to the need for flexibility around CC licences.

* **Highly scattered publishing landscape.** The subset of funded publishers in the abovementioned FWF dataset [7] has a strong national character and shows little overlap with the equivalent list of publishers that funders in other countries and linguistic areas might provide. This feature is characteristic of the disciplinary bias towards the SSH and makes OA book publishing conceptually much closer to the Diamond OA publishing landscape than to mainstream journal publishing in the natural sciences. The benefits of involving in the discussion initiatives working in the Diamond OA sphere become evident, as does the value of an organisation bringing together funders from different provenances like cOAlition S.

<footnote n="16">¹⁶ The University of Bielefeld-hosted OpenAPC initiative to collect and share Open Access costs includes a specific section devoted to Open Access Monograph Charges (BPCs) at https://treemaps.openapc.net/apcdata/bpc/#institution/. The FWF top this section, but there is no other cOAlition S funder on the long list of data providers despite the fact that some cOAlition S funders have for many years provided Gold Open Access funding for books and book chapters.</footnote>

<footnote n="17">¹⁷ Guidance for UKRI's open access fund for long-form publications, https://www.ukri.org/publications/guidance-for-ukris-open-access-fund-for-long-form-publications/</footnote>