Organisation for Economic Co-operation and Development

Unclassified

ENV/CBC/MONO(2021)22
English - Or. English
8 December 2021

ENVIRONMENT DIRECTORATECHEMICALS AND BIOTECHNOLOGY COMMITTEE

Cancels & replaces the same document of 3 September 2021

Guidance Document for the scientific review of Adverse Outcome Pathways
Series on Testing and Assessment,
No. 344
Series on Adverse Outcome Pathways
No. 20

JT03487133
OFDE

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to thedelimitation of international frontiers and boundaries and to the name of any territory, city or area.

2  ENV/CBC/MONO(2021)22

SERIES ON TESTING AND ASSESSMENT
NO. 344

Guidance Document for the scientific review of Adverse Outcome Pathways

Environment Directorate
ORGANISATION FOR ECONOMIC COOPERATION AND DEVELOPMENT
Paris 2021

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  3

About the OECDThe Organisation for Economic Co-operation and Development (OECD) is an intergovernmentalorganisation in which representatives of 36 industrialised countries in North and South America,
Europe and the Asia and Pacific region, as well as the European Commission, meet to coordinate and harmonise policies, discuss issues of mutual concern, and work together to respondto international problems. Most of the OECD’s work is carried out by more than 200 specialisedcommittees and working groups composed of member country delegates. Observers fromseveral countries with special status at the OECD, and from interested internationalorganisations, attend many of the OECD’s workshops and other meetings. Committees andworking groups are served by the OECD Secretariat, located in Paris, France, which is organised
into directorates and divisions.
The Environment, Health and Safety Division publishes free-of-charge documents in elevendifferent series: Testing and Assessment; Good Laboratory Practice and Compliance
Monitoring; Pesticides; Biocides; Risk Management; Harmonisation of Regulatory
Oversight in Biotechnology; Safety of Novel Foods and Feeds; Chemical Accidents;
Pollutant Release and Transfer Registers; Emission Scenario Documents; and Safety of
Manufactured Nanomaterials. More information about the Environment, Health and Safety
Programme and EHS publications is available on the OECD’s World Wide Web site(www.oecd.org/chemicalsafety/).

This publication was developed in the IOMC context. The contents do not necessarilyreflect the views or stated policies of individual IOMC Participating Organizations.
The Inter-Organisation Programme for the Sound Management of Chemicals (IOMC)
was established in 1995 following recommendations made by the 1992 UN Conference
on Environment and Development to strengthen co-operation and increase international
co-ordination in the field of chemical safety. The Participating Organisations are FAO,
ILO, UNDP, UNEP, UNIDO, UNITAR, WHO, World Bank and OECD. The purpose of the
IOMC is to promote co-ordination of the policies and activities pursued by theParticipating Organisations, jointly or separately, to achieve the sound management ofchemicals in relation to human health and the environment.

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified4  ENV/CBC/MONO(2021)22

This publication is available electronically, at no charge.
Also published in the Testing and Assessment link
For this and many other Environment,
Health and Safety publications, consult the OECD’s
World Wide Web site (www.oecd.org/chemicalsafety/)or contact:

OECD Environment Directorate,
Environment, Health and Safety Division
2 rue André-Pascal
75775 Paris Cedex 16
FranceFax: (33-1) 44 30 61 80
E-mail: ehscont@oecd.org© OECD 2021
Applications for permission to reproduce or translate all or part of this material should be madeto: Head of Publications Service, RIGHTS@oecd.org, OECD, 2 rue André-Pascal, 75775 Paris
Cedex 16, FranceOECD Environment, Health and Safety Publications

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  5

Table of contents

Foreword

6

Abbreviations

7

1. Background: The OECD AOP Development Programme

8

1.1. Objectives of the AOP Development Programme
1.2. Development of AOPs under the OECD AOP Development Programme

88

2. Pre-requisite for the AOP scientific review

11

3. The AOP Scientific review

12

3.1. The management of the review: Review organiser and review manager
3.2. Conflict of interest
3.3. Call for reviewers and recruitment of candidate reviewers
3.4. Selection of reviewers from the pool of candidate reviewers
3.5. The review process
3.6. Public information related to the reviewers

13
14
14
15
16
17

4. Subsequent AOP endorsement by OECD

17

References

19

Annex 1: Example of a template for the development of an AOP scientific review
report20

Annex 2: Declaration of interest

21

FIGURES
Figure 1. The three phases of the AOP development process
Figure 2. Detailed description of the AOP development process
Figure 3. Illustration of the AOP scientific review process

9
10
13

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified6  ENV/CBC/MONO(2021)22

Foreword
The Adverse Outcome Pathway (AOP) framework has been developed to facilitate the collection of
mechanistic information derived from toxicological science in a structured manner, to assist in establishingcausal relationships between molecular and cellular events that lead from stressor perturbation of thebiology to adverse effects, and to identify critical data gaps in the understanding of those pathways.
AOPs are intended to help regulatory agencies and risk assessors utilise a broader range of mechanistic
data concerning the effects of stressors on various test systems (e.g. in silico, in vitro, in vivo) for decisionmaking. The widespread acceptance of AOPs as a source of information to guide interpretation, generation
and application of data from alternative methods depends on the confidence in the quality of the AOP and
therefore, on the review process used to evaluate whether the scientific evidence underpinning the AOP
is reliable, credible, and balanced and thus that the science used to inform policy is trustworthy.
In order to maintain the development of high quality AOPs, it is important to standardise the way in which
AOPs are developed and reviewed while retaining an appropriate degree of flexibility to ensure efficiency.
The existing Users' Handbook supplement to the Guidance Document for developing and assessing
Adverse Outcome Pathways (OECD, 2018) focuses on practical aspects of AOP development andassessment. The objective of the present document is to provide guidance on the quality standards
required for the scientific review of an AOP on the AOP-Wiki. It defines the core principles associated with
AOP scientific review in order to enable consistent scientific reviews to be conducted, regardless of who
is doing the review, and thus will facilitate OECD endorsement.
This Guidance Document has been developed by the Extended Advisory Group for Molecular Screening
and Toxicogenomics (EAGMST) based on the experience gained with the AOP scientific review over the
past years. It was circulated to the Working Party of the National Coordinators of the Test Guidelines
Programme (WNT) and the Working Party on Hazard Assessment (WPHA) for a commenting round in July
2020. Comments were discussed and addressed by the EAGMST and the document was approved by the
WNT and the WPHA by written procedure in June 2021. It is published under the responsibility of the
Chemicals and Biotechnology Committee.

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  7

Abbreviations
AOP: Adverse Outcome Pathway
AOP-KB: AOP Knowledge-Base (https://aopkb.oecd.org/)
AOP-Wiki: Collaborative Adverse Outcome Pathway Wiki (https://aopwiki.org/)
CBC: Chemicals and Biotechnology Committee
CRPPH: Committee of Radiological Protection and Public Health
EAGMST: Extended Advisory Group for Molecular Screening and Toxicogenomics
FAQs: Frequently Asked Questions
IATA: Integrated Approaches to Testing and Assessment
ITS: Integrated Testing Strategy
KE: Key Event
KER: Key Event RelationshipOECD: Organisation for Economic Co-operation and Development
SAAOP: Society for the Advancement of AOPs
WoE: Weight of Evidence
WNT: Working Group of the National Coordinators of the Test Guidelines Programme
WPHA: Working Party on Hazard Assessment

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

8  ENV/CBC/MONO(2021)22

1. Background: The OECD AOP
Development Programme
1.1. Objectives of the AOP Development Programme1.
The objectives of the chemical safety programme of the Organisation for Economic Co-operation
and Development (OECD) are to assist countries in developing and implementing policies and instrumentsthat make their systems for managing chemicals as efficient and robust as possible, while protecting
human health and the environment.
2.
In this context, the OECD launched in 2012 a new programme on the development of Adverse
Outcome Pathways (AOP). The AOP concept is expected to guide decision-makers, such as risk
assessors in their work to use existing and emerging information on the effects of chemicals on various
test systems (e.g. in silico, in vitro, in vivo), and to target the generation of additional information forregulatory decision-making.
3.
A variety of potential uses has been described for AOPs. AOPs can, for example, inform the workof the OECD Test Guideline Programme by describing the rationale for the use of particular methods andalso by identifying potentially more predictive methods for development. AOPs can be used as a basis fordeveloping an Integrated Approaches to Testing and Assessment (IATA) or an Integrated Testing Strategy(ITS) (OECD, 2016a). The Defined Approaches for skin sensitisation are also based on the corresponding
AOP (OECD, 2012, 2016b). They can also be used for further development and application of alternative
approaches, such as read-across, as well as in a number of other regulatory contexts, such as priority
setting for further testing, hazard identification (e.g. EFSA-ECHA, 2018), classification and labelling, andrisk assessment (OECD, 2017).
4.
Any chemical or non-chemical stressors that perturb biological pathways and/or functions are partof the AOP framework. Although the scientific review principles apply to all stressors, endorsement
processes may differ (e.g. AOPs should be chemical-relevant for OECD Environment Health and Safety
endorsement - see paragraph 11 and section 4).

1.2. Development of AOPs under the OECD AOP Development Programme
5.
The development of an AOP under the OECD AOP Programme consists of three main phases(Figure 1). The first phase is the assembly of the knowledge in the AOP-Wiki (Collaborative Adverse
Outcome Pathway Wiki, https://aopwiki.org/; a module of the AOP Knowledge-Base AOP-KB;
https://aopkb.oecd.org/). The second phase is the review of the AOP, the subject of this GuidanceDocument. The third phase is the endorsement of the AOP by the responsible OECD Committees.

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  9

Figure 1. The three phases of the AOP development process

The role of the Extended Advisory Group for Molecular Screening and Toxicogenomics
6.
The OECD AOP Development Programme is guided by the Extended Advisory Group for
Molecular Screening and Toxicogenomics (EAGMST) which is under the oversight of the Working Groupof the National Coordinators of the Test Guidelines Programme (WNT) and the Working Party on Hazard
Assessment (WPHA).
7.
The EAGMST has primary responsibility for (i) examining AOP proposals and deciding on their
inclusion in the work plan of the AOP Development Programme, (ii) reviewing the AOPs for theirconsistency with principles and guidance set out in the User's Handbook (OECD, 2018) and (iii) approving
the release of AOPs to WNT and WPHA for endorsement (see section 4 for a description of the process),
after the completion of the scientific review. In addition, many EAGMST members also play an active rolein the development of AOPs. With the scientific review of the AOPs being possibly outsourced, the above
listed responsibilities of the EAGMST will become even more prominent. The various points of control
EAGMST has over the process via its subgroups and collective decisions ensures a transparent control of
the AOP Development programme.

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified10  ENV/CBC/MONO(2021)22
The various steps and interlinkages between the parties involved in AOP development
8.
Figure 2 provides an overview of the OECD AOP development process. The OECD Secretariat
serves as a liaison between the various actors and steps.

Figure 2. Detailed description of the AOP development process

9.
Assembly phase (Step 1 Figure 2): An AOP is assembled by its authors in the AOP-Wiki. Thisshould be done in compliance with the AOP development principles (OECD, 2018). If the AOP is includedin the OECD AOP development work plan (paragraphs 12-14), coaching in AOP development is offeredto the authors by EAGMST (the OECD Secretariat can be contacted). AOP authors will subsequently
update their AOP in the AOP-Wiki to address comments during the review and endorsement phases.
10.

Review phase (Steps 2 and 3 in Figure 2):

A compliance check (Step 2 in Figure 2) is performed by the coaches who provided support tothe authors during the assembly phase.
Scientific review (Step 3 in Figure 2): It is conducted after the AOP has been assembled in the
AOP-Wiki and checked for compliance (see section 2).

11.
Endorsement phase (Steps 4 and 5 in Figure 2): This phase is OECD-specific and requires
approval by EAGMST that the AOP can be submitted to the WNT and WPHA for endorsement. This isfollowed by the declassification by the Chemicals and Biotechnology Committee (CBC) as a last step. The

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  11

endorsement phase is based on the outcome of the scientific review, as well as authors’ responses and
AOP revisions to address reviewer comments. The three-step endorsement of the AOP can involve further
revisions from the AOP authors to address OECD-specific comments (see section 4). AOP endorsement
is followed by the publication of the AOP on the OECD dedicated Series on AOPs in i-Library.

2. Pre-requisite for the AOP scientificreview
Inclusion of a project in the OECD AOP Development Programme work plan
12.
Developers of an AOP who wish their project to be included in the OECD AOP development work
plan should submit a completed AOP Project proposal form via the OECD Secretariat to the EAGMST.
The EAGMST decides whether the proposed AOP should be included in the OECD AOP development
work plan, and submits it for consultation to WPHA/WNT to help assess regulatory relevance and raiseawareness about AOP developments. WPHA/WNT may also help identify opportunities for collaboration
between groups and consider how AOPs can be better aligned to support regulatory needs of countries
and how resources can be focused.
13.
In principle, the submission of a project proposal for inclusion of an AOP in the OECD AOP
development work plan can be done before any of the three-phases in the development of an AOP (Figure1); however, in practice, the submission of a project proposal is encouraged before assembly of the AOP
into the AOP-Wiki platform, so that authors can benefit from the coaching provided by EAGMST (seeparagraph 16).
14.
Contributing to the AOP-Wiki via the OECD AOP Development Programme allows authors to besupported during AOP assembly, provides more visibility and confidence in the AOP developed, andfacilitates its potential use by regulators and more broadly by the scientific community. It is however
possible for scientists to initiate and assemble an AOP in the AOP-Wiki outside of the OECD AOP
Development Programme. In particular, AOP developers can still make valuable contributions even if their
AOP is covering aspects of biology that do not have immediate regulatory application. AOPs developed
outside the OECD AOP Development Programme can receive ad-hoc mentoring from the Society for the
Advancement of AOPs (SAAOP). Practical details on how to request author access, either via OECD or
SAAOP requests, are provided on the AOP-Wiki -‘start a new AOP’ page.

Coaching and compliance check15.
Before scientific review, AOPs assembled in the AOP-Wiki are checked for compliance (seeparagraph 16) with the principles and guidance set out in the Users' Handbook supplement to the GuidanceDocument for developing and assessing Adverse Outcome Pathways (OECD, 2018). The Users’
Handbook details how to structure the elements of an AOP in the AOP-Wiki, and provides guidance on
how to assemble and assess the weight of evidence (WoE) supporting the AOP (OECD, 2018).
16.
The compliance check approach has evolved over time and is now the final step of an individual
support offered during the AOP assembly process. Until 2018, this compliance check was conducted by

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified12  ENV/CBC/MONO(2021)22
experts within the EAGMST (so-called "internal review"), after the authors had assembled an AOP in the
AOP-Wiki and notified EAGMST that it was ready for review. In 2019, EAGMST introduced a coachingprocess for the authors of an AOP, starting when an AOP project is accepted into the work plan. The
coach, a member of the EAGMST who is familiar with AOP guidance and development, guides developers
during the assembly phase and helps them to adhere to the AOP development principles. At the end of
this process, the coach fills in a compliance check form that ensures that the requirements of the Users’
Handbook are met and that the AOP can thus undergo scientific review. The coach will not participate in
the scientific review as a reviewer to avoid any potential conflict of interest.

3. The AOP Scientific review
17.
The scientific review of an AOP, developed in the AOP-Wiki, is initiated after completion of theUsers’ Handbook compliance check by coaches. It is based on the following principles:The scientific review should be independent Avoid conflicts of interestThe scientific review should be transparent
 Transparent selection of reviewers through diverse recruitment channels – e.g.
making use of OECD expert groups with expertise in relevant fields.
 Public disclosure on AOP-Wiki or e.AOP.portal of the names of the reviewers, thecollective outcome of their individual declaration of interest analysis, theircomments and the responses of the AOP authors to the reviewers’ comments



The collective scientific expertise of the Review panel should cover the full scope of
the AOP (e.g. technical, biological, toxicological aspects)

The scientific review should address a standard set of pre-defined charge questions(see paragraph 30 below)

18.
Figure 3 provides a simple diagram illustrating interactions and roles of the key players in the AOP
scientific review process.

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  13

Figure 3. Illustration of the AOP scientific review process

Note: The review organiser which may be supported by a review manager is responsible to establish a review panel. This panel carries out thescientific review and provides reports to the organiser. Reviewers and organiser/manager discuss the review outcome with the authors of the
AOP. Finally, the organiser/manager provides a report summarising the outcome of the scientific review. The scientific review reports are madepublically available on the AOP wiki.

3.1. The management of the review: Review organiser and review manager
19.
A review organiser is in charge of the organisation of the scientific review, as illustrated in Figure
3. It may necessitate providing financial or staff support for the various tasks to be completed. In the firstyears of the OECD AOP Programme, scientific reviews were organised by the OECD Secretariat.
However, in principle, any organisation or individual can act as a review organiser and conduct anacceptable review by following the principles listed in paragraph 17 and further described in section 3. As
appropriate, the OECD Secretariat, on an ad hoc basis, may consult with the WNT and the WPHA toidentify potential review organisers.
20.
Prospective review organisers are encouraged to inform the OECD Secretariat of their interestand intention to conduct a scientific review and in doing so can benefit from advice and potential support
from EAGMST.
21.
The review organiser may wish to delegate tasks to a review manager (e.g. internal staff,
consultant, journal editor) who ensures coordination between the reviewers during the review and asnecessary, between the AOP authors and the reviewers once reviewers have submitted their initialcomments. The review manager should have sufficient general knowledge in the field of the AOP underreview in order to be able to facilitate the review, but should not personally contribute to the review. The
review manager should have no conflict of interests associated with the AOP under review and should

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified14  ENV/CBC/MONO(2021)22
ideally be affiliated with an independent organisation (e.g. government body or scientific journal). It is
acknowledged that the role of the review manager in the coordination of the review is highly valuable.

3.2. Conflict of interest
22.
Confidence in the AOP evaluation process depends on a high scientific quality of the review andon an independent and transparent process throughout evaluation at any step. The scientific review needs
to be free of any conflict of interest in order not to undermine the credibility of a future published AOP.
23.
This Guidance Document adopts the definitions of a conflict of interest (COI) proposed by the US
National Academies (National Academies, 2003) in the context of committees developing reports, but amodel similar to that established for the Persistent Organic Pollutants Review Committee of the United
Nations (United Nations, 2019) could also be used. COI is defined as being any financial or other interest
that conflicts with the service of an individual because it (1) could significantly impair the individual's
objectivity or (2) could create an unfair competitive advantage for any person or organisation.
24.
Both financial and other interests are important (ICMJE, 2018). In the context of the AOP review,
review organiser and reviewer COIs include, in addition to financial interests, having participated in thedevelopment of the AOP under review or having a collaboration with the AOP developer or being from thesame team or laboratory as the AOP developer, because this is susceptible to impair the individual's
objectivity (see paragraph 23).
25.
The review organiser’s responsibility is to ensure the quality and integrity of the review. A review
should not be organised by a group/entity with a potential COI (i.e. groups/entities directly impacted
financially or otherwise by the outcome of the review), which could result in the review being compromised.
It is thus recommended that the review organiser requests EAGMST to check the absence of COI of thereview organiser prior to the scientific review. Otherwise, absence of COI of the review organiser will bechecked by EAGMST after the scientific review, before the AOP is submitted to WNT and WPHA for
endorsement. A proposed declaration of interest form is available in Annex 2. Other conflict of interest
forms that capture potential relevant conflicts could be utilised (e.g. The International Committee of Medical
Journal Editors disclosure of interest checklist, Persistent Organic Pollutants Review Committee of theUnited Nations, World Health Organisation declaration of interests for WHO experts, National Academiesof Science Conflict of interest policies and procedures).
26.
Review managers and reviewers have to declare any interests that are relevant to the functions tobe performed in the context of the review, in the form of a declaration of interest or a COI questionnaire on
potential conflicts. This form serves to document whether interests are of a significantly conflicting nature,or may be perceived as such, thus potentially preventing participation in the review process of the AOP
under consideration. The forms will be analysed by the review manager or organiser and the review reportwill include a general statement that the review organiser can confirm that there are no potential COIs of
reviewers based on the analysis of the declarations made by the reviewers. If there is a COI, participationin the review is not possible. A proposed declaration of interest form is available in Annex 2.

3.3. Call for reviewers and recruitment of candidate reviewers
27.
It is up to the review organiser to launch a call for relevant reviewers and contact a broad networkof experts via various appropriate channels. If the credentials and qualifications of the candidate reviewers
are not publicly available, a short CV will be requested. Situations where an AOP developer suggests
potential reviewers should be avoided.

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  15

28.
The call for reviewers can be made via various channels depending on the organisation
responsible for the scientific review. For example, when the review is organised by the OECD Secretariat,
the WNT is invited to nominate relevant experts via the different expert groups of the Test Guideline
programme; independent scientific societies with access to subject matter expertise may also be solicited.
The WNT and the WPHA will be consulted when a call for reviewers is circulated, in order to be given thepossibility to nominate experts and thus broaden the pool of candidates for the review. This can be donein coordination with the OECD Secretariat.

3.4. Selection of reviewers from the pool of candidate reviewers
29.
Following the call for reviewers, a balanced Review panel for the scientific review will beestablished by the review organiser. The EAGMST recommends Review panels of 3 to 5 reviewers. This
ensures diversity of opinions but also a manageable and efficient process. The number of reviewers may
be adapted to provide sufficient expertise to evaluate the entire pathway being reviewed.
30.
The selection of the reviewers will be based on both individual and Review panel criteria. Priorwork with AOPs is not required, since the review of the AOP should focus on the scientific aspects
presented in this AOP. Once enroled in the Review panel though, the reviewers will familiarise themselveswith AOP principles (see para graph 31).
- Individual criteria:oThe reviewers’ expertise should be relevant to the AOP under review, notonly for the hazard/endpoint (i.e. the AO) but at the various levels of
biological organisation of the key events leading to the AO. This can beevaluated, for example, by examining the publications that a candidate
reviewer has authored or co-authored and their relevance to the AOP
under review;o

The reviewers should have no conflict of interest.

- Review panel criteriaoAppropriate collective scientific expertise of the Review panel is needed toensure that the various parts of the AOP are covered and can be fully
evaluated, i.e. addressing the various levels of biological organisation
characterised within the AOP for the relevant endpoint;oBalanced representation of research and regulatory fields and affiliation(country/region level) is desirable, but should not be achieved at theexpense of scientific expertise.

31.
Prior to the review, the review organiser should ensure that the reviewers are sufficiently informedof the AOP conventions. This could be done by consulting some of the training material available on-line.
Videos, slide presentations, kick off review webinars, OECD webinars on AOPs, and/or the online training
course could for example be consulted. This is available from the AOP-Wiki Forum, section on AOP
training, which includes current training resources. In addition, a full list of available AOP training at the
Animal Free Safety Assessment is available under the Tox21 workstream, at this link:
https://www.afsacollaboration.org/tox21/get-trained/#training-resources.
Readingtheavailable
documentation, especially the Guidance Document on Developing and Assessing Adverse Outcome
Pathways (OECD, 2017), the Users’ Handbook (OECD, 2018) and the Frequently Asked Questions(FAQs) is also strongly recommended.

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified16  ENV/CBC/MONO(2021)22

3.5. The review process
32.
The online version of an AOP on the AOP-Wiki can evolve over time. Therefore, the AOP-KB offers
the possibility to generate a “snapshot” of the AOP at a certain point in time, in the form of a versioned,
date-stamped PDF document. An AOP snapshot generated before the start of the review is the documentof reference that should be used for the review.
33.

The scientific reviewers’ tasks are to:Take note of comments that may have been made on previous versions of the AOP
during the compliance check; these are available in the discussion pages of the
AOP in the AOP-KB;



Review the scientific evidence that has been presented to substantiate the AOP;

Respond specifically to the following charge questions (additional questions may
be added on a case-by-case basis, for example depending on the outcome of theUsers’ Handbook compliance check):
1. Scientific quality:o

Does the AOP incorporate all appropriate scientific literature andevidence?

o

Does the scientific content of the AOP reflect current scientific knowledge
on this specific topic?

2. Weight of evidence (WoE):oIs the WoE judgement/scoring well described and justified based on theevidence presented? If not please explain.

oPlease consider WoE for each Key Event Relationship (KER) and for the
AOP as a whole.

34.
Assessment criteria for performing the scientific review are provided in the Users’ Handbook(OECD, 2018). Reviewers should avoid recommending changes that, if adopted, would cause the AOP
under review to lose compliance with the Users’ Handbook or with the Guidance Document.
35.
The reviewers send written responses and comments back to the review organiser, who is
responsible for review collation and correspondence with the AOP authors.
36.
It is recommended that during the review phase and until the submission of the reviewers’comments to the organiser of the review, the AOP authors are not informed of the membership of the
Review panel and any direct communication between the reviewers and the authors should be avoided.
After AOP organisers have received the reviewers’ comments however, interactions between authors andreviewers can be organised and are encouraged. Bringing together the reviewers and the AOP authors
has proven to be helpful because it enables them to discuss, exchange views and share experience about
the AOP after the review. It also facilitates the revision of the AOP by their authors, in line with thereviewers’ comments.
37.
The reviewers’ comments are processed by the review organiser according to its organisation's
standard process. If no formal process exists, the outcome of the scientific review could take the form of ascientific review report (see example of a template in Annex 1).

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  17

3.6. Public information related to the reviewers
38.
The names of the reviewers, their affiliation and comments will be made publicly available on the
AOP-Wiki after completion of the review. The EAGMST recommends that the reviewers’ names beprovided as an AOP specific list, but not associated with their individual comments. This option was chosen
as a compromise between having individual comments linked to individual reviewers (which somereviewers may not feel comfortable with as the comments become publicly available) and having a masterlist of reviewers across all AOPs (which may dilute the visibility of a reviewer's participation). The reviewers
should be informed about this before the start of the review.
39.
Reviewers are considered as critical contributors to AOP development and thus participation in
the review of an AOP is regarded as a notable scholarly activity and achievement. Consequently, reviewers
are encouraged to cite their contributions to the review of AOPs as part of their professional credentials(e.g., listed on a CV as evidence of scholarly activity and expertise).

4. Subsequent AOP endorsement byOECD
40.
At the end of the review process, the authors of an AOP in the OECD AOP development work plan
will be invited either by the review organiser or by the OECD Secretariat to revise their AOP in order tomove to the next step, i.e. endorsement by OECD and subsequent publication of their AOP on the OECDpublic website. The AOP revision should take into consideration the comments from the Review panel andthe AOP authors should provide written responses addressing the comments. Responses to reviewer
comments will also be made publicly available on the AOP-Wiki.
41.
The comments from the scientific review, responses from the AOP author, and the revised AOP
are collated by the review organiser and submitted to EAGMST. EAGMST will then determine if the AOP
can move to the next endorsement step. This decision is based on the verification that therecommendations from the scientific review have been adequately addressed in the revised AOP (anycomments made at this stage are uploaded in the AOP discussion page in the AOP-Wiki). If this is thecase, the AOP can be released to the WNT and the WPHA for endorsement by written procedure 1. If thisis not the case, further work may be needed from the AOP authors, as appropriate.
42.
The OECD procedure that is described below applies to chemical-relevant AOPs and ionisingradiations induced AOPs. Before their publication, AOPs may require other procedures for endorsement,
depending on the overseeing regulatory bodies and the context of application.
43.
The WNT and WPHA may consult their expert networks during the endorsement phase andquestions or comments may be submitted to the AOP authors. The scope of AOP endorsement by the
WNT and the WPHA was clarified in 2016, and the WNT and WPHA agreed on a disclaimer, which has
subsequently been updated and which is included in the foreword of published AOPs:
1

Regarding radiation-related or radiation-induced AOPs, the NEA CRPPH will be the standing Technical Committee
in charge of the endorsement.

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

18  ENV/CBC/MONO(2021)22“This Adverse Outcome Pathway (AOP) has been developed under the auspices of the OECD AOP
Development Programme, overseen by the Extended Advisory Group on Molecular Screening andToxicogenomics (EAGMST), which is an advisory group under the Working Group of the National
Coordinators for the Test Guidelines Programme (WNT). The AOP has been reviewed internally by the
EAGMST, its scientific review has been conducted following the principles established in the GuidanceDocument for the scientific review of Adverse Outcome Pathways, and it has been endorsed by [the WNTand the Working Party on Hazard Assessment (WPHA)]2 [the Nuclear Energy Agency Committee ofRadiological Protection and Public Health (CRPPH) in parallel to the WNT and the WPHA]3.
Through endorsement of this AOP, [the WNT and the WPHA]2 [the CRPPH, as well as the WNT and the
WPHA]3 express confidence in the scientific review process that the AOP has undergone and accept therecommendation of the EAGMST that the AOP be disseminated publicly. Endorsement does notnecessarily indicate that the AOP is now considered a tool for direct regulatory application.”
44.
After WNT and WPHA endorsement, the last step in the AOP development process (as depicted
in Figure 2), is declassification4 by the OECD's Chemicals and Biotechnology Committee (CBC) followedby publication on the OECD public website on the OECD Series on Adverse Outcome Pathways. As far
as ionising radiation exposure is concerned in the AOP, after the CRPPH endorsement, the last step in
the AOP development process is declassification by the NEA followed by publication on the OECD Series
on Adverse Outcome Pathways. The three-step endorsement of the AOP (EAGMST, WNT/WPHA andCBC) can involve further need for AOP authors to address OECD-specific revisions.
45.
AOPs are viewed as living documents. Crowdsourcing is one principle of the AOP Development
programme, and AOPs thus may continue to evolve on the AOP-KB after their OECD endorsement andpublication, as new evidences supporting or rejecting AOPs are generated and/or new knowledge isgained. The purpose of publication in the Series on AOPs is to provide a stable version over time, i.e. theversion which has been reviewed and revised based on the outcome of the review. An AOP published in
the OECD series on AOPs may be considered for update when significant additional information isavailable in the AOP-Wiki that justifies the update.

2

For AOPs induced by chemical stressors

3

For AOPs induced by ionising radiations

4

Official OECD information shall be either unclassified or classified as: a) For Official Use -- for information whichshould not be communicated except for official purposes; or b) Confidential -- for information the unauthorised
disclosure of which would seriously prejudice the interest of the Organisation or any of its Member countries. When aclassified official document under the responsibility of the CBC is ready for publication, the CBC is responsible for its
declassification, such that it obtains an unclassified status and can be published.

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  19

References
EFSA-ECHA (2018), Guidance for the identification of endocrine disruptors in the context of Regulations(EU) No 528/2012 and (EC) No 1107/2009, EFSA Journal 16(6):5311, DOI: 10.2903/j.efsa.2018.5311
ICMJE5 (2018), International Committee of Medical Journal Editors, Author Responsibilities—Conflicts ofInterest[Webpageavailableat
http://www.icmje.org/recommendations/browse/roles-andresponsibilities/author-responsibilities--conflicts-of-interest.html ]National Academies, National Academy of Sciences, National Academy of Engineering, Institute ofMedicine, National Research Council (2003), “Policy on committee composition and balance and conflictsof interest for committees used in the development of reports”.
[Available athttps://www.nationalacademies.org/about/institutional-policies-and-procedures/conflict-of-interestpolicies-and-procedures ; Policy on Committee Composition and Balance and Conflicts of Interest ]United Nations. SC-1/8: Rules of procedure for preventing and dealing with conflicts of interest relating toactivities of the Persistent Organic Pollutants Review Committee [Available at (select SC-1/8 in thelanguageofchoice):http://www.pops.int/%20TheConvention/POPsReviewCommittee/%20OverviewandMandate/tabid/2806/D
efault.aspx]
World Health Organisation Declaration of Interests
https://www.who.int/about/doi-form-blank-en.doc?ua=1]

for

WHO

experts[AvailableatOECD (2012), “The Adverse Outcome Pathway for Skin Sensitisation Initiated by Covalent Binding toProteins”, OECD Series on Testing and Assessment, No. 168, OECD Publishing, Paris.
OECD (2016a), “Guidance Document for the use of Adverse Outcome Pathways in developing Integrated
Approaches to Testing and Assessment (IATA)”, OECD Series on Testing and Assessment, No. 260,
OECD Publishing, Paris.
OECD (2016b), “Guidance Document on the Reporting of Defined Approaches and Individual Information
Sources to be Used Within Integrated Approaches to Testing and Assessment (IATA) for Skin
Sensitisation”, OECD Series on Testing and Assessment, No. 256, OECD Publishing, Paris.
OECD (2017), “Revised Guidance Document on Developing and Assessing Adverse Outcome Pathways”,
OECD Series on Testing and Assessment, No. 184, OECD Publishing, Paris.
OECD (2018), “Users' Handbook supplement to the Guidance Document for developing and assessing
Adverse Outcome Pathways”, OECD Series on Adverse Outcome Pathways, No. 1, OECD Publishing,Paris. http://dx.doi.org/10.1787/5jlv1m9d1g32-en
Frequently Asked Questions (FAQs): draft document

5

International Committee of Medical Journal Editors

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified20  ENV/CBC/MONO(2021)22

Annex 1: Example of a template for thedevelopment of an AOP scientific review report

Foreword

6

Abbreviations

7

1. Background: The OECD AOP Development Programme

8

1.1. Objectives of the AOP Development Programme
1.2. Development of AOPs under the OECD AOP Development Programme
The role of the Extended Advisory Group for Molecular Screening and Toxicogenomics
The various steps and interlinkages between the parties involved in AOP development

8
8
9
10

2. Pre-requisite for the AOP scientific review

11

3. The AOP Scientific review

12

3.1. The management of the review: Review organiser and review manager
3.2. Conflict of interest
3.3. Call for reviewers and recruitment of candidate reviewers
3.4. Selection of reviewers from the pool of candidate reviewers
3.5. The review process
3.6. Public information related to the reviewers

13
14
14
15
16
17

4. Subsequent AOP endorsement by OECD

17

References

19

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  21

Annex 2: Declaration of interest

Conflict of Interest Questionnaire6:
The following questionnaire is intended to ensure that individuals who organise, manage, or
participate in the scientific review of an adverse outcome pathway (AOP) are not compromisedby any significant conflict of interest. Conflict of interest means any financial or other interest(including intellectual property) that conflicts with the participation of an individual in particulardecisions and evaluation of the scientific content of an AOP because that interest could 1) impair
the individual’s objectivity or 2) create an unfair competitive advantage for any person ororganisation.
Conflict of interest applies to the individual’s personal interests, as well as to the interests ofothers with whom the individual has substantial common financial interests if these interests are
relevant to the functions to be performed (e.g., consider individual’s employer, business partners,
spouse and other family members, etc). Consider also the interests for whom one is acting in afiduciary or similar capacity (e.g., officer or director of a corporation, serving as a trustee).
The intent of the following questionnaire is to determine and eliminate certain potentiallycompromising situations from occurring, protecting the OECD and/or other institutions,individuals (organiser, manager, or reviewer), and the public interest.
A copy of the declaration should be retained that can be disclosed should questions of a conflictarise.
1. Information:
1.
Date:
2.
Given Name:
3.
Surname:4.
Role in review (organiser, manager, reviewer):5.
AOP number:
6.
AOP title:
2. Affiliations: please list your organisational affiliations (relevant business relationships andrelevant remunerated or volunteer non-business relationships).

6

THIS COI QUESTIONNAIRE WAS DEVELOPED BASED ON THE INTERNATIONAL COMMITTEE OF
MEDICAL JOURNAL EDITORS (ICMJE) COI CHECKLIST (HTTP://WWW.ICMJE.ORG/CONFLICTS-OFINTEREST/) AND THE UNITED NATIONS PERSISTENT ORGANIC POLLUTANTS REVIEWCOMMITTEE RULES AND PROCEDURES FOR PREVENTING AND DEALING WITH CONFLICTS OF
INTEREST RELATING TO ACTIVITIES OF THE PERSISTENT ORGANIC POLLUTANTS REVIEWCOMMITTEE..

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified22  ENV/CBC/MONO(2021)22
3. Direct links to the AOP under consideration for scientific review: these questions are about
links that existed at any time to the AOP under consideration including the stages of initialconception and planning to the present.
1.
Did you or your institution at any time receive payment or services from a third
party (government, commercial, private foundation, etc) for any aspect of the AOP
under review (including but not limited to grants, data monitoring board, study
design, AOP preparation/drafting, statistical analysis, etc.)? If yes, please list
funders.
2.
Did you or your institution at any time directly fund or provide services (whether
for free or for payment) for any aspect of the AOP under review (including but notlimited to grants, data monitoring board, study design, AOP preparation/drafting,statistical analysis, etc.)? If yes, please explain.
3.
For a) and b) above, please list:
1.
Name of entity
2.
Type of funding (grant, personal fees, non-financial support, other)3.
Provide any relevant comments
4.
If you have nothing to report, please state “Neither I nor any institution with which
I am affiliated have received direct financial payment, financial or otherwise forwork directly related to the development of the AOP under consideration for
scientific review”
4. Relevant financial activities outside the AOP considered for scientific review: This section
asks about financial interests outside of this AOP that could be perceived to influence, or
give appearance of potentially influencing the scientific review, organisation, ormanagement of the review of the AOP under review. All sources of revenue paid (orpromised to be paid) directly to you or your institution on your behalf over the 36 months
prior to review of the AOP under consideration (including all monies from sources with
relevance to the specific AOP under review). Also list interactions with the AOP’s sponsor
that are outside the submitted AOP. If unsure, you must disclose the relationship for
review. For grants received related to the content of this AOP, you should disclose
support from entities that could be impacted financially by publishing the AOP underreview or that could reasonably be perceived to be affected financially by publishing the
AOP under review. This section would include any conflicts of interest identified during
current or previous employment by a government agency (e.g. national conflicts ofinterest restrictions that may be applicable to service in connection to this activity, is thisproject sponsored by your current government agency or other employer or sponsor?).
1.
Indicate whether you have financial relationships (regardless of the amount ofcompensation) with entities described above. List each entity you have had arelationship with in the last 36 months prior to this review process.
1.
Name of entity
2.
Type of funding (grant, personal fees, non-financial support, other)3.
Provide any relevant comments
2.
If you have no financial activities outside of the AOP being considered for scientificreview, please state “I have had no relevant financial relationships in the last 36
months outside the AOP being considered for scientific review”
5. Intellectual property – patents and copyright. Do you or your institution have any patents,
whether planned, pending, or issued, that are relevant to this AOP?
1.
If yes, please list and describe.
2.
If no, please state, “I have no patents, planned, pending or issued that are relevantto the AOP to be reviewed”

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified

ENV/CBC/MONO(2021)22  23

6. Public statements and positions or other relevant aspects of your background or present
circumstances not addressed that might be reasonably construed by others to affect yourjudgment in organising, managing or reviewing the AOP under consideration for scientificreview. Please list any relevant articles, testimony, speeches, assigned tasks or activities
that could be perceived to indicate a position on an issue or problem relevant to the AOP
under consideration. Please list by date, title, and publication (if relevant) and a brief
description of groups or activities of concern. If none, please state "None".
7. Lobbying activities. You must disclose any activities that are considered lobbying so that they
can be considered for their potential relationship to the development and publishing of
the AOP under scientific review. Lobbying activities are those that are performed by aperson who receives compensation or reimbursement from another person, groups, or
entity for the purpose of promoting, opposing, or in any manner influencing or attempting
to influence the introduction, defeat, or enactment of legislation before any legislative
body, or the practice of promoting, opposing, or in any manner influencing or attempting
to influence the enactment, promulgation, modification, or deletion of regulations before
anyregulatorybody(http://www.ncsl.org/research/ethics/50-state-chart-lobbydefinitions.aspx). If none, please state "None".
8. Relationships not covered. Are there other relationships or activities that AOP users or the
public at large could perceive to have influenced, or that give the appearance ofpotentially influencing your ability to organise, manage the review of, or review the AOP
under consideration, e.g. participation in planned, ongoing or follow up research project
collaborations dealing with AOP and respective test method development? Have you
been involved in the development of the AOP at any stage?
1.
If yes, please list and describe, e.g. possible influence by governmental deadlines,
priorities or programs, colleagues, or program pressures, friends on staffs ofjournals/publications
2.
If no, please state “No other relationships, considerations, or circumstances exist
that present a potential conflict of interest”.
Disclosure statement:
NAME HERE has the following potential conflicts to disclose related to the LIST AOP TITLE AND
NUMBER HERE:OR
NAME HERE has nothing to disclose.
I understand that any inaccuracies or omissions, whether intentional or unintentional, could havea serious impact not only on the AOP being reviewed but on the AOP programme and the OECD
more generally. I confirm that I have carefully read and understood the above questions and thatmy responses above are accurate and complete and do not contain any misleading statements
or information. If I at any point before completion of the AOP review realise that I have omitted
information from this form, I will immediately contact [X] in order to update this form.
____________________________________________
Signature of organiser, review manager, or reviewer______________
Date

GUIDANCE DOCUMENT FOR THE SCIENTIFIC REVIEW OF ADVERSE OUTCOME PATHWAYS
Unclassified