Annex
Annex 11

SIRC-CVS Test Method
Report of the Peer Review Panelon

a JaCVAM coordinated study programme addressing thevalidation status of the SIRC-CVS test method for theprospective identification of eye irritating substances

Report completed by the Peer Review Panel on August 19, 2019.
1
1

27

Table of Contents

Executive Summary

3

Peer Review Panel Composition

4

Background

5

SIRC-CVS Test Method Definition

5

Within Laboratory Reproducibility

6Interlaboratory Transferability

6

Between Laboratory Reproducibility

6

Predictive Capacity

6

Applicability Domain

7

Performance Standards

7

Additional Comments

8

Conclusions and Recommendations

8

Acknowledgements

9References

9

Annex: Short professional bios of the member of the peer review panel

10

2
2

28

Executive Summary

The Statens Seruminstitut Rabbit Cornea–Crystal Violet Staining Cytotoxicity Test Method with
Triethanolamine (SIRC-CVS Test Method) has been proposed as an in vitro test method todiscriminate eye irritating chemicals from non-eye irritating chemicals. It could contribute to
replacing the Draize eye irritation test as being used as the first test method in a bottom-up
approach. The test method main advantages are the use of a rabbit cornea cell line and that it isrelatively simple to perform. Coordinated and sponsored by the Japanese Center for the
Validation of Alternative Methods (JaCVAM), the SIRC-CVS Test Method was prospectively
validated by a Validation Management Team (VMT).
The peer review panel (PRP) found the Validation Management Team’s report presented thenecessary information for an independent review. The PRP concluded that the SIRC-CVS Test
Method was, with the exception of the applicability domain, sufficiently well defined, with a clear
protocol and criteria for data interpretation. Both within and between laboratory reproducibility
information were considered to be satisfactory. However, the predictive capacity could not be
assessed, as the applicability domain of the test method was adjusted several times without
providing clear mechanistic insights and justification of the proposed domain restrictions.
Accordingly, the PRP concluded that, based on the provided information, the scientific validity of
the SIRC-CVS Test Method could not be demonstrated for its use as a part of an integrated testing
strategy for distinguishing chemicals classified from chemicals not classified according to the UN
GHS classification system.

3
3

29

Peer Review Panel Composition

Sebastian Hoffmann (chair)

seh consulting + services, Paderborn, Germany

Chantra Eskes

Swiss 3R Competence Centre, Bern, Switzerland(until August 2018 SeCAM, Agno, Switzerland)

Pertti Hakkinen

National Institutes of Health, Bethesda, USA

Tae Cheon Jeong

Yeungnam University, Gyeongsan, South Korea

Tadashi Kosaka

Institute of Environmental Toxicology, Ibaraki, Japan

Jill Merrill

U.S. Food and Drug Administration, Silver Spring, USA

Sanae Takeuchi

P&G Innovation Godo Kaisha, Kobe, Japan

4
4

30

Background
The SIRC-CVS Test Method was evaluated in a prospective validation study by a Validation
Management Team (VMT), which was chaired, coordinated and sponsored by the JapaneseCenter for the Validation of Alternative Methods (JaCVAM). The study was designed to assessthe usefulness of the SIRC-CVS Test Method as an alternative to the in vivo Draize test method
to identify ocular non-irritants in a bottom-up testing strategy approach according to the UN
GHS classification scheme (Scott et al., 2010).
The PRP was assembled at the end of 2014 and met in March 2015 to discuss the outcomeof the validation study with the VMT, and to review, in the absence of the VMT, the validation
study of the SIRC-CVS Test Method. The peer review was conducted based on 14 evaluation
criteria. As these criteria were derived from the modular approach (Hartung et al., 2014), the
PRP review is structured by the seven validation modules. Upon repeated requests forclarifications by the PRP on a number of elements, but most importantly on the applicability
domain, the VMT provided replies that were discussed via several teleconferences and meetingsfrom 2015 to 2018. In March 2019, the PRP held a final telephone conference. With the provisionof all of the amended, updated and additional material, including the final VMT report, this PRP
report was prepared.

SIRC-CVS Test Method Definition
The PRP concluded that the SIRC-CVS Test Method has been fully described in the report of
the Validation Management Team (VMT) and in the associated detailed test protocol. Protocol
amendments made during and after the validation study were traceable over the various
versions. The validation report adequately stated the need for the assay in the current regulatory
context. Furthermore, a rationale for the test method has been given and helpfully included
references to existing in vitro methods for eye irritation that have been validated and adopted
into OECD guidelines. The PRP agreed that the mechanistic basis of the method and how it
related to eye irritation was sufficiently described in the report although further details would
have been appreciated.
The PRP agreed that this method is intended to contribute to the replacement of animal
usage for eye irritation assessment and that, when compared to other in vitro cell-based
methods, which aim at discriminating irritating from non-irritating chemicals, it is likely to offer
time, throughput, and cost benefits. Furthermore, the PRP was of the opinion that the use of
corneal cells in the SIRC-CVS Test Method can present mechanistic advantages as compared toother cell types for predicting ocular irritation.

5
5

31

Within Laboratory Reproducibility
Based on information provided in the VMT report, the PRP agreed that the results
demonstrated a good within-laboratory reproducibility. The PRP achieved this conclusion on thebasis of the data of Phase II of the validation study, in which 20 substances were tested threetimes in three laboratories, regardless of any applicability domain considerations. As all
substances were consistently predicted in the laboratories, resulting in a within-laboratory
reproducibility of 100%, the success criterion of a within-laboratory reproducibility of at least
80% set by the VMT was met.
In addition, the results provided in the validation report indicate that bovine serum and TEA
from different manufacturing lots have no effect on the reproducibility.

Transferability
The PRP noted that the fact that the three participating laboratories were all naïve and thatno practical training was provided is a good indication of the robustness of the test method.

Between Laboratory Reproducibility
As with within laboratory reproducibility, the PRP based its evaluation of the betweenlaboratory reproducibility on the information provided in the VMT report. The PRP agreed that theresults demonstrated a good between-laboratory reproducibility. The PRP achieved this conclusionon the basis of the data of Phase II and of 10 substances of Phase III of the validation study, inwhich 30 substances were tested in three laboratories, regardless of any applicability domain
considerations. As 27 substances were consistently predicted across the three laboratories,
resulting in a between-laboratory reproducibility of 90%, the success criterion of a betweenlaboratory reproducibility of at least 80% set by the VMT was met.

Predictive Capacity
Demonstration of a test method’s performance should be based on the testing ofrepresentative, preferably coded, reference chemicals. The PRP concluded that the validation
study used an appropriate level of test chemical coding to ensure fully blinded evaluation. With
respect to predictive capacity, the PRP confirms that a suitable balance of stronger, weaker, andnon-classified (according to the GHS classes of 1, 2A, 2B and no class) test chemicals was selected.
The PRP agreed that the rabbit was considered as the target species, noting that prediction of6632

human eye irritation is the ultimate goal and acknowledging that high quality human data are in
general not available. The PRP notes that in vivo data have been reviewed by ICCVAM.
Additionally, the majority of test chemicals were also used in other validation studies (e.g. of the
RhCE and STE). Finally, the majority of tested chemicals differed from those used in initial
method development.
The PRP agreed that the predictive capacity for an unrestricted applicability domain was
based on a sufficiently large and representative set of test chemicals. However, the accuracy of
55% (64/116) did not meet the VMT success criteria of 80%, and the false negative rate of 40%(28/70) did not meet the success criterion of <5%.
The PRP understood that this poor predictive performance triggered the test developer toexplore various physical and chemical properties to identify possible reasons for
misclassification. Various restricted applicability domains, all of which resulted in the exclusionof a substantial amount of substances, but also in improved predictive performance, were
presented to the PRP. However, the VMT failed to provide a clear mechanistic justification for
the restrictions. Consequently, the PRP could not conclusively assess the predictive capacity of
the SIRC-CVS Test Method.

Applicability Domain
Due to unsatisfactory predictive capacity with an unrestricted applicability domain, the VMTexplored different potential reasons for misclassification, including chemical classes andphysicochemical properties, but not in vivo drivers of classification as suggested by the PRP(Adriaens et al., 2014). The proposals presented to the PRP restricted the applicability, for example
using a combination of molecular weight and chemical classes, the dissociation constant (pKa) andthe distribution coefficient (log D). However, the VMT failed to provide a clear mechanistic
justification for any of these. Consequently, the PRP concluded that the applicability domain of the
SIRC-CVS Test Method was not sufficiently defined.

Performance Standards
Because the assay does not include components, equipment, or other scientific proceduresthat are covered by (or pending) intellectual property rights, the PRP initially agreed that
performance standards are not mandatory at this stage, but could be useful if similar or modified
test methods become available. However, due to the lack of a clearly justified applicability domain,
performance standards are not required.

7
7

33

Additional Comments
The PRP concluded that the validation study management and conduct met the criteria set
out in OECD GD 34 (2005). However, based on the information provided to the PRP, including a
dedicated discussion at a PRP meeting, the PRP concluded also that it is unclear whether the
study was conducted in accordance with the principles of GLP.
The PRP notes that during the conduct of the review, access to the full raw data files
associated with SIRC-CVS Test Method validation work was provided.

Conclusions and Recommendations
The PRP concluded that, based on the provided information, the scientific validity of the
SIRC-CVS Test Method could not be demonstrated for its use as a part of an integrated testing
strategy for distinguishing chemicals classified from chemicals not classified according to the UN
GHS classification system.

8
8

34

Acknowledgements
The PRP is grateful to the members of the VMT for their hard work and patience and to
JaCVAM for their support in setting up and hosting meetings in Japan, as well as for the setting
up of several telephone conferences.

References
Adriaens E, Barroso J, Eskes C, Hoffmann S, McNamee P, Alépée N, Bessou-Touya S, De Smedt A,
De Wever B, Pfannenbecker U, Tailhardat M, Zuang V. (2014). Retrospective analysis of the
Draize test for serious eye damage/eye irritation: importance of understanding the in vivo
endpoints under UN GHS/EU CLP for the development and evaluation of in vitro test methods.
Arch Toxicol. 88(3):701-723.
Hartung T, Bremer S, Casati S, Coecke S, Corvi R, Fortaner S, Gribaldo L, Halder M, Hoffmann S, Roi
AJ, Prieto P, Sabbioni E, Scott L, Worth A, Zuang V. (2004). A modular approach to the ECVAM
principles on test validity. Altern Lab Anim. 32(5):467-472.
OECD (2005). Guidance document on the validation and international acceptance of new or
updated test methods for hazard assessment. OECD Series on Testing and Assessment
Number 34. ENV/JM/MONO(2005)14, Paris.
Scott L, Eskes C, Hoffmann S, Adriaens E, Alepée N, Bufo M, Clothier R, Facchini D, Faller C, Guest
R, Harbell J, Hartung T, Kamp H, Varlet BL, Meloni M, McNamee P, Osborne R, Pape W,
Pfannenbecker U, Prinsen M, Seaman C, Spielmann H, Stokes W, Trouba K, Berghe CV,
Goethem FV, Vassallo M, Vinardell P, Zuang V. (2010). A proposed eye irritation testing
strategy to reduce and replace in vivo studies using Bottom-Up and Top-Down approaches.
Toxicol In Vitro.24(1):1-9.

9
9

35

Annex: Short professional bios of the member of the peer review panel
Dr. Sebastian Hoffmann (PhD, ERT)
is running the independent consultancy ‘seh consulting + services‘ in Paderborn, Germany, since2009, specialised in validation and assessment of new approach methods (NAM) and test
strategies as well as in the regulation of chemicals (REACH) and cosmetic ingredients. Being
statistician by training, he received a Ph.D. from the University of Konstanz (Germany) in 2005
for his thesis ‘Evidence-based in vitro toxicology’. He has being working for the European Centre
for the Validation of Alternative Methods of the European Commission for five years,
contributing inter alia to the management of various validation studies of NAM. His research
activities focus on methodological challenges in the assessment of in vitro tests, theconstruction and evaluation of integrated testing strategies and in exploring the application ofevidence-based approaches to toxicology. He is an appointed member of the European
Commision’s Scientific Committee on Occupational Exposure Limits and member of the Boardof Trustees of the Evidence-based Toxicology Collaboration.
Dr. Chantra Eskes (PhD)
is director of the Swiss 3R Competence Centre (www.swiss3rcc.org). With a background in foodsciences, in vitro neurotoxicity and topical toxicity, Chantra has over 20 years of experience inthe development, optimization, validation, peer-review and regulatory acceptance ofalternative methods to animal toxicity testing conducted in Europe and abroad. Her activities
contributed to the formal validation and international acceptance of a number of test methods,
and to the development of a number of international test guidelines and guidance documents.
Chantra currently acts as the chairwoman of the EURL-ECVAM Scientific Advisory Committee(ESAC), and as a Swiss Nominated Expert for the Organisation for Economic Co-operation andDevelopment (OECD). In the past she has been the president of the European Society of
Toxicology In Vitro and Executive Secretary of the Animal Cell Technology Industrial Platform on
the production of biopharmaceuticals (ACTIP).
Dr. Pertti Hakkinen (PhD, S-FRA)
is the Senior Toxicologist and the Toxicology and Environmental Health Science Advisor at the(U.S.) National Institutes of Health’s National Library of Medicine. His job includes providing
leadership in the development of new resources in exposure science, toxicology, risk assessment,
and risk communication, and enhancements to existing resources in these fields. In addition, heis the project leader for the Chemical Hazards Emergency Medical Management (CHEMM) tooland the recently updated and enhanced ToxTutor educational resource. Further, he is an Adjunct
Associate Professor in Preventive Medicine & Biostatistics in the F. Edward Hébert School of
Medicine at the Uniformed Services University of the Health Sciences (USU) in Bethesda, and aco-leader of the Environmental Health Sciences graduate level course offered by the Foundation
for Advanced Education in the Sciences at the NIH (FAES@NIH). Dr. Hakkinen earned a B.A. in
Biochemistry and Molecular Biology from the University of California, Santa Barbara, andreceived a Ph.D. in Comparative Pharmacology and Toxicology from the University of California,
10
10

36

San Francisco. He is a member of the Society of Toxicology, a Councilor of the International
Society of Exposure Science, and a Fellow of the Society for Risk Analysis. Recent publications
include a) Exploring Global Exposure Factors: Resources for Use in Consumer Exposure
Assessments and b) New Studies About Everyday Types of Chemical Exposures: What Readers
Should Consider. He is a co-editor of an upcoming (mid-2019) book on the Practice of Consumer
Exposure Assessment.
Prof. Tae Cheon Jeong (PhD)
is a professor of toxicology in College of Pharmacy at Yeungnam University, Gyeongsan, Republicof Korea, since 2000. He majored in pharmacy at Sungkyunkwan University in 1986 and received
his Master and Ph.D. degree (1992) at the Korea Advanced Institute of Science and Technology(KAIST), majoring toxicology. As a post-doctoral fellow, he worked with Dr. Michael P. Holsapple
in the Department of Pharmacology and Toxicology at the Medical College of Virginia, located in
Richmond, Virginia, USA. Thereafter, he worked for the Korea Institute of Toxicology (KIT) as asenior scientist for 6 years. His research area has been the toxicology in liver and immune
systems and the development of alternative testing methods. He has published 227 research
papers since 1992. He is the president of the Korean Society for Alternatives to Animal
Experiments (KSAAE) at present.
Dr. Jill C. Merrill (PhD, DABT)
is a senior reviewer for the Division of Dermatology and Dental Products, Center for Drug
Evaluation and Research (CDER), USFDA. She received her doctorate in toxicology from Texas
A&M University and was a postdoctoral fellow at the University of Texas Southwestern Medical
Center and at Monsanto Co. In 1990 she joined the Gillette Medical Evaluation Laboratories as a
product safety toxicologist. Under Gillette’s no animal testing policy she coordinated thecompany’s first use of the BCOP assay to further the development of innovative products
otherwise sidelined by ocular irritation concerns. Subsequently she served as Study Director(2000-2003) at the Institute for In Vitro Sciences, Inc., specializing in alternative ocular anddermal irritation assays conducted under contract for pharmaceutical and personal carecompanies. Since 2003 she has evaluated the significance of results from both nonclinical andalternative toxicology studies conducted in support of dermal drug products prior to first-inhuman studies. Dr. Merrill proactively supports CDER’s use of alternative test methods.
Sanae Takeuchi (MSc)
joined Procter & Gamble Far East Inc. (current: P&G Innovation Godo Kaisha) in 1992 aftergetting her master’s degree of science in biology from Ochanomizu University in Tokyo. After
working in Clinical Development department for years, she has been working for Central Product
Safety function in Global Product Stewardship, since 2001, supporting Beauty Care sector.
From 2008, she has been working with JaCVAM (Japanese Center for the Validation of
Alternative Methods) as a member of Editorial Committee of alternatives for ocular irritation
testing. She is a member of International Committee and Council for The Japanese Society for
Alternatives to Animal Experiments, and also a member of the Japan SOT.
11
11

37

Annex22
Annex

SIRC-CVS:TEA の予測性一覧
表 1．
．Overall analysis by the judgment based on IC50 value of Triethanolamine (TEA) in UN GHS
classification system in a bottom-up approach in the validation study
Regulatory System

Analysis

Accuracy

55.2% (64/116)

Sensitivity

60.0% (42/70)

Specificity

47.8% (22/46)

False Negative Rate

40.0% (28/70)

False Positive Rate

52.2% (24/46)

表 2．
．Analysis after cut Molecular weight <180 for alcohol, ester, ether , ketone heterocyclic compoundand carboxylic acid , and purity ≧80% (GHS, Bottom-up, TEA) in the validation study.
Regulatory System

Analysis in applicability domain

Accuracy

64.9% (37/57)

Sensitivity

92.3% (24/26)

Specificity

41.9% (13/31)

False Negative Rate

7.6% (2/26)

False Positive Rate

58.1% (18/31)

表 3．
． Analysis after cut LogP (-1.5<log D<2) and pKa 4 or less (GHS, Bottom-up, TEA) in thevalidation study(see appendix 15).
Regulatory System

Analysis in applicability domain

Accuracy

57.7% (30/52)

Sensitivity

93.8% (15/16)

Specificity

41.7% (15/36)

False Negative Rate

6.3% (1/16)

False Positive Rate

58.3% (21/36)

1

39

表 4．
．Overall analysis by the judgment based on IC50 value of Triethanolamine (TEA) in UN GHS
classification system in a bottom-up approach in the Shiseido in house 46 data (see appendix 16) andvalidated data
Regulatory System

Analysis

Accuracy

57.4% (93/162)

Sensitivity

64.8% (59/91)

Specificity

47.9% (34/71)

False Negative Rate

35.2% (32/91)

False Positive Rate

52.1% (37/71)

表 5．
．Analysis after cut Molecular weight <180 for alcohol, ester, ether , ketone heterocyclic compoundand carboxylic acid , and purity ≧80% (GHS, Bottom-up, TEA) in the Shiseido in house 22 data (seeappendix 11) and validated data
Regulatory System

Analysis in applicability domain

Accuracy

64.6% (51/79)

Sensitivity

94.6% (35/37)

Specificity

38.1% (16/42)

False Negative Rate

5.4% (2/37)

False Positive Rate

61.9% (26/42)

表 6．
． Analysis after cut LogP (-1.5<log D<2) and pKa 4 or less (GHS, Bottom-up, TEA) in thevalidation study(see appendix 15) (GHS, Bottom-up, TEA) in the Shiseido in house 31 data (seeappendix 16) and validated data
Regulatory System

Analysis in applicability domain

Accuracy

65.1% (41/63)

Sensitivity

96.3% (26/27)

Specificity

42.9% (24/56)

False Negative Rate

3.7% (1/27)

False Positive Rate

57.1% (32/56)

2

40

表 7．
．Overall analysis by the judgment based on IC50 value of Triethanolamine (TEA) in UN GHS
classification system in a bottom-up approach in the validated data based on Barroso’ data (seeappendix 14)
Regulatory System

Analysis

Accuracy

50.0% (49/98)

Sensitivity

52.9% (27/51)

Specificity

46.8% (22/47)

False Negative Rate

47.1% (24/51)

False Positive Rate

53.2% (25/47)

表 8．
．Analysis after cut Molecular weight <180 for alcohol, ester, ether , ketone heterocyclic compoundand carboxylic acid , and purity ≧80% (GHS, Bottom-up, TEA) in the validated data based on
Barroso’ data (see appendix 14)
Regulatory System

Analysis in applicability domain

Accuracy

56.5% (26/46)

Sensitivity

87.5% (14/16)

Specificity

40.0% (12/30)

False Negative Rate

12.5% (2/16)

False Positive Rate

60.0% (18/30)

341