EUROPEAN COMMISSIONJOINT RESEARCH CENTRE
Institute for Health and Consumer Protection
European Centre for the Validation of Alternative Methods (ECVAM)ECVAMSCIENTIFIC
ADVISORYCOMMITTEE(ESAC)ESAC Working Group Peer Review Consensus Reporton an ECVAM-coordinated prevalidation study
concerning three protocols of
the Cell Transformation Assay (CTA)
for in vitro carcinogenicity testing
Title page information
File name
AbbreviatedrequestTEMPLATE_ESAC-WG_REPORT.doc
title

of

ESAC

ESAC peer review of and ESAC opinion on the ECVAM-led study ofthree cell transformation assays for carcinogenicity testing:1) SHE pH 6.7 assay
2) SHE pH 7.0 assay
3) BALB/c 3T3 assay.
Relating to ESAC REQUEST Nr.
2010-02
Request discussed through
Plenary at ESAC33 at 2010/10/12
Report to be handed over to 25.01.2011
ESAC Chair and Secretariat by
Version tracking
File name1102-18 ESAC WG CTA_approved.doc

TABLE OF CONTENTS
ESAC WORKING GROUP....................................................................................................................................4
NOTE ON THIS REPORTING TEMPLATE..............................................................................................................5
ABBREVIATIONS USED IN THE DOCUMENT.......................................................................................................5
EXECUTIVE SUMMARY......................................................................................................................................6
1. DATA COLLECTION ........................................................................................................................................7
1.1 INFORMATION / DATA SOURCES USED ..............................................................................................................7
1.2 SEARCH STRATEGY .......................................................................................................................................8
1.3 SELECTION CRITERIA APPLIED TO THE AVAILABLE INFORMATION ..............................................................................9
2. STUDY OBJECTIVE AND DESIGN ..................................................................................................................10
2.1 CLARITY OF THE DEFINITION OF THE STUDY OBJECTIVE ........................................................................................10
2.2 ANALYSIS OF THE SCIENTIFIC RATIONALE PROVIDED ...........................................................................................10
2.3 ANALYSIS OF THE REGULATORY RATIONALE PROVIDED ........................................................................................11
2.4 APPROPRIATENESS OF THE STUDY DESIGN .......................................................................................................12
2.5 APPROPRIATENESS OF THE STATISTICAL EVALUATION .........................................................................................14
3. TEST DEFINITION (MODULE 1) ....................................................................................................................17
3.1 QUALITY AND COMPLETENESS OF THE OVERALL TEST DEFINITION ..........................................................................17
3.2 QUALITY OF THE BACKGROUND PROVIDED CONCERNING THE PURPOSE OF THE TEST METHOD ......................................18
3.3 QUALITY OF THE DOCUMENTATION AND COMPLETENESS OF SOPS AND PREDICTION MODELS ......................................18
4. DATA QUALITY............................................................................................................................................20
4.1 OVERALL QUALITY OF THE EVALUATED DATA ....................................................................................................20
4.2 SUFFICIENCY OF THE EVALUATED DATA IN VIEW OF THE STUDY OBJECTIVE ...............................................................21
4.3 QUALITY OF THE REFERENCE DATA FOR EVALUATING RELIABILITY AND RELEVANCE .....................................................22
5. TEST MATERIALS.........................................................................................................................................23
5.1 SUFFICIENCY OF THE NUMBER OF EVALUATED TEST ITEMS IN VIEW OF THE STUDY OBJECTIVE ........................................23
5.2 REPRESENTATIVENESS OF THE TEST ITEMS WITH RESPECT TO APPLICABILITY .............................................................24
6. WITHIN-LABORATORY REPRODUCIBILITY (MODULE 2) ...............................................................................30
6.1 ASSESSMENT OF REPEATABILITY AND REPRODUCIBILITY IN THE SAME LABORATORY ...................................................30
6.2 CONCLUSION ON WITHIN-LABORATORY REPRODUCIBILITY AS ASSESSED BY THE STUDY ...............................................31
7. TRANSFERABILITY (MODULE 3)...................................................................................................................32
7.1 QUALITY OF DESIGN AND ANALYSIS OF THE TRANSFER PHASE ...............................................................................32
7.2 CONCLUSION ON TRANSFERABILITY TO A SECOND LABORATORY AS ASSESSED BY THE STUDY ........................................33
8. BETWEEN-LABORATORY REPRODUCIBILITY (MODULE 4)............................................................................34
8.1 ASSESSMENT OF REPRODUCIBILITY IN DIFFERENT LABORATORIES ...........................................................................34
8.2 CONCLUSION ON REPRODUCIBILITY AS ASSESSED BY THE STUDY ............................................................................35
9. PREDICTIVE CAPACITY (MODULE 5) ............................................................................................................37
10. APPLICABILITY DOMAIN (MODULE 6) .......................................................................................................37
11. PERFORMANCE STANDARDS (MODULE 7).................................................................................................37

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 2 of 95

12. READINESS FOR STANDARDISED USE........................................................................................................38
12.1 ASSESSMENT OF THE READINESS FOR REGULATORY PURPOSES ............................................................................38
12.2. ASSESSMENT OF THE READINESS FOR OTHER USES ..........................................................................................41
12.3 CRITICAL ASPECTS IMPACTING ON STANDARDISED USE ......................................................................................41
12.4 GAP ANALYSIS ........................................................................................................................................42
13. OTHER CONSIDERATIONS .........................................................................................................................43
14. CONCLUSIONS ON THE STUDY ..................................................................................................................45
14.1 SUMMARY OF THE RESULTS AND CONCLUSIONS OF THE STUDY ............................................................................45
14.1.1 Test items ...................................................................................................................................45
14.1.2 Summary of study results............................................................................................................49
14.2 EXTENT TO WHICH STUDY CONCLUSIONS ARE JUSTIFIED BY THE STUDY RESULTS ALONE .............................................51
14.3 EXTENT TO WHICH CONCLUSIONS ARE PLAUSIBLE IN THE CONTEXT OF EXISTING INFORMATION ...................................55
15. RECOMMENDATIONS ...............................................................................................................................56
15.1 GENERAL RECOMMENDATIONS CONCERNING THE SHE ASSAYS ...........................................................................56
15.2 RECOMMENDATIONS FOR IMPROVEMENT OF THE SOPS ASSOCIATED WITH THE SHE ASSAYS ....................................58
15.3 GENERAL RECOMMENDATIONS CONCERNING THE BALB/C 3T3 ASSAY ................................................................59
15.3 RECOMMENDATIONS FOR IMPROVEMENT OF THE SOPS ASSOCIATED WITH THE BALB/C 3T3 ASSAY ..........................60
16. REFERENCES..............................................................................................................................................61
17. ANNEXES...................................................................................................................................................63
ANNEX 1 – ESAC WG ANALYSIS OF THE SHE RESULTS OF THE PREVALIDATION STUDY ....................................................63
A) Morphological transformation frequencies of vehicle and positive controls......................................63
B) Predictive capacity of the test methods used in the present prevalidation study...............................66
C) Comparison of transformation frequencies .......................................................................................67
ANNEX 2 – ANALYSIS OF SHE PROTOCOL SIMILARITY: HISTORICAL VS. PREVALIDATION ...................................................69
ANNEX 3 – PRESENTATION OF STUDY RESULTS BY THE VMT (COPIED FROM STUDY REPORTS) ..........................................72
ANNEX 4 – LIST OF DETAILED RECOMMENDATIONS FROM THE ESAC WG REPORT REGARDING THE SOPS OF THE THREE CTAS 74
A) Detailed recommendations concerning the SOP of the SHE pH6.7 CTA..............................................74
B) Detailed recommendations concerning the SOP of the SHE pH7.0 CTA..............................................77
C) Detailed recommendations concerning the SOP of the BALB/C 3T3 CTA............................................79
ANNEX 5 – ESAC REQUEST CONCERNING THE REVIEW OF THE CTA PREVALIDATION STUDY .............................................82

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 3 of 95

ESAC Working Group
This report was prepared by the "ESAC Working Group CTA" (ESAC WG), charged with conducting adetailed scientific peer review of the ECVAM prevalidation study concerning three protocols of the
Cell Transformation Assay (CTA) for carcinogenicity testing.
The ESAC WG had been set up by the ESAC during its meeting on 12 October 2010. Basis for thescientific review was the ECVAM request to ESAC concerning the scientific review (ESAC request
2010-02, see Annex 5).
The ESAC WG conducted the peer review from 14 December 2010 to 25 January 2011. This report
was endorsed by the ESAC WG on 16.2.2011 and represents the consensus view of the ESAC WG.
This ESAC WG peer review consensus report was endorsed by the ESAC on 18.2.2011.
The ESAC WG had the following members:•

Dr. Erwin ROGGEN (ESAC member, Chair of the ESAC Working Group)•

Dr. Rodger CURREN (ESAC member)•Dr. David LOVELL (invited expert; EEP1 member)•Dr. Edgar RIVEDAL (invited expert; EEP member)•Dr. Takeki TSUTSUI (invited expert following an ICATM proposal from JaCVAM; EEP member)

ESAC Secretariat:

1•

Dr. Claudius GRIESINGER (EC-ECVAM, ESAC Secretariat)•

Dr. Pascal PHRAKONKHAM (EC-ECVAM, specific support to ESAC Secretariat)

EEP = ECVAM Expert PoolESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 4 of 95

NOTE ON THIS REPORTING TEMPLATE
The template follows the ECVAM modular approach and allows at the same time for the descriptionof the analysis and conclusions concerning more specific questions. The template was approved by
the ESAC through written procedure on 29 October 2010.
The template can be used for various types of validation studies (e.g. prospective full studies,retrospective studies, performance-based studies and prevalidation studies).
Depending on the study type and the objective of the study, not all sections may be applicable.
However, for reasons of consistency and to clearly identify which information requirements have notbeen sufficiently addressed by a specific study, this template is uniformly used for the evaluation ofvalidation studies.
•

Explanatory notes to the paragraph titles (in green) have been added on 17 November 2010.
These notes provide guidance on the type of information / analysis expected under eachsection. Depending on the purpose and scope of the study to be reviewed, some of the
aspects mentioned in the explanatory notes may not be applicable or only be applicable tosome extent. Moreover, the explanatory notes are not intended to represent an exhaustive
list of possible issues to be addressed under the respective heading, but are thought toprovide some guidance with respect to the considerations typically expected.

•

For all of the template’s numbered sections the summary view of ESAC WG is given in bold
followed by more detailed comments ("general observations" and "specific observations").

ABBREVIATIONS USED IN THE DOCUMENT•

BLR

Between-laboratory reproducibility•

CTA

Cell Transformation Assay

•

ECVAM

European Centre for the Validation of Alternative Methods

•

ESAC

ECVAM Scientific Advisory Committee•

ESAC WG

ESAC Working Group

•

GCCP

Good Cell Culture Practice•

GLP

Good Laboratory Practice•HClHydrochloric acid•

MTF

Morphological Transformation Frequency

•

OECD

Organisation for Economic Cooperation and Development•

OECD DRP

OECD Detailed Review Paper•

PC

Positive Control•

SHE

Syrian hamster embryo•

SOPStandard Operating Procedure (used here as equivalent to 'protocol')•VC

Vehicle Control•

VMT

Validation Management Team•

WLR

Within-laboratory reproducibility

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 5 of 95

Executive summary
Following a request from ECVAM to ESAC for peer review of and scientific advice on an ECVAMcoordinated prevalidation study concerning three protocol variants of the Cell Transformation Assay(CTA) used as in vitro predictors of carcinogenicity, an ESAC Working Group (ESAC WG) was set up by
ESAC. The ESAC WG was charged with conducting a detailed scientific peer review of this study which
had addressed protocol refinement of these three CTA protocols and assessed protocol
transferability and reproducibility.
The ESAC WG met once in person at ECVAM in December 2010 and communicated further by email
and teleconferences in December 2010 and January 2011. The ESAC WG reviewed the prevalidation
study reports on the Syrian hamster embryo (SHE) pH6.7, SHE pH7.0 and BALB/c 3T3 CTA tests. The
ESAC WG considered the scientific work presented was of good quality, despite some weaknesses instudy design and execution.
The ESAC WG concluded that for both the SHE pH6.7 and SHE pH7.0 assays the results showed that asufficiently standardised protocol is available which appears to be transferable, and that althoughdata on within-laboratory variability was insufficient, there was reproducibility between laboratories
on the basis of the six chemicals tested. The protocols were considered appropriate to serve as abasis for an OECD Guideline. However, future work using these protocols should aim at a completecharacterisation of test method performance based on a larger set of chemicals.
Since a preliminary comparison made by the ESAC WG demonstrated appreciable similarity between
the historical SHE protocols and those produced as a result of the prevalidation study (Annex 2), it isconceivable that historical data could be used to supplement or even substitute prospective testingdata for the characterisation of test method performance. This characterisation should include
information on reproducibility, predictive capacity, applicability and limitations of the SHE protocols.
The ESAC WG concluded that an improved BALB/c 3T3 CTA protocol had been developed. The WG
considered that, although transferability had been achieved, within-laboratory reproducibility had
not been sufficiently addressed and the between-laboratory reproducibility, while promising, had notbeen fully demonstrated. The protocol was considered suitable for further development towards the
production of an OECD Guideline. However, the WG recommended that, prior to possible regulatory
use there was a need to refine the acceptance and assessment criteria for the assay and to evaluate
the test performance through dedicated test trials.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 6 of 95

1. Data collection
1.1 Information / data sources used
NOTE: (Pre)validation studies typically make use of existing data, e.g. either as reference data(prospective studies) or as reference data and testing data as well (retrospective study). Have otherdata been used during the studies that were not generated during the study? If yes, for which purpose(e.g. reference data etc.)? What were that data sources?
All reference data used in the prevalidation study are derived from the Detailed Review Paper on
Cell Transformation Assays for Detection of Chemical Carcinogens published by the OECD in 2007
(hereunder abbreviated as "OECD DRP").
The current prevalidation study makes use, for each variant of the assay, of data from six of thechemicals reported in the OECD DRP as reference data to assess reproducibility as measured throughthe concordance of predictions within and between laboratories and in reference to in vivo
carcinogenicity classifications as published in the OECD DRP, which in turn – in case of thesechemicals – refers to classifications by the International Agency for Research on Cancer (IARC), Goldand Zeiger (1997) and the U.S. National Toxicology Program (NTP) database.
Briefly, the predictions generated by the Cell Transformation Assay (CTA) protocols in this study allow
classification of the test chemical either as a "transforming agent" or as a "non-transforming agent"(for Syrian hamster embryo SHE, based on calculation of "Morphological Transformation Frequency",
MTF; for BALB/c 3T3, based on measurement of the number of type III foci). Predictions(transforming/non-transforming agent) obtained in different laboratories were assessed forconsistency (concordance) between the laboratories and were compared with in vivo carcinogenicity
data as reported in the OECD DRP as the "reference standard"– see Figure 1.
Moreover, relevant ECVAM workshop reports and recent research papers related to (a) the scoringof observed effects, (b) the mechanistic understanding and (c) between-laboratory reproducibility
are discussed in the prevalidation study reports (sections 1.4 to 1.7 pp.10-13 in all three reports), buthave not been used as reference data.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 7 of 95

Figure 1: Schematic depiction of the test procedure and the prediction generated by the SHE andBALB/c 3T3 CTAs and the reference data ("standard") used.

Cell Transformation Assays (test procedure)a) Parametermeasuredb) Parameter
expressed as

SHE assay
Number oftransformed
colonies

MorphologicalTransformation
Frequency

BALB/c assay
Number offoci

Mean numberof type III
foci per dishc) Test result: prediction(based on ‘Assay
Assessment Criteria’)

Reference data
Reference
Data (predictions
considered true)

Transforming agent in
SHE CTA (“Positive”)
Non-transforming agent in
SHE CTA (“Negative”)

Transforming agent in
BALB/c CTA (“Positive”)

In vivo carcinogenicity of
substance according toOECD DRP and based on
1) IARC
2) Gold & Zeiger3) NTP database

Non-transforming agent in
BALB/c CTA (“Negative”)

1.2 Search strategy
NOTE: How was the search for existing data planned, organised and executed? Has a search strategy
been described and consistently applied?
The ESAC working group (ESAC WG) noted that there was apparently no detailed search strategy
established for identifying suitable reference data. However, taking into account that this was asmall-scale prevalidation study which did not attempt to define the predictive capacity or theapplicability domain of the three CTAs studied, but focused on protocol refinement andreproducibility, this fact was not considered relevant in this context.
The information referred to in the CTA prevalidation study was extracted from the OECD DRP and,for purposes of discussing the background and status quo of the methods, from reports of ECVAM
workshops and to a lesser extent from recent literature.
The OECD DRP was the sole source for reference data used for assessing within- and betweenlaboratory reproducibility (measured through concordance of predictions obtained in different
laboratories in reference to published in vivo carcinogenicity predictions).
It was noted by the ESAC WG that the OECD DRP does not specify (1) what the search strategy for
retrieving literature was, (2) whether there were selection criteria (e.g. based on study quality) toselect or reject retrieved data and (3) whether studies that were originally retrieved had been indeed
rejected because of quality issues (see point 2). This may affect the quality of the data andinformation represented in the OECD DRP and thus the quality of the reference data in theprevalidation study.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 8 of 95

1.3 Selection criteria applied to the available information
NOTE: Have consistent evaluation/decision criteria been pre-defined and applied in order to select thedata and has data selection been explained in a transparent manner?
The ESAC WG noted that there was apparently no detailed set of selection criteria established toreject/accept retrieved data. The OECD DRP was taken as reliable source although it is not clearly
described in the OECD DRP how the quality of the data had been controlled.
The ESAC WG noted that for the purposes of the current prevalidation study, CTA test data from theOECD DRP and relating to six chemicals were selected on the basis of criteria established by the
Validation Management Team (VMT) for selecting test chemicals (section 2.4.1 p.16 in all threereports). Once the chemicals had been identified, the relevant CTA reference data were extracted
from the OECD DRP, apparently without further critical review of the data quality. Thus, there was no
quality control of the reference data per se.
The OECD DRP does not mention whether the data reported were analysed for their quality andwhether some of the data retrieved were rejected (i.e. on the basis of predefined minimum criteria
for study quality). Thus, the issue of a lack of control on the reference data may have been
propagated through to the prevalidation study.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 9 of 95

2. Study objective and design
2.1 Clarity of the definition of the study objective
NOTE: Is the objective of the study clearly and comprehensibly defined?
The objective of the studies was considered clear and comprehensive: standardisation of CTA
protocols and subsequent assessment of these protocols for reproducibility and transferability.
The objective was to establish a harmonized standard operation procedure (SOP) for each of theselected CTAs and to assess whether these method protocols are reproducible within laboratories,
transferable and reproducible between laboratories.
It is stated in the study objective, that this information may facilitate the development of an OECD
guideline based on these tests. The precise purpose of a possible future test guideline using CTAs is,however, not further elaborated on. In the opinion of the ESAC WG this may have affected the study
design. For instance, considerations of regulatory needs, for instance the need to have additional
information on predictive capacity, could have influenced chemical selection. Chemicals may havebeen selected based more on the amount of information available for their carcinogenic potential
than on their potential to be good challenges for a reproducibility study.

2.2 Analysis of the scientific rationale provided
NOTE: Is the scientific rationale for the test method AND (consequently) for conducting the study
clearly explained? How does the test method contribute scientifically to the scientific understanding /prediction of the specified health/environmental effect or aspects of it?
The intended scientific rationale was explained as far as our current understanding of the cellularmechanisms involved in carcinogenesis (primarily in rodent cells) allows. The reported
prevalidation study does not contribute to this scientific understanding, but builds upon evidence(provided primarily by the OECD DRP) that genotoxic as well as non-genotoxic carcinogens induce
cell transformation in SHE and BALB/c 3T3 cells while non-carcinogenic substances do not.
The prevalidation study reports discuss the scientific background in some detail (sections 1.1 to 1.4
pp.8-11 in all three reports). According to this brief overview, the CTAs could be used as a cell-based
tool for carcinogenic hazard identification.
As outlined in the reports, the CTAs have been shown to involve a multistage cellular process thatclosely models some stages of in vivo carcinogenesis and have the potential to detect both genotoxic
and non-genotoxic carcinogens. In addition, they are conducted to screen for potential
carcinogenicity as well as to investigate mechanisms of carcinogenicity of industrial chemicals, agrochemicals, cosmetics, pharmaceuticals, environmental hormones (xenobiotics), exhaust gas etc.
The suitability of the test for expanding the scientific understanding (i.e. studying in detailmechanisms of action) is discussed to some extent in the prevalidation study reports (section 1.4
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 10 of 95

pp.10-11 in all three reports). This is exemplified by mechanistic studies of di(2-ethylhexyl)phthalate
genotoxicity in SHE pH6.7 and SHE pH7.0 CTAs (last paragraph of section 1.4, p.11 of bothprevalidation study reports).

2.3 Analysis of the regulatory rationale provided
NOTE: Is a regulatory rationale specified, i.e. a specific application of the test method for purposes ofgenerating data with respect to regulatory requirements as specified in legislation or internationallyagreed guidelines etc.? If so, how does the study and its objective and design relate to this regulatory
rationale?
The regulatory rationale remains somewhat open although it is acknowledged by the ESAC WG
that even screening data and supportive data within a weight of evidence framework can be used
for regulatory purposes and may thus constitute a "regulatory rationale". However,recommendations for a more precise definition of the regulatory usability of these tests shouldhave been made in the reports since such use is mentioned as one of the motives for the study (seealso section 15).
The current study was performed from 2005 to 2010 and planned on the background of past ECVAM
activities towards the use of CTAs (Combes et al., 1997) and the OECD project of developing an OECD
DRP which took place from 1997 to 2007. Both projects therefore overlapped to some extent andinformation on progress in both projects was mutually taken into consideration: while the ECVAM
studies relied to a great extent on reference data compiled in the OECD DRP, the recommendations
made in the OECD DRP took already into account possible results of the prevalidation study
conducted by ECVAM at that time and aiming at the refinement of selected CTA protocols.
According to these recommendations of the OECD DRP, the reported studies (if successful) should
facilitate the incorporation of the tests into OECD guidelines by demonstrating their reproducibility
and transferability.
However, the specific purpose of the tests within the framework of, for example, an OECD test
guideline was neither defined in the OECD DRP nor by the VMT when planning the current study.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 11 of 95

2.4 Appropriateness of the study design
NOTE: This includes an analysis of the selection of test items, the number of test items, the number oflaboratories involved in the study, retesting in case of unqualified tests and other technical aspects ofthe study.
Overall, the study design was considered appropriate for assessing the reproducibility andtransferability of the standardised protocols, despite shortcomings relating to the design/planningof (1) the test item selection (even when considering that this is a prevalidation study, there areconcerns regarding the number representativeness of test chemicals with regard to chemical classand mechanism of action), (2) the within-laboratory variability phase and (3) the transferability
phase.
General observations:(a)Test item selection: The selection criteria as defined and applied for choosing the six test
items (chemical substances) may lead to a preponderance of strong positive/strong negative
chemicals, which could give more reproducible results across different laboratories (section
2.4.1.1 p.16 in all three reports). In general, more than six substances should be used in a
prevalidation study to assess reproducibility, although the number is sufficient to allow
statistical evaluation of reproducibility. A more detailed discussion of the ESAC WG
concerning number and representativeness of test items is provided in sections 5.1 and 5.2,
respectively.

(b)

Within-laboratory reproducibility: Only one chemical was used to assess the withinlaboratory reproducibility. Moreover, this chemical was the positive control chemical, which
increases the likelihood of obtaining reproducible results.

(c)Transfer and transferability: In agreement with the provisions for prevalidation (ECVAM
1995; OECD 2005), transfer of the protocol (formal prevalidation phase II) was ensured
through appropriate measures (e.g. training, generation of photo catalogues for consistentscoring).
The reports furthermore indicate that the "transferability module" was assessed (e.g. p.9, 2ndparagraph in the SHE pH7.0 CTA report). This statement is however not substantiated by thestudy design and conduct: since all laboratories involved in the prevalidation studies had
substantial experience with performance of the SHE and BALB/c 3T3 CTAs and since
performance in such naive laboratories was not assessed through empirical testing of, atleast a subset, of the test items, ease of transferability (which is part of the "transferability"
assessment) was in fact not assessed.
Two issues should be acknowledged in this context:(1) While testing the ease of transferability is not a specific requirement of prevalidation
studies (ECVAM 1995; OECD Guidance Document Nr. 342 ), it may be considered goodpractice in validation to include such a testing phase to give information about the possible

2

The guidance document outlines that "Additional activities at the prevalidation stage may include anassessment of the transferability of the test protocol to laboratories inexperienced in the test or the necessary
techniques and to resolve questions or inconsistencies arising in this phase" (Paragraph 75, OECD GuidanceDocument Nr. 34 on the Validation and International Acceptance of new or updated test methods for hazard
assessment, OECD 2005)
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 12 of 95

problems that may arise in transferring a method and/or its robustness in view of possible
minor (uncontrolled) alterations of test procedure. In particular, when considering that thetest methods depend on visual scoring, data on ease of transferability would have been
helpful.
(2) It should be noted however, that the VMT has obviously given this issue some thought
and the ESAC WG acknowledges that the reports discuss, for example, to which extent
within-laboratory variability data of the different laboratories (and all based on testing thepositive control only) could be useful in making conclusions on ease of transferability. See for
instance sections 2.4.2 p.17 and 2.6, 5th paragraph on pp.18-19 in the SHE pH7.0 CTA report,
the latter reproduced below:"Following the preliminary phase of optimisation of the protocols, the transferability and thewithin- laboratory reproducibility were assessed by evaluating results obtained for one noncoded test chemical and a coded one. These two chemicals were the same (Benzo(a)pyrene),allowing an analysis of the within-laboratory reproducibility as well as the transferability of
the assay."(d)

The number of laboratories involved in each study was considered sufficient (n=3-4) (section
2.2 p.15 in all three reports).

(e)

While it is recognized that using the same cell batch in all laboratories involved in a study isgood practice in the context of validation (section 3.5.1 p.21 and p.22 in the SHE pH6.7 CTA
and SHE pH7.0 CTA reports, respectively), this does not reflect the real testing situation once
the tests are used in applied safety testing/research. It, therefore, does not provide
information on the extent to which batch-to-batch variability may influence thereproducibility of these tests.

(f)

The studies were performed by GLP-certified laboratories, but not under GLP (GoodLaboratory Practice). The use of the expression "GLP-like" (section 2.3 p.15 in all threereports) can be criticised as potentially implying a quality-controlled study, while this can not
be derived from the mere fact that some laboratories participating in the studies have GLP
certification and work under GLP. The ESAC WG holds that it should have been possible tostate in which ways the study conduct differed from full compliance with GLP.

Specific observations:(a)

SHE pH7.0 CTA report, p.22, section 3.5.4 Controls:
The reason why the concentration of benzo(a)pyrene was different between the Universityof Metz and the other laboratories should be specified.

(b)

SHE pH7.0 CTA report, p.53, Figure 14:
The University of Metz obtained high MTFs with o-toluidine HCl at 20 •g/ml and 40 •g/ml.
The VMT concluded that a statistical significant MTF at the lower doses could be related to
the University of Metz using a different batch of cells and serum than the other laboratories.
However, in order to confirm this, it may be necessary to re-check the colonies byinvestigators from other laboratories. It is possible that the University of Metz used criteria
different from those used by other laboratories but this has not been specified.

(c)

In the BALB/c 3T3 CTA report, p.26, section 3.5.3 Conclusion of the Validation Management
Team on preliminary experiments, the VMT stated that “the maximum number of Type III fociin the entire set of vehicle control dishes should not exceed five”. The specific reason forrequiring a maximum of “five” foci should be explained and justified.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 13 of 95

2.5 Appropriateness of the statistical evaluation
NOTE: Are the statistical methods used for evaluating the study data appropriate. Is there a sufficientjustification for the use of the methods chosen? Was the statistician independent from the test
method submitter/developer?
The statistical evaluation of the test data generated during the study appears appropriate.
However, the methods of statistical analysis used in the test method procedures (SOP) and theassay assessment criteria need a critical revision as outlined below:
General observations:(a)Plate variability is not taken into account by the Fisher exact test, i.e. Fisher exact test
assumes the colony is the experimental unit (and not the plate).

(b)

The use of the negative binomial analysis should be critically assessed and justified. Evidence
that it gives comparable results to those using the Fisher exact test should be provided.

(c)Anomalous results in the tables need explanations.

(d)Individual plate data should be available.

Specific observations:
For all reports:(a)Terminology issues: Care is needed as there are two different uses of the termtransformation: statistical transformation of data and cell transformation by a series ofstages (in this context assessed as morphological transformation).

(b)

Statistical analysis seems to have been independent of the conduct of the study and has
been carried out "blinded". However, there is some subjectivity or expert judgement applied
to the final decision. The criteria for determining a positive result were not completely
followed with the use of other information to overrule criteria that were not met. It wasnoted that a lack of statistical significance was in some cases overridden by a positive
biological interpretation which suggests a low power for the studies and/or appreciable
uncertainty over what constitutes a positive or negative result.

(c)Note that a statistical analysis is not "negative". A statistical test relates to acceptance orrejection of a null hypothesis. A "negative" result means that there is insufficient evidence toreject the Null Hypothesis. It is not that the Null Hypothesis is true.

For the SHE pH6.7 and SHE pH7.0 CTA reports:(d)

The statistical methods for the SHE pH6.7 and SHE pH7.0 CTAs are simple (section 3.5.6 p.23
and p.24 in the SHE pH6.7 CTA and SHE pH7.0 CTA reports, respectively). In spite of being thestandard test in the context of the SHE CTA methods, there are some limitations to the use of
the Fisher exact test and it may not be the most appropriate statistical method.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 14 of 95

(e)The decision rule using the Fisher exact test is "any two comparisons significant" and a
Cochran-Armitage trend test (if only one Fisher exact test is significant). The statistical
criteria for a significant result is if P <0.05. No correction was made for multiple comparisons.

(f)

The decision rule used could lead to some idiosyncratic "calls". It assumes, for instance, thatthe dose-response is linear (different from "general positive trend" (p.23 and p.24 in the SHE
pH6.7 and pH7.0 CTA reports, respectively)). It is not exactly clear what the statisticalproperties of this decision rule are. As the decision rule is defined there is no specific dose
relationship needed between the two significant dose comparisons required for a "positive"
call.

(g)

The Fisher (not Fischer! p.38 in the BALB/c 3T3 CTA report) exact test is vulnerable to false
positives in that it expects each unit to be independent. In fact, it is not clear how manycolonies are independent of one another. It is unclear how many plates are independent.
"Rogue" plates could make an effect seemingly statistically significant.

(h)

It is unclear whether zero results (e.g. table 9 p.36 for 2.5 µg/ml and table 30 p.57 for 55 and110 µg/ml in the SHE pH7.0 CTA report) considered acceptable results.

(i)

It is not clear that a "complete concordance" of results of statistical conclusions (section 6
p.34 and p.35 in the SHE pH6.7 and pH7.0 CTA reports, respectively) is a "data quality control
check". Two copies of the same data may be identical but that does not mean that the data
are of high quality.

For the BALB/c 3T3 CTA report:(j)

The acceptable number of vehicle control foci is <6. This should be justified in more detail.

(k)

The recommended statistical methods suggested for the BALB/c 3T3 CTA (section 3.6.5.3
pp.29-30 in the BALB/c 3T3 CTA report) are complex and have not been fully investigated toallow use in practice.

(l)These statistical methods are somewhat specialized and not widely used. The exact
properties of the methods are not defined and need some discussion. The methods are
based upon either the Negative Binomial combined with a Williams-type test or the
Nishiyama transformation. The only reference to this transformation seems to be a paper by
Nishiyama (section 3.6.5.2, 3rd paragraph p.29 in the BALB/c 3T3 CTA report).

(m)

For the within-laboratory reproducibility phase (section 4 p.32), the text is ambiguous
because the statistical analysis is described as being initially by Fisher exact tests (section 4.3p.37), which were superseded by the Negative Binomial combined with a Williams-type test
after the statistician ad hoc expert meeting, and both approaches "gave comparable results"
with 3-methylcholanthrene. This seems unlikely given the properties of the different tests(i.e. a series of pair-wise comparisons for Fisher tests vs. identification of a significant trend
for the Negative Binomial combined with a Williams-type test), although the calls based upon
the decision criteria set for each test were found to be the same. It may be that thisstatement relates, in fact, to giving comparable "calls" given the decision criteria used (table
9 p.37).

(n)

The provision of the code for the "R programme" is good practice (annex 13.3.2 pp.101-104
in the BALB/c 3T3 CTA report).

(o)

It is not clear how significant result can be obtained when values at dose level are the sameas those with the vehicle control (e.g. in table 10 p.41, 1.2 vs. 1.2 foci per dish) or lower thanthe vehicle control (e.g. in table 14 p.47, 0.20 vs. 0.40 foci per dish). These are indicated anddescribed as having "contributed to a significant result i.e. which were present in the

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 15 of 95

downward-protected Williams contrast that resulted in the lowest p-value (negative binomial
analysis) …". This may be a feature of the decision criteria used. This appears to relate to thecriteria for the detection of a positive trend which can include results from dose levels which
are as low or lower than the vehicle control values. Reporting positive effects partly based
upon values at or below the vehicle control values can, however, be difficult to envisage andmay be liable to misinterpretation.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 16 of 95

3. Test definition (Module 1)
3.1 Quality and completeness of the overall test definition
NOTE: This included an analysis of the description of the test system, the protocol, test acceptancecriteria etc.
Overall the tests were adequately defined considering the objective of this study.
Specific observations:
For the SHE pH6.7 and SHE pH7.0 CTA reports:(a)

The endpoint is the number of transformed colonies defined by cells with alteredmorphology and disorganized patterns of cell growth (a potential "subjectivity" issue).

(b)

The test system is well described as far as the current mechanistic understanding allows. The
reasons for changing the pH from 7.0 to 6.7 is adequately explained (specific for SHE pH6.7
CTA, section 3.4 p.21 in the SHE pH6.7 CTA report).

(c)

The protocol is detailed and clear (section 3.5 pp.21-24 and pp.22-25 in the SHE pH6.7 CTA
and SHE pH7.0 CTA reports, respectively), and addressing the important issues (e.g. medium,
serum) individually. The controls were specified.

(d)Acceptance criteria are listed and are clear (section 3.5.7 p.23 and pp.24-25 in the SHE pH6.7
CTA and SHE pH7.0 CTA reports, respectively).

(e)Section 9 Recommendation, 2nd paragraph p.61 and p.64, in the SHE pH6.7 CTA and SHE
pH7.0 CTA reports, respectively: “since they only differ by the pH used to culture…” shouldbe “since they only differ by the pH and medium used to culture…”.

For the BALB/c 3T3 CTA report:(f)

The endpoint is focus formation (number of foci) (a potential "subjectivity" issue).

(g)

The test system is well described as far as the current mechanistic understanding allows(section 3.3 pp.20-21 in the BALB/c 3T3 CTA report).

(h)

The protocol is detailed and clear (section 3.6 pp.26-31 in the BALB/c 3T3 CTA report), andaddressing the important issues (e.g. medium, serum) individually. The controls were
specified.

(i)

Acceptance criteria are listed for both cell growth and transformation assays, and are clear(section3.6.6 p.30 in the BALB/c 3T3 CTA report). Type III foci (as well as type I and II) should
be better defined.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 17 of 95

3.2 Quality of the background provided concerning the purpose of the test method
NOTE: What is the overall purpose of the test method (scientific use, regulatory application,
guidelines, etc.)
The overall purpose (development of OECD guidelines) was clear, but the specific purpose of thetests was neither defined by the OECD nor by the VMT.
The purpose of the prevalidation studies is to assess whether the three CTA protocols (SHE pH6.7;
SHE pH7.0 and BALB/c 3T3) have been sufficiently standardised to be transferable to otherlaboratories and reproducible between different laboratories and may therefore be fit for future use
and to address the suitability of the three CTAs/protocols to be used as a basis for the developmentof OECD test guidelines.

3.3 Quality of the documentation and completeness of SOPs and prediction models
NOTE: Are the SOPs sufficiently detailed and complete? Are the prediction models sufficiently well
explained to be applied in the correct manner?
The SOPs were found acceptable provided some minor revisions, including the ones recommended
by the VMT.
General observations:(a)Three SOPs with specific weaknesses and strengths (section 12.2 pp.69-88, section 12.2
pp.73-98 and section 13.2 pp.87-98 in the SHE pH6.7 CTA, SHE pH 7.0 CTA and BALB/c 3T3
CTA reports, respectively). The use of the same template for the different SOPs would havebeen useful for comparison and in view of developing, in case of the SHE assays, a common
protocol by merging the pH6.7 and pH7.0 protocols.

(b)

The terms "replicate", "independent" and "random" should be better defined.

Specific observations:
For the SHE pH6.7 and SHE pH7.0 CTA reports:(a)

The prediction model (assay assessment criteria) was described in detail for the SHE pH6.7
CTA (section 3.5.8 p.24). Why was the dose-response (trend) effect not included in 2nd bullet
point, 1st item ("increased transformation frequency at at least 2 dose levels")?

(b)

For the SHE pH7.0 CTA, the prediction model is poorly described in the SOP (section 6.6
p.86), while it is well described in the report text (section 3.5.8 p.25).

(c)

The SOPs should specify that the cell cultures should be regularly tested for mycoplasma to
exclude contamination.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 18 of 95

(d)Adjusted target cell seeding for SHE pH 6.7 and pH 7.0 CTAs (section 2.4.2 p.77 and section
5.2.2 p.82 in the SHE pH 6.7 CTA and SHE pH7.0 CTA reports, respectively): a clear descriptionof the procedure is needed on how to do the target cell seeding in a reproducible way.

(e)

Clearly describe the definition of passage and the meaning of "early passage" (passage
number) SHE cells (section 3.5.5 p.22 and p.24 in the SHE pH 6.7 CTA and SHE pH7.0 CTA
reports, respectively), e.g. SHE cells in secondary or tertiary culture.

(f)Provide better guidance concerning the maximum duration of storage of cryopreserved SHE
cells. For example, the SHE pH6.7 CTA report states in section 4.4. p.85, 2nd paragraph that"the storage period should not exceed 24 months". It should be explained why this period
should not exceed 24 months.

SHE pH 6.7 CTA SOP:(g)

Section 2.4.4 p.78, 3rd paragraph: "10-15 % aqueous Giemsa" should be "10% aqueous
Giemsa". The concentration of Giemsa was defined p.80, section 2.4.11 Fixing and staining of
the colonies.

(h)Section 3.1.2 p.82: Add the criteria of normal, slightly reduced, and greatly reduced colony
size and density.

SHE pH7.0 CTA SOP:(i)

Section 3. p.77, line 8, section 5.2.1. p.82, line 6, section 5.2.2 p.82, line 2 and section 5.3.3
p.82, line 1: "24 hours" should be "approximately 24 hours".

(j)

It is noted that the SHE pH6.7 and the SHE pH7.0 use different fixatives (e.g. section 5.6 p.84,
4th paragraph). For the SHE pH 7.0 ethanol is being used, while for the SHE 6.7"methanol was
the fixative. In view of a possible harmonisation of the SHE assay SOPs, a decision to use oneof the two fixatives is recommended.

For the BALB/c 3T3 CTA report:(k)

The SOP is adequately described (section 3.6 pp.26-31 and section 13.2 pp.87-98).

(l)

The assay assessment criteria (prediction model) was described in detail and anticipates theoccurrence of inconclusive results (section 3.6.7 pp.30-31).

(m)

A high dose was used for the positive control (BALB/c 3T3 CTA) which may hide variations intest performance (Tables 12 and 13 in Annex 1).

(n)Status of the cells regarding mycoplasma should be clarified (section 3.6.1 p.26).

(o)Section 3.6.1 p.26, 2nd paragraph: BALB/c 3T3 cells were "…used for the CTA within 3 to 4
passages". An explanation would be helpful as to why "within 3 to 4 passages"?

(p)

Figure 31 p.67:
HRI "TA1" should be "TA2".

(q)

HRI "TA2" should be "TA3".

p.95, Appendix 1, Culture vessels: 100x20 mm dishes should be 90x20 mm dishes (p.91, line
24 from the bottom: Authors defined the size of dishes as 90mm).

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 19 of 95

4. Data quality
The data obtained for the SHE CTAs during this study were processed and are summarized in Annex 1(Tables 7 to 10),

4.1 Overall quality of the evaluated data
NOTE: What is the quality of the data evaluated (testing data).
In general, the data quality was good. Acceptance criteria are broad enough to anticipate different
outcomes of the assays when applied properly. Discrepancies were explained.
General observations:
It is not clear from the reports whether there were pre-defined procedures for data
management/handling and whether such procedures, if in place, had been followed. This concerns
mainly issues such as compliance of the generated data with the defined acceptance criteria andpossible corrective measures in case the acceptance criteria were not met. There seem to be cases
where the criteria were indeed not met.
Specific observations:(a)Phthalic anhydride gave conflicting results in the transforming activity. It was shown to bepositive in SHE pH6.7 CTA but negative in SHE pH7.0 CTA, which was the expected result
based on previous results from the literature. A possible explanation for this discordantresult was offered on p.59, 2nd paragraph in section 7.6 in the SHE pH6.7 CTA report:"This difference could be due to pH dependent instability of phthalic anhydride in the aqueous media
used in the SHE assay. According to the OECD Screening Information Data Set (SIDS) document onphthalic anhydride (April 2005), phthalic anhydride is unstable in water, hydrolyzing within minutes
completely to phthalic acid which is non-genotoxic. Importantly, experiments with phthalic anhydride
performed in the presence of buffer showed a half-life for phthalic anhydride of 30.5 seconds at pH
7.24 and at pH 6.8 the half-life of phthalic anhydride in water was prolonged to 61 seconds. Thus, small
differences in the dose preparation, timing and pH stability of phthalic anhydride in the SHE assays
could have contributed to the conflicting results obtained in this study."

The ESAC WG believes that more discussion may be necessary to clarify the different
response for phthalic anhydride, because one cannot ignore the possibility that thedifference may stem from the substantial difference of SHE cells cultured in low pH medium
rather than a suspected immediate effect of pH on the test items (as suggested by the VMT's
explanation).
(b)

Retesting in case of unqualified tests
pp.65-66, module 4, section 6.5.2.3. in the BALB/c 3T3 CTA report: The VMT requested HRI to
repeat the experiments with BALB/c 3T3 cells treated with o-toluidine HCl. The results
obtained from the cells treated with 800 to 1200 •g/ml o-toluidine HCl were negative inTransformation Assay 2 and positive in Transformation Assay 3. The VMT concluded thattreatment with o-toluidine HCl in the repeated tests of HRI and in the other studies

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 20 of 95

conducted by the other participating laboratories produced a positive response inmorphologically transformed foci; however the ESAC WG questions why the VMT gave apositive decision for the results from HRI based on only two contradictory experiments. HRI
should have conducted further repeat experiments until reproducible results were obtained
without exceeding the number of admissible retests which should have been predefined
beforehand.
The same issue was observed in the results shown in Tables 22 and 23, p.59, in whichtransformation assay results from ECVAM, testing phenanthrene conducted with BALB/c 3T3
are described.
More generally, the ESAC WG noted that the number of admissible retesting runs in case of
unqualified tests apparently had not been defined which should ideally be done in any typeof prevalidation or validation study.

4.2 Sufficiency of the evaluated data in view of the study objective
NOTE: Are the data and their quality sufficient in view of the stated objective of the study?
The data generated and evaluated did not allow for either a proper assessment of withinlaboratory reproducibility or the success of the test transfer within the context of a dedicatedstudy phase (transferability) for all three assays.
In contrast, the data produced for assessing between-laboratory reproducibility were consideredsufficient for the SHE assays and may moreover be used to infer the success of transfer. The lack ofappropriate testing for both modules - within-laboratory reproducibility and transferability - is,however, not considered compliant with standard practice in validation.
Specific observations:(a)Within-laboratory reproducibility was established using a single compound, which was also
the positive control. This may lead to an overestimation of reproducibility due to thecompound’s high intrinsic transforming activity.

(b)

An experiment specifically assessing the success of the training and transfer from the leadlaboratory to the other laboratories was neither described nor performed.

(c)

The description of the transfer module for all three assays is patchy: information on thetimeline and the specific actions undertaken would have been helpful. In particular, more
information on how the photo catalogues have been developed as a result of the"transferability module" would be helpful.

(d)

For the SHE assays, considerable differences were observed between the dose-response
curves produced by the different laboratories.
The ESAC WG notes that for some substances the SHE assay has been observed historically togive non-monotonic (so-called "bell shaped") curves, i.e. that the number of transformed
colonies drops off when further increasing the dose. The reason for this is not fully
understood, but may be caused by disturbed conditions for expression of the transformed
phenotype at increased doses. Thus, there may be competition between the ability of acompound to induce transformed morphology, and the ability to inhibit the effect at higher
concentrations. The sensitivity of the assay may for similar reasons vary between
experiments.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 21 of 95

Nevertheless, despite these differences, the final predictions were in most cases concordant.
This shows that the assay assessment criteria are robust enough to manage such differences
which may be expected in the real testing situation, when no information on dosage andincrements is available.
Examples for the SHE pH6.7 CTA: MTF caused by benzo(a)pyrene (Figure 5 p.29 in the SHE
pH6.7 CTA report), MTF caused by 2,4-diaminotoluene (Figure 12 p.41 in the SHE pH6.7 CTA
report), MTF caused by o-toluidine HCl (Figure 18 p.50 in the SHE pH6.7 CTA report).
Examples for the SHE pH7.0 CTA: MTF caused by 2,4-diaminotoluene (Figure 8 p.43 in the
SHE pH7.0 CTA report), MTF caused by o-toluidine HCl (Figure 14 p.53 in the SHE pH7.0 CTA
report)(e)

For the BALB/c 3T3 assay refinement of the assessment criteria is required.

4.3 Quality of the reference data for evaluating reliability and relevance3
NOTE: What is the quality of the reference data used? Are the data and their quality sufficient in viewof the study objective?
The quality of the reference data was assumed sufficient for assessing reproducibility, being basedon the OECD DRP and well-regarded sources (i.e. IARC, Gold & Zeiger and NTP database). However,it was noted that there were apparently no provision for assessing the quality of data reported in
the OECD DRP and consequently in the present study.
Specific observations:
Although not specified, the small amount of human data available means that the CTAs assessed
here are used as prediction tools for rodent rather than human carcinogenicity.

3OECD guidance document Nr. 34 on validation defines relevance as follows: "Description of relationship of thetest to the effect of interest and whether it is meaningful and useful for a particular purpose. It is the extent towhich the test correctly measures or predicts the biological effect of interest. Relevance incorporates
consideration of accuracy (concordance) of a test method."
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 22 of 95

5. Test materials
5.1 Sufficiency of the number of evaluated test items in view of the study objective
NOTE: Is the number of test items tested during the study sufficient in order to draw conclusions withrespect to the objective of the study? If not, are there reasons for deviations and are these explained
and justified?
The number of chemicals (n=6) is judged to be sufficient, with respect to statistical requirements,
to assess reproducibility (the main study objective).
However, although it is acknowledged that this is not a full validation study, the number ofsubstances tested is low. More specifically, the number appears low to adequately cover, also forthe purposes of a prevalidation study, the range of possible types of chemicals in view of the mostprominent underlying mechanism of action (i.e. genotoxic/non-genotoxic) for an endpoint ascomplex as carcinogenicity.
Thus, the reproducibility assessment is restricted in this case to substances that belong to the samechemical classes (i.e. organic substances; inorganic compounds have not been tested) and whichhave the same mechanism of action as the ones tested in the prevalidation study. Considering themore advanced SHE assays, 3/6 of the substances were clear genotoxic carcinogens, only 1/6 of thesubstances was a possible non–genotoxic carcinogen (more details in section 5.2).
Specific observations:(a)

The chemicals for assessing reproducibility were selected on the basis of available in vivo andin vitro data (see comments under section 1. Data Collection, see also table 35, p.68 in the
SHE pH6.7 CTA report).

(b)

The number of compounds is small and includes only compounds with historically clear
positive or clear negative test results in the literature. It is debatable whether the
performance of the test methods was adequately challenged by such clear
transforming/non-transforming chemicals that are more likely to give concordant and hence
reproducible results.
The inclusion of chemicals where there are discordant results in the literature might haveprovided information on the test performances where transferability or between-laboratory
reproducibility could be more difficult to confirm.

(c)

The number of compounds was sufficient for the main purpose of the study (reproducibility
and transferability) although it is noted that transferability was not assessed directly by
testing chemicals in dedicated experiments.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 23 of 95

5.2 Representativeness of the test items with respect to applicability
NOTE: Analysis of how well the test items were selected in order to gain – through empirical testing
during the study – insight into the applicability domain / limitations of the test method OR analysis towhich extent the test items used during the study map an applicability domain already known.
It is not the objective of this study to assess the limitations of the tests. However, based on the
ESAC WG's analysis (see section 12.1; Annex 2) showing the apparent similarity of the historical
protocols compared with the protocols from this prevalidation study, it is conceivable that
historical data as reported in the OECD DRP could help describe the applicability domain of the test
methods in the future.
When considering the test items of this study, it appears that mainly clear positives (transforming
agents) and clear negatives (non-transforming agents) have been tested, but no "equivocal"
substances (known to be able to lead to discordant results within and between laboratories) which
may have challenged the reproducibility of the protocols more adequately.
For the two SHE protocols, a few materials, such as reserpine, cinnamyl anthranilate, or ethylene
thiourea, which have given discordant results in previous interlaboratory studies (Tu et al., 1986;
Jones et al., 1988) might have helped better define the transferability and reproducibility of these
newly standardised protocols.
When considering the complexity of the endpoint (i.e. regarding decisions on "genotoxic/nongenotoxic" and "carcinogenic/non-carcinogenic") as well as considering the small number of test
items (n=6), the test items covered a range of the possible combinations of (non)genotoxic and(non)carcinogenic (see Figure 2 "categorisation tree" and Box 1 reproducing table 35, p.68 in the
SHE pH6.7 CTA report; see also Figure 3 "categorisation tree" and Box 2 reproducing table 36, p.86
in the BALB/c 3T3 CTA report)4.
Briefly, in case of the SHE assays, 4/6 substances tested are carcinogens. These are
benzo(a)pyrene, 2,4-diaminotoluene, o-toluidine HCl and 3-methylcholantrene. 2/6 are noncarcinogens when considering reference data from the rodent bioassay (anthracene and phtalic
anhydride). One of these non-carcinogens (Anthracene) is currently not classifiable according to
IARC.
Furthermore, 2/4 carcinogenic substances studied are clearly genotoxic in in vivo and in vitro
assays (benzo(a)pyrene and 2,4-diaminotoluene), while for one the overall evidence suggests that
it is a genotoxic carcinogen despite some inconclusive in vivo genotoxicity data (3methylcholantrene)5. The remaining substance has equivocal data from in vivo and in vitro
genotoxicity tests (o-toluidine HCl).
As the concept of non-genotoxic carcinogens has only been accepted rather recently and many
substances found to be carcinogens have been tested repeatedly for genotoxicity, it is possible thatsuch substances with equivocal genotoxicity data could be regarded as non-genotoxic carcinogens(e.g. o-toluidine HCl).

4

For an example of the aspects to be taken into consideration with regard to chemical selection for
carcinogenicity/genotoxicity testing, see Vinken et al. (2008). For a full bibliographic reference see section 16.
5
Evidence for the genotoxicity of 3-Methylcholantrene is provided for instance in: Moorthy et al., 2007; Xu etal., 2005; Rihn et al., 2000; Moorthy et al., 1993; Bryla & Wyand, 1992. For bibliographic references see section
16.
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 24 of 95

Figure 2: Categorisation tree of the SHE test items showing the possible combinations of(non)genotoxic, (non)carcinogenic and inconclusive/equivocal and the placing of the test items
assessed for reproducibility within these categories. Theoretically at least 9 different categories ofsubstances are conceivable based on the mutual combinations of (non)genotoxic, (non)carcinogenic
and inconclusive/equivocal. Substances with clear conclusive data in blue [Genotoxic carcinogens:benzo(a)pyrene and 2,4-diaminotoluene; Non-genotoxic non-carcinogen: Phtalic anhydride];
substances with some degree of uncertainty regarding their belonging to one of the logicalcategories in pale green. Two of the four carcinogenic substances (shaded in grey) have equivocal
data or some inconclusive data (highlighted in red) on genotoxicity. While o-toluidine HCl can be
regarded as a possible non-genotoxic carcinogen (dashed arrow) based on its equivocal data, 3methylcholantrene is generally seen as a genotoxic carcinogen on the basis of the overall evidence
available despite some inconclusive in vivo genotoxicity data.

Test item (reference data)
Decision layer 1:
Genotoxicity
Genotoxic

Inconclusive or
equivocal (+/-)

Non-genotoxic

Decision layer 2:
Carcinogenicity *Carcinogenic

NonNotcarcinogenic classifiable

Carcinogenic

Noncarcinogenic

benzo(a)pyrene

o-toluidine HCL

anthracene

Genotoxicic based onin vitro and in vivo
data.

Genotoxicity
equivocal basedon in vitro andin vivo data.
non-genotoxic. Can
be considered
possible nongenotoxic substance.

Genotoxicity
Inconclusive basedon in vitro and invivo data

Carcinogenic
based on rodent
assay and
IARC class 1
2,4-diaminotoluene
Genotoxic based on
majority of in vivo
data and in vitro
data.

Carcinogenic
based on rodent
assay and IARC
class 2A

Not
classifiable

Non-Carcinogenic
based onrodentbioassay, but
IARC class 3.

Carcinogenic
o-toluidine HCL

Noncarcinogenic

Notclassifiable

phtalic
anhydride
Non-genotoxic
based on in vivo
data and in vitro
data.
Non-carcinogenic
based
on rodent
bioassay (but notdetermined byIARC).

Carcinogenic basedon rodent assay and
IARC class 2B.
3-methylcholantrene
Genotoxic based onin vitro and in vivo
data. Some in vivo
data howeverinconclusive.
Carcinogenic basedon rodent bioassay,
but not determinedby IARC*) based on the rodent bioassay and the overall evaluation of data by IARC, using human, in vivo animal and in vitro data.
Carcinogen = certain (class 1), probable (class 2A) and possible (class 2B)
Non-carcinogenic = probably not carcinogenic (class 4)
Not classifiable = class 3

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 25 of 95

Box 1: Table 35 of the SHE pH6.7 CTA report, outlining the carcinogenicity data and genotoxicity
data available for the substances tested. The data are graphically summarised in Figure 2.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 26 of 95

Figure 3: Categorisation tree of the BALB/c 3T3 test items showing the possible combinations of(non)genotoxic, (non)carcinogenic and inconclusive/equivocal and the placing of the test items
assessed for reproducibility within these categories. Theoretically at least 9 different categories ofsubstances are conceivable based on the mutual combinations of (non)genotoxic, (non)carcinogenic
and inconclusive/equivocal. Substances with clear conclusive data in blue [Genotoxic carcinogens:benzo(a)pyrene and 2-acetylaminofluorene]; substances with some degree of uncertainty regarding
their belonging to one of the logical categories in pale green. Two of the four carcinogenic substances(shaded in grey) have equivocal data or some inconclusive data (highlighted in red) on genotoxicity.
While o-toluidine HCl can be regarded as a possible non-genotoxic carcinogen (dashed arrow) basedon its equivocal data, 3-methylcholantrene is generally seen as a genotoxic carcinogen on the basis of
the overall evidence available despite some inconclusive in vivo genotoxicity data.

Test item (reference data)
Decision layer 1:
Genotoxicity
Genotoxic

Inconclusive or
equivocal (+/-)

Non-genotoxic

Decision layer 2:
Carcinogenicity *Carcinogenic

NonNotcarcinogenic classifiable

Carcinogenic

Noncarcinogenic

benzo(a)pyrene

o-toluidine HCL

anthracene

Genotoxicic based onin vitro and in vivo
data.

Genotoxicity
equivocal basedon in vitro andin vivo data.
non-genotoxic. Can
be considered
possible nongenotoxic substance.

Genotoxicity
Inconclusive basedon in vitro and invivo data

Carcinogenic
based on rodent
assay and
IARC class 1
3-methylcholantrene
Genotoxic based onin vitro and in vivo
data. Some in vivo
data howeverinconclusive.
Carcinogenic basedon rodent bioassay,
but not determinedby IARC
2-acetylaminofluorene
Genotoxic based onin vivo data and invitro data.
Carcinogenic basedon rodent bioassay(but not determinedby IARC).

Carcinogenic
based on rodent
assay and IARC
class 2A

Not
classifiable

Carcinogenic

Noncarcinogenic

Notclassifiableo-toluidine HCL

Non-Carcinogenic
based onrodentbioassay, but
IARC class 3.
phenantrene
Genotoxicity
inconclusive basedon in vivo and invitro data.
Non-carcinogenic
based on rodent
assay, but IARC
class 3.

*) based on the rodent bioassay and the overall evaluation of data by IARC, using human, in vivo animal and in vitro data.
Carcinogen = certain (class 1), probable (class 2A) and possible(class 2B)
Non-carcinogenic = probably not carcinogenic (class 4)
Not classifiable = class 3

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 27 of 95

Box 2: Table 36 of the BALB/c 3T3 CTA report, outlining the carcinogenicity data and genotoxicity
data available for the substances tested. The data are graphically summarised in Figure 3.

Specific observations:
The ESAC WG has the following observations concerning the selection criteria applied for identifying
chemicals:
1. By requiring materials to be positive in both SHE and BALB/c 3T3 assays, one limits thenumber of eligible chemicals to those most widely studied. Most widely studied chemicals
however are likely to be those that give clear responses. Thus, this selection criterion may
introduce a bias towards good reproducibility.
However, using identical chemicals for all three protocols may also be seen as an attempt tosecure comparability of the data of the validation study which is good practice.
2. By requiring a set of chemicals negative in both SHE and BALB/c 3T3 assays, one limits thenumber of eligible substances again to those most widely studied, possibly introducing bias
(see point 1).
However, as stated under 1., also this criterion may be seen as good practice from theviewpoint of comparability of the data generated in the study.
3. Requiring two references for each chemical may again increase the bias towards more widely
studied substances (see point 1).

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 28 of 95

4. Requiring a clear classification of the test items as an in vivo carcinogen or non-carcinogen
may again lead to a bias towards selection of materials with unequivocal results andtherefore to the selection of substances that will not challenge transferability or between
laboratory reproducibility to the extent possible.
However, while in vivo reference data were not necessary for the declared objective of thestudy (i.e. reproducibility assessment) reference data on in vivo effects allow evaluating thepredictive capacity of the protocols. Furthermore, these reference data provide a reference
point in case of discordant results during a (pre)validation study when assessing
reproducibility. These reference data can thus facilitate the identification of possible problemsof study conduct in one of the participating laboratories.
5. Both the SHE and BALB/c 3T3 assays were assessed on the basis of organic compounds only.
There are a significant number of inorganic compounds in the database which could havebeen used to inform about reproducibility of the new protocols for inorganic substances.
However, it is acknowledged that it is not the scope of this prevalidation study to investigate
applicability or limitations of these protocols. Furthermore, the limitation to a small numberof 6 test items does not allow for the covering of a broad range of chemical classes.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 29 of 95

6. Within-laboratory reproducibility (Module 2)
6.1 Assessment of repeatability and reproducibility in the same laboratory
NOTE: How were repeatability and reproducibility assessed? Are the conclusions justified by
the data as evaluated?

The ESAC WG feels that within-laboratory reproducibility was not clearly established due toinadequate study design: only one chemical was tested. Moreover, this substance was the positive
control which may lead to an overestimation of reproducibility due to the clear effects to be
expected.
General observations:(a)

The design for assessing the within-laboratory reproducibility was variable with regard to thedifferent protocols assessed (e.g. coded versus non-coded for SHE pH6.7 CTA while three
dosed repetitions for SHE pH7.0 CTA).

(b)

Within-laboratory reproducibility was based on only one chemical which was, moreover, thepositive control (producing a possible bias towards reproducible results).

Specific observations:(a)

In the SHE pH6.7 CTA, each of the three laboratories performed the assay using
benzo(a)pyrene first as a non-coded and then as a coded chemical. There was no clear dosedependent response in terms of MTF. Comparison of MTF and relative plating efficiency data
within the laboratories showed a good reproducibility, although only the non-coded
benzo(a)pyrene from one laboratory gave a higher MTF value, probably due to a highcytotoxicity (pp.29-30, Figure 6).

(b)

In the SHE pH7.0 CTA, one laboratory performed three independent assays using coded
benzo(a)pyrene. In addition, benzo(a)pyrene was used as the positive control for the assay
conducted with coded chemicals. There was no clear dose-dependent increase in MTF (Table
11 in Annex 1). The results obtained by the University of Metz were quite different fromthose obtained by the other laboratories involved in the study.
All the experiments conducted in the same laboratory (University of Metz) were reproducible
showing significant increases in MTF compared to the vehicle control at all test
concentrations equal to or higher than 0.1 •g/ml (section 4.11 pp.26-27 in the SHE pH7.0
CTA report).

(c)

In the BALB/c 3T3 CTA, each of the three laboratories performed the assay using 3methylcholanthrene as a non-coded and coded chemical. There was no significant dosedependent increase in the mean number of foci (Table 12 in Annex 1).
The trend in the formation of transformed foci by non-coded and coded 3methylcholanthrene was similar in all laboratories.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 30 of 95

6.2 Conclusion on within-laboratory reproducibility as assessed by the study
NOTE: How was within-laboratory reproducibility assessed? Are the conclusions justified by the data
as evaluated?
The VMT's conclusion of acceptable within-laboratory reproducibility is not justified because of
inadequate study design: only one chemical had been tested.
As outlined in section 6.1, only one chemical was tested to assess within-laboratory reproducibility.
This chemical was the positive control (benzo(a)pyrene for SHE CTAs, 3-methylcholanthrene for the
BALB/c 3T3 CTA), which may, due to its strong transforming potency, lead to an overestimation of
reproducibility.
Within-laboratory reproducibility for this substance was, not surprisingly, high. However, one
chemical only (being in addition the positive control) cannot be considered a sufficient dataset toconclude on within-laboratory reproducibility in compliance with what is generally considered
standard practice in validation.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 31 of 95

7. Transferability (Module 3)
7.1 Quality of design and analysis of the transfer phase
NOTE: Was the transfer phase appropriately planned, e.g. transfer instructions, training, minimum
requirements, training SOP (if appropriate). Where evaluation / decision criteria defining a successfultransfer established beforehand and consistently applied during the analysis?
The transfer phase was adequately described and appropriately executed so allowing proper test
method conduct in the other laboratories for the subsequent analysis of between-laboratory
reproducibility. However, how the success of the transfer was assessed and what criteria wereused to judge the transfer successful was not clearly described. Moreover, ease of transferability
was not assessed through the testing of test items. While this is not a prerequisite forprevalidation studies, the current study nevertheless did not fully address one of its objectives (i.e.
assessment of the transferability module).
Specific observations:(a)

It was not described how the success of the transfer was evaluated before the assessment of
the between-laboratory reproducibility. The reports suggest however that within-laboratory
reproducibility data of the positive control were intended to provide evidence ontransferability (see for instance section 2.4.2 p.17 and 2.6, 5th paragraph on pp.18-19 in the
SHE pH7.0 CTA report).

(b)

Photo catalogues were established for assuring consistency for the assessment of themorphology of transformed colonies (for the SHE assays) and transformed foci (for the
BALB/c 3T3 assay) during the scoring of the experiments performed to assess the betweenlaboratory reproducibility(c)Representatives from all laboratories involved (including technical staff and study directors)
participated in a training for harmonizing the procedures across the laboratories for theprevalidation studies.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 32 of 95

7.2 Conclusion on transferability to a second laboratory as assessed by the study
NOTE: Are the conclusions justified by the data generated? Have critical issues that may impact ontransferability been identified?
The success of the transfer programme was not demonstrated in separate experiments. All
participating laboratories had some experience with CTAs. Thus, the ease of transferability to alaboratory without any CTA experience was not demonstrated. However, this is in any case not aformal requirement for a prevalidation study (OECD guidance document Nr. 34), and in this case it
is noted that successful transfer may be inferred from the good between-laboratory reproducibility(see below).
The study design was appropriate for ensuring transfer of the protocols to other laboratories and thegeneration of photo catalogues likely to support a more consistent approach to visual scoring is
considered one of the merits of the study.
In contrast, the ease of transferability to naive/inexperienced laboratories was not assessed throughempirical testing of, at least a subset, of the test items, contrary to reports which state that one ofthe objectives was to "assess" the transferability module (e.g. p.9, 2nd paragraph in the SHE pH7.0
CTA report). While ease of transferability is not a specific requirement of prevalidation studies (OECD
Guidance Document Nr. 34)6, it may be considered good practice in validation to include such atesting phase to give information about the possible problems that may arise in transferring amethod and/or its robustness in view of possible minor (uncontrolled) alterations of test procedure.
This would have been important in this specific case, since the test method relies on an observational
readout (visual scoring) from which issues of transferability are likely to arise. However, since ease oftransferability is not a formal requirement for such a study, lack of testing is not considered crucial
for the conclusions of the study.
Moreover, it should not be omitted that the VMT has obviously given this issue some thought andthe ESAC WG acknowledges that the reports discuss, for example, to which extent within-laboratory
variability data of the different laboratories (and all based on testing the positive control only) could
be useful in making conclusions on ease of transferability. See for instance sections 2.4.2 p.17 and
2.6, 5th paragraph on pp.18-19 in the SHE pH7.0 CTA report, the latter reproduced below:"Following the preliminary phase of optimisation of the protocols, the transferability and the withinlaboratory reproducibility were assessed by evaluating results obtained for one non-coded test
chemical and a coded one. These two chemicals were the same (benzo(a)pyrene), allowing ananalysis of the within-laboratory reproducibility as well as the transferability of the assay."

6

The guidance document outlines that "Additional activities at the prevalidation stage may include anassessment of the transferability of the test protocol to laboratories inexperienced in the test or the necessary
techniques and to resolve questions or inconsistencies arising in this phase" (Paragraph 75, OECD GuidanceDocument Nr. 34 on the Validation and International Acceptance of new or updated test methods for hazard
assessment, OECD 2005)
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 33 of 95

8. Between-laboratory reproducibility (Module 4)
8.1 Assessment of reproducibility in different laboratories
NOTE: How was reproducibility between laboratories assessed?

The final outcome following the implementation of the assessment criteria was considered
reproducible for the SHE assays. For the BALB/c 3T3 assay, refinement of the assessment criteria isrequired.
General observations:(a)

The dose-ranges differed between laboratories. The VMT had decided to indicate clear dose
ranges and increments for some compounds, while for others no clear indications of thedose points and increments had been given (see citation from Validation Study Report
below). The latter case was intended to assess performance of dose-range finding in arealistic testing situation, when dose range and increments are not clear. This provides
information on the potential transferability of the test methods.
In the case of the SHE pH6.7 CTA the VMT report states (p.17, paragraph below table 1):"The doses of benzo(a)pyrene, 3-methylcholanthrene and o-toluidine HCl to be used weresuggested by the VMT based on data from the literature to optimise the use of resources(either due to high chemical cost or lack of cytotoxicity) for timely completion of thesestudies. For the other chemicals the laboratories were asked to select the dose ranges on
their own in order to check their ability to identify the critical doses for the transformation
assay."(b)

In the context of the dose range finding, the ESAC WG recommends to include guidance fordose selection in the SHE protocols to ensure proper use of the protocols in testinglaboratories. The following approach is suggested:
The highest dose should be determined base on relative plating efficiency and the lower
doses spaced out for instance with two concentrations per log (for instance: 100, 30, 10, 3,
0.3, 0.1 etc.).

(c)

The doses may have to be spaced differently for the BALB/c 3T3 assay since the responsive
concentrations are spaced in a narrower band as compared to the SHE. This could be thereason why repetitions of BALB/c 3T3 CTA experiments with more narrow doses resulted inchange from 'negative' to 'positive' calls/predictions.
Based on the study data, there is therefore a need to optimize the number and spacing ofdoses in the BALB/c 3T3 CTA SOP (e.g. 8 doses with what spacing?). For instance, in the
BALB/c 3T3 CTA report some of the dose-responses are quite steep and could be missed if
the dose spacing is not correct.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 34 of 95

Specific observations:
For the SHE pH6.7 and SHE pH7.0 CTA reports:(a)Reproducibility is acceptable.

(d)

It is not clear why the concentration of benzo(a)pyrene for the positive control differed
between laboratories.

(b)

SHE pH6.7 CTA: It is standard practice to repeat inconclusive results (e.g. o-toluidine HCl),however, the number of retesting runs should be defined in a study and, moreover, guidanceshould be provided in the SOP.

(c)

SHE pH7.0 CTA: The phthalic anhydride result produced by BASF is different from the results
produced by the other laboratories. Although the experiment met the assay assessment
criteria (specified for testing in one laboratory), it would have been perhaps advisable torepeat this experiment since discordant results had been produced when considering theresults of all laboratories.

(d)After implementation of the acceptance criteria, reproducibility is good.

For the BALB/c 3T3 CTA report:(e)

The mean numbers of foci per dish in each of the three laboratories were higher than thoseobserved during the within-laboratory reproducibility assessment (Table 13 in Annex 1).

(f)

It is not clear why incorrect solvents were used by one laboratory (e.g. DMEM-L and water
rather than DMSO). It is not clear whether this indicates (1) a lack of appreciation of thestudy objective, (2) errors at a technical level or (3) disagreement followed by a unilateralchoice of what was believed to be the best method.

(g)

Only three (out of six) chemicals produced concordant results across all laboratories before
the repetition of some experiments.

(h)

The ESAC WG agreed with the conclusion of the VMT that the between-laboratory
reproducibility was not satisfactory before retesting and improved after retesting.

8.2 Conclusion on reproducibility as assessed by the study
NOTE: Are the conclusions justified by the data generated?
Between-laboratory reproducibility was assessed through analysis of the concordance ofpredictions for the six test substances obtained by the involved laboratories. The predictions
concerned the classification of test substances as potential transforming agents/non-transforming
agents in the CTA assays. The CTA predictions were compared with the reference data associatedwith the test chemicals. These data are in vivo carcinogenicity predictions taken from the OECD
DRP report which, for the test chemicals, are based on IARC classifications, the Gold & Zeiger andthe NTP databases.
Based on the data generated and reported, the ESAC WG agrees with the VMT that the two SHE
protocols yield results which are concordant between laboratories and hence reproducible for thesubstances tested.
paragraph continued on next page
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 35 of 95

In contrast, evidence supporting reproducibility of the results between laboratories for the BALB/c
3T3 protocol was considered insufficient, as suggested by the need to refine assay assessment
criteria and to repeat some experiments to obtain concordant results across the laboratories.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 36 of 95

9. Predictive capacity (Module 5)
Although predictive capacity was outside the scope of the study objective, it is noteworthy that thepredictions made by the SHE assays for the six chemicals were in most cases correct (6/6
predictions were correct in the SHE pH7.0 CTA study, while 5/6 were correct in the SHE pH6.7 CTA
study). While the chemicals selected may have a bias towards reproducible results (clear negatives
and strong positives), the results are nevertheless reassuring and add to the database of CTA
testing data.
However, based on the ESAC WG's analysis (section 12.1; Annex 2) showing apparent similarity of
the historical protocols with the protocols from the prevalidation study, it is conceivable that
historical data as reported in the OECD DRP could help describe the predictive capacity of the SHE
CTA test methods in the future.

10. Applicability domain (Module 6)
Since this study is not a full validation study, the assessment of the applicability domain is rather
limited.
However, based on the ESAC WG's analysis (section 12.1; Annex 2) showing apparent similarity of
the historical protocols with the protocols from the prevalidation study, it is conceivable that
historical data as reported in the OECD DRP could help describe the applicability and limitations of
the SHE CTA test methods in the future.

11. Performance standards (Module 7)
Not applicable to the current CTA study.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 37 of 95

12. Readiness for standardised use
12.1 Assessment of the readiness for regulatory purposes
NOTE: Is the test method ready for regulatory purposes? If yes, why? If no – what impediments
currently exclude application for regulatory purposes?
The data generated during this prevalidation study, when viewed on their own, are insufficient todraw conclusions on readiness for regulatory use of the SHE assay protocols, although theseshowed acceptable reproducibility.
However, an analysis by the ESAC WG identified considerable similarity in the historical SHE
protocols and the standardised protocols in this prevalidation study. This supports the view that asubstantial amount of the existing testing information from the SHE assays could be used for
future considerations on their performance (e.g. predictive capacity, applicability/limitations)required to define their regulatory utility.
This view is further supported by the apparent robustness of the SHE assays as demonstrated in
the OECD DRP: the data show considerable concordance with regard to the predictions made eventhough these older protocols may have differed to some extent and clearly no standardised test
procedures had been used. These predictions are moreover relevant when compared with in vivo
carcinogenicity data derived from well-respected sources (e.g. IARC, NTP database).
The ESAC WG, therefore, believes that future activities aimed at more precise definition of test
method performance of the SHE assays and possible regulatory utility of the associated SHE
protocols can be based on both, prospective testing but also on the analysis of existing historicalinformation (e.g. a meta-analysis using defined search and data selection criteria based on study
quality).
The ESAC WG notes, based upon current opinion, that no single method can provide sufficient
information for an unequivocal assessment of the carcinogenicity potential of a substance tosatisfy regulatory requirements fully. The SHE assays may provide information about possible
genotoxic and non-genotoxic carcinogens for use in conjunction with other data (e.g. in the context
of a "weight-of-evidence" approach). Some recommendations on possible approaches towards the
expansion of the performance characterisation of these methods are made in section 15,
notwithstanding the fact that the specific regulatory use needs to be defined by the relevant
authorities for the purpose in mind.
The study results show that, in contrast to the SHE data, the BALB/c 3T3 protocol still requires
optimisation (concerning for example the assessment criteria for the assay) and is at present
neither ready to enter full validation nor consideration for regulatory use based on existing
information.
Typically an assay can be considered for regulatory purposes if the three following items are
sufficiently well described: (1) the toxicological effect the assay is intended to predict (its relevance),(2) how well the assay does predict that endpoint (its predictive capacity) or measure relevantmechanisms contributing to this toxicity effect and (3) how reproducible the assay is (its reliability).
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 38 of 95

The regulatory authority, however, makes the final decision of whether the level of relevance,
predictive capacity and reproducibility are "good enough" for the specific purpose envisioned. On the
background of these reflections, the ESAC WG holds the following views regarding the readiness of
the SHE and BALB/c 3T3 assays for regulatory purposes:
The ESAC WG is of the opinion that the reliability of the BALB/c 3T3 protocol was not adequately
addressed in this study. For example, repeat testing was executed without blinding and also theassay assessment criteria require further refinement. The ESAC WG understands why further
refinements were made to the BALB/c 3T3 protocol, but believes that these modifications should betested in further trials.
In contrast, acceptable reliability has been shown for the two SHE assay protocols for the compounds
that were tested. Although the chemical domain of this set of compounds and the number ofcompounds is very limited, the ESAC WG feels that it is both likely and plausible that this level of
reproducibility would extend to other chemical domains as well. Therefore, the SHE assay protocols
as standardised during this study are at least sufficiently reproducible, for those chemicals tested, tobe considered for eventual use in a regulatory setting.
Nevertheless, robust conclusions on the readiness for regulatory purposes of the SHE assays cannot
be drawn on the basis of this prevalidation study alone. The dataset generated during this study is
too small to allow sufficient characterisation of key items of test method performance. These
include: reproducibility on the basis of a larger and different set of chemicals (including weaklytransforming agents), predictive capacity, applicability and possible limitations.
Therefore, the next step following on from this prevalidation exercise would normally be to plan afull prospective validation study. This would comprise a set of test substances which cover a wide
range of chemical classes / possible mechanisms of action (e.g. genotoxic/non-genotoxic) and is large
enough for a statistical evaluation of predictions into two (dichotomous) classes: transforming or
non-transforming agents.
However, it is conceivable that historical SHE testing data can be used with to help get robust
performance characteristics of the SHE tests to support possible regulatory use. This is based, firstly,on the apparent robustness of the SHE assays as demonstrated by an analysis of published data in
the OECD DRP (predictions were obtained using non-standardised protocols and showed
nevertheless a high degree of concordance) and, secondly, the appreciable similarity of the historical
SHE protocols and the standardised protocols in this study shown by the analysis carried out by the
ESAC WG (see Annex 2). It should also be noted that, about 500 coded and un-coded compounds
have been tested up to now using the SHE assay by many laboratories. Consideration should be given
to what extent these historical data, which include validation studies, could supplement or substitutefor a new full prospective validation study of the SHE assay. The ESAC WG draws attention to two
publications that highlight the good predictive capacity of the SHE assays (Isfort et al., 1996; Mauthe
et al., 2001).
The ESAC WG also has the opinion, that any further activities should pay attention to the chemical
selection and, if using existing information, to an appropriate description of the toxicity potency of
substances and their physicochemical properties and chemical class. The ESAC WG has two concernswith respect to the information compiled in the OECD DRP: (a) the lack of explicitness regarding thecompleteness of the existing data presented and whether data selection criteria based on study
quality were defined and had been applied and (b) the lack of a description of the transforming
potency of the chemicals analysed. If most data are based on studies on strong transforming agentsor clear non-transforming agents (i.e. substances with no record of discordant results), thenreproducibility could have been overestimated. Future activities using this information should, thus,
analyse these issues before concluding on test performance and/or the planning of new prospective
studies.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 39 of 95

Specific observations:(a)This ESAC WG considers that its responsibility is to state whether or not the studies
conducted by the VMT have demonstrated sufficient reliability of the CTAs so that regulatory
authorities can then make a decision of whether that level of reproducibility is “goodenough” for the regulatory purpose envisioned.

(b)

The ESAC WG feels that acceptable reliability has been shown for the two SHE assay
protocols for the compounds that were tested. Although the chemical domain of this set ofcompounds is very limited, the ESAC WG is of the opinion that it is likely that this level of
reproducibility extends to other chemical domains as well. The predictive capacity of the SHE
assay was only addressed in a very limited fashion in the studies that we were asked toreview.

(c)

The ESAC WG concludes that the reliability of the BALB/c 3T3 protocol was not adequately
addressed in this study because several repeat assays had to be conducted and that therepeat assays were conducted without blinding. The ESAC WG understands why further
refinements were made to the BALB/c 3T3 CTA protocol, but believes that thesemodifications should be tested in further trials.

(d)None of the tests are currently ready as stand-alone methods for regulatory purpose.
However, it is the view of the ESAC WG that they may provide useful supplementary
information relevant for instance to a "weight of evidence" approach. Positive results may
have more "weight" than negative ones. It should moreover be considered that due to thecomplexity of carcinogenicity, no single method is sufficient for a complete hazard or risk
assessment.

(e)

Of the different alterations introduced in historical SHE CTA protocols, the change in pH of
the medium is by far the most influential factor on SHE cell morphological transformation.
Nevertheless, historical data using protocols with different pH show reasonable concordanceof the results obtained.
For example, the OECD DRP provides an analysis of the predictions yielded for 48 substancestested in various protocols that varied with regard to the pH of the medium (table 1-3 of theOECD DRP). The pH values were pH 6.7 and pH •7.
40 of these 48 substances (83%) gave the same response with both protocols, while 8/48
(17%) gave discordant results. Of these 8 compounds, 3 were positive (=transforming) at
pH6.7 and 5 at pH •7. It is noteworthy that 4 of these 8 substances were in the category"non-carcinogenic and inconclusive for carcinogenicity" when considering the reference data(in vivo carcinogenicity predictions).
Taken together, this supports the view that historical alterations in SHE CTA protocols overtime have had little impact on the reported results of chemicals tested (See section 16).

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 40 of 95

12.2. Assessment of the readiness for other uses
NOTE: Is the test method ready for other uses (e.g. screening purposes, testing to gain mechanistic
insight, to generate supportive information for hazard/risk assessment).
The ESAC WG considers the CTAs useful for testing compounds belonging to the same class ofchemicals as those used in the reported prevalidation studies (screening purposes) and to generate
supporting information for hazard identification and risk assessment (weight of evidence).
Moreover, the CTAs will continue to be useful also for mechanistic studies of the transformation
process.

12.3 Critical aspects impacting on standardised use
Note: What are the factors that may impact on standardised use (in regulatory or non-regulatory
settings)?
The performance characteristics of the SHE CTA methods need to be carefully analysed throughprospective testing and/or analysis of existing information (protocol similarity supports the use ofhistorical data) before the SHE CTA protocols can be used in standardised applications (regulatory
or non-regulatory). This analysis should include a careful examination of the chemical classes
tested. Moreover, some improvement of the SHE CTA protocols such as the development ofcommon protocol for the two pH variants and a better description of some of the protocol steps(cell preparation) should be performed before standardised use is considered.
Concerning the BALB/c 3T3 CTA, further optimisation of the protocol is needed. These
modifications, including those suggested by the VMT, should be tested in further trials before
standardised use is considered.
Specific observations:(a)Test protocols were harmonised and standardised in order to provide a basis for thedevelopment of CTA OECD test guidelines.

(b)

The visual scoring of the colonies/foci had been considered one of the greatest weaknessesof the SHE and BALB/c 3T3 CTAs. Training in the conduct and scoring the colonies/focidescribed in the CTA reports is very likely to have made the CTAs more objective and hence
more reproducible. The production of the photo catalogues is a commendable achievementof the study.

(c)Implementation of minor modifications (e.g. concerning dose selection) and alignment of the
two SHE protocols are required.

(d)

Further development of BALB/c 3T3 CTA protocol is required (e.g. to include proper guidanceon how to approach dose spacing).

(e)Several methods to automate the scoring of morphological transformation of SHE cells havebeen reported on (Trevisan et al., 2010; Walsh et al., 2009; Ridder et al., 1997). It isreasonable to assume that increased recognition, acceptance and use of the assay will
provide the necessary incentives to speed up this work, which could result in a low cost,
objective and high throughput assay. This is especially important at a time with increased

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 41 of 95

needs for testing (e.g. the REACH challenge) and increasing restrictions on the use of animalbioassays.

12.4 Gap analysis
NOTE: Identify, if appropriate, gaps in the study design and/or execution that impact on the statedstudy objective or the conclusions drawn.
Specific observations:(a)Some gaps in the study design and execution were found. These include insufficient data forwithin-laboratory reproducibility, no dedicated testing for transferability (although it isacknowledged that this is not always required in a prevalidation study) and issues withretesting (in particular for the BALB/c 3T3 CTA).

(b)

It is not known yet how assays using these protocols will perform with equivocal (e.g. weak
positive) chemicals.

(c)Procedure for data management (separated from statistician) should be established.

(d)Training programme: Criteria for success and mechanisms for assessing success were notdescribed.

(e)

It would have been appropriate, if the VMT had presented the extent to which thestandardised SHE CTA protocols differed from those used historically. This would support theuse of existing information when considering test method performance on a more generallevel.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 42 of 95

13. Other considerations
NOTE: Please address any other consideration you might have in relation to the proposed approach
under this section.
Concerning the conduct of the validation study and its conclusions:
All protocols:(a)

The high concentrations used for the positive control may hide reduced sensitivity during
test performance. The concentrations should thus be reconsidered.

SHE pH6.7 CTA and SHE pH7.0 CTA:(b)

The following statement concerning the SHE pH6.7 CTA could not be confirmed by the ESAC
WG (section 3.4 p.21 in the SHE pH6.7 CTA report): “Optimised cell growth reduces
variation…, and also increases the number of transformants in control and treated cultures
which allow for the application of robust statistical methods”.
The ESAC WG carefully re-analysed MTFs of vehicle control and positive control between SHE
pH6.7 and pH7.0 assays performed in the present prevalidation studies (Annex 1). There
were apparently no significant differences in the MTFs of vehicle control and positive control
between the cells (Tables 7-9 in Annex 1).

(c)

It was noted that some dose response curves (SHE CTAs) were flat (it is conceivable that
there was saturation) so not permitting the dose responses of the test to be addressed
adequately.

BALB/c 3T3 CTA:(d)Laboratory did not always follow instruction (e.g. wrong vehicle used).

(e)

The criteria for an inconclusive result need strengthening.

(f)

An appreciable number of experiments were repeated although the assay assessment
criteria had been met. These decisions were taken on the basis of expert judgement andretesting led to changed predictions.
For example: Transformation Assay No. 1 for phenanthrene and Transformation Assay No. 2
for o-toluidine HCl were repeated due to an increase in the number of foci at the highest
concentration although the results were considered negative according to the assessment
criteria (see table 34 p.72 in the BALB/c 3T3 CTA report). The use of more closely spaced
concentrations in the repeated experiments (Table 23 p.58 and Table 31 p.66 forphenanthrene and o-toluididne HCl, respectively, in the BALB/c 3T3 CTA report) allowed forthe confirmation of the transforming effect of these two chemicals. The choice of dose levels
to be tested in the transformation assay is critical to get a reliable result and should be
carefully made by narrowing the inter-dose spacing, especially with chemicals displaying very
steep cytotoxicity curves, as discussed by the VMT (section 8 pp.74-75 in the BALB/c 3T3 CTA
report).

(g)

Phenathrene was found a positive transforming agent which is, according to the VMT report"in contrast with previously published results" (section 7.5, p73). This has implications fortesting of unknown substances. The ESAC WG is concerned that the use of the BALB/c 3T3

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 43 of 95

CTA protocol as evaluated in this study may result in an appreciable number offalse/misleading positives.
General considerations concerning the assays:
All protocols:(a)Despite the development of the photo catalogues supporting consistent scoring, there
remains a degree of subjectivity in the assessment of colonies. The ESAC WG suggests toconsider automated approaches (e.g. image analysis) as possibly more objective ways toscore.

(b)

It should be considered whether the SOPs should provide instructions regarding the codingof test dishes to improve objectivity of the scoring process (including positive control dishes)(c)

There is some concern that despite the development of standardised protocols as a result of
this study, variants of CTA protocols may still be used. It is therefore recommended that the
protocols developed here (pending some improvements, see recommended changes of SOPs
in section 15) be accepted world-wide. Some attention should be paid to possible
implications of results obtained previously using different protocols, in particular with regard
to those protocol changes which are more likely to have an impact on the test result.

(d)

The argument made in the prevalidation study reports that the lack of knowledge about the
specific mechanism of action is an advantage seems a little weak. Although tests that
empirically seem to work may be very useful, there is ultimately a need for tests based uponmechanisms.

BALB/c 3T3 CTA:(e)

The criteria for a positive/negative decision are not completely defined. It is not clear
whether these should be solely based on a decision rule using a currently un-refereed
statistical method. The ESAC WG suggests to consider the development of a decision chart
which may aid consistent and transparent decision-making.

(f)

The ESAC WG is concerned about the implication of the comment that "experiments
requiring repetition because a biological effect was observed although the statistical analysis
was negative": This suggests that the design of the statistical evaluation as described in the
SOP may not be sufficient to detect effects of size that are considered biologically important.
More work may need to be done on the SOP to overcome this problem.

(g)

The ESAC WG is concerned about the apparent correlation between cytotoxicity observedwith positive and negative compounds and the development of foci.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 44 of 95

14. Conclusions on the study
NOTE: This section should presents an overview over the study results and conclusions as described in
the study reports (subsection 14.1), discuss to which extent the conclusions drawn in the study reports
are justified by the study results on their own (subsection 14.2) and evaluate to which extent theconclusions are plausible with respect to other information (subsection 14.3).

14.1 Summary of the results and conclusions of the study
The ESAC WG's description of the results of the study is summarized in Table 2 in this section. A
detailed recalculation of the study results has been performed by the ESAC WG and is summarised in
Annex 1. Table 3 in section 14.2 shows the extent to which the information requirements have beensatisfied by the study results, taking the study type and study objectives into account. The summary
results as presented by the VMT are reproduced in Annex 3.
14.1.1 Test items
The 6 items tested in the SHE CTAs were:
1) benzo(a)pyrene2) 2,4-diaminotoluene3) o-toluidine HCl4) 3-methylcholantrene5) anthracene6) phthalic anhydride
The 6 items tested in the BALB/c 3T3 CTA were:
1) 3-methylcholantrene2) 2-acetyl aminofluorene3) benzo(a)pyrene
4) anthracene5) phenanthrene6) o-toluidine HCl
Table 1 shows to which extent the same test items were used for the assessment of the SHE andBALB/c CTAs and which were tested either only in the SHE CTAs or only in the BALB/c 3T3 CTA. The
criteria used to select test items had been defined in a way to ensure greatest possible overlap
between test items used to assess the three protocol variants (for a critical reflection on the chemical
selection criteria see section 5.2).

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 45 of 95

Table 1: List of specific and common test items used for assessing the three CTA variants. Two
substances were tested only in the SHE CTAs, two only in the BALB/c CTA, while four substances wereused in all protocol variants.
Nr.

1

Test items used in

Tested in SHE CTAs

Tested in BALB/c CTA

OVERLAP: tested
in SHE CTAs AND
BALB/c 3T3 CTA

the study (n=8)

(n=6)

(n=6)

benzo(a)pyrene

TESTEDTESTEDYES(PC for SHE CTAs)

2

2,4-diaminotoluene

TESTEDNOT TESTED

NO

3o-toluidine HCl

TESTEDTESTEDYES

4

3-methylcholantrene

TESTEDTESTEDYES(PC for BALB/c)

5anthracene

TESTEDTESTED

YES

6phthalic anhydride

TESTEDNOT TESTED

NO

7

2-acetylaminofluorene

NOT TESTEDTESTEDNO

8phenanthrene

NOT TESTEDTESTEDNO

The test items used for assessing the SHE protocols covered a range of the possible combinations of(non)genotoxic and (non)carcinogenic (see section 5: Figure 2 "categorisation tree" and Box 1
reproducing table 35, p.68 in the SHE pH6.7 CTA report; also Figure 3 "categorisation tree" and Box 2
reproducing table 36, p.86 in the BALB/c 3T3 CTA report), in particular, when considering thecomplexity of the endpoint (i.e. regarding decisions on "genotoxic/non-genotoxic" and"carcinogenic/non-carcinogenic") as well as considering the small number of test items (n=6).
The classification of test substances used for the SHE CTAs and BALB/c CTA is shown in Box 3 and Box
4, respectively.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 46 of 95

Box 3: Categorisation of test items used in the SHE CTAs with respect to their (non)genotoxic and(non)carcinogenic profile:A) CARCINOGENICITY7
A1. Carcinogenic substances:
4/6 substances tested are carcinogenic:
1) Benzo(a)pyrene2) 2,4-Diaminotoluene3) o-Toluidine HCl4) 3-Methylcholantrene
A2. Non-carcinogenic substances:
2/6 are non-carcinogens when considering reference data from the rodent bioassay. 1 ofthese non-carcinogens (Anthracene) is currently not classifiable according to IARC.
1) Anthracene2) Phtalic anhydrideB) GENOTOXICITY8
B1. Genotoxicity of the carcinogenic substances tested:
B1.1 Genotoxic carcinogens
2/4 carcinogenic substances studied are clearly genotoxic in vivo and in vitro assays:
1) Benzo(a)pyrene2) 2,4-Diaminotoluene
For 1/4 the overall evidence suggests that it is a genotoxic carcinogen despite someinconclusive in vivo genotoxicity data:
3-Methylcholantrene
B1.2 Non-genotoxic carcinogens
The remaining substance has equivocal data from in vivo and in vitro genotoxicity
tests and may be regarded non-genotoxic9.
o-Toluidine HCl
B2. Genotoxicity of the non-carcinogenic substances tested:
B2.1 Non-genotoxic non-carcinogens
1/2 of the non-carcinogenic substances tested is non-genotoxic based on in vitro andin vivo data.
Phtalic anhydride
B2.2 Non-carcinogens with inconclusive genotoxicity results:
1/2 of the non-carcinogenic substances tested has inconclusive data from in vivo andin vitro tests:
Anthracene

7

The conceptual term carcinogenicity encompasses both carcinogenic and non-carcinogenic substances.
The conceptual term genotoxicity encompasses both genotoxic and non-genotoxic substances.
9
As the concept of non-genotoxic carcinogens has only been accepted rather recently and many substances
found to be carcinogens have been tested repeatedly for genotoxicity, it is possible that such substances withequivocal genotoxicity data could be regarded as non-genotoxic carcinogens (e.g. o-toluidine HCl).
8

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 47 of 95

Box 4: Categorisation of test items used in the BALB/c 3T3 CTA with respect to their (non)genotoxic
and (non)carcinogenic profile:A) CARCINOGENICITY10
A1. Carcinogenic substances:
4/6 substances tested are carcinogenic:
1) 3-Methylcholantrene2) 2-Acetylaminofluorene
3) Benzo(a)pyrene4) o-Toluidine HCl
A2. Non-carcinogenic substances:
2/6 are non-carcinogens when considering reference data from the rodent bioassay. Both ofthese non-carcinogens are currently not classifiable according to IARC.
1) Anthracene2) PhenanthreneB) GENOTOXICITY11
B1. Genotoxicity of the carcinogenic substances tested:
B1.1 Genotoxic carcinogens
2/4 carcinogenic substances studied are clearly genotoxic in vivo and in vitro assays:
1) 2-Acetylaminofluorene
2) Benzo(a)pyrene
For 1/4 the overall evidence suggests that it is a genotoxic carcinogen despite someinconclusive in vivo genotoxicity data:
3-Methylcholantrene
B1.2 Non-genotoxic carcinogens
The remaining substance has equivocal data from in vivo and in vitro genotoxicity
tests and may be regarded non-genotoxic12.
o-Toluidine HCl
B2. Genotoxicity of the non-carcinogenic substances tested:
B2.1 Non-carcinogens with inconclusive genotoxicity results:
2/2 of the non-carcinogenic substances tested have inconclusive data from in vivo
and in vitro tests:
Anthracene
Phenanthrene

10

The conceptual term carcinogenicity encompasses both carcinogenic and non-carcinogenic substances.
The conceptual term genotoxicity encompasses both genotoxic and non-genotoxic substances.
12
As the concept of non-genotoxic carcinogens has only been accepted rather recently and many substances
found to be carcinogens have been tested repeatedly for genotoxicity, it is possible that such substances withequivocal genotoxicity data could be regarded as non-genotoxic carcinogens (e.g. o-toluidine HCl).
11

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 48 of 95

14.1.2 Summary of study results
Study results are summarised below. Table 2 (overleaf) presents graphically the results with respect
to the study objective.
For the SHE pH6.7 and SHE pH7.0 CTA reports:(a)

The study results show that sufficiently standardised protocols are available which appears
to be transferable (at least to experienced laboratories) and which are reproducible between
laboratories on the basis of the six chemical tested.

(b)

Moreover, as result of the transfer phase, a photo catalogue supporting the consistentscoring of visually assessed effects (number of transformed colonies) has been produced. It isconceivable that this catalogue may have supported between laboratory reproducibility,
although this has not been tested (i.e. by including another laboratory working without
photo catalogue). Clearly, colony scoring appeared to be less of an issue as compared withconcerns raised in the past when no photo catalogue was available.

For the BALB/c 3T3 CTA report:(c)

An improved BALB/c protocol has been developed as a result of this prevalidation study andrecommendations of the VMT made during the course of this prevalidation study. The
recommended changes related to data interpretation only (refinement of acceptance andassessment criteria) and not to any procedural aspects related to the practical execution of
the test method. However, more experimental data are required to further refine the BALB/c
3T3 CTA protocol and assess its within- and between-laboratory reproducibility and its
transferability.

(d)

In particular the statistical methods used requires further attention, better definition andrefinement of the decision criteria based upon them. Considerations concerning thestatistical methods may affect other aspects of the design of possible future prospective
studies (number of plates, concentrations tested, requirements for repeat studies, etc).

(e)Moreover, as result of the transfer phase, a photo catalogue supporting the consistentscoring of visually assessed effects has been produced. With the appropriate training and theuse of the photo catalogue, the scoring of foci was not problematic despite the concerns
raised in the past.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 49 of 95

Table 2: The overview to which extent the information requirements were addressed/assessed inview of the objective of the prevalidation study:
CTA variant

Test definition /Protocolstandardisation

Within laboratory variability

Transferability

Between laboratory variability

SHE pH 6.7

Scientific basis of thetest defined

One chemical (positive
control benzo(a)pyrene)tested coded and non-coded,in 1 laboratory (for alllaboratories)

No dedicated testing

Six chemicals (4 carcinogens, 2
non-carcinogens) tested in 3
laboratories

SOP and photocatalogue produced

Same prediction (call) andsimilar morphologicaltransformation frequency
induced for coded and noncoded chemical

SHE pH 7.0

Scientific basis of thetest defined
SOP and photocatalogue produced

BALB/c 3T3

Scientific basis of thetest defined
SOP and photocatalogue produced
New statistical method
applied (General linear
model with a negative
binomial distribution
and identity as linkfunction)Laboratory training
organised
Photo catalogue
produced during thetraining to supportconsistent scoring

One chemical (positive
control benzo(a)pyrene)tested coded in 1 laboratory
with 3 repeats

No dedicated testing

Same prediction (call) andsimilar dose-dependent
induction of morphologicaltransformation frequency for
all 3 repeats

Photo catalogue
produced during thetraining to supportconsistent scoring

One chemical (positive
control 3methylcholanthrene) tested
coded and non-coded, in 1
laboratory (for alllaboratories)

No dedicated testing

Same prediction (call) anddose-dependent induction ofmorphologicaltransformation frequency for
coded and non-codedchemicalLaboratory training
organised

Laboratory training
organised
Photo catalogue
produced during thetraining to supportconsistent scoring

6/6 chemicals were concordantly
identified by the laboratories (butone non-carcinogenic chemical
phthalic anhydride consistently
identified as carcinogen by alllaboratories). Although the SHE
pH6.7 showed higher concordance
than the pH7.0 variant, it should
be noted that one of theconcordant predictions wasincorrect with respect to thereference data.
Six chemicals (4 carcinogens, 2
non-carcinogens) tested in 4
laboratories
5/6 chemicals were concordantly
identified by the laboratories (butone non-carcinogenic chemical
phthalic anhydride identified as
carcinogen by one laboratory).
Although the SHE pH7.0 showed
lower concordance than the pH6.7
variant, it should be noted that allof the concordant predictions
were correct with respect to thereference data.
Six chemicals (4 carcinogens, 2
non-carcinogens) tested in 3
laboratories
Without considering repeat
experiments, 3/6 chemicals wereconcordantly identified by thelaboratories (1 chemical 2acetylaminofluorene withinconclusive call in one laboratory
but experiment not repeated, 2chemicals phenanthrene and otoluidine HCl with different calls
among the laboratories)
When considering repeatedexperiments for 2 chemicals(phenanthrene and o-toluidine
HCl) 5/6 chemicals wereconcordantly identified by thelaboratories (1 chemical 2acetylaminofluorene withinconclusive call in one laboratory
but experiment not repeated)
Recommendations by the VMT torefine acceptance and assessment
criteria

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 50 of 95

14.2 Extent to which study conclusions are justified by the study results alone
As a scientific piece of work the study is impressive. This is probably a "as good as it gets" set ofexperiments considering the difficulty of these assays (e.g. time and cost considerations).
Table 3 summarises how well the information requirements were addressed/assessed in view of theobjective of the prevalidation study. From Table 3 the following is concluded:
Table 3: Extent to which information requirements were addressed and fulfilled in view of theobjective of the prevalidation study

CTA variant

Protocolstandardisation

Within-laboratory
reproducibility

Transferability

Between-laboratory
reproducibility

SHE pH 6.7

Achieved.

Not sufficiently
addressed

Successfullytransferred toexperiencedlaboratories.

Satisfactorily
demonstrated*

Single reporting format
would have been
beneficial.
Development of thephoto catalogue isconsidered a major
merit of the study.

SHE pH 7.0

Achieved
Single reporting format
would have been
beneficial.
Development of thephoto catalogue isconsidered a major
merit of the study.

BALB/c 3T3

Not finalised
Assessment criteria wereinsufficient at outset ofstudy. Further definition
suggested by VMT and
ESAC WG. These
improvements need now
to be assessed by testing.

Only one substancetested. In some cases
this was the PC(intrinsic propensity togenerate reproducible
results). Study design of
WLR phase was veryvariable.

Not sufficiently
addressed
Only one substancetested. In some cases
this was the PC(intrinsic propensity togenerate reproducible
results). Study design of
WLR phase was veryvariable.

Not sufficiently
addressed
Only one substancetested. In some cases
this was the PC(intrinsic propensity togenerate reproducible
results). Study design of
WLR phase was veryvariable.

Satisfactory for thesubstances tested*

Success of transfer was
not tested empirically
but can be deduced
from information on
BLR.

Successfullytransferred toexperiencedlaboratories.

Satisfactorily
demonstrated*
Satisfactory for thesubstances tested*

Success of transfer was
not tested empirically
but can be deduced
from information on
BLR.

Successfullytransferred toexperiencedlaboratories.

Promising but
insufficient
Further refinement of thetest method required.

Success of transfer was
not tested empirically
but can be deduced
from information on
BLR.

*) However due note should be taken that the number of substances was small (n=6), and that only
one non-genotoxic carcinogen had been tested (vs. 3 genotoxic carcinogens).
PC: positive control; WLR: within-laboratory reproducibility; BLR: between-laboratory reproducibility

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 51 of 95

General observations:(a)

For the SHE pH6.7 and SHE pH7.0 CTA reports:
Not all the criteria for a prevalidation study are met. Important merits of the studies include
the availability of standardised protocols and transferability of the CTAs to partly
experienced laboratories. The between-laboratory reproducibility seems very promising forthe selected chemicals.

(b)

For the BALB/c 3T3 CTA report:
More work is required in order to satisfactorily address protocol standardisation (including
definition of a positive response), within-laboratory reproducibility, transferability andbetween-laboratory reproducibility.

(c)Within-laboratory reproducibility:
For the three CTAs evaluated, the confidence in the within-laboratory reproducibility was notestablished because of the use of a single compound which in part was tested uncoded.
For the BALB/c 3T3 CTA, results obtained during the between-laboratory reproducibility
assessment seem to indicate low within-laboratory reproducibility in the laboratories
involved.

(d)Transferability:
The transfer of the SHE protocols was successful as judged by the acceptable betweenlaboratory reproducibility. The same holds true for the BALB/c 3T3 CTA when taking therepeat experiments into account.

(e)

Between-laboratory reproducibility:
There was good between-laboratory reproducibility for the SHE CTA.
For the BALB/c 3T3 CTA, there were discordant results for two of the chemicals (twolaboratories with a positive call, one laboratory with a negative call) when assessed according
to the criteria included in the protocol. Only when the VMT intervened with more detailed
testing instructions did the outlying laboratories obtain the same results as the other two.

Specific observations:(a)Data collection:
The data on which the prevalidation study was based (OECD DRP) were adequate. The ESAC
WG has no evidence suggesting obvious mistakes in data collection.

(b)

Study objective and design:
The definition of the study objective is clear. The scientific rationale was described as far as
our understanding of the cellular mechanisms involved in carcinogenesis allows it.

(c)Regulatory rationale:
The specific purpose of these tests was not defined by VMT. Similarly, in the OECD DRP, no
specific purpose was defined although it is indicated that these cell assays can be used for
identification of potential carcinogens.

(d)Study design:•

The labs involved in the prevalidation studies had variable experience with SHE andBALB/c 3T3 CTAs. Therefore we have no information about the challenges related to

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 52 of 95

transferability and reproducibility to inexperienced laboratories. Ease oftransferability has thus not been assessed. This is regrettable since these tests
contain a considerable element of judgement with regard to the parametersassessed (visual scoring) and information on transferability to naive laboratories
would be of value. It is remarked in this context that, although there seems to beapparent robustness of these assays when considering the OECD DRP, mostlaboratories that produced data reported in the OECD DRP are most likely "expert"
laboratories having considerable experience in conducting the test. However, one
may also argue that the CTAs were transferred over time to many laboratories, which
testifies to the transferability of the assay.
In summary, considering the nature of the readout (visual scoring), it is suggested
that the SOPs contain a specific subsection on training and transfer to naivelaboratories. Moreover, proficiency chemicals should be defined at some stage
allowing the self-assessment of laboratories.

(e)•

It was questioned whether the criteria applied to select the test chemicals wereappropriate in view of the study objective, i.e. assessment of reproducibility. The
selection criteria applied may lead to a preponderance of strong positive/strong
negative chemicals that may be more likely to give similar results in different
laboratories. Therefore the chemicals selected may lead to an overestimation of
reproducibility.

•

For assessing the within-laboratory reproducibility only the positive control wasused, partly coded and partly uncoded.

•

There may be issues related to the specific decision rules based on statistical tests(assessment criteria) leading to the calls as positive and negative that need to berevisited. Statistical tests used (Fisher exact test) with the SHE assays may lead toincreased false positive results. Statistical tests used with the BALB/c 3T3 assay seemto lead to some inexplicable results. The properties of the various statistical methods
used need further evaluation.

Test definition:
The ESAC WG appreciates the efforts to standardise the assay and subsequently the SOPs.
The development of a photo catalogue supporting consistent scoring is considered a very
positive result of this study because scoring was previously identified as a critical step in the
conduct of the assay. While it is plausible that the photo catalogues did improve theconsistency of scoring across laboratories, this is not yet entirely clear since this was not
empirically tested (e.g. by including additional laboratories working without the catalogues
during the between-laboratory reproducibility phase). The SOPs were considered to be
acceptable provided some revisions including the ones recommended by the VMT are made.

(f)Data quality:
The ESAC WG has no evidence suggesting problems with data quality. The ESAC WG was
informed that the independent statistician checked the data to see whether the acceptancecriteria were met. This process worked well in this study, however, in the future perhaps
clear distinction should be made between roles concerning data management and statisticalanalysis.

(g)

Test materials:
The ESAC WG considers six chemicals as the absolute minimum for assessing reproducibility
in a prevalidation study. There were concerns in the ESAC WG that the selected chemicals
may produce a bias towards good reproducibility (see previous comments).

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 53 of 95

(h)Within-laboratory reproducibility:
The ESAC WG feels that within-laboratory reproducibility was not clearly established for anyof the assays. Only a single substance, the positive control partly coded and partly uncoded,
was tested for assessing the within-laboratory reproducibility. Moreover, the design forassessing the within-laboratory reproducibility was variable with regard to the different
protocols assessed.
For the SHE pH6.7 assay, the identification as a positive result from the results with thepositive control is a lenient test of within-laboratory reproducibility. For the SHE pH7.0 assay
three repeats with complete dosing were performed.
In the SHE pH6.7 assay, single doses coded and non-coded were compared. In the SHE pH 7.0
assay, three dose response curves all derived using coded chemicals were compared.
In the BALB/c 3T3 assay, two dose response curves, one coded and one non-coded, werecompared.

(i)Transferability:
The main finding from this transferability exercise was the importance of the scoring process.
It is assumed that this process was significantly improved by the implementation of thephoto catalogue. The success of the transfer was not assessed in separate experiments.
Acceptable between-laboratory reproducibility however suggests that there was successfultransfer of the assays – albeit to experienced laboratories.

(j)

Between-laboratory reproducibility:
For both SHE assays some of the dose response curves varied appreciably. The final outcome
after the implementation of the assessment criteria was considered reproducible. For the
BALB/c 3T3 assay, the dose response curves seem less variable compared with those fromthe SHE assays, but refinement of the assessment criteria may be required.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 54 of 95

14.3 Extent to which conclusions are plausible in the context of existing information

In case of the SHE assays, the ESAC WG considers the observed between-laboratory reproducibility
plausible with respect to existing data on the assays. This is based upon a consideration of theextensive body of existing data produced with protocols which the ESAC WG analysis indicated arevery similar and based upon the apparent robustness of the SHE assays (e.g. as reviewed in theOECD DRP). In case of the BALB/c 3T3 assay, the ESAC WG considers the observed betweenlaboratory reproducibility insufficiently supported by existing data on the assay. This is based upona consideration of the substantial differences between the protocol variants reported e.g. in theOECD DRP (e.g. use of a modified medium, two stage-protocol, etc) and the present one (e.g. useof a new statistical method), in spite of the apparent robustness of the assay (e.g. as reviewed in
the OECD DRP).
The results of the SHE assays are in good agreement with the existing data related to this assay:
For the SHE pH 6.7 assay these data include 1) the reproducibility evaluations of a similar protocol(LeBoeuf et al., 1989; Engelhardt et al., 2004) and, 2) the overall evaluation of the SHE datacontained in the OECD DRP, which reported consistent results for 87.7% (57/65) of chemicals which
had been tested in more than one laboratory (OECD, 2007).
For the SHE pH 7.0 assay these data include 1) the reproducibility evaluations of similar protocols asreported in the literature (Isfort et al., 1996c) and, 2) the overall evaluation of the SHE datacontained in the OECD DRP, which reported consistent results for 87.7% (57/65) of chemicals which
had been tested in more than one laboratory (OECD, 2007).
In addition, the data for both SHE variants add to the understanding of the predictive capacity of the
CTA, which was previously addressed by the OECD DRP evaluation (OECD, 2007) of non-standardised
protocols.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 55 of 95

15. Recommendations
Note: This section should provide recommendations on the test method (e.g. further work, possibleuse) and their constituting elements (e.g. test system, prediction model, SOP).

15.1 General recommendations concerning the SHE assays
Although the present study succeeded in generating standardised protocols which appear
reproducible, the SHE assays are at present not yet ready for regulatory use.
In any case, a revision of the protocols with the aim of incorporating the two SHE CTA protocols
into one single protocol describing both pH variants (pH6.7 and pH7.0) should be considered. As aminimum, the two protocols should be harmonized as much as possible. Moreover, considering
the nature of the readout (visual scoring), it is recommended that the SOPs contain a specificsubsection on training and transfer of the assays to naive laboratories13. The definition of
proficiency chemicals would support such transfer and help laboratories to assess whether they
are capable of conducting the assay.
More importantly, the assays require still a complete description of their performance on the basisof a considerably larger set of chemicals including, if necessary for the envisaged purpose,
pharmaceuticals and food additives. Future test substances should include substances thatchallenge the transferability and reproducibility (i.e. substances with discordant results between
laboratories) as well as substances representing a range of possible mechanisms of action. Suchperformance characterisation should include information on (a) predictive capacity, (b)applicability and, more importantly, limitations of the assays, (c) reproducibility, as well as (d) easeof transferability.
When planning future steps of performance characterisation, the extent to which historical data(including earlier validation studies) can be taken into account, should be carefully considered, as
these data could supplement or even substitute for a full new validation study of the SHE assays.
Moreover, the extent to which prospective testing is required to fully characterise the SHE assays
will depend on (a) the intended purpose of the assays including a more precise concept concerning
their possible regulatory use and (b) the information that may have – in the meantime – become
available in the literature.
The following strategy is recommended in order to gain more robust information towards acomplete test performance characterisation of the SHE cell assays especially for regulatory
purposes:

STEP 1 – Analysis of existing information:
Any future activity towards the standardised/regulatory use of the SHE method should
start with a critical analysis of the considerable body of existing testing information (either
published or residing with stakeholders).

13

The word 'naive laboratories' in the context of validation refers to laboratories that are inexperienced withregard to the use of a specific test method, i.e. they have not conducted this method or variants of it before.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 56 of 95

It is conceivable that, after analysis of the historical datasets (e.g. with respect to chemicalclass, mechanism of action, carcinogenic potency) test performance can be satisfactorily
described through retrospective validation and meta-analysis of data alone, without
further need for prospective testing.
Importantly, such an analysis should also go back to original data and not only rely on
processed data such as contained in the OECD DRP. Moreover, an evidence-based approachshould be employed using a predefined search strategy for retrieving all relevant
information and minimum acceptance criteria for data quality.
Should this analysis show that there are gaps in the existing data sets (e.g. with regard tochemical classes, transforming potency), STEP 2 or STEP 3 should be considered.

STEP 2 – Targeted prospective testing of gap substances:
Should the retrospective evaluation of existing information performed in STEP 1 not suffice
for a satisfactory description of test performance in view of the intended purpose, a small
and targeted prospective study should be conducted providing information on assay
performance for those "gap substances" identified in STEP 1. The testing information
generated during STEP 2 may then supplement the existing information compiled in STEP 1.

STEP 3 – Full prospective validation:
Should the information generated during STEP 1 and/or STEP2 not suffice for the intended
purpose, a full prospective validation study should be conducted using the SHE CTA
protocol(s) produced during this prevalidation study but taking into account the
improvements of the SOPs as suggested by the ESAC.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 57 of 95

15.2 Recommendations for improvement of the SOPs associated with the SHE assays
The following tables 4 and 5 provide an overview over the ESAC WG's recommendations concerning
improvement of the SOPs of the SHE pH6.7 and SHE pH7.0 CTAs, respectively. Annex 4 provides
detailed information on these recommendations including justifications why the improvements havebeen recommended, references to the sections of the ESAC WG report and citations of the relevant
passages of the ESAC WG.
Table 4: Recommendations concerning improvements of the SOP of the SHE pH6.7 CTA:
Recommendation
1.

Recommendation 1: Merger of the two SOP protocols into one single protocol describing both pH
conditions (6.7 and 7.0)

2.

Recommendation 2: Include mandatory testing requirement for mycoplasm

3.

Recommendation 3: Better description of how to adjust target cell seeding in case of cytotoxic effects ofthe test item

4.

Recommendation 4: Define parameters relating to cell passage and cell storage more precisely

5.

Recommendation 5: Further refine the description of scoring parameters to facilitate a consistent
approach6.

Recommendation 6: Include guidance on dose-range finding

7.

Recommendation 7: Include guidance concerning retesting and the maximum number retesting runs
before results should be considered inconclusive

8.

Recommendation 8: Reconsider the recommended concentration for the Positive Control

9.

Recommendation 9: Provide guidance for training and transfer

10. Recommendation 10: Consider a more objective approach to visual scoring: use of image analysis, codingof test dishes

Table 5: Recommendations concerning improvements of the SOP of the SHE pH7.0 CTA:
Recommendation
1.

Recommendation 1: Merger of the two SOP protocols into one single protocol describing both pH
conditions (6.7 and 7.0)

2.

Recommendation 2: Include mandatory testing requirement for mycoplasm

3.

Recommendation 3: Better description of how to adjust target cell seeding in case of cytotoxic effects ofthe test item

4.

Recommendation 4: Define parameters relating to cell passage and cell storage more precisely

5.

Recommendation 5: Better description of the prediction model

6.

Recommendation 6: Include guidance on dose-range finding

7.

Recommendation 7: Include guidance concerning retesting and the maximum number retesting runs
before results should be considered inconclusive

8.

Recommendation 8: Reconsider the recommended concentration for the Positive Control

9.

Recommendation 9: Provide guidance for training and transfer

10. Recommendation 10: Consider a more objective approach to visual scoring: use of image analysis, codingof test dishes

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 58 of 95

15.3 General recommendations concerning the BALB/c 3T3 assay
The BALB/c 3T3 assay is at present and following this prevalidation study not yet ready for
regulatory use requires further optimisation, including refinement of the acceptance andassessment criteria.
However, considering the specificities of the BALB/c 3T3 assay (e.g. use of a well-established cellline, no feeder cells needed so no irradiation facility required) compared to the SHE assays, further
use of the refined protocol is encouraged to expand the data on assay reproducibility and theappropriateness of the assay assessment criteria (including statistical methodology used) for
generating relevant predictions. These steps should precede a more complete test performance
characterisation which may follow the same strategy as outlined for the SHE assays (see A).
Moreover, specific recommendations can be made for the BALB/c 3T3 CTA:(a)

The introduction of exogenous metabolic activation systems into the BALB/c 3T3 CTA would
support the applicability of the assay to a broader range of chemicals.

(b)Considering the nature of the readout (visual scoring), it should be considered whether the
SOPs should also contain a specific subsection on training and transfer. Moreover,proficiency chemicals should be defined at some stage allowing the self-assessment oflaboratories.

(c)

The recommended statistical method suggested for the BALB/c 3T3 CTA should be
investigated more in depth to provide a better description of its properties and to allow its
use in practice.

(d)

The modifications of the protocol, including those suggested by the VMT, should be tested infurther trials before standardised use is considered.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 59 of 95

15.3 Recommendations for improvement of the SOPs associated with the BALB/c 3T3
assay
The following table (Table 5) provides an overview over the ESAC WG's recommendations concerning
improvement of the SOPs of the BALB/c 3T3 CTA. Annex 4 provides detailed information on theserecommendations including justifications why the improvements have been recommended,references to the sections of the ESAC WG report and citations of the relevant passages of the ESAC
WG.
Table 5: Recommendations concerning improvements of the SOP of the BALB/c 3T3 CTA:
Recommendation
1.

Recommendation 1: Include mandatory testing requirement for mycoplasm

2.

Recommendation 2: Define parameters relating to cell passage more precisely

3.

Recommendation 3: Further evaluate the proposed statistical method and refine the prediction model

4.

Recommendation 4: Include guidance on dose-range finding

5.

Recommendation 5: Include guidance concerning retesting and the maximum number retesting runs
before results should be considered inconclusive

6.

Recommendation 6: Define criteria for the vehicle control more precisely

7.

Recommendation 7: Reconsider the recommended concentration for the Positive Control

8.

Recommendation 8: Consider the use of exogenous metabolic activation system

9.

Recommendation 9: Provide guidance for training and transfer

10. Recommendation 10: Consider a more objective approach to visual scoring: use of image analysis, codingof test dishes

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 60 of 95

16. References
Barrett JC and Ts'o PO (1978) Relationship between somatic mutation and neoplastic transformation. Proc. Natl.
Acad. Sci. USA, 75: 3297-301.
Berwald Y and Sachs L (1965) In vitro transformation of normal cells to tumor cells by carcinogenichydrocarbons. J. Natl. Cancer Inst., 35: 641-61.
Bryla P and Weyand EH (1992) Detection of PAH:DNA Adducts from Auto-oxidation Using 32P-Postlabeling.
Cancer Letters, 65: 35-41.
Coecke S, Balls M, Bowe G, Davis J, Gstraunthaler G, Hartung T, Hay R, Merten OW, Price A, Schechtman L,
Stacey G and Stokes W ( 2005) Guidance on good cell culture practice. A report of the second ECVAM task force
on good cell culture practice. Altern. Lab. Anim., 33: 261-287.
Combes R, Balls M, Curren R, Fischbach M, Fusenig N, Kirkland D, Lasne A, Landolph J, LeBoeuf R, Marquardt H,
McCormick J, Mueller L, Rivedal E, Sabbioni E, Tanaka N, Vasseur P and Yamasaki H (1999) Cell transformation
assay as predictors of human carcinogenicity. Alter. Lab. Anim., 27: 745-67.
Cruciani V, Rast C, Durand MJ, Nguyen-Ba G and Vasseur P (1997) Comparative effects of clofibrate and methyl
clofenapate on morphological transformation and intercellular communication of Syrian hamster embryo cells.
Carcinogenesis, 18: 701-6.
DiPaolo JA, Nelson RL and Donovan PJ (1972) In vitro transformation of Syrian hamster embryo cells by diverse
chemical carcinogens. Nature, 235: 278-80.
Dunkel VC, Pienta RJ, Sivak A and Traul KA (1981) Comparative neoplastic transformation responses ofBalb/3T3 cells, Syrian hamster embryo cells, and Rauscher murine leukemia virus-infected Fischer 344 rat
embryo cells to chemical compounds. J. Natl. Cancer Inst., 67: 1303-12.
Engelhardt G, Schwind KR and Mu•ler B (2004) The testing of chemicals in Syrian hamster embryo (SHE) celltransformation assay for assessment of carcinogenic potential. Toxicol. in vitro, 18: 213-8.
Gold LS and Zeiger E, Eds. (1997) Handbook of carcinogenicity and genotoxicity databases. CRC Press, Boca
Raton (FL), USA 754 pp.
Harvey JS, Howe JR, Lynch AM and Rees RW (2005) The results of five coded compounds: genistein,
metaproterenol, rotenone, p-anisidine and resorcinol tested in the pH 6.7 Syrian hamster embryo cellmorphological transformation assay. Mutagenesis, 20: 51-6.
IARC (2009) Agents reviewed by the IARC Monographs Volumes 1-100A (alphabetical order).
http://monographs.iarc.fr/ENG/Classification/Listagentsalphorder.pdf
Isfort RJ, Kerckaert GA and LeBoeuf RA (1996) Comparison of the standard and reduced pH Syrian Hamster
Embryo (SHE) cell in vitro transformation assays in predicting the carcinogenic potential of chemicals. Mutat.
Res., 356: 11-63.
Jones CA, Callaham MF and Huberman E (1984) Enhancement of chemical-carcinogen-induced celltransformation in hamster embryo cells by 1 alpha,25-dihydroxycholecalciferol, the biologically active
metabolite of vitamin D3. Carcinogenesis, 5: 1155-9.
Jones CA, Huberman E, Callaham MF, Tu A, Hallowell W, Pallota S, Sivak A, Lubet RA, Avery MD and Kouri RE(1988) An interlaboratory evaluation of the Syrian hamster embryo cell transformation assay using eighteen
coded chemicals. Toxicol. In Vitro, 2: 103-116.
Kerckaert GA, Isfort RJ, Carr GJ, Aardema MJ and LeBoeuf RA (1996) A comprehensive protocol for conducting
the Syrian hamster embryo cell transformation assay at pH 6.70. Mutat. Res., 356: 65-84.
LeBoeuf RA, Kerckaert KA, Poiley JA and Raineri R (1989) An interlaboratory comparison of enhanced
morphological transformation of Syrian hamster embryo cells cultured under conditions of reduced
bicarbonate concentration and pH. Mutat. Res., 222: 205-18.
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 61 of 95

Lu LJ (1990) Transplacental DNA damage: effects of chemical types, gestation stage, Ah Locus, and species. J
Cancer Res. Clin. Oncol., 116(Suppl Pt 2): 1002.
Mauthe RJ, Gibson DT, Bunch RT and Custer L (2001) The Syrian Hamster Embryo (SHE) Cell Transformation
Assay: Review of the Methods and Results. Toxicol Pathol. 29 Suppl:138-46.
Moorthy B, Chen S, Li D and Randerath K (1993) 3-Methylcholanthrene-inducible liver cytochrome(s) P450 infemale Sprague-Dawley rats: possible link between P450 turnover and formation of DNA adducts and Icompounds. Carcinogenesis. 14: 879-86.
Moorthy B, Muthiah K, Fazili IS, Kondraganti SR, Wang L, Couroucli XI and Jiang W (2007) 3-Methylcholanthrene
elicits DNA adduct formation in the CYP1A1 promoter region and attenuates reporter gene expression in rat
H4IIE cells. Biochem. Biophys. Res. Commun., 354: 1071-7.
NTP, NTP website at http://www.ntp-server.niehs.nih.gov.
OECD (2005) Guidance document on the validation and international acceptance of new or updated test
methods for hazard assessment. OECD Series on testing and assessment Nr. 34.
OECD (2007) Detailed review paper on cell transformation assays for detection of chemical carcinogens, OECD
Environment, Health and Safety Publications, Series on Testing and Assessment, No. 31.
http://appli1.oecd.org/olis/2007doc.nsf/linkto/env-jm-mono(2007)18.
Pienta RJ, Poiley JA and Lebherz WB 3rd (1977) Morphological transformation of early passage golden Syrian
hamster embryo cells derived from cryopreserved primary cultures as a reliable in vitro bioassay for identifying
diverse carcinogens. Int. J. Cancer, 19: 642-55.
Ridder GM, Stuard SB, Kerckaert GA, Cody DB, LeBoeuf RA, Isfort RJ (1997) Computerized image analysis ofmorphologically transformed and nontransformed Syrian hamster embryo (SHE) cell colonies: application toobjective SHE cell transformation assay scoring. Carcinogenesis, 18: 1965-72.
Rihn BH, Bottin MC, Coulais C, Rouget R, Monhoven N, Baranowski W, Edorh A and Keith G (2000) Genotoxicityof 3-methylcholanthrene in liver of transgenic big Blue mice. Environ. Mol. Mutagen., 36: 266-73.
Rivedal E and Sanner T (1981) Metal salts as promoters of in vitro morphological transformation of hamster
embryo cells initiated by benzo(a)pyrene. Cancer Res., 41: 2950-3.
Trevisan J, Angelov PP, Patel II, Najand GM, Cheung KT, Llabjani V, Pollock HM, Bruce SW, Pant K, Carmichael PL,
Scott AD and Martin FL (2010) Syrian hamster embryo (SHE) assay (pH 6.7) coupled with infrared spectroscopy
and chemometrics towards toxicological assessment. Analyst, 135: 3266-72.
Tsutsui T and Barrett JC (1997) Neoplastic transformation of cultured mammalian cells by estrogens andestrogenlike chemicals. Environ. Health Perspect., 105(Suppl. 3): 619-24.
Tsutsui T, Watanabe E and Barrett JC (1993) Ability of peroxisome proliferators to induce cell transformation,
chromosome aberrations and peroxisome proliferation in cultured Syrian hamster embryo cells. Carcinogenesis
14: 611-18.
Tu A, Hallowell W, Pallotta S, Sivak A, Lubet RA, Curren RD, Avery MD, Jones C, Sedita BA, Huberman E, Tennant
RW, Spalding J and Kouri RE (1986) An interlaboratory comparison of transformation in Syrian hamster embryo
cells with model and coded chemicals. Environ. Mutagen., 8: 77-98.
Walsh MJ, Bruce SW, Pant K, Carmichael PL, Scott AD and Martin FL (2009) Discrimination of a transformation
phenotype in Syrian golden hamster embryo (SHE) cells using ATR-FTIR spectroscopy. Toxicology, 258: 33-8.
Xu M, Nelson GB, Moore JE, McCoy TP, Dai J, Manderville RA, Ross JA and Miller MS (2005) Induction of Cyp1a1
and Cyp1b1 and formation of DNA adducts in C57BL/6, Balb/c, and F1 mice following in utero exposure to 3methylcholanthrene. Toxicol. Appl. Pharmacol., 209: 28-38.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 62 of 95

17. Annexes
Annex 1 – ESAC WG analysis of the SHE results of the prevalidation studyA) Morphological transformation frequencies of vehicle and positive controls
Comparison of the morphological transformation frequencies of vehicle and positive controls
between SHE pH6.7 and pH7.0 CTAs performed in the present prevalidation study
To analyze whether the morphological transformation frequencies (MTFs) were increased in SHE
pH6.7 CTA when compared with those in SHE pH7.0 CTA, the MTFs of vehicle and positive controls
between the CTAs were compared (Tables 7-9).
The MTFs of vehicle control (VC) and positive control (PC) in the case of the SHE pH6.7 or SHE pH7.0
dataset of this prevalidation study are shown in Table 7 or Table 8, respectively.
Table 7. Morphological transformation frequencies (MTFs) of vehicle control (VC1)and positive control (PC1) in the present prevalidation study with SHE pH6.7
Laboratory

BASF

Harlan CCR

BioReliance

Mean ± S.D.

MTF (%) in the experiment with
Benzo(a)pyrene

VC1*1
0.37

PC1*2
2.47

Anthracene
2,4-Diaminotoluene
3-Methylcholanthrene
O-Toluidine HCl

0.24
0.39
0.17
0.36

0.99
1.38
2.52
2.21

Phthalic anhydride

0.23

1.71

Benzo(a)pyrene
Anthracene

0.27
0.06

2.09
1.58

2,4-Diaminotoluene
3-Methylcholanthrene

0.14
0.28

2.06
2.30

O-Toluidine HCl

0.13

1.84

Phthalic anhydride

0.13

1.46

Benzo(a)pyrene
Anthracene
2,4-Diaminotoluene

0.47
0.42
0.45

2.29
1.28
2.05

3-Methylcholanthrene

0.44

1.99

O-Toluidine HCl

0.34

1.54

Phthalic anhydride

0.45

2.21

0.30 ± 0.13*3

1.89 ± 0.44*3

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 63 of 95

*1

0.2% DMSO; *2 5•g/ml benzo(a)pyrene; *3 n=18

Table 8. Morphological transformation frequencies (MTFs) of vehicle control (VC2)and positive control (PC2) in the present prevalidation study with SHE pH7.0
MTF (%) in the experiment with
Anthracene
2,4-Diaminotoluene
3-Methylcholanthrene
O-Toluidine HCl
Phthalic anhydride

VC2*1
0.40
0.39
0.51
0.49
0.56

PC2*2
(2.85)(1.82)(2.46)(3.02)(1.60)

BASF

Anthracene
2,4-Diaminotoluene
3-Methylcholanthrene
O-Toluidine HCl
Phthalic anhydride

0.25
0.28
0.25
0.26
0.26

1.36
1.70
1.36
1.43
1.43

Harlan CCR

Anthracene
2,4-Diaminotoluene
3-Methylcholanthrene
O-Toluidine HCl
Phthalic anhydride

0.27
0.20
0.32
0.35
0.44

1.85
2.34
2.04
2.29
2.66

BioReliance

Anthracene
2,4-Diaminotoluene
3-Methylcholanthrene
O-Toluidine HCl
Phthalic anhydride

0.21
0.21
0.38
0.30
0.52

1.03
1.03
1.59
1.37
1.41

0.34 ± 0.11*3

1.66 ± 0.48*4

Laboratory

Uni. Metz

Mean ± S.D.
*1

0.2%DMSO; *2 5•g/ml benzo(a)pyrene except for PC2 in Uni. Metz (1•g/ml
benzo(a)pyrene) ; *3 n=20; *4 n=15 (The MTFs from Univ. Metz were not included
because the concentration of benzo(a)pyrene used in the university was different
from that used in the other laboratories.)

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 64 of 95

When compared the MTFs of VC1 and PC1 in SHE pH6.7 CTA with those of VC2 and PC2 in SHE pH7.0
CTA, there were no significant differences in the MTFs between VC1 and VC2 and PC1 and PC2 (Table
9).

Table 9. Comparison of the morphological transformation frequencies of vehicle
controls (VC1and 2) and positive controls (PC1 and 2) between SHE pH6.7 andSHE pH7.0 CTAs
SHE
pH6.7

SHE
pH7.0

SHE
pH6.7

SHE
pH7.0

VC1

VC2

PC1

PC2

Mean
Standard deviation

0.30
0.13

0.34
0.11

Mean
Standard deviation

1.89
0.44

1.66
0.48

Number of samples

18

20

Number of samples

18

15

Between VC1 and VC2 (P = 0.2572); Between PC1 and PC2 (P = 0.1697) (Unpaired t
test with Welch's correction).
The results indicate that there were no significant differences in the morphological transformation
frequencies of vehicle and positive controls between the SHE pH6.7 and SHE pH7.0 CTAs.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 65 of 95

B) Predictive capacity of the test methods used in the present prevalidation study
Although the number of chemicals tested is limited, the relevance of the test methods was analyzed(Table 10).
Table 10. The relevance of the test methods used in the present prevalidation study
Performance
Concordance(Accuracy)
Sensitivity
Specificity
Positive
predictivity
Negative
predictivity

SHE pH6.7

Test method
SHE pH7.0

BALB/c 3T3

83.3% (5/6)*1

100% (6/6)*2

83.3% (5/6)*3

100% (4/4)
50% (1/2)

100% (4/4)
100% (2/2)*2

100% (4/4)*4
50% (1/2)*5

80% (4/5)

100% (4/4)

80% (4/5)

50% (1/2)

100% (2/2)*2

50% (1/2)*1(#)shows the number of chemicals.
One of four laboratories gave a conflicting result on phthalic anhydride.
*3
One of three laboratories gave diverging results (positive or negative) in
each of three out of six chemicals.
*4
One of three laboratories gave a diverging result on either 2acetylaminofluorene or o-toluidine HCl.
*5
One of three laboratories gave a positive result on phenanthrene.
*2

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 66 of 95

C) Comparison of transformation frequencies
Comparison of the transformation frequencies induced by low or high concentrations of chemicals
used as positive controls in the prevalidation study
The morphological transformation frequencies (MTFs) induced by low or high concentrations ofbenzo(a)pyrene in SHE pH7.0 CTA (SHE pH7.0 CTA: pp.27-29, Tables 3-6 and Figure 2C) are shown in
Table 11.
Table 11. MTFs (%) induced by low or high concentrations of benzo(a)pyrene in SHE
pH7.0 CTA (SHE pH7.0 CTA: pages 27-29, Tables 3-6 and Figure 2C)
Benzo(a)pyrene
Laboratory

1.0 or 1.25•g/ml

5•g/ml

BASF

1.87

1.57

Harlan CCR

1.45

3.01

BioReliance

1.38

2.17

Mean ± S.D.

1.57 ± 0.27 (A)2.25 ± 0.72 (B)Between (A) and (B) (P=0.2658 by t-test).
There were no significant differences were observed between low (1.0 or 1.25•g/ml) and high(5•g/ml) concentrations of benzo(a)pyrene in SHE pH7.0 CTA. The result suggests that a highconcentration of benzo(a)pyrene used as the positive control may not hide variations in test
performance.
Conversely, although there were little, if any, significant differences between the mean number offoci per dish induced by low (1•g/ml) or high (3 or 4•g/ml) concentrations of 3-methylcholanthrene
in the within-laboratory reproducibility tests with BALB/c 3T3 cells, the mean number of foci per dishin BALB/c 3T3 cells treated with 3 or 4•g/ml 3-methylcholanthrene were much higher than those incells treated with 1•g/ml 3-methylcholanthrene (Tables 12 and 13).

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 67 of 95

Table 12. Mean number of foci per dish induced by low or high concentrations of
3-methylcholanthrene in BALB/c 3T3 CTA (BALB/c 3T3 CTA, 4 Module 2:
Within-laboratory reproducibility: pages 32-34, Tables 3-8)
Concentration of 3-methylcholanthrene
Laboratory1•g/ml3 or 4•g/ml

ECVAM

1.20, 1.30

4.70, 1.50

Harlan CCR

4.40, 4.60

6.70, 5.00

HRI

3.40, 3.90

11.90, 8.90

3.13 ± 1.52 (A)6.45 ± 3.62 (B)Mean ± S.D.

Between (A) and (B) (P=0.026 by Mann-Whitney U test; P=0.083 by t-test).

Table 13. Mean number of foci per dish induced by 4•g/ml 3-methylcholanthrene usedas positive controls (BALB/c 3T3 CTA, 6 Module 4: Between-laboratory
reproducibility: pp.41-67, Tables 10-31)

Laboratory

Mean number offoci per dishMean ± S.D.

ECVAM

20.22, 11.40
13.10, 16.10
15.60, 11.90

14.72 ± 3.30
(n=6)

Harlan CCR

11.30, 11.50
15.00, 13.60
13.70, 10.90

12.67 ± 1.66
(n=6)

HRI

16.10, 10.33
25.00, 11.25
20.70, 16.10

16.58 ± 5.58
(n=6)

Mean ± S.D.

14.66 ± 3.99 (n=18)

The results indicate that (1) the mean number of foci per dish varied between the within- andbetween-laboratory reproducibility tests, and (2) treatment of BALB/c 3T3 cells with a highconcentration of 3-methylcholanthrene used as the positive control may hide variations in test
performance, as described in Section 3.3 (m) p.19 in this report.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 68 of 95

Annex 2 – Analysis of SHE protocol similarity: historical vs. prevalidation
Analysis of the degree of similarity between the historical SHE cell transformation protocols andthose standardised in this study
This analysis was performed by the ESAC WG in the course of its scientific review. Due to time
restraints it should not be regarded as a comprehensive analysis, but rather as a study of publishedprotocols from a selection of the more active laboratories.
A) Assay principle: Test the effect of substances on the morphology of colony forming early passage
Syrian hamster embryo (SHE) cells.
The assay principle has been unaltered for more than 40 years, and the main points in the procedure
are therefore also unaltered. Some changes in protocols have however been introduced in different
laboratories.
B) Variation in the SHE cell transformation protocols 1965-2010:
1. Frozen cells. The assay was originally developed in the laboratories of Leo Sachs and Joe
DiPaolo using freshly prepared cells from hamster embryos. Pienta introduced in 1976 theuse of frozen cells which thereafter has been adopted by basically all laboratories.
2. Medium used has been DMEM with minor modifications such as alterations in glucose (highlevel (4.5 g/l) or low level (1 g/l)) and with or without phenol red. No reproducible effects ofthese alterations have been reported.
3. Medium pH has been modified by alteration of the medium bicarbonate concentration in
combination with level of CO2 in the incubator.
i. pH 7.3 was obtained using DMEM with 3.7 g/l bicarbonate and 5% CO2.
ii. pH 7.0 was obtained using 2-2.2 g/l bicarbonate and 10% CO2.
iii. pH6.7 was obtained using 0.75 g/l bicarbonate and 10% CO2.
pH is by far the single most effective factor in the protocols to influence morphologicaltransformation. The effect of altered pH has in general been that more extensivelytransformed colonies, both in exposed and control dishes, are formed when pH is decreased.
When using pH 7.3, extremely few (if any) transformed colonies were formed in unexposed
dishes. When decreasing the pH to 7.0 and 6.7, the frequency of transformed colonies in
unexposed dishes increases to about 0.3-0.4%. Still, this modification has been consideredadvantageous since a higher number of transformed colonies also were obtained in the
exposed dishes.
4. Serum. 15 - 20% fetal bovine serum has been used. Most laboratories have tested for serumbatches with optimum support of plating efficiency and expression of transformed
morphology.
5. Syrian hamsters are out-bred animals originating from one litter of animals. Some differencein cell quality between animals has been reported, and made it relevant to test for optimalcell preparations. The advantage of having a stable source of cells (Syrian hamsters) for thiscell transformation assay can not be over-estimated.
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 69 of 95

6. DMSO is the most dominantly used solvent. When necessary acetone or other solvents havebeen used.
7. X-irradiation of feeder cells has been performed at 4-5000R and 40-60 000 feeder cells
seeded in 2 ml medium.
8. 150-500 target cells have been seeded in 2 ml medium the day after the feeder cells. The
number of seeded cells has varied due to varying plating efficiency of the cells and protocols
used.
9. Chemicals have been added in 4 ml complete medium the day after the target cells. The
general routine has been to leave the dishes undisturbed for 7 days. Chemical exposure
strategy has also been modified for specific mechanistic studies (sequential exposures, 24h
exposure).
10. Fixation of colonies has been performed in 100% methanol, and the colonies thereafterstained in 10-15% Giemsa.
11. Scoring for morphological transformation. The manual scoring of individual colonies formorphological alteration involves subjectivity. Several intra- and inter-laboratorial exercises
have however concluded that following training by experienced personnel, 5-10% interindividual variance in scored colonies occur. The use of photo catalogs showing different
types of normal and transformed colonies has further improved the scoring process, andcontributed to the general appreciation among participants in scoring exercises that theprocess is less problematic than expected.
Conclusion:
Of the different alterations introduced in the SHE cell protocols, the change in medium pH is by far
the factor with the largest influence on SHE cell morphological transformation. At pH 7.3, none orclose to none transformed colonies were observed in unexposed dishes, while when decreasing the
pH to 7.0 and 6.7, the frequency of transformed colonies increased both in unexposed and exposed
dishes. Comparative data from protocols using different pH have been shown to provide reasonableconcordance of the results obtained. In table 1-3 in the OECD DRP, 48 of the chemicals could becompared for response at pH 6.7 and pH •7. 40 (83%) of the chemicals gave the same response atboth protocols while 8 (17%) gave different results. Of the 8 compounds giving different results, 4were in the category "non-carcinogenic and inconclusive for carcinogenicity", 3 were positive at
pH6.7 and 5 at pH •7.
Modifications other than pH have been introduced to simplify the procedure or to improve the SHEcell plating efficiency. It can obviously not be excluded that there exist chemicals where specificalterations in the protocol may influence the final result. There are however to our knowledge nodata suggesting that such alterations have influenced the final assessment of chemicals as being SHEcell transforming or non-transforming.
There are no elements in the pre-validation protocols suggesting that the experiments carried out
under the present study stand out from the experiments reported in the OECD DRP. The threepresent SHE cell protocols are adopted from protocols in laboratories that have been directly
involved in providing significant parts of the data presented in the OECD DRP.
The analysis was based on the following references , listed in ascending chronological order.
For full bibliographic information, see section 15:• Berwald Y, Sachs L. J Natl Cancer Inst. 1965, 35, 641-61.
• DiPaolo JA et al. Nature. 1972, 235, 278-80.
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 70 of 95

•
•
•
•
•
•
•
•
•
•
•

Pienta RJ et al. Int J Cancer. 1977, 19, 642-55.
Barrett JC, Ts'o PO. Proc Natl Acad Sci U S A. 1978, 75, 3297-301.
Rivedal E, Sanner T. Cancer Res. 1981, 41, 2950-3.
Dunkel VC et al. J Natl Cancer Inst. 1981, 67, 1303-12.
Jones CA et al. Carcinogenesis. 1984, 5, 1155-9.
Kerckaert GA et al. Mutat Res. 1996, 356, 65-84.
Cruciani V et al. Carcinogenesis. 1997, 18, 701-6.
Tsutsui T et al. Int J Cancer. 1997, 70, 188-93.
Engelhardt G et al. Toxicol In Vitro. 2004, 18, 213-8.
Harvey JS et al. Mutagenesis. 2005, 20, 51-6.
Walsh MJ et al.Toxicology. 2009, 258, 33-8.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 71 of 95

Annex 3 – Presentation of study results by the VMT (copied from study
reports)
The Validation Management Team has summarised the results of this prevalidation study as follows(dark red font):"SHE pH 6.7:
The VMT concluded that in keeping with the objectives of this CTA effort, the SHE pH 6.7 CTA had
been prevalidated in accordance with modules 1-4 (Hartung et al., 2004). It has been demonstrated
that a standardised protocol is available that should be the basis for future use. This protocol and theassay system itself are transferable between laboratories, and reproducible within- and betweenlaboratories.
This conclusion is substantiated by the existing body of knowledge related to this assay. In particular,by 1) the reproducibility evaluations of a similar protocol as reported in the literature (LeBoeuf et al.,
1989; Engelhardt et al., 2004) and, 2) the overall evaluation of the SHE data contained in the OECD
DRP, which reported consistent results for 87.7% (57/65) of chemicals which had been tested in morethan one laboratory (OECD, 2007). Moreover, the VMT concluded that with the appropriate training
and the use of the photo catalogue, colony scoring was not problematic despite the concerns raised in
the past.
In addition, the data produced add to the understanding of the predictive capacity (module 5) of the
CTA, which was previously addressed by the OECD DRP evaluation (OECD, 2007).
The VMT supports the conclusions of the OECD DRP and the generation of an OECD SHE celltransformation test guideline.
SHE pH 7.0:
The VMT concluded that in keeping with the objectives of this CTA effort, the SHE pH 7.0 CTA had
been prevalidated in accordance with modules 1-4 (Hartung et al., 2004). It has been demonstrated
that a standardised protocol is available that should be the basis for future use. This protocol and theassay system itself are transferable between laboratories, and reproducible within- and betweenlaboratories.
This conclusion is substantiated by the existing body of knowledge related to this assay. In particular,by 1) the reproducibility evaluations of similar protocols as reported in the literature (Isfort et al.,1996c) and, 2) the overall evaluation of the SHE data contained in the OECD DRP, which reported
consistent results for 87.7% (57/65) of chemicals which had been tested in more than one laboratory(OECD, 2007). Moreover, the VMT concluded that with the appropriate training and the use of thephoto catalogue, colony scoring was not problematic despite the concerns raised in the past.
In addition, the data produced add to the understanding of the predictive capacity (module 5) of
theCTA, which was previously addressed by the OECD DRP evaluation (OECD, 2007).
The VMT supports the conclusions of the OECD DRP and the generation of an OECD SHE celltransformation test guideline.
BALB/c 3T3:

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 72 of 95

On the basis of the outcome of this prevalidation study, an improved protocol, incorporating therecommendations made by the VMT has been developed. The recommended changes to the protocol
relate only to data interpretation (refinement of acceptance and assessment criteria). Moreexperimental data are required to allow for further refinement and evaluation of the statisticalmethod which may impact future study design (number of plates, concentrations tested,requirements for repeat studies, etc).
If the repeated experiments and the modifications to the data interpretation of the improved protocol
are taken into consideration, including the importance of considering biological relevance, it can beconcluded that the assay is transferable between laboratories and, reproducible within and between
laboratories. Moreover, this study demonstrated that with the appropriate training and the use of thephoto catalogue, the scoring of foci was not problematic despite the concerns raised in the past. It is
recommended that this improved protocol be used in the future in order to confirm its utility.
Furthermore, although limited, these prevalidation data add to the fifth module i.e. predictive
capacity, which has been addressed by the OECD DRP evaluation."

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 73 of 95

Annex 4 – List of detailed recommendations from the ESAC WG report
regarding the SOPs of the three CTAs
The recommendations are in bold and numbered. Below each recommendation, the relevant
sections of the ESAC WG report are in cited (italics) and referenced in the right column. The middle
column provides a summary justification for the recommendation (blue).

A) Detailed recommendations concerning the SOP of the SHE pH6.7 CTA
Recommendation

Justification

1. Recommendation 1: Merger of the two SOP
protocols into one single protocol describing both
pH conditions (6.7 and 7.0)

The two SHE protocols differ only withrespect to the pH used to harvest theembryonic cells and culture them
subsequently. All other protocol
parameters are identical or very similar(e.g. source of medium, fixatives etc). In
the interest of the use of both protocols,
harmonisation of the protocols of the
SHE assay pH variants or merger into one
single protocol would be of benefit theconsistent use of this CTA.

The use of the same template for the different SOPs would havebeen useful for comparison and in view of developing, in case of the
SHE assays, a common protocol by merging the pH 6.7 and pH 7.0
protocols.
Implementation of minor modifications (e.g. concerning dose
selection) and alignment of the two SHE protocols are required.
A revision of the protocols with the aim of incorporating the two
SHE cell protocols into one single protocol describing both pH
variants (pH6.7 and pH7.0) should be considered. As a minimum,
the two protocols should be harmonized as much as possible.

Corresponding section of
the ESAC WG report

It is noted that the SHE pH6.7 and the SHE pH7.0 use different
fixatives. For the SHE pH7.0 ethanol is being used, while for the SHE
6.7 methanol was the fixative. In view of a possible harmonisationof the SHE assay SOPs, a decision to use one of the two fixatives isrecommended.

2. Recommendation 2: Include mandatory testing
requirement for mycoplasm

Adjusted target cell seeding for SHE pH 6.7 and pH 7.0 CTAs: a clear
description of the procedure is needed on how to do this in a
reproducible way.

4. Recommendation 4: Define parameters relating
to cell passage and cell storage more precisely
Clearly describe the definition of passage and the meaning of "earlypassage" (passage number) SHE cells, e.g. SHE cells in secondary or
tertiary culture.

12.3
14.4

3.3

In agreement with Good Cell Culture
Practice (GCCP) (Coecke et al., 2005)

The SOPs should specify that the cell cultures should be regularly
tested for mycoplasma to exclude contamination.

3. Recommendation 3: Better description of how
to adjust target cell seeding in case of cytotoxic
effects of the test item

3.3

3.3

A precise approach for target cellseeding is crucial to counterbalance
possible loss of colonies due tocytotoxicity and, if not adjusted
appropriately, may lead to anunderestimation of the carcinogenicity
potential. Therefore, this procedureneeds to be well described.
It cannot be excluded that the cells willchange their properties with increasing
passage number or storage duration.
Precision on passage parameters
including maximum admissible passage
number is important.

Provide better guidance concerning the maximum duration ofstorage of cryopreserved SHE cells. For example, the SHE pH6.7 CTA
ndreport states in section 4.4. p.85, 2 paragraph that "the storage
period should not exceed 24 months". It should be explained why
this period should not exceed 24 months.

3.3

3.3

3.3

Continued on next page
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 74 of 95

Continued from previous page
Recommendation

Justification

Corresponding section of
the ESAC WG report

5. Recommendation 5: Further refine thedescription of scoring parameters to facilitate aconsistent approach

Colony size provides additional
information on cytotoxicity andcomplements Relative Plating Efficiency.

Section 3.1.2 p.82: Add the criteria of normal, slightly reduced, andgreatly reduced colony size and density.

6. Recommendation 6: Include guidance on doserange finding
In the context of the dose range finding, the ESAC WG recommends
to include guidance for dose selection in the SHE protocols toensure proper use of the protocols in testing laboratories. The
following approach is suggested:
The highest dose should be determined base on RPE (%) and thelower doses spaced out for instance with two concentrations perlog (for instance: 100, 30, 10, 3, 0.3, 0.1 etc.).

7. Recommendation 7: Include guidance
concerning retesting and the maximum number
retesting runs before results should be considered
inconclusive

3.3

Dose range finding is a crucial step for
defining the test dosage. Inappropriate
dose spreading may lead to missing therelevant dose.

Substances that give inconclusive results
after one test should be retested.
Retesting should however be limited to areasonable number.

SHE pH6.7 CTA: It is standard practice to repeat inconclusive results(e.g. o-toluidine HCl), however, the number of retesting runs should
be defined in a study and, moreover, guidance should be provided
in the SOP.

8. Recommendation 8: Reconsider therecommended concentration for the Positive
Control (PC)The high concentrations used for the Positive Control may hide
reduced sensitivity during test performance. The concentrations
should thus be reconsidered.

9. Recommendation 9: Provide guidance fortraining and transfer
It should be considered whether the SOPs should provide
instructions regarding the coding of test dishes to improve
objectivity of the scoring process (including positive control dishes).

8.1

While the PC is not used fornormalisation in this assay, it should also
provide information on the relative
sensitivity of the test system. Highconcentrations are however likely toyield effects. Thus, a second PC using alower concentration or use of a 'batch
control' should be considered.
The readout of the CTAs is visual scoring.
A training and transfer subsection should
describe an approach to transferring thetest to an inexperienced laboratory.

Proficiency chemicals should be defined at some stage allowing theself-assessment of laboratories.

10. Recommendation 10: Consider a more objective
approach to visual scoring: use of image analysis,
coding of test dishes
Considering the nature of the readout (visual scoring), it issuggested that the SOPs contain a specific subsection on training
and transfer to naive laboratories.

8.1

13

14.2,
14.4
14.2,
14.4

The readout of the CTAs is visual scoring.
As with all visual assessments there is adegree of subjectivity and possible
inconsistency which should be minimised
to the extent possible.

Despite the development of the photo catalogues supporting
consistent scoring, there remains a degree of subjectivity in theassessment of colonies. The ESAC WG suggests to consider
automated approaches (e.g. image analysis) as possibly moreobjective ways to score.

13

13

Continued on next page

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 75 of 95

Continued from previous page
Recommendation

Justification

11. Minor recommendation-

Section 2.4.4 p.78, 3rd paragraph: "10-15 % aqueous Giemsa"
should be "10% aqueous Giemsa". The concentration of Giemsa
was defined in the SHE pH6.7 SOP on p.80, section 2.4.11 Fixing andstaining of the colonies.

-

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Corresponding section of
the ESAC WG report

3.3

Page 76 of 95

B) Detailed recommendations concerning the SOP of the SHE pH7.0 CTA
The recommendations are in bold and numbered. Below each recommendation, the relevant
sections of the ESAC WG report are in cited (italics) and referenced in the right column. The middle
column provides a summary justification for the recommendation (blue).
Recommendation

Justification

1. Recommendation 1: Merger of the two SOP
protocols into one single protocol describing both pH
conditions (6.7 and 7.0)

The two SHE protocols differ only withrespect to the pH used to harvest theembryonic cells and culture them
subsequently. All other protocol
parameters are identical or very
similar (e.g. source of medium,fixatives etc). In the interest of the useof both protocols, harmonisation of
the protocols of the SHE assay pH
variants or merger into one single
protocol would be of benefit theconsistent use of this CTA.

The use of the same template for the different SOPs would have been
useful for comparison and in view of developing, in case of the SHE
assays, a common protocol by merging the pH 6.7 and pH 7.0
protocols.
Implementation of minor modifications (e.g. concerning dose
selection) and alignment of the two SHE protocols are required.
A revision of the protocols with the aim of incorporating the two SHE
cell protocols into one single protocol describing both pH variants(pH6.7 and pH7.0) should be considered. As a minimum, the two
protocols should be harmonized as much as possible.

Corresponding section of
the ESAC WG report

It is noted that the SHE pH6.7 and the SHE pH7.0 use different
fixatives. For the SHE pH7.0 ethanol is being used, while for the SHE
6.7 methanol was the fixative. In view of a possible harmonisation of
the SHE assay SOPs, a decision to use one of the two fixatives isrecommended.

2. Recommendation 2: Include mandatory testing
requirement for mycoplasm

Adjusted target cell seeding for SHE pH 6.7 and pH 7.0 CTAs: a clear
description of the procedure is needed on how to do this in a
reproducible way.

4. Recommendation 4: Define parameters relating tocell passage and cell storage more precisely
Clearly describe the definition of passage and the meaning of "earlypassage" (passage number) SHE cells, e.g. SHE cells in secondary or
tertiary culture.

The prediction model is poorly described in the SOP.

14.4

In agreement with Good Cell Culture
Practice (GCCP) (Coecke et al., 2005)
3.3

A precise approach for target cellseeding is crucial to counterbalance
possible loss of colonies due tocytotoxicity and, if not adjusted
appropriately, may lead to anunderestimation of the carcinogenicity
potential. Therefore, this procedureneeds to be well described.
It cannot be excluded that the cells
will change their properties withincreasing passage number. Precision
on passage parameters including
maximum admissible passage number
is important.

Provide better guidance concerning the maximum duration of storageof cryopreserved SHE cells. For example, the SHE pH6.7 CTA reportndstates in section 4.4. p.85, 2 paragraph that "the storage period
should not exceed 24 months". It should be explained why this period
should not exceed 24 months.

5. Recommendation 5: Better description of theprediction model

12.3

3.3

The SOPs should specify that the cell cultures should be regularly
tested for mycoplasma to exclude contamination.

3. Recommendation 3: Better description of how toadjust target cell seeding in case of cytotoxic effects ofthe test item

3.3

3.3

3.3

3.3

The threshold for spontaneous
morphological transformation
frequency (0.6%) is an importantcriterion of the prediction model andshould be added to the assessment
criteria as has been done in the SHE
pH6.7 SOP.

3.3

Continued on next page
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 77 of 95

Continued from previous page
Recommendation

Justification

6. Recommendation 6: Include guidance on doserange finding

Dose range finding is a crucial step for
defining the test dosage.
Inappropriate dose spreading maylead to missing the relevant dose.

In the context of the dose range finding, the ESAC WG recommends toinclude guidance for dose selection in the SHE protocols to ensure
proper use of the protocols in testing laboratories. The following
approach is suggested:
The highest dose should be determined base on RPE (%) and the lower
doses spaced out for instance with two concentrations per log (forinstance: 100, 30, 10, 3, 0.3, 0.1 etc.).

7. Recommendation 7: Include guidance concerning
retesting and the maximum number retesting runs
before results should be considered inconclusive

The high concentrations used for the Positive Control may hide
reduced sensitivity during test performance. The concentrations
should thus be reconsidered.

9. Recommendation 9: Provide guidance for training
and transfer
It should be considered whether the SOPs should provide instructions
regarding the coding of test dishes to improve objectivity of thescoring process (including positive control dishes).

4.1

While the PC is not used fornormalisation in this assay, it should
also provide information on therelative sensitivity of the test system.
High concentrations are however
likely to yield effects. Thus, a second
PC using a lower concentration or useof a 'batch control' should beconsidered.
The readout of the CTAs is visualscoring. A training and transfer
subsection should describe an
approach to transferring the test to an
inexperienced laboratory.

Proficiency chemicals should be defined at some stage allowing theself-assessment of laboratories.

10. Recommendation 10: Consider a more objective
approach to visual scoring: use of image analysis,
coding of test dishes
Considering the nature of the readout (visual scoring), it is suggested
that the SOPs contain a specific subsection on training and transfer tonaive laboratories.

8.1

Substances that give inconclusive
results after one test should be
retested. Retesting should however belimited to a reasonable number.

The ESAC WG noted that the number of admissible retesting runs incase of unqualified tests apparently had not been defined. This,
however, should ideally be done in any type of prevalidation orvalidation study.

8. Recommendation 8: Reconsider therecommended concentration for the Positive Control(PC)

Corresponding section of
the ESAC WG report

13

14.2,
14.4
14.2,
14.4

The readout of the CTAs is visualscoring. As with all visual assessments
there is a degree of subjectivity andpossible inconsistency which should
be minimised to the extent possible.

Despite the development of the photo catalogues supporting
consistent scoring, there remains a degree of subjectivity in theassessment of colonies. The ESAC WG suggests to consider automatedapproaches (e.g. image analysis) as possibly more objective ways toscore.

13

13

11. Minor recommendation
Section 3. p.77, line 8, section 5.2.1. p.82, line 6, section 5.2.2 p.82,
line 2 and section 5.3.3 p.82, line 1: "24 hours" should be"approximately 24 hours".

3.3

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 78 of 95

C) Detailed recommendations concerning the SOP of the BALB/C 3T3 CTA
The recommendations are in bold and numbered. Below each recommendation, the relevant
sections of the ESAC WG report are in cited (italics) and referenced in the right column. The middle
column provides a summary justification for the recommendation (blue).
Recommendation

Justification

Corresponding section of
the ESAC WG report

1. Recommendation 1: Include mandatory testing
requirement for mycoplasm

In agreement with Good Cell Culture
Practice (GCCP) (Coecke et al., 2005)

Status of the cells regarding mycoplasma should be clarified.

2. Recommendation 2: Define parameters relating tocell passage more precisely
Section 3.6.1 p.26, 2nd paragraph: BALB/c 3T3 cells were "…used for
the CTA within 3 to 4 passages". An explanation would be helpful as towhy "within 3 to 4 passages".

3. Recommendation 3: Further evaluate theproposed statistical method and refine the prediction
model
Refinement of the assessment requirements is required.
The criteria for an inconclusive result need strengthening.
The criteria for a positive / negative decision are not completely
defined. It is not clear whether these should be solely based on adecision rule using a currently un-referred statistical method. The
ESAC WG suggests to consider the development of a decision chart
which may aid consistent and transparent decision-making.

3.3

Precision on passage parameters
including maximum admissible
passage number is important.
However, it should be specified
whether this number relates to therisk that cells change their properties
with increasing passage number.
The prediction model is based on therecommended statistical method.
Further use of his method and further
investigation of its properties shouldbe performed to confirm theappropriateness of the assay
assessment criteria. Further testing
using this protocol and the associatedassay assessment criteria is
encouraged to gain further
information on protocol performance
including the statistical method.

3.3

4.2
13
13

Implication of comment that "experiments requiring repetition
because a biological effect was observed although the statisticalanalysis was negative": This suggests that the design of the statisticalevaluation as described in the SOPS may not be sufficient to detect
effects of size that are considered biologically important. More work
may need to be done on the SOPs to overcome this problem.

13

The statistical methods used requires further attention, better
definition and refinement of the decision criteria based upon them.
Considerations concerning the statistical methods may affect otheraspects of the design of possible future prospective studies (number ofplates, concentrations tested, requirements for repeat studies, etc).

14.1

There may be issues related to the specific decision rules based on
statistical tests (assessment criteria) leading to the calls as positive
and negative that need to be revisited. Statistical tests used with the
BALB/c 3T3 assay seem to lead to some inexplicable results. The
properties of the various statistical methods used need further
evaluation.

14.2

The dose response curves seem less variable compared with thosefrom the SHE assays, but refinement of the assessment criteria may berequired.

14.2

The recommended statistical method suggested for the BALB/c 3T3
CTA should be investigated more in depth to provide a better
description of its properties and to allow its use in practice.

14.4

Continued on next page

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 79 of 95

Continued from previous page
Recommendation

Justification

4. Recommendation 4: Include guidance on doserange finding

Dose range finding is a crucial step for
defining the test dosage.
Inappropriate dose spreading maylead to missing the relevant dose.

The doses may have to be spaced differently for the BALB/c assay
since the responsive concentrations are spaced in a narrower band ascompared to the SHE. This could be the reason why repetitions of
BALB/c experiments with more narrow doses resulted in change from'negative' to 'positive' calls/predictions.

Corresponding section of
the ESAC WG report

8.1

Based on the study data, there is therefore a need to optimize thenumber and spacing of doses in the BALB/c 3T3 SOP (e.g. 8 doses withwhat spacing?). For instance, in the BALB/c 3T3 CTA report some ofthe dose-responses are quite steep and could be missed if the dose
spacing is not correct.

5. Recommendation 5: Include guidance concerning
retesting and the maximum number retesting runs
before results should be considered inconclusive

Substances that give inconclusive
results after one test should be
retested. Retesting should however belimited to a reasonable number.

The ESAC WG noted that the number of admissible retesting runs incase of unqualified tests apparently had not been defined. This,
however, should ideally be done in any type of prevalidation orvalidation study.

6. Recommendation 6: Define criteria for the vehicle
control more precisely
The maximum number of Type III foci in the entire set of vehicle
control dishes should not exceed five”. The specific reason forrequiring a maximum of “five” foci should be explained and justified.
The acceptable number of vehicle control (VC) foci is <6. This perhaps
should be justified in more detail.

7. Recommendation 7: Reconsider therecommended concentration for the Positive Control(PC)
A high dose was used for the positive control which may hidevariations in test performance.

8. Recommendation 8: Consider the use ofexogenous metabolic activation system
The introduction of exogenous metabolic activation systems into the
BALB/c 3T3 CTA would support the applicability of the assay to abroader range of chemicals.

9. Recommendation 9: Provide guidance for training
and transfer
It should be considered whether the SOPs should provide instructions
regarding the coding of test dishes to improve objectivity of thescoring process (including positive control dishes).

4.1

An acceptable background level of
spontaneous foci should be definedand justified. If this criterion should bebased on historical data,
inexperienced laboratories should begiven instructions to build their own
database in a reliable way.

2.4

While the PC is not used fornormalisation in this assay, it should
also provide information on therelative sensitivity of the test system.
High concentrations are however
likely to yield effects. Thus, a second
PC using a lower concentration or useof a 'batch control' should beconsidered.

3.3

Some compounds need priorbioactivation to exert theirtransforming properties. The BALB/c
3T3 cells are known to have a limited
metabolic activity which could restrict
their use to certain compounds only.
This could be mitigated by theaddition of an exogenous metabolic
activation system to the test system.
The readout of the CTAs is visualscoring. A training and transfer
subsection should describe an
approach to transferring the test to an
inexperienced laboratory.

Proficiency chemicals should be defined at some stage allowing theself-assessment of laboratories.

2.5

14.4

13

14.2,
14.4

Continued on next page
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 80 of 95

Continued from previous page
Recommendation

Justification

10. Recommendation 10: Consider a more objective
approach to visual scoring: use of image analysis,
coding of test dishes

The readout of the CTAs is visualscoring. As with all visual assessments
there is a degree of subjectivity andpossible inconsistency which should
be minimised to the extent possible.

Considering the nature of the readout (visual scoring), it is suggested
that the SOPs contain a specific subsection on training and transfer tonaïve laboratories.
Despite the development of the photo catalogues supporting
consistent scoring, there remains a degree of subjectivity in theassessment of colonies. The ESAC WG suggests to consider automatedapproaches (e.g. image analysis) as possibly more objective ways toscore.

Corresponding section of
the ESAC WG report

14.2,
14.4
13

11. Minor recommendation
p.95, Appendix 1, Culture vessels: 100x20 mm dishes should be 90x20
mm dishes.

3.3

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 81 of 95

Annex 5 – ESAC Request concerning the review of the CTA prevalidation study

ESAC Request 2010-02
ECVAM Scientific Advisory Committee(ESAC)

ECVAM REQUEST FOR ESAC ADVICE
on an ECVAM-coordinated prevalidation study concerning theprotocols of three Cell Transformation Assays (CTA) for
carcinogenicity testing

INSTRUCTIONS FOR IVM/ST STAFF:
Blue text: to be filled in by the ECVAM Scientific Officer completing the draft request in collaboration
with ESAC Secretariat.
Green text: to be filled in by the ESAC Secretariat.

Title page information
Abbreviated title of ESAC
requestESAC REQUEST Nr.
Filename
Template used for preparing
requestDate of finalising request
Date of submitting request to
ESAC
Request discussed throughOpinion expected at (date)

ESAC peer review of and ESAC opinion on the ECVAM-led
prevalidation study of three cell transformation assays for
carcinogenicity testing:1) SHE pH 6.7 assay
2) SHE pH 7.0 assay
3) BALB/c 3T3 assay.
2010-02
ESAC REQUEST_2010 02_CTA+ESAC-WG-Mandate-approved.doc
EP 2.01
2010-09-14
2010-10-02
Plenary at ESAC33 at 2010/10/12
Through written procedure in January 2011 (before OECD WNT in
March)ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 82 of 95

1.

TYPE OF REQUEST

Request Type

Identify request ("YES")

R1 ESAC Peer Review

YESof a Prevalidation Study or Validation Study
If R1)applies please specify further:•Prevalidation StudyYES
The study is a complement to the recommendationsof the OECD Detailed Review Paper on Cell
Transformation Assays. The study addressed
protocol standardisation, transferability andreproducibility (but not performance) of three
protocols of cell transformation assays in view of
establishing standardised protocols for future
consistent use, e.g. through the development ofOECD test guidelines for in vitro carcinogenicity
testing.

•Prospective Validation Study•Retrospective Validation Study•Validation Study based on PerformanceStandards

R2 Scientific Advice on a test method submitted to
ECVAM for validation(e.g. the test method's biological relevance etc.)

R3 Other Scientific Advice(e.g. on test methods, their use; on technical issues such as cell
culturing, stem cells etc.)

2.

TITLE OF STUDY OR PROJECT FOR WHICH SCIENTIFIC ADVICE OF THE
ESAC IS REQUESTEDPrevalidation of three cell transformation assays for carcinogenicity testing:1) SHE pH 6.7 assay
2) SHE pH 7.0 assay
3) BALB/c 3T3 assay

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 83 of 95

3.

BRIEF DESCRIPTION OF THE STUDY OR PROJECT

1) Background to carcinogenicity testing and available alternative methods
The potential for a compound to induce carcinogenicity is a crucial consideration when establishinghazard and risk assessment of chemicals and pharmaceuticals in humans. To date, the standard
approach to assess carcinogenicity at a regulatory level is the 2-year bioassay in rodents (OECD TG
451; Ref. 1).
Several in vitro alternatives have been developed for predicting carcinogenicity. Of these, the in vitro
genotoxicity tests address only one mechanism involved in carcinogenicity, the induction of genetic
damage. In contrast, in vitro Cell Transformation Assays (CTAs) have been shown to involve a
multistage process that closely models some stages of in vivo carcinogenesis: CTAs can detect
phenotypic changes of cultured cells as a result of exposure to test materials (i.e. chemicals, productsetc.). These phenotypic/morphological changes are a result of the transformation of cultured cells
which involves changes in cell behaviour and proliferation control (e.g. altered cell morphology,changed colony growth patterns and anchorage –independent growth). Moreover, transformed cells
can evolve to be tumorigenic when injected in a suitable host. Importantly, CTAs are to date the only
optimised tests that allow the detection of both genotoxic and non-genotoxic carcinogens. CTAs havebeen in use for about 40 years and are currently being performed by academia, the chemical, agrochemical, cosmetic and pharmaceutical industries. CTAs are conducted in-house as well as at
contract research organisations to screen for potential carcinogenicity as well as investigate
mechanisms of carcinogenicity. While CTAs are currently not used routinely for regulatory testing,they are frequently used for internal (in-house) safety assessment of chemicals, drugs, etc. and areconsidered to provide additional useful information to the prevailing tests that are used for assessing
carcinogenic potential.
2) The OECD Detailed Review Paper as the basis for this prevalidation study
In order to systematically assess the performance of the CTAs, the Organisation for Economic Cooperation and Development (OECD) finalised in 2007 a "Detailed Review Paper on Cell
Transformation Assays For Detection of Chemical Carcinogens" (OECD DRP). The OECD DRP focusedon the analysis of the predictive capacity (relevance) of three CTAs and addressed also someelements of reliability: (1) the Syrian hamster embryo (SHE) assay, (2) the BALB/c 3T3 assay and (3)the C3H10T1/2 assay. A substantial body of existing and published data was evaluated (SHE n=264
chemicals; BALB/c 3T3 n=184; C3H10T1/2 n=141). The OECD DRP concluded that the performancesof two of the assays, the SHE assay and BALB/c 3T3 assay, were sufficiently adequate and should be
developed into formal OECD test guidelines (OECD DRP, Ref. 2). Further, the same OECD DRP
recommended that although considerable data on the performance of the assays were available, aformal assessment of the assays, in particular focusing on development of a standardisedtransferable and reproducible protocol, would be important for preparation of such OECD test
guidelines.
3) Study objectives and design
Based on the OECD DRP and several ECVAM expert meetings (Combes et al., 1999, Ref. 3), ECVAM
initiated a study on the two CTAs found most relevant by the OECD DRP on the basis of the available
information, the SHE and the BALB/c 3T3 assays. The study constitutes a complement to theextensive OECD DRP and its conclusions. In agreement with the conclusions of the OECD DRP, ECVAM
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 84 of 95

focused on the development and evaluation of standardised, well-documented protocols that could
serve as a basis for an OECD test guideline. In summary, the study was organised and designed taking
into account:• the objective of the study to address protocol standardisation and an assessment oftransferability and reproducibility of the standardised CTA protocols but not their predictive
capacity (which is addressed by the OECD DRP) and• the high costs and considerable time required to perform the assays as well as the limited
funding and resources which could be made available by ECVAM.
The study addressed the three classical aspects of Prevalidation: I) protocol
refinement/standardisation; II) protocol transfer and III) protocol performance (ECVAM 1995, Ref.4;
OECD guidance document on validation, 2005, Ref. 5). With respect to the modular approach ofvalidation (Hartung et al., 2004, Ref. 6), the study assessed information concerning module 1) test
definition, module 2) within-laboratory reproducibility, module 3) transferability, module 4)
between-laboratory reproducibility.
The study addressed three variants of CTA protocols: two SHE protocol variants (cells at pH 6.7 and at
pH 7.0, respectively) and the CTA based on the BALB/c 3T3 A31 cell line. Each protocol was assessedusing six chemicals. In contrast to the BALB/c 3T3 protocol which required more substantialrefinement, both SHE protocols were already available in the literature and results of these havebeen reported in the OECD DRP. Between-laboratory reproducibility was examined in threelaboratories except for the SHE 7.0 protocol, where four laboratories were involved.
4) Results and Conclusions
The Validation Management Team (VMT) concluded that, for the SHE pH 6.7 and the SHE pH 7.0
CTAs, the study had demonstrated that standardised protocols were available which could be thebasis for future use. These protocols and the assay system itself have been shown to be transferable
between laboratories, and are reproducible within- and between-laboratories. For the BALB/c 3T3
method, an improved protocol has been developed, which allowed obtaining reproducible results.
However, further testing of the improved BALB/c 3T3 protocol is recommended (see Validation Study
Reports, Ref. 7-9). Moreover, the VMT concluded that the appropriate training and the use of thephoto catalogues (see Photo Catalogues, Ref. 10-12) developed during the protocol refinement
phase, led to a consistent scoring of transformed colonies and foci.
Overall, these results in combination with the extensive database summarized in the OECD DRP
support the utility of in vitro CTAs for the assessment of carcinogenicity potential.

References
1

OECD TG 451 on rodent long term carcinogenicity testing2

OECD. Detailed Review Paper on Cell Transformation Assays for Detection of Chemical Carcinogens,OECD Environment, Health and Safety Publications, Series on Testing and Assessment, No. 31 (2007).

3

Combes R., Balls M., Curren R., Fischbach M., Fusenig N., Kirkland D., Lasne A., Landolph J., LeBoeuf R.,
Marquardt H., McCormick J., Mueller L., Rivedal E., Sabbioni E., Tanaka N., Vasseur P. and Yamasaki H.
Cell transformation assay as predictors of human carcinogenicity. Alter. Lab. Anim., 27 (1999) 745-67.

4

ECVAM Prevalidation Task Force Report 1: The role of prevalidation in the development, validation
and acceptance of alternative methods. ATLA 23, 211-217 (1995)

5

OECD Series on Testing and Assessment Number 3: Guidance document on the validation andinternational acceptance of new or updated test methods for hazard assessment. OECD, Paris, 2005.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 85 of 95

6Hartung T., Bremer S., Casati S., Coecke S., Corvi R., Fortaner S., Gribaldo L., Halder M., Hoffmann S.,
Roi A.J., Prieto P., Sabbioni E., Scott L., Worth A. and Zuang V. A modular approach to the ECVAM
principles on test validity. Alter. Lab. Anim., 32 (2004) 467-72.

7

VMT-study report SHE pH 6.7

8

VMT-study report SHE pH 7.0

9

VMT-study report BALB/c 3T3

10

CTA SHE pH 6.7 photo catalogue

11

CTA SHE pH 7.0 photo catalogue

12

CTA BALB/c 3T3 photo catalogue

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 86 of 95

4.

OBJECTIVES, QUESTIONS, TIMELINES

4.1

OBJECTIVE

Objective
Why does ECVAM
require advice on
the current issue?

Given the background in Section 3, the opinion of the ESAC should provide expert
advice to ECVAM on three studies that ECVAM conducted in view of assessing
whether the three CTA protocols (SHE 6.7; SHE 7.0 and BALB/c) have been
sufficiently standardised to be transferable to other laboratories and reproducible
between different laboratories and may therefore be fit for future use.
In providing this advice, ESAC is requested to take account of the existing
information (in particular the OECD DRP) and address also the suitability of thethree CTA assays/protocols in question to be used as a basis for the developmentof OECD test guidelines as foreseen by the OECD in the context of the OECD DRP
which led to the present study.

4.2

QUESTION(S) TO BE ADDRESSEDQuestions

The ESAC is requested to address the following three questions:

What are thequestions andissues that should
be addressed inview of achieving
the objective of
the advice?

1) to review whether the study of the three CTAs was conducted appropriately inview of the stated purpose, i.e. of assessing whether the CTA protocols are
sufficiently standardised to be transferable and reproducible.
In particular the following issues should be addressed:a) Clarity of the definition of the study objective.
b) Appropriateness of the study design (e.g. chemical selection, number ofchemicals used, number of laboratories, acceptance criteria).
c) Appropriateness of the study execution (e.g. were there pre-defined
acceptance criteria, were these respected? How were exceptions /deviations handled, e.g. retesting?).
d) Appropriateness of the statistical analysis as used in the protocols and foranalysing reproducibility.

2) to assess whether the conclusions as presented in the Study Reports by the
Validation Management Team are justified by the information generated during
the study and whether they are plausible with respect to existing information andcurrent views (e.g. literature), in particular the OECD DRP on CTAs.
In particular the following issues should be addressed:a) Provide a qualitative discussion of the study results/deliverables
achieved within the limits of this prevalidation study:•

Clarity and completeness of the standardised protocol.

•

Within laboratory reproducibility•

Transferability (critical issues and how they werehandled)ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 87 of 95

• Between laboratory reproducibilityb) Provide a clear presentation of the conclusions presented in the study
reportsc) Evaluate to which extent the conclusions are justified by the study
results aloned) Discuss the plausibility of the conclusion in the light of the study
results AND existing historical information as available to the EWG (inparticular the OECD DRP which led to this study).

3) to express its opinion with regard to the question whether the CTA protocols
standardised and evaluated during the study could indeed be recommended toserve as a basis for an OECD test guideline on in vitro carcinogenicity testing.
In particular the following issues should be addressed:a) Similarity of the standardised protocols with respect to the historical
protocols (provide to the extent possible a direct comparison anddiscuss the relative importance of any difference identified).
b) Other critical issues and gap analysis (what further work may be
useful/required). Please provide a rationale for your proposed
position.

4.3

TIMELINES

Timelines
concerning thisrequestTimeline

Indication

Finalised ESAC Opinion required by:

The ESAC opinion should be available
latest during the second half of
February 2011 (e.g. 20.1.2011).

When does
ECVAM require
the advice?

An attempt will be made to finalise theopinion by written procedure.
[N.B. Following ESAC33 (12 October2010) the entry has been revised(original
text
below
in
grey/strikethrough): before February
2011, if possible (i.e. before the OECDWNT meeting)]
Request to be presented to ESAC bywritten procedure (e.g. due tourgency) prior to the next ESACNO

Request to be presented to ESAC at
ESAC plenary meeting

October 2010

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 88 of 95

5.

ECVAM PROPOSALS ON HOW TO ADDRESS THE REQUEST WITHIN ESAC

5.1

ECVAM PROPOSAL REGARDING REQUEST-RELATED STRUCTURES REQUIREDSpecific
structures
required within
ESAC to address
the request

Structure(s) required

Required according to ECVAM?

S1 ESAC RapporteurNO

S2 ESAC Working GroupYES

Does the advice
require an ESAC
working group, an
ESAC rapporteuretc.?

S3 Invited Experts

NO

Ad S3: If yes – list names andaffiliations of suggested
experts to be invited andspecify whether these aremember of the EEPNO

If other than above (S1-S3):

NO

5.2

DELIVERABLES AS PROPOSED BY ECVAM

Deliverables
What deliverables(other than the
ESAC opinion) are
required for
addressing therequest?

Title of deliverable otherthan ESAC opinion

Required?

D1 ESAC Rapporteur Report NO
and draft opinion

D2 ESAC Peer Review Report YES (ECVAM proposal)and draft opinion
If other than above (D1-D2):

NO

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 89 of 95

6.

LIST OF DOCUMENTS TO BE MADE AVAILABLE TO THE ESACCount

Description of document

Available (YES/NO)

File name1

VMT-study report SHE pH 6.7

YES1)ER2010-02_SHE6.7.pdf

2

VMT-study report SHE pH 7.0

YES2)ER2010-02_SHE7.0.pdf

3

VMT-study report BALB/c 3T3

YES3)ER2010-02_Balb.pdf

4

CTA SHE pH 6.7 photo catalogue

YES4)ER2010-02_SHE6.7photo.pdf

5

CTA SHE pH 7.0 photo catalogue

YES

6

CTA BALB/c 3T3 photo catalogue

YES

7OECD. Detailed Review Paper on Cell
YES
Transformation Assays for Detection ofChemical Carcinogens, OECD Environment,
Health and Safety Publications, Series on
Testing and Assessment, No. 31 (2007).

5)ER2010-02_SHE7.0photo.pdf
6)ER2010-02_Balbphoto.pdf
7)ER2010-02_OECDDRPonCTAs.pdf

8Combes R., Balls M., Curren R., FischbachM., Fusenig N., Kirkland D., Lasne A.,
Landolph J., LeBoeuf R., Marquardt H.,
McCormick J., Mueller L., Rivedal E.,
Sabbioni E., Tanaka N., Vasseur P. andYamasaki H. Cell transformation assay as
predictors of human carcinogenicity. Alter.
Lab. Anim., 27 (1999) 745-67.

YES8)ER2010-02_ECVAMWS-Report-on-CTAs.pdf

9OECD TG 451 on rodent long term
carcinogenicity testing

YES9)ER2010-02_OECD-TG451.pdf

YES10)ER2010-02_ECVAMprevalidation.pdf

YES11)ER2010-02_OECDGuidanceDocument.pdf

YES12)ER2010-02_ECVAMmodular-approach.pdf

10

11

12

ECVAM Prevalidation Task Force Report 1:
The role of prevalidation in thedevelopment, validation and acceptance ofalternative methods. ATLA 23, 211-217(1995)
OECD Series on Testing and Assessment
Number 3: Guidance document on thevalidation and international acceptance of
new or updated test methods for hazard
assessment. OECD, Paris, 2005.
Hartung T., Bremer S., Casati S., Coecke S.,
Corvi R., Fortaner S., Gribaldo L., Halder M.,
Hoffmann S., Roi A.J., Prieto P., Sabbioni E.,
Scott L., Worth A. and Zuang V. A modular
approach to the ECVAM principles on test
validity. Alter. Lab. Anim., 32 (2004) 467-72.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 90 of 95

7.

TERMS OF REFERENCE OF THE ESAC WORKING GROUP

7.1

ESTABLISHMENT OF THE ESAC WORKING GROUP

During its 33rd meeting on 12 October 2010 the ESAC plenary unanimously decided to establish an
ESAC Working Group charged with the detailed scientific review of a study on three Cell
Transformation (CTA) protocols.

7.2

TITLE OF THE ESAC WORKING GROUP

Full title:"ESAC Working Group on the scientific review of 3 Cell Transformation Assay (CTA) prevalidation
studies (SHE 6.7, SHE 7.0, BALB)".
Abbreviated title:"ESAC Working Group CTA"

7.3

MANDATE OF THE ESAC WG

The EWG is requested to conduct a scientific review of the ECVAM study concerning three protocolsof the Cell Transformation Assay (CTA). The review needs to address the questions put forward to
ESAC by ECVAM.
The review should focus on the appropriateness of design and conduct of the study in view of thestudy objective and should provide an appraisal to which extent the conclusions of the Validation
Management Team (VMT) are substantiated by the information generated during the study and how
the information generated relates to the scientific background available.

7.4

DELIVERABLE OF THE ESAC WG

The ESAC WG is requested to deliver to the chair of the ESAC and the ESAC Secretariat a detailed
ESAC Working Group Report outlining its analyses and conclusions. A reporting template has been
appended (Appendix 1) intended to facilitate the drafting of the report.
The conclusions drawn in the report should be based preferably on consensus. If no consensus canbe achieved, the report should clearly outline the differences in the appraisals and provide
appropriate scientific justifications.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 91 of 95

7.5

PROPOSED TIMELINES OF THE ESAC WG

The Secretariat has proposed timelines which should be agreed upon during the first Teleconference(Item 1 in the table):
Item Proposed date/time
ActionDeliverable
1
Teleconference
Kick-off teleconference to• Agreed timelines
5 November 2010, 14:00 CET• discuss the mandate,• Agreed work plandeliverables, timelines, study
and distribution
background• agree on timelines andmeeting dates/times (seeitem2)• distribute (if appropriate)
work and agree on further
communication (e.g. TCs if
required)2
First ESAC WG meeting in Ispra
Possibly preliminary• Discussions of the relevant
versions of• Option 1 (preferred):material and preliminary
30.11. – 2.12.2010 (3 days)
analysis and possible• ESAC WG Reportconclusion.
• Option 2:
6.12 - 7.12.2010 (2 days)• Identification of unresolved
issues and disagreements• Option 3 (least preferred):
14.12. – 16.12.2010 (3 days)• Identification of process toresolve possible
disagreements• Further work distribution andcommunication means(e.g. TCs)
• Beginning of drafting processof report
3
Teleconference
Refresher teleconference (if
10. January 2011, 14:00 CET
required) to revisit the status of
the work, plan what remains to bedone before the second meeting.
4
Second (last) ESAC WG meeting
Finalisation of ESAC WG Report
Final versions ofin Ispra• ESAC WG Report• Option 1:
12.1. – 14.1.2011 (3 days)• Option 2:• 19.1. – 21.1.2011 (3 days)
5

Tuesday 25.1.2011

Handover of report to ESAC chair
and Secretariat

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Final edited versions(ready for distribution
to ESAC):• ESAC WG Report

Page 92 of 95

7.6

QUESTIONS WHICH SHOULD BE ADDRESSED BY THE ESAC WG

The ESAC WG is requested to address the three questions posed to the ESAC which have beenbroken down further in more specific questions by the ESAC chair, the chair of the ESAC WG and theSecretariat (see section 4.2).
When preparing the final ESAC WG report to address these questions, the ESAC WG is requested touse a pre-defined reporting template. This template (see appendix 1) follows ECVAM's modular
approach and addresses to which extent the standard information requirements have beenaddressed by the study. The template allows moreover for addressing the issues specific studies
outlined in section 4.2. The Secretariat will provide guidance if necessary.

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 93 of 95

APPENDIX 1REPORTING TEMPLATE FOR THE ESAC WG REPORTThe following suggested template follows the ECVAM modular approach and allows at the same time
for the description of the analysis and conclusions concerning more specific questions. The template
can be used for various types of validation studies (e.g. prospective full studies, retrospective studies,
performance-based studies and prevalidation studies). Depending on the study type and theobjective of the study, not all sections may be applicable. However, for reasons of consistency and toclearly identify which information requirements have not been sufficiently addressed by a specificstudy, this template is uniformly used for the evaluation of validation studies.
Text in red is explanatory, not intended to be part of the title.
One section is clearly not applicable to the present CTA study (identified).
1. Data collection
1.1 Information / data sources used (e.g. reference data)
1.2 Search strategy
1.3 Selection criteria applied to the available information
2. Study objective and design
2.1 Clarity of the definition of the study objective
2.2 Analysis of the scientific rationale provided
2.3 Analysis of the regulatory rationale provided
2.4 Appropriateness of the study design(selection of test items, number of test items, number of laboratories, retesting in case of
unqualified tests etc.)
2.5 Appropriateness of the statistical evaluation(independence of statisticians, statistical method)
3. Test definition (Module 1)
3.1 Quality and completeness of the overall test definition(test system, protocol, test acceptance criteria etc.)
3.2 Quality of the background provided concerning the purpose of the test method
3.3 Quality of the documentation and completeness of (a) standardised protocols (SOPs) and(b) prediction models
4. Data quality
4.1 Overall quality of the evaluated data
4.2 Sufficiency of the evaluated data in view of the study objective
4.3 Quality of the reference data for evaluating reliability and relevance14
5. Test materials
5.1 Sufficiency of the number of evaluated test items in view of the study objective
5.2 Representativeness of the test items with respect to the applicability domain

14

OECD guidance document Nr. 34 on validation defines relevance as follows: "Description of relationship of
the test to the effect of interest and whether it is meaningful and useful for a particular purpose. It sis theextent to which the test correctly measures or predicts the biological effect of interest. Relevance incorporates
consideration of accuracy (concordance) of a test method."
ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 94 of 95

6. Within-laboratory reproducibility (Module 2)
6.1 Assessment of repeatability and reproducibility in the same laboratory
6.2 Conclusion on within-laboratory reproducibility as assessed by the study
7. Transferability (Module 3)
7.1 Quality of design and analysis of the transfer phase
7.2 Conclusion on transferability to a second laboratory as assessed by the study
In particular: where critical issues that may impact on transferability identified or addressed?
8. Between-laboratory reproducibility (Module 4)
8.1 Assessment of reproducibility in different laboratories
8.2 Conclusion on reproducibility as assessed by the study
9. Predictive capacity (Module 5) N.B. Predictive capacity was outside the scope of the study
9.1 Adequacy of the assessment of the predictive capacity in view of the purpose
9.2 Overall relevance (biological relevance and accuracy) of the test method in view of the
purpose
10. Applicability domain (Module 6) N.B. Since this study is not a full validation study, theassessment of the applicability domain is rather limited
10.1 Appropriateness of study design to conclude on applicability domain, limitations andexclusions
10.2 Quality of the description of applicability domain, limitations, exclusions
11. Performance standards (Module 7) N.B. Not applicable to the current study.
11.1 Adequacy of the proposed Essential Test Method Components
11.2 Adequacy of the Reference Chemicals
11.3. Adequacy of the defined Accuracy Values
12. Readiness for standardised use
12.1 Assessment of the readiness for regulatory purposes
12.2. Assessment of the readiness for other uses (in house screening etc.)12.3 Critical aspects impacting on standardised use
12.4 Gap analysis
Identify, if appropriate, gaps in the study design and/or execution that impact on the statedstudy objective or the conclusions drawn.
13. Other considerations
Please address any other consideration you might have in relation to the proposed approach
under this section.
14. Conclusions and recommendation
14.1 Summary of the study results and conclusions
14.2 Extent to which conclusions are justified by the study results alone
14.3 Extent to which conclusions are plausible in the context of existing information
14.4 Recommendations

ESAC WG REPORT on 3 protocol variants of the Cell Transformation Assay (CTA)

Page 95 of 95