1 Appendix E: Section 5 Supplemental, HTP Data Processing Pipeline

Description of Workflow and “Level” files The ToxCast workflow is divided into three main portions: 1. pipeline; 2. fitting/hit

- calling; 3. ACToR/ToxCastDB . The pipeline portion includes QA / data processing / sample mapping / normalization steps (called Level0

- Level4). The fitting/hit

- calling portion includes curve fitting / hit

- calling / filtering (called Level5

- Level8). The pipeline information is accompa nied by Support Files that provide auxiliary information for all Level output files. The ACToR/ToxCastDB portion organizes relevant information from all “Levels” into a standardized database that will then become Web

- accessible . The “Levels” in the workfl ow are graphically summarized in Figure E

- 1, following textual descriptions of all file types.

Level0 Level0 "Raw" data are files from sources/vendors prior to any normalization/correction done by EPA. These are usually multiple files of different types a nd are not yet in a computable format.

Level1 Level1 "Raw" data are computable files (matrices, plate maps, etc.) from sources/vendors prior to any normalization/correction done by EPA. These are usually multiple files of different types. As many of the incoming data sets correspond to contract task orders, these initial raw data files and any accompanying summary reports from the contractors often provide the EPA with an overview of work completed.

Level2 Level2 "Processed Raw" files are raw data that have been reshaped into a single file. Columns have been added for meta

- data such as assay_name, sample_id, chemical_id, chemical_casrn, chemical_name, chemical_plate_id, assay_plate_id, row_index, col_index, row_letter. No data values have been altered. F or corresponding contractual purposes, the processed raw files are used to ensure complete coverage of ordered sample

- assay combinations as well as cursory quality control checks. Early quality control checks include requiring the vast majority of technica l replicates to be within 20% of each other and for negative and positive control to demonstrate expected response.

Level3 Level3 "Primary Long” files are standardized long

- format data files, wherein each line represents a single data point. Beginning wit h this level, data are subject to correction/normalization based upon plate/batch effects, positive/negative control normalization. The default normalization method for the vast majority of the assays utilizes the positive control (e.g., estradiol) raw

res ponse values for each plate or run and either the DMSO plate or run raw response values or the median plate response. The basic formula for percent activity is the treatment raw response minus the negative control raw response divided by the positive contr ol raw response minus the negative control raw response multiplied by one

- hundred [ % Activity = (Treatment

- Neg)/(Pos

Neg)*100 ]. This default method subtracts out background while providing a percent of positive control normalized value.2 Level4 Level4 " Primary Wide” files are standardized wide

- format data files, wherein each line represents concentration

- response data for a given sample technical replicate (sample_tech_rep_id) in a given assay. This level is fed into the curve

- fitting/hit

- calling portion of the workflow. There are two types of data permitted in the curve

- fitting/hit

- calling portion of the workflow and they are: 1. percent activity data with a baseline of 0% (often normalized to DMSO control or median plate raw values) and top end values o f 100% or

- 100% (often normalized to a positive control, e.g., estradiol for estrogen agonist assays); 2. Fold induction data which are used when a good scalar reference (positive control) is not available and therefore values are normalized only to DMSO n egative control as a fold change value from baseline of 1. For estrogen receptor assays, estradiol was used as a positive control and therefore percent activity values were generated across all assay formats.

Level5 Level5 "Fit Mask" files are wide

- format files that contain information on data points that are masked from fitting in subsequent levels Masking may be based upon cytotoxicity information, outlier algorithms, or chemical contamination information depending on the assay platform. In addition to outlier detection, negative

- mode transformation, adding of an assay postfix and baseline correction can also be performed. Since all curve

- fitting is performed with positive Hill

slopes (i.e., forcing the curve to increase), negative

- mode transformation in volves a simple step that positively transforms the data (e.g., describe an example). This is often accompanied by an addition of variables that describes the assay signal and curve

- fitting direction. For instance, if values range from 0% to

- 100% then one would apply the negative

- mode transformation making the value 0% to 100% and add an assay postfix of “negative” or “neg” or “inhibition” clearly identifying the signal direction the data was analyzed. Baseline correction simply transforms the low concentr ation values of a sample technical replicate (sample_tech_rep_id) to straddle the intended baseline; 0 for percent activity data and 1 for fold induction data.

Level6 Level6 "Fit Results (Internal)" files include tabular output and graphics files. The ta bular output files contain results of the curve

- fitting/hit

- calling procedure. The graphics files are PDFs containing diagnostic concentration

- response plots for all chemical sample

- assay pairs. The diagnostic plots are for analysis quality control and for iterative exploration of optimal curve

fitting and hit

- calling parameters. The “Internal” designation means that Level6 results use only the custom fitting algorithm on data from within an assay. Level6 results provide hill model parameters, an initial hi t

- call based on statistically

- derived global assay response cutoffs from baseline, and a characterization of the curve

- fitting and hit

- calling determinations. Due to the homogeneous analysis approach, the curve

- fitting and hit

- calling algorithms attempt to handle all special cases that can be addressed or identified in a systematic manner. The special case identification also assists with subsequent filtering and finalized activity calls.

Level7 Level7 "Fit Results (External)" files are tabular output foll owing all automated meta/external filters on the Level6 results. The “External” designation means that meta

- data, external data from other assays, or information from previous projects may have been used to filter Level6 results. Level7 data represent the highest level of automated fit/hit calls. In Section 6, the specific level 73 filters used in this project are described. Level 7 also produces graphic output files which are used laster to display data.

Level8 Level8 "Final Results" files are tabular output that are ready for loading into the ACToR/ToxCastDB portion of the workflow. Level8 files have been subjected to expert/manual curation of results.

Support Files "Support Files" include all auxiliary information about the “Levels” files. This includes meta

data needed for processing raw data, such as files for mapping vendor assay IDs to ToxCastDB

style IDs, vendor chemical (TX code) mapping, chemical plate concent ration corrections, maps of assay

- wise control compounds, etc. Support Files also include "Log Files", which are summary files containing information on data contents (chemicals, number of concentrations, assays run, etc.), summary statistics, error logs, plate

- wise enumeration of controls, etc.

Figure E

- 1 File types and descriptions in the ToxCast data workflow.

ACToR/ToxCastDB Files ACToR/ToxCastDB files contain summary information from across all “Levels” of the workflow. These files are created from the standardized database that is published online
at http://actor.epa.gov . All ToxCast / Tox21 data is made available in the ToxCastDB portion of4 ACToR for browsing, as well as in a series of downloadable files. Current ly, only summary text data is available through the ToxCastDB web site either for browsing or downloading, but text and graphics are being built into the newest version. All data files (including all concentration

response plots) described in this document will be available to download through the ToxCastDB and NCCT web sites. The ToxCastDB “export” files contain several file types, each containing consolidated information on: Chemicals, Chemical Structure, In Vitro Data (Long format), In Vitro Data (AC50 m atrix format), and Assay Definitions.

Data headers / column names in files The “Levels” files use a consistent set of column names. Although individual assay platforms necessitate diverse analytical approaches

— meaning that certain assay data files may not contain all columns

— a column name represents a consistent data concept. ACToR/ToxCastDB files having the same column name also represent a consistent data concept.

Workflow “Levels” Files

sample_tech_rep_id = Index of the sample_rep_id (see below) based on the number of technical replicates for that assay; General assay reproducibility statistics performed at this level (e.g., correlation among technical replicates) sample_rep_id = Individual sample replicate ID; This is the blinded ID sent to assay vend ors. Sample replicates come from the same lot/batch and were intended additions to the sample

chemical library; Replicate analysis (from same lot/batch) performed at this level (e.g., hit

- call concordance). sample_id = Base sample ID which equates to a sam ple vial from one lot/batch; Primary analysis performed at this level. chemical_id = DSSTox Generic Substance ID; Additional reproducibility performed at this level to evaluate impact of separately sourced chemicals. chemical_casrn = CAS registration numbe r. chemical_name = Chemical name; This may be a common name, IUPAC name, or some combination thereof. assay_name = Unique assay component endpoint name (breaks out by readout, timepoint, curve fit direction, etc). AC50.mod = AC50 modifier used to disting uish hit confidence at a terse level;

“==”

- met basic requirements of hill model with some minimal confidence in T and B

“><”

- Not a hit and no model fit

“<=”

- AC50 set to low concentration tested as no model could be determined but high levels of activity w ere observed

“>=”

- Hill model was fit but with little confidence in T (top) of curve as the model had a hill slope of 8 and had only a single point determining the hit call5 “<>”

- Significant activity was observed but the data was noisy or the hill model did not meet certain criteria. This assumes the AC50 should be somewhere in the tested concentration range but there is little confidence in the point estimate B = Bottom (minimum asymptote) of curve (parameter of hill model) T = Top (maximum asymptote) of curve (parameter of hill model) logAC50 = 50% maximal response concentration (log uM) (parameter of hill model) AC50 = Converted 50% maximal concentration from hill model (uM) W = Hill slope (parameter of hill model) Emax = Maximal average response at a given concentration (outlier removed)

T2B = T

- B (Top of curve minus Bottom of Curve) Rsq = R

- squared of the hill model B.std.error = Standard Error of the B parameter (from summary(nls)) T.std.error = Standard Error of the T parameter (from summary(n ls)) logAC50.std.error = Standard Error of the logAC50 parameter (from summary(nls)) W.std.error = Standard Error of the W parameter (from summary(nls)) B.pval = P

- value (F

- statistic) of the B parameter (from summary(nls)) T.pval = P

- value (F

- statistic ) of the T parameter (from summary(nls)) logAC50.pval = P

- value(F

- statistic) of the logAC50 parameter (from summary(nls)) W.pval = P

- value (F

- statistic) of the W parameter (from summary(nls)) logAC50.confint.lower = Lower 95%

- tile confidence interval a round logAC50 parameter (confint(nls[“logAC50”])) logAC50.confint.upper = Upper 95%

- tile confidence interval around logAC50 parameter (confint(nls[“logAC50”])) AC50.confint.lower = Transformed value from logAC50 AC50.confint.upper = Transformed value fr om logAC50 AC50.2se.lwr = AC50 minus two times the transformed logAC50 standard error AC50.2se.upr = AC50 plus two times the transformed logAC50 standard error

B.confint.lower = Lower 95%

- tile confidence interval around B parameter (confint(nls[“B”]))

B .confint.upper = Upper 95%

- tile confidence interval around Bparameter (confint(nls[“B”]))

T.confint.lower = Lower 95%

- tile confidence interval around T parameter (confint(nls[“T”])) T.confint.upper = Upper 95%

- tile confidence interval around T parameter (c onfint(nls[“T”]))

W.confint.lower = Lower 95%

- tile confidence interval around W parameter (confint(nls[“W”])) W.confint.upper = Upper 95%

- tile confidence interval around W parameter (confint(nls[“W”]))6 Min.conc and max.conc = minimum and maximum tested concentration with a usable (not flagged) data point Min.resp and max.resp = minimum and maximum response of all non

- flagged responses (max.resp does not equal Emax as Emax is the max average response at a concentration and no t the single maximal response) Lec.resp.cutoff = the resp value required to determine an LEC and a minimal requirement to be considered a hit Ac50.resp.cutoff = the resp value required to determine an AC50 and a minimal requirement to be considered a hit ( e.g., 10 times the baseline MAD); these values generally range from 15

- 50% and vary primarily due to the background noise level of the assay. Rsq.cutoff = Secondary hit

- calling cutoff which requires the hill model to have an r

- squared value above this cuto ff to determine an AC50 and have an AC50.mod equal to “==”. LEC = Lowest Effective Concentration, First concentration in which the average response at a concentration is above the lec.resp.cutoff.

AC10

- 90 = ACXX <<

- function(T2B,AC50,W,xx){XX <

- xx/100;F <

- XX*(T2B);ACXX <

- ((F/(T2B

- F))^(1/W))*AC50; return(s3(ACXX))}AC20 y50 = Efficacy at the AC50 [(0.5 * T2B)+B] line.x = Hill model line X (concentration) coordinates, Captured to assist in plotting curve if equation is not available line.y = Hill model line Y (efficacy) coordinates, Captured to assist in plotting curve if equation is not available converged = Did the hill model converge? TRUE/FALSE convergence.message = Convergence message from NLS line.y.pi.lwr & upr = Hill model line of the predict ion interval (below and above the hill model) AC50.pi.lwr = Lower prediction interval around AC50 AC50.pi.upr = Upper prediction interval around AC50 RSQUARED_TEST = TRUE or FALSE; Did the hill model meet the R

- Squared criteria (>0.5) T2B_TEST = TRUE o r FALSE; Did the hill model meet the T

- B test based on the response cutoff (ac50.resp.cutoff) Emax_TEST = TRUE or FALSE; Did the max response meet the response cutoff (ac50.resp.cutoff) OVERALL_TEST = TRUE or FALSE; Did the model meet all 3 criteria or a special case (such as the “<>” AC50.mod case) Level6.ac50.hitcall = TRUE or FALSE; the model meet all 3 criteria or a special case (may override special case from TRUE to FALSE)

Level6.lec.hitcall = TRUE or FALSE; did the Emax rise above the lec.resp.cutoff7 Level7.ac50.hitcall = TRUE or FALSE; Based on external data or confounding properties of the assay level6.ac50.hitcall can be changed systematically Level7.lec.hitcall = TRUE or FALSE; Based on e xternal data or confounding properties of the assay level6.lec.hitcall can be changed systematically Level7.ac50.notes = Provides reason for any change Level7.lec.notes = Provides reason for any change Level8.ac50.hitcall = TRUE or FALSE; Based on manu al review, hit calls can be changed; justification required Level8.lec.hitcall = TRUE or FALSE; Based on manual review, hit calls can be changed; justification required Level8.ac50.notes = Provides reason for any change

Level8.lec.notes = Provides reason for any change

ACToR/ToxCastDB Files

The column

- wise ACToR/ToxCastDB files are exported from the ToxCastDB and are especially useful in providing summaries data across chemicals and assays in a variety of custom formats.