Machine Learning Enabled “RASAR” Toxicity
Models Outperform Animal Test Counterparts
UL Cheminformatics Suite

Model Development: Simple RASAR
Jan 2016

Dec 2016

SOT 2016

Simple
ECHA Database

JHU Publications

NLP Collection of ECHA C&L

Skin sens., Eye irrit., Oral models

REACHAcross Launchhttps://ulreachacross.com

Model Development: Simple RASAR
Jan 2016

Dec 2016

Toxic?

SOT 2016

Simple
ECHA Database

JHU Publications

NLP Collection of ECHA C&L

Skin sens., Eye irrit., Oral models

REACHAcross Launchhttps://ulreachacross.com

Model Development: Data Fusion RASAR
Jan 2016

Dec 2016

Toxic?

SOT 2016Simple
ECHA Database

JHU Publications

NLP Collection of ECHA C&L

Skin sens., Eye irrit., Oral models

2017

July 2018

REACHAcross Launchhttps://ulreachacross.com

Fall 2018

Data Fusion
Algorithm UpgradeTox Sci Publication

Data Fusion & Potency“RASAR” Models

UL Cheminformatics

https://psi.ul.com/en/products/cheminformatics/

Model Development: Data Fusion RASAR
Jan 2016

Dec 2016

Toxic?

SOT 2016

Simple
ECHA Database

JHU Publications

NLP Collection of ECHA C&L

Skin sens., Eye irrit., Oral models

REACHAcross Launchhttps://ulreachacross.com

Toxic?
2017

July 2018

Fall 2018

Data Fusion
Algorithm UpgradeTox Sci Publication

Data Fusion & Potency“RASAR” Models

UL Cheminformatics

https://psi.ul.com/en/products/cheminformatics/

Data Sources:
LegacyData Sources:
LegacyProduction

Data Sources:
LegacyProduction

80,908 Chemicals, 74 Properties, 833,844 End-Points

Similarity

Data Source

Similarity

Fingerprinter
010110…

Graph Algorithms

MetricM ( 0 1 0 0 … , 0 0 1 0 ) = -∞ ≤ x ≤ ∞Data Fusion “RASAR” Model

Acute Oral?

TargetTargetAcid

Muta.

Corr.

TTF

Data Fusion “RASAR” Model

Acute Oral?

0.8Target

Pos. Source

Target

Acid

Oral

Acid

Muta.

Corr.

0.8

0.8

TTF

Data Fusion “RASAR” Model

0.8

Pos. Source

Acute Oral?

0.4

Target0.6

Neg. Source

Target

Acid

Oral

Acid

Muta.

Corr.

Oral

Acid

0.8

0.8

TT

F

0.4

0.6

Data Fusion “RASAR” Model
Hazard & Properties
74 (e.g. H225 - flammable liquid)

0.8

Pos. Source

Acute Oral?

0.4

Target0.6

Features
74 x 3 = 222 (target, pos, neg)
Database
ECHA C&L, Pubchem, NTP
Learning
Random Forest, Grad. Boost Trees,
Multilayer Perceptron

Neg. Source

TargetAcid

OralAcid

Muta.

Corr.

Oral

Acid

0.80.8TTF

0.40.6

Data Fusion “RASAR” Model
Hazard & Properties
74 (e.g. H225 - flammable liquid)

0.8

Pos. Source

Acute Oral?

0.4

Target0.6

Features
74 x 3 = 222 (target, pos, neg)
Database
ECHA C&L, Pubchem, NTP
Learning
Random Forest, Grad. Boost Trees,
Multilayer Perceptron

Neg. Source

TargetAcid

OralAcid

Muta.

Corr.

Oral

Acid

0.80.8TTF

0.40.6

Advantages

Familiar ConceptAdvantages

Familiar ConceptMore Data

Advantages

Familiar ConceptMore Data

Endpoint Agnostic

Advantages

Familiar Concept

Endpoint AgnosticMore Data

Multi-Task LearningPerformanceHazard

OECD TG

Acute Oral

401

Reproducibility of OECD
Animal Test Guidelines1
Studies92%

Acute Dermal

402

78%

Skin Irritation

404

75.5%

Eye Irritation

405

83.5%

406

82.5%

429

85.5%

474

74%

475

75%

Skin Sensitization

Mutagenicity

1. Study of REACH Registrations was used to extract multiple guideline studies on
the same chemical. Conditional pairwise probabilities were calculated to derive
accuracy of a repeat experiment.
2. Calculated using leave-one-out cross-validation. Sensitivities constrained at 80%
Specificity range is 50-75%
3. Calculated using Five-fold cross-cross-validation results. All chemicals werepredicted, i.e. coverage is 100%PerformanceHazard

OECD TG

Acute Oral

401

Reproducibility of OECD
Animal Test Guidelines1
Studies92%

Acute Dermal

402

78%

Acute Dermal Binary

73%

77%

90%

Skin Irritation

404

75.5%

Skin Irritation Binary

75%

77%

97%

Eye Irritation

405

83.5%

Eye Irritation Binary

88%

79%

98%

406

82.5%

429

85.5%

Skin Sensitization
Binary

85%

68%

84%

47474%

47575%

Mutagenicity Binary

83%

59%

88%

1. Study of REACH Registrations was used to extract multiple guideline studies onthe same chemical. Conditional pairwise probabilities were calculated to derive
accuracy of a repeat experiment.

Chronic Aquatic
Binary

80%

77%

98%

2. Calculated using leave-one-out cross-validation. Sensitivities constrained at 80%
Specificity range is 50-75%

Acute Aquatic

82%

67%

95%

Acute Inhalation

83%

74%

90%

Skin Sensitization

Mutagenicity

HazardSimple RASAR
SimpleData Fusion
CoverageRASAR BAC2 RASAR BAC3

Acute Oral Binary

87%

77%

93%

PerformanceHazard

OECD TG

Acute Oral

401

Reproducibility of OECD
Animal Test Guidelines1Studies
92%

Acute Dermal

402

78%

Acute Dermal Binary

73%

77%

90%

Skin Irritation

404

75.5%

Skin Irritation Binary

75%

77%

97%

Eye Irritation

405

83.5%

Eye Irritation Binary

88%

79%

98%

406

82.5%

429

85.5%

Skin Sensitization
Binary

85%

68%

84%

474

74%

475

75%

Mutagenicity Binary

83%

59%

88%

Chronic Aquatic
Binary

80%

77%

98%

Acute Aquatic

82%

67%

95%

Acute Inhalation

83%

74%90%

Skin Sensitization

Mutagenicity

1. Study of REACH Registrations was used to extract multiple guideline studies on
the same chemical. Conditional pairwise probabilities were calculated to derive
accuracy of a repeat experiment.
2. Calculated using leave-one-out cross-validation. Sensitivities constrained at 80%
Specificity range is 50-75%
3. Calculated using Five-fold cross-cross-validation results. All chemicals werepredicted, i.e. coverage is 100%

HazardSimple RASAR
SimpleData Fusion
Coverage
RASAR BAC2 RASAR BAC3

Acute Oral Binary

87%

77%

93%Performance
HazardAcute Oral
Acute Dermal
Skin Irritation
Eye Irritation
Skin Sensitization

Reproducibility of OECD
Simple RASAR
SimpleData Fusion
HazardOECD TG
1Animal Test Guidelines
Coverage
RASAR BAC2 RASAR BAC3
Studies Test Performance
40192%
Acute Oral Binary
87%
77%
93%
Animal Test Reproducibility of 74-92%
73%coverage 77%
90%
Acute Dermal
Binary on 100%
402
78%
Data Fusion RASAR
Model balanced accuraciesof 84-98%
77%

97%

79%
EyeBinary405
Toxicological“Six Pack”83.5%
accounted for 55% ofallIrritation
animals usedin toxicity88%testing in EU 201198%

$3BB Euro
spent on animal tests
toxicology each year
82.5%
Skin forSensitization
85%
Binary85.5%

84%404406429

75.5%

Skin Irritation Binary

Cost of Animal Tests

75%

68%

REACH Registration Case Study~12,000
474 and ~13,000 chemicals
74%
were received for Phase 1 & 2 registration, respectively
83%
59%
Mutagenicity
Mutagenicity
ECHA expects 60,000
registrationsBinary
in 2018475
75%
If we had stopped animal studies for REACH after the 2013 deadline (the data we are using)
Chronic Aquatic
77%
Usingthestudies
Data onFusion RASAR would have saved 80%
1. Study of REACH Registrations was used to extract multiple
guideline
Binarythe same chemical. Conditional pairwise probabilities were calculated to derive2.8 million animals
accuracy of a repeat experiment.
$490 Euro inAcutetesting
82%
67%
Aquatic
2. Calculated using leave-one-out cross-validation.
3. Calculated using Five-fold cross-cross-validation results. All chemicals werepredicted, i.e. coverage is 100%

Acute Inhalation

83%

74%

88%
98%
95%
90%

Performance and Impact
HazardAcute Oral
Acute DermalSkin Irritation
Eye Irritation
Skin Sensitization

Reproducibility of OECD
Simple RASAR
SimpleData Fusion
HazardOECD TG
1
Animal Test Guidelines
Coverage
RASAR BAC2 RASAR BAC3
Studies Test Performance
40192%
Acute Oral Binary
87%
77%
93%
Animal Test Reproducibility of 74-92%
73%coverage 77%
90%
Acute Dermal
Binary on 100%
402
78%
Data Fusion RASAR
Model balanced accuraciesof 84-98%
77%

97%

79%
EyeBinary405
Toxicological“Six Pack”83.5%
accounted for 55% ofallIrritation
animals usedin toxicity88%
testing in EU 201198%

$3BB Euro
spent on animal tests
toxicology each year
82.5%
Skin forSensitization
85%
Binary85.5%

84%404406429

75.5%

Skin Irritation Binary

Cost of Animal Tests

75%

68%

REACH Registration Case Study~12,000
474 and ~13,000 chemicals
74%
were received for Phase 1 & 2 registration, respectively
83%
59%
Mutagenicity
Mutagenicity
ECHA expects 60,000
registrationsBinary
in 2018475
75%
If we had stopped animal studies for REACH after the 2013 deadline (the data we are using)
Chronic Aquatic
77%
Usingthestudies
Data onFusion RASAR would have saved 80%
1. Study of REACH Registrations was used to extract multiple
guideline
Binarythe same chemical. Conditional pairwise probabilities were calculated to derive2.8 million animals
accuracy of a repeat experiment.
$490 Euro inAcutetesting
82%
67%
Aquatic
2. Calculated using leave-one-out cross-validation.
3. Calculated using Five-fold cross-cross-validation results. All chemicals werepredicted, i.e. coverage is 100%

Acute Inhalation

83%

74%

88%
98%
95%
90%

Performance and Impact
HazardAcute Oral
Acute Dermal
Skin Irritation
Eye Irritation
Skin Sensitization

Reproducibility of OECD
Simple RASAR
SimpleData Fusion
HazardOECD TG
1
Animal Test Guidelines
Coverage
RASAR BAC2 RASAR BAC3
Studies Test Performance
401
92%
Acute Oral Binary
87%
77%
93%
Animal Test Reproducibility of 74-92%
73%coverage 77%
90%
Acute Dermal
Binary on 100%
402
78%
Data Fusion RASAR
Model balanced accuraciesof 84-98%
77%

97%

79%
Eye
Binary405
Toxicological“Six Pack”83.5%
accounted for 55% of
allIrritation
animals used
in toxicity88%
testing in EU 201198%

$3BB Euro
spent on animal tests
toxicology each year82.5%
Skin forSensitization
85%
Binary
85.5%

84%

404

406
429

75.5%

Skin Irritation Binary

Cost of Animal Tests

75%

68%

REACH Registration Case Study~12,000
474 and ~13,000 chemicals
74%
were received for Phase 1 & 2 registration, respectively
83%
59%
Mutagenicity
Mutagenicity
ECHA expects 60,000
registrationsBinary
in 2018475
75%
If we had stopped animal studies for REACH after the 2013 deadline (the data we are using)
Chronic Aquatic
77%
Using
thestudies
Data onFusion RASAR would have saved 80%
1. Study of REACH Registrations was used to extract multiple
guideline
Binarythe same chemical. Conditional pairwise probabilities were calculated to derive2.8 million animals
accuracy of a repeat experiment.
$490 Euro inAcutetesting
82%
67%
Aquatic
2. Calculated using leave-one-out cross-validation.
3. Calculated using Five-fold cross-cross-validation results. All chemicals werepredicted, i.e. coverage is 100%

Acute Inhalation

83%

74%

88%
98%
95%
90%

Acknowledgements

Acknowledgements