Generic Drug Evaluation and R-package 

SABE

Elena Rantou, PhD

GASP, September 23, 2019 

Washington, DC

Elena.Rantou@fda.hhs.gov
2

Disclaimer

o This presentation reflects the views of the 

presenter and should not be construed to 

represent the United States Food and Drug 

Adマiミistratioミげs ┗ie┘s or poliIies

o All data sets shown in this presentation have 

been previously de-identified

www.fda.gov
3

Outline

o Office of Biostatistics/DBVIII

o Office of Generic Drugs/ORS/DQMM

o R-paIkage けSABEげ
4

Office of Biostatistics / DBVIII

o power simulations 

o generate the distribution of certain statistics of 

interest

o assess the similarity of and cluster amino-acid 

sequences 

o determine the validity of data sets categorized for 

genotoxicity 

o characterize outliers in replicated, crossover design

PK studies

o compare bioequivalence assessment approaches

o determine important features for identifying 

clinical sites for inspection
5

Similarity of 

amino-acid 

sequences

Use weighted sampling 

and select sequences 

using their frequencies 

as weights.

Tanimoto Distance

劇 =

軽凋∩喋

軽凋 + 軽喋 − 軽凋∩喋
6

Clustering using aminoacid frequencies

• Sample sequences using either random or

weighted sampling

• For each sequence define mean similarity

score across all other sequences

• For each sequence define the frequency of

each amino-acid, i.e., けAげ, けKげ, けEげ and けYげ

Self-Organizing 

Maps 

(package

けSOMげぶ
7

Genotoxicity data integrity

o Examining data from the Ames test on different 

genotoxic impurities. Such data demonstrated 

suspicious patterns and unusual degree of 

replication

o The objective was to analyze the reported 

positive control data in order to investigate the 

existence, pattern and likelihood of lack of 

variation and assess the probability of the 

occurrence of such outcomes 
8

Genotoxicity data integrity


11

Outliers in replicated crossover PK studies

When formulations are compared with respect to 

their PK-characteristics, there may exist 

o けuミusualげ suHjeIts or 

o けuミusualげ oHser┗atioミs ┘ithiミ a Iertaiミ 

formulation

with extremely high or low bioavailability values

13

Comparison of two BE-assessment 

approaches

Passing Rates

Equivalence Superiority Both

o This is for an abbreviated new drug application for a generic topical cream. A traditional 

approach for establishing BE relies on a clinical endpoint study and uses success 

proportion (where success = at least 2-grade improvement based on 5-point scale of the 

condition severity) as a study endpoint.

o An applicant proposed a new approach based on AUEC/Emax for establishing BE.

o The three graphs above help us comparing the chances of passing 1) equivalence test, 2) 

superiority test and 3) both tests when using the two approaches, when the test and 

reference products are indeed equivalent based on simulation. 
14

Comparison 

of methods 

for clinical 

investigator 

site 

inspection 

selection

Objective is to determine if data mining 

techniques and / or unsupervised 

statistical monitoring can assist with 

the process of identifying potential 

clinical sites for inspection

. Summary of methods used to predict site inspection outcomes. 

Data mining 

 SMARTTM CISST 2016 2017

Description Detects outliers 

using 

distributional 

assumptions 

about the data. 

Expert opinions 

used to develop 

a risk-based 

model. 

Historical data 

used to train 

classification 

models for 

prediction. 

Historical data 

used to train 

classification 

models for 

prediction. 

Predictions Uses p-value to 

identify atypical 

sites. 

Assigns risk 

score to each 

site. 

NAI, VAI, or 

OAI. 

NAI or VAI/OAI 

Journal of biopharmaceutical statistics. 2019 Aug 30:1-4. 
15

Comparison of methods for clinical 

investigator site inspection selection

Journal of biopharmaceutical statistics. 2019 Aug 30:1-4. 
16

Office of Generic Drugs/Office of Research and 

Standards/Division of Quantitative Methods 

and Modeling

o Machine learning (ML) methodology to predict 

Abbreviated New Drug Application (ANDA) 

submissions

o Application of ML for Time-to-Event analysis

o Equivalence Testing of Complex Particle Size 

Distribution Profiles
17

Predictive analysis of first ANDA 

submission for new chemical entities 

based on machine learning methodology

o Random Survival Forest (RSF) ML method is employed to forecast the time to first ANDA 

submission, referencing a new chemical entities (NCE) drug product

o RSF is superior in predictive performance comparing to conventional time-to-event methodology 

o Variable importance of predictors (e.g., drug product, regulatory and pharmacoeconomic 

information variables) is assessed

Clin Pharmacol Ther. 2019 Jul;106(1):174-181. doi: 10.1002/cpt.1479. 
18

Big data toolsets to pharmacometrics: 

Application of machine learning for time-toevent analysis

o Big Data tools (machine learning, ML) are applied to address pharmacometric problems

o The predictive performance of ML methods is superior compared to the Cox regression model 

under various simulated scenarios 

o ML methods demonstrate less sensitivity to data sizes and censoring rates 

Clin Transl Sci. 2018 May;11(3):305-311. doi: 10.1111/cts.12541.
19

Equivalence testing of complex particle size 

distriHutioミ profiles Hased oミ Earth Mo┗erげs 

Distance

o EMD approach is employed to compare complex PSD profiles for equivalence 

assessment 

o The developed approach is both effective and sensitive to pass equivalent products 

and reject inequivalent products in cases of multimodal PSD

AAPS J. 2018 Apr 12;20(3):62. doi: 10.1208/s12248-018-0212-y.
20

Bioequivalence assessment for topical 

dermatological products and the In-Vitro 

Permeation Test (IVPT)

Package けSABEげ*

*Scaled Average BioEquivalence
21

IVPT Study Design

Donor 1 Donor 2 Donor 3 Donor 4 Donor 5 Donor n

Test

Reference

0 5 10 15 20 25 30 35 40 45

0

1

2

3

4

5

6

Percent Dose/hr

Time (hr)

 Benzoic acid in Petrolatum

In Vitro Rate of Absorption

In Vivo Rate of Excretion

Source: Bronaugh and Franz (1986) 



25

Based on the this 

criterion, the two 

products are declared 

equivalent if 

BE assessment 

2. The upper 95% bound 

of the scaled confidence 

interval is ≤ 0

1. The point estimate 

(GMR) is contained within 

the limits [

怠

陳

, 兼]
26

R-package 

けSABEげ

o Tests for BE using the mixed 

scaled criterion

o Estimates statistical power as a 

function of the sample size

o Compares statistical power using 

the mixed scaled criterion (SABE) 

vs. that of using regular average 

BE (ABE)

o Estimates statistical power for 

different levels of the BE margin

o Estimates the size of the test 

(alpha-level)
27

R-package 

けSABEげ

o Conducts sensitivity analysis 

with varying the number of 

replicates per donor, as well as, 

the inter-donor and withinreference variability levels

o Balances an unbalanced data set 

using different criteria

o Produces graphical displays that 

demonstrate the variability 

levels and potential extreme 

replicate values (outliers) 
28

R-package 

けSABEげ

IVPT.outcome(DataSet)

Bioequivalence assessment
29

R-package 

けSABEげ

Power analysis

Power with 

respect to PKmetric

Power with 

respect to BE 

assessment 

method
30

R-package 

けSABEげ

alphaTest(PE,matrixT,matrixR,n,r,trialn)

--------------------------------

SABE ABE n 

---------- ---------- ----

0.03128 0.005038 4 

0.03054 0.00245 6 

0.02752 0.001334 8 

0.02387 0.000756 10 

0.02037 0.000432 12 

0.01721 0.00024 14 

0.01346 0.000128 16 

0.01083 9.8e-05 18 

---------------------------------
31

References

Hu, M., Jiang, X., Absar, M., Choi, S., Kozak, D., Shen, M., Weng, Y.T., Zhao, L. and Lionberger, R., 2018. Equivalence 

testiミg of Ioマple┝ partiIle size distriHutioミ profiles Hased oミ earth マo┗erげs distaミIe. The AAPS journal, 20(3), p.62.

Gong, X., Hu, M. and Zhao, L., 2018. Big Data Toolsets to Pharmacometrics: Application of Machine Learning for 

Tiマe‐to‐E┗eミt Aミal┞sis. Clinical and translational science, 11(3), pp.305-311.

FDA, Guidance for Acyclovir cream 5%, 

http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM428195.pdf. 

(recommended December 2014; revised in December 2016)

Hein, N., Rantou, E., & Schuette, P. (2019). Comparing methods for clinical investigator site inspection selection: a 

comparison of site selection methods of investigators in clinical trials. Journal of biopharmaceutical statistics, 1-14.

Hu, M., Babiskin, A., Wittayanukorn, S., Schick, A., Rosenberg, M., Gong, X., Kim, M.J., Zhang, L., Lionberger, R. and 

Zhao, L., 2019. Predictive analysis of first Abbreviated New Drug Application submission for new chemical entities 

based on machine learning methodology. Clinical Pharmacology & Therapeutics.

Stead, A. G., Hasselblad, V., Creason, J. P., & Claxton, L. (1981). Modeling the Ames test. Mutation 

Research/Environmental Mutagenesis and Related Subjects, 85(1), 13-27.

Sellers, K. F., Borle, S., & Shマueli, G. ふヲヰヱヲぶ. The COM‐Poissoミ マodel for Iouミt data: a sur┗e┞ of マethods aミd 

applications. Applied Stochastic Models in Business and Industry, 28(2), 104-116.

https://cran.r-project.org/web/packages/compoisson/compoisson.pdf

Wang, W., & Chow, S. C. (2003). Examining outlying subjects and outlying records in bioequivalence trials. Journal 

of biopharmaceutical statistics, 13(1), 43-56.
32

Acknowledgements

www.fda.gov

Office of 

Biostatistics/OTS

Stella Grosser

Fairouz Makhlouf

Nam Hee Choi

Sungwoo Choi

Paul Schuette

Office of Generic Drugs

Meng Hu

Liang Zhao