FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 1

FDA Virtual Town Hall Series –

Immediately in Effect Guidance on 

Coronavirus (COVID-19) Diagnostic Tests

Moderator: Irene Aihie

October 21, 2020

12:15 pm ET

Coordinator: Good afternoon and thank you all for standing by. For the duration of today's 

conference all participants' lines are on a listen-only mode until the questionand-answer session. At that time, if you would like to ask a question press 

star 1. Today's call is being recorded. If you have any objections you may

disconnect at this time. It is my pleasure to introduce Ms. Irene Aihie. Thank 

you ma'am, you may begin. 

Irene Aihie: Thank you. Hello. I'm Irene Aihie of CDRH's Office of Communication and 

Education. Welcome to the FDA's 31st in a series of virtual town hall 

meetings to help answer technical questions about the development and 

validation of tests for SARS-CoV-2 during the public health emergency. 

Today Timothy Stenzel, Director of the Office of In Vitro Diagnostics and 

Radiological Health in the Office of Product Evaluation and Quality, and 

Toby Lowe, Associate Director of the Office of In Vitro Diagnostics and 

Radiological Health, both from CDRH, will provide a brief update. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 2

Following opening remarks, we will open the lines for your questions related 

to today's discussion. Please remember that we are not able to respond to 

questions about specific submissions that might be under review. Now I give 

you Timothy. 

Dr. Timothy Stenzel: Welcome everyone. A pleasure to be on the call again with all of you. So I 

appreciate all of your creative minds and dedication to addressing unmet 

COVID pandemic needs. We look forward to working with you. 

You may start to realize that you have additional folks who you've not met 

before. We've been hiring and we've been shifting some resources to try to 

work through all of the volumes of submissions as quickly as possible. We've 

been doing that since day one and we continue to do that. So hopefully that 

has a net positive influence on all of us. 

Just to start off, I wanted to talk about something that's very important to all of 

us as test developers and those involved with validation and authorization and 

use of these tests in this pandemic. It also applies to non-pandemic situations. 

I do want to help enlist your support and help in educating all of the various 

stakeholders who utilize these tests in different situations. 

So I wanted to talk about false positives again. We continue to hear about 

challenges relating to getting false positive results. And I would add, even in 

situations where it's totally expected that might seem odd. So first of all, there 

is no - I'm not aware of any perfect test. And I want to go back to my example

of using a test, whether it be an antigen test or a molecular test. 

Neither - I don't know a single one that's perfect, that is 100% specific that is 

every time a patient is truly negative the test is negative. It just doesn't exist 

to my knowledge. So if we just assume for a second just for the math, that 99 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 3

- the specificity of any of these tests is 99%. It is 99 times out of 100 a patient 

who is negative will test negative. And only one time out of a 100 will a 

patient who is negative test positive. 

I think that's a very high specificity for almost any category of test. The 

challenge of course is if we begin to test in large scale screening, whether it's 

antigen or molecular, and we're screening populations that in particular, have 

a low percent positive rate, if that percent positive rate falls below the 1% 

false positive rate, it means that even for a perfectly well-functioning test it's a 

well-designed test, it's a well-validated test and the users who are performing 

the test are following the instruction, this, you know, according to the package 

insert, they are going to get a 1% false positive rate if the test is 99% specific. 

And it really doesn't matter if the patient is symptomatic or not. If the 

asymptomatic it applies the same. And it doesn't even really matter if a test 

has achieved a claim for screening asymptomatics. Specificity - is specificity 

typically going to be the same in all of these different populations for a given 

test format, a given sample type? 

And so the challenge is what do you do when the false positives which are 

totally expected, outnumber the true positives? I think it's important to say 

this is a totally expected situation and we plan for it. And we plan for it by -

in those situations where the prevalence of positive disease, where the percent 

positive is low, significantly less or even in a, you know, even if it's a 1% 

positive rate you're going to have - half your positives are going to be false 

positives and half your positives are going to be true positives. 

So unless your percent positive rate is pretty high the idea that you might have 

as false positive is going to - if it's going to impact somebody's livelihood that 

is they may be asked to quarantine themselves for 14 plus days, or it involves 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 4

putting a patient into a risky situation or healthcare workers aren't alerted to a 

patient having this or it determines outcome. You know, if they have flu 

they're going to treat somebody this way; if they have SARS they're going to 

treat them another. So all those situations matter. 

And you know your population is such that the false positive rate is not 

insignificant. And it's important and I would recommend that you get some 

sort of orthogonal confirmatory test done. I would prefer it be a molecular 

test of high specificity and sensitivity. You know, there is the option 

potentially of using another direct antigen test if your primary test is a direct 

antigen test. 

What I don't think we have a good handle on yet is whatever causes a false 

positive for one test, in particular a test that doesn't have an extraction of 

nucleic acid step up front, you know, if there's something in that sample that 

lends itself to a false positive result in a given individual, if you just fully go 

to another different orthogonal or direct antigen test, I just don't know at the 

moment and maybe somebody does, if you do please reach out to us through 

our Templates email address, you know, if that second direct antigen test will 

give you a true result based on the sample having certain potential 

characteristics that may lend itself in a direct antigen test to giving you a false 

positive. So I don't know. I think - I personally lack the information to give a 

recommendation on that. I would be very interested in seeing data about two 

orthogonal direct antigen tests and when you have a false positive, you know, 

can you use that orthogonal test to determine that? 

The other thing is if there is any sort of loss of sensitivity with a direct antigen 

second test, a high sensitive - a highly, highly specific molecular test is more 

likely to give you an accurate confirmation of whether you have disease or 

not. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 5

So that's the introductory remarks for today. Thanks again for joining us. 

And I look forward to your questions. Thank you so much. 

Irene Aihie: Operator, we'll now take questions. 

Coordinator: Thank you. If you would like to ask a question please make sure your phone 

is not muted, please press 1, star 1 and when prompted, clearly record your 

first and last name so I may introduce you. Again, to ask a question press star 

1. Our first question is from (Alberto Gambini). Sir, you may go ahead. 

(Alberto Gambini): Hello. Hi. Can you hear me? 

Dr. Timothy Stenzel: Yes. 

(Alberto Gambini): Yes. So my question is related to the - once the staff, the NIH staff ensure 

the FDA - to ensure that the validation done by the NIH is done according to 

the instructions of the packages of the test being tested. We know in at least 

one case the NIH has not followed instruction as far as for example, using 

sample matrices not intended to be used in this assay for example, using 

different test plasma coagulated with a different anti-coagulant than the one 

the sponsor actually providing and validated. 

It seems to be to me a pretty curious evaluation of the instruction of use. And 

also to now follow in some cases, at least one case, you know, do not use the 

proper specific controls to ensure that the operator knows how to use the test. 

Some of these tests may be new technology and so the operator would have to 

sort of train on it. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 6

And I’m wondering this can have a serious impact because you conducted

basically, you know, validating incorrectly either in a good way, a bad way or 

whatever you like to see it as an assay. So I would like to know what kind of 

steps you may take and if there's any concern about the NIH's mishandling of 

the assay during the validation. 

Is there any possible way for the sponsors to discuss that either with the NIH 

or with the FDA or all three together? Thank you. 

Dr. Timothy Stenzel: Yes. Well first of all, the NCI is doing a fantastic job and such an 

important job. And we have processes and procedures in place to try to 

ensure that in all cases package inserts are followed and that the correct 

sample types are used. 

We don't have the luxury of getting a wide variety of panels for each different 

kind of sample type as far as serum or plasma and then as far as the plasma for 

the anticoagulant that's used. And so we are somewhat beholden to the panel 

as it gets formulated and therefore, we do want to look to those situations 

where, you know, a non-ideal sample type was used. 

We typically can look to (ADAS), look at as subset of the result that match 

and line up with the IFU for the product. If that is missed we - when we get 

the results we do reach out to sponsors with their NCI results. And that is an 

opportunity for sponsors to engage in a conversation with the FDA to address 

those specific questions. And a few times it has come to my attention of this 

and we've done our best to address those concerns. 

So that's an opportunity for sponsors when we give you that feedback, on the 

results. The panel does use serum and plasma and it does -for plasma it does 

use, you know, the common anticoagulants, you know, in general. So, you 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 7

know, and there may be some tests that have limitations on serum versus 

plasma and vice versa and for plasma, the type of anticoagulation. 

So we appreciate working with sponsors to help us if we overlook something 

either at the NCI or at the FDA. And we'll do our best to properly address that 

situation. So if you feel that's not the case you can always reach out to Toby 

and me through the Templates email address for a specific discussion and a 

specific situation. 

(Alberto Gambini): Thank you.

Coordinator: And our next question is from (Louis Perlmutter). Sir, you may go ahead. 

(Louis Perlmutter): Yes. Thank you, Tim and Toby. The question really concerns whether 

there are any approvals for the recent - for recently submitted rapid antigen 

immunoassay tests for the SARS-CoV-2. 

I'm not thinking of the BinaxNOW test by Abbott, but any others. Are there 

any in the works? 

Dr. Timothy Stenzel: So we have posted all of those that we've authorized. The IVD EUA 

authorized at the FDA. I believe there are six and I believe Quidel came in 

subsequent to the BinaxNOW test with a Flu A+B SARS test. So you can 

always see what's within, you know, we post that within, you know, an hour 

to 24 hours or, you know, and business days after authorization. If it happens 

over the weekend then it usually happens on - it may happen not until on 

Monday. 

And there are a number of developers working on additional rapid antigen 

tests. And I can't go into any more specifics than that, but...
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 8

(Louis Perlmutter): Yes. Okay. I was aware of those. There are a few submissions. Yes. 

Okay. Thanks very much. 

Coordinator: And our next question is from Alexis Sauer-Budge. You may go ahead. 

Alexis Sauer-Budge: Hi. This is Alexis Sauer-Budge from Exponent. I was hoping that you 

could give us a little bit more detail with regard to the announcement about 

the FDA declining to review LDTs. You've also stated that it's a high priority 

to review high throughput tests. So if a CLIA certified lab is developing an 

ultra-high throughput molecular test would you still review the submission? 

Dr. Timothy Stenzel: So we are currently declining to review LDTs and for the reasons that we 

stated on our FAQ page. And that is largely because they - the FDA cannot 

require an EUA and - according to the HHS statement. As far as high 

throughput goes those are for those tests that then we choose to review due to 

the, you know, unprecedented number of applications. 

We want to focus our resources on those that do require an EUA. And so a 

single lab that's developed their own test and it happens to be high throughput, 

you know, doesn't - if that's, you know, sort of the specifications of that its' 

our understanding it doesn't require an EUA. And so we are focusing on high 

throughput situations now. 

Some of that's, you know, going to be, you know, kits that have incredible 

amounts of automation where you load samples and step away and they have 

a high throughput. So we talk about high throughput, you know, being a high 

priority. It's that sort of situation where in particular, distributed kits. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 9

So if, you know, the kit is high throughput, the instrument is high throughput 

and it's able to be offered in a lot of different labs, that's going to have a huge 

positive public health benefit and that's where our attention is going to be 

focused among the other high priority items and those items that do require an 

EUA prior to launch. 

Toby, do you have any other comments? 

Toby Lowe: No. I think exactly what you said. We're focused on distributed tests and that 

includes high throughput distributed...

Alexis Sauer-Budge: So...

Toby Lowe:

Alexis Sauer-Budge: Thank you for that. If I could just have a follow up then. If the intent 

though was to provide the protocols to other laboratories a - so, you know, 

buy these particular pieces of equipment and use this protocol sort of similar 

to the Yale SalivaDirect protocol, is that an avenue that would make it not an 

LDT and the FDA would review it? 

Toby Lowe: So without all of the information for a specific scenario it's difficult to give an 

absolute answer. But I can tell you that the SalivaDirect test that you're 

referring to is considered to be a distributed test. 

Dr. Timothy Stenzel: And not an LDT. 

Alexis Sauer-Budge: Great. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 10

Dr. Timothy Stenzel: In fact I don't believe that Yale actually has a lab that - the developers 

don't have a lab that they're doing clinical testing in. So they are basically a 

kit that has the instructions and then everything needed for the kit is required 

but not provided and that's the way we view that. 

Alexis Sauer-Budge: Great. Thank you so much. And thank you for all the hard work that 

you're doing to help us with this pandemic. 

Dr. Timothy Stenzel: You're most welcome. 

Toby Lowe: Thank you. 

Dr. Timothy Stenzel: Thanks for those kind words. 

Coordinator: And our next question is from (Christopher Benson). Sir, you may go ahead. 

(Christopher Benson):Hi Tim. I'm calling again to update you since we last spoke about this, 

about two weeks ago, about shipping the SARS-CoV-2 FDA panel overseas. 

We've made substantial progress. I work with Dr. (Mayra Garcia) at your 

office. But we did receive a request from the foreign country for a certificate 

of inactivation from FDA for the panel. I thought this was a reasonable 

request. Do you know if you have that kind of certificate? 

Dr. Timothy Stenzel: Toby may know. You know, this - we have processes and procedures 

within the FDA. We received live virus, we grew it up with - on the FDA 

campus in a BSL-3 facility. We followed a protocol, an institutional protocol 

that was reviewed by the FDA to make sure that it was safe for our employees 

and I think we did the same for the inactivation. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 11

What I don't know is - and so we have complete confidence that it is safe to 

distribute as a BSL-2 level panel. But what I don't know is if we have the 

ability to provide such a certificate. And there may be an opportunity to write 

a letter rather than say provide a certificate. 

So - and I’m not sure if you've had that conversation with (Mayra). It's not 

coming to mind. I meet with (Mayra) a couple of times a week if not more 

frequently. Toby, do you know anything more and...

Toby Lowe: I don't know whether or not we have that certificate. But we can - (Mayra) 

would definitely know. She's just not online at the moment because she's in 

the lab I believe. But we can definitely find that out. 

(Christopher Benson):All right, thank you. Yes. I wrote to (Mayra) and...

Toby Lowe: Okay. 

(Christopher Benson):...most of that's - so maybe a letter would be sufficient. The other thing 

they recommended is a limit of 10 kilos for the dry ice shipment which I'm 

sure is due to international flights. But I'll follow up with (Mayra) on that. 

Thank you Tim and Toby. 

Dr. Timothy Stenzel: Well that's a, you know, that's not a lot of dry ice especially if you're 

shipping international, which I - fortunately or unfortunately substantial 

amounts of experience with, having been a Chief Operating Officer who has 

shipped internationally in my previous role. 

So the other thing is that we have worked with some sponsors who have found 

a facility or lab that can work with - in the US, to do the testings for them. 

And, you know, where there were challenges in shipping internationally. So 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 12

that is a potential avenue as well. It's not, you know, required or even 

recommended. It's just a statement of fact. 

(Christopher Benson):Okay. I appreciate that. 

Dr. Timothy Stenzel: Thank you. 

(Christopher Benson):Thank you. 

Coordinator: And our next question is from (Elliott Millinton). You may go ahead, sir. 

(Elliott Millinton):I had a couple of questions. One is, is there a panel for an antigen test 

validation for a rapid antigen test? 

Dr. Timothy Stenzel: Not yet to my knowledge. And there have been fairly detailed discussions 

about what we can do in this area. One of the concerns relates to the prior 

conversation about the fact that this is a BSL-3 level virus and we - most folks 

don't have a BSL. Most developers in labs don't have a BSL-3 facility so that 

they can safely handle it. There are some, but most don't. 

And therefore, I'm just going to tell you that transparent issues that we're 

facing. So you can go ahead and inactivate that virus in a number of different 

ways. And then you want to make sure it actually still functions at the antigen 

level. And it's going to be challenging to, you know, provide that kind of 

assurance that it's going to behave just like Wild-Type, live virus. 

But we are looking into it or, you know, our methods of inactivation to make 

it safe. We are looking into how well it performs. We're also looking at 

potentially the opportunity to express antigen and make protein. And then 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 13

obviously you get away from the need for inactivation because it's not whole 

virus and should be very nontoxic. 

So we're working on it and I'll say that we're working on it within the US 

government and there may be others outside the government who are working 

on it. We're just not ready to move forward but we certainly are very 

interested. So thank you for your question. 

(Elliott Millinton):Thank you. And when you say you'd have to make sure the antigen still 

functions, first of all, do you have any preference for heat inactivated versus 

radiation inactivated? And how would you recommend determining whether 

it's still functioning? 

Dr. Timothy Stenzel: Yes. So I don't know that I have a preference there as far as I think, you 

know, the way to go about this is to try a potentially different method and 

conditions and see if it - see how robust that is. And then see how it impacts, 

you know, at least the EUA authorized antigen test. 

But again, they're not cookie cutter. The direct antigen tests are not cookie 

cutter. And if they are using antibodies that are different those antibodies then 

in most cases, are going to see a different epitope. And that could be very -

different epitopes can be variably impacted by different methods of 

inactivation. 

Likewise if you began expecting proteins, if it doesn't - if the proteins aren't 

processed and get the non-nucleic acid directed modifications in the same way 

as in human cells then there could be alterations in performance. 

So they're just very clearly additional challenge when you try to develop a 

direct antigen reference panel as compared to a reference panel for molecular 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 14

that we can basically inactivate, know that we inactivate, and know that if the 

RNA is there and stable that we'll have uniform good results. 

(Elliott Millinton):Thank you. And one more question - for a - for validating an antigen test with 

saliva, can it be compared to a PCR test with saliva and do you have a 

preference as to whether it's a saliva sample for PCR or a nasal swab sample? 

Dr. Timothy Stenzel: I think that's a great question. I think it's important to - whenever you're 

doing kind of comparative tests, so just to make sure with the FDA staff in our 

office that it's an okay comparator. I would say that if it's an EUA authorized 

high sensitivity molecular method there’s good chances that you could use 

saliva as the comparator, molecular saliva for the antigen comparison. But I 

would say it can depend and just double check with our team. 

Toby Lowe: I don't have it up in front of me right now, but I believe the antigen template 

discusses appropriate comparators and specifically mentions no saliva. So if 

you are particularly interested in using saliva as a comparator I would 

recommend reaching out to the antigen team prior to beginning that study, to 

discuss that. 

(Elliott Millinton):Great. Thank you. 

Dr. Timothy Stenzel: And that would be to an alteration to what's recommended. And, you 

know, over time our thinking can change and we try to remain open. Now 

that we have a lot more saliva experience the team's thoughts may have 

changed. I'm not going to speak for them though. 

(Elliott Millinton):Thank you very much. 

Coordinator: Our next question is from Edward Strong. Go ahead sir. You may go ahead. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 15

Edward Strong: Hi there. Thank you. Yes. Edward Strong from iAssay in San Diego. So we 

have a handheld portable reader instrument that reads lateral flow devices and 

other formats of rapid tests. You know, iAssay's model is we make that 

instrument and then we work with part and developers of other testing 

devices. 

I think you probably remember, on last week's call you indicated that there 

may soon be some guidance on instruments, especially now that there's 

recognition that instruments like the reader, would be beneficial to improving 

or preserving the sensitivity and reporting in offsite testing. I was just 

wondering if there was any update that you might have to provide on that 

guidance on that front? 

Dr. Timothy Stenzel: Yes. So Toby wasn't here last week. I terribly missed her. And she may 

know the latest. It was my desire to have some additional recommendations -

oh, two weeks ago Toby was here - wasn't here. And I believe that we may -

we made some adjustments to the appropriate templates but those have not 

been posted yet. And I don't think we can say when it can be posted. 

But I do believe that guidance can be given and this would - if I remember 

correctly, this was for a serology test. But potentially the same could be 

applied to direct antigen test. But I think it was around, and my input was 

around what to do for serology tests. 

So I believe an email to the FDA staff can provide some recommendations 

based on our current thinking until that guidance is posted. And we're 

obviously - Toby, do you have anything else to add? 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 16

Toby Lowe: Yes. I think we've had some emails with you as well on this topic. I think that, 

you know, the main point is going to be that we need to see evidence that the 

reader works with a specific test. And so that's what we're going to be most 

interested in. We do have some experience with readers outside of the public 

health emergency where the readers have had a - potentially can have a 

negative impact on a test performance. 

So we do want to see those paired together. And there, you know, we would 

be happy to work with you on the best pathway to do that, whether it's 

working directly with the EUA holder for the test, where they could come in 

with an amendment to, or a supplement rather, to their EUA. 

We can, you know, we're still having some discussions internally about 

different options there. But that's something that we can work with you on. 

But the biggest point is that we really do need to see the data with the test and 

the reader paired together. 

Edward Strong: Okay. Yes. If I could ask a follow up question. That's particularly...

Dr. Timothy Stenzel: Yes, go ahead. 

Edward Strong: ...the area that we were thinking about, you know, obviously working with 

some developers with their EUA submission. But yes, we were looking at 

iAssay for guidance on if we could supply data that we preserve the 

performance of these tests but do it independently of the developers because 

their model's very much like an independent open platform for our reader to 

be functional in many different serology or antigen tests. 

So it sounds like you're having those discussions internally. And maybe we'll 

reach out again and get some guidance on that direction. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 17

Dr. Timothy Stenzel: Yes. And so Toby was talking about accuracy determination. And I’m - I 

was talking more - that's important as well. But I was thinking a lot more 

towards the flexibility and regulatory pathways for some, for reader 

developers and, you know, what might be required of the reader developer 

that could be independent - or recommended reader developers that could be 

independent from the lateral flow test itself. 

In the end, you know, just to be transparent and clear, it would - and unless 

you were to repeat all of the validation studies that the original manufacturer 

did for their EUA authorization, get your own EUA authorization, you're 

going to want to rely on the data that's already been accumulated and EUA 

authorization, if it's authorized, for that test. 

And that will require some sort of letter from the test developer to the FDA

saying that they give permission for a particular instrument developer or 

reader developer, to use that data in their submission to have a reader that can 

specify that it works with that particular lateral flow. 

So that was a lot of words, but basically in the end, even if you do things 

independently and I suggested edits, that would allow some independent so 

that it makes it as easy as possible to add readers, it still requires a 

collaboration on some level with the company that developed the lateral flow

device. 

Edward Strong: Of course. This is perfect. Very helpful. And I echo what everybody else has 

said. Thank you, guys for the sleepless nights and you all deserve a vacation. 

Thank you. 

Dr. Timothy Stenzel: Thanks so much. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 18

Toby Lowe: Thanks. 

Coordinator: And our next question is from (Kotamudi Venkep). Go ahead, sir. Your line 

is open. 

(Kotamudi Venkep): Hi. Good afternoon. Can you hear me? 

Dr. Timothy Stenzel: Yes. 

(Kotamudi Venkep): Yes. Thank you for taking the question and thank you for all the updates. 

My question is related to the initial remarks that you mentioned about the 

false positives. If you are giving an example of 99% specificity, one out of 

ten is a false positive. And at the end, you know you were mentioning about 

the - if an antigen test is there you can do orthogonal test with the secondary 

test. 

Is it only for the antigen test or even for molecular test? So any positive test if 

there is a - it can be potentially false positive that needed to be confirmed with 

another test? I did not get what you were mentioning clearly. Can you please 

elaborate and clarify that now please? 

Dr. Timothy Stenzel: Yes. Yes, no, it - this also applies to molecular tests. They're not perfect 

either. Developers do their best, you know, and users of those tests do their 

best. And, you know, in my conversations with those who are doing pure 

surveillance which for this pandemic, is not something under the purview of 

the FDA but yet we still have a lot of conversations, this is typically done on a 

lot of different educational campuses around the US. They oftentimes use 

molecular means. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 19

And in the ones that I've spoken to specifically about this, they do have false 

positives and they do - first of all, if they have a positive, you know, they want 

to move it into the CLIA lab to be able to - in some situations at least, to 

provide a CLIA result rather than an unregulated surveillance result which, 

you know, is cautioned against obviously. 

But they have found that they do want to do their confirmatory testing. So and 

particularly as I said, in a low percent positive population, you know, and I've 

talked to some who have rates of detection of positives that are well below 

.1%. 

So when you start to get into a positivity rate, you know, well below not even 

1%, below .1%, you are absolutely going to get into territory with even 

extremely high specificity tests you're going to have some false positives. So 

you do want to do some sort of confirmation. Hopefully that's helpful. 

(Kotamudi Venkep): So it is only, you know, when if it is a - you are 99% specificity. So only 

if you are testing a population where it is less than 1% then this approach is 

useful. It is not for other diagnostic tests even if it is...

Dr. Timothy Stenzel: I think it's something to - that every clinician or healthcare worker that 

interprets positive and negative results of testing, no matter what kind of 

testing it is, needs to keep in mind. It's a very important thing to keep in 

mind. The - it's one result and it could be a false positive, it could be a false 

negative. 

What we do at the FDA is make sure that these tests perform at a certain level 

and we make public that performance information so that the users of these 

tests whether it be labs or it be clinicians, can make an informed decision 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 20

about what does a positive result mean in this patient, in the population that 

I'm testing? 

You know, is the patient symptomatic or are they asymptomatic? You know, 

have they been exposed or not exposed? So this is a very complex science to 

properly interpret the results of tests. What I was really urging folks to do is 

not to assume in low incident populations, that a positive is a true positive, 

just is a chance of false positives. 

So as your percent positive rate increases, say it increases - I mean there are 

some states I'm saying wow, they're over 30%. All right. Let's just say 20%. 

Let's say percent positive in the population is 20% and your false positive rate 

is 1%. It means that you're going to have 21% positive rate, 1% of that is a 

false positive, the rest of it is true positive. 

So it means that - or it was just say overall, the measured rate is 20 to make 

the math easier. So that, you know, 19% true positive rate, 1% false positive 

rate, your percent positive overall including the false positives is 20%. One 

out of 20 is a false positive or 5% is a false positive. 

But you just, you know, if you're a clinician interpreting results or you're a 

laboratory interpreting results, you just want to know that. What are the 

chances that this individual result based on the performance of the test I'm 

using that it's a false positive versus a true positive? 

And so you never get rid of the false positive issue because no test is perfect. 

You never get rid of the false negative problem no matter what the incidence 

is because every - no test is perfect. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 21

And so I'm sorry I can't give you a really cookie cutter applies in all situations 

answer. It's just important that if it really matters for the result, you know, it 

might be good to get a confirmatory test. 

(Kotamudi Venkep): So just like how you recommended the serology test you have certain 

disclaimers. So do you mean these molecular tests should then also have to 

come up with these situations or the disclaimers? 

Dr. Timothy Stenzel: We are looking at - Toby's working on an FAQ update as we speak. In 

fact I think she has sent me language to review and I haven't done it yet. You 

sent me language yesterday and I forget when. So we're going to try to further 

explain this. 

And we may look at additional information in the intended use statement, 

especially when let's say asymptomatic claims are made about a test. So it 

really is situational. It depends. It depends. So what we saw is that - recently, 

is nursing homes - we're seeing a lot of positives. When we went and tried to 

confirm it roughly they were only able to confirm about half of those 

positives. 

And they were shocked. They were frightened that the test wasn't performing. 

What I’m saying is the test performed. To my knowledge in those situation, 

the knowledge - the situation that I'm aware of I don't know that the test - I 

believe the test was performing as intended. It's just that this math says that, 

in a low percent population you're going to have false positives. 

So don't - expect false positives in that situation. And therefore, if it's really 

important to know whether that patient is positive or negative get a 

confirmatory test. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 22

(Kotamudi Venkep): Thank you very much for your...

Dr. Timothy Stenzel: I hope that's helpful. 

(Kotamudi Venkep): ...explanation. I appreciate it. 

Dr. Timothy Stenzel: And - you're welcome. And the reason I enlist this audience of developers 

is that it can negatively impact what we do. If assumptions are made based on 

performance that's entirely expected that there's actually a problem. There 

isn't a problem with the test. 

And it's - if someone views there's a problem with a specific test or a class of 

tests like direct antigen tests when there isn't, just became they have false 

positives, then it has potential for the public to not be supportive of the testing 

schemes that we want to roll out in order to help patients, especially those 

who are most vulnerable. 

We all read stories of how SARS can go around a nursing home in no time 

flat. And everybody's infected including some of the staff. And that's what 

we're trying to prevent with some of the schemes that we're using. And not 

just us, but others. All right. 

(Kotamudi Venkep): Thank you. 

Dr. Timothy Stenzel: Next question. 

Coordinator: And our next question is from (Dana Hummel). You may go ahead. 

(Dana Hummel): Hi. My question is regarding the sale of COVID diagnostic tests to 

researchers for research use only. And I wanted to ask specifically, if there's 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 23

any restriction by the FDA to sell a non-EUA kit to a research lab for research 

only. I understand the FDA recommends the use of EUA kits at researchers 

but I'm wondering if researchers are allowed to purchase non-EUA kits for 

their research. 

Dr. Timothy Stenzel: Toby, do you want to handle that question? 

Toby Lowe: Yes, sure. So if it is truly a research lab doing research, then absolutely they 

can use research kits, research tests, research components. What we don't 

want to see is kits that are labeled for RUO being sold to a clinical laboratory 

for clinical use. 

And we have discussed previously the guidance document that we have that 

talks about appropriate labeling for RUO and that is a really good resource to 

take a look at how we've laid out what is appropriate for an RUO labeled 

device versus a device that should be IVD labeled. 

(Dana Hummel): Perfect. Thank you so much. 

Toby Lowe: Sure. 

Irene Aihie: We will take our next question. 

Coordinator: And our next question is from (Win Lee Ju). Your line is open. 

(Win Lee Ju): Hello. Thank you for taking the call. So I just have a question continue the 

discussion on the molecular test false positives. So right now it's very - I 

think it's very tricky here to determine the false positive because we don't 

really have a good reference panel. And what we do is just compare two 

molecular tests with each other. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 24

So I could see one false positive or another one is just a false negative, you 

know? It's always hard to see which one is a true false positive or false 

negative. So what I found is that with a higher - this could be a problem with 

a very high CT value. 

And so with that being said, the false negative can I say that false positive 

could just have more - could happen at a higher CT more and then just lower 

CT? 

Dr. Timothy Stenzel: Yes. I think that may be dependent on the test and the platform and in 

particular, the (unintelligible) chemistry because if you have, you know, a 

baseline that say is set such to sort of maximize sensitivity that baseline can be 

inadvertently in a small, in a well-performing test a small number of cases, 

you know, that the baseline can be crossed earlier on too and just the case 

based on the detection of the technology. 

So really my statement has to do with when you are - when you know you 

have a population that has a low percent positive, you know, and it's critical to 

properly - to know whether that patient is truly positive or not, in the - and the 

chance of a false positive is significant even with the test performing as 

intended, that's when I recommend that you take a look at confirming. 

This is not a confirming all false negatives, potential false negatives and all 

potential false positives. That would be not workable. And but when you are 

interpreting results of any test yes, the clinician wants to take that into the 

context of the situation. So if you think this test is falsely negative I know 

that some of them are ordering a second test. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 25

Or if they think it's falsely positive I know that some of them are checking to 

make sure. So this is - this - at least for some percentage of the clinical 

population of healthcare providers, they are taking this into account. I just 

think that due to recent news reports and other reports, there are elements of 

the healthcare system that we're trying to bring up to speed on in particular, 

interpreting positive results in a very low percent positive environment. 

(Win Lee Ju): Yes. So yes, you see have a very low positive rate of prevalence then you can 

afford to check every positive. But if you have higher prevalence and also if 

the CT value are high in many people and then it's kind of hard to do that. 

And also, if you don't check them right away and that if you check like a few 

days later and that could just be that first time is real and the second time is 

still real but the virus just clear out. You know? 

Dr. Timothy Stenzel: You're exactly right. You're exactly right. 

(Win Lee Ju): Yes. 

Dr. Timothy Stenzel: It's why basically every test that we've authorized to date is by 

prescription. Why is that? Because having a clinician involved, somebody 

who can order the test involved, they can help interpret the results correctly. 

Is it a perfect system? No. But even the direct antigen tests that are being 

used in nursing homes across the country are still by prescription only. 

And so we are wanting to augment what we're saying the CDC has guidelines 

on this that are published on their Web site. And CMS and CLIA are 

supportive of this. We're going to augment what we're saying about this, to 

help educate the full community. And on this situation where there's a 

significant ratio of false positives in a population where it really matters. 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 26

And there's huge disruption if there is a positive and you want to make sure 

it's a true positive. Okay? 

(Win Lee Ju): Yes. Yes. Thank you. So yes, just one last comment on that is so on this 

molecular test, unlike antigen tests or antibody tests, we all have those 

sensitivity specifically - or specificity test given to the public. But for 

molecular test there's no such thing given. I think it's harder to determine the 

specificity as you just discussed. 

And because some clinicians - some physicians ask me what's the specificity 

of your test? Well if I look at the EUA and they all say well, this is highly 

specific or 100% specificity, it's really - but I know it's not. So is there any 

effort actually putting some - at least specificity for some of the sensitivity for 

our molecular test from this well known EUA? 

Dr. Timothy Stenzel: Yes. So, you know, even in the early days when positive patient samples 

were not readily available, and we allowed a virus to be diluted into negative 

patient matrix, there was still negative patient matrix allowed. And our bar 

which is an EUA bar, it's not a typical FDA bar for a test, during a pandemic 

and this pandemic we allow as few as 30 negative patient - independent 

negative patient samples to be used in the validation. 

In the IFU it's listed as negative percent agreement. That's for technical 

reasons because you don't have a - not a true comparator and we call it 

negative percent agreement. So if you look at negative percent agreement 

from the very beginning, all of - to my knowledge, all of the tests authorized 

have a specificity determination. 

It however, may be on as few as 30 negative samples in which case the - in 

which case the confidence intervals around that result are quite large. And we 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 27

do provide the confidence intervals in the instructions for use with the test. So 

you can look at that confidence interval as well. 

And that's another thing that's perhaps not well understood is okay, we're 

measuring the performance of these tests for this EUA and, you know, and the 

confidence interval is dependent in large part, on the number of samples that 

are tested. If only 30 are tested and you may be - you may measure 100% 

specificity or 100% NPA. 

But, you know, statistics tells us that it could be lower than that and we 

provide a 95% confidence interval so that you can say oh, you know, a 95% 

confidence interval means you can fall as low as X% on the specificity based 

on the small sample number that you've tested in the validation study. 

(Win Lee Ju): Okay. Thank you so much. Thank you. 

Dr. Timothy Stenzel: You're welcome. 

Coordinator: I would now like to turn the call back to Irene Aihie as there are no additional 

questions at this time. 

Irene Aihie: Thank you. This is Irene Aihie. We appreciate your participation and 

thoughtful questions. Today's presentation and transcript will be made 

available on the CDRH Learn Web page at 

www.FDA.gov/Training/CDRHLearn, by Thursday, October 29. If you have 

additional questions about today's presentation, please email CDRH-EUATemplates@FDA.HHS.gov. 

As always, we appreciate your feedback. Following the conclusion of the 

presentation please complete a short 13 question survey about your FDA 
FDA Virtual Townhall

Moderator: Irene Aihie

10-21-20/12:15 pm ET

Page 28

CDRH virtual town hall experience. The survey can be found at 

www.FDA.gov/CDRHWebinar immediately following the conclusion of 

today's live discussion. 

Again, thank you for participating and this concludes today's discussion. 

Coordinator: And this concludes today's conference. Thank you for participating. You may 

disconnect at this time. Speakers, please standby. 

END