NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 1

NWX-FDA OC

Moderator: Irene Aihie

February 19, 2016

1:00 pm ET

Operator: Welcome and thank you for standing by. At this time, all participants are in a

listen only mode until the question and answer portion of today's conference. 

If you would like to ask a question today, please press star followed by the 

number one on your touchtone phone. You will be prompted to record your 

first and last name. Today's conference is being recorded. If you should have 

any objection you may disconnect at this time. Now, I'd like to turn the call 

over to your host, Miss Irene Aihie. Thank you, ma'am. You may begin.

Irene Aihie: Thank you. Hello and welcome to today's FDA webinar. I am Irene Aihie, of 

CDRH's Office of Communication and Education. On February 3, 2016 the 

U.S. Food and Drug Administration issued the final version of the guidance

document, Applying Human Factors and Usability Engineering to Medical 

Devices. This guidance seeks to assist medical device developers in following 

appropriate human factors and usability engineering processes to maximize 

the likelihood that using medical devices will be safe and effective for the 

intended users, uses, and use environment.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 2

The focus of today's webinar is to review the recommendations on this 

guidance document for manufacturers and other interested stakeholders. Your 

presenters will be members of the human factors community, (Shannon 

Hoste), (Dr. Hanniebey Wiyor), and (Dr. Xin Feng), all of the office of the 

device evaluation. Following the presentation, we will open the line for 

questions related to topics in this guidance only. Additionally, there are other 

subject matter experts available to assist with the Q and A portion of our 

webinar. Now I give you (Shannon).

(Shannon Hoste): Hello everybody. It's great to be here with you today and to talk through the 

final release guidance, human factors guidance. And the other opportunity we 

wanted to have today other than reviewing the guidance is to introduce you all 

to the human factors team here at CDRH. And so that is myself, (Shannon 

Hoste), (Dr. Xin Feng), who will be speaking a little bit later and (Dr. 

Hanniebey Wiyor), who will be wrapping up our discussion today. After we 

get a chance to walk through the guidance we will go over the basic tenets of 

the guidance and then we will be opening to the question and answers. So I 

look forward to that. So we'll get started.

First, we'll be going over the relevant regulations and standards that are 

applicable to human factors and we will be talking through the guidance. And 

finally, we'll have some conversation around the draft guidance that was 

(issued) at the same time -- on February 3rd -- of the (highest priority) devices 

for human factors review. 

So to get started, the regulatory basis for human factors. So for human factors 

there is basic tenets within design control 21 CFR 820.30. Design input is 

asking for the intended uses for the device including the needs of the user and 

the patient.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 3

This can be identified and defined through human factors methods and it's 

fundamental to the development of the project and then design verification 

and design validation. And one thing I would like - want to note here is 

design validation -- to read the quote -- is to ensure that the device conforms 

with the design user needs and intended users. One thing I want to make sure 

we point out here is we'll be talking about human factors validation and 

human factors validation is looking to show that the user interface supports 

safe and effective use. And so that is -- when you're looking at those 

definitions side by side -- that is a subset of design validation.

They're not synonymous per se but human factors validation may answer 

some of your design validation needs. Make sure we're clear on that. Also, in 

the preamble to (cGMP) you'll note that human factors is mentioned 

specifically. We identify that the manufacturer should conduct appropriate 

human factors study, analyses, and tests in the early stages of design process. 

So again, here is outlining that human factors, is a portion of product 

development that follows through the product development process. And also 

later in the preamble, it's identified that use error is considered to be nonconformity. The idea here is that this is emphasizing that human factors are

part of risk management. It's part of understanding how the device can 

perform to a manner that is safe and effective for use. 

A quick overview of human factors standards and what's out there in the 

landscape right now. So HE75 -- I'm sure many of you are familiar with this -

- is a large standard document that goes over, it's more of a design tool and 

standard. It goes over some human factor processes as well as specific 

elements of how human factors is important in the control design and software 

design, but it also has chapters on specific applications such as mobile health 

and home health care. So this is a great reference as you're conducting a 

human factors activity. Also recently released is (ANSI AAMI 62366-1). 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 4

This is an update to 62366 and what this document does is it focuses more on 

the safety aspects of usability engineering

Again, as we go through this discussion, the usability engineering term in this 

document is known factors engineering, which is the term we use in our 

guidance. There are more - they're synonymous for the purpose of this 

discussion. This document will walk you through the usability engineering

process. And an important thing to note on this, while this recognized 

consensus standard at the agency here, this is not a test standard per se. It's a 

horizontal standard. It identifies a process to implement usability engineering. 

So as we review submissions, when we see conformance to the standard, what 

we may ask for is the data behind that. Because conformance to the standard 

doesn’t indicate a human pass/fail type result. It indicates that the process is 

valid.

Also, while not a usability standard, 14971, the risk management standard, is 

integral to everything that we'll be discussing today because usability

engineering and risk management activities are interwoven throughout 

product development and the product lifecycle. We've arrived at our guidance 

document. So we have the final document was released on February 3rd of 

this year and as of April 3rd will supersede the 2000 version of Human 

Factors Engineering and Risk Management. And then the draft also, which 

was issued in 2011. I wanted to give you a quick overview of some of the 

changes from 2011. I know that’s been put out there and widely read. So one 

thing we aim to do with this new final guidance is to clarify the scope of when 

human factors is requested for a pre-market submission.

One important thing to note here is when we're discussing this we're not 

discussing whether or not human factors work is pivotal to product
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 5

development. I think we were clear on that, back when we were looking at 

the design controls. . Human factors is part of robust design controls. 

What we're discussing here and what the question on the table is whether --

when and whether or not human factors data should be submitted with a premarket submission. Okay. So the criteria we're looking at there that we aim 

to clarify with this guidance is that if a task of use could result in serious 

harm. So a use error could result in serious harm. That’s where we can say as 

we're looking to protect public health we want to review human factors data.

So some quick terms. We've added a definition section to this document. 

Some key terms that I wanted to touch on before we get going because it will 

come up in this session and it shows up throughout the guidance document are 

critical task. And so this is a task, a user task if performed incorrectly or not 

performed at all, would or could cause serious harm. The keyword there is 

could. The idea here is we want to look at all of the critical tasks that could 

lead to serious harm. And then user interface, and when we say user interface, 

what we're referring to is all points of interaction between the user and the 

device or the product.

And so that could be packaging, labeling, training, the graphical user 

interface, any controls. If the device is portable it could be the way it's 

handled, so if a handle is on it -- all of those items we're referring to when 

we're talking about user interface. So as we look at the guidance document, 

this is a graphic from that document and this walks through the general flow 

of human factors engineering and it also is the way that the document is laid 

out. So we'll be talking through this today. 

A quick overview of human factors engineering. Human factors is a systems 

engineering type discipline whereas if you are looking at a device you are 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 6

looking at the system boundaries at that device. When you move that out and 

you're considering users and other folks with interactions, you move the 

boundaries out; you look at all of the interactions within those boundaries. 

That’s what we're talking about. We're talking about human factors. 

So the key things that come into and can affect that device use are the use

environments, the user, and the characteristics of the users, the device user 

interface. All of those items interact and can result in correct use or incorrect 

or unsafe use. . And then if we break that down further, we want to look at --

this is a basic model that I'm sure most of are familiar with -- of understanding 

the mechanisms behind that interaction.

So this is just a basic systems model. Coming into the user interface is an

input of some sort. The device processes that information and it generates an 

output. This output could be information. It could be energy. This 

information or energy is received by the user. They process that. They are 

recognizing it. They understand it. They need to make any decisions and then 

they're choosing to take an action on this. A few things I want to point out 

here is when you're looking at this diagram, when we talk about use error, use

error would happen right here at this star. Use error happens when the person 

takes the action. And then that information goes into the system. The system 

generates an output. That is where the hazard can occur.

That is the output of energy too high, too low; it could be put in incorrect 

information. So when we're referring back to these and looking at even test 

data, we're talking about use error happening at this action point. We're 

talking about potential hazardous situations starting here at this output point. 

And then we can go back as we see those events and start to understand why 

they might have happened. So if we see a use error we can start looking back 

and seeing if it was a problem in perception, was an item noticed was it, you 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 7

know visually perceived? The processing was it understood so the correct

decisions could be made on it, and then finally the action. 

So we'll talk through a little bit here through hazard analysis and the key 

things here is as these preliminary analyses are going on, as you're starting to 

understand the product you're developing and understand what risks are there

is understanding what are the critical tasks of use and identifying them. 

Again, when looking at these critical tasks, they should be identified based on 

their severity of potential harm and not necessarily the probability. And the 

caveat here is because first of all, it's very difficult to estimate probability of a 

use error or of an item that could lead to use error. But also at this point, 

you're trying to identify all the potential hazardous situations. So as you're 

going through that, you're looking at the things that could lead to harm.

And tools to do this are typically bottom up tools. So you're looking at failure 

modes and effects analysis. This is where you're walking through on a task 

level what could go wrong at each task, this would be an FMEA or (FMECA) 

and then fault free analyses is also another method that’s used for this.

And also as you're going through this there is evaluations of known use

problems that can lend insight into this -- that can be your internal complaint 

files. It could be knowledge that your sales staff or training staff may have. It 

could be information you have from previous human factors studies. Another 

good source are databases such as (MAUDE) database or (MedSun) here at 

the FDA. But there's also industry sites. There's even YouTube sometimes 

has some interesting information on how your device is being used. So there's 

lots of places to gather this data.

Then in the evaluation, you want to start looking at analytical approaches to 

work through your user interface and understand what all of the pieces of safe
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 8

use. And so that could be a through task analysis or heuristic analysis, as well 

as an expert review. The next level as you're starting to develop prototypes is 

to go through contextual inquiry or interviews or formative evaluations -- so 

cognitive walkthroughs or simulated use testing at some level there. 

And again, each of these are targeted at slightly different pieces of 

information, slightly different understanding of your user interface. So 

depending on what you're trying to discover at this point is the method that 

would be best for you. So we tried to explain some different options here. 

And then we'll get into the next step of the process, which is then trying to 

implement mitigations and risk control measures to reduce use error, reduce 

the potential high severity harms. 

So again, as with general risk management, the best thing to do is design 

modifications to remove the hazardous situation. So that would be inherent 

safety by design. The next step would be to consider protective measures. So

this is allowing a fault to happen but not allowing it to lead to a hazardous 

situation. And then the next level would be information for safety, and again 

as with the risk literature, these are listed in order of effectiveness typically.

And then as those updates are made, whether it's on the design of the product 

itself or whether it's in additional information being provided to the users, 

these should be evaluated to confirm that they are effective -- for verification 

of effectiveness and that they don't introduce any new risks. So at this point, 

you've evaluated the user interface. You've started to identify your critical 

tasks that could lead to harm.

You've started -- you've mitigated some of those and you have some 

mitigation in place that need to be evaluated and you're ready to move on to 

validation and testing. And at this point, I am going to hand it over to my 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 9

colleague, (Dr. Feng), and he'll be talking you through the validation. Thank 

you.

(Dr. Feng): Welcome everyone. This is (Xin Feng) from human factors team at the FDA. 

So I was here to talk about the human factors validation testing in the 

guidance. Now, the goal of the human factors validation testing is to 

demonstrate that the user interface of the device can support the intended use 

and intended user, and intended user environment without causing serious 

harm to the user or patient. To achieve that goal, the testing setup should 

match the representative intended user, intended use, intended user interface, 

and intended use environment. 

So more specifically speaking, the test participants should represent the 

intended user of the subject device. All the critical tasks from your analysis 

should be included in the human factors validation testing. The device that

includes the physical interface as well as the training, packaging, and labeling 

material should represent the final design and the use in your human factors 

validation testing.

The test conditions of the validation testing should match the representative 

use environment of the subject device. So in the next few slides, we will have 

more details into the test setup, the data collection, as well as data analysis of 

your human factors validation testing.

So first, regarding the test participants. As we mentioned, the test participants 

should match the intended user’s population of the subject device. . In 

general, minimal 15 participants should be included for each user population. 

Now, the guidance gives two examples of what is the single population. One 

example is between pediatric population and the adult population, and the 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 10

second one is the health care providers and lay users. Those are two examples 

from the guidance regarding the distinct population.

To avoid the bias introduced in the testing, your employees and partners 

should not be served as the test participants as a general rule. And then the 

test participants should be U.S. residents. We saw some exceptions in the test 

and then the exceptions to this rule should be considered on a case by case 

basis.

Regarding the tasks and use scenarios, you should present the test participant

with representative use scenarios and you can group the tasks into use 

scenarios in the logical order, and you should present the use scenarios to the 

test participant following a natural flow. Then prior to the human factors

testing, you should define the pass/fail criteria for each critical task. 

As mentioned before, you should identify the critical task based on the 

severity level instead of the probability and you should include all of the 

critical tasks in your human factors testing. 

So next one, regarding the instructions for use and labeling material. As we 

mentioned labeling and (IFU) is considered as a part of use interface of the 

subject device. All the labeling and the (IFU) and packaging, when they are 

used in the human factors validation testing, they should represent of final 

design. At the representative scenario (unintelligible) your users will make 

their choice to use the (IFU). That means some users may use the (IFU) 

during the actual use, some users may choose not. So the human factors

validation testing can include the (IFU) as part of the setup. However, the test 

participants should make their own choice whether to use the (IFU) during the 

testing or not.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 11

And last, if you decide to use (IFU) and change to (IFU) as part of the risk 

mitigation for the user related issue identified by the human factors testing, 

you need to provide follow up testing data to demonstrate that the (IFU) will

be effective to reduce or eliminate the user related risks as well as not to 

introduce any new use related issues. 

Regarding the participant training, again, the training offered in the human 

factors validation testing should match the representative training. Between 

the training and the human factors performance testing, the guidance 

recommends a minimum of one hour training delay. Similar as the labeling 

change, if you tend to use training as risk mitigation measure you need to 

provide the follow-up testing data to demonstrate the 

effectiveness(unintelligible) of that risk mitigation measure.

So now, we are going to cover the data collection part of the human factors

validation testing. There are two basic categories -- the objective data 

collection and the subjective data collection. So for the objective data 

collection, first of all, because you have defined the performance criteria for 

each critical task, you should be able to record the participant's performance 

for all the critical use scenarios. The typical data task includes success, use 

error, close call, operational difficulty, as well as anticipated user error. 

Now, for certain types of use related tasks, for example, the warnings, 

cautions, contraindications in the (IFU), which may be difficult to quantify 

and format in a performance task. Now, for those types of tasks, you can 

formulate them into knowledge questions and you can collect the participant's 

response to those questions and record them as knowledge task data.

The second big category of your data collection is subjective data and you can 

generally can collect this up to date subjective data by the interview. So there 

are a few things or categories that you can collect by subjective questions and 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 12

the first one is you should get participant feedback on the overall device use in 

each use scenario. You can also get participant's assessment regarding the 

participant's own thinking on what are the user related issues including user 

error, difficulty confusion occurred during the human factors testing.

The last but not least, you should get participants perspective on why those 

user related issues occurred during the human factors testing and that will be 

used for the root cause analysis for those user related issues. 

So now, you have the test completed. You collected the subjective data and 

the objective data. Very important is you use those data for the analysis of 

your human factors testing results. By looking at those objective data and 

subjective together, it should help you to identify the potential user errors 

occurred during the testing as well as to determine the root causes of those 

user related issues. You should be able to look at each user related issue 

through your risk management methodology and identify whether risk 

mitigation measure is required.

Now, if you determine additional risk mitigation measure is required for those 

user related issues identified by the testing then you need to provide follow-up

testing data to demonstrate the effectiveness of those proposed risk mitigation 

measures.

So there are cases that the simulated use testing might not be sufficient, as an 

example that for certain type of home use dialysis systems. Those certain 

human factors issues can only be validated it in the actual use environment. 

So for that case, we need to consider conduct a human factors testing in an 

actual use environment. Most likely for that, an IDE study might be required, 

and you can consider to conduct the actual use human factors testing at the 

part of the clinical study. And for those scenarios we highly recommend you 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 13

to submit a pre-submission to have a discussion with the agency before you 

launch your human factors testing.

So now, you have your testing done and last but not least, you should 

document your human factors process and human factors testing into a 

document and include it as part of your pre-market submission. And to make 

that process more effectively and consistently, the human factors guidance

include a human factor report outline. And we truly believe that this outline 

follows the order of how the human factors should be incorporated as part of 

the design control process. So in an outline as was shown here, you should 

first talk about the conclusion of your human factors testing, which is the user 

interface of the subject device, the intended use -- your intended use 

environment without causing serious harm to the user or the patient.

And then you should talk about your intended user, intended use, intended use 

environment describe the device user interface, and document the known use 

problem, your preliminary analysis, the critical tasks you have identified, and 

finally, the details of your human factor testing results. It is worth to point out 

that it's equally important to discuss the intended user, intended use, intended 

use environment as well as describe your user interface because those 

information are necessary for the agency to review your human factors testing 

data.

So next, we will have (Dr. Hanniebey Wiyor) to discuss the other draft

guidance, which is focusing on when to submit human factors data.

(Dr. Hanniebey Wiyor): Thank you very much, (Dr. Xin Feng) and you're all welcome once 

again. Good afternoon. My name is (Dr. Hanniebey Wiyor) and it is my duty 

to walk you through the new draft guidance -- the contents of the new draft 

guidance. Well, as you are all aware at the beginning of our presentation, in 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 14

addition to that, the release of the final human factors guidance on February 3, 

2016 FDA issued a new draft guidance document titled list of highest priority 

devices for human factors review. And the goal for this new draft guidance is 

to set clear expectations and streamline the human factors review process for 

pre-market submissions for the agency. And when this new draft guidance is 

finalized, it will be represent the Agency’s current thinking on this issue. So I 

would like to pause here and give a few reminders as to our listeners.

As you are all aware or may not be aware, the guidance development at 

agency is always a collaborative effort between the agency who presented this 

and the other representatives. So for instance, you as a sponsor may be an

industry or a device manufacturer. So as part of the agency policy, this 

guidance is available for 90 days for you to provide your comments or your 

suggestions. And be reminded that on May 3rd, all your comments must be 

received on May 3, 2016. That is when the 90 day period is up. 

You also have to be reminded that the official channels have to be used for 

your comments -- to communicate your comments or suggestions to us. And 

this guidance has a reference document number 1500052. So when you're 

making a comment to this guidance document, this draft guidance, you have 

to reference document number 1500052 written or electronic communication 

will be welcome as well. 

So I'm going to walk you through the rest of the presentation, which is the list 

of the highest priorities devices for human factors review. As you all are 

aware, medical devices are considered one of the safety critical technologies 

and the expectation is that human factors validation testing it should be part of 

your robust design control process. And it is our belief at the agency here that 

appropriate human factors engineering or usability engineering will surely 

improve the safety and effectiveness of medical devices for effective use -- I 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 15

mean for intended users, uses and use environments. So the question here is 

not when to do it, but when does it need to be part of your submission? And 

then the question again is -- the question is not when to do it -- when does it 

have to be part of your submission. That is part of this -- the guidance 

document that is (unintelligible) step.

So the next one is -- now, I'm going to talk about the list of the highest priority 

devices for us in describing this guidance. As you can see on the list there, 

this list or these devices were selected because they clearly have high 

potential for serious resulting from use error, and that information is actually 

gleaned from a medical device report or records. For instance, on initial issue

of this draft guidance or list, duodenoscopes were not part of the list. But upon 

a class 2 recall on September 17, 2015, the list was specifically updated to add 

these devices on a reprocessing side. So you can see this list is not all 

inclusive lists and most likely will be revised or more lists will be added to the 

general list here. 

Now, as you can see on the list I would like to walk you through how this list 

developed. On the list, there is a general device area in the device 

classification under the CFR regulation, and this is actually followed by an 

example of a product code. And here, I'd like to discuss the product code, 

specific descriptors to identify a group of devices within that class area under 

the CFR regulation. So for instance, the first on your left-hand column -- the 

first row, which is ablation generators. The product code (LPB, OAD, OAE, 

OCM) were product code for devices on ablation generators at the PMA 

submission. And one OCL was a 510(k) submission. 

And I'm going walk - let's take a typical example. On the left hand side -- on 

the left column, the third row which is artificial pancreas systems. I'm going 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 16

to use this to put a question and I'll provide a response to that question. That 

probably will clear a lot of things on this list. 

So let's take a typical example, which is the artificial pancreas systems on the 

left-hand side on the third row. This system, if your device falls within this is 

a type of submission, artificial pancreas systems and does not have any of 

these examples of product code (OZO, OZP, and OZQ). Does it mean that 

you have to submit a human factors data for this particular device? The 

question is you -- the response you answer is that you have to -- you may want 

to ask that. So by sending Q-sub questions to ask based on this specification, 

say that your product is a pancreas -- it falls under an artificial pancreas 

system, but does not have any of these product codes then the proper feedback 

will be provided to you for a response. 

Now, so once again I would like to emphasize that your comments or 

suggestions is absolutely necessary very, very extremely important. So please 

be reminded your comments must be received on May 3rd, 2016 as we 

develop this guidance. 

Now, the list of. Now, how do you apply this list to your pre-market 

submission? What this guidance does or present to us, it presents two classes 

or two criteria generated by this guidance. One is the devices on the list. The 

other ones is the devices not on the list, and on this slide I would like to talk 

about -- I'm going to talk about the devices on the list. So if your device is on 

the list or if your device is clearly on the list. there are two options here. 

Your first option is that you provide a human factors report and data based on 

appendix A of the final human factors guidance. Or the other option is that 

you have to provide detailed rational supporting your conclusion that human 

factors data is not needed. 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 17

What that means is that you have to conduct a risk analysis and evaluation 

with intended users, uses and the use environment of your device. And your 

analysis should indicate that the severity of potential harm resulting from use 

error is not serious or is very minimum. 

Now, the second one is -- I'm going to talk about the devices not on the list. 

So if your device is not on the list and your results of risk analysis indicate 

that if you user fails to perform a critical task or does not perform the task at 

all and could or would lead to harm, then you are required to submit a human 

factors data and a report based on appendix A of our final guidance released. 

Now, in addition to that ODE may also determine how human factors may be 

needed on a case by case basis based on the following criteria. 

One of them is your submission type so with the submissions that what we're

describing is a premarket application or de novo petition and that is clearly 

indicating that there's a potential for serious harm from a user error then you 

have to submit a human factors data for Premark review. One another criteria 

is a user interface modifications. If you made a user interface modification or 

new design based on as a response to satisfy special control then you have to 

submit the human factors data for a review. One of another criteria is 

different users. So if your device has -- is intended for different user, which is 

different from the predicate intended user then you have to submit a human 

factors data for review. 

Another is if your device is associated with a use error and that use error is 

what led to a recall or adverse event of medical device report then you are 

required to submit a human factors data for review. Finally, and not the least 

the device modification. So if your device is being modified and the 

modification is related to a user interface being modified, whether it's a simple 

modification, you have to submit a human a factors review. Or your user 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 18

tasks has been changed or added, you have to submit a human factors review. 

And then the severity of harm. 

And then finally, if your device is -- your subject device is for a new use 

environment, which is different from the predicate device's environment -- for 

instance, if your device, the predicate device was used in the hospital and you 

are (your subject device was supposed to be use in the home, then you have to 

submit a human factors review for premarket submission.

So I would like to conclude by saying that you are highly encouraged to 

actually engage the FDA in your product development so that appropriate 

human factors data will be submitted for premarket review. . And once 

again, the human factors team would like to thank you very much for you 

attending this seminar or this webinar. Thank you.

Irene Aihie: We'll now open the line for questions.

Operator: Thank you. If you would like to ask a question, please press star followed by 

the number one on your touchtone phone. Please unmute your phone and 

record you first and last name when you are prompted. You'll be announced 

at your turn. If you're in the queue and then decide to withdraw your question, 

you can do so by pressing star followed by two. Thank you. And we do have 

a party waiting with a question. I'll open the line. (Sue) your line is open.

(Sue): Yes, we could not find the slides on the website from the URL. Where can we 

find the slides?

Irene Aihie: The webinar slides can be found at www.fda.gov/ training/crdhlearn and you 

will see them under the specialty topics heading.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 19

(Sue): Okay, what was the last after training slash what? CDR what? 

Irene Aihie: CDRH Learn. 

Operator: Does that give you information you need, ma’am?

(Sue): We're looking.

Operator: Okay. Thank you.

(Sue): We're not finding it.

Irene Aihie: Okay. So can we take the next question and we will figure out how to get you 

the slides, but we have to move onto the next question. 

Operator: Thank you. (Steven Rosenfeld), your line is open.

(Steven Rosenfeld): Thank you. I wanted to know how much of our (unintelligible) testing 

results should be referenced in the final validation report, if at all.

(Shannon Hoste): Okay. Hi, this is Shannon. So with the formative, part of what we're looking 

at when we're reviewing premarket submissions and we're looking at the 

human factors data, part of what we're looking at is that the information is 

there to show that the work was done to elicit and identify potential use errors. 

And so formative evaluation provides some of that information. Again, we're 

just looking for the overview for a discussion of what may have been learned 

along the way so that we have a better understanding as we're reviewing the 

final summative testing, what are those critical tasks and what are the things 

that could impact those. So that's the level of information we're looking for in 

the formative.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 20

(Steven Rosenfeld): Okay, great, thank you.

Operator: Thank you. Next, we'll take a question from (Dr. Rajal Folako). 

(Dr. Rajal Folako): Yes, good afternoon. This is (Rajal Folako) from (unintelligible) in 

Germany and we would like to ask a question regarding dry powder inhalers 

and MDI inhalers. We would like to know from your list what priority 

classification in this list will have these devices in terms of human factors

submission. Thank you.

Erin Keith: Hi, this is (Erin Keith). I just wanted to stress -- I'm sorry, I'm the division 

director in which the human factors team resides in the office of device 

evaluation. I want to stress that the draft guidance documents that describes 

the conditions under which we would like to see human factors data addressed 

in a premarket submission is just that. It's a draft guidance document and it's 

not a requirement at this point. We look at human -- we typically see human 

factors data in submission at this point when there is a clear safety issue 

associated with the user interface with the device.

So if you have identified that that would be the case for those devices and 

those devices that you're talking about are being reviewed by the Center for 

Devices and Radiological Health and not a different center, then you would 

consider submitting human factors data in association with your submission. I 

would recommend that you discuss the actual submission and the specifics of 

it with the review division, which would receive that submission and the 

human factors team to make sure that your protocol would address what 

would be needed to support your specific application.

(Dr. Rajal Folako): Okay, thank you very much.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 21

Operator: Next, we'll move to (Laura Storm). Your line is open.

(Laura Storm): Good afternoon, FDA, and thank you for this very informative presentation. I 

have a question on slide 27. Could you please go back to that slide please?

Irene Aihie: Just a moment.

(Laura Storm): Thank you. Oh, 47. I'm sorry, 47. 

Irene Aihie: Oh, 47.

(Laura Storm): Yes, sorry. Thank you very much. Thank you. So on the left hash mark it 

talks about the device modification. Could you give me a little bit more 

information on what the FDA's expectations are for human factors studies 

based upon device modifications, which may be made, you know, while post-

(5TK) clearance.. 

(Erin Keith): Hi, this is (Erin Keith) again. I just want to again stress that that guidance 

document that (Dr. Wiyor) was walking everyone through is a draft guidance 

document. So it's not an expectation automatically for all of the specific 

trigger points for submitting human factors data as a final policy. What again 

we would be looking at right now would be if -- with the user interface, you 

observe a safety issue associated with the way that a person is reacting to a 

device through human factors, we would like to see that sort of information in 

a submission. 

The modifications that you're referring to -- if you are modifying the device 

user interface, we would want you to assess whether or not that device user

interface change would be one in which human factors testing should re-
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 22

perform to assess the modification or have you address why you would expect 

prior testing that has been done to cover that particular change. 

(Laura Storm): Okay, great. Thank you. That answered my question.

Operator: Thank you. (Nathan Hogan) you may ask your question. Nathan, do you 

have your line muted?

(Nathan Hogan): So sorry. Yes, I had my line muted. 

Operator: Okay, go ahead.

(Nathan Hogan): I had a question about -- it came up -- it came to mind on slide 31 where it 

talks about tasks and use scenarios. The last bullet talks about critical tasks 

that have low frequency of occurrence should be included in the testing. I've 

had a couple of interactions with the (ODE) in the past where we had 

extremely rare use errors and the correct method was determined to use expert 

review to evaluate those extremely rare use errors. And I was wondering if 

your thinking has changed on this or is that meant to be covered in kind of the 

catch-all pre-submission meetings comment?

(Shannon Hoste): Hi, Nathan. This is Shannon. In response to that, there is various types of 

data that can be generated in these tests and some of that can be subjective

data that's used for performance. And as we're looking at expected use you 

can capture information and critical tasks along those ways and other 

information can be captured through knowledge tasks. And that's about the 

understanding and the comprehension of specific information with the new 

user interface. 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 23

And so sometimes I think what you might be referring to as expert use might 

be something along those lines of knowledge tasks, confirming that the 

information is clear and understandable by the user.

(Nathan Hogan): Okay. Yes, that makes sense. If you have information for safety risk 

mitigation that you need to evaluate that could be evaluated through a 

knowledge task where you make sure they understand the posted 

contraindication for example.

(Shannon Hoste): Yes, in the part of the -- the challenge of some of summative testing is finding 

the best way to get at the data that you need, and actually throughout the 

whole human factors process. And so coming up with ways to get that 

information through simulated use or through knowledge comprehension 

tasks, and all of that is part of laying out the protocol and is also another 

reason that we suggest -- if we can assist the protocol review through presubmission process, we can provide some feedback on that before the testing 

is run. 

(Nathan Hogan): Right. Thank you.

Operator: Thank you. We will move onto (James Taylor).

(James Taylor): Hello. Thank you. I work with automated external defibrillators and with 

regards to the (AAMI -TIR) guidance for external defibrillators and human 

factors studies, what weight or use will the FDA make of the specific (TIR) 

guidance from (AAMI)? 

(Shannon Hoste): So several devices have special controls as in guidance documents released

specifically for those device types and again, those guidance documents are 

FDA or FDA recognized. Those would apply to part of the process of 

understanding the human factors, but all of those are process-type standards 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 24

that provide additional information that fit into the overall human documents 

process.

(James Taylor): That'll be used to inform the process but not necessarily to limit it?

(Shannon Hoste): Correct. Not necessarily to replace it. Then again those are areas where you 

would want to work with your review division and the human factors staff

assisting with those reviews prior to some of the testing to review some of 

those methodologies because you may have questions to answer from the 

human factors process and the use error questions in addition to questions you 

may need to answer according to any device specific standards or guidance 

documents that you need to satisfy. 

(Nathan Hogan): Thank you.

Operator: Our next question comes from (Chris Iesovitch). One moment, I'll open your 

line. Now it's open. Go ahead.

(Chris Iesovitch): Hi. I work for a division of a company that makes software medical devices 

and I had a question regarding the timing of the use of the human factor 

studies of data. Typically, in design validation, which earlier in the 

presentation you emphasized that human factors fell under design validation. 

That's typically done toward the end when your design has been locked down,

but in modern software development, it's iterative and some parts of the 

design are still evolving while other parts have been locked down. 

Do we have to wait until the end or toward the end to obtain our human 

factors data or can we do that iteratively in line with the rest of our product 

development methodology?
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 25

(Dr. Xin Feng): This is (Xin Feng), human factors team, FDA. So as we discussed during the 

presentation, during the summative testing or the human factors validation 

testing, the final design of the user interface should be used in the validation 

testing. Because we need to have enough validate between the testing setup 

material and the actual use device on the market and that's why we're required

to use the final user interface in your validation testing. 

However, I think using the human factors report outline, we also mentioned 

that you can include your preliminary analysis and formative study results in 

the report. And we think that's where you should include your iterative testing 

on different stages of your software development into the report and describe 

how those study results and preliminary analysis inform you to make certain 

design decisions and use that in your final summative testing.

(Shannon Hoste): This is (Shannon Hoste. I just wanted to add one thing. As you're looking at 

our requested outlines for human factor reports, a majority of that outline is 

created through the product development process. So a majority of that 

outline, sections two through seven, I believe, is the background information 

that goes into that summative validation -- summative validation testing. And 

so that is all information that is developed iteratively throughout the 

development process and then it culminates, just like your design validation 

will culminate your design product in the final validation.

Chris Iesovitch): Okay, thank you.

Operator: Thank you. Next we have a question from (Becky Leibowitz). Your line is 

open, ma'am.

(Becky Leibowitz): Yes, hi. I just had a question about what prompted the change of

terminology from summative testing to validation testing.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 26

(Shannon Hoste): It's not necessarily a change in terminology. It's a synonymous term. So we're 

trying to -- the terms commonly used in the industry, there's human factors

validation; there's human factors summative validation, usability validation, 

usability summative. We hear a lot of terms as we review these submissions. 

What we're getting at is the crux of is that final evaluation of the user interface 

in a simulated use type scenarios. So as we're referring to human factors

validation or human factors summative validation, those are really 

synonymous terms.

(Becky Leibowitz): Okay, thank you.

Operator: (Robert Stevens). Your line is open.

(Robert Stevens): Hi, there. Yes, I have a couple questions. So related to one question the 

gentleman asked earlier, you had stated earlier that manufacturers should 

revalidate in case changes are made to, say, training or the (IFU) in response 

to the validation study findings. Previously, it seemed the agency would treat 

this more on a case by case basis, whether revalidation would be needed. Can

you elaborate on that? And then the second question was can you comment 

on when it's appropriate or required to have IRB) review for a usability study.

(Dr. Xin Feng): So this is (Xin Feng). For your first question, I think during the presentation 

as well as in the guidance we do say that if you're proposing to have certain 

design modifications that including types of leveling change or training 

change, you use that design modification as risk mitigation measure. Then 

you need to provide follow-up testing data to demonstrate that proposed risk 

mitigation measure is effective as you suggested, as well as not to introduce 

any new use related issue. But I think in more generally speaking, as long as 

your design modification has direct or indirect impact on the critical task, then 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 27

you should provide follow-up testing data to demonstrate that this

modification is effective in eliminating or reducing the risk as well as not to 

introduce any new risks.

And then I think (Erin) will answer you...

(Shannon Hoste): This is (Shannon). I just wanted to note that there are cases where 

revalidation may not be necessary and again, it is somewhat of a case-by-case 

basis just as evaluating how you would establish effectiveness of that control 

measure can be on a case-by-case if, for example, if you're designing a hazard 

out and you're showing that you're totally eliminating the possibility of a use 

error, that may be a case where it can be discussed what level of assurance 

would be needed to show that effectiveness of that risk control measure. So I 

think it is a bit of a case-by-case, but again the first thing to think about is 

does that need to be evaluated and tested to show effectiveness.

(Erin Keith): Hi, this is (Erin Keith). I just wanted to put this sort of request type of testing 

and resubmission of information in context in that you're still going to make 

decisions about when a resubmission for a specific design change to your 

device fits the criteria for submission to the FDA for reevaluation. And then 

when that change meets those criteria, one of the things that you want to 

consider in that is how that impacts your human factors data. Your human 

factors data is then impacted then that should be addressed in the submission 

how you either revalidated it or determined that it didn't need to be 

revalidated. 

(Robert Stevens): Okay, thank you. And then for the IRB review question, is -- when is that 

appropriate or required for a usability study?
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 28

(Shannon Hoste): The IRB question again is it depends on the studies and the study design. So 

that will be -- again that's a case-by-case decision as to whether or not your 

human factors study should be going through an IRB or not. So I don't have 

a direct answer to that. Again, I would suggest working with your reviewing 

division to identify if there is a need for that for your study type.

(Robert Stevens): Okay, thank you.

Operator: We have a question from (Poi). Your line is open.

(Poi): Hi. I was just wondering -- I'm looking at slide 30 and 31 -- and it talks about 

tasks and use scenarios. So for a medical software company, where we have 

validation tests at the end of each software iteration or prior to release, could 

we get this human factors data from the same validation tests that we use to 

verify and validate the software?

(Shannon Hoste): I think the difficulty you may have with that, or some of the challenges that 

may be with that is being able to compile and show that at the end of your 

final product design that you're testing the final interface and understanding 

that all of the features may have some interaction when it comes to use of the 

overall product.

So some of that may end up being more formative with the final summative 

test that looks at the complete user interface. But I think it would depend on 

your design of your product development and roll out on that.

(Poi): Okay so it is possible? It's not a strict separation between, like, the software 

development process versus this usability engineering process?
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 29

(Shannon Hoste): Again, at the end of the day the goal is to show that the final design supports 

safe and effective use. So it's a matter of how that's demonstrated. 

(Point of impact): Okay. Sorry, I just have one more question. You also state that employees 

shouldn't serve as test participants and you should have about 15 for each 

distinct population. How strict is that rule? Because let's say we employ 

software testers. Would they be -- could we use them as human factors data 

subjects?

(Dr. Xin Feng): This is (Xin Feng) Human Factors Team, FDA. So I think if you look at the 

human factors report again that you have preliminary analysis, formative 

study, and analysis of known use these problems. All those processes of

iterative design and development process before your final human factors 

validation testing.

I think what the guidance is saying is that in your final human factors 

validation testing you should avoid using your own employee and use a 

minimum of 15 participants per each use group, and this is a way to avoid 

introducing bias into your final human factors testing results.

And during your preliminary analysis or design development process, if you 

choose to use your own separate test or your partners into your testing, and I 

think that's depending on what kind of results and objectives you're trying to 

achieve for those preliminary studies. As long as it meets your objective and 

serves the purpose, I think this is case by case discussion.

But again, for the final human factors validation testing, the guidance says that 

you should avoid using your own employees and use minimum of 15 

participants per each user group.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 30

(Poi): Okay. So then just to confirm it’s a guideline but not so much a requirement 

then?

(Shannon Hoste): Well, I think it’s the case where for summative validation to use the software 

testers from the subject device company, the case for that would be if the end 

users were the software testers of your subject device

(Poi): I see. Okay.

(Shannon Hoste): At the end of the day, you're showing that the final user interface can be used 

with representatives, your expected users in expected use scenarios. So that's 

where even -- one, you need the representative results from the users. So 

whether or not those test engineers could represent your actual users, but also 

that they are using it in a scenario where they're not biased -- that you think 

they're using it as it would be in the real world.

(Poi): Okay. Thank you very much.

Operator: Thank you. (Tom), your line is open.

(Tom): Yes, my question is there some type of protocol report template or guideline 

in terms of what this study is supposed to look like?

(Dr. Hanniebey Wiyor):Hello once again this is (Dr. Hanniebey Wiyor) and I think that the 

protocol, the guidance of how your template should look like is as its 

explained here in the guidance that you're testing as well as the user 

environment and the design interface should represent the actual use setting 

of your device. So as much as possible the fidelity of your testing should be 

what is your concentration here. 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 31

Take into concentration the representative user, the use the environment as 

well as intended use of your device. So what you have to think in advance is 

how you have to put all these three simultaneous interaction, uses users and

the environment and then how you can implement this in your validation test.

(Tom): Yes. So I see that it's going to mimic your (IFU) so as you're executing 

through the (IFU). You're going to be making observations along the way and 

also picking up other things in terms of how it's being misused. So that's how 

the protocol I would assume would look.

(Dr. Hanniebey Wiyor): Can you repeat the question again please? I mean your comment 

again?

(Tom): Again, when you're executing these usability studies, is there a specific 

template in terms of the detail that you want to see in the report for the premarket submission?

(Dr. Hanniebey Wiyor): Yes.

(Shannon Hoste): The report is outlined in the appendix.

(Dr. Hanniebey Wiyor): Yes.

(Shannon Hoste): So within the appendix there is an outline that goes though some of the detail 

within the report structure. Again, that's a recommendation of the report and 

how that information can be laid out so that it conveys easily to the reviewers

and they can quickly understand your critical tasks and what your focus was 

on during the summative testing.

(Tom): Okay thank you.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 32

Operator: We have a question from (Enba Trum). (Mr. Trum)?

(Enba. Trum): Yes. At the last slide, I think you mentioned about submitting protocols to the 

Agency for review. Is that for both summative and formative?

(Dr. Hanniebey Wiyor): It's only for summative. Formative protocol is probably not 

allowed to be submitted but because we are only going to basically 

concentrate on your human factors validation testing or usability validation 

testing. So the protocol is to concentrate on your summative human factors 

validation testing.

(Enba Trum): Okay, thank you.

(Erin Keith): Hi this is (Erin Keith). The focus of the guidance document itself is from a 

request perspective from the agency on what we would like to see to support 

the pre-market program is the summative information. We think that the 

formative information is very helpful and useful. But we're not requesting and 

requiring companies to submit it. We would appreciate a summary 

description of what you did and what you learned from it, but not the detailed 

protocol or results.

Operative: Are you ready for the next question?

(Erin Keith): Sure.

Operator: Thank you. (Susan Needle) your line is open.

(Susan Needle): Hi. Thank you for the presentation. It was very good. Quick question, on 

Slide 45, when you are talking about the highest priority device types I'm 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 33

curious why there is a distinction on whether or not CDRH is the lead center 

or not when it comes to the devices and having the human factors submission?

Erin Keith: Hi this is Erin Keith. . Could you repeat your question please?

(Susan Needle): Yes. I'm curious about the distinction on why for some of the devices it talks 

about depends on whether CDRH is the lead center for whether or not it is 

part of the highest priority device types to include human factors as part of the 

submission. Is there a reason for the distinction?

Erin Keith: Yes, there is. So there are combination products that are -- have other centers 

have the lead review over. Those would be things that are reviewed in the 

Center for Biologics or The Center for Drug Research and Evaluation. So this 

guidance document is a CDRH guidance document that covers the CDRH 

lead submissions.

In the event that you have a product that is a combination product that 

includes let's say a drug delivery system such as a pen injector or a jet injector 

and things like that, those submissions -- the lead centers are either CDER or 

CBER depending upon whether or not they are biological or drug.

Those submissions aren't governed by this particular guidance document. We 

believe that the information contained in it is very helpful and useful for you 

in the development of your products. But in understanding what the 

expectations are for the reviewing center about what they would like to see in 

a submission, you should be contacting that specific center and having a 

dialogue with them regarding what information they need for -- as it relates to 

human factors testing.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 34

There is a draft guidance document out at the moment that the office of 

combination products has issued related to human factors or combination 

products and I would suggest that you review that and consider commenting 

on that guidance document while it is open.

(Susan Needle): Okay, thank you.

Operator: Thank you. The next party, I did not hear the name but they said they were 

from (Leva Nova).

(April): Hi this is (April). I have a quick question and it wasn't really covered in the 

presentation. But I wondered if you had any guidance or best practices on 

how to show traceability of your validation testing back to maybe task only or 

design requirements or user needs. If you could give any comments on that.

(Shannon Hoste): This is (Shannon Hoste) from Human Factors. On the traceability side, I think 

that that is always helpful for understanding especially if you're trying to map 

out how those use errors, and risks, and user needs were identified, kind of 

how they were carried through the process, mitigated, and finally tested and 

validated. That is helpful to communicate the message of the work that was 

done and that can be helpful. But it is not required per se. 

We don't mention it in the guidance as such. But that is a tool that's 

commonly used to enable that work to happen and kind of tie everything 

together, especially if it is done electronically to link those things and assist 

you in the product development effort. But from the regulatory submissions 

standpoint, it may or may not be useful. Depending on how complex the 

product is, those trace matrices can become quite cumbersome.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 35

So it's a question of whether it can convey that information or whether it's 

more of an internal tool. So we don't discuss it in the guidance per se.

Operator: Thank you. (Terry Bogucki), your line is open.

(Terry Bogucki): Hi. I have two questions somewhat related to the comment that was just made 

because this wasn't addressed in the guidance, but two things.

One is if you're going for over the counter labeling, we've gotten some 

feedback from the agency that our usability testing would have to incorporate 

the ability for the participants to, like, self-select out of the testing, which is 

really sort of independent of the errors and the use error that you could run 

into. So I wanted your feedback on that.

And then the second question is if a company is going to offer case support 

for the initial uses of the device, what impact does that have on your 

interpretation of your usability testing results?

(Shannon Hoste): Could you elaborate a little bit more on what you mean by case support?

(Terry Bogucki): Well for example, if you are, you know, you have a product; you've done the 

usability testing. But then you can maybe see some mistakes are being made. 

But when you go to sell the device, when the initial uses are done -- say if 

there's five cases for example --. You would have someone there to help 

guide the physician or whoever through the product use.

(Shannon Hoste): Okay. I think what you would need to evaluate at that point is understanding 

the critical tasks of use. Understanding if you are expecting -- somewhat like 

a training program, right? So if you are expecting that the first five uses are 

gathered by a master user or something like that, then maybe that needs to be 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 36

built into an understanding of your simulated use scenario, so that might be 

presented in a training type scenario within your summative testing.

So again, thinking back to again what are the critical tasks? What are the 

things where an error could be made that could lead to harm? And trying to 

effectively implement those within your summative study so that you can get 

a clear understanding of how your device supports those issues.

(Terry Bogucki): So are you saying that I would model the case support scenario in my usability 

study?

(Shannon Hoste): If you have identified critical tasks, you may need to consider that if you are 

identifying the need for that, for some reason, as part of your training program 

then you may need to consider to show how that is effective and that if it’s a 

fixed case, these errors aren't introduced.

So again it all depends on your application, understanding your use scenarios 

and understanding those critical tasks of use and evaluating based on those.

(Terry Bogucki): Okay.

(Shannon Hoste): Thank you. The next question is from (Patty Cole).

(Patty Cole): Yes. Thank you, in a similar vein to the question on device modifications. 

What is the agency's current thinking on the applicability of the final guidance 

document to reprocess devices?

(Erin Keith): So hi, this is (Erin Keith). So modifications to reprocess devices or just 

reprocess devices in general?
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 37

(Patty Cole): Reprocess devices in general.

(Erin Keith): I think that you should consider what the impact is on the user interfaces 

associated with the reprocessing of the device and make a determination 

whether or not it needs meets individual criteria associated with reprocessing 

devices.

I think that there are some instances where human factors data might be 

important associated with that to support applications, and I think that there 

could be others where it isn't. And we don’t have a specific across the board 

policy related to all reprocessed devices that you should submit.

Again, the draft guidance document is open for comment about under which 

the conditions in which human factors data should be considered in it, but in a 

marketing application. And if you have any comments related to that issue,

feel free to send them to the docket.

(Patty Cole): Okay, thank you.

Operator: (Jennifer Sigh) you may ask your question.

(Jennifer Sigh): Hi. Can you speak a little more as to what is the threshold for serious harm? 

Like what is an example of serious harm? And what is an example of a harm 

that is not serious? Because when I think through things, almost any medical 

device can cause patient injury of some sort or at least it is providing therapy 

and if someone can't use it then the patient is not getting their therapy. So I 

guess I am trying to get a better understanding of that because the processing 

is very either or right now -- either the device can cause serious harm and you 

want a validation study, you want the data submitted in the report -- or it 
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 38

seems the alternative is that nothing needs to be submitted to FDA. You don't 

need to see or ever hear about any of the human factors work that was done in 

that case. 

So there's not even a rational as to why we aren't submitting anything. So is 

that a correct understanding?

(Shannon Hoste): We're looking at high severity harm. So what we are looking at -- again, it is 

subjective because risk management is a bit subjective. It's -- a lot of times 

uses the numeric quantifications, but it's a pseudo quantitative process, it's a 

qualitative process of understanding risks.

And so as you're looking at high severity harms, we're looking at from the

standpoint of is the device safe and effective? Can a clinical harm be created 

by a misuse or an error in the use of this device? If not being treated it could 

have a harmful clinical outcome, then that may be something else that flags 

that perspective.

Again, it's looking at the overall risk of the device and understanding what 

aspects of that device -- where could a misuse or a use error cause that clinical 

harm -- that high severity harm.

(Dr. Xin Feng): And I think some of that is also coming from looking at post market signal 

data of your predicate device and similar devices on the market. What we 

have learned from the market and use of those devices so far, that it 

introduced any serious harm -- issues to the patient or user. So looking at the

post market signal may help you determine whether there is serious harm to 

the patient or user.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 39

(Jenifer Sigh): Okay, thank you. And if we make the determination that this is something 

that does not cause serious harm then if you don't need to see any other human 

factors work that we might do on the product. So nothing in terms of human 

factors submission, is that true?

(Dr. Xin Feng): Well, I think if you reach that conclusion, our suggestion would be you submit 

a pre-submission Q-sub to submit your justification as why you think there is 

no serious harm issue with your device. Basically I think at that time you're 

saying is there is no critical tasks associated with your device and then you 

can submit that justification for the agency to review.

(Jenifer Sigh): Okay, thank you.

(Shannon Hoste): Yes, as we said, this is the criteria for when we may be requesting human 

factors data. So if there is a risk consideration, that's where the humans 

factors data would be requested as part of the submission. So again, it's 

whether or not it would come in with the package for submission.

(Dr. Hanniebey Wiyor): Hello, this is (Hanniebey) and I'd like to add something to what my 

colleagues have just talked about. It's like the concentration (on clinical harm. 

Sometimes it can be delay of therapy or some of the harm may be a little 

benign because if a device makes a diagnosis a disease or condition, that is 

also a harm because if a misdiagnosis and we have to have the data for that. 

So you have to look at the whole device as total risk management. It’s a little 

bit subjective but it may different from the acceptable on clinical harm 

concentration.

So you don't have to pay attention to clinical harm but also pay attention to 

some of the potential harms that are made, maybe as a result of delay of 

therapy or misdiagnosis.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 40

(Jenifer Sigh): Okay, thank you.

Operator: As we are approaching the bottom of the hour, this will be our last question 

and it comes from (Sue). (Sue) your line is open. Did you still have a 

question? Please check your mute button. We'll move on to (Dave Zack) for 

the last question.

(Dave Zack): Yes, hi. Developing a mobile medical app for dose guidance for a 

pharmaceutical company and interested in how the human factors and 

usability guidance would apply in the case of a mobile medical app for 

smartphones.

(Erin Keith): Hi this is (Erin Keith). So I would say, one, it depends on whether or not this 

is a product that is regulated by CDRH or a different center, and again you 

would need to confirm that it is a product that is regulated by CDRH or...

(Dave Zack): Yes.

(Erin Keith): in this and to know what the sort of expectations are for the submission.

(Dave Zack): Yes.

(Erin Keith): But mobile medical apps would fall into the same kind of categories as every 

other device. If it is something that would be received as meeting one of those 

proposed criteria in the draft guidance document, as having a trigger point for 

requesting human factors data then we would want to understand how human 

factors data was addressed or why you came to the conclusion that you didn't 

need to submit it in that specific application.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 41

(Dave Zack): Okay, fair enough. Just one other quick question related to that is the device 

class -- class one through three -- - all the same considerations regarding this 

guidance or is there some differentiation there that you would consider?

(Erin Keith): The differentiation is really along whether or not it's something that we would 

be looking at in the pre-market versus exempt from pre-market. If it's 

something that we see in pre-market we see it because selectively as a whole 

or see the relative risk profile associated with that device is something that is 

higher than those in which that we don't -- that we don’t request to see through 

the pre-market program.

So the probabilities are a little bit higher that you might have a device user 

interface with a higher risk device that results in something that you might 

want to see the human factors data on, but it's not a guarantee. There could 

be class three devices that we would say that it isn’t necessary to support the 

application and there could be class two through class one to carry them.

(Dave Zack): Okay, thank you.

Operator: And that was our last question as our time is concluded.

Irene Aihie: Thank you. This is Irene Aihie. We appreciate your participation and 

thoughtful questions. Today's presentation and transcript will be made 

available on the CDRH Learn webpage at www.fda.gov/training/cdrhlearn by 

Friday, February 26th.

If you have additional questions about the final guidance document, please use 

the contact information provided at the end of this live presentation. As 

always, we appreciate your feedback.
NWX-FDA OC

Moderator: Irene Aihie

02-19-16/12:00 pm CT

Confirmation # 6874928

Page 42

Again, thank you for your participation and this concludes today's webinar.

END