**Transcript of Cyberseminar**

**Mixed Methods Methodology**

**A Practical Approach to Designing, Conducting, and Analyzing Focus Groups**

**Presenter: Susan Zickmund**

**December 10, 2012**

**A Practical Approach to Designing, Conducting, and Analyzing Focus Groups**

**December 10, 2012**

MODERATOR:  We are just past the top of the hour here, so I am going to introduce our speaker.  Today’s speaker is Susan Zickmund, she is the director of qualitative – at the Qualitative Research Core at CHERP – the Center for Health Equity Research and Promotion at the VA Pittsburgh Healthcare System – and she is an associate professor of medicine and clinical and translational science the University of Pittsburgh.  Susan, can I turn things over to you?

DR. ZICKMUND:  Certainly.  

MODERATOR:  OK.  

DR. ZICKMUND:  Welcome everyone.  Thanks so much for joining.  [Inaudible] discussing the goal for this cyber seminar, and that is provide the practical details needed to conduct and analyze focus group data.  The organization of the seminar is – I’d like to focus first on a definition and a design of focus groups then move on to a step by step guide to conducting focus groups within the VA.  Next, I’d like to discuss the code book construction and coding process and then finally, just a very brief mention of something described as a virtual focus group.  

OK.  I’d like to start with my very first audience poll.  And, I’m curious about your level of experience in terms of conducting focus groups.  So if you could share whether or not you have no experience, some experience, or a great deal of experience, that would be very helpful to me.  All right.  So it appears that the majority has some experience although when it comes to individuals with a great deal of experience, that’s a relatively small number of the individuals.  So, I will share with you, and I believe now I can click and become the presenter again.  OK.  Show my screen [laughs].  So, the goal that I had and I predicted that the audience would be individuals who had really used sort of a basic approach to focus groups with the thought that maybe we can just sort of walk through the step by step, so hopefully this will meet the needs of the audience.  

So let me start with my first part which is the definition and design of focus groups.  OK.  Providing a definition of focus groups is a form of data collection with a goal of describing an understanding perception interpretations and beliefs of a select population to gain understanding of a particular issue from the perspective of the group’s participants.  And what I’d really like to draw your attention to is this idea of trying to gain something about the perception, the beliefs, and that really having a targeted group that you’re interested in learning that information from.  OK.  So, I’m a qualitative methodologist and I sit down with a variety of different – PI s– investigators who are interested in collecting qualitative data and we often have conversations about when to choose a focus group and when not to.  So I’d like to share with you a little bit about what the conversation is often like.  So I really encourage individuals to use a focus group over the other most typical form of data collection which is an interview when the goal is to discuss attitudes about products, devices, campaigns and recognize that focus groups came out of the marketing industry.  And if you think about perhaps one of the ways that we frequently see focus groups used, such as at CNN when there’s the presidential debates and they bring people on for focus groups and they ask them what to do they think.  Focus groups are especially good at that kind of data collection.  They’re very good for pilot testing, surveys, study designs, all manner of things.  They’re great for generating new ideas.  Also in terms of gaining competing viewpoints; when you thought everyone feels this way but oh, you have a focus group and you realize no, there’s a lot of diversity of opinion.  You can also shed light on quantitative study findings if you’re really trying to understand the insights particularly of a certain population.  They key and the choosing of focus group is that you really want something that you’re going to gain through the interactions with the participants.  That’s the most exciting thing – element that comes from focus groups, and the best reason to choose them.  

Let me also share moments when I often recommend to investigators perhaps not to choose a focus group, and instead perhaps to choose an interview instead.  And so, let me pause just to say I’m giving some dos and don’ts and its – you know, qualitative researchers are often very flexible and I have to say that every do and every don’t, there’s going to be an alternative situation where some other design might work.  So I want to be, you know, responsive and sensitive to the fact that there are many cases that you know, there might be some design that runs contrary to the designs that actually work.  

That being said, collecting views in private, stigmatizing, incriminating behaviors, focus groups is often very difficult.  All right, if you think about how difficult it is to get individuals to start talk about almost anything in a group, if you add something that involves shame, it’s often difficult to collect information in that way.  And I would encourage someone to use an interview.  Think about bringing providers together, having a focus group about why they choose not to wash their hands before they see the next patient.  You could see that it might be easier to have that conversation in an interview rather than in a group.  

Also, in terms of capturing qualitative themes from each participant, it’s important to realize that we never know if a particular participant is going to talk in a group.  You might have somebody who talks says a lot things; you may have somebody who says absolutely nothing.  So, if your goal is to link themes to individual outcomes, let’s say comorbidities, depression scores or demographic feature on the demographic sheet – it’s much better to be able to use an interview, because the linkages are a lot clearer. 

Also, I would encourage somebody to consider a telephone interview over a focus group when the goal is to recruit busy participants with demanding or you know, conflicting schedules.  For example, oftentimes, we start talking about having focus groups with providers and then realize providers – they’re very busy; if you’re working with a team that’s part of a hospital, the schedules may be very different.  Administrators often have very busy schedules that demand their time and it’s much easier to use a telephone interview so that they can structure according to their busy schedule.  Or also closer geographically dispersed.  So, for example, a focus group with 14 diabetes educators across the state, where you’re trying to bring everyone together in one place, this becomes very difficult.  A telephone interview may work better.  Or even potentially with an at risk population who have limited impact – limited access to transportation, they may have privacy concerns or other aspects – a telephone interview might actually function better for them.  

Also I just want to emphasize that focus groups are not a good choice when the goal is to reach consensus.  Focus groups are wonderful in terms of brainstorming but when your goal is to really try to bring the conversation bound to the point where you can  make decisions about the right choice or the next action step, there are other techniques.  For example, something that is nominal group technique would work better than a focus group.  Just something to consider.  So another question that participants would have or investigators would have, would be how many participants do I want per focus group.  And I can share that the ideal number of participants is about 10 – eight to ten for a group; I would prefer eight, but up to ten.  The maximum number should be around 12.  The larger the group, the harder it becomes for everyone to talk.  And the minimum number should be, you know, around three.  I would much prefer to have four or five than even three.  Again, the fewer the people, the more it functions like a shared interview.  I can share with you an experience where we had an interview where – we had a focus group and many people didn’t show up and we had two individuals who did, one was very reticent and one was dominant.  And I can share that that really was much more like having a sort of an interview with participants around it.  So, one of the things that typically is a good idea with focus groups, try to over recruit.  Like ten to 20%, I’m just sending a little bit about the likely barriers to participating that your particular subjects might have.  And so, and then if they all come, which rarely happens but once in a while it does, your focus group may be larger.  

Also in terms of the length of the focus group – about 60 minutes between health services research is typical.  An hour and a half is also not an atypical length within the kind of research that we do here in health services within the VA.  Discussions can extend to two hours, 160 minutes, often in you know, marketing and business focus groups, they tend to be longer.  Also, it may have a lot to do with the nature of your audience, how many questions that you have.  So I think what’s very important is that if you have longer discussions, please include a break.  Typically when we have a 60 minute focus group, we don’t give a break.  But we do inform the participants of the length during the ground rules stage, just to help define the expectations.  And always think about decorum in the focus group; whoever you recruit, you hope at some future moment they might want to come back.  And what’s important is to begin and end on time.  And that’s challenging because people are coming in, they’re eating, there’s forms to fill in, other people are coming late.  Also, in the middle of the conversation, you love what they’re saying and then you’re starting to run out of time.  You want to keep going, but it’s really important to make sure that you begin and end on time, especially when you’re fitting into a predefined time slot such as a lunch hour.  We do focus groups with providers, and it’s very important I know, that when that lunch hour’s done that we need to let them go.

In terms of the numbers of focus groups, I – the literature emphasizes that you should have about three to four discussions on a single topic and a single topic also talks about a fairly homogenous population.  Now I know that many individuals don’t have the budget to be able to do three or four focus group and so it’s possible to bring the number down to two.  And there are limitations to that, but it’s certainly feasible.  Do note that when you have group comparisons – let’s say you’re doing a gender based comparison – what you want to do is to take the number and you need to double it.  So, instead of having three to four discussions and you’re doing gender based comparisons, then you would have six to eight.  Also, if you’re using different sites, almost – no matter how many focus groups you have, I would really encourage somebody to have at least two focus groups per site.  And sort of a rule of thumb is always avoid a single focus group if you can because the group dynamics are such that a single individual may really skew the discussions, so that it would be difficult to really effectively use that focus group. 

Also, just to share, if you’re thinking well, why do I need these numbers.  The ultimate goal of all qualitative research in terms of it determining the numbers involved is to achieve somatic saturation.  In this case, across the focus group discussion.  Somatic saturation is the means of determining sample size in qualitative research.  What it means is saturation occurs when new transcripts reveal no new themes.  Or you’ve been sitting in all the focus groups and you’re hearing the same kind of thing.  The notion of thematic saturation really emerge from the social sciences back when people could do the research without necessarily having to write the grant in advance, and to sort of come up with a guestimation of what may be the sufficient number of focus groups.  And so what I would encourage you is, if you’re going to plan your own project or your own grants, take a look at the slide before in term of the sheer numbers.  Think about how homogenous your population is, or how really diverse your population may be.  And try to do some of the numbers – how many sites are involved, are there comparisons, just try to get a sense of how many focus groups you should conduct.

And let me also share something about consistency of data collection.  To help achieve somatic saturation, try to keep the script questions.  So your focus group script questions in the same order for all focus groups.  This allows for comparisons across group.  But let me share that there are a lot of different philosophies in qualitative research.  I and my team fall on the side that embraces consistency, particularly consistency in the interview script more than other methodologists would.  And you’ll probably see more why that is when I start to share a little bit about our approach to data analysis.  So consistency is a theme that will frequently show up that I would encourage, personally, teams try to do.  Because I really think that it helps for analytic precision.  Also, if you’re – let’s imagine you’re doing gender comparison.  You have a group of women and you have a group of men and you’re interested in asking a few gender specific questions.  Let’s say just for the women.  What I would really encourage is take the same script and then add those questions on at the end.  It’s particularly in situations of comparisons – it’s important to try to keep the focus groups as similar as possible so that you know that what’s really changing is the answers from the participants rather than anything about the design that might be different.  

Also it’s important I think about the composition of the focus groups.  You always want to avoid a chilling effect on the discussion.  And, separating participants out is one approach to being able to deal with that.  Now, like I said, there aren’t many fast rules and qualitative research – more rules of thumbs and suggestions – I would say something that’s pretty close to being a fast rule, is that you should always try to separate participants based on different levels of power and authority.  Again, you want everyone to feel comfortable talking.  Sort of the classic example is when you have, you know, back in the day, we were thinking, the MD as the providers and non-MDs is maybe somewhat of an old fashioned notion.  But typically, we’ll separate anyone who has, you know, some kind of power over other participants and separate them into different focus groups.  So, for example, if you had someone who was the boss, you know, it was the leadership of a certain organization and employees.  You would most likely try to separate them out – those groups out, just so that – the employees feel comfortable being able to share information and the leaders don’t have to wonder, you know, am I looking, you know, maintaining my authority and making these disclosures.  In a similar way, if you happen to know the participants have a negative view of each other, and this doesn’t tend to come up very much, it would be important to separate them out.  I, in fact, even struggle to come up with an example here of management and union employees doing contract talks, because in general, there isn’t this open hostility that I’ve confronted in medical topics.  But if you were to, the best way to address it is to separate the participants.

I also recommend that if you have participants with vastly different levels of experiences or levels of training, that you would want to try to separate them out as well.  Unless you have a good reason.  I mean, there’s always situations where that’s the particular effect you’re trying to get.  But let’s imagine you were trying to get feedback on some new software and you had bioinformatics experts and you had members of the lay public.  It would be important to separate them out.  So in terms of the composition of the focus groups, here I’m talking about where the goal of the research is to actually make comparisons as opposed to just having different groups.  When you have comparisons, it’s always important to separate them out into different focus groups.  And, I’ll just give some examples.  You could be doing gender, you could be looking at race and ethnicity and then you might have multiple groups of which, of course, you would want to have, you know, three to four of each, one of the groups if you possibly can.  I’ve seen and I’ve worked with focus groups that have control groups and intervention groups; even one where we had control groups and intervention groups and then almost an intervention plus group which are more intensive – and then we talk to people separately.  

Now I’d like to chat a little bit about script development, and much of what I’m going to say is specific to interviews as well.  To sort of, you know – best practices.  The first one is to be clear on the information needed from the discussion.  And I know that sounds obvious; however, when I work with investigators I always have them give me sort of bullet points of what kinds of questions you would like built into a script and oftentimes, I’ll find almost a reluctance to, you know – be very clear, this is the question that will give us the kind of information we’re seeking.  As if maybe being more subtle, or you don’t want to be overt; try to be really clear on the information that you need and formulate your questions around that.

Also know that for focus groups, you want fewer questions than is typical for an interview.  So for a one hour focus group, you may, depending on the level of complexity of the questions, four to seven questions; so if you were doing an interview, you could certainly have many more of those questions, so just to be aware of that.  Also, it’s really important to develop probes for the moderator to clarify, enrich the discussion.  And even though I fall on the spectrum of trying to have consistency of data, I do want to emphasize that the role of qualitative data is to allow participants to talk, to follow them, to let them think through; and you would want to have various probes and the probes may differ very much between the different focus groups, and that’s fine.  What I really just think is important is then you bring the discussion back to a specific question and then you probe.  So in terms of the script development, I always encourage investigators to have a pilot testing phase, certainly in this case for the focus group.  The best thing that works is first, do a mock focus group with the team members just to see what’s the wording like, do people get it, obviously the team understands in general what the topic is, and then to go to participants who are similar to the topic population.  Just so you can pilot, recognize your most likely needs; include that.  My sense is you probably would need to include that on your IRB protocol.  And then also make revisions based on the team feedback.

Also, again sort of mechanics of script development, always try to avoid closed questions; and I think you always hear that, but it’s so easy to have a closed questions without realizing it.  So this example, did you receive diabetes education – if a moderator asked that, there’d be probably a lot of head nodding or shaking – wouldn’t – shoulder shrugging, what do you say, it would be important then to say did you receive diabetes education and if you did, could you share with me a little bit about your experiences.  Because by doing that, you’re able to open up the discussion.  Also make sure that you eliminate double barreled questions that really probe two different topics.  So an example, do you understand how to take your medications and do you follow your doctor’s advice.  I – if you do that particular in a focus group because you have so many people, it’s likely to end up that either one thing gets answered or the other, or it becomes muddled.  So, especially in the focus group discussions, it’s important that the questions are short and understandable.  

Also, when including differing populations, tailor the scripts to meet the needs of those individuals with the lowest education level.  So let me give an example.  So let’s imagine there was a change in some kind of eligibility or service connectedness, and you want to get an understanding from patients, but you always want to understand the experiences that providers – and potentially do some kind of comparison.  On those questions that are the same between the providers and the patient, what I also will describe is sort of anchor questions that link them.  It would be very important to make sure that you lean in the direction of the classic eighth grade education level because the providers can always answer those questions, as opposed to having language that might potentially be not as clear, not as understandable for the more – the lay public.

OK.  So that has taken us through the definitions, and a little bit about the mechanics and the reasons for why we would choose and not choose focus groups.  Now my goal for part two is to really take a step by step guide to conducting focus groups within the VA and try to make this as helpful for an individual who’s thinking OK I want to do this, now what do I do next.  So, one of the things that you need to do is you need to think about the environment.  You need to choose a comfortable room with a large table that’s large enough to hold all the participants.  Be sure the room is quiet.  You’re going to be tape recording, and so, if they have decided that they are going to, you know, renovate, and there’s a – you know, there are people with hammers right outside your window, you need to find a different room because it will really effect not only the conversation but the tape recorder as well.  Also think about privacy.  If you have participants, particularly if it’s a more sensitive topic, but in general, they are expecting a degree of confidentiality and if they have to go through a clinic and a waiting room and say I’m here for the focus group or they have to walk through the police station to get to the back room, you want to think about these aspects when you choose that room.  A round table is ideal because this is a – you want to decrease hierarchy as much as you can to create that facilitating type of environment.  However, most of us don’t end up with a round table; we end up with a board room table, and those are rectangle – rectangular.  So, what I really suggest is that your moderator sit in the middle, not at the head, because the moderator is already running the discussion; you need to empower the participants so at a certain point, they take over and the moderator is there to facilitate.  And the more the decentralized the environment is, the better it is for that ultimate outcome.  Also, make sure to have the participants turn off any of their electronic devices.  There are a lot of gizmos that go off, and anything that breaks the flow of the conversation you want to avoid.

Also think about the form.  So bring all the inform consent documents that you need signed.  If a participant comes in while the discussion is underway, try to have a team member who can, you know, welcome them, take them aside, have them complete all their informed consent documents for joining the discussions so that you have all of those matters effective taken care of.  Do be aware that the VA has a special form – special consent form for the recording of voice and also photography.  It’s very important that you get their signature on that form as well.  And there are likely to be other forms; typically, people do demographics and I encourage you to have a demographic that tells you something about your participant.  If you’re publishing at this, it gives you a, you know, table one – typically they’re done beforehand; they can be done after as well.  I really encourage, however, if you have surveys which often people do, that will in any way bias or inform the discussion in a very specific way, reserve it for the end.  You don’t want to somehow have created a unique situation because of your survey when you could have avoided that easily.  Also recognize that all focus groups should provide food.  That’s just – that’s part of the decorum of focus groups.  Allow participants time to eat before the focus groups begin.  And, that’s sort of titrated based on how elaborate your food might be.  So, if you have for example, a boxed lunch, I would – and they also have to fill out inform consent documents – I would usually ask people to come about 30 minutes early.  If you have something like you know, a beverage and a cookie tray, that doesn’t take very long.  I would hope that the focus group’s during the middle of the day and not during lunch.  Give more food during the breakfast and lunch typical eating time.  But then, 15 minutes would usually be sufficient.  Also recognize that some VA sites, maybe it’s a national regulation, we have it here, require that if you’re going to use your VA funds to purchase the food that there’s – that there are forms to fill out, just be aware of that.  Try to organize that at the same time you’re doing your IRB protocols just so that you don’t suddenly realize that you can’t use the VA funds you had anticipated to purchase the food.  

Compensation is always an important point; and  typically, the participants will receive $25 to $45 per focus group, depending on the length or maybe something about the unique expertise of the participants.  Try to be clear with your IB – IRB – what they define as coercion.  If you really wanted participants, you may not be able to say oh just pay them a hundred so that they will come.  There are constraints on that.  Things that you can do to help is to offer to pay for transportation, parking vouchers; in fact, if you have a population that has financial deprivation, it would often be very helpful and nice and just to be able to help sponsor transportation.  Just to share that the VA typically does not allow providers to be compensated for research – what I find is the most effective way to engage providers is to find a champion for the study.  Somebody who is respected, known, or perhaps even the buy in from leadership where they find this is an important focus group and they really encourage people so, there are different ways of recruiting rather than just having compensation.  Also, discuss whether VA providers are allowed to participate during work time, their work time, or lunch break.  There are differing policies about that and I think it’s important to be aware of that before you’re in the midst of scheduling things and then you find out.

Let’s discuss audio recording.  It’s a – it seems like a fairly simple topic but it’s a really important one and having advanced planning would be helpful.  On – so one of the great things is that digital audio recorders work very well.  I’ve been in qualitative long enough; I used to use those mini cassette recorders and they never worked and they ate the tapes and you had to flip them, the sound quality is very poor.  Digital tape recorders, particularly high quality ones, do a marvelous job.  Many times, people will choose to have two audio recorders for their focus groups.  We typically have one and it tends to – we haven’t had a problem, but I can understand wanting to be certain, making sure that one doesn’t stop working.  Some teams also use external microphones.  They feel that it helps with the sound quality.  We haven’t had a problem, thought I’d share that with you.  Also, I’m working with a new study that is going to use a VA – encrypted VA laptop to record, and so there may be different mechanisms for recording to share with you. 

So, one of the things is that you – it’s going to be important to have a conversation with your IRB about what types of recorders are allowed at your site.  Many require encryption, password protection for it to be HIPAA compliant.  I know we have a single tape recorder that we’re allowed to use at the VA and it has all these protections.  These encrypted and password protected tape recorders tend to be about $350 to $450.  So, it’s not like going into Radio Shack and having a $15 tape recorder.  So, just – to be aware, they also on the other hand, are higher quality and I think one of the reasons our sound quality is so strong is we have very good tape recorders.  Also, tape recorders can be defined at VAs as ITs, so they may take a while.  There may be forms you need to complete – I know that at our site, there’s a six signature memo, so it’s very important that when you’re developing your study that you start to think about how can you purchase your audio recording device rather than simply assuming you can do it at the last minute.  In addition, just be aware that tape recorders are transportable devices.  They use a USB report, they have software because they have to download the interviews; and the memory card, even when you just delete your focus group there is sort of a ghost of your interview or your focus group left on the recorder and voice is an identifiable characteristic of participants.  So as a result, the tape recorders must be kept secure at all times.  So check with your IRB to see, for example, if you need to carry, if you need to go from building to building, if you need to carry the tape recorders in a locked briefcase – that’s what we do at our center.  Also, check with the IRB and also IT about downloading the digital file, just to be aware of some of the issues involved with audio recordings, and then it can all work very smoothly.

So let’s talk about the beginning of the focus group.  Once the focus group begins, the moderator – we’ll talk a little bit about these roles soon – introduces him or herself along with a note taker.  The introduction should include an explanation of the purpose of the study and the ground rules.  The purpose of the study is the reason why you’re bringing the participants together.  So in terms of the ground rules, there’s a script the moderator begins, and it needs to include these elements, and there may be others to include as well; once you’ve developed a script, you can often tailor it for future focus groups.  But it’s going to be important to tell the participants how long the discussion will last, a disclaimer to make it really clear that the discussion will be audio recorded, a statement that participation is voluntary.  In addition, you need to suggest participants write an identifier on a folded index card that they then place in front of themselves; you give them this card.  They can use a nickname when privacy is important.  They can use their own name.  Especially when participants know each other – we do focus groups with providers who all work together, and so, the likelihood is that no one will use the name Mickey Moue at that moment.  Or, oftentimes what I hear is – you can do what you’re most comfortable with, but they’ll have this card in front of them – that’s what the moderator will see and, as well as the other participants and the note taker.  

So continuing with the ground rules, you need to share that there are no right or wrong responses.  It’s important that everyone feel very comfortable and they’ve probably haven’t ever been in a focus group and they might be nervous about talking [inaudible] need to emphasize that the goal was to hear from everyone so that everyone can feel empowered.  Respectfulness is important; turn taking, that everyone has to take turns for those who – that might be a little bit more of a challenge.  It’s important to stay on topic.  Again, remember you only have an hour, hour and a half probably and you need to be able to stay focused on the questions.  You need to share with participants that they must maintain confidentiality about the discussion, and that’s important because we’re all accustomed to hearing conversations and going to parties or, you know, being in the hallway, and we don’t necessarily think that there’s a problem.  And sharing, you know, the experience that we had – but this is research and it’s – the rules are different.  So, be sure to share that with participants.  And then finally, any logistical information, always tell them where the restrooms are.  Tell them whether or not there’s going to be a break, share it whether or not it’s OK to go and have a cookie or go to where the food is if it’s elsewhere and come back, the rules of how you would like people to behave.

Let me share a little bit about the moderator.  The moderator is critical.  And it’s critical in my mind to have a trained moderator if you’re able to.  I encourage funded investigators to avoid saving money by having an untrained research assistant function as the moderator.  And I can share that trained moderators can be quite pricey, upwards of $250, sometimes $400 per focus group.  Not always; but they can run at that rate.  So, I think it’s important to realize just what they really bring to the table.

Also, work for consistency across moderators, across all the focus groups.  Again, my team is very much into consistency.  There’s an alternative philosophy that if you have different groups – men and women for example, you might choose to have a female moderator and a male moderator.  Typically what we’ll do, is we’ll find an individual who is likely going to be accepted by all of the individuals and stay consistent.  So there are different philosophies and that’s where I and my group tend to come down on that issue.

Also, the job of the moderator is to, as I’ve explained, introduce the focus group topic, establish the rules, have the participants introduce themselves, and that would be brief; ask the focus group script questions, that’s the bulk of everything, you know those questions are the most important and then draw the discussion to a close.  Moderators insist this is very difficult.  Must complete the questions on the script without running out of time.  They need to be able to come up with a way that they move – they transition from question so that each question gets adequate and, most likely, equal attention from the participants before moving forward.  And again, that’s why they – having a trained moderator is so important.  The moderator should remain neutral so as not to bias the discussion.  If somebody brought up an idea and the moderator said that’s a great idea, they might cause everybody else to say, oh that’s what they want to hear.  The moderator should always be encouraging, should try to enable people to feel very relaxed so that they can talk.  

Also it’s important to have techniques to further the discussion.  Moderators use silence after questions to encourage participants to talk.  They, for example, particularly the first question when nobody wants to talk, one of the things that they do is that they use silence; they fold their hands and they wait.  And it may only be 30 seconds but it seems agonizing and inevitably, somebody’s going to come in.  And if a group is reticent to talk, it might have to use silence after each question.  Oftentimes, after that, the group just sort of starts to take the task over themselves.  And as I mentioned, they certainly want to use pros to continue and enrich the conversation.  For non-talkers, the moderator should draw participants out by nicely asking them to share their perspective.  I’ll share that I was trained in focus group technique from Dr. Martha Terry here at the University of Pittsburg, and – who is a very seasoned moderator herself and has been doing this for decades – and she says that her approach is that she’ll give a lot of positives.  You know, encouraging eye contact with the person who has not yet said anything, nodding, and then eventually, directly ask the person.  And if the person doesn’t answer something, she may answer – do that one more time, maybe a third time, but then she’ll stop because she wants to make sure that that person feels comfortable; the door has been opened, they should feel welcomed to talk.  But you don’t want people to feel almost stuck in a conversation by the moderator.  

Then, moving to the very end, closing the session; once the final question has been asked and answered, the moderator informs the participants that the session is over, thanks them for their participation.  It’s really important to mention how they’ll be paid.  Here, we have a check for you, we’re going to be sending it in the mail, you need to go to the registrar, whatever that happens to be.  Welcome them to the food, make certain that they filled out the last survey – whatever it is you have to do.  And one thing that – to consider, if safety is an issue, make certain that individuals get to their car or whatever is needed to make sure that the participants are all taken care of.  

Now, it is important for a moderator to be able to deal with some personality issues.  For participants who dominate or behave inappropriately, and I can fortunately share that it’s – our experience is just almost always with someone who is so excited to talk that they’re kind of dominating the conversation.  But nonetheless, the moderator needs to be able to address that.  Again, turning to some of the suggestions that I learned from Dr. Martha Terry, she said that she had sort of a gradual approach.  So if somebody is talking or talking inappropriately, she said the first thing I’ll do is I’ll look at my watch, that I’ll withdraw eye contact.  I’ll physically turn my body and my chair away from the participant so that they start to get what it is they ought to do – that they’re doing something wrong.  And then, sort of as an escalation, and very calmly and very, you know, firm but politely, use language to discourage the inappropriate behavior such as, let’s hear from someone else, it may be time now to move on.  Remember that we need to be respectful of everyone or we need to make sure that everyone is heard and while I don’t have personal experience with it I’m so happy to say, in having to confront somebody because of a problem.  And I think it is relatively rare.  The literature indicates that what you want to do is, you want to allow the person to share their – the reason they may be upset, but you also want to make sure you don’t personalize it, by focusing on the topic.  You appear to feel strongly about this, can you share what it is about the topic that causes you to feel this way.  So something to help – allowing them space and diffusing it at the same time.  

Now let’s move on to the note taker. The note taker is a member of the team, trained to record content and non-verbal behavior.  The note taker needs to be very well trained.  It’s a very hard job and sometimes I find in teams that thought is well the – you know, that’s a leftover RA that he or she can go and take the notes.  So, let me share all of the things that the note taker needs to do and how important this person is.  So, the note taker first of all, develops the seating chart identifying the participants by their selective IDs, whether it’s their names or whatever they might be; I’ve drawn one from the web because ours are a little bit messy.  But, it’s interesting in looking at this diagram, if you notice, the moderator is actually at the head of the table or the moderator – by the way, facilitator is another term for moderator that’s basically commensurate, or in this case the observer, also the name for the note taker, is away from the table.  And it’s ideal for the observer to be as, you know, to blend into the background as much as possible.  If it’s one of these rooms where the table has absorbed every bit of space, I’d run into that situation before, that you can have the note taker sit at the table but just try to be as, you know, non-obtrusive as possible.  The note taker writes notes as a backup, in case the audio recorder malfunctions; this is probably less of an issue these days.  Particularly if you have two, but it certainly is helpful under those, you know, circumstances where something like that happens.  Also, if he or she records the first few words of response to help the transcriptionist link the statements to an ID.  So, basically, imagine that you have a focus group that is divided by gender, where you have males and females.  It’s already hard to tell the differences between voices, but sometimes the voices may be very similar.  And the note taker – the transcriptionist is going to be listening to the tape and having a hard time most likely.  So, what a good note taker does is that they hear a statement, so for example, the person identified as R3 says, “And my mom said to me,” – if they just jot down the first few words and then that ID, and then the next person who talks, the next and the next; you may not be able to get every single phrase and there’ll be people talking over each other and there’ll be hums, etcetera if it’s yeah, yeah, it might – all of the group – just ways that the note taker can help the transcriptionist to effectively capture that discussion.  Also, the note taker needs to capture emotional reaction and body language during the discussion.  Especially the things that cannot be captured on a tape recorder.  So, head nodding, shaking of heads, anger – certainly if anyone is laughing to make sure it’s clear on the tape it’s laughter.  If anybody is getting teary eyed, that certainly wouldn’t be being captured.  You need to try to record anything that you possibly can; sarcasm might not sound like sarcasm, so, those are all the various aspects that the note taker provides to the study. 

At the very end of the study, there is the debriefing period.  This begins after the discussion, once the participants have left.  It also involves the moderator and the note taker.  Certainly, other team members can come in as well, and the goal is to debrief on the overall impression about the focus group.  This will include discussions about the interpersonal dynamics, something about the non-verbal communication, those aspects not captured on the tape.  The goal is also to get a sense of summarizing what kind of topics are coming out; and in this debriefing process, it’s the beginning of the analytic work.  It’s a little bit like a film critic before he or she writes a review of a film, might be talking to a friend, maybe even another critic, somebody who – or just friends.  And starting to think about, what do I really think about this film, what are the strengths, the weaknesses – it’s the same kind of situation here.  And you want to record this.  You want to record it perhaps on the tape recorder as well as notes, or you can think about them as field notes in terms of writing down, you know, the experiences.  So this will become part of the text of the focus group.  

So, we’ve completed the focus group, introducing the different roles, the different ways of approaching and setting up and then eventually concluding in addition to the mechanics of informed consent and tape recorders, what I’d like to do now is to move into the code book construction and coding process.  Because part of that – and I know that’s something that’s very important to many people who are doing qualitative research, is thinking about transcription.  Coding requires a high quality verbatim transcription of the focus group.  Transcription is expensive.  It can range from $15 to $40 an hour – 30’s a little high – but 35, depending on the level of experience and I can share that the national reaction is that I’ll go with the $15 person but, a really – a good transcriptionist knows what he or she is doing, they often transcribe much faster because – you’re paying by the hour.  So sometimes, I would say, that a trained transcriptionist is actually a better deal.  And certainly the output is better.  There are professional services; they often require going outside of the VA.  I know that we know – we always do our own transcripts, but we are contracting with a university center here for some transcriptions for a new study and – I know there’s IRB protocols that need to be accepted, there’s a lot of data safety issues, there is a need to deal with contract language, so it’s – you know, there’s a lot that’s involved with trying to find a service.  But of course, if you can find a good service, it certainly is helpful.  Focus groups, just to share with you, are more difficult to transcribe because there are multiple participants to distinguish between.  The transcriber must incorporate the note taker’s notes to help determine which participant is talking.  So, it’s really, as I said, it’s good to use a highly trained person.  Also an opportunity is to use the moderator or note taker, if they’re trained, or worked – try to train that person.  As the investigator, it’s important to do quality controls, listen to the tapes to read it against the transcript, just to make sure you have high quality transcription – transcripts to work with.

Now there’s a great deal that can be said about qualitative coding and qualitative methodology and I wanted to just sort of baby step our way into some approaches to qualitative analysis.  I can share that there have been some other archived cyber seminars if you’re interested; I’ve done a couple in the past, one on coding, one on Atlas that – the coding one is actually tied to one on Atlas TI which I’ll talk about briefly here.  I also know that many of my colleagues, Jane Forman, I believe Barbara Bakhour, Allison Hamilton, have also done cyber seminars and qualitative research.  And so, that’s a real repository that can help to supplement the very – sort of a brief discussion that I can provide here.  But let me share that all qualitative coding must extend from a qualitative methodology.  It helps to frame how it is you collect the data and analyze it.  And there’s multiple types of methodologies you can use.  You may have heard of grounded theory – content analysis can work fairly well for focus groups.  There’s a wonderful piece by Margarete Sandelowski on descriptive qualitative analysis.  I would recommend – I am very fond of the editing style, which is grounded theory-esque.  Written and developed by Crabtree and Miller and it’s very grounded in medical research and I found it very helpful.  So, just think about the fact that having a methodology would be important.  In terms of the analysis, everyone should begin by reading the transcripts.  The whole team should read the transcripts.  Really get a feel for the data, and the interpretations really must be grounded in the – in those texts.  It’s important to address qualitative bias.  You would never want to pursue a certain hypothesis that you have that’s not born out in the next.  So, always allow yourself to be grounded in what the participants are saying.

Now, there are two different approaches for doing a final analysis for focus groups.  And I wanted to share both with you.  I’ll share that I and my team always do the latter, but I want to share with you that there’s another way as well.  One’s a summary report and the second one is a qualitative analysis.  The summary report is basically from the debriefing and all the documents that you have; create a document describing each focus group.  So, you know, you’re debriefing, you’ve experienced that one focus group, you talk about it, you have the transcript for that focus group, you’ve kind of come up with a summary document for that one focus group.  And then do it for every single one.  With a goal then of synthesizing these summaries so that you come up with one overarching summary of the focus group – very specific to the research questions and then also including insightful quotations.  So the goal of the summary report is to convey information in a condensed format rather than having something very in depth, very, very specific defined gains with a goal of moving toward publication.  This is more – let’s say you’re doing a quality improvement project, and you want to know, should we move forward or not, and we will need to do this in a fairly quick amount of time.  The summary report may be the way to go.  Another one is a more traditional qualitative analysis.  I can encourage that if there are members of the team who are trained in qualitative research or you have a consultant, have those individuals really be the ones to focus on the code book construction; although, other team members can always give input.  The goal is to come up with a final code book that contains inclusions, exclusion criteria for each of the codes you have, with clear and borderline quotations.  You should have a definition – know what it is, is not, and some quotations.  When we do a code book, it can extend five, up to 50, 60 pages.  And we do a very fine grade coding.  So I wouldn’t say, you know, 10 pages maybe, 10, 20, depending on the sheer number of codes that you have.  But you want to be really very clear, because it’s a roadmap for excellent coding, and that’s why you need a good code book.

Now there’s so much to say about coding – developing a code book.  I’m going to provide one slide; recognize that there are limitations of a single slide, and I want to share the approach that my and my qualitative research core are increasingly moving to – and are really enjoying – this particular approach, which is a question specific format.  So let me, and let me share, I’m choosing the easiest example as I – that I can – I have six on a slide, but just recognize that.  Examine – so a question specific code book examines the question and uses its structure and the answers given by the participants to organize the coding.  So, here we have a very small question from a focus group in the VA, talking about CPRS notes and there was a question, is anything missing in the patient’s notes.  And, automatically before reading the transcripts, anyone can look and say well they’re likely to say yes, they might say I don’t know, and they may be likely to say no.  So, I’m only sharing with you the yes portion.  So we develop that structure and then developing a code book, we read all the transcripts.  And – this is a very small question – so whatever they said that was specific, like yes, what’s missing are related diagnoses such as MST, how the diagnosis came about in the first place, who diagnosed and why, anything that they might have said, we add down as a topic or what I’ll call is a theme under that structure.  And we would do the same thing with don’t know, and the same thing with no, which would extend beyond this one slide.  So the codes are organized in the categories in terms of how they answered the question, and then are broken down into specific themes.  So that’s one approach that almost everybody can take with their focus group, presuming that there’s a degree of stability at the question level.

We also – and it also allows for fine-grain coding at the level of the question, so you really know your data.  In a way, that’s different than a summary report.  Summary reports can be great, they’re just – they just function differently.  We then also combine with what I call global, or sometimes they call floater codes that capture themes across the focus groups.  So for example, let’s imagine I have a reason that any time someone talks about suicide, in any way, I want to capture it.  It’s not question specific, it comes up now and then, and I capture it using the software program that I do – and I now know, one of the five focus groups that we had, every time someone talked about suicide, I have a code called discussion of suicide.  And I capture that quotation.  So what we do, is we do global codes that we think are important to capture and that we – that emerge through the reading process, and then we have these question specific, structured by the question, and then filled in based on the topics that the patients – the participants share with us.  So that’s our approach.  And again, I recognize that that’s limited, but it’s what I can share in the amount of time that I have.  What also aids us are software packages; we use Atlas TI.  It allows for the conversion of themes into spreadsheet formats and the software also captures all the quotations associated with a code.  So we can walk away having all that structure, having it in a database, knowing exactly what the codes are at the click of a button.  And at that point, you really – with all that technology, you’re ready for the coding.  And the coding classically involves two coders.  You should have two coders for the focus group with determination of numbers of double – of cases to double code.  I – is it the large, let’s say 15 cases, so 50% that we’re coding perhaps.  Especially with focus groups, the N is just usually not very high; smaller studies we’ll usually just double code everything because again, the N is small.  So we have three focus groups – I need both coders reading them.  So, the process then is the coders independently code each focus group.  We then have them discuss each code and if they have differences, they adjudicate those differences.  There’s a master file; and so if they have all the codes that they agree on and then they say, I say yes, I say no, or I say that this is this thing, I say no and then they agree, they add that into the master file so that we have a master file for our final analysis.  With those separate coding sheets that they brought to the table, they don’t change those, and those are used to calculate inter-coder reliability.  So, this qualitative approach is certainly much more time consuming than the summary reports.  You have a much greater handle on your data, which you may or may not need, depending on your project, and it certainly is something that you can move and use toward peer reviewed publication.  OK, so that’s a little bit – just a small amount of information about coding.

So the last thing I’d like to do is to just briefly mention virtual focus groups.  I see this as an important type of data collection for the future, although I don’t know that it’s as feasible at the current moment.  So, the idea is to have a new approach to the focus groups which uses computer technology.  So, this will include – can include online sort of blackboards where individuals can come and fill in information that is not occurring at the same time.  But, then a moderator can eventually come back and see the input from the group.  You can also have chat sessions which are occurring live, so please come at noon and we’re going to chat until one.  And so then you actually have the interaction and again, obviously, you don’t have the human element of seeing your participants. What I think is particularly exciting is the prospect of having teleconferencing; in the literature they mostly focus on Skype, and we need to develop our approach, our unique approach within the VA, that enables us to have access to hard to reach populations such as providers, administrators who would be fabulous in a focus group but have limited abilities to get away from their office or from the clinic, to participate.  Almost everybody has a computer that they can sit in front of.  Limitations as far as I know, even using the Skype technology, you really need to see the entire group to really get to the point where you can replicate that focus group experience.  Also the question is at the current time, it’s unclear how feasible it is right now within the VA; there may be investigators using teleconferencing.  There are issues related to data safety and confidentiality that certainly are not fully clear to myself and that need to be worked out.  But I think it’s important to note that these are forms of data collection for focus groups that may become important in the future.  So in conclusion, focus groups can be effective ways to capture viewpoints from participants.  Not all data is captured using focus groups; however, when well designed and analyzed, focus groups can provide important insights into a subject.  OK, thank you.

MODERATOR:  Are you ready for some questions now?

DR. ZICKMUND:  Indeed.

MODERATOR:  Fantastic.  I know a lot of our audience just only scheduled one hour to join us for the session, so I know some people will need to drop off now.  Just to let the audience know, we are recording the session and if you do miss anything here, we will have this available on our archive and we will be sending that out to everyone within the next day or two.  So I am just going to look through these questions here.  I’m just trying to figure out where we started at.

OK.  The first question we received: can you focus on the difference between focus group questions and probes?

DR. ZICKMUND:  Sure, sure.  So a focus – and pardon me, I’m starting to lose my voice – a focus group question is what you would look at if you looked at a basic script.  So, imagine you have five main questions.  One of the questions, let’s see – by the – tell me how you feel related [coughs] – I am so sorry.  Tell me how you feel when your provider is giving you advice related to your diabetes.  And, individuals can talk about that question.  And then, maybe somebody says something and then the interviewer – well I don’t know, I hate to talk to my doctor about any of this.  Well that’s interesting, can you tell me more – that’s a fairly standard probe – can you tell me more about that.  What is it that makes you feel like you really don’t want to talk to your doctor about this.  Can anyone share – are they having a similar kind of experience when they talk to their doctors.  So the difference is one is a question; and it’s what you want answered.  The probes are ways of being able to solicit information, extend, bring other people in, and sometimes there may be – I want to know what they think about when the doctor asks, you know, asks information related to diabetes, and then the investigator stated I want to know what they hate the most.  And then there might be a content probe.  This is the difference between – can you share more with me, and what is it that you really prefer your doctor not to say.  So that may be a probe that may or may not – there may or may not be enough time to work that in, but the investigator thinks that would be an interesting, you know, augmentation to that main question.  

MODERATOR:  OK. Great, thank you.  The next question we have here: if VA doesn’t allow providers to be compensated with money, how would one recruit/entice and otherwise respect time and effort?

DR. ZICKMUND:  That’s a very good question.  So, I really do think that the best way is to work with leadership and a champion – and a champion often can be somebody who participants – of participants – providers look up to.  You know, this is sort of the person we all really like in our clinic and if he or she says you know, this is really important, they’re likely to go.  I think also, your providers are very concerned about important research and making sure that they improve patient care.  So, I think often, make it very clear about what this – the reasons behind the research and what is – what are the important elements of good that can come from their one hour.  And then making it work for them.  And that’s why figuring out is it your individual site, if you can do like a morning meeting or, it’s wonderful to work with leadership, you always have a mandatory meeting on Monday at 10 and the leader says I’m going to give you that meeting time, and you bring the food and I’ll encourage people to go and we’ll turn it into a focus group – the thing is that of course, it needs to be a smaller group.  You can’t have 30 people show up.  But, working within the infrastructure of the clinic and making connection is the best way to really bring participants in.  I have heard people argue outside the VA, that by the time you have enticed a provider, given the level of their salary, you may actually have also reached the level of coercion.  So, it probably – the lessons learned within the VA are probably important even outside.

MODERATOR:  OK great, thank you.  The next question was sent in by several different people.  How does one become a trained moderator?

DR. ZICKMUND:  [Laughs].  Well, for example, here, at University of Pittsburgh, there’s a certificate course in focus group moderation.  And it’s the one that I took with Dr. Martha Terry.  So, that was the best way for me to be trained, and she trained us.  There probably are also courses – I’m fortunate in that many of the people who work for me have had that experience.  Or if I bring someone on and I’m interested in having them become a moderator, I can send them to that course.  So I – you know, I don’t have to worry as much about how to do it outside this kind of wonderful opportunity that we have here.  There may be individual courses that you can look at for the – on the web.  It’s possible, particularly if you have a very talented interviewer, to try to do a situation where they sit in it – focus groups – if you have a large enough team or if you have people who are already focus group moderators, they watch, and then eventually they will run it and the person who was the moderator is the person watching.  You can always baby step into training if you don’t have a more formalized approach.  What certainly is tricky is if you don’t have an infrastructure and you don’t have a class.  But, I would encourage people to look – if they’re connected to a university, and oftentimes, it’s in the nursing schools, public health, psychology, social work, qualitative research sort of comes up in odd places.  I don’t want to say odd – different.  Not that it’s always part of the standard medical curriculum.  So you need to be resourceful, so hopefully that can work.

MODERATOR:  Great, thank you.  The next question we have here, can you require confidentiality among participants or can you only simply request confidentiality?  If you can require it, how do you go about doing that?

DR. ZICKMUND:  Right.

MODERATOR:  We’ve had participants ask how you as a moderator can enforce confidentiality and my thought has always been that you cannot. 

DR. ZICKMUND:  I agree.  There is no way you can enforce, probably, confidentiality.  When an individual goes home and talks to a family member, you know, we are free agents.  I think the best thing is to impress upon them the, you know, the implications that it might have.  Well, maybe we’re all strangers here, but I think it’s a show of respect for other individuals to keep the stories – their feelings secure enough to share, and quiet, because that’s really showing that you respect that person.  So, I don’t think you can require it, no.  Inspire.

MODERATOR:  OK.  Thank you.  And next here, can you go back to Slide 47?  We’ve got a couple people looking for references on the citations.

DR. ZICKMUND:  Oh, sure.  And I can also – 47 sure – 

MODERATOR:  Yeah, sorry.  You’re at the beginning.

DR. ZICKMUND:  That’s ok, that’s ok.  Oh certainly.  I can share, and maybe there’s something we can do for this cyber seminar, I have an extensive reading list that I have posted on a prior cyber seminar and maybe we can talk about whether or not I can upload that.  Certainly, if you type in grounded theory or if you type most of these into Google, it will give some references, but I may be able to make sure that people have access to it, fairly extensive literature – bibliography that I put together previously for this.

MODERATOR:  OK.  Great.  Thank you.  Next question here:  you mentioned that thematic saturation is the main goal for qualitative analyses, I thought it depended on the approach.  For example, with IPA, I thought the goal was to come up with as many themes as possible; can you please clarify?

DR. ZICKMUND:  OK.  So, I’m addressing the question of how do you know you have enough, and, so it’s a sample size question.  So, what you would want to do in terms of just thinking through, how do I know that I have enough focus groups and discussions and that’s a frequent question that I’m asked, you want to be able to have an idea that you’ve got a chance to know that you have all of those different viewpoints.  Because, there may be differences in how different methods look at things, but I think all of them would want to make sure that you didn’t stop the discussion before you hurt it all.  So, this is just – one is more of a methodological issue and thematic saturation is when is it enough.  

MODERATOR:  Great.  Thank you.  The next question is: what do you think of video taping sessions?

DR. ZICKMUND:  So, before I was formally trained, I thought video taping seemed like a fabulous, fabulous idea.  And in terms of data analysis, which is usually where my head is – sort of, I come out of rhetoric, which is a very, you know, textually based field, it’s a wonderful thing.  That they not only do – potentially read into, you know, what does that voice mean, but we also have an opportunity to see the actual body language.  Within focus group – within the literature of focus groups, the concern is that, if you video tape, that you could create one of those chilling effects.  That people don’t want to be video taped.  That – and that’s why in marketing, they had those mirrors so they could actually look at them without individuals feeling...  We don’t really do that in health services, but, where individuals feel that they – they don’t realize that they’re being looked at so they can feel comfortable.  It’s just they can’t tell what that mirror might be.  So, the idea is the tape recorder is a – really is obstructive as you want to be and it’s most effective not to use the video tape.  I also don’t know what the IRB issues might be if you’re video taping.  That’s the basic idea.  

MODERATOR:  Yes.  Thank you.  The next question: are there any good references – articles or books – on how to do focus groups?

DR. ZICKMUND:  Sure.  So, I’ll make certain that the bibliography that I have, I can share and put that on so people can definitively know.  

MODERATOR:  Great.  Thank you.  The next question here: excellent conference, but what if you are starting out, don’t have the resources for multiple staff.  Is it absolutely necessary to have separate note taker, two coders, etcetera.  I understand this is much more robust but when not feasible, can such – could such research be done?

DR. ZICKMUND:  Oh, absolutely.  And that’s what I’m trying to describe what the ideal ought to be, and that’s when I mentioned you can go down to two.  I mean, the limitation you might run into is that if you do a focus group with a goal of publishing it and you go through all the methods that you pursued, it – you know, there may be some criticism and concerns.  I think it also, you need to note in the limitations sections that, you know, there wasn’t a note taker, they – the moderator – you know, whatever stuff you didn’t enough resources to be able to meet.  I think, you know, advancing important research is important and oftentimes we start out without much of a budget and then we grow as investigators and you need to do whatever you need to do and as long as there’s transparency and a recognition of, you know, limitations in terms of the ideal, I think people should push forward.  But also look for pilot studies; sometimes there are funds one can tap into just to help out.

MODERATOR:  Great.  Thank you.  What are your thoughts regarding participants taking notes during a focus group?

DR. ZICKMUND:  Personally, I’m not a very directive person in the focus group.  So, I will emphasize, you know, treating people well and decorum is very important, being a responsive – typically people don’t take notes, but I think if somebody was taking notes, I don’t – it’s nothing that I felt that I would need to critique them, certainly not in the beginning of a – or in the middle of a focus group.  If they were walking out, I might emphasize in the closing, reiterate, you know, that it’s very important that we you know, maintain confidentiality so that maybe that person’s just trying to jog their own memory and they’re not going to go and share those notes with anybody.  But, you know, I appreciate peoples’ time and I want them to have a good experience and to feel good and that’s sort of – that’s where I personally am coming from.

MODERATOR:  Great.  Thank you.  The next question here: we use a professional transcription service outside the VA.  Would you recommend sending the room diagram and notes taken by the note taker to the transcription company along with the audio files, or would you reconcile and attach participant codes when the transcripts come back?

DR. ZICKMUND:  You know, I’ve never worked with – using a service is new, and even when we do, this will be for interviews.  So, I’ve always known the transcriptionist for our focus groups.  I never had a situation where I couldn’t sit down with a transcriptionist in advance.  It’s also important, I don’t think I mentioned that there are rules you need to give them, such as I want to know when people are laughing or they’re crying, you know, I don’t want you to give me a dot, dot, dot where every dot is one sentence of pausing.  You know, you need to give them information.  I think what I would want to do is to provide the diagram – particularly the notes from the note taker related to who said what, in advance.  Because, you know, somebody transcribing a focus group is going to try to have – usually it’s in like an R, R1, R2 – they’re going to try to match up those voices, and why not give them the most information as possible and then you can do spot checking on the other end.  Trying – not providing it to them and then going back, I think there may be assumptions that the note taker, rather the transcription is using that might be different from the research team.  And at least, I think everything should be incorporated, at least the first time, in a consistent fashion.  So, I’d give them everything I can, and I’d try to be able to talk to them on the phone – if not the transcriptionist then at least a supervisor. 

MODERATOR:  Great, thank you.  And Susan I just wanted to check in; we’re just past a quarter after the top of the hour, I just want to check to see if you are still OK staying for more questions, or if you want to wrap things up and handle some of these offline.

DR. ZICKMUND:  I’m fine.  I’m fine.

MODERATOR:  [Laughs].  OK, sounds good.  The next question here: as far as analysis, is there more or less to be gained by using an exploratory approach to coding versus looking for predetermined or defined themes.

DR. ZICKMUND:  So, with qualitative coding – and I want to make sure that I didn’t say something that indicated that I was in agreement with predefined – unless you’re using something that Crabtree and Miller described as a template coding where you’re coming in with your codes, you always want the codes to be iteratively emerging from your discourse.  But with the one thing that’s slightly different, is now that we use this code – this question specific structure, there’s structure around the codes or the themes – you know, and the way that people answer the questions.  But, the structure doesn’t determine what those themes ought to be.  So somebody can say anything, but we’re not going to walk in presuming that they will say it, other than if I say is this major or minor, I’m likely to organize things in advance: major and minor.  And then, if nobody says anything about minor, then I might say you know, not mentioned or NA.  So, one is structure versus the iterative approach that always allows the themes – the topics, to emerge from the discourse, so.  And in that respect we’re always exploratory.  I mean, there are differences between the different qualitative methods but – unless I’m missing something other than the templates, I think they are unified in that we ground ourselves in that discourse and that determines what we code and what we present.  

MODERATOR:  OK.  Thank you.  The next question here: our research team is developing focus groups with an aim of understanding more about the decision making within military units.  Would you recommend having service members of the same unit in different groups or in the same group?  Can you speak to the pros and cons of each option.

DR. ZICKMUND:  OK.  So, part of it may be me trying to envision – let’s say if that same unit that are going to know each other – so I guess it, part of it has to do with the question.  If you – if, for example, this is the larger discussion here, if this was a gender based comparison, you know, you would certainly want to make sure that the women are with the women and the men are with the men, regardless of what that unit happens to be.  I also know that within the unit, there were for example officers and enlistees.  Let’s imagine there are you know, struggles or power issues and there may not be but I’m just envisioning it – or there’s a leadership and those who fall underneath the leadership – again, if those are sensitive issues that might cause people to not want to talk, then separating them out may make a lot of sense.  I think then it comes down to the investigators trying to understand what you’re trying to learn; if what you really think is important is that there’s something about the cohesiveness of a unit that will enable you to really answer your question, then keeping a unit together makes a lot of sense.  If what you’re going to do is probe the issues, let’s imagine like military sexual trauma that you imagine or may be sort of a chilling effect, by knowing that there is the same person from the same unit, then it may make sense to separate it.  So I think I struggle just because I don’t know enough about that – the goals of the research.  You can certainly know people in a discussion and it can still work.  You just need to decide what it is that you’re trying best to capture and do you want strangers for that or do you want people who know each other.

MODERATOR:  Great.  Thank you.  The next question: what are your recommendations on training or preparing your note taker for their role?

DR. ZICKMUND:  Sure.  So the ideal situation that I feel very happy about it – that I often get people who are trained, [laughs], that helps, again, having a fabulous class where individuals come with a knowledge of how to do note taking is best.  If that’s not the case, the same type of scenario that I used for the moderator where you have a note taker and they train, and their person sits in and then they sort of, you know, slowly the person takes over the responsibility – that’s the best, that’s always the best way to train.  If that’s not the case, hopefully the investigator has experience or can gain experience in how to do note taking.  You might even see if a person has different levels of experience outside of focus groups, so for example, I had spent years in high school as a debater and you have to learn how to flow – very – arguments are spoken very quickly.  So, that kind of training – finding somebody who has been in debate, who is accustomed to being able to capture information enables you to have someone with a leg up in terms of being trained.  And then you’d need to help refine aspects such as, you know, how do you catch the elements of the conversation that’s being – you know, the first part of the sentence.  You know, how do you make sure you are sensitive to what’s going on in the room.  That the debriefing – that this is a person who analytically is attuned to what’s happening – so, I think that’s what I would do if I didn’t have any training mechanism at all. 

MODERATOR:  Great.  Thank you.  We have a comment here from someone following up on the focus group moderator training.  They believe that VHA’s National Center for Organization Development may offer focus group moderator training.  So that would be something to check up on.  And the next question we have here: what is your timeline we can expect to get approval from IRB?

DR. ZICKMUND:  [Laughs].  Well, it depends on your own IRBs, and that deviates widely.  But let’s assume that somebody knows – let’s imagine it takes a month at a given site at that person’s site.  I think if you address the security issues in advance, that your focus groups don’t have to be that sensitive.  It’s more that if you don’t know that you have to have the right – you don’t know what the right tape recorder is and you don’t include it or have a plan in terms of data safety, they want it to be – you know, put in a briefcase – you don’t have the provisions for that.  So I think being – having a conversation with the IRB chair – whoever is the representative of the IRB who intersects with investigators – being very clear on what do I need to do for focus groups, I would hope at this point, the methodology is common enough that they might have expanded the path and can give some pretty concrete advice on what we need you to do, X, Y and Z.  And, as long as it’s part of the protocol, I don’t think it has to necessarily extend the length of getting IRB at that particular site. 

MODERATOR:  Great.  Thank you.  The next question here: you did not mention the requirement to get OMB approval per the paper reduction act; have you gone through that process?

DR. ZICKMUND:  [Laughs].  Yes, I have.  I’ve had some merit reviews – there was one short term project that we were requested to keep the number of participants below nine, and we did – that wasn’t for focus groups, that was for interviews.  On other occasions, I’ve requested, I believe the language is an exemption from OMB which enables a person to go beyond the nine and use certain kinds of surveys – I think there are other elements as well that I think I may not be thinking about quite at the moment.  So I – for the most part, the recent work has not been impacted by OMB other than, you know, going through multiple drafts to request an exemption.  So, that’s – those are my experiences.  I’ve never directly applied for OMB and went through that process, so. 

MODERATOR:  Great.  Thank you.  The next question here: would the quantitative survey after the focus group be a good opportunity to identify any potential bias?  For example, the patient satisfaction or patient centered care focus groups, and previous experiences patients may have that would influence their perspective.

DR. ZICKMUND:  So, I think using mixed message designs where you have both the qualitative and the quantitative is a great idea.  The one thing I’ll share is that qualitative research is never designed to be generalizable; it’s designed to provide insights into unique experiences.  And, so, it’s – I just would use different language then.  Will it show the biases – because bias is usually a way of describing something that’s – you know, that’s wrong or that takes this particular data away from being generalizable.  I think it provides unique insights; I think it’s something that you could engage the survey and say something about the survey in the context of a larger focus group.  The one thing I will share is that linking an individual’s – you know, individuals’ statements in a focus group and trying to link it back to a survey data is very difficult.  Just because you’re never quite sure that when your transcriber writes down R1 and R1 is the person who filled out the form who had, you know, certain kind of results – you’re never totally positive, so that linkage is slippery.  I mean, you can try and note it in the limitations.  You’re more likely to be able to say something about the focus group in general and then look at the surveys that came from that group overall.  So, just the answers.  I think it’s a great idea; I think that – just, the focus on bias is just something different than I think – as opposed to the experiences and then, making certain that there’s not a hope or presumption that you can link.  

MODERATOR:  Great, thank you.  Next question here: in relation to descriptive quality studies, you mentioned an author whose first name is Margarete, but I did not –

DR. ZICKMUND:  Sandel –

Q: – catch her last name.  [Would you] re-say it.

DR. ZICKMUND:  And I’ll – her work is on the sheets that I’ll share with you.  Margarete Sandelowski.  She is probably, I think, one of the more – perhaps the most innovative and interesting methodologist within our larger realm of clinical medicine, health services research, there’s many things that she does very, very nicely.  And then there’s this piece on descriptive qualitative research that I’ve always enjoyed and so I encourage you to read it and I’ll put it on that bibliography I share.  

MODERATOR:  Great.  Thank you.  And, the last question: where can we find the VA consent form for recording voice?  Are there guidelines for when it’s required?

DR. ZICKMUND:  Absolutely.  I should have brought up the specific language of what that is.  You know what I can do is, I can also include that on to the bibliography as just sort of – under an asterisk – so it’s form ten something, something.  Those numbers never completely stay with me.  I would imagine that asking a member of the IRB – because my sense is that this is a national form.  And, even if – whether or not it’s required at that individual VA, I would imagine somebody who works in the IRB would be very aware of what that form meant.  My understanding is that I thought nationally, it was required for anybody that included capturing participants’ voice.  But it is pulled basically with a consent for form ten something, something; consent for recording of voice and photograph or photographic voice.  

MODERATOR:  Fantastic.  And Susan, if you are able to get that bibliography, the information you’re putting together over within the next – before like 10 or so tomorrow morning – I can include that in the archive notice that I’ll send out to everyone –

DR. ZICKMUND:  OK.

MODERATOR:  – so will get that information with the archive notice.

DR. ZICKMUND:  Wonderful.  I’d be happy to. 

MODERATOR:  Fantastic.  And that is all of the questions that we have.  I really want to thank the 144 audience members who have stuck with us for this hour and a half.  We really appreciate, and we really admire your desire to get as much information as possible.  Thank you for joining us.

DR. ZICKMUND:  Thanks everyone.

MODERATOR:  Susan, thank you so much for presenting for us today.  We are thrilled that this series is starting back up and obviously, you saw our audience also is very excited, so. 

DR. ZICKMUND:  [Laughs].

MODERATOR:  And to our audience, this is a series – we will be running two or three times a year, so we will be sending you registration information for our next session as soon as we get that on the catalogue.  It will be probably two or three months down the road, but we will be getting another one scheduled at some point probably in late winter, early spring, summer, somewhere around there.

DR. ZICKMUND:  OK.

MODERATOR:  Sounds good.  Susan, thank you very much for putting this together and presenting for us today; we really appreciate everything you put into this.  And –

DR. ZICKMUND:  My pleasure.

MODERATOR:  Great.  And for audience, as you are leaving the webinar today, you will be prompted with a feedback form.  If you could take a few moments and fill that out, we will very much appreciate it.  We take that into consideration for our current and future cyber seminars.  Thank you everyone for joining us today and this concludes today’s HSR&D cyber seminar.  Thank you.

DR. ZICKMUND:  Thanks everyone.  Bye.

MODERATOR:  Thanks.

[End of Recording]

Page 1 of 24

Page 24 of 24