**Transcript of Cyberseminar**

**Patient-aligned Care Team Demonstration Labs**

**Making sense of the ideal and the real in PACT implementation using qualitative research Part II: The importance of being there.**

**Presenters: Samantha Solimeo, Jane Forman, Molly Harrod, Anais Tuepker, Sarah Ono**

**October 16, 2013**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact:*

 HYPERLINK "mailto:" ***Anais.Tuepker@va.gov***

*Sarah.Ono@va.gov*

*Samantha.Solimeo@va.gov***

*Molly.Harrod@va.gov***

*Jane.Forman@va.gov*

*Idamay.curtis@va.gov*

Moderator:And, I do realize it’s about a minute before the top of the hour, but as I said, we do have five speakers so it’s going to take me a moment to introduce them all. So, I’m going to go ahead and just get started now. We have Dr. Samantha Solimeo and she’s a medical anthropologist in VISN 23 PACT Demo Lab, Center for Comprehensive Access and Delivery Research and Evaluation – known as CADRE. And, that’s at the Iowa City VA Healthcare System. Joining her we have Anais Tuepker, a Core Investigator, Health Services R&D at the Portland VAMC, and Assistant Professor in the Division of General Internal Medicine at Oregon Health & Sciences University. We also have Dr. Jane Forman, a Research Scientist and Director of Qualitative/Mixed Methods Core at the VA Center for Clinical Management Research, and the Qualitative Evaluation Group at the VISN 11 PACT Demo Lab in VA Ann Arbor Healthcare System. Also, we have Dr. Molly Harrod, a Research Scientist for the VA Center for Clinical Management Research and Qualitative Evaluation Group also in VISN 11 at the PACT Demo Lab also in VA Ann Arbor Healthcare System. And finally, we have Dr. Sara Ono, Qualitative Core Director at the Center for Comprehensive Access and Delivery Research and Evaluation (CADRE) and VISN 23 PACT Demo Lab Formative Evaluation Core. 

So, I’d like to very much thank all of our presenters for lending their expertise to the field today. And with that, I would like to turn it over to you ladies. 

Samantha Solimeo:Thank you Molly. In part one of our cyber seminar last month we addressed mythological diversity and the role of qualitative investigators in developing research operations, partnerships. We talked about some examples of the valuation methodology and collaboration from VISN 22 and 4. Today we’re going to demonstrate sort of the role and importance of observational techniques in in-person interviews to data collection and collaboration. 

In our last session, I post the question how do we understand why some PACTs are more successful than others. For qualitative investigators, we build our understanding through listening, observing, reading the literature, training materials, and team reports and then by analyzing those data. As I noted last time, we like to say we draw our insight from the field. And today, I want to use the term field in two slightly different ways. First, we build understanding by regarding VA staff adapting and practicing PACT in the field as experts. And second, we build understanding and collaborative relationships by actually going out to the field in person. I think some qualitative methods such as open-ended surveys or interviews are really amendable to collecting remotely. There’s a great deal to be learned by actually going there. 

Moderator:I’m sorry for interrupting. Can I ask you to just speak up a little bit louder? 

Samantha Solimeo:Yes, I’m sorry. 

Moderator: Thank you. 

Samantha Solimeo:It wouldn't be a talk unless I had to speak louder. Sorry everyone. So one quick example: In VISN 23 our lab conducted focus groups with team members by role. We could have certainly done this over the phone but we opted to conduct theses at PACT training events where participants were already away from their clinics to focus on PACT. I think that had the advantage of both better response rates and fewer cancelations and interruptions. But as you can see from their comments on those sessions, collecting these data face-to-face enabled us to build repore with the participants. We also demonstrated our commitment to the work by just showing up. We created a safe, confidential environment for people to discuss implementation, and in a sense we fostered a sense of community. 

Now we call it the PACT Transformation for a reason. VA staff are being asked to adapt new clinical processes as well as a population based approach to panel management. I think these changes in turn affect the organizational culture and professional identity of the staff involved. All of that is happening in a broader context of accountability. Our PACTs are being measured to a greater degree than ever before. This means that we have to be aware of social desirability when analyzing our data but that we also have the opportunity to utilize qualitative approaches in ways that help us understand the growing body of measurement data. 

So one last note, and it’s a long one, before we dive into today’s talk. Being there is important but it’s also expensive. It’s time and staff intensive. Being there comes with travel costs. But when it comes to qualitative data the old adage you get what you pay for is partly true. So when you invest in being there you get a certain number, a certain select intangibles. This list that I have up here on slide seven is not exhaustive but I wanted to walk through some of the things that really are advantages of in person data collections. 

So, the first is rapport. The staff recognized the cost involved in your being there and they really do appreciate our time. By going to the site you demonstrate respect for the people you're working with an a commitment to understanding their work. And, you begin to build long-term relationships out of this. So, out of sight, out of mind right. When you interview someone over the phone you're more reliant on your interview questions. You don't know what you don't know. So, in person interviews benefit both the interviewee and the interviewer because they provide environmental queues and I think really provide richer data. 

Seeing is believing – interviewees can describe what’s going on in their clinics, but we also take those descriptions with a preverbal grain of salt. When you visit the clinic in person you're really able to reconcile what people are saying with what’s actually going on in reality. That’s really important. This idea of contextualizing performance, observations as I noted a moment ago really help us to identify the factors driving performance that people might not think of or might be hidden. So, things like spacial orientation, IT issues, person environment fit – those kinds of issues. 

Belief vs. behavior – one common refrain about interview data that we hear is that we can only discover what we already know in part because respondents say because what they think you want to hear. We all know that what we say and what we do are different things, you know especially with how much you might report eating and how much you actually eat. Collecting data in person provides a window into how different beliefs and behavior are in actuality. 

Finally, meaningful recruitment and purposeful sampling – we sometimes assume that everyone has equal ability to participate in the interviews and surveys we invite them to. We’ve learned, and Sara Ono will talk about this, that LPNs and clerks for example rarely have such control over their daily work life. Even though they might like to participate in an interview or survey they can’t get the time to do so. So by being there to collect the data you not only foster meaningful recruitment, you figure out who to talk to next. It informs your purposeful sampling. 

Now I’m going to pass it over to Jane Foreman and Molly Harrod. 

Jane Forman:Thank you Samantha. This is Jane. I want to thank everyone for being here. Molly Harrod and I are going to present an analysis of field notes form observations of teamlet coaching sessions that we conducted as part of a locally designed PACT teamlet coaching model. The model was developed by clinic leadership at the Ann Arbor VA Medical Center. Here’s a brief overview of the setting: The Ann Arbor VA Primary Care Clinic is a large clinic with over 20,000 patients and 20 Teamlets. There are a lot of part-time providers and residents. So, that 70 PCPs and residents comprise 20 FTEE. There is a lot going on in the clinic and work processes are quite complicated. There’s an average of 3.5 PCPs per teamlet and at least one resident per teamlet. One thing to note is that residents are not able to attend coaching sessions because of competing responsibilities. That’s a limitation of this coaching model. That’s something that would need to be addressed and worked out in academic medical centers. 

I’ll give you a little background before we get to the coaching study itself. PACT implementation in Ann Arbor primarily meaning forming teamlets that were able to conduct practice redesign took over two years, largely because of the complexity of the clinic and started with lower staffing ratio is our VISN. It took time to build the foundation so that teamlets were able to do practice redesign work. I’ll try to use the pointer here. Yeah, there it goes. You can see on the schematics that the red bars at the bottom represent the foundation, meaning the kinds of things that you need to implement PACT - in our case particularly, staff base and communication which you can see on the bottom bar. The pillars in this schematic represent the ability to do practice redesign. That is redesigning roles, tasks, and relationships and work processes to meet PACT goals. So the coaching sessions were established to help teamlets once they were formed to develop and implement these new work processes. 

This is what the coaching model looks like. It was developed and is delivered by clinic leadership and is consistent with national requirements. There are six coaches, including the director of primary care and the ACOS for ambulatory care. And the program brings each teamlet together for one to two sessions that is two hours per month. The sessions are agenda based with current clinic wide goals – for example, to address access and continuity – and the teamlets in these sessions develop and then pilot QI interventions. 

Our evaluation of a coaching model was designed with input from our clinical operations leadership. We’re doing a formative evaluation with an eye to program improvement and spread to other sites. We’re looking at several things in our evaluation. In addition to interactions among teamlet members during the coaching session, which we’ll talk about today, we’re also looking at session content and attendance, development and implementation of new work processes over time, spread of best practices, and the effect of coaching sessions on part-time provider engagement and PACT metrics. 

To do this, we are doing a longitudinal observation of 9 out of the 20 teamlets in the clinic. The sessions we are observing are led by five different coaches and two of them are in leadership positions. We’ve done over 16 hours of observations to date. Our research design also includes semi-structured interviews with teamlet members and coaches to understand their experiences with these sessions and the coaching program value to their day-to-day work. We are also doing outcomes measurement, for example, changes in PACT metrics. Today, we present our findings form our preliminary analysis of observation data, focusing on teamlet interaction during the coaching sessions. My colleague Molly Harrod will take it from here. 

Molly Harrod:Thank you Jane. The preliminary findings I’m going to present relate to teamlet members learning from each other, building relationships within the teamlet, spreading best practices across teamlets, and teaching and learning with the coaches. In the interest of time, I will focus on the first two points. 

The first finding, teamlet members learn from each other, has three sub themes: Learning about each other’s roles, working out new work processes, and taking the initiative. As Jane mentioned, because teamlets were newly formed they were not used to working together in practice redesign. They didn't always know each other’s specific roles and tasks and what each was allowed to do under their scope of practice. They learned about this during these sessions. Second, we saw them learning together about new ways to accomplish patient care and clinic goals. And third, while they were talking about practice redesign, we say individual teamlet members taking the initiative to suggest new ways of doing things. 

The first finding, learning about each other’s roles, reflects teamlet members asking one another about the task they performed and what was possible for them to do under their scope of practice. Here’s an example of a physician learning about the LPNs perceived scope of practice. 

Part-time MD: What tasks do you enjoy doing or what should I be doing or what should I be giving to you?

LPN1: I would like to not have to turn everything over to RN. I can’t even take blood pressure readings over the phone. If I take someone’s blood sugar and even if it is normal, I have to hand it over to an RN. 

MD1: So you can’t call, take the information, and put it in a note? 

LPN1: Nope

Part-time MD: I think you should be able to. 

MD2: But will you get into trouble if you just do it? 

Coach: That’s what we have to find out. 

Second, we saw teamlets working out new work processes during these coaching sessions. There was a lot of back and forth about what each team member would need to do in order to make the process work. Here’s an example of a physician and LPN working out their own process to streamline giving injections. 

MD1: I’ll work on a list like we did for labs that has my most used injections and what I usually give. 

LPN1: If you could look at the schedule that day and write down any injections that you want I can have them ready and that way we don’t have to go searching for the injection tray. We have a bunch of new residents so you never know where they put things. It messes with the flow of the clinic. 

In developing these new work processes at times individual teamlet members took the initiative to present new ideas to the rest of the teamlet. These proposed changes are more likely to be successful because individual teamlet members are really the experts in their own work. They are most familiar with how they work and what possible solutions will work best for them. So in the case below the LPN suggests a change in her role. 

LPN1: What I would like to be able to do is print out the medication reconciliation form beforehand, call them (meaning the patient) and tell them I am putting it in the mail.

Part-time MD referring to LPN1: She has picked up adherence issues with patients before I even did. 

Coach: That’s a really good idea. That would help because it takes 20 minute of the visit to go over it with them. 

MD1: It takes so long. This would help so much. Is this beyond your scope of practice? 

LPN1: No, it is not. 

Here you can also see...it’s also an example of learning about each other’s roles, where the coach is asking about the scope of practice. 

The second main finding, building relationships within the teamlet, also has three sub themes: Affirmation within the teamlet, advocating for teamlet members, and flattening the hierarchy. Coaching sessions occurred at a time when most teamlets were fully staffed and beginning to work as a teamlet on PACT goals. In the interactions we observed we saw evidence of important relationship building and transformation during these sessions. Some of the interactions that occurred during the coaching sessions really revolved around teamlet members affirming to one another that they were doing a good job during this time of change. As they learned to work together, physicians in particular discovered the value of having dedicated teamlet members and expressed appreciation. 

MD referring to the LPN: She’s very good. She’s very efficient. It is literally impossible for me to do my job without these two (meaning the RN and the LPN). 

MD2: Having a dedicated LPN has been great. 

MD1 to LPN1: You are making a difference. You're getting rave reviews from patients. 

In addition, it was also common to hear physicians and RNs really advocating for the LPNs to be able to do work to the top of their license. Working to one’s top of license requires not only the appropriate skill set but also support from the rest of the teamlet. It is also key to work being appropriately delegated. Appropriate delegation of task is one of the cornerstones of affective team functioning. So, in this example a physician is advocating for the LPN to do patient education. 

MD2: I’d be happy to facilitate the LPNs doing the blood pressure teaching. Tell them (meaning primary care leadership) things are going great and it’s working well. It’s about reallocating our resources. LPNs shouldn’t be doing all check ins and check outs. Doing patient education is totally appropriate. 

And finally, we heard instances that indicated that teamlets are transforming from the traditional physician centered care team to a more team based patient care approach. Occasionally we heard non-physician teamlet members teaching physicians organizational processes that have a direct effect on PACT implementation and metrics. We have also heard physicians openly admitting that they do not know how to do things or asking how they should do things to make the teamlet work a little bit better. For example, in this field note the LPN proposes that she and the physician learn together. 

LPN1: That day you had a pap, I should have had everything ready beforehand. So now I’m going to look ahead and have it ready. 

MD2: I don't even know what the procedure is for doing paps. 

LPN1: On a slow day we can sit with the Women’s Clinic and go over everything. 

MD2 to LPN1: We’ll work on that together. 

So in conclusion, I would like to point out the value of using observation as a data collection technique. Some of the advantages of using observations are that it really allows us to see processes in action. We were able to document in real time and in detail how change occurs. Interviews, in contrast, can only give us a retrospective reconstructed account as told. We saw the process in which all teamlet members were both teachers and learners. It wasn’t physicians or clinic leadership deciding how teamlet members were going to do their jobs. It was the teamlet itself, and through their interactions they decided what the best practices for them were. 

Through observations we can also understand the nuances involved in teamlet transformation. As Nadine and others have described, the delineation of roles and responsibilities aided each teamlet member in better understanding what the colleague did and could do. This in turn lead to support and advocacy for everyone to be able to work to the top of their license. And finally, it was within teamlet conversations and how they really related to one another where we could see a flattening of a hierarchy. If we had only done interviews we may have heard that they worked better together but not how they worked better together. 

This concludes our presentation and I will now send it over to Dr. Tuepker. Thank you. 

Anais Tuepker: Thank you very much Molly and Jane. That was fantastic. So I...oh, I skipped onto my first slide. I think I’m skipping around too fast. So thank you everyone for being here and joining us today. I am going to talk about how our demo lab used qualitative methods to identify and then pursue some unanticipated research questions. So although I’m going to be talking about things which are interesting, I think, I’m really going to be focused on the process and how we came to those findings so that you can see kind of an example of how these methods can be useful. 

So very briefly a little background – our demo lab in VISN 20 had an initial focus on PACT innovations to improve care for patients with complex chronic conditions. When we were designing interviews and focus groups, which were our main means of collecting qualitative data, we did not do as much direct observation as was described in the last presentation by Jane and Molly. We organized sessions, like I say here. We organized focus groups and interviews with 242 employees in 15 primary care clinics. We did 31 focus groups and 26 interviews. That ended up being 242 people, which was about...a little over 70% of all of the employees in primary care. So, we were trying to get feedback from a really broad sample of people. 

But what’s important for what I’m going to present today is because our initial focus was care for complex chronic conditions, we didn't has any questions about performance metrics. We made a very loose use of the consolidated framework for implementation research, which I think many of you might be familiar with. If not, Laura Damschroder and colleagues have written some great articles and they originated it and the PACT pillars that were in the previous presentation. We used those to develop an initial code book. That code book is how we made sense of what we were hearing people say in these focus groups and interviews. We, instead of field notes – and you saw some great examples of very detailed and extensive field notes in the last presentation – we audio recorded all of these and then we transcribed them, and we coded them using qualitative software that basically just allows you to apply codes that group your data under different definitions. 

So, we then pulled together findings across codes that it turned out had something to do with performance metrics. So, what I mean by that is we did not have just one code that related to performance metrics because as I said, this was not really a theme that we had anticipated. That’s...when people talk about iterative coding, that’s what they’re talking about – something that kind of you did not start off with an idea that this is what you're looking for. It is something that is so compelling in your data that you have to find a way to make sense of it. So what I’m going to do... usually I’m a believer in the advice that I always get, that you shouldn’t have too much text on screens. But today, I’m doing something a little bit different. So I have a lot of text today. 

We would have these transcripts that would have quotes. I want you to be able to see the kind of data that we’re working on. So these are very minimally edited. So for example, we’d have someone say something like:

Well on one had we’d say we want the PACT team. We want to have more phone clinic. We want to have more kind of non-face-to-face stuff so we can manage these people better. On the other side of the coin it’s like boy, that veteran really wants an appointment and you had better give it to him on the day he wants it. So everybody gets an appointment and all of the slots fill up. You're giving two directives that really do not...they just bounce off of each other. You can’t make that fit.

So this was a kind of comment that we didn't have a pre-existing framework for making sense of. So we had to think about what does it mean? And, there are many was you could make sense of it. What we started to think was okay, metrics are sometimes in conflict. These are both things people are getting measured on. They’re actually in conflict. 

Similarly, here’s a comment from a different site. I’ve identified people by their role, acronym. So PCP is primary care provider at an anonymized site. 

We started scrubbing our panels. I changed the ratio of in-person to telephone calls, things of that sort. Then we get this backlash from administration that we’re not allowing patients to come in. We received this memorandum that we will see a certain number in person and we will do a certain number of phone calls. That it will be that way. 

So, again you know how would you make sense of this? Well again, we thought of this as metrics are sometimes in conflict. But we started to see even more here, which was also present in the past quote. Metrics are in conflict and who sets the priority? Who makes the decision about which metrics should be prioritized? Here’s another example: 

On one hand we’re saying we’re a PACT team and we're a team, but we’re suing the individual’s performance as the metric for how well the team is doing. So to be able to say the PACT model doesn't work because these people haven’t made their metrics, where’s the breakdown? Is it the individual provider that’s not really providing leadership and guidance? Is it the nurse care manager who’s not really good? Is it that people don’t understand their roles very well? 

So here again metrics, but there’s something different going on here. We typified this as metrics are missing something essential about PACT. And then as kind of a sub category of that, what they’re missing is the idea of team based care. They can’t capture... the metrics that we have right now can’t really capture what team based care is all about. Another one:

I’m also frustrated when his...when his – the primary care provider’s – numbers are clean and better, he gets the pat on the back for it. And I just say okay, well now that you've got that done here’s what else I need for you to do. So, that’s very frustrating for me too because what am I chasing numbers for? – This is from a clinical associate. 

So here again, metrics are missing something essential about PACT that kind of team based cared, different people are doing the work. But also, here there’s some evidence of some unintended consequences for staff satisfaction. You can really see that it goes back to actions taken to meet performance metrics having an impact on staff satisfaction. I think this is the last one I have for you here:

And then the clinic also has these things that say well, gee, nobody hasn’t had is cholesterol checked in 150 days. Let’s get that to your team to add. And then that generates an appointment. Whether the patient even needs to have is cholesterol or his A1C checked or any of that nonsense, there’s still those little boxes that we're expected to check. And, the patients are expected, either by themselves or by the clinic managers to come in once a year whether or not they need an appointment. That’s a system thing. So we have this system, these system requirements that don’t speak to the PACT model. 

So how would you make sense of this? Well, here we’re getting at maybe the metrics are not aligned with PACT? Maybe, for example in this case, the underlying question which we sort of key on that whether or not they need the appointment is is it patient centered? I think there’s still a lot of questions about how PACT is going to be defining patient centered, but clearly by tying that definition or attempting to tie that to metrics, there’s an influence from the performance metrics on how we define patient centeredness. So...oh sorry, one more:

I think we're reaching some of those core performance measures that we’re able to help our providers reach some of those goals and attain them. But it’s almost like, with some we’re able to but others are left behind. Other patients are being left behind. It’s like we can focus on some of them but not all of them. It feels like maybe our diabetics and our congestive heart failures, COPDers, those kinds of people, we’re attaining those goals with them. But it doesn't seem like we’re giving the same amount of attention and care to all of them across the board. 

This is kind of...I think you could say this is also about being patient centered for all patients. But what we thought was really relevant here is the opportunity cost that metrics may carry. When you focus on some metrics that relate to some patients, there may not be time to give that same attention to other patients. That was, we thought, a really important concept. So the point of giving all this detail is just to drive home the fact that again, there was not a question we asked about this. This was information that came in the context of other questions but was spread very evenly through all of our focus groups, which were role stratified. We did it mostly by role. Sometimes we would do team focus groups. And this theme kept coming up again and again. 

So we then took this data and what we had thought about it...people did not articulate here’s how metrics should be different. Here’s how they’re impacting our work. They talked about those things but they didn't sort of systematically organize those thoughts. That’s the job of research. So here’s a simple graphic of what we’re trying to do both in terms of research learning and also advice and insights that we can give to our implementation partners. 

So, we're focused now on trying to build a research focus and implementation focus on improving metrics. What do we need to do, why would we do it, and how are we going to do it? We have three general areas. You need to align metrics with PACT, you need to streamline the metrics that we’re using, and some of them you need to redefine so that they can actually capture the PACT concept. I won’t talk about all of these things because the middle column there is kind of implicated by the slides before. This is what we learned about why we need to do these things. And then in the last column, you know this is what we’re trying to work on. We’re trying to develop a tool that can help assess some of those opportunity costs, some of those conflicts of existing performance metrics and how they relate to the PACT pillars. 

In terms of streamlining we’re trying to help develop communication tools so that clinics will have a clearer idea about how to prioritize metrics and making the system more transparent so there’s not as much confusion about where metrics are coming from, who they’re important to, what they’re being use for. Throughout PACT I think we're learning there’s a lot of work to be done in system transparency to build trust and commitment to the model. And finally, sometimes we’ll need to redefine or develop new measures such as those that can capture team performance. 

So, our next steps as a demo lab, what do you do next once you've identified this new research question that you weren’t interested in before, is you need to do more focused data collect. So the next step would be interviews for example, with teams about the specific impact of the actions they’re taking that are intended to address performance metrics. We also want to do interviews with patients on this topic. How would they measure team performance? What matters to them? The voice of patients has been missing from our work on this and that’s an important thing that we need to address. Also because metrics go across our system, they’re not just in something that is generated and used solely by primary care clinics – far from it. We need to do interviews with VISN and facility leadership and really understand better that process of metric selection and dissemination. And we’ll develop this next time around a more focused interview guide but using the same process that we did before of directed-to-inductive coding – meaning we start off with the questions we know we're interested in and then who knows what we might find. 

So just a couple comments before I had it over. I’ve tried to show here how an inductive and a directed analytic approach can actually...you can use both of them. They’re not mutually exclusive. You can set out to answer certain questions and you can find answers to questions that you didn't really know you had. But you should try and be clear about which one you're doing at which point in the process.

Just two other things – the validity of qualitative themes comes through consensus. By that I mean, you need to involve multiple researchers and perspectives. Double coding your data, having a lot of iterative discussion, and going back to your participants to talk about your findings. All of those things really help create or show the validity of your findings. Because, you know as I’ve shared with you this raw data, the comments, there are different ways you can interpret it. Really, the only way you're going to find something that has resonance and meaning for many people is by involving multiple perspectives in the analysis. 

Finally, just a comment – I think we often see this language that findings emerge from qualitative data. I try never to say that because I really feel that you work at it and it’s a conscious process. There are always other interpretations that are available. So, I hope this presentation has kind of showed how that’s the case rather than just me saying it. And with that, I’m going to turn it over to Dr. Sara Ono whose presentation about the role of clerks I’m really fascinated to hear. 

Dr. Sara Ono: Thanks Anais. For this final portion of today’s cyber seminar I’ll be talking about qualitative insight into PACT dynamics and the role of clerks. This is work that I’ve done with Samantha Solimeo and the Formative Evaluation Core in Iowa City. So, if there are errors, they are mine, but the work is a credit to the larger team. 

Samantha and I first presented this work to an anthropological audience.  I mention this because thinking about audience has a big impact on these qualitative findings. We’ve become accustomed to what is the norm in our VA setting based on past precedence and knowledge of how data is used and valued in this VA system. But what often circulates is a very small fraction of what is available in the qualitative data in most cases. Qualitative data is a wealth of riches. Our methodology is one of abundance. It’s a direct result of a holistic approach that’s foundational to ethnographic work. While we can rarely present the whole of our data, we use this comprehensive perspective in the analysis process to boil it down into focused findings that are appropriate for the setting or the audience. 

In my time today I want to talk about clerks, who are also known as MSAs, PSAs, and clerical associates. Clerks are one of the four roles that make up a PACT team. The role of clerks is rarely the primary focus of evaluation analysis. This is likely from multiple reasons. One reason is that there are unique challenges associated with collecting clerk data. Samantha eluded to these at the beginning. Among the challenges are clerk’s time and willingness. It’s hard for clerks to get way for a phone interview or fill out a survey. And also, because of their public locations and clinics, they may not have a private space to talk with researchers. 

Participation in research may mean something different to different roles. For example, clerks may not think their input will be valued or believed that it matters. The nature of the interaction changes based on perceived status or power dynamics. What I mean by this is that during data collection we found that clerks were often suspicious of the interaction and that they worried about repercussion and what might happen if they talk to us, especially if they were to say anything critical. 

Now, if you've encountered the television program Sesame Street while you were growing up or through kids in your life, you might remember one of the songs – one of these things is not like the others. Qualitative data allows us to see how this scenario applies to clerks on PACT teams. Why people become clerks is different than other team members. Unlike other roles, say nurse or providers, clerks do not tend to talk about their role – their job – as a calling. Working as a clerk is often an entry level position and it’s a way to get into the system. It’s a place to move up from. This contributes to high turnover and lack of team continuity. Well, clerks are one of the four roles on a PACT team, they are the only role without any clinical licensure. This makes them different. It’s this difference that’s reflected in the two findings I want to share today. Both of these findings drawing data collected through interviews and sight visits, but also demonstrate the value of in-person or on-site data collection. 

Finding number one: Clerks report high work load and low status. So in other words, for clerks there’s a lot to do and the expectations are high. Clerks need to perform a broad range of tasks that may include panel management, data collection, triaging phone calls, screening secure messages, and sending faxes. Clerks are viewed by their fellow PACT team members – and these are taken from quotes – as being overwhelmed at the front desk or too busy to do a great job, but as also needing to be more involved in data collection and to provide better customer service. So while it is recognized that the clerks often have too many tasks, the message sustains that they should be doing more and better in spite of this. 

Finding two: Spatial dislocation and timing of tasks lead to feeling disconnected. And, to quote one PACT provider who said, “Clerks would interact more with the rest of the team if not spatially apart.” So there is a level of understanding that comes with having someone tell you they feel disconnected from their team or that they’re limited in what they can do by the workspace they have. There’s another level of understanding that comes with seeing four people jockey for counter space that was designed for one to two people. Or from watching a clerk negotiate greeting patients in one room while retrieving faxes in a room down the hall. Now, we know that every clinic is different. Seeing that difference can help put reported data into context. 

The feeling of being undervalued and the recipients of undesirable tasks is only exacerbated by the notion that clerks have of being out of sight, out of mind. The clerk role creates isolation in two ways – being physically removed from the team and the timing of tasks related to PACT. The nature of the role is that clerks are the frontline in clinics. They are the first point of contact when veterans arrive for care or call to make an appointment. In order to do this, they have to be available and visible to veterans, therefore typically unavailable for team huddles and meetings, which often takes place at a slow time for providers and nurses. To no surprise, this is the busiest time for clerks – such as first thing in the morning. 

PACT teams report that clerks are disconnected. They can also identify how clerks play a critical role on teams. In their role, they are most often located at the front desk. Clerks are the face of the office, and in this they provide the first impression of the clinic in the VA to veterans upon arrival. If well incorporated, clerks make valuable contributions to their teams. If they’re role in a PACT team is clear, they can incorporate team goals into scheduling practices and prioritization of tasks. 

A few last thoughts of this idea of being there as a method, which I think all of the presenters have come at from different directions, which is terrific. Not only does this approach provide different kinds of data in the sense of observational verses outcome, but it also provides data on different kinds of things. In this case, the role least addressed impact implementation, that of clerks. Collecting data form a distance may result in knowledge of who is in a clinic, but not necessarily how those individuals relate to each other in practice and in their day-to-day work. 

Another advantage, especially in a territory like VISN 23 where I’m situated, is that being there and traveling to clinics provides a better understanding of distances involved for veterans using these facilities and potential barriers. It also builds rapport with clinic staff who appreciate the effort involved in sight visits. And that it shows them that their contributions are important to us and we’re willing to make the effort to get that information from them. 

Being there allows researchers to identify and incorporate information that’s otherwise invisible or might often go unseen. It can fill in spaces between what happens on the ground in the day-to-day practice and what is reflected in the quantitative data or outcome measures. These contributions, along with all the benefits Dr. Solimeo outlined at the beginning of today’s presentation and those introduced by my fellow presenters can aid in the developing of further understanding in our evaluations. 

So, in thinking about contribution of qualitative methods we have a pole question for you. The question is: Which aspects of qualitative observation methods would you be most interested in learning or learning more about? The choices are: Different approaches – for example direct participant or structured, selecting the best approach for your study, strategies for gaining entry and developing rapport, or field net recording and analysis? I believe Molly is going to help me out with this pole part. 

Moderator: Thank you Dr. Ono. Would you like me to make it so that people can select all that apply to them? 

Sara Ono:Sure. 

Moderator: Okay great. So, for our attendees, please just click the box next to any of the answers that you would like to report. For those of you who are raising your hand, you are muted and will remain muted. So simply please just click the box and submit your vote. And, it looks like most of the answers have stopped streaming in. So, I am going to broadcast the results, and Dr. Ono you should be able to see those now if you want to talk through them real quick. 

Sara Ono:Okay, they’re not up yet. 

Moderator: Okay, you may be in full screen mode. That’s perfectly fine. I can read through it. 

Sara Ono:Do you want to do it? 

Moderator: Sure you can click out of it if you want. But, I can go through it real quick. It looks like 33 people are requesting different approaches – for example direct participant, structured, etc. Thirty-five people also requested selecting the best approach for your study. Twenty-two people have requested strategies for gaining entry and developing rapport. And, 31 people are interested in learning a field note, recording, and analysis. So thank you to our attendees for answering that. We’ll go right back to the slides. 

Sara Ono:Great. That’s really...it’s interesting that it’s as evenly divided as it is. I think it’s a testament to the multiple approaches that you can apply qualitative to, the multiple topics and different approaches available. So maybe this will be something for a future session. And, we have hit the point for questions if there are any. 

Moderator: Thank you very much. Yes, we do have some pending questions. I noticed a lot of our attendees joined us after the top of the hour. So, to submit your question or comment please type it into the Q&A box that’s located in upper right hand corner – simply type it into the lower box and press send. With that our first question, I believe this was done during the initial speaker: As an aside from gathering information for program evaluation for the need for separate role of facilitating community within new and emerging PACT how could those sites which do not have you team’s presence translate the focus/support groups? 

Samantha Solimeo: Oh, that’s a really great question. This is Samantha in VISN 23. One of the things our VISN has done is develop community practice model for some of the roles. So, I know there’s one for providers that’s national and there’s one for RNs that’s regional in our VISN. The challenge with that I think is, kind of going back to what Sarah and I talked about with clerical staff and LPNs is they often don’t have the support of their local clinic leadership to take time away from their demanding jobs to participate in that kind of thing. So, I’d say one: recognizing that clerical staff and even the LPNs do benefit and enjoy the comradely and sort of peer mentoring that those kinds of things can provide – that they have a professional identity that can be supported. I think that’s really important, and then providing some kind of mechanism for them to actually take part in things. There are a lot of resources out there but I think many PACT staff don’t have verbal and sort of technical support to participate. 

Moderator: Thank you for that reply. We do have another question. This one is for Dr. Ono and your presentation on clerks. Now that have insight to the dynamics of challenges related to clerk integration and involvement on a PACT team, what are you doing with the results of that research? 

Sarah Ono:That’s a great question. What we’ve been doing throughout the evaluation process is trying to feedback findings as we get them. So at each of the different learning collaborative for the VISN that we participated in a report was generated and sent back to VISN leadership. There’s also been a real effort by the formative e evaluation team to feedback findings to the PACT teams and clinics themselves. So you know, information is going back to the teams that they’re at least made aware of the dynamics that often I think they just take to be part of the day-to-day and don’t necessarily recognize how it’s affecting the PACT team function. I know that between visits - Samantha and I have done sight visits repeatedly to a number of places. We saw between first visit and second visit that in some cases they were making changes or they had plans in the pipeline to redo front counters or to make modifications to waiting areas to try and help facilitate some of the space issues that clerks in particular encountered. 

I think the progress is ongoing and sometimes the steps take time and are small. But, I think that what we’ve seen is a responsiveness to the findings in whatever ways people are able to. 

Moderator: Thank you for that reply. Just a reminder, we have several people trying to use the hand raising function. I cannot unmute your line. So please type your question or comment into the Q&A box that’s in the upper right hand corner of your screen. If you're in full screen mode, you need to click out of that in order to see the Q&A box. A lot of people are asking if this data is available and also if the slides are available? The slides you can retrieve from the hyperlink in your reminder email that you used to enter today’s session. Also, they’ll be available from the follow-up email you will receive. But regarding the results, I’m going to turn that over to the presenters. 

Anais Tuepker:This is Anais Tuepker. I’ll just jump in. We have a paper that’s under review right now. I don't know if there’s an easier way to let people know when it becomes available, but if you are interested, send me an email and I’ll let you know if and when it gets published. 

Moderator:Thank you very much. That is the final pending question, but we do have just a minute or two left. I would like to give you ladies the opportunity to make any concluding comments if you'd like. 

Jane Forman: Hi, this is Jane Forman. I just wanted to thank everyone for their interest. I will address the question that someone asked just now about products. We also have a paper that’s in press about the kind of evolution of PACT implementation in our medical center. And we are continuing with our evaluation of the coaching model here and also feeding back information to operations about it. They were very interested in our results, and as I said, they were involved in designing the evaluation. 

Moderator: Thank you. Would anybody else like to make any concluding comments? 

Sarah Ono: This is Sarah Ono, and you know like the other authors, I think all of us are working on getting our manuscripts out in addition to the PACT reports that have circulated to a lot of leadership. I think you can contact any of us directly or if we know there’s interest, we also are in communication with each other and we’ll do our best to try to get all the work out. 

Moderator:Wonderful. Well, I just want to ask our participants to hold on for one more second. I’m going to close the meeting momentarily and you'll be redirected to an audience survey. I ask you to please answer those few questions as it is your opinion that helps us decide what presentations to support. I also want to plug the next PACT session, which will be taking place on the 20th of November. Please join us at that time for that session. The topic for that one will be special populations, homeless veterans, and veterans experience intimate partner violence. That will be at noon on the 20th presented by doctors Melissa Victor and Sonya Gabriellen. So, once again, I want to really thank our presenters for sharing their expertise and I want to thank all of our attendees for joining us today. And again, please do fill out our survey. So thank you everyone and have a lovely rest of the day. 

00:55:03END OF TAPE 

Page **1** of **16**