**Transcript of Cyberseminar**

**Spotlight on Women’s Health**

**Integrating Patient or Provider Experience into Organizational Initiatives: An Introduction to Q-methodology**

**Presenter:  Melissa Naiman, PhD, PMP, EMT-B**

**November 12, 2013**

*Moderator:*Our presenter today is Melissa Naiman.  She’s the Associate Director, Center for Advanced Design, Research and Exploration.  Research Assistant Professor, School of Public Health, Division of Health, Policy and Administration.  Research Program Manager, Advanced Cooling Therapy, LLC, and a Professor at the University of Illinois, Chicago.  I would like to turn things over to Dr. Naiman. 

*Dr. Naiman:*Thank you.  Good morning, or afternoon I guess depending on where everyone’s sitting.  I’m very pleased to have been invited to share a bit about, really, methodology that I’ve used and found very helpful in better understanding user needs and subcultures within various health care environments.  

Just to quickly jump in, talking about Q-methodology, and really what’s going to be the center of my talk today.  Q is a mixed-methods technique.  It’s a dependency analysis, so that’s distinctive from other statistical models that look more at differences between individuals.  Instead, we’re looking at significances between people.  As we go through today, I’ll hope to point out some advantages to using this sort of strategy in research, especially in health care and women’s health within the DA.  

First audience poll, before I get too far into this, I was just wondering how many people are familiar at all with factor analysis?  If so, if anyone’s heard of Q-methodology before right now?

*[Pause 01:38 - 01:43]*

*Moderator:*I’m actually going to have to pull those polls up separately, but we’ll go one by one here. 

*Dr. Naiman:*That’s fine.  Starting off, factor analysis.  

*[Pause 01:50 - 02:12]*

*Moderator:*It looks like the responses may be slowing down here a little bit if you want to read through those here. 

*Dr. Naiman:*It looks like we’re at about 70 percent have heard of factor analysis, or are familiar with it.  32 or 30 percent-ish have not. 

*[Pause 02:29 - 02:35]*

Specifically, has anyone heard of Q-methodology previously?

*[Pause 02:39 - 02:54]*

It looks like almost 80 percent of you are sayin, “No.”  *[Chuckles]*

*Moderator:*Just flips the responses there.  

*Dr. Naiman:*There we go, well that’s excellent.  You should understand the—I’m glad that the majority of you are familiar with the mathematics that underpin this methodology, but it’s a very different approach, and an interesting way to use factor analysis, I think.  

Just a quick background of Q-methodology, it was initially published by William Stephenson in *Nature* in 1935.  He was a physicist by training, and ended up getting his second Ph.D. in psychology, and was a student of Fisher.  There was another scholar at almost the same time, literally a week apart, Godfrey Thomson, who published a very similar premise.  William Stephenson really spent the majority of his career developing this as a methodology and philosophy. 

Q-methodology is probably the most central of the mixed-methods methodologies.  Aside from its subjective purpose because you're really focusing on perception, everything else about Q-methodology falls kind of right in between the qualitative and quantitative extremes in the way that you can approach a research question.

What this leads to, in terms of psychometric distinction, when you’re looking at end-users or *[interference 04:21],*is that there are some very distinct outcomes that you would expect from Q versus what we call R-methodology, which would be more like any sort of survey technique, for example, that’s looking for an R-value.  I never realized what a tongue-in-cheek jab it was, but that’s kind of the rough distinction.  

The important part of these differences to point out, is that Q, first of all, is dependency analysis.  You’re really, again, looking at similarities.  The population is statements.  That has some very important implications that I’ll touch on later in terms of power, and sample size, and things like that.  

The third very important distinction is the requirement of forced choice.  Because that has other philosophical implications that are important.  In comparison with more standard survey techniques that have, typically, a Likert scale, where you can pick as many one’s or five’s or threes as you’d like.  

In order to demonstrate how Q-methodology works, I think the easiest way to explain it distinctly is to just give an example of a study that I did looking at radical innovation adoption within health care.  In my case it was within an emergency department in a Chicago-land large hospital.  The reason that I chose radical innovation as a study topic was because technology use in health care settings is, in my mind, a public health issue.  The way that a hospital or health care organization decides to adopt technology and use technology has very direct implications on their quality of care they provide, access to care within a community, and other issues along those lines.

Working in a health policy department, a lot of my colleagues look at me and say, “Yeah, well Melissa, we spend a lot of money on buying all of this technology, and the problem is that if you measure innovation based on an organizational purchase, you're really not getting the full picture.  Because technology purchase is really not equivalent to technology use.”  Which kind of falls under the *Field of Dreams*acquisition strategy,  as I refer to it, where a lot of administrators feel that if you buy it, the users will fall in line.  At this point, especially given a lot of the research in electronic medical records, it’s very clear that that’s not the case.

I approached technology adoption from two different views.  First there are organizational variables that assist with adoption, or can hinder adoption of technology.  Then also individual variables, where in health care settings would be clinicians their feelings, and attitudes towards the technologies, and also what functions and tasks the technology is supposed to assist with.

It is my hypothesis that in systematically gathering opinions of clinicians, you are able to link these organizational variables and individual variables that are associated with technology adoption into sort of a virtuous cycle that would, overall, reduce technology rejection.  

I’m not the first person to come up with the idea of asking clinicians what they need before going into a technology purchase decision.  The Institute of Medicine, American Medical Association, and the American College of Physicians all have issued different papers and studies advocating for larger and broader clinician participation in technology purchase decisions.  What they don’t really cover is how do you do this?  Boots on the ground, as an administrator, or as a hospital leader, or a leader in an organization, how do you actually come to understand what it is that clinicians really need?

My specific research questions were can clinician opinions be used to identify radical innovation in the first place?  Can these opinions then guide prioritization of these radical innovations because in most cases you really can’t do every single thing that people would like to have implemented.  I define radical innovation as a major departure from standard practice that may change workflow or professional roles.  Those were the qualities I was looking for in a specific innovation.  

I worked with Advocate Christ Medical Center’s Emergency Department, which is a very near suburb to Chicago.  It’s in Oak Lawn.  It’s a pretty large teaching hospital, and has a level 1 trauma center designation, so they have a lot of very disparate technology needs in order to serve their patient population.

The overall study design that is consistent between a Q studies first is concourse development, second is developing the Q-set, which is that sample of statements, which is your population, and condition of instruction.  Third is having your participants complete Q-sorts, and fourth is data analysis.  

For this particular study, concourse development for me had a couple of qualitative steps.  I use one-on-one interviews that I conducted within an empty treatment area where I guided clinicians through all of the stuff, essentially, in this room, and asked them to comment on what they liked, and what they didn’t like, and why.  I was able to gather from that—conduct a somatic analysis and identify certain characteristics and functions that they felt were particularly useful and easy to use within what they already had.  Then I conducted a series of focus groups with nurses and physicians, and focused on clinical challenges they faced, and where they felt technology should be better to help them do their jobs more effectively.  

Between the outcomes from those two things, I did a market analysis and identified a series of 53 products that were either on the market, or were coming to market, that embodied both the ergonomic characteristic, ease-of-use characteristic, and then clinical indications that clinicians were most concerned about in this emergency department. 

From there, I had a bunch of different technologies.  You put together some generic statements.  This is just a couple of examples, the actual statements are a little bit longer, but they were all generic, so I didn’t mention any brands so that people weren’t biased against whatever company.  I tried to get the gist across of exactly what it was that the technology would do, when it would be used, and how quickly it would provide results.  The full concourse will be published in a journal called *Operant Subjectivity*.  It’s just not out yet, but if you wanted to see the whole instrument, you can let me know, and I’m happy to share that. 

One of the most distinct parts of Q-methodology is the Q-sort.  What you have people do is take these statements that you’ve developed, the population of ideas that are concourse, and you sort them.  They synthetically do this, so you hand them cards, or you have them do it in a virtual environment.  There are a couple of programs online that you can do to facilitate these.

Basically, it looks like this.  You provide them with a condition of instruction, so for this study it’s saying, “When thinking about technology and techniques to support improving care in emergency department, which of the following do you feel would be most likely or most unlikely to improve the care that you provide?”  This question can be whatever you want, and should be, at least, be grammatically consistent with your statement, and basically reflect how you would like them to sort their preferences in terms of these items.  

Again, you’re handing them cards.  Each card has a statement on it, and you ask them to do an initial sort where they sort of do a gut check of whether they either agree or disagree with the statement, or whether it’s more likely to improve care, or more unlikely to improve care.  Just do a rough sort into two piles.  There’s no restrictive rules on this set.  They can put them wherever they want, however many they want.  

The final sort actually does have them place each of these cards into a grid which is always in some form of Gaussian distribution because this facilitates the statistics that are about to happen with the factor analysis.  They take each of these cards, and they go through and place each card in a box.  They’re allowed to move them around, and so the goal is to make these statements reflect their personal beliefs as closely as possible.  

After this step, you often include some sort of demographic survey, asking for professional background, years of experience, those kinds of things that are pretty standard demographic questions.  You can append a survey.  All of the online applications allow this as well.  

Then, once you have all of these sorts filled out,  all of these grids are completed, you’re able to do correlation coefficients between each of the individuals.  You can see here the matrix, have a person-by-person matrix and you can see the correlation coefficients.  You would imagine that if somebody, or two people sort all of these cards—in my case it was 43 total statements that they sorted in a similar fashion, then there should be a high correlation between persons.  It’s this grid, across all of your people, that you end up completing factor analysis.  

Just to give some details on the data collection for this study, I used a convenience sample of 40 participants.  It ended up being 30 physicians and 24 nurses.  Three people did not fill out the supplemental survey, so I know that they’re either a doctor or a nurse because I checked their badges before the study.  All of them were at least 50 percent FTE.

We used email and in-person recruitment which actually turned out to be a lot more efficient, and I basically sat in a break room with a bunch of Dunkin Donuts Munchkins, which is the best way to get anybody in an emergency department to do anything, in case you were wondering, and had them fill out these Q-sorts on computers that I provided.  

Going into the factor solution, those of you who are familiar with factor analysis know that this is really where things get tricky.  I came into this part with a lot of trepidation*[chuckles],*and came up with a set of conditions that I felt needed to be satisfied before I even looked at the numbers, or even started playing with different factor solutions.  I felt that it really needed to be statistically convincing.  If this technique was going to be useful in a health care policy setting, it had to really jive with a lot of the other highly quantitative methods.  You’re talking about return on investment, cost-benefit analysis, which very quantitative-type functions.

In order to include this as  part of decision making, I felt that the result had to be statistically convincing.  At least 50 percent of the variants explained.  It had to be inclusive.  You needed to have a lot of people who participated in the study be part of the solution, otherwise it’s really not reflecting the department.  Probably, if it came down to it, favoring senior staff members because these are the individuals who would be most likely to be called upon as champions during a real implementation.  Probably Senior MDs tend to hold that role more closely, but I think that’s changing a lot.  There are quite a few nurses who have a lot of respect within this particular department. 

I went through and did a bunch of step-wide comparisons of different factor solutions.  These were my top four.  At the end of the day, I chose the four component principal component analysis as the factor solution that I would interpret.  Because it had the largest number of people who significantly aligned within the solution.  There were 11 clinicians with more than 10 years of experience, which is *[interference 16:34]*clinicians.  Five of those were Senior MDs.  It explained more than 50 percent of the variance within the study.  

Again, before even looking at the statements that created these factors, I looked at the demographics within each of these factors.  What was interesting, to me, in looking at all of this is first that physicians and nurses aligned together.  If you think about the way that technology tends to be designed and implemented, each of the professional groups tend to be addressed separately.  You’ll have a nursing representative, you’ll have a physician representative, you’ll have a pharmacist representative.  Based on those types of practices, one would have expected that physicians would have grouped together in terms of which technologies they felt were most likely to improve care, and nurses would have had their own groups of technology.  That really wasn’t the case.  They very clearly spanned across different factors, so that was a pretty interesting result.  The senior staff also was spread across three of the four factors.  

This is all based on Rogers Innovation Diffusion Theory, I asked at the end for them to—there were little vignettes, and they selected which was most like the way that they approached new technology.  Almost everyone identified as either early majority or earlier in terms of when they adopt innovation.  I had one lagger, one person who went and said that they just waited forever.  That also contradicts a lot of the talk that it’s just that clinicians are slow to adopt.  I don’t think that they perceive that as true.  I think that needs—whatever it is that’s adopted needs to, in some way in their minds, address an issue of care, and improving care.  

From these factors, I interpret them two ways.  First, we’re with innovation profiles.  Factor one ended up being speed oriented.  They wanted fast access to everything.  In the following slides, I’m going to go over exactly how I came to these conclusions, so just very quickly introducing you to these concepts.

The smallest group but the youngest group were holism oriented.  The third group was acuity oriented, and they were most interested in the sickest patients, and helping those people out favorably.  Fourth was information oriented, where their highest priority was communicating with everyone.  

This is what comes out of the factor analysis.  Once you have done the correlations, and you’ve defined these factors, the software called PQ Method, that is used for data analysis and Q-methodology—or the most widely used, develops an ideal-type factor array.  The ideal or most common answer for people in factor one are in this factor one column here.  

Is this showing my pointer?  Here, how about I—

*Moderator:*Yes it is, yep. 

*Dr. Naiman:*Okay, there we go.  Over there.  Basically it goes through and shows that the average answer for somebody in factor one would be three on all of these.  You go through this factor array, and this ideal-type response, and see which statements are associated with what they feel is most likely to improve care, or most unlikely to improve care.  You can get a sense of what it is that shapes their perceptions of improvement of care.  

In factor one, which was the group that was speed oriented, you’ll notice that point of care tests were really important to this group.  They felt it was really likely to improve care.  In terms of what they felt were unlikely to improve care, they were less interested in items that helped with patient dignity, amongst other issues.  They were less interested in different types of imaging and things like that.  In short, they were really interested in fast access to blood levels; really fast access to quantitative results.  They were interested in fast access to patient history.

In contrast, factor two were really interested in non-invasive techniques and more qualitative outcomes in terms of results in the ED.  They were the only group that felt that a needle-less injection system—which explicitly, in the full statement, discussed reducing pain during administration—they were the only ones who felt that that was likely to improve care.  They were interested in being able to share a lot more.  In terms of what they felt was unlikely to improve care, they—in extreme contrast to the speed oriented—were not interested in these quantitative tests at all, and they were interested necessarily in video streaming.  They voted that the least likely to improve care of all of these technologies.  Not being able to see what was going on in the ambulance as the patient was coming in.  Access to imaging technologies, that was clearly one of the themes which provided qualitative results. 

One of the other things that came up as part of the focus groups was that the delays in imaging were particularly uncomfortable for patients.  It may have linked in with that concept of patient experience overall, when they get stuck in the emergency department for several hours waiting for some sort of scanner to open up, they felt really bad about putting patients in that position. 

They also liked fast access to patient history, and also fast access to other clinicians.  This may be a reflection of the fact that this group, in particular, was younger and less experience, than any of the other factors. They were the only group to really emphasize a technology that would decrease pain. 

Third group, they really focused in on all of the technologies that would address basically stroke, heart attack, and trauma were their top picks.  They did want to see the EMS encounter, and see what was going on inside the ambulance, and get as much information about what was going on before the patient even reached the emergency department, moreso than any other factor.  

Very interestingly, they felt strongly that a government-controlled health record, which the actual statement reflected a health information exchange wouldn’t be helpful.  They felt that was very unlikely to improve care.  During the focus groups, one of the issues that physicians brought up—and if you think about this from a high acuity patient standpoint, knowing what somebodies A1C level was last week is not helpful if their leg has just gotten blown off.  Depending on what it is that you’re trying to trying to treat somebody for, the extensive background on their health may or may not be of much utility to you in the emergency department. 

They also were particularly disinterested, more so than any other group in collaborating with other physicians through video conferencing, or being able to contact their primary care physician.  They also were not terribly interested in some of the drug-delivery techniques, again, that would reduce pain during drug delivery.  That’s why I ended up interpreting this as an acuity-oriented group.  They wanted faster diagnosis for monitoring the sickest patients.  They wanted improved monitoring of EMS activities.  They really wanted to know what was going on in the pre-hospital setting.  

Finally, the fourth factor focused a lot on communication more so than any of the other factors.  They really wanted to be able to reach out within their facility, outside of their facility, share videos, interact with EMS.  They favored the patient-controlled medical records.  A personal medical record rather than, again, the government controlled, or the concept of the health-information exchange.  

What they felt was unlikely to improve care were a lot of the imaging.  Again, they did not feel that it was going to improve care to have faster access to some of the imaging.  This is distinctive from the high acuity group who felt that these items would be most likely to improve care.  Also unique to this group were a particular dislike of some of the sepsis-treatment modules that were coming onto the market.  That was just unique to them, but I wasn’t sure what that added to the interpretation per se.  

In short, the information-oriented factor was really interested in real-time collaboration.  They wanted faster diagnostics, even when slower, more definitive tests were available.  That was the over-arching individual interpretations of the factor.

Then I took a step back, and looked at more of the departmental perspective, and looked at what was considered likely to improve care, or unlikely to improve care across these different factors.  Even though they’re coming at their preferences from very different perspectives, at the end of the day, a lot of them did want to see similar technologies implemented in the department.  

I identified the consensus innovations that were either positive, which several of the groups felt were more likely to improve care, and everybody else was neutral.  We’re not looking at things where there were extremes on either end because those would probably be more challenging for implementation.  I was looking at where people felt it was most likely to improve care, and the other groups were kind of like, “Whatever.”  The same with the negative, so looking at which technologies groups ranked as very unlikely to improve care, and the remaining factors felt neutrally towards.  

In terms of the positive-consensus technologies, there were seven total.  I was able to, then, go back and link which vendors and which products inspired each of those statements.  I was able to provide the Director of the department with a concise list, and an explanation of, “This is what the people that participated in the study felt were most likely to improve care.”  In contrast, there were seven items that were considered unlikely to improve care, and I was able to provide a similar chart that just showed where—you may not want to waste your political capital right away.  

It’s not to say that these will never be implemented, or they shouldn’t be implemented, or they’re bad, or anything like that.  Just understanding where the clinicians are in their perceptions at this moment is helpful in terms of getting together any sort of implementation plan or planning to do any sort of change management.  

Overall, the way you can apply these kinds of results are from the departmental perspective, looking at an innovation strategy.  If you’re looking at what you’re going to purchase over the next five years, then maybe the next time you have some extra capital equipment investment money, look at one of these items in preference to something else that might be out there.  Also, looking at the individual profiles can help with change management, and most specifically, in terms of communication, and communicating the value of a given innovation, which is a very important part of change management.

Putting that together, the change management part in particular, puts me in the mind of Pepe Le Pew.  If you remember the premise behind that cartoon, there was a black cat, and by some strange happenstance, she’d get some sort of white stripe on her.  All of a sudden, Pepe Le Pew would absolutely fall in love.  It can be like that with technology products.  Where you’re really not making any sort of fundamental change to the product or what it does, but you can make a very superficial observation about the technology itself.  In doing so, make that value resonate with the people that you would like to be your users.  I feel that the results from this kind of study can really help you get to that point.

That concludes my introduction to Q-methodology *[interference 30:15].*The second half of the talk is going to be going over potential applications in the VAs Women’s Health research.  I am not, in particular, a women’s health researcher.  I am more of a methodologist, as you will see in a moment.  I am most interested in applying Q-methodology in a lot of different contexts.  

When Becky Yano invited me to do this talk, I asked her to give me a little bit of background on what’s going on in VAs Women’s Health, and I would be able to do a little bit of armchair quarterbacking in terms of how Q-methodology might be applied to help understand some of the issues facing this community.  

Becky provided me with three review articles, essentially, covering a variety of topics that are research concerns and research questions that remain unanswered within the Women’s Health Research community in the VA.

In general, one of the things that struck me, that would be a major benefit of implementing a Q-study within the VA is this recurring efficiency argument that came up in the literature, that since you don’t have enough women, you won’t be able to reach a sufficiently-powered study, and therefore it’s not worth doing, and therefore, we’re just not going to worry about what’s going on in women’s health because we don’t have any results.  Then when you try to do something, we’re going to say, “Well, you don’t have the research to back that up.”  It’s just this ugly situation.  

Q is multivariate analysis, so there is, actually, statistics behind what comes out and how your interpretations go.  It’s a little more quantitative.  I don’t want to say robust than qualitative research because that’s not really true.  It gives you a bit more of an idea of scope and presence of a belief.  Since there aren’t any ranking requirements within qualitative research, you can identify themes but you can’t necessarily identify how important they are.  

Q-methodology does allow you to start getting at those types of questions, and not just what opinions are out there, but how many people hold these opinions?  How strongly do they hold these opinions?  You don’t have to have a priority-variable definition.  Since you’re evaluating similarities and not differences, the requirements for the statistic are different.

You don’t have a null hypothesis to reject, first of all; that’s a major difference.  In doing that, you’re not worried about power size or *[interference 33:01]*power analysis.  Smaller sample sizes are accessible, in that respect it is similar to a qualitative study, where if you have 40 individuals, you definitely have enough participants to start understanding where their priorities are, and where their similarities and differences may exist.  You can use that information to then help and inform design of different interventions that can help, specifically, with women’s health, and women’s issues.   

I’m not going to read through every single one of these, but based on the women’s health issues article that Becky provided, there was a big, big chart full of all sorts of different research questions.  I went through and tried to say, “Realistically, if I were to design a Q-study, which would be most appropriate?”  These are the ones I identified.  These are not going to use all of the same exact design.  There were three groups as far as my armchair quarterbacking would go. 

The first group that goes into this prioritization; very similar to the study that I just presented where you’re looking at different options in terms of, say, program components, or features, or things like that, and prioritizing them amongst the population of interest.  The second group is looking more at barriers, and more of an individual component, understanding what’s going on at a personal level.  The third is looking at more of a longitudinal approach.  I’m going to go through the design factors, and what I think could work in terms of these specific research questions.  

Design one is, again, looking at prioritization and strategy.  The overall goal would be to understand the user profiles and understand group needs.  This is similar to what I just presented.  The concourse could be developed from, let’s say, possible interventions from structures of departments, from various features of organizations, really anything that is currently in existence, and options that you might implement, for example.

You could look at the literature, you could look at the way your organization currently handles these things.  There’s often variations.  There are several instances where certain clinics were very good, and worked well for certain types of women’s health issues; other clinics that didn’t.  You could develop statements based on what it is that they do, or physicians, or something along those lines.  You could also interview potential VA users, either on past experiences, or why they’re not using the VA.  

The Q-set could, again, be generic descriptions of these interventions of possible innovations and so on.  Then the condition of instruction could be in terms of something along the lines of what you feel or what the participant feels is most needed or least needed.  What is most useful or least useful to them along those terms.  Then, when you do the interpretation, you would look for the different personality profiles, if you will, in terms of potential users.  Then looking across these different profiles, what types of structures, or features, or services would be of most value to the greatest number of these people.  Start using that as a place to begin development-type work.  

Just to slim it down, again, back to which research questions I thought would be appropriate for setting Q in this specific design, it would be along the lines of evaluating comprehensive women’s primary care models, examining structure care models that support the patient-centered medial home.  Evaluating variations in mental health care needs, use and outcomes of subgroups of women Veterans.  Understanding the aging issues of women Veterans, including needs, use, and preferences.  Evaluating needs and care for disabled women Veterans, and determining reproductive health needs of women Veterans.  

In design two, to my mind, the goal would be focusing, again, on the significance to individuals.  This is a more personal-oriented design.  You would be looking for these “ideal types” within a population, trying to understand what each of these groups are looking for, and then use them to develop and tailor interventions.  Again, looking more of that Pepe Le Pew paradigm that I suggested, using the same interventions, maybe making a little tweak so that it’s more palatable to a given sub-audience or subculture within the women’s health population.

Q-concourse would be more along the lines of experiential themes.  This would be something where you would probably derive it from interviews, or focus groups, and understanding what experiences are out there.  Understanding, broadly, what the experience is to be a woman within the VA system.

Then you can do an inductive design for the Q-set.  Once you identify your theme, you can purposefully create items that reflect mixes of themes.  If one theme was access, and another was the way they’re treated, or something along those lines, you can start to mix and match different themes in ways that make sense so that you can see where the actual breaking point is in terms of the importance.  If somebody really focuses in on their access to care issues, and someone else is focusing more on, say, their experience and respect, you’ll be able to purse that our as part of your interpretation. 

The condition of instruction could likely be structured in terms of what represents me.  It’s all in terms of a person’s experience and, again, this representative concept.  

Specific research priorities that I felt would align with this sort of design would be assessing factors related to women’s Veterans trust of the VA, and other providers in clinic environments.  Understanding similarities and differences between male and female Veterans with sexual trauma, including barriers, needs, and outcomes.  Determining variant to care for woman who attempt or contemplate suicide.  Identify risk factors for suicide among Veterans, and develop combat-exposure measures that reflect women’s experiences.  

Finally, you can also design a Q study the longitudinal development of an individual.  In these types of studies, you’re actually looking for a change within an individual sort over time.  You could combine this with demographic data to analyze both a group of people and individuals across time. 

The Q-concourse could be some sort of metric of function.  You can also use an existing measure, for example.  If you have a validated measure for some sort of health issue, you can adapt those exact same items into a Q-sort just by sticking them on cards and having people sort them according to what best describes them, rather than using a Likert-style scale.  

The Q-set would be a deductive design, so you’d be reflecting that population of metrics.  You wouldn’t have to necessarily use just one survey, you could combine items from many, depending on what it is that you were trying to understand.  Again, the condition of instruction could include something along the lines of what best describes me, or least describes me.  Again, using that personal frame of reference, and having them use items that are already validated in the literature to create more of a broad explanation of, really, what most describes me as opposed to just taking a five down the line.  

Then the data collection would be repeating the Q-sort over intervals.  Just to illustrate this, what would you do is have each person fill out the Q-sorts, and then collect them over some period of time; six months, a year, however long it is that you’re able to do it.  

You could do analysis over time, across each of these groups.  Then you could also look at it across one individual and how their perceptions change, or how their experience changes over time.  It might be interesting to see if there were any consistent patterns among an individual experience that could give you some insight into what’s going on with people over time that might be linked to, say, resilience, for example, or other concepts with importance to you.

The specific research priorities that I felt would work with this sort of design would be understanding the impact of mental health on sexual health and reproductive health over the lifetime.  Evaluating functional status, quality of life, and resilience post-deployment, in addition to physical and mental health.  Evaluating impact of multiple deployments on women Veterans and their families.  Then examining impacts of first experiences with reproductive health services, on their perceptions of care, and on later use.  

That’s basically all of my thoughts, my armchair quarterbacking, and an introduction to the methodology, what it can do.  Posing to the audience, are you interested in possibly using Q in the future?  I’ll take a yes even if it’s just because you’re a Trekkie and you liked my Q reference at the beginning. 

*[Pause 43:33 – 43:49]*

It looks like there’s a significant group of you, about 85 percent.  We’re dropping.  80 to 85 percent, it’s fluctuating a little bit, who are interested in using Q in the future.  I’m very gratified to hear that.  I’m glad that I have convinced more people to, hopefully, implement this type of design in the future.  If you are interested, please do reach out and contact me.  I’m always looking for new people with whom I can collaborate.

I am here as an Assistant Professor here at UIC, so I am young and hungry.  If you need young investigator street cred, I am your girl.  I also have a small niche consulting firm, so if it is easier for you to hire me strictly on a consulting basis, I can accommodate that as well. 

Here is my contact information.  Mnaima1@uic.edu, and my phone number.  

If you’re interested in reading more about Q-methodology, I have two books that I would highly recommend: *Doing Q-Methodological Research*by Watts and Stenner, and *Q Methodology*by McKeown and Thomas.

They just actually came out with a brand new one.  It was just released in June or something.  Do the links work on here?  On the actual slides, there are links to Amazon if you want to see the exact ISBN and all that information.  

Also if you have any questions, please do contact me.  I’m happy to talk to you and talk your ear off about Q-methodology.  I think that’s it for me. 

*Moderator:*Fantastic.  We’ll turn it over to questions at this point.  We have gotten a few in.  

*Dr. Naiman:*Oh, excellent. 

*Moderator:*What is concourse development? 

*Dr. Naiman:*Concourse development is trying to understand the flow of communicability.  One of the big developers of Q-methodology, Dr. Brown out of Iowa.  He referred to it as the flow of communicability surrounding a topic.  It’s really just trying to understand all of the constructs that surround your topic of interest in order to be able to sample effectively within the Q-set. 

*Moderator:*Correlations between where people put the cards on the distribution. 

*Dr. Naiman:*Yes.  That’s the correlation.  If all of their cards versus all of the other person’s cards is how the correlation is, essentially, constructed.  It’s not just one card *[inaudible 46:33].*It’s all of their cards versus somebody else’s all of their cards, and an average that way. **

*Moderator:*I thought PCA was used to confirm hypothesis, not for exploratory work? 

*Dr. Naiman:*Yes and no.  *[Chuckles]*It can be used—and it is controversial.  You touched on a very big controversy within the Q community.  The reason I justified using PCA for this particular study was because what PCA does is to come up with the optimized solution, essentially, based on the number of factors that you assign.

For this particular study, it was really an optimization problem that I was using Q to get information on.  Because it was about understanding where there balance was in terms of likelihood or preference in terms of providing care.  It would be utilized as such.  From that perspective, I felt PCA was appropriate.

Now, that being said, within the Q-methodology community, there are people who say you should absolutely never, ever, ever, ever, ever use anything besides Centroid because it is an exploratory technique.  Whereas PCA is a mathematically optimized solution, and therefore you can’t do hand rotation.  It limits your theoretical play, essentially.

Typically, that’s correct.  You usually do not use PCA, but you can if there is a good reason to do so.  For this particular study, I felt it was consistent, but I think in general you would probably stick with Centroid if you were going for something that was a lot more exploratory, like trying to understand women’s experiences more broadly within the context of the VA system.  That’s my rambling answer.  *[Chuckles]*

*Moderator:*What were the factor loadings for the four factors you showed? 

*Dr. Naiman:*I would have to go back and look at that.  If whoever that is wants to send me an email, I can send them the full analysis.  I don’t remember that off the top of my head.  *[Chuckles]*Good question. 

*Moderator:*We don’t expect you to know everything, so I always say that’s a fair response to say that.  The next question or comment here looks like applying this to rollout of Telehealth to Veterans would also be an important application of this method. 

*Dr. Naiman:*Sure, yeah.  Anything that involves people’s perceptions or preferences regarding a technology could definitely be an application. 

*Moderator:*Would it be accurate to compare Q-methodology to cluster or class analysis?

*Dr. Naiman:*In as much as you’re looking—full disclosure, not as familiar with cluster or class analysis.  I’m a Q-person.  In as much as you are looking for this similarities, yes.  I think part of the difference would be in the getting to those items, I think is one major difference.  That we have more of a structured way of getting to the actual Q-sets.  I’d have to think more, but I think that would be the first thing that jumps out at me.  I mean, I think cluster analysis—I’m trying to remember exactly how the math works*[chuckles]*I’m failing because again I don’t—I’m glad I don’t have to know everything.  Yeah, I think it’s more along those lines of the underlying philosophy and process is a little different even if the end result is looking at those similarities.  

*Moderator:*Is the sort and grid restricted to five, three, one?

*Dr. Naiman:*No. 

*Moderator:*How did you get from 53 statements to this small grid?

*Dr. Naiman:*This is just an example.  I couldn’t fit all 53 on the PowerPoint slide.  The actual 53 statements was 43, and there was a 43-item grid that was spread out in that distribution.  You can set it up—as long as it’s symmetrical—really any way that you want.  

Some of the earlier studies had just a square where, let’s say, you had 50 items.  You would have ten columns of five, for example.  You can do it however you want, it’s just matter of how specific you want to be in terms of how much space you give people to express their extreme opinions.  

*Moderator:*Can you recommend literature on issues of design such as determining what are the factors and what is the process like? 

*Dr. Naiman:*Yeah, doing Q-methodological research is really great in terms of really taking you step-by-step in about what you should consider in the factor solution.  I haven’t read the newer Q-methodology book, but the old one from the ‘80s, it was also pretty good, but it was really, really step-wise and clearly articulated in the Watson/Stenner book.  I would highly recommend that. 

*Moderator:*That actually concludes all of the questions that we currently have.  Melissa, did you have any final remarks you wanted to make before we close things out?

*Dr. Naiman:*Just thank everyone for their participation and interest.  Please, again, I encourage you to contact me if you are really interested in trying to implement a Q-methodology study yourself. 

*Moderator:*We really want to thank you for taking the time to prepare and present today.  I know we really do appreciate it.  For the audience, thank you everyone for joining us today.  As I close out the session, you will have a feedback form pop up on your screen.  We would really appreciate if you would take a few moments to feel that out.  We really do read through all of your feedback, and use it to implement in our current and upcoming session.  Thank you everyone for joining us for today’s HSR&D cyber seminar, and we hope to see you at a future session.  

*Dr. Naiman:*Thank you. 

*[End of Audio]*

Page 1 of 16