Using Negatively Framed Questions to Evaluate Nursing Home Care 

Elizabeth M. Frentzel, MPH1 

Judith A. Sangl, ScD2 

Carol Cosenza, MA3 

Julie Brown4 

Chris Evensen, MS1

San Keller, PhD1

1American Institutes for Research, 101 Conner Drive, Suite 301, Chapel Hill, NC 27514 

2Agency for Healthcare Research and Quality, Rockville, MD 

3University of Massachusetts, Boston, MA 

4RAND Corporation, Santa Monica, CA 

Introduction 

Negatively framed survey items use a negative construction such as ―are you angry?‖ and ―are you not happy?‖ and are not 

commonly used in surveys. This paper discusses our rationale for including negatively framed survey items in the CAHPS® 

Nursing Home Survey for Family Members (Nursing Home Family Survey). We describe the process of how we weighed 

several statistical measures for items and their composites against (a) results and recommendations from our literature review, 

(b) focus groups and (c) different stakeholders on our Technical Expert Panel to decide on including or excluding a set of 12 

negative items. 

Background on CAHPS® Surveys and the Family Member Survey 

The Consumer Assessment of Healthcare Providers and Systems (CAHPS®) is a family of survey instruments designed to 

capture and report people’s experiences obtaining medical care. These surveys are available at the facility level (such as 

hospitals, dialysis centers) and ambulatory level (such as doctors’ offices and health plans). The CAHPS surveys are 

developed by following a set of design principles to ensure that the data are reliable, valid, credible and useful. These 

principles include: a) focusing on consumer/patient reports and assessment of the quality of their experience; b) focusing on

items for which the respondent is the best or only source of information; c) obtaining stakeholders’ input; and d) conducting 

rigorous cognitive and psychometric testing (Crofton, Lubalin, & Darby, 1999). 

Because the government (federal and state combined) pays for almost two-thirds of the $131 billion of total nursing home 

costs, the Centers for Medicare & Medicaid Services (CMS) are interested in the consumers’ perspective on the quality of 

care they receive (CMS, 2008). As the federal agency responsible for nursing home quality oversight, CMS has supported the 

development of a consumer experience survey for both residents and their family members. The family member survey was 

designed to complement the resident survey, not serve as a proxy for those who cannot respond. 

The nursing home setting is unique in that quality-of-care (QoC) is comingled with quality-of-life (QoL) since this is the 

residents’ home as well as a place where they receive health care. Residents receive care including medical and rehabilitation 

care and also personal care assistance with activities of daily living (ADLs) such as eating, toileting, walking, dressing, 

bathing–so the nursing home experience is unlike a person visiting a doctor as a discrete medical event but is experienced as 

a continuous/seamless health and personal care experience. 

Another important feature of the nursing home environment is the imbalance of power between residents and staff. Because 

of their physical and/or mental limitations, residents are deeply dependent on staff for personal and medical care needs 

everyday. Most long stay residents (and their family members as well) are unable to do more for themselves (or their family 

members), cannot easily leave or are not empowered enough to complain to change conditions about which they are not 

happy (Nelson, 2000). Applying the social exchange theory to the nursing home setting, residents’ inability to reciprocate is 

the major element of their powerlessness and vulnerability (Nelson, 2000). Because of their vulnerability, fear of retaliation is 

often mentioned as a reason for nursing home residents not to complain (National Citizens’ Coalition for Nursing Home 

Reform, 1985; Institute of Medicine, 1995; Monk, Kaye, & Litwin, 1984). This theme of fear of retaliation was expressed in 
the focus groups we conducted (Frentzel, Dardess, & Carman, 2005; Rand, 2005). Formal complaints are considered to be an 

underestimate since residents and family members have concerns about filing a formal complaint, in part because of fear of 

reprisals and because complaining is perceived to be futile (Hawes, 2002). 

Use of Negative Items in Surveys 

Use of both negative and positive items to reduce response set bias or guard against acquiescent behaviors has been used 

extensively historically (Benson & Hocevar, 1985; Chronbach, 1950; Nunnally, 1978; and Wright & Masters, 1982). One 

purpose is to prevent respondents from generally agreeing with survey statements. The assumption is that both positive and 

negative items are perceived as the same construct by respondents. However, the value of using negative and positive items 

has been questioned by some studies in the past two decades that show using both negative and positive items have not

measured the same construct, have decreased internal consistency and/or have confused respondents due to changing from

positive and negative statements. (Melnick & Gable, 1990; Pilotte & Gable, 1990; and Barnette, 2000). 

Negatively framed items are not common in CAHPS: Most CAHPS surveys use only positively framed or neutral items. 

Negative items are found in three CAHPS surveys: the CAHPS Health Literacy Supplemental set for clinician and group 

practices (n=4); the CAHPS Health Literacy Supplemental for the hospital survey (n=8) (in development); and the CAHPS 

Cultural Competency surveys (n=12), the latter two are in development. An example of a negatively worded CAHPS item in 

these surveys include, ―how often did the nurses use a rude tone or manner with you?‖ In the pilot version of the Nursing 

Home Family Survey, there were 12 negatively framed items (Table 1), of which we retained nine in the final version. 

Table 1. List of Negatively Framed Items in pilot version 

Item 

Number 

Item text 

15. In the last 6 months, did you ever see any nurses or aides be rude to your family member or any other resident? 

Yes/No (Y/N) 

16. In the last 6 months, did you ever see any nurses or aides treat your family member or any other resident 

roughly? Y/N 

17. In the last 6 months, did you ever see another resident be rude to your family member (or any other resident)? 

Y/N 

19. [If you helped your family member with eating,] was it because the nurses or aides either didn’t help or made 

him or her wait too long? Y/N 

21. [If you helped your family member with drinking,] was it because the nurses or aides either didn’t help or made 

him or her wait too long? Y/N 

23. [If you helped your family member with toileting,] was it because the nurses or aides either didn’t help or made 

him or her wait too long? Y/N 

31. In the last 6 months, did the nurses and aides ever try to discourage you from asking questions about your 

family member? Y/N 

36. In the last 6 months, did you ever see the nurses and aides fail to protect any resident’s privacy while the 

resident was dressing, showering, bathing, or in a public area? Y/N 

37. Personal medical belongings are things like hearing aids, glasses, and dentures. In the last 6 months, how often 

were your family member’s personal medical belongings damaged or lost? Never, Once, Two or more times 

39. In the last 6 months, when your family member used the laundry service, how often were clothes damaged or 

lost? Never, Once or twice, three or more times 

40. In the last 6 months, were you ever unhappy with the care your family member received at the nursing home? 

Y/N 

43. In the last 6 months, did you ever stop yourself from talking to any nursing home staff about your concerns 

because you thought they would take it out on your family member? Y/N 

For the Nursing Home Family Survey, one purpose of using negative items has been to measure unique constructs that could 

not be framed as a positively written item. Because of the significant issue of staff failing to respond to requests for assistance 

in nursing homes (NORS, 2008), the CAHPS team developed items that would explore the issue of staff responsiveness on 

the most essential ADLs – eating, drinking fluids and toileting. For example, the survey asks a screener question, ―In the last 

6 months, during any of your visits, did you help your family member with eating?‖ and if yes, the survey then asks, ―Was it 
because the nurses or aides either didn’t help or made him or her wait too long?‖ Two similar items to this on drinking and 

toileting are also asked. Had these items been written in a positive frame, e.g., ―How often did the nurses or aides help your 

family member with eating?‖ the items would capture a different construct. 

Similarly, the item, ―in the last 6 months, did the nurses and aides ever try to discourage you from asking questions about 

your family member?‖ would not have the same meaning if positively framed, for example, ―in the last 6 months, did the 

nurses and aides ever try to encourage you to ask questions about your family member?‖ When positively framed, it 

measures a different construct that does not help to elucidate a potentially significant problem in the nursing home. Monk, 

Kaye, and Litwin (1984) found that about one-fifth of the patient population held back from talking about a problem because 

of fear of reprisals. 

Methods 

The Nursing Home Family Survey followed a similar process as all other CAHPS survey development projects (see Exhibit 

1, below). The Nursing Home Family Survey was the second nursing home survey that the CAHPS team developed, the first 

being the CAHPS Nursing Home Resident Survey (Sangl et al., 2007). Drawing on literature from the resident survey, a 

literature review previously conducted by the University of Colorado, and additional more recent research, the team 

developed a list of domains and ultimately a draft survey (Congdon, et al, 2004; Frentzel, Evensen, Keller, & Garfinkel, 

2008). 

Exhibit 1: Process for CAHPS Survey Development 

Cognitive Interviews

Focus groups with 

family members

Call for measures

Stakeholder input

Draft Survey

Draft Pilot Survey

Revise Survey

Cognitive Interviews Conduct Pilot Study

Analyze Data

Stakeholder Input 

Finalize Survey

Literature review

The team supplemented the items with data from focus groups with family members conducted in 2005 and a call for 

measures in the Federal Register in 2004. We used the results from the call for measures, the literature review, and the focus 

groups, to develop a list of initial domains and questionnaire items within each domain. A total of 90 items were developed 

for this initial survey instrument. 

We then conducted two rounds of cognitive testing. The purpose of the cognitive testing is to identify and eliminate problems 

associated with comprehension and navigation of the questionnaire items. The interviews will mitigate problems associated 

with ambiguous wording, respondent knowledge, and literacy, and will reduce respondent burden. The team conducted a total 

of 27 interviews in the first testing round in June 2005 and conducted another 27 interviews in the second round in June 

2006. 

Following the first round of cognitive tests, we revised the items and prepared a protocol for the second round of cognitive 

testing. This round tested the items as both self-administered items and as interviewer-administered items under the 

assumption that the final instrument would probably be administered by both mail and telephone. The instrument was revised 

again after the second round of cognitive testing. 

Between the two rounds of testing, the team met with a Technical Expert Panel (TEP) in late 2005 to obtain their guidance 

and suggestions about the draft survey and composites or groups of related items. The TEP included representatives from the 

nursing home industry, regulators and quality improvement organizations, consumer advocates (including nursing home 

ombudsman), providers, and long-term care researchers. 
The instrument was field tested in east Texas from early October 2006 to January 2007. The Texas State Long Term Care

Ombudsman recruited 15 Texas nursing homes and the Health Quality Council. In addition, the Health Quality Council of 

Alberta conducted an independent field test in Alberta and contributed their data to AIR for the psychometric analysis 

conducted by AIR. The TEP provided a review afterwards to ensure that necessary substantive items were included. 

Psychometric Analysis 

Prior to data collection and in consultation with stakeholders, we organized 31 of the substantive survey items into a set of 

five domains (Getting Care Quickly; Quality of Care by Nurses and Aides; Communication of Nurses and Aides; 

Communication—Other Staff and Administrators; and Nursing Home Environment). We conducted a confirmatory 

psychometric analysis using both the Alberta and Texas data and calculated a Cronbach’s alpha for each of the five domains. 

The alphas for three of the five domains were greater than the standard 0.70, but the alphas for two of the domains were 

lower. While these results provided some support for the hypothesized relationships, the item-total correlations for the 

domains indicated that each domain’s alpha would improve if certain items were dropped, which indicated that it was likely 

that there was a better item-domain structure to be specified. Separate analyses of the Alberta and Texas data replicated these 

findings. 

We then conducted exploratory analyses to identify composites. To make use of all available data, we obtained maximum 

likelihood estimates of the covariance matrix under the Missing at Random (MAR) model using a multiple imputation 

procedure (Rubin, 1976; Rubin, 1987; see also Hurtado, Angeles, Blahut & Hays, 2005; Keller, et al, 2005; and O’Malley, et 

al, 2005 for the use of the MAR model and SAS PROC MI in CAHPS surveys).

 

We conducted an exploratory factor analysis (EFA) on the correlation matrix produced by PROC MI. The EFA used the 

principle factor method with squared multiple correlations as initial communality estimates, and oblique rotation (promax) 

with Kaiser normalization. The number of factors was determined by the eigenvalues and the interpretability of the rotated 

factor pattern matrix. The EFA results did not demonstrate a definitive underlying factor structure for these 31 items. The 

instrument development team considered these results along with prior research on nursing home quality assessment and 

input from stakeholders such as nursing home administrators as well as nursing home patient ombudsman. The alternative 

composite structures that were suggested and evaluated using multi-trait multi-item analysis (Hays, Hayashi, Carson, & 

Ware, 1988; Ware, Harris, Gandek, Rogers, & Reese, 1997). This analysis computes the correlation of each item with each

hypothesized scale corrected for overlap (item-to-total correlation), and produces statistics that indicate whether an item is 

more highly related to its own scale than to the other composites (scaling success). In other words, the analysis measures

convergent and discriminant validity of an item as an indicator of the quality composite to which it is hypothesized to belong. 
Findings 

The team agreed on a final set of 21 total items (nine of which were negatively framed with the ninth item becoming a new 

global) organized into four composites. Not all items were included. The team, in consultation with the Technical Expert 

Panel eliminated ten items from the survey, two of which were negative items. In order to determine which items to keep and 

which items to eliminate, we closely reviewed the psychometric properties of the composites and items. We used an interunit reliability statistic to determine how well items and composites were able to detect differences – or discriminate – across 

nursing homes. We assessed the convergent validity of the items within each composite by examining the item-to-total 

correlations and the factor loadings.. In every case these ten items had either poor convergent or discriminant validity with 

their composite, did not discriminate among nursing homes, and/or contained content that was included elsewhere in the 

survey. Two of the negative items that were retained among the 21 had marginal measurement properties but were very 

important to the consumer advocates and nursing home resident ombudsman in our stakeholder panel and were also indicated 

as a significant problem by Ombudsman data (NORS, 2008). These were included to maintain the content validity of the 

survey. The eight negatively-framed items were kept either because they had good or excellent measurement properties or 

because they discussed aspects of care that ombudsman had identified as particularly important to patients (see Table 2, last 

column). 

Overall, the negative items have similar nursing home reliability scores and factor loadings compared to positive items. For 

item to total correlations, negative items have lower correlations than positive items but generally above the targeted or 

recommended minimum (targets are noted in parentheses at the top of the columns in Table 2 (Targets for inter-unit 

reliability are based on previous work cited in Keller et al., 2005; Targets for factor loadings are from Nunnally, 1978; and 

targets for the item-to-total correlation came from are based on previous work cited in Keller et al., 2005). However, for the 

purpose of developing composites, the attributes the CAHPS instrument team cares strongly about are the composite-level 

scores. 

Table 2: Statistics for Negative Items and Their Composites 

Item 

Nursing 

home 

reliability 

(≥ 0.70) 

Factor 

Loadings 

(>0.30) 

Item-to-total 

Correlation 

(>0.40) 

Rationale for keeping 

Composite 1: 

Meeting Basic 

Needs 

0.48 

19. Wait too long 

[for eating help]? 

Y/N 

0.46 0.65 0.83 These items represent a common 

complaint to ombudsmen and 

measure the vulnerability of 

residents; also, NH-level reliability 

will increase if a larger sample size 

is feasible. 

21. Wait too long 

[for drinking 

help]? 

0.32 0.62 0.83 

23. Wait too long 

[for toileting 

help]? 

0.55 0.78 0.79 

Composite 2: 

Kindness and 

Respect 

0.83 

15. Nurses or aides 

rude to family 

member? 

0.64 

(0.16 – 0.82 

for positive 

items) 

0.51 

(0.68 – 0.85) 

0.50 

(0.72 – 0.82) 

No statistical or substantive 

concerns 

Composite 3: Info 

& Encourage 0.85 

31. Nurses or aides 

discourage 

questions about 

family member? 

0.48 

(0.63 – 0.85 

for positive 

items) 

0.31 

(0.47 – 0.74) 

0.31 

(0.58 – 0.74) 

Focus group data and 

advocates/ombudsmen strongly 

supported item 
Table 2: Statistics for Negative Items and Their Composites 

Item 

Nursing 

home 

reliability 

(≥ 0.70) 

Factor 

Loadings 

(>0.30) 

Item-to-total 

Correlation 

(>0.40) 

Rationale for keeping 

Y/N 

43. Ever stop 

yourself from 

talking to any 

nursing home 

staff? Y/N 

0.50 

(0.63 – 0.85) 

0.48 

(0.47 – 0.74) 

0.47 

(0.58 – 0.74) 

Literature review, focus groups and 

advocates/ombudsmen strongly 

supported item 

Composite 4: 

Staffing, 

Belongings, & 

Cleanliness 

0.89 

37. Family member’s 

personal medical 

belongings 

damaged or lost? 

0.21 

(0.76 – 0.88 

for positive 

items) 

0.33 

(0.58 – 0.71) 

0.25 

(0-49 – 0.60) 

Focus group data and 

advocates/ombudsmen strongly 

supported item 

39. Family member’s 

clothes damaged 

or lost? 

0.82 

(0.76 – 0.88) 

0.46 

(0.58 – 0.71) 

0.35 

(0.49 – 0.60) 

Slightly lower than recommended 

item-total correlation and focus 

group and advocate/ombudsmen 

strongly supported item 

For the 9th negative question, item 40, ―In the last 6 months, were you ever unhappy with the care your family member 

received at the nursing home?‖ the team decided to keep this as a new global item. In the CAHPS team opinion, this item 

acted more like a global item although it has a high nursing home reliability, it did not did not scale well with any of the 

composites. 

The internal consistency reliability (Cronbach’s alpha) was relatively high for all four composites, which indicates that the 

scores would provide reliable data. Notably, the composite with the highest alpha is also the first CAHPS composite 

composed entirely of negative items. The mean scores indicate that responses were well distributed. 

Table 3: Final Composite Decisions and Composite Level Psychometrics 

Composite Total 

# of items Total number of negative items Mean (SD) 

Alpha 

(>0.70) 

1. Basic Needs 3 3 73.5 (39.3) 0.90 

2. Staff Kindness 

and Respect 5 1 84.8 (19.9) 0.88 

3. NH Info & Encouragement 6 2 87.4(17.0) 0.78 

4. NH Staffing, 

Care & Cleanliness 7 2 80.5(17.1) 0.79 

Three negatively framed items were removed from the final survey primarily because of poor item-level statistics. Both the 

nursing home reliability scores were low as were the correlation with the global questions (were you ever unhappy with the 

care, would you recommend the nursing home, and the overall global rating of the nursing home). In addition to the poor 

statistics, the team and the Technical Expert Panel also agreed that these items were not as critical as other items. 

Table 4. Items Removed from The Survey and Justification 

# Items 

Nursing 

Home 

Reliability 

(> 0.70) 

Correlation with 

Global Questions Justification for Removing 
16 

Nurses or aides 

treat family 

member roughly 

0.14 0.24 – 0.32 

Marginal measurement characteristics: Low nursing homereliability and did not correlate highly with global questions. 

In addition, item 15 (―rude‖ question) facilitated assessing 

issues similar to this concept. 

17 

Another resident 

rude to family 

member 

0.08 0.20 –0.26 

Marginal measurement characteristics: Low nursing homereliability and did not correlate highly with global questions. 

This was also not considered critical by the team or the 

technical expert panel. 

36 

Nurses or aides 

didn’t protect 

modesty 

0.56 0.20 – 0.23 

Marginal measurement characteristics: Low nursing homereliability and did not correlate highly with global questions. 

This was also not considered critical by the team or the 

technical expert panel. 

Discussion 

The composite structure that the team agreed to presents the best combination of conceptual properties and statistical support. 

Table 5 presents the final composite structure. In some cases, the team opted to retain items that were not as statistically 

strong, but based on input from the Technical Expert Panel, focus group data, cognitive interview, and background 

information from the national ombudsman program, these items were retained. 

Table 5. Final Composite Structure and Items 

# Composite or Item Handle 

Composite 1: Meeting Basic Needs: Help with eating, drinking, toileting 

19* Wait too long for help with eating 

21* Wait too long for help with drinking 

23* Wait too long for help with toileting 

Composite 2: Nurses or Aides' Kindness and Respect Towards Resident 

12 Nurses or aides treat resident with respect 

13 Nurses or aides treat resident with kindness 

14 Nurses or aides really cared about resident 

15* Nurses or aides rude to resident 

26 Nurses or aides appropriate with violent resident 

Composite 3: Nursing Home Provides Info/ Encourages Respondent Involvement 

29 Nurses or aides give respondent information about resident 

30 Nurses or aides explain things to respondent 

31* Nurses or aides discourage respondents from asking questions 

43* Respondent stops self from complaining 

45 Respondent involved in decisions about care 

53 Respondent given information about payments or expenses 

Composite 4: Nursing Home Staffing, Care of Belongings, and Cleanliness 

11 Can find a nurse or aide 

51 Enough nurses or aides 

32 Room looks and smells clean 

24 Resident looks and smells clean 

35 Public areas look and smell clean 

37* Family member’s medical belongings lost 

39* Family member’s clothes lost 

*Negative items 

In 2008, the top complaint of nursing home residents and their families, eliciting some 14,329 complaints to ombudsmen, 

was failing to respond to requests for assistance. The first composite, meeting basic needs, covers the top complaint identified 

by ombudsmen, indicating a critical need to assess how well and how poorly a nursing home provides basic care. Specific 

complaints relating to these items include lack of assistance with toileting which had 3,404 complaints; lack of assistance 
with drinking which had 2,899 complaints; and lack of assistance with eating which had 1,529 complaints (NORS, 2008). 

Similarly, most of the other negative items were also major sources of complaints. While no specific complaint used the word 

―rude‖, complaints relating to dignity, respect and staff attitudes totaled 9,075. Fear of reprisals totaled 687—which may not 

seem high, but given the research indicating that people seldom complain about fear of reprisals, it suggests a significant 

issue. Finally, loss of laundry was mentioned 1,771 times in 2008. 

Our final model rested both on statistical findings and substantive issues raised by multiple sources, including family 

members of residents via our focus groups, cognitive testing, and our Technical Expert Panel. While a few items were not 

statistically strong, we ultimately opted to keep these items because they were substantively meaningful despite poor 

statistics. As one focus group participant stated, ―I want to know those [nursing home] places that are terrible...I want to 

know that as well‖ (Rand, 2005). Retaining negatively framed items that meet many of the psychometric criteria for either 

item or composite level achieves this goal of providing information about the less positive aspects of nursing homes which 

consumers may wish to know to avoid selecting such homes. 

References 

Barnette, J. J. (2000). Effects of stem and likert response option reversals on survey internal consistency: If you feel the need, 

there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 

60 (3): 361-370.

Benson, J. & Hocevar, D. (1985). The impact of item phrasing on the validity of attitude scales for elementary school

children. Journal of Educational Measurement, 22: 231-240. 

Centers for Medicare & Medicaid Services. (2008). National Health Expenditures: 2007 Highlights. Retrieved December 31,

2009 from http://www.cms.hhs.gov/NationalHealthExpendData/downloads/highlights.pdf. 

Congdon, J. G., Magilvey, J.K., Jones, K.R., Morgenstern, N. E., Vojir, C.P., Dingley, C. E., Scott, J. A., & Kramer, A.M. 

(2004). Quality Factors in Nursing Home Choice. A technical report submitted to the Agency for Healthcare 

Research and Quality under grant # R18 HS10926-03. University of Colorado Health Sciences Center. 

Crofton, C., Lubalin, J. & Darby, C. (1999). Foreword (March 1999 supplement) Medical Care, 37(3): MS 1-MS9. 

Cronbach, W. J. (1950). Further evidence on response sets and test design. Educational and Psychological Measurement, 10: 

3 – 31. 

Frentzel, E., Dardess, P., & Carman, K., (2005). Reporting nursing home quality: Draft results from focus groups with people

who have not yet chosen a nursing home. Report submitted to the Agency for Healthcare Research and Quality. 

Washington DC: American Institutes for Research. 

Frentzel, E., Evensen, C., Keller, S., Garfinkel, S. (2008). CAHPS survey for family members of nursing home residents. 

Final Report submitted to the Agency for Healthcare Research and Quality. Washington DC: American Institutes for 

Research. 

Hawes, C. (2002). Elder abuse in residential long-term care facilities: What is known about prevalence, causes, and 

prevention. Testimony Before the U.S. Senate Committee on Finance. 

Hays, R. D., Hayashi, T., Carson, S., & Ware, J. E. (1988). User's guide for the multitrait analysis program (MAP). Santa 

Monica, CA: Rand Corporation. 

Hurtado, M., Angeles, J., Blahut, S.A. & Hays, R., (2005). Assessment of the equivalence of the Spanish and English 

Versions of CAHPS® Hospital Survey. Health Services Research, 40 (6) Part II: 2140–2161. 

Institute of Medicine. (1995). Real People, real problems: Evaluation of the long-term care ombudsman program of the 

Older Americans Act. Washington, D.C.: National Academy Press. 
Keller, S.D., O’Malley, A.J., Hays, R.D., Zaslavsky, A.M., Hepner, K.A. & Clearly, P.D. (2005). Methods used to streamline 

the CAHPS® Hospital Survey. Health Services Research ,40(6) Part II: 2057-2077. 

Melnick, S. A. & Gable, R. K. (1990). The use of negative item stems: A cautionary note. Educational Research Quarterly, 

14 (3): 31-36. 

Monk, A., Kaye, L.W., & Litwin, H. (1984). Resolving grievances in the nursing home: A study of the ombudsmen program. 

New York: Columbia University Press. 

National Citizens’ Coalition for Nursing Home Reform. (1985). A consumer perspective on quality care: The residents’ point 

of view. Washington DC: National Citizens’ Coalition for Nursing Home Reform. 

National Ombudsmen Reporting System. (2008). Top 20 complaints by category for nursing facilities (FFY 1996-2008). 

2008 National Ombudsman Reporting System Data Tables (Unlettered Tables in Appendix B). Retrieved on 

December 31, 2009 from 

http://www.aoa.gov/AoARoot/AoA_Programs/Elder_Rights/Ombudsman/National_State_Data/2008/Index.aspx. 

Nelson, H. W. (2000). Injustice and conflict in nursing homes: Toward advocacy and exchange. Journal of Aging Studies, 14 

(1): 39-61. 

Nunnally, J. C. (1978). Psychometric theory (2nd ed). New York: McGraw Hill. 

O’Malley A.J., Zaslavsky, A.M., Hays, R.D., Hepner, K.A., Keller, S.A. & Cleary, P.D. (2005). Exploratory factor analyses 

of the CAHPS Hospital Pilot Survey responses across and within medical, surgical and obstetric Services. Health 

Services Research. 40(6), Part II: 2078-2095. 

Pilotte, W. J. & Gable, R. K. (1990). The impact of positive and negative item stems on the validity of a computer anxiety 

scale. Educational and Psychological Measurement, 50: 603-609. 

RAND Corporation (2005) Focus group with family members of nursing home residents, Los Angeles, conducted between 

4/21/05 to 4/26/05. 

Rubin, D.B. (1976). Inference and missing data. Biometrika, 63: 581-592. 

Rubin, D.B. (1987). Multiple imputation for nonresponse in surveys. New York: John Wiley & Sons, Inc. 

Sangl J., Buchanan J., Cosenza C., Bernard S., Keller S., Mitchell N., Brown J., Castle N., Sekscenski E., & Larwood D. 

(2007). The Development of a CAHPS Instrument for Nursing Home Residents (NHCAHPS). Journal of Aging and 

Social Policy, 19(2): 63-82. 

Ware J.E., Harris W.J., Gandek B., Rogers B.W., & Reese P.R. (1997). MAP-R for Windows: Multitrait/multi-item analysis 

program – revised user’s guide. Boston, MA: Health Assessment Lab. 

Wright, B. D. & Masters, G., N. (1982). Rating scale analysis. Chicago: Mesa Press.