CMS’s goals for Nursing Home Compare and its Five-Star System are consistent with its strategy to improve the quality of health care by providing transparent information about the quality of health care services, including those delivered in nursing homes. According to the strategy, to achieve better care, patients must be given access to understandable information and decision support tools that help them manage their health and navigate the health care delivery system. Since 1998, CMS has publicly reported information on nursing home quality on its Nursing Home Compare website and has increased the amount of information reported on the website over time. CMS initially reported information only about nursing home characteristics and nursing home health inspection results on Nursing Home Compare. Later, CMS began reporting additional information on the website, such as the ratio of nursing staff to residents, nursing homes’ performance on various quality measures, and the number of complaints registered against nursing homes. Additionally, CMS has updated the appearance and functionality of the Nursing Home Compare website over time, with the most significant change being the introduction of the Five-Star System in 2008. In December 2008, CMS made the Five-Star System available to the public on its Nursing Home Compare website in order to help consumers compare nursing homes more easily. The Five-Star System assigns each nursing home participating in the Medicare or Medicaid programs an overall “star” rating, ranging from one to five. Nursing homes with five stars are considered to have much above average quality, while nursing homes receiving one star are considered to have much below average quality. Calculation of the overall star rating is based on separate ratings that nursing homes receive for each of three components: health inspections, staffing, and quality measures. Health inspection rating. CMS contracts with state survey agencies to conduct unannounced, on-site nursing home health inspections— known as standard surveys—to determine whether nursing homes meet federal quality standards. Every nursing home receiving Medicare or Medicaid payment must undergo a standard survey not less than once every 15 months, and the statewide average interval for these surveys must not exceed 12 months. State surveyors also conduct complaint investigations in response to allegations of quality problems. If nursing homes are found to be out of compliance with any requirements, state surveyors issue deficiency citations that reflect the scope (number of residents affected) and severity (level of harm to residents) of the deficiency. Surveyors conduct revisits to the nursing home to ensure that the deficiencies identified have been corrected. A nursing home’s health inspection rating is relative to other nursing homes’ health inspection ratings in their state. As such, health inspection ratings are assigned to generally achieve the following distribution within each state: the top 10 percent of nursing homes receive five stars, the bottom 20 percent receive one star, and the middle 70 percent of nursing homes receive two, three, or four stars. Staffing rating. Nursing homes self-report staffing hours worked for a 2-week period at the time of the standard survey. CMS converts the reported point-in-time staffing hours for nursing staff—registered nurses, licensed practical nurses, and certified nursing assistants— into measures that indicate the number of registered nurse and total nursing hours per resident per day. CMS adjusts the staffing levels for differences in the level of complexity of nursing services required to care for residents across nursing homes—referred to as resident acuity. Each nursing home’s staffing rating is assigned based on how its total nursing and registered nurse staffing levels compare to the distribution of staffing levels for freestanding homes in the nation and staffing level thresholds identified by CMS. Quality measure rating. Nursing homes regularly collect assessment information on all their residents, including information on the residents’ health, physical functioning, mental status, and general well-being. Nursing homes self-report this information to CMS. CMS uses some of the assessment information to measure the quality of certain aspects of nursing home care, such as the prevalence of pressure sores and changes in residents’ mobility. At the time of our analysis, CMS calculated this rating for each nursing home based on 11 of the 18 quality measures posted on Nursing Home Compare. Information on the remaining 7 quality measures is posted on the website but not used in the calculation of the rating. A nursing home’s quality measure rating is assigned based on national thresholds established by CMS. The overall star rating is calculated using a process that combines the star ratings from the health inspection, staffing, and quality measure components—with the greatest weight given to the health inspection rating. The overall rating is assigned based on the following steps: 1. Start with the number of stars for the health inspection rating. 2. Add one star if the staffing rating is four or five stars and also greater than the health inspection rating. Subtract one star if the staffing rating is one star. The overall rating cannot go above five stars or below one star. 3. Add one star if the quality measure rating is five stars. Subtract one star if the quality measure rating is one star. The rating cannot go above five stars or below one star. See figure 1 for an example of how a nursing home’s overall rating is calculated. CMS updates the ratings on a monthly basis; however, a particular home’s overall rating will only change if it had new data that affected any one of the component ratings. For example, when a home has a health inspection survey, either a standard survey or a complaint investigation, the deficiency data from the survey will become a part of the calculation for the health inspection rating and the overall rating will also be adjusted, if necessary. We found that CMS utilizes three standard mechanisms for collecting information on the use of the Nursing Home Compare website: website analytics, website user surveys, and website usability tests. Website analytics. CMS utilizes website analytics to gauge the performance of Nursing Home Compare and improve the visibility of the website in search engine listings. Through this mechanism, CMS is able to track data such as how many users, sessions, bounce rates, and page views Nursing Home Compare has per year. For example, these data show that from 2013 to 2015, Nursing Home Compare averaged 1.5 million sessions per year and 914,000 users per year. The website analytics also track the average session duration and the average number of pages that are viewed per session. For this same time period, the average session duration was 5.8 minutes and the average number of pages that were viewed per session was 4.8. Website user surveys. CMS utilizes website user surveys to collect information about Nursing Home Compare users, how they use the site, and their opinions about the site. According to CMS officials, these surveys, which CMS began using in 2013, randomly pop up in web browsers to 50 percent of the website’s visitors. The surveys ask the user to identify themselves (for example, a caregiver or a researcher), the primary purpose for visiting the site, and the user’s experience in using the site. These surveys provide the only way CMS determines the type of users who come to the website, according to CMS officials. In October 2015, website survey data showed that 59 percent of users of Nursing Home Compare identified themselves as consumers, and the majority of users report coming to the site to research or select nursing homes for themselves or a family member. Website usability tests. CMS utilizes usability tests—in the form of one-on-one sessions with nine consumer participants—to assess how well consumers navigate the website. CMS has conducted four usability tests from 2011 through 2015. The tests focus on the navigability of the website; however, they also include a few background questions about consumer use. For example, the usability tests ask if participants were previously aware of Nursing Home Compare, what factors consumers find most important in searching for a nursing home, and what expectations consumers have for a nursing home comparison website such as this. In addition to these three standard mechanisms, CMS officials also told us that they gain insight into the use of Nursing Home Compare by holding ad-hoc meetings with a variety of stakeholders that are familiar with Nursing Home Compare to discuss the website. CMS held three stakeholder meetings from 2010 through 2015. According to CMS documents, stakeholders have included groups that represent consumers, such as ombudsmen, consumer advocate groups, provider advocate groups, and others that are involved in nursing home services. Information exchanged during stakeholder meetings includes CMS presentations on pending changes, such as the development of new quality measures, and stakeholder feedback. The mechanisms that CMS utilizes to collect information about the use of Nursing Home Compare provide the agency with valuable information. However, these mechanisms do not provide CMS with information on the usefulness of the website to a broader range of consumers. Specifically, the usability tests are not designed to assess the website’s usefulness to consumers, and the website analytics and user surveys only provide information about consumers who access the website. Therefore, the mechanisms do not provide CMS with information on nursing home consumers who have not used the website because they are unaware of it, or choose not to use it, as well as the reasons why. In stakeholder interviews we conducted, some nursing home stakeholders noted that many consumers do not know about the website and that consumers collect information from other sources. Obtaining information from consumers who do not access Nursing Home Compare would likely require the dedication of resources to, for instance, consumer-oriented focus groups or broader surveys. We identified five key areas of improvement CMS could make to Nursing Home Compare in order to make it more helpful for consumers. Specifically, we reviewed over 300 individual improvements identified in CMS documents—in part resulting from the mechanisms described above—and in interviews with national and state stakeholders; for example, one internal CMS analysis included over 40 individual recommendations for improvement. Through our analysis, we found the key areas of improvement are: 1) explanation of how to use the website, 2) additional information about the nursing home, 3) community and consumer outreach, 4) clarity of the website, and 5) navigability of the website. Table 1 below provides more information about these key areas of improvements.