evekhm/cms_iom_500 huggingface dataset:
2023-07115.pdf
recall period. That is, for measures derived from the HOS, the disaster policy adjustment is for 3 years after the extreme and uncontrollable circumstance. For example, we noted at 84 FR 1577215773 that the 2023 Star Ratings would adjust measures derived from the HOS for 2020 extreme and uncontrollable circumstances. We proposed to clarify in  422.166(i)(3)(iv) the timing for HOS measure adjustments for extreme and uncontrollable circumstances. VerDate Sep<11>2014 21:20 Apr 11, 2023 Jkt 259001 PO 00000 Frm 00176 Fmt 4701 Sfmt 4700 E:\FR\FM\12APR2.SGM 12APR2ddrumheller on DSK120RN23PROD with RULES222295 Federal Register / Vol. 88, No. 70 / Wednesday, April 12, 2023 / Rules and Regulations 169See the Announcement of Calendar Year (CY) 2022 Medicare Advantage (MA) Capitation Rates and Part C and Part D Payment Policies, page 97, which delayed the return of the Plan All-Cause Readmissions measure to the Star Ratings for an additional year due to the disruption to data collection posed by the COVID19 pandemic. We solicited comments on this proposal. Comment: All commenters supported our proposal to clarify the timing for HOS disaster adjustments. Response: CMS appreciates the support for clarifying the timing of the HOS measure adjustments for extreme and uncontrollable circumstances at  422.166(i)(3)(iv). Comment: A commenter suggested that CMS provide additional clarification on the recall period for HOS measures and the hold harmless timing for the adjustment for extreme and uncontrollable circumstances. Response: The measurement period or recall period is defined by the measure steward. NCQA is the measure steward for the three HEDISHOS measures derived from the HOS. As noted by the title of NCQAs technical manual for the 2021 HOS data collection, HEDIS MY 2020 Volume 6: Specifications for the Medicare Health Outcomes Survey, the measurement period is one year prior to data collection. Since 2020, HOS survey administration occurs in late summer through fall. HOS is currently fielded from late July through early
2023-07115.pdf
HEDIS MY 2020 Volume 6: Specifications for the Medicare Health Outcomes Survey, the measurement period is one year prior to data collection. Since 2020, HOS survey administration occurs in late summer through fall. HOS is currently fielded from late July through early November. For Part C measures derived from HOS, the disaster adjustment is three years after the extreme and uncontrollable circumstance. That is, contracts affected by an extreme and uncontrollable circumstance in the year prior to data collection are essentially held harmless and receive the higher of the previous or current years Star Rating for each HOS and HEDISHOS measure (and corresponding measure score) for the Star Ratings 3 years after the eligible extreme and uncontrollable circumstance. For example, contracts affected by an extreme and uncontrollable circumstance in 2021 will receive the higher of their 2023 or 2024 measure-level Star Rating (and corresponding measure score) for each HOS and HEDISHOS measure in the 2024 Star Ratings as described at  422.166(i)(3)(iv). After considering the comments we received and for the reasons outlined in the proposed rule and our responses to comments, we are finalizing the clarification at  422.166(i)(3)(iv) as proposed without modification. H. Calculation of Star Ratings ( 422.166(a)(2)(i) and 423.186(a)(2)(i)) In the June 2020 final rule, we finalized use of Tukey outlier deletion effective for the Star Ratings issued in October 2023 and subsequent years. (85 FR 3383336). In the rulemakings since that time, we have not proposed to eliminate the Tukey outlier deletion aspect of the Star Ratings methodology. In a final rule that appeared in the Federal Register on January 19, 2021, we noted how the Tukey outlier deletion provision had been adopted for the Part C and Part D Quality Star Ratings. (86 FR 5917). As we stated in May 2022 final rule (87 FR 27766), we will implement Tukey outlier deletion beginning with the 2024 Star Ratings to help improve stability of cut points and prevent cut points from being influenced by outliers. We further stated that with Tukey outlier deletion, extreme
2023-07115.pdf
we will implement Tukey outlier deletion beginning with the 2024 Star Ratings to help improve stability of cut points and prevent cut points from being influenced by outliers. We further stated that with Tukey outlier deletion, extreme outliers will be removed from measure scores prior to clustering to prevent outliers from impacting cut points for all contracts. However, it appears that the sentence in  422.166(a)(2)(i) and 423.186(a)(2)(i) (Effective for the Star Ratings issued in October 2023 and subsequent years, prior to applying mean resampling with hierarchal clustering, Tukey outer fence outliers are removed.) was inadvertently removed from the codified regulation text. At no point did CMS propose removal of the Tukey outlier provision and CMS has, since its adoption in the June 2020 final rule, discussed implementation and application of the Tukey outlier provision when applicable. We proposed a technical amendment to fix this codification error from the May 2022 final rule. In addition, although the provision regarding application of the Tukey outlier deletion policy was originally at the end of paragraph (a)(2)(i) in each regulation, we also proposed a non-substantive technical change to move the sentence about removal of Tukey outer fence outliers earlier in  422.166(a)(2)(i) and 423.186(a)(2)(i) since Tukey outlier deletion is applied prior to the other steps. We believe that this makes the regulation text clearer. We solicited comments on this proposal. Comment: Some commenters were concerned that Tukey outlier deletion would result in disproportionate losses of QBPs among DSNPs. They cite an analysis performed by ZAHealth that found that the outlier policy would result in 14 percent of DSNP contracts losing their QBPs compared to 7 percent of non-DSNPs, and 27 percent of D SNPs losing rebate dollars compared to 20 percent of non-DSNPs. Response: We are unable to replicate the findings of ZAHealth. Based on our simulations, we do not believe that contracts with DSNPs will be
2023-07115.pdf
losing rebate dollars compared to 20 percent of non-DSNPs. Response: We are unable to replicate the findings of ZAHealth. Based on our simulations, we do not believe that contracts with DSNPs will be disproportionately impacted by Tukey outlier deletion. Using the 2023 Star Ratings data, we examined the impact of introducing Tukey outlier deletion assuming no guardrails. In the simulation, approximately 8.2 percent of contracts with DSNPs and 9.5 percent of contracts without DSNPs would lose a QBP by their overall rating decreasing from 4 to 3.5 stars overall with Tukey outlier deletion compared to without Tukey outlier deletion. The percentage of contracts losing a QBP is slightly higher for non-DSNP contracts. In the simulation, 13.6 percent of contracts with DSNPs would have a decrease in rebates or lose rebates compared to 10.5 percent of contracts without DSNPs, a very small difference. Comment: A commenter stated that CMS should withdraw its proposed Tukey outlier deletion for the 2024 Star Ratings as it will create new hurdles for plans that are trying to improve their ratings. Another commenter supported Tukey outlier deletion but raised challenges in the industry implementing multiple changes in the Star Ratings over the next few years, while another commenter suggested CMS delay implementation. Response: Tukey outlier deletion is the only methodological enhancement to the 2024 Star Ratings. The only other changes for the 2024 Star Ratings are the addition of two new measures, Transitions of Care and Follow-up after Emergency Department Visit for Patients with Multiple Chronic Conditions measures, to the Part C Star Ratings that have been on the display page since the 2020 Star Ratings (2018 measurement year) finalized in the January 2021 final rule (86 FR 592126) and the return of the updated Plan All-Cause Readmissions measure finalized in the April 2019 final rule,169a measure that has been included in the Star Ratings program since the 2012 Star Ratings. CMS finalized the application of Tukey outlier deletion for non-CAHPS measures beginning with the 2024 Star Ratings in the CY 2021 final rule published in June 2020 so this is not a new enhancement and contracts have been on notice of this
2023-07115.pdf
finalized the application of Tukey outlier deletion for non-CAHPS measures beginning with the 2024 Star Ratings in the CY 2021 final rule published in June 2020 so this is not a new enhancement and contracts have been on notice of this upcoming change. For the 2025 Star Ratings, there are no additional measures or methodological enhancements. The primary goal of setting cut points is to disaggregate the distribution of scores into discrete categories such that each grouping accurately reflects true performance. (85 FR 15752). Tukey outlier deletion supports this goal by helping stabilize VerDate Sep<11>2014 21:20 Apr 11, 2023 Jkt 259001 PO 00000 Frm 00177 Fmt 4701 Sfmt 4700 E:\FR\FM\12APR2.SGM 12APR2ddrumheller on DSK120RN23PROD with RULES222296 Federal Register / Vol. 88, No. 70 / Wednesday, April 12, 2023 / Rules and Regulations 170See COVID19 interim final rule (IFC) (CMS 1744IFC) issued on March 31, 2020. measure-level cut points since they will not be influenced by one or more contracts with outlier scores. Tukey outlier deletion does not change what contracts need to do to improve. Interested parties have requested that CMS minimize changes in cut points from year to year; the implementation of Tukey outlier deletion supports this goal, so we do not believe that the implementation of Tukey outlier deletion needs to be delayed. Comment: A commenter stated that an agency must use the same procedures when they amend or repeal a rule as they used to issue the rule. Additionally, this commenter noted that CMS cannot make a change in the substantive rules that apply to Star Ratings without undertaking rulemaking and cannot rely on the Regulatory Impact Analysis used in 2020, given that other policies related to the mechanism for calculating the Star Ratings continue to evolve. Response: CMS proposed and provided public notice and an opportunity to comment on revising  422.166(a)(2)(i) and 423.186(a)(2)(i) to include the Tukey outlier deletion provision that was inadvertently removed from the regulation text in a May 2022 final rule. The commenter responded to that proposal
2023-07115.pdf
and 423.186(a)(2)(i) to include the Tukey outlier deletion provision that was inadvertently removed from the regulation text in a May 2022 final rule. The commenter responded to that proposal and had the opportunity to submit comments on the substance of the Tukey outlier deletion. Comments on the proposed correction were submitted and are being addressed in this final rule. After the adoption of the Tukey outlier deletion provision in the June 2022 final rule (85 FR 33833 36), CMS would need additional rulemaking to change that policy and change the Star Ratings methodology to eliminate that provision, which did not happen. In addition, the June 2020 final rule adopting the Tukey outlier deletion step in the Star Ratings methodology (85 FR 3389133893) adequately discussed the cost estimates for the implementation of Tukey outlier deletion. Those estimates were projected based on initial implementation for the 2024 Star Ratings, which the regulation text adopted in this rule at  422.166(a)(2)(i) and 423.186(a)(2)(i) provides for, so the projected cost analysis remains relevant and accurate. We still measure performance in the same way at the measure-level and calculate the Star Ratings in a similar manner. We believe more recent data from 2020 and 2021 performance years would be less useful to simulate the impact of the Tukey outlier deletion process in future years. First, the performance data from the early years of the pandemic have been more impacted by COVID19, and we would expect that there would be more fluctuations in scores during this time, including potentially more outliers. Second, some of the changes we made to the Star Ratings to account for the uncertainties caused by COVID19, including expanding the existing hold harmless provision for the Part C and D improvement measures to include all contracts for the 2022 Star Ratings,170 and all contracts qualifying for the disaster adjustment for the 2022 Star Ratings, make it difficult to use more recent data to predict future performance. Comment: A commenter claimed there would be significant impact on 3, 4, and 5-star cut points. Response: Outlier deletion does not significantly impact 3, 4, and 5-star cut points for most measures. For example, we examined the 2023 Star Ratings data with no
2023-07115.pdf
on 3, 4, and 5-star cut points. Response: Outlier deletion does not significantly impact 3, 4, and 5-star cut points for most measures. For example, we examined the 2023 Star Ratings data with no guardrails to focus on the effect of Tukey outlier deletion at the measure level. While we note that the 2023 Star Ratings data may still show some impacts of the COVID19 pandemic and therefore may have more outliers than data not impacted by the pandemic, we still found that outlier deletion did not significantly impact 3, 4, and 5-star cut points for most measures. In our analyses, 8 Part C measures (40 percent of the non-CAHPS measures) have no changes across all Star Rating thresholds. Similarly, for Part D measures, there are 4 measures (44 percent of the non-CAHPS measures) with no changes for MAPD contracts and 4 measures (44 percent) with no changes for PDP contracts. Of the remaining measures, most of the changes were for the 12 star cut points, with most measures having no significant impact at the 3, 4, and 5-star cut points. The Tukey outlier approach lessens the influence of a few outliers on cut point formation, leading to more reliable and stable thresholds, especially for the 12 star cut points. This analysis is based on data available at https://www.cms.gov/Medicare/Prescription- Drug-Coverage/PrescriptionDrugCovGenIn/PerformanceData in the Downloads section under the Tukey Outlier Deletion Simulations. Comment: A couple of commenters claimed that Tukey outlier deletion will decrease predictability and stability of cut points. A commenter gave as an example the Plan Makes Timely Decisions about Appeals measure where there would have been more significant changes to cut points with outliers removed for the 2023 Star Ratings. Response: The primary effect of Tukey outlier deletion is to make thresholds more accurate, reliable, and stable. Outlying contract scores can have undue influence on cut points, and this can lead to a single contract having a major influence on cut point values used to assign stars for all contracts. Removing outliers means the thresholds are more stable since they are not influenced by scores on either extreme of the distribution. The
2023-07115.pdf
and this can lead to a single contract having a major influence on cut point values used to assign stars for all contracts. Removing outliers means the thresholds are more stable since they are not influenced by scores on either extreme of the distribution. The commenter cited the Plan Makes Timely Decisions about Appeals measure where there were more significant changes to the thresholds in CMSs simulation using 2023 Star Ratings; this is an extreme example of how outliers influence cut points and is rare in nature. For Part C measures in the 2023 simulations, 8 measures (40 percent of the non-CAHPS measures) had no observed outliers, while only 2 measures (10 percent), including Plan Makes Timely Decisions about Appeals, had greater than 4 percent of contracts being classified as outliers, and the remaining 50 percent of measures had 3.5 percent or less (and generally less than 1 percent) of contracts being classified as outliers; similar trends were observed for Part D measures. Additionally, we compared year over year stability of thresholds between simulations that included outlier deletion and simulations that did not remove outliers (without guardrails). The changes in thresholds between 2022 and 2023 were much smaller when outliers were removed as compared to when they were not removed. For 10 of the 20 non-CAHPS Part C measures, there were thresholds that changed by more than 5 percentage points if outliers were not removed, whereas only 5 measures had this property when outliers were removed. Outlier deletion does stabilize Part C cut points, and results in more year-to-year stability when outliers are deleted compared to simulations that do not use outlier deletion. For MAPD contracts, 4 of 9 Part D measures had thresholds that change by more than 5 percentage points if outliers are not removed compared to only 2 measures with thresholds that change by more than 5 percentage points if outliers were removed. Outlier deletion stabilizes Part D cut points for MAPD contracts, and results in more year-to-year stability when outliers are deleted compared to simulations that do not use outlier deletion. Outlier deletion had a smaller effect on Part D thresholds for PDP contracts. Comment: A couple of commenters claimed that outlier deletion will increase all cut points significantly and VerDate
2023-07115.pdf
use outlier deletion. Outlier deletion had a smaller effect on Part D thresholds for PDP contracts. Comment: A couple of commenters claimed that outlier deletion will increase all cut points significantly and VerDate Sep<11>2014 21:20 Apr 11, 2023 Jkt 259001 PO 00000 Frm 00178 Fmt 4701 Sfmt 4700 E:\FR\FM\12APR2.SGM 12APR2ddrumheller on DSK120RN23PROD with RULES222297 Federal Register / Vol. 88, No. 70 / Wednesday, April 12, 2023 / Rules and Regulations move them closer together, decreasing reliability. Response: Outlier deletion may increase or decrease cut point thresholds, depending on the shape of the measures score distribution. Closer cut points do not necessarily imply lower reliability or lessen the ability to distinguish between contracts. Tukey outlier deletion does not increase thresholds for all measures. In a simulation using the 2023 Star Ratings data, we calculated Tukey outlier deletion before applying guardrails as will be done when Tukey outlier deletion is implemented. This also allows us to distinguish the impact of Tukey outlier deletion from the impact of applying guardrails. In this simulation, there were 8 Part C measures (40 percent of measures) that had no change at all in the thresholds. For Part D there were 4 measures (44 percent of measures) with no changes for MAPD contracts and 4 measures (44 percent) with no changes for PDP contracts. Tukey outlier deletion refines measurement by ensuring cut points reflect true variation in performance and are not unduly influenced by low or high performance of a few outlying contracts. Lessening the influence of outliers on cut point formation leads to more reliable and stable cut points. Comment: A commenter claimed that Tukey outlier removal will harm plans performing at lower star levels. Response: Tukey outlier removals primary effect is to make thresholds (that is, cut points) more accurate, reliable, and stable. Removing outliers means the thresholds are more stable and more accurately categorize performance across the industry into measure-level Star Ratings, especially at lower levels, since they are not influenced by outlier scores, and means that a single
2023-07115.pdf
stable. Removing outliers means the thresholds are more stable and more accurately categorize performance across the industry into measure-level Star Ratings, especially at lower levels, since they are not influenced by outlier scores, and means that a single contract has limited impact on thresholds. These are desirable properties of thresholds. Contracts performing at the lower level are still incentivized to improve performance through the improvement measure, which is highly weighted in the calculation of Star Ratings. Additionally, the more such contracts improve their performance on any given measure, the higher their measure rating can be. Comment: A commenter recommended replacing the clustering methodology with percentile thresholds. Response: The Star Ratings system uses the clustering methodology for non-CAHPS measures because this groups contracts into natural clusters based on the distribution of performance, whereas percentile thresholds would force a certain percent of contracts to receive each star level, regardless of how similar or different those contracts perform. There are situations where it would not make sense to force a fixed percent of contracts into the highest star level, and other contracts into lower star levels, because there may not be meaningful differences between the top group of contracts based on percentile ranking and the next few contracts. The clustering methodology avoids the need to specify percentile thresholds and instead places contracts into their natural groupings based on actual plan performance. After considering the comments we received and for the reasons outlined in the proposed rule and our responses to comments, we are finalizing the technical amendment to fix the Tukey outlier deletion codification error from the May 2022 final rule and the non- substantive technical change to move the sentence about removal of Tukey outer fence outliers earlier in  422.166(a)(2)(i) and 423.186(a)(2)(i), since Tukey outlier deletion is applied prior to the other steps. The Tukey outlier deletion will be applied beginning with the 2024 Star Ratings. VI. Updates to Programs of All- Inclusive Care for the Elderly (PACE) Policy A. Contract Year Definition ( 460.6) Sections 1894(a)(9) and 1934(a)(9) of the Act define the trial period for PACE organizations as the first 3 contract years operating a PACE program under a PACE program agreement. Sections 1894(e)(4) and 1934(e)(4) of