2023-07115.pdf

with RULES222296 Federal Register / Vol. 88, No. 70 / Wednesday, April 12, 2023 / Rules and Regulations 170See COVID19 interim final rule (IFC) (CMS 1744IFC) issued on March 31, 2020. measure-level cut points since they will not be influenced by one or more contracts with outlier scores. Tukey outlier deletion does not change what contracts need to do to improve. Interested parties have requested that CMS minimize changes in cut points from year to year; the implementation of Tukey outlier deletion supports this goal, so we do not believe that the implementation of Tukey outlier deletion needs to be delayed. Comment: A commenter stated that an agency must use the same procedures when they amend or repeal a rule as they used to issue the rule. Additionally, this commenter noted that CMS cannot make a change in the substantive rules that apply to Star Ratings without undertaking rulemaking and cannot rely on the Regulatory Impact Analysis used in 2020, given that other policies related to the mechanism for calculating the Star Ratings continue to evolve. Response: CMS proposed and provided public notice and an opportunity to comment on revising  422.166(a)(2)(i) and 423.186(a)(2)(i) to include the Tukey outlier deletion provision that was inadvertently removed from the regulation text in a May 2022 final rule. The commenter responded to that proposal and had the opportunity to submit comments on the substance of the Tukey outlier deletion. Comments on the proposed correction were submitted and are being addressed in this final rule. After the adoption of the Tukey outlier deletion provision in the June 2022 final rule (85 FR 33833 36), CMS would need additional rulemaking to change that policy and change the Star Ratings methodology to eliminate that provision, which did not happen. In addition, the June 2020 final rule adopting the Tukey outlier deletion step in the Star Ratings methodology (85 FR 3389133893) adequately discussed the cost estimates for the implementation of Tukey outlier deletion. Those estimates were projected based on initial implementation for the 2024 Star Ratings, which the regulation text adopted in this rule at  422.166(a)(2)(i) and 423.186(a)(2)(i) provides for, so the projected cost analysis remains relevant and accurate. We still measure performance in the same way at the measure-level and calculate the Star Ratings in a similar manner. We believe more recent data from 2020 and 2021 performance years would be less useful to simulate the impact of the Tukey outlier deletion process in future years. First, the performance data from the early years of the pandemic have been more impacted by COVID19, and we would expect that there would be more fluctuations in scores during this time, including potentially more outliers. Second, some of the changes we made to the Star Ratings to account for the uncertainties caused by COVID19, including expanding the existing hold harmless provision for the Part C and D improvement measures to include all contracts for the 2022 Star Ratings,170 and all contracts qualifying for the disaster adjustment for the 2022 Star Ratings, make it difficult to use more recent data to predict future performance. Comment: A commenter claimed there would be significant impact on 3, 4, and 5-star cut points. Response: Outlier deletion does not significantly impact 3, 4, and 5-star cut points for most measures. For example, we examined the 2023 Star Ratings data with no guardrails to focus on the effect of Tukey outlier deletion at the measure level. While we note that the 2023 Star Ratings data may still show some impacts of the COVID19 pandemic and therefore may have more outliers than data not impacted by the pandemic, we still found that outlier deletion did not significantly impact 3, 4, and 5-star cut points for most measures. In our analyses, 8 Part C measures (40 percent of the non-CAHPS measures) have no changes across all Star Rating thresholds. Similarly, for Part D measures, there are 4 measures (44 percent of the non-CAHPS measures) with no changes for MAPD contracts and 4 measures (44 percent) with no changes for PDP contracts. Of the remaining measures, most of the changes were for the 12 star cut points, with most measures having no significant impact at the 3, 4, and 5-star cut points. The Tukey outlier approach lessens the influence of a few outliers on cut point formation, leading to more reliable and stable thresholds, especially for the 12 star cut points. This analysis is based on data available at https://www.cms.gov/Medicare/Prescription- Drug-Coverage/PrescriptionDrugCovGenIn/PerformanceData in the Downloads section under the Tukey Outlier Deletion Simulations. Comment: A couple of commenters claimed that Tukey outlier deletion will decrease predictability and stability of cut points. A commenter gave as an example the Plan Makes Timely Decisions about Appeals measure where there would have been more significant changes to cut points with outliers removed for the 2023 Star Ratings. Response: The primary effect of Tukey outlier deletion is to make thresholds more accurate, reliable, and stable. Outlying contract scores can have undue influence on cut points, and this can lead to a single contract having a major influence on cut point values used to assign stars for all contracts. Removing outliers means the thresholds are more stable since they are not influenced by scores on either extreme of the distribution. The commenter cited the Plan Makes Timely Decisions about Appeals measure where there were more significant changes to the thresholds in CMSs simulation using 2023 Star Ratings; this is an extreme example of how outliers influence cut points and is rare in nature. For Part C measures in the 2023 simulations, 8 measures (40 percent of the non-CAHPS measures) had no observed outliers, while only 2 measures (10 percent), including Plan Makes Timely Decisions about Appeals, had greater than 4 percent of contracts being classified as outliers, and the remaining 50 percent of measures had 3.5 percent or less (and generally less than 1 percent) of contracts being classified as outliers; similar trends were observed for Part D measures. Additionally, we compared year over year stability of thresholds between simulations that included outlier deletion and simulations that did not remove outliers (without guardrails). The changes in thresholds between 2022 and 2023 were much smaller when outliers were removed as compared to when they were not removed. For 10 of the 20 non-CAHPS Part C measures, there were thresholds that changed by more than 5 percentage points if outliers were not removed, whereas only 5 measures had this property when outliers were removed. Outlier deletion does stabilize Part C cut points, and results in more year-to-year stability when outliers are deleted compared to simulations that do not use outlier deletion. For MAPD contracts, 4 of 9 Part D measures had thresholds that change by more than 5 percentage points if outliers are not removed compared to only 2 measures with thresholds that change by more than 5 percentage points if outliers were removed. Outlier deletion stabilizes Part D cut points for MAPD contracts, and results in more year-to-year stability when outliers are deleted compared to simulations that do not use outlier deletion. Outlier deletion had a smaller effect on Part D thresholds for PDP contracts. Comment: A couple of commenters claimed that outlier deletion will increase all cut points significantly and VerDate Sep<11>2014 21:20 Apr 11, 2023 Jkt 259001 PO 00000 Frm 00178 Fmt 4701 Sfmt 4700 E:\FR\FM\12APR2.SGM 12APR2ddrumheller on DSK120RN23PROD with RULES222297 Federal Register / Vol. 88, No. 70 / Wednesday, April 12, 2023 / Rules and Regulations move them closer together, decreasing reliability. Response: Outlier deletion may increase or decrease cut point thresholds, depending on the shape of the measures score distribution. Closer cut points do not necessarily imply lower reliability or lessen the ability to distinguish between contracts. Tukey outlier deletion does not increase thresholds for all measures. In a simulation using the 2023 Star Ratings data, we calculated Tukey outlier deletion before applying guardrails as will be done when Tukey outlier deletion is implemented. This also allows us to distinguish the impact of Tukey outlier deletion from the impact of applying guardrails. In this simulation, there were 8 Part C measures (40 percent of measures) that had no change at all in the thresholds. For Part D there were 4 measures (44 percent of measures) with no changes for MAPD contracts and 4 measures (44 percent) with no changes for PDP contracts. Tukey outlier deletion refines measurement by ensuring cut points reflect true variation in performance and are not unduly influenced by low or high performance of a few outlying contracts. Lessening the influence of outliers on cut point formation leads to more reliable and stable cut points. Comment: A commenter claimed that Tukey outlier removal will harm plans performing at lower star levels. Response: Tukey outlier removals primary effect is to make thresholds (that is, cut points) more accurate, reliable, and stable. Removing outliers means the thresholds are more stable and more accurately categorize performance across the industry into measure-level Star Ratings, especially at lower levels, since they are not influenced by outlier scores, and means that a single contract has limited impact on thresholds. These are desirable properties of thresholds. Contracts performing at the lower level are still incentivized to improve performance through the improvement measure, which is highly weighted in the calculation of Star Ratings. Additionally, the more such contracts improve their performance on any given measure, the higher their measure rating can be. Comment: A commenter recommended replacing the clustering methodology with percentile thresholds. Response: The Star Ratings system uses the clustering methodology for non-CAHPS measures because this groups contracts into natural clusters based on the distribution of performance, whereas percentile thresholds would force a certain percent of contracts to receive each star level, regardless of how similar or different those contracts perform. There are situations where it would not make sense to force a fixed percent of contracts into the highest star level, and other contracts into lower star levels, because there may not be meaningful differences between the top group of contracts based on percentile ranking and the next few contracts. The clustering methodology avoids the need to specify percentile thresholds and instead places contracts into their natural groupings based on actual plan performance. After considering the comments we received and for the reasons outlined in the proposed rule and our responses to comments, we are finalizing the technical amendment to fix the Tukey outlier deletion codification error from the May 2022 final rule and the non- substantive technical change to move the sentence about removal of Tukey outer fence outliers earlier in  422.166(a)(2)(i) and 423.186(a)(2)(i), since Tukey outlier deletion is applied prior to the other steps. The Tukey outlier deletion will be applied beginning with the 2024 Star Ratings. VI. Updates to Programs of All- Inclusive Care for the Elderly (PACE) Policy A. Contract Year Definition ( 460.6) Sections 1894(a)(9) and 1934(a)(9) of the Act define the trial period for PACE organizations as the first 3 contract years operating a PACE program under a PACE program agreement. Sections 1894(e)(4) and 1934(e)(4) of the Act require CMS, in cooperation with the State administering agency, to conduct a comprehensive annual review of the PACE organizations operation of the PACE program during the trial period to assure compliance with all significant requirements. The rule titled Medicare and Medicaid Programs; Programs of All-Inclusive Care for the Elderly (PACE), which appeared in the November 24, 1999 issue of the Federal Register (64 FR 66234) (hereinafter referred to as the 1999 PACE interim final rule) defined a contract year at  460.6 as the term of the PACE program agreement, which is a calendar year, except that a PACE organizations initial contract year may be from 12 to 23 months, as determined by CMS. This enables CMS to adjust the length of the initial contract year so that it always ends on December 31 and subsequent contract years align with a standard annual calendar year consisting of 12 months (64 FR 66236). As discussed in the proposed rule (87 FR 79635), CMS is required to conduct comprehensive reviews during a PACE organizations trial period to assess all significant regulatory requirements, and these reviews must be conducted on an annual basis for the first 3 contract years. CMS must conduct the first trial period review (for example, audit) within the first contract year in order to comply with the statutory and regulatory requirements. However, CMSs ability to schedule and conduct the first trial period audit is limited by when a PACE organization enters into a program agreement, when the PACE organization begins enrolling participants during their first contract year, and the initial contract year timeframe in the current contract year definition in  460.6. The timing of the initial contract year audit impacts the timing of subsequent audits, leaving CMS with increasingly narrow timeframes to audit within statutory and regulatory requirements. We proposed to amend the definition of contract year at  460.6 to state that a PACE organizations initial contract year may be 19 to 30 months, as determined by CMS, but in any event will end on December 31. Under the proposed contract year definition, although the duration of the initial contract year of the trial period would change, the initial contract year would continue to begin when the program agreement goes into effect on the first day of the relevant month and end on December 31 to ensure subsequent contract years follow the standard annual calendar year cycle. For PACE organizations with an initial contract year start date of January 1 through June 1, CMS would extend the initial