36232 Federal Register / Vol. 74, No. 139 / Wednesday, July 22, 2009 / Notices 

TABLE 2.—ESTIMATED ANNUAL RECORDKEEPING BURDEN1

21 CFR Section No. of 

Recordkeepers 

Annual Frequency 

per 

Recordkeeping 

Total Annual 

Records 

Hours per 

Record Total Hours 

203.23(a) and (b) 31,676 5 158,380 .25 39,595 

203.23(c) 31,676 5 158,380 .08 12,670 

203.30(a)(2) and 203.31(a)(2) 2,208 100 220,800 .50 110,400 

203.31(d)(1) and (d)(2) 2,208 1 2,208 40 88,320 

203.31(d)(4) 442 1 442 24 10,608 

203.31(e) 2,208 1 2,208 1 2,208 

203.34 90 1 90 40 3,600 

203.37(a) 50 4 200 6 1,200 

203.37(b) 50 40 2,000 6 12,000 

203.39(d) 65 1 65 1 65 

203.39(e) 3,221 1 3,221 .50 1,610 

203.39(f) 3,221 1 3,221 8 25,768 

203.39(g) 3,221 1 3,221 8 25,768 

203.50(a) 125 100 12,500 .17 2,125 

203.50(b) 125 100 12,500 .50 6,250 

203.50(d) 691 1 691 2.0 1,382 

Total 332,769 

1There are no capital costs or operating and maintenance costs associated with this collection of information. 

Dated: July 15, 2009. 

Jeffrey Shuren, 

Associate Commissioner for Policy and 

Planning. 

[FR Doc. E9–17394 Filed 7–21–09; 8:45 am] 

BILLING CODE 4160–01–S 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Agency for Healthcare Research and 

Quality 

Agency Information Collection 

Activities: Proposed Collection; 

Comment Request 

AGENCY: Agency for Healthcare Research 

and Quality, HHS. 

ACTION: Notice. 

SUMMARY: This notice announces the 

intention of the Agency for Healthcare 

Research and Quality (AHRQ) to request 

that the Office of Management and 

Budget (OMB) approve the proposed 

information collection project: ‘‘Study 

of Factors Influencing Consumer 

Choices Among Health Plans and 

Clinicians.’’ In accordance with the 

Paperwork Reduction Act of 1995, 

Public Law 104–13 (44 U.S.C. 

3506(c)(2)(A)), AHRQ invites the public 

to comment on this proposed 

information collection. This proposed 

information collection was previously 

published in the Federal Register on 

September 3rd, 2008 and allowed 60 

days for public comment. No comments 

were received. The purpose of this 

notice is to allow an additional 30 days 

for public comment. This notice differs 

from the 60 day notice in the following 

ways: (1) The number of responses has 

been decreased from 6,000 to 4,950, (2) 

the burden hours are decreased from 

838 to 709, and (3) the descriptions of 

each experimental arm in the sections: 

Clinician Choice Experimental Design 

and Health Plan Choice Experimental 

Design were removed. 

DATES: Comments on this notice must be 

received by August 21, 2009. 

ADDRESSES: Written comments should 

be submitted to: AHRQ’s OMB Desk 

Officer by fax at (202) 395–6974 

(attention: AHRQ’s desk officer) or by email at OIRA_submission@omb.eop.gov 

(attention: AHRQ’s desk officer). 

Copies of the proposed collection 

plans, data collection instruments, and 

specific details on the estimated burden 

can be obtained from the AHRQ Reports 

Clearance Officer. 

FOR FURTHER INFORMATION CONTACT: 

Doris Lefkowitz, AHRQ Reports 

Clearance Officer, (301) 427–1477, or by 

e-mail at doris.lefkowitz@ahrq.hhs.gov. 

SUPPLEMENTARY INFORMATION: 

Proposed Project 

‘‘Study of Factors Influencing Consumer 

Choices Among Health Plans and 

Clinicians’’ 

AHRQ proposes to use an 

experimental design to determine 

factors that influence consumers’ 

understanding and use of performance 

information to select among health 

plans and clinicians. The experimental 

design will include two parallel 

experiments, one designed to assess 

factors influencing choice of health 

plans and one designed to assess factors 

influencing choice of individual 

doctors. For both the health plan and 

clinician choice experiments 

respondents will be randomly assigned 

to one of six experimental arms that 

vary according to the type and 

complexity of performance information 


jlentini on DSKJ8SOYB1PROD with NOTICES
Federal Register / Vol. 74, No. 139 / Wednesday, July 22, 2009 / Notices 36233 

and the size of the choice set (number 

of plans or doctors) included in the 

Web-based report. Respondents will 

complete the experiment through a 

secure online connection from their 

homes. Data will be derived from pre 

and post-test questionnaires and from 

server logs that record the web pages 

visited and viewing times. 

The results of this study will be used 

to develop recommendations for helping 

consumers to better understand and 

more effectively use complex 

information to select health plans and 

providers, with the aim of making 

performance information less 

burdensome and more accessible, 

useful, and transparent to the public. 

This study, funded through cooperative 

agreements with the RAND Corporation 

and Harvard University, is being 

conducted pursuant to AHRQ’s 

statutory mandate to promote health 

care quality improvement by conducting 

and supporting research that develops 

and presents scientific evidence 

regarding all aspects of health care, 42 

U.S.C. 299(b)(l), and to conduct research 

on health care and on systems for the 

delivery of such health care, 42 U.S.C. 

299a. 

Method of Collection 

Participants in this study will be 

recruited through the Knowledge 

Networks national online panel of 

consumers. The experimental designs of 

the clinician choice experiment and of 

the health plan choice experiment are 

discussed separately below. 

Clinician Choice Experimental Design 

Participants will see a web page 

labeled ‘‘Performance Overview’’ that 

presents performance information for a 

set of primary care doctors in a way that 

allows them to compare doctor ratings. 

Performance is summarized by 

assigning one to five stars to show how 

each doctor compares with others in the 

same geographic area. Participants can 

click on hyperlinks or a tab to see more 

detailed results. The six experimental 

arms differ in the type and amount of 

performance information presented and 

the number of doctors listed. 

The goals of the experiment are to 

assess the process of consumer choice 

and the extent to which the Consumer 

Assessment of Healthcare Providers and 

Systems (CAHPS)-type measures are 

consulted, and to examine how 

consumers respond to different types of 

information about doctor quality, 

including quantitative patient 

experience measures, anecdotal reports 

from individual patients, and clinical 

performance indicators. The post-test 

questionnaire will elicit participants’ 

understanding and impressions of the 

material they saw on the Web site and 

inquire about how they made their 

choice. Therefore, the post-test 

questions will differ across 

experimental arms. 

Health Plan Choice Experimental 

Design 

The basic design of the health plan 

choice experiment is similar to that 

used for the clinician choice 

experiment. The key difference in the 

choice set is that as is true in real-world 

choices health plan choice is made 

complex in the experiment by 

introducing a larger number of measures 

of performance, compared to those 

available to inform clinician choice. 

Even the simplest experimental arm has 

twice as many component measures for 

health plans as for clinicians. Reports 

from consumers include both anecdotes 

and a count of aggregate complaints that 

have been filed against the plan. 

Potentially offsetting the cognitive 

burdens caused by additional measures, 

health plan choices typically involve 

fewer options than do clinician choices; 

in this choice experiment participants 

will face choice sets involving either 4 

or 8 health plans. 

A second substantial difference exists 

between the health plan and clinician 

choice experiments: the former assesses 

in an explicit manner the ways in which 

emotionality affects how consumers 

make use of information. It will do so 

in two ways. First, the counts of 

complaints mentioned above as an 

additional measure of plan performance 

represent a quantitative score with a 

stronger emotional valence than the 

other measures. Second, two of the 

experimental arms will ‘‘prime’’ 

respondents to think about health 

outcomes in a more emotionally laden 

manner, to see if this alters the way in 

which they process this information. 

The goals of the experiment are to 

assess the process of consumer choice 

and the extent to which CAHPS-type 

measures are consulted, and to examine 

how consumers respond to different 

types of information about health plan 

quality, including customer services and 

accessibility of care issues, selected 

Healthcare Effectiveness Data and 

Information Set (HEDIS) measures for 

preventive care and treatment of chronic 

conditions, and selected reports on 

enrollee complaint rates and other 

issues. The post-test questionnaire will 

elicit participants’ understanding and 

impressions of the material they saw on 

the Web site and inquire about how they 

made their choice. Therefore, the posttest questions will differ across 

experimental arms. 

Estimated Annual Respondent Burden 

Exhibit 1 shows the estimated 

annualized burden hours for the 

respondents’ time to participate in this 

one year experiment. All participants 

will complete the pre-test, which is 

estimated to require 5 minutes. As 

explained above, the experimental 

website varies by experimental arm; 

however, based on preliminary testing, 

each participant will require an average 

of 10 minutes to review the information 

on the site. The post-test questionnaires 

will require between 7 to 14 minutes to 

complete, depending on the 

experimental arm. The total burden 

hours are estimated to be 709 hours. 

Exhibit 2 shows the respondents’ cost 

burden associated with their time to 

participate in this experiment. The total 

cost burden is estimated to be $13,887. 

EXHIBIT 1—ESTIMATED ANNUALIZED BURDEN HOURS

Experimental group Number of 

responses 

Number of 

responses per 

respondent 

Hours per 

response 

Total burden 

hours 

Clinician Choice Experiment: Pretest 750 1 5/60 63 

Experimental Web site ............................................................................. 750 1 10/60 125 

Baseline/Control Arm Post-test ................................................................ 125 1 7/60 15 

Experimental Arm #1 Post-test ................................................................. 125 1 8/60 17 

Experimental Arm #2 Post-test ................................................................. 125 1 8/60 17 

Experimental Arm #3 Post-test ................................................................. 125 1 12/60 25 

Experimental Arm #4 Post-test ................................................................. 125 1 12/60 25 

Experimental Arm #5 Post-test ................................................................. 125 1 14/60 29 

Health Plan Choice Experiment: Pretest 900 1 5/60 75 

Experimental Web site ............................................................................. 900 1 10/60 150 


jlentini on DSKJ8SOYB1PROD with NOTICES
36234 Federal Register / Vol. 74, No. 139 / Wednesday, July 22, 2009 / Notices 

EXHIBIT 1—ESTIMATED ANNUALIZED BURDEN HOURS—Continued 

Experimental group Number of 

responses 

Number of 

responses per 

respondent 

Hours per 

response 

Total burden 

hours 

Baseline/Control Arm Post-test ................................................................ 150 1 7/60 18 

Experimental Arm #1 Post-test ................................................................. 150 1 8/60 20 

Experimental Arm #2 Post-test ................................................................. 150 1 12/60 30 

Experimental Arm #3 Post-test ................................................................. 150 1 12/60 30 

Experimental Arm #4 Post-test ................................................................. 150 1 14/60 35 

Experimental Arm #5 Post-test ................................................................. 150 1 14/60 35 

Total ................................................................................................... 4,950 na na 709 

EXHIBIT 2—ESTIMATED ANNUALIZED COST BURDEN

Experimental group Number of 

respondents 

Total burden 

hours 

Average hourly wage rate* Total cost 

burden 

Clinician Choice Experiment: Pretest 750 63 $19.56 $1,232 

Experimental Web site ............................................................................. 750 125 19.56 2,445 

Baseline/Control Arm Post-test ................................................................ 125 15 19.56 293 

Experimental Arm #1 Post-test ................................................................. 125 17 19.56 333 

Experimental Arm #2 Post-test ................................................................. 125 17 19.56 333 

Experimental Arm #3 Post-test ................................................................. 125 25 19.56 489 

Experimental Arm #4 Post-test ................................................................. 125 25 19.56 489 

Experimental Arm #5 Post-test ................................................................. 125 29 19.56 567 

Health Plan Choice Experiment: Pretest 900 75 19.56 1,467 

Experimental Web site ............................................................................. 900 150 19.56 2,934 

Baseline/Control Arm Post-test ................................................................ 150 18 19.56 352 

Experimental Arm #1 Post-test ................................................................. 150 20 19.56 391 

Experimental Arm #2 Post-test ................................................................. 150 30 19.56 587 

Experimental Arm #3 Post-test ................................................................. 150 30 19.56 587 

Experimental Arm #4 Post-test ................................................................. 150 35 19.56 685 

Experimental Arm #5 Post-test ................................................................. 150 35 19.56 685 

Total ................................................................................................... 4,950 709 na 13,887 

*Based upon the mean of the average wages, ‘‘National Compensation Survey: Occupational Wages in the United States, May 2007,’’ U.S. 

Department of Labor, Bureau of Labor Statistics. 

Estimated Annual Costs to the Federal 

Government 

Exhibit 3 shows the total and 

annualized cost for developing and 

conducting both the health plan and 

clinician choice components of this 

study, including the cost of designing 

the experiments, developing the 

simulated Web-based reports, 

conducting usability testing of the Web 

reports, pilot testing the experiment, 

collecting the data, analyzing the data, 

preparing reports and papers for journal 

submission, and the cost for AHRQ staff 

to oversee the project. The total and 

annual costs are identical since data 

collection will not exceed one year. The 

total cost is estimated to be $844,000. 

EXHIBIT 3—TOTAL AND ANNUALIZED COSTS

Cost components Total cost Annual cost 

Experimental design ................................................................................................................................................ $168,900 $168,900 

Development of simulated Web-based reports ....................................................................................................... 157,900 157,900 

Pilot testing .............................................................................................................................................................. 56,000 56,000 

Usability testing of Web-based reports ................................................................................................................... 56,300 56,300 

Data collection via Knowledge Networks ................................................................................................................ 126,000 126,000 

Data analysis ........................................................................................................................................................... 56,300 56,300 

Preparation of reports and journal papers .............................................................................................................. 112,600 112,600 

AHRQ project management .................................................................................................................................... 110,000 110,000 

Total .................................................................................................................................................................. 844,000 844,000 

Request for Comments 

In accordance with the above-cited 

Paperwork Reduction Act legislation, 

comments on AHRQ’s information 

collection are requested with regard to 

any of the following: (a) Whether the 

proposed collection of information is 

necessary for the proper performance of 

AHRQ health care research and health 

care information dissemination 

functions, including whether the 

information will have practical utility; 

(b) the accuracy of AHRQ’s estimate of 

burden (including hours and costs) of 

the proposed collection(s) of 

information; (c) ways to enhance the 

quality, utility, and clarity of the 

information to be collected; and (d) 

ways to minimize the burden of the 

collection of information upon the 

respondents, including the use of 


jlentini on DSKJ8SOYB1PROD with NOTICES
Federal Register / Vol. 74, No. 139 / Wednesday, July 22, 2009 / Notices 36235 

automated collection techniques or 

other forms of information technology. 

Comments submitted in response to 

this notice will be summarized and 

included in the Agency’s subsequent 

request for OMB approval of the 

proposed information collection. All 

comments will become a matter of 

public record. 

Dated: July 8, 2009. 

Carolyn M. Clancy, 

Director. 

[FR Doc. E9–17203 Filed 7–21–09; 8:45 am] 

BILLING CODE 4160–90–M 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Agency for Healthcare Research and 

Quality 

Agency Information Collection 

Activities: Proposed Collection; 

Comment Request 

AGENCY: Agency for Healthcare Research 

and Quality, HHS. 

ACTION: Notice. 

SUMMARY: This notice announces the 

intention of the Agency for Healthcare 

Research and Quality (AHRQ) to request 

that the Office of Management and 

Budget (OMB) approve the proposed 

information collection project: ‘‘Health 

Literacy Item Set Supplemental to 

CAHPS Hospital Survey—Pretest of 

Proposed Questions and Methodology.’’ 

In accordance with the Paperwork 

Reduction Act of 1995, Public Law 104– 

13 (44 U.S.C. 3506(c)(2)(A)), AHRQ 

invites the public to comment on this 

proposed information collection. 

This proposed information collection 

was previously published in the Federal 

Register on May 13th, 2009 and allowed 

60 days for public comment. The 

purpose of this notice is to allow an 

additional 30 days for public comment. 

This notice differs from the 60 day 

notice in the following ways: (1) The 

burden hours are increased from 200 to 

250, and (2) an incentive experiment 

has been added. 

DATES: Comments on this notice must be 

received by August 21, 2009. 

ADDRESSES: Written comments should 

be submitted to: AHRQ’s OMB Desk 

Officer by fax at (202) 395–6974 

(attention: AHRQ’s desk officer) or by email at OIRA_submission@omb.eop.gov 

(attention: AHRQ’s desk officer). 

Copies of the proposed collection 

plans, data collection instruments, and 

specific details on the estimated burden 

can be obtained from the AHRQ Reports 

Clearance Officer. 

FOR FURTHER INFORMATION CONTACT: 

Doris Lefkowitz, AHRQ Reports 

Clearance Officer, (301) 427–1477, or by 

e-mail at doris.lefkowitz@ahrq.hhs.gov. 

SUPPLEMENTARY INFORMATION: 

Proposed Project 

‘‘Health Literacy Item Set Supplemental 

to CAHPS Hospital Survey—Pretest of 

Proposed Questions and Methodology’’ 

AHRQ proposes to conduct a pretest 

of the Consumer Assessment of 

Healthcare Providers and Systems 

(CAHPS®) Hospital Survey health 

literacy module. The CAHPS program is 

a multi-year initiative of the Agency for 

Healthcare Research and Quality. AHRQ 

first launched the program in October 

1995 in response to concerns about the 

lack of good information about the 

quality of health plans from the 

enrollees’ perspective. Numerous public 

and private organizations collected 

information on enrollee and patient 

satisfaction, but the surveys varied from 

sponsor to sponsor and often changed 

from year to year. The CAHPS® 

program was designed to make it 

possible to compare survey results 

across sponsors and over time, and to 

generate tools and resources that 

sponsors can use to produce 

understandable and usable comparative 

information for consumers. 

Over time, the program has expanded 

beyond its original focus on health 

plans to address a range of health care 

services to meet the various needs of 

health care consumers, purchasers, 

health plans, providers, and 

policymakers. Based on a literature 

review and an assessment of currently 

available questionnaires, AHRQ 

identified the need to develop a health 

literacy module for the CAHPS® 

Hospital Survey. The intent of the 

planned module is to examine patients’ 

perspectives on how well health 

information is communicated to them 

by healthcare professionals in the 

hospital setting. The objective of the 

new module is to provide information to 

health plans, hospitals, clinicians, group 

practices, and other interested parties 

regarding the quality of health 

information delivered to patients. The 

set of questions about health literacy 

will be evaluated as a supplement to the 

CAHPS® Hospital Survey. 

This study will be conducted for 

AHRQ by its contractor, RAND 

Corporation. It is being conducted 

pursuant to AHRQ’s statutory authority 

to conduct research and evaluations on 

health care and systems for the delivery 

of such care, including activities with 

respect to (1) the quality, effectiveness, 

efficiency, appropriateness and value of 

health care services. See 42 U.S.C. 

299a(a)(1). 

This study is a one-time field test to 

be completed in the calendar years 2009 

and 2010. The field test to be conducted 

under this request will be done for the 

following purposes: 

a. Analysis of item wording—Assess 

candidate wordings for items. 

b. Analysis of participation rate— 

Evaluate the overall response rate and 

the proportion of that obtained from 

mail versus telephone modes of data 

collection. 

c. Case mix adjustment analysis— 

Evaluate variables that need to be 

considered for case mix adjustment of 

scores. 

d. Psychometric analysis—Provide 

information for the revision of the 

health literacy item set based on the 

assessment of the reliability and 

validity. 

e. Incentive experiment—Provide 

information on the effectiveness of a 

post-paid, $5 incentive as a mechanism 

to enhance response by randomizing 

half the sample at one site to an 

experiment in which a post-paid 

incentive of $5 is provided for 

completing the survey. 

The end result will be collection of 

the data related to the assessment of 

patients’ perspective on how well health 

information is communicated to them 

by health care professionals in a 

hospital setting. The field testing will 

ensure that future data collections yield 

high quality data and minimize 

respondent burden, increase agency 

efficiency, and improve responsiveness 

to the public. The survey items will be 

added to currently available CAHPS® 

surveys and will enhance the ability of 

hospitals to assess the quality of their 

services. 

Method of Collection 

The potential respondent universe is 

persons who had at least one overnight 

stay at a hospital within the previous 

five months. Excluded from the study 

will be those who were less than 18 

years old at the time of their admission, 

had a psychiatric diagnosis, were 

discharged to a hospice facility or died 

during the hospitalization. Testing sites 

will be selected purposively based on 

several considerations, including ability 

to execute the activities necessary to 

participate in the pilot, number of beds, 

number of discharges for medical, 

surgical, and obstetric patients, average 

length of stay, location (urban versus 

rural), profit status, and academic 

medical center status. 

The draw will be a sample large 

enough to yield approximately 600 

completes. It is assumed that 


jlentini on DSKJ8SOYB1PROD with NOTICES