62152 Federal Register / Vol. 79, No. 200 / Thursday, October 16, 2014 / Notices 

and implementation of the agency’s 

strategic plan that establishes long and 

short-range goals, objectives, strategies 

and action plans for advancing the 

agency’s policy and program agenda. 

Reviews and coordinates all policy and 

program development documents, 

regulations and activities to ensure 

consistency with ACL’s strategic plan; 

and adjusts goals and strategies as 

appropriate. Coordinates the 

identification and analysis of emerging 

policy issues and trends and 

appropriate Federal responses. 

Formulates an agency-wide policy and 

program development strategy 

consistent with the priorities 

established by the Administrator and 

the Principal Deputy Administrator. 

Plans and directs the evaluation of 

ACL programs designed to provide 

planning, coordination and services to 

older Americans and people with 

disabilities. The Director serves as the 

Performance Improvement Officer and 

is the primary liaison with the Office of 

the Assistant Secretary for Planning and 

Evaluation (ASPE), the Office of the 

Assistant Secretary for Financial 

Resources (ASFR), and the Office of 

Management and Budget (OMB) for 

program performance and evaluation 

activities. 

2. Office of Policy Analysis and 

Development (BFB). The Office of 

Policy Analysis and Development 

(OPAD) analyzes trends in 

demographics, service needs, public 

policies and program development, and 

translates those trends into new policies 

and initiatives in long-term services and 

supports and health care that assist 

people with disabilities and older 

individuals to remain in their own 

homes and communities. 

Directs intergovernmental activities as 

they relate to the agency’s policy and 

program development agenda, and 

develops and maintains effective 

relationships with other governmental 

departments and agencies. Plans, 

negotiates, facilitates and updates, as 

appropriate, memoranda of 

understanding with other departments 

and agencies to promote agreements and 

cooperative relationships. Maintains 

information on, and pursues 

collaborative opportunities with, other 

Federal agencies, non-profit 

organizations and private corporations 

that have the potential to contribute to 

the agency’s policy and program 

development priorities. 

Provides technical, program and 

policy development input on legislative 

activities and the annual budget. 

Participates in Departmental and interdepartmental activities that concern 

health and long-term care; reviews and 

comments on Departmental regulations 

and policies regarding health programs, 

institutional and non-institutional longterm care services, and those designed 

to enhance community living. 

Conducts relevant policy research, 

carries out periodic reviews of needs 

and resources in the fields of aging and 

disability, and undertakes qualitative 

and quantitative analyses to develop 

policy options and recommendations for 

the Administrator and the Principal 

Deputy Administrator. Develops policy 

reports based on the needs and 

circumstances of older people, their 

family members and the aging 

population. Develops and coordinates 

initiatives with other Federal agencies, 

national aging organizations, national 

disability organizations, and 

universities to fill gaps in information in 

the field of aging and disability. 

3. Office of Performance and 

Evaluation (BDC). The Office of 

Performance and Evaluation (OPE), in 

collaboration with the respective ACL 

program offices, implements, oversees 

and manages ACL’s program 

performance responsibilities, data 

collection systems, and program 

evaluation activities. Develops plans 

and priorities for evaluation of ACL 

programs, with subject matter input 

from appropriate units. Manages 

contracts for mandated evaluation 

projects and performs intramural 

evaluation studies. Prepares reports of 

the results of program and impact 

evaluations conducted by and for ACL, 

with technical input from other ACL 

units. Provides technical guidance on 

evaluation activities conducted as part 

of ACL’s discretionary grants programs. 

Implements the requirements of the 

Government Performance and Results 

Act of 1993 (GPRA) and the GPRA 

Modernization Act of 2010. Interprets 

ACL goals, priorities, and strategies for 

consistency with ACL long-range GPRA 

goals and strategies, and adjusts GPRA 

goals and strategies accordingly. 

Provides guidance and technical 

assistance to ACL organizational units 

in developing operational plans, 

particularly in developing measurable 

objectives and indicators reflecting 

program and organizational 

performance. Prepares annual GPRA 

plans and reports and coordinates with 

the Office of Budget and Finance on the 

development of the ACL performance 

budget. 

Coordinates ACL activities related to 

the collection, analysis, and 

dissemination of national and program 

data on older individuals and 

individuals with disabilities. Develops 

and manages data requirements; designs 

the criteria for collecting, analyzing and 

disseminating program performance 

data; and prepares the data for reporting 

to Congress and the public. Designs, 

implements and provides guidance and 

technical assistance to funding 

recipients on data collection and 

analysis. Works with the Office of 

Information Resources Management to 

coordinate mandated Office of 

Management and Budget (OMB) 

approvals required under the Paperwork 

Reduction Act of 1980, as amended. 

Compiles, publishes, and 

disseminates information on 

demographic data and data from other 

Federal agencies on the health, social 

and economic status of older persons 

and persons with disabilities. Performs 

routine and special statistical analyses 

of data for ACL offices, other Federal 

and non-Federal organizations, and the 

general public. 

II. Delegations of Authority: All 

delegations and re-delegations of 

authority made to officials and 

employees of affected organizational 

components will continue in them or 

their successors pending further redelegations. 

III. Funds, Personnel and Equipment: 

Transfer of organizations and functions 

affected by this reorganization shall be 

accompanied in each instance by direct 

and support funds, positions, personnel, 

records, equipment, supplies and other 

resources. 

Dated: August 29, 2014. 

Sylvia M. Burwell, 

Secretary. 

[FR Doc. 2014–24639 Filed 10–15–14; 8:45 am] 

BILLING CODE P 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Agency for Healthcare Research and 

Quality 

Agency Information Collection 

Activities: Proposed Collection; 

Comment Request 

AGENCY: Agency for Healthcare Research 

and Quality, HHS. 

ACTION: Notice. 

SUMMARY: This notice announces the 

intention of the Agency for Healthcare 

Research and Quality (AHRQ) to request 

that the Office of Management and 

Budget (OMB) approve the proposed 

information collection project: ‘‘Care 

Coordination Quality Measure for 

Patients in the Primary Care Setting.’’ In 

accordance with the Paperwork 

Reduction Act, 44 U.S.C. 3501–3521, 

AHRQ invites the public to comment on 

this proposed information collection. 


asabaliauskas on DSK5VPTVN1PROD with NOTICES
Federal Register / Vol. 79, No. 200 / Thursday, October 16, 2014 / Notices 62153 

This proposed information collection 

was previously published in the Federal 

Register on July 30th, 2014 and allowed 

60 days for public comment. AHRQ 

received and responded to comments 

from two members of the public. The 

purpose of this notice is to allow an 

additional 30 days for public comment. 

DATES: Comments on this notice must be 

received by November 17, 2014. 

ADDRESSES: Written comments should 

be submitted to: AHRQ’s OMB Desk 

Officer by fax at (202) 395–6974 

(attention: AHRQ’s desk officer) or by 

email at OIRA_submission@

omb.eop.gov (attention: AHRQ’s desk 

officer). 

FOR FURTHER INFORMATION CONTACT: 

Doris Lefkowitz, AHRQ Reports 

Clearance Officer, (301) 427–1477, or by 

email at doris.lefkowitz@AHRQ.hhs.gov. 

SUPPLEMENTARY INFORMATION: 

Proposed Project 

Care Coordination Quality Measure for 

Patients in the Primary Care Setting 

Proposed Project 

‘‘Care Coordination Measure 

Development—Phase III’’ 

This project is Task Order #11 under 

the Agency for Healthcare Research and 

Quality (AHRQ) Prevention and Care 

Management Technical Assistance 

Center Indefinite Delivery Indefinite 

Quantity contract. The project, entitled 

‘‘Care Coordination Measure 

Development—Phase III’’, will develop 

a patient survey of the quality of care 

coordination for adults in primary care 

settings, i.e., the Care Coordination 

Quality. Measure for Primary Care 

(CCQM–PC). The project will update the 

Care Coordination Measures Atlas 

(http://www.ahrq.gov/professionals/

systems/long-termcare/resources/

coordination/atlas/index.html). In 

combination with primary research, the 

project will use the Atlas and prior work 

that identified gaps in the measurement 

of care coordination to develop and 

pilot test a rigorous and 

psychometrically sound patient 

assessment (from the perspective of 

patient and family) of the quality of care 

coordination for adults within primary 

care settings—the CCQM–PC. The 

survey will address key care 

coordination domains; be appropriate 

for research; will set the stage for the 

future development of measures for 

quality reporting, accountability, and 

payment purposes; and be consistent 

with Consumer Assessment of 

Healthcare Providers and Systems 

(CAMPS) 0 principles. The instrument 

is to be developed, cognitively tested, 

revised and pilot tested. A stakeholder 

panel will provide input throughout the 

phases of the project. 

There are five explicit objectives for 

our analysis of the pilot-test data: 

• Evaluate the quality of the 

responses to the CCQM–PC survey 

(through item functioning analysis). 

• Determine how the items that ask 

for reports of patient experiences could 

be summarized into a smaller set of 

composite measures (through factor 

analysis). 

• Evaluate the measurement 

properties of the composite scales 

(assessment of reliability, validity, and 

variability of the measure). 

• Identify information (i.e., case mix 

adjusters) that should be used to adjust 

scores to ensure valid comparisons 

among primary care practices (PCPs). 

• Determine how CCQM–PC scores 

vary among practices that self-report 

processes of care that are more or less 

aligned with a medical home model. 

This study is being conducted by 

AHRQ through its contractor, American 

Institutes for Research (AIR), pursuant 

to AHRQ’s statutory authority to 

conduct and support research on 

healthcare and on systems for the 

delivery of such care, including 

activities with respect to quality 

measurement and improvement. 42 

U.S.C. 299a(a)(1) and (2). 

Method of Collection 

Thirty primary care practices of 

different types and ownership 

configurations will be recruited to 

provide a patient sample to AIR for the 

purpose of establishing the 

psychometrics of the CCQM–PC and 

understanding the relation of its 

domains to a practice-level measure of 

processes of care, the Medical Home 

Index (Long Version, MHI–LV). The 

CCQM–PC will be conducted by mail 

with phone follow-up for 

nonrespondents. Survey operations for 

the CCQM–PC will follow standard 

CAHPS practice: 

• Mail the questionnaire package, 

including a personalized letter 

introducing the study and explaining 

the respondent’s rights as a research 

participant. Include a postage-paid 

envelope to encourage participation. 

• Send a postcard reminder to 

nonrespondents 10 days after sending 

the questionnaire. 

• Send a second questionnaire with a 

reminder letter to those who have still 

not responded thirty days after the first 

mailing. 

• Begin follow-up by telephone with 

nonrespondents three weeks after 

sending the second questionnaire. 

Interviewers will attempt to locate 

respondents who have not responded to 

the mailed survey. 

• Verify telephone numbers for 

sample respondents prior to calling. 

• Make a maximum of 9 attempts by 

phone. 

• Include a toll-free number in the 

cards and letters for respondents to call 

if they have questions about the survey. 

The firm responsible for fielding the 

survey will establish a helpdesk that 

will start operating at the first mailing 

and that will remain open until close of 

fieldwork. 

• Answer incoming calls live during 

business hours and a recording machine 

will capture after hours calls. The afterhours calls will be returned next 

business day. 

• Ask two clinicians from each 

participating practice complete the 

MHI–LV by paper-and-pencil jointly 

and return the form to the AHRQ 

contractor. 

The information collected in the pilot 

survey will be used to test and improve 

the draft survey. The pilot design will 

support the standard suite of 

psychometric analyses conducted to 

identify and develop composite scoring 

algorithms as well as to provide 

evidence of the reliability and construct 

validity of the composite scores and any 

scores based on individual items. 

Additionally, the variations in 

composite scores and total CCQM–PC 

scores will be examined for any 

differences that may be correlated with 

variations in the practice’s selfassessment of its engagement in 

processes of care that are consistent 

with the medical home model. The 

analyses will include the following 

components: 

• Item functioning analysis 

• Confirmatory Factor Analysis 

• Exploratory Factor Analysis 

• Evaluation of the reliability, validity, 

and variability of composite and 

single-item scores 

• Case mix adjustment (if the data 

indicate this is needed). 

Because the survey items are being 

developed to measure specific aspects of 

care coordination in accordance with 

the domain framework developed 

through previous phases of AHRQ’s 

Care Coordination Measure 

Development portfolio, the factor 

structure of the survey items will be 

evaluated through multilevel 

confirmatory factor analysis. On the 

basis of the data analyses, items or 

factors may be dropped. Exploratory 

factor analysis is also planned. 

Data from the pilot survey will be 

used to make final adjustments to the 

CCQM–PC. The final survey instrument 


asabaliauskas on DSK5VPTVN1PROD with NOTICES
62154 Federal Register / Vol. 79, No. 200 / Thursday, October 16, 2014 / Notices 

will be made publicly available, at no 

charge, to prospective users, for use in 

research projects that aim to assess care 

coordination as it relates to quality care 

and healthcare outcomes, thereby 

helping to expand the evidence base for 

the care coordination construct and its 

associated processes. There is value, 

given where the field is now, in 

developing a survey of reasonable 

length that can be used for research 

purposes, but also can serve as the 

‘‘parent’’ survey from which a smaller 

subset of items appropriate for quality 

improvement could be drawn. 

A well-developed, psychometrically 

sound, practical survey of adult 

patients’ experiences of care 

coordination in primary care settings, 

that covers key conceptual domains 

articulated through AHRQ’s past work 

in this area, will help generate evidence 

that is needed to understand the 

relationship between care coordination 

processes and health outcomes, in 

addition to offering a way to explore 

other critical questions regarding care 

coordination. 

The development of this researchfocused survey is a critical step in 

moving toward the future development 

of measures of care coordination in 

primary care settings that can be used 

for accountability purposes, including 

those submitted for consideration of 

endorsement by the National Quality 

Forum. This will ensure that the 

measures or measure set is useful from 

a public reporting perspective to a 

variety of potential stakeholders, 

including patients seeking providers 

that engage in care coordination 

practices supported by the evidence 

base. The key target audiences for the 

use of the survey are researchers and, 

ultimately, payers (including health 

insurance plans, employers, and entities 

such as the Centers for Medicare & 

Medicaid Services), although use by 

health systems and individual primary 

care practices is also envisioned. 

Estimated Annual Respondent Burden 

Exhibit 1 shows the total estimated 

annualized burden hours for the 

CCQM–PC pilot survey (2,022 hours), 

including burden for survey 

respondents (1,890 hours) and practice 

staff (132 hours). With respect to the 

burden on CCQM–PC survey 

respondents, thirty practices will be 

sampled, with the survey sent to 375 

prospective respondents per sample. A 

40% response rate (in keeping with 

response rates on other CAHPS II+ and 

CAHPSS-like surveys of similar length 

and mode) will yield 150 respondents 

per practice. Total respondents were 

calculated by multiplying the number of 

practices by the respondents per 

practice, for a total of 4,500 (i.e., 150 × 

30 = 4,500). The survey has 102 items 

(79 assessment items, 4 items about 

healthcare services sought in the past 12 

months, and 19 items that assess 

participant characteristics such as 

demographics), with an estimated 

completion time of 25 minutes (.42 

hours) per survey response. This 

estimate is based on the length of 

previous CAHPS® surveys of 

comparable length that have been 

administered to similar populations. 

Burden hours for participating 

practices are calculated based on the 

total burden to one physician/

administrator and one other clinician to 

complete the MHI–LV. The measure 

author recommends that both physician 

and non-physician viewpoints are 

considered in the PCP’s response, thus 

the estimate is based on an assumption 

that two clinicians per practice will 

complete the MHI–LV process of care 

items together, with only one of the 

clinicians (i.e., the physician/

administrator) completing the items on 

practice characteristics. Contract staff 

from AIR will ensure that practices 

realize there is no burden to them on the 

MHI–LV other than the time required to 

fill out the MHI–LV tool (i.e., they can 

ignore the measure author’s reference in 

the instructions to a companion patient 

tool associated with the MHI–LV). 

EXHIBIT 1—ESTIMATED ANNUALIZED BURDEN HOURS FOR CCQM–PC SURVEY PILOT TEST BY ENTITY

CCQM–PC survey ........................................................................................... 4,500 1 0.42 1,890 

MHI–LV: 1 Physician/administrator ................................................................... 30 1 2.33 70 

MHI–LV: Non-physician clinician ..................................................................... 30 1 2.08 62 

Total .......................................................................................................... ........................ ........................ ........................ 2,022 

1The Instructions for completing the MHI–LV recommend that a physician/administrator and a non-physician clinician each fill out the index 

separately. So, even though it is one form as reproduced in Appendix B, we have two rows in the table to describe the burden of the two individuals. There are a series of questions on the first two pages of the index which simply require administrative information and would only need to 

be completed once. We assume that the administrator would complete these and so the time required for the administrator to 

Exhibit 2 shows the estimated 

annualized cost burden associated with 

the pilot survey administration. The 

total cost burden is estimated to be 

$51,228 for the one-time survey pilot. 

EXHIBIT 2—ESTIMATED ANNUALIZED COST BURDEN FOR CCQM–PC SURVEY PILOT TEST BY ENTITY

Survey Respondents ................................................................................................................... 1,890 1 $22.33 $42,204 

Physician/Administrator ............................................................................................................... 70 2 88.43 190 

Non-physician Clinician ............................................................................................................... 62 3 45.71 2,834 

Total Overall ......................................................................................................................... 2022 n/a 51,228 

1Average wage for civilian workers, http://www.bls.gov/news.release/ocwage.htm. 2Average wage for family and general practitioners, http://www.bls.gov/news.release/ocwage.htm. 3Average wage for nurse practitioners, http://www.bls.gov/news.release/ocwage.htm. 

Request for Comments 

In accordance with the Paperwork 

Reduction Act, comments on AHRQ’s 

information collection are requested 

with regard to any of the following: (a) 

Whether the proposed collection of 

information is necessary for the proper 

performance of AHRQ health care 

research and health care information 

dissemination functions, including 

whether the information will have 

practical utility; (b) the accuracy of 

AHRQ’s estimate of burden (including 

hours and costs) of the proposed 


asabaliauskas on DSK5VPTVN1PROD with NOTICES
Federal Register / Vol. 79, No. 200 / Thursday, October 16, 2014 / Notices 62155 

collection(s) of information; (c) ways to 

enhance the quality, utility, and clarity 

of the information to be collected; and 

(d) ways to minimize the burden of the 

collection of information upon the 

respondents, including the use of 

automated collection techniques or 

other forms of information technology. 

Comments submitted in response to 

this notice will be summarized and 

included in the Agency’s subsequent 

request for OMB approval of the 

proposed information collection. All 

comments will become a matter of 

public record. 

Dated: October 6, 2014. 

Richard Kronick, 

Director. 

[FR Doc. 2014–24513 Filed 10–15–14; 8:45 am] 

BILLING CODE 4160–90–M 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Agency for Healthcare Research and 

Quality 

Agency Information Collection 

Activities: Proposed Collection; 

Comment Request 

AGENCY: Agency for Healthcare Research 

and Quality, HHS. 

ACTION: Notice. 

SUMMARY: This notice announces the 

intention of the Agency for Healthcare 

Research and Quality (AHRQ) to request 

that the Office of Management and 

Budget (OMB) approve the proposed 

information collection project: 

‘‘Continuing Education for Comparative 

Effectiveness Research Survey.’’ In 

accordance with the Paperwork 

Reduction Act of 1995, Public Law 104– 

13 (44 U.S.C. 3506(c)(2)(A)), AHRQ 

invites the public to comment on this 

proposed information collection. 

This proposed information collection 

was previously published in the Federal 

Register on August 4th, 2014 and 

allowed 60 days for public comment. 

AHRQ did not receive any substantive 

comments. The purpose of this notice is 

to allow an additional 30 days for public 

comment. 

DATES: Comments on this notice must be 

received by November 17, 2014. 

ADDRESSES: Written comments should 

be submitted to: AHRQ’s OMB Desk 

Officer by fax at (202) 395–6974 

(attention: AHRQ’s desk officer) or by 

email at OIRA_submission@

omb.eop.gov (attention: AHRQ’s desk 

officer). 

FOR FURTHER INFORMATION CONTACT: 

Doris Lefkowitz, AHRQ Reports 

Clearance Officer, (301) 427–1477, or by 

email at doris.lefkowitz@AHRQ.hhs.gov. 

SUPPLEMENTARY INFORMATION: 

Proposed Project 

Continuing Education for Comparative 

Effectiveness Research Survey 

Patient-centered outcomes research 

(PCOR) is an area that has seen 

increased focus from research agencies 

and other government entities. Also 

known as comparative effectiveness 

research, PCOR is the focus of AHRQ’s 

Effective Health Care (EHC) program, 

which has the mission of providing 

health care decisionmakers (e.g., 

patients, health care providers, 

purchasers, and policymakers) with 

recent evidence-based information 

about the harms, benefits, and 

effectiveness of various treatment 

options by comparing medical devices, 

surgeries, tests, drugs, or ways to deliver 

health care. 

The EHC program was created in 

response to Section 1013 of the 

Medicare Prescription Drug, 

Improvement, and Modernization Act of 

2003 and became the first federal 

program to conduct PCOR and 

disseminate those findings to the public. 

AHRQ works with researchers, 

academic organizations, and research 

centers through the EHC program on 

work relating to methods, training, and 

dissemination of products to a variety of 

stakeholders to help spread awareness 

and knowledge about PCOR. It is 

important for AHRQ to be able to 

measure the effectiveness of these 

products, which include training 

modules and publications, specifically 

around how they are affecting health 

care professionals’ understanding, 

awareness, and use of PCOR and its 

related concepts. It is also important for 

AHRQ to be able to identify ways to 

improve how this information is being 

disseminated to the medical 

community. 

The Continuing Education for 

Comparative Effectiveness Research 

Project is designed to provide online 

continuing education materials that 

inform physicians and other health care 

providers about patient-centered health 

research from the EHC Program, 

specifically comparative effectiveness 

research reports, and other governmentfunded comparative clinical 

effectiveness research. Online 

multimedia continuing education 

modules based on the Effective Health 

Care Program http://www.

effectivehealthcare.ahrq.gov/tools-andresources/cmece-activities/ comparative 

effectiveness research reports will be 

planned, developed, disseminated, and 

promoted. In addition, data will be 

collected on the modules to assess their 

effectiveness and impact. 

This study is being conducted by 

AHRQ through its contractor, Hayes Inc. 

(Hayes) and Hayes’ subcontractors, 

Deloitte Consulting LLP (Deloitte), 

pursuant to AHRQ’s statutory authority 

to support the agency’s dissemination of 

comparative clinical effectiveness 

research findings. 42 U.S.C. 299b–37(a)– 

(c). 

Method of Collection 

To achieve the goals of this project, 

the following data collection will be 

implemented: (1) Each training module 

will involve one follow-up 

questionnaire that would be 

administered six months after the 

completion of the course for the 

purposes of tracking the longer-term 

effectiveness of the modules. 

This data collection will help to meet 

AHRQ’s objectives to: 

1. Understand the extent to which 

these online continuing education 

modules based on the EHC Program 

comparative effectiveness research 

reports improve knowledge of each 

topic and change participants’ 

awareness of, attitude towards, and/or 

confidence to apply CER in their 

clinical practice. 

2. Track information about the 

dissemination efforts employed for CE/ 

CER information specific to the 

modules, and the uptake of AHRQ’s 

other EHC Program materials as a result 

of the project, including the Clinician 

and Consumer Summaries when 

available. 

3. Determine implementation 

practices (e.g. changes in practice 

behavior or implementation of the 

information conveyed in the modules) 

that occur as a result of the learning. 

4. Identify opportunities for 

improving the presentation and delivery 

of CE modules by gathering information 

on the participants’ reactions to the 

modules and to the faculty presenters 

through the post-event evaluation 

assessment. 

AHRQ will use the information 

collected through this Information 

Collection Request to assess the shortand long-term progress in achieving the 

dissemination and implementation aims 

of the Continuing Education project. 

Estimated Total Respondent Burden 

Exhibit 1 provides information on the 

estimated time to complete the data 

collection survey. These educational 

activities are enduring training modules 

and will be available for a 2-year period. 

The AHRQ Continuing Education for 

Comparative Effectiveness Research 


asabaliauskas on DSK5VPTVN1PROD with NOTICES