Federal Register / Vol. 78, No. 18 / Monday, January 28, 2013 / Notices 5807 

will last about one hour. The student 

survey will include 1,800 students and 

takes 10 minutes to complete. The total 

burden is estimated to be 324 hours 

annually. 

Exhibit 2 shows the estimated 

annualized cost burden associated with 

the respondents’ time to participate in 

this research. The total cost burden is 

estimated to be $4,790 annually. 

EXHIBIT 1—ESTIMATED ANNUALIZED BURDEN HOURS

Form name Number of 

respondents 

Number of 

responses per 

respondent 

Hours per 

response Total 

Faculty Interview .............................................................................................. 24 1 1 24 

Student Survey ................................................................................................ 1,800 1 10/60 300

Total .......................................................................................................... 1,824 Na Na 324 

EXHIBIT 2—ESTIMATED ANNUALIZED COST BURDEN

Form name Number of 

respondents 

Total burden 

hours 

Average hourly wage rate * Total cost 

burden 

Faculty Interview .............................................................................................. 24 24 $47.70 $1,145 

Student Survey ................................................................................................ 1,800 300 12.15 3,645 

Total .......................................................................................................... 1,824 324 Na 4,790 

* Based on the mean wages for Health Specialties Teachers, Postsecondary (25–1071; 47.70/hour) and Teacher Assistants (25–9041; $12.15/ 

hour. Many of the students will be teaching and research assistants, making this the best occupational code for them), National Compensation 

Survey: Occupational wages in the United States May 2011, ‘‘U.S. Department of Labor, Bureau of Labor Statistics.’’ http://www.bls.gov/oes/current/oes_nat.htm#25-0000. 

Estimated Annual Costs to the Federal 

Government 

Exhibit 3 shows the total and 

annualized cost to the Federal 

Government for conducting this 

research. The total cost to the Federal 

Government is $683,335. The total 

annualized cost is estimated to be 

approximately $341,667. The total 

annual costs include the questionnaire 

development, administration, analysis, 

and study management. 

EXHIBIT 3—ESTIMATED TOTAL AND ANNUALIZED COST

Cost component Total cost Annualized 

cost 

Project Development ............................................................................................................................................... $144,707 $72,353 

Data Collection Activities ......................................................................................................................................... 283,667 141,833 

Data Processing and Analysis ................................................................................................................................. 135,523 67,762 

Publication of Results .............................................................................................................................................. 9,012 4,506 

Project Management ................................................................................................................................................ 65,722 32,861 

Overhead ................................................................................................................................................................. 44,704 22,352 

Total .................................................................................................................................................................. 683,335 341,667 

Request for Comments 

In accordance with the Paperwork 

Reduction Act, comments on AHRQ’s 

information collection are requested 

with regard to any of the following: (a) 

Whether the proposed collection of 

information is necessary for the proper 

performance of AHRQ health care 

research and health care information 

dissemination functions, including 

whether the information will have 

practical utility; (b) the accuracy of 

AHRQ’s estimate of burden (including 

hours and costs) of the proposed 

collection(s) of information; (c) ways to 

enhance the quality, utility, and clarity 

of the information to be collected; and 

(d) ways to minimize the burden of the 

collection of information upon the 

respondents, including the use of 

automated collection techniques or 

other forms of information technology. 

Comments submitted in response to 

this notice will be summarized and 

included in the Agency’s subsequent 

request for OMB approval of the 

proposed information collection. All 

comments will become a matter of 

public record. 

Dated: January 16, 2013. 

Carolyn M. Clancy, 

Director. 

[FR Doc. 2013–01342 Filed 1–25–13; 8:45 am] 

BILLING CODE 4160–90–M 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Agency for Healthcare Research and 

Quality 

Agency Information Collection 

Activities: Proposed Collection; 

Comment Request 

AGENCY: Agency for Healthcare Research 

and Quality, HHS. 

ACTION: Notice. 

SUMMARY: This notice announces the 

intention of the Agency for Healthcare 

Research and Quality (AHRQ) to request 

that the Office of Management and 

Budget (OMB) approve the proposed 

information collection project: 

‘‘Applying Novel Methods To Better 


tkelley on DSK3SPTVN1PROD with 
5808 Federal Register / Vol. 78, No. 18 / Monday, January 28, 2013 / Notices 

Understand the Relationship Between 

Health IT and Ambulatory Care 

Workflow Redesign.’’ In accordance 

with the Paperwork Reduction Act, 44 

U.S.C. 3501–3521, AHRQ invites the 

public to comment on this proposed 

information collection. 

DATES: Comments on this notice must be 

received by March 29, 2013. 

ADDRESSES: Written comments should 

be submitted to: AHRQ’s OMB Desk 

Officer by fax at (202) 395–6974 

(attention: AHRQ’s desk officer) or by 

email at 

OIRA_submission@omb.eop.gov 

(attention: AHRQ’s desk officer). 

Copies of the proposed collection 

plans, data collection instruments, and 

specific details on the estimated burden 

can be obtained from the AHRQ Reports 

Clearance Officer. 

FOR FURTHER INFORMATION CONTACT: 

Doris Lefkowitz, AHRQ Reports 

Clearance Officer, (301) 427–1477, or by 

email at doris.lefkowitz@AHRQ.hhs.gov. 

SUPPLEMENTARY INFORMATION: 

Proposed Project 

Applying Novel Methods To Better 

Understand the Relationship Between 

Health IT and Ambulatory Care 

Workflow Redesign 

The Agency for Healthcare Research 

and Quality (AHRQ) requests that the 

Office of Management and Budget 

(OMB) approve, under the Paperwork 

Reduction Act of 1995, AHRQ’s 

collection of information for the project 

‘‘Applying Novel Methods To Better 

Understand the Relationship Between 

Health IT and Ambulatory Care 

Workflow Redesign.’’ The data to be 

collected consists of interviews and 

focus groups with clinical, non-clinical, 

and management staff about their 

experiences with new health 

information technology (IT) in an 

ambulatory care facility. The overall 

goal of this study is to characterize the 

relationship between health IT 

implementation and health care 

workflow in six (6) small and mediumsized ambulatory care practices 

implementing patient-centered medical 

homes (PCMH), with a focus on the 

influence of behavioral and 

organizational factors and the effects of 

disruptive events. 

AHRQ is a lead Federal agency in 

developing and disseminating evidence 

and evidence-based tools on how health 

IT can improve health care quality, 

safety, efficiency, and effectiveness. 

Health IT has been widely viewed as 

holding great promise to improve the 

quality of health care in the U.S. Health 

IT can improve access to information for 

both patients and providers, 

empowering patients to become 

involved in their own self-care. 

Increased patient safety can result from 

health IT when records are shared, 

medications are reconciled, and adverse 

event alerts are in place. When health IT 

improves efficiency, providers can 

spend more time directly caring for 

patients, ultimately improving the 

quality of care patients receive. 

In redesigning an ambulatory office 

practice as a patient-centered medical 

home (PCMH), health IT is intended to 

allow for a seamless and organized flow 

of information among providers. The 

health IT system is critical, because 

under the PCMH model, a team of 

clinicians aims to provide continuous 

and coordinated care throughout a 

patient’s lifetime. 

Unfortunately, health IT systems can 

fail to generate anticipated results and 

even carry unintended consequences 

which undermine usability and 

usefulness. Directly or indirectly, health 

IT may create more work, new work, 

excessive system demands, or 

inefficient workflow (the sequence of 

clinical tasks). Electronic reminders and 

alerts may be timed poorly. Software 

may require excessive switching 

between screens, leading to cognitive 

distractions for end users. Providers 

may spend more time on health IT 

system-related tasks than on direct 

patient care. 

The literature also suggests that the 

ambulatory health care environment is 

full of unpredictable yet frequently 

occurring events requiring actions that 

deviate from normal practice. 

Unpredictable events such as 

interruptions requiring a provider’s 

immediate attention, or disruptions in 

the normal functioning of the health IT 

system (exceptions) divert health care 

workers from the usual course of 

workflow. The inability of health IT to 

properly accommodate these events 

could cause compromises to clinical 

work. 

Because of adverse, unintended and 

disruptive consequences, developing an 

understanding of how health IT 

implementation alters clinical work 

processes and workflow is crucial. 

Unfortunately, research is scarce, and 

methods of investigation vary widely. 

Empirical evidence of health IT’s impact 

on clinical workflow has been 

‘‘anecdotal, insufficiently supported, or 

otherwise deficient in terms of scientific 

rigor’’ (Carayon and Karsh, 2010). 

This study aims to examine more 

systematically the impact of health IT 

on workflow in six (6) small and 

medium-sized ambulatory care practices 

varying in their characteristics but all 

implementing PCMH. All of the 

practices will be in the process of 

implementing a new health IT system 

during the course of the study, but some 

may have an existing, baseline system 

such as an electronic health record 

system. The focus of the study will be 

on the new systems being implemented. 

It will employ the complementary 

quantitative and qualitative methods of 

previous research. The combination of 

methods produces quantitative results 

and allows validation through 

observation and solicitation of 

qualitative participant opinions. 

The specific goals of this study are to 

identify (1) the relationship between 

health IT implementation and 

ambulatory care workflow; (2) the 

behavioral and organizational factors 

and the role they play in mitigating or 

augmenting the impact of health IT on 

workflow; and (3) how the impacts of 

health IT are magnified through 

disruptive events such as interruptions 

and exceptions. 

This study is being conducted by 

AHRQ through its contractor, Billings 

Clinic, pursuant to AHRQ’s statutory 

authority to conduct and support 

research on healthcare and on systems 

for the delivery of such care, including 

activities with respect to the quality, 

effectiveness, efficiency, 

appropriateness and value of healthcare 

services and with respect to clinical 

practice, including primary care and 

practice-oriented research. 42 U.S.C. 

299a(a)(1) and (4). 

Method of Collection 

To achieve the goals of this project the 

following data collection will be 

implemented: 

(1) Mapping of Study Practices. This 

activity will detect any changes made to 

the physical layout as a result of 

implementing PCMH and health IT. 

Practices will be mapped at the 

beginning of the study and maps will be 

updated as needed. Recording this 

information will not burden the clinic 

staff and is not included in the burden 

estimates. 

(2) Staff Observation. Clinicians 

(physicians, nurse practitioners, 

physician assistants, nurses, medical 

assistants, pharmacists, and case 

managers) and non-clinical office 

personnel will be observed to delineate 

the overall characteristics of clinical 

workflow before, during, and after 

health IT implementation. Particular 

attention will be paid to interruptions 

and exceptions. If necessary and if the 

situation allows, observers will as 

unobtrusively as possible ask clinic staff 

to clarify certain observed actions. 

Recording this information will not 


tkelley on DSK3SPTVN1PROD with 
Federal Register / Vol. 78, No. 18 / Monday, January 28, 2013 / Notices 5809 

burden the clinic staff and is not 

included in the burden estimates. 

(3) Before—After Time and Motion 

Study. This activity quantifies staffs 

time expenditures on different clinical 

activities and delineates the sequence of 

task execution. It will be conducted 

before and after health IT 

implementation. This data will be 

collected by observation only. 

Recording this information will not 

burden the clinic staff and is not 

included in the burden estimates. 

(4) Extraction of Clinical Data. Logs, 

audits trails, and time-stamped clinical 

data will be extracted from the health IT 

system to reconstruct clinical workflow 

related to the health IT system. This 

information validates and supplements 

the data recorded by human observers. 

Extracting this data will not burden the 

clinic staff and is not included in the 

burden estimates. 

(5) Semi-Structured Interviews. This 

data collection will be conducted posthealth IT implementation to solicit 

attitudes and perceptions by health IT 

end users including clinical staff, nonclinical personnel, and management 

regarding how health IT has changed 

their workflow. Particular attention will 

be paid to behavioral and organizational 

factors. 

(6) Focus Group. A focus group will 

be conducted post-health IT 

implementation with the clinical staff, 

non-clinical personnel, and 

management team to ensure the research 

findings, as well as the interpretation of 

the findings, accurately reflect their 

experiences using health IT. 

On-site data collection will be 

conducted over a 5-day period during 

each of three phases. Preimplementation data collection 

activities will be conducted prior to user 

training. During-implementation data 

collection will begin when staff are 

instructed to start using the health IT 

system. Post-implementation data 

collection will be conducted 

approximately 3 months after 

implementation at each study practice. 

The qualitative study components of 

this project, namely staff observations, 

semi-structured interviews, and focus 

groups, will generate qualitative data in 

the form of observation notes and 

interview transcripts. The time-andmotion study and the electronic clinical 

data will produce quantitative 

information in the form of sequences of 

clinical activities and information about 

the duration, location, and performer of 

each action. Mapping will create 

annotated floor plans delineating the 

physical layout of each study clinic, 

which will be incorporated in the 

collection and analysis of the data of the 

other study components. 

Estimated Annual Respondent Burden 

Exhibit 1 shows the estimated annual 

burden hours for participation in this 

study. The semi-structured interview 

will be completed by 60 respondents 

across the 6 clinics (10 per practice) and 

requires one hour. Sixty (60) clinic staff 

members will be asked to participate in 

the focus group across all 6 clinics (10 

per practice). The focus group requires 

no more than 45 minutes. The total 

annual burden is estimated to be 105 

hours. 

Exhibit 2 shows the estimated annual 

cost burden associated with the 

respondents’ time to participate in this 

research. The total annual burden is 

estimated to be $5,505. 

EXHIBIT 1—ESTIMATED ANNUALIZED BURDEN HOURS

Form name Number of 

respondents 

Number of 

responses per 

respondent 

Hours per 

response 

Total burden 

hours 

Semi-Structured Interview ............................................................................... 60 1 1 60 

Focus Group .................................................................................................... 60 1 45/60 45 

Total .......................................................................................................... 120 ........................ ........................ 105 

EXHIBIT 2—ESTIMATED ANNUALIZED COST BURDEN

Form name Number of 

respondents 

Total burden 

hours 

Average hourly wage rate * Total cost 

burden 

Semi-Structured Interview ............................................................................... 60 60 $55 $3,300 

Focus Group .................................................................................................... 60 45 49 2,205 

Total .......................................................................................................... 120 105 ........................ 5,505 

* Based upon the mean of the average wages, National Compensation Survey. Occupational wages in the United States July 2010, U.S. Department of Labor, Bureau of Labor Statistics, http://www.bls.gov/ncs/ocs/sp/nctb1477.pdf. For the semi-structured interviews, hourly wage is an 

average including 2 physicians or surgeons ($85.67), 1 registered nurse ($32.42), 2 non-physician providers (measured here as physician assistants, $43.44), and 1 senior administrator (measured here as ‘‘Medical and health services managers,’’ $42.28). For focus groups, 3.34 physicians or surgeons ($85.67), 1.66 non-physician providers (measured here as physician assistants, $43.44), 3.34 registered nurses ($32.42), and 

1.66 medical assistants ($14.46). 

Estimated Annual Costs to the Federal 

Government 

The total cost of this study is 

$799,014 over a 36-month time period 

from June 1, 2012 through May 31, 2015 

for an annualized cost of $266,338. 

(Because the project entails gathering 

data before, during, and after health IT 

implementation, a period of 21 months 

is planned for data collection.) Exhibit 

3 provides a breakdown of the estimated 

total and average annual costs by 

category. 

EXHIBIT 3—ESTIMATED TOTAL AND ANNUALIZED COST

Cost component Total cost Annualized cost 

Project Development ....................................................................................................................................... $135,759 $45,253 

Data Collection Activities ................................................................................................................................. 177,460 59,153 


tkelley on DSK3SPTVN1PROD with 
5810 Federal Register / Vol. 78, No. 18 / Monday, January 28, 2013 / Notices 

EXHIBIT 3—ESTIMATED TOTAL AND ANNUALIZED COST—Continued 

Cost component Total cost Annualized cost 

Data Processing and Analysis ......................................................................................................................... 239,426 79,809 

Publication of Results ...................................................................................................................................... 51,779 17,260 

Project Management ........................................................................................................................................ 67,729 22,576 

Overhead ......................................................................................................................................................... 126,861 42,287 

Total .......................................................................................................................................................... 799,014 266,338 

Request for Comments 

In accordance with the Paperwork 

Reduction Act, comments on AHRQ’s 

information collection are requested 

with regard to any of the following: (a) 

Whether the proposed collection of 

information is necessary for the proper 

performance of AHRQ healthcare 

research and healthcare information 

dissemination functions, including 

whether the information will have 

practical utility; (b) the accuracy of 

AHRQ’s estimate of burden (including 

hours and costs) of the proposed 

collection(s) of information; (c) ways to 

enhance the quality, utility, and clarity 

of the information to be collected; and 

(d) ways to minimize the burden of the 

collection of information upon the 

respondents, including the use of 

automated collection techniques or 

other forms of information technology. 

Comments submitted in response to 

this notice will be summarized and 

included in the Agency’s subsequent 

request for OMB approval of the 

proposed information collection. All 

comments will become a matter of 

public record. 

Dated: January 16, 2013. 

Carolyn M. Clancy, 

Director. 

[FR Doc. 2013–01345 Filed 1–25–13; 8:45 am] 

BILLING CODE 4160–90–M 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Agency for Healthcare Research and 

Quality 

AHRQ Standing Workgroup for Quality 

Indicator Measure Specification 

AGENCY: Agency for Healthcare Research 

and Quality (AHRQ), HHS. 

ACTION: Notice of request for 

nominations. 

SUMMARY: The Agency for Healthcare 

Research and Quality (AHRQ) is seeking 

nominations for both a time-limited 

work group and a standing work group 

to be convened by an AHRQ contractor. 

The work groups shall be comprised of 

individuals with knowledge of the 

AHRQ Quality Indicators (QIs), their 

technical specifications, and associated 

methodological issues. The overarching 

goals of each group are to provide 

feedback to AHRQ regarding 

refinements to the Qls. The time-limited 

workgroup is more restricted to specific 

clinical or methodological issues, while 

the standing workgroup addresses 

broader issues related to the 

measurement cycle. 

DATES: Please submit nominations on or 

before March 15, 2013. Self-nominations 

are welcome. Third-party nominations 

must indicate that the individual has 

been contacted and is willing to serve 

on the workgroup. Selected candidates 

will be contacted by AHRQ no later than 

April 5, 2013. Please include the 

committee of interest. Candidates may 

apply for both. 

ADDRESSES: Nominations can be sent in 

the form of a letter or email, preferably 

as an electronic file with an email 

attachment, and should specifically 

address the submission criteria as noted 

below. Electronic submissions are 

strongly encouraged. Responses should 

be submitted to: ATTN: Pamela Owens, 

Agency for Healthcare Research and 

Quality, Center for Delivery, 

Organization and Markets, 540 Gaither 

Road, Rockville, MD 20850, Email: 

pam.owens@AHRQ.hhs.gov. 

FOR FURTHER INFORMATION CONTACT: 

Pamela Owens, Ph.D., Senior Research 

Scientist, Agency for Healthcare 

Research and Quality, Center for 

Delivery, Organization and Markets, 540 

Gaither Road, Rockville, MD 20850, 

Email: pam.owens@AHRQ.hhs.gov; 

Phone: (301) 427–1412; Fax: (301) 427– 

1430. 

SUPPLEMENTARY INFORMATION: These 

workgroups are being administered by 

AHRQ’s contractor as part of a 

structured approach to formally and 

broadly engage stakeholders, and to 

enhance and expand transparency about 

the scientific development of the AHRQ 

QIs. 

Time-Limited Work Group 

Time-limited workgroups are 

formative in nature, providing feedback 

on significant measure improvement 

issues and representing a broad range of 

stakeholders. The focus for this 

upcoming year will be the Prevention 

Quality Indicators (PQI). The role of 

time-limited group members is to: (1) 

Provide technical guidance on the PQI 

specifications and rationales, risk 

adjustment strategies, and other quality 

measurement issues; (2) provide input 

on critical information gaps, as well as 

research methods to address them; (3) 

provide guidance on draft 

recommendations for the PQI measure 

refinements; (4) offer scientifically 

rigorous recommendations for the 

evaluation and validation efforts 

required to ensure the accuracy of the 

PQIs; and, (5) provide input on and 

review of the contractor’s technical 

report resulting from the workgroup’s 

discussions. 

The time-limited workgroup will 

consist of 8–12 members consisting of: 

• One or more statisticians 

specialized in the relevant statistical 

methods and applications 

• One or more individuals with 

expertise in community health care and 

prevention, and access to and quality of 

care 

• One or more individuals with 

experience using AHRQ PQI measures 

for assessing health system performance 

and public reporting 

• One or more individuals with 

expertise in developing algorithms 

using ICD–9–CM codes to construct or 

modify quality indicators using 

administrative data is desirable, but not 

mandatory 

In addition, the work group is 

expected to include representatives 

from impacted provider groups and 

their professional organizations, other 

stakeholders, consumers and other 

users, quality alliances, medical or 

specialty societies, measure developers, 

accrediting organizations, and public 

and private payers. 

Standing Work Group 

The standing workgroup is part of a 

structured approach to bring together 

individuals from multiple disciplines 

for the purpose of providing technical 

feedback on proposed updates to the 

AHRQ QIs. The intent is to collect 


tkelley on DSK3SPTVN1PROD with