Federal Register / Vol. 76, No. 231 / Thursday, December 1, 2011 / Notices 74785 

higher per-foot dockage fee than other 

vessels’’ is unlawfully discriminatory. 

Complainant asserted this claim in the 

United States District Court for the 

District of Alaska, which court referred 

the claim to the Commission upon 

Respondent’s motion in that court 

‘‘alleging that the dispute was within 

the Commission’s primary jurisdiction.’’ 

Thus Complainant alleges that 

Respondent has violated the Shipping 

Act of 1984 ‘‘by unreasonably 

prejudicing and disadvantaging Minto 

and unreasonably preferring and 

advantaging others in violation of 46 

U.S.C. 41106(2), and by failing to 

establish, observe, and enforce just and 

reasonable regulations and practices 

relating to or connected with receiving, 

handling, storing, or delivering 

property, in violation of 46 U.S.C. 

41102. Complainant also presents its 

state law discrimination claim at the 

direction of the District Court. 

Complainant requests that 

Respondent be ordered ‘‘after due 

hearing, to answer the charges herein, to 

cease and desist from the aforesaid 

violations of the Shipping Act, to 

establish and put in force such practices 

as the Commission determines to be 

lawful and reasonable, and to pay Minto 

reparations for PARN’s violations of the 

Act, including the amount of the actual 

injury, plus interest, costs and attorneys 

fees, and any other damages to be 

determined; and that the Commission 

order any such other relief as it 

determines proper.’’ The full text of the 

complaint can be found in the 

Commission’s Electronic Reading Room 

at http://www.fmc.gov. 

This proceeding has been assigned to 

the Office of Administrative Law Judges. 

Hearing in this matter, if any is held, 

shall commence within the time 

limitations prescribed in 46 CFR 502.61, 

and only after consideration has been 

given by the parties and the presiding 

officer to the use of alternative forms of 

dispute resolution. The hearing shall 

include oral testimony and crossexamination in the discretion of the 

presiding officer only upon proper 

showing that there are genuine issues of 

material fact that cannot be resolved on 

the basis of sworn statements, affidavits, 

depositions, or other documents or that 

the nature of the matter in issue is such 

that an oral hearing and crossexamination are necessary for the 

development of an adequate record. 

Pursuant to the further terms of 46 CFR 

502.61, the initial decision of the 

presiding officer in this proceeding shall 

be issued by November 23, 2012 and the 

final decision of the Commission shall 

be issued by March 25, 2013. 

Karen V. Gregory, 

Secretary. 

[FR Doc. 2011–30895 Filed 11–30–11; 8:45 am] 

BILLING CODE 6730–01–P 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Agency for Healthcare Research and 

Quality 

Agency Information Collection 

Activities: Proposed Collection; 

Comment Request 

AGENCY: Agency for Healthcare Research 

and Quality, HHS. 

ACTION: Notice. 

SUMMARY: This notice announces the 

intention of the Agency for Healthcare 

Research and Quality (AHRQ) to request 

that the Office of Management and 

Budget (OMB) approve the proposed 

information collection project: ‘‘Use of 

Deliberative Methods to Enhance Public 

Engagement in the Agency for 

Healthcare Research and Quality’s 

(AHRQ’s) Effective Healthcare (EHC) 

Program and Comparative Effectiveness 

Research (CER) Enterprise.’’ In 

accordance with the Paperwork 

Reduction Act, 44 U.S.C. 3501–3521, 

AHRQ invites the public to comment on 

this proposed information collection. 

DATES: Comments on this notice must be 

received by January 30, 2012. 

ADDRESSES: Written comments should 

be submitted to: Doris Lefkowitz, 

Reports Clearance Officer, AHRQ, by 

email at doris.lefkowitz@AHRQ.hhs.gov. 

Copies of the proposed collection 

plans, data collection instruments, and 

specific details on the estimated burden 

can be obtained from the AHRQ Reports 

Clearance Officer. 

FOR FURTHER INFORMATION CONTACT: 

Doris Lefkowitz, AHRQ Reports 

Clearance Officer, (301) 427–1477, or by 

email at dorislefkowitz@AHRQ.hhs.gov. 

SUPPLEMENTARY INFORMATION: 

Proposed Project 

Use of Deliberative Methods To Enhance 

Public Engagement in the Agency for 

Healthcare Research and Quality’s 

(AHRQ’s) Effective Healthcare (EHC) 

Program and Comparative Effectiveness 

Research (CER) Enterprise 

With this project, AHRQ seeks 

evidence on the feasibility and 

usefulness of public deliberation as an 

approach to obtaining public input on 

questions related to the conduct and use 

of comparative effectiveness research 

(CER). Although stakeholder 

engagement has been central to the 

Effective Healthcare (EHC) prop-am to 

date, public input has not traditionally 

been used to inform and guide broad 

strategies related to the use of evidence 

to inform decisions. This study would 

provide a research base to address this 

gap. This project closely ties to AHRQ’s 

efforts to improve the rigor of methods, 

as it will generate methodological 

evidence through a randomized 

controlled experiment comparing five 

distinct methods of public deliberation 

to find the most effective approaches for 

involving the general public, including 

members of AHRQ’s priority 

populations, in questions related to the 

research enterprise. Public deliberation 

is a strategy for engaging lay people in 

informing decisions when these 

decisions require consideration of 

values and ethics in addition to 

scientific evidence. It includes three 

core elements: 

(1) Convening a group of people 

(either in person or via online 

technologies to connect people in 

remote locations), 

(2) Educating the participants on the 

relevant issue(s) through dissemination 

of educational materials and/or the use 

of content experts, and 

(3) Having the participants engage in 

a reason-based discussion, or 

deliberation, on all sides of the issue(s). 

AHRQ wishes to study the 

effectiveness of public deliberation, 

because it offers the opportunity to 

obtain public input on complex topics 

in an environment that encourages 

participants to educate themselves 

about the topic and discuss it in a 

thoughtful, respectful manner. 

Information about the topic is 

intentionally neutral and respectful of 

the full range of underlying values and 

experience with healthcare issues in the 

population. This approach is designed 

to improve upon the sometimes 

superficial or ‘‘top of mind’’ responses 

that are often provided by public 

opinion surveys. AHRQ views public 

deliberation as a potential source of 

higher quality public input on issues 

fundamental to the Agency’s mission, 

such as the best and most effective ways 

to use comparative effectiveness 

research, than has heretofore been 

available. 

Several distinct deliberative methods 

have been developed and used 

previously. They share the three core 

elements of public deliberation, but 

differ on key features of implementation 

such as duration, whether they take 

place in-person or online, and the use 

of content experts. Although there is 


emcdonald on DSK5VPTVN1PROD with NOTICES
74786 Federal Register / Vol. 76, No. 231 / Thursday, December 1, 2011 / Notices 

considerable theoretical and case study 

literature endorsing the value of public 

deliberation, there has been little 

empirical research about its 

effectiveness and even less about the 

comparative merits of different 

deliberative methods (Community 

Forum Deliberative Methods Literature 

Review, 2010). 

The objectives of this study are to: 

1. Obtain informed and deliberated 

input from lay people on important 

questions underlying AHRQ’s research 

program; and 

2. Expand the evidence base for the 

use of public deliberation methods for 

exploring issues relevant to healthcare 

research by comparing the outcomes of 

five distinct deliberative methods to a 

control condition and to each other. 

This study is being conducted by 

AHRQ through its contractor, the 

American Institutes of Research (AIR), 

pursuant to AHRQ’s statutory authority 

to (1) promote healthcare quality 

improvement by conducting and 

supporting both research that develops 

and presents scientific evidence 

regarding all aspects of healthcare and 

the synthesis and dissemination of 

available scientific evidence for use by 

policymakers, among others, and (2) 

conduct and support research, provide 

technical assistance, and disseminate 

information on healthcare and on 

systems for the delivery of such care. 

See 42 U.S.C. 299(b)(1)(A), (D), (F), and 

(G); 42 U.S.C. 299(b)(2); 42 U.S.C. 

299a(a)(1)–(4). 

Method of Collection 

To achieve the objectives of this study 

the following activities and data 

collections will be implemented: 

(1) Participant recruitment—A short 

screening questionnaire, including a 

brief overview of the study, will be used 

to recruit persons for the study. 

(2) Educational Materials— 

Educational materials are designed to 

inform participants about the topics that 

are being deliberated and will be 

provided to all 1,685 participants 

recruited before the implementation of 

any of the methods, but after the 

administration of the Knowledge and 

Attitudes Pre-test Survey (described 

below). Additional content provided 

during the deliberative method sessions 

includes an overview of the study and 

the background materials needed by 

participants to competently deliberate 

the issues. For two methods (ODP and 

IDP; see below) educational materials to 

be used during the sessions will be sent 

to participants before the sessions (but 

after administration of the pre-test). 

(3) Deliberative Discussion Groups 

and Control Group—The purpose of the 

discussion groups is to obtain informed 

and deliberated input from lay people 

on an important set of issues underlying 

healthcare research. Participants will be 

randomly assigned to one of the five 

deliberative methods or a control 

condition. The five methods were 

selected because they have been 

previously implemented and vary on 

key features that may affect the 

scalability and effectiveness of the 

methods, including: duration (from two 

hours to three days), mode of 

implementation (online versus in 

person), role of content experts, and 

time between sessions allowing 

participants to seek additional 

information on the issues and 

communicate informally with other 

participants. The subject of the 

deliberations is the use of research 

evidence in healthcare decision-making. 

This deliberative topic encompasses 

several themes or ‘‘variations’’ that will 

be elaborated in the deliberations: 

1. Use of evidence to encourage better 

healthcare: Is evidence useful (or, what 

kind of evidence is useful) to a 

physician and a patient who are 

considering a test or treatment that has 

been found to be ineffective, less 

effective than another, riskier than 

another, or for which effectiveness has 

not been demonstrated? 

2. Use of evidence to encourage better 

value: Is evidence useful (or, what kind 

of evidence is useful) to a physician and 

a patient who are considering a test or 

treatment that is effective even though 

an equally effective but less expensive 

alternative is available? 

3. Decision-making when evidence 

shows more complex trade-offs: Is 

evidence useful (or, what kind of 

evidence is useful) in treatment 

decisions that involve the balancing of 

effectiveness, risk, and value? 

The issues involved in each variation 

will be discussed in the context of 

specific comparative effectiveness 

research (CER) examples. These 

‘‘vignettes’’ illustrate the issues and 

elicit participants’ input on the issues 

and the values employed by participants 

in the deliberations. 

(4) Knowledge and Attitudes Pre-test 

Survey—This survey will measure 

knowledge of and attitudes about the 

health issues discussed in the 

deliberations. It will be administered to 

deliberation participants and controls 

before educational materials are sent or 

the methods are implemented. 

As described, study participants will 

be provided with educational materials 

related to the deliberative topic. In order 

to assess whether or not participants 

were sufficiently informed on the topics 

addressed in the materials, the 

Knowledge and Attitudes Survey 

contains items assessing knowledge of 

medical research and medical evidence, 

of comparative effectiveness research, 

and of healthcare costs. The attitudinal 

questions refer to the use of medical 

evidence in healthcare decision making. 

They include attitudes about health care 

decision-making when research findings 

can provide no support for, or conflict 

with patient and doctor preferences for 

particular treatments. 

The questionnaire will also gather 

demographic and other information 

necessary to characterize the study 

sample, test the success of the 

randomization, and define population 

subgroups for which variation in 

outcomes will be examined. The 

demographic variables also will be used 

to control for participant and group 

characteristics that may influence the 

outcomes. Even though the design 

involves randomization, and these 

characteristics should be balanced 

across groups, including them in the 

statistical models guards against 

inadequate results from randomization. 

The variables to be measured in the 

Knowledge and Attitudes Pre-test 

Survey include: 

• Sociodemographic characteristics: 

Gender, age, marital status, education, 

employment status, household 

income, race/ethnicity, priority 

population, languages spoken (in 

addition to English) 

• General health status 

• Recent experience with the healthcare 

system (e.g., seeing a healthcare 

provider more than three times for the 

same condition in the last 12 months) 

• Health insurance coverage 

• Health information-seeking behavior 

(e.g., the extent to which people seek 

healthcare information or rely on their 

doctors to provide information) 

(5) Knowledge and Attitudes Post-test 

Survey—This survey will measure 

knowledge of and attitudes about the 

issues discussed in the deliberations 

after the deliberations take place. It will 

be administered to deliberation 

participants and controls within one 

week following conclusion of the 

deliberative methods and will include 

the same knowledge and attitude 

questions as the pre-test questionnaire. 

(6) Deliberative Experience Survey— 

As described above, the five deliberative 

methods being tested vary in terms of 

duration, mode, use of educational 

materials, and time between deliberative 

sessions. A one-time survey will be 

administered to participants in the 

deliberative methods after 

implementation of the experimental 

conditions to compare deliberative 


emcdonald on DSK5VPTVN1PROD with NOTICES
Federal Register / Vol. 76, No. 231 / Thursday, December 1, 2011 / Notices 74787 

methods to each other. Levels of 

discourse quality and implementation 

quality achieved will be assessed. Using 

multi-item scales, the survey will 

measure the following: 

Discourse quality 

• Equal participation in the 

discussions 

• Respect for others’ opinions and 

tolerance of differing perspectives 

• Appreciation of perspectives other 

than their own 

• Reasoned justification of ideas: 

Sharing the reasoning or rationale 

for positions, opinions, beliefs, or 

preferences 

Implementation quality 

• Quality of group facilitation 

• Quality. of the educational 

materials provided 

• Quality of the experts 

• Transparency of the process and 

use of the results 

• Participants’ perceived value of 

method 

• Participants’ view of the influence 

the results will have on programs 

In sum, information collection in this 

study will entail qualitative transcript 

review and quantitative surveys. This 

information will be used to describe and 

summarize the input obtained from the 

participants in the deliberative groups 

concerning the use of evidence, 

presenting the findings in reports for 

AHRQ and the public. 

The information from the surveys also 

will be used to expand the evidence 

base for public deliberation. The 

experiment is designed to: (1) Compare 

the effectiveness of the five deliberative 

methods to the control condition and to 

each other, (2) compare the quality of 

the discourse achieved by the 

deliberative methods to each other, (3) 

assess the quality of implementation of 

the five methods, and (4) test for 

variation in effectiveness and discourse 

quality by features of the deliberations 

and for population subgroups defined 

by sociodemogiaphic characteristics of 

the participants. 

Estimated Annual Respondent Burden 

Exhibit 1 shows the estimated 

annualized burden associated with the 

respondents’ time to participate in this 

research. The total annualized burden 

hours are estimated to be 11,647 hours. 

The burden estimate comprises the 

following activities: 

Participant Recruitment—The 

screening questionnaire and recruitment 

letter and materials will be sent to 1,685 

participants. We estimate that it will 

take 15 minutes to complete the 

questionnaire and review the 

recruitment letter and materials. 

Educational materials—Educational 

materials will be provided to all 1,685 

participants recruited before the 

implementation of any of the methods. 

We estimate that it will take up to 1 

hour to review the materials. 

Short Citizens’ Deliberation (SCD): 

This method will be tested with 192 

participants (12 groups). Participants 

will attend a single, 2-hour in-person 

meeting. 

Online Deliberative Polling® (ODP): 

This method will be tested with 288 

participants (24 groups) and will consist 

of 4 online sessions over the course of 

4 weeks; in total, this method will take 

about 5 hours per person. 

In-Person Deliberative Polling® (IDP): 

This method will be tested with 288 

participants (16 groups); participants 

will attend a single in-person meeting, 

lasting a full day. 

Citizens’ Panel (CP): This method will 

be tested with 96 participants (4 

groups); participants will attend a 3-day, 

in-person meeting. 

Interrupted Deliberation (ID): This 

method will be tested with 192 

participants (12 groups). Participants 

will attend 2 in-person meetings, lasting 

3 hours each, a week apart. Between 

meetings, participants will be asked to 

access an online platform. In total, this 

method will take about 6 hours per 

person. 

Knowledge and Attitudes Pre-test 

Survey: This survey will be 

administered to 1,685 participants and 

will take an estimated 30 minutes to 

complete. 

Knowledge and Attitudes Post-test 

Survey: This survey will be 

administered to 1,685 participants and 

will take an estimated 20 minutes to 

complete. 

Deliberative Experience Survey: This 

survey will be administered to 1,056 

deliberative methods participants at the 

conclusion of the deliberative method. It 

will take about 15 minutes to complete. 

EXHIBIT 1—ESTIMATED ANNUALIZED BURDEN HOURS

Form name/Deliberative method Number of 

respondents 

Number of 

responses per 

respondent 

Hours per 

response 

Total burden 

hours 

Recruitment and Consent Materials ................................................ 1685 1 15/60 421 

Short Citizens’ Deliberation (SCD) .................................................. 192 1 2 384 

Online Deliberative Polling® (ODP) ................................................ 288 1 5 1440 

In-Person Deliberative Polling® (IDP) ............................................. 288 1 9 2592 

Citizens’ Panel ................................................................................. 96 1 24 2304 

Interrupted Deliberation (ID) ............................................................ 192 1 6 1152 

Educational Materials ...................................................................... 1685 1 1 1685 

Knowledge and Attitudes Pretest Survey ........................................ 1685 1 30/60 843 

Knowledge and Attitudes Posttest Survey ...................................... 1685 1 20/60 562 

Deliberative Experience Survey ...................................................... 1056 1 15/60 264 

Total .......................................................................................... 8852 N/A N/A 11647 

EXHIBIT 2—ESTIMATED ANNUALIZED COST BURDEN

Form name/Deliberative method Number of 

respondents 

Total burden 

hours 

Average hourly 

wage rate Total cost burden 

Recruitment and Consent Materials ................................................ 1685 421 $21.35 $8,988 

Short Citizens’ Deliberation (SCD) .................................................. 192 384 21.35 8,198 

Online Deliberative Polling® (ODP) ................................................ 288 1440 21.35 30,744 

In-Person Deliberative Polling® (IDP) ............................................. 288 2592 21.35 55,339 

Citizens’ Panel ................................................................................. 96 2304 21.35 49,190 

Interrupted Deliberation (ID) ............................................................ 192 1152 21.35 24,595 


emcdonald on DSK5VPTVN1PROD with NOTICES
74788 Federal Register / Vol. 76, No. 231 / Thursday, December 1, 2011 / Notices 

EXHIBIT 2—ESTIMATED ANNUALIZED COST BURDEN—Continued 

Form name/Deliberative method Number of 

respondents 

Total burden 

hours 

Average hourly 

wage rate Total cost burden 

Educational Materials ...................................................................... 1685 1685 21.35 35,975 

Knowledge and Attitudes Pretest Survey ........................................ 1685 843 $21.35 $17,998 

Knowledge and Attitudes Post-test Survey ..................................... 1685 562 21.35 11,999 

Deliberative Experience Survey ...................................................... 1056 264 21.35 5,636 

Total .......................................................................................... 8852 N/A N/A 248,662 

* Based upon the mean of the wages for 00–000 All Occupations ($21.35), May 2010 National Occupational Employment and Wage Estimates. 

United States, ‘‘U.S. Department of Labor, Bureau of Labor Statistics.’’ http://www.bls.gov/oes/current/oes_nat.htm#00–0000. 

Estimated Annual Costs to the Federal 

Government 

Exhibit 3 below breaks down the costs 

related to this study. These are the costs 

associated with the portion of the 

contract awarded to AIR to conduct the 

experiment. Since the implementation 

and evaluation periods will span 24 

months, the costs have been annualized 

by taking the total cost and dividing by 

2. 

EXHIBIT 3—ESTIMATED ANNUALIZED COST TO THE FEDERAL GOVERNMENT

Cost component Total cost Annualized cost 

Project Management ........................................................................................................................................ $60,106 $30,053 

Technical Expert Panel .................................................................................................................................... 117,793 58,896 

Technology Tools ............................................................................................................................................ 177,580 88,790 

Develop Educational Materials ........................................................................................................................ 368,624 184,312 

Evaluation Plan ................................................................................................................................................ 214,566 107,283 

Implement Methods ......................................................................................................................................... 1,624,169 812,085 

Conceptual Framework .................................................................................................................................... 50,195 25,098 

Data Processing and Analysis ......................................................................................................................... 566,846 283,423 

Reporting ......................................................................................................................................................... 135,693 67,847 

Overhead ......................................................................................................................................................... 1,281,340 640,670 

Total .......................................................................................................................................................... 4,596,914 2,298,457 

Request for Comments 

In accordance with the Paperwork 

Reduction Act, comments on AHRQ’s 

information collection are requested 

with regard to any of the following: (a) 

Whether the proposed collection of 

information is necessary for the proper 

performance of AHRQ healthcare 

research and healthcare information 

dissemination functions, including 

whether the information will have 

practical utility; (b) the accuracy of 

AHRQ’s estimate of burden (including 

hours and costs) of the proposed 

collection(s) of information; (c) ways to 

enhance the quality, utility, and clarity 

of the information to be collected; and 

(d) ways to minimize the burden of the 

collection of information upon the 

respondents, including the use of 

automated collection techniques or 

other forms of information technology. 

Comments submitted in response to 

this notice will be summarized and 

included in the Agency’s subsequent 

request for OMB approval of the 

proposed information collection. All 

comments will become a matter of 

public record. 

Dated: November 16, 2011. 

Carolyn Clancy, 

Director. 

[FR Doc. 2011–30795 Filed 11–30–11; 8:45 am] 

BILLING CODE 4160–90–M 

DEPARTMENT OF HEALTH AND 

HUMAN SERVICES 

Agency for Healthcare Research and 

Quality 

Patient Safety Organizations: 

Voluntary Relinquishment From 

HealthWatch, Inc. 

AGENCY: Agency for Healthcare Research 

and Quality (AHRQ), HHS. 

ACTION: Notice of delisting. 

SUMMARY: AHRQ has accepted a 

notification of voluntary relinquishment 

from HealthWatch, Inc. of its status as 

a Patient Safety Organization (PSO). The 

Patient Safety and Quality Improvement 

Act of 2005 (Patient Safety Act), Public 

Law 109–41, 42 U.S.C. 299b–21—b–26, 

provides for the formation of PSOs, 

which collect, aggregate, and analyze 

confidential information regarding the 

quality and safety of health care 

delivery. The Patient Safety and Quality 

Improvement Final Rule (Patient Safety 

Rule), 42 CFR part 3, authorizes AHRQ, 

on behalf of the Secretary of HHS, to list 

as a PSO an entity that attests that it 

meets the statutory and regulatory 

requirements for listing. A PSO can be 

‘‘delisted’’ by the Secretary if it is found 

to no longer meet the requirements of 

the Patient Safety Act and patient Safety 

Rule, including when a PSO chooses to 

voluntarily relinquish its status as a 

PSO for any reason. 

DATES: The directories for both listed 

and delisted PSOs are ongoing and 

reviewed weekly by AHRQ. The 

delisting was effective at 12 Midnight 

ET (2400) on November 1, 2011. 

ADDRESSES: Both directories can be 

accessed electronically at the following 

HHS Web site: http:// 

www.pso.AHRQ.gov/index.html. 

FOR FURTHER INFORMATION CONTACT: 

Susan Grinder, Center for Quality 

Improvement and Patient Safety, AHRQ, 

540 Gaither Road, Rockville, MD 20850; 

Telephone (toll free): (866) 403–3697; 

Telephone (local): (301) 427–1111; TTY 

(toll free): (866) 438–7231; TTY (local): 

(301) 427–1130; Email: 

pso@AHRQ.hhs.gov. 

SUPPLEMENTARY INFORMATION: 


emcdonald on DSK5VPTVN1PROD with NOTICES