**[THE GALLUP ORGANIZATION\
]{.smallcaps}**

General Services Administration (GSA)

*Office of Government-wide Policy*

*(OGP)*

*Advisory Committee Engagement Survey (ACES)*

*Best Practices Report\
\
*

March 2005

Submitted by:

THE GALLUP ORGANIZATION

901 F Street N.W.

Washington, D.C. 20004

**Copyright Standards**

> This document contains proprietary research, copyrighted materials,
> and literary property of The Gallup Organization. It is for the
> guidance of your company only and is not to be copied, quoted,
> published, or divulged to others outside of your organization.
> Gallup^®^ is a trademark of The Gallup Organization, Princeton, NJ.
> All other trademarks are the property of their respective owners.
>
> This document is of great value to both your organization and The
> Gallup Organization. Accordingly, international and domestic laws and
> penalties guaranteeing patent, copyright, trademark, and trade secret
> protection protect the ideas, concepts, and recommendations related
> within this document.
>
> No changes may be made to this document without the express written
> permission of The Gallup Organization.

**Contents**

# **Introduction and Overview**  {#introduction-and-overview .TOC-Heading-1}

## Introduction 2 {#introduction-2 .TOC-Heading-2}

## Methodology 2 {#methodology-2 .TOC-Heading-2}

## Overall Findings 3 {#overall-findings-3 .TOC-Heading-2}

###  {#section .TOC-Heading-3}

##   {#section-1 .TOC-Heading-2}

# **Detailed Findings** {#detailed-findings .TOC-Heading-1}

## Composition of the Survey Respondents 8 {#composition-of-the-survey-respondents-8 .TOC-Heading-2}

## Most Important Ingredients for Committee Success 9 {#most-important-ingredients-for-committee-success-9 .TOC-Heading-2}

## Comments on Items with the Highest Scores 12  {#comments-on-items-with-the-highest-scores-12 .TOC-Heading-2}

## Comments on Items that Increased Significantly 17  {#comments-on-items-that-increased-significantly-17 .TOC-Heading-2}

## Comments on Low-Scoring Items 21 {#comments-on-low-scoring-items-21 .TOC-Heading-2}

## General Best Practices and Advice 23  {#general-best-practices-and-advice-23 .TOC-Heading-2}

###  {#section-2 .TOC-Heading-3}

###  {#section-3 .TOC-Heading-3}

# **Conclusions**  {#conclusions .TOC-Heading-1}

## Conclusions 25

Appendix A: Composition of the Survey Respondents (Table A)

Appendix B: Best Practice Questionnaire

Appendix C: Advisory Committee Engagement Survey (ACES) Questionnaire

**Introduction**

The General Services Administration's (GSA) Office of Government-wide
Policy (OGP) contracted with the Gallup Organization to conduct a survey
assessing the level of engagement of members of Federal advisory
committees. The goal of this study, called the GSA Advisory Committee
Engagement Survey (ACES), was to better understand the opinions of
committee members about their committee's performance.

As a follow-up to the ACES survey, Gallup conducted telephone interviews
with members of Federal advisory committees that were identified as
being exceptional on the basis of their ACES scores. These interviews
investigated what "best practice" committees do that differentiates them
from other committees.

The purpose of this report is to showcase the ideas and methods for
success used by members of outstanding committees, and to offer an
analysis of the information gathered with the intention of developing a
model for committee success.

**Methodology**

In September and October 2004, Gallup conducted 51 telephone interviews
with members of advisory committees that received the highest overall
scores on the ACES 2004 survey. The survey participants represented 11
committees.

Gallup Executive Interviewers conducted the interviews. The interviewers
were selected from a team that has extensive experience with indepth
interviewing.

**Overall Findings**

# Most Important Ingredients for Committee Success

Interview participants credit the following with the success of their
advisory committees:

-   Good selection of members

-   Effective chairperson

-   Interface with stakeholders

-   Positive relationship between the agency and the committee

-   Resourceful liaison/administrative staff

# Comments on Items With the Highest Scores

Interview participants were asked about two items for which their
committee received the highest scores.

## Our Committee\'s Operating Procedures and Guidelines are Fair

Interview participants offered the following guidelines for success on
this attribute:

-   Maintain focus on the committee's mission

-   Consider the personal dynamics between members as well as expertise
    when organizing task forces

-   Promote a "participative attitude" among the committee members

-   Provide sufficient advance warning of committee meetings and/or
    allow members to select dates and times that work for all schedules.

-   Develop guidelines for site visits/stakeholder meetings

## Our Committee Meetings are Well Run 

The committee chairperson is primarily responsible for ensuring that
meetings are well run. Committee chairs who run successful meetings
commonly streamline them in the following ways:

-   Plan a meaningful agenda and carefully organize the meeting's
    logistics

-   Stick to the agenda

```{=html}
<!-- -->
```
-   Brief members about relevant issues and protocol prior to meetings
    when guests will be attending

-   Brief any guests about the specific mission of the committee, the
    most effective way to approach the topic that they will be
    presenting on, and how best to deliver what information they have

```{=html}
<!-- -->
```
-   Turn over leadership of the meeting to the co-chair if it is
    necessary to leave a meeting

## Our Committee Fairly Considers Both Majority and Minority Opinions

Interview participants from committees who received high scores on this
item indicated that it hinges on the chairperson's moderation. Chairs
who are successful encourage discussion of all viewpoints and often have
open-door policies with committee members.

Well-structured meetings also contribute to the open exchange of ideas.

## Our Committee\'s Recommendations or Other Contributions are Responding to the Agency\'s Needs

Working closely with the agency helps the committee focus on the
critical issues. To keep the committee on track, it is necessary for the
agency to keep them informed about what their direction is in terms of
implementing, assessing, or even seeking support for various programs.
Committee members would welcome time with the agency's decision-makers
to discuss the committee's work relative to the agency's needs.

## Our Committee Communicates Effectively With Senior Managers and External Stakeholders

Many of the interview participants stressed the importance of
interaction with stakeholders as a way of maintaining focus and
energizing the committee. Some committees regularly conduct site visits.
All participants agreed that inviting guests present at committee
meetings is invaluable.

## Our Committee's Recommendations or Contributions Have a Positive Impact on the Public or External Stakeholders

Success on this item depends upon maintaining close ties with the public
and external stakeholders in terms of their opinions and input and
experiences surrounding the committee's mission.

## The Results of Our Committee\'s Work are Available to Others as Needed 

Committee members indicated that their reports are public record once
they have been submitted. Some said that their reports are posted on a
Web site. Interview participants suggested that strong relationships
between the advisory committee and professional societies help to
disseminate the committee's findings and recommendations.

# 

## Our Committee Has Access to Senior Managers and Agency Technical Experts When Needed

Committees who received high scores on this item have agency
representatives and senior analysts attend meetings so the committee has
immediate access to them.

# 

# Our Committee is a Positive Influence Within its Area of Expertise 

# 

An opinion or recommendation endorsed by a collection of experts in a
particular field has credibility. The impact of a committee's work can
extend beyond agency plans and have broader influence on the field at
large.

# Comments on Items That Increased Significantly 

Interview participants were asked to identify what their committee did
to impact items whose scores increased significantly since the last ACES
survey administration.

## Our Committee Members are Well-Prepared for Meetings

Distribution of an agenda and materials in advance of the meetings helps
committee members to prepare for meetings. Interview participants agreed
on the importance of not wasting time in meetings with inefficient
organization. Some interview participants suggested that preparation for
meetings has less to do with the actual mechanics of the meetings than
the accountability of the committee members to the larger community.

## Our Committee Meetings are Well-Run

The chairperson of the committee is responsible for arbitrating the
meeting. One important key to efficiently-run meetings is to make sure
that members clearly understand what falls within the scope of the
committee.

Interview participants also suggest that keeping the minutes of the
meeting helps to keep them on track.

## Our Committee Communicates Effectively With Senior Managers as Well as With External Stakeholders, if Needed. 

In order to facilitate effective communication with senior managers or
external stakeholders, senior managers and/or external stakeholders
should be invited to attend committee meetings and interact directly
with the members. To make the best use of time, the committee should be
briefed prior to a visit from senior managers and/or external
stakeholders. Similarly, presenters should be briefed about the specific
goals of the committee so that they can tailor their presentation
accordingly.

## Thanks to Our Committee, the Agency is More Effective

Committee members know how their recommendations and other contributions
are used and they receive direct feedback. Interview participants
indicated that their committee improved on this item simply by being an
honest representation of their constituents.

## Our Committee's Work Helps to Build Trust in Government

Maintaining close ties with stakeholders promotes their sense that they
are being "championed," that the government is working on their behalf.
Citizens are also encouraged to discover that all Federal committee
sessions are accessible to interested parties.

# Comments on Low-Scoring Items

Interview participants were asked for their comments regarding items
that received relatively low scores government-wide.

## Feedback on Recommendations or Other Contributions

While this item received a relatively low score, most interview
participants indicated that they do receive feedback from the agency
about their recommendations or other contributions.

Most members said that they receive follow-up, often in the form of a
written response, from the agency explaining what steps they have taken
as a result of recommendations, although feedback is not always provided
so formally.

Some committee members said that if the agency does not respond to a
recommendation, the committee's liaison person explains why no action
has been taken. The liaison person also reintroduces recommendations
that had been overlooked previously to determine if they are still
viable.

A few interview participants suggested that they would like more detail
in the feedback they receive.

## Effective Use of Your Committee's Recommendations or Other Contributions

A few of the interview participants mentioned that this item could be
improved by having the committees direct their recommendations toward a
specific outcome. They should identify problems and the necessary
outcomes and then allow the agency to determine how to address the
problems within their resources.

**General Best Practices and Advice**

Interview participants were asked to provide any further best practices
employed by their committee and hypothetical advice they would offer to
a newly forming committee.

## Other Best Practices

Other than those already discussed, the most common best practice
mentioned by committee members is the construction of a committee Web
site. Web sites facilitate communication among the members and keep the
public informed of the committee's work.

## Advice to a Newly Forming Committee

Interview participants offered the following advice to newly forming
committees:

-   Define your goal/Stick to your charter

-   Maintain open communications with stakeholders

-   Maintain open communications with the agency

-   Select a good leader

-   Select good members

-   Orient new members

-   Be fiscally responsible

-   Meet as frequently as possible

**Detailed Findings**

This section of the report details the composition of the survey
respondents and then organizes the survey results into sections that
match the structure of the survey instrument.

# Composition of the Survey Respondents[^1]

Fifty-one advisory committee members representing 11 distinct committees
were interviewed.

Six of the committees represented are non-scientific program advisory
committees, four are scientific technical program advisory committees,
and one has a different committee function.

Congress created eight of the committees, two were established by agency
authority and one was authorized by law.

Only one of the represented advisory committees, the National Institute
for Literacy Advisory Committee, is a Presidential appointment.

# Most Important Ingredients for Committee Success

## Most Important Ingredients for the Success of Committees Over the Last Year

Interview participants credit the following with the success of their
advisory committees:

-   Good selection of members

The members of the committee are vital to its success. Interview
participants said that when members are being selected, it is important
to consider the mix of knowledge and talents, the "team dynamic," and
their dedication to the committee's focus.

> "Everyone \[on the committee\] is extremely conscientious about what
> we\'re doing, and very serious and dedicated." (Advisory Committee on
> the Readjustment of Veterans)
>
> "\[I have been on other committees that people were there for\]
> personal, professional reasons, or there to impress other people or
> something, they had another agenda, and those people delayed and
> disrupted the committee." (Advisory Committee on the Readjustment of
> Veterans)

Interview participants believe that it is important for people with
diverse backgrounds and a mix of expertise to be included.

> "\[The agency\] went out to the private sector with people that were
> experienced, having been there, done it, and looking for understanding
> what their processes are, and then comparing them to what the
> governmental processes were, and choosing the best approach. And I
> think the zeroing in on getting the panel from the area of the market
> that was in fact the subject matter of the questions they wanted
> answered I think was the biggest single factor. (Industry Advisory
> Panel)
>
> "It was really an eye opener for me that everyone\'s different walk of
> life fit together so perfectly to make a cohesive group and a strong
> group who had strengths in getting things done in a different way."
> (National Institute For Literacy Advisory Committee)
>
> "We all come from such diverse backgrounds and from across the country
> and the majority of us are there because of our activism if you will,
> so we have connections in our own communities, in our own states that
> support these programs within that agency." (Advisory Committee on the
> Readjustment of Veterans)

Ideally, a committee's success relies on the healthy turnover of its
members, blending long-term members who offer experience with new
members who offer fresh insight that is invaluable.

> "We have some long-term members in the committee ... and those people
> can give us a perspective on the history and the kinds of issues we
> might anticipate." (Advisory Committee on the Readjustment of
> Veterans)
>
> "Our chair has done an outstanding job in coaching new members of the
> committee, reminding them about what our charter says that we will do,
> and what our commitment to Congress is, and he also briefs them before
> we do site visits, explains to them why the teams were divided and
> made up as they were so that they know their particular role whenever
> we do a site visit." (Advisory Committee on the Readjustment of
> Veterans)

-   Effective chairperson

The chairperson of an advisory committee controls the direction of the
group. He or she is not only responsible for overseeing the actual
content of the committee's work but also facilitating the meetings and
organizing logistics. Interview participants agreed that this role plays
a large part in the effective functioning of a committee.

> "It\'s extremely important to have a chairperson that is organized,
> dedicated, knowledgeable and has the ability to manage and get along
> with a multidisciplinary group of people." (Advisory Committee on the
> Readjustment of Veterans)
>
> "\[The chair\] was very good at directing discussion and identifying
> problems ... and identifying what we would need to bring to the
> committee to get some resolution on controversial issues, whether we
> needed content experts to come and give a presentation. He was just
> very good at directing the discussions so that we came out with
> something." (Healthcare Infection Control Practices Advisory
> Committee)

-   Interface with stakeholders

While direct interaction with stakeholders is not applicable to all
committees, it was considered to be crucial for success by some
interview participants, particularly those in the Advisory Committee on
the Readjustment of Veterans.

> "We get out and meet with the population that we\'re supposed to be
> serving, i.e., \[STAKEHOLDERS\]. I think we get much more benefit in
> hearing what \[STAKEHOLDERS\] say than we do in getting reports from
> people who are in Washington. While those are helpful about a lot of
> administrative and legislative issues going on, it\'s getting face to
> face with the \[STAKEHOLDERS\] where I think we\'re able to determine,
> assess and convey their needs." (Advisory Committee on the
> Readjustment of Veterans)
>
> "We get to put a face on real people in terms of the clients that we
> ultimately service. ... It helps us to prioritize." (Advisory
> Committee on the Readjustment of Veterans)

-   Positive relationship between the agency and the committee

A strong relationship between the committee and the agency helps the
committee to succeed. Both parties must consider each other's direct and
indirect priorities and assist each other in reaching their goals.

> "There\'s a great deal of trust on both sides, first from the trust on
> the agency side that they can ask priority questions, and that the
> committee will broadly reflect the community and give the best
> scientific answers. And there\'s trust on the sides of us on the
> advisory committee that the agencies place very high value on the
> advice we give, and of course they generated that by showing that when
> we make recommendations, those have been followed through." (DOE/NSF
> Nuclear Science Advisory Committee)

-   Resourceful liaison/administrative staff

Several interview participants indicated that the committee's liaison
person plays a critical role. Specifically, the liaison person
streamlines interaction between the committee and the agency and works
directly with the committee's chair to organize any applicable resources
or schedule committee "guests" who can provide needed information.
Interview participants agreed that the advisory committees could not
perform efficiently without this preparation and administrative
coordination.

> "When we are at the crux of a decision or an issue, we really need the
> person in authority who has the best understanding of that situation
> to come and talk to us." (Advisory Committee on the Readjustment of
> Veterans)
>
> "The single most important ingredient is the key administrative
> staffer that they\'ve assigned to us. ... He is just a super
> facilitator." (Advisory Committee on the Readjustment of Veterans)

# Comments on Items With the Highest Scores

Interview participants were asked about two items for which their
committee received the highest scores.

## Our Committee\'s Operating Procedures and Guidelines are Fair

Interview participants offered the following guidelines for success on
this attribute:

-   Maintain focus on the committee's mission

> "We're not playing politics with the problems that we\'re trying to
> deal with." (Former POWs)

-   Consider the personal dynamics between members as well as expertise
    when organizing task forces

-   Promote a "participative attitude" in the committee

-   Provide sufficient advance warning of committee meetings and/or
    allow members to select dates and times that work for all schedules.

-   Develop guidelines for site visits/stakeholder meetings

## Our Committee Meetings are Well Run 

Interview participants agree that the committee chairperson is primarily
responsible for ensuring that meetings are well run.

Committee chairs who run successful meetings commonly streamline them in
the following ways:

-   Plan a meaningful agenda and carefully organize the meeting's
    logistics

> "Just by nature of our structure, we do not have the luxury of missing
> deadlines, and when you\'re in that position, you have to be operating
> efficiently in terms of the logistics of the meeting and also the
> quality of the people that we\'re bringing on to help address these
> questions." (Federal Insecticide, Fungicide, and Rodenticide Act
> Scientific Advisory Panel)
>
> "\[The committee chair\] always has his finger on the pulse of what\'s
> going on in terms of \[STAKEHOLDER\]. And he has been able to craft an
> agenda that is realistic, and so when we get there we have an agenda,
> we have a timeline, and we can then just focus on getting those issues
> addressed and accomplishing something as opposed to just allowing a
> free for all." (Advisory Committee on the Readjustment of Veterans)
>
> "We know what we\'re going to be talking about ahead of time, we have
> the opportunity to add to the agenda. They allocate enough time for
> everybody to do or say what they need to do at each meeting. We have
> the subcommittee meeting two weeks prior, so we\'re well aware of
> that. They announce the venues in enough time to get everyone there."
> (Houston/Galveston Navigation Safety Advisory Committee)

-   Stick to the agenda

> "\[The committee chair\] lets us discuss hot issues, but he always
> brings us back to the agenda." (Advisory Committee on the Readjustment
> of Veterans)
>
> "We are so serious about what we\'re doing, and so we walk in there
> with what we see as a mountain of work to be accomplished, and we
> never waste a second." (National Institute For Literacy Advisory
> Committee)
>
> "\[The chair's\] ability to focus the discussions, and also to keep
> everyone engaged." (Healthcare Infection Control Practices Advisory
> Committee**)**

-   Brief members about relevant issues and protocol prior to meetings
    when guests will be attending

-   Brief any guests about the specific mission of the committee, the
    most effective way to approach the topic that they will be
    presenting on, and how best to deliver what information they have

```{=html}
<!-- -->
```
-   Turn over leadership of the meeting to the co-chair if it is
    necessary to leave a meeting

## Our Committee Fairly Considers Both Majority and Minority Opinions

Interview participants from committees who received high scores on this
item indicated that it hinges on the chairperson's moderation. Chairs
who are successful encourage discussion of all viewpoints and often have
open-door policies with committee members.

> "The chairman is very fair, and she\'s always very intent on everyone
> being heard from. ... There is great interest in everyone being
> heard." (National Institute For Literacy Advisory Committee)
>
> "The chairs ... often times don\'t push for any kind of consensus.
> It\'s just taking the broad array of advice that they get, and using
> that to shape their work. So even people who don\'t feel like that
> they\'re on the same page with the rest of the committee members in
> terms of whatever is being recommended by the majority, feel like
> their voice is being heard, and their cautionary notes or their
> concerns are being heard and will receive some attention." (Healthcare
> Infection Control Practices Advisory Committee)
>
> "You can get your point in if you're persistent." (Former POWs)

Well-structured meetings also contribute to the open exchange of ideas.

> "We have very well defined procedures. ...The procedures themselves
> are designed to make sure that all the different viewpoints are
> brought forward in an orderly fashion." (National Advisory Committee
> on Institutional Quality and Integrity)

## Our Committee\'s Recommendations or Other Contributions are Responding to the Agency\'s Needs

Working closely with the agency helps the committee focus on the
critical issues.

> "If the charges are well written, then answering the charge \[responds
> to the agency's needs\] naturally. The committee works hard to stay on
> target and respond to the charge." (DOE/NSF Nuclear Science Advisory
> Committee)

Close interaction between the committee's members and people in the
agency also supports the agency\'s ability to prioritize. To keep the
committee on track, it is necessary for the agency to keep them informed
about what their direction is in terms of implementing, assessing, or
even seeking support for various programs. Committee members would
welcome time with the agency's decision-makers to discuss the
committee's work relative to the agency's needs.

> "It would be good ... if not only the Assistant Secretary but the
> Secretary or the Deputy were to say 'let me just remind you of what my
> objectives are in this, let me thank you for doing these following
> things, let\'s make sure that we\'re on the same wavelength' because
> our intent as an advisory body that makes recommendations is to make
> sure that we\'re actually helping push forward the work of the agency.
> And so having a very clear personal description of what those goals
> are from the most senior members of the agency would be helpful."
> (National Advisory Committee on Institutional Quality and Integrity)

## Our Committee Communicates Effectively with Senior Managers and External Stakeholders

Many of the interview participants stressed the importance of
interaction with stakeholders as a way of maintaining focus and
energizing the committee. Some committees regularly conduct site visits.
All participants agreed that inviting guests present at committee
meetings is invaluable.

> "They\'ve been very good at identifying the people within \[LOCATION\]
> that can contribute or can make decisions, and those are the people
> that we solicit for the various meetings and the various
> subcommittees." (Houston/Galveston Navigation Safety Advisory
> Committee)

## Our Committee's Recommendations or Contributions Have a Positive Impact on the Public or External Stakeholders

Success on this item depends upon maintaining close ties with the public
and external stakeholders in terms of their opinions and input and
experiences surrounding the committee's mission.

## The Results of our Committee\'s Work are Available to Others as Needed 

Committee members indicated that their reports are public record once
they have been submitted. Some said that their reports are posted on a
Web site. Interview participants suggested that strong relationships
between the advisory committee and professional societies help to
disseminate the committee's findings and recommendations.

# 

## Our Committee Has Access to Senior Managers and Agency Technical Experts When Needed

Committees who received high scores on this item have agency
representatives and senior analysts attend meetings so the committee has
immediate access to them.

> "The policy deciders, the policy decision makers have all also been
> quite receptive to input and interaction with the committee in
> appropriate venues." (National Advisory Committee on Institutional
> Quality and Integrity)

# Our Committee is a Positive Influence Within its Area of Expertise 

# 

An opinion or recommendation endorsed by a collection of experts in a
particular field has credibility. The impact of a committee's work can
extend beyond agency plans and have broader influence on the field at
large.

# *"\[NAME\] took the information that we provided him, infiltrated it into his presentations, and his answers to Congress and went forward with our recommendations and on many times he has come back and said it turns the corner, it helps him explain to the people in the governmental community why what he\'s asking for was the right thing because he got the consensus of and the backing of people in the industry that were experts in those fields and that it helped clarify many issues for him, and gave him the words used to help explain those issues to the proper panels and committees that he in turn has to report to." (Industry Advisory Panel)*

## 

## Our Committee is Made Up of the Right Mix of Individuals

Members of committees who received high scores on this item said that
their members were selected by the agency based on their expertise.
Other contributors are appointed.

## Our Committee\'s Recommendations Or Other Contributions Are Used Effectively

The key to this item is concentrating on the specific charter on which
the committee is established and maintaining strict focus on the issues
posed to the committee.

# 

# *"What we do is make recommendations that are clear and well-defined, and focus \[on\] ... aspects of the mission of the committee." (Advisory Committee on the Structural Safety of VA Facilities)* 

# Comments on Items That Increased Significantly

Interview participants were asked to identify what their committee did
to impact items whose scores increased significantly since the last ACES
survey administration.

## Our Committee Members are Well-Prepared for Meetings

Distribution of an agenda and materials in advance of the meetings helps
committee members to prepare for meetings. Interview participants agreed
on the importance of not wasting time in meetings with inefficient
organization.

> "Be prepared when you get to the committee in order to discuss
> whatever subject needs to be discussed." (Former POWs)
>
> "I think very honestly ... that typically these people are very, very
> involved in this particular field, and I think that the preparation
> that they\'ve put in reflects not only their professionalism as
> scientists, but also the fact that they want to show up prepared.
> They\'re going to be sitting around a table with an ad hoc group of
> their colleagues, people that they have known from their field, and I
> think they want to be prepared and ready to contribute." (Federal
> Insecticide, Fungicide, and Rodenticide Act Scientific Advisory Panel)
>
> "Prior to any site visit in particular, we receive packages of
> information from the staff that include our last annual report, so we
> can sort of keep in mind the kinds of questions you\'re asking and the
> issues that you\'re looking at." (Advisory Committee on the
> Readjustment of Veterans)

Some interview participants suggested that preparation for meetings has
less to do with the actual mechanics of the meetings than the
accountability of the committee members to the larger community.

> "It\'s the sense of responsibility in the individual members that
> because they know what they say is going to be listened to, they take
> it very seriously." (DOE/NSF Nuclear Science Advisory Committee)
>
> "I think that just a belief on the part of the members of the
> committee, to make the meetings as productive as possible, we need to
> be well prepared and ... be on top of the issues that are being
> addressed." (Advisory Committee on the Structural Safety of VA
> Facilities)

## Our Committee Meetings are Well-Run

The chairperson of the committee is responsible for arbitrating the
meeting. One important key to efficiently-run meetings is to make sure
that members clearly understand what falls within the scope of the
committee.

> "We have over the years past had instances where someone brings some
> of their personal politics into the committee dealings with a senior
> official, and they are reminded by the chair that that is maybe very
> important to you and it is valued, but it can\'t be a part of what we
> do in this committee." (Advisory Committee on the Readjustment of
> Veterans)

Interview participants also suggest that keeping the minutes of the
meeting helps to keep them on track.

## Our Committee Communicates Effectively With Senior Managers as Well as With External Stakeholders, if Needed. 

In order to facilitate effective communication with senior managers or
external stakeholders, senior managers and/or external stakeholders
should be invited to attend committee meetings and interact directly
with the members.

> "The committee feels we can ask and get straight answers from the
> agencies ... they don\'t keep agendas concealed from us, and so we
> feel we can get the information we need, and we believe the
> information. We don\'t need to spend time second guessing the material
> we get." (DOE/NSF Nuclear Science Advisory Committee)

To make the best use of time, the committee should be briefed prior to a
visit from senior managers and/or external stakeholders. Similarly,
presenters should be briefed about the specific goals of the committee
so that they can tailor their presentation accordingly.

## Thanks to Our Committee, the Agency is More Effective

> "We feel that our advice is followed, and that we are asked
> substantive questions, and those substantive questions really
> determine how well the \[AGENCY\] is able to carry out its mission."
> (DOE/NSF Nuclear Science Advisory Committee)

Committee members know how their recommendations and other contributions
are used and they receive direct feedback.

> "Our chair and the liaison person ... bring back to us things that
> have changed in the \[AGENCY\] as a result of recommendations that we
> made." (Advisory Committee on the Readjustment of Veterans)

Interview participants indicated that their committee improved on this
item simply by being an honest representation of their constituents.

> "We have a sense in terms of what are critical issues, and as we go
> out and talk to the various clients or stakeholders, some of these get
> confirmed or reaffirmed and others get kind of put in a further
> development category, and then when we do come back, always we are
> able to in our report to the agency in Congress, we say this is not
> something we\'re making up, this is something that we have direct
> input on, or not input, but we have direct feedback, so this is a
> reality as opposed to something that we perceive might be going on."
> (Advisory Committee on the Readjustment of Veterans)

## Our Committee's Work Helps to Build Trust in Government

Maintaining close ties with stakeholders promotes their sense that they
are being "championed," that the government is working on their behalf.

> "Our committee, through the expert sessions and through other
> opportunities that we had, we really made an effort to explain the
> process to the people so that people would understand it and
> understand why we were participating in it and what the charge to us
> was." (Healthcare Infection Control Practices Advisory Committee)
>
> "We always report back to \[the stakeholders\] based on our meeting
> and the information you provided, this is what we\'ve done, this is
> how the agency has responded, and this is what\'s going to happen. So
> they can see a direct relationship between their input and some
> result." (Advisory Committee on the Readjustment of Veterans)

Citizens are also encouraged to discover that all Federal committee
sessions are accessible to interested parties.

> "All the hearings are public, they\'re open, we are careful not to
> transact public business except in public settings so that anyone who
> wants to be a part can be a part. I think everyone who watches and is
> involved in any way can say the process is very fair, so I hope in
> that way it does build some confidence or trust in government."
> (National Advisory Committee on Institutional Quality and Integrity)

Committee participants also noted that the intentions of their
committees are forthright and that they approach their subjects with
openness and sincerity.

# *"All of us believe in the process that we\'re going through. In other words, that we have honest government and honest registrants and businesses and commercial enterprises who have different interests, different objectives sometimes, but at least we are approaching this as an open forum to try to come to answers that not only represent the best scientific interpretation on the issue but also take into account the social and economic factors of these issues as well." (Federal Insecticide, Fungicide, and Rodenticide Act Scientific Advisory Panel)*

# *"If we are doing our job right, then these entities from the independent or private sector will see that the government that they encounter in this instance is well functioning and approaches the judgment that they may make with respect to that entity in a fair and judicious fashion." (National Advisory Committee on Institutional Quality & Integrity)* 

# Comments on Low-Scoring Items

Interview participants were asked for their comments regarding two items
that received relatively low scores government-wide. These two items
are:

-   Feedback on recommendations or other contributions

-   Effective use of your committee's recommendations or other
    > contributions

## Feedback on Recommendations or Other Contributions

While this item received a relatively low score, most interview
participants indicated that they do receive feedback from the agency
about their recommendations or other contributions.

> "It\'s more a question of trying to protect us from a core dump of
> information than any attempt to not provide feedback." (DOE/NSF
> Nuclear Science Advisory Committee)

Most members said that they receive follow-up, often in the form of a
written response, from the agency explaining what steps they have taken
as a result of recommendations.

> "The secretary will directly send a report back to us with his
> comments on what is being done or what can\'t be done to what we said.
> It\'s a very direct dialog. We\'re not guessing." (Advisory Committee
> on the Readjustment of Veterans)

Feedback is often provided less formally. An interview participant from
the Houston/Galveston Navigation Safety Advisory Committee said that she
receives feedback "either at other industry functions, either by phone,
by email." A representative from the Advisory Committee on the
Structural Safety of VA Facilities said that the agency provides
feedback in the most basic and obvious way ... by incorporating or not
incorporating their recommendations in the code of the agency.

Some committee members said that if the agency does not respond to a
recommendation, the committee's liaison person explains why no action
has been taken. The liaison person also reintroduces recommendations
that had been overlooked previously to determine if they are still
viable.

A few interview participants suggested that they would like more detail
in the feedback they receive.

> "I\'m trying to balance what I think would be sufficient with the
> \[AGENCY\]'s ability to do it. ... At every meeting we get a list of,
> and we actually spend time going over, ... all the recommendations
> that we made at the previous meeting, along with a narrative
> underneath it that says this is how \[AGENCY\] has responded to it.
> And so in that sense, I know that \[AGENCY\] is taking the
> recommendation seriously because they\'re spending the time to put
> this whole thing together, and we actually spend time going over it in
> the meeting. But sometimes some of the responses that we get are
> fairly vague like '\[AGENCY\] agrees with this, and we are in process
> of putting programs in place to rectify this problem.' That\'s a vague
> hogwash answer, and while it may be true, I think it\'d be nice if we
> spent a little bit more time on that." (Advisory Committee on
> Cemeteries and Memorials)

## Effective Use of Your Committee's Recommendations or Other Contributions

A few of the interview participants mentioned that this item could be
improved by having the committees direct their recommendations toward a
specific outcome. They should identify problems and the necessary
outcomes and then allow the agency to determine how to address the
problems within their resources.

> "Most of the actions are extremely visible. We recommend that one
> should increase funding in a particular area compared to another area,
> and six months later we are given the break down of funding across
> areas and we can see did it happen or not. We recommend that if you
> want to do this, you need to close this facility. Well that facility
> is closed. And the other new facility is started. So most of the
> actions are quite visible." (DOE/NSF Nuclear Science Advisory
> Committee)
>
> "Our sense is that the agency does its best to follow all our
> recommendations under the other constraints they have dealing with
> federal regulations, federal law, federal budget " (DOE/NSF Nuclear
> Science Advisory Committee)

Interview participants did not have standardized measurements for
monitoring the use of their contributions.

**General Best Practices and Advice**

Interview participants were asked to provide any further best practices
employed by their committee and hypothetical advice they would offer to
a newly forming committee.

## Other Best Practices

The most common best practice mentioned by committee members is the
construction of a committee web site. Web sites facilitate communication
among the members and keep the public informed of the committee's work.

## Advice to a Newly Forming Committee

Interview participants offered the following advice to newly forming
committees:

-   Define your goal/Stick to your charter: A clear, agreed-upon
    framework will tighten the committee's focus.

> "Make sure that the committee is formed around a well-defined and
> worthwhile mission, and then for the committee to stay focused on that
> and believe in itself and its mission." (Advisory Committee on the
> Structural Safety of VA Facilities)

-   Maintain open communications with stakeholders: Involving
    stakeholders in committee work keeps members focused and boosts the
    trust of the stakeholders.

-   Maintain open communications with the agency: Make sure the
    expectations between the agencies and the committees are extremely
    well understood.

-   Select a good leader: The chairperson's leadership style sets the
    tone for the committee.

-   Select good members: Balance experience and expertise. Dedication to
    the committee's goals is the most important trait for a committee
    member.

> "Put people on the committee that have a vested interest, not
> political, in what is trying to be accomplished ... and good for the
> country itself." (Advisory Committee on the Readjustment of Veterans)

-   Orient new members: Have long-term members mentor newer members.

-   Be fiscally responsible: Plan very carefully for site visits and
    consider the cost. Doing so will allow you to make the most use of
    time with the least expense. Fight for your budget.

-   Meet at frequently as possible: Budgets often limit the number of
    meetings a committee can hold, and the number of ideal meetings
    varies according to the committee's mission, but interview
    participants generally agree that advisory committees should try to
    meet at least twice, preferably three times a year.

**Conclusions**

Many consistencies emerged in the interviews with the best practice
advisory committee members. The most noteworthy is the importance placed
on the communication. More specifically, member engagement and committee
success are perhaps most critically dependent on the consistency and
quality of communication between the committee members and three key
groups:

-   The committee chairperson and administrative coordinator

-   The sponsoring agency

-   External stakeholders

**The Committee Chairperson and Administrative Coordinator:** A
successful committee requires a chairperson who encourages active
participation and a cooperative atmosphere. As the moderator of
committee meetings, the chairperson controls the direction and dynamic
of the committee. He or she is responsible for keeping the committee
focused on the specific goals it was created to achieve. To do so, he or
she must maintain continual communication with panel members, including
updates and feedback throughout the year.

An administrative coordinator or agency liaison ensures that the
meetings run smoothly and efficiently from a more logistical standpoint.
Without well-run meetings, a committee formed of the top experts in a
field with the noblest of intentions cannot function effectively. To
facilitate the meetings, a resourceful administrative coordinator must
work directly with committee members to schedule the meetings and
distribute agendas set by the chairperson along with any other relevant
materials. The coordinator also schedules meetings, guests, and site
visits, and can assist in communication between committee members.

The Sponsoring Agency: Maintaining a positive relationship with the
agency is crucial to achieving the committee's mission. This
relationship is most effectively forged through a sharing of
perspectives regarding critical issues, which can be accomplished by
having agency or department representatives attend committee meetings
and engage in discussions about the agency's needs relative to the
committee's work.

External Stakeholders: Maintaining open communication with stakeholders
helps advisory committees to focus on their mission. Committee members
are invigorated by interactions with their "clients," whether on site
visits or presentations during committee meetings. Soliciting
stakeholder feedback is also an extremely effective means of promoting
trust in government by fostering a belief that the government is working
on their behalf.

# Appendix A

+---+--------------------------------------+-----+------------+--------+
| * |                                      |     |            |        |
| * |                                      |     |            |        |
| T |                                      |     |            |        |
| a |                                      |     |            |        |
| b |                                      |     |            |        |
| l |                                      |     |            |        |
| e |                                      |     |            |        |
| A |                                      |     |            |        |
| * |                                      |     |            |        |
| * |                                      |     |            |        |
|   |                                      |     |            |        |
| # |                                      |     |            |        |
| # |                                      |     |            |        |
| # |                                      |     |            |        |
| # |                                      |     |            |        |
| # |                                      |     |            |        |
|   |                                      |     |            |        |
| C |                                      |     |            |        |
| o |                                      |     |            |        |
| m |                                      |     |            |        |
| m |                                      |     |            |        |
| i |                                      |     |            |        |
| t |                                      |     |            |        |
| t |                                      |     |            |        |
| e |                                      |     |            |        |
| e |                                      |     |            |        |
| s |                                      |     |            |        |
|   |                                      |     |            |        |
| I |                                      |     |            |        |
| n |                                      |     |            |        |
| c |                                      |     |            |        |
| l |                                      |     |            |        |
| u |                                      |     |            |        |
| d |                                      |     |            |        |
| e |                                      |     |            |        |
| d |                                      |     |            |        |
|   |                                      |     |            |        |
| i |                                      |     |            |        |
| n |                                      |     |            |        |
|   |                                      |     |            |        |
| t |                                      |     |            |        |
| h |                                      |     |            |        |
| e |                                      |     |            |        |
|   |                                      |     |            |        |
| B |                                      |     |            |        |
| e |                                      |     |            |        |
| s |                                      |     |            |        |
| t |                                      |     |            |        |
|   |                                      |     |            |        |
| P |                                      |     |            |        |
| r |                                      |     |            |        |
| a |                                      |     |            |        |
| c |                                      |     |            |        |
| t |                                      |     |            |        |
| i |                                      |     |            |        |
| c |                                      |     |            |        |
| e |                                      |     |            |        |
|   |                                      |     |            |        |
| A |                                      |     |            |        |
| n |                                      |     |            |        |
| a |                                      |     |            |        |
| l |                                      |     |            |        |
| y |                                      |     |            |        |
| s |                                      |     |            |        |
| i |                                      |     |            |        |
| s |                                      |     |            |        |
+---+--------------------------------------+-----+------------+--------+
| # | ##### Committee Name                 | **A | *          | ####   |
| # |                                      | CES | *Committee | Commit |
| # |                                      | S   | Type**     | tee Cr |
| # |                                      | cor |            | eation |
|   |                                      | e** |            |        |
|   |                                      |     |            |        |
| # |                                      |     |            |        |
| # |                                      |     |            |        |
| # |                                      |     |            |        |
| # |                                      |     |            |        |
|   |                                      |     |            |        |
| A |                                      |     |            |        |
| g |                                      |     |            |        |
| e |                                      |     |            |        |
| n |                                      |     |            |        |
| c |                                      |     |            |        |
| y |                                      |     |            |        |
+---+--------------------------------------+-----+------------+--------+
| V | Advisory Committee on Cemeteries and | 4   | Non        | Sta    |
| A | Memorials                            | .78 | Scientific | tutory |
|   |                                      |     | Program    |        |
+---+--------------------------------------+-----+------------+--------+
| H | Healthcare Infection Control         | 4   | Scientific | Auth   |
| H | Practices Advisory Committee         | .78 | Technical  | orized |
| S |                                      |     | Program    | by Law |
+---+--------------------------------------+-----+------------+--------+
| D | Industry Advisory Panel              | 4   | Committee  | Agency |
| O |                                      | .77 | Functi     | Aut    |
| S |                                      |     | on\--Other | hority |
+---+--------------------------------------+-----+------------+--------+
| V | Advisory Committee on the            | 4   | Non        | Sta    |
| A | Readjustment of Veterans             | .73 | Scientific | tutory |
|   |                                      |     | Program    |        |
+---+--------------------------------------+-----+------------+--------+
| V | Advisory Committee on Structural     | 4   | Scientific | Sta    |
| A | Safety of Department of Veterans     | .66 | Technical  | tutory |
|   | Affairs Facilities                   |     | Program    |        |
+---+--------------------------------------+-----+------------+--------+
| V | Advisory Committee on Former         | 4   | Non        | Sta    |
| A | Prisoners of War                     | .63 | Scientific | tutory |
|   |                                      |     | Program    |        |
+---+--------------------------------------+-----+------------+--------+
| E | Federal Insecticide, Fungicide, and  | 4   | Scientific | Sta    |
| P | Rodenticide Act Scientific Advisory  | .58 | Technical  | tutory |
| A | Panel                                |     | Program    |        |
+---+--------------------------------------+-----+------------+--------+
| E | National Advisory Committee on       | 4   | Non        | Sta    |
| D | Institutional Quality and Integrity  | .57 | Scientific | tutory |
|   |                                      |     | Program    |        |
+---+--------------------------------------+-----+------------+--------+
| E | National Institute for Literacy      | 4   | Non        | Sta    |
| D | Advisory Committee                   | .55 | Scientific | tutory |
|   |                                      |     | Program    |        |
+---+--------------------------------------+-----+------------+--------+
| D | DOE/NSF Nuclear Science Advisory     | 4   | Scientific | Agency |
| O | Committee                            | .54 | Technical  | Aut    |
| E |                                      |     | Program    | hority |
+---+--------------------------------------+-----+------------+--------+
| D | Houston-Galveston Navigation Safety  | 4   | Non        | Sta    |
| H | Advisory Committee                   | .54 | Scientific | tutory |
| S |                                      |     | Program    |        |
+---+--------------------------------------+-----+------------+--------+

# Appendix B

PAPER - OUTBOUND

**[FINAL DRAFT - SEPTEMBER 20, 2004]{.underline}**

PROJECT REGISTRATION #139617

**GSA** APPROVED BY CLIENT

City Center:

Best Practice Qualitative APPROVED BY PROJECT MANAGER

Montemarano/Sorensen/Bechtolt

[September, 2004 n=]{.underline}

Hello, my name is \_\_\_\_\_ and I'm calling from The Gallup
Organization for the General Services Administration's Office of
Government-wide Policy. We're following up on a survey that we conducted
last month, called the GSA Advisory Committee Engagement Survey (ACES).

The results of the survey indicate that YOUR committee, **[(COMMITTEE
NAME from call record card)]{.underline}** is highly successful. Your
agency CMO is extremely pleased with your results and has recommended
that we contact you. We're interested in talking with you specifically
about what you do to achieve your mission. We are gathering information
from the "best practice" committees so that we can share your successes
with other committees and agencies to help them increase their
effectiveness.

S1. We would like to record this conversation. We will use your input in
the analysis we do. We will NOT, however, identify you or any of your
comments. With the understanding that Gallup will keep your comments
confidential, may we have permission to record our conversation?

1 Yes - **(Continue)**

2 No - **(Thank and Terminate)**

# SECTION I

1\. First of all, as you think about the success your committee has
enjoyed over the last year, what do you think have been the most
important ingredients to your success? **[(INTERVIEWER NOTE: Try to get
three open ended responses)]{.underline}**

2\. Why do you think this/these ingredient(s) was/were so important to
your success?

3\. What specific steps did your committee take?

# SECTION II

**[(READ:)]{.underline}** While your committee achieved extremely high
scores throughout the entire survey, I want to ask you specifically
about two items where your group scored the highest -- we would like to
find out how your committee operates in these areas and what leads to
your success.

4\. First, what does your committee do specifically to impact **[(first
question number and wording from call record card)]{.underline}**?

5\. What does your committee do specifically to impact **[(second
question number and wording from call record card)]{.underline}**?

# SECTION III

**[(READ:)]{.underline}** In more general terms, we have seen some
significant increases in the data during this survey iteration and would
like to find out what you are doing as a "best practice" committee to
impact these items. What does your committee do specifically with
regards to the following questions:

6\. **[(If not already mentioned in #4 or #5, read:)]{.underline}** Our
committee members are well-prepared for meetings.

8\. **[(If not already mentioned in #4 or #5, read:)]{.underline}** Our
committee meetings are well-run.

13\. **[(If not already mentioned in #4 or #5, read:)]{.underline}** Our
committee communicates effectively with senior managers and, if needed,
with external stakeholders.

20\. **[(If not already mentioned in #4 or #5, read:)]{.underline}**
Thanks to our committee, the agency is more effective.

21\. **[(If not already mentioned in #4 or #5, read:)]{.underline}** Our
committee's work helps to build trust in government.

# SECTION IV

**[(READ:)]{.underline}** There are two items on the survey that have
received relatively low scores government-wide. Making improvements in
these areas are of particular interest to your agency and we'd like to
gather your feedback on them.

22\. First, do you feel that your committee receives sufficient feedback
from the agency on your recommendations or other contributions?

23\. \[**[(If Yes in #22, ask:)]{.underline}** Why?/**[(If No in #22,
ask:)]{.underline}** Why not?

24\. What do you feel your agency and/or committee can do to improve in
this area?

25\. How does your committee receive feedback on your recommendations
and other contributions?

26\. How would you like to receive feedback on recommendations or other
contributions?

27\. Does your committee have any "Best Practices" to share regarding
getting/disseminating feedback back to the committee?

28\. Next, do you feel that your committee's recommendations or other
contributions are used effectively?

29\. \[**[(If Yes in #28, ask:)]{.underline}** Why?/**[(If No in #28,
ask:)]{.underline}** Why not?

> 30\. How do you know? Do you have any concrete measurements in place?
>
> 31\. What do you feel your agency and/or committee can do to improve
> in this area?

# SECTION V

> 32\. And finally, thinking about the last year, are there any other
> "best practices" that your committee engaged in that you would like to
> share?

33\. **[(If Yes in #32, ask:)]{.underline}** What were they? Please try
to be specific.

34\. Finally, if you were to offer advice to a newly forming committee
on how to be successful, what advice would you offer?

35\. Why would you offer this advice?

**[(READ:)]{.underline}** Thank you for your time. We appreciate your
input and are looking forward to sharing your insights and expertise
during the GSA training session in October.

Thank you again for your time and input.

jlw\\2004\\GSA\\GSA Best Practices Qualitative 0409

# Appendix C

WEB

**App_ID#: 2719860**

**URL: <https://gx.gallup.com/gsaaces.gx>**

  -----------------------------------------------------------------------
  **[FIELD FINAL -- JUNE 1, 2004]{.underline}**

  -----------------------------------------------------------------------

+----------------------------------+-----------------------------------+
|                                  | The Gallup Organization, Inc.     |
+----------------------------------+-----------------------------------+
| Project Registration #139617     |                                   |
+----------------------------------+-----------------------------------+
| # GSA                            | \_\_\_\_\_ APPROVED BY CLIENT     |
+----------------------------------+-----------------------------------+
| City Center: Government          |                                   |
+----------------------------------+-----------------------------------+
| Advisory Committee Engagement    | \_\_\_\_\_ APPROVED BY PROJECT    |
| Study                            | MANAGER                           |
+----------------------------------+-----------------------------------+
| Larsen/Montemarano/Bechtolt      |                                   |
+----------------------------------+-----------------------------------+
| Kate Armstrong, Programmer       | [X]{.underline} SURVEY DESIGN:    |
|                                  | Deb Brown                         |
+----------------------------------+-----------------------------------+
| May, 2004                        | n=                                |
+----------------------------------+-----------------------------------+

***(PROGRAMMER NOTE: Privacy Policy hyperlinks to standard Gallup
privacy policy on the www site. <galluppoll@gallup.com> spawns e-mail
page. The following is to appear at the bottom of each screen:)***

If you need assistance in completing this survey, please contact Gallup
Client Support by sending an e-mail to <galluppoll@gallup.com> or by
calling 1-888-297-8999, from 8:00 a.m. to 8:00 p.m. Eastern Time, Monday
through Thursday, or 8:00 a.m. to 6:00 p.m. Eastern Time, on Fridays.

Do not print, store, or copy this page.

Copyright © 2004 The Gallup Organization, Princeton, NJ. All rights
reserved. [Privacy Policy]{.underline}

***[(PROGRAMMER NOTE: BANNER)]{.underline}***

Advisory Committee Engagement Survey (ACES)

Welcome to the Advisory Committee Engagement Survey (ACES), sponsored by
the Interagency Committee on Federal Advisory Committee Management. This
survey will ask your opinions about the performance of your Federal
Advisory Committee. Your responses will be completely confidential and
will only be used in statistical summaries.

Please enter your Access Code and click the \"Begin Survey\" button to
continue.

**[(PROGRAMMER NOTE: If incorrect Access Code, display the following
error message:)]{.underline}**

The Access Code you have entered is invalid. Please carefully re-enter
your Access Code.

***[(PROGRAMMER NOTE: If no Access Code entered, display the following
error message:)]{.underline}***

You must enter your Access Code to continue.

**[(PROGRAMMER NOTE: If completed survey found in database for this
Access Code, display the following error message:)]{.underline}**

A survey has already been completed with this Access Code. Thank you for
your participation.

If you feel you have received this message in error, please contact
<galluppoll@gallup.com> for assistance.

Please indicate the extent to which you agree or disagree with each
statement.

1\. Overall, I am satisfied with the work of this committee.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

2\. If given the opportunity, I would choose to work with this committee
again.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

3\. The mission and goals of this committee are clearly defined.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

4\. Our committee is made up of the right mix of individuals.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

5\. Our committee's operating procedures and guidelines are fair.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

6\. Our committee's members are well prepared for meetings.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

7\. Our committee's staff is well prepared for meetings.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

8\. Our committee meetings are well run.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

9\. Our committee has access to senior managers and agency technical
experts when needed.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

10\. Our committee has access to adequate resources.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

Please indicate the extent to which you agree or disagree with each
statement.

11\. Our committee meets the right amount to accomplish its work.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

12\. Our committee sets reasonable timelines for producing outputs.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

13\. Our committee communicates effectively with senior managers and, if
needed, with external stakeholders.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

14\. Our committee's recommendations or other contributions are
responding to the agency's needs.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

15\. Our committee receives sufficient feedback from the agency on our
recommendations or other contributions.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

16\. Our committee fairly considers both majority and minority opinions.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

17\. Our committee's recommendations or other contributions are used
effectively.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

18\. Our committee's recommendations or other contributions have a
positive impact on the public and/or external stakeholders.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

19\. The results of our committee's work are available to others as
needed.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

20\. Thanks to our committee, the agency is more effective.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

21\. Our committee's work helps to build trust in government.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

22\. Our committee is a positive influence within its area of expertise.

1 Strongly disagree

2

3

4

5 Strongly agree

6 Don\'t know/Does not apply

23\. How long have you served on this committee?

1 Less than 6 months

2 6 months to 1 year

3 More than 1 year, but less than 3 years

4 3 years or more

24\. How many meetings did you attend in the past 12 months?

0 None

1 1-2

2 3-5

3 6 or more

25\. Which of the following best describes your relationship to the
advisory committee? Please select one.

1 Member of committee

2 Former member of committee

3 DFO

4 Agency decision maker/recipient of committee recommendations

26\. How easy or difficult was this questionnaire to complete?

5 Very easy

4

3

2

1 Very difficult

> When answering the next question, keep in mind that your individual
> response will be kept confidential. Your comments will be combined
> with comments from all respondents in your Agency and sent to your
> Agency without any identifying information attached. Your comments
> will not be linked to you or your individual survey responses, or to
> any other information that may identify you. You will have 600
> characters for your response.

27\. Please tell us what else you would like us to know. (Fill in
response)

**[(PROGRAMMER NOTE: Right above the \"Submit Button\",
display:)]{.underline}**

Please submit your survey to Gallup by clicking the button below. Once
you close your browser, you will no longer be able to view the survey or
change your responses.

Thank you for your participation in the Advisory Committee Engagement
Survey.

You may now go to another Web page or [close]{.underline} your browser.

jlw\\2004\\GSA\\GSA Advisory Committee Engagement WEB 0405

[^1]: See Appendix A for a tabular representation of the survey
    respondents' composition.