# Presentation: 495386

## Expert Committees

- R. F. Shangraw, Jr., Ph.D.
- Chief Executive Officer
- Project Performance Corporation

- Methods and Approaches in Screening Proprietary Drug Names Public Meeting
- Renaissance Washington DC Hotel
- June 26, 2003

## Topics

- Framing the Problem
- Identifying Relevant Research
- Addressing the Key Questions
- Raising Two Other Concerns

## Framing the Problem

- Using expert committees to review proposed proprietary drug names for potential sound alike and look alike confusion with existing drug names
- Using expert committees to review multifactoral studies on potential sound alike and look alike confusions
- Using expert committees to  provide input to a single factor study

- Bound the Decision

- Identify Alternatives

- Evaluate Alternatives

- Consensus Decision

## The Key Questions

- Is an expert committee necessary to review information from studies?
- How many people should staff an expert committee?
- What credentials are important for expert committee members?
- Should expert committees meet in person, via videoconference, teleconference, email?

## Identifying Relevant Research

- Psychology
- Sociology
- Law
- Jury Deliberations
- Expert Witness
- Science Court
- Policy Sciences
- Forecasting
- Game Theory
- Organizational Behavior

- Medicine
- Appropriate and Necessary Care
- Group Decision-Making Methods
- Delphi Method
- Nominal Group Technique
- NIH Consensus Development Program
- RAND/UCLA Appropriateness Method

## Necessity of Expert Committees:  Research

- Empirical research suggests group decisions are more consistent – individual judgments are prone to personal bias (McDonnell et al, 1996)
- There is some empirical evidence that groups make quicker and better decisions (Blinder & Morgan, 2000; Hogarth, 1978)
- However, some empirical research on group decision making suggests a systematic bias in the outcomes (Janis & Mann, 1977)
- But, a recent study found that groups are better than individuals in interpreting letters and number substitutions (Laughlin, et al., 2002)

## Necessity of Expert Committees:  Practice

- Use an Expert Panel When:
- Historical data does not exist or is limited
- Future events are likely to invalidate the results of historical analysis
- Ethical and moral factors are sufficiently important to the decision

## Optimal Size of Expert Committees:  Research

- Assuming specific voting rules, committee size is negatively correlated with decision accuracy (Gabel & Shipan, 2000)
- Communication quality and complexity is greater in small groups (n=5) than in larger groups (n=10) (Fay, et al., 2000)
- Limit to 12 – 15 members for effective functioning of the group (Shekelle, et al., 1999)

## Optimal Size of Expert Committees: Practice

| Group Decision-Making Method | Recommended Expert Committee Size |
| --- | --- |
| RAND/UCLA Appropriateness Method | 7 – 15 Panelists |
| Nominal Group Technique | 8 – 12 Participants |
| NIH Consensus Development Program | 9 – 16 Panelists |

## Credentials for Committee Participation:  Research

- Need a baseline level of expertise but additional expertise does not improve accuracy (Armstrong, 1980)
- Committees should include participation from a variety of relevant disciplines (Fitch et al., 2001)
- Participants’ status affects group dynamics (Murphy, et al., 1998)
- Participants must be justifiable as "expert" on the matter under discussion (Jones & Hunter, 1995)

## Credentials for Committee Participation:  Practice

- Establish baseline qualifications
- Review conflicts of interest
- No advocacy or promotional position
- No financial interests
- Minimize domineering personalities
- Increase diversity

## Optimal Media for the Expert Committee:  Research

- Recent research focuses on decision quality, satisfaction, and media richness
- Computer-mediated group decision systems decreases group effectiveness, increases time to complete task, and decreases member satisfaction in the process (Baltes, et al., 2002)
- However, groups with a history can be just as effective with a computer-mediated system (Alge, et al., 2003)
- Group decision support systems lead to more complex communications than simple chat systems (Brandy & Young, 2002)

## Optimal Media for the Expert Committee:  Research

- Adding video to an audio-only system improves decision quality (Baker, 2002)
- Participants are less likely to be committed to computer-mediated processes (Shangraw & Bozeman, 1989)
- After reviewing 200 studies, “no difference” between face-to-face meetings and specific collaborative technologies (Fjermestad & Holtz, 1998)

## Optimal Format for the Expert Committee:  Practice

- Combine computer-mediated and face-to-face discussions if economically feasible
- Match media to the type and structure of the decision

## Addressing the Concerns with  “Groupthink”

- Facilitator should be impartial
- Facilitator should assign the role of “critical evaluator” to all committee participants
- One or more participants should be rotated through the role of “devil’s advocate”
- If feasible, sub-divide the group to work under different facilitators
- Immediately following preliminary consensus, hold a “second chance” meeting to express any residual doubts

## Using a Structured Approach:  The Nominal Group Technique

- Briefing provided on topic and method
- Ideas are silently generated on paper
- Participants share one idea from his/her list
- Review and consolidate ideas
- Voting is done privately
- Votes are tabulated

## Direction of Future Research

- Intense focus on the value of computer-mediated group decision-making system
- Specific interest in web-based, distributed computer-mediated group decision support systems
- Increasing interest in the value of combining expert panels with empirical, data-driven models