QUALITY IMPROVEMENT IN HIV-PREVENTION RESEARCH A rights-based and participatory approach

A toolkit for HIV-prevention research centres ACKNOWLEDGEMENTS

The Quality Improvement Toolkit represents the committed effort of many staff members and consultants at the International AIDS Vaccine Initiative (IAVI) and partner organizations. Acknowledgment for extensive writing and review goes to Maj-Britt Dohlie, Prince Bahati, Heather Clark, Julie Becker and Rachel Kagel. Thank you to Jennifer Lehrman, Leslie Nielsen, William Kidega, Judie Mbogua, Maaza Seyoum, Tsietsi Mokhele, Richard Rwanyonga, Simon Sigirenda, Surita Roux, Jane Odada, Hilda Ogutu, Joseph Nzioka and Mecthilde Mukankuku for their valuable input. Acknowledgment goes to our partner organizations—the Desmond Tutu HIV Foundation, Kenya AIDS Vaccine Initiative, Kenya Medical Research Institute, Medical Research Council, Projet San Fancisco, Uganda Vaccine Research Institute and Zambia-Emory HIV Research Project—for their participation and feedback in developing this toolkit.

IAVI is grateful to the organizations working in international health that are committed to quality improvement and that have developed a wide variety of materials in this area. This toolkit draws on these earlier efforts, integrating and adapting specific issues related to HIV-prevention research. In particular, IAVI would like to recognize EngenderHealth, which developed Client-Oriented Provider-Efficient Services (COPE®), and Liverpool VCT, which developed Quality Assurance Resource Pack for voluntary counseling and testing service providers, both of which were drawn upon for this manual. IAVI would also like to acknowledge the United States Agency for International Development’s PRIME II Project and Maximizing Access and Quality initiative, which provided additional guidance.

All information in this document may be reproduced or copied without permission, provided that the International AIDS Vaccine Initiative is acknowledged as the source.

This Quality Improvement Toolkit can be found online at http://www.iavi.org.

IAVI’s mission is to ensure the development of safe, effective, accessible, preventive HIV vaccines for use throughout the world.

This publication is made possible by the generous support of the American people through the United States Agency for International Development. The contents are the responsibility of the International AIDS Vaccine Initiative and do not necessarily reflect the views of USAID or the United States government. INTRODUCTION

What is quality improvement?

Quality improvement is the process of continuously assessing and improving the quality of goods or services to ensure client satisfaction and loyalty. The concept of quality improvement—QI for short—was conceived in industrial and commercial sectors and has been successfully adapted by community health, reproductive health and counseling services, often with different approaches and names. But it has yet to be applied widely or documented in HIV-prevention research, including clinical trials.

Generally, HIV-prevention research centres have relied on traditional approaches to quality control for services provided to volunteers (e.g., checklists completed by counseling, medical and laboratory staff). For the most part, these approaches document what has been done and not the quality of what has been done. They often fail to empower staff to identify areas of concern and participate in bringing about needed changes. Moreover, these methods might not provide any indication of volunteers’ satisfaction or provide a basis for improving services from both volunteer and staff perspectives.

Many services offered to volunteers in epidemiology studies and clinical trials (e.g., HIV counseling and testing, family planning and medical care) are similar to those offered by health centres. Unlike in health centres, however, participation and follow-up visits in HIV-prevention research are not necessarily motivated by the need for health services. HIV-prevention research also differs in that it targets specific groups of volunteers, has a target number of volunteers to enroll and involves a set of inclusion/exclusion criteria, informed-consent processes and community outreach activities that require development or adaptation of quality indicators specific to the research setting. Ensuring the quality of study-related services is critical to retaining volunteers and guaranteeing the ethical conduct of HIV-prevention research. Collective involvement by staff in identifying areas that need improvement and fostering their ownership of the process are critical to the success of quality improvement programmes.

The concept of QI in this toolkit applies to the overall environment of the research centre and the processes a volunteer experiences, from outreach and recruitment through the conclusion of a study. It does not apply to the clinical science itself, but to all of the services and processes that surround and enable the conduct of research.

The Quality Improvement Toolkit

This toolkit has been developed to facilitate the implementation of QI in HIV-prevention research. It is devised to help all research centre staff meet the following goals in accordance with international standards and local regulations:

- Maintain the highest level of ethical research standards
- Ensure effective, customized and community-centred outreach and engagement
- Ensure the highest standard of quality in volunteer-centred clinical and counseling services

Most HIV-prevention research centres are engaged in quality-related activities, including but not limited to following national and international standards such as Good Clinical Practices (GCP) and Good Participatory Practices (GPP) and monitoring visits. This guide does not compete with or replace any activities necessary for the ethically and scientifically sound conduct of research, including the provision of medical services to volunteers. Instead, it complements these guidelines and supports the research teams in implementing key aspects through a participatory approach. This toolkit is designed to help research centre staff go beyond existing efforts to ensure that volunteers’ perspectives and needs are taken into consideration. It aims to support a QI process that involves all research centre staff in making improvements at all stages of HIV-prevention research. Those research centres that have already adopted a participatory approach to QI can use this toolkit to reflect on their efforts and build on their successes.

1 Taegtmeyer, M., Doyle, V. (2003). Quality Assurance Resource Pack for Voluntary Counseling and Testing Service Providers, Liverpool VCT and Care Kenya, p4. Adapting the Toolkit

QI is a flexible process, and should be adapted and integrated into a research centre’s daily processes to ensure its sustainability. Every research centre is different and will need to use this toolkit in ways that make sense within its context. Country setting, size of the research facility and study objectives can all affect the implementation of QI. For example, some research centres might be conducting clinical trials, while others focus on epidemiology or other kinds of clinical studies. Even those conducting clinical trials differ in important ways. Some focus on vaccine development, while others assess microbicides or other strategies of HIV prevention, such as pre-exposure prophylaxis (PrEP). Similarly, some research centres might routinely work with relatively vulnerable volunteers (who may be stigmatized or whose behavior might be criminalized), while others might be working with low-risk cohorts. Some research centres will have fully staffed programmes (e.g., full-time community workers, on-site family planning services, pharmacists, etc.), while others will use alternative strategies for conducting community outreach (e.g., peer leaders, peer recruiters or community-based recruiters or agents) and for ensuring services to volunteers (e.g., nurse counselors who provide counseling and medicine to volunteers, referrals for family planning services, etc). Yet, although the contexts vary in these significant ways, all research centres share common ethical ground, based on international guidelines on rights and responsibilities that constitute the framework for QI in HIV-prevention research.

Toolkit contents

Part 1: Manual The Manual introduces the concept of quality, the rights and responsibilities framework, the typical HIV-prevention research process and the four-step quality improvement process.

Part 2: Sample Tools Sample tools are provided for use at the research centre and in the community. These tools can be adapted as necessary.

Part 3: Facilitator’s Guide The Facilitator’s Guide provides a step-by-step approach to introducing QI. It also contains handouts and visuals to help the facilitator convey key concepts to research centre staff and community agents.

# TABLE OF CONTENTS

Section I: Quality improvement in the research setting ................................................................. 2\
Concept of quality improvement in the research setting .......................................................... 2\
Common challenges to implementing quality improvement in HIV-prevention research .......... 3

Section 2: Rights and responsibilities framework ........................................................................ 4\
Proposed framework for ensuring the rights and responsibilities of volunteers ...................... 4\
Rights and responsibilities of volunteers .................................................................................. 4\
Ethical responsibilities of the staff ............................................................................................ 6\
Explanation of staff ethical responsibilities .............................................................................. 6

Section 3: The HIV-prevention research process ........................................................................ 8\
Stage 1: Formative research and community outreach and recruitment .................................. 8\
Stage 2: Pre-enrollment ............................................................................................................ 8\
Stage 3: Enrollment/study participation .................................................................................... 9\
Stage 4: Post-participation follow-up ...................................................................................... 9\
Figure 3.1: Key stages of HIV-prevention research ................................................................. 10

Section 4: The quality improvement process: overview and preparations .................................. 11\
Introduction to the quality improvement process .................................................................... 11\
Figure 4.1: The QI process and tools ....................................................................................... 11\
Setting the stage for the QI process .......................................................................................... 12\
Preparing research centre managers ....................................................................................... 12\
Identifying, selecting and training an effective QI facilitator .................................................... 13\
Preparing to introduce QI to research centre staff and community agents .............................. 13\
Introductory meeting with research centre staff ...................................................................... 13\
Introductory meetings with community agents (non-staff) .................................................... 14\
Establishing a QI committee .................................................................................................... 15

Section 5: The four-step quality improvement process ............................................................... 16\
Step 1: Identifying area to be improved/problem to be solved .............................................. 16\
Introduction to quality improvement tools .............................................................................. 17\
Team self-assessment – research centre .................................................................................. 17\
Team self-assessment – community ....................................................................................... 20\
Volunteer feedback interview ................................................................................................. 21\
Group discussion .................................................................................................................... 23\
Suggestion boxes .................................................................................................................... 25\
Step 2: Analyzing areas to be improved/problems to be solved ........................................... 26\
Step 3: Developing an action plan ........................................................................................... 27\
Step 4: Implementing the action plan ...................................................................................... 31\
Final Considerations on the QI Process .................................................................................. 33

References and resources ........................................................................................................ 34 SECTION I: QUALITY IMPROVEMENT IN THE RESEARCH SETTING

Concept of quality improvement in the research setting

Definitions of quality improvement in HIV-prevention research

Quality is a function of the degree to which customer or client needs are met. It requires that the providers of products and services be aware of the unique needs and preferences of their customers or clients. In the case of HIV-prevention research, including clinical trials, the client is the volunteer, and each volunteer’s and community’s needs and preferences must be addressed in the context of larger research goals. Quality in HIV prevention is thus more completely defined as the degree to which research-related processes meet volunteers’ and communities’ expectations and adhere to international and national guidelines of quality.

Quality improvement, meanwhile, generally requires a systematic process to assess, continuously improve and monitor products or services to ensure their effectiveness and safety. In HIV-prevention research, then, QI necessitates a concerted effort to improve research processes and participant experience through better community-centred outreach and engagement and volunteer-centred services at the research centre. Research centres might want to adopt the definitions of Quality and Quality Improvement in HIV-prevention research presented in Box 1.1, this page, or they may choose to modify them to better suit their situations.

Rationale for quality improvement in HIV-prevention research

Volunteers deserve consistently respectful and ethical treatment and the highest quality of services that can be provided by research centres. At a minimum, research centres must provide the communities from which volunteers are recruited with accurate information about HIV/AIDS and prevention research, devise interventions to reduce the potential stigma associated with participation in prevention research, offer volunteer-centred counseling services to study participants, maintain national/international standards of medical practice and establish mechanisms to ensure that all categories of staff are trained to provide quality services to volunteers within the framework of internationally agreed ethics and rights.

Why quality is important:

- It ensures compliance to international standards and increases efficiency of research conduct.
- Respectful treatment and effective delivery of services by research staff can make it easier to recruit and retain volunteers and ensure that enrolled volunteers follow their appointment schedules and regimens. High-quality counseling can increase the likelihood that volunteers adopt risk-reduction strategies and other positive health practices. It also helps those who are diagnosed with HIV to adopt effective measures to prevent HIV transmission, communicate with partners and seek treatment and other services as needed. Finally, it can enhance understanding of concepts such as “unknown efficacy” or “partial efficacy” and so reduce risk-compensation behavior (i.e. increased risk behavior resulting from the false belief that a study product will prevent HIV).
- Providing assurance of confidentiality to volunteers and protecting their rights can help ensure accurate reporting of behavioral data and adverse psychosocial and medical events.
- Appropriate information helps volunteers make informed decisions about participation, correctly use effective family planning methods while participating in research that requires pregnancy prevention, and take prescriptions correctly for sexually transmitted infections or other medical problems.

Why quality is important to research centre staff:

- Co-workers and volunteers tend to notice when staff performance improves. Such recognition creates a sense of accomplishment and increases motivation and job satisfaction.

Box 1.1: Definitions

Quality in HIV-prevention research

The degree to which formative research, community education, recruitment, enrollment, study participation and post-participation follow-up and access adhere to the highest standards of quality based on international & national guidelines and meet volunteers’ expectations.

Quality Improvement in HIV-prevention research

The concerted effort to continuously and systematically monitor quality of services offered in HIV-prevention research and to promote a participatory approach (involving volunteers, community agents and research staff) to problem-solving. Active involvement in QI by all research centre staff and community outreach workers leads to effective problem-solving and sustainable solutions grounded in the realities of the workplace and the social context in which the research occurs. This contributes to a better work environment.

Continuous and systematic QI can unleash creativity and innovation that otherwise might remain untapped for lack of a forum to discuss quality-related issues. This creates a more interesting and productive workplace.

Common challenges to implementing quality improvement in HIV-prevention research

Community context and research literacy In some communities where HIV-prevention research takes place, many people do not fully understand the rights and responsibilities of volunteers or the research process itself. This knowledge gap, along with the possible difference in social status between research centre staff and volunteers, can impede the candid discussion of issues and concerns essential to the QI process. Additionally, in communities where medical and psychosocial services are limited or not available, community members and volunteers might consider mere access to such services a privilege and may be reluctant to question the quality of the services they receive.

Staff may be unfamiliar with QI or lack ownership of the process Some research centre staff (including community outreach workers and community agents such as peer leaders and peer recruiters) might not be fully conversant with international standards for the rights and responsibilities of volunteers and staff. Staff members may find interactions with some volunteers challenging, especially when they conflict with personal values or their nation’s laws (e.g., working with vulnerable and marginalized populations such as sex workers, men who have sex with men or injecting drug users). In addition, some research centre staff might fear that the outcomes of QI will be used as a basis for disciplinary action, which can discourage honest feedback and detract from staff ownership of the process.

Management ownership of QI and the role of sponsoring agencies Management support and leadership is crucial for effective and continuous implementation of QI. Sponsors of HIV-prevention research can also play an important role in QI, but the process must be owned by the research centre, and sponsors must ensure that it is not used as a means to intrude on research centre management.

External factors that affect QI Volunteers render an exceptional service by participating in research. They give their time generously and risk high social costs, such as stigmatization and discrimination that can come with participation. The psychosocial safety of volunteers, one of the pillars of QI, depends on factors that research centres can control (e.g., confidentiality, community education) and those that they cannot (e.g., stigma related to volunteer’s disclosure of participation or a breach of confidentiality by another volunteer). Research centre staff must be conversant with external factors that can affect the quality of their research and take them into account when conducting QI. SECTION 2: RIGHTS AND RESPONSIBILITIES FRAMEWORK

Proposed framework for ensuring the rights and responsibilities of volunteers

Health service organizations have developed quality-of-care frameworks and patients’ bill of rights to ensure high-quality services and the ethical conduct of research. The proposed framework in Box 2.1 draws heavily on this earlier work and should help guide HIV-prevention research centres to improve the quality of their volunteer services, support high standards for ethics in HIV-prevention research and inform both potential volunteers and staff about the rights and responsibilities of study participants. Each of the rights and responsibilities is explained in more detail on the following pages.

Box 2.1: Charter of volunteers’ rights and responsibilities in HIV-prevention research

| Volunteers have the right to: | Volunteers have the responsibility to: | |---------------------------------------------------------------------------------------------|---------------------------------------------------------------------------| | Privacy and confidentiality | Make an informed decision about research participation | | Give voluntary informed consent based on sufficient and appropriate information | Provide complete and accurate personal and medical information to research staff | | Refuse to join or decide to withdraw from the study at any time | Make efforts to use appropriate HIV-prevention methods | | Protection from preventable biomedical and social harms | Make efforts to prevent pregnancy (if required for participation in the study) | | Access to selected services at the research centre or through referral | Follow study instructions and attend scheduled appointments | | Respectful treatment and comfort to express their opinions | Treat research centre staff with respect | | | Keep knowledge about participation of other volunteers confidential |

Rights and responsibilities of volunteers

Volunteers in HIV-prevention research have the right to:

- **Privacy and confidentiality.** Volunteers have the right to privacy and confidentiality from recruitment throughout the study and any follow-up services. In most cases, volunteers are identified by numbers, not their names. They also have a right to expect that counseling, testing, physical examinations, clinical procedures and the dispensation of medications will be done in a private area where they cannot be seen or heard by others. Volunteer records are confidential, and are shared only as needed to conduct the study. All categories of staff (including security guards, drivers, receptionists and administrators) are responsible for keeping volunteer participation confidential and should be trained or sensitized to that effect.

- **Give voluntary informed consent based on sufficient and appropriate information.** Volunteers have the right to culturally sensitive, clear and accurate information, including why the study is being conducted, what will happen during the study and what volunteers will be asked to do. They also have the right to receive information on reasonably foreseeable risks and possible benefits of participation.

- **Refuse to join or to decide to withdraw from the study at any time.** There is no penalty for refusing to join or withdrawing from a study.

- **Protection from preventable biomedical and social harms.** Volunteers have the right to a clean and safe research centre environment that can provide effective medical services and follows appropriate procedures to prevent infection and dispose of biomedical waste. The research must be based on sound ethical principles with careful consideration of benefits and risks. All research centre staff and those conducting

______________________________________________________________________

1 Volunteer rights and responsibilities have been adapted from the following sources: HIV Vaccine Trials Network’s “Patient’s Bill of Rights”, UNAIDS/WHO’s “Ethical Considerations in HIV Biomedical Research,” New York University’s “Research Participant’s Bill of Rights,” and EngenderHealth’s “COPE Handbook: A Process for Improving Quality in Health Services.” outreach are responsible for ensuring that their actions minimize the potential for physical, psychosocial and social harm to volunteers from their participation in HIV-prevention research.

- **Access to selected services at the research centre or through referral.** Volunteers have the right to receive:

  - Standard HIV-prevention services (appropriate counseling and access to all HIV risk-reduction methods) throughout the study, including new methods as they are scientifically validated or approved and become available.
  - Treatment and care for participants who become HIV positive during the study(^2) (such services will be provided either at the study centre or through agreements with other facilities; volunteers who screen out of a study because of existing HIV infection also have the right to referrals for these services).
  - Treatment, services and follow-up care for study-related illnesses, such as adverse events.
  - Appropriate family planning methods and counseling for volunteers who are required to avoid becoming pregnant during participation.
  - Special HIV testing at the research centre for any reason—including applications for health or life insurance, travel or employment—should participation in an AIDS vaccine trial result in a false-positive reading in standard HIV tests. Candidate AIDS vaccines often provoke the production of antibodies against key parts of HIV. These antibodies in turn produce an HIV-positive signal on some HIV tests. The detection of such antibodies does not necessarily mean that the individual is infected with HIV. In such cases, additional tests need to be done to discern a false-positive result from a true HIV infection. It is very important to note that in all cases, it is impossible for candidate HIV vaccines themselves to cause HIV infection.
  - Referral for required medical and psychosocial services unavailable at the research centre.

- **Respectful treatment and comfort to express their opinions.** Volunteers have the right to be respected by staff regardless of their personal choices, values, beliefs, lifestyles, sexual orientation, gender and backgrounds. Staff must ensure that volunteers are as comfortable as possible, physically and psychologically, during procedures. Staff should encourage volunteers to express their opinions, and do all they can to provide a nonjudgmental and supportive environment for different views.

**Volunteers in HIV-prevention research have the responsibility to:**

- **Make an informed decision about research participation.** Volunteers have the responsibility to weigh the risks and benefits, based on written materials such as the informed consent form and information provided during counseling sessions, before making the decision to participate in prevention research. They are also responsible for asking questions about concepts they have not understood and for requesting additional information at any time during the study to ensure they understand the procedures. This is essential to the ability of the research centre to provide them with appropriate counseling and, should the need arise, effective treatment.

- **Provide complete and accurate personal and medical information to research staff.** Volunteers have the responsibility to respond as accurately as possible to questions from research staff about personal information required by the study and to inform research staff of any changes in contact or health information during the study. Participants are encouraged to report any experience of discrimination or social harm related to study participation, and should inform staff if they are unable to participate or want to withdraw.

- **Make efforts to use appropriate HIV-prevention methods.** Volunteers have the responsibility to reduce their risk of HIV infection and transmission to the best of their ability by communicating with partners about risk where possible, using or asking partners to use male or female condoms, reducing risky sexual practices, reducing numbers of partners and using safe injection practices. HIV-prevention methods and counseling on the use of these methods are provided by the research centre. Adult male circumcision is an option male volunteers might consider in addition to other existing methods and newly proven interventions. Research staff will help volunteers learn to discuss HIV status and risk-reduction strategies with their partners.

- **Make efforts to prevent pregnancy if required for participation.** Female volunteers have the responsibility to use effective birth control methods (provided by the research centre or through referral) to prevent pregnancy if required to do so by the study protocol. This is important during clinical trials, as it is typically unknown what effect, if any, an experimental product is likely to have on fetal development.

______________________________________________________________________

(^2) While volunteers are counseled extensively regarding risk-reduction behaviours and provided with condoms, some volunteers may still engage in risky behaviour and become infected with HIV. HIV vaccine candidates do not contain live virus and cannot cause infection. Follow study instructions and attend scheduled appointments. Volunteers have the responsibility to follow instructions related to the research product to the best of their ability and work with study staff to maintain health and safety during the study. Volunteers are encouraged to attend all scheduled appointments or reschedule the appointment if a conflict arises.

Treat all research centre staff and other volunteers with respect.

Keep knowledge about the participation of other volunteers in the study confidential. Volunteers have the responsibility to respect the privacy of others participating in the study. The volunteer alone has the right to decide whether to inform his or her partner(s) or others about participation in the study.

Ethical responsibilities of the staff

Research centre staff members have ethical responsibilities as shown in Box 2.2. Each responsibility is explained in more detail below.

Box 2.2: Charter of staff ethical responsibilities in HIV-prevention research

Staff have the responsibility to:

- Treat volunteers and colleagues with respect
- Ensure that volunteers understand the study and informed consent materials before and during their participation in clinical research
- Maintain volunteers’ confidentiality
- Ensure volunteers are as safe as possible from biomedical and social harms
- Ensure volunteers’ access to appropriate HIV-prevention methods and health services either on-site or through referral
- Adhere to the study protocol and international standards (e.g. GCP, GCLP, GPP, UNAIDS ethical guidelines)

Explanation of staff ethical responsibilities

Staff in HIV-prevention research have the responsibility to:

- Treat volunteers and colleagues with respect. Staff members should respect the personal choices, values, beliefs, lifestyles and backgrounds of volunteers. They must also do all they can to ensure that volunteers are as comfortable as possible, physically and psychologically, during procedures.

- Ensure volunteers understand the study and informed consent before and during participation. Staff members are responsible for explaining informed-consent principles and information about the study—including objectives, procedures, risks and benefits—in a culturally appropriate manner, using language and formats that the volunteer can understand. It is recommended that staff revisit the informed consent form at least once after the enrollment visit and review it again when new study-related results or other relevant information become available. The need for additional explanation and reiteration of those principles should be assessed on an individual basis (e.g., a volunteer expresses concerns during a counseling session). Staff must also ensure that potential volunteers do not feel pressured or coerced into participating in research.

- Maintain volunteers’ confidentiality. Staff members have the responsibility to protect the privacy of volunteers and keep their information confidential throughout the study and over the course of any follow-up services. In most cases volunteers are identified by number, not name. Staff should ensure recruitment, follow-up, counseling, testing, physical examinations, clinical procedures and medications are provided in a private area where volunteers cannot be seen or overheard by others. Volunteer records are confidential and are shared only as needed to conduct the study. All employees of the research centre (including security guards, drivers, receptionists, and administrative staff) are responsible for keeping volunteer participation confidential and should be trained or sensitized to that effect.

- Ensure volunteers are as safe as possible from biomedical and social harm. Staff members have the responsibility to ensure a safe research centre environment, including effective medical care, infection-prevention and waste-disposal procedures. Clinical staff members have the responsibility to adhere to Good Clinical Practices and Good Clinical Laboratory Practice. The research must be based on sound ethical principles with careful consideration of benefits and risks as established by international and national guidelines. All research centre staff and those conducting outreach are responsible for ensuring that their actions minimize the potential for social harms that volunteers might encounter from participating in HIV-prevention research. Research centres have the responsibility to engage communities and apply, to the extent possible, the recommendations of the Good Participatory Practices.

- **Ensure volunteers’ access to appropriate HIV-prevention methods and health services either on-site or through referral:** Staff members have the responsibility to provide volunteers with:

  - Standard HIV-prevention services (appropriate counseling and access to all HIV risk-reduction methods) throughout the study, including new methods as they are scientifically validated or approved and become available.
  - Treatment and care for participants who become HIV positive during the study (such services will be provided either at the study centre or through agreements with other facilities; volunteers who screen out of a study because of existing HIV infection also have the right to referrals for these services).
  - Treatment, services and follow-up care for study-related illnesses, such as adverse events.
  - Appropriate family planning methods counseling for volunteers who are required to avoid pregnancy during participation.
  - Special HIV testing at the research centre for any reason—including applications for health or life insurance, travel or employment—should participation in an AIDS vaccine trial result in a false-positive reading in standard HIV tests. Candidate AIDS vaccines often provoke the production of antibodies against key parts of the HIV virus. These antibodies in turn produce an HIV-positive signal on some HIV tests. The detection of such antibodies does not necessarily mean that the individual is infected with HIV. In such cases, additional tests need to be done to discern a false-positive result from a true HIV infection. It is very important to note that in all cases, it is impossible for candidate HIV vaccines themselves to cause HIV infection.
  - Referrals for required medical and psychosocial services that are unavailable at the research centre.

- **Adhere to the study protocol and international standards.** Staff members have the responsibility to follow the requirements of the study protocol, including data collection methods, reporting adverse events, providing proper storage of products, maintaining accurate records and ensuring their confidentiality.

______________________________________________________________________

3 Good Participatory Practice: Guidelines for Biomedical HIV Prevention Trials. UNAIDS. 2011. http://data.unaids.org/pub/Manual/2007/jc1364_good_participatory_guidelines_en.pdf SECTION 3: THE HIV-PREVENTION RESEARCH PROCESS

Maintaining high standards of ethics, protecting the rights of volunteers and providing them with quality services should be an integral part of the research process. Although every study is unique, and specifics of the process vary from place to place, most research related to HIV prevention—whether a clinical trial or an epidemiology study—follows the four stages shown in Box 3.1.

HIV-prevention research begins and ends with diligent community engagement. This is achieved mainly by educating people in the community about the proposed research and HIV prevention in general, and via consultative mechanisms such as advisory boards. The objectives of such efforts are to build support for the research and increase knowledge in the community about HIV prevention, learn about the community’s perspectives and needs and support the recruitment and retention of potential volunteers. Note that community engagement is a long-term exercise that continues for the duration of a program of research and is not confined to any particular study protocol.

Stage 1: Formative research and community outreach and recruitment

Collaborating with community stakeholders on formative research activities ensures their perceptions and cultures inform study design and conduct. Community outreach ensures that local populations understand, support and are prepared for the research. The community receives information primarily through seminars, fliers, gatherings and radio announcements, as well as through individual outreach. Providing general education about HIV-prevention research decreases the probability of there being any stigma associated with participation in such research. People who express an interest in participating in the research are invited to visit the research centre for pre-enrollment.

Stage 2: Pre-enrollment

Pre-enrollment begins when a potential volunteer arrives at the research centre. This stage includes informed consent, screening and additional education about the study and the research process. Note that education begins during the community outreach and recruitment stage and continues throughout pre-enrollment and study participation.

Each potential volunteer must give informed consent at pre-enrollment before entering the study. Researchers are responsible for ensuring that each potential volunteer fully understands all aspects of study participation through extensive individual education and group sessions. The informed-consent process during pre-enrollment generally includes the following steps:

- Potential volunteer receives and reviews a copy of the informed-consent form.
- Potential volunteer attends a presentation explaining the study, followed by a group or individual discussion, including a question-and-answer session to clarify key concepts.

Box 3.1: Key stages of HIV-prevention research

Stage 1: Formative research and community outreach and recruitment Stage 2: Pre-enrollment Stage 3: Enrollment/study participation Stage 4: Post-participation follow-up

4 Adapted from IAVI’s AIDS Vaccine Literacy Toolkit At the completion of the question-and-answer session, the potential volunteer has a one-on-one session with a nurse/counselor to receive individual counseling and review the informed consent document again.

Appropriate research centre employee conducts an assessment of understanding to ensure that the potential volunteer fully comprehends key information about participation.

If interested in participating, the potential volunteer signs the informed-consent form and the screening process begins.

The screening process determines whether the individual is eligible to participate in the study. It includes one or more visits, depending on the requirements of the study. During screening, volunteers:

- Receive detailed information on HIV, risk-reduction counseling and family planning counseling when appropriate.
- Complete a pre-screening questionnaire that collects behavioral and basic medical information.
- Undergo an HIV test, including pre- and post-test counseling.
- Undergo additional medical assessments and blood tests when applicable.

If the volunteer is eligible and still wishes to participate, he or she is enrolled in the study. If the volunteer is ineligible, he or she is referred to appropriate services, such as further counseling or HIV treatment, care and support if the person is infected with HIV.

Stage 3: Enrollment/study participation

Only volunteers who have provided informed consent, met all eligibility requirements and expressed the desire to participate are enrolled in the study. In a clinical trial, each volunteer is randomly assigned to receive either the experimental candidate product or a placebo. In most studies volunteers will not know whether they received the candidate product or the placebo until the study is completed.

The volunteer might be required to make several visits to the research centre. At each scheduled visit, a counselor is available to provide HIV risk-reduction counseling. HIV-prevention methods, medical care, psychosocial support and family planning counseling are provided on site or by referral. Ongoing education, including review of the informed-consent materials, also occurs at each visit.

In a clinical trial, staff members collect blood samples from the volunteers at different times. The blood is examined in the laboratory to determine whether the candidate product is safe and elicits a vigorous immune response. In both clinical trials and epidemiology studies, researchers may collect blood samples at various times to test for HIV and other sexually transmitted infections pertaining to eligibility requirements, to enhance understanding of the immune response to HIV infection and to meet other objectives of the specific protocol.

Stage 4: Post-participation: follow-up and access

Staff members may monitor the health of volunteers for a predetermined follow-up period after the study has been completed. Volunteers will continue to receive referrals for off-site counseling and HIV prevention or treatment services, as necessary. They may also be given the opportunity to enroll in additional studies. Researchers communicate the results of the study to the volunteers and the community as soon as such announcements are possible. When appropriate, researchers should involve stakeholders in discussions about access should the product or intervention prove efficacious. This helps sustain the trust and goodwill that allows for future research to take place in that community. Figure 3.1: Key stages of HIV prevention research

1. Formative research and community outreach and recruitment

   - Community members are consulted and engaged in the research process. Information about the research is disseminated through seminars, fliers, radio announcements and individual outreach.
   - Interested community members are invited to visit the research centre for more information.

2. Pre-enrollment

   - Potential volunteer learns more about the research from general information sessions or private discussions with staff at the research centre.
   - Understands all details of screening and research study participation; completes pre-screening questionnaire.
   - Signs informed consent.
   - HIV test with pre- and post-test counseling
     - Positive HIV test
       - Post-test counseling and referral for care
     - If test is negative, screening questionnaire
       - Does not meet requirements of study
       - Medical history and exam
         - Does not meet requirements of study

3. Enrollment / study participation

   - Fully eligible, knowledgeable and willing volunteer enters study.
   - Random assignment to candidate product or placebo group (clinical study).
   - Delivery of study candidate product or placebo (clinical study).
   - Medical exam, lab tests, including HIV test with pre- and post-test counseling.
   - Volunteer returns for follow-up visits to report on health or problems and/or for further doses/regimen according to protocol.

4. Post-participation: follow-up & access

   - Volunteer returns to report on health or concerns.
   - Results of the study are communicated to the volunteer, CAB and community.
   - Access to trial products and procedures if applicable. SECTION 4: THE QUALITY IMPROVEMENT PROCESS: OVERVIEW AND PREPARATIONS

Introduction to the quality improvement process

Implementing QI in prevention research requires a systematic process to monitor, assess and continuously improve the quality of services offered in HIV-prevention research and to promote a participatory approach to problem solving to ensure that services meet the needs and preferences of the volunteers and community. Research-related processes occur both in the community and at the research centre, and the quality of community engagement affects services at the research centre and vice versa. The QI approach in HIV-prevention research thus addresses both community and research centre activities.

Figure 4.1 depicts the continuous four-step QI process and tools used in each step. The process includes:

- Identifying an area to be improved or a problem to be solved
- Analyzing the area to be improved or problem to be solved
- Developing an action plan
- Implementing the action plan

Figure 4.1: The quality improvement (QI) process and tools

Setting the stage for the QI process:

- Prepare research centre managers
- Identify and train QI facilitator
- Hold introductory meeting for research centre staff
- Hold introductory meetings for community agents (non-staff)
- Establish QI committee
- Ensure compliance with local ethics committee requirements, if applicable

The QI Process

Step 1: Identify area to be improved or problem to be solved

Tools:

At research centre level

- Team self-assessment
- Feedback interviews
- Group discussion
- Others as desired

At community level

- Team self-assessment
- Group discussion
- Others as desired

Step 2: Analyze area/problem

- Root-cause analysis

Step 3: Develop action plan

- Sample action plans

Step 4: Implement action plan

Ongoing monitoring and evaluation by the QI committee Setting the stage for the QI process

Preparing research centre managers

Efforts to involve staff in QI are usually successful if such efforts are a high priority for management, and management creates a supportive environment for staff to engage in the process. Initially, some managers might not feel comfortable with participatory approaches to quality, or might not be well equipped to provide effective support for their staff. Lack of awareness regarding the importance of such support and how to provide it, combined with lack of good supervisory skills, sometimes leads to weaknesses in this area. Box 4.1 highlights some of the issues and recommendations that managers should consider to create a supportive and safe environment for staff to be actively involved in QI. These issues also apply to members of the QI committee.

Box 4.1: Creating an environment conducive to good employee performance and quality services:

- **Ensure that there are no negative consequences for positive actions.** Do not punish someone for identifying a problem.
- **Provide feedback to individual employees and teams in a non-judgmental manner.** Avoid resorting to personal criticism.
- **Make sure that all employees know what is expected of them.** Never assume that employees know what is expected of them or how important particular tasks and behaviors are to maintaining high-quality services.
- **Show support for innovation and initiative in employees.** Showing staff members that innovation and initiative are valued will encourage them to participate in the QI process.
- **Cultivate team spirit.** Use language such as, “We face this difficult situation/problem together. Let’s work together to fix it…”
- **Create an environment conducive to learning.** Ensure that there are opportunities for all staff members to receive needed training, including opportunities for professional development. Many people are motivated to improve their performance when they are given opportunities to learn.
- **Take the mental health of staff members seriously.** Ensure staff members receive needed psychological support. Working in HIV-prevention research can be emotionally distressing.
- **Show appreciation to staff members when they exceed expectations.** Be specific so they understand that you have noticed their good work.
- **Take time to listen to staff.** Pay attention to the ideas and opinions of staff members and give them credit when their ideas lead to improvements.
- **Show respect to all levels of staff.** Staff members are your resident experts on their lines of work and often have the best ideas for improvements in their areas of responsibility.
- **Ensure staff members have the infrastructure and materials they need.**
- **Involve staff in finding realistic solutions, and be creative with the resources you have.**
- **Keep your promises to staff.** Credibility, once lost, can be hard to recover.
- **Delegate appropriate tasks.** Most people enjoy solving problems and being in control of their work environment. Delegate responsibilities appropriately, and hold people accountable for what they can realistically accomplish.

______________________________________________________________________

5 This section draws on EngenderHealth’s 1) COPE® Handbook: A Process for Improving Quality in Health Services, 2) Facilitative Supervision, as well as work done by USAID’s MAQ initiative and the PRIME II Project. Managers should keep in mind that some problems exposed in the QI process may reveal flaws in management and may leave them feeling put on the spot. In the end, however, appropriate delegation of tasks and engagement of staff members will lead to a better functioning research centre. And the effectiveness of managers is often reflected in the performance of the people they oversee.

**Identifying, selecting and training an effective QI facilitator**

Identifying an effective facilitator for QI is critical. Depending on the extent to which the research centre has used QI tools, it might already have such a facilitator in all but name. For example, a nurse counselor who is responsible for quality-related issues may be able to use the *Facilitator’s Guide* to initiate QI. If not, the research centre needs to identify one or two facilitators that can be trained. This facilitator should not be a high-level manager; it is important to involve all categories of staff in the QI process, including facilitation. Consider engaging external assistance to train selected facilitator(s), if necessary.

**Criteria for selecting a facilitator:**

- Experience with participatory facilitation
- Understanding of QI or international guidelines (e.g. GCP, GPP)
- Demonstrated commitment to quality
- Well respected by both management and staff
- Good communication and listening skills
- Sufficient influence at the research centre to implement changes

**Preparing to introduce QI to research centre staff and community agents**

Research centre managers and the QI facilitator need to prepare before introducing QI to research centre staff and community outreach workers. This includes holding an introductory meeting for staff, presenting QI tools to staff, forming a QI committee and planning for QI implementation, among other things. Managers and the QI facilitator must familiarize themselves with the entire toolkit. The choice of managers responsible for QI will vary from one research centre to another. Options include the Principal Investigator, Research/Trial Coordinator or Project Director.

**Introductory meeting with research centre staff**

Once the managers and QI facilitator are familiar with the tasks and issues related to introducing QI, it is time to involve research centre staff. The introduction of the process and tools can take place at one or more regular staff meetings or at a special meeting for this purpose (see *Facilitator’s Guide* for different options).

**The QI introductory meeting(s) will aim to ensure that staff members:**

- Understand the definition of QI and its rationale (Section 1)

- Understand that QI is a continuous process (Figure 4.1).

- Are familiar with the proposed Volunteer’s Rights and Responsibilities Framework (Section 1).

- Understand that they will be involved in the QI process and will receive adequate support.

- Establish a set of rules that create an environment conducive to QI (Box 4.1).

- Are familiar with QI tools and have provided suggestions for their appropriate adaptation (Section 5).

- Understand that they will be responsible for using the tools.

- Provide suggestions for effective implementation of QI.

- Recognize issues related to QI sustainability. At the end of this meeting, management and staff will have to decide:

- Which tool(s) to use and how to proceed. Some research centres might decide to use multiple tools concurrently, while others might prefer to use them sequentially.

- A date and time when staff will begin to use the tool(s) (allow time for the appropriate adaptation of tools by staff).

- Whether to first implement QI in the research centre or the community, or both simultaneously.

Introductory meetings with community agents (non-staff)

Some research centres use non-staff community members (e.g., peer leaders, influential network agents/leaders, community advisory board [CAB] members) for community education, community engagement or recruitment (e.g., referrals from existing volunteers, peer recruitment). In this toolkit, such stakeholders are referred to as community agents. The purpose of meeting with community agents is to request specific assistance with strengthening outreach activities. This should be an interactive meeting. Participants should be encouraged to ask questions and engage in open discussion (see Facilitator’s Guide for more details).

Depending on the audience, the QI introductory meeting will aim to ensure that community agents:

- Are aware of the rights and responsibilities of volunteers.
- Are aware of the research centres’ commitment to improving the quality of their services.
- Become more aware of their role in improving quality.
- Become more aware of the QI process and tools.
- Become familiar with the underlying principles that guide QI community tools and QI community activities.
- Recognize that the research centre wants to cultivate an environment conducive to eliciting quality-related feedback.
- Develop rules of conduct for collaboration.
- Become familiar with the research centre’s plans for QI and mechanisms for collecting feedback from them.

At the end of the introductory meeting, the facilitator, the community liaison officers and community agents will:

- Choose which quality-improvement tool to use: community team self-assessment or group discussion (explained below)
- Decide whether to start using the chosen tool immediately (after it has been adapted to suit local needs) or to convene a meeting to apply the tools.

To prepare the diverse groups involved in community outreach for QI, research centres should consider:

- The tools they adopt: Research centres that use different strategies and agents for community outreach will need to determine which tool (community team self-assessment or group discussion) is appropriate for each group and select questions relevant to the role that each plays in community outreach and recruitment.
- Mixed groups or homogenous group: Research centres that use two or more categories of community agents need to decide whether they will conduct community team self-assessment or group discussion with a homogenous group of community agents (e.g., peer leaders, CAB members or peer recruiters only) or with a mixed group of community agents (e.g., representatives from influential networks, peer recruiters, CAB members). Although each research centre will evaluate the suitability of each approach with reference to its needs, all of them must ensure that community agents (especially peer volunteers) are aware that joining the QI process could expose their participation in research and that they provide consent in light of this knowledge before taking part in the team self-assessment or group discussion.
- Selection of individuals for participation in the QI process: Research centres with a large number of peer leaders, peer volunteers or CAB members should select only a few participants. Give priority to individuals who can provide honest feedback, have significant experience in their role and, to the extent possible, are representative of the people who participate in research at the centre. Role of the research centre staff in charge of community outreach (commonly referred to as community liaison officer or CLO): The community liaison officer often works closely with community agents and supervises their work. The role and responsibility of a CLO in the community is similar to that of a manager for QI at the research centre level. He or she must help cultivate an environment that is conducive to QI with community stakeholders and provide support to the QI facilitator(s).

Duration and cost: The introductory meeting and use of one of the community QI tools very likely can take place in one day (see Facilitator’s Guide). However, research centres might have reason to conduct introductory sessions over the course of a few days or hold separate sessions for different types of community agents. If so, research centres that compensate community agents for time spent in training and at meetings will need to keep costs in mind when planning.

Establishing a QI committee At the end of the introductory meetings, the QI facilitator(s) ensures that a QI committee comprising different levels of staff is convened. Depending on the size of the research centre, the QI committee can include the research centre’s QI facilitator and representatives from the following teams: management, counseling, laboratory, clinical, reception and cleaning staff, and from the community outreach team. The QI committee is responsible for sustaining, monitoring and evaluating the QI process. Their main roles are:

- Finalizing the adaptation of tools (including exploring other QI tools as necessary) based on recommendations from the introductory meetings.
- Preparing the initiation of QI based on dates approved by management.
- Supervising the implementation of QI and troubleshooting any process-related challenges.
- Monitoring implementation of action plans.
- Disseminating progress made on QI and appreciations to management, staff, community agents and volunteers.
- Cultivating an environment conducive to QI.

Issues that should be considered regarding the QI committee include:

- Length of time members will serve.
- Method of assigning or electing new members.
- Roles of the different QI committee members. (Do they need to have formal roles, such as secretary, chairperson, general members? Do they establish rules or adopt existing rules for other committees at the research centre? Does the research centre need a standard operating procedure for conducting QI on site?)
- Establishment of a regular meeting schedule to monitor the action plan.
- Method for communicating regularly with those responsible for implementing changes. Committee members need to intervene if a given problem is not being solved. The proposed solution might need to be reconsidered if it cannot be implemented.
- Maintenance of records on solved and unsolved problems.

Ensure compliance with local ethics committee requirements In some contexts, it might be important to ensure that proposed QI activities comply with the requirements and regulations established by local ethics committees. For example, some ethics committees might require a review of any material that requires asking volunteers questions (e.g., feedback interview, group discussion or publishable data resulting from QI activities). Step 1: Identifying area to be improved/problem to be solved

The first step requires research centre staff to apply a tool or combination of the tools, shown in Box 5.1, to identify problems or areas that need improvement. An explanation of these tools follows; various samples of the tools are in Part 2 of the toolkit.

Consideration before selecting tools

Should QI tools be used at the research centre and community levels simultaneously? QI teams are encouraged to use a selection of tools from both categories in Box 5.1 simultaneously, using a team approach to identify problems. This can be valuable because staff members providing research centre services often have insight into the problems and issues raised by those conducting community outreach activities—and vice versa. For example, counselors might be aware of concerns that volunteers have regarding community outreach and be able to contribute to improving the quality of services in this area. Similarly, community outreach workers might be aware of concerns that volunteers have regarding counseling or health care at the research centre, and so provide a completely different perspective to the research centre team. For this approach to work, good communication must be developed, if it does not already exist, between research centre staff and community outreach workers and agents. This ensures that each of the teams is well informed about the activities and challenges of the other. However, this approach can pose challenges for a centre that is new to QI. In such cases, research centres might consider starting the process of problem identification with one category and expanding it to include both as they become familiar with QI processes. Even if problem identification is conducted separately, consider integrating the findings from the two teams in action-plan development and implementation.

How many tools should be used?

All the tools presented in this toolkit do not need to be used during every QI exercise. Apply a feasible number of tools and take on a realistic number of problems during each QI exercise to avoid overwhelming centre staff and outreach workers. Keep in mind that although some proposed solutions could require complex changes, others could be quite simple.

Should the research centre use only the tools mentioned in this toolkit?

There are many other ways to identify problems or areas in need of improvement. The research centre can use tools other than those included in this kit, including those developed by other organizations, and tools designed for a specific purpose (e.g., use of Volunteer Flow Analysis if volunteers’ waiting time is a recurrent problem or has been identified in previous QI exercises). It is also important to note that some problems can still be identified without using the QI tools. When problems are discovered outside regular QI exercises, research centre staff, community workers and managers should follow the problem-analysis process discussed in the next step.

______________________________________________________________________

Box 5.1: Tools included in the toolkit

**Research centre services:**

- Team self-assessment—research centre
- Volunteer feedback interview
- Group discussion
- Others, if desired. For example, volunteer flow analysis (from COPE®)

**Community outreach activities:**

- Team self-assessment—community
- Group discussion
- Others, if desired

______________________________________________________________________

6 If volunteers indicate that waiting times are long, or if staff members identify this as a problem, research centres may use Volunteer Flow Analysis if they think this will contribute to understanding the problem. The COPE methodology has a tool of this kind that may be adapted and used. It can be found at http://www.engenderhealth.org/files/pubs/qi/handbook/cope_handbook-a.pdf Additionally, areas that require improvement often become apparent in a number of ways, including:

- Research centre staff members or those responsible for outreach discover problems as they go about their daily tasks, or they hear rumors in the community about the research centre.
- Volunteers mention concerns or problems they encounter because of research participation.
- Staff members themselves realize that the centre is falling short of its standards, or that its operating procedures are not being closely followed.

**Tool selection and adaptation**

Once the research centre has decided to initiate QI, the QI committee must:

1. **Review all the tools.** An in-depth introduction to the tools is presented on the following pages.

2. **Select the tools to be used.** The recommendation is to combine service provider-oriented tools and volunteer/community-oriented tools at research centre and community levels. For example, research centres can choose to use team self-assessment (a service provider-oriented tool) and one of the volunteer-oriented tools (say, volunteers’ feedback interviews or group discussions) to explore the volunteers’ perspectives during the first and subsequent QI exercises. If the research centre wishes to use additional tools after the first exercise, it may do so. In any case, it should use a variety of tools for different QI exercises to ensure that QI participants remain interested. For example:

   - Research centre “A” might start with team self-assessment at the research centre, then conduct volunteer feedback interviews for the first QI exercise. For subsequent QI exercises, it might use group discussion at the research centre and team self-assessment with the community.
   - Research centre “B” might conduct volunteer feedback interviews, followed by community team self-assessment for the first QI exercise. For subsequent exercises, it might use team self-assessment at the research centre and group discussion in the community.
   - Research centre “C” might start with a team self-assessment at the research centre, followed by a group discussion with selected volunteers. It might then decide to use group discussion in the community, followed with volunteer feedback interviews for the next QI exercise.

3. **Adapt the tools.** Each research centre’s QI committee needs to review the questions and ensure they are appropriate. It should feel free to remove, change or add questions as needed.

4. **Incorporate the process into the work schedule.** Although it is necessary to apply the tools and go through the other steps described in the QI process, disruptions to the research centre and its services can be minimized by adapting and incorporating QI activities into existing meetings and committees.

**Introduction to quality improvement tools**

**Team self-assessment – research centre**

The self-assessment tools in this kit encourage self-assessment in a team or teams (as opposed to individual self-assessment), involving all categories of research centre and community outreach staff.

There are two versions of the team self-assessment tool: one for the research centre, and one for those involved in community outreach. The research centre self-assessment consists of questions that the team and its members ask themselves to capture the volunteers’ experience at the centre as they go from one station to the next for different types of services. “Station” here means a service point (e.g., reception, counseling room, laboratory, exam room, etc.). These questions are devised to reflect national and international standards for conducting clinical research. When staff respond “no” to a question, it is likely that they have identified a problem (see the tools in Part 2).

The team self-assessment should not be used when it interferes with staff performance or volunteer appointments. Team self-assessment should be planned for times when there are few or no scheduled volunteer visits at the research centre. Instructions: How to apply team self-assessment at the research centre

1. **Adapt the team self-assessment tool to fit your program of research.** The QI committee, in collaboration with staff, should adapt the team self-assessment tool to the research in which they’re involved and the populations with which they work. This can be done simply by crossing out questions that are not relevant and skipping them when using the tool. Staff can also add questions that they believe to be relevant to their particular circumstances.

2. **Determine the optimal composition of the QI team.** Depending on the size of the research centre, it might be more effective to divide staff into multiple QI teams when using the team self-assessment tool. QI teams should have a maximum of eight members. If the research centre is small, all or most of the staff can be part of the QI team and use the self-assessment together. If the research centre is large, each department can send a representative to form a QI team. If a QI team is too large (more than eight members), divide it into two smaller teams and assign each to complete part of the tool. To provide an opportunity for everyone to participate and develop professionally, different staff members from departments can take turns being on the teams. Research centres that have community outreach workers as part of the staff team are encouraged to involve them in this team self-assessment. As research centres become more experienced in using the self-assessment tool, they may consider whether to engage community agents, potential volunteers (e.g., community members who attended education sessions, screening visits, and those who opted not to enroll), volunteers who dropped out of a study or enrolled volunteers in subsequent self-assessment exercises to identify problems.

3. **Determine the most appropriate and efficient way to conduct team self-assessment.** Choose one of these options, depending on the size of the research centre:

   - In the case where all or most research centre staff constitute one QI team, the team uses the self-assessment together, answering the questions through discussion as they visit one station after another (mostly following the volunteer’s typical progress) over one or several days, as time permits.
   - In the case of multiple teams composed of departmental representatives, each team can be assigned a section of the self-assessment and the corresponding station to visit. Each team should complete the general part of the self-assessment in addition to the assigned section. This means that there are several QI team members working in different areas. However, each team must ensure that it has adequate competency to review the area(s) in the centre that it visits, and that its membership represents all categories of staff (e.g., the team cannot consist of managers only).

4. **Introduce the process and prepare QI team.** Have the QI team members reflect on the traits and needs of the current cohort of volunteers in the context of national/international standards. Go from station to station, completing the questions on the assigned section of the self-assessment by interacting with staff at each location. In addition, the QI team should observe quality-related issues at each station (e.g., crowded reception area, cleanliness, privacy). In the case of small research centres, where the QI team is composed of all staff, at least one staff member should be present at each station (e.g., at least one counselor should be at the counseling station) to respond to questions from the QI team. Then that person can rejoin the QI team when it moves to the next station.

**Role of research centre staff at each station**

The staff at the station should answer questions posed by the QI team objectively. They should be prepared to provide additional insights, documentation and data if needed. Co-workers and peers from other departments are more likely to see the different stations with fresh eyes.

**Role of the QI team**

The QI teams should encourage those working at each station to identify problems. If staff are unable to do so, team members should ask open-ended questions (beginning with words such as “what” or “how”) to explore issues. Possible questions include:

- How do you think volunteers feel about privacy in the reception area?
- What do you think volunteers feel when they see, for example, a photo of a person dying of AIDS in a physician’s room, a stigmatizing poster, a lengthy informed-consent form, etc.?
- What do you do to make people comfortable before drawing blood? How do you think volunteers react to all the blood samples when they enter the laboratory? (if blood samples are visible).

How do you think volunteers feel about picking up condoms from the reception area?

Role of research centre and QI teams when volunteers and community agents are involved

As mentioned previously, research centres with extensive experience in QI might want to include community agents in their self-assessment. Here are some considerations:

- The QI committee should adapt the process as needed.
- The QI team facilitating this process must listen to the volunteers/community agents and encourage them to speak. Research centre staff on the QI team should not identify problems during this visit, but leave that to the volunteers/community agents, only asking questions as necessary. Staff at each station visited should do the same.
- The QI committee should adapt the tools as needed. The QI team should change the questions in the team self-assessment to ensure they are of relevance to the respondents. If the QI team does not receive adequate feedback from the respondents, they can consider open-ended questions. Appropriate open-ended questions with volunteers, potential volunteers and community agents include: “How can we make the services volunteers receive more friendly and meet the needs of this population?” “What information is needed to ensure true informed consent?” Other possible open-ended questions can be adapted from those listed on the group discussion tool (without making it a group discussion) later in this section.
- Staff at each station must never become defensive when volunteers or community agents identify a problem. Similarly, staff should do all they can to avoid making volunteers and community agents defensive.
- Staff should not make promises the research centres cannot keep. For example, if volunteers request more money for their participation or make other suggestions that conflict with ethical standards and policies or are unaffordable or impossible for the research centre to fulfill, the team should immediately and clearly explain why such changes are not possible.
- In the end, staff should thank the volunteers and community agents for helping the research centre make improvements, and explain how their suggestions will be used and how they might be made aware of the steps taken to improve the services. The timing for follow-up will depend on the problems identified. If the research centre has decided to use the bulletin board to communicate changes, encourage volunteers or community agents to check the bulletin board.

Interpretation of answers and identification of areas to improve

A “no” response to a question on the team self-assessment indicates a need for improvement. In some instances, team members will be divided between “yes” and “no.” At such times, teams should try to build consensus and ask the staff at the station to clarify issues. In general, if there is lack of consensus, consider the question as a problem identified until proved otherwise. If no problems are identified at the station, the QI team should ask questions such as:

- Are there any improvements you have wanted to make in the work here but have not been able to?
- Do you have any concerns that were not raised by the team?
- Is there an area you would like to work on to provide (even) better services?
- Is there an area that you are particularly proud of? How did you accomplish that?

Documentation of outcomes

The visiting QI team makes a note of the problem(s) identified and areas to be improved. When staff members become more proficient, they may develop good problem statements and conduct root-cause analysis as they go about identifying areas to be improved, particularly for less complicated problems. (This is explained in Step 2 of the QI process).

A suggestion for managers and QI facilitators

Managers and QI facilitators should generally leave problem identification to staff. However, if they notice a problem that has not been identified, they should ask open-ended questions rather than point out the problem: e.g. “What do you think about all the blood samples clients see as soon as they arrive in the laboratory? How will they feel?” or “What do you think about the cleanliness of the toilets used by volunteers? How would you feel about using them?” “How do you feel about the way that volunteers coming for CD4 count are seated separately by the receptionist?” Team self-assessment – community

Many of the challenges and problems faced by research centres arise from conditions prevailing in the surrounding communities. Often, community agents learn of volunteers’ concerns related to understanding of the study objectives, confidentiality, quality of services and stigma and discrimination in the community. Community agents can also perpetuate misinformation about the study, and might compromise confidentiality and the safety of volunteers if they have not been well trained. For these reasons, community agents should always be involved in a research centre’s QI initiatives.

Self-assessment at the community level is challenging because of the cultural and social diversity in any given community. Most of the principles that guide team self-assessment at the research centre apply at the community level. However, self-assessment questions at the community level will stress outreach activities, engagement strategies and education tools, as opposed to the process flow from one station to the next at the research centre. Each research centre should determine the benefit of engaging community agents in the team self-assessment, either with community outreach staff or separately. (See tool in Part 2).

Instructions: How to apply team self-assessment at the community level

1. Decide who will facilitate the first exercise. This could be the research centre’s QI facilitator, along with a selected community outreach staff member (e.g., senior community outreach staff, community liaison officer etc.) and representative from management. However, if the CLO is a trained QI facilitator, he or she can serve as the person who regularly conducts self-assessment with other community outreach staff or community agents during routine monthly or quarterly meetings.

2. Review and adapt the self-assessment tool. Adapt the tool for community outreach activities. Select only relevant questions or develop additional questions as needed.

3. Follow steps as described under self-assessment at research centre, where appropriate. Volunteer feedback interview

The Volunteer Feedback Interview (sometimes called “volunteer exit interview” or “feedback form”) is commonly used to explore volunteer perspectives on research centre services. Although it is common for research centres to administer a volunteer exit interview at the end of a study, a volunteer feedback interview is intended to gather feedback while volunteers are participating in the study. This is of value because volunteers’ opinions about their experiences often shape their decisions to participate, continue to participate or advocate for the study or research centre in the community. For better representation of volunteers’ opinions and perspectives, research centres should target potential volunteers, enrolled volunteers and those who have dropped out when possible.

Feedback interviews should include questions that are quantitative and qualitative, closed and open-ended. Although each participant might have a different perspective on services and activities, the feedback interview provides a basis for discovering common trends, and identifying strengths and areas for improvement.

Staff must choose whether the interviews should be conducted by other staff members or an independent QI consultant. Samples of different feedback interviews are included in Part 2. Research centres determine how many feedback interviews are administered, and how often, based on the study schedule, number of targeted volunteers for screening and number of volunteers needed for the study. The QI committee should determine the frequency, target volunteers (potential volunteers or enrolled volunteers) and target number, making sure that the resulting volume of data will not overwhelm the QI team.

Unlike the self-assessment tool, feedback interviews are recommended for use at the research centre only.

Instructions: How to apply the volunteer feedback interview

1. Adapt the feedback interview. The QI committee should select which type of feedback interview to use and determine how it may be adapted to suit the needs of the research centre. The tool should be simplified whenever possible. For example, if a study does not require family planning, the feedback interview might not need to have questions on this topic.

2. Determine the purpose. The rationale for frequency and target number of feedback interviews will depend on each research centre’s needs. Research centres that conduct multiple studies over long time frames (two to three years) might choose to conduct routine feedback interviews (adapting them to different studies or cohorts) while research centres that are conducting single studies for short time frames (12 to 18 months) might choose to conduct one feedback interview at the beginning and another toward the end of the study.

3. Prepare the interviewer. The QI committee should determine whether to use a staff member or an independent QI consultant to conduct the interview, and make appropriate preparations (e.g., space for interview, means of informing staff and volunteers about the exercise, etc.). The QI committee must ensure that the person conducting the interview understands the feedback form, as well as the concerns of the research centre and the social and cultural context in which the interview is being conducted. Preparation could include basic information on counseling processes and available care at the research centre, confidentiality, research centre study processes, principles of QI (purpose, methods and tools), strategies

Box 5.3: Advantages and disadvantages of conducting team self-assessment at the community level

Advantages: Team self-assessment at the community level has many of the same benefits as self-assessment at the research centre—and a few additional ones:

- Community outreach staff and community agents are sensitized about quality and empowered to devise strategies to improve community education, recruitment and engagement through a retrospective assessment of their activities.
- When used with additional stakeholders, community self-assessment can enhance trust and ownership of research and facilitate input on areas for improvement.

Disadvantages:

- Challenges associated with convening people for an exercise that might take significantly longer than a group discussion.
- Difficulty of involving community agents or stakeholders without unduly raising their expectations. for creating rapport and ensuring a sense of safety with volunteers, communication skills and need for standardization in administration of feedback interviews. The interviewer should pre-test the interview with at least four QI committee members before conducting feedback interviews with volunteers.

4. **Conduct the interview.** The interview should take place in a private area. The QI committee needs to ensure that the person conducting the interview carefully explains the purpose of the interview, obtains permission from the volunteer as needed, informs the volunteer that his or her name will not be used, and explains that the volunteer can stop the interview at any time with no repercussions. The interviewer must put the volunteer at ease and listen carefully to what the volunteer has to say, and must document the volunteer’s comments. At the end, the interviewer should thank the volunteer.

5. **Disseminate the data.** The QI committee should design simple tools to analyze data (e.g., Excel-generated spreadsheets) and disseminate data (e.g., Excel-generated or hand drawn graphs, PowerPoint presentations) gathered through the feedback interviews. As with the team self-assessments, the QI committee should convene a staff meeting (or reserve time during a staff meeting) to present the results of the interviews, including problems identified, areas needing improvement in the action plan(s) and areas found to be in compliance with quality standards and meeting volunteers’ satisfaction. This can be done with the dissemination of team self-assessment exercises if a research centre has decided to conduct team self-assessments and feedback interviews at the same time.

6. **Analyze the data.** The data must be studied carefully before problem analysis can take place among staff. The QI committee and staff should also discuss whether there are issues identified in the feedback interview they do not understand. This might include an area identified as a big problem by volunteers (e.g., avoid tribalism, racism, discrimination) or a recurring suggestion that is unclear (e.g., consider the needs of people like us) and explore ways to gain additional insights into those issues. In some cases, the QI committee might recommend the use of additional tools to improve their understanding of issues revealed through the feedback interviews. For example, they can recommend the use of team self-assessment with volunteers in a walk-through at the research centre (as described above), or they can conduct a targeted group discussion (discussed later in this section).

7. **Show appreciation for volunteer feedback and recommendations.** The QI committee should consider the best ways to communicate results to volunteers. Volunteers need to know that their input is appreciated and has been taken into account to improve services. This can be done through “thank you for helping us serve you better” messages at the reception, or by posting findings and actions on a bulletin board or in counseling rooms.

______________________________________________________________________

**Box 5.4: Advantages and disadvantages of the volunteer feedback interview**

**Advantages:**

- Helps research centre staff make improvements that are important to volunteers.
- Leads to improvements that staff might not have thought of on their own.
- Provides a learning experience for staff (when they conduct the interviews) by raising their awareness of volunteer perspectives. This direct experience is often more instructive than being told by an outsider what the volunteers said.

**Disadvantages:**

- Volunteers might not be willing to speak openly—to either an independent QI interviewer or a staff member.
- There is a perception that volunteers might not want to spend the extra time required for an interview. However, at research centres that have conducted feedback interviews, more than 98% of volunteers that were approached were willing to provide feedback.
- Challenges with sustainability if the research centre uses an independent QI interviewer.
- Requisite technical skills for basic quantitative and qualitative data analysis may be hard to find.

______________________________________________________________________

7 Prince Ngongo Bahati et al. Ensuring quality of services in HIV prevention research settings: findings from a multi-centre quality improvement pilot in East Africa. AIDS Care. 2010. Vol 22, issue 1 pp119-125. **Group discussion**

Group discussions allow the research centre to explore the perspectives of volunteers or community agents in more depth. Group discussions for QI purposes are relatively less demanding than a focus group discussion organized for research purposes. Group discussions for QI should seek to address a reasonable number of questions and issues so as to allow sufficient time to explore each in the depth required. Group discussion promotes better understanding of the opinions, preferences and challenges regarding quality of services and experiences at a research centre. For example, research centre staff might learn that there are rumors in the community about the research being done at the centre, and want to learn more about what people are saying. Or the research centre might have received information in the suggestion box (suggestion boxes are discussed later in this section) or from volunteer feedback interviews that needs further analysis. Even if it is not possible or desirable to find the person who provided the suggestion, other volunteers might be able to shed light on the issue. In-depth information helps inform possible interventions and ensures that such measures make the kinds of improvements that volunteers and community agents actually had in mind.

Note that this tool can be used by both research centre staff and community outreach staff to explore the views of volunteers, potential volunteers and community agents (see sample questionnaire in Part 2).

**Considerations before conducting a group discussion**

1. **Select a facilitator and an assistant.** This will often be the research centre’s QI facilitator. A staff member who can assist the facilitator by taking notes, keeping track of time and helping to manage the group should also be identified. Some research centres have obtained better outcomes (based on volunteer assessments) when they use an outsider to facilitate group discussions. Consider using an internal staff member who does not professionally interact with volunteers (e.g., a data clerk, pharmacist, etc.) to conduct the discussions.

2. **Determine the target group.** Make sure the participants are likely to be comfortable with each other. To facilitate open dialogue, the research centre might want to avoid mixing people from different cohorts, such as, sex workers, discordant couples or men who have sex with men. The centre may also choose to avoid mixing men and women, or young and old participants in group discussions, especially if they have reason to believe that a diverse spectrum of gender identities, sexual orientations and ages is likely to impede open discussion.

3. **Ensure participation is voluntary.** When selecting participants, emphasize that participation is voluntary and will not affect a volunteer’s involvement or potential to be involved in a study.

4. **Select an appropriate group size.** Generally, the group should include a minimum of six and a maximum of 15 people to ensure that everyone has the opportunity to participate in the discussion in a meaningful way. Research centres conducting small studies can convene discussion groups with smaller numbers of volunteers or community agents when appropriate.

5. **Prepare questions.** Prepare appropriate questions to obtain the information required (see suggestions in Part 2 and adapt as needed).

   **Note:** The questions developed for group discussion can also be used in an informal, non-structured one-on-one interview by staff if confidentiality is an issue, or if it is difficult to convene a group.

______________________________________________________________________

**Instructions: How to conduct group discussions**

01. **Welcome and introductions.** Welcome the group, introduce the facilitator and any assistant(s), make other introductions as desired (use first names only), and explain the purpose of the discussion.

02. **Ensure participation is voluntary.** Emphasize that participation is voluntary and that anyone is free to leave at any time without consequence.

03. **Emphasize confidentiality.** Assure participants that their names will not be used in the notes. The notes help document the suggestions and opinions that will guide the research centre in improving its services.

04. **Warm-up.** As a warm-up, consider introducing the volunteers’ rights and responsibilities framework and ask participants their opinions about it. Alternatively, use any other warm-up of your choice.

05. **Create ground rules.** Encourage the group to develop ground rules for the discussion, and make sure to stress the importance of respecting confidentiality and the opinions of other participants. Other rules might include, for example, not interrupting one another, etc.

06. **Listen carefully.** Listen to the participants’ responses and to the discussion as it develops.

07. **Delve deeper.** Ask open-ended questions to obtain more information (such as “what do you think about...?” or “how do you think this problem can be solved...?” or “how does that make you feel...?” “what other reasons are there for...?”).

08. **Moderate responses.** Ensure that everyone has a turn to speak if they want to, and try to avoid having one person dominate the discussion by asking others questions directly or setting the stage for others to talk (“Let’s see what the others think about that” or “Akinyi, what is your opinion on...?”).

09. **Enforce the rules.** If conflict arises, encourage the group to help enforce the rules established by the group.

10. **Avoid arguments.** Do not argue with the participants. If necessary, make sure that someone from the research centre with technical competence is present to correct misinformation.

11. **Do not make promises.** Do not make promises the research centre cannot keep, and correct unrealistic expectations among the participants about the outcomes of the discussion.

12. **Thank the group.** At the end of the discussion, thank the group for participating and for their openness.

13. **Communicate results.** Explain how you will communicate results from the discussion, if appropriate, and whether interventions will follow as a result of the discussion.

**Different options to consider for group discussions**

- Discussions can be planned for specific groups of volunteers. For example, if the research centre would like to recruit more young women, outreach workers could approach specific enrolled volunteers or young women in the community and invite them to a group discussion on a particular topic, such as the quality of the information and educational materials provided by the centre.

- Group discussions can also be organized around recruitment activities, information sessions or other events as long as the purpose of the discussion is clear and the facilitator is prepared. For example, during a community gathering, outreach staff might ask a few people to stay after the event (consider providing incentives such as tea and refreshments or additional items that research centres use, e.g. condoms, packets of water-based lubricant, T-shirts or pens with HIV-prevention messages, etc.). Outreach staff can then start a group discussion about the community education session the volunteers just attended, as well as potential misinformation and community concerns.

- Counselors can be asked to identify volunteers with a particular profile and ask them to join a discussion group planned for later that day. Community agents can also be involved in group discussions after community education or recruitment events if appropriate.

**Box 5.5: Advantages and disadvantages of the group discussion**

**Advantages:**

- People feel safer and more willing to open up when they are in a group.
- Respondents often react to other participants and share ideas and observations that they might not have had in a one-on-one interview.
- Provides in-depth information and an opportunity for immediate clarification.
- The experience often raises the awareness of staff.

**Disadvantages:**

- Requires a skilled facilitator to maintain order and make it a pleasant experience for all.
- Might need to schedule a special visit for volunteers or have community agents travel to participate in the group discussion.
- Requires careful consideration if there is an issue of confidentiality for the target group. **Suggestion boxes**

Many research centres use suggestion boxes to obtain ideas from volunteers and staff.

**Instructions: How to use suggestion boxes**

1. Place the suggestion box in an area outside the view of those in the waiting room.
2. Ensure that there is always a pencil and paper by the suggestion box.
3. Encourage people to provide suggestions by having staff indicate the option is available.
4. Encourage staff to use the suggestion box between QI exercises if they do not want to make a suggestion in front of others.
5. Designate a staff member to collect and review the suggestions regularly.

**Box 5.6: Advantages and disadvantages of using suggestion boxes**

**Advantages:**

- Requires minimal staff time.
- Easy to implement.
- Conveys to volunteers and staff that the research centre values feedback.

**Disadvantages:**

- Response rate is often low.
- Comments received are varied and not focused on a specific area.
- It can be difficult to interpret suggestions and target responses if the same suggestion box is used by staff and volunteers.
- Requires a cohort of literate volunteers to be effective. Step 2: Analyzing areas to be improved/problems to be solved

Once these tools have been used to identify problems or areas to be improved, the QI facilitator, with the help of the QI committee, schedules a meeting at the research centre to analyze the findings and determine root causes of identified problems. In preparation for this meeting, the QI team(s) should develop a list of these problems. They should seek agreement among team members on whether these are truly problems. If there is disagreement, they need to consider why there is disagreement, and how to proceed:

- Is there disagreement between the findings from different tools? (For research centres that use multiple tools or tools at multiple levels)
- Is the issue a difference of perspective between the volunteer and research centre staff?
- Is more information needed?

For research centres that have multiple QI teams, a comparison of findings made by the different teams might clarify a disagreement or confirm the problem. The list of problems might be long, which can discourage staff, so it might be necessary to prioritize items in the action plan (the next step of the QI process).

Using “multiple whys”

During the first exercise, research centre and community outreach staff will probably need help in formulating good problem statements and executing root-cause analyses. In this manual we will use a simple technique called “multiple whys” or “three whys.” If the team runs out of answers to “why” questions, they should ask “what other reasons are there?” (see Box 5.7). Stop if the responses touch on factors that are beyond the control of the research centre. Once problems have been identified and prioritized, and their root causes identified (both those that are within the control of research centre and those that are not), the next step is to develop an action plan.

Box 5.7: Formulation of problem statements and the use of multiple whys

Example

Group discussion: Discovery: In a recent group discussion, several volunteers expressed that they are too embarrassed to pick up condoms in the reception area, where everyone can see them.

Potential statement: Most (10 of 15) volunteers at the group discussion stated that they are too embarrassed to pick up condoms in the reception area, where everyone can see them.

Why do most volunteers and potential volunteers not want to pick up condoms in the waiting room? Because they do not have privacy and are embarrassed.

Why do they not have privacy? Because staff did not identify a private area for condom pick-up.

Why didn’t they select a private area? Because the research centre does not have much space to allocate as private areas.

What other reasons are there? Other staff (e.g., cleaners, other counselors) were not involved in the decision about the choice of the distribution area. Counselors do not extensively explore barriers to the use of condoms when they conduct counseling on condom use.

This root cause opens the door for possible solutions that are feasible and inexpensive and over which the research centre has control.

Why else might volunteers feel embarrassed about picking up condoms in the waiting room? Because of the stigma attached to condoms in our society.

Why? Because men use them with sex workers, not with their wives.

This root cause is more difficult for the research centre to take action on and see immediate results.

Continue formulating problem statements and “multiple whys” until you have one or more root causes where interventions are feasible for the research centre or outreach workers and where they are likely to see results. For more examples of how QI discovery can be used to practice formulation of a problem statement and multiple whys, see Facilitator’s Guide; Handout #13. Step 3: Developing an action plan

An action plan documents a problem and the steps that will be taken to improve the issue and keep it from recurring. The plan should be written, and templates are provided later in this section. Developing an action plan entails:

1. Identifying issues discovered in Step 2 of the QI process.
2. Formulating and selecting appropriate actions.
3. Assigning actions to appropriate staff members.
4. Determining a date by which the action will be completed.

Preparing for the action plan meeting

1. A small centre can choose to have one or several meetings for all staff. A larger facility might have this activity take place within each department, with assistance from the QI facilitator.
2. Decide on an action plan format (see samples at the end of this section). Make sure there is a person assigned to implement each remedy (distribute responsibilities fairly) and a date by which the required steps are to be completed.
3. Discuss plans to evaluate the outcome.
4. Discuss how the QI committee will monitor progress. In larger centres, the committee needs to work closely with each department and delegate some of this responsibility.
5. Inform staff when you plan to do the next QI exercise.

The action plan can be any format, but it should include the components mentioned above. If desired, each problem can be on one or several sheets of paper and placed in a binder that all staff members or each department can access. It can also contain an outcome section so that the research centre can keep track of progress and assist with monitoring. See Boxes 5.8 and 5.9 for suggested formats. For the purpose of illustration, the problem related to insufficient privacy for condom pick-up presented earlier is used in these samples. The research centre should select and adapt the format that best fits their needs.

Designing and selecting appropriate interventions

Much consultation will go into the design of most interventions. The staff likely to be affected by the changes must be included in the design and development of action plans, because their input is typically invaluable to the formulation of practical and effective solutions. For example, if volunteers express concerns about the ability of receptionists to maintain confidentiality, receptionists need to be involved in analyzing the problem and developing a solution.

Note: Interventions should be designed to not only solve a problem but to prevent that problem from recurring.

Many effective interventions can be implemented at little or no cost and with minimal effort. In fact, the problem in Box 5.7 can be solved, at least partly, by leaving condoms in bathrooms, where volunteers can pick them up without being observed.

The problem of a lack of privacy in condom access could also lead to changes in counseling: counselors might emphasize to all clients that condoms are not exclusively for use with sex workers and for heterosexual men only, but are to be used as a precaution by all. Counselors might additionally decide to explore barriers to condom use and provide the condoms directly to volunteers during their sessions. Another staff member, perhaps a member of the cleaning crew, might assume responsibility for checking and refilling condom containers in the bathrooms. The problem could also require changing the information provided to the community to reduce the stigma associated with condoms. These interventions might not require any new skill sets, training or infrastructure.

Avoid selecting and developing interventions that are more expensive than necessary. Examples include use of training when it is not appropriate, changes to infrastructure when the process can be changed at less cost, etc. Research centre managers should, however, be supportive when there is a genuine need for additional resources.

Sound analyses and appropriate interventions help avoid unnecessary expense and waste. Keep in mind that the goal is to develop the most effective, least costly and least labor-intensive solutions. If there are many problems, set priorities and a realistic timeline for addressing them. Identifying a few “quick wins”—simple changes that can be made quickly at low or no cost—is a good way to start.

If staff members at a specific station are not able to address problems that have been identified, flag them as areas to be improved and ask medical staff, counselors, laboratory technicians, pharmacists or other experts with relevant technical competence to work on the issue.

Although research centres have limited control over what happens in the communities around them, interventions at the community level are extremely important. Outreach workers should take care to establish creative partnerships for specific QI interventions and build enduring relationships with appropriately selected community groups and organizations. Keep in mind that it can take time to see results from community-level interventions.

Sometimes interventions will need to be revised due to unacceptably high costs or other considerations. Don’t rush the process.

The design of some interventions will continue to evolve even after research centre staff have developed an action plan. Box 5.8: Sample action plan format 1

**ACTION PLAN**

**Date problem was identified:** 1 September 2011

**Problem identified:** Most volunteers at the group discussion stated that they are too embarrassed to pick up condoms in the reception area of the research centre.

**Root cause 1:** Staff did not provide private areas to make condoms available.

**Proposed intervention:** W.C. (QI facilitator) will identify private areas within the research centre by 15 September 2011. T.K. (cleaner) will ensure there are condoms (both male and female) in bathrooms and in a visible area in counseling rooms by 30 September 2011. Management will ensure adequate supply of female condoms from the ministry of health (ongoing).

**Actually completed:** 15 September 2011

**Root cause 2:** Counselors do not routinely explore barriers to condom use with volunteers during counseling sessions.

**Proposed intervention:** J.O. and R.Z. (counselor supervisors) will conduct a special session on condoms (male condoms, female condoms, condom use by men who have sex with men) during the next monthly counseling supervision, to be scheduled by 30 September 2011.

**Actually completed:** 30 September 2011

**Root cause 3:** There is stigma attached to condoms in the community because people think men use condoms for sex with sex workers. Most men who have sex with men think condoms are for sex between a man and a woman, not between two men. Female condoms are not always available and women are embarrassed to take them.

**Proposed intervention:** 1) R.O. (nurse counselor) will work with the other counselors to include specific, culturally sensitive discussion guides on condom use in counseling sessions (by 25 October 2011, after next month’s counseling supervision).

**Actually completed:** 30 October 2011

**Proposed intervention:** 2) S.K. and L.H. (nurse counselor and outreach worker supervisor) will guide outreach workers to be more specific when they discuss condom use for HIV prevention in the community (by 15 October 2011).

**Actually completed:** 8 October 2011

**Proposed intervention:** 3) M.T. (trial coordinator) and S.K. (community liaison officer) will review the posters on condoms used by the research centre to identify parts that may be confusing, evaluate whether the posters are appropriate for the newly developed cohort (men who have sex with men) and propose suggestions for improvement (by 25 October 2011).

**Actually completed:** 30 October 2011

**Intervention outcomes:** The issue of privacy was discussed at the staff meeting, and the discussion continued among staff who decided to take other actions. T.K. worked with the pharmacist to find containers in the storage area, filled them with condoms and placed them in each counseling room and bathroom. After the condoms ran out in the counseling rooms a few times, he decided the containers need to be checked daily and refilled as needed. Counselors state that they now always have condoms in the counseling rooms. More female condoms are being taken by the volunteers (T.K. estimates the use of three boxes of 200 condoms a week and recommends ordering at least 20 boxes a month).

R.O. has facilitated role-playing with the other counselors and has developed a script for the counselors to use to ensure that they are specific in their information about condoms in order to decrease the stigma associated with their use.

S.K. and L.H. held a session with the outreach workers to address stigma. They have made some additions (e.g., condoms and anal sex) to what they say about condoms, according to L.H., who has accompanied several outreach workers when they organize information sessions in the community.

M.T. and S.K. have presented suggestions to management for making current posters MSM-friendly while ensuring that volunteers’ safety is not compromised and that the material conforms with the law of the land.

### Box 5.9: Sample action plan format 2

| Problem | Cause(s) | Recommended intervention | By whom | By when | Date completed | |------------------------------------------------------------------------|--------------------------------------------------------------------------|------------------------------------------------------------------------------------------|------------------------------|---------------|----------------| | Most volunteers at the group discussion stated that they are too embarrassed to pick up condoms in the reception area of the research centre, where everyone can see them | No extensive consultation on private areas was conducted among staff before deciding on the reception area | Discuss importance of privacy at next staff meeting, and identify private areas where condoms can be distributed | W.C. (QI facilitator) | 02 June 2011 | 10 July 2011 | | Counselors do not extensively and routinely explore barriers to condom use with volunteers | | Ensure there are condoms in counseling rooms and bathrooms | T.K. (cleaner) | 05 June 2011 | 05 June 2011 | | Stigma attached to condoms in the community | | Facilitate a session on condom use during monthly counseling support supervision | J.O. and R.R. (counselor supervisors) | 25 June 2011 | 20 June 2011 | | Posters on condom use received from the Ministry of Health do not address specific needs of men who have sex with men | | Develop a counseling guide on condom use | R.Z. (nurse counselor) | 30 June 2011 | 30 June 2011 | | | | Provide outreach workers with guidance on being more specific when they discuss condom use for HIV prevention | S.K. & L.H. (nurse counselor & outreach worker supervisor) | 25 June 2011 | 30 June 2011 | | | | Review posters and propose changes | M.T. and S.K. | 05 June 2011 | June 10 2011 |

______________________________________________________________________

8 The COPE® process uses this action plan format and recommends that the action plan should be on a wall for all to see. See earlier reference to COPE® in the Introduction and Section I. Step 4: Implementing the action plan

QI-related interventions are similar to other interventions in terms of implementation, monitoring and evaluation. What is different with a QI action plan is that more staff are involved and assigned responsibilities. Implementation of the action plan takes time, and the QI team should convene periodically to monitor implementation and identify additional problems and interventions.

During monitoring, consider these questions at regular intervals:

- Is the problem being solved? If not, why not?
- Did the centre select the correct interventions? Are additional interventions needed?
- Does the person responsible need help? If yes, what are the results and how are they being recorded?
- Have any new problems developed as a result of the interventions?

Many issues require attention as implementation proceeds and QI is institutionalized. Everything will not and cannot be settled immediately. In a large centre, the QI committee must work closely with department heads to ensure that they are involved and supportive during implementation. In small and large centres, QI committee members must have the flexibility and time to take on their new responsibilities.

Measuring improvement

Organizations often try to measure too much, which takes time and can be costly. If the indicators selected are of questionable value, the research centres will waste time and resources that could be spent on more useful activities. It is, however, very important to examine trends as more QI exercises are implemented to ensure that areas needing improvement have been dealt with and that action plans were effective.

For this reason, after careful consideration pick a few good indicators and be open to the possibility of combining or complementing quantifiable indicators with other qualitative issues. Box 5.10 presents possible indicators. Some might be collected through existing data as part of research protocol (e.g., behavioral or medical data). If so, it is not necessary to collect them again with the QI tools described in this kit. Nevertheless, such data can inform the QI process and confirm the findings of the exercises.

Quantitative indicators

The indicators on the next page focus on examples of outcomes associated with quality HIV-prevention research. Some research centres might choose to ask specifically about satisfaction with services. While they do not measure volunteer satisfaction or cause-effect relationships, they provide insight into volunteers’ compliance to the goals and objectives of the research.

However, the concept of quality changes over time, and research centres will need to adjust their action plans continually to meet the changing needs of volunteers within available resources.

The indicators in Box 5.10 represent just some of the examples of ways research centres can measure success of quality-improvement programs. Box 5.10: Quantitative indicators

Collecting and analyzing data can be time-consuming. For this reason, it is a good idea to keep the number of quantitative indicators to a minimum. An indicator is generally expressed as a number or percentage. The indicators selected should be in an area that is important and where performance is relatively low.

A) Examples of measurable indicators that can be collected from volunteers through feedback interviews:

- % of volunteers enrolled (or members within the selected volunteers in group discussion) who can report that they understand study objectives or % of volunteers who have correctly described the study objectives
- % of volunteers enrolled (or of selected volunteers in a group discussion) who report receiving counseling on condom use and being offered condoms at each visit
- % of volunteers enrolled who state they have no concerns about confidentiality at the research centre
- % of volunteers enrolled who state they are satisfied with privacy during medical exam, counseling or both
- % of volunteers enrolled who state that a research centre staff member has revisited their consent to participate at least once after enrollment
- % of volunteers enrolled who state that they were reminded of “unknown efficacy” of the investigational product or HIV-prevention tool
- % of volunteers enrolled (or of selected volunteers in a group discussion) who state their medical complaints have been promptly and effectively attended
- % of volunteers enrolled (or of selected volunteers in a group discussion) who report that they have not been discriminated against by research centre staff based on race, social status, sexual orientation, behavior or profession
- % of volunteers enrolled (or of selected volunteers in a group discussion) who report that they are given adequate opportunity to ask questions and report concerns about their participation

B) Examples of quality indicators that can be collected through other research centre study tools/forms:

- % of female volunteers using and adhering to an effective method of family planning (in research where this is a requirement)
- % of female volunteers dropping out because of pregnancy (if this incident happens relatively frequently)
- % of volunteers who adhere to prescribed regimen related to the investigational product (e.g. PrEP, microbicides) or drugs prescribed by trial physicians
- % of volunteers provided with referrals who used those referrals
- % of volunteers who remain in the study until the end (retention rate)

There are many other possible indicators. It is up to each research centre to choose which ones they prefer to use. Means to complement these quantifiable indicators with qualitative observations are explored in the tools (group discussion and feedback interviews). Final considerations on the QI process

How often should QI exercises take place? It is important not to lose momentum after a QI exercise, and to steadily implement the action plan and monitor its progress. It is also important to add problems identified between QI exercises to the action plan. When many of the problems in the action plan have been solved, it might be time for the next QI exercise. QI exercises should be organized regularly but should be planned in accordance with study schedules and in sync with volunteers’ scheduled visits. HIV-prevention research centres that have conducted QI recommend that exercises be conducted at least twice a year. With time and experience, some research centres might consider conducting quarterly follow-up QI exercises, as these are shorter than the introductory QI exercises.

How do the research centres sustain the QI process? Some organizations have initiated QI only to see their efforts wane. But that can be avoided. Above all, managers must prioritize QI and involve all levels of staff in the exercise. Sponsors of HIV-prevention research must prioritize QI and provide the support necessary for research centres to conduct QI. Other issues are reviewed below. Research centre managers and staff should keep in mind that they “own” the QI process, and that these tools are theirs to use and further adapt and develop. The benefits of QI extend far beyond present research partners or sponsors. However, the research centre might want to discuss the problems identified and share the challenges and results of their QI efforts with partners and sponsors. This can enable sponsors and partners to provide more targeted support for the improvement efforts.

Communication about progress and challenges is important to keep employees involved and committed. Use regular staff meetings, call additional meetings if needed for all staff or for a department or staff members who are particularly involved, and meet with employees one-on-one as needed. If the research centre has other ways of communicating with staff, they should be used.

Action plan, monitoring and evaluation. Continuous monitoring of the action plan is important. Action plans are not useful unless they become truly important documents that guide the work done at the research centre. Action plans should be accessible to research centre staff (either posted on a wall or in a binder available for staff to consult) and should be incorporated into workplans and budgets as necessary.

Celebrate good results. Celebrate improvements at the research centre and better services for the volunteers. This enhances team spirit and motivation. Celebrations do not need to be elaborate. Small acts of appreciation usually go a long way if they are sincere and done at the right time. For example, give staff 20 minutes to get together over a cup of tea at the end of a staff meeting, or a similar activity that is feasible and that managers think their staff will appreciate.

Sample QI schedule Below is a sample timeline for conducting a QI cycle. Keep in mind that the timeline is flexible, and that this is just one way of scheduling the process.

| Activity | Week 1 | Week 2 | Weeks 3-11 | Week 12 | Week 13 | Week 14 | Weeks 15-22 | Week 24 | |-----------------------------------------------|--------|--------|------------|---------|---------|---------|-------------|---------| | Introduction meeting | | | | | | | | | | Team self-assessment | | | | | | | | | | Problem analysis & action plan meeting | | | | | | | | | | Implement interventions | | | | | | | | | | Monitoring meeting & plan use of next tool | | | | | | | | | | Volunteer feedback interview/group discussion/team self-assessment | | | | | | | | | | Problem analysis & action plan meeting | | | | | | | | | | Implement interventions | | | | | | | | | | Monitor meeting & plan use of next tool | | | | | | | | | REFERENCES AND RESOURCES

International AIDS Vaccine Initiative’s AIDS Vaccine Literacy Training Manual http://www.iavi.org/publications-resources/pages/publicationdetail.aspx?pubid=2e2d5419-050b-4814-a135-f9b0d20a5b56

EngenderHealth. 2003. COPE® Handbook: A Process for Improving Quality in Health Services, Revised Edition. EngenderHealth: New York, United States (This is a series that comes with different tools related to different areas of reproductive health, including HIV/AIDS, many of which may be relevant to research centres. The materials can be downloaded for free at http://www.engenderhealth.org/files/pubs/qi/handbook/cope_handbook-a.pdf )

EngenderHealth. 1999 and 2001. Facilitative Supervision Handbook. EngenderHealth: New York, United States. http://www.engenderhealth.org/pubs/quality/facilitative-supervision-handbook.php

For all the COPE tools and other quality-related resources, see: http://www.engenderhealth.org/pubs/quality/index.php

UNAIDS/AVAC Good Participatory Practice Guidelines (GPP). http://www.avac.org/ht/d/sp/i/304/pid/304

International Conference on Harmonization. Guidelines for Good Clinical Practice (GCP) http://ichgcp.net/LOB/media/MEDIA482.pdf

World Health Organization. Good Clinical Laboratory Practice (GCLP). http://apps.who.int/tdr/publications/tdr-research-publications/gclp-web/pdf/gclp-web.pdf

HIV Vaccine Trial Network “Participants’ Bill of Rights and Responsibilities.” http://www.hvtn.org/community/rights.html

NYU Medical Center Research Participants’ Bill of Rights. http://www.med.nyu.edu/patients-visitors/clinical-services/clinical-trial-info/participant-bill-rights

UNAIDS/WHO guidance document “Ethical Considerations in Biomedical HIV Prevention Trials”. http://data.unaids.org/pub/Manual/2007/jc1349_ethics_2_11_07_en.pdf

USAID supports many different organization and issues to improve quality. For example: www.maqweb.org http://www.infoforhealth.org/pr/q02/index.shtml http://www.infoforhealth.org/pr/q01/index.shtml http://www.prime2.org/prime2/section/53.html http://www.hciproject.org QUALITY IMPROVEMENT IN HIV-PREVENTION RESEARCH A rights-based and participatory approach

Part 2: Tools

# TABLE OF CONTENTS

Sample 1.a: Volunteer Feedback Interview: for volunteers enrolled in a clinical trial ........................................ 2\
Sample 1.b: Volunteer Feedback Interview: for volunteers enrolled in an epidemiology study .......................... 5\
Sample 1.c: Volunteer Feedback Interview: for use during recruitment and screening visits .......................... 8\
Sample 2.a: Team Self-Assessment: for use at research centre level ................................................................. 10\
Sample 2.b: Team Self-Assessment: for use at community level ........................................................................ 16\
Sample 3: Group Discussions: possible questions/topics for volunteers and community agents .................. 18 Sample 1.a Volunteer feedback interview: for volunteers enrolled in a clinical trial

**Date:** / /\
**Time:**\
**Site:**\
**Couple**\
**Indiv.**\
**Sex:** F M

**Explain:** We are conducting a survey to find out what volunteers think about our services. Your honest responses will help us improve the quality of our research services. It will take about 15-20 minutes. Your name and responses will be kept strictly confidential. If you choose not to participate, this will have no effect on the services you receive at the centre. Do you agree to participate? (Agreement is required to proceed). Thank you for your openness and time.

### Study understanding

1. How would you describe the kind of information you received before joining this study?

   - Adequate [ ]
   - Inadequate [ ]

   In your own words, what is this study about?

### During today’s visit, did the counselor discuss the following issues with you?

2. How to reduce risks of contracting HIV

   - Yes [ ]
   - No [ ]
   - NA [ ]

3. That the product under investigation may not protect you and that you need to use condoms and/or other safe sex practices

   - Yes [ ]
   - No [ ]
   - NA [ ]

4. Possible side effects/benefits that may be associated with your participation in the trial

   - Yes [ ]
   - No [ ]
   - NA [ ]

5. How to avoid pregnancy during the trial (when applicable)

   - Yes [ ]
   - No [ ]
   - NA [ ]

6. Your right to continue or discontinue participation in the study at any time without negative consequences

   - Yes [ ]
   - No [ ]
   - NA [ ]

7. Blood drawing test(s) and what each of the result(s) means for you

   - Yes [ ]
   - No [ ]
   - NA [ ]

8. Concerns you may have about your visit or participation in the study

   - Yes [ ]
   - No [ ]
   - NA [ ]

### Doctors or trial physicians

09. Did the doctor explain to you the purpose of each test/examination conducted today, and what each of the results might mean to you?

    - Yes [ ]
    - No [ ]
    - NA [ ]

10. Did you understand the doctor’s advice on health-related matters, and the purpose of each test conducted today?

    - Yes [ ]
    - No [ ]
    - NA [ ]

11. Did you understand the information you received from other staff members (pharmacists, laboratory personnel, etc.)?

    - Yes [ ]
    - No [ ]
    - NA [ ]

If no, what did you not understand? 12. The length of today’s visit was:

- Just right [ ]
- Too long [ ]
- Too short [ ]

During today’s visit, did the counselor or other staff members:

13. Offer you condoms?

- Yes [ ]
- No [ ]
- NA [ ]

14. Offer you treatment and/or family planning options that you require?

- Yes [ ]
- No [ ]
- NA [ ]

15. Provide you with a referral for services that you need but that are not offered at the research centre?

- Yes [ ]
- No [ ]
- NA [ ]

How long did you wait to see the following personnel?

16. The receptionist

- \<5mins [ ]
- 5-10mins [ ]
- 10-15mins [ ]
- > 15mins [ ]

17. The nurse/counselor

- \<5mins [ ]
- 5-10mins [ ]
- 10-15mins [ ]
- > 15mins [ ]

18. The doctor

- \<5mins [ ]
- 5-10mins [ ]
- 10-15mins [ ]
- > 15mins [ ]

19. Laboratory staff

- \<5mins [ ]
- 5-10mins [ ]
- 10-15mins [ ]
- > 15mins [ ]

20. The pharmacist

- \<5mins [ ]
- 5-10mins [ ]
- 10-15mins [ ]
- > 15mins [ ]

Do you feel you were treated respectfully throughout your visit by the following:

21. Security guards?

- Yes [ ]
- No [ ]
- NA [ ]

22. Receptionists?

- Yes [ ]
- No [ ]
- NA [ ]

23. The nurse/counselor?

- Yes [ ]
- No [ ]
- NA [ ]

24. The doctor?

- Yes [ ]
- No [ ]
- NA [ ]

25. The pharmacist?

- Yes [ ]
- No [ ]
- NA [ ]

26. Laboratory staff?

- Yes [ ]
- No [ ]
- NA [ ]

If no, please explain:

Privacy and confidentiality

27. Do you feel you had enough privacy at every stage of your visit?

- Yes [ ]
- No [ ]

If no, please explain:

28. Do you trust that you have complete confidentiality at the research centre (that only the study staff will have access to the information you provide)?

- Yes [ ]
- No [ ] If no, please explain:

| Satisfaction | |--------------| | 29. Will you return for your next visit? | Yes [ ] No [ ] Maybe [ ] | | If no or maybe, please explain: |

| 30. Will you recommend joining the study to others? | Yes [ ] No [ ] Maybe [ ] | | If no or maybe, please explain: |

| 31. Overall what do you think about the services you received today? | Very good [ ] Good [ ] Poor [ ] | | 32. What did you think of the quality of services provided by community mobilizers/workers? | Very good [ ] Good [ ] Poor [ ] |

33. If there is one improvement you would like to see at the research centre, what would it be?

Thank you very much for your time and participation!

Date administered: \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ by:\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Sample 1.b Volunteer feedback interview: for volunteers enrolled in an epidemiology study (not involving administration of an investigational product)

Instructions for interviewer: Interviewer should complete this questionnaire based on the volunteer’s answers. It should not be self-administered.

Explain: We are conducting a survey to find out what volunteers think about our services. Your honest responses will help us improve the quality of our research services. It will take about 15-30 minutes. Your name and responses will be kept strictly confidential. If you choose not to participate, this will have no effect on the services you receive at the centre. Do you agree to participate? (Agreement is required to proceed). Thank you for your openness and time.

PLEASE CIRCLE ONE CHOICE UNLESS INDICATED OTHERWISE

| First Visit | Made 2-4 visits | Made more than 4 visits | |-------------|-----------------|-------------------------| | 1. How long did you wait before seeing the receptionist? | | | | Less than 5 minutes | 10-15 minutes | | | 5-10 minutes | More than 15 minutes | | | Waiting time was: | | | | Just right | Too long | | | 2. How long did you wait before receiving counseling services? | | | | Less than 5 minutes | More than 30 minutes but less than 1 hr. | | | 10-30 minutes | More than 1 hour | | | Waiting time was: | | | | Just right | Too long | | | 3. Do you feel that staff at our clinic will keep the issues you discussed during your visit confidential? | | | | Yes | No | | | If no, please comment: | | | 4. Do you feel that you had enough privacy during your visit? During counseling: Yes No During physical exam: Yes No When (if) you picked up condoms: Yes No If no, please comment:

05. Do you feel that all staff members treated you respectfully throughout your visit? Yes No If no, please comment:

06. Did you find staff at the reception desk welcoming and helpful during your visit? Yes No If no, please comment:

07. Did you receive the following during the visit? PLEASE CIRCLE ITEMS RECEIVED TODAY Condoms Lubricants Referral cards (to invite others) Medications Referral to other services Other – please note:

08. Did the counselor discuss HIV-prevention options during the visit? Yes No If yes, please list which ones:

09. Is there information you would have liked to receive today that you did not receive? Yes No If yes, please explain:

10. Did you receive a schedule of follow-up visits from the counselor or other staff (if applicable)? Yes No

11. Did the counselor explain the schedule of visits properly and clearly? Yes No If no, please explain:

12. Do you think you will come back for your scheduled follow-up visits? Likely Unlikely Undecided If unlikely or undecided, please explain the reasons:

13. Will you recommend the research centre to your family members and friends? Yes No Please explain:

14. Do you have any further comments or suggestions that may help us improve our services?

Date administered: \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ by:\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Sample 1.c Volunteer feedback interview: for use during recruitment and screening visits

Instructions for interviewer: Interviewer will complete questionnaire based on volunteer’s answers. It should not be self-administered.

Explain: We are conducting a survey to find out what potential volunteers think about our services. Your honest responses will help us improve the quality of our research services. It will take about 15-20 minutes. Your name and responses will be kept strictly confidential. If you choose not to participate in this survey, this will have no effect on the services you receive at the centre or your participation in the study. Do you agree to participate? (Agreement is required to proceed). Thank you for your openness and time.

1. How long did your visit take today?

   PLEASE CIRCLE ONE (change time if necessary)

   Less than 1 hour 1 – 2 hours 2-3 hours More than 3 hours

   Length of the visit: Just right Too long Too short

2. Did you find staff at the reception desk welcoming and helpful during your visit today?

   Yes No

   If no, please comment:

3. Do you feel that all staff members treated you respectfully throughout your visit today?

   Yes No

   If no, please comment:

4. How would you describe the information that you received about the study today?

   Adequate Not adequate

   If not adequate, please comment why:

5. In your own words, what is this study about?

6. Do you feel that the research centre gave you all the information and the time required to make an informed decision about whether to participate?

Yes No

Please explain:

7. What is your main motivation to participate/not to participate in this study/trial?

8. Do you feel that staff at our clinic will keep the issues you discussed during your visit confidential?

Yes No

If no, please comment:

9. Do you feel that you had enough privacy during your visit today?

Yes No

If no, please comment:

10. Did you receive services today that were useful to you?

Yes No

Please explain:

11. What, if anything, could the research centre do to improve its services?

Please explain:

12. Would you recommend participation in the study to family and friends?

Yes No

Date administered: \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ by:\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Sample 2.a Team self-assessment at the research centre level

Instructions: The team(s) responds to the questions. Circle “Y” for “yes” and “N” for “no.” A “no” answer indicates there is a possible problem.

General questions for all staff

01. Have all staff members (including security guards, drivers, cleaners and administrators) been sensitized to volunteers’ rights and responsibilities and the ethical conduct of HIV-prevention research? ......................... Y / N

02. Have all staff members (including security guards, drivers, cleaners and administrators) received instruction on the basics of HIV-prevention research? ................................................................. Y / N

03. Do we have the Volunteers’ Rights and Responsibilities posted throughout the research centre? ............ Y / N

04. Do we effectively engage community stakeholders in the design of the study/trial during implementation and after research is completed? ................................................................. Y / N

05. Are our current advisory mechanisms (community advisory boards [CABs] and other) independent and representative of the community? ................................................................. Y / N

06. Do we engage stakeholders in determining policies and practices on HIV prevention and care, non-related HIV care, and research-related harm? ................................................................. Y / N

07. Do we treat all volunteers respectfully at all times? ........................................................................ Y / N

08. Do we attend to volunteers as soon as we can and minimize waiting times? .................................. Y / N

09. Do we provide to volunteers and community members correct, clear and easy-to-understand information about the objectives of the research and the risks, benefits and process of participation? ................................................................. Y / N

10. Do we communicate our policy on research related harm to volunteers and stakeholders? ................. Y / N

11. Does our informed consent process ensure that participants have understood all that is explained to them and that their consent is genuinely voluntary? ................................................................. Y / N

12. Do we evaluate the informed consent throughout the course of the volunteer’s participation? ............ Y / N

13. Do we pilot-test our IEC materials, informed consent materials etc., before using them with volunteers? .... Y / N

14. Do we ensure that all areas of the research centre are safe and clean for volunteers and staff? ............ Y / N

15. Do we listen to volunteers and encourage them to ask questions? ................................................................. Y / N

16. Do we treat volunteers respectfully and non-judgmentally regardless of their choices, values, beliefs and practices? ........................................................................................................ Y / N

17. Do we ensure that the privacy of volunteers is adequately protected throughout their visit? ................. Y / N

18. Do we ensure that the volunteers’ personal information is kept confidential at all times? ......................... Y / N

19. Are all clinic and laboratory staff trained (including cleaning personnel and those who dispose of medical waste) in infection-prevention procedures, so that we are able to recognize the risks associated with our respective activities? ................................................................. Y / N

20. Are all clinic and laboratory staff trained in Good Clinical Practice? ................................................................. Y / N

21. Are all laboratory staff trained in Good Clinical Laboratory Practice? ................................................................. Y / N

22. Have all staff been sensitized on Good Participatory Practice? ................................................................. Y / N

23. Do we have a policy on post-HIV exposure prophylaxis (PEP) for staff in case of needle injury and for volunteers in case of rape? ................................................................. Y / N

24. Is the research centre environment friendly to both male and female volunteers, as well as to all social, cultural and demographic subgroups that may be participating in a study or trial? ......................... Y / N

25. Do we have a system to accommodate volunteers with babies/young children (if applicable)? ................. Y / N

26. Do we communicate our policies on post-trial access to trial products and procedures if applicable? .... Y / N

Receptionist/Office staff

(please add questions if needed, but don’t repeat from general section)

01. Are volunteers and visitors greeted promptly when they enter the research centre and clearly informed where and for how long they will have to wait? ................................................................. Y / N

02. Do we protect the privacy of volunteers and the confidentiality of their personal information when discussing the objective of their visit in the reception area? ................................................................. Y / N

03. Do we provide timely reimbursement for transportation? ................................................................. Y / N

04. Do we provide refreshments in a timely manner when it is appropriate to do so? ................................................................. Y / N

05. Do we always keep files and records confidential? ................................................................. Y / N

06. Are the IEC or entertainment materials (TV, videos) appropriate for the types of volunteers who visit the centre? ................................................................. Y / N

07. Do we move volunteers from service provider to service provider with maximum efficiency? ................. Y / N Counseling staff (please add questions if needed, but don’t repeat from general section)

08. Do we provide information and volunteer-centred counseling on all proven HIV-prevention options and family planning options (if applicable)? ................................................................. Y / N

09. Do we provide on-site (or refer when appropriate) all nationally approved and relevant HIV-prevention options for men and women in our study? ................................................................. Y / N

10. Do we revisit the following issues with volunteers at least once after enrollment? a. The goals and processes of the study................................................................. Y / N b. The right to informed consent........................................................................ Y / N c. The right to discontinue participation in the study at any time without negative repercussions ......................................................................................................................... Y / N d. The right to confidentiality................................................................................ Y / N e. Unknown efficacy of the product under development (when applicable)........ Y / N f. Partial effectiveness of some of the existing HIV-prevention methods (e.g. Adult Male Circumcision) ................................................................................................................................. Y / N

Do we review the following at each visit? a. What the results of each of the different clinical tests mean............................ Y / N b. HIV risk-reduction plan.................................................................................... Y / N c. New study findings or change of procedures that may have an impact on volunteers’ willingness to continue participation ................................................................. Y / N

04. Do we explain clearly why blood is being drawn for the study, the process (including safety) and what will be done with the blood sample? ................................................................. Y / N

05. Do we counsel volunteers and teach them techniques for safe disclosure of HIV status and/or research participation to partners, family and friends? ................................................................. Y / N

06. Do we explore with volunteers the possible challenges and social harms that could stem from their participation in the research or because of their HIV status, and help them identify their sources of social support? ................................................................. Y / N

07. Do we explore the needs of volunteers for family planning and provide family planning counseling on all effective and locally available methods, whether offered at the research centre or not, when appropriate? ................................................................. Y / N

08. Do we follow up on referrals made for volunteers prior to or during their next visit? ................................................................. Y / N

09. Do we have a clear system for making and following up on referrals? .......................................................... Y / N

10. Do we explore issues of adherence to research requirements (contraception, regimens and dosages, or potential disinhibition) at every visit? .......................................................... Y / N

11. Do we have sufficient time to conduct effective counseling sessions? .......................................................... Y / N

12. Do we document key counseling issues appropriately and confidentially for each volunteer to ensure effective follow-up counseling? .......................................................... Y / N

13. Do we share relevant issues and concerns of volunteers discreetly and confidentially with trial physicians? .......................................................................................................................... Y / N

14. Do we participate in any sort of supportive supervision (e.g. counseling supervision) to ensure that burnout or personal issues do not affect the quality of counseling we provide to volunteers? ........ Y / N

15. Do we have the skills and knowledge to provide effective HIV counseling and testing in the context of: a. Clinical research? .......................................................................................................................... Y / N b. Current cohort? .......................................................................................................................... Y / N

Clinical staff (please add questions if needed, but don’t repeat from general section)

01. Do we minimize waiting time for volunteers? .......................................................................................... Y / N

02. Do we review the following with each volunteer: a. The right to informed consent and confidentiality? ........................................................................... Y / N b. Concerns and clarifications needed about the study? ........................................................................ Y / N c. Family planning needs, as applicable? ............................................................................................. Y / N

03. Do we establish a good rapport with volunteers before beginning a medical examination or delivering a diagnosis? ........................................................................................................... Y / N

04. Do we listen and attend to medical complaints and provide medical services and advice in accordance with the best current medical practices? ........................................................................... Y / N

05. Do we provide medical advice and instructions on how to take medications in understandable and non-technical language? .......................................................................................... Y / N

06. Do we ask volunteers about possible concerns, encourage them to ask questions and listen carefully to what they have to say? .......................................................................................... Y / N

07. Do we provide appropriate referrals for services that we do not provide at the research centre, including: a. Family planning .................................................................................................................. Y / N b. Medical services ................................................................................................................ Y / N c. ART ........................................................................................................................................ Y / N d. Psychosocial services ........................................................................................................... Y / N

08. Do we follow up on referrals made to volunteers when they come back for the next visit? ................. Y / N

09. Do we provide research centre clinical staff with guidance on PEP and emergency contraception for volunteers who report a case of rape? ........................................................................................................... Y / N

10. Do we explore issues of adherence to research requirements (contraception, regimens and dosages, and disinhibition) during every visit? ........................................................................................................... Y / N

11. Do we have sufficient time to conduct medical examinations? .......................................................... Y / N

12. Do we have the clinical skills and knowledge to provide appropriate health services for each of the cohorts enrolled in the study? ........................................................................................................... Y / N

13. Do we always use adequate infection-prevention practices, including appropriate hand-washing and glove-changing between contacts with different volunteers? ........................................................................................................... Y / N

**Laboratory technicians/phlebotomists**

*(please add questions as needed, but don’t repeat from general section)*

01. Do we establish a rapport with volunteers prior to beginning procedures? ........................................... Y / N

02. Do we explain to the volunteers the procedure for drawing blood? ......................................................... Y / N

03. Do we make efforts to ensure volunteers feel as comfortable as possible? ............................................. Y / N

04. Are we skilled in drawing blood, and do we inflict the least discomfort possible? ................................ Y / N

05. Do we use adequate infection prevention practices, including proper hand-washing and glove-changing following contact with each volunteer? ........................................................................................................... Y / N

06. Do we have and consistently use the Laboratory Health and Safety Resource Pack? ............................ Y / N

07. Does the lab environment minimize anxiety for volunteers by storing blood and certain tools out of view? ........................................................................................................... Y / N Pharmacy staff (please add questions as needed, but don’t repeat from general section)

08. Do we have a system that minimizes volunteers’ waiting time to receive drugs? ........................................... Y / N

09. Do we consistently review instructions for taking medications with volunteers, taking care to protect their privacy? ........................................................................................................................................................................... Y / N

10. Do we maintain an appropriate stock of: a. Study drugs and products? ................................................................................................................................. Y / N b. HIV-prevention options (e.g., male and female condoms, PEP)? ................................................................. Y / N c. Lubricants (if applicable)? ................................................................................................................................. Y / N d. HIV and non-HIV related care (e.g., family planning products if applicable)? ........................................... Y / N e. Other products that are needed by volunteers? ................................................................................................. Y / N

11. Do we keep the assignment of study products and relevant documents confidential? ........................................... Y / N Sample 2.b Team self-assessment at the community level

Instructions: Outreach workers at the research centre as well as community agents use this tool to conduct team self-assessments. Circle “Y” for “yes” and “N” for “no.” A “no” answer indicates there is a possible problem.

01. Do we inform (potential) volunteers about their rights and responsibilities as volunteers? ......................... Y / N

02. Do we provide accurate and easy-to-understand information to the community to help potential volunteers assimilate information essential to their voluntary informed consent? ........................................... Y / N

03. Do we clearly explain the purpose and goal of the study? ........................................................................... Y / N

04. Do we explain to community members and volunteers that they can refuse to join or withdraw from the study at any time? ........................................................................................................... Y / N

05. Do we encourage community members and volunteers to ask questions and express their opinions? ........ Y / N

06. Do we monitor social harm and have a system in place to help volunteers if they encounter stigma or other social harm? ........................................................................................................... Y / N

07. Do we help to identify organizations and service providers where volunteers can be referred for psychosocial and other services not provided at the research centre? ................................................................. Y / N

08. Do we always ensure that (actual and potential) volunteers have privacy when we discuss sensitive issues with them? ........................................................................................................... Y / N

09. Do we always ensure the absolute confidentiality of information provided to us by research volunteers? ................................................................................................................................. Y / N

10. Do we treat volunteers and community members respectfully and non-judgmentally, regardless of their choices, values, beliefs and practices? ........................................................................................................... Y / N

11. Do we engage the community in developing effective strategies to recruit vulnerable, hard-to-reach target groups for HIV-prevention research? ........................................................................................................... Y / N

12. Are the education and recruitment materials (IEC, T-shirts etc.) appropriate for the community and target volunteers? ........................................................................................................... Y / N

13. Do we refrain from using any techniques or strategies that may be seen to create an “undue” incentive to participate? ........................................................................................................... Y / N

14. Do we have a strategy to identify the main myths and misperceptions common in the community regarding vaccine research and participation in such studies, and do we have the tools and knowledge to counter them? ........................................................................................................... Y / N

15. Do we have a standard procedure on how to help people with decisions about participation when they are concerned about stigma, violence or their spouses’ or partners’ reactions but still want to participate? ................................................................. Y / N

16. Do we review our strategies and assess the results of our efforts on a regular basis? ........................................ Y / N

17. Do we work closely with research centre staff to understand their day-to-day challenges and help them overcome those challenges through effective work in the community? ........................................ Y / N

18. Do we coordinate with other research centre staff in addressing the needs and concerns of the volunteers and follow-up appropriately as needed? ................................................................. Y / N

19. Do we conduct follow-up based on choices of volunteers/potential volunteers? ........................................ Y / N

20. Have we built strong relationships with influential individuals/community leaders who are in a position to hamper or contribute to HIV-prevention research? ........................................ Y / N

21. Have we built strong relationships with different types of community-based, faith-based, women’s, youth and other organizations to help inform the general community about HIV-prevention research and to minimize stigmatization of those who participate or are associated with the research centre? ........ Y / N

22. Do we leverage the CAB (or a similar board/committee) that represents the interests of the group(s) targeted by our research to inform our strategies for working with volunteers and identify areas for improvement? ........................................................................................................................................................................ Y / N

23. Does the CAB/committee reflect the diversity of the community? ................................................................. Y / N

24. Do we have a mechanism for collecting timely feedback on community concerns through volunteers, CAB members, peer leaders, and other community agents? ................................................................. Y / N

25. Do we regularly conduct effective community outreach and recruitment meetings and seminars? ........ Y / N

26. Do we provide regular feedback to the community about the progress, outcomes and challenges of HIV-prevention research? ........................................................................................................................................................................ Y / N Sample 3

Group discussions: Possible questions/topics for volunteers or community agents

Possible questions for group discussion with both volunteers and community agents

Instructions: First ask the numbered, closed questions below to determine what the participants think. Then explore their answers further with the open-ended probing statements bulleted at the end of this tool.

01. Do you believe volunteers receive competent counseling and sufficiently clear and correct information to provide truly informed consent?

02. Do the volunteers feel free to ask questions and express their opinions to staff at the research centre, even when it means bringing up embarrassing issues?

03. Do volunteers experience social harm due to study participation in the family/community/work place?

04. What worries community members about participation in the study?

05. Do volunteers have adequate access to relevant services at the research centre or at other facilities?

06. Are there services that the volunteers need but do not receive?

07. Does the referral mechanism work for volunteers?

08. Are volunteers pleased with the quality of care at facilities to which they are referred?

09. Do volunteers feel they have sufficient privacy when they visit the research centre?

10. Do volunteers trust that the research centre will keep their information and participation in research confidential?

11. Do volunteers think that they are treated with respect by all staff members on their visits to the research centre?

12. Is the research centre’s environment comfortable for volunteers? Can you think of ways to make the research centre more comfortable for the volunteers? Is there anything at the research centre that makes volunteers feel uncomfortable?

13. Do volunteers find any of the procedures psychologically or physically uncomfortable? Possible additional questions for use in group discussion with community agents only

Instructions: First ask the numbered questions below to determine what the community agents think. Then explore what is working and what can be improved.

01. What does the community (or what do you) know about this research study?

02. What does the community (or what do you) know about eligibility criteria and benefits and risks of participating?

03. How should we identify members of XYZ group for recruitment into this study?

04. Where is the best place to find potential volunteers for this study? What is the best time to find them?

05. How do we help members of XYZ group participate in this study?

06. How do we encourage women to participate in this study?

07. How do we encourage men to participate in this study?

08. What are the reasons many community members give for being unsure about participating in HIV-prevention research?

09. How can we work more closely together to encourage community members to participate in HIV-prevention research? What can you do to help encourage community members to participate?

10. What are some of the reasons there is stigma surrounding HIV-prevention research in this community?

11. How can we work together to reduce the stigma surrounding HIV-prevention research? What can you do personally to help reduce stigma?

12. What can we do to ensure the safety of those who participate in HIV-prevention research, particularly in terms of negative reaction or violence by a partner or the community? What can you personally do to help ensure the safety of those who participate in HIV-prevention research?

13. What are we doing well, and what can we do better? Possible additional questions for use in group discussion with volunteers only

14. What benefits have you experienced from participating in HIV-prevention research?

15. What are the challenges/disadvantages of participating in HIV-prevention research?

16. What can the research centre (or community workers) do to make it easier for volunteers to participate in this study?

17. What do you or other volunteers think of the services received at the research centre (or the services obtained through referrals?)

18. Did you or other volunteers receive the services needed? If not, what services are needed that were not received?

19. What can the research centre (or community workers) do to better serve you?

20. What is your opinion of the counseling you receive at the research centre? Does the counseling help you to practice safer sex? If not, what can we do to make the counseling more useful? What, besides counseling, would encourage you to practice safer sex?

21. What can we do to recruit more volunteers from your community into this study?

Probing Statements

Verbal probes improve the accuracy of information obtained through interview. The following are examples of probing statements the facilitator may use to keep the discussion going:

- Tell me some more about that
- What can you tell me about that?
- Please explain further
- Please, give me an example of a situation when... QUALITY IMPROVEMENT IN HIV-PREVENTION RESEARCH A rights-based and participatory approach

Part 3: Facilitator’s Guide

# TABLE OF CONTENTS

| Section | Page | |------------------------------------------------------------------------|------| | Objectives | 2 | | How to conduct QI introductory sessions | 3 | | Sample agendas | 5 | | Research centre staff: three-day agenda | 5 | | Research centre staff: two-day agenda | 6 | | Research centre staff: one-day agenda (for management team) | 7 | | Community agents: one-day agenda | 8 | | Quality improvement introductory sessions | 9 | | Section I: Introducing QI: Setting the stage | 9 | | Session 1: Climate setting: expectations, objectives, group norms | 9 | | Session 2: Concept and perception of quality in HIV-prevention research | 10 | | Session 3: Rights and responsibilities framework | 12 | | Session 4: Concepts and process of QI and requirements for establishing a QI program | 14 | | Session 5: Ensuring a conducive environment for QI at the research centre | 15 | | Section II: Identifying an area to be improved/problem to be solved | 17 | | Session 6: Identifying an area to be improved: Introduction of QI tools | 17 | | Session 7: Developing a research centre-specific volunteer feedback interview | 18 | | Session 8: Using feedback interviews | 19 | | Session 9: Using team self-assessment at the research centre level | 21 | | Session 10: Using team self-assessment at the community level | 22 | | Session 11: Using group discussion at the research centre level | 23 | | Session 12: Using group discussion at the community level | 24 | | Section III: Analyzing an area to be improved | 26 | | Session 13: Analyzing an area to be improved or a problem to be solved for research centre staff | 26 | | Session 14: Analyzing an area to be improved or a problem to be solved with community agents | 27 | | Section IV: Developing an action plan and monitoring and evaluation | 28 | | Session 15: Developing an action plan with research centre staff | 28 | | Session 16: Developing an action plan with community agents | 30 | | Session 17: Monitoring and evaluation of QI | 31 | | Session 18: Developing research centre-specific standards of quality and quality indicators | 32 | | Session 19: Planning for implementation of QI at the research centre | 33 | This Quality Improvement Facilitator’s Guide is designed to help navigate effective introduction of Quality Improvement (QI) at the research centre and in the community. This guide includes a series of QI introductory sessions that convey, using a participatory approach, key concepts and guidelines in the QI manual. These include the concept of quality, the four-step QI process, and how to use the tools in the context of HIV-prevention research and within the framework of volunteers’ rights and responsibilities. Use of this guide demands substantial understanding of the concepts and information in the Quality Improvement Manual.

Objectives

When used to support the introduction of QI at the research centre, this guide can help ensure that research centre staff:

- Are familiar with the framework of quality in HIV-prevention research based on the rights and responsibilities of volunteers
- Are familiar with the QI tools to be used at both the research centre level and the community level
- Know how to adapt and use the QI tools
- Develop skills in identifying quality problems, analyzing their root cause(s) and developing action plans to solve problems or prevent their recurrence
- Recognize issues related to effective implementation and sustainability of QI programs

When used to support the introduction of QI at the community level, this guide can help to ensure that community agents:

- Are aware of quality issues pertaining to community mobilization, education and recruitment within the framework of volunteer rights and responsibilities
- Are familiar with expected standards of quality and recognize that research centres are committed to improving the quality of community outreach activities
- Are aware of their role in improving the quality of services in HIV-prevention research
- Develop skills in identifying problems, analyzing causes and developing action plans to solve or prevent identified problems from recurring
- Are familiar with the research centre’s QI plan and feedback mechanisms How to conduct QI introductory sessions

Each of the 19 QI introductory sessions includes the topic, approximate time it takes to conduct the session, learning objectives, materials required and the content. The content is broken down into easily followed steps, which may include different options to suit different audiences.

QI sessions should be tailored to the characteristics, cohorts and protocols at each research centre. When designing and preparing QI sessions, the following should be considered:

Facilitator(s) Select QI facilitators based on their core competencies and expertise. Facilitators must have experience in participatory facilitation and a thorough understanding of all concepts in the Quality Improvement Manual. A research centre can work with an external QI facilitator when first implementing QI, and/or appoint staff members as QI facilitators if s/he has sufficient experience in implementing QI (See Manual, page 13).

Criteria for facilitator selection:

- Experience in participatory facilitation
- Understanding of the QI process
- Demonstrated commitment to QI
- Well respected by both management and staff
- Good communication and listening skills
- Sufficient influence at the research centre to implement changes

The QI introductory sessions can involve one or more facilitators, depending on the experience, skills and expertise of each. When an external facilitator has experience with QI but lacks experience working in a research context, an internal staff member with experience in HIV-prevention research should co-facilitate.

Target participants Introductory sessions should involve all research centre staff members. This helps to facilitate and mainstream effective QI implementation. When all staff members cannot be called to such sessions at once, multiple introductory sessions may be scheduled. In such cases, ensure that each session has representatives from different departments or sections of the research centre to ensure adequate exploration of quality issues in the continuum of the research process and to aid group dynamics.

Venue QI introductory sessions, especially those that identify areas for improvement, should be conducted at or near the research centre, so that relevant QI tools are applied within the specific context of the research centre and the surrounding community. In such cases, the QI facilitator and management need to ensure that the sessions do not disrupt research-related activities and that the rights and privacy of the volunteers are protected.

Training methodology and tips This facilitator’s guide is designed to promote the involvement of staff and community agents and maximize their ownership and implementation of QI. The guide outlines suggested approaches and plans for each session topic; however, the experience and approach of the facilitator should be taken into account in designing the sessions. Key tips are provided for facilitators throughout the introductory sessions. The facilitator is guided to encourage staff to initiate problem identification and problem solving, and to avoid lecturing them or handing out solutions. The facilitator must remain neutral to ensure that the QI process is staff-led. Finally, although the agenda and sessions should be planned in advance, facilitators should be flexible and willing to adapt to group needs and dynamics. The session evaluations and closing should be designed by the facilitator with reference to the agenda and approach used (for example, 2-3 successive days or 3 separate days). Logistics, resources and preparation Prior to presenting the introductory sessions, facilitators should identify participants and ensure an optimal group size (ideally 12 to 16 staff members per session), locate a convenient venue with sufficient space to conduct the sessions, provide an agenda to participants, arrange for refreshments, design the venue space layout to enable group learning and prepare handouts and other materials. Materials required for each session are listed in session descriptions.

Suggested agendas This facilitator’s guide is devised to be adapted by each facilitator to meet the needs of various participants (e.g. research centre staff or community agents) as well as the time and resources available for the QI introductory sessions. The following agendas are suggested:

Three-day agenda for research centre staff Three days (not necessarily successive) are required to effectively introduce QI at the research centre and to conduct all sessions in this facilitator’s guide. The first day focuses on introduction of QI and tools for problem identification. The second focuses on analysis of identified problems. The third focuses on action plan development and implementation.

Two-day agenda for research centre staff A two-day version of the agenda for introducing QI is provided as an option for research centres that have limited time available.

One-day agenda for management A one-day agenda for introducing QI to research centre management is also provided, but this does not preclude management participation in two- or three-day introductory sessions scheduled for all staff.

One-day agenda for community agents One full day is required to introduce QI at the community level. While the approach at the research centre is on the integration of QI into its processes, the approach at the community level emphasizes the incremental introduction of QI and application of QI tools (a choice of group discussion or team self-assessment) to identify problems, analyze root causes and develop action plans. Different tools should be introduced at different meetings with community agents.

## RESEARCH CENTRE STAFF: THREE-DAY AGENDA

### Day 1: Introduction to quality improvement and quality improvement tools

| TIME | TOPIC | SESSION # | |------------------|----------------------------------------------------------------------|--------------------| | 8:30am – 10:30am | Climate Setting: expectations, objectives, group norms | Session 1 | | | Concept and perception of quality in HIV-prevention research | Session 2 | | 10:30am – 11:00am| Tea Break | | | 11:00am – 1:00pm | Rights and responsibilities framework | Session 3, option 1| | | Concepts and process of QI and requirements for establishing a QI program | Session 4 | | 1:00pm – 2:00pm | Lunch | | | 2:00pm – 4:00pm | Identifying an area to be improved: introduction of QI tools | Session 6 | | | Developing a research centre-specific feedback interview | Session 7 | | 4:00pm – 4:30pm | Closing | | | 4:30pm – 5:00pm | Afternoon Tea | |

### Day 2: Development of research centre-specific tools and experimenting with the tools

| TIME | TOPIC | SESSION # | |------------------|----------------------------------------------------------------------|--------------------| | 8:30am – 10:30am | Recap | | | | Using feedback interviews | Session 8 | | | Analyzing an area to be improved identified through feedback interview| Session 13 | | 10:30am – 11:00am| Tea Break | | | 11:00am – 1:00pm | Using team self-assessment tools at the research centre level | Session 9 | | 1:00pm – 2:00pm | Lunch | | | 2:00pm – 3:00pm | Using team self-assessment tools at the research centre level (cont’d)| Session 9 (cont’d) | | 3:00pm – 4:30pm | Analyzing an area to be improved identified through team self-assessment | Session 13 | | 4:30pm – 5:00pm | Afternoon Tea | |

### Day 3: Data analysis, interpretation, and development of action plan

| TIME | TOPIC | SESSION # | |--------------------|----------------------------------------------------------------------|-----------| | 8:30am – 10:30am | Recap | | | | Using group discussion | Session 11| | | Developing research centre-specific standards of quality and quality indicators | Session 18| | 10:30am – 11:00am | Tea Break | | | 11:00am – 1:00pm | Developing an action plan | Session 15, option 1| | | Monitoring and Evaluation of QI | Session 17| | 1:00pm – 2:00pm | Lunch | | | 2:00pm – 3:30pm | Ensuring a conducive environment for QI at the research centre | Session 5, option 2| | | Planning for implementation of QI at the research centre | Session 19| | 3:30pm – 4:30pm | Evaluation and closing | | | 4:30pm – 5:00pm | Afternoon tea | |

### RESEARCH CENTRE STAFF: TWO-DAY AGENDA

#### Day 1: Introduction to quality improvement and quality improvement tools

| TIME | TOPIC | SESSION # | |--------------------|----------------------------------------------------------------------|-----------| | 8:30am – 9:45am | Climate Setting: expectations, objectives, group norms | Session 1 | | | Concept and perception of quality in HIV-prevention research | Session 2 | | 9:45am – 10:30am | Rights and responsibilities framework | Session 3, option 1| | | Concepts and process of QI and requirements for establishing a QI program | Session 4 | | 10:30am – 11:00am | Tea Break | | | 11:00am – 1:00pm | Identifying an area to be improved: team self-assessment and feedback interviews | Session 6 | | | Using feedback interview | Session 8 | | 1:00pm – 2:00pm | Lunch | | | 2:00pm – 4:15pm | Analyzing an area to be improved identified through feedback interview | Session 13| | | Using team self-assessment tools | Session 9 | | 4:15pm – 4:30pm | Closing | | | 4:30pm – 5:00pm | Afternoon Tea | | Day 2: Development of research centre-specific tools, experimenting with the tools, and action plan development

| TIME | TOPIC | SESSION # | |------------------|----------------------------------------------------------------------|-----------| | 8:30am – 9:00am | Recap | | | 9:00am – 10:30am | Analyzing an area to be improved identified through team self-assessment | Session 13| | 10:30am – 11:00am| Tea Break | | | 11:00am – 11:30am| Analyzing an area to be improved identified through team self-assessment (cont’d) | Session 13 (cont’d) | | 11:30am – 1:00pm | Developing research centre-specific standards of quality and quality indicators | Session 18| | 1:00pm – 2:00pm | Lunch | | | 2:00pm – 3:00pm | Developing action plan | Session 15, option 1| | 3:00pm – 4:00pm | Planning for implementation of QI at the research centre | Session 19| | 4:00pm – 4:30pm | Evaluation and closing | | | 4:30pm – 5:00pm | Afternoon Tea | |

RESEARCH CENTRE STAFF: ONE-DAY AGENDA (FOR MANAGEMENT TEAM)

| TIME | TOPIC | SESSION # | |------------------|----------------------------------------------------------------------|-----------| | 8:30am – 9:00am | Climate Setting: expectations, objectives, group norms | Session 1 | | 9:00am – 9:45am | Concept and perception of quality in HIV-prevention research | Session 2 | | 9:45am – 10:30am | Rights and responsibilities framework | Session 3, option 2| | | Concepts and process of QI and requirements for establishing a QI program | Session 4 | | 10:30am – 11:00am| Tea Break | | | 11:00am – 12:00pm| Identifying an area to be improved: introduction to QI tools | Session 6 | | 12:00pm – 1:00pm | Analyzing areas to be improved | Session 13| | 1:00pm – 2:00pm | Lunch | | | 2:00pm – 3:00pm | Developing research centre-specific standards of quality and quality indicators | Session 18| | | Developing an action plan | Session 15, option 2| | 3:00pm – 4:00pm | Ensuring a conducive environment for QI at the research centre | Session 5, option 1| | 4:00pm – 4:30pm | Evaluation and closing | | | 4:30pm – 5:00pm | Afternoon Tea | |

## COMMUNITY AGENTS: ONE-DAY AGENDA

| TIME | TOPIC | SESSION # | |------------------|----------------------------------------------------------------------|----------------------------| | 8:30am – 9:00am | Climate Setting: expectations, objectives, group norms | Session 1 | | 9:00am – 9:45am | Concept and perception of quality in HIV-prevention research and in the community | Session 2 | | 9:45am – 10:30am | Rights and responsibilities framework | Session 3, option 3 | | 10:30am – 11:00am| Tea Break | | | 11:00am – 1:00pm | Identifying an area to be improved: using team self-assessment or group discussion at the community level | Session 10 or 12 | | 1:00pm – 2:00pm | Lunch | | | 2:00pm – 3:00pm | Analyzing areas to be improved | Session 14 | | 3:00pm – 4:00pm | Developing action plan | Session 16 | | 4:00pm – 4:30pm | Evaluation and closing | | | 4:30pm – 5:00pm | Afternoon Tea | | QUALITY IMPROVEMENT INTRODUCTORY SESSIONS

Note: the following sessions are not meant to be read in succession. Please follow the suggested agendas on the previous pages and refer to the corresponding session number listed on the agenda.

SECTION I: INTRODUCING QI: SETTING THE STAGE

Session 1 Climate Setting: expectations, objectives, group norms

Climate setting is done differently depending on the affiliation of the facilitator (external or staff member), the size of the research centre (large or small number of staff), the type of agenda and approach (e.g. three successive days, or two- or three-day agendas facilitated over multiple days) and the type of audience (research centre managers, research centre staff, or community agents). For instance, an internal facilitator might already know the names and roles of each participant and might therefore require less time for participant introductions, whereas an external facilitator might need to use this time to establish roles in addition to obtaining the names of participants as part of the introduction. In general, a flexible approach is recommended. But it is always necessary to establish group expectations, objectives and norms.

Time: 30 minutes

Learning objectives:

- To introduce the agenda and participants
- To gain insight into participants’ perceptions of the concept of quality improvement and their expectations for the session
- To present objectives of the QI introductory sessions, and expected outcomes
- To establish an environment conducive to discussing quality-related issues, basic teamwork and adult learning principles

Advance preparation/materials:

- Visuals: #1 or #2, Objectives of Introductory Sessions
- Handouts: #1, #2 or #3 Agenda (based on target audience)

Content: STEP ONE Welcome participants, introduce facilitators and ask participants to introduce themselves (participants provide their name and role/title).

STEP TWO (5 – 10 minutes) Option 1 (recommended for three- and two-day agendas): In groups of two or three, have participants discuss three expectations for the QI introductory sessions and write them down on paper. Option 2 (recommended for one-day agendas for managers): Ask each participant to write what they understand “quality improvement” to mean and what they expect to achieve/contribute by the end of the day.

Option 3 (recommended for one-day agendas for community agents): Prompt discussion by asking what came to mind when they were invited to attend the meeting on quality in community outreach activities. What were the expectations?

STEP THREE (10 – 15 minutes) Present the introductory QI session objectives (Visual #1 or #2) and agenda (Handout #1, #2 or #3), and ask for reactions (e.g. overlap between objectives and expectations mentioned above, consensus on timing of lunch and breaks, etc.). Briefly describe some of the quality-related activities the research centre is already involved in, including the results (if applicable) and the added value of QI.

STEP FOUR (5 – 10 minutes) Option 1 (recommended for three- and two-day agendas): Briefly state that discussing quality-related work issues requires open and honest discussion about the research centre, their own work, and that of their colleagues. Ask the group (you may choose to organize participants in groups of two or three, or conduct a plenary discussion based on the number of participants, your judgment of group dynamics and the time you have available) to come up with norms or principles that will foster a conducive environment for learning and discussing ways to improve our work. If not mentioned, emphasize confidentiality and the need to respect opinions and views as part of group norms, and their value for the QI process. Also stress that Quality Improvement is not about blaming individuals or teams for mistakes or problems, or pointing fingers at others, but rather identifying areas that need to be improved and collectively making improvements.

Option 2 (recommended for one-day agenda for community agents): Briefly emphasize the principles that will guide the community agents’ relationship and collaboration with the research centre. Explain that you want to ensure a safe environment to raise problems, challenges and identify solutions related to community outreach and recruitment, community education and the quality of services at the research centre. Emphasize that their honest opinions are of utmost importance. Finally, ask them to devise rules that will ensure honest discussion during the meeting.

Session 2 Concept and perception of quality in HIV-prevention research

Time: 45-60 minutes

Learning objectives:

- To gain insights on the term “quality”
- To define quality in HIV-prevention research
- To introduce the framework of quality in HIV-prevention research Advance preparation/materials: Visuals: #3 Key messages on quality, #4 Definitions of Quality, #5 Definition of Quality in HIV-prevention Research

Content:

STEP ONE Have participants think of a time when they have experienced either very good or very bad quality services. A large group may want to break into smaller groups for this exercise. Encourage them to think of personal experiences outside the research centre, for example, in a restaurant, hair salon, taxi or hospital. Ask them to share their experiences with one another for about 5 minutes.

STEP TWO Bring the group together and ask three or four people to share the situations described in their discussions.

STEP THREE Ask the whole group the following questions:

- How did you know that you were provided with bad or good services? Write (or ask a co-facilitator for help) responses to this question on a flip chart.
- How did you respond to each experience? Write down responses on a flip chart with two columns indicating “Outcomes of good services” and “Outcomes of bad services.”

STEP FOUR Conclude by remarking, or in your own words, “Just like us, most volunteers have expectations about the quality of services they are provided. They will also take actions similar to those just discussed, depending on whether they are satisfied or dissatisfied with our services.”

STEP FIVE Brainstorm the following: Option 1: For community agents:

- How do quality problems in our community work affect work at the research centre?

Option 2: Management and research centre staff

- How do quality problems at the research centre affect work in the community (i.e. community outreach, recruitment, etc.)?

STEP SIX Ask the group to define “quality.” Acknowledge their answers and then provide the brief definition of “quality” based on Visual #4 Definition of Quality.

Show Visual #3 and reiterate the key messages on quality.

Show Visual #4 with suggested definition of quality.

STEP SEVEN Ask if anyone can define “quality of HIV-prevention research” based on above definitions of quality. Acknowledge their answers and then move to Visual # 5 on definition of “quality of HIV-prevention research” and ask for their reactions to the definition. Session 3 Rights and responsibilities framework

Time: 30-60 minutes

Learning objectives:

- To build understanding of quality based on rights, responsibilities and ethics in HIV-prevention research
- To discuss rights and responsibilities as they pertain to each staff member, station, department or the research centre in general
- To discuss ways to solidify commitment to upholding rights and responsibilities in HIV prevention

Advance preparation/materials: Handout: #4 Volunteer Rights and Responsibilities

Content: Option 1 (60 minutes) (recommended for two- or three-day agendas with research centre staff):

STEP ONE Explain that before discussing QI in depth, it is important to look at the research centre’s research processes within the context of the rights and responsibilities framework based on international and national guidelines, the core of QI (see Manual, page 4).

Divide the participants according to their professional roles (e.g. community outreach workers, administrative staff including drivers, security, receptionists, and cleaners, nurse counselors/counselors, phlebotomists/laboratory technicians, trial physicians and pharmacists). With a smaller group you may divide them into two categories: clinical staff and non-clinical staff.

STEP TWO Ask each professional group or individual (in some cases, you might only have one person representing a profession) to come up with a brief description of their roles and responsibilities vis-à-vis volunteers and record it on one flip chart, and of volunteers’ rights vis-à-vis their roles, which should be recorded on a second flip chart. For example, the receptionist/s might indicate the following:

Receptionist roles & responsibilities

- Make volunteers feel welcome
- Ensure volunteers’ confidentiality during the intake process
- Etc.

Volunteers’ rights vis-à-vis the role of the receptionist

- Right to respectful treatment and to feel as comfortable as possible
- Right to privacy and confidentiality
- Etc. STEP THREE Each group presents what it discussed and the key roles and responsibilities of its sub-groups. Encourage groups to present in order of their interactions with volunteers. Encourage members of other groups to react to the presentations by asking questions, or providing additional insights about rights and responsibilities.

Display the flip charts on the wall to reflect research processes and volunteers’ experience from first interaction (e.g. community level) to the last.

STEP FOUR Probe participants about their opinions of how well the research centre has protected volunteers’ rights and its compliance with ethical responsibilities mentioned on the flip charts.

Provide Handout #4 on volunteers’ rights and responsibilities and discuss with the group whether there is any right that was not mentioned, or what they think of the responsibilities of volunteers.

Briefly discuss, with reference to the definition of quality, how they feel volunteers would rate the research centre in complying with volunteers’ rights and helping volunteers fulfill their responsibilities.

Option 2 (30 minutes) (for the management team):

STEP ONE Brainstorm the elements of volunteers’ rights in HIV-prevention research with the group and record their responses on a flip chart.

STEP TWO Provide Handout #4 on rights and responsibilities of volunteers and record how the group feels about them.

STEP THREE Have the group identify all strategies that the research centre has put in place to ensure that the rights mentioned above are protected and that it complies with its responsibilities in that regard.

Probe to find out what problems participants have identified and what solutions they have put in place to solve those problems and/or ensure that they do not recur.

Probe to find out whether any of their strategies included a systematic exploration of volunteers’ perceptions, as discussed earlier in the section covering the definition of quality.

Probe to discover whether staff members have been involved in monitoring compliance and providing solutions to problems.

Option 3 (45 minutes) (for community agents):

STEP ONE Divide the group in two to answer the following questions:

- Group 1: What are the Do’s and Don’ts when recruiting volunteers or conducting community outreach for HIV-prevention research?
- Group 2: What are the rights of volunteers when participating in HIV-prevention research?

STEP TWO Ask each group to present its answers and ask other members of the group for comments, questions and additions. Provide your own additions and clarifications only after the rest of the group has reacted.

STEP THREE Conclude by telling participants that the research centre has introduced a process to continuously monitor the quality of services both at the research centre level and at the community level. This approach is based on identifying quality problems and solving them together as a team. Encourage participants to keep in mind the Do’s and Don’ts and volunteers’ rights for the remainder of the workshop. Session 4 Concepts and process of QI and requirements for establishing a QI program

Time: 30-60 minutes

Learning objectives:

- To define QI
- To introduce the QI process
- To discuss benefits and challenges of QI implementation

Advance preparation/materials: Visuals: #6 Definition of Quality Improvement in HIV-prevention Research, #7 The Quality Improvement Process

Content: STEP ONE Ask if anyone can try to define the term “quality improvement” based on what was defined as “quality in HIV-prevention research.”

STEP TWO Review Visuals #6 and #7.

Emphasize:

- That QI is a systematic, planned and continuous process
- The difference between the approaches at the research centre level and at the community level (see Manual, page 16)
- That the introductory meeting is part of the recommended steps in preparing a research centre for implementing QI effectively (see Manual, page 13)

Ask if there are questions, comments, additions or concerns.

Steps three and four below are recommended for three-day agenda only.

STEP THREE Divide the participants into two groups:

- Group 1 - Discuss the potential benefits of implementing QI programs.
- Group 2 - Discuss the anticipated challenges to implementing QI programs (if a research centre has started implementing QI but has experienced challenges, this session could also focus on those challenges). STEP FOUR Ask each group to present and solicit comments and questions from other groups. Then brainstorm with the group about potential benefits and potential solutions to challenges.

STEP FIVE Let the group know that the rest of this program is going to focus on the first three steps of the QI process by applying select QI tools to identify a problem, analyze a problem and develop an action plan to correct the problem and to prevent it from happening again. The fourth step of the QI process, implementing an action plan, will take place after the QI introduction is completed. It will, however, be discussed as part of the broader process of implementation of QI at the research centre.

Session 5 Ensuring a conducive environment for QI at the research centre

Time: 45-60 minutes

Learning objectives:

- To discuss ways to ensure that research centres create a conducive environment for QI

Advance preparation/materials: Handout: #16 Creating an Environment Conducive to Good Employee Performance and Quality Services

Content: Option 1: Recommended for managers:

STEP ONE Provide Handout #16 on creating a conducive environment and ask each participant to mark what might be easy to implement and what might be challenging to implement.

STEP TWO Facilitate discussion by asking participants to share with the group what would be easy to implement (or what the research centre management has been doing effectively), then ask participants to discuss what would be challenging. Ask the group to brainstorm potential solutions to the challenges.

Encourage each manager to identify issues they can work on in order to create an environment conducive to QI.

STEP THREE Conclude by stating that both management and staff have the responsibility to create an environment conducive to QI. Emphasize that implementation of QI may involve changes to management or supervision approaches and how staff interact with their colleagues and supervisors. Emphasize that this requires incremental change and that it has been a learning process for most managers when they adapt QI approaches. Option 2: Recommended for three-day agenda

STEP ONE Divide the group into three smaller groups with mixed representation.

STEP TWO Assign each group a specific stakeholder listed below and ask each group to answer the question: What should this stakeholder do to facilitate effective implementation of QI?

- Staff (vis-à-vis volunteers, community agents, other staff and management)
- Management (vis-à-vis staff, other managers, community agents, volunteers and donors)
- Donors/Sponsors (vis-à-vis the management of the research centre)

STEP THREE Ask each group to present its answers and request additional comments or questions from the other groups.

STEP FOUR Ask what can be done to ensure that the above stakeholders adhere to the suggested recommendations in Step two.

STEP FIVE Conclude by stating that ensuring an environment conducive to sustaining QI programmes requires a team effort by many stakeholders. While donors can play a role in encouraging QI, it is mostly the responsibility of management and staff—the individuals conducting HIV-prevention research. State that effective implementation of QI requires a change in management’s approach and a change in how staff relate to volunteers, community agents and colleagues. Finally, it is important to acknowledge the need to change approaches and to do so incrementally. Session 6 Identifying an area to be improved: Introduction of QI tools

Time: 90-120 minutes

Learning objectives:

- To introduce a variety of QI tools
- To explore advantages of specific tools and approaches in an HIV-prevention research setting
- To differentiate between volunteer-oriented and service provider-oriented tools
- To provide a basis for research centres to choose tools appropriate to their contexts
- To explore the interdependence of QI tools at the research centre level and the community level

Advance preparation/materials:

- Visuals: #8 Methods and Tools to Identify a Problem and #9 Tools in Three Columns
- Handouts: #6 Suggested Tools at the Research Centre Level and Community Level and #7-11 Tools: Definitions, Advantages and Disadvantages (for distribution at end of session)

Content: STEP ONE On a sheet of paper, write down each of the following methods to identify a quality problem (listed on Visual #8) and put them on a table facing down: Rumors, Checklist, Household questionnaire, Suggestion box, Volunteer/Client feedback, Supervision, On-the-job observation, Feedback interviews, Analysis of routine data, Self-assessment (team and individual), Mystery (dummy)-volunteer, Records review, Case studies and role-plays, and Group discussion.

STEP TWO Ask each participant to pick one method and discuss the advantages and disadvantages of the method as a way to identify a quality problem. Encourage additional discussion from the group.

STEP THREE On a flipchart, make three columns—“volunteer-oriented” in the left column, “service provider-oriented” in the right column, and “both” in the middle column (see visual #9). Ask the group to classify tools as either volunteer-oriented or service provider-oriented (i.e. tools that provide volunteer perspectives and those that collect the perspectives of service providers). Ask the group to put those that they think can do both in the middle. STEP FOUR Ask the group to prioritize two or three methods to be applied at their research centre. Allow debate, but ensure that it remains focused on the practicality of each method for the research centre (in terms of data collection and data analysis, integrity of data collection, volunteers’ vs. service providers’ perspectives, action plan development, etc.).

Acknowledge the value of other methods but mention that for the purpose of these QI introductory sessions, the focus will be on the ones presented in the QI Toolkit. Additional methods can be added by the research centres as needed.

STEP FIVE Provide Handout #6 on Suggested Tools at the Research Centre Level and Community Level. Reemphasize the connection between quality at the research centre level and quality at the community level (see Manual, page 16). Provide the rationale for use of each tool at each level. Ask if there is any tool discussed that they feel needs to be added in Handout #6.

STEP SIX Conclude by saying that the subsequent sessions will focus on tools recommended in Handout #6. It is recommended that the research centre experiment with the tools and prioritize the ones that will help identify quality problems from the perspectives of both service providers and volunteers. Due to the interdependence of quality at the research centre level and the community level, it is recommended that problems be identified at both levels. Some QI tools can be used to complement other QI tools (e.g., group discussion can complement feedback interview). It is recommended that research centres use different tools from one QI exercise to another to maintain interest of QI participants and to explore different perspectives.

Distribute Handouts #7-11 on Tools: Definitions, Advantages and Disadvantages.

Session 7 Developing a research centre-specific volunteer feedback interview

For three-day agenda only

Time: 60 minutes

Learning objectives:

- To familiarize participants with the feedback interview approach
- To adapt the QI tools to the particular research centre and its environment
- To discuss modalities for the effective implementation of feedback interviews

Advance preparation/materials: Handouts: #18, #19 and #20 Samples of Volunteer Feedback Interviews Content:

STEP ONE Explain that there are three sample feedback interviews suggested in the toolkit: one for volunteers enrolled in clinical trials, one for volunteers enrolled in an epidemiology study, and another for use during recruitment and screening visits. Briefly explain that these are very general samples and not all questions will apply to all research centres. Each will need to adapt these tools to suit its context and cohorts.

STEP TWO Divide the group into two. (Note: If the research centre is conducting both clinical trials and epidemiology studies at the same time, consider dividing the group into three: one looking at the feedback interview for a clinical trial, another considering the feedback interview for an epidemiology study and another examining the feedback interview during recruitment).

Provide Group 1 with the sample feedback interview for use during clinical trials or epidemiology studies, depending on the current focus of the research centre. Provide Group 2 with the sample feedback interview for use during recruitment or screening visits. Ensure that all staff roles are represented in each group, if possible.

STEP THREE Ask each group to consider which questions apply to its research centre and which ones do not, and why. Ask them to suggest additional questions that may be crucial in helping the research centre identify a quality problem. Recommend that they keep written notes about their suggestions because the outcome of this discussion will form the basis for adapting the tools. Encourage one person in each group to take notes on the suggested changes to the sample feedback interview. Collect these notes at the conclusion of Step three.

STEP FOUR Ask each group to present the outcome of its discussion to the entire group. Document all suggested modifications and incorporate changes into a feedback interview tool adapted for the research centre prior to Session eight.

Session 8 Using feedback interviews

Time: 60 minutes

Learning objectives:

- To practice using feedback interviews and familiarize participants with the experience of being an interviewer
- To practice entering data from volunteer feedback interviews

Advance preparation/materials:

- Visuals: #10 Instructions on How to Apply Volunteer Feedback Interviews Handouts: Adapted sample of Volunteer Feedback Interview (see outcome of Session seven) and Handout #17 Sample Excel Spreadsheet for Volunteer Feedback Interview Data Analysis.

Setting up a role-play Prepare two desks and chairs (depending on group size, see Step two) arranged at each corner of the room with laptops equipped with the sample Excel data-collection spreadsheet. As an alternative to using the laptops, split the participants into two groups to discuss the feedback interviews.

Content:

STEP ONE Provide a brief overview presentation (Visual #10) on how to use feedback interviews (see Manual, page 21).

STEP TWO Instruct the group to break up in pairs for a role-play exercise in which one person will act as the interviewer and one will act as the volunteer.

STEP THREE Decide which type of feedback interview to use, depending on the type of study conducted by the research centre (see outcome of Session seven). If the research centre is conducting a clinical trial or epidemiology study, then use the adapted feedback interview. If it is in the middle of recruitment or is otherwise preparing for a study or trial, then use the adapted feedback interview for recruitment and screening. Provide each interviewer with one questionnaire to use during the interview.

STEP FOUR Ask the participants playing the role of volunteers to answer the questions as if they are volunteers or potential volunteers at the research centre. Remind them to assume that they have gone through all the steps of their scheduled visit, and that this interview is happening at its very end. Instruct them to answer some questions positively and some negatively for the purpose of analyzing the data in the following steps.

STEP FIVE At the end of the role-play, collect all filled feedback interview forms. Discuss their experience conducting interviews or being interviewed. Divide the whole group in two. Send each group to one laptop or computer that has the Excel file for feedback interview data input downloaded.

Briefly explain how to insert the data into the Excel sheet. Explain that the Excel sheet is set to match the answers in the volunteers’ feedback interview with option one as the most desirable quality outcome and option four as the least desirable quality outcome. State that the Excel sheet will automatically calculate a summary of percentages after data from the volunteer feedback interview has been entered. Provide Handout #17 as a sample of an Excel data collection sheet filled out. As an alternative to using Excel, participants can calculate the answers manually.

STEP SIX Ask each group to summarize its findings based on the excel sheet summary or hand calculations, and to highlight problems. If time allows, the groups can make graphs and presentations (using software such as PowerPoint) out of their findings. Ask each group to try to define the quality problem(s).

STEP SEVEN Conclude by thanking participants for doing the role-play and data entry. State that the data is going to be used to analyze the root cause of the problem. Session 9 Using team self-assessment at the research centre level

Time: 90-120 minutes

Learning objectives

- To familiarize staff with the team self-assessment approach
- To practice using the sample self-assessment tools
- To adapt sample tool questions to the particular context of the research centre

Advance preparation/materials:

- Visuals: #11 Instruction on How to Apply Team Self-Assessment at the Research Centre Level
- Handout: #21 Sample Team Self-Assessment for Use at Research Centre Level

Content: Ideally, this exercise should be conducted at the research centre. If that is not possible (e.g. the venue where you are holding the QI introductory sessions is too far from the research centre) set up the room to reflect the different stations at the research centre. The latter approach deprives participants of the advantage of actual research centre observation during this process. To compensate for that disadvantage and get the most out of this exercise, encourage them to think of the research centre environment as they go through this exercise or to repeat it when they get back to the research centre.

STEP ONE Provide an overview on how to apply team self-assessment at the research centre level (see Manual, page 17). Explain that the whole team will go through a series of questions related to the quality of their work. A “no” answer means the team needs to discuss how to improve that area. Remind the group of the importance of providing honest feedback and that there will be no penalty for identifying a problem. (The goal is not to assign blame to individuals but to identify processes that can be improved).

STEP TWO Divide the group into QI teams based on the number and composition of the participants (see Manual, page 18).

STEP THREE Explain that each QI team will have a section (or multiple sections) of the team self-assessment tool to focus on and a station (or multiple stations) to visit. A research centre staff member should remain at each station representing their professional role to answer additional questions that QI teams might have. STEP FOUR Briefly go through the role of research centre staff at each station, as well as the role of QI teams. Ask participants to mark any question that is not relevant to the context of the research centre with an “N/A” for not applicable.

STEP FIVE Ask the QI team to imagine that they are volunteers (emphasize that they must identify as the specific cohort enrolled at the research centre) going from station to station and completing the questions on the assigned section of the self-assessment tool by interacting with staff at the station. In addition, the QI team should make observations of quality-related issues at each station (e.g. crowded reception area, cleanliness, display of volunteers’ rights, privacy, etc.) and use that information and their own knowledge to help answer the questions on the team self-assessment tool. If the research centre is small, and the QI team is comprised of all staff members, at least one relevant staff member should be present at each station (e.g. at least one counselor should be at the counseling station) to respond to any questions that the QI team might have. Then they can rejoin the QI team when they move on to the next station.

STEP SIX Option 1: At the end of the exercise, bring the group together and ask staff to discuss how it felt to be in the volunteers’ shoes.

Walk through the questions with the whole group. Congratulate the teams for all questions answered “yes” and emphasize that each indicates an area where the research centre is doing quality work. List all questions answered “no” on a flipchart, all questions on which the group could not reach consensus, and any additional problem areas identified through the interactions with the research centre staff at the station.

Option 2: Provide Handout #21 team self-assessment to all staff members and ask each staff member to answer individually. Then have the group compare answers with each other and discuss.

Session 10 Using team self-assessment at the community level

Time: 120 minutes

Learning objectives:

- To use team self-assessment with community agents to identify areas to be improved at the community level
- To familiarize community agents with expected standards for community outreach and recruitment

Advance preparation/materials: Handout: #22 Sample Team Self-Assessment at the Community Level Content:

**STEP ONE** Explain that the whole team will go through a series of questions related to the quality of our work. A “no” answer means the team needs to discuss how to improve that area. Remind the group to give honest feedback and that there will be no penalty for identifying a problem. (The goal is not to point fingers at individuals, but to identify processes that can be improved).

**STEP TWO** Depending on the group size and composition, you may divide the group into small teams to go through the questionnaire (see Manual, page 20).

**STEP THREE** Encourage community agents to imagine that they are in the shoes of community members or volunteers when responding to questions.

**STEP FOUR** Analyze the answers to questions by highlighting what the team is doing well (questions answered “yes”) and by identifying quality problems (questions answered “no”). Inform the group that the focus will be on questions answered “no” in the following session on group discussion (Session 12).

______________________________________________________________________

**Session 11**

**Using group discussion at the research centre level**

**Time:** 75 minutes

**Learning objectives:**

- To familiarize staff with the group discussion approach
- To practice group discussion
- To adapt the sample group discussion tool to the particular research centre

**Advance preparation/materials:**

- Visuals: #12: Guide to Conducting a Group Discussion
- Handout: #23 Sample Group Discussion Questions

**Content:**

**STEP ONE** Give an overview of group discussion as a tool for identifying quality problems (see Manual, page 23). Then briefly present the approach to conducting a group discussion (Visual #12) and solicit questions and comments. STEP TWO Ask for two volunteers to lead group discussions, and ask each volunteer discussion leader to choose a note taker to document the discussion. Split the rest of the group into two and have them play the role of research volunteers in the group discussion. Responses should genuinely reflect their feelings about the issues raised by the questions. If the group is small, you may choose to have only one group discussion instead of conducting two simultaneously.

STEP THREE Give the two discussion leaders the sample group discussion handout and have them choose which questions to use, focusing on those that apply to the research centre. Facilitators may focus on questions related to the issues emerging from volunteer feedback interviews or the elements of the team self-assessment that need clarification. Discussion leaders should also identify a main facilitator and note taker. Give them 15 to 20 minutes to prepare.

STEP FOUR (45 min) Ask the group discussion facilitator to start the group discussion. S/he might skip the introduction section during the role-playing to save time.

STEP FIVE Ask each group about its experience during the group discussion in identifying areas that need to be improved.

STEP SIX Ask the note taker to present the key findings with a focus on major problems identified.

Session 12 Using group discussion at the community level

Time: 120 minutes

Learning objectives:

- To use group discussion to identify areas to be improved at the community level.
- To familiarize community agents with expected quality standards for conducting community outreach and recruitment.

Advance preparation/materials: Review how to conduct a focus group discussion with community agents (see Manual, page 23).

Content: STEP ONE Tell the participants that we will have an open discussion on the quality of our work so that we can learn how to do our work better.

STEP TWO Briefly discuss rules to ensure effective and honest discussion. Emphasize the importance of confidentiality and respect for the opinions of others. STEP THREE Start the discussion selecting questions from the Group Discussion tool three that pertain to the issues identified on the team self-assessment exercise (Session 10). Given time constraints, it is strongly preferable to cover fewer discussion questions on major quality issues than to rush through many questions. Include follow-up issues identified at the research centre or through other QI exercises.

STEP FOUR Thank the participants for their answers. Present a summary of the discussion and answers to the group. Inform the group that the next session will look at the problems that were identified and determine the causes so that a decision can be made on how to prevent or solve the problems (Session 14). Session 13 Analyzing an area to be improved or a problem to be solved for research centre staff

Time: 60-90 minutes

Learning objectives:

- To establish the process for analyzing area(s) to be improved or problem(s) to be solved
- To familiarize participants with root-cause analysis

Advance preparation/materials: Visuals: #13 Formulation of Problem Statement and Multiple Whys

Content: STEP ONE With a brief presentation, introduce the concept of multiple whys (Visual #13 and Manual, page 26).

STEP TWO Choose one of the two following options:

Option 1 (recommended for managers): provide Handout #13, “Sample of Quality Discovery Problems,” and ask each participant, or pairs of participants, to apply multiple whys to identify the root cause of the problem. While these are case studies, encourage managers to think of the cases as if they happened at the research centre.

Option 2 (recommended for two- and three-day agendas): Note: This session is implemented based on the tool used to identify the problem. For example, if you introduced the team self-assessment and you had participants practice using it, then you should apply the multiple whys to the data from the team self-assessment.

Select key findings (two to three) from practice with feedback interviews, self-assessments, or group discussions. Include a minimum of two volunteer-oriented discoveries and two service provider-oriented discoveries. Divide the group accordingly and ask each small group to apply multiple whys to different identified problems that need to be improved.

STEP THREE Ask each group to present its analysis of the root cause. Ask other groups to respond with alternative explanations or confirm the root-cause analysis.

STEP FOUR Thank the group and discuss how it felt to conduct the root-cause analysis.

When participants have applied root-cause analysis of the volunteer feedback interview, team self-assessment or group discussion, discuss the difference between conducting root-cause analysis for a volunteer-oriented tool (e.g. volunteer feedback interview) and service provider-oriented tool (e.g. team self-assessment).

**STEP FIVE** Conclude by saying that once problems have been identified, prioritized, and their root cause(s) have been analyzed (both those that are within the control of research centre and those that are not) through multiple whys, the next step is to discuss how to solve or prevent the identified problems by developing an action plan, which is covered in Session 15.

______________________________________________________________________

**Session 14**

**Analyzing an area to be improved or a problem to be solved with community agents**

**Time:** 60 minutes

**Learning objectives:**

- To analyze identified area(s) to be improved or problem(s) to be solved with community agents
- To familiarize community agents with root-cause analysis

**Advance preparation/materials:** Review formulation of problem statements and multiple whys (Manual, page 26).

**Content:**

**STEP ONE** Review the summary of key quality problems identified through team-self assessment or group discussion. Ask participants to identify the two most pressing problems that need to be solved to better serve volunteers. Explore the rationale for their choices.

**STEP TWO** Tell participants that we will try to better understand the cause of this problem in order to fix it. To do that you will ask them a couple of questions, starting with why. Inform them that the process of asking multiple whys might feel a bit intrusive but that it is necessary to understand the causes of problems before solving them or preventing them from happening again. Start with problem #1 and ask up to three why questions, obtaining consensus from the group on answers to each. Write them on a flip chart or ask the note taker to document the answers. Then facilitate the same discussion for problem #2.

**STEP THREE** Ask participants how they felt about the “why” question-and-answer session. Thank participants for their insights and honesty.

**STEP FOUR** Conclude by saying that now that we have a sense of what the causes of the quality problems might be, the next step is to discuss how to solve or prevent the identified problems by developing an action plan. Session 15 Developing an action plan with research centre staff

Time: 60 minutes

Learning objectives:

- To familiarize participants with the process of developing an action plan
- To practice the development of research centre-specific action plans
- To initiate action plan development for identified quality problems

Advance preparation/materials:

- Visuals: #14: Considerations and Preparations for Action Plan Meeting
- Handouts: #14: Sample Action Plan 1 and #15 Sample Action Plan 2

Content: Option 1 (recommended for two- and three-day agendas for research centre staff and one-day agendas for management teams):

STEP ONE Review the third step of the QI process, the development of an action plan to solve and prevent identified quality problems. Preparing an action plan involves the development and selection of interventions. State that the goal is to develop the intervention that is most effective in solving the identified problems within available resources. If there are many problems, it may be necessary to prioritize the interventions and set a realistic timeline.

STEP TWO State that there are different ways of developing action plans. However, in principle, all QI action plans must include the following:

- Date the problem was identified
- Problem identified
- Root-cause analysis
- Proposed intervention
- Individual/s responsible for implementation
- Implementation timeframe
- Intervention outcomes Provide samples of action plans (Handouts #14 and #15) and allocate 15 minutes for the whole group to review and critique them.

**STEP THREE** Divide the group into two and provide each with at least two of the problems identified (one through feedback interview and another through team self-assessment) and its root-cause analysis from Session 13. Ask each group to come up with a proposed intervention.

Groups should focus on the following:

- How effective will the proposed intervention be?
- How does this intervention prevent this problem from recurring?
- Does the research centre have resources to implement the suggested intervention?

**STEP FOUR** Ask each group to present its proposed intervention. Then give the other group the opportunity to respond to the presentation, answering the following questions:

- Will the proposed intervention(s) solve the identified problem or prevent it from recurring?
- Are there better alternatives? What are they, and why are they better?

**STEP FIVE** Conclude by saying that most interventions require considerable consultation with relevant staff throughout their design and development. It is therefore crucial to include them in action plan development. When those affected by a problem are involved, the solution ultimately chosen is more likely to be appropriate. Different teams might sometimes have different perspectives on the best course of action regarding suggested interventions. Similarly, the QI committee and management team may have differing opinions on suggested interventions. It might be necessary to rethink an intervention due to its cost or other considerations. It is therefore important not to rush the process of intervention design. It is recommended that the QI committee and management hold a specific meeting to develop interventions and action plans.

**Option 2** (Recommended for research centre managers):

**STEP ONE** Based on the outcomes of Session 13 and root-cause analysis of the sample discovery quality problem, provide each participant with one discovery problem and ask each to come up with an action plan to solve the problem and/or prevent it from recurring.

**STEP TWO** Ask each participant to present his or her finding, and ask other participants to react to the presentation by considering the three questions below:

- Would the proposed intervention(s) solve the identified problem or prevent it from recurring?
- Does the research centre have resources to implement the intervention?
- Are there better alternatives? What are they and why are they better?

**STEP THREE** Conclude by saying that most interventions require considerable consultation with relevant staff throughout their design and development. It is therefore crucial to include them in action plan development. When those affected by a problem are involved, the solution ultimately chosen is more likely to be appropriate. Different teams might sometimes have different perspectives on the best course of action regarding suggested interventions. It is important that management and QI committee consult each other during action plan development and implementation of action plans. Session 16 Developing action plan with community agents

Time: 60 minutes

Learning objectives:

- To familiarize participants with the process of developing an action plan
- To practice the development of research centre-specific action plans
- To initiate action plan development for identified quality problems

Advance preparation/materials:

- Choose a sample action plan (Manual, pages 29-30).

Content:

STEP ONE State quality problem #1 and the root causes identified by the group. Then ask the group: what can be done to solve this problem or ensure that this problem does not recur?

Probe with the following queries:

- What can the team do better? Who can take the lead on this action?
- What can different individuals do? How can we ensure that they do it?
- What can the research centre do? How can we ensure that it follows through?

STEP TWO Write down different interventions and specific suggestions for persons responsible.

STEP THREE Repeat Steps one and two for quality problem #2. If time allows, develop action plans for up to three or four quality problems.

STEP FOUR Conclude by thanking participants and highlighting next steps (for example, presenting the outcome of this discussion to the QI committee), taking care to inform them whether you are going to give feedback to this particular group, to a larger group or give no feedback. If you will provide feedback, explain how you are going to disseminate the feedback (next community meeting, communication board, etc). Session 17 Monitoring and evaluation of QI

Time: 60 minutes

Learning objectives:

- To establish a framework for monitoring and evaluating QI
- To initiate discussion on roles and terms of the QI committee
- To organize a QI committee

Advance preparation/materials: Handouts: #5 Role of QI Committee and Consideration for Creating a QI Committee

Content: Recommended for three-day agendas for research centre staff only.

STEP ONE Review the four steps in the QI process, the systematic approach to QI and the need for continuous monitoring and evaluation.

STEP TWO Ask participants what they think “monitoring and evaluation of QI” entails. Write down their answers on a flip chart. Provide additional input based on page 31 of the QI Manual.

Ask participants who they think should be in charge of monitoring and evaluation of QI.

STEP THREE Introduce the concept of a QI committee. Provide Handout #5 on the role of the QI committee and considerations for establishing a QI committee.

STEP FOUR Ask participants to select a QI committee for ensuring the launch and effective implementation of QI. The group can decide if they want to elect members or solicit volunteers. Session 18 Developing research centre-specific standards of quality and quality indicators

Time: 60 minutes

Learning objectives:

- To discuss the need for monitoring key indicators of quality
- To explore key indicators of quality and to discuss their meaning and relevance to the research centre
- To discuss adaptation of indicators

Advance preparation/materials: Handouts: Handout #12 Sample of Quality Indicators

Content:

STEP ONE Brainstorm with the whole group how to measure quality improvement from one QI exercise to another. Briefly, state that depending on the tools used and problems identified, a research centre might have many quality issues to monitor. However, not all identified problems require monitoring over time, and not all problems will be recurrent. Furthermore, there are other indicators of quality that are not necessarily captured by the QI tools. It is therefore important for the research centre to identify key indicators of quality to monitor over time as they conduct different QI exercises.

STEP TWO Distribute Handout #12 on sample quality indicators. Divide the group in two. Group 1 will look at measurable indicators that can be collected through QI tools, and Group 2 will consider measurable indicators that can be collected through existing research centre forms and data. Ask the groups to evaluate each indicator and decide whether it is a relevant indicator of quality for the specific research centre, and whether the indicator should be monitored over time. Encourage the group to explore and identify other relevant measurable indicators for quality.

STEP THREE Ask each group to present its thoughts on the sample quality indicators. Presentations should encourage consensus and discussion among groups. Document the different groups’ suggestions to form a cumulative list of initial measurable indicators to observe over time.

STEP FOUR Conclude by saying that these indicators may need to be revised from time to time based on experience and observations through the continuation of the QI process. The QI committee should ensure appropriate monitoring of such indicators and review them using cause analysis in different QI exercises. State that this is important so that the research centre does not get caught in a measurement trap (see Manual, page 31). Session 19 Planning for implementation of QI at the research centre

For two- and three-day agendas only

Time: 60 minutes

Learning objectives:

- To formulate a specific plan to implement the QI process at the research centre after the introductory sessions
- To discuss a way forward on quality problems identified during the introductory sessions

Advance preparation/materials: Review “Tool selection and adaptation” (see Manual, page 17).

Content: STEP ONE Ask the QI committee to facilitate discussions and present a consensus on the following issues:

- How to proceed with problems identified and action plans developed during the QI introductory sessions
- Which tool(s) to use and how to proceed with use or adaptation of the tool(s) for identification of other areas to be improved
- A date/time when staff will use the tool(s)
- Whether to start with QI at the research centre level first, begin with the community level, or conduct both simultaneously
- How to approach the management regarding the four issues above if they were not part of the QI introductory sessions QUALITY IMPROVEMENT IN HIV-PREVENTION RESEARCH TOOLKIT