RESEARCH

IN ACTION

Agency for Healthcare Research and Quality • www.ahrq.gov

Introduction

Americans of all ages suffer from mental illness, and many

do not receive the care they need. For example, about 44

million people 18 and over suffer from mental disorders,

which can result in disability and death.1 Major depressive

disorder is the leading cause of disability among adults, and

suicide ranks as the second leading cause of death among

U.S. adolescents ages 10-17.2 In any given year, over 2

million Americans suffer the symptoms of schizophrenia,

but more than half do not receive appropriate treatment.3,4

Furthermore, little is known about the quality of mental

health programs, how well they work, or how to measure

their effectiveness.5 Mental health professionals have

expressed concern that education and training programs

have not kept up with changes in health and health care

over the past 15 years.6 As a result, those receiving care

may not be benefiting from recent advances.

The Agency for Healthcare Research and Quality (AHRQ)

has a broad portfolio of mental health research. This report

focuses specifically on AHRQ-funded research that has led

to the development of programs, methods, and tools for

evaluating and improving the quality of mental health

services and improving the education of mental health

professionals. AHRQ-funded research covers areas such

as: 

• Depression–The Partners In Care program increases

treatment for depression and improves outcomes.

• Schizophrenia–Two toolkits were developed to improve

treatment for schizophrenia.

• Adolescents at risk–New tools can be used to screen

adolescents at risk for suicide and to evaluate school

mental health programs.

• Quality assessment–Tools have been developed that

promote quality improvement programs at managed

behavioral health care organizations.

Programs and Tools to Improve the Quality of Mental Health Services

Making a Difference

Quality improvement programs improve care and

outcomes for depression...Page 3

Treatment guidelines for schizophrenia are helpful in

developing toolkits to improve care...Page 5

A screening tool helps identify children and adolescents

at risk for suicide...Page 6

Evaluation of a school mental health program shows that

it lowers costs...Page 6

Consumer ratings assessment can improve behavioral

health services...Page 7

A conference provides solutions to improve provider

training and continuing education...Page 8

Issue #16 July 2004

Author: Barbara L. Kass-Bartelmes, M.P.H., C.H.E.S.

Managing Editor: Margaret Rutherford

Design and Production: Joel Boches

Suggested citation: Kass-Bartelmes BL, Rutherford MK. Programs and tools to improve the quality of mental health services. Rockville (MD): Agency for

Healthcare Research and Quality; 2004. Research in Action Issue 16. AHRQ Pub. No. 04-0061.

Advancing Excellence in Health Care
• Professional education–Researchers have identified

solutions that can improve education and training for

mental health care professionals.

Background

Research in mental health services has long indicated the

need to improve care for people who have or are at risk for

developing mental health disorders. 

Depression

Between 1987 and 1997, the rate of outpatient treatment for

depression tripled.7 However, depression is difficult to treat

even in diagnosed patients because depressed patients are

less likely to complain of their problem than patients

diagnosed with physical ailments. In fact, depressed

patients may not even know that they are depressed or that

they need treatment.8 Furthermore, treatment itself requires

careful management for up to 1 year.8 Research shows that

patients often do not receive the best quality of care for

depression.9 For example, research sponsored by AHRQ has

found that: 

• About 20 percent of patients in primary care settings

have depression and 6 percent require treatment, yet it

often is not detected.10,11

• Less than half of patients who were diagnosed as

depressed actually received treatment.12

• Only 29 to 43 percent of patients with symptoms of

depression received counseling when they were initially

screened for depression, and only 35 to 42 percent had

used the correct dosage of antidepressant medication.9

Schizophrenia

Schizophrenia is a chronic, disabling medical illness, and

those who suffer from it have high rates of under- and

unemployment, often resulting in poverty and reliance on

publicly funded health care, such as Medicaid.4 No specific

cause of schizophrenia has been identified4 nor is there any

known cure for this disease.3 About 10 percent of people

diagnosed with schizophrenia eventually commit suicide.3

Even though effective medications and therapies exist to

treat people with schizophrenia, the Schizophrenia Patient

Outcomes Research Team (PORT) funded by AHRQ found

that: 

• Less than half of patients suffering from schizophrenia

received proper doses of antipsychotic medications or

appropriate psychosocial interventions. 

• Younger schizophrenic patients were more likely than

people ages 65 and over to be offered psychotherapy

and vocational rehabilitation. 

• African-American patients were less likely to report

being diagnosed with schizophrenia and, once

diagnosed, more likely to be on higher doses of

antipsychotic medications. However, they were less

likely to receive individual therapy or to be prescribed

an antidepressant medication when depressed.4,13,14 

Adolescents at risk

The suicide rate in young people has increased dramatically

since 1980: among adolescents ages 10-14 years, the rate

increased 100 percent.15 In 2000, over 1,900 adolescents

ages 10-19 years committed suicide.16 It is estimated that,

for every young person who completes a suicide, 8 to 25

others have attempted suicide.16 Research indicates that it

is imperative to identify and treat both suicidal thoughts

and depression among adolescents. Findings from AHRQfunded studies include the following: 

• One study showed that about 20 percent of high school

students admitted to having thoughts of suicide and 10

percent had attempted suicide.17

• Forty-seven percent of physicians in another study stated

that at least one of their adolescent patients attempted

suicide in the prior year. However, only 23 percent of

these physicians routinely screened adolescent patients

for suicide risk factors.18

Depression increases the risk of adolescent deaths from

suicide.19-21 But even if an adolescent is diagnosed with

depression or found to be at risk for suicide, access to

mental health services can be difficult. For example:

• Attempts to reduce health costs have resulted in

decreased access to mental health services for children.20

• Expanded school mental health programs that offer

assessment, treatment, case management, referral

services, and preventive services can help fill the gap,

but they need evaluation to demonstrate their value and

effectiveness to help ensure funding and continued

growth.22-24

Quality assessment

Insurance plans often provide mental health services

through managed behavioral health care organizations

(MBHOs).5 Consumer opinions of the services they receive 

2 www.ahrq.gov
from MBHOs are used to evaluate the quality of these

services.25 However, research indicates that:

• Evaluation instruments that have not been standardized

or validated are often used.25

• The National Committee for Quality Assurance

(NCQA) accreditation process requires that MBHOs

show that they collect, analyze, and use consumer

opinions to develop quality improvement programs, but

few instruments have been developed to collect this

information.5

Professional education

Education and training of behavioral health practitioners

have not kept up with changes in health care.26 As a result,

students are not trained to work under the conditions they

will face in actual practice, such as using evidence-based

guidelines and working in managed care practices.6

Updating graduate school curricula competes with other

demands that are made on faculty, such as competition for

research funds.27 In addition, continuing education

requirements for licensed professionals are not

standardized.28 As a result:

• Community-based mental health services are often not

available to people with mental illness, partially because

mental health professionals do not know about these

services.26,29

• New practices based on evidence are not implemented.26

• Students do not receive training in new methods and

treatment and get little experience working in outpatient

settings, substance abuse treatment, community

services, or primary care settings.27

Impact of AHRQ research

Collaborative care program improves care and

outcomes for depression

Under the AHRQ-funded Partners in Care (PIC)

collaborative-care program conducted by the PORT II for

depression, the rates of antidepressant medication use and

psychotherapy increased among patients who were

diagnosed with depression.10,30

The PIC program conducted a randomized trial comparing

quality improvement programs in clinics for medications

(QI-meds) and for psychotherapy (QI-therapy).8 Two groups

of clinics were randomly assigned to either the QI-meds or

QI-therapy group. A third group of clinics served as a

control by offering usual care for depression.8

Patients who enrolled in the PIC intervention were

encouraged by nurses and providers to choose which

treatment they preferred–either the QI-meds or the QItherapy, or no treatment.10,30,31

• The QI-meds intervention provided patients with nurse

followup assessments and support for medication

compliance for 6 or 12 months.8,30,32 QI-meds patients

could still receive psychotherapy, but they had to pay the

normal insurance copayment.8,30,32

• The QI-therapy intervention provided patients with 12-

16 sessions of cognitive behavioral therapy on a reduced

copayment basis.8,30 QI-therapy patients could still take

antidepressant medications, but they did not receive the

nurse followup.8,30

• Clinics that were providing usual care received

depression practice guidelines.30

Local expert leaders in primary care, nursing, and mental

health specialties were selected to implement the PIC

interventions and received 2-day training on

implementation.8,10, 31,33 Primary care expert leaders then

hired depression nurse specialists,who received a 1-day

training session.8 The depression nurse specialists initially

assessed patients for both the QI-meds and QI-therapy

programs and handled case management for the QI-meds

only patients.8 Psychotherapists were also hired and trained

in short-term psychotherapy for depression in patients who

chose the QI-therapy only program.8

When compared to patients who received usual care,

patients in the PIC program were twice as likely to begin

treatment within the first 6 months of being diagnosed,

were less likely to be depressed during the first year, were

more likely to remain employed after 1 year, and were more

likely to state that they had good quality of life.10,33 Results

showed that:

• Rates of counseling and appropriate use of

antidepressant medications were 30-40 percent higher

during the first 6 months and 20-30 percent higher after

1 year for intervention patients than for patients who

received usual care (Figures 1 and 2).33

• At 6 and 12 months, 7-10 percent fewer intervention

patients had depression compared to usual care patients

(Figure 3).33

www.ahrq.gov 3
4 www.ahrq.gov

Intervention Usual care

10

20

30

40

50

6 months

12 months

Baseline

29.5%

38.2%

27.3% 26.9% 25.6%

20.9%

Intervention Usual care

10

20

30

40

50

6 months

12 months

Baseline

27.6%

34.7%

31.0%

27.0% 25.1% 24.0%

20

40

60

80

6 months

12 months

Baseline

75.4%

55.4% 54.5%

75.7%

64.4% 61.4%

Figure 1. Percent of patients with specialty counseling: Partners in Care project

Source: Wells KB, Sherbourne C, Shoenbaum M, et al. Impact of disseminating quality improvement programs for depression in managed primary care. JAMA 2000; 283(2):212-20.

Intervention Usual care

10

20

30

40

50

6 months

12 months

Baseline

29.5%

38.2%

27.3% 26.9% 25.6%

20.9%

Intervention Usual care

10

20

30

40

50

6 months

12 months

Baseline

27.6%

34.7%

31.0%

27.0% 25.1% 24.0%

20

40

60

80

6 months

12 months

Baseline

75.4%

55.4% 54.5%

75.7%

64.4% 61.4%

Figure 2. Percent of patients with appropriate antidepressant medication: Partners in Care project

Source: Wells KB, Sherbourne C, Shoenbaum M, et al. Impact of disseminating quality improvement programs for depression in managed primary care. JAMA 2000; 283(2):212-20.
www.ahrq.gov 5

• Of patients who were employed at the beginning of the

intervention, 89.7 percent of intervention patients and

84.7 percent of usual care patients were still employed

after 1 year.33

• Of patients who were unemployed at the beginning of

the intervention, 16.4 percent of intervention patients

and 11.4 percent of usual care patients were working at

6 months; at 1 year, 17-18 percent of all patients had

started working.33

• After 6 months, 72 percent of patients who received

appropriate treatment (i.e., counseling or medication)

were employed, compared with 53 percent of patients

who did not receive appropriate treatment.32

• At the end of 2 years, patients who chose the QI-therapy

intervention had improved emotional well-being

compared to patients who received usual care.34

The PIC program was also successful in treating depressed

patients who had additional medical conditions such as

back problems, arthritis, hypertension, asthma, diabetes,

and heart disease.35 After 6 and 12 months, these

intervention patients were less likely to report depressive

disorders.35 In addition, after 18 months, the clinicians who

participated in the intervention training programs

demonstrated more knowledge about assessing and treating

depression than clinicians who did not participate in the

intervention training.31 

More information on the Partners in Care program can be

found at http://www.rand.org/publications/RB/RB4528/.

PIC materials, including user’s guide, clinician’s guide, and

implementation guides, are available through the PIC Web

site at http://www.rand.org/health/partners.care/portweb/.36

Toolkits to improve schizophrenia care are based on

treatment guidelines 

Based on a complete and thorough review of studies

conducted on the outcomes of treatment for schizophrenia,

AHRQ’s PORT developed evidence-based

recommendations, published in 1995, to improve the

quality and cost effectiveness of care for patients with

schizophrenia.37 Specifically, the recommendations address

interventions with antipsychotic medications; drug therapy

for anxiety, depression, and aggression/hostility;

electroconvulsive therapy; psychological interventions;

family interventions; vocational rehabilitation; and assertive

community treatment and case management.37 They have

been used in 9 States to improve statewide treatment

approaches and are the basis of two toolkits for treatment of

schizophrenia developed by the Human Services Research

Institute (HSRI) and the Department of Health and Human

Services of the State of New Hampshire.

Using AHRQ’s evidence-based recommendations on

schizophrenia, HSRI, in Cambridge, Massachusetts,

developed a toolkit with two primary purposes. First, the

Intervention Usual care

Intervention Usual care

20

40

60

80

6 months

12 months

Baseline

75.4%

55.4% 54.5%

75.7%

64.4% 61.4%

Source: Wells KB, Sherbourne C, Shoenbaum M, et al. Impact of disseminating quality improvement programs for depression in managed primary care. JAMA 2000; 283(2):212-20.

Figure 3. Percent of patients with depression: Partners in Care project
6 www.ahrq.gov

toolkit measures how well an organization implements a set

of practice recommendations. Second, it illustrates how

evaluation evidence can be used to develop

recommendations for specific disorders. Specifically, the

HSRI toolkit shows how designing a measurement strategy

based on recommendations can help promote

implementation of those recommendations. While the

toolkit is not a guide to the treatment of schizophrenia

(primarily because the recommendations will be updated),

the process it describes illustrates how to build a

recommendation-based quality assessment system. The

PORT tools for assessing care (the PORT Mental Health

Survey and the Schizophrenia PORT Inpatient and

Outpatient Record Review forms) are included in the

toolkit. The resulting performance measures and evaluation

studies should contribute to quality improvement,

enhancing outcomes, and preventing adverse events. The

toolkit is available through HSRI’s Web site at

http://www.hsri.org/.38

The New Hampshire Department of Health and Human

Services used the Schizophrenia PORT recommendations

for key policy initiatives. The New Hampshire Division of

Public Health used the PORT findings to highlight racial

disparities in the provision of care, directly influencing New

Hampshire’s Healthy People 2010 plan, which is aimed at

eliminating health care disparities. Partnering with the

Dartmouth College Psychiatric Research Center, the

Division of Mental Health restructured care protocol for

patients with schizophrenia based on the PORT findings.

The restructuring effort included the development of

evidence-based toolkits for retraining community mental

health staff.

A screening tool helps identify children and

adolescents at risk for suicide

A Risk of Suicide Questionnaire (RSQ) that can be used

during emergency room visits takes 2 minutes to detect

children and adolescents at risk for suicide.39,40 An AHRQfunded study showed that these four questions identified 98

percent of children at risk:

• Are you here because you tried to hurt yourself?

• In the past week, have you been having thoughts about

killing yourself?

• Have you ever tried to hurt yourself in the past other

than this time?

• Has something very stressful happened to you in the

past few weeks?39

The RSQ was the result of a clinical practice guideline

(CPG) program conducted at Children’s Hospital Boston.40

The program provided training to emergency room nurses

on psychiatric issues such as how to perform patient

searches, de-escalating techniques, and risk factors for

suicide.40 At the beginning of the training, emergency room

nurses at Children’s Hospital stated that they were

uncomfortable dealing with children who had psychiatric

problems or asking parents or children about thoughts of

suicide.40 In addition, 94.4 percent of nurses stated that

suicidal patients were the most difficult patients to treat.40

Two years into the CPG program, nurses reported a

significant increase in their confidence and less stress when

dealing with psychiatric patients.39,40 They also stated that

they preferred using the screening tool to the previous

method of simply judging when to ask about suicidal

behavior.39,40 Parents also expressed relief that emergency

room clinicians were asking about suicide.39

Evaluation shows that a school mental health

program lowers costs

AHRQ-supported studies helped to develop a systematic

approach to evaluating expanded school mental health

(ESMH) programs in middle and high school 22,23 and

discovered that the costs of ESMH programs were low

compared to the costs of programs in the community or

private sectors.24 A cost-of-care evaluation conducted for

one school mental health program found that clinician

services cost less than $50 per hour–much less than private

services, which were estimated to cost $100 to $120 per

hour.24

A typical program provides assessment; individual, group,

and family counseling; crisis intervention; prevention and

case management activities; teacher consultation; and

collaboration with school staff and the community.24

According to researchers funded by AHRQ, an ESMH

evaluation plan should systematically measure the

outcomes of the program’s goals, such as identifying

emotional, behavioral, and academic problems early and

improving school attendance. The following steps are

recommended in conducting ESMH program evaluations:

• Focus the evaluation on both quantitative measures

(such as changes in behavioral problems) and qualitative

measures (such as satisfaction ratings) obtained from 
www.ahrq.gov 7

adolescents, parents, teachers, school personnel, etc.

These measures should be associated with a positive

outcome (for example, improved grades or attendance).

• Ensure cultural competence, taking into consideration

program philosophy, staff ethnicity, cultural competence

training, and cultural values of the target population.

• Obtain support from the school and community by

selecting goals that are important to stakeholders.

• Use outcome indicators that address life stressors and

risk factors, protective or resilience factors, emotional

and behavioral problems, and life functioning.

• Use specific measurement strategies, which can include

self-report, reports by parents and teachers, and clinician

ratings.

• Measure the cost effectiveness of the ESMH program in

relation to traditional mental health programs.22

An advisory board of stakeholders (families, school staff

and teachers, mental health professionals, community

leaders, the community health/mental health system, and

funders) should review and revise the evaluation plan and

provide feedback on the results. This feedback should be

used for program improvement.22,23

Consumer ratings can improve behavioral health

services

An AHRQ-funded study that used the Consumer

Assessment of Behavioral Health Services (CABHS)

prompted quality improvement efforts in several managed

behavioral health care organizations.5 CABHS is based on

CAHPS®a and was developed to collect ratings from

consumers about services received from MBHOs.25

Consumer ratings of behavioral health care services can

help to evaluate and identify ways to improve behavioral

health insurance plans.5 The 56 questions in the CABHS

survey ask consumers about their experience in finding a

clinician, getting approval for treatment, getting prompt

counseling or treatment, communication with clinicians,

and customer service, as well as staff, paperwork burden,

written materials, and how much the treatment helped

patients.5 CABHS also collects information from clinicians

about medications, treatment options, and patient rights.5

Examples of questions and responses from consumers are

shown in Table 1.

The survey assessed patient satisfaction in five commercial

and five public assistance plans.5 Based on the results of

these surveys, three MBHOs implemented quality

improvement efforts.

• Patients at one MBHO reported problems in getting help

from customer service, telephone answer response time,

finding a clinician, getting care while waiting for

treatment approval from the plan, and filling out

paperwork. As a result, the MBHO added more staff

and trained its customer service personnel to answer

calls for information, check patient eligibility, authorize

an initial treatment session, help patients find a

clinician, and ensure that patients receive appropriate

and timely care. The MBHO also introduced centralized

scheduling and increased the number of clinician hours

to reduce waiting times for new patient evaluations.5

• A second MBHO found that 13-30 percent of its

members were not given adequate information about

patient rights. The MBHO implemented projects to

improve dissemination of information about patient

rights, including consulting with members and

clinicians, posting patient rights in clinics, and

interviewing providers to find out if patient rights

information is being provided.5

• A third MBHO developed educational materials for its

members and clinicians to improve the patients’ ability

to get timely appointments.5

Based on the CABHS data collected in this AHRQ-funded

study, along with data from a previous study using the

Mental Health Statistics Improvement Program (MHSIP),

researchers developed the Experience of Care and Health

Outcomes (ECHOTM).5

The ECHOTM 3.0 has two surveys–one for MBHOs and

one for health plans.5,41 Both surveys ask the same

questions, but the health plan survey includes questions

about administrative services, such as filling out paperwork

and finding information in written materials.42 The surveys

can be downloaded for free from the Web site:

http://www.hcp.med.harvard.edu/echo/home.html.

Consumers, clinicians, MBHOs, health care plans,

purchasers, States, and Federal agencies can collect

consumers’ ratings of their behavioral health treatment,

a

CAHPS® (Consumer Assessment of Health Plans) is a family of

rigorously tested and standardized questionnaires and reporting formats

that can be used to collect and report meaningful and reliable

information about the experiences of consumers with a variety of health

services. For more information, see AHRQ’s Web site at

http://www.ahrq.gov/qual/cahpsix.htm.
8 www.ahrq.gov

including mental health and alcohol, drug, and other

substance abuse services.42 ECHOTM assesses specific

aspects of care, including getting care quickly,

communication with clinicians, information provided by

clinicians on medication side effects, family involvement in

care, information about self-help groups and treatment,

cultural competence, perceived improvement in

functioning, patient rights, and health plan or MBHO

administrative services.42

ECHOTM has been adopted as a CAHPS® instrument.43 In

addition, NCQA adopted ECHOTM 1.1H, which excludes

13 questions from the original MBHO version,41 and made

it a part of the NCQA Quality Assurance Plan HEDIS®

(the Health Plan Employer Data and Information Set) 2003

Survey Measures.44 ECHOTM 1.1H was designed to target

quality assessment by MBHOs and assesses aspects of care

such as overall satisfaction with counseling and treatment,

getting treatment quickly, communication with clinicians,

access to treatment and information from the MBHO,

office wait times, ability to refuse treatment, and receipt of

information about treatment options, medication side

effects, managing conditions, and patient rights.44

A conference provides solutions to improve provider

training and continuing education 

AHRQ cosponsored the National Conference on Behavioral

Health Workforce Education in Annapolis, Maryland,

September 10-11, 2001. The proceedings from this

conference resulted in a special double-issue volume of

Administration and Policy in Mental Health in May 2002.

The proceedings present problems and solutions to

educating behavioral health professionals, including:

• Clarifying the issues of the behavioral health care

system and the need for change.6,45,46

• Educational best practices for practitioners, patients,

consumers, and families.26,47

• Recommendations to enhance graduate education so

that practitioners develop the competencies necessary

for current practice.27

• Methods to improve continuing education practices.28

Table 1. Consumer Assessment of Behavioral Health Services results

Plan rating (range)

Question Commercial Public assistance 

Percent stating “Not a problem”

How much of a problem did you have with paperwork for

your health plan? 28-52% 33-50%

How much of a problem was it to get help when you called 

your health plan’s customer service? 42-62% 39-79%

How much of a problem were delays in treatment while 

waiting for approval from your health plan? 59-86% 64-84%

With the choices your health plan gave, how much of a 

problem was it to get a clinician you are happy with? 48-75% 50-80%

Percent stating “Yes”

Did your clinicians tell you that you have the right to refuse 

treatment that you do not want? 66-73% 65-80%

Percent stating “Always”

When you needed to see a clinician right away, how often did 

you get the treatment or counseling as soon as wanted? 42-68% 48-57%

Source: Shaul JA, Eisen SV, Stringfellow VL, et al. Use of consumer ratings for quality improvement in behavioral health insurance plans. Jt Comm J Qual Improv 2001; 27(4):216-29.
Ongoing research

AHRQ continues to fund and sponsor many research

projects involving mental health services. For a

comprehensive list and summary of past and ongoing

projects, please see the Compendium of AHRQ Research

Related to Mental Health, AHRQ Pub. No. 03-0001. The

HTML version can be found on AHRQ’s Web site at

http://www.ahrq.gov/research/mentalcomp/. The PDF

version can be downloaded at http://www.ahrq.gov/

research/mentalcomp/mentalcompend.pdf. The PDF

version is indexed by topic as well as by principal

investigators and authors.

Conclusion

AHRQ’s wide range of mental health research has produced

programs, methods, and tools that can improve the quality

of mental health services. The PIC program allows people

suffering from depression to collaborate in their care with

their providers and has been shown to improve outcomes.

Providers who treat schizophrenia patients have toolkits that

use evidence-based treatment recommendations to assess

the care and treatment they provide. AHRQ research

supported the development of a simple tool to screen

adolescents at risk for suicide. Research is also directed

toward finding ways to evaluate and promote quality

improvement programs for mental health services in

school-based programs and MBHO’s. In addition, AHRQ’s

research helps identify solutions that can improve education

and training for mental health care professionals. 

For more information

For further information on mental health research, 

contact:

Charlotte Mullican, B.S.W., M.P.H., at cmullica@ahrq.gov

or by telephone at 301-427-1495.

References

1. National Institute of Mental Health. The numbers count.

Mental disorders in America. NIH Pub No. 01-4584.

Web site:

http://www.nimh.nih.gov/publicat/numbers.cfm.

Accessed Apr 7, 2003.

2. Centers for Disease Control and Prevention. Web-based

Injury Statistics Query and Reporting System

(WISQARSTM) [online database]. National Center for

Injury Prevention and Control; 2002. Web site:

www.cdc.gov/ncipc/wisqars. Accessed Apr 8, 2003.

3. National Institute of Mental Health. Schizophrenia. Web

site: http://www.nimh.nih.gov/publicat/schizoph.cfm.

Accessed Apr 2, 2003.

*4. Lehman AL. Quality of care in mental health: the case

of schizophrenia. Health Aff 1999; 18(5):52-65.

*5. Shaul JA, Eisen SV, Stringfellow VL, et al. Use of

consumer ratings for quality improvement in behavioral

health insurance plans. Jt Comm J Qual Improv 2001;

27(4):216-29.

*6. Hoge MA. The training gap: an acute crisis in behavioral

health education. Adm Policy Ment Health 2002;

29(4/5):305-17.

*7. Olfson M, Marcus SC, Druss B, et al. National trends in

the outpatient treatment of depression. JAMA 2002;

287(2):203-9.

*8. Rubenstein LV, Jackson-Triche M, Unutzer J, et al.

Evidence-based care for depression in managed primary

care practices. Health Aff 1999; 18(5):89-105.

*9. Wells KB, Schoenbaum M, Unutzer J, et al. Quality of

care for primary care patients with depression in

managed care. Arch Fam Med 1999; 8:529-36.

10. RAND Health. Partners In Care. Hope for those who

struggle with hope. Research Highlights. Web site:

http://www.rand.org/publications/RB/RB4528. Accessed

Oct 23, 2002.

*11. Garrard J, Rolnick SJ, Nitz NM, et al. Clinical detection

of depression among community-based elderly people

with self-reported symptoms of depression. J Gerontol

1998; 53A(2):M92-M101.

*12. Callahan CM, Dittus RS, Tierney WM. Primary care

physicians’ medical decision making for late-life

depression. J Gen Intern Med 1996; 11:218-25.

*13 Dixon L, Green-Paden L, Delahanty J, et al. Variables

associated with disparities in treatment of patients with

schizophrenia and comorbid mood and anxiety disorder.

Psychiatr Serv 2001; 52(9):1216-22.

*14. Dixon L, Lyles A, Smith C, et al. Use and costs of

ambulatory care services among Medicare enrollees

with schizophrenia. Psychiatr Serv 2001; 52(6):786-92.

15. Public Health Service [U.S.]. The Surgeon General’s call

to action to prevent suicide, 1999. Web site:

http://www.surgeongeneral.gov/library/calltoaction/

default.htm. Accessed Apr 15, 2003.

*AHRQ-funded/sponsored research

www.ahrq.gov 9
10 www.ahrq.gov

16. National Institute of Mental Health. Suicide facts. 2003.

Web site: http://www.nimh.nih.gov/research/suifact.htm.

Accessed Apr 15, 2003.

17. Hacker K, Drainoni ML. Mental health and illness in

Boston’s children and adolescents: one city’s experience

and its implications for mental health policy makers.

Public Health Rep 2001; 116(4):317-26.

18. Frankenfield DL, Keyl PM, Gielen A, et al. Adolescent

patients—healthy or hurting? Missed opportunities to

screen for suicide risk in the primary care setting. Arch

Pediatr Adolesc Med 2000; 154(2):162-8.

19. Nock MK, Kazdin AE. Examination of affective,

cognitive, and behavioral factors and suicide-related

outcomes in children and young adolescents. J Clin

Child Adolesc Psychol 2002; 31(1):48-58.

20. Sampson SM, Mrazek DA. Depression in adolescence.

Curr Opin Pediatr 2001; 13:586-90.

21. Catallozzi M, Pletcher JR, Schwarz DF. Prevention of

suicide in adolescents. Curr Opin Pediatr 2001; 

13:417-22.

*22. Weist MD, Nabors LA, Myers CP, et al. Evaluation of

expanded school mental health programs. Community

Ment Health J 2000; 36(4):395-411.

*23. Nabors LA, Weist MD, Reynolds MW. Overcoming

challenges in outcome evaluations of school mental

health programs. J Sch Health 2000; 70(5):206-9.

*24. Nabors LA, Leff SS, Mettrick JE. Assessing the costs of

school-based mental health services. J Sch Health 2001;

71(5)199-200.

*25. Eisen SV, Shaul JA, Clarridge B, et al. Development of a

consumer survey for behavioral health services.

Psychiatr Serv 1999; 50(6):793-8.

*26. Stuart GW, Burland J, Ganju V, et al. Educational best

practices. Adm Policy Ment Health 2002; 29(4/5):

325-33.

*27. Hoge MA, Jacobs S, Belitsky R, et al. Graduate

education and training for contemporary behavioral

health practice. Adm Policy Ment Health 2002;

29(4/5):335-57.

*28. Daniels AS, Walter DA. Current issues in continuing

education for contemporary behavioral health practice.

Adm Policy Ment Health 2002; 29(4/5):359-76.

29. Department of Health and Human Services [U.S.].

Mental health: a report of the Surgeon General—

executive summary . Rockville, MD: Substance Abuse

and Mental Health Services Administration, Center for

Mental Health Services, National Institutes of Health,

National Institute of Mental Health. 1999. Web site:

http://www.surgeongeneral.gov/library/mentalhealth/

summary.html. Accessed Apr 9, 2003.

*30. Dwight-Johnson M, Unutzer J, Sherbourne C, et al. Can

quality improvement programs for depression in primary

care address patient preferences for treatment? Med

Care 2001 Sep; 39(9):934-44.

*31. Meredith LS, Jackson-Triche M, Duan N, et al. Quality

improvement for depression enhances long-term

treatment knowledge for primary care clinicians. J Gen

Intern Med 2000 Dec; 15(12):868-77.

*32. Schoenbaum M, Unutzer J, McCaffrey D, et al. The

effects of primary care depression treatment on patients’

clinical status and employment. Health Serv Res 2002;

37(5):1145-58.

*33. Wells KB, Sherbourne C, Shoenbaum M, et al. Impact

of disseminating quality improvement programs for

depression in managed primary care. JAMA 2000;

283(2):212-20.

*34. Sherbourne CD, Wells KB, Duan N, et al. Long-term

effectiveness of disseminating quality improvement for

depression in primary care. Arch Gen Psychiatry 2001;

58:696-703.

*35. Koike AK, Unutzer J, Wells KB. Improving the care for

depression in patients with comorbid medical illness.

Am J Psychiatry 2002; 159:1738-45.

*36. Vaiana M. RAND. E-mail correspondence to Barbara

Kass-Bartelmes, Agency for Healthcare Research and

Quality. July 17, 2003.

*37. Agency for Healthcare Quality and Research. The

Schizophrenia Patient Outcomes Research Team (PORT)

treatment recommendations. Rockville, MD. 1998. Web

site: http://www.ahrq.gov/clinic/schzrec.htm. Accessed

Sept 9, 2003.

*38. Lehman A, Steinwachs D. Measuring conformance to

treatment guidelines: the example of the schizophrenia

PORT. Aug 2001. Pub. Code PN-46. Schizophrenia

Patient Outcomes Research Team, University of

Maryland School of Medicine. Web site:

http://www.hsri.org. Accessed Apr 2, 2003.

*39. Horowitz LM, Wang PS, Koocher GP, et al. Detecting

suicide risk in a pediatric emergency department:

development of a brief screening tool. Pediatrics 2001;

107(5):1133-7.

*AHRQ-funded/sponsored research
*40. O’Neill K, Horowitz LM, Smith MF, et al. Recognizing

suicide risk in a pediatric emergency department: a

change in nursing care. Pediatr Emerg Care 2001;

17(4):306-9.

41. Shaul J. Department of Health Care Policy, Harvard

Medical School. E-mail communication to: Barbara

Kass-Bartelmes. Mar 18, 2003.

42. Department of Health Care Policy, Harvard Medical

School. ECHOTM Experience of Care and Health

Outcomes Survey. Web site: http://www.hcp.med.

harvard.edu/echo/home.html. Accessed June 18, 2004.

43. Darby C. Agency for Healthcare Research and Quality.

E-mail communication to: Barbara Kass-Bartelmes.

Mar 18, 2003.

44. National Committee for Quality Assurance. NCQA

Quality Assurance Plan. HEDIS® 2003 Survey

Measures. Web site: http://www.ncqa.org/programs/

hedis/2003qap.doc. Accessed Mar 18, 2003.

*45. Huey LY. Problems in behavioral health care: leapfrogging the status quo. Adm Policy Ment Health 2002;

29(4/5):403-19.

*46. O'Connell MJ, Gill D, Artar A, et al. Student voices:

perspectives on behavioral health education. Adm Policy

Ment Health 2002; 29(4/5):421-34.

*47. Morris JA, Stuart GW. Training and education needs of

consumers, families, and front-line staff in behavioral

health practice. Adm Policy Ment Health 2002;

29(4/5):377-402.

*AHRQ-funded/sponsored research

www.ahrq.gov 11

Previous issues of Research in Action are available for free from the AHRQ Publications Clearinghouse: 1-800-358-9295.

Please specify the AHRQ publication number when you call.

Issue Title Publication Number

15 Women and Domestic Violence: Programs and Tools That AHRQ 04-0055

Improve Care for Victims

14 Hospital Nurse Staffing and Quality of Care AHRQ 04-0029

13 Dental Care: Improving Access and Quality AHRQ 03-0040

12 Advance Care Planning: Preferences for Care at the End of Life AHRQ 03-0018

11 AHRQ Tools for Managed Care AHRQ 03-0016

10 AHRQ Tools and Resources for Better Health Care AHRQ 03-0008

9 Reducing Costs in the Health Care System: Learning From AHRQ 02-0046

What Has Been Done

8 Prescription Drug Therapies: Reducing Costs and Improving AHRQ 02-0045

Outcomes

7 Improving Treatment Decisions for Patients with Community- AHRQ 02-0033

Acquired Pneumonia

6 Medical Informatics for Better and Safer Health Care AHRQ 02-0031

5 Expanding Patient-Centered Care to Empower Patients and AHRQ 02-0024

Assist Providers

4 Managing Osteoarthritis: Helping the Elderly Maintain Function AHRQ 02-0023

and Mobility

3 Preventing Disability in the Elderly With Chronic Disease AHRQ 02-0018

2 Improving Care for Diabetes Patients Through Intensive AHRQ 02-0005

Therapy and a Team Approach

1 Reducing and Preventing Adverse Drug Events To Decrease AHRQ 01-0020

Hospital Costs
U.S. Department of

Health and Human Services

Public Health Service

Agency for Healthcare Research and Quality

540 Gaither Road

Rockville, Maryland 20850

Official Business

Penalty for Private Use $300

www.ahrq.gov

AHRQ Pub. No. 04-0061

July 2004