Evaluating

Your Program

MedTEAM

U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES

Substance Abuse and Mental Health Services Administration

Center for Mental Health Services

www.samhsa.gov

Evaluating MedTEAM

Your Program

U.S. Department of Health and Human Services

Substance Abuse and Mental Health Services Administration

Center for Mental Health Services
Acknowledgments

This document was prepared for the Substance Abuse and Mental Health Services 

Administration (SAMHSA) by the New Hampshire-Dartmouth Psychiatric Research Center under 

contract number 280-00-8049 and Westat under contract number 270-03-6005, with SAMHSA, 

U.S. Department of Health and Human Services (HHS). Pamela Fischer, Ph.D., and Crystal 

Blyler, Ph.D., served as SAMHSA Government Project Officers.

Disclaimer

The views, opinions, and content of this publication are those of the authors and contributors 

and do not necessarily reflect the views, opinions, or policies of the Center for Mental Health 

Services (CMHS), SAMHSA, or HHS.

Public Domain Notice

All material appearing in this document is in the public domain and may be reproduced 

or copied without permission from SAMHSA. Citation of the source is appreciated. However, 

this publication may not be reproduced or distributed for a fee without the specific, written 

authorization from the Office of Communications, SAMHSA, HHS.

Electronic Access and Copies of Publication

This publication may be downloaded or ordered at http://www.samhsa.gov/shin. Or, please call 

SAMHSA’s Health Information Network at 1-877-SAMHSA-7 (1-877-726-4727) (English 

and Español).

Recommended Citation

Substance Abuse and Mental Health Services Administration. MedTEAM: Evaluating Your Program. 

HHS Pub. No. SMA-10-4548, Rockville, MD: Center for Mental Health Services, Substance Abuse 

and Mental Health Services Administration, U.S. Department of Health and Human Services, 2010.

Originating Office

Center for Mental Health Services

Substance Abuse and Mental Health Services Administration

1 Choke Cherry Road

Rockville, MD 20857

HHS Publication No. SMA-10-4548

Printed 2010
MedTEAM

Evaluating Your Program

Evaluating Your Program shows quality assurance team 

members how to evaluate the effectiveness of Medication 

Treatment, Evaluation, and Management (MedTEAM). 

It includes the following:

 A Readiness Assessment; 

 The MedTEAM Organizational Fidelity Scale;

 The MedTEAM Prescriber Fidelity Scale; and

 Outcome measures that are specific to MedTEAM. 

You will also find instructions for conducting assessments 

and tips on how to use the data to improve your 

MedTEAM initiative.

For references, see the booklet, The Evidence.
This KIT is part of a series of Evidence-Based Practices KITs created 

by the Center for Mental Health Services, Substance Abuse and 

Mental Health Services Administration, U.S. Department of Health 

and Human Services.

This booklet is part of the MedTEAM KIT that includes a DVD, 

CD-ROM, and seven booklets:

How to Use the Evidence-Based Practices KITs

Getting Started with Evidence-Based Practices

Building Your Program

Training Frontline Staff

Evaluating Your Program

The Evidence

Using Multimedia to Introduce Your EBP
MedTEAM

What’s in Evaluating Your Program

Why Evaluate MedTEAM?................................................... 1

Conduct a Readiness Assessment ......................................... 3

Conduct a Process Assessment ............................................. 5

Monitor Outcomes ........................................................... 13

Use Data To Improve MedTEAM........................................ 19

Appendix A: Cover Sheet—MedTEAM Fidelity Scales .......... 23

Appendix B: MedTEAM Organizational 

Fidelity Scale and Scoresheet............................................. 27

Appendix C: Interview Guides: MedTEAM 

Organizational Fidelity Scale ............................................. 35

Appendix D: MedTEAM Prescriber Fidelity 

Scale and Scoresheet ........................................................ 55

Appendix E: Chart Review Guide: MedTEAM 

Prescriber Fidelity Scale .................................................... 61

Appendix F: Outcomes Report Form .................................. 85

Appendix G: Instructions for the Outcomes 

Report Form .................................................................... 89

Appendix H: Assessor Training and Work 

Performance Checklist....................................................... 97

Evaluating Your Program 1 Why Evaluate MedTEAM

Evaluating Your Program

Why Evaluate MedTEAM?

Key stakeholders who implement 

MedTEAM may find themselves asking 

two questions: 

	Has MedTEAM been implemented 

as planned?

	Has MedTEAM resulted in the 

expected outcomes?

Asking these two questions and using the 

answers to help improve MedTEAM are 

critical for ensuring the success of your 

MedTEAM initiative. 

To answer the first question, collect 

process measures (by using the 

MedTEAM Fidelity Scales). Process 

measures capture how services are 

provided. To answer the second question, 

collect outcome measures. Outcome 

measures capture the results 

or achievements of your initiative. 

As you prepare to implement MedTEAM, 

we strongly recommend that you develop 

a quality assurance system using both 

process and outcome measures to monitor 

and improve the quality of your MedTEAM 

initiative beginning during the startup 

phase and continuing thereafter. 
Why Evaluate MedTEAM 2 Evaluating Your Program

Why you should collect 

process measures 

Process measures give you an objective, structured 

way to determine if you are implementing 

MedTEAM in the way that research has shown 

will result in desired outcomes. Process measures 

allow agencies to understand whether they are 

providing medication management that is faithful 

to the evidence-based model. Programs that adhere 

closely to the model are more effective than those 

that do not follow the model. Adhering to the model 

is called fidelity. 

Collecting process measures is an excellent way 

to diagnose program weaknesses while helping 

to clarify program strengths. Once MedTEAM 

reaches high fidelity, ongoing monitoring allows 

you to test local innovations while ensuring that 

your agency does not drift from the core principles 

of the evidence-based model.

Process measures also give mental health 

authorities a comparative framework to evaluate 

the quality of MedTEAM across the state. 

They allow mental health authorities to identify 

statewide trends and exceptions to those trends. 

Why you should collect 

outcome measures

While process measures capture how services are 

provided, outcome measures capture the program’s 

results. Every mental health service intervention 

has both immediate and long-term consumer goals. 

In addition, consumers have goals for themselves, 

which they hope to attain by receiving mental 

health services. These goals translate into outcomes, 

and the outcomes translate into specific measures. 

Consumer outcomes are the bottom line for 

mental health services, like profit in business. 

No successful businessperson would assume that 

the business was profitable just because employees 

work hard.

Research Has Shown That You 

Can Expect These Outcomes

	Improved symptoms

	Improved quality of care

	Increased retention of employment

	Improved satisfaction and adherence 

to treatment

(Katon et al., 1995; Simon, Von Korff, Rutter, 

& Wagner, 2000; Wells et al., 2000; Worrall, 

Angel, Chaulk, Clarke, & Robbins, 1999.)

Why develop a quality 

assurance system?

In your mental health system, you should develop 

a quality assurance system that collects not only 

process measures, such as those on the MedTEAM 

Fidelity Scales, but also outcome measures, such 

as those specified above to show the effect of 

MedTEAM. Developing a quality assurance 

system will help you do the following:

	Diagnose your program’s strengths 

and weaknesses; 

	Formulate action plans for improving 

your MedTEAM initiative;

	Help consumers achieve their goals 

for recovery; and

	Deliver medication treatment both 

efficiently and effectively.
Evaluating Your Program 3 Conduct a Readiness Assessment Why Evaluate MedTEAM

Evaluating Your Program

Conduct a Readiness Assessment

Let’s assume that administrators and 

MedTEAM leaders have read Building 

Your Program. How do you get started 

with putting MedTEAM into place?

The Readiness Assessment on the next 

page will help quality assurance team 

members, advisory group leaders, and 

MedTEAM leaders track the processes 

and administrative tasks required to 

develop your MedTEAM initiative. 

Answering these questions will help 

you generate an ongoing to-do list 

(or implementation plan) to guide your 

steps in implementing MedTEAM.
Why Evaluate MedTEAM Conduct a Readiness Assessment 4 Evaluating Your Program

Readiness Assessment

Check any areas that you feel you do NOT completely understand.

q	Who will participate in MedTEAM?

q	What are the roles of the MedTEAM leader, 

prescribers, and other MedTEAM staff? 

q	Who will supervise MedTEAM staff and direct 

the program?

q	What is the MedTEAM supervisory structure 

(for example, how often does the MedTEAM leader 

meet with prescribers and other MedTEAM staff 

and the agency director)?

q	How will MedTEAM staff communicate with 

one another?

q	How will you measure fidelity to the 

MedTEAM model?

q	Who will participate on your MedTEAM advisory 

committee or the group that will review your 

fidelity assessment results and develop your 

implementation plan?

q	How will MedTEAM staff relate to your 

advisory committee?

q	Who will lead the process of reviewing and 

changing documentation?

q	Who will lead the process of reviewing and developing 

procedures related to accessing information from 

outside agencies?

q	Who will lead the process for reviewing and 

developing treatment guidelines or algorithms 

including those for consumers with treatmentrefractory illnesses? 

q	Who will lead the process for reviewing and 

developing procedures for consumer education 

and participation?

q	Who will lead the process for reviewing 

and developing procedures about 

scheduling appointments?

q	Who will lead the process for reviewing and 

developing procedures for integrating outcome 

measures into clinical assessments?

q	Who will be responsible for ensuring that 

outcomes data are also available for the purpose of 

evaluating MedTEAM?

Note areas where you still are unclear or have questions. 

Arrange to speak to an expert consultant or experienced MedTEAM leader.
Evaluating Your Program 5 Conduct a Process Assessment

Evaluating Your Program

Conduct a Process Assessment

In addition to the Readiness Assessment, 

conduct your first process assessment 

to determine whether your agency has 

core components of MedTEAM in place. 

During the first 2 years of implementing 

MedTEAM, conduct assessments 

every 6 months.

After your MedTEAM initiative has 

matured and achieved high fidelity, you 

may choose to conduct assessments once 

a year. Agencies that have successfully 

implemented MedTEAM indicate that 

you must continue to evaluate the process 

to ensure that you do not revert to 

previous practice patterns.

Once your agency has achieved high 

fidelity to the evidence-based model, 

MedTEAM staff may tailor the program 

to meet individual needs of the community. 

If you continue to use process evaluations 

along with outcomes monitoring, you will 

be able to understand the extent to which 

your changes result in your agency’s 

departure from model fidelity and whether 

the changes positively or negatively 

affect consumers.
Conduct a Process Assessment 6 Evaluating Your Program

How to use process measures

Two tools have been developed to monitor how 

medication management is provided under the 

MedTEAM approach:

	The MedTEAM Organizational Fidelity Scale; 

and 

	The MedTEAM Prescriber Fidelity Scale. 

You may administer both tools at the same time. 

The MedTEAM Organizational Fidelity Scale 

has 15 program-specific items. Each item is rated 

on a 5-point scale, ranging from 1 (meaning not 

implemented) to 5 (meaning fully implemented). 

The items assess whether MedTEAM is provided 

as the evidence-based model prescribes.

The MedTEAM Prescriber Fidelity Scale is a 

second set of process measures that has been 

developed. In contrast to the Organizational 

Fidelity Scale that assesses agency’s procedures 

and policies, the Prescriber Fidelity Scale assesses 

prescriber practices by reviewing documentation. 

The MedTEAM Prescriber Fidelity Scale has 

22 items that may be scored. Each item is rated 

0 (meaning not implemented) or 1 (meaning 

fully implemented).

For the MedTEAM Fidelity Scales, see 

Appendices B and D. You can also print 

these forms from the CD-ROM in the KIT.

About the process measures that are included in the KIT

Quality assurance measures have been developed and are included in all Evidence-Based Practices KITs. 

The MedTEAM Fidelity Scales were developed by an expert panel. The panel members had expertise in 

psychopharmacology and in delivering medication services in public mental health settings. The standards 

used for establishing the anchors for the fully implemented ratings were determined through a variety of 

expert sources as well as through empirical research.

The scale has undergone numerous drafts and review by many groups. Revisions were made based on 

feedback from a variety of sources. The scales were field tested in 26 community mental health centers 

in four states. A psychometric evaluation of the MedTEAM Organizational Fidelity Scale indicated that for 

the total scale the reliability across all sites was .89 (Howard et al., in press; Taylor et al., 2009). Item-level 

interrater reliability was generally satisfactory with most intraclass correlations exceeding .80.

A psychometric evaluation of the MedTEAM Prescriber Fidelity Scale indicated that for the total scale the 

interrater reliability was .87 (Howard et al., in press; Taylor et al., 2009). Item-level interrater reliability 

was generally satisfactory with all but four items achieving intraclass correlations of .80 or higher. Testretest reliability for 12 sites for the total scale was .94. Test-retest reliability for individual fidelity items 

ranged from .15 to .95 with a mean of .77. Further testing is underway.
Evaluating Your Program 7 Conduct a Process Assessment

Who can conduct process assessments?

We recommend enlisting two assessors to conduct 

your process assessment. Data collected by two 

assessors simultaneously increases the likelihood 

that the information will be reliable and valid. 

Agencies that have successfully implemented 

MedTEAM have taken different approaches to 

identify assessors. Some agencies train MedTEAM 

Advisory Committee members as assessors and 

rotate the responsibility of completing assessments. 

Others have pre-existing quality assurance teams 

and simply designate members of the team to 

complete the assessments. In other cases, the 

mental health authority has designated staff 

to conduct assessments. 

You can conduct assessments either internally 

through your agency or externally by a review 

group. External review groups have a distinct 

advantage because they use assessors who are 

familiar with the MedTEAM model but, at the 

same time, are independent. The goal is to select 

objective and competent assessors. 

Although we recommend using external assessors, 

agencies can also use fidelity scales to rate their 

own MedTEAM initiative. The validity of these 

ratings (or any ratings, for that matter) depends 

on the following:

	The knowledge of the person making 

the ratings; 

	Access to accurate information pertaining 

to the ratings; and 

	The objectivity of the ratings. 

If you do conduct your assessments using internal 

staff, beware of potential biases of raters who are 

invested in seeing the agency look good or who do 

not fully understand the MedTEAM model. It is 

important that ratings are made objectively and 

that they are based on hard evidence. 

Circumstances will dictate decisions in this area, 

but we encourage agencies to choose a review 

process that fosters objectivity in ratings, for 

example, by involving a practitioner who is not 

centrally involved in the MedTEAM initiative. 

Only people who have experience and training 

in interviewing and data collection procedures 

(including chart reviews) should conduct 

assessments. Additionally, assessors must 

understand the nature and critical ingredients 

of the evidence-based model. 

If your agency chooses to use a consultant or 

trainer to help implement MedTEAM, involving 

that person in the assessment process will enhance 

the technical assistance you receive. Whichever 

approach you choose, we encourage you to make 

these decisions early in the planning process. 

For a checklist to help evaluate assessors’ training 

and work performance, see Appendix H.

How to conduct process assessments

A number of activities take place before, during, 

and after a process assessment. In general, 

assessments include the following:

	Interviewing administrators, the MedTEAM 

leader, prescribers, other MedTEAM staff, 

consumers, and families;

	Interviewing other agency staff (such as intake 

specialists, appointment schedulers, or medical 

records staff);

	Observing a treatment team and supervisory 

meeting; and 

	Conducting a chart review.

Collecting information from multiples sources 

helps assessors more accurately capture how 

services are provided. A daylong site visit is the 

best way to learn this information. The following 

suggestions outline steps in the assessment process.
Conduct a Process Assessment 8 Evaluating Your Program

Before the process assessment

			Prepare your assessment questions

The Interview and Chart Review Guides 

in Appendices C and E provide questions 

that you may use to collect information 

during your assessment visit. 

The guide helps you understand each item 

on the MedTEAM Fidelity Scales and the 

types of information to collect during your 

assessment. Use the guides to prepare the 

questions that you will ask during your 

assessment visit. 

While we expect that quality assurance 

teams will select which outcome measures 

meet your agency’s needs, you should use 

the MedTEAM Fidelity Scales in full. 

Collecting data for all the items on these 

scales will allow your agency to gain a 

comprehensive understanding of how 

closely your program resembles the 

evidence-based approach. 

			Create a timeline for the assessment 

List all the necessary activities leading 

up to and during the visit and create 

a timeline for completing each task. 

Carefully coordinating efforts, particularly 

if you have multiple assessors, will help 

you complete your assessment in a 

timely fashion. 

			Establish a contact person

Have one key person arrange your visit 

and communicate beforehand the purpose 

and scope of your assessment to people 

who will participate in interviews. 

Typically, this contact person will be 

the MedTEAM leader. 

Exercise common courtesy and show 

respect for competing time demands by 

scheduling well in advance and making 

reminder calls to confirm interview dates 

and times.

			Establish a shared understanding with 

the MedTEAM staff

The most successful assessments are those 

in which assessors and the MedTEAM 

staff share the goal of understanding 

how medication treatment is progressing 

according to evidence-based principles. 

If administrators or prescribers fear that 

they will lose funding or look bad if they 

don’t score well, then the accuracy of 

the data may be compromised. The best 

agreement is one in which all parties are 

interested in learning the truth.

			Indicate what you will need from 

respondents during your visit

In addition to explaining the purpose 

of the assessment, briefly describe what 

information you need, with whom you 

must speak, and how long each interview 

will take to complete. The visit will be 

most efficient if the MedTEAM leader 

gathers beforehand as much of the 

following information as possible:

	Roster of MedTEAM staff (roles, 

full-time equivalents);

	Copies of any rating scales 

used to document outcomes of 

medication treatment;

	A copy of the agency’s admissions form;

	A copy of the agency’s annual 

update form (if different from the 

admissions form);

	A copy of the agency’s ongoing 

treatment form; and

	A copy of consumer 

education materials.

Reassure the MedTEAM leader that you 

will be able to conduct the assessment, 

even if all of the information you requested 

is unavailable. 
Evaluating Your Program 9 Conduct a Process Assessment

Tell the contact person that you must 

observe a treatment team meeting and 

group supervision meeting during your 

visit. These are important factors in 

determining when you should schedule 

your visit. 

			Alert your contact person that you will 

need to sample 10 charts

From an efficiency standpoint, it is 

preferable that the charts be drawn 

beforehand, using a random selection 

procedure. A concern may arise that 

the evaluation may be invalidated if 

MedTEAM staff members handpick charts 

or update them before the visit. If you 

both understand that the goal is to learn 

how the program is implementing services, 

this is less likely to occur. 

You can further ensure random selection 

by asking for 20 charts and randomly 

selecting 10 to review. Other options 

include asking the MedTEAM leader 

for a de-identified list of consumers 

(that is, with names removed) and using 

the list to choose 10 charts to review. 

Since the MedTEAM Prescriber Fidelity 

Scale was designed for consumers with 

schizophrenia and adapted for those with 

major depression and bipolar disorders, 

limit your chart selection to consumers 

with those illnesses. 

			Clarify reporting procedures

With the appropriate people (agency 

administrators, the mental health 

authority, or the MedTEAM leader), 

clarify who should receive a report of the 

assessment results. Recipients may include 

the following:

	Agency administrators; 

	Members of the agency’s quality 

assurance team;

	Members of the MedTEAM 

advisory committee; 

	The MedTEAM leader;

	MedTEAM staff; and

	Consumers and families. 

Assessors should also clarify how the 

agency would like the report to be 

distributed. For example, assessors may 

mail or fax the report and follow up to 

discuss the results in a meeting or by 

conference call. 

			Organize your assessment materials

Three forms have been created to help you 

conduct your assessment: 

	The first form is a cover sheet for the 

MedTEAM Fidelity Scales, which is 

intended to help you organize your 

process assessment. It captures general 

descriptive information about the 

agency and data collection. 

	The second and third forms are 

scoresheets for the two scales. They 

help you compare assessment ratings 

from one time period to the next. They 

may also be useful if you are interested 

in graphing results to examine your 

progress over time.

For the MedTEAM Fidelity Scales, cover 

sheet, and scoresheets, see Appendices A, 

B, and D. You can also print these forms 

from the CD-ROM in the KIT. 
Conduct a Process Assessment 10 Evaluating Your Program

During your assessment visit

			Tailor your terminology

To avoid confusion during your interviews, 

tailor the terminology you use. For example, 

an agency may use member for consumer

or clinician for practitioner. Every agency 

has specific job titles for particular staff 

roles. By adopting the local terminology, 

you will improve communication.

			Conduct your chart review

It is important that your chart review is 

conducted from a representative sample 

of charts. When you begin your chart 

review, note whether your sample 

includes consumer charts from each 

prescriber’s caseload. If your random 

sample is not representative in this 

manner, consider supplementing your 

sample with selected charts that will 

increase its representativeness. 

While the MedTEAM Prescriber Fidelity 

Scale was developed specifically for 

consumers with schizophrenia, items have 

been adapted for consumers with major 

depressive and bipolar disorders. Start by 

limiting your chart review to records of 

consumers with these diagnoses. 

As your MedTEAM initiative evolves, 

consider adding more charts to your 

review if you are interested in expanding 

your review to those with other disorders. 

In this case, use only those items in the 

fidelity scales that are not specific to 

particular diagnoses and medications. 

This has the obvious disadvantage that 

important aspects of medication treatment 

are not measured, but at least it provides 

a metric of some key aspects of medication 

treatment that can be used with any chart 

review for any serious mental illness.

			If discrepancies between sources occur, 

query the MedTEAM leader

The most common discrepancy is likely 

to occur when the MedTEAM leader’s 

interview gives a more idealistic picture 

of the team’s functioning than the 

chart and observational data indicate. 

For example, on the MedTEAM 

Organizational Fidelity Scale, Consumer 

Education (Item 8) assesses whether 

educational materials are available and 

the procedures developed for distributing 

them. Your interviews may indicate 

that little educational information is 

distributed, while the MedTEAM leader 

may relay that prescribers provide 

educational information. 

To understand and resolve this discrepancy, 

ask the MedTEAM leader about it by saying: 

Our interviews indicate that 20 

percent of prescribers are involved 

in consumer education, but your 

estimate is higher. Would you help 

us understand the difference? 

Often the MedTEAM leader can provide 

information that will resolve the discrepancy.

			Before you leave, check for missing data

Fidelity scales should be completed in full, 

with no missing data on any items. Check 

in with the MedTEAM leader at the end 

of the visit to collect any additional 

information you may need.
Evaluating Your Program 11 Conduct a Process Assessment

 After your assessment visit

			Follow up

It is important to collect any missing 

data before completing your rating. 

If necessary, follow up on any missing 

data (for example, by calling or sending 

an e-mail). This would include discussing 

with the MedTEAM leader any 

discrepancies between data sources that 

you notice after you’ve completed the visit. 

			Score your scales

Use the Interview and Chart Review 

Guides in Appendices C and E to score 

MedTEAM. If you assess an agency for the 

first time to determine which components 

of MedTEAM the agency already has in 

place, some items may not apply. If an item 

cannot be rated, code the item as “1” on the 

MedTEAM Organizational Fidelity Scale. 

For the MedTEAM Prescriber Fidelity 

Scale, complete one scoresheet for every 

chart that you review. The scoresheet 

contains qualifying questions to help you 

assess whether the item may be scored 

based on the information in the chart.

For example, if you are reviewing a chart 

for a consumer who is newly diagnosed, 

item P11: Medication Visit Frequency, 

which captures whether consumers are 

contacted after medication changes, will 

not apply. If an item does not apply, 

withhold it from scoring. 

Total number of points scored

Total number of applicable items

Find an average for each chart reviewed. 

After you complete your chart review, 

average all scores. 

Sum of scores

Number of charts reviewed

			Complete scales independently

If you have two assessors, both of 

you should independently review 

the data collected and rate the scales. 

Then compare your ratings, resolve 

any disagreements, and devise a 

consensus rating.

			Complete the scoresheets 

Tally the item scores and determine which 

level of implementation was achieved.

Evaluating Your Program 13 Monitor Outcomes

Evaluating Your Program

Monitor Outcomes

Unlike process measures, which must 

be used in full to comprehensively 

understand how services are provided, 

you must decide which outcome measures 

will be most informative for your 

MedTEAM program. Initially, your 

outcomes monitoring system should be 

simple to use and maintain. Complexity 

has doomed many well-intended attempts 

to collect and use outcomes data. 

One way to simplify is to limit the 

number of outcome measures used. 

Select your outcome measures based 

on the type of information that will be 

most useful to your agency. 

Based on the research literature, we 

suggest that you monitor a core set 

of outcomes such as the following:

	Improved symptoms;

	Improved quality of care;

	Increased retention of employment; 

and

	Improved satisfaction and adherence 

to treatment. (Katon et al., 1995; 

Simon, Von Korff, Rutter, & 

Wagner, 2000; Wells et al., 2000; 

Worrall, Angel, Chaulk, Clarke, & 

Robbins, 1999).

Symptoms targeted by medications are 

the primary outcome measures used 

to track direct outcomes of medication 

treatment. Thus, psychotic symptoms are 

a measure of antipsychotic medications, 

anxiety symptoms are a measure of 

antianxiety medications, and so on.

It is important to measure symptoms 

for both the purposes of evaluating your 

MedTEAM program and informing the 

clinical assessment.
Monitor Outcomes 14 Evaluating Your Program

While quantifying symptoms takes more time 

than recording clinical impressions (for example, 

“Consumer seems less psychotic”), research has 

shown that prescribers do a better job of treating 

symptoms when they have the capacity to track and 

efficiently and effectively record symptom levels. 

For more information on measuring psychiatric 

symptoms, see “Integrating Outcomes into Clinical 

Assessments” in Training Frontline Staff in 

this KIT. 

In addition to targeted symptoms, it is also 

important to look at global life areas to assess the 

effectiveness of treatment as a whole. You can use 

measures of quality of care or outcomes that relate 

to consumers’ recovery goals, such as employment. 

For data to be useful, they must be valid. That is, 

the data must accurately represent what they were 

intended to measure. Thus, the outcomes must be 

few and concrete for MedTEAM staff to focus on 

key outcomes, to understand them in a similar way, 

and to make their ratings in a consistent and 

error-free fashion. 

To enhance validity, we recommend using simple 

ratings initially (such as, Did the consumer hold a 

competitive job in this quarter?), rather than more 

detailed ones (such as, How many hours during 

this quarter did the consumer work competitively?). 

For a sample Outcomes Report Form that is an 

example of a simple, paper-based way to collect 

participation and overall treatment outcomes data 

regularly, see Appendix F. For instructions for 

using the Outcomes Report Form, see Appendix G. 

What Is the Consumer Outcomes Monitoring Package? 

Sponsored in part by the Substance Abuse and Mental Health Services Administration (SAMHSA), the 

Consumer Outcomes Monitoring Package (COMP) was designed by a team at the School of Social Welfare, 

University of Kansas. This computer application allows agencies to choose from a pre-established list 

of outcomes developed for various evidence-based practices. Although not developed specifically for 

MedTEAM, the outcomes included in COMP represent global life areas that are relevant for assessing 

overall effects of treatment as a whole. Data may be entered for the chosen outcomes, and reports can 

be generated quarterly or monthly. The COMP also allows agencies to view their outcomes data using 

a variety of tables and graphs. 

The designers of COMP tried to make the computer application as easy and as flexible to use as possible. 

You may access COMP through the Web. Agencies can download the computer application and print out 

Installation Instructions and a User Manual, which provides definitions and forms. 

To download COMP—

	Go to http://research.socwel.ku.edu/ebp

	Click the link to the download page;

	Click the links to download the Installation Instructions and the User Manual; and 

	Follow the instructions to install the application.
Evaluating Your Program 15 Monitor Outcomes

 Develop procedures 

Agencies may choose to develop the outcomes 

portion of their quality assurance system from 

scratch or use existing outcomes monitoring 

systems. A number of electronic evaluation 

programs are available to help you develop 

comprehensive, integrated, user-friendly 

outcomes monitoring systems. Examples 

include the following:

	Publicly available tools such as the Consumer 

Outcomes Monitoring Package (see the 

previous page), and Decision Support 

2000+ Online (http://www.ds2kplus.org); or 

	Various commercially available products.

When deciding whether to use an existing 

outcomes monitoring package or to design your 

own, it is important to keep your organization’s 

capabilities in mind. The system must not create 

undue burden for MedTEAM staff, and it must 

provide information to them that is useful in 

their jobs. 

The system should fit into the workflow of the 

organization, whether that means making ratings 

on paper, using the COMP computer application, 

or developing your own outcomes monitoring 

package. Start with whatever means are available 

and expand the system from there. In the 

beginning, you may collect data with a simple 

report form and you can report hand-tallied 

summaries to MedTEAM staff.

Computer software that allows for data entry and 

manipulation (such as Microsoft Access, Excel, or 

Lotus) makes tabulating and graphing data easier 

than if it is done by hand. A computerized system 

for data entry and report generation presents a 

clear advantage and may be the goal, but do not 

wait for it. Feedback does not have to come from 

a sophisticated computer system to be useful. It is 

more important that it is meaningful and frequent. 
Monitor Outcomes 16 Evaluating Your Program

Expanding Your Outcome Measures 

Once you have established your core outcomes 

monitoring system, learned how to routinely 

collect data, and are accustomed to using it to 

improve MedTEAM, you will be ready to expand 

your outcomes measures. Consider asking 

consumers and families for input on how to 

improve medication management. Consumers 

and families are important informants for 

agencies that are seeking to improve outcomes. 

Agencies may want to know the following:

	Whether consumers and families are satisfied 

with their services; 

	How services have affected their quality of life; 

and 

	Whether consumers believe the services are 

helping them achieve their recovery goals. 

While collecting data from consumers and 

families requires more staff time than collecting 

the information that may be reported quickly 

by MedTEAM staff, consumers and families give 

valuable feedback. 

We recommend the following surveys for 

collecting information from consumers 

and families:

	The Mental Health Statistics Improvement 

Program (MHSIP) Consumer Satisfaction Survey 

at http://www.mhsip.org; and 

	Recovery measurement instruments such as 

those described in Measuring the Promise: A 

Compendium of Recovery Measures, Volume II, 

available from http://www.tecathsri.org

It is difficult to obtain a representative sample 

of consumer and family respondents since mailed 

surveys are often not returned and interviews may 

only be done with people who are cooperative 

and easy to reach. Samples that are not 

representative may be biased. 

Avoid bias in your consumer and family data by 

using a variety of mechanisms to conduct your 

assessments. For example, consider combining 

feedback collected through surveys with that 

obtained from focus groups. Another option is to 

hire a consultant to conduct qualitative interviews 

with a small group of consumers or families.
Evaluating Your Program 17 Monitor Outcomes

How often should you collect outcomes data? 

Plan to monitor outcomes every 3 months 

and share the data with your MedTEAM staff. 

Collecting data at regular and short intervals will 

enhance the reliability of your outcomes data. 

While we recommend that you design a system 

for collecting outcomes early in the implementation 

process, agencies should not expect to see the 

desired results until the MedTEAM approach 

is fully operational. Depending on the resources 

available to your agency, this may take anywhere 

from 6 to 18 months to accomplish.

How should you identify data collectors? 

Agency administrators or mental health authorities 

may assign the responsibility for collecting 

outcomes data to the following:

	The MedTEAM leader;

	Members of the MedTEAM advisory 

committee;

	The quality assurance team;

	Independent consultants, including consumers 

and family members; and 

	Other staff. 

Unlike collecting process measures, collecting 

outcome measures does not require a daylong 

assessment process. Many standard outcome 

measures, such as symptoms and employment, can 

be collected by MedTEAM staff from their daily 

work with consumers. This is especially true if your 

MedTEAM staff have been trained to integrate 

outcome measures into their clinical assessments. 

It is important to develop a quick, easy, standardized 

approach to collect outcomes data. For example, 

create a simple form or computer database that 

MedTEAM staff can routinely update.

Evaluating Your Program 19 Use Data to Improve MedTEAM

Evaluating Your Program

Use Data To Improve MedTEAM

As you develop a quality assurance system, 

MedTEAM leaders and staff will weave 

it into the fabric of their daily routines. 

Process assessments will give you a 

window into the demanding work done 

every day. Outcome reports will give you 

tangible evidence of the use and value of 

services, and they will become a basis for 

decisionmaking and supervision. 

At some point, your MedTEAM staff may 

wonder how they did their jobs without an 

information system as they come to view 

it as an essential ingredient of wellimplemented evidence-based practices. 
 Use Data to Improve MedTEAM 20 Evaluating Your Program

			Create reports from your assessments

For your process data, in addition to 

completing the MedTEAM Fidelity Scales 

and scoresheets, assessors should write a 

report explaining their scores. The report 

should include the following:

	An interpretation of the results 

of the assessment; 

	Strengths and weaknesses of the 

agency’s approach to medication 

management; and 

	Clear recommendations to help the 

MedTEAM initiative progress. 

The report should be informative, factual, 

and constructive. Since some process 

measures assess adherence to the evidencebased model at both the agency and 

MedTEAM staff levels, remember to target 

recommendations to administrators, 

MedTEAM leaders, and staff.

When summarizing outcomes data, start 

with simple, easy-to-read reports. Then 

let experience determine what additional 

reports you need. You can design your 

reports to give information about individual 

consumers, a single prescriber’s caseload, 

or the program as a whole. For example, 

reports generated for individual consumers 

may track the consumer’s progress using 

specific outcomes over time. You could 

enter these reports in consumers’ charts 

and they could be the basis for discussions 

about consumers’ progress.

			Use tables and graphs to understand your 

outcomes data

After the first process and outcomes 

assessments, it is often useful to provide 

a visual representation of an agency’s 

progress over time. We recommend 

that you use tables and graphs to report 

the results. 

By graphing your fidelity score, you have 

a visual representation of how medication 

management has changed over time. 

For an example, see Figure 1. For your 

process data, you may simply graph the 

results using a spreadsheet and include 

this in your report. 

When your agency shows greater fidelity 

over time, the graph will display it and 

reinforce your efforts. Additionally, as you 

can see in Figure 1, the graph allows you 

to quickly compare one team to another. 

In this example, Team A struggled in the 

first 6 months. Understanding Team A’s 

progress compared to Team B’s allowed 

the teams to partner and share strategies. 

Consequently, Team A improved 

dramatically over the next 6-month period.

Another feature of graphing assessment 

scores is to examine the cutoff score for 

fair (56) or good (66) implementation. 

Your program can use these scores 

as targets.
Evaluating Your Program 21 Use Data to Improve MedTEAM

			Share your results

The single factor that will most likely 

determine the success of an information 

system is its ability to give useful and 

timely feedback to key stakeholders. 

It is fine to worry about what to enter into 

an information system, but ultimately its 

worth is in converting data into meaningful 

information. For example, the data may 

show that 20 consumers worked in a 

competitive job during the past quarter, 

but it is more informative to know that 

this represents only 10 percent of the 

consumers in the agency. 

For information to influence practice, it 

must be understandable and meaningful, 

and it must be delivered in a timely way. 

In addition, the quality assurance system 

must tailor the information to suit the 

needs of various users and to answer 

their questions.

Sharing results with MedTEAM staff 

After each assessment, dedicate time 

during a supervisory meeting to discuss 

the results. Numbers that reflect above 

average or exceptional performance should 

trigger recognition, compliments, or other 

rewards. Data that reflect below average 

performance should provoke a search for 

underlying reasons and should generate 

strategies that offer the promise of 

improvement. By doing this regularly, 

MedTEAM leaders will create a learning 

organization characterized by adaptive 

responses to information that aim to 

improve consumer outcomes.

30

35

40

45

50

55

60

65

Feb-07

Apr-07

Jun-07

Aug-07

Oct-07

Dec-07

Feb-08

Apr-08

Jun-08

Aug-08

Oct-08

Note: 66-75 = good implementation 

 56-65 = fair implementation 

 55 and below = not evidence-based practice
 Use Data to Improve MedTEAM 22 Evaluating Your Program

Sharing results with your MedTEAM 

Advisory Committee or quality 

assurance team 

You may also use this information to keep 

external stakeholders engaged. Sharing 

information with vested members of 

the community, staff from your mental 

health authority, and consumers and family 

advocates can be valuable. Through these 

channels, you may develop support 

for MedTEAM, increase consumer 

participation, and raise private funds 

for your agency.

Sharing results internally

Agencies may distribute reports during 

all staff and manager-level meetings to 

keep staff across the agency informed and 

engaged in the process of implementing 

MedTEAM. Agencies with successful 

evidence-based practice programs 

highlight the importance of developing 

an understanding and support for the 

evidence-based model across the agency. 

Additionally, integrating consumer-specific 

reports into clinical charts may help you 

monitor consumers’ progress over time. 

Reporting consumer-specific outcomes 

information at the treatment team 

meetings also helps keep the team 

focused on consumers’ recovery goals.

Sharing results with consumers 

and families

Agencies may highlight assessment 

results in consumer and family meetings. 

Increasing consumers’ and families’ 

understanding of MedTEAM may 

motivate them to participate in the 

treatment process and build trust in 

the consumer-practitioner relationship. 

Also, sharing results may create hope and 

enthusiasm for your MedTEAM initiative. 

Sharing information motivates people and 

stimulates changes in behavior. Sharing the 

results of your assessments with a variety 

of stakeholders is the key to improving 

medication management in your agency.
Appendix A 23 MedTEAM Fidelity Scales

Evaluating Your Program

Appendix A: Cover Sheet— 

MedTEAM Fidelity Scales

Appendix A 25 MedTEAM Fidelity Scales

Cover Sheet: MedTEAM Fidelity Scales 

Assessors’ names: ___________________________________________ Today’s date: _____/___/_____

___________________________________________

___________________________________________

Program name (or program code): __________________________________

Agency name: ___________________________________________

Agency address: ______________________________________________________________________________

Street

______________________________________________________________________________

City State ZIP code

MedTEAM leader or contact person: ___________________________________________________________________

Names of the MedTEAM practitioners: __________________________________________________________________

Telephone: ( ___) _____–_______ E-mail: _________________________________________________

Sources used for assessments: q Chart review Number reviewed: _______

q Progress Notes reviewed Number reviewed: _______

q MedTEAM curriculum review

q Brochure review

q MedTEAM session/group observation

q MedTEAM director/coordinator interview

q MedTEAM leader interview

q MedTEAM practitioner interviews Number interviewed: ____

q Consumer interviews Number interviewed: ____

q Family member interviews Number interviewed: ____

q Other staff interviews Number interviewed: ____

q Other ______________

Number of MedTEAM practitioners: _______

Number of current MedTEAM consumers: _______

Number of consumers served last year: _______

Funding source: ______________________________________________________________________________

Agency location: q Urban

q Rural

Date program was started: _____/_____/______

Appendix B 27 Fidelity Scale and Scoresheet

Evaluating Your Program

Appendix B: MedTEAM Organizational 

Fidelity Scale and Scoresheet

Appendix B 29 Fidelity Scale and Scoresheet

MedTEAM Organizational Fidelity Scale 

Criteria

Ratings / Anchors

1 2 3 4 5

Staffing

O1. Standardized Admissions Form

Agency has a standardized 

admissions form covering the 

following 11 areas:

 Diagnoses

 Symptoms and severity

 Illness history (including age 

of onset, hospitalizations, and 

suicide attempts)

 Past medication history 

(dose, duration, interactions, 

tolerability, and response)

 Current medications (dose, 

duration, interactions, 

tolerability, and response)

 Assessment of the effectiveness 

of current medications 

and any plans for future 

medication changes

 Current medication adherence

 Current side effects and 

treatments for them

 Current consumer functioning

 Consumer preferences 

and goals

 Contact information for 

previous providers

Agency has no 

standardized 

admissions form 

Agency has a 

standardized 

admissions form 

covering fewer 

than 6 areas

Agency has a 

standardized 

admissions form 

covering 6–7 areas

Agency has a 

standardized 

admissions 

form covering 

8–10 areas

Agency has a 

standardized 

admissions form 

covering all 

11 areas

O2. Standardized Annual 

Update Form

Agency has a standardized annual 

update form covering the same 

11 areas as the standardized 

admissions form. (See O1.)

Agency has no 

standardized 

annual update 

form

Agency has a 

standardized 

annual update 

form covering 

fewer than 6 areas

Agency has a 

standardized 

annual update 

form covering 

6–7 areas

Agency has a 

standardized 

annual update 

form covering 

8-10 areas

Agency has a 

standardized 

annual update 

form covering all 

11 areas

O3. Standardized Ongoing 

Treatment Form:

Agency has a standardized 

ongoing treatment form covering 

the following 7 areas:

 Diagnoses

 Symptoms and severity

 Current medications (dose, 

duration, interactions, 

tolerability, and response)

 Rationale for any 

medication changes

 Current medication adherence

 Current side effects and 

treatments for them

 Current consumer functioning

Agency has no 

standardized 

ongoing treatment 

form or form is 

open-ended

Agency has a 

standardized 

ongoing treatment 

form covering 3 or 

fewer areas

Agency has a 

standardized 

ongoing treatment 

form covering 

4–5 areas

Agency has a 

standardized 

ongoing treatment 

form covering 

6 areas

Agency has a 

standardized 

ongoing treatment 

form covering all 

7 areas
Fidelity Scale and Scoresheet 30 Appendix B

MedTEAM Organizational Fidelity Scale 

Criteria

Ratings / Anchors

1 2 3 4 5

O4. Prescriber Access to Information 

at First Medication Visit After 

Admission (New Consumers 

Only):

Updated charts with all pertinent 

information are available 

at the time of consumers’ 

first appointment with the 

prescriber. (See O1 for list of 

pertinent information.)

Less than 60% 

of charts with 

adequate 

information 

available at first 

medication visit 

60–69% of charts 

with adequate 

information 

available at first 

medication visit

70–79% of charts 

with adequate 

information 

available at first 

medication visit

80–89% of charts 

with adequate 

information 

available at first 

medication visit

More than 

90% of charts 

with adequate 

information 

available at first 

medication visit 

O5. Prescriber Access to Information 

After Hospital or Emergency 

Room Visit: 

Updated charts with all pertinent 

information are available at the 

time of consumers’ appointment 

after discharge from hospital 

or emergency room for a 

psychiatric emergency.

Less than 60% 

of charts with 

adequate 

information 

available at first 

medication visit

60–69% of charts 

with adequate 

information 

available at first 

medication visit

70–79% of charts 

with adequate 

information 

available at first 

medication visit

80–89% of charts 

with adequate 

information 

available at first 

medication visit

More than 

90% of charts 

with adequate 

information 

available at first 

medication visit

O6. Prescriber Access to Information 

at Each Routine Visit:

Updated charts with all pertinent 

information are available at the 

time of consumers’ appointment.

Less than 60% 

of charts with 

adequate 

information 

available at time 

of appointment

60–69% of charts 

with adequate 

information 

available at time 

of appointment

70–79% of charts 

with adequate 

information 

available at time 

of appointment

80–89% of charts 

with adequate 

information 

available at time 

of appointment

More than 

90% of charts 

with adequate 

information 

available at time 

of appointment

O7. Treatment Refractory:

Outcomes are routinely monitored 

to identify consumers with 

treatment-refractory conditions 

(multiple medication failures) and 

to ensure that they are offered 

appropriate treatments. The 

system includes the following:

 Specific operational criteria for 

treatment refractoriness

 Regular review (at least every 

6 months)

 Process for 

informing prescribers

Agency has 

no criteria or 

process to identify 

consumers whose 

symptoms have 

inadequately 

responded 

to medication 

Agency has a 

system to identify 

consumers whose 

symptoms have 

inadequately 

responded to 

medication, but 

it falls short on 

all 3 standards 

Agency has a 

system to identify 

consumers whose 

symptoms have 

inadequately 

responded to 

medication, but 

it fails to satisfy 2 

of 3 standards 

Agency has a 

system to identify 

consumers whose 

symptoms have 

inadequately 

responded to 

medication, but it 

fails to fully satisfy 

1 of 3 standards 

(for example, the 

review process 

is annual)

Agency fully 

satisfies the 3 

standards to 

identify consumers 

whose symptoms 

have inadequately 

responded 

to medication 

O8. Consumer Education:

Agency has educational materials 

for consumers and families that 

cover the following topics: 

 The purpose of 

medication treatment

 Benefits and risks

 Potential side effects

 Alternative treatments

The agency also has procedures 

to distribute educational 

materials and to document 

prescriber involvement in 

consumer education. 

Agency has no 

educational 

materials to 

distribute

Educational 

materials are 

available, but 

serious deficiencies 

in content, 

distribution, and/

or prescriber 

involvement exist

Educational 

materials have 2 

of the following 

deficiencies: 

1. 1 or more 

critical elements 

missing 

2. Distribution is 

not systematic

3. No agency 

forms exist 

to document 

prescriber 

involvement

Educational 

materials have 1 

of the following 

deficiencies: 

1. 1 or more 

critical elements 

missing 

2. Distribution is 

not systematic

3. No agency 

forms exist 

to document 

prescriber 

involvement 

Agency has 

educational 

materials and 

procedures to 

distribute them 

and document 

prescriber 

involvement 

in consumer 

education 
Appendix B 31 Fidelity Scale and Scoresheet

MedTEAM Organizational Fidelity Scale 

Criteria

Ratings / Anchors

1 2 3 4 5

O9. Agency Medication Guidelines:

Agency has written guidelines that 

include the following:

 Definition of what constitutes 

an adequate trial for each 

medication (that is, dose and 

duration)

 Recommended medication 

sequences for inadequate 

responses

 Plan to annually update the 

guidelines 

Agency has no 

written guidelines

Agency has written 

guidelines that 

have not been 

reviewed in the 

last 3 years, or the 

guidelines have 

major deficiencies 

Agency has written 

guidelines that 

have not been 

reviewed in 2–3 

years, or the 

guidelines have 

major deficiencies

Agency has written 

guidelines that 

have not been 

reviewed in 1–2 

years, or the 

guidelines have 

minor deficiencies

Agency has 

written medication 

guidelines that 

meet all 3 criteria

010. Scheduling Flexibility for 

Unscheduled Urgent Visits:

Agency has explicit scheduling 

policies allowing consumers to be 

seen within 7 days for urgent care.

No explicit policy 

exists or most 

consumers are 

required to wait 

more than 14 days 

to be seen

Agency has explicit 

scheduling policies 

that are not used 

or most consumers 

are seen within 

8–14 days

Agency has explicit 

scheduling policies 

and 50–79% of 

consumers are 

seen within 7 days

Agency has explicit 

scheduling policies 

and 80-89% of 

consumers are 

seen within 7 days

Agency has explicit 

scheduling policies 

allowing > 90% 

of consumers to be 

seen within 7 days 

for urgent care

011. Scheduling Flexibility After 

Hospital Visits:

Agency has and adheres to 

explicit scheduling policies 

allowing consumers to be seen 

within 3 days after discharge 

from a hospital for psychiatric 

treatment.

Agency has no 

explicit policy or 

consumers are 

seen after more 

than 28 days

Agency has explicit 

scheduling policies 

that are not used 

or consumers 

are seen within 

15–28 days

Agency has explicit 

scheduling policies 

that are not used 

or consumers 

are seen within 

8–14 days

Agency has explicit 

scheduling policies 

that are not used 

or consumers 

are seen within 

4–7 days

Agency has 

explicit scheduling 

policies allowing 

consumers to be 

seen within 3 days 

after discharge 

from a hospital 

for psychiatric 

treatment

012. Scheduling Flexibility After 

Emergency Room Visits:

Agency has and adheres to explicit 

scheduling policies allowing 

consumers to be seen within 3 

days after an emergency room 

visit for a psychiatric emergency.

Agency has no 

explicit policy or 

consumers are 

seen after more 

than 28 days

Agency has explicit 

scheduling policies 

that are not used 

or consumers 

are seen within 

15–28 days

Agency has explicit 

scheduling policies 

that are not used 

or consumers 

are seen within 

8–14 days

Agency has explicit 

scheduling policies 

that are not used 

or consumers 

are seen within 

4–7 days

Agency has 

explicit scheduling 

policies allowing 

consumers to 

be seen within 

3 days after 

an emergency 

room visit for 

a psychiatric 

emergency

013. Integration of Services:

MedTEAM staff (including 

prescribers and those who work 

directly on MedTEAM) are part of 

a mental health treatment team 

(for example, case management, 

nursing, residential services, 

or employment services). They 

attend weekly treatment team 

meetings (not replaced by 

administrative meetings) and 

have frequent contact with 

treatment team members. 

MedTEAM staff 

are separate from 

other treatment 

team members. 

No regular, direct 

contact or only 

phone or faceto-face contact 

per month

MedTEAM staff 

occasionally attend 

treatment team 

meetings, but less 

than monthly

MedTEAM staff 

have several 

contacts with 

treatment team 

members each 

month and attend 

monthly treatment 

team meetings

MedTEAM staff 

have several 

contacts with 

treatment team 

members each 

month and attend 

biweekly treatment 

team meetings

MedTEAM staff 

are part of a 

mental health 

treatment team, 

attend weekly 

treatment team 

meetings, and 

have frequent 

contact with 

treatment 

team members
Fidelity Scale and Scoresheet 32 Appendix B

MedTEAM Organizational Fidelity Scale 

Criteria

Ratings / Anchors

1 2 3 4 5

014. Clinical Supervision:

MedTEAM staff receive 

structured, monthly supervision 

(group or individual format) from 

an experienced MedTEAM leader. 

Supervision should be consumercentered and should explicitly 

address the MedTEAM model 

and its application to specific 

consumer situations.

20% of MedTEAM 

staff receive 

supervision

21–40% of 

MedTEAM staff 

receive monthly, 

structured, 

consumer-centered 

supervision, or all 

MedTEAM staff 

receive informal 

supervision 

41–60% of 

MedTEAM staff 

receive monthly, 

structured, 

consumer-centered 

supervision, or all 

MedTEAM staff 

receive quarterly 

supervision

61–80% of 

MedTEAM staff 

receive monthly, 

structured, 

consumer-centered 

supervision, or all 

MedTEAM staff 

receive bimonthly 

supervision

More than 80% 

of MedTEAM staff 

receive monthly, 

structured, 

consumer-centered 

supervision, 

focusing on 

specific consumers, 

in sessions that 

explicitly address 

the MedTEAM 

model and its 

application

015. Quality Assurance (QA):

Agency has QA committee 

that comprehensively reviews 

MedTEAM (using all items on 

MedTEAM fidelity scales) every 

6 months, and assessment 

information is used to improve 

the program. 

There is no review 

or committee

QA committee has 

been formed but 

no reviews have 

been completed

Comprehensive QA 

review occurs less 

than annually, or 

QA review is not 

comprehensive

Comprehensive 

QA review occurs 

annually, or QA 

review is not 

comprehensive

Agency has QA 

committee that 

comprehensively 

reviews MedTEAM 

(using all items 

on MedTEAM 

fidelity scales) 

every 6 months, 

and assessment 

information is 

used to improve 

the program
Appendix B 33 Fidelity Scale and Scoresheet

Organizational Scoresheet: MedTEAM Fidelity Scale 

MedTEAM leader: ___________________________________________ Today’s date: _____/___/_____

Assessors’ names: ___________________________________________

___________________________________________ Site number: _______________

1 2 3 4 5

O1 Standardized Admissions Form

O2 Standardized Annual Update Form q q q q q

O3 Standardized Ongoing Treatment Form q q q q q

O4 Prescriber Access to Information at First Medication Visit After Admission 

(New Consumers Only) q q q q q

O5 Prescriber Access to Information After Hospital or Emergency Room Visit q q q q q

O6 Prescriber Access to Information at Each Routine Visit q q q q q

O7 Treatment Refractory q q q q q

O8 Consumer Education q q q q q

O9 Agency Medication Guidelines q q q q q

O10 Scheduling Flexibility for Unscheduled Urgent Visits q q q q q

O11 Scheduling Flexibility After Hospital Visits q q q q q

O12 Scheduling Flexibility After Emergency Room Visits q q q q q

O13 Integration of Services q q q q q

O14 Clinical Supervision q q q q q

015 Quality Assurance q q q q q

Total Score

Items not rated due to insufficient data

Appendix C 35 Interview Guides: MedTEAM Organizational Fidelity Scale

Evaluating Your Program

Appendix C: Interview Guides: 

MedTEAM Organizational 

Fidelity Scale

Appendix C 37 Interview Guides: MedTEAM Organizational Fidelity Scale

MedTEAM Leader Interview Guide: 

MedTEAM Organizational Fidelity Scale

Ask the MedTEAM leader (and the agency administrator when necessary) these questions. 

Collect information that is as detailed as possible. If possible, keep copies of any forms you reviewed.

General Questions

Are your forms electronically accessible or able 

to be indexed?

q All

 Some

q None

Of your electronic forms, how many prescribers 

use them? 

 All

 Some

 None

O1. Standardized Admissions Form

Do you have a standardized admissions form 

or admission packet? 

 Yes. What is the name of this document? 

May I see a copy?

 No

To the Interviewer: If you receive the form, examine it 

for the elements listed below. If any elements are not 

included, ask the MedTEAM leader if they appear in 

the chart and if they are recorded systematically. 

On the 

form

Not on the 

form or not 

explicit

Diagnoses  

Symptoms and severity  

Illness history (including 

hospitalizations and 

suicide attempts)

 

Past medication history 

(dose, duration, interactions, 

tolerability, and response)

 

Current medications (dose, 

duration, interactions, 

tolerability, and response)

 

Assessment of the effectiveness 

of current medications 

and any plans for future 

medication changes

 

Current medication adherence  

Current side effects and 

treatments for them  

Current consumer functioning  

Consumer preferences and goals  

Contact information for 

previous providers  

Total number present

 

Where do I find this form in the chart? (If it is an 

electronic form, where do you find it?)
Interview Guides: MedTEAM Organizational Fidelity Scale 38 Appendix C

O2. Standardized Annual Update Form

Do you have a standardized update form? 

 Yes

 No

Is this completed annually? 

 Yes

 No

To the Interviewer: Examine the form for the elements 

listed below. Updates can be completed using the 

Standardized Admissions Form. Updates can also 

refer to the Initial Admission Form for history that 

is unchanged. 

On the 

form

Not on the 

form or not 

explicit

Diagnoses  

Symptoms and severity  

Illness history (including 

hospitalizations and 

suicide attempts)

 

Past medication history 

(dose, duration, interactions, 

tolerability, and response)

 

Current medications (dose, 

duration, interactions, 

tolerability, and response)

 

Assessment of the effectiveness 

of current medications 

and any plans for future 

medication changes

 

Current medication adherence  

Current side effects and 

treatments for them  

Current consumer functioning  

Consumer preferences and goals  

Contact information for 

previous providers  

Total number present

O3. Standardized Ongoing Treatment Form

Do you have a standardized ongoing treatment form 

that prescribers complete at every medication visit? 

 Yes. What is the name of this form? 

May I see a copy?

 No

Where do I find this form in the chart? (If it is an 

electronic form, where do you find it?)

To the Interviewer: Examine the form for the elements 

below. The form must be structured, not a free-flowing 

narrative or open-ended questions. If any elements are 

missing, ask the MedTEAM leader if they are recorded 

systematically and located elsewhere in the chart.

On the 

form

Not on the 

form or not 

explicit

Diagnoses  

Symptoms and severity  

Current medications (dose, 

duration, interactions, 

tolerability, and response)

 

Rationale for any medication 

changes  

Current medication adherence  

Current side effects and 

treatments for them  

Current consumer functioning  

Total number present

 
Appendix C 39 Interview Guides: MedTEAM Organizational Fidelity Scale

O4. Prescriber Access to Information 

at First Medication Visit After 

Admission (New Consumers Only) 

Who admits consumers? 

Do you have an intake specialist? 

 Yes

 No

How is the information transmitted or 

incorporated into the chart?

Do you have procedures for obtaining illness and 

medication history on admission? 

 Yes. What are they? 

What information do you obtain? 

Do you have policies and procedures for how 

rapidly admission data are put into charts?

 Yes

 No

Do you monitor them?

 Yes

 No

Are they adequate? 

 Yes

 No

 No

Is the chart information always available (from 

referring institution or previous treatment setting, 

if it applies)? 

 Yes

 No

Is it complete? 

 Yes

 No. About how often is it incomplete?

What procedures do you have to 

ensure completeness? 

Can you think of situations in which this 

information may be delayed (for example, 

the hospital does not forward information)?
Interview Guides: MedTEAM Organizational Fidelity Scale 40 Appendix C

O5. Prescriber Access to Information After 

Hospital or Emergency Room Visits

What kind of information do prescribers have 

about consumers after hospital or emergency room 

visits (for example, lab tests, dictations, item 

symptom list, etc.)? 

Is the chart information always available from 

the hospital? 

 Yes

 No

Is it complete? 

 Yes

 No. What percentage of the time is the chart

incomplete or unavailable?

 percent

What procedures do you have for ensuring that 

complete chart information is available?

O6. Prescriber Access to Relevant 

Information at Each Routine Visit 

During each routine visit, what kind of information 

do prescribers have? 

Is the chart information always available? 

 Yes

 No

Is it complete? 

 Yes

 No. What percentage of the time is the chart

incomplete or unavailable?

 percent

What procedures do you have for ensuring that 

complete chart information is available?
Appendix C 41 Interview Guides: MedTEAM Organizational Fidelity Scale

O7. Treatment Refractory 

Does your agency have a formal method to identify 

consumers who have treatment-refractory illnesses? 

 Yes. What is your definition of treatment refractory? 

[Probe for operational criteria.]

 No

Do you have a current list of consumers who have 

treatment-refractory illnesses? 

 Yes. May I see it?

 No

What does the agency do when a consumer is 

identified as having a treatment-refractory illness? 

How are outcomes monitored for consumers with 

treatment-refractory illnesses? How often? 

If a monitoring process exists, how are 

prescribers informed? 

To the Interviewer: Are there specific operational criteria 

for treatment refractory? 

 Yes

 No

Does the agency have a regular review process 

(at least every 6 months)? 

 Yes

 No

Does the agency have a method for 

informing prescribers?

 Yes

 No
Interview Guides: MedTEAM Organizational Fidelity Scale 42 Appendix C

O8. Consumer Education

Do you have written materials about 

each medication? 

 Yes. May I see samples?

 No. Do you provide consumer education?

Yes. What elements are included in your 

consumer education?

No

Do you have procedures to distribute 

education materials? 

 Yes. May I see samples?

 No

Who distributes educational materials? 

Is distribution systematic (that is, every time a new 

medication is prescribed)? 

 Yes

 No

How are prescribers involved in 

consumer education? 

Do agency forms document the prescriber 

involvement? [It doesn’t have to be the prescriber who 

educates as long as he or she acknowledges or reviews it.] 

 Yes

 No

To the Interviewer: Probe for the following elements: 

On the 

form

Not on the 

form or not 

explicit

Purpose of medication treatment  

Benefits and risks  

Potential side effects  

Alternative treatments  

Procedures to distribute 

educational materials  

Procedures to document 

prescriber involvement  

Total number present
Appendix C 43 Interview Guides: MedTEAM Organizational Fidelity Scale

O9. Agency Medication Guidelines 

Does your agency have medication guidelines 

specifying what constitutes an adequate trial for 

each medication? 

 Yes. May I see a copy? Examine the guidelines for the 

presence of recommendations about—

Dose

 Yes

 No

Duration

 Yes

 No

 No

Do your medication guidelines recommend 

medication sequences for consumers who are not 

adequately responding to medication (that is, 

treatment refractory)? 

 Yes

 No

How often are the medication guidelines reviewed 

or updated? 

When was the last update?

_______/__________/______

[Use this date to rate frequency of updates. For example, 

if the date is less than a year old, then rate it as annual.]
Interview Guides: MedTEAM Organizational Fidelity Scale 44 Appendix C

O10. Scheduling Flexibility for 

Unscheduled Urgent Visits 

To the Interviewer: The intent is to determine the balance 

between the demands for productivity and the desirability 

of scheduling flexibility. Some possible probe questions:

Describe your scheduling procedures. 

Do you seek to fill completely your prescribers’ 

appointment times? 

 Yes

 No

How are unfilled appointment time slots viewed at 

your agency? 

How does your agency handle unscheduled or 

urgent consumer visits (for example, if consumers 

experience troublesome side effects that are not 

life-threatening but would require an evaluation 

and possible medication change)? [Ask for recent 

examples and specifics of the manner in which they 

were handled.]

Do you “hold back” a certain number of time slots 

in prescribers’ schedules for such purposes? 

 Yes. Number of time slots held back: _____________

If held back by prescriber, how many per 

time period:

____ per day/week

If held back for agency as a whole, how many 

per time period: 

____ per day/week

 No

How far in the future is a typical appointment date 

for consumers with an urgent (but not emergency) 

request to see a doctor? 

______ business days 

Do you have an explicit policy about unscheduled 

or urgent visits? 

 Yes. Please describe it.

 No

Estimated number of days from urgent request 

to appointment: 

_______ days

To the Interviewer: Estimate the percent of consumers who 

are seen within 7 days.
Appendix C 45 Interview Guides: MedTEAM Organizational Fidelity Scale

O11. Scheduling Flexibility After 

Hospital Visits

When consumers are discharged from the hospital, 

how long is the wait time before they have a 

medication visit?

Do you have an explicit policy about visits after 

discharge from the hospital? 

 Yes. Please describe.

 No

O12. Scheduling Flexibility After 

Emergency Room Visits 

When consumers are seen in the emergency room 

for a psychiatric emergency, how long is the wait 

time before they have a medication visit?

Do you have an explicit policy about visits after 

emergency room visits? 

 Yes. Please describe.

 No
Interview Guides: MedTEAM Organizational Fidelity Scale 46 Appendix C

O13. Integration of Services 

How many treatment teams does your agency 

have? (Typically, a team comprises a team leader, 

several case managers, a psychiatrist, and nurses.)

_______ Number of teams

How often does each meet? Do not include 

administrative meetings. Count only meetings that 

review individual consumers regarding treatment 

planning, outcomes, etc.

Team 1 _______

Team 2 _______

Team 3 _______ 

Team 4 _______

Team 5 _______

Are prescribers and other MedTEAM staff 

expected to attend these meetings? 

 Yes. How often? ________ 

Is it the same for each prescriber?

 Yes 

 No

 No

Does a process exist to monitor attendance at 

these meetings? 

 Yes. Please describe.

 No

Do other ways exist in your agency in which 

MedTEAM staff have regular contact with other 

members of the treatment team? 

 Yes. Please describe.

 No

To the Interviewer: The intent of this question is to 

determine if an equivalent method to ensure regular, 

consumer-specific treatment planning and coordination 

exists involving all relevant staff.

__________ Average number of treatment team 

meetings (or equivalent) attended per 

week by MedTEAM staff:

__________ Average duration of meetings
Appendix C 47 Interview Guides: MedTEAM Organizational Fidelity Scale

O14. Clinical Supervision

Do prescribers and other MedTEAM staff receive 

clinical supervision? 

 Yes. How often? 

What types of issues are discussed?

 No

Who provides clinical supervision? 

Is this person experienced in the evidence-based 

approach to medication management? 

 Yes

 No

O15. Quality Assurance 

To the Interviewer: If appropriate, interview the director of 

quality assurance or quality assurance team members in 

addition to the MedTEAM leader, asking similar questions, 

or interview the two together.

Do you have a department or committee 

responsible for quality assurance of 

medication management? 

 Yes. Name of the department or committee:

_______________________________________

 No

Please describe its composition, function, 

and procedures.

How often do you conduct reviews? 

What specific information is examined 

in the review? 

What corrective actions have been taken?
Interview Guides: MedTEAM Organizational Fidelity Scale 48 Appendix C

















Prescriber/MedTEAM Staff Interview Guide: 

MedTEAM Organizational Fidelity Scale

Use these questions for your interviews with prescribers and MedTEAM staff. Collect information that is 

as detailed as possible. If possible, keep copies of any forms you reviewed. Note: Some questions duplicate 

those asked of the MedTEAM leader; DO NOT share the MedTEAM leader’s responses with anyone else 

on the team. Also, tell the prescriber or MedTEAM staff member that any disclosure will be confidential 

and in no way will affect his or her relationship with the MedTEAM leader, the staff, or the agency. 

O1. Standardized Admissions Form

Do you use one standard form or admission packet 

to summarize illness and medication history? 

Yes. May I see an example from a consumer chart?

Is this information typically fully filled out?

Yes

No

No

O2. Standardized Annual Update Form

Do you have a standardized update form? 

Yes. May I see an example from a consumer chart?

No

How often are updates completed? 

O3. Standardized Ongoing Treatment Form

Do you complete a standardized ongoing 

treatment form at every medication visit? 

Yes. May I see an example from a consumer chart?

No

O4. Prescriber Access to Information 

at First Medication Visit After 

Admission (New Consumers Only)

What kind of information do you have 

about consumers at the first medication visit 

after admission? 

Can you think of situations in which this 

information may be delayed (for example, 

the hospital does not forward information)?

About what percentage of the time would you 

estimate that you have all the information you need 

at the time of first medication visit?

_______ percent

To the Interviewer: Get an estimate of the percentage of all 

charts with adequate information available at admission. 

Make fidelity rating based on the prescribers’ estimate, 

but take into account the other information as it informs 

the credibility of the estimate.
Appendix C 49 Interview Guides: MedTEAM Organizational Fidelity Scale

O5. Prescriber Access to Information After 

Hospital or Emergency Room Visits

What kind of information do you have about 

consumers on visits after hospital discharge? 

Is the chart information always available from 

the hospital? 

 Yes

 No. What percentage of time would you say that 

it is unavailable?

Is it complete? [Use list from Fidelity Item 01 to review 

the types of information considered to be pertinent.]

 Yes

 No. What percentage of time would you say that 

it is not complete?

What procedures do you have for 

ensuring completeness?

To the Interviewer: Get an estimate of the percentage of all 

charts with adequate information available at appointments 

after discharge from the hospital. Estimate a percentage of 

all charts with adequate information available to prescribers 

at appointments after discharge from the hospital. 

O6. Prescriber Access to Information 

at Each Routine Visit 

During routine medication visits, do you have 

access to the consumer’s chart? 

 Yes

 No

What percentage of time is the chart incomplete 

or unavailable? 

_______ percent

To the Interviewer: Estimate the percent of all charts with 

adequate information available to prescribers at each 

routine visit. 
Interview Guides: MedTEAM Organizational Fidelity Scale 50 Appendix C

O7. Treatment Refractory

Does your agency have a formal method to identify 

consumers who have treatment-refractory illnesses? 

 Yes. What is your definition of treatment refractory? 

[Probe for operational criteria.]

 No

Do you have a current list of consumers who have 

treatment-refractory illnesses? 

 Yes. May I see it?

 No

What do you do when a consumer is identified as 

having treatment-refractory illness? 

Are there specific treatments offered? 

 Yes

 No

How are outcomes monitored for consumers with 

treatment-refractory illnesses? 

How often?

Does a clinic-wide monitoring process exist to 

inform you? 

 Yes

 No

To the Interviewer: Are there specific operational criteria for 

treatment refractory? 

 Yes

 No

Does a regular review process exist (at least every 

6 months)? 

 Yes

 No

Does a method exist for informing prescribers? 

 Yes

 No

O8. Consumer Education

Do you have written materials about 

each medication? 

 Yes. May I see samples?

How is this information distributed? [Probe for 

whether the materials are routinely offered or if 

they are given only when requested.] 

 No. Do you provide consumer education? 

 Yes. Please describe.

 No

Do agency forms document prescriber 

involvement? [It doesn’t have to be the prescriber that 

provides education as long as he or she acknowledges 

or reviews it.] 

 Yes

 No
Appendix C 51 Interview Guides: MedTEAM Organizational Fidelity Scale

O9. Agency Medication Guidelines 

Does your agency have medication guidelines 

specifying what constitutes an adequate trial for 

each medication? 

 Yes. May I see a copy? Examine the guidelines for the 

presence of recommendations about—

Dose

 Yes

 No

Duration

 Yes

 No

 No

Do your medication guidelines recommend 

medication sequences for consumers who are not 

adequately responding to medication (that is, 

treatment refractory)? 

 Yes

 No

How often are the medication guidelines reviewed 

or updated? 

When was the last update?

_______/_______/_______

[Use this date to rate frequency of updates. For example, 

if the date is less than a year old, then rate it as annual.]

O10. Scheduling Flexibility for 

Unscheduled Urgent Visits

How does your agency handle unscheduled 

or urgent consumer visits (for example, if 

consumers experience troublesome side effects 

that are not life-threatening but would require an 

evaluation and possible medication adjustment or 

additional medication)? Are time slots reserved in 

your schedule? 

 Yes

 No

How far into the future is a typical appointment 

date for consumers with an urgent (but not 

emergency) request to see you?

________ business days 

Describe the policy for unscheduled 

or urgent visits.

________ Number of time slots reserved

________ Number of days from request to appointment

To the Interviewer: Get an estimate of the percentage 

of all consumers who are seen within 7 days.
Interview Guides: MedTEAM Organizational Fidelity Scale 52 Appendix C

O11. Scheduling Flexibility After 

Hospital Discharge

When consumers are discharged from the hospital, 

how long is the wait time before they have a 

medication visit?

Do you have an explicit policy about scheduling 

medication appointments after discharge from 

the hospital?

O12. Scheduling Flexibility After 

Emergency Room Visits 

When consumers are seen in the emergency room 

for a psychiatric emergency, how long is the wait 

time before they have a medication visit?

Do you have an explicit policy about scheduling 

medication appointments after emergency 

room visits?

O13. Integration of Services

Do you have contact with other members of the 

treatment team (for example, case managers, 

residential staff, and vocational staff)? 

 Yes. Please describe.

 No

What is the average duration and frequency 

of contact? 

________ Duration

________ Frequency of contact

Do you attend team meetings? 

 Yes

 No

________ Frequency of team meeting attendance
Appendix C 53 Interview Guides: MedTEAM Organizational Fidelity Scale

O14. Supervision

Describe ongoing supervision that prescribers 

and others who are involved in medication 

treatment receive about recommended medication 

practices. Supervision refers to ongoing review of 

professional development. [There may be a focus 

on specific consumers or guidelines; it does not include 

administrative meetings in which no clinical issues 

are discussed.] 

How often does the staff receive medicationspecific supervision? 

Is it in a group format? 

 Yes

 No

How long are these meetings? 

Who leads these meetings? 

Does a designated person run the meetings? 

 Yes

 No

What is the content of the meetings? 

What does a typical supervision session look like?

_________ Frequency of supervision

O15. Quality Assurance 

Does your agency have a department or 

committee responsible for quality assurance of 

medication management? 

 Yes. Name of the department or committee:

_______________________________________

 No

How often do they conduct reviews? 

Have you seen this information used to improve 

the way medication management is provided? 

 Yes. Please describe.

 No

Appendix D 55 MedTEAM Prescriber Fidelity Scale and Scoresheet

Evaluating Your Program

Appendix D: MedTEAM Prescriber 

Fidelity Scale and Scoresheet

Appendix D 57 MedTEAM Prescriber Fidelity Scale and Scoresheet

MedTEAM Prescriber Fidelity Scale and Scoresheet

Agency name: ___________________________________________ Today’s date: _____/___/_____

Assessors’ names: ___________________________________________

___________________________________________ Site number: _______________

Note:

Yes No

Does 

not 

apply

P1. Accessible and Accurate 

Summary of Illness and 

Medication History

A. Timeliness of Summary of Illness and Medication History   

B. Diagnosis   

C. Illness History

Age of onset   

Number of hospitalizations   

Most recent hospitalization   

Summary of course of illness   

D. Past Medications

History present  

Medication   

Highest daily dose   

Duration (years taken)   

Response   

Comments   

P2. Current Comprehensive 

Medication Documentation

A. Timeliness of Current Medication Documentation   

B. Detailed Prescriber Summary of Current Medications

Medication   

Highest daily dose   

Duration (date started)   

Response   

Rationale and comments   

C. Current Side Effects   

D. Current Medication Adherence   

E. Consumer Education

If standardized form, covered medication risks?   

If standardized form, covered medication benefits?  

If nonstandardized form, covered medication risks?   

If nonstandardized form, covered medication benefits?   

 Answers are not scored. These are qualifying questions. Answers cause the item to be excluded from the scoring.
MedTEAM Prescriber Fidelity Scale and Scoresheet 58 Appendix D

Yes No

Does 

not 

apply

P3. Treatment of All Psychiatric 

Diagnoses and Conditions

Treatment plan updated within last 6 months?  

Diagnosis or condition specified?  

Intervention specified?   

Response specified?   

P4. Treatment Guided by Outcomes Medication specified  

Desired outcome specified?   

Rating method specified?   

P5. Documentation of Outcomes Four visits documented?  

Visit 1 target system rated?   

Visit 2 target system rated?   

Visit 3 target system rated?   

Visit 4 target system rated?   

P6. Documentation of Side Effects Medication side effects documented?  

Medication specified?  

Four side effects monitored?   

P7. Treatment of Side Effects One or more side effects documented? 

Intervention specified?   

P8. Review of Treatment for 

Side Effects

Side effect treatment reviewed regularly in last 6 months?  

P9. Simplification of Medication 

Regimen

More than one medication for a single condition?  

If Yes, is there justification within last 6 months?   

Is each medication administered once or twice daily?  

If No, is there justification within last 6 months?   

P10. Recommended Dose Range Four visits documented?  

Medication doses documented?  

Doses within recommended range?   

If No, rationale given?   

P11. Medication Visit Frequency Medication changed within last 12 months? 

If No, has consumer been seen at least 3 times in past 12 months?   

If Yes, was consumer contacted weekly for at least 1 month after the change?   

 Answers are not scored. These are qualifying questions. Answers cause the item to be excluded from the scoring.
Appendix D 59 MedTEAM Prescriber Fidelity Scale and Scoresheet

Yes No

Does 

not 

apply

P12. Treating Refractory 

and Persistently 

Symptomatic Consumers

A. Treatment-Refractory

Do consumers with schizophrenia meet the criteria for being treatment-refractory? 

If consumers are treatment-refractory, have they been prescribed or offered 

clozapine for their condition?  

B. Persistent Symptoms

Are consumers with schizophrenia, major depressive disorder, or bipolar disorder 

experiencing more than mild symptoms at most recent visit? 

If Yes, have these symptoms persisted over all visits in past 6 months? 

If Yes, has consumer been offered a medication change, or an evidence-based 

augmentation treatment, or is a convincing rationale for not doing so documented 

in the last 6 months?

  

P13. Consumer Involvement 

in Treatment Planning

Consumer Involvement

Does documentation exist of consumer goals, preferences, and shared 

decisionmaking?

  

P14. Consumer Medication 

Adherence Strategies

Regular provision of evidence-based strategies to enhance medication adherence, 

such as behavioral tailoring and motivational interviewing, documented for all 

consumers not identified as fully adherent to medications?  

Total number of items scored:

_____

22

Average score:

 Answers are not scored. These are qualifying questions. Answers cause the item to be excluded from the scoring.

Appendix E 61 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

Evaluating Your Program

Appendix E: Chart Review Guide: 

MedTEAM Prescriber Fidelity Scale

Appendix E 63 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

P1. Accessible and Accurate Summary 

of Illness and Medication History 

Before completing the chart review, ask the 

MedTEAM leader:

Does the agency have a standardized admissions 

or annual update form that summarizes consumers’ 

psychiatric history? 

 Yes

 No

What is the form’s name and location in the chart?

An annual update should give complete medication 

history, although it may refer to other documents 

(for example, an admissions form). Give the agency 

credit if it directs you to another location within 

the chart to find the information.

P1 consists of four separate ratings (A – D). If you 

cannot find the relevant information in a given 

chart within 15 minutes, then rate item as absent 

or incomplete. 

A. Timeliness of summary of illness 

and medication history 

A comprehensive summary of consumer psychiatric 

history, updated within the last 12 months, is easily 

found in every chart. (Use the admission form if 

consumers are admitted to the agency in the last 

12 months.)

___/___/___ Date of last comprehensive summary 

of consumer psychiatric history

Is the most recent summary within last 12 months? 

 Yes

 No

Answer the remaining questions for P1 with respect 

to most recent comprehensive review.

B. Diagnosis

 Yes. Present

 No. Absent

List Current Diagnoses
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 64 Appendix E

C. Illness history

Age of onset, number of hospitalizations, most 

recent hospitalization, and summary of the course 

of illness EITHER since onset (if onset was 

less than 5 years ago) OR in last 5 years

Age of onset: 

 Yes. Specify _____________

 No

Number of psychiatric hospitalizations (from 

Structured Clinical Interview for DSM Disorders 

(SCID) or best estimate): 

 Yes. Specify:

_____ Past year

_____ Past 5 years

_____ Lifetime

 No

Most recent hospitalization: 

 Yes. Specify _____________

 No

Summary of course of illness:

 Yes

 No

D. Past medications 

Past psychotropic medication treatments 

are documented including the following: 

	Highest daily dose;

	Duration (years taken);

	Response; and

	Comments. 

Response is defined as assessing the consumer’s 

degree of medication benefit (includes side effects 

that the consumer experiences; nonresponse; and 

inability to adequately gauge response due to 

polypharmacy, nonadherence, or short duration 

of medication therapy). 

For a chart to be scored Yes on this item, it must 

at least have the list of medications taken and one 

other category of information shown below, 

summarized in a single location within the chart. 

If the chart shows at least one prior psychotropic 

medication, fill out the table on the next page. 

History present: 

 Yes

 No

 Does not apply. Go to the next item.

If there is no medication history (consumer has 

never been on medication before), rate the item 

as Does not apply. Go to next item.
Appendix E 65 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

Past Medications 

Review medications for at least the past 5 years, listing the most recent medications first.

Medication Highest Daily 

Dose

Duration 

Years Taken

Response Comments

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 66 Appendix E

P2. Current Comprehensive 

Medication Documentation

Detailed prescriber summary of consumer’s 

current medication status, updated within last 6 

months, including the following:

	Medications (name, highest daily dose, duration, 

response, rationale and comments);

	Current side effects; and

	Medication adherence.

Before completing the review, ask the MedTEAM leader:

Does the agency have a standardized ongoing 

treatment form or another form that summarizes 

the consumer’s current medication? 

 Yes

 No

What is the form’s name and location in the chart?

You may use annual update form found in P1 if this 

form has been completed within the last 6 months 

and covers the elements of P2. A copy of the 

prescription in the chart is not sufficient for any 

item under P2.

P2 consists of five separate ratings (A – E). If you 

cannot find the relevant information in a given 

chart within 15 minutes, then rate the item as 

absent or incomplete.

A. Timeliness of current 

medication documentation 

Prescriber summary of current medications 

completed within last 6 months?

 Yes

 No

Answer remaining questions for P2 with respect to 

most recent prescriber summary. If no prescriber 

summary exists within the past 2 years, score the 

items as No, except for those found in any of the 

last four Progress Notes.

B. Detailed prescriber summary 

of current medication 

Include highest daily dose, duration, response, 

and rationale and comments. 

About rationale and comments, are there 

medications for conditions that may no 

longer exist? 

 Yes

 No
Appendix E 67 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

Current Medications 

Review current medications, listing the most recent medications first.

Medication Total Daily Dose Duration 

Date Started

Response Rationale and 

Comments

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 68 Appendix E

C. Current side effects

Examine the most recent summary (if it was done 

in last 6 months). If no timely summary was done, 

examine the Progress Notes in the last 6 months. 

The chart must list at least one side effect or it must 

show “None reported by consumer.” You cannot 

substitute a Progress Note of a nonprescriber; 

prescribers must complete the Progress Note. 

If chart has the Abnormal Involuntary Movement 

Scale (AIMS) checklist that is completed by the 

prescriber or signed by the prescriber, then rate 

this chart as Yes (Present). 

 Yes. Present

 No. Absent

List Current Side Effects

D. Current medication adherence 

Must be explicitly noted. Any explicit comment on 

level of adherence would be acceptable, including 

“Yes. Mostly.” Examine the most recent summary 

(if done in last 6 months) or if no timely summary 

was done, examine prescriber Progress Notes from 

the last 6 months. 

 Yes. Present

 No. Absent

E. Consumer education

Prescriber discusses therapeutic options and 

associated risks and benefits with consumer 

(and with consumer’s family if consent is given).

To the Reviewer: Examine the chart for last 6 months 

until you find a Progress Note about medication 

education. If the chart documents consumer education 

for therapeutic options, medication risks, and 

medication benefits discussed, rate the item Yes

(can be standardized medication education checklist 

covering these elements). 

Does evidence exist that medication education 

was done in the past 6 months? 

 Yes

 No

If this is a standardized medication education 

checklist, does it explicitly mention:

Medication risks? 

 Yes

 No

Medication benefits?

 Yes

 No

If no standardized medication education checklist 

exists covering both elements, does consumer 

education cover:

Medication risks? 

 Yes

 No

Medication benefits?

 Yes

 No
Appendix E 69 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

P3. Treatment of All Psychiatric 

Diagnoses and Conditions

For all identified psychiatric conditions (for 

example, schizophrenia, schizoaffective disorder, 

anxiety, depression, insomnia, substance abuse, 

mood instability, impulsivity) treated with a 

medication, a specific medication treatment plan is 

documented and updated within the last 6 months. 

For identified psychiatric conditions not treated 

with medications, the treatment plan specifies 

a nonmedication intervention or rationale for 

no intervention.

To the Reviewer: Examine prescriber progress notes 

for last 6 months.

The underlying question for P3 is: Are conditions 

not being treated? 

Rating this item requires clinical judgment. 

For example, if depression is noted and 

antidepressants are prescribed, rate the item as 

Yes (satisfactory) even if no explicit rationale is 

given. In addition, if the consumer is diagnosed 

with a schizophrenia-spectrum disorder and is not 

prescribed an antipsychotic, the chart must give 

a rationale to be rated Yes (satisfactory). If the 

consumer is diagnosed with a schizophreniaspectrum disorder and is not prescribed an 

antipsychotic, and no rationale is present, rate 

the item as No (unsatisfactory). 

Is there an updated treatment plan within the last 

6 months?

 Yes

 No. Go to the next item.

Are psychiatric conditions or diagnoses identified 

in at least one Progress Note?

 Yes

 No

It does not have to be a formal clinical diagnosis. 

Include any condition identified as a clinical problem. 

Examples of conditions identified 

(Check, if present):

Core symptoms: 

 Mania

 Depression

 Positive symptoms or psychosis 

 Negative symptoms

Other symptoms: 

 Irritability 

 Mood lability 

 Agitation 

 Anxiety

 Level of interest

 Appetite 

 Energy level 

 Insomnia 

 Other (specify): ___________________________

An acceptable treatment plan would give the 

intervention and the response to the intervention.

Condition

From above list

Intervention Response 

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 70 Appendix E

P4. Treatment Guided by Outcomes

Treatment plan specifies desired outcomes from 

each medication and a systematic rating method on 

target outcomes.

For each psychotropic medication identified in 

P2B, determine if desired outcomes specified. 

Examples include: 

Consumer will be substantially free of psychotic 

symptoms as measured by the psychosis items 

in the Brief Psychiatric Rating Scale.

Consumer’s depression will be relieved to the point 

of no longer having suicidal thoughts.

(Must be a quantitative measure. Global Assessment 

of Functioning (GAF) is not adequate.) 

Documentation of desired outcome and rating 

method must be documented within last 6 months.

Medication Specifies desired 

outcome?

Specifies rating 

method?

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

q Yes

q No

q Don’t know

P5. Documentation of Outcomes

Prescriber makes or reviews ratings of target 

symptom severity (using any quantitative scale) 

at every medication visit (for last four visits).

Note: Another member of the treatment team can 

make the actual rating as long as the prescriber reviews 

the rating.

Examine the last four medication visits or the last 6 

months if fewer than four visits occurred in the last 

6 months. Check the box next to each visit if it 

applies to the consumer. 

How are symptoms measured (that is, it must be at 

least a 3-point scale)? (GAF is not considered 

adequate measurement of outcomes.)

How many visits were there? 

 No visits

 1 visit 

 2 visits 

 3 visits 

 4 visits 

Visit

Corresponding to 

number of visits

Date Target symptom 

rated?

q Visit 1 ___/___/___ q Yes

q No

q Visit 2 ___/___/___ q Yes

q No

q Visit 3 ___/___/___ q Yes

q No

q Visit 4 ___/___/___ q Yes

q No

q Visit 5 ___/___/___ q Yes

q No
Appendix E 71 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

P6. Documentation of Side Effects

Prescriber or other MedTEAM staff rates severity 

of side effects. 

Are medication side effects documented in at least 

one Progress Note or summary in last 6 months? 

 Yes. Fill out the table below.

 No. Chart fails on this item. Go to next item.

Medication Class 

Does the consumer take this?

Type of Side Effect

For Schizophrenia

Atypical Antipsychotic Tardive Dyskinesia Extrapyramidal 

Symptoms

Blood Glucose Weight Other Symptoms

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

Typical Antipsychotic

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

For Major Depressive Disorder

Antidepressant Weight Sexual Dysfunction Amotivation 

or Apathy

Other Symptoms

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

Antidepressant Augmenting 

Agent

q Yes

q No

q Yes

q No

For Bipolar Disorder

Atypical Antipsychotic Tardive Dyskinesia Extrapyramidal 

Symptoms

Blood Glucose Weight Other Symptoms

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

Typical Antipsychotic

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

q Yes

q No

Antimanic

q Yes

q No

q Yes

q No

q Yes

q No

Antidepressant

q Yes

q No

q Yes

q No

Note: To get credit for monitoring, prescribers cannot simply say “No side effect complaints.” The chart needs to specify. 
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 72 Appendix E

P7. Treatment of Side Effects

If a consumer has side effects, for each side effect 

medication or psychosocial intervention is prescribed.

Does the consumer have one or more side effects 

identified in the last 6 months? 

 Yes

 No. Go to the next item.

Has the consumer been prescribed a side effect 

medication or a psychosocial intervention for each 

side effect (for example, diet and exercise for 

weight control)? 

 Yes

 No

Since side effects differ by medication, a single rule 

for what to measure and how often is impossible. 

Moreover, the likelihood is that some side effects 

decrease over time (for example, orthostasis), whereas others increase (for example, tardive dyskinesia). 

The following chart lists side effects of antipsychotics 

and their relative potency in causing these side 

effects. Each consumer chart should specify side 

effects being monitored (at least four) and how 

they are being monitored in at least one Progress 

Note or summary within the last 6 months. 

For typical and atypical antipsychotics, tardive 

dyskinesia, extrapyramidal symptoms, glucose, 

and weight should be included in the parameters 

being monitored. Other possible side effects to be 

monitored could be blood pressure, lipids, or other 

symptoms specific to the medication.

Examples of prescribed side effect medications 

for extrapyramidal symptoms are the following:

	Cogentin (benztropine);

	Artane (trihexyphenidyl); 

	Amantadine; and

	Benadryl (diphenhydramine). 

For Schizophrenia

Comparative Side Effect Risk of Antipsychotic Agents

Drug

(Trade Name)

Chlorpromazine

(Thorazine)

Haloperidol

(Haldol)

Aripiprazole

(Abilify)*

Clozapine

(Clozaril)*

Olanzapine

(Zyprexa)*

Paliperidone

(Invega)*

Quetiapine

(Seroquel)*

Risperidone

(Risperdal)*

Ziprasidone

(Geodon)*

Anticholinergic +++ + + ++++ ++ + + + +

Extrapyramidal 

symptoms

+++ ++++ + 0 + ++ +/- ++ +

Orthostasis ++++ + + ++++ ++ ++ ++ ++ +

Hyperprolactinemia++ +++ 0 0 + +++ 0 +++ +

Sedation ++++ + + ++++ +++ + +++ + +

Tardive 

dyskinesia

+++ ++++ ? 0 + + +/- + ?

Weight gain ++ + +/- ++++ +++ ++ ++ ++ +/-

Glucose 

intolerance

+ + ? +++ +++ ++ ++ ++ +/-

* indicates atypical antipsychotics

Key 

+/- Negligible

+ Minimal risk of occurrence

++ Low risk of occurrence

+++ Moderate risk of 

occurrence

++++ Highest risk of occurrence

? Inadequate data to assess 

relative risk
Appendix E 73 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

Common Side Effects for Medications

Medication Common Side Effects

For Treating Hypomania and Mania

Lithium Tremor, drowsiness, nausea and vomiting, polyuria, muscle weakness, thirst, 

dry mouth, cognitive impairment

Anticonvulsants Oxcarbazepine Dizziness, somnolence, diplopia, fatigue, nausea, vomiting, ataxia, abnormal 

vision, abdominal pain, tremor, dyspepsia, abnormal gait

Divalproex sodium Nausea and vomiting, increased appetite with weight gain, sedation

Carbamazepine extended-release Dizziness, somnolence, nausea and vomiting, ataxia, pruritus, dry mouth, 

blurred vision, speech disorder

Atypical Antipsychotics Aripiprazole Anxiety, akathisia, constipation, insomnia, nausea, and vomiting

Clozapine Sedation, anticholinergic effects, hypotension, weight gain, hypersalivation, 

constipation, nausea, vomiting, hyperlipidemia, hyperglycemia

Olanzapine Weight gain, sedation, anticholinergic effects, mild extrapyramidal 

symptoms (EPS), hypotension, potential tardive dyskinesia (TD), 

hyperlipidemia, hyperglycemia

Paliperidone/Risperidone EPS, weight gain, mild sedation, anticholinergic effects, changes in blood 

pressure, sexual dysfunction, potential TD, hyperprolactinemia, hyperglycemia

Quetiapine Sedation, blood pressure, weight gain, hyperlipidemia, hyperglycemia

Ziprasidone Nausea and vomiting, constipation, somnolence, EPS, dizziness, 

akathisia, insomnia

For Treating Bipolar Depression

SSRIs Citalopram Dizziness, dry mouth, insomnia, agitation, nausea, sexual dysfunction, 

headache, sweating Escitalopram

Fluoxetine

Paroxetine

Sertraline

Fluvoxamine

Bupropion SR

Bupropion 

(immediate release)

Headache, agitation, weight loss, insomnia, nausea, sweating

Duloxetine Headache, nausea, dizziness, ataxia, somnolence, rhinitis, rash

Nefazodone Dizziness, headache, nausea, somnolence, insomnia

Desvenlafaxine

Venlafaxine XR

Venlafaxine

Dizziness, somnolence, insomnia, decreased appetite, anxiety, headache, 

nausea, sexual dysfunction, sweating

Tricyclic antidepressants 

(TCAs)

Amitriptyline Sedation, dizziness, dry mouth, nausea, insomnia, anxiety, anticholinergic 

effects, tremor, constipation, blurred vision, arrhythmias Clomipramine

Desipramine

Imipramine

Nortriptyline

Amoxapine Sedation, dizziness, dry mouth, nausea, anticholinergic effects, anxiety, 

insomnia, extrapyramidal reactions, seizures

Mirtazapine Dizziness, diarrhea, increased appetite, drowsiness, dry mouth

Monoamine oxidase 

inhibitors (MAOIs)

Phenelzine Restlessness, dizziness, blurred vision, diarrhea, insomnia, weakness, 

arrhythmias, headache, sexual dysfunction Tranylcypromine
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 74 Appendix E

Common Side Effects for Medications

Medication Common Side Effects

For Treating Psychotic Depression

Atypical Antipsychotics Aripiprazole Anxiety, akathisia, constipation, insomnia, nausea, and vomiting

Clozapine Sedation, anticholinergic effects, hypotension, weight gain, hypersalivation, 

constipation, nausea, and vomiting

Olanzapine Weight gain, sedation, anticholinergic effects, mild EPS, hypotension, 

potential TD

Paliperidone/Risperidone EPS, weight gain, mild sedation, anticholinergic effects, changes in blood 

pressure, sexual dysfunction, potential TD

Quetiapine Sedation, blood pressure, weight gain

Ziprasidone Nausea and vomiting, constipation, somnolence, EPS, dizziness, 

akathisia, insomnia

For Augmentation Agents

Lithium Cognitive impairment, tremor, drowsiness, muscle weakness, nausea and 

vomiting, thirst, polyuria

Buspirone Dizziness, nausea and vomiting, insomnia, dry mouth, nervousness

Cytomel Insomnia, diarrhea, tremor, increased or decreased appetite, headache, heat 

intolerance, nausea

*For more information about potential side effects, please consult the Physician’s Desk Reference (PDR) or medication package inserts.

P8. Review of Treatment for Side Effects

Treatments for side effects are reviewed regularly 

to determine their effectiveness and the need for 

continuing treatment.

If the consumer has been prescribed a side effect 

medication, has it been reviewed regularly within 

the last 6 months to determine its effectiveness and 

the need for changing or discontinuing treatment? 

 Yes

 No. Go to next item.

 Does not apply. Go to next item.

P9. Simplification of Medication Regimen 

A single medication prescribed once a day is the 

simplest regimen. Justification for more complex 

medication regimens (more than one medication 

within a class of medication, more than twice a 

day) should be documented and updated within 

the last 6 months. Do not include PRN 

medications (those taken as needed) and antianxiety medication in this analysis. Focus on the 

core symptoms of the illness and the medication 

used to treat those symptoms. 

Does the consumer have more than one 

medication currently prescribed for any single 

identified psychiatric condition (for example, 

two antipsychotics)? 

 Yes. Is there a justification within the last 6 months?

 Yes

 No

 No
Appendix E 75 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

Is each medication in the regimen administered 

once or twice daily? 

 Yes

 No

If no, is there a justification within the last 

6 months? 

 Yes

 No

An appropriate justification for a more complex 

regimen documents that a simpler regimen is 

unacceptable, for example, the consumer has failed 

on a simpler regimen.

Medication Dose Codes

Code Dose

HS Hour of sleep (before bed)

QD Daily

BID 2 times a day

TID 3 times a day

QID 4 times a day

QOD Every other day

RF Refills

ø None

T 1 tablet

P10. Recommended Dose Range

Medication doses are within recommended ranges 

or, when dose falls outside the range (either above 

or below), prescriber documents rationale for 

the deviation.

Examine the last four medication visits (that is, 

the same visits examined in P5). 

Has the consumer been prescribed any 

psychiatric medication? 

 Yes

 No

 Does not apply

If the consumer is on two or more antipsychotics, 

do not score the visit because we do not have 

evidence-based recommended ranges for 

combinations of antipsychotics. 

Rate the item as Does not apply if the consumer 

has not been prescribed any medication for 

schizophrenia or bipolar disorder or if the 

consumer is on two or more antipsychotics. 
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 76 Appendix E

For Antidepressants

Antidepressants

Visit Dose 

Documented?

Dose Within 

Recommended Range?

 Visit 1  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 2  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 3  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 4  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

For Antidepressant Augmenting Agents

Antidepressant 

Augmenting 

Agents

Visit Dose 

Documented?

Dose Within 

Recommended Range?

 Visit 1  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 2  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 3  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 4  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale
Appendix E 77 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

For Treating Bipolar Disorder

Medication 1

Antimanic

Antipsychotic

Antidepressant

Visit Dose 

Documented?

Dose Within 

Recommended Range?

 Visit 1  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 2  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 3  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 4  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

For Treating Bipolar Disorder

Medication 2

Antimanic

Antipsychotic

Antidepressant

Visit Dose 

Documented?

Dose Within 

Recommended Range?

 Visit 1  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 2  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 3  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale

 Visit 4  Yes

 No

 Yes

 No

 1. Rationale given

 2. No rationale
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 78 Appendix E

For Depression

Recommended Antidepressant Dosing Used for Acute Phase Treatment of Depression

Type/Class

Medication 

Brand Name

Initial Target 

Dose Level

Maximum

Dose Level

Recommended 

Administration 

Schedule

Selective serotonin reuptake 

inhibitors (SSRI)

Citalopram (Celexa) 20 mg 60 mg QAM

Escitalopram (Lexapro) 10 mg 30 mg QAM

Fluoxetine (Prozac) 20 mg 40–80 mg QAM

Paroxetine (Paxil) 20–30 mg 40–60 mg QAM

Sertraline (Zoloft) 50–100 mg 150–200 mg QAM

Tricyclic antidepressants 

(TCA)

Amitriptyline 150–200 300 mg QHS

Clomipramine 100–150 250 mg QHS

Desipramine 150 mg 

(>125ng/ml)

300 mg QHS

Imipramine 150 mg 

(IMI+DMI>200 ng/ml)

300 mg 

(200–400ng/ml)

QHS

Nortriptyline 75–100 mg

(50–150 ng/ml)

150 mg 

(50–150 ng/ml)

QHS

Others Amoxapine 200–300 mg 400 mg QHS

Bupropion SR (Wellbutrin 

SR)

200–300 mg 400 mg BID<200 mg/dose

Bupropion (Wellbutrin) 225–300 mg 450 mg TID<150mg/dose

Desvenlafaxine 50 mg 400 mg QD

Duloxetine (Cymbalta) 40 mg 60 mg BID

Mirtazapine (Remeron) 30 mg 60 mg QHS

Nefazodone (Serzone) 200–400 mg 600 mg BID

Venlafaxine (Effexor) 150–225 mg 375 mg BID

Venlafaxine XR(Effexor XR) 75–225 mg 225 mg QD

Monoamine oxidase 

inhibitors (MAOIs)

Phenelzine 45–60 mg 90–120 mg QD–TID

Tranylcypromine 30–40 mg 60–80 mg QD–TID

Recommended Doses of Augmenting Agents for Inadequate Response

Type/Class

Medication 

Brand Name

Initial Target 

Dose Level

Maximum

Dose Level

Recommended 

Administration 

Schedule

Recommended 

Augmentation Agents

Lithium 600–1200 mg 

(0.4–0.6 mEq/L)

1200–1800 mg 

(0.8–1.0 mEq/L)

BID

T3– Cytomel 25–50 micrograms 50 micrograms QAM

Buspirone 25–50 mg 45–60 mg BID-TID

Other Augmenting Agents Dextroamphetamine 5–30 mg 60 mg QAM

Methylphenidate 5–30 mg 40–60 mg BID
Appendix E 79 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

For Mania and Hypomania

Recommended Doses of Medications Used for Acute Phase Treatment of Mania and Hypomania*

Type/Class

Medication 

Brand Name

Initial Target 

Dose Level

Maximum

Dose Level

Recommended 

Administration 

Schedule

Oxcarbazepine 600-2100 2400 BID or TID

Divalproex Sodium (100 ug/mL) (150 ug/mL) BID or QHS

Carbamazepine ER 400-1600 1600 BID

Aripiprazole 15-30 30 QD

Clozapine 300 900 QHS

Olanzapine 10-15 20 BID or QHS

Paliperidone 3 12 QD

Risperidone 2 8 BID or QHS

Quetiapine 200-600 800 BID or QHS

Ziprasidone 40 160 BID

*Doses used for maintenance treatment may be lower.

For Bipolar Depression

Recommended Doses of Medications Used for Acute Phase Treatment of Bipolar Depression*

Type/Class

Medication 

Brand Name

Initial Target 

Dose Level

Maximum

Dose Level

Recommended 

Administration 

Schedule

Selective Serotonin Reuptake 

Inhibitors (SSRIs)

Citalopram 20-40 60 QD

Escitalopram 10-20 20 QD

Fluoxetine 20 80 QD

Fluvoxamine 150-250 250 QD

Paroxetine 20 60 QD

Sertraline 50 200 QD

Anticonvulsant Lamotrigine 200** 600 QD

Second Generation 

Antidepressants

Bupropion SR 300 400 (200/dose max) BID

Desvenlafaxine 50 400 QD

Duloxetine 40-60 60 QD or BID

Nefazodone 300-600 600 QD

Venlafaxine XR 75 225 QD

*Doses used for maintenance treatment may be lower.

**Dose titration is necessary to reach this dose to avoid serious side effects.
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 80 Appendix E

For Depression

Recommended Dosing of Medications for Treating Associated Symptoms of Depression

Associated Symptoms Medication

Usual Dose

Range mg/day Schedule

Insomnia Lorazepam 0.5–2.0 mg QHS; taper after 7 –10 days or as soon as possible

Clonazepam 0.5–2.0 mg

Temazepam 15–30 mg

Zolpidem 5–10 mg

Zaleplon 5–20 mg

Eszopiclone 2–3 mg

Remelteon 8 mg

Trazodone 25–100 mg

Anxiety Lorazepam1 0.5–4.0 mg Q4–6h prn throughout the day

Alprazolam 0.75–4.0 mg

Clonazepam 1.5–3.0 mg

Anxiety 

If history of substance 

abuse or if benzodiazepines 

are contraindicated2

Buspirone 15–60 mg BID-TID

Severe Agitation Lorazepam 0.5–2.0 mg QD

Clonazepam 0.5–2.0 mg

Alprazolam 0.75–4.0 mg

Propranolol 10–30 mg

1 In general, treatment emergent side effects should be addressed first by dose reduction or medication switching, as pharmacological intervention may 

increase the risk of drug interaction and additional adverse effects, thus decreasing consumer compliance.

2 Benzodiazepines are best avoided in consumers with prior history of substance abuse and dependence or who are at risk for substance abuse. Nonaddicting 

agents such as zolpidem or buspirone may be preferred.
Appendix E 81 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

For Antipsychotic Medications

Recommended Dose Ranges

Drug Trade Name Usual Dose 

Range

Aripiprazole (Abilify) 10-30 mg/d

Chlorpromazine (Thorazine) 300-1000mg/d

Clozapine (Clozaril) 300-900mg/d

Fluphenazine (Prolixin/Permitil) 5-20mg/d

Fluphenazine 

Decanoate

6.25-50mgIM/ 

2–4weeks

Haloperidol (Haldol) 2–20mg/d

Haloperidol 

Decanoate

50-200mg/ 

2–4weeks

Loxapine (Loxitane) 50-150mg/d

Molindone (Moban) 50–150mg/d

Olanzapine (Zyprexa) 10–20mg/d

Paliperidone (Invega) 3–12mg/d

Perphenazine (Trilafon) 8–32mg/d

Quetiapine (Seroquel) 300–800mg/d

Risperidone (Risperdal) 2-8mg/d

Thioridazine (Mellaril) 300–800mg/d

Thiothixene (Navane) 15–50mg/d

Trifluoperazine (Stelazine) 5–40mg/d

Ziprasidone (Geodon) 40–160mg/d

P11. Medication Visit Frequency

Consumers are seen at least three times every 12 

months and more frequently when medications are 

being changed or when the prescriber requests it.

Examine the chart for last 12 months. Scan for 

instances where the prescriber has requested 

additional visits. Rate the item according to the 

prescriber’s intentions. 

Has the psychiatric medication changed during last 

12 months? 

 Yes

 No. Has the consumer been seen at least three times 

in the past 12 months?

 Yes

 No

If psychiatric medication changed, was the 

consumer contacted weekly for at least 1 month? 

 Yes

 No. How often was the consumer seen when the 

medications were changed? (Note: You will not 

need this information to rate P11.) _________
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 82 Appendix E

P12. Treating Refractory and Persistently 

Symptomatic Consumers

All consumers who are identified as having 

treatment-refractory illnesses are offered state-ofthe-art treatment for their treatment-refractory 

or persistently symptomatic condition.

To the Reviewer: Rating this item requires some clinical 

judgment and detective work. 

First, identify if the consumer has treatment-refractory 

illness or is persistently symptomatic. If he or she does 

not meet these definitions, rate the item as No and go 

to next item. 

Second, for consumers who have treatment-refractory 

illnesses or are persistently symptomatic, determine 

whether they have been offered an evidence-based 

change in treatment.

A. Treatment refractory

Does the consumer with schizophrenia meet the 

criteria for having a treatment-refractory illness?

Have the consumer’s symptoms failed to respond to 

monotherapy with two or more antipsychotics during 

the course of illness? 

 Yes

 No. Go to the next item.

If a treatment-refractory illness is identified, has 

the consumer been offered clozapine at any time? 

 Yes

 No. Are there contraindications for clozapine? 

 Yes

 No

Evidence of response to current antipsychotic:

If treatment-refractory and not offered clozapine 

and no contraindications for clozapine, is the 

consumer prescribed another antipsychotic and 

does he or she have a documented positive 

response (that is, symptoms are improving or not 

severe) according to the last two Progress Notes?

 Yes

 No
Appendix E 83 Chart Review Guide: MedTEAM Prescriber Fidelity Scale

B. Persistent symptoms

Prescriber systematically identifies consumers 

who are experiencing persistent symptoms 

and prescribes or offers a medication change. 

Is the consumer currently experiencing more than 

mild psychotic symptoms at the time of the most 

recent visit (as documented in Progress Notes, or 

other documented evidence of elevated symptoms, 

for example, Positive and Negative Syndrome 

Scale (PANSS) scores over 60)? 

 Yes. Have these symptoms persisted over all visits 

in past 6 months? (Note: If no visits occurred 

in the preceding 6 months, go to the last visit 

before the current one.)

 Yes. Has the consumer been prescribed 

or offered a medication change?

 No. The consumer is not experiencing 

persistent symptoms. Go to the next 

item.

 No. The consumer is not experiencing persistent 

symptoms. Go to the next item.

For schizophrenia:

Has dosing with the current antipsychotic been 

adjusted upward, as tolerated, or has a switch 

occurred to a different antipsychotic if dose 

adjustments have been unsuccessful? 

 Yes

 No

For depression:

Has the consumer been treated with two 

pharmacologically different antidepressants 

and, if so, has he or she been offered combination 

therapies with an antidepressant plus an augmenting 

agent or cognitive behavioral therapy? 

 Yes

 No

For bipolar:

Has the consumer failed to respond to optimizing 

the dose of monotherapy with a mood stabilizer? 

If so, depending on whether the persistent 

symptoms are depressive, manic, or mixed, have 

appropriate combination treatments been tried? 

 Yes

 No. Has the prescriber documented a rationale for 

not prescribing/offering a medication change 

within the past 6 months? 

 Yes

 No
Chart Review Guide: MedTEAM Prescriber Fidelity Scale 84 Appendix E

P13. Consumer Involvement 

in Treatment Planning

Consumer involvement is documented for all 

medication treatment decisions and evidence 

shows shared decisionmaking between the 

prescriber and the consumer, or a rationale 

for deviation is documented. This may include 

eliciting consumer goals, preferences, and 

ongoing experience with medication treatment. 

To the Reviewer: Examine the last four medication 

Progress Notes or documentation over the last 6 months, 

whichever is shorter. Check the box next to each visit if it 

applies to the consumer. 

Rate the item Yes if your global impression is that 

decisionmaking is shared, based on the presence 

of all or most of the important factors in shared 

decisionmaking, at least once within last four 

medication visits. 

Visit Consumer involvement? 

Or if not, rationale given?

 Visit 1  Yes

 No, but rationale given

 No, no rationale

 Don’t know

 Visit 2  Yes

 No, but rationale given

 No, no rationale

 Don’t know

 Visit 3  Yes

 No, but rationale given

 No, no rationale

 Don’t know

 Visit 4  Yes

 No, but rationale given

 No, no rationale

 Don’t know

P14. Consumer Medication 

Adherence Strategies

Regular provision of evidence-based strategies 

to enhance medication adherence, such as 

behavioral tailoring and motivational interviewing, 

documented for all consumers not identified as 

fully adherent to medications.

To the Reviewer: Examine the chart for each medication 

until you find a note about consumer medication 

adherence strategies, but no longer than 12 months 

previous to date of chart review. (The note does not have 

to be from a prescriber. It could be from a case manager, 

nurse, or anyone else from the treatment team). 

If you cannot find the information within 5 

minutes, rate the item as No (unsatisfactory). 

If the consumer has been prescribed a longacting injection medication, this does NOT 

qualify as a medication adherence strategy. If the 

documentation shows the consumer is completely 

adherent, rate the item as “Does not apply.” 

If evidence shows that the consumer is only 

partially adherent to medication regimen 

(or if the chart is unclear whether the consumer 

is adherent), the chart must document examples 

of consumer medication adherence strategies. 

Examples of consumer medication adherence 

strategies are the following:

	Behavioral tailoring;

	Discussing pros and cons;

	Enlisting support of family members 

and others in support network;

	Exploring consumer’s reasons for wanting 

to take and to avoid medications; and

	Relating adherence to consumer’s 

personal goals.

Rate the item Yes if you find at least one example. 

If evidence shows that the consumer is not adherent 

and the chart has no documentation of medication 

adherence strategies, rate the item No.
Appendix F 85 Outcomes Report Form

Evaluating Your Program

Appendix F: Outcomes Report Form

Appendix F 87 Outcomes Report Form

Outcomes Report Form

Quarter:  January, February, March Year: ______

 April, May, June

 July, August, September

 October, November, December

Reported by: ________________________________________

Agency: ________________________________________ Team: _______________________________

About the consumer

Consumer ID: __________________________ Discharge date: ____/___/____ Date of birth: ____/___/____

  Male Ethnicity: 

  Female Primary diagnosis: 

What was the consumer’s evidence-based service status on the last day of the quarter?

Unknown Not Eligible Eligible Enrolled

MedTEAM    

Integrated Treatment for Co-Occurring Disorders    

Supported Employment    

Assertive Community Treatment    

Illness Management and Recovery    

Family Psychoeducation    

In the past 3 months, how often has the consumer... Number of days Number of incidents

Been homeless?

Been incarcerated?

Been in a state psychiatric hospital?

Been in a private psychiatric hospital?

Been hospitalized for substance abuse reasons?
Outcomes Report Form 88 Appendix F

In the past 3 months, how many days was the consumer 

competitively employed? (Use 0 if the consumer has not been 

competitively employed.)

_______________ Days

Was the consumer competitively employed on the last day 

of the reporting period?

 Yes

 No

What was the consumer’s stage of substance abuse 

treatment on the last day of the quarter? Check one.

 Not applicable

 Pre-engagement

 Engagement

 Early persuasion

 Late persuasion

 Early active treatment

 Late active treatment

 Relapse prevention

 In remission or recovery

What was the consumer’s living arrangement on the last 

day of the quarter? Check one.

 Not applicable or unknown

 Psychiatric hospital

 Substance abuse hospitalization

 General hospital psychiatric ward

 Nursing home 

 Family care home

 Living with relatives (heavily dependent for personal care) 

 Group home

 Boarding house 

 Supervised apartment program 

 Living with relatives (but is largely independent)

 Living independently

 Homeless

 Emergency shelter

 Other (specify): ____________________________ 

What was the consumer’s educational status on the last day 

of the quarter? Check one.

 Not applicable or unknown 

 No educational participation 

 A vocational/educational involvement 

 Pre-educational explorations 

 Working on GED 

 Working on English as a Second Language

 Basic educational skills

 Attending vocational school, vocational program, 

apprenticeship, or high school

 Attending college: 1 to 6 hours per term

 Attending college: 7 or more hours per term

 Other (specify): _________________________________

What is the consumer’s highest level of education? 

Check one. 

 No high school 

 High school diploma or General Educational Development 

(GED) diploma 

 Some college 

 Associate degree 

 Vocational training certificate 

 Bachelor of Arts or Bachelor of Science

 Master’s degree or Ph.D.
Appendix G 89 Instructions for the Outcomes Report Form

Evaluating Your Program

Appendix G: Instructions for the 

Outcomes Report Form

Appendix G 91 Instructions for the Outcomes Report Form

Instructions for the Outcomes Report Form

Before you fill out the Outcomes Report Form, become familiar with the definitions of the data elements to 

provide consistency among reporters.

General data

Quarter: Check the time frame for the 

reporting period.

Year: Fill in the current year.

Reported by: Fill in the name and title of the person 

who completed the form.

Agency: Identify the agency name.

Team: Write the team name or number.

About the consumer

Consumer ID: Write the consumer ID that is used at 

your agency, usually a name or an 

identifying number. This information will 

be accessible only to the agency providing 

the service.

Discharge date: If the consumer has been discharged 

during this report period, fill in the 

discharge date.

Date of birth: Fill in the consumer’s date of birth 

(example: 09/22/1950).

Gender: Check the appropriate box.

Ethnicity: Fill in the consumer’s ethnicity. 

Primary 

diagnosis:

Write the Diagnostic and Statistical 

Manual of Mental Disorders (DSM) 

diagnosis. 

Evidence-based service status

What was the consumer’s evidence-based service status 

on the last day of the quarter? Check the appropriate 

boxes according to these definitions:

Eligible: Does the consumer meet the participation 

criteria for a specific evidence-based 

practice (EBP)? Each EBP has criteria 

for program participation that should be 

used to determine eligibility.

Enrolled: Is the consumer participating in a particular 

EBP service or has the consumer participated 

in the EBP in the past period? Note: 

Aggregate data about eligibility and 

enrollment can be used to determine 

the percent of eligible consumers who 

received services.

Incident reporting

For the following outcomes, record the number 

of days and number of incidents that the consumer 

spent in each category during the reporting period.

Categories:

Been homeless: Number of days that the consumer 

was homeless and how many times 

the consumer was homeless during 

the reporting period. Homeless refers 

to consumers who lack a fixed, regular, 

and adequate nighttime residence. 

Been 

incarcerated:

Number of days and incidents that 

the consumer spent incarcerated in jails 

or in other criminal justice lockups.

Been in a state 

psychiatric 

hospital:

Number of days and incidents that the 

consumer spent hospitalized primarily 

for treatment of psychiatric disorders 

in a state psychiatric hospital.
Instructions for the Outcomes Report Form 92 Appendix G

Been in 

a private 

psychiatric 

hospital:

Number of days and incidents that the 

consumer spent hospitalized primarily 

for treatment of psychiatric disorders 

in a private psychiatric hospital.

Been 

hospitalized 

for substance 

abuse reasons:

Number of days and incidents that the 

consumer spent hospitalized primarily 

for treatment of substance use disorders, 

including both public and private 

hospitals whose primary function 

is treating substance use disorders. 

Competitive employment

In the past 3 months, how many days was the consumer 

competitively employed? Competitive employment

means working in a paid position (almost always 

outside the mental health center) that would 

be open to all community members who wish 

to apply. Competitive employment excludes 

consumers working in sheltered workshops, 

transitional employment positions, or volunteering. 

It may include consumers who are self-employed 

but only if the consumer works regularly and is 

paid for the work.

Stage of substance abuse treatment

What was the consumer’s stage of substance abuse 

treatment on the last day of the quarter? Record the 

consumer’s stage of substance abuse recovery, 

according to the following nine categories: 

	Not applicable: No history of substance 

abuse disorder.

	Pre-engagement: No contacts with a case 

manager, mental health counselor, or substance 

abuse counselor. 

	Engagement: Contact with an assigned case 

manager or counselor, but does not have regular 

contacts. The lack of regular contact implies lack 

of a working alliance. 

	Early persuasion: Regular contacts with a case 

manager or counselor, but has not reduced 

substance use for more than a month. 

Regular contacts imply having a working 

alliance and a relationship in which substance 

abuse can be discussed.

	Late persuasion: Engaged in a relationship with 

a case manager or counselor, is discussing 

substance use or attending a group, and shows 

evidence of reducing use for at least 1 month 

(fewer drugs, smaller quantities, or both). 

External controls (such as Antabuse) may 

be involved in reduction.

	Early active treatment: Engaged in treatment, is 

discussing substance use or attending a group, 

has reduced use for at least 1 month, and is 

working toward abstinence (or controlled use 

without associated problems) as a goal, even 

though consumer may still be abusing. 

	Late active treatment: Engaged in treatment, 

has acknowledged that substance abuse 

is a problem, and has achieved abstinence 

(or controlled use without associated problems) 

but for less than 6 months.

	Relapse prevention: Engaged in treatment, 

has acknowledged that substance abuse 

is a problem, and has achieved abstinence 

(or controlled use without associated problems) 

for at least 6 months. Occasional lapses, not days 

of problematic use, are allowed.

	In remission or recovery: No problems related to 

substance use for more than 1 year and is no 

longer in any type of substance abuse treatment.
Appendix G 93 Instructions for the Outcomes Report Form

Living arrangement

What was the consumer’s living arrangement on the last 

day of the quarter? These data give your agency an 

ongoing record of the consumer’s residential status. 

	Not applicable or unknown

	Psychiatric hospital: Those hospitals, both public 

and private, whose primary function is treating 

mental disorders. This includes state hospitals 

and other freestanding psychiatric hospitals.

	Substance use hospitalization: Those hospitals, both 

public and private, whose primary function is 

treating substance use disorders. 

	General hospital psychiatric ward: Psychiatric wards 

located in general medical centers that provide 

short-term, acute crisis care.

	Nursing home: Facilities that are responsible 

for the medical and physical care of consumers 

and have been licensed as such by the state. 

	Family care home: Consumers live in single-family 

dwellings with nonrelatives who provide 

substantial care. Substantial care is determined 

by the degree to which nonrelatives are 

responsible for the daily care of consumers. 

Such things as medication management, 

transportation, cooking, cleaning, restrictions 

on leaving the home, and money management 

are considered. Nonrelatives may have 

guardianship responsibilities. If consumers 

are unable to do most daily living tasks without 

the aid of caretakers, consider caretakers to be 

providing substantial care.

	Lives with relatives (heavily dependent for personal 

care): Consult consumers and relatives about 

how much family members are responsible for 

consumers’ daily care. An important distinction 

between this status and supervised apartment 

program is to ask, “If the family were not 

involved, would the consumer be living in a 

more restrictive setting?” In assessing the extent 

to which family members provide substantial 

care, consider such things as taking medication, 

using transportation, cooking, cleaning, having 

control of leaving the home, and managing 

money. If consumers are unable to 

independently perform most daily living 

functions, consider family members to be 

providing substantial care. 

	Group home: A residence that is run by staff 

who provide many functions (shopping, meal 

preparation, laundry, etc.) that are essential 

to living independently. 

	Boarding house: A facility that provides a place to 

sleep and meals but is not seen as an extension 

of a mental health agency, nor is it staffed with 

mental health personnel. These facilities are 

largely privately run and consumers have a high 

degree of autonomy. 
Instructions for the Outcomes Report Form 94 Appendix G

	Supervised apartment program: Consumers 

live (fairly independently) in an apartment 

sponsored by a mental health agency. 

In determining whether someone fits this 

category, look at the extent to which mental 

health staff has control over key aspects of the 

living arrangements. Example characteristics of 

control include the following:

	The mental health agency signs the lease;

	The mental health agency has keys to the 

house or apartment;

	Mental health agency staff provides onsite 

day or evening coverage; and

	The mental health agency mandates that 

consumers participate in certain mental 

health services—medication clinic, 

day program, etc., to live in the house 

or apartment.

Note: Consumers who receive only case 

management support or financial aid are NOT 

included in this category; they are considered 

to be living independently.

	Lives with relatives (but is largely independent): 

An assignment to this category requires having 

information from consumers and families. 

The key consideration relates to the degree 

to which consumers can perform most tasks 

essential to daily living without being supervised 

by family members.

	Living independently: Consumers who live 

independently and are capable of self-care, 

including those who live independently with 

case management support. This category also 

includes consumers who are largely 

independent and choose to live with others for 

reasons unrelated to mental illness. They may 

live with friends, a spouse, or other family 

members. The reasons for shared housing could 

include personal choice related to culture or 

financial considerations. 

	Homeless: Consumers who lack a fixed, regular, 

and adequate nighttime residence.

	Emergency shelter: Temporary arrangements due 

to a crisis or misfortune that are not specifically 

related to a recurrence of the consumer’s illness. 

While many emergency shelters provide 

emotional support, the need for emergency 

shelter is due to an immediate crisis unrelated 

to the consumer’s mental illness. 

	Other: Those who complete the form should 

clearly define this status in the space provided.
Appendix G 95 Instructions for the Outcomes Report Form

Educational status

What was the consumer’s educational status on the last 

day of the quarter? These data give your agency an 

ongoing record of the consumer’s educational status. 

	Not applicable or unknown

	No educational participation: Consumer is not 

participating in educational activities.

	Avocational/educational involvement: Organized 

classes in which consumers enroll consistently 

and expect to take part for the purpose of life 

enrichment, hobbies, recreation, etc. These 

classes must be community-based, not run 

by the mental health center. Classes are those 

in which anyone could participate, not just 

consumers. If any of these activities involve 

college enrollment, use the categories below. 

	Pre-educational explorations: Consumers in this 

status are engaged in educational activities 

with the specific purpose of working toward an 

educational goal. This includes consumers who 

attend a college orientation class with the goal 

of enrolling, meet with the financial aid office 

to apply for scholarships, or apply for admission 

to enroll. This status also includes consumers 

who attend a mental health center-sponsored 

activity focusing on an educational goal (for 

example, campus visits with a case manager 

to survey the location of classrooms or meetings 

with the case manager and college staff to 

secure entitlements). 

	Working on General Educational Development (GED):

Consumers who are taking classes to earn their 

GED diploma. 

	Working on English as Second Language: Consumers 

who are taking classes in English as a Second 

Language in a community setting. 

	Basic educational skills: Consumers who are taking 

adult educational classes focused on basic skills, 

such as math and reading. 

	Attending vocational school or apprenticeship, vocational 

program, or attending high school: Consumers 

who are—

	Participating in community-based 

vocational schools; 

	Learning skills through an apprenticeship, 

internship, or in a practicum setting;

	Involved in on-the-job training to acquire 

more advanced skills;

	Participating in correspondence courses 

which lead to job certification; and 

	Young adults attending high school.

	Attending college: 1 to 6 hours: Consumers who 

attend college for 6 hours or fewer per term. 

This status continues over breaks, etc., if 

consumers plan to continue enrollment. This 

status suggests that consumers regularly attend 

college and includes correspondence, TV, or 

video courses for college credit. 

	Attending college: 7 or more hours: Consumers 

attend college for 7 or more hours per term. 

This status continues over breaks, etc., if 

consumers plan to continue enrollment. 

Regular attendance with expectations of 

completing course work is essential for 

assignment to this status. 

	Other: Those who complete the form should 

clearly define this status in the space provided.

Appendix H 97 Assessor Training and Work Performance Checklist

Evaluating Your Program

Appendix H: Assessor Training and 

Work Performance Checklist

Appendix H 99 Assessor Training and Work Performance Checklist

Assessor Training and Work Performance Checklist

Assessment date: ____/___/____

Assessor’s name ____________________________________________________________ ______________________

First Middle Initial Last Title

Agency visited ____________________________________________________________

Agency address __________________________________________________________________________________________________

Street

 __________________________________________________________________________________________________

City State ZIP code

EBP assessed __________________________________________________________________________________________________

Assessor qualifications

Yes

 1a. Data collection and skills: Assessor’s skills are evidenced by his or her prior work experience, credentials, 

or supervisor’s observations.

 1b. Evidence-based practices (EBP) knowledge: Assessor’s knowledge is evidenced by his or her prior work 

experience, credentials, or passing a knowledge test on a specific EBP.

 1c. Training: Assessors receive at least 8 hours of systematic training on chart review, interviewing techniques, 

and process assessment.

 1d. Shadowing: Assessors complete at least one assessment with an experienced assessor before the first official 

process assessment.

 1e. Practice rating: Assessors co-rate as practice before being official assessors and agree exactly with an 

experienced assessor on ratings for at least 80 percent of items.

____/5 Subtotal
Assessor Training and Work Performance Checklist 100 Appendix H

Data collection

Yes

 2a. Contact and scheduling: With contact person, assessors identify a date convenient to visit site, explain 

purpose of the assessment, identify information to be assembled ahead of time, and develop specific 

schedule of interviews and assessment activities. 

 2b. Number of assessors: Two or more assessors are present during the assessment visit and 

independently rate all items. If agency is working with a consultant, assessor may join with consultant to 

conduct assessments.

 2c. Time management: Sufficient time is allotted and all necessary materials reviewed (2 days 

for two assessors).

 2d. Interviewing: Interview all the sources stipulated in the protocol (for example, interviews with the program 

leader, team members, and consumers).

 2e. Completion of documents: Complete scoresheet, cover sheet, and any other supplemental documents 

relating to the agency.

 2f. Documentation supporting rating: Each assessor provides written documentation for evidence supporting 

the rating for each item (such as marginal notes). 

 2g. Chart selection and documentation: Chart selection follows guidelines provided in the protocol 

(for example, the appropriate type and number of charts). Assessors note discrepancies (such as 

chart unavailability).

 2h. Chart review: Both assessors review all charts and rate them independently.

 2i. Resolution of discrepancies: When a discrepancy exists between sources (such as charts and MedTEAM 

team members), assessors follow up with an appropriate informant (typically the MedTEAM leader or 

relevant staff members).

 2j. Independent ratings: No later than 1 day after the assessment, assessors independently complete scales 

before discussing ratings.

____/10 Subtotal

Post-assessment visit

Yes

 3a. Timely consensus: Within 5 working days after the assessment, assessors discuss their ratings to determine 

consensus ratings, identifying any followup information needed. A third assessor (for example, supervisor) 

may be consulted to resolve difficult ratings. 

 3b. Interrater reliability: Raters agree exactly on ratings for at least 80 percent of the items. Sources of 

unreliability are discussed with supervisor and strategies developed to reduce future unreliability. 

 3c. Follow up on missing data: If followup calls are needed to complete an item, information obtained within 

3 working days.

____/3 Subtotal
Appendix H 101 Assessor Training and Work Performance Checklist

Comprehensive report writing

Yes

 4a. Documentation of background information: 

 List recipients of report in the header (usually the agency director and MedTEAM leader; add others by 

mutual agreement).

 Summarize time, place, and method.

 Provide background about scale.

 4b. Site and normative fidelity data: Provide a table with item-level (consensus) scores, along with normative 

data (if available). Normative data include both national and state norms. In this table, provide comparative 

site data from prior assessments. On second and later assessments, provide a graph of global fidelity ratings 

over time for the site (trend line).

 4c. Quantitative summary: Provide narrative summary of quantitative data. List strengths and weaknesses.

 4d. Score interpretations: 

 Interpret overall score

 Include other pertinent observations

 Provide overall summary

 Provide opportunity for site to comment and clarify

 4e. Report editing: If agency is working with a consultant, consultant may write report. Assessor and 

supervisor review draft of the report before it is submitted to the agency.

____/5 Subtotal

Report submission and followup

Yes

 5a. Timely report: Report sent to agency director within 2 weeks of visit.

 5b. Follow up on report: If agency is working with a consultant, consultant discusses report with designated 

agency staff within 1 month of assessment.

____/2 Subtotal

Quality control

Yes

 6. Quality control: Supervisor reviews assessments and gives feedback, as necessary, to assessors. Depending 

on skill level of assessors, supervisor periodically accompanies assessors on assessment for quality 

assurance purposes. 

____/1 Subtotal

____/27 Total—Add the subtotals.


28609.0510.8712010401

HHS Publication No. SMA-10-4548

Printed 2010