# Evidence-based assessment/Prediction phase

HGAPS is finding new ways to make psychological science conferences more accessible!
Here are examples from APA 2022 and the JCCAP Future Directions Forum. Coming soon... ABCT! ~ More at HGAPS.org ~
* Medical disclaimer: This page is for educational and informational purposes only and may not be construed as medical advice. The information is not intended to replace medical advice offered by physicians. Please refer to the full text of the Wikiversity medical disclaimer.
The first phase of assessment involves making rapid decisions about contending hypotheses, deciding which to evaluate further to build a case formulation and a treatment plan. Listing the most common disorders and benchmarking the base rates are the preamble to the process. They create a shortlist of hypotheses that will be worth considering precisely because they are commonplace. The list functions as a baseline set of hypotheses. We then look for disconfirming evidence as well as confirmatory evidence. The top panel of Figure 1 illustrates a graphical way of viewing the common issues as leading initial hypotheses that warrant assessment.
Studies of clinical decision making find that when we use unstructured interviews, we tend to formulate one hypothesis based on the presenting problem (usually in the first few minutes of the interview!) and then we do an excellent job of searching for confirmatory data. We tend not to look for disconfirming evidence, and we also rarely consider competing or augmenting hypotheses. These dynamics play into our tendency to underestimate comorbidity and to have “favorite” diagnoses that we identify at high rates. The cognitive heuristics can be particularly error prone when working with minority groups, who may use different language to describe the presenting problem – leading to a different starting hypothesis. Consider the case of pediatric bipolar disorder: Black, low income parents are more likely to describe their concerns as focused on the youth’s behavior, and white middle class families are more likely to describe their main worry as mood swings. One description pulls for an initial hypothesis of conduct problems, and the other for a mood disorder conceptualization. The confirmatory bias kicks in immediately, and if we do not systematically assess for potentially disconfirming information, then the black child winds up diagnosed with conduct disorder, and the equally labile white youth diagnosed with bipolar – exactly the pattern we see in services data. In normal clinical practice, we do not receive corrective feedback – there are no structured diagnostic interviews of a subset of cases, it is not common to hear contrasting formulations or contradictory opinions at case conferences, and if treatment does not progress because the initial assessment was off, there are a host of other reasons that are likely to come to mind first (e.g., family is too busy, not ready for change). The benchmarks remind us that these disorders are equally common in both demographic groups and deserve equal initial consideration.

## Rationale
### EBM Decision Making and Zones of Clinical Action

We have extended the EBM model by adding the traffic light color metaphor to label the zones, so that the region below the Wait-Test threshold is the “Green Zone,” the middle region betwixt the Wait-Test and Test-Treat thresholds is the “Yellow Zone,” and above the Test-Treat is the “Red Zone." The color labels are easy for families to comprehend. The bars in Figure 1 are shaded to indicate into which zone they fall. Another refinement was to mash up the EBM thresholds with the community mental health idea of primary, secondary, and tertiary intervention. Whereas classic EBM only thinks of treatment in the Red Zone, the community mental health model of levels of intervention encourages us to think about primary prevention options in the green zone (low cost, low risk, and likely to avert later problems). Similarly, the Yellow Zone could be the place for targeted intervention for at risk groups, or the deployment of broad spectrum treatments that are unlikely to harm and that may have some efficacy while we continue intensive assessment to refine the diagnosis. In the case of mood disorder, suggesting that someone with a family history of bipolar but no current symptoms take fish oil supplements would be an example of a Green Zone recommendation. Someone with mood symptoms but insufficient information to determine whether it is a unipolar or bipolar depression could be a good candidate for psychotherapy emphasizing sleep hygiene, coping skills, and CBT components – these are likely to be helpful whether the mood disorder follows a unipolar or bipolar course, and unlikely to cause harm; so they can be started while the evaluation process is ongoing. Atypical antipsychotics would be an example of a Red Zone treatment that should wait in abeyance until the probably of a bipolar diagnosis is high enough to justify the attendant risks and side effects.
Some of the base rates will be high enough to start in the Yellow Zone. At a typical outpatient clinic, the base rates of ADHD, disruptive behavior, anxiety, and mood will be high enough that they should routinely be considered in evaluation. The Yellow Zone issues define the targets for our routine evaluation. If we build a core battery, we should match the measures to the Yellow Zone issues. If we use a semi-structured interview, we want to make sure that the modules cover the Yellow Zone topics, as well as less common ones that could get kicked up to Yellow Zone levels of probability via screening or identification of risk factors.

## Steps to put into practice

The next layer of assessment consists of brief screens, key factors from developmental history, and gathering information from collateral informants’ perspectives. The screening measures can include instruments with broad content coverage, such as the Achenbach checklists or the Strengths and Difficulties Questionnaire. These include subscales that address symptoms associated with many of the most common issues: Internalizing or emotional problems scores inform about whether anxiety or mood disorder might be present; externalizing scores scout for disruptive behavior disorders; and attention problems provide data related to ADHD or learning issues.

   ### Interpreting screening measure scores
   #### Overview

   The purpose of this subsection is to use Bayesian probability theory in order to accurately predict diagnoses, given base diagnosis rate in the region and diagnostic likelihood ratios.

   #### Likelihood Ratios

   Likelihood ratios (also known as likelihood ratios in diagnostic testing) are the proportion of cases with the diagnosis scoring in a given range divided by the proportion of the cases without the diagnosis scoring in the same range. [17] The table below shows area under the curve (AUCs) and likelihood ratios for potential screening measures.
   | Likelihood Ratio | Comments |
   | Larger than 10, smaller than 0.10 | Frequently clinically decisive |
   | Ranging from 5 to 10, 0.20 | Helpful in clinical diagnosis |
   | Between 2.0 and 0.5 | Rarely result in clinically meaningful changes of formulation |
   | Around 1.0 | Test result did not change clinical impressions at all |
   "LR+" refers to the change in likelihood ratio associated with a positive test score, and "LR-" is the likelihood ratio for a low score. Likelihood ratios of 1 indicate that the test result did not change impressions at all[16]. On the other hand, likelihood ratios larger than 10 or smaller than 0.10 are frequently clinically decisive, 5 or 0.20 are helpful, and between 2.0 and .5 are small enough that they rarely result in clinically meaningful changes of formulation.

#### Probability Nomogram

Once we know the LR, the next step is to combine it with other information about the client. One way of doing this is using a probability nomogram. A nomogram uses geometry to turn the math steps of updating a probability into a "connect the dots" exercise.

## Steps for researchers

* ROC Party

## Tables and figures
### Psychometric properties of common screening instruments

| Measure | Format (Reporter) | Age Range | Administration/ Completion Time | Coverage | Test-Retest Reliability | Free and Accessible Measures |
| Child Behavior Checklist (CBCL) *not free | Parent-report | | | The CBCL, YSR, and TRF each has all of the following subscales: anxious/depressed withdrawn/depressed somatic complaints social problems thought problems attention problems rule-breaking behavior aggressive behavior affective problems, anxiety problems somatic problems ADHD oppositional defiant problems conduct problems Higher order: internalizing and externalizing The CBCL and YSR are have competence scales for: activities social relations school total competence | G-E | *not free* Website to purchase |
| Youth Self Report (YSR) *not free | Self-report | 11-18 years old | 10 minutes | The CBCL, YSR, and TRF each has all of the following subscales: anxious/depressed withdrawn/depressed somatic complaints social problems thought problems attention problems rule-breaking behavior aggressive behavior affective problems, anxiety problems somatic problems ADHD oppositional defiant problems conduct problems Higher order: internalizing and externalizing The CBCL and YSR are have competence scales for: activities social relations school total competence | | *not free* Website to purchase |
| Teacher’s Report Form (TRF) *not free | Teacher-report | | | The CBCL, YSR, and TRF each has all of the following subscales: anxious/depressed withdrawn/depressed somatic complaints social problems thought problems attention problems rule-breaking behavior aggressive behavior affective problems, anxiety problems somatic problems ADHD oppositional defiant problems conduct problems Higher order: internalizing and externalizing The CBCL and YSR are have competence scales for: activities social relations school total competence | | *not free* Website to purchase |

### Likelihood ratios and AUCs of common screening instruments

| Screening Measure (Primary Reference) | AUC | LR+ Score | LR- Score | Clinical generalizeability | Study description |
| Child Behavior Checklist (CBCL) - Attention and Aggression Problems T-Score | Boys: .86 (N=111) | 10.2 (>55) | 0.41 (<55) | Somewhat High | Utilized sample ages 6-18 which consisted of 219 brothers and sisters of children who were referred to a hospital pediatric unit for ADHD or other symptoms. Half of these siblings had brothers and sisters who had ADHD, half did not. |
| Child Behavior Checklist (CBCL) - Attention and Aggression Problems T-Score | 11.2 (>55) | 0.35 (<55) | | Somewhat High | Utilized sample ages 6-18 which consisted of 219 brothers and sisters of children who were referred to a hospital pediatric unit for ADHD or other symptoms. Half of these siblings had brothers and sisters who had ADHD, half did not. |
| Teacher Response Form (TRF) - Attention Problems T-Score | Not reported (N=184) | 3.66 (>70) | 0.73 (<70) | Somewhat High | Utilized sample ages 5-12 years referred to a research clinic for assessment of ADHD. 108 children were ultimately diagnosed with ADHD, while 76 were not. LR's are calculated for discriminating between those two groups. |
| Teacher Response Form (TRF) - Attention and Aggression Problems T-Score | Not reported (N=184) | 4.33 (>70) | 0.89 (<70) | Somewhat High | Utilized sample ages 5-12 years referred to a research clinic for assessment of ADHD. 108 children were ultimately diagnosed with ADHD, while 76 were not. LR's are calculated for discriminating between those two groups. |
| Child Behavior Checklist (CBCL) - Attention Problems T-Score | .84 (N=187) | 6.92 (>55) | 0.19 (<55) | Somewhat High | Utilized sample ages 6-18 recruited from local pediatricians, psychiatrists, and community advertisements. Included 95 children who met criteria for ADHD. 70 of these children also met criteria for ODD/CD. |
| Child Behavior Checklist (CBCL) - Attention Problems T-Score | .84 (N=187) | 12.2 (>60) | 0.41 (<60) | Somewhat High | Utilized sample ages 6-18 recruited from local pediatricians, psychiatrists, and community advertisements. Included 95 children who met criteria for ADHD. 70 of these children also met criteria for ODD/CD. |
| Child Behavior Checklist (CBCL) - Attention Problems T-Score | .84 (N=187) | 47 (>65) | 0.53 (<65) | Somewhat High | Utilized sample ages 6-18 recruited from local pediatricians, psychiatrists, and community advertisements. Included 95 children who met criteria for ADHD. 70 of these children also met criteria for ODD/CD. |
| Child Behavior Checklist (CBCL) - Attention Problems T-Score | .84 (N=187) | 34 (>70) | 0.66 (<70) | Somewhat High | Utilized sample ages 6-18 recruited from local pediatricians, psychiatrists, and community advertisements. Included 95 children who met criteria for ADHD. 70 of these children also met criteria for ODD/CD. |
| Child Behavior Checklist (CBCL) Anxious/Depressed Scale T-score | .70 (N=470) | 3.78 (60+) | .39 (<60) | High. Large diverse sample with mixed depression sample compared to samples without depression. | |
| CBCL Anxious/Depressed Scale T-score | .75 (N=1445) | 1.49 (raw score 9+) | .67 (raw score ≤) | | |
| CBCL Affective Problems Scale T-score | .78 (N=1445) | 1.49 (raw score 9+) | .67 (raw score ≤) | | |
| Youth Self Report (YSR) | .81 (N=207) | -- | -- | | |
| WHO-Five Well-being Index (WHO-5) 3 | .885 (N=294) | 4.40 (raw score 11+) | .15 (raw score ≤) | General sample of adolescents from Norway and Denmark | |
Note: “LR+” refers to the change in likelihood ratio associated with a positive test score, and “LR-” is the likelihood ratio for a low score. Likelihood ratios of 1 indicate that the test result did not change impressions at all. LRs larger than 10 or smaller than .10 are frequently clinically decisive; 5 or .20 are helpful, and between 2.0 and .5 are small enough that they rarely result in clinically meaningful changes of formulation (Sackett et al., 2000).