bluuebunny/crossref_metadata_2025_split huggingface dataset:
B - 127 Equivalence of In-Person and Remotely Administered Performance Validity Tests: Evidence from a Forensic Sample
Authors: Bart Thomas A , Nghiem Duyen M , Carlew Anne R , Wisdom Nick , Collins Robert L , Sullivan-Baca Erin 
Published: September 2024

Abstract

Objective
While neuropsychologists are increasingly recognizing the importance of tele-neuropsychology (teleNP), existing research has not comprehensively explored equivalency of commonly used performance validity tests (PVTs) and symptom validity tests (SVTs) across in-person and remote administration. This is despite the fact that such measures are a crucial component of any neuropsychological assessment battery and are, as such, considered medically necessary. Furthermore, the existing teleNP research base is primarily limited to clinical and other non-forensic samples, with limited applicability to forensic teleNP evaluations. The current study examined the equivalency of in-person and remote administration of widely used PVTs and SVTs in a forensic sample.

Method
Archival data between July 2021 and December 2023 were collected from a private practice setting in which board-certified neuropsychologists (ABPP/ABCN) conducted independent medical evaluations (IMEs) in controlled settings with utilization of an on-site proctor. Mean PVT and SVT scores across in-person and remote evaluations were compared.

Results
All included PVTs and SVTs (TOMM, DCT, WAIS-IV RDS, Word Choice, WMT, MENT, MSVT, IOP-29, SIMS, MMPI-2-RF scales) showed mean equivalency across in-person and remote administration (p &gt; 0.05).

Conclusions
Findings provide preliminary support for the adaptation of both PVTs and SVTs for teleNP. Implications are present for forensic and clinical practice, although future research should continue to establish preliminary trends.
B - 128 Examining Word Choice (WC) and Test of Memory Malingering Trial 1 (T1) Cutoffs by Ethnicity, Language, and Education
Authors: Hausman Hanna , Roop Samantha , Hromas Gabrielle , Rolin Summer , Davis Jeremy 
Published: September 2024

Abstract

Objective
Optimal cutoffs on performance validity tests can vary by examinee characteristics including ethnicity, language, and education. We examined WC and T1 cutoffs in Hispanic/Latino (H/L) and non-H/L participants with consideration of language and education.

Method
Local clinical data were retrospectively examined. Cases with WC and T1 data were included if there was no diagnosis of dementia, intellectual disability or psychosis yielding a sample (N = 580) in which 41% of participants identified as H/L and 26% of participants with language data (n = 530) reported bilingualism. Outcome measures included WC and T1 scores and dichotomous cutoffs.

Results
Education showed significant associations with WC and T1 (r = 0.12–0.20) and varied by ethnicity and bilingualism (pH/L) with a small effect (d = 0.21). No language differences were observed on WC or T1. Logistic regression models predicting dichotomous WC and T1 with education, ethnicity, and language as predictors were significant overall, but ethnicity and language were not significant. Grouping with independent validity indicators, WC false positives (5–6%) did not vary significantly by ethnicity or language; T1 false positives were not significantly different but were slightly higher in H/L and bilingual (14–15%) participants compared to non-H/L and monolingual (9–10%) participants.

Conclusions
WC and T1 cutoffs from prior research in mixed clinical and forensic samples appear appropriate for use with H/L and bilingual examinees. Lower educational levels remain an area in need of further validity measure research.
B - 129 Examining the Influence of Age and Other Demographics on Word Choice (WC)
Authors: Roop Samantha , Hausman Hanna , Hromas Gabrielle , Rolin Summer , Davis Jeremy J 
Published: September 2024

Abstract

Objective
Performance validity tests (PVTs) often vary by examinee characteristics including age and education among other demographics. Normative data on WC are limited to adults in the 16–69 age range. We examined relationships among demographic variables and WC in a mixed clinical sample.

Method
Local clinical data were retrospectively examined. Cases with WC and four independent PVTs were included; cases with dementia, intellectual disability or psychosis were excluded yielding a sample (N = 362) in which 61% of participants were female and 44% identified as Hispanic/Latino. Average age and education were 48 and 13 years, respectively. Outcome measures included WC and PVT scores and dichotomous cutoffs.

Results
WC was associated with age (r = −0.27) but not education. WC scores were significantly different by gender with a small effect (d = 0.39) but not by ethnicity. Using logistic regression with WC as outcome, age, education, and gender were significant predictors. Significantly more males (11%) scored below the WC cutoff than females (5%) in the valid group. Among participants 70 and older, 17% scored below the standard WC cutoff, but only 7% scored below expectation on two or more criterion PVTs. Lowering the WC cutoff resulted in improved specificity in the 70 and older subset: 15% with WC ≤ 40, and 11% with WC ≤ 36.

Conclusions
WC scores varied by age and gender. The standard WC cutoff showed an elevated false positive rate among adults 70 and older. Consideration might be given to adjusting the cutoff if subsequent research supports these preliminary findings. Further research might also examine relationships among gender, education, and WC.
B - 130 Application of Item Response Theory to the MMPI-2-RF Response Bias Scale-19 (RBS-19)
Authors: Childers Luke , Herring Tristan , Albertorio Arianna , Ingram Paul 
Published: September 2024

Abstract

Objective
RBS is a cognitive overreporting scale on the MMPI-family of instruments. Recent efforts to refine the scale removed nine items lacking conceptual and empirical relationships to cognitive dysfunction (Ingram et al., 2024; Ratcliffe et al., 2022). This revised scale version (RBS-19) demonstrated improved utility and classification accuracy, although the improvements were small. Albertorio et al (2024) found that the RBS could be reduced to 14 items and identified potential “critical items”. This study uses IRT in a database of Active-Duty personnel to explore reduced item RBS forms.

Method
Using a 2-parameter IRT model, we examined cognitive overreporting across the 19-item MMPI-2-RF RBS-19 Scale. Participants (n = 326; 96% male) were divided into pass-all vs fail 2+ standalone PVTs for criterion-based comparisons (e.g., ROC analysis and mean difference), with indetermined profiles removed.

Results
Results indicate good theta coverage. Two items (24 and 26) were removed due to low α (2). RBS-14/17/19 all had similar AUC (0.72 vs 0.71 vs 0.72) and similar effect sizes between groups.

Conclusions
IRT analyses supported a 17-item RBS scale rather than the 14 or 19 item versions in this sample. However, no “critical items” emerged in this sample. Such differences may reflect changes in factor structure based on referral reason (Boress et al., 2023).
B - 131 Validation of the California Verbal Learning Test-II-Short Form Forced Choice Paradigm as an Indicator of Performance Validity
Authors: Do Cardinal , Milam Alicia L , Soble Jason R , Webber Troy A 
Published: September 2024

Abstract

Objective
Although the Forced Choice (FC) paradigm of the standard California Verbal Learning Test-Second Edition (CVLT-II) has been extensively validated as an embedded performance validity test (PVT), no studies have validated the CVLT-II Short Form (SF) against a standard battery of external PVTs.

Method
Participants included 98 veterans (Mage = 69.7 ± 9.0; 86.7% cisgender-male) who completed the CVLT-II-SF FC and the following independent criterion freestanding/embedded PVTS as part of comprehensive outpatient neuropsychological evaluations: the Advanced Clinical Solutions Word Choice Test, Test of Memory Malingering-Trial 1, and Reliable Digit Span from the Wechsler Adult Intelligence Test-Fourth Edition Digit Span subtest. Invalid performance was identified using two different criteria: (1) failing ≥2 PVTs in addition to other clinical indicators of poor engagement, or (2) failing one PVT. Receiver Operator Characteristic (ROC) curves tested the classification accuracy of identifying participants in the invalid group versus those with valid presentations and with or without cognitive impairment.

Results
Using both criteria, ROC analyses accurately identified invalidity (AUCs = 0.708–0.807) in the total sample, with cut-scores of ≤7 or ≤ 8 on the CVLT-II-SF FC exhibiting adequate specificities (0.900–0.986) and sensitivities of 0.407–0.500. Classification accuracy was acceptable for differentiating the invalid and cognitively intact groups (AUCs = 0.734–0.829) at a cut score of ≤8 (specificities = 0.923–0.958, sensitivities = 0.500–0.704), but was variably acceptable for differentiating the invalid and cognitively impaired groups (AUCs = 0.684–0.794; cut-score of ≤7 with specificities = 0.962–0.978 and sensitivities = 0.229–0.407).

Conclusions
Our findings support the CVLT-II-SF FC trial as an embedded PVT, particularly when differentiating cognitively intact examinees from those with noncredible presentations.
B - 135 Wisconsin Card Sorting Test 64-Card Computer Version (WCST-64-CV) Failure to Maintain Set (FMS) as an Embedded Performance Validity Indicator in a Mixed Clinical Sample
Authors: Cation Bailey , Schoenberg Michael 
Published: September 2024

Abstract

Objective
The current objective was to determine the sensitivity and specificity of the Wisconsin Card Sorting Test 64-card Computer Version (WCST-64-CV) Failure to Maintain Set (FMS) as an embedded validity indicator.

Methods
Data were from an archival database of an academic medical center neuropsychology service. Three hundred sixty-eight adult participants completed the WCST-64-CV and Word Memory Test (WMT). Participants were defined as noncredible if all three of their WMT Immediate Recall (IR), Delayed Recall (DR), and Consistency (CNS) scores were 82.5% or below. Participants were defined as credible if all three of IR, DR, and CNS scores were above 82.5%. Forty-seven participants with scores both above and at/below 82.5% were excluded (N = 321). The sample was mean age 45 years (standard deviation [SD] = 17), mean education 13 years (SD = 2), 50% female, and 77% Caucasian. Primary diagnoses were 56.8% epilepsy and 18.1% mild cognitive impairment. No participants had dementia or an acute stroke.

Results
Twenty-eight participants were noncredible (8.7%) and 293 were credible (91.3%). One hundred sixty-five participants got 0 FMS (51.4%), 95 participants got 1 (29.6%), 41 participants got 2 (12.8%), and 20 participants got 3 or more (6.2%). The lowest cut-off that achieved at least 90% specificity was failure defined as ≥3. The specificity was 93.5% and the sensitivity was 3.5%. A cut-off of ≥2 achieved 80.8% specificity and 17.8% sensitivity. A cut-off of ≥1 achieved 50.8% specificity and 42.8% sensitivity.

Conclusions
There is no cut-off of WCST-64-CV FMS that achieves adequate specificity and sensitivity as an embedded validity indicator in this mixed clinical sample.
B - 136 A Validity Study of an Embedded Performance Validity Measure of Wide Range Assessment of Memory and Learning, Third Edition
Authors: Franke Michelle L , Adams Wayne , Black Jacy , Cantley Jessica , Gordon Taryn 
Published: September 2024

Abstract

Objective
This study investigated chance levels of responding on the Recognition Memory component of the performance validity test 2 (PVT2) measure embedded within the Wide Range Assessment of Memory and Learning, Third Edition (WRAML3; Adams & Sheslow, 2021). While standardized norms are provided, the WRAML3 has yet to examine PVT2 scores obtained through random responding. Given the impact of suboptimal effort and performance on test scores, providing scores obtained through guesstimation can help differentiate low scores due to impairment from chance responding.

Methods
Participants included 50 volunteer undergraduates and graduate students. To assess levels of random responding on PVT2 items, participants responded to the four Recognition memory subtests of the WRAML3 without the usual prior exposure to immediate and delayed subtest content. One-sample t-tests were conducted to compare participant performance scores to statistical chance levels of responding and the established PVT2 cut-off scores.

Results
Analyses revealed that the mean total PVT2 score was significantly lower than the established WRAML3 PVT2 cut-off score of 17 (t(49) = −26.88, p &lt; 0.001). Regarding chance levels of responding, participants’ random responses were significantly different from the hypothesized chance mean of 5 (t(49) = 6.24, p &lt; 0.001).

Conclusions
Findings support the overall utility of the available WRAML-3 embedded PVT2 cutoff scores while indicating individuals who rely on guesstimation techniques may be able to answer better than chance. These results establish a new standard of random response on the WRAML3, which may be utilized to determine whether scores below the PVT2 cutoff are invalid or may warrant further investigation.
B - 137 Analyzing Completion Time on a Created Forced Choice Trial of the Hopkins Verbal Learning Test – Revised (HVLT-R) to Measure Engagement in a Mixed Clinical Sample
Authors: Kapoor Arunima , Conrad Alexis , Thomas Aaron , Kristinsson Hayley , Lopez Michael , Chen Patrick , Lopez Alex 
Published: September 2024

Abstract

Objective
We created a forced choice recognition trial (FCR) for the HVLT-R, including recording completion time (FCompT), to evaluate the utility of combining completion time with accuracy scores when using performance validity tests (PVTs).

Method
Participants included 29 clinically referred patients at a large medical center who completed the HVLT-R in addition to multiple free-standing and embedded PVTs including the TOMM, Word Choice Test (WCT), Reliable Digit Span (RDS), and the Dot Counting Test (DCT) as part of a larger battery of neuropsychological tests in an outpatient setting. Data from the sample (M age = 64.79, SD = 13.86; M education = 15.65, SD = 2.47; 62.1% females) were analyzed using correlation and linear regression.

Results
The HVLT-R FCR trial correlated with the TOMM-10, r(19) = 0.75, p = &lt;0.001, TOMM trial 1, r(17) = 0.75, p = &lt;0.01, the DCT r(12) = −0.71, p = &lt;0.01, and the FCompT, r(29) = −0.72, p = &lt;0.001. The FCompT was correlated with RDS, r(28) = −0.46, p = 0.03, TOMM-10, r(19) = −0.64, p = &lt;0.01, and TOMM trial 1, r(17) = −0.61, p = &lt;0.01. FCompT significantly predicted FCR accuracy b = −0.72, t(29) = −5.41, p &lt; 0.001.

Conclusions
The HVLT-R FC and FCompT strongly correlated with free-standing and embedded PVTs. Data demonstrates that a longer completion time predicts poorer performance on FCR. Results demonstrate the value of measuring completion time for specific PVTs, in addition to providing support for the addition of a FCR trial to the HVLT-R.
B - 139 A Bayesian Approach to Performance Validity in Neuropsychological Testing
Authors: Serrano-Salcedo Juan A , Wang Yishi , Forde Jessica , Zambrana-Bonaparte Hjalmar M , Carlson Kate E , Puente Antonio E 
Published: September 2024

Abstract

Objective
Using the Test of Memory Malingering (TOMM), this study seeks to explore the feasibility of a continuous approach to performance validity by simulating experiments using a Bayesian statistical algorithm.

Methodology
A literature review was carried out to conduct statistical analyses from data of two known groups (those putting forth credible effort and those not). TOMM scores were modeled using various statistical distributions, with the inflated Conway-Maxwell-Poisson (ICMP) distribution providing the best fit. Simulation experiments were conducted to test the Bayesian statistics model. This involved generating artificial data from known distributions and then seeing how well the algorithm could recover the probability of a score belonging to the credible effort group. We tested the algorithm’s robustness by intentionally starting with incorrect assumptions and seeing if it could still converge on the correct probabilities. Finally, the Bayesian model was applied to real TOMM data to estimate the probability of credible effort for each possible score.

Results
The simulation studies showed that the Bayesian approach was generally robust, even when starting with incorrect assumptions. When applied to real data, TOMM scores of 40, 45, and 50 were essential, with approximately 5% change occurring with each score. A score of 40 covers approximately 50% of scores, 45 covers approximately 75%, and, finally, 50 covers 95% of scores. A perfect score of 50 encompassed ~95% of credible performances but not 100%.

Conclusion
This study provides initial statistically derived evidence for the feasibility and potential value of a continuous, probabilistic approach to performance validity.
B - 140 Comparison of Performance Validity, Symptom Validity, Cognitive, and Psychological Symptom Report Measures in an Active-Duty Military Sample
Authors: Armistead-Jehle Patrick , Ingram Paul , Keen Megan 
Published: September 2024

Abstract

Objective
Performance and symptom validity (PVT/SVT) are distinct but related constructs (Nelson et al., 2007; Ruocco et al., 2008). However, studies have rarely examined PVT and SVT via factor analytic techniques, which offer stronger understanding of concurrent performance patterns across tests. Ord et al (2021) conducted one such study and concluded that PVT/SVT each “contribute unique information about examinees’ performance.” This and previous studies, however, have not included cognitive or symptom measures along with SVTs/PVTs, which limits interpretation.

Methods
In a large sample of Active-Duty (ad) personnel (n = 629), EFA and CFA modeling evaluated underlying patterns between different forms of testing (PVT, SVT, symptom report, and cognitive report). Measures include the MSVT, NV-MSVT, RBANS, and PAI.

Results
EFA (Maximum Likelihood and Geomin rotation) evidenced shared method variance patterns across PVT, SVT, and both cognitive/symptom scales without good fit (X2 = 4643.0, RMSEA = 0.15, CFI = 0.58). When modeled with CFA, these domains provided good fit, X2(145) = 634.92, CFI = 0.94, RMSEA = 0.07, SRMR = 0.05. Standardized loadings were large for PVT (0.40–0.97), SVT (0.62–0.98), symptom report (0.43–0.91), and cognitive ability (0.48–0.87). Higher order factor structure suggests convergence of PVT and Cognitive Testing (0.70); however, SVT and were distinct from cognitively-focused testing (−0.43).

Conclusions
Validity testing in neuropsychology has accelerated (Sweet et al., 2021), and understanding relationships between validity and symptom domains is critical to this effort. Our study suggests differentiation between domains, with weaker relationships between validity and clinical symptoms.
B - 141 Research Review of Symptom and Performance Validity Tandem: Inventory of Problems-29 and Memory Module
Authors: Viglione Donald J , Giromini Luciano 
Published: September 2024

Abstract

Objective
Consistent with the NAN expectations, the American Academy of Clinical Neuropsychology (Sweet et al., 2021) asserted that neuropsychologists should include symptom validity tests (SVT) and performance validity tests (PVT) to assess the credibility of psychological and cognitive complaints. Aligned with this declaration, the Inventory of Problems–29 (IOP-29), an SVT, and its Memory Module (IOP-M), a PVT based on the respondent’s recall of the IOP-29 items. Our objective is to evaluate the research support of these measures to discriminate credible from noncredible presentations.

Data Selection
Since the IOP was introduced in 2017, more than 20 peer-reviewed research studies on the IOP-29 and 10 studies including both the IOP-29 and IOP-M have been published with 4 meta-analyses or systematic reviews. These publications present various, commonly used statistical analyses of validity and efficacy, e.g., sensitivity and specificity, along with comparisons to other SVTs and PVTs.

Data Synthesis
In the quantitative research literature, the IOP-29 and IOP-M have demonstrated efficacy across countries and languages in discriminating credible from noncredible presentations related to psychosis, PTSD, depression, anxiety, and neuropsychological disorders.

Conclusions
These findings support adding this tandem to the multi-method evaluation of the credibility of mental health and neuropsychological presentations. The brevity of administering these two tests, i.e., approximately 20 minutes total, suggests that the cost–benefit might be strong. More research with biomarkers and specific neuropsychological disorders might be needed along with more publications for which the authors do not include the test developers.
B - 143 A Comparison of Structured Inventory of Malingered Symptomology (SIMS) Performance to Multiple Validity Measures in a Criminal Forensic Sample
Authors: Hernandez Katherine A , Ikonomou Vasilios C , Campbell Ivan A , Wagaman Bailey N , Sullivan Sydney , Ton-Loy Adan , Kockler Timothy , Hill Benjamin D 
Published: September 2024

Abstract

Objective
We compared valid or invalid SIMS performance to other performance and symptom validity measures.

Methods
Data from 119 forensic neuropsychological evaluations [competency (n = 106), forensic (n = 5), insanity (n = 5), commitment review (n = 2), and danger (n = 1)] were analyzed. Three chi-squares tests compared group membership (valid/invalid) for SIMS total score, affective (AF) subscore, or low intelligence (LI) subscore to a separate valid or invalid group membership based on MMPI-2-RF (VRIN-r and TRIN-r), WAIS-IV Reliable Digit Span, and WMT (IR, DR, and CONS) validity cutoff scores (74% failed &gt;2 measures).

Results
Sample was predominately male (91.6%) and Caucasian (53.8%). Mean age was 35.1 years; mean education was 11.36 years. For SIMS total, 36% failed the SIMS but passed all other validity measures [χ 2(1) = 21.668; p &lt; 0.001; V = 0.427] for a 64% concordance rate for overall valid performance. Individuals with valid scores on the SIMS AF subscale were likely (66%) to have valid performance on all other validity measures [χ 2(1) = 12.418; p &lt; 0.001; V = 0.323]. Individuals with valid scores on the SIMS LI subscale were essentially at chance (52%) to have performed validly overall [χ 2(1) = 9.406; p = 0.002; V = 0.0.281].

Conclusions
In this forensic criminal sample, a third of individuals failed the SIMS total score and AF subscale while having credible performance otherwise. Approximately half failed the LI subscale while passing other validity measures. Results suggest that SIMS performance is not strongly related to SVT measures that focuses on response consistency or PVT performance; there is still significant classification overlap. However, high base rate of PVT/SVT failure in this sample may limit generalizability.