Assessing the Utility of MMPI-2-RF-EX in Detecting Simulated Underreporting of Current Suicide Risk in Military Veterans
Phase: NA
Conditions: Suicidal Ideation, Suicidal
Sponsor: Ohio State University

 
 
 
 
 
 
 
 
 
Assessing the Utility of MMPI-2-RF-EX in Detecting Simulated Underreporting of 
Current Suicide Risk in Military Veterans 
 
 
NCT: 2020B0414 
 
Study Protocol and Statistical Analysis Plan  
 
Approved 2/25/2021 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
Assessing the Utility of MMPI-2-RF-EX in Detecting Simulated Underreporting of 
Current Suicide Risk in Military Veterans 
 
1. Objectives 
The aims of the current project are: 1) to examine the incremental predictive utility of the 
MMPI-2-RF-EX validity scales (L and K) in detecting simulated underreporting of suicide risk on the 
SUI scale and extratest measures of suicide risk (Suicide Cognitions Sale, Beck Scale for Suicide 
Ideation – Current) and 2) compare the use and preliminary effectiveness of a self-administered 
version of the Crisis Response Plan (CRP) in decreasing suicidal/death ideation and distress and 
increasing positive affect when compared to a clinician-administered version of the protocol in a 
sample of 150 military Veterans experiencing current death or suicidal ideation. Hypotheses include: 
(H1) Veterans following instructions to underreport suicide risk (Simulated Underreporting) would 
endorse significantly higher MMPI-2-RF-EX L and K scale scores than those administered the MMPI-
2-RF-EX and extra test measures under standard conditions (Standard Instructions); (H2) Veterans 
randomized to the Simulated Underreporting group will produce significantly lower scores on MMPI-2-
RF-EX (SUI) and extra test self-report measures of suicide risk (Suicide Cognitions Scale, Beck Scale 
for Suicide Ideation – Current) than the Standard Instructions group; (H3) lastly, L and K would 
contribute incremental predictive validity over one another in differentiating between veterans in the 
Standard Instructions and Simulated Underreporting group. Exploratory analyses will be conducted to 
examine whether those randomized to Simulated Underreporting endorse lower mean scores on 
extratest measures that do not assess suicide risk.  
 
2. Background and Rationale 
 
Military veterans comprise 7.9% of the US population but account for 13.5% of adult deaths by 
suicide; additionally, the Veteran suicide rate is 1.5 greater than that of the general population (US 
Department of Veterans Affairs, 2019). A disproportionate number of suicides are attributed to 
Veterans who have not recently received healthcare within the Veterans Affairs (VA) Healthcare 
System. Indeed, the age- and sex-adjusted suicide rates among these Veterans increased by 11.8% 
between 2016 and 2017, compared to an increase of 1.3% among Veterans recently utilizing VA 
health care services, highlighting the increasing pace of suicide rates among Veterans not receiving 
health care within VA, a group which constitutes the majority of Veterans (US Department of Veterans 
Affairs, 2019). Given the heightened suicide rate among this group of Veterans, implementation of 
empirically-based suicide assessment procedures would be well-served in other Veteran-serving 
community settings. However, research and clinical efforts to identify Veterans at highest risk of 
suicide have been met with limited success (Nelson et al., 2017). Further complicating suicide risk 
assessment in the Veteran population is the tendency for high risk populations to not disclose current 
levels of suicide risk (Anestis & Green, 2015). Therefore, it is critical that empirically-derived 
methods of assessing suicide risk accurately identify Veterans outside the VA setting who are 
at risk of suicide, but do not accurately report their current experiences of suicidal ideation or 
intent.  However, current measures of suicide risk commonly administered in research and clinical 
settings are not well-suited to accomplish this task. The proposed study is well-suited to address 
these limitations and expand upon preliminary findings in Veteran samples through two of its 
aims: 1) examine whether simulated underreporting on the MMPI-2-RF-EX (indexed by K and L 
scores) impacts SUI scores and generalizes to underreporting on extratest self-report 
measures of suicide risk and 2) assess the incremental predictive utility of MMPI-2-RF-EX L 
and K scales in differentiating between these groups. 
 
In response to enhanced detection of suicide risk, there is a need to identify psychotherapeutic 
interventions targeting suicide risk among military veterans who may be unable or unwilling to seek 

follow-up mental health services. Indeed, reasons for refusal to seek mental health services among 
veterans include public and self-stigma (Held & Owens, 2013), adherence to masculine norms (Burns 
& Mahalik, 2011). However, Wilson et al. (2008) observed that the majority of Army soliders who were 
unwilling to speak to a clinician about mental health care have were willing to utilize technology-based 
approaches to mental health care. 
 
 
2.1.1 False negatives and the current state of suicide risk assessment. The issue of false 
negatives in suicide risk assessment has been well-documented. In the VA setting, half of suicides 
occur in the group classified by physicians as being at low risk of suicide, calling the utility of suicide 
risk screening instruments into question (Pokorny, 1983).  In accordance with Pokorny’s (1983) 
findings, Denneson et al. (2010) observed that 72% of Veterans who died by suicide denied 
experiencing suicide ideation during their last suicide risk assessment by a VA provider. These 
findings are also consistent with those observed when suicide risk assessment is more proximal to 
death by suicide:  Roughly 80% of suicide decedents whose death occurred in the hospital or shortly 
after discharge denied experiencing suicide ideation during their last suicide risk assessment (Bush, 
Fawcett, & Jacobs, 2003). The heightened rate of false negatives in suicide risk assessment are 
partly attributed to responders’ concealment of suicidal intent (Nielssen, Wallace, & Large, 2017).  
 Suicide risk assessment is one instance in which individuals are motivated to underreport or 
minimize distress in order to avoid hospitalization or decrease time hospitalized (Oquendo & 
Bernanke, 2017; Petrik, Gutierrez, Berlin, & Saunders, 2015). Additionally, stigma related to receiving 
metal health care for suicide ideation (Hom, Stanley, Podlogar, & Joiner, 2016), past negative 
experiences related to disclosing suicide ideation (Fulginiti, Pahwa, Frey, Rice, & Brekke, 2016), 
negative repercussions during military service (Ganzini et al., 2013), and ambivalence about these 
thoughts (Harris, McLean, Sheffield, & Jobes, 2010) are other reasons for underreporting or 
nondisclosure of suicide ideation. Additionally, underreporting has been observed in instances when 
individuals are forced to select item responses (Podlogar et al., 2016; Podlogar & Joiner, 2019), 
which is common in clinical settings that administer self-report measures of suicide risk. Despite the 
documented prevalence of false negatives in suicide risk assessment and the tendency of 
respondents to underreport on these measures for a variety of reasons, the majority of measures and 
interviews assessing suicide risk are not equipped to assess underreporting. Additionally, limitations 
regarding the administration and interpretation of commonly administered measures of suicide risk 
have been noted.  
While recent research has sought to identify better indicators of suicide risk using objective 
measures commonly administered in clinical practice (Levy et al., 2019), the design of commonly 
administered standardized interviews assessing suicide risk hinders the ability to accurately identify 
those at various levels of suicide risk (Tabares, Butner, Bryan, & Harris, In Press). Additionally, 
interview-based methods of suicide risk assessment rely on clinical judgment to determine an 
individual’s current level of suicide risk and the corresponding course of intervention; however, 
clinicians’ reliance on clinical judgment in suicide risk assessment results in heterogeneous 
determinations of suicide risk (Berman, Stark, Cooperman, Wilhelm, & Cohen, 2015). Further, suicide 
risk assessment methods that rely on clinical judgment do not add predictive validity above validated 
self-report measures of suicide risk (Lindh et al., 2020).  In addition to the respective limitations of 
these methods of suicide risk assessment, each relies on patient self-report, but is not designed to 
accurately identify those who fail to disclose current suicide ideation or plans, preparations, or intent 
to attempt suicide. This proposed research aims to address these limitations through 1) 
incorporating MMPI-2-RF-EX validity scales to detecting underreporting of suicide risk on the 
MMPI-2-RF-EX SUI scale and 2) assessing whether underreporting identified on the MMPI-2-
RF-EX generalizes to extratest measures of suicide risk.  
 
2.1.2 The MMPI in detecting underreporting and its extension to extratest measures 

A growing body of evidence indicates that underreporting on the MMPI instruments 
generalizes to conjointly administered self-report measures. Forbey and Lee (2011) observed that 
individuals who produced significantly elevated MMPI-2 K and L scale scores endorsed significantly 
lower mean scores on several conjointly administered self-report measures assessing various 
symptoms of psychopathology with roughly medium to large sized effects. Additionally, Forbey and 
colleagues (2013) observed that scores on internalizing self-report measures, namely the Beck 
Depression Inventory (Beck, Ward, Mendelson, Mock, & Erbaugh, 1961), were also significantly lower 
for the content-based underreporting group with medium to large effects. Indeed, individuals indicated 
as underreporting on the MMPI-2 and MMPI-2-RF respond in a similar manner to extratest measures 
of psychopathology in university, VA psychiatric inpatient, and correctional settings (Forbey & Lee, 
2011; Forbey et al., 2013).               
Underreporting may result in false negative findings, particularly in high-stakes assessments 
(Crighton, Marek, Dragon, & Ben-Porath, 2017). However, limited quantitative research has focused 
on identifying underreporting in suicide risk assessment, a limitation noted in the current research 
base (Burchett and Ben-Porath, 2019). Indeed, preliminary evidence indicates that underreporting on 
the MMPI-2 may have implications for suicide risk assessment. During clinic intake procedures, 
individuals who denied current suicide ideation/behavior on both a clinician-administered phone 
interview and the MMPI-2 produced higher K, S, and ODep scores than those who responded 
affirmatively to only MMPI-2 questions or both the interview and MMPI-2 items (Glassmire, Stolberg 
,Greene, & Bongrar, 2001). However, no research has extended this research to Veterans outside of 
the VA setting or examined whether the MMPI-2-RF-EX possesses similar clinical utility of previous 
iterations of the MMPI to accurately identify Veterans who underreport current suicide risk.  
Divergent findings regarding differences in SUI scores between those classified as 
underreporting and those producing scores within normal limits on content-based validity scale scores 
support the further clinical need for examining the impact of MMPI-indexed underreporting on suicide-
related criterion, including those embedded into the MMPI. While no significant mean differences in 
MMPI-2-RF SUI scores between these two groups have been observed (Crighton et al., 2017), 
including in Veteran samples (Forbey et al., 2013; Khazem et al., In Press), significant differences 
with small-sized effects have also been observed in other samples (Brown & Sellbom, 2020; Khazem 
et al., In Preparation). Further, as these studies were not aimed at identifying patterns of responding 
for individuals attempting to conceal suicidal ideation or intent, it is unclear how these individuals, 
particularly within the Veteran population, would respond to SUI items - and extratest measures of 
suicide risk and psychopathology. Research employing a simulated groups design in Veteran-serving 
clinics unaffiliated with VA is well-suited to address these limitations and expand upon previous 
findings.  
MMPI-based research involving simulation designs have indicated the need for including 
indicators of underreporting when administering extratest self-report measures. Such research 
designs allow examination of the construct validity of the MMPI across response-style groups while 
maintaining experimental control over study conditions (Burchett & Ben-Porath, 2019; Dhillon, Bagby, 
Kushner, & Burchett, 2016). Sellbom and Bagby (2008) examined the impact of simulated 
underreporting on MMPI-2-RF substantive scales in undergraduate students and individuals with 
schizophrenia and observed that K-r and L-r added incrementally to each other in predicting 
underreporting in both groups, with greater effects in those diagnosed with schizophrenia. These 
findings were replicated in undergraduate students (Creighton et al., 2017). Other research utilizing 
simulation designs have noted differences in extratest measures of trait negative affectivity (Dhillon, 
Bagby, Kushner, & Burchett, 2016). Despite the need for research assessing whether underreporting 
negatively impacts the sensitivity of suicide risk assessment through simulation research designs 
(Burchett & Ben-Porath, 2019; Rogers, 2018), such research has not been conducted. This research 
is particularly timely for populations at heightened risk of suicide - including Veterans. The 
proposed study addresses this absence in the research base by employing a simulation 
design centered on explicit underreporting of suicide risk to identify how Veterans respond to 
SUI and other MMPI-2-RF-EX scales when intending to conceal suicidal ideation or intent. 

Additionally, this study design affords the additional advantage of comparing SUI scores and 
scores of extratest measures of suicide risk between those attempting to underreport suicide 
risk and those testing under standard instructions.  
 
 
2.3 Anticipated Results and Potential Pitfalls It is anticipated that this research will advance the 
detection and treatment of suicide risk among military veterans who seek clinical services outside the 
Veterans Affairs setting. More specifically, based on the aforementioned findings regarding the MMPI, 
we anticipate that the MMPI-2-RF-EX will demonstrate clinical utility in detecting simulated 
underreporting of suicide risk among individuals with current distress or suicidal ideation. 
Participants could develop mild to moderate emotional discomfort or frustration associated with 
completing questionnaires or answering interview questions assessing thoughts about suicide. This 
potential risk is expected to be comparable to the discomfort experienced when talking with a friend 
or acquaintance about these same topics. If discomfort is experienced, it is not expected to be severe 
or to last for more than a few minutes. Participants’ confidentiality could be breached if their identifiers 
are inadvertently released or accessed by a third party. This risk is expected to be low because the 
data are not stored or analyzed in ways that are likely to reveal a subject’s identity. Breach of 
confidentiality could also occur if an external party or individual hacks into the Zoom interface during a 
participant’s session. Safeguards will be implemented to mitigate this risk, including using password-
protected Zoom sessions. 
 
2.3. Benefits of the Proposed Research  Results of this proposed research will have 
implications for future suicide risk assessment and suicide prevention efforts in Veterans through 
enhancing methods of identifying and responding to those at risk of suicide that would otherwise be 
missed by traditional methods of suicide risk assessment (i.e. false negatives).  
 
Procedures 
 
3.1. Research Design. A two-arm randomized controlled simulation design will be employed 
in this proposed research. For both components of the study, participants will be randomized using 
stratified blocks of 6 or 8 participants, per recommendations by Krenan et al. (1999), based on gender 
and suicide attempt history (no history of suicide attempts, history of one suicide attempt, or history of 
multiple suicide attempts).  
 
3.2. Sample. A sample of 150 Veterans will be included in the study and randomized to study 
conditions (75 in the Simulation condition and 75 in the Standard condition). Results of a priori power 
analyses conducted using G*Power software (Faul et al., 2007) indicate that a total sample of 150 will 
results in a highly-powered study (𝛽 =.98-.99) to detect medium-sized effects in all regression-based 
analyses. Therefore, although we anticipate low attrition from this study, the proposed sample size is 
sufficiently powered for the proposed analyses. Additionally, this sample size accounts for a roughly 
15% rate of Non-Content-Based Invalid Responding observed in other simulation-based MMPI 
research (Crighton et al., 2017).  
Online recruitment methods for the study are include wed-based survey and research 
recruitment systems (e.g., SurveyMonkey), listservs and forums, and social media sites (e.g., 
Facebook, Twitter, Reddit). Additionally, we are requesting use of ResearchMatch (RM), which  is an 
NIH sponsored national registry for research volunteers. ResearchMatch (RM) and OSU have had a 
fully executed agreement since August 2009. The IRB of record for ResearchMatch is Vanderbilt 
University. The Principal Investigator will be authorized to recruit study participants.  
 
We will enroll any prospective Veteran meeting inclusion criteria (18 years of age or older, US 
military veteran, death or suicidal ideation/suicide attempt within the past month) in the proposed 
study. Exclusion criteria include: non-Veteran status, acute intoxication or active psychosis precluding 

provision of informed consent, an inability to communicate and comprehend English, residence 
outside the United States, and lack of past-month death/suicide ideation or attempt. Participants 
recruited through web-based survey and research and recruitment systems will provide consent for 
study personnel to initiate contact through email or phone call, while those recruited through other 
online methods (e.g., listservs, forums) will be provided contact information for study personnel in 
order to inquire about the study or schedule participation.  
 
3.3 Measurement/Instrumentation. The following self-report measures will be administered 
through Qualtrics online survey software during baseline procedures as part of the proposed 
research: 
MMPI-2-RF-EX. The MMPI-2-RF-EX The MMPI-2-RF-EX was developed for research 
purposes in order to develop the MMPI-3 (Ben-Porath & Tellegen, 2020), which is comprised of a 
subset of MMPI-2-RF-EX items. While completing the MMPI-2-RF-EX. all participants will be 
monitored remotely by study personnel using a web camera and OSU CarmenZoom to ensure test 
integrity, as required by the study funding agency. While competing the MMPI-2-RF-EX, participants 
will simultaneously select the MMPI-2-RF-EX item numbers corresponding to those they believe are 
assessing current suicide risk through a separate internet browser or tab opened to a link in Qualtrics 
provided by study personnel.  
  Suicide Cognitions Scale (SCS; Bryan et al. 2014). The SCS is an 18-item survey 
measuring suicide-related thoughts and beliefs, unclouding hopelessness, perceived 
burdensomeness, entrapment, and unbearability. Respondents rate their agreement with each 
statement on a 5-point Likert scale. Item responses are summed, with higher scores indicating more 
severe suicide risk.  
Beck Scale for Suicide Ideation-Current (BSS; Beck & Steer, 1991). The BSS is a 21-item 
measure of past-week suicide ideation and desire for death. Responses to items are scored on a 3-
point Likert scale (range of 0- 2), with higher scores indicating greater suicide risk.  
Short Grit Scale (Duckworth & Quinn, 2009). The Short Grit Scale is being administered to 
examine whether those randomized to the Simulated Underreporting condition endorse lower mean 
scores when compared to those randomized to the Standard Instructions condition (e.g., whether 
simulated underreporting of suicide risk extends to extratest measures that do not explicitly measure 
suicide risk). 
PTSD Checklist for DSM-5 (PCL-5; Weathers et al., 2013). The PCL-5 is a self-report 
measure assessing the severity of 20 PTSD symptoms. Respondents rate the severity of each 
symptom during the past week on a 5-point Likert Scale ranging from 0 (Not at all) to 4 (Extremely). 
Similar to the Short Grit Scale, the PCL-5 is being administered to examine whether Veterans in the 
Simulated Underreporting groups endorse lower mean scores than those in randomized to the 
Standard instructions group.  
 
 Posttest Inquiry. All participants will be administered the MMPI-2-RF-EX Posttest Inquiry 
(PTI) that has been administered in previous MMPI simulation studies (Creighton et al., 2017), based 
on recommendations by Rogers (2008). The PTI assesses participants’ compliance with condition-
specific instructions for completing the MMPI and was adapted for the proposed research. Responses 
to three questions from this survey (“What instructions did you receive for completing the research 
study”; “I tried to answer the questions so I would look like I am not suicidal?”; “I tried to appear as 
suicidal as I could when I took the test”) will be examined to assess compliance with instructions. 
Simulators answering “To fake not having thoughts of suicide” to the first question will be classified as 
Simulation- Compliant. Alternatively, Simulators answering at least “most of the time” to the second 
item and “not at all” to the third item will be classified as Simulation-Noncompliant and excluded from 
analyses. Veterans in the Standard Instructions condition who comply with instructions (e.g., do not 
produce profiles indicative of content-based invalid responding) are classified as Standard - 
Compliant while those who do not will be classified as Standard - Noncompliant and excluded from 
analyses.  
3.4. Informed Consent and Confirmation of Eligibility 

Informed consent will be obtained through Qualtrics survey software from each participant prior 
to initiation of study tasks. Study personnel will obtain informed consent from participants and answer 
questions about the study or informed consent process prior to beginning study activities.  Following 
informed consent, participants will complete a demographics questionnaire through Qualtrics that also 
includes items assessing history of suicide attempts from the Self-Injurious Thoughts and behaviors 
Interview (Nock, Holmberg, Photos, & Michel, 2007) for study eligibility determination and 
randomization purposes as well as the Suicide Visual Analog Scale (Bryan, 2019). Regardless of 
eligibility, all participants will be provided with contact information for crisis resources including the 
National Suicide Prevention Lifeline and Veterans Crisis Line.  
3.5. Randomization to Study Conditions.  
REDCap, a secure online survey and data management system maintained by The Ohio State 
University, will be utilized to randomize participants to study conditions immediately after obtaining 
demographic information (man, woman, other) and information about suicide attempt history (no 
suicide attempts, one suicide attempt, more than one suicide attempt) obtained from the SITBI). 
Participants will be randomized using stratified blocks of 6 or 8 participants, per recommendations by 
Krenan et al. (1999). After randomization, each participant will complete the remainder of the 
research study, but will be presented with one of the following pages containing instructions for 
completing the MMPI-2-RF-EX. 
MMPI Condition 1 (Standard Instructions): “We want you to imagine that you are 
undergoing psychological testing as part of standard intake procedures for receiving services in a 
mental health clinic, which involves completing some questionnaires and a personality test before the 
first session. Please answer the questionnaires as honestly as you can, as your mental health care 
provider will use the results of your testing to develop your treatment plan. Please note, the 
questionnaire has some “lie” scales built into it that detect false responding. Additionally, as part of 
the intake process, you agreed to identify questions that you think are assessing your current risk of 
suicide. You will open the link to this portion of the study that was sent to you by study personnel to 
complete this.” 
MMPI Condition 2 (Simulated Underreporting): “We want you to imagine that you are 
undergoing testing as part of standard intake procedures for receiving services in a mental health 
clinic, which involves completing some questionnaires and a personality test before the first session. 
You have recently had recurring thoughts of suicide and have thought of ways to attempt suicide. 
However, you are concerned that you will be hospitalized if you disclose this during testing and do not 
want to be hospitalized. Please answer the questionnaires as though you are trying to hide or 
downplay any thoughts of suicide or intent to make a suicide attempt. The questionnaire has some 
“lie” scales built into it, so try to avoid being detected. If you are able to avoid detection, you will 
receive an additional $20 gift card. Additionally, as part of the intake process, you agreed to identify 
questions that you think are assessing your current risk of suicide. You will use a computer to mark 
these while taking the questionnaires.” 
 
3.6. Additional incentives following completion of the first portion of the study.  
Following the study session, all participants will receive payment in the form of $50 for 
participation. Additional incentives are condition-dependent, will be described to participants at the 
beginning of the study as part of the informed consent process, and administered no later than two 
weeks following completion of the study protocol: 
 
Simulated Underreporting: Additional payment for undetected underreporting on MMPI-
2-RF-EX. Individuals in the Simulated Underreporting condition who do not produce clinically 
significant MMPI-2-RF-EX K or L scale elevations and are classified as Simulation Compliant based 
on Posttest Inquiry responses will receive an additional electronic $20 gift card through email, 
following recommendations and procedures outlined in previous research (Brown & Sellbom, 2020; 
Crighton et al., 2017).  
3.4. Detailed Study Procedures. One hundred and fifty adult military Veteran participants 
residing in the United States will complete eight study components during the baseline study session: 

consenting, demographics survey and self-reported history of suicide attempts, randomization to 
study conditions, the MMPI-2-RF-EX, self-report measures, and the post-test survey to assess 
compliance with study instructions. Consenting procedures will always be completed first, and 
demographic and suicide attempt history data will always be collected second for randomization to 
conditions in both components of the study through REDCap online software. We estimate that the 
total participation time will be approximately 2.5 hours (2 hours to complete the MMPI-2-RF-EX and 
extra-test self-report measures. In order to protect participant confidentiality, password-protected 
Zoom or phone-based sessions will be conducted during the baseline session, in order to ensure 
security of MMPI-2-RF-EX items. All data will be collected through the Qualtrics, and non-PHI 
demographic information will be entered into REDCap system for randomization purposes. ID 
numbers will be linked to all data collected in the study, including identifying information collected in a 
separate data set as part of safety procedures (See Section 3.7), which will be deleted at the 
conclusion of the study. All deidentified data will be downloaded and deleted from Qualtrics servers at 
the conclusion of the study and stored on the password protected OneDrive server hosted by OSU 
indefinitely for the proposed analyses and secondary analysis. Additionally, in order to protect 
participant privacy during the study, all study personnel will conduct study procedures from a private 
location, using a secure Internet connection and instruct all participants to do the same, if possible, 
and outline the risks of loss of privacy or confidentiality if the participant is unable to so. 
3.7. Internal Validity.  A number of measures will be implemented to mitigate threats to internal 
and external validity. Specifically, the randomization procedures employed in the proposed research 
study (See Section 4.1.2) were selected to account for the potential influences of gender and suicide 
attempt status on study findings.  
 
3.8 Data Analysis. 
 
 
3.8.1.MMPI-2-RF-EX scale scores and extra test measures.  Following methods 
employed in prior simulation research designs (Sellbom & Bagby, 2008), mean differences in scores 
on MMPI-2-RF-EX L, K, and SUI scales, and extratest measures of suicide risk (BSS), suicide-related 
cognition intensity (SCS), and measures of other psychopathology (PCL-5, Short Grit Scale) will be 
compared between Standard and Simulated Underreporting groups through a one-way Multivariate 
Analysis of Variance (MANOVA). Additionally, two stepwise hierarchical regression analyses will be 
conducted in order to assess the incremental predictive utility of MMPI-2-RF-EX K and L scales in 
identifying individuals classified in the Simulated Underreporting condition, consistent with previous 
literature (Sellbom & Bagby, 2008). The first model will contain the K scale in the first step and the L 
scale in the second step. The analysis will be repeated with the K and L scales entered into the 
opposite step. 
3.9. Safety Procedures.  During enrollment in the study, all participants, regardless of 
agreement to participate in both portions of the study, will provide their contact information (home 
address, telephone number, email address) and a telephone number for an alternate person that 
study personnel may contact in case of imminent suicide risk. Home addresses may be used to 
contact the participant’s local emergency services in case of imminent suicide risk. If a subject reports 
suicidal ideation, or a research staff member becomes aware that the subject is at imminent risk to 
harm himself/herself, the following questions will be asked to clarify the nature of risk (and to identify 
those at imminent risk requiring consideration for hospitalization): (1) Do you have a plan for killing 
yourself and do you intend to act on the plan?; (2) Do you have a desire to kill yourself that you 
believe you might act on?; (3) Have you already taken steps to act on your plan? If so, what steps 
have you taken? Both iterations of the CRP, which is a recommended and empirically-supported 
strategy for reducing suicide risk, will include contact information for professional resources and crisis 
services, the National Suicide Prevention Lifeline phone number, and other local behavioral health 
clinics and emergency departments. These procedures have been used successfully by our research 
team in dozens of studies with acutely suicidal individuals, to include multiple clinical trials. All 
participants will be provided with contact information to use in case of crisis (National Suicide 
Prevention Lifeline, emergency services). 

 
3. Bibliography 
 
Anestis, M. D., & Green, B. A. (2015). The impact of varying levels of confidentiality on disclosure of 
suicidal thoughts in a sample of United States National Guard personnel. Journal of Clinical 
Psychology, 71 , 1023-1030.   
Arroll, B., Goodyear-Smith, F., Crengle, S., Gunn, J., Kerse, N., Fishman, T., ... & Hatcher, S. (2010).  
 
Validation of PHQ-2 and PHQ-9 to screen for major depression in the primary care population. 
The Annals of Family Medicine, 8(4), 348-353. 
Beck, A. T., & Steer, R. A. (1991). Manual for the Beck scale for suicide ideation. San Antonio, TX:   
 
Psychological Corporation.  
Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring 
depression. Archives of General Psychiatry, 12, 57-62. 
Ben-Porath YS, Tellegen A. The Minnesota Multiphasic Personality Inventory-3: technical manual. 
Minneapolis: University of Minnesota Press; 2020. 
Berman, N. C., Stark, A., Cooperman, A., Wilhelm, S., & Cohen, I. G. (2015). Effect of patient and 
therapist factors on suicide risk assessment. Death studies, 39(7), 433-441. 
Brown, T. A., & Sellbom, M. (2020). The Utility of the MMPI–2–RF Validity Scales in Detecting 
Underreporting. Journal of personality assessment, 102(1), 66-74. 
Bryan CJ. (2019). A preliminary validation study of two ultra-brief measures of suicide risk: the 
Suicide and Perceived Burdensomeness Visual Analog Scales Suicide and Life-Threatening 
Behavior.  
Bryan, C. J., Mintz, J., Clemans, T. A., Leeson, B., Burch, T. S., Williams, S. R., ... & Rudd, M. D. 
(2017). Effect of crisis response planning vs. contracts for safety on suicide risk in US Army 
soldiers: a randomized clinical trial. Journal of affective disorders, 212, 64-72. 
Bryan, C. J., Mintz, J., Clemans, T. A., Burch, T. S., Leeson, B., Williams, S., & Rudd, M. D. (2018). 
Effect of crisis response planning on patient mood and clinician decision making: A clinical trial 
with suicidal US soldiers. Psychiatric Services, 69(1), 108-111. 
Bryan, C. J., & Rudd, M. D. (2018). Brief cognitive-behavioral therapy for suicide prevention. Guilford 
Publications. 
Bryan, C. J., Rudd, M. D., Wertenberger, E., Etienne, N., Ray-Sannerud, B. N., Morrow, C. E., ... & 
Young-McCaughon, S. (2014). Improving the detection and prediction of suicidal behavior 
among military personnel by measuring suicidal beliefs: An evaluation of the Suicide 
Cognitions Scale. Journal of affective disorders, 159, 15-22. 
Burchett, D., & Ben-Porath, Y. S. (2019). Methodological considerations for developing and 
evaluating response bias indicators. Psychological assessment, 31(12), 1497-1511. 
Burns, S. M., & Mahalik, J. R. (2011). Suicide and dominant masculinity norms among current and 
former United States military servicemen. Professional Psychology: Research and 
Practice, 42(5), 347. 
Busch, K. A., Fawcett, J., & Jacobs, D. G. (2003). Clinical correlates of inpatient suicide. The Journal 
of clinical psychiatry, 64, 14-19.  
Crighton, A. H., Marek, R. J., Dragon, W. R., & Ben-Porath, Y. S. (2017). Utility of the MMPI-2-RF 
validity scales in detection of simulated underreporting: Implications of incorporating a 
manipulation check. Assessment, 24(7), 853-864. 
Denneson, L. M., Basham, C., Dickinson, K. C., Crutchfield, M. C., Millet, L., Shen, X., & Dobscha, S. 
K. (2010). Suicide risk assessment and content of VA health care contacts before suicide 
completion by veterans in Oregon. Psychiatric Services, 61(12), 1192-1197. 
Department of Veterans Affairs (2019, September). 2019 National Veteran Suicide Prevention Annual 
Report. https://www.mentalhealth.va.gov/docs/data-
sheets/2019/2019_National_Veteran_Suicide_Prevention_Annual_Report_508.pdfb 

Dhillon, S., Bagby, R. M., Kushner, S. C., & Burchett, D. (2017). The impact of underreporting and 
overreporting on the validity of the Personality Inventory for DSM–5 (PID-5): A simulation 
analog design investigation. Psychological assessment, 29(4), 473. 
Duckworth, A. L., & Quinn, P. D. (2009). Development and validation of the Short Grit Scale (GRIT–
S). Journal of personality assessment, 91(2), 166-174. 
Forbey, J. D., & Lee, T. T. (2011). An exploration of the impact of invalid MMPI–2 protocols on 
collateral self-report measure scores. Journal of Personality Assessment, 93(6), 556-565.  
Forbey, J. D., Lee, T. T., Ben-Porath, Y. S., Arbisi, P. A., & Gartland, D. (2013). Associations between 
MMPI-2-RF validity scale scores and extratest measures of personality and psychopathology. 
Assessment, 20(4), 448-461. 
Fulginiti, A., Pahwa, R., Frey, L. M., Rice, E., & Brekke, J. S. (2016). What factors influence the 
decision to share suicidal thoughts? A multilevel social network analysis of disclo- sure among 
individuals with serious mental illness. Suicide and Life-Threatening Behavior, 46, 398-412.  
tGanzini, L., Denneson, L. M., Press, N., Bair, M. J., Helmer, D. A., Poat, J., & Dobscha, S. K. (2013). 
Trust is the basis for effective suicide risk screening and assessment in veterans. Journal of 
General Internal Medicine, 28(9), 1215-1221. 
Glassmire, D. M., Stolberg, R. A., Greene, R. L., & Bongar, B. (2001). The utility of MMPI-2 suicide 
items for assessing suicidal potential: Development of a suicidal potential scale. Assessment, 
8(3), 281-290. 
Held, P., & Owens, G. P. (2013). Stigmas and attitudes toward seeking mental health treatment in a 
sample of veterans and active duty service members. Traumatology, 19(2), 136-143. 
Hom, M. A., Stanley, I. H., Podlogar, M. C., & Joiner, T. E. (2017). Are you having thoughts of 
suicide? Examining experiences with disclosing and denying suicidal ideation. Journal of 
Clinical Psychology, 73, 1382-1392.  
Kernan, W. N., Viscoli, C. M., Makuch, R. W., Brass, L. M., & Horwitz, R. I. (1999). Stratified 
randomization for clinical trials. Journal of clinical epidemiology, 52, 19-26. 
Khazem, L.R., Anestis, J.C., Erbes, C.R., Ferrier-Auerbach, A.G. Schumacher, M.M., &  
Arbisi, P.A. (In Press). Further assessment of the clinical utility of MMPI-2-RF in detecting 
suicidal ideation in a partially-hospitalized veteran sample. Journal of Personality Assessment 
Levy AG, Scherer AM, Zikmund-Fisher BJ, Larkin K, Barnes GD, Fagerlin A. Assessment of Patient 
Nondisclosures to Clinicians of Experiencing Imminent Threats. JAMA Netw Open. Published 
online August 14, 20192(8):e199277. 
Lindh, Å. U., Beckman, K., Carlborg, A., Waern, M., Renberg, E. S., Dahlin, M., & Runeson, B. 
(2020). Predicting suicide: A comparison between clinical suicide risk assessment and the 
Suicide Intent Scale. Journal of Affective Disorders, 263, 445-449. 
Mackinnon, A., Jorm, A. F., Christensen, H., Korten, A. E., Jacomb, P. A., & Rodgers, B. (1999). A 
short form of the Positive and Negative Affect Schedule: Evaluation of factorial validity and 
invariance across demographic variables in a community sample. Personality and Individual 
differences, 27(3), 405-416. 
Nelson, H. D., Denneson, L. M., Low, A. R., Bauer, B. W., O’Neil, M., Kansagara, D., & Teo, A. R. 
(2017). Suicide risk assessment and prevention: a systematic review focusing on veterans. 
Psychiatric Services, 68, 1003-1015.  
Nielssen, O., Wallace, D., & Large, M. (2017). Pokorny's complaint: the insoluble problem of the   
overwhelming number of false positives generated by suicide risk assessment. BJPsych 
bulletin, 41(1), 18-20. 
Nock, M. K., Holmberg, E. B., Photos, V. I., & Michel, B. D. (2007). Self-Injurious Thoughts and  
Behaviors Interview: development, reliability, and validity in an adolescent sample. 
Psychological Assessment, 19(3), 309-317. 
Oquendo, M. A., & Bernanke, J. A. (2017). Suicide risk assessment: tools and challenges. World 
Psychiatry, 16(1), 28. 

Petrik, M. L., Gutierrez, P. M., Berlin, J. S., & Saunders, S. M. (2015). Barriers and facilitators of 
suicide risk assessment in emergency departments: a qualitative study of provider 
perspectives. General hospital psychiatry, 37(6), 581-586. 
Pokorny, A. D. (1983). Prediction of suicide in psychiatric patients: report of a prospective study.  
 
 
Archives of general psychiatry, 40(3), 249-257. 
Rogers, R. (2008). Detection strategies for malingering and defensiveness. In R. Rogers (Ed.), 
Clinical assessment of malingering and deception (3rd ed., pp. 14-35). New York: Guilford 
Press. 
Rogers, R. (2018). Researching response styles. In R. Rogers & S. D. Bender (Eds.), Clinical 
assessment of malingering and deception (4th ed.). New York, NY: Guilford Press. 
Rozek, D. C., & Bryan, C. J. (2020). Integrating crisis response planning for suicide prevention into 
trauma‐focused treatments: A military case example. Journal of clinical psychology, 76(5), 
852-864. 
Sellbom, M., & Bagby, R. M. (2008). Validity of the MMPI-2-RF (restructured form) Lr and Kr scales in 
detecting underreporting in clinical and nonclinical samples. Psychological Assessment, 20(4), 
370. 
Stuck, A. R., Wilson, M. P., Chalmers, C. E., Lucas, J., Sarkin, A., Choi, K., & Center, K. (2017).  
 
Health care usage and suicide risk screening within 1 year of suicide death. The Journal of  
 
 
emergency medicine, 53(6), 871-879. 
Tabares, J. V., Butner, J. E., Bryan, C. J., Harris, J. A. (In press). Mokken scale analysis of lifetime 
responses on the Columbia Suicide Severity Rating Scale’s Severity of Ideation Subscale. 
Assessment.  
Tellegen, A., & Ben-Porath, Y. (2008). Minnesota Multiphasic Personality Inventory-2-Restructured 
 
 
Form: Technical Manual. Minneapolis, MN: University of Minnesota Press. 
Wilson, J. A., Onorati, K., Mishkind, M., Reger, M. A., & Gahm, G. A. (2008). Soldier attitudes about 
technology-based approaches to mental health care. CyberPsychology & Behavior, 11(6), 767-
769.