Validating the Military Service Identification Tool in Its Ability to Correctly Identify Civilians and Those Who Have Served in the Military
Conditions: Mental Health Disorder
Sponsor: King's College London

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 1 of 16 
Military Service Identification Tool 
Validating the Military Service Identification Tool in its ability to correctly identify civilians and those 
who have served in the military. 
Sponsors 
Institute of Psychiatry, Psychology & Neuroscience  
King's College London  
16 De Crespigny Park  
London  
SE5 8AF  
+44 (0) 20 7848 0002  
Chief Investigator  
Dr Sharon Stevelink 
King's College London, Department of Psychological Medicine  
Institute of Psychiatry, Psychology and Neuroscience  
Weston Education Centre,  
10 Cutcombe road,  
London SE5 9RJ  
0207 848 5817 
Sharon.stevelink@kcl.ac.uk 
Co-Investigators 
Professor Nicola T. Fear 
King's College London, Department of Psychological Medicine  
Institute of Psychiatry, Psychology and Neuroscience  
Weston Education Centre,  
10 Cutcombe road,  
London SE5 9RJ  
0207 848 5265 
Nicola.t.fear@kcl.ac.uk 
 
Dr Dominic Murphy 
Tyrwhitt House,  
Oaklawn Road,  
Leatherhead,  
Surrey,  
KT22 0BX 
Dominic.murphy@combatstress.org 
 
 
 
 
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 2 of 16 
CONTENTS 
Study Synopsis 
3 
Introduction 
5 
Study Objectives and Design 
7 
Sample Size, Statistics, Selection and Withdrawal of Subjects 
8 
Study procedures 
9 
Assessment of Safety 
11 
Study oversight arrangements 
12 
Ethics & Regulatory Approvals 
13 
Data Handling 
14 
Finance and Publication Policy 
15 
 
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 3 of 16 
Study Synopsis 
 
Full Title  
 
Validating the Military Service Identification Tool in its 
ability to correctly identify civilians and those who have 
served in the military. 
Short Title/Acronym
 
Military Service Identification Tool 
Protocol Version number and Date
 
Version 1 (02/03/2020) 
Study Duration  
12 months 
Study Design
 
Questionnaire, interview or observation study 
Sponsor/Co-sponsors  
King’s College London 
Chief Investigator
 
Dr Sharon Stevelink 
IRAS number
 
278987 
Primary objective
 
The principal research question is to ascertain whether 
individuals identified by the Military Service Identification 
Tool have or have not served in the military (Armed 
Forces).
Secondary objective (s)
 
N/A 
Number of Subject
 
Expected sample n=1168. This represents an estimation 
that 20% of our cohort (n=5844) will have given consent 
for contact.  
Main Inclusion Criteria
 
Only those participants who formed part of our original 
study will be screened for eligibility. This corresponds to 
2,922 probable veterans and 2,922 civilians. The following 
inclusion criteria will apply: 
1. Participant are listed as ‘Alive’ in the NIHR Maudsley 
Biomedical Research Centre Clinical Record Interactive 
Search database; 
2. Participants aged 18 years and older; 
3. Participant has given Consent for Contact; 
4. Participant has a valid postal address or telephone 
number; 
5. Participants have capacity to consent.  
Statistical Methodology and Analysis
 
Aggregation of data to estimate accuracy of the Military 
Service Identification Tool 
 
 
 
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 4 of 16 
Glossary of Terms and Abbreviations 
 
BRC 
 
Biomedical Research Centre 
EHR   
Electronic Healthcare Record 
KCMHR 
King’s Centre for Military Health Research 
MSIT  
Military Service Identification Tool 
SLaM  
South London and Maudsley 
UK 
 
United Kingdom 
 
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 5 of 16 
1. Introduction 
Summary of the study: 
 
Estimates of the UK’s military veteran population, defined by the British Government as those who have served in 
the military for at least one day [1], is approximately 2.5 million, equivalent to around 5% of household residents aged 
16 years or over in the UK [2]. UK military veterans receive healthcare provision from the National Health Service 
(NHS), with care recorded in local, regional and national EHRs [3]. EHRs – structured and unstructured (i.e. free text) 
– can be used to evaluate disease prevalence, surveillance, to perform epidemiological analyses and investigate quality 
of care and to improve clinical decision-making [4], [5].  
 
There is no national marker in UK EHRs to identify veterans, nor is there a requirement for healthcare professionals 
to record it, making it difficult to evaluate the unique healthcare needs of those who have served in the UK Armed 
Forces [6]. This study, funded by Forces in Mind Trust, seeks to validate the Military Service Identification Tool, an 
open-source computer program that searches through free-text clinical notes to make a prediction on a person’s 
military status. It is in the public interest to know the health of our Armed Forces. The Tool has been validated using 
manually annotated datasets, but we now need to valid an individual’s military status by contacting them via post or 
telephone and asking, “Have you ever served in the Armed Forces”. The research team will work closely with the 
CRIS Patient Advisor Group and local healthcare professionals.   
 
The current study: 
 
Currently, there is no military service marker in electronic healthcare records to identify who is a veteran in England 
and Wales [3]. This makes it difficult to evaluate the unique healthcare needs of those who have served in the UK 
military. In a previous study, we set out to investigate whether it was feasible to identify veterans who accessed 
secondary mental healthcare services via their anonymized electronic healthcare records using a manual, text search 
based, method. To do this, we used the SLaM BRC Case Register. This is a novel data resource, derived directly from 
the routine electronic healthcare records of the SLaM NHS Foundation Trust. SLaM is one of Europe’s largest mental 
health providers, serving over 1.2 million residents in four South London boroughs. This Case Register holds patient’s 
electronic healthcare records for all secondary mental healthcare provisions within SLaM. All these electronic 
healthcare records have been anonymized and a system has been developed to enable researchers to access and search 
through these records called the clinical record interactive system (CRIS).  
 
As part of the first study we developed a manual, text search based, method using commonly used terms and phrases 
found in the free text clinical notes to identify veterans. Running this manual method resulted in the identification of 
n = 6,039 potential veteran records, of which n = 1,600 were selected to scrutinize in more detail, whilst considering 
time and manpower restrictions. This resulted in n = 693 veteran records, suggesting an identification rate of 43% 
when using the manual method. Therefore, we concluded that it was feasible to identify veterans in electronic 
healthcare records, but time consuming. Each potential veteran record was manually verified by the research team by 
reading through each patient’s notes. This took on average 11 minutes per record.  
 
To address the time it took to identify veterans and improve sensitivity and reach we developed the MSIT, and it 
proved to be quicker, more accurate and cheaper than the manual method. The Tool was designed to detect military 
service, not just veteran status. We took a systematic approach to developing and testing the MSIT. We used different 
subsets of all the electronic healthcare records available to us, to ensure the MSIT was developed and trained on a 
different set of electronic healthcare records (named the training dataset). Subsequently the MSIT was verified on 
another subset of the data (named the gold standard dataset) to ensure it would still be able to identify veterans if a 
different set of electronic healthcare records would be used.  
 
We managed to identify 2,922 veterans in the electronic healthcare records when applying MSIT to inspect 150,000 
individual records. The MSIT took only 20 minutes to go through all these records and had an identification rate of 
88% when compared to the manual approach. We also matched these veterans to 2,922 non-veterans based on age and 
gender to compare their mental health treatment pathways.  
 
Despite the success of our initial developments we have faced a significant barrier that may prevent the widespread, 
dissemination, roll-out and implementation of the MSIT. Various academics and stakeholders (e.g. NHS England, 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 6 of 16 
Cobseo) are interested in the MSIT. However, they have highlighted an important limitation, namely it is still unclear 
whether the veterans identified by the MSIT are ‘actual’ veterans. MSIT identifies veterans based on the notes 
provided by the clinician in the electronic healthcare record. It is also important to understand if higher rates of physical 
and mental health are represented in a veteran population compared to civilians. By including both samples – civilian 
and veteran – we’re ensuring that fair comparisons are able to be made. This There is currently no opportunity to 
verify their veteran status. For example, no service number is provided in their medical records.  
 
Therefore, to overcome these limitations and to demonstrate that the MSIT is correctly identifying military service 
(serving personnel, veterans or civilian) it is important to reach out to those who have been identified to ask about 
their previous Service. 
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 7 of 16 
2 Study Objectives and Design 
2.1. Study Objectives 
The principal research aim is to ascertain whether individuals identified by the MSIT have or have not served in the 
military (Armed Forces). This study will ask up to 5 questions to eligible participants, with the responses used to 
validate the MSIT algorithm predictive ability.  
 
Primary Outcome: 
The primary outcome of interest will be the military serving status of participants. We will use the information 
provided to cross-check the performance of the MSIT. Where incorrect classifications have been performed, 
refinement of the algorithm will take place to improve the accuracy. The questions that will be asked have been 
designed to elicit as descriptive information as possible to help us refine the MSIT in the event it has made an incorrect 
prediction. 
2.2 Study Design & Flowchart 
The study will employ a questionnaire approach by asking a set of (up to 5) questions on previous military service.  
These questions are as follows: 
 
Question 1: Have you ever served in the Armed Forces? 
 
Yes 
 
No 
 
Question 2: Which part of the Armed Forces did you serve in? 
 
Royal Navy 
 
Royal Marines 
 
Army 
 
Royal Air Force 
 
Question 3: What was the highest rank when you left the military? 
 
Senior Commissioned Officer (Cdr/Lt Col/Wg Cdr and above) 
 
Commissioned Officer (to Lt Cdr/Maj/Sqn Ldr) 
 
Senior Non-Commissioned Officer 
 
Junior Non-Commissioned Officer 
 
Other ranks (AB/Pte/AC/JT or equivalent) 
 
Other (please specify) 
 
Question 4: How long did you serve in the Armed Forces? 
 
Question 5: When you left the Armed Forces, were you: 
 
Regular 
 
Reserve 
 
Both 
 
If the participant responds NO to Question 1, the questionnaire will end. If they answer YES, they will be asked 
questions relayed to military service. The participant can skip any question. These questions will be asked only once. 
No follow-up will be performed. Participants who take part in the study will be asked to supply an email address at 
the end of the questionnaire to enter a prize draw. This is voluntary. If they decide to take part, they will be entered a 
prize draw. At the end of the study, we will randomly select the winners. The prizes are: 1 £50 Amazon Gift voucher, 
5 £25 Amazon Gift voucher and 20 £10 Amazon Gift voucher. 
 
The study is using a cross-section, single point in time, questionnaire approach. There is no control comparison group, 
or randomised group allocation.  

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 8 of 16 
3.  Sample Size, Statistics, Selection and Withdrawal of Subjects 
Sample Size:  
No formal power calculation was performed as the development of the MSIT was part of a feasibility study. In total, 
150,000 patient medical records were examined by MSIT, which resulted in 2,922 probable veterans being identified. 
These were matched on age and gender to a probable civilian cohort of 2,922. Hence, a total sample of 5,844. Due to 
resource limitations we did not apply the Tool to the full database of the SLaM NHS Foundation Trust. It is expected 
that 20% of the sample will have Consent for Contact. This estimate is based on previous studies.  
 
Participant Eligibility: 
Only those participants who formed part of our original study will be screened for eligibility. This corresponds to 
2,922 probable veterans and 2,922 civilians. 
 
The following inclusion criteria will apply: 
 
1. Participant are listed as ‘Alive’ in the SLaM BRC Clinical Record Interactive Search database; 
2. Participants will be aged 18 years and old; 
3. Participant has given Consent for Contact; 
4. Participant has a valid postal address or telephone number; 
5. Participants have capacity to consent.  
 
The following exclusion criteria will apply: 
 
Participants will be excluded if a clinician is required to be notified of the research prior to contact, as this may indicate 
at risk participants. Further participants who are not fluent in spoken English will be unable to take part in the study 
due to inability to understand the participant information and consent form. Therefore, non-English speaking 
participants will not be included in this study. 
 
Withdrawal: 
 
Participants are able to withdraw at any point throughout the study up until data analysis. To ensure the participants 
do not feel unduly burdened by the requirements of the study at any time, they will be reminded of the voluntary 
nature of their participant, and that they do not need to answer the question and can withdraw from the research at any 
time. Once a participant has withdrawn from the study, no further contact will be made. Data from withdrawn 
participants will not be used in the analyses.  
 
Any protocol deviations or violations will be reported to Dr Sharon Stevelink (Chief Investigator) who will act 
accordingly. If the protocol deviation/violation relates to a participant, they will be withdrawn from the study and 
appropriate support given (ie. contact by the study clinical team).  If the deviation/violation relates to a member of the 
research team, the NHS Research Ethics Committee and Research and Development will be informed.  
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 9 of 16 
4.   Study procedures 
Participants will be asked to complete a single questionnaire via a bespoke study website or take part in a short 
telephone interview. 
 
Informed Consent Procedures:  
 
Once eligible participants have been identified a Post-Doctoral Research Associate (to be appointed) will, in the first 
instance, send an email to the participant informing them of the study and asking for consent to the main research 
question (question 1). Informed content will be obtained prior to any questionnaires or procedures being employed. 
This will be performed either via the study website or via telephone. Time and date of consent will be recorded 
alongside a unique study identifier. Participants will be informed that participant is voluntary, that non-participant or 
withdrawal from the study will not affect their medical care in anyway. Prior to consent being sought, participants 
will be given time to read and consider the Participant Information Sheet.  
 
Questionnaire Approach: 
 
Where an email address is not present, a letter will be sent instead, where an email an email address and postal address 
are not present, the Post-Doctoral Research Associate (to be appointed) will make contact via telephone. Through all 
forms of communication, participants will be directed to record their response via the study website using a proxy_id, 
which is unique to the participant. If participants do not respond after 2 weeks a maximum of 3 attempts to contact 
the participant using contact information will be made. If we are not able to make contact, they will be removed from 
the study. 
 
No personal information will be retained on the website, and participants will be required to provide two unique sets 
of ‘identifiers’ to log-in. If a participant fails to complete the online questionnaire or would prefer to undertake the 
study via telephone, a research assistant will make contact. The Post-Doctoral Research Associate (to be appointed) 
will log participant responses via the study website.    
 
Risk and Burdens: 
 
We do not anticipate their being any consequences for taking part in this research. However, there is a potential for 
participant to have negative perceptions towards the concept of military service due to negative experiences in some 
way. To manage and mitigate these risks, we will adopt the following process: 
 
1) The Post-Doctoral Research Associate (to be appointed) employed on the study will have relevant research 
and clinical experience on how to deal with distressed participants; 
2) The Post-Doctoral Research Associate (to be appointed) will be trained to encourage the participant to seek 
help if they are experiencing problems; 
3) A formal risk protocol will be implemented to ensure that participants who are deemed at risk are 
appropriately managed;  
4) For those who are seriously distressed, or deemed at risk, advice will be sought from and as appropriate a 
call back will be offered by Dr Dominic Murphy, Clinical Psychologist at Combat Stress. 
 
In terms of participant burden, it is estimated that to take part in the study will take no more than 15 minutes. If during 
the telephone discussion the participant become distressed, the Post-Doctoral Research Associate (to be appointed) 
will halt the discussion and re-arranged to an alternative time or day. Additionally, to ensure the participants do not 
feel unduly burdened by the requirements of the study at any time, they will be reminded of the voluntary nature of 
their participant, and that they do not need to answer the question and can withdraw from the research at any time. 
 
Screening Procedures: 
 
Except applying the eligibility criteria described previously, participants will not undergo any screening.  
Treatments and Follow-up: 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 10 of 16 
This study does not seek to test any intervention or treatment. We will not perform any follow-up, nor require the 
participant to provide any further information above and beyond the questionnaire.  
End of Study Definition: 
 
Once all participants have been contacted using the stated procedure, or after 12 months, whichever comes first, the 
study will we be defined as complete. Participants who have provided an email address will be randomised and winners 
selected. They will be contacted within 3 months of the study completing.  
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 11 of 16 
5. Assessment of Safety  
It is possible, but unlikely due to the nature of interaction, that some participants may perceive the interview as 
distressing or upsetting as we are asking about their military service. It will be stressed to participants during the 
telephone discussion, email or via post that they do not need to answer the question and are able to decline to partake 
in the study. The Post-Doctoral Research Associate (to be appointed) contacting the participants will be trained to 
signpost wherever needed and will be able to act as a listener. A robust risk protocol has been developed in 
collaboration with SLaM and BRC and Dr Dominic Murphy (Clinical Psychologist at Combat Stress) to ensure that 
the Post-Doctoral Research Associate (to be appointed) understands how to respond to a situation.  This risk protocol 
includes key contact information for a range of charitable organisations. It is anticipated that a participants GP 
manages any identified risk but where that is not possible for any reason, then participants who are seriously distressed 
or deemed at risk then advice will be sought from and as appropriate a clinical call back offered to participants by Dr 
Dominic Murphy, Clinical Psychologist at Combat Stress. 
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 12 of 16 
6. Study oversight arrangements  
The study proposal has undergone extensive review and evaluation within the research team. Study conduct and 
progress will be monitored by a steering committee representing the research team. 
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 13 of 16 
7. Ethics & Regulatory Approvals 
This application will be reviewed by the NHS Research Ethics Committee and has received approval from the 
Research and Development office. 
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 14 of 16 
8. Data Handling 
Data collected for this study will be encrypted and stored on a secure server maintained by the sponsor organisation 
and will not identifiable by participant name. The bespoke website for this study will be developed prior to data 
collection and will use servers located in London, England. The information stored on this system will be: 
 
1. Proxy ID: A unique identifier will be created to allow users to ‘log-in’ and complete the questionnaire. 
The mapping key to link responses and contact details will be held by the SLaM BRC Research 
Administrator.  
2. Questionnaire responses: Participant questionnaire responses will be recorded in the system and will be 
saved with the ‘proxy id’ identifier. 
3. Prize draw: Participants are able to provide an email address to enter the prize draw. This will be saved 
in a separate database. At no time will the ‘proxy id’ and email address be linked.  
 
Upon completion of data collection, all data will be downloaded from the server and the server be decommissioned. 
At any time, up until data analysis, participants can withdraw from the study.  
 
The research team will keep legible and accurate documents to ensure thorough documentation of study conduct. The 
highest degree of confidentiality will be maintained for managing data collected throughout the course of the study. 
However, to meet legal responsibilities we will permit authorised representatives of the health authorities to examine 
records, to satisfy quality assurance reviews, audits and evaluations of study safety and progress.  
 
For data analysis, the non-identifiable data acquired will be downloaded and stored on a secure network managed by 
SLaM BRC. Only the immediate research team will have access to data collected with the Chief Investigator Dr 
Sharon Stevelink being the custodian of the data. Data will be used to assess the performance of MSIT, and further 
refine the Tool to improve classification rates. 
 
This study will be compliant with the principles of the Declaration of Helsinki, compliant with the Data Protection 
Act (2018) and follow local and national regulations.  
 
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 15 of 16 
9. Finance and Publication Policy 
This study is being undertaken by KCMHR based at King’s College London and is funded by the Forces in Mind 
Trust. Results will be disseminated via social media, KCMHR mailing list and scientific journals. Participants will 
not be identified in any report or publication.  
 
 

IRAS Project: 278987 – version 1 (02/03/2020) 
 
 
 
Page 16 of 16 
References 
[1] 
“Veterans: Key facts,” 2016. [Online]. Available: 
https://www.armedforcescovenant.gov.uk/wp-content/uploads/2016/02/Veterans-Key-
Facts.pdf. [Accessed: 12-Mar-2019]. 
[2] 
“Population Projections: UK Armed Forces Veterans residing in Great Britain, 2016 to 
2028,” London, UK, 2019. 
[3] 
D. Leightley et al., “Integrating electronic healthcare records of armed forces personnel: 
Developing a framework for evaluating health outcomes in England, Scotland and 
Wales,” Int. J. Med. Inform., vol. 113, pp. 17–25, May 2018. 
[4] 
R. A. Payne, G. A. Abel, B. Guthrie, and S. W. Mercer, “The effect of physical 
multimorbidity, mental health conditions and socioeconomic deprivation on unplanned 
admissions to hospital: a retrospective cohort study,” Can. Med. Assoc. J., vol. 185, no. 5, 
pp. E221–E228, Mar. 2013. 
[5] 
S. J. Simmonds et al., “Understanding NHS hospital admissions in England: linkage of 
Hospital Episode Statistics to the Hertfordshire Cohort Study,” Age Ageing, vol. 43, no. 5, 
pp. 653–660, Sep. 2014. 
[6] 
V. A. Morgan and A. V. Jablensky, “From inventory to benchmark: quality of psychiatric 
case registers in research,” Br. J. Psychiatry, vol. 197, no. 01, pp. 8–10, Jul. 2010.