Can a Tailored Quality Assurance Program Help General Practitioners Maintain POCUS Scanning Competence? A Cohort Study From Denmark
Conditions: Point-of-care Ultrasonography (POCUS)
Sponsor: Aalborg University

 
 
 
 
  
 
Study protocol 
Can a tailored quality assurance program 
help general practitioners maintain POCUS 
scanning competence? 
A cohort study from Denmark 
Center for General Practice at Aalborg University 
15-11-2022 
 

Maintaining POCUS scanning competence  
1
Study protocol according to the SPIRIT 2013 Checklist 
Administrative information ............................................................................................................. 3 
1 
Title ...................................................................................................................................... 3 
2a 
Trial registration ............................................................................................................... 3 
2b 
 The World Health Organization Trial Registration Data Set .......................................... 3 
3 
Protocol version ................................................................................................................... 5 
4 
Funding ................................................................................................................................ 5 
5 
Roles and responsibilities .................................................................................................... 5 
5a  
Names, affiliations, and roles of protocol contributors .......................................................... 5 
5b 
Name and contact information for the trial sponsor .............................................................. 5 
5c 
Role of study sponsor and funders .............................................................................................. 5 
5d 
Composition, roles, and responsibilities of the steering committee and safety committee)6 
Introduction ..................................................................................................................................... 6 
6 
Background and rationale .................................................................................................... 6 
7 
Specific objectives ............................................................................................................... 6 
8 
Trial design .......................................................................................................................... 8 
Methods - Participants, interventions, and outcomes ..................................................................... 8 
9  Study settings ....................................................................................................................... 8 
10 
Participants ....................................................................................................................... 8 
11a 
Description of the intervention ......................................................................................... 9 
11b 
Criteria for discontinuing or modifying allocated interventions ...................................... 9 
11c 
Strategies to improve adherence to intervention protocols ............................................ 10 
11d 
Relevant concomitant care and interventions that are permitted or prohibited .............. 10 
12 
Outcome measures ......................................................................................................... 10 
13 
Time schedule ................................................................................................................ 12 
14 
Sample size .................................................................................................................... 12 
15 
Recruitment .................................................................................................................... 12 
Assignment of interventions ......................................................................................................... 12 
16a  Allocation Sequence generation ..................................................................................... 12 
16b 
Allocation concealment mechanism ............................................................................... 12 
16c 
Implementation............................................................................................................... 12 
17 
Blinding .......................................................................................................................... 12 
Data collection, management, and analysis .................................................................................. 13 

Maintaining POCUS scanning competence  
2
18 
Data collection - Plans for assessment and collection of outcome, baseline, and other 
trial data .................................................................................................................................... 13 
Education of POCUS expert assessors .................................................................................. 14 
19 
Data management ........................................................................................................... 15 
20a 
Statistical methods for analysing primary and secondary outcomes.............................. 15 
20b Sub-analyses . ........................................................................................................................ 15 
20c 
Missing data ................................................................................................................... 15 
Data monitoring ............................................................................................................................ 15 
21a 
Data monitoring committee (DMC). .............................................................................. 15 
21b 
Description of any interim analyses and stopping guidelines, including who will have 
access to these interim results and make the final decision to terminate the trial .................... 16 
22 
Harms ............................................................................................................................. 16 
23 
Auditing .......................................................................................................................... 16 
Ethics and dissemination............................................................................................................... 16 
24 
Research ethics approval ................................................................................................ 16 
25 
Protocol amendments ..................................................................................................... 16 
26a 
Consent or assent ............................................................................................................ 16 
26b 
Additional consent provisions for collection and use of participant data ...................... 16 
27 
Confidentiality ................................................................................................................ 16 
28 
Declaration of interests................................................................................................... 17 
29 
Access to data ................................................................................................................. 17 
30 
Ancillary and post-trial care ........................................................................................... 17 
31a 
Dissemination policy ...................................................................................................... 17 
31b 
Authorship eligibility guidelines .................................................................................... 17 
31c 
Plans for granting public access to the full protocol, ..................................................... 17 
32 
Informed consent materials ............................................................................................ 17 
References ..................................................................................................................................... 17 
 
 
 

Maintaining POCUS scanning competence  
3
Administrative information 
 
1 
Title 
Can a tailored quality assurance program help general practitioners maintain POCUS scanning 
competence? A cohort study from Denmark 
 
2a 
Trial registration 
The trial will be registered on clinicaltrials.org 
 
2b 
 The World Health Organization Trial Registration Data Set 
 
Data category 
Information 
Primary registry and trial 
identifying number 
Clinicaltrials.gov number: Nor provided yet 
Date of registration in 
primary registry 
Uploaded to clinical trials  
Secondary identifying 
numbers 
Red-CAP registration number ID-242-5. Center for General Practice at Aalborg 
University (CAM AAU) 
Source(s) of monetary or 
material support 
n/a 
Primary sponsor 
Center for General Practice at Aalborg University (CAM AAU) 
Secondary sponsor(s) 
Novo Nordisk Foundation 
The General Practice Foundation in Denmark 
Contact for public queries 
Søren Kæseler Andersen  skandersen@dcm.aau.dk 
Contact for scientific queries 
Camilla Aakjær Andersen caakjaer@dcm.aau.dk 
Public title 
General practitioners’ ability to maintain POCUS competence  
Scientific title 
Can a tailored quality assurance program help general practitioners maintain 
POCUS scanning competence? A cohort study from Denmark 
Countries of recruitment 
Denmark 
Health condition(s) or 
problem(s) studied 
The use of point-of-care ultrasonography (POCUS) in general practice 
Intervention(s) 
A quality assurance program with an online multiple-choice quiz including still-
pictures and video sequences of 10 application specific POCUS examinations. 
Through 15-20 questions, the quiz tests participants (1) knowledge about 
indications for performing POCUS examinations, (2) applied knowledge of 
ultrasound equipment, (3) ability to optimize images, (4) ability to recognize and 
present structures, (5) ability to interpret images, (6) ability to describe and 
document findings and (7) medical decision-making. Following the quiz, the 
participants are offered guidance to improve their performance.   
Key inclusion and exclusion 
criteria 
Inclusion criteria:  
1. GP, i.e. postgraduate medical doctor with a specialization in general practice. 
2.Work in office-based general practice in Denmark 
3.Access to an ultrasound device in the practice during the study period  
Exclusion criteria:  
1. GPs with a possible conflict of interest  
2. No signed informed consent to participate. 
Study type 
A cohort study 
Date of first enrolment 
Expected: 15-11-2022 
Target sample size 
Fifteen general practitioners  
Recruitment status 
Recruiting finished 14-11-2022 

Maintaining POCUS scanning competence  
4
Primary outcome(s) 
Primary outcome 1 (P1): For GPs, participating in a POCUS quality assurance 
program, the ultrasound competence score (summarized OSAUS score across ten 
scanning modalities), 12 months after baseline.    
 
Primary outcome 2 (P2): The proportion of GPs, participating in a POCUS 
quality assurance program, with an ultrasound competence score (summarized 
OSAUS score across ten scanning modalities) at 12 months larger than or equal to 
the ultrasound competence score at baseline.   
  
Key secondary outcomes 
Secondary outcome 1 (S1): For GPs, participating in a POCUS quality assurance 
program, the OSAUS scores and the item scores, 12 months after baseline, for 
each of the ten scanning modalities included in the curriculum. 
 
Secondary outcome 2 (S2): The proportion of GPs, participating in a POCUS 
quality assurance program, who have an ultrasound competence score (OSAUS 
score) of three or above, 12 months after baseline, for all of the seven OSAUS 
items of each of the ten scanning modalities included in the curriculum.  
 
Secondary outcome 3 (S3): The proportion of GPs, participating in a POCUS 
quality assurance program, who rate themselves to be competent to perform un-
supervised POCUS in general practice, 12 months after baseline, for each of the 
ten scanning modalities included in the curriculum. 
 
Secondary outcome 4 (S4): The number of performed POCUS examinations by 
each GP each month during the study (months 1-12 after baseline). 
 
Secondary outcome 5 (S5): The proportion of GPs, participating in a POCUS 
quality assurance program, who complete the online quiz at months 3, 6, 9 and 12 
after baseline.  
 
Secondary outcome 6 (S6):  For GPs, participating in a POCUS quality assurance 
program, the test scores in the online quiz at months 3, 6, 9 and 12 after baseline.  
 
Secondary outcome 7 (S7): For GPs, participating in a POCUS quality assurance 
program, the association between OSAUS score at 12 months and total number of 
performed POCUS examinations for each of the ten scanning modalities included 
in the curriculum. 
 
Secondary outcome 8 (S8): For GPs, participating in a POCUS quality assurance 
program, the association between OSAUS score at 12 months and the number of 
completes quizzes in the quality assurance program.  
 
Secondary outcome 9 (S9): For GPs, participating in a POCUS quality assurance 
program, the association between OSAUS score at 12 months and the test score in 
the quizzes in the quality assurance program.  
 
Secondary outcome 10 (S10): For GPs, participating in a POCUS quality 
assurance program, the association between OSAUS score at 12 months and the 
OSAUS score at baseline.  
 
Secondary outcome 11 (S11): For GPs, participating in a POCUS quality 
assurance program, their evaluation of the program after 12 months.. 
  
 

Maintaining POCUS scanning competence  
5
3 
Protocol version 
Version 5 November 14th 2022 
 
4 
Funding  
This study will be conducted as independent research at Center for General Practice at Aalborg 
University and is financially supported by The Novo Nordisk Foundation (grant number 
0061821) and The General Practice Foundation in Denmark (grant number A3495). 
 
5 
Roles and responsibilities 
 
5a  
Names, affiliations, and roles of protocol contributors 
Søren Kæseler Andersen (SKA) will be the principal investigator.  
SKA, Camilla Aakjær Andersen (CAA) and Martin Bach Jensen (MBJ) will oversee the choice 
of and collection of outcome measures. 
CAA, SKA and MBJ wrote the first draft of the protocol. 
Søren Lundby Christensen (SLC) will assist in statistical guidance. Allan Riis (AR) will draft the 
statistical analysis plan.  
CAA will handle participant correspondence. AR will handle the data cleansing and do the data 
analysis.   
Ole Graumann (OG), MBJ, Louise Pihl (LP), Christian Sjernebjerg (CS) and SKA will perform 
the OSAUS evaluations of scanning competence at 12 months.  
CAA, MBJ, SLC, OG, LP, CS and SKA are all expected to make valuable scientific additions to 
the draft and will be co-authors on subsequent manuscripts based on these data. 
The definition of author is defined on ICMJE´s four criteria1:  
 
Substantial contributions to the conception or design of the work; or the acquisition, analysis, or 
interpretation of data for the work; AND 
Drafting the work or revising it critically for important intellectual content; AND 
Final approval of the version to be published; AND 
Agreement to be accountable for all aspects of the work in ensuring that questions related to the 
accuracy or integrity of any part of the work are appropriately investigated and resolved. 
   
The expected author list is: Andersen SK, Jensen MB, Riis A, Christensen SL, Pil L, Stjernebjerg 
C, Graumann O, Andersen CA. 
 
5b 
Name and contact information for the trial sponsor 
Trial sponsor: Center for General Practice at Aalborg University (CAM AAU) 
Contact name: Camilla Aakjær Andersen 
Address: Center for General Practice in Aalborg, Fyrkildevej 7, 1., 9220 Aalborg East, Denmark  
E-mail: caakjaer@dcm.aau.dk 
 
5c 
Role of study sponsor and funders 
Sponsor (CAM AAU) is part of the study design, data analyses and writing of the manuscript. 
Sponsor will ensure that the results will be submitted for publication. Sponsor is non-commercial 
and declares no conflict of interest. 
 

Maintaining POCUS scanning competence  
6
Other sponsors have no part in the study design, data analyses or writing of the manuscript. 
 
The authors declare no conflict of interest. 
 
5d 
Composition, roles, and responsibilities of the steering committee and safety 
committee (see Item 21a for data monitoring committee) 
 
A steering committee affiliated with CAM AAU (TL, UM, TMJ, SKA, MBJ and CAA) will 
handle and oversee the development of the quality assurance tool.   
 
A safety committee will be set up to handle all reports of adverse and suspected adverse events 
(AEs) and serious adverse events (SAEs). MBJ will head the committee.  
 
Introduction 
6 
Background and rationale 
Point-of-care ultrasound (POCUS) examinations are increasingly used in general practice2 3. 
General practitioners (GPs) can learn to perform ultrasound examinations by attending 
ultrasound courses, but after course attendance they are left to maintain and develop scanning 
competence by themselves4. Maintaining scanning competence in general practice is challenged 
by (1) a low frequency of performed examinations as not all patients seen in general practice 
present conditions suited for POCUS5 6, (2) a low pre-test probability of disease in general 
practice7 which challenges the ability to recognise different pathologies, (3) the lack of 
supervision as GPs often work alone in small offices8, and (4) the short consultations9 leaving 
little room for practice.    
 
An ultrasound course tailored for Danish office-based GPs has been developed offering 
continuous practice under supervision during the three months that the GPs are enrolled in the 
course. Previous research10 have shown that this ultrasound course leads to scanning competence 
within the ten scanning modalities that are included in the course curriculum. The course is 
designed to make the participants move towards self-directed learning11, but we do not know if 
scanning competence obtained on the course can be maintained over time.       
 
Within some medical specialities, the number of specific POCUS examination performed have 
been used as an indicator for scanning competence in re-certification12. However, little is known 
about the number of performed POCUS examinations needed to secure skills. In general 
practice, continuous medical education (CME) programs have been found to increase GPs 
knowledge and skills in other areas13, but as POCUS is just a smaller part of the clinical work in 
general practice, CME programs for GPs cannot be too time-consuming.   
 
7 
Specific objectives  
The overarching aim of this study is to investigate whether a quality assurance program, tailored 
for GPs who use POCUS as part of their patient examination in general practice, can lead to 
maintained scanning competence 12 months after baseline.   
 

Maintaining POCUS scanning competence  
7
Primary outcome 1 (P1): For GPs, participating in a POCUS quality assurance program, the 
ultrasound competence score (summarized OSAUS score across ten scanning modalities), 12 
months after baseline.    
 
Primary outcome 2 (P2): The proportion of GPs, participating in a POCUS quality assurance 
program, with an ultrasound competence score (summarized OSAUS score across ten scanning 
modalities) at 12 months larger than or equal to the ultrasound competence score at baseline.   
 
Secondary outcome 1 (S1): For GPs, participating in a POCUS quality assurance program, the 
OSAUS scores and the item scores, 12 months after baseline, for each of the ten scanning 
modalities included in the curriculum. 
 
Secondary outcome 2 (S2): The proportion of GPs, participating in a POCUS quality assurance 
program, who have an ultrasound competence score (OSAUS score) of three or above, 12 
months after baseline, for all of the seven OSAUS items of each of the ten scanning modalities 
included in the curriculum.  
 
Secondary outcome 3 (S3): The proportion of GPs, participating in a POCUS quality assurance 
program, who rate themselves to be competent to perform un-supervised POCUS in general 
practice, 12 months after baseline, for each of the ten scanning modalities included in the 
curriculum. 
 
Secondary outcome 4 (S4): The number of performed POCUS examinations by each GP each 
month during the study (months 1-12 after baseline). 
 
Secondary outcome 5 (S5): The proportion of GPs, participating in a POCUS quality assurance 
program, who complete the online quiz at months 3, 6, 9 and 12 after baseline.  
 
Secondary outcome 6 (S6):  For GPs, participating in a POCUS quality assurance program, the 
test scores in the online quiz at months 3, 6, 9 and 12 after baseline.  
 
Secondary outcome 7 (S7): For GPs, participating in a POCUS quality assurance program, the 
association between OSAUS score at 12 months and total number of performed POCUS 
examinations for each of the ten scanning modalities included in the curriculum. 
 
Secondary outcome 8 (S8): For GPs, participating in a POCUS quality assurance program, the 
association between OSAUS score at 12 months and the number of completes quizzes in the 
quality assurance program.  
 
Secondary outcome 9 (S9): For GPs, participating in a POCUS quality assurance program, the 
association between OSAUS score at 12 months and the test score in the quizzes in the quality 
assurance program.  
 
Secondary outcome 10 (S10): For GPs, participating in a POCUS quality assurance program, the 
association between OSAUS score at 12 months and the OSAUS score at baseline.  
 

Maintaining POCUS scanning competence  
8
Secondary outcome 11 (S11): For GPs, participating in a POCUS quality assurance program, 
their evaluation of the program after 12 months.  
 
 
8 
Trial design  
This is a cohort study designed as a follow-up for a hybrid effectiveness-implementation study ( 
ClinicalTrials.gov Identifier: NCT05274581). In the previous study, 18 GPs attended an 
ultrasound course targeted to office-based GPs. The ultrasound course consisted of three 
teaching seminars over three months (Marts – June 2022), a curriculum of 10 POCUS 
applications, an online learning platform providing educational support before, during and after 
the teaching sessions (see Figure 1). At the end of the ultrasound course (June 2022) and again 3 
months after the ultrasound course (September 2022), all participants had their scanning 
competences within the 10 POCUS applications assessed by external experts. These experts were 
blinded to the participants previous experience and learning process. The assessment of scanning 
competence was done using the OSAUS assessment tool14. Following this study participants 
have implemented POCUS in their clinical practice, where they use POCUS as part of the patient 
examination.  
 
Before data collection starts, this trial will be registered on clinicaltrials.gov.  
Methods - Participants, interventions, and outcomes 
 
9  
Study settings  
This study will be conducted in office-based general practice in Denmark. GPs in Denmark are 
self-employed and work in office-based general practice clinics. Denmark has a public health 
care system where almost all patients are listed with a GP for primary health care. Consultations 
and treatments are free-of-charge for patients. GPs act as gatekeepers for other primary care 
healthcare providers and secondary care specialists. GPs are paid through a combination of 
remuneration and fee-for-service financed through taxes8. There is no fee for performing POCUS 
in primary care and GPs must cover expenses for the ultrasound device and their ultrasound 
education themselves. However, during this study participating GPs will have a fee for 
performing POCUS examinations. POCUS scanning-fees will be registered in the medical record 
system in the practice.  
 
The study will be coordinated from CAM AAU and data will be collected by the research team 
and by the participating general practitioners in their clinics. All study data will be stored at a 
secure server at Aalborg University. 
 
 10 
Participants   
Eighteen GPs working in office-based general practice in Denmark, who have attended the PLO-
e ultrasound course10, will be invited to participate in the study. To participate a GPs must fulfill 
the inclusion criteria and not the exclusion criteria 
 
Inclusion criteria:  
1. GP, i.e., be a postgraduate medical doctor with a specialization in general practice. 

Maintaining POCUS scanning competence  
9
2. Work in office-based general practice in Denmark 
3. Have access to an ultrasound device in the practice during the study period 
 
Exclusion criteria:  
1. GPs with a possible conflict of interest (e.g., industry affiliation related to the use of 
ultrasound)  
2. No signed informed consent to participate. 
 
The quality assurance program will be developed and delivered by research team, who are 
experienced GPs, POCUS users and/or teachers. The competence assessment (OSAUS score) 
will be delivered by a team of POCUS experts (SKA, OG, MBJ, CS and LP), who have 
extensive POCUS experience.   
 
11a 
Description of the intervention  
The research team has developed a Quality Assurance program as a CME-model for POCUS-
using GPs. The program consists of an online quiz using a multiple choice questionnaire (MCQ) 
design. The participants are expected to complete the online MCQ at 3, 6, 9, and 12 months 
following baseline. The MCQ includes still-pictures and video sequences of the 10 application 
specific POCUS examinations from the course curriculum. Through 15-20 questions, the quiz 
tests participants (1) knowledge about indications for performing POCUS examinations, (2) 
applied knowledge of ultrasound equipment, (3) ability to optimize images, (4) ability to 
recognize and present structures, (5) ability to interpret images, (6) ability to describe and 
document findings and (7) medical decision-making. Each question has four possible answers of 
which up to three answers are correct. Each MCQ will have ten questions pertaining to 
pathologies within the ten applications of the curriculum. Further, there will be five to ten 
questions relating to image recognition, image optimisation, ultrasound artifacts. In each MCQ 
one or two of the questions will test the GP’s knowledge in less frequent findings. We will 
predefine each included question in the MCQ according to: (1) scanning modality, (2) OSAUS 
dimension.  
 
Once each MCQ is finalized, the participating GP’s will receive feed-back on the answers with 
suggestions for improvement and guidance toward continuing self-directed learning. 
The MCQ part of the intervention is meant to function as a system of continuing medical 
education (CME). 
 
11b 
Criteria for discontinuing or modifying allocated interventions for a given trial 
participant  
The trial may be discontinued for a GP in case of withdrawal of an informed consent. Hence, 
data collection will stop for that person at the time of withdrawal of the informed consent. 
Otherwise, this is an intention-to-treat study hence participants will not be excluded because of 
low adherence to the educational elements. However, we will collect data of the degree of 
participation for each participant.  
 

Maintaining POCUS scanning competence  
10 
11c 
Strategies to improve adherence to intervention protocols, and any procedures for 
monitoring adherence  
CAA will monitor participants’ scanning activity and responses to the quizzes and reach out to 
participants, who fail be active or fail to complete the quizzes.  
 
11d 
Relevant concomitant care and interventions that are permitted or prohibited 
during the trial 
The GPs are asked to declare any additional POCUS course activity or education during the 
study. 
 
12 
Outcome measures  
Background characteristics of participating GPs 
At baseline the following participant characteristics will be collected: Age (years), gender (M, F, 
other), previous use of ultrasound (number of months with regular use), previous ultrasound 
courses of minimum 1 day duration (yes/no), scanner type (low range, mid range, high end), type 
of practice (collaboration, partnership, solo), location of practice (urban, rural, mixed), number 
of patients assigned to the practice, and number of GPs working in the practice. 
 
In addition, the results from the OSAUS assessment of scanning competences conducted on all 
participants in September or October 2022 will be included.  
The OSAUS assessment tool has been developed and validated as a generic tool for assessing 
scanning competence14. The OSAUS scale consists of seven items: ‘indication for the 
examination’, ‘applied knowledge of ultrasound equipment’, ‘image optimization’, ‘systematic 
examination’, ‘interpretation of images’, ‘documentation of the examination’ and ‘medical 
decision-making’ and each item is rated using a five-point Likert-scale with descriptions of 
performance ranging from very poor (score = 1) to excellent (score = 5). All items are weighted 
equally, as high inter-item correlation have been found previously15. Hence, for each scanning 
modality a total score from 7 to 35 points may be achieved.  
The OSAUS ultrasound competence assessment tool have been developed as a generic tool to 
assess ultrasound competence across medical specialities. The assessment tool was developed 
through a Delphi study14 and it has been used to assess clinicians’ ability to transfer learning 
from an ultrasonography course into diagnostic performance on patients. Furthermore, the tool 
has been found valid and reliable to distinguish between novice POCUS users and experts16.    
 
For the primary outcome(P1) and secondary outcomes (S1-S2) we will use the Objective 
Structured Assessment of Ultrasound Skills (OSAUS) score scored after 12 months. For the 
primary outcome (P2) we will use the OSAUS score at baseline and at 12 months.  
 
Primary outcomes:  
(P1): The summarized OSAUS score after 12 months for all ten modalities will be calculated as 
percentage of maximum score. To summarize normal and non-normal distributed data similarly 
mean(SD) as well as median(IQR) will be presented together with minimum and maximum 
scores.  
 
Primary outcome 2 (P2):  

Maintaining POCUS scanning competence  
11 
The summarized OSAUS score at baseline and at 12 months for each participating GP will be 
compared and the proportion of GPs, with an OSAUS score at 12 months larger than or equal to 
the ultrasound competence score at baseline, will be reported.   
 
 
Secondary outcomes 
(S1): For GPs, participating in a POCUS quality assurance program, the OSAUS scores and the 
item scores after 12 months, for each of the ten scanning modalities included in the curriculum 
will be calculated and presented as median, IQR, minimum and maximum score. 
 
(S2): We calculate the proportion of GPs, who achieve a minimum OSAUS score of three for all 
seven domains for each of the ten scanning modalities after 12 months. As variation in scores 
can occur between the four expert assessors. We will normalize the scores by multiplying a given 
accessors score with (0.2 x sum of scores of all five accessors/the given accessors mean score) 
prior to calculating the proportion of successful GPs. 
 
For the secondary outcome S3, we will use the GPs assessment of their own scanning 
competence. The GPs will declare to which degree they have POCUS scanning competence 
within a specific scanning modality to perform the scan un-supervised in general practice (To a 
very high degree, to a high degree, to some degree, to a lesser degree, not at all, unsure).    
 
(S3): We calculate the proportion of GPs, who rate themselves as competent to perform POCUS 
un-supervised in general practice, after the educational period (three months after baseline), 
within each of the ten scanning modalities. 
 
For the secondary outcome S4, we will use fee-specific codes registered by the GP during 
consultation in general practice. Following the use of POCUS in the general practice 
consultations the GPs will follow normal procedures for registration of activities in general 
practice using remuneration codes and fee-specific codes for using POCUS in the medical record 
system. Prior to this study, 10 fee-specific codes for the 10 different POCUS examinations in the 
curriculum will be installed in the medical record system. The Primary Sector Data Provider 
Platform (PLSP) will develop an algorithm that allows for the following data extraction on each 
participating GP in a given time frame: (1) number of POCUS examinations performed and (2) 
number of consultations. PLSP will deliver monthly aggregated data for each participating GP 
from baseline to 12 months after baseline.  
 
(S4): The number of performed POCUS examinations by each GP each month during the study 
(months 1-12 after baseline) will be summarized to demonstrate variation over time.  
 
For the secondary outcome S5 and S6, we will use the results of distributed online quizzes to the 
participants. During the study, four online quizzes will be send to participants by email (Months 
3, 6, 9, and 12). A reminder will follow after two weeks to participants who fail to respond the 
quiz. The results of these quizzes will be summarized as the total number of completed quizzes 
for each participant (S5) and the test score for each participant at months 3, 6, 9, and 12 (S6).  
 

Maintaining POCUS scanning competence  
12 
For the secondary outcomes S7, S8 and S9, we will use a linear regression model to test 
associations between OSAUS score after 12 months and the total number of performed POCUS 
examination for each scanning modality (S7), the number of completed quizzes in the quality 
assurance program (S8), and the tests scores in the quality assurance program (S9). Changes in 
OSAUS scores between baseline and 12 months (S10) are calculated using linear regression. 
Scores at baseline and after 12 months will also be compared by a paired t-test. 
 
13 
Time schedule  
 
Figure 2 Study timeline   
 
2022 
2023 
Month 
Mar 
Apr 
May 
Jun 
Jul 
Aug 
Sep 
Oct 
Nov 
Dec 
Jan 
Feb 
Mar 
Apr 
May 
Jun 
Jul 
Aug 
Sep 
Oct 
Nov 
Dec 
Ultrasound course   
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Skills assessment 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Registration of fee 
for using POCUS  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Quiz  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Evaluation  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14 
Sample size  
 
Our potential study population is the 18 GPs who participated in the ultrasound course organized 
by PLO-e. We expect a participation rate of 80% corresponding to 15 GP. 
 
With 15 participants the total number of possible completed 7-item OSAUS questionnaires will 
be 15 for each modality and 150 for the combined estimate after 12 months. This is considered 
sufficient to estimate mean estimates and provide confidence intervals with acceptable spread.  
 
15 
Recruitment  
All participants, who attended the PLO-e ultrasound course were invited to participate in the 
study. The ultrasound course was listed in PLO-e course catalogue for 2022 and as such offered 
to all GP working in primary care in Denmark. 
Assignment of interventions  
16a  
Allocation Sequence generation 
n/a 
16b 
Allocation concealment mechanism 
n/a 
16c 
Implementation 
n/a 
 
17 
Blinding  
Expert assessors performing the competence assessment (primary outcomes) have a medical 
background and are considered experts in the field. They have not been teaching participants in 
the training program. They will be blinded to the participants prior experience with POCUS, the 

Maintaining POCUS scanning competence  
13 
number of performed POCUS examinations, and any other elements in the participants’ learning 
process. 
The principal investigator (SKA) and MBJ acts as expert assessors in this study and as such, they 
will be blinded to the background information on participants, the data collection and the 
analysis of data. CAA will be in charge of all participant correspondence, AR will be cleaning 
the data set and analyzing the data. 
Data collection, management, and analysis 
 
18 
Data collection - Plans for assessment and collection of outcome, baseline, and other 
trial data 
 
Multiple Choice Questionnaire (MCQ) 
MCQ’s will be distributed by email to participating GPs at 3, 6, 9, and 12 months. The email 
includes a link to an online questionnaire prepared and collected using Forms (Microsoft Office, 
Redmond, USA). A reminder will follow after two weeks to participants failing to complete the 
MCQ after the initial invitation. Data will be transferred and saved on a secure server at Aalborg 
University.    
 
Assessment of scanning competence  
The assessment of participants scanning competence using the OSAUS score will be performed 
in the same manner using the same expert assessors at baseline and after 12 months.  
 
During the assessment, only the participant and the assessor are present. Each assessment will 
follow the same structure, where experts will assess the participants by asking the following 
questions: 
- 
In which clinical scenarios would you perform this POCUS examination (Item 1 in the 
OSAUS) 
The experts will ask participants to demonstrate the POCUS examination (for maximum five 
minutes) to assess the following:  
- 
Applied knowledge of the ultrasound equipment (Item 2 in the OSAUS)  
- 
Image optimization (Item 3 in the OSAUS)  
- 
Systematic examination (Item 4 in the OSAUS)  
- 
Interpretation of images (Item 5 in the OSAUS)  
The experts will present the participants with a picture of common pathology and ask the 
participants the following questions:   
- 
How would you interpret these ultrasound findings? (Item 5 in the OSAUS)  
- 
If you were to describe this examination in the medical record, what would you write? 
(Item 6 in the OSAUS) 
- 
What would you do if you found it? (Item 7 in the OSAUS) 
- 
 
Following the assessment, the participant will receive feedback on the level of performance and 
educational advice. The total duration of the assessment is expected to last 90 minutes for each 
participant. The assessor completes an OSAUS score for each of the ten examinations performed 
by the participant. 

Maintaining POCUS scanning competence  
14 
 
 
The OSAUS assessment tool measures competency according to seven key domains: 1) 
Indication for the examination 2) Applied knowledge of ultrasound equipment 3) Image 
optimization 4) Systematic examination 5) Interpretation of images 6) Documentation of 
examination and 7) Medical decision making.  Each domain is rated using a five-point Likert 
scale with descriptions of performance ranging from very poor (score=1) to excellent (score=5).  
 
Figure 3. The OSAUS competence assessment tool 
 
 
From: Tolsgaard MG, Todsen T, Sorensen JL, Ringsted C, Lorentzen T, Ottesen B, et al. (2013) International Multispecialty Consensus on How 
to Evaluate Ultrasound Competence: A Delphi Consensus Survey. PLoS ONE 8(2): e57687. https://doi.org/10.1371/journal.pone.005768 
(reference 13) 
 
Education of POCUS expert assessors 
The POCUS expert assessors all have prior experience using the OSAUS score. Still, prior to the 
assessment of the participants’ performance in this study, the POCUS experts participate in a 90-
minutes online training session with CAA who has prior experience with the education of a team 
of assessors. During this training session, the POCUS experts are presented to the OSAUS score 
and the procedure for assessing the participants in this study. The POCUS experts are instructed 

Maintaining POCUS scanning competence  
15 
to rate according to the standard expected from a GP, who is capable of performing POCUS 
unsupervised in general practice.   
 
Self-rated scanning competence  
In relation to the expert assessment of scanning competence, participants were asked to fill out a 
questionnaire where they declare whether or not they have POCUS scanning competence within 
this scanning modality to perform the scan un-supervised in general practice. The expert 
assessors were blinded to this declaration.   
 
Evaluation questionnaire  
After 12 months following the final MCQ-quiz and the final evaluation of scanning competence, 
we will circulate an evaluation questionnaire for participants. This questionnaire will be 
developed based on collected experiences during the study.  
  
19 
Data management 
The OSAUS assessment will be filled out by the expert assessors using paper versions of the 
registration sheet. The registration sheets and questionnaires will be collected by CAA and 
checked for completeness. CAA will bring the registration sheets to CAM AAU, where they will 
be safely stored in a locked cupboard. Two research assistants will independently impute the 
results in a Microsoft Excel fil along with the results from the MCQ-quiz. The two datasets will 
then be compared for inconsistencies and these will be resolved by involving a third party.    
 
Hereafter, all digital study related data will be pseudo-anonymized and stored on secure server 
on Aalborg University and handled according to the General Data Protection Regulation. 
 
20a 
Statistical methods for analysing primary and secondary outcomes.  
All statistical analyses will be performed using STATA version 17 (StataCorp, Texas, USA) and 
analyzed according to a predefined statistical analysis plan, which will be uploaded prior to the 
data collection for the primary outcome.   
 
20b 
Methods for any additional analyses (e.g. subgroup and adjusted analyses). 
Sub-analyses will be performed to explore the importance of different educational elements in 
the intervention. These sub-analyses will be defined in the statistical analysis plan.   
 
20c 
Missing data  
A predictive mean matching (pmm) imputation method will be used to fill in missing values by 
the command “mi impute pmm” in Stata17, for all OSAUS scores after 3 and 6 months. 
Predictive mean matching (PMM) is a partially parametric method that matches the missing 
value to the observed value with the closest predicted mean18. 
Data monitoring 
 
21a 
Data monitoring committee (DMC). 
CAA will monitor the data collection and contact the project steering committee in case of 
problems related to data collection or the validity of the collected data. CAA will also contact the 
steering committee in case of participant drop-out.    

Maintaining POCUS scanning competence  
16 
 
21b 
Description of any interim analyses and stopping guidelines, including who will have 
access to these interim results and make the final decision to terminate the trial 
n/a 
 
22 
Harms  
 
The participating GPs will be obliged to report any Suspected Unexpected Serious Adverse 
Reactions (SUSARs), Serious Adverse Events (SAEs) and Adverse Events (AEs) related or 
possibly related to the use of POCUS to the adverse event committee.  
 
23 
Auditing 
n/a 
Ethics and dissemination 
24 
Research ethics approval 
The study will be performed in accordance with the Declaration of Helsinki. The project was 
notified to The North Denmark Region Committee on Health Research Ethics  (Den 
Videnskabsetiske Komité for Region Nordjylland, reference number 2022-000764) who 
responded that according to Danish Law (komitélovens § 14, stk. 2), no ethical approval is 
needed for this project.  
Prior to the collection of register data, we will seek permission from the Danish Regions  
The project has been registered and conducted according to the regulations of the Danish Data 
Protection Agency (registration number ID-242-2).  
 
25 
Protocol amendments 
 
The registration on clinicaltrials.gov will be updated if any of the above-mentioned 
modifications are made. 
 
26a 
Consent or assent 
 
Informed consent form the participating GPs will be collected by the principal investigator prior 
to the study.  
During the study, participating GPs will inform patients about the study by making a written 
description of the study available in the clinic.   
 
26b 
Additional consent provisions for collection and use of participant data and 
biological specimens in ancillary studies, if applicable 
Prior to the collection of register data, we will submit the research project to the Regional 
Council for approval of data collection through PLSP from patients' medical records without 
patient consent.  
 
27 
Confidentiality 
 
Personal information about participants consenting to participate will be stored on a secure 
server at Aalborg University. All data will be kept for 10 years after completion of the study 
which in accordance with The European Code of Conduct for Research Integrity.  

Maintaining POCUS scanning competence  
17 
 
28 
Declaration of interests 
The authors declare that they have no competing interests. 
 
29 
Access to data 
During the study, only CAA and a research assistant will have access to the data. After the 
pseudo-anonymization, the following authors will have access to collected data stored at the 
secure server at Aalborg University (SKA, MBJ and CAA).  
 
30 
Ancillary and post-trial care 
 
Any participants who suffer harm from trial participation will be eligible to seek compensation 
by The Patient Compensation Association.  
 
31a 
Dissemination policy  
We aim to publish positive, negative or inconclusive results of the study in a peer-reviewed 
journal. The project group will also present results at conferences.  
 
31b 
Authorship eligibility guidelines  
We aim to follow the definition of authors defined on ICMJE´s four criteria1:  
 
• 
Substantial contributions to the conception or design of the work; or the acquisition, 
analysis, or interpretation of data for the work; AND 
• 
Drafting the work or revising it critically for important intellectual content; AND 
• 
Final approval of the version to be published; AND 
• 
Agreement to be accountable for all aspects of the work in ensuring that questions related 
to the accuracy or integrity of any part of the work are appropriately investigated and 
resolved. 
 
31c 
Plans for granting public access to the full protocol, participant-level dataset, and 
statistical code 
It is unsure if data can be anonymized sufficiently to be made publicly available. 
 
32 
Informed consent materials 
The Danish informed consent form will be attached as an appendix   
 
References 
 
1 International Committee of Medical Journal Editors. Defining the Role of Authors and Contributors. Available 
from: http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-
contributors.html 
 
                                                 
 
                                                 

Maintaining POCUS scanning competence  
18 
                                                                                                                                                             
2 Myklestul HC, Skonnord T, Brekke M. Point-of-care ultrasound (POCUS) in Norwegian general practice. Scand J 
Prim Health Care. 2020 Jun;38(2):219-225. 
3 Aakjær Andersen C, Holden S Vela J, Rathleff MS, and Jensen MB. Point-of-Care Ultrasound in General Practice: 
A Systematic Review. Ann Fam Med January/February 2019 vol. 17 no. 1 61-69. 
4 Andersen CA, Espersen M, Brodersen J, Thomsen JL, Jensen MB, Davidsen AS. Learning strategies of general 
practitioners striving to achieve point-of-care ultrasound competence: a qualitative study. Scand J Prim Health Care. 
2022 Feb 15:1-11. doi: 10.1080/02813432.2022.2036483. 
5 Andersen CA, Brodersen J, Davidsen AS, Graumann O, Jensen MB. The use and impact of point-of-care 
ultrasonography in general practice: a prospective observational study. BMJ Open. 2020 Sep 17;10(9):e037664. doi: 
10.1136/bmjopen-2020-037664 
6 Andersen CA, Frandsen AK, Valentiner-Branth C, Lykkegaard J, Løkkegaard T, Thomsen JL, Jensen MB, Hansen 
MP. Introducing point-of-care ultrasound in Danish general practice – elucidating the use through a medical audit. 
Fam Pract. 2020 Aug 25:cmaa080. doi: 10.1093/fampra/cmaa080 
7 Guyatt G, Rennie D, Meade MO, Cook DJ. User’s guide to the medical literature. Amanual for evidence-based 
clinical practice. Third edition. 2008. JAMA evidence, ISBN: 978-0-07-179071-0 
8 Pedersen KM, Andersen JS, Søndergaard J. General practice and primary health care in Denmark. J Am Board 
Fam Med. 2012 Mar;25 Suppl 1:S34-8. 
9 Irving G, Neves AL, Dambha-Miller H, Oishi A, Tagashira H, Verho A, Holden J. International variations in 
primary care physician consultation time: a systematic review of 67 countries. BMJ Open. 2017 Nov 
8;7(10):e017902. 
10 Unpublished data. Study registration on ClinicalTrials.org: 
https://clinicaltrials.gov/ct2/show/NCT05274581?term=camilla+aakj%C3%A6r+andersen&draw=2&rank=2 
(accessed 15-11-2022) 
11 Exley, K. & Dennick, R. (2004). Small group teaching: Tutorials, seminars and beyond (Elektronic Edition) 
London: RoutledgeFalmer 
12 Minimum training recommendations for the practice of medical ultrasound. Ultraschall Med. 2006;27(1):79-105. 
13 Löffler C, Altiner A, Blumenthal S, Bruno P, De Sutter A, De Vos BJ, Dinant GJ, Duerden M, Dunais B, Egidi G, 
Gibis B, Melbye H, Rouquier F, Rosemann T, Touboul-Lundgren P, Feldmeier G. Challenges and opportunities for 
general practice specific CME in Europe - a narrative review of seven countries. BMC Med Educ. 2022 Nov 
7;22(1):761. 
14 Tolsgaard MG1, Todsen T, Sorensen JL, et al. International multispecialty consensus on how to evaluate 
ultrasound competence: a Delphi consensus survey. PLoS One. 2013;8(2):e57687.   
15 Tolsgaard, M.G., Ringsted, C., Dreisler, E., Klemmensen, A., Loft, A., Sorensen, J.L., Ottesen, B. and Tabor, A. 
(2014), Reliable and valid assessment of ultrasound operator competence in obstetrics and gynecology. Ultrasound 
Obstet Gynecol, 43: 437-443. https://doi.org/10.1002/uog.13198 
16 Todsen T, Tolsgaard MG, Olsen BH, Henriksen BM, Hillingsø JG, Konge L, Jensen ML, Ringsted C. Reliable 
and valid assessment of point-of-care ultrasonography. Ann Surg. 2015 Feb;261(2):309-15. doi: 
10.1097/SLA.0000000000000552. 
17 Schenker, N., and J. M. G. Taylor. 1996. Partially parametric techniques for multiple imputation. Computational 
Statistics & Data Analysis 22: 425–446. https://doi.org/10.1016/0167-9473(95)00057-7. 
18 Little, R. J. A. 1988. Missing-data adjustments in large surveys. Journal of Business & Economic Statistics 6: 
287–296. 
https://doi.org/10.2307/1391878.