Detection of the Occurrence of Infiltration of Gadolinium Injection in Brain MR Scans Using Artificial Intelligence
Conditions: Magnetic Resonance Imaging
Sponsor: Mayo Clinic

Minimal Risk Protocol Template [IRB 10429.010] 
 
Effective: 9/20/2017 
 
 
 
 
 
 
 
 
Page 1 of 5 
 
IRB Minimal Risk Protocol Template  
 
General Study Information 
 
Principal Investigator: James Pipe, PhD, Magnetic Resonance Technology and Use Design Lab, Department 
of Radiology, Mayo Clinic, Rochester 
Other Study Staff:  
Jacinta Browne, PhD, 
Norbert Campeau, MD 
Jared Verdoorn, MD 
 
Study Title:    Detection of the occurrence of infiltration of gadolinium injection in brain MR scans using 
artificial intelligence. 
 
Protocol version number and date:     Version 2: January 20th, 2022 
 
Research Question and Aims 
 
Hypothesis: Machine learning can be used to make a binary decision regarding the occurrence of infiltration of 
the MR contrast agent, gadolinium, through analysis of MR Brain scan images.   
 
Aims, purpose, or objectives: The aim of this study is to develop machine-learning algorithms and 
infrastructure through the analysis of MR image data for training and testing, for the detection of intravenous 
infiltration of MR contrast agent. This aim will be achieved through the analysis of both non-infiltrated and 
infiltrated brain MR images.  This analysis will involve segmenting and analyzing the nasal mucosa region 
following contrast agent administration. The developed machine learning model will be trained to be able to 
differentiate between non-infiltrated (contrast-enhanced) and infiltrated MR images and thus determine if 
infiltration has occurred.  The training and testing data will include both prospective and retrospective MR data.  
Note: If this study establishes a human specimen repository (biobank) for research 
purposes, do not use this template. Use the Mayo Clinic Human Specimen Repository 
Protocol Template found on the IRB home page under Forms and Procedures at 
http://intranet.mayo.edu/charlie/irb/ 
 
First-time Use: Use this template to describe your study for a new IRB submission. 
1. Complete the questions that apply to your study. 
2. Save an electronic copy of this protocol for future revisions.    
3. When completing your IRBe application, you will be asked to upload this document to the protocol 
section. 
Modification:  To modify this document after your study has been approved: 
1. Open your study in IRBe. Click on the study ‘Documents’ tab and select the most recent version of the 
protocol. Save it to your files. 
2. Open the saved document and activate “Track Changes”.  
3. Revise the protocol template to reflect the modification points , save the template to your files 
4. Create an IRBe Modification for the study and upload the revised protocol template. 

Minimal Risk Protocol Template [IRB 10429.010] 
 
Effective: 9/20/2017 
 
 
 
 
 
 
 
 
Page 2 of 5 
 
This IRB application outlines the part of the study which will involve the collection prospective data for 
training the machine learning model. 
 
 
Background (Include relevant experience, gaps in current knowledge, preliminary data, etc.):   
 
In 2015, an estimated 17 million magnetic resonance imaging (MRI) examinations were performed in the 
United States using intravenous (IV) contrast agents [1], reflecting the essential role of IV contrast in the 
detection, characterization, and staging of disease.  On occasion, the IV line is inadequate and causes the 
contrast medium to infiltrate into the muscle tissue in the arm, which can cause discomfort to the patient, delays 
with the patient scan as well as the need for a repeat exam to be performed or patient call-back, if not detected 
in a timely manner.  The risk of undetected contrast medium extravasation is increased when automated 
mechanical injectors or higher injection rages are used for the contrast material injection [2-3].  Thus, if the 
occurrence of IV infiltration could be detected during the patient MR examination before the patient is removed 
from the scanner this would significantly improve both the patient experience and the clinical workflow.  To 
that end, this study aims to develop a machine learning algorithm to detect intravenous infiltration, into the 
muscle of MR contrast agent used for routine brain MR examinations, through the analysis of the nasal mucosa. 
 
Study Design and Methods 
 
Methods:  Describe in lay terms, completely detailing the research activities that will be conducted by Mayo 
Clinic staff under this protocol. 
 
We will recruit 40 patients presenting for a clinical Magnetic Resonance Imaging (MRI) brain without and with 
IV contrast agent. Informed written consent will be obtained prior to the participation in the study and recorded 
in clinical documents. Each patient will be contacted by the principal investigator or one of his/her designees. 
The potential risks to participants will be minimal. The specific absorption rate (SAR) used in this study is well 
under the current FDA safety limits for MRI. No additional drugs or invasive procedures will be used in the 
study.  
 
Medical charts will be abstracted for imaging and pathologic findings. Patients will undergo their one additional 
MR sequence as part of their clinical MRI brain scan. This additional sequence will be included prior to IV 
contrast administration as part of the standard MR brain without and with IV contrast examination.  
Commercially available MRI scanners (General Electric (GE) Medical Systems, Wauwatosa, WI, USA at both 
1.5T and 3T and Siemens, Siemens Healthcare, Germany) will be used.  
 
Study Procedure 
Commercially available MRI scanners (General Electric (GE) Medical Systems, Wauwatosa, WI, USA at both 
1.5T and 3T and Siemens, Siemens Healthcare, Germany at both 1.5T and 3T) will be used. A total of 40 
patients will be recruited and randomly selected from the cohort of patients undergoing a MR brain without and 
with IV contrast examination on the Siemens 1.5T and 3T, and GE 1.5T and 3T MR scanners within the clinic.  
The additional sequence will be a sagittal T1 CUBE FS for the GE scanners and a sagittal T1 SPACE FS for the 
Siemens scanners.    
 

Minimal Risk Protocol Template [IRB 10429.010] 
 
Effective: 9/20/2017 
 
 
 
 
 
 
 
 
Page 3 of 5 
 
The additional sequence will be anonymized and stored for subsequent analysis using the machine learning 
model as part of the training phase.  
 
This study involves the development of an AI algorithm which analyze images from routine Brain without and 
with IV contrast agent (Gadolinium (Gd)) examinations to confirm that Gd contrast agent has been correctly 
administered. The training and testing data will include both prospective and retrospective MR data. We require 
additional data which simulates the occurrence of IV infiltration during a routine brain without and with IV 
contrast exam as currently there is a low number of previous studies (n= 50) with associated MR images 
demonstrating IV infiltration. The data collected as part of this innovation study will be used to augment the 
retrospective data already collected for the training and testing of the developed AI algorithm.   
The machine learning training and testing process is described below and illustrated in Figure 1. 
i. 
Training Phase - Predictive machine learning models for detection of infiltration will be developed and 
the performance optimized. Machine Learning Tasks will be to (a) differentiate nasal mucosa with Gd 
contrast agent uptake vs. nasal mucosa or eye fat ratio without Gd contrast agent uptake. 
ii. 
Test Phase - Evaluate the machine learning model and then compare them with the assessment by 
radiologists using McNemar’s test and Receiver Operator Curve (ROC) analysis with specificity 
threshold equivalent to clinical reporting. 
 
 
Figure 1: (a) Radiomics Machine Learning model development and training workflow.  (b) (i) During training, the network weights 
will be optimized according to the error calculated by the binary loss function; (ii) After each training epoch, the model performance 
will be assessed by the area under the receiver-operating characteristic curve (AUC) on the validation set; (iii) The parameters of the 
model predictions with be recalibrated using the unseen calibration dataset and (iv) Model testing will be carried out in the unseen test 
cohort using the best performing model identified from the validation set and the performance will be evaluated using receiver-
operator characteristic curve analysis, net benefit analysis and intra-class correlation coefficient. 
 
 
(i) MRI images
e.g. Brain Gad Sag 
T1 CUBE (SPACE) FS
(n=300)
Brain Sag T1 CUBE 
(SPACE) FS 
(n=100) 
(ii) 3D 
Segmentation
e.g. nasal 
mucosa
(iii) Image 
Normalization
(iv) Feature 
Extraction
e.g. nasal 
mucosa -
Shape 
Features, 
texture 
features 
and First-
Order 
Features
(v) Analysis
e.g.
Radiomic 
Features 
combined 
with 
Machine 
Learning 
Features. 
ROC 
Analysis
(a)
Data Pre-processing
Training
(i) Training 
(65% exams) –
adjustment of 
model weights 
to reduce the 
loss error
(ii) Validation 
(15% exams) 
– after each 
training 
epoch, the 
model 
performance 
will be 
assessed by 
ROC analysis 
on the 
validation 
exams 
dataset
(iii) Calibration 
(10% exams) –
the unseen 
calibration 
dataset will be 
applied to the  
best 
performing 
model to
estimate 
recalibration 
parameters of 
the model 
predictions
(iv) Model testing (10% 
exams) will be carried out in 
the unseen test cohort using 
the best performing model 
identified from the validation 
set.
Model Performance 
Evaluation – Net benefit 
analysis, ROC AUC, sensitivity, 
specificity, positive predictive 
value (PPV) and intra-class 
correlation coefficient. 
(b)
Machine Learning Model 
Training Cohort
Machine Learning Model 
Test Cohort

Minimal Risk Protocol Template [IRB 10429.010] 
 
Effective: 9/20/2017 
 
 
 
 
 
 
 
 
Page 4 of 5 
 
 
Subject Information 
 
Target accrual is the proposed total number of subjects to be included in this study at Mayo Clinic. A “Subject” 
may include medical records, images, or specimens generated at Mayo Clinic and/or received from external 
sources.    
 
Target accrual: 40 
 
Subject population (children, adults, groups):  Adults undergoing MR brain without and with IV contrast 
examination as part of their care plan. 
 
Inclusion Criteria:  
Patients undergoing a MR brain without and with contrast examination on any GE or Siemens 1.5T and 
3T MRI system within Mayo Clinic, Rochester, as part of their care plan. 
 
Exclusion Criteria:   
Pregnant Women and individuals under the age of 18 years. 
 
Review of medical records, images, specimens  
 
 
Check all that apply (data includes medical records, images, specimens).  
 
  Only data that exists before the IRB submission date will be collected.   
 
Date Range for Specimens and/or Review of Medical Records:   
 
Note: The Date Range must include the period for collection of baseline data, as well as follow-up data, 
if applicable. 
 
  The study involves data that exist at the time of IRB submission and data that will be generated after IRB 
submission. Include this activity in the Methods section.  
Examples 
• The study plans to recruit subjects undergoing a MR brain without and with IV contrast examination 
as part of their care plan.  
 
  The study will use data that have been collected under another IRB protocol. Include in the Methods section 
and enter the IRB number from which the research material will be obtained. When appropriate, note when 
subjects have provided consent for future use of their data and/or specimens as described in this protocol.  
 
Enter one IRB number per line, add more lines as needed 
 
 Data    
 Specimens  
 Data & Specimens  ______________________________________ 

Minimal Risk Protocol Template [IRB 10429.010] 
 
Effective: 9/20/2017 
 
 
 
 
 
 
 
 
Page 5 of 5 
 
 
 Data    
 Specimens  
 Data & Specimens  ______________________________________ 
 
 Data    
 Specimens  
 Data & Specimens  ______________________________________ 
 
 
Data Analysis 
 
 
Power analyses may not be appropriate if this is a feasibility or pilot study, but end-point analysis plans are 
always appropriate even if only exploratory. Provide all information requested below, or provide justification if 
not including all of the information.  
 
Power Statement:  Not applicable – pilot study 
 
Data Analysis Plan:  
Receiver operating characteristic (ROC) curves will be generated for radiologist diagnosis and the machine 
learning models decisions and compared using the Delong test. Cut-off values for the machine learning models 
will be selected to match the sensitivity of radiologist clinical assessment in the training cohort, in order to 
construct working points of the models that maintain clinically achieved detection rates for the specific clinical 
exams. The sensitivity and specificity of the machine learning models will be compared based on the reduction 
of false-positive (FP) decisions compared with the clinical reference. The performance of the machine learning 
models will be compared to that of the radiologist performance using the McNemar test.  
 
Endpoints 
Primary: Area-under-curve performance for each machine learning model and the performance measure “net 
benefit”. Net benefit is defined as:  
 
The weighting factor, w, corresponds to the odds (i.e. the harm-to-benefit ratio) of the chosen risk threshold.  
Retest-repeatability of the model determined from intra-class correlation coefficient (ICC).  
Secondary: Sensitivity and specificity of each machine learning model and comparison with radiologist 
performance using the McNemar test. 
 
References 
1. OECD Health Statistics. Health care utilisation. OECD. Stat Web site. Available at: 
http://stats.oecd.org/Index.aspx?DataSetCode=HEALTH_ PROC.  
2. Bullard MA, Cohan RH, Ellis JH, et al. Extravasation of intravenous contrast material: incidence, 
management, outcome. Acad Radiol 1997;4:711–8. 
3. Nicola R, Shaqdan KW, Aran S, et al. Contrast media extravasation of computed tomography and 
magnetic resonance imaging: management guidelines for the radiologist. Curr Probl Diagn Radiol 
2016;45:161–4.