<figure><img src="image_1.png" /></figure>

HAL
open science

# Use of a Pre-Trained Neural Network for Automatic Classification of Arterial Doppler Flow Waveforms: A Proof of Concept

Antoine Guilcher, Damien Lanéelle, Guillaume Mahe

► To cite this version:

Antoine Guilcher, Damien Lanéelle, Guillaume Mahe. Use of a Pre-Trained Neural Network for Automatic Classification of Arterial Doppler Flow Waveforms: A Proof of Concept. Journal of Clinical Medicine, 2021, 10 (19), 10.3390/jcm10194479. [https://doi.org/10.3413306]

HAL Id: hal-03413306

https://hal.science/hal-03413306

Submitted on 3 Nov 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Article

# Use of a Pre-Trained Neural Network for Automatic Classification of Arterial Doppler Flow Waveforms: A Proof of Concept

Antoine Guilcher<fnref n="1" />, Damien Laneelle<fnref n="2" /> and Guillaume Mahé<fnref n="1" />,<fnref n="3" />,<fnref n="4" />,*

<figure><img src="image_3.png" /></figure>

check for
updates

**Citation:** Guilcher, A.; Laneelle, D.; Mahé, G. Use of a Pre-Trained Neural Network for Automatic Classification of Arterial Doppler Flow Waveforms: A Proof of Concept. *J. Clin. Med.* **2021**, *10*, 4479. https://doi.org/10.3390/jcm10194479

Academic Editor:
Ferdinando Mannello

Received: 13 August 2021
Accepted: 28 September 2021
Published: 28 September 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_4.png" /></figure>

**Copyright:** © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Abstract:** Background: Arterial Doppler flow waveform analysis is a tool recommended for the management of lower extremity peripheral arterial disease (PAD). To standardize the waveform analysis, classifications have been proposed. Neural networks have shown a great ability to categorize data. The aim of the present study was to use an existing neural network to evaluate the potential for categorization of arterial Doppler flow waveforms according to a commonly used classification.
Methods: The Pareto efficient ResNet-101 (ResNet-101) neural network was chosen to categorize 424 images of arterial Doppler flow waveforms according to the Simplified Saint-Bonnet classification.
As a reference, the inter-operator variability between two trained vascular medicine physicians was also assessed. Accuracy was expressed in percentage, and agreement was assessed using Cohen's Kappa coefficient. Results: After retraining, ResNet-101 was able to categorize waveforms with 83.7 ± 4.6% accuracy resulting in a kappa coefficient of 0.79 (0.75-0.83) (CI 95%), compared with a kappa coefficient of 0.83 (0.79-0.87) (CI 95%) between the two physicians. Conclusion: This study suggests that the use of transfer learning on a pre-trained neural network is feasible for the automatic classification of images of arterial Doppler flow waveforms.

**Keywords:** Doppler waveform; peripheral artery disease; neural network classification

## 1. Introduction

With an aging population worldwide, lower extremity peripheral artery disease (PAD) has become a major health issue [1,2] with significant financial implications [3–6]. Consequently, optimization of the management of patients suffering from PAD is a matter of upmost importance.

Along with ankle brachial pressure index (ABI) and toe brachial pressure index (TBI), arterial Doppler waveform analysis is a tool recommended to diagnose and manage PAD [7–9]. For years, researchers have described the relationship between waveform contours and PAD severity [10–13]. Arterial Doppler waveforms are usually described according to the number of phases (i.e., alternance between forward and backward flow) and “sharpness” (i.e., acceleration/deceleration of blood flow) [14–17]. It is commonly accepted that the greater the number of phases and the sharper the signal the healthier the artery, and vice versa [17,18]. However, the interpretation and description of Doppler waveforms is highly subjective [19].

As a result, an abundance of terms has surfaced in the literature to describe Doppler waveforms, often leading to confusion and misunderstanding between medical staff, resulting in a differing management of PAD [19–21]. In order to standardize arterial Doppler waveform description, and improve Doppler waveform analysis, several classifications have been proposed [14–16] to categorize arterial flow contours and assess PAD severity.

<footnote n="1">¹ Vascular Medicine Unit, CHU Rennes, 35000 Rennes, France; antoine.guilcher@chu-rennes.fr</footnote>

<footnote n="2">² Vascular Medicine Unit, CHU Caen-Normandie, 14033 Caen, France; laneelle-d@chu-caen.fr</footnote>

<footnote n="3">³ Clinical Investigation Center, University Rennes, INSERM CIC 1414, 35033 Rennes, France</footnote>

<footnote n="4">⁴ M2S-EA 7470, University Rennes, 35170 Bruz, France</footnote>

<footnote>* Correspondence: maheguillaume@yahoo.fr or guillaume.mahe@chu-rennes.fr; Tel.: +33-2-99-28-43-21</footnote>
However, currently, none of the classifications propose an exhaustive description of the various arterial Doppler flow waveforms observed by vascular physicians in a clinical setting. Thus, reproducibility errors remain. One method to prevent reproducibility errors is to automate the categorization process.

Since the 1960s there has been a growing emphasis on computer aided diagnosis (CADg) in medicine. An increasingly utilized CADg technique is the application of neural networks in the classification of patients' conditions [22], including in patients suffering from PAD [23,24]. A neural network trained by experts to categorize arterial Doppler flow waveforms could potentially remove the remaining subjectivity in the classification process. However, creating and training a dedicated neural network is a complex process which requires a specific skill set and computational power, which for many medical teams could be unattainable.

One way to overcome these issues is a technique called transfer learning. Transfer learning utilizes a pre-trained network which is “retrained” to perform a new task. As the architecture of the neural network is already built, only a few parameters have to be adapted, making the technique relatively accessible to a novice user. Among the pre-trained networks freely available are the networks participating in the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) [25]. These networks are trained on more than a million images from the ImageNet database (http://www.image-net.org) to classify images according to a thousand categories.

The aim of the present study was to use transfer learning on a pre-trained neural network in order to automatically categorize images of arterial Doppler flow waveforms, and to assess its feasibility.

## 2. Materials and Methods

Arterial Doppler waveforms were acquired from patients who attended the vascular clinic of the University Hospital in Rennes from January 2019 to February 2020. The acquisition was performed with Philips iU22, Philips Epiq 7 and Mindray Resona 7 ultrasound machines using a linear probe in pulse-wave Doppler mode. Waveforms were selected only if they displayed at least one full cardiac cycle, if they did not exhibit signal loss or high velocity shift artifact, if the pulse-wave Doppler spectrum was not suffering from an excess of overgain and if they were not masked by low frequency noise. For each of the selected waveforms, a full cardiac cycle was then manually selected using a custom-made software developed using MATLAB (version R2190b, The MathWorks, Natick, MA USA). The complete dataset consisted of 424 waveforms, each representing a single cardiac cycle. These waveforms fitted 9 categories after categorization by a vascular physician according to the simplified Saint-Bonnet classification [14]. One hundred and forty-five waveforms were categorized as N, 15 as N-CF, 125 as A, 0 as A-CF, 19 as B, 29 as B-CF, 29 as CD, 35 as CD-CF, 13 as E and 14 as E-CF, according to the simplified Saint-Bonnet classification (Table 1).

<table><caption>**Table 1.** Full dataset waveform distribution according to the simplified Saint-Bonnet classification.</caption><thead><tr><td>Category</td><td>Number</td><td>%</td></tr></thead><tbody><tr><td>N</td><td>145</td><td>34.2</td></tr><tr><td>N-CF</td><td>15</td><td>3.5</td></tr><tr><td>A</td><td>125</td><td>29.5</td></tr><tr><td>A-CF</td><td>0</td><td>0.0</td></tr><tr><td>B</td><td>19</td><td>4.5</td></tr><tr><td>B-CF</td><td>29</td><td>6.8</td></tr><tr><td>CD</td><td>29</td><td>6.8</td></tr><tr><td>CD-CF</td><td>35</td><td>8.3</td></tr><tr><td>E</td><td>13</td><td>3.1</td></tr><tr><td>E-CF</td><td>14</td><td>3.3</td></tr></tbody></table>
## 2.1. Reference Classification

The simplified Saint-Bonnet classification (Figure 1) was chosen as the reference classification for two reasons: (i) The College of the French Vascular Medicine Teachers (Collège des Enseignants de Médecine Vasculaire, CEMV) recommends the use of the simplified Saint-Bonnet Classification in its most recent guidelines [9,26] and (ii) The simplified Saint-Bonnet Classification categories describe extensive types of arterial Doppler flow waveform representing different progressing grades of PAD severity [14].

<figure><img src="image_5.png" /><figcaption>**Figure 1.** The simplified Saint-Bonnet classification. There was no waveform in categories A-CF and 0 in the present study.</figcaption></figure>

## 2.2. Network Choice and Settings

The neural network was chosen among Pareto efficient networks (i.e., the best of its class for the combination of accuracy and computational time) participating in the ILSVRC. The computational power available for the study being quite modest (AMD 3700X CPU and NVIDIA GTX 1660 Super GPU with 6GB of RAM), the ResNet-101 network was selected. Training time for the task, using transfer learning, was around three and a half minutes.

Neural network training, testing and data processing were completed using MATLAB software and its Deep Learning Toolbox (version R2190b, The MathWorks, Natick, MA, USA).

Neural networks consist of several layers. An input layer, which matches the data format, stem layers which condition the data, hidden layers which extract the key characteristics of the data and final layers which perform the classification. As ResNet-101 accepts only 224 × 224 pixel images as inputs, the grayscale images of the selected waveforms were resized to match this format. Doppler flow waveform images were padded with black colored pixels (either along the x-axis or the y-axis) in order to obtain a square image, thus avoiding additional distortion. These images where then scaled to 224 × 224 pixels. In addition, as ResNet-101 is designed to sort images into a thousand object categories, much greater than what is required for the objective of this study, two of the final layers of the neural network were replaced by new classification layers. The new layers matched the number of arterial Doppler flow waveform categories found within the 424 selected waveforms according to the simplified Saint-Bonnet Classification (9 categories in the present study).

Default values were used for all parameters for transfer learning with the exception of: (i) the number of epochs (a full pass of the data through the network), which was set to 7 at which point the network accuracy was plateauing; (ii) the mini batch size (a sub sample of the dataset used to estimate and reduce prediction errors) which was set to 8, meeting Masters and Luschi criteria [27], with samples being shuffled before each epoch to avoid discarding the same data every epoch as the number of waveforms in the training set was not necessarily a multiple of 8; and (iii) the initial learning rate which was set to 3 × 10⁻⁴.
Several measures were implemented to reduce overfitting. Due to the small size of the dataset, no training was performed on the stem layers of Resnet101. The number of maximum epochs was chosen so that very little training would be done after the network accuracy had started to converge. Finally, waveforms of the training set were randomly modified by applying small offset and scaling coefficients, a process known as data augmentation [28].

## 2.3. Network Training and Validation

The small size of the dataset and the uneven sample size across categories lead to a risk of unstable prediction when training neural networks. For this reason, and the fact that it provides a better estimate of how the neural network will perform in a real-life setting, k-fold cross validation (with *k* = 10) was used to assess the accuracy of the neural network in categorizing arterial Doppler flow waveforms [29]. Ten stratified sub-datasets were randomly generated from the original dataset. Training was performed using 9 of the 10 datasets, with validation performed on the remaining, unknown to the network, dataset. The process was repeated 10 times, changing the validation set and the training sets at each iteration (Figure 2).

<figure><img src="image_6.png" /><figcaption>**Figure 2.** Illustration of the 10-fold cross-validation process. Legend: For the 10-fold cross validation, 10 sub datasets were used and for each of the 10 training and validation procedures, training was performed on 9 sub datasets (white boxes) and tested on the remaining sub dataset (black box). The subset used for testing changed between each procedure. Average accuracy and agreement compared with a vascular physician (physician 1) were calculated.</figcaption></figure>
## 2.4. Comparison with Humans

In order to have a point of reference on which to assess the neural network performance, two vascular physicians (physician 1 and physician 2) familiar with the simplified Saint-Bonnet classification (daily use for at least 2 years) categorized the same 424 waveforms used as inputs for the neural network. The categorization was performed according to the simplified Saint-Bonnet classification, and the agreement between the two physicians was assessed.

## 2.5. Statistics

The accuracy of the neural network classification was expressed in percentage ± standard deviation (SD). The agreement between the neural network and the reference vascular physician, as well as the agreement between the two trained vascular medicine physicians, were assessed with Cohen's Kappa coefficient [30] using MedCalc (version 18.5, MedCalc Software, Ostend, Belgium).

# 3. Results

In comparison with the waveform categorization performed by physician 1, the average accuracy of the ResNet-101 neural network to categorize arterial Doppler flow waveforms, after the transfer learning process, was 83.7 ± 4.6%. The resulting Cohen's Kappa coefficient was 0.79 (0.75–0.83) (CI 95%), indicating a moderate agreement [30]. Specifically, 86.9% of the N waveforms, 40.0% of the N-CF waveforms, 94.4% of the A waveforms, 68.4% of the B waveforms, 69.0% of the B-CF waveforms, 72.4% of the CD waveforms, 80.0% of the CD-CF waveforms, 76.9% of the E waveforms and 92.9% of the E-CF waveforms were categorized correctly (Table 2).

<table><caption>**Table 2.** Comparison of ResNet-101 waveform categorization with the reference vascular medicine physician according to the simplified Saint-Bonnet classification.</caption><thead><tr><th rowspan="2">ResNet-101</th><th rowspan="2">N</th><th rowspan="2">N-CF</th><th colspan="7">Physician 1</th></tr><tr><th>A</th><th>B</th><th>B-CF</th><th>CD</th><th>CD-CF</th><th>E</th><th>E-CF</th></tr></thead><tbody><tr><td>N</td><td>126</td><td>1</td><td>4</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>N-CF</td><td>7</td><td>6</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><td>A</td><td>7</td><td>0</td><td>118</td><td>2</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td></tr><tr><td>B</td><td>0</td><td>0</td><td>1</td><td>13</td><td>0</td><td>4</td><td>0</td><td>0</td><td>0</td></tr><tr><td>B-CF</td><td>2</td><td>3</td><td>0</td><td>1</td><td>20</td><td>0</td><td>4</td><td>0</td><td>0</td></tr><tr><td>CD</td><td>3</td><td>0</td><td>2</td><td>3</td><td>0</td><td>21</td><td>2</td><td>2</td><td>0</td></tr><tr><td>CD-CF</td><td>0</td><td>5</td><td>0</td><td>0</td><td>7</td><td>1</td><td>28</td><td>0</td><td>1</td></tr><tr><td>E</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>2</td><td>0</td><td>10</td><td>0</td></tr><tr><td>E-CF</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>13</td></tr><tr><td>Agreement (%)</td><td>86.9</td><td>40.0</td><td>94.4</td><td>68.4</td><td>69.0</td><td>72.4</td><td>80.0</td><td>76.9</td><td>92.9</td></tr></tbody></table>

ResNet-101 waveform categorization pooled from the 10-fold cross validation tests; Physician 1: reference physician. Numbers in bold represent the number of arterial Doppler waveforms that were in the same category according to the Physician 1 and the ResNet-101.

An example of the neural network waveform categorization can be seen in Figure 3.
<figure><img src="image_7.png" /><figcaption>**Figure 3.** Example of waveforms categorized by ResNet-101 in comparison with the waveform categorization performed by physician 1, according to the simplified Saint-Bonnet classification. Physician 1: reference physician.</figcaption></figure>

When comparing the categorization of the waveforms performed by the two trained vascular physicians, 86.5% (n = 367) of the waveforms were categorized identically. The resulting Cohen's Kappa coefficient between the two physicians was 0.83 (0.79–0.87) (CI 95%), indicating a strong agreement [30]. Specifically, 81.4% of the N waveforms, 93.3% of the N-CF waveforms, 98.4% of the A waveforms, 78.9% of the B waveforms, 51.7% of the B-CF waveforms, 72.4% of the CD waveforms, 97.1% of the CD-CF waveforms, 100% of the E waveforms and 100% of the E-CF waveforms were categorized identically (Table 3).
<table><caption>**Table 3.** Comparison of waveform categorization between the two trained vascular medicine physicians according to the simplified Saint-Bonnet classification.</caption><thead><tr><th rowspan="2">Physician 2</th><th rowspan="2">N</th><th colspan="8">Physician 1</th></tr><tr><th>N-CF</th><th>A</th><th>B</th><th>B-CF</th><th>CD</th><th>CD-CF</th><th>E</th><th>E-CF</th></tr></thead><tbody><tr><td>N</td><td>118</td><td>0</td><td>2</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>N-CF</td><td>27</td><td>14</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>A</td><td>0</td><td>0</td><td>123</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>B</td><td>0</td><td>0</td><td>0</td><td>15</td><td>0</td><td>6</td><td>0</td><td>0</td><td>0</td></tr><tr><td>B-CF</td><td>0</td><td>1</td><td>0</td><td>0</td><td>15</td><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><td>CD</td><td>0</td><td>0</td><td>0</td><td>4</td><td>0</td><td>21</td><td>0</td><td>0</td><td>0</td></tr><tr><td>CD-CF</td><td>0</td><td>0</td><td>0</td><td>0</td><td>14</td><td>1</td><td>34</td><td>0</td><td>0</td></tr><tr><td>E</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>0</td><td>13</td><td>0</td></tr><tr><td>E-CF</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>14</td></tr><tr><td>Agreement (%)</td><td>81.4</td><td>93.3</td><td>98.4</td><td>78.9</td><td>51.7</td><td>72.4</td><td>97.1</td><td>100.0</td><td>100.0</td></tr></tbody></table>

Physician 1: reference physician; Physician 2: trained vascular medicine physician. Numbers in bold represent the number of arterial Doppler waveforms that were in the same category according to the Physician 1 and the Physician 2.

## 4. Discussion

Transfer learning using the ResNet-101 network enabled the categorization of arterial Doppler flow waveforms according to the simplified Saint-Bonnet classification with an accuracy of 83.7 ± 4.6% and a Cohen's Kappa coefficient of 0.79 (0.75–0.83) compared with a vascular medicine physician familiar with the simplified Saint-Bonnet classification. Although this agreement does not reach that obtained between two trained vascular physicians (Kappa = 0.83 (0.79–0.87), strong agreement) it is very similar and without any potential reproducibility error. These results are in keeping with those of previous studies where neural networks were used to diagnose PAD [23,24] and the prospect of perfect reproducibility is of importance, especially as an arterial Doppler waveform description is frequently debated amongst vascular medicine physicians [17]. Neural networks could indeed provide an objective approach for categorization and a description of arterial Doppler waveforms.

The detailed results of the waveform classification performed by ResNet-101 (Table 2) indicates that the chosen neural network underperformed for the N-CF waveform category with 53% of these waveforms being incorrectly categorized as B-CF or CD-CF. This result could indicate that the neural network may have problems distinguishing between waveforms exhibiting a continuous flow. However, the results for waveform types B-CF (69.0%), CD-CF (80.0%) and E-CF (92.9%) demonstrate greater accuracy, with only one waveform wrongly categorized as N-CF. The neural network behaves as if it is overlooking the N-CF category. This is an expected result, as the N-CF category contains a low sample number (15 waveforms). Indeed, to optimize the training process, a neural network requires a large training set to ensure all waveforms observed in clinical practice are represented, and the number of samples within each category must be similar in order to limit bias towards, or against, certain categories. Therefore, the results of the present study could be improved by introducing a larger and more balanced dataset.

The main discrepancy in waveform categorization between the two vascular physicians resulted from difficulties in differentiating between B-CF and CD-CF waveforms. This disagreement is typical in general practice, as these two categories are sequential in the simplified Saint-Bonnet classification (Figure 1) and in terms of degree of severity of PAD. The main difference between B-CF and CD-CF categories resides in a change of the upstroke and the downstroke of the systolic velocity peak. This change is progressive, with no set threshold and is thus prone to inter-operator reproducibility error.

Concerning the simplified Saint-Bonnet classification, the results of the present study suggest that its use may be highly reproducible with a Cohen's Kappa coefficient of 0.83 (0.79–0.87), indicating a strong inter-operator agreement. The results also highlight the fact
that some categories, such as A-CF, may be theoretical or have a very low prevalence in a
population of patients consulting vascular clinics.

Although the use of the simplified Saint-Bonnet classification is recommended by The College of the French Vascular Medicine Teachers, it has not been endorsed internationally. However, compared to other classifications [15,16] the simplified Saint-Bonnet classification includes the most categories. Therefore, the use of a neural network to categorize arterial Doppler waveforms according to other classifications, with an equal or a lower number of categories, should provide similar accuracy, as long as those categories are of relevance.

Moving forward, the ideal situation would be to generate a large, freely accessible database of waveforms. This would enable a better description of waveform categories, potentially leading to the creation of a more robust classification, which could be utilized globally. It would also provide larger training and testing datasets for more precise automatic categorization. To achieve this aim, vascular medicine societies across the world should be encouraged to collaborate and share waveform data.

The present study has several limitations. As previously discussed, the dataset used was suboptimal due to the small number of waveforms and the lack of equality between waveform categories. Furthermore, as this study was designed as a proof of concept, the patient characteristics were unknown and although, in theory, a large number of different waveforms should correlate to differing underlying conditions and degree of disease, a selection bias cannot be excluded. Finally, as mentioned previously, the neural network was only trained to categorize waveforms according to the simplified Saint Bonnet classification, and therefore the results from this study cannot be extrapolated to other classifications.

**5. Conclusions**

Computer-aided diagnostics will undoubtedly be a part of the future of medicine, and medical practitioners should not only be “end users”, but must also play a part in the development of such tools for clinical practice. The results of the present study suggest that it is possible to use an existing neural network with the transfer learning technique to categorize arterial Doppler waveform with good accuracy. The creation of a worldwide, freely accessible, arterial Doppler waveform database would greatly improve these preliminary results. Such a tool could then be key in standardizing arterial Doppler waveforms analysis and thus improve PAD management.

**Author Contributions:** Conceptualization, A.G. and G.M.; methodology, A.G. and G.M.; validation, A.G., D.L. and G.M.; formal analysis, A.G. and G.M.; investigation, A.G. and G.M.; data curation, A.G.; writing—original draft preparation, A.G. and G.M.; writing—review and editing, A.G., D.L. and G.M.; supervision, G.M.; project administration, G.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding. The APC was funded by CHU Rennes.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the local institutional ethics committee of the CHU Rennes on 14 November 2018 under reference n° 18.88.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data that support the findings of this study are available from the corresponding author upon reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.
## References

1. Fowkes, F.G.R.; Rudan, D.; Rudan, I.; Aboyans, V.; Denenberg, J.O.; McDermott, M.M.; Norman, P.E.; Sampson, U.K.; Williams, L.J.; Mensah, G.A.; et al. Comparison of Global Estimates of Prevalence and Risk Factors for Peripheral Artery Disease in 2000 and 2010: A Systematic Review and Analysis. *Lancet* **2013**, *382*, 1329–1340. [CrossRef]

2. Hirsch, A.T.; Criqui, M.H.; Treat-Jacobson, D.; Regensteiner, J.G.; Creager, M.A.; Olin, J.W.; Krook, S.H.; Hunninghake, D.B.; Comerota, A.J.; Walsh, M.E.; et al. Peripheral Arterial Disease Detection, Awareness, and Treatment in Primary Care. *JAMA* **2001**, *286*, 1317–1324. [CrossRef]

3. Riviere, A.B.; Bouée, S.; Laurendeau, C.; Torreton, E.; Gourmelen, J.; Thomas-Delecourt, F. Outcomes and Management Costs of Peripheral Arterial Disease in France. *J. Vasc. Surg.* **2018**, *67*, 1834–1843. [CrossRef]

4. Hirsch, A.T.; Hartman, L.; Town, R.J.; Virnig, B.A. National Health Care Costs of Peripheral Arterial Disease in the Medicare Population. *Vasc. Med.* **2008**, *13*, 209–215. [CrossRef] [PubMed]

5. Mahoney, E.M.; Wang, K.; Keo, H.H.; Duval, S.; Smolderen, K.G.; Cohen, D.J.; Steg, G.; Bhatt, D.L.; Hirsch, A.T. Vascular Hospitalization Rates and Costs in Patients with Peripheral Artery Disease in the United States. *Circ. Cardiovasc. Qual. Outcomes* **2010**, *3*, 642–651. [CrossRef]

6. Scully, R.E.; Smith, A.D.; Arnaoutakis, D.J.; Semel, M.; Nguyen, L.L. Annual Health Care Expenditures in Individuals with Peripheral Arterial Disease. *J. Vasc. Surg.* **2016**, *64*, 1180–1181. [CrossRef]

7. Aboyans, V.; Ricco, J.-B.; Bartelink, M.-L.E.L.; Björck, M.; Brodmann, M.; Cohnert, T.; Collet, J.-P.; Czerny, M.; De Carlo, M.; Debus, S.; et al. 2017 ESC Guidelines on the Diagnosis and Treatment of Peripheral Arterial Diseases, in Collaboration with the European Society for Vascular Surgery (ESVS)Document Covering Atherosclerotic Disease of Extracranial Carotid and Vertebral, Mesenteric, Renal, Upper and Lower Extremity Arteries Endorsed by: The European Stroke Organization (ESO)The Task Force for the Diagnosis and Treatment of Peripheral Arterial Diseases of the European Society of Cardiology (ESC) and of the European Society for Vascular Surgery (ESVS). *Eur. Heart J.* **2018**, *39*, 763–816. [CrossRef] [PubMed]

8. Gerhard-Herman, M.D.; Gornik, H.L.; Barrett, C.; Barshes, N.R.; Corriere, M.A.; Drachman, D.E.; Fleisher, L.A.; Fowkes, F.G.R.; Hamburg, N.M.; Kinlay, S.; et al. 2016 AHA/ACC Guideline on the Management of Patients with Lower Extremity Peripheral Artery Disease: Executive Summary: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. *J. Am. Coll. Cardiol.* **2017**, *69*, 1465–1508. [CrossRef]

9. Mahé, G.; Boge, G.; Bura-Rivière, A.; Chakfé, N.; Constans, J.; Goueffic, Y.; Lacroix, P.; Le Hello, C.; Pernod, G.; Perez-Martin, A.; et al. Disparities between International Guidelines (AHA/ESC/ESVS/ESVM/SVS) Concerning—Lower Extremity Arterial Disease: Consensus of the French Society of Vascular Medicine (SFMV) and the French Society for Vascular and Endovascular Surgery (SCVE). *Ann. Vasc. Surg.* **2020**, *S0890–S5096*. [CrossRef]

10. Campbell, W.B. Doppler Waveform Analysis in the Management of Lower Limb Arterial Disease. *Ann. R. Coll. Surg. Engl.**1986**, *68*, 103–106.

11. Poe, P. Lower-Extremity Arterial Continuous-Wave Doppler Evaluation. *J. Vasc. Ultrasound* **2012**, *36*, 123–134. [CrossRef]

12. Strandness, D.E., Jr.; Schultz, R.D.; Sumner, D.S.; Rushmer, R.F. Ultrasonic Flow Detection a Useful Technic in the Evaluation of Peripheral Vascular Disease. *Am. J. Surg.* **1967**, *113*, 311–320. [CrossRef]

13. Dean, R.H.; Yao, J.S.T. Hemodynamic Measurements in Peripheral Vascular Disease. *Curr. Probl. Surg.* **1976**, *13*, 1–76. [CrossRef]

14. Boulon, C.; Desormais, I.; Boissier, C.; Bressollette, L.; Constans, J.; Lacroix, P.; Guilmot, J.L.; Le Hello, C.; Sevestre, M.-A.; Pernod, G.; et al. Analyse des Flux Doppler: Classification de Saint-Bonnet. *J. Des Mal. Vasc.* **2015**, *40*, 305. [CrossRef]

15. Spronk, S.; den Hoed, P.T.; de Jonge, L.C.W.; van Dijk, L.C.; Pattynama, P.M.T. Value of the Duplex Waveform at the Common Femoral Artery for Diagnosing Obstructive Aortoiliac Disease. *J. Vasc. Surg.* **2005**, *42*, 236–242. [CrossRef]

16. Descotes, J.; Cathignol, D. Classification des Altérations du Profil de Vitesse Ultra-Sonore dans Les Artères des Membres Inférieurs. *Nouv. Presse Med.* **1975**, *4*, 2091–2093.

17. Kim, E.S.; Sharma, A.M.; Scissons, R.; Dawson, D.; Eberhardt, R.T.; Gerhard-Herman, M.; Hughes, J.P.; Knight, S.; Marie Kupinski, A.; Mahe, G.; et al. Interpretation of Peripheral Arterial and Venous Doppler Waveforms: A Consensus Statement from the Society for Vascular Medicine and Society for Vascular Ultrasound. *Vasc. Med.* **2020**, 484–506. [CrossRef]

18. Lau, J.F.; Weinberg, M.D.; Olin, J.W. Peripheral Artery Disease. Part 1: Clinical Evaluation and Noninvasive Diagnosis. *Nat. Rev. Cardiol.* **2011**, *8*, 405–418. [CrossRef]

19. Scissons, R. Characterizing Triphasic, Biphasic, and Monophasic Doppler Waveforms: Should a Simple Task Be So Difficult? *J. Diagn. Med. Sonogr.* **2008**, *24*, 269–276. [CrossRef]

20. Scissons, R.; Comerota, A. Confusion of Peripheral Arterial Doppler Waveform Terminology. *J. Diagn. Med. Sonogr.* **2009**, *25*, 185–194. [CrossRef]

21. Omarjee, L.; Stivalet, O.; Hoffmann, C.; Scissons, R.; Bressollette, L.; Mahé, G.; Jaquinandi, V. Heterogeneity of Doppler Waveform Description Is Decreased with the Use of a Dedicated Classification. *Vasa* **2018**, *47*, 471–474. [CrossRef] [PubMed]

22. Yamashita, R.; Nishio, M.; Do, R.K.G.; Togashi, K. Convolutional Neural Networks: An Overview and Application in Radiology. *Insights Imaging* **2018**, *9*, 611–629. [CrossRef] [PubMed]

23. Ara, L.; Luo, X.; Sawchuk, A.; Rollins, D. Automate the Peripheral Arterial Disease Prediction in Lower Extremity Arterial Doppler Study Using Machine Learning and Neural Networks. In Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, Association for Computing Machinery, New York, NY, USA, 4 September 2019; pp. 130–135.
24. Jana, B.; Oswal, K.; Mitra, S.; Saha, G.; Banerjee, S. Detection of Peripheral Arterial Disease Using Doppler Spectrogram Based Expert System for Point-of-Care Applications. *Biomed. Signal Process. Control.* **2019**, *54*, 101599. [CrossRef]

25. Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. ImageNet Large Scale Visual Recognition Challenge. *Int. J. Comput. Vis.* **2015**, *115*, 211–252. [CrossRef]

26. Mahé, G.; Boulon, C.; Désormais, I.; Lacroix, P.; Bressollette, L.; Guilmot, J.L.; Le Hello, C.; Sevestre, M.A.; Pernod, G.; Constans, J.; et al. Recommandations du Collège des enseignants de médecine vasculaire (CEMV): Analyse des flux artériels Doppler (classification de Saint-Bonnet simplifiée). *JMV-J. Médecine Vasc.* **2018**, *43*, 255–261. [CrossRef]

27. Masters, D.; Luschi, C. Revisiting Small Batch Training for Deep Neural Networks. *arXiv* 2018, arXiv:1804.07612.

28. Wong, S.C.; Gatt, A.; Stamatescu, V.; McDonnell, M.D. Understanding Data Augmentation for Classification: When to Warp? In *Proceedings of the 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA)*, Gold Coast, QLD, Australia, 30 November–2 December 2016; pp. 1–6.

29. Cunningham, P.; Carney, J.; Jacob, S. Stability Problems with Artificial Neural Networks and the Ensemble Solution. *Artif. Intell. Med.* **2000**, *20*, 217–225. [CrossRef]

30. McHugh, M.L. Interrater Reliability: The Kappa Statistic. *Biochem. Med.* **2012**, *22*, 276–282. [CrossRef]