[image] [image] Ressource protégée - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu

[image] [image]

* Documentation

* FR
  Français (FR)

* Anglais (EN)
  Se connecter

* Portail HAL Inria

* Recherche
  Loading...

## Recherche avancée

Information de documents

Titres

○ Titres

○ Sous-titre

○ Titre de l'ouvrage

○ Titre du volume (Série)

○ Champ de recherche par défaut (multicritères) + texte intégral des PDF

○ Résumé

○ Texte intégral indexé des documents PDF

○ Mots-clés

○ Type de document

○ Sous-type de document

○ Tous les identifiants du document

○ Identifiant HAL du dépôt

○ Langue du document (texte)

○ Pays (Texte)

○ Ville

○ À paraître (true ou false)

Ajouter
<figure><img src="image_1.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
## Auteur

### Auteur (multicritères)

○ Auteur (multicritères)

○ Auteur : Nom complet

○ Auteur : Nom de famille

○ Auteur : Prénom

○ Auteur : Complément de nom, deuxième prénom

○ Auteur : Organisme payeur

○ Auteur : IdHal (chaîne de caractères)

○ Auteur : Fonction

○ Auteur : personID (entier)

○ Auteur : Identifiant de l'organisme payeur

○ Auteur : Identifiant de la structure

○ Directeur de thèse

○ Éditeur

○ Éditeur scientifique

○ Éditeur de la série

## Ajouter

### Structure

### Structure (multicritères)

○ Structure (multicritères)

○ Structure : Acronyme

○ Structure : Nom

○ Structure : Code

○ Structure : Pays

○ Structure : Type

○ Structure : État dans le référentiel

○ Structure : Identifiant HAL de la structure

○ Structure/équipe de recherche : Pays

○ Structure/regroupement d'équipes : Pays

○ Structure/laboratoire : Pays

○ Structure/regroupement de laboratoires : Pays

○ Structure/institution : Pays

○ Structure/regroupement d'institutions : Pays

## Ajouter
# Automated Annotation and Classification of Catheters in Chest X-rays

Akash K¹[0000-0002-1568-0155] and Saravana Perumaal S¹[0000-0002-6545-2783]

Department of Mechanical Engineering, Thiagarajar College of Engineering, Madurai
akashk1@tce.student.edu, sspmech@tce.edu

**Abstract.** Catheters are usually used to deliver drugs and medications close to the heart and to monitor the vital organs around the chest region for patients who undertook critical surgery. Radiologists often check for the presence of catheters, puncture-needles, guiding sheaths, and various other tube-like structures in interventional radiology. The clinical analysis of X-ray requires a manual pixel-wise annotation which is an excruciating process. In order to address this issue, we attempt to auto- annotate the CXRs using a Self-Supervised Learning approach. Further, the classification task on the catheter is performed based on semantic and perceptual clues (object shapes, colors, and their interactions) of color and class distributions. A generative adversarial network is utilized to learn a mapping to annotate (colorize and identify end-tip points) and classify the given grayscale CXR. The additional number of classes, custom loss function, and attention heads introduced in the model is a unique attempt to ensure robust results in the radiological inferences. It is evident that the qualitative and quantitative results of annotation and classification are viable which resembles how humans perceive such problems. The results are consistent and outperform's the state-of-the- art supervised learning models in terms of metrics and inference dura- tions. The model being end-to-end in nature, can be integrated along with the existing in-hospital pipeline and will be ready to use instantly.

**Keywords:** GAN · Chest X-rays · Self-Supervised Learning.

## 1 Introduction

Chest X-rays are commonly performed radiologic examination of the human body for patients kept under critical care. Portable anterior-posterior (AP) CXRs are often used to detect malpositions if any and verify the placement of catheters. These catheters are inserted through the subclavian or jugular veins and are typically blindly operated upon [12]. After placement Chest X-Rays (CXRs) are obtained to analyze their presence, and identity to avoid any mispositioning or other complications. Traditionally it requires years of training, experience, and skill-set to accomplish such a task. With the recent developments in computer vision and the advent of Artificial Intelligence, autonomous report generation in radiology will considerably expedite clinical workload, Such a system would be able to remotely generate reports for CXRs in a matter of seconds.
Autres

Champ de recherche par défaut (multicritères)

○ <u>Champ de recherche par défaut (multicritères)</u>

○

○ <u>Revues (multicritères)</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Titre abrégé</u>

○ <u>Revue : Titre</u>

○ <u>Revue : Identifiant interne</u>

○ <u>Revue : Date de début de publication</u>

○ <u>Revue : ISSN électronique</u>

○ <u>Revue : ISSN</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Couleur dans SHERPA/RoMEO</u>

○ <u>Revue : État dans le référentiel</u>

○

○ <u>Colloque (multicritères)</u>

○ <u>Colloque : Titre</u>

○ <u>Colloque : Organisateur</u>

○ <u>Colloque : date de début (Année)</u>

○ <u>Colloque : date de fin (Année)</u>

○ <u>Conférence invitée (oui/non)</u>

○

○ <u>Projets ANR (multicritères)</u>

○ <u>Projet ANR : Acronyme</u>

○ <u>Projet ANR : Acronyme du programme</u>

○ <u>Projet ANR : Code décision (référence)</u>

○ <u>Projet ANR : Nom</u>

○ <u>Projet ANR : Identifiant interne</u>

○ <u>Projet ANR : État dans le référentiel</u>

○

○ <u>Projets européens (multicritères)</u>

○ <u>Projet européen : Acronyme</u>

○ <u>Projet européen : Identifiant de l'appel à projet</u>

○ <u>Projet européen : Référence</u>

○ <u>Projet européen : Nom</u>

○ <u>Projet européen : Date de fin</u>

○ <u>Projet européen : Financement</u>

○ <u>Projet européen : Date de début</u>
<table><caption>**Table 1.** Distribution of the samples over various classes of catheters based on their position</caption><thead><tr><th colspan="2">Description</th><th>No. Samples</th></tr></thead><tbody><tr><td rowspan="3">Endo-Tracheal Tube</td><td>Abnormal</td><td>16</td></tr><tr><td>Borderline Abnormal</td><td>192</td></tr><tr><td>Normal</td><td>1423</td></tr><tr><td rowspan="4">Naso-Gastric Tube</td><td>Abnormal</td><td>61</td></tr><tr><td>Borderline Abnormal</td><td>95</td></tr><tr><td>Incompletely Imaged</td><td>507</td></tr><tr><td>Normal</td><td>887</td></tr><tr><td rowspan="3">Central venous catheter</td><td>Abnormal</td><td>640</td></tr><tr><td>Borderline Abnormal</td><td>1596</td></tr><tr><td>Normal</td><td>4220</td></tr><tr><td>Swan-Ganz</td><td>Presence of Swan-Ganz Catheter</td><td>139</td></tr><tr><td colspan="2">Total Classes: 11</td><td>Total : 30083</td></tr></tbody></table>

Previous attempts at producing such reports, under in-hospital conditions - lack the diagnostic interpretation and accuracy. The need for consistent and structured reports are imminent and plays a pivotal role in clinical care. Failure to report (multilabel classifications as shown in Table. 1, and corresponding problems based on their positioning and insertions if any) in a clear and concise manner reflects in sub-optimal care.

The popular approach involves, the use of Deep Convolutional models. To be able to learn unique discriminative spatio-temporal features for CXRs is a difficult task. Hence recognition and classification of catheters from direct whole images yield poor results, as these needle-like structures account for less than 1% of the footprint in the whole image [12]. It is evident from the Class Activation Map as shown in Figure. 1 of traditional convolutional models struggles with classification tasks due to overlap in receptive fields. Different classes of catheters arise, owing to the difference in tip locations, functions, and the target organ as shown in Table. 1

While conventional CNNs could be used for this task, they usually require a large amount of paired data which is to be manually annotated, yet will still suffer from class imbalances. Moreover, almost all the CNNs are used to 3-channel input rather than single-channel input in the case of CXRs limiting the use of ImageNet weights of state-of-the-art models.

This paper presents an approach to address the automatic classification of unla-beled samples and the classification of peripheral and central catheter positions through a GAN model that is based on semantic and perceptual (end-tip points, pins, object shapes, colors, and their interactions) nature. We also explore how colorization affects the class distribution in CXRs. The model is designed in an end-to-end fashion and trained in a weakly supervised manner. Initial tests in-
○ Projet européen : État dans le référentiel

○ Projet européen : Identifiant interne

○

○ Date de production : année

○ Date de mise en ligne : année

○ Date de publication : année

○ Date d'écriture : année

○ Date de modification du dépôt : année

○ Date de dépôt : année

○ Date de publication électronique : année

○

○ Collection HAL (multicritères)

○ Collection HAL : catégorie

○ Collection HAL : Code

○ Collection HAL : Nom

○ Collection HAL : Identifiant interne

○

○ Identifiant interne du contributeur/déposant

○ Nom complet du contributeur/déposant

○

○ Domaines

○ Domaine primaire

○ Domaine racine

○ Sous-domaine niveau 1

○ Sous-domaine niveau 2

○ Sous-domaine niveau 3

○

○ Statut du document

○ Version du document

○ Type de dépôt

○ Type de document

○ ISBN

○ Numéro - référence

○ Identifiant DOI

○

○ Classification

○ Audience

○ Vulgarisation

○ Comité de lecture - texte (oui ou non)

○ Actes de colloque
<figure><img src="image_2.png" /><figcaption>Fig. 1. Class Activation Maps of a multi-labelled sample, with logits</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Fig. 2. Example of annotated sample, Multi-labeled and annotated sample</figcaption></figure>

volving perpetual cues provided good initializations. In particular, adding color
to the classification task leads to better initializations and forces the model to
learn proper feature representations as seen in warm-starting of ImageNet mod-
els for various transfer learning applications.

1.1 Related Work

The absence of colors being a major reason for the failure of ImageNet mod-
els in CXRs. Colors also seemed to add class-specific semantic clues which are
known to boost performance. Zhang et al. proposed an automatic colorization al-
gorithm where colorization is treated as a multinomial classification problem [16]
proposed a colorization model that learns to map a monochromatic image along
with user-provided hints and cues that later is used to fuse along with high-level
○ Référence interne

○ Financement

○ Collaborations

Ajouter

Lancer la recherche
Recherche experte (SolR)

Recherche experte (SolR)

Lancer la recherche
Vers la recherche avancée

• Déposer

Portail INRIA.HAL.SCIENCE
Les publications, logiciels... des scientifiques Inria

• Accueil

• Publications Inria

○ Consulter les derniers dépôts

○ Par équipe de recherche

○ Par auteur

○ Rapports de recherche et techniques

○ Livres blancs

○ Rapports d'activité des équipes

○ Statistiques

• Boite à Outils

○ X2Hal : importer par lot

○ Logiciels : créer le Codemeta.json

○ Créer mon IDHal

○ Créer mon CVHal

○ Haltools : créer ma page de publications

○ Haltools : extraire des données (réservé Inria)

○ Les bonnes pratiques

■ Comment bien s'affilier ?
semantic information to provide realistic outputs. These processes again make
the task multistage or involve humans in a loop making it difficult to build a
standalone system.

"Generative Adversarial Networks (GANs) have been a popular choice to learn a mapping between gray-scale photographs. In recent times, Wasserstein GANs have been used with the gradient penalty to generate paired image-mask samples from CXRs" [10]. Authors of [2] proposed to use such generated masks to be paired along with real-time images to train in a supervised manner. A few methods were employed for non-paired data generation of medical and clinical images, entailing these data from an auxiliary imaging modality in a domain adaptation setting [8]. Performance evaluation of these techniques has not been evaluated. The author [6] proposed the "use of conditioned GANs to map the images using U-Net-based architecture. They adopted Dice Loss and Wasserstein loss to generalize the same to high-resolution images, stabilize and converge faster" [9].

Notice that GANs were essentially utilized to generate synthetic data which is used to provide better domain adaptation in various transfer learning tasks utilizing deep convolutional networks. Upon using a shared module trained in a joint manner (Semantic annotation and Classification) in an adversarial method yields better results and reduced inference period, as these learned representations aid in the classification as well as a form of additional supervision.

2 Method

2.1 Annotation-GAN

When the gray-scale image *L* which resembles the CXR, the model aims to
learn a mapping *G*: *L* → *c* such that *I* = (*L*, *c*) which represent a probable color
distribution, and π(*z*) denotes real image distribution. A bijective mapping such
that *c* = *G*⁻¹(*z*; *L*) is near the ground truth values is learned. Thus, the generated
annotated sample is expected to encompass semantic, perceptual, and geometric
characters to that of the ground truth image.

As in [14] a generator module learns a function *G* which is also invertible in
nature and in an end-to-end fashion. While parallelly a discriminator module
validates the similarity between the colorized output to actual ground truth.
*I* = (*L*, *c*) of *L*. The GAN model tries to tune the *Ψ* and *w*, parameters of the
model. This ensures a conditioned GAN trained in a weakly-supervised nature
using the annotated samples, with model initialized with weights from ImageNet
[4], Especially when using an annotated sample *I*<sub>*z*</sub> = (*L*, *c*), the model learns the
color information in a split fashion : The mono-chromatic channel *L* and the
chrominance channels (*c*) similar to the method in [14].

The generator $G_{\Psi}$ also tries to learn a $[nx1]$ classification matrix. The advantage of using such an approach enables us to learn the perpetual and semantic image distribution contained in $L$ that makes the model similar to how humans perceive color. Let us subnets of generator be defined as follows, $G_{\Psi_1}: L \rightarrow (c)$, and $G_{\Psi_2}: L \rightarrow y$.
* Quelle version puis-je déposer ?

* Choisir le bon « type de document »

* Mettre en valeur ses publications

○ Les tutos IES Inria

* Lier les notices dans HAL

* Umbrhal Inria pour l'accessibilité

* Personnaliser son CVHAL

* Créer et Personnaliser sa collection

• Documentation

○ Aide en ligne HAL

○ Aide en ligne Haltools

○ Aide en ligne X2hal

○ Tutoriels video HAL

○ Documentation API HAL

○ Ajouter des vignettes

○ Identifiant auteur IdHAL et CV

○ Compléter son dossier ORCID avec ses dépôts HAL

○ Inria et la Science Ouverte

X

X

X

Loading...

X

Le document que vous demandez est en access restreint
jusqu'au 2025-01-01 !

Revenir a la notice

Retourner à l'accueil

Contact

•

Ressources
<figure><img src="image_4.png" /><figcaption>Fig. 3. Auto-Annotation GAN can colorize the tubular structures of catheters in CXR's based on different classes, It is a combination of Discriminator Network, $D_A$ in pink, and the Generator Network, $G_\Psi$ which consist of two-subnets: $G_{\Psi_1}^1$ (yellow, aqua-green, purple and gray layers) and $G_{\Psi_2}^2$ (yellow, purple and gray layers).</figcaption></figure>

## 2.2 Objective Formulation

The error function is given by:

$$L(G_\Psi, D_A) = L_\epsilon(G_{\Psi_1}^1) + \lambda_g L_g(G_{\Psi_1}^1, D_A) + \lambda_s L_s(G_{\Psi_2}^2) \quad (1)$$

The initial term represents **reconstruction loss** which is defined as follows:

$$L_{\epsilon}(G_{\Psi_1}^{1}) = E_{(L,c)\sim P_c}[-\log(q)(L, c')||G_{\Psi_1}^{1}(L) - c||_2^2] \quad (2)$$

where, $P_c$ represents the distribution of ground truth images, the initial part represents negative log-likelihood loss added along with $|| \cdot ||_2$ for the euclidean norm to retain the structural similarity in the image and prevent from losing data. To improve the sensitivity to perpetual color, we use **class distribution loss**

$$L_s(G_{\Psi_2}^2) = E_{L \sim P_z}[KL(y_v||G_{\Psi_2}^2(L))] \quad (3)$$

Herein, $P_z$ represents the distribution of CXR input, and $y_v \in labels$ as in Table.1 the softmax output is obtained from VGG-16 classification branch [11]. $KL(\cdot||\cdot)$ represents the Kullback-Leibler divergence. The $L_g$ denotes the **WGAN loss** which represents Wasserstein GAN [1]. WGAN provides a few niceties like avoiding vanishing gradient issues, preventing mode collapse, and faster convergence.
* Documentation

* FAQ

* API

* OAI-PMH

* AuréHAL

## Informations

* À propos

* Données personnelles

* Mentions légales

* Accessibilité

* Conformité RGAA

## Questions juridiques

* Je publie, quels sont mes droits ?

* Loi pour une République numérique

* Stratégie de non-cession des droits

* SHERPA/RoMEO

## Portails

* Portails institutionnels

* HAL

* HAL SHS

* HAL Thèses

* MédiHAL

## CCSD

* CCSD

* Episciences

* Sciencesconf
$$L_g(G_{\Psi_1}^1, D_\Lambda) = E_{L' \sim P_c}[D_\Lambda(L')] - E_{(c) \sim P_{G_{\Psi_1}^1}}[D_\Lambda(c, L)] \\ - E_{L' \sim P_i}[(\|\nabla_i D_\Lambda(\hat{I})\|_2 - 1)^2] \quad (4)$$

with $P_{G_{\Psi_1}^1}$ representing the probability density of the model $G_{\Psi_1}^1(L)$ denoting the generator distribution, with $L \sim P_z$. $P_{L'}$ sampled uniformly along straight lines between pairs of points from data distribution $P_c$ and $P_{G_{\Psi_1}^1}$. The negative sign in (Eq. 4) ensures that the model minimizes the loss with respect to discriminator parameters.

This now reduces to a min-max problem that tries to converge in an alternate fashion by adjusting the weights of generator and discriminator modules. Later in the ablation experiments, it was also observed that the reconstruction loss enables quicker convergence.

$$\min_{G_{\Psi}, D_{\Lambda} \in D} \max_{L} L(G_{\Psi}, D_{\Lambda}) \quad (5)$$

**Conditional GAN Loss and adversarial strategy $L_g$.** The general adversarial min-max game between a generator and discriminator deals with learning suitable parameters so that the generator mimics the probability distribution of real data.

The WGAN is a better alternative to the non-overlapping supports approach. In contrast, KL divergence causes a vanishing gradient problem. The JS divergence may be non-continuous with parameters.

The results for objective formulation were in agreement with [7]. This approach outperforms the colorization produced by only $L_2$ or $L_1$ color loss terms and prevents model collapse.

Learning per-pixel probability distribution allows the use of a variety of classification losses. **Classification Loss.** The $L_s(G_{\Psi_1}^1)$ Eq. 3 tries to minimize the difference between generated data with respect to actual distribution, to accomplish that we adapt convolutional layers from VGG-16 model pre-trained on ImageNet dataset. So to make the input competent with the model we create copies of the grayscale channel and reshape the input as $(L, L, L)$ such that it could be used to generate the density distribution as stated earlier.

## 2.3 Model Architecture

The model architecture is as shown in **Figure. 3**. It is comprised of 3 sub-parts. Two of those focusing on chrominance information and classification are the same. The last one belongs to the discriminator network which to distinguishes ground truth from synthetic data.
**Generator Architecture $G_\Psi$.** The generator subnetworks $G_{\Psi_1}^1$ and $G_{\Psi_2}^2$ outputs chrominance related data, ($c$) = $G_{\Psi_1}^1(L)$ and classification vectors, $y$ = $G_{\Psi_2}^2(L)$. Both the subnetworks are trained jointly through a single step backprop as in proposed in [14].

The basic blocks responsible for global feature extraction are shared by both subnets. Which is initialized with pre-trained ImageNet weights.

The first subnetwork (displayed aqua-green in Figure. 3) proceeds with a form of Convolutional(3x3)-BatchNorm-AReLU to learn $G_{\Psi_1}^1$ and similarly the second subnetwork (displayed purple in Figure. 3) learns the $G_{\Psi_2}^2$ using four modules of form Convolutional(1x1)-BatchNorm-AReLU, sufficed by fully connected layers (in purple) providing us the classification vector.

The first stage results (displayed yellow in Figure. 3) are shared to both the subnetworks and has the same architecture of VGG-16 [17] and are initialized with ImageNet weights [17]. Once $G_{\Psi_1}^1$ is learned it is used to generate useful information to help the colorization process. Later these two subnetworks are fused and are used to predict ($c$) by up-sampling with the help of Convolution-AReLU layers. The class distribution loss Eq. 3 only subnet $G_{\Psi_2}^2$ is affected. Whereas the color error loss Eq. 1 affects the whole network. We use AReLU [3] as activation for all layers and use a Softmax in the final layer to obtain the logits distribution. The AReLU enables a learnable activation function and formulates an element wise attention mechanism. The attention map forwards scaled positive elements which enable us to capture discriminative features amongst different classes. This amplifies positive elements and suppresses the negative ones.

**Discriminator Architecture $D_\Lambda$.** The discriminator is adapted from PatchGAN [7] a markovian discriminator. It operates in a sliding window fashion instead of considering the CXR as a whole to focus on local patches. Instead of classifying a whole image the discriminator convolutionally runs across the image and provides the average of those responses.

# 3 Experimentation and Results

Quantitatively studying the effect of colorization is a difficult task so in order to analyze how each term in the loss function affects the model, we can perform an ablation study and evaluate the different variants. GAN model using adversarial learning and classification approach and GAN w/o class [$\lambda_s$ = 0]. We also compare the results with EfficientNet backbones and Knowledge Distilled models for endpoint detection and classification with custom loss function as shown in Figure. 4.

## 3.1 Dataset

We took the RANZCR CLIP dataset which is in-turn built upon Chest X Ray14 dataset [15], one of the largest publicly available datasets, where 30000 labeled (but non- annotated) CXRs spread across 11 different classes as in Table. 1 were
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Teacher - Student model with catheter endpoint heatmap, Source: kaggle</figcaption></figure>

taken into consideration. A subset of these samples, around 950 were manually annotated with catheters and tube positions as shown in Figure. 2 to indicate pixels that belong to catheters or tubes. The class wise distribution of the labeled samples are as shown in Table. 1. The CXRs are of varied resolutions (> 2048 × 2048). We resize these samples to 1024 × 1024 by means of learned image resizing technique [13]. Replacing the typical linear resizers with learned resizers can substantially improve the results as they produce machine friendly visual manipulations. This resizer model is trained jointly with the classifier subnet model.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Segemeted results show the presence of a multi-labeled sample (i.e. ETT - Abnormal, ETT - Borderline positive sample), Image to the left represents un-annotated sample and the one on right represents auto-annotation</figcaption></figure>
## 3.2 Implementation Details

<table><caption>**Table 2.** Optimal Training Parameters and Hyper-parameters. Note that first and last layers consist of 7 × 7 kernels. With a larger kernel size allows more receptive fields, it is also verified from [13] and provides about 1-2 % increase in accuracy. We also use batch normalization layers followed by AReLU activations[3].</caption><tr><td>Optimizer</td></tr><tr><td>GAN: AdamW 1e-5</td></tr><tr><td>Hyperparameter</td></tr><tr><td>λ<sub>g</sub> = 0.1 &nbsp;&nbsp;&nbsp;&nbsp; λ<sub>s</sub> = 0.003</td></tr><tr><td>Activation</td></tr><tr><td>GAN: ReLU &nbsp;&nbsp;&nbsp;&nbsp; G<sub>ψ<sub>2</sub><sup>2</sup></sub>: AReLU</td></tr><tr><td>Kernel</td></tr><tr><td>n = 16 &nbsp;&nbsp;&nbsp;&nbsp; 3 × 3</td></tr><tr><td>Device: NVIDIA P100</td></tr><tr><td>Batch-Size: 10</td></tr></table>

We trained the Annotation - GAN Model with about 950 annotated CXRs resized to 2048 × 2048 with parameters as in Table. 2. A single epoch took about 5 hours to train. We had to retain the original image size to make the annotations less ambiguous. The inference phase takes around 2 seconds Figure. 5. We minimize the objective loss using AdamW optimizer with the parameters as given in Table. 2. The intensity regularization makes the model develop a mean color response over the regions of the catheter and tubes. For reliable evaluation of the data, we took a 5-fold cross-validation using F1 scores, pixel-wise precision, and recall metrics Table. 3, owing to the thin structures of the catheters and tubes we enlarged the manually annotated samples to a 5-pixel dilation radius which resulted in 70 % cases with over 50% of overlap.

Metrics such as Structural Similarity Index (SSIM) check for variation in contrast, luminance in high frequency region. Similarly, the Hausdorff distance measures the closeness to the ground truth thus representing the resemblance amongst the images.

<table><caption>**Table 3.** Evaluation of Auto-Annotated Samples with respect to ground truth annotation. Note these are the results of auto-annotated data averaged over a batchsize of 10.</caption><thead><tr><th>Data</th><th>SSIM</th><th>PSNR</th><th>Hausdorf</th></tr></thead><tbody><tr><td>Auto - Annotated CXR</td><td>0.98184</td><td>28.5766</td><td>4 ± 0.3</td></tr><tr><td>GANw/oclass</td><td>0.99411</td><td>33.9304</td><td>4 ± 0.5</td></tr></tbody></table>
<figure><img src="image_7.png" /><figcaption>**Fig. 6.** Results of the Annotation GAN for each class. Mean values of the same are as shown in Table. 4</figcaption></figure>

## 3.3 Limitations

While the model offers quick inference and has a higher potential to improve workflow efficiency, there are a few drawbacks. The system confuses the catheters with tips with bone edges, often around the ribs. This can be avoided by adding additional background annotation classes (i.e., ribs, collar bones) and retraining the network so that the model will learn a differential feature to separate out the various classes.

The class imbalance can be mitigated via class frequency weights as an additional loss term. Further data augmentation techniques can be studied to provide better initializations to the model and prevent the mixing of background with labeled data.

## 3.4 Future Scope and Direction

Chest radiography analysis is an important stage in post-surgical care it involves detection and identification of tip locations, endpoints of catheters. The proposed
<table><caption>**Table 4.** Results of the catheter classification (Mean ± standard deviation) AUC:Area under ROC, Acc: Accuracy, P: Precision, R: Recall). MTSS: Multi-Teacher Single Student trained and distilled network model with EfficientNet backbone.</caption><thead><tr><th>Model</th><th>AUC</th><th>Accuracy</th><th>P</th><th>R</th><th>F1</th></tr></thead><tbody><tr><td>Resnet200D*</td><td>0.884 ± 0.055</td><td>0.822</td><td>0.33</td><td>0.27</td><td>0.61</td></tr><tr><td>EfficientNet* - B0*</td><td>0.883 ± 0.055</td><td>0.732</td><td>0.4</td><td>0.39</td><td>0.62</td></tr><tr><td>EfficientNet - MTSS</td><td>0.917 ± 0.005</td><td>0.903</td><td>0.50</td><td>0.55</td><td>0.65</td></tr><tr><td>MoCo [5]</td><td>0.815 ± 0.5</td><td>0.711</td><td>0.56</td><td>0.58</td><td>0.51</td></tr><tr><td>AnnotationGAN</td><td>0.969 ± 0.005</td><td>0.952</td><td>0.77</td><td>0.68</td><td>0.73</td></tr></tbody></table>

model is capable of detecting tip locations of various different catheters and the inference takes about 2s this could accelerate the mundane but essential tasks required in the health care system.

Our initial target was to detect catheter endpoints and reduce the false negatives, such as to make the solution viable in real-time applications. However, as a part of distinguishing the catheters from one another, it will work to our benefit if we could capture all the tubular structures present in the given sample. This also aids the clinical setting without having to redesign the system to incorporate the model. The current approach involves analyzing the CXR from a single viewpoint. It could benefit us to analyze both frontal and lateral views for detecting any abnormalities.

## 4 Conclusion

We propose an autonomous report generation system that can autonomously classify and identify unlabeled samples and provide reports for central and peripheral catheter positions (Table. 4, Figure. 6). Experiments show that our end-to-end model performs equivalent to the state-of-the-art models even though it was trained 0.033% (950 annotated samples) of labeled data. Auto-annotation GAN can further be explored with the help of shape constraints and incorporating spatial priors to improvise the results. Future prospects include autonomous report generation and validation by various trials, which results in robust reporting with minimal costs which is well under regulatory requirements. However, the proposed approach could be expanded scope for similar tasks in ophthalmology, 3D volume colorization, motion forecasting or satellite image analysis, and beyond.

## References

1. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein gan. ArXiv abs/1701.07875 (2017)

2. Bailo, O., Ham, D., Shin, Y.M.: Red blood cell image generation for data augmentation using conditional generative adversarial networks. In: 2019 IEEE/CVF
Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). pp. 1039–1048 (2019). https://doi.org/10.1109/CVPRW.2019.00136

3. Chen, D., Xu, K.: Arelu: Attention-based rectified linear unit. ArXiv abs/2006.13858 (2020)

4. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848

5. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning (2020)

6. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 5967–5976 (2017). https://doi.org/10.1109/CVPR.2017.632

7. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. CVPR (2017)

8. Lei, Y., Wang, T., Tian, S., Fu, Y., Patel, P., Jani, A.B., Curran, W.J., Liu, T., Yang, X.: Male pelvic CT multi-organ segmentation using synthetic MRI-aided dual pyramid networks. Physics in Medicine & Biology 66(8), 085007 (apr 2021). https://doi.org/10.1088/1361-6560/abf2f9, https://doi.org/10.1088/1361-6560/abf2f9

9. Nazeri, K., Ng, E., Ebrahimi, M.: Image colorization using generative adversarial networks. In: International Conference on Articulated Motion and Deformable Objects. pp. 85–94. Springer (2018)

10. Neff, T.: Data augmentation in deep learning using generative adversarial networks. Ph.D. thesis, Master’s thesis, Graz University of Technology. https://www.tugraz.at ... (2018)

11. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2015)

12. Subramanian, V., Wang, H., Wu, J.T., Wong, K.C.L., Sharma, A., Syed-Mahmood, T.: Automated detection and type classification of central venous catheters in chest x-rays. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. pp. 522–530. Springer International Publishing, Cham (2019)

13. Talebi, H., Milanfar, P.: Learning to resize images for computer vision tasks (2021)

14. Vitoria, P., Raad, L., Ballester, C.: Chromagan: Adversarial picture colorization with semantic class distribution. In: The IEEE Winter Conference on Applications of Computer Vision. pp. 2445–2454 (2020)

15. Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 3462–3471 (2017)

16. Zhang, R., Zhu, J.Y., Isola, P., Geng, X., Lin, A.S., Yu, T., Efros, A.A.: Real-time user-guided image colorization with learned deep priors. ACM Transactions on Graphics (TOG) 9(4) (2017)

17. Zheng, H., Fu, J., Mei, T., Luo, J.: Learning multi-attention convolutional neural network for fine-grained image recognition. In: 2017 IEEE International Conference on Computer Vision (ICCV). pp. 5219–5227 (2017). https://doi.org/10.1109/ICCV.2017.557