# Per-COVID-19: A Benchmark Dataset for COVID-19 Percentage Estimation from CT-Scans

Fares Bougourzi, Cosimo Distante, Abdelkrim Ouafi, Fadi Dornaika, Abdenour Hadid, Abdelmalik Taleb-Ahmed

► To cite this version:

Fares Bougourzi, Cosimo Distante, Abdelkrim Ouafi, Fadi Dornaika, Abdenour Hadid, et al. Per-COVID-19: A Benchmark Dataset for COVID-19 Percentage Estimation from CT-Scans. Journal of Imaging, 2021, 7 (9), pp.189. 10.3390/jimaging7090189. hal-03429748

HAL Id: hal-03429748

https://hal.science/hal-03429748

Submitted on 12 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Per-COVID-19: A Benchmark Dataset for COVID-19 Percentage Estimation from CT-Scans

Fares Bougourzi, Cosimo Distante, Abdelkrim Ouafi, Fadi Dornaika, Abdenour Hadid, Abdelmalik Taleb-Ahmed

► To cite this version:

Fares Bougourzi, Cosimo Distante, Abdelkrim Ouafi, Fadi Dornaika, Abdenour Hadid, et al. Per-COVID-19: A Benchmark Dataset for COVID-19 Percentage Estimation from CT-Scans. Journal of Imaging, 2021, 7 (9), pp.189. 10.3390/jimaging7090189. hal-03429748

HAL Id: hal-03429748

https://hal.science/hal-03429748v1

Submitted on 12 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Article

# Per-COVID-19: A Benchmark Dataset for COVID-19 Percentage Estimation from CT-Scans

Fares Bougourzi <fnref n="1" />, Cosimo Distante <fnref n="1" />,* , Abdelkrim Ouafi <fnref n="2" />, Fadi Dornaika <fnref n="3" />,<fnref n="4" /> , Abdenour Hadid <fnref n="5" /> and Abdelmalik Taleb-Ahmed <fnref n="5" />

<figure><img src="image_2.png" /></figure>

check for
updates

**Citation:** Bougourzi, F.; Distante, C.; Ouafi, A.; Dornaika, F.; Hadid, A.; Taleb-Ahmed, A. Per-COVID-19: A Benchmark Dataset for COVID-19 Percentage Estimation from CT-Scans. *J. Imaging* **2021**, *7*, 189. https://doi.org/10.3390/jimaging7090189

Academic Editors: Yudong Zhang, Juan Manuel Gorriz and Zhengchao Dong

Received: 13 August 2021

Accepted: 14 September 2021

Published: 18 September 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_3.png" /></figure>

**Copyright:** © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).

**Abstract:** COVID-19 infection recognition is a very important step in the fight against the COVID-19 pandemic. In fact, many methods have been used to recognize COVID-19 infection including Reverse Transcription Polymerase Chain Reaction (RT-PCR), X-ray scan, and Computed Tomography scan (CT- scan). In addition to the recognition of the COVID-19 infection, CT scans can provide more important information about the evolution of this disease and its severity. With the extensive number of COVID-19 infections, estimating the COVID-19 percentage can help the intensive care to free up the resuscitation beds for the critical cases and follow other protocol for less severity cases. In this paper, we introduce COVID-19 percentage estimation dataset from CT-scans, where the labeling process was accomplished by two expert radiologists. Moreover, we evaluate the performance of three Convolutional Neural Network (CNN) architectures: ResneXt-50, Densenet-161, and Inception-v3. For the three CNN architectures, we use two loss functions: MSE and Dynamic Huber. In addition, two pretrained scenarios are investigated (ImageNet pretrained models and pretrained models using X-ray data). The evaluated approaches achieved promising results on the estimation of COVID-19 infection. Inception-v3 using Dynamic Huber loss function and pretrained models using X-ray data achieved the best performance for slice-level results: 0.9365, 5.10, and 9.25 for Pearson Correlation coefficient (PC), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE), respectively. On the other hand, the same approach achieved 0.9603, 4.01, and 6.79 for *PC<sub>subj</sub>*, *MAE<sub>subj</sub>*, and *RMSE<sub>subj</sub>* respectively, for subject-level results. These results prove that using CNN architectures can provide accurate and fast solution to estimate the COVID-19 infection percentage for monitoring the evolution of the patient state.

**Keywords:** COVID-19; deep learning; convolutional neural network; CT-scans; dataset generation

1. Introduction

Since the end of 2019, the World has faced a health crisis because of the COVID-19 pandemic. The crisis has influenced many aspects of human life. To save the infected persons' lives and stop the spread of COVID-19, many methods have been used to recognize the infected persons. These methods include Reverse Transcription Polymerase Chain Reaction (RT-PCR) [1], X-ray scan [2-4], and CT-scan [5,6]. Despite the fact that the RT-PCR test is considered as the global standard method for COVID-19 diagnosis, this method has many downsides [7,8]. In detail, the RT-PCR test is time-consuming, expensive, and has a considerable False-Negative Rate [1]. Using X-ray scan and CT-scan methods can replace RT-PCR test and give an efficient result in both time and accuracy [2,8]. However, both of these methods need an expert radiologist to identify COVID-19

<footnote n="1">¹ Institute of Applied Sciences and Intelligent Systems, National Research Council of Italy, 73100 Lecce, Italy; fares.bougourzi@isasi.cnr.it</footnote>

<footnote n="2">² Laboratory of LESIA, University of Biskra, Biskra 7000, Algeria; ou_karim@yahoo.fr</footnote>

<footnote n="3">³ University of the Basque Country UPV/EHU, 20018 San Sebastian, Spain; fadi.dornaika@ehu.eus</footnote>

<footnote n="4">⁴ IKERBASQUE, Basque Foundation for Science, 48009 Bilbao, Spain</footnote>

<footnote n="5">5 University Polytechnique Hauts-de-France, University Lille, CNRS, Centrale Lille, UMR 8520-IEMN, F-59313 Valenciennes, France; abdenour.hadid@ieee.org (A.H.); Abdelmalik.Taleb-Ahmed@uphf.fr (A.T.-A.)</footnote>

<footnote>* Correspondence: cosimo.distante@cnr.it; Tel.: +39-0832-1975300</footnote>
<figure><img src="image_2.png" /></figure>

Article

# Per-COVID-19: A Benchmark Dataset for COVID-19 Percentage Estimation from CT-Scans

Fares Bougourzi <fnref n="1" />, Cosimo Distante <fnref n="1" />,* , Abdelkrim Ouafi <fnref n="2" />, Fadi Dornaika <fnref n="3" />,<fnref n="4" /> , Abdenour Hadid <fnref n="5" /> and Abdelmalik Taleb-Ahmed <fnref n="5" />

<figure><img src="image_3.png" /></figure>

check for
updates

**Citation:** Bougourzi, F.; Distante, C.; Ouafi, A.; Dornaika, F.; Hadid, A.; Taleb-Ahmed, A. Per-COVID-19: A Benchmark Dataset for COVID-19 Percentage Estimation from CT-Scans. *J. Imaging* **2021**, *7*, 189. https://doi.org/10.3390/jimaging7090189

Academic Editors: Yudong Zhang, Juan Manuel Gorriz and Zhengchao Dong

Received: 13 August 2021

Accepted: 14 September 2021

Published: 18 September 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_4.png" /></figure>

Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).

**Abstract:** COVID-19 infection recognition is a very important step in the fight against the COVID-19 pandemic. In fact, many methods have been used to recognize COVID-19 infection including Reverse Transcription Polymerase Chain Reaction (RT-PCR), X-ray scan, and Computed Tomography scan (CT- scan). In addition to the recognition of the COVID-19 infection, CT scans can provide more important information about the evolution of this disease and its severity. With the extensive number of COVID-19 infections, estimating the COVID-19 percentage can help the intensive care to free up the resuscitation beds for the critical cases and follow other protocol for less severity cases. In this paper, we introduce COVID-19 percentage estimation dataset from CT-scans, where the labeling process was accomplished by two expert radiologists. Moreover, we evaluate the performance of three Convolutional Neural Network (CNN) architectures: ResneXt-50, Densenet-161, and Inception-v3. For the three CNN architectures, we use two loss functions: MSE and Dynamic Huber. In addition, two pretrained scenarios are investigated (ImageNet pretrained models and pretrained models using X-ray data). The evaluated approaches achieved promising results on the estimation of COVID-19 infection. Inception-v3 using Dynamic Huber loss function and pretrained models using X-ray data achieved the best performance for slice-level results: 0.9365, 5.10, and 9.25 for Pearson Correlation coefficient (PC), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE), respectively. On the other hand, the same approach achieved 0.9603, 4.01, and 6.79 for *PC<sub>subj</sub>*, *MAE<sub>subj</sub>*, and *RMSE<sub>subj</sub>* respectively, for subject-level results. These results prove that using CNN architectures can provide accurate and fast solution to estimate the COVID-19 infection percentage for monitoring the evolution of the patient state.

**Keywords:** COVID-19; deep learning; convolutional neural network; CT-scans; dataset generation

1. Introduction

Since the end of 2019, the World has faced a health crisis because of the COVID-19 pandemic. The crisis has influenced many aspects of human life. To save the infected persons' lives and stop the spread of COVID-19, many methods have been used to recognize the infected persons. These methods include Reverse Transcription Polymerase Chain Reaction (RT-PCR) [1], X-ray scan [2-4], and CT-scan [5,6]. Despite the fact that the RT-PCR test is considered as the global standard method for COVID-19 diagnosis, this method has many downsides [7,8]. In detail, the RT-PCR test is time-consuming, expensive, and has a considerable False-Negative Rate [1]. Using X-ray scan and CT-scan methods can replace RT-PCR test and give an efficient result in both time and accuracy [2,8]. However, both of these methods need an expert radiologist to identify COVID-19

<footnote n="1">¹ Institute of Applied Sciences and Intelligent Systems, National Research Council of Italy, 73100 Lecce, Italy; fares.bougourzi@isasi.cnr.it</footnote>

<footnote n="2">² Laboratory of LESIA, University of Biskra, Biskra 7000, Algeria; ou_karim@yahoo.fr</footnote>

<footnote n="3">³ University of the Basque Country UPV/EHU, 20018 San Sebastian, Spain; fadi.dornaika@ehu.eus</footnote>

<footnote n="4">⁴ IKERBASQUE, Basque Foundation for Science, 48009 Bilbao, Spain</footnote>

<footnote n="5">5 University Polytechnique Hauts-de-France, University Lille, CNRS, Centrale Lille, UMR 8520-IEMN, F-59313 Valenciennes, France; abdenour.hadid@ieee.org (A.H.); Abdelmalik.Taleb-Ahmed@uphf.fr (A.T.-A.)</footnote>

<footnote>* Correspondence: cosimo.distante@cnr.it; Tel.: +39-0832-1975300</footnote>
infection. Artificial Intelligence (AI) can provide the right solution to make this process automatic and limit the need of the radiologist to recognize the COVID-19 infection from these medical imaging. Indeed, computer vision and machine learning communities have proposed many algorithms and frameworks which have proved their efficiency on this task. Especially by using deep learning methods which already have proved their efficiency on different computer vision tasks [9] including medical imaging tasks [10,11].

Compared with the other two aforementioned diagnosis methods, the CT scan method has many advantages, as shown in Table 1. In addition to the use of CT scans to recognize COVID-19 infection, they can be used for other important tasks, which include quantifying the infection and monitoring the evolution of the disease, which can help with treatment and save the patient's life [12]. Moreover, the evolution stage can be recognized, where the typical signs of COVID-19 infection could be ground-glass opacity (GGO) in the early stage, and pulmonary consolidation in the late stage [7,8]. According to the estimated COVID-19 infection percentage from the CT-scans, the patient state can be classified into Normal (0%), Minimal (<10%), Moderate (10-25%), Extent (25-50%), Severe (50-75%), and Critical (>75%) [13].

<table><caption>**Table 1.** COVID-19 recognition methods with pros and cons for each method.</caption><thead><tr><td>COVID-19 Recognition Method</td><td>Pros</td><td>Cons</td></tr></thead><tbody><tr><td>RT-PCR</td><td>The most used method</td><td>Requires specific equipment, Time-consuming, Expensive, Considerable False-Negative Rate</td></tr><tr><td>X-ray scans</td><td>Low cost, Available in most Hospitals, fast acquisition time</td><td>Require an experienced Radiologist</td></tr><tr><td>CT scans</td><td>Available in most Hospitals, Very accurate recognition, Can be used for monitoring the evolution of the infection, Useful for Infection Severity Prediction</td><td>Require an experienced Radiologist, Expensive, Medium acquisition time</td></tr></tbody></table>

Most of the state-of-the-art methods have been concentrating on the recognition of COVID-19 from the CT scans or segmentation of the infected regions. Despite the huge efforts that have been made, the state-of-the-art methods have not provided many helpful tools to monitor the patient state, the evolution of the infection, or the response of patient to the treatment, which can play a crucial role in saving the patient's life. In this paper, we propose a fully automatic approach to evaluate the evolution of COVID-19 infection from the CT scans as regression task which can provide a richer information about the COVID-19 infection evolution. The estimation of COVID-19 percentage can help intensive care workers to identify the patients that need urgent care, especially the critical and severe cases. With the extensive number of COVID-19 infections, estimating the COVID-19 percentage can help intensive care workers free up resuscitation beds for the critical cases and follow other protocol for less severe cases.

Unlike the mainstream that dealt with COVID-19 recognition and segmentation, this paper addresses the estimation of COVID-19 infection percentage. To this end, we constructed the Per-COVID-19 dataset, then we used it to evaluate the performance of three CNN architectures with two loss functions and two pretrained models scenarios. In summary, the main contributions of this paper are as follows:

* We introduce the Per-COVID-19 dataset for estimating the COVID-19 infection percentage for both slice-level and patient-level. The constructed dataset consists of 183 CT scans with the corresponding slice-level COVID-19 infection percentage which were estimated by two expert radiologists. To the best of our knowledge, our work
infection. Artificial Intelligence (AI) can provide the right solution to make this process automatic and limit the need of the radiologist to recognize the COVID-19 infection from these medical imaging. Indeed, computer vision and machine learning communities have proposed many algorithms and frameworks which have proved their efficiency on this task. Especially by using deep learning methods which already have proved their efficiency on different computer vision tasks [9] including medical imaging tasks [10,11].

Compared with the other two aforementioned diagnosis methods, the CT scan method has many advantages, as shown in Table 1. In addition to the use of CT scans to recognize COVID-19 infection, they can be used for other important tasks, which include quantifying the infection and monitoring the evolution of the disease, which can help with treatment and save the patient's life [12]. Moreover, the evolution stage can be recognized, where the typical signs of COVID-19 infection could be ground-glass opacity (GGO) in the early stage, and pulmonary consolidation in the late stage [7,8]. According to the estimated COVID-19 infection percentage from the CT-scans, the patient state can be classified into Normal (0%), Minimal (<10%), Moderate (10-25%), Extent (25-50%), Severe (50-75%), and Critical (>75%) [13].

<table><caption>**Table 1.** COVID-19 recognition methods with pros and cons for each method.</caption><thead><tr><td>COVID-19 Recognition Method</td><td>Pros</td><td>Cons</td></tr></thead><tbody><tr><td>RT-PCR</td><td>The most used method</td><td>Requires specific equipment, Time-consuming, Expensive, Considerable False-Negative Rate</td></tr><tr><td>X-ray scans</td><td>Low cost, Available in most Hospitals, fast acquisition time</td><td>Require an experienced Radiologist</td></tr><tr><td>CT scans</td><td>Available in most Hospitals, Very accurate recognition, Can be used for monitoring the evolution of the infection, Useful for Infection Severity Prediction</td><td>Require an experienced Radiologist, Expensive, Medium acquisition time</td></tr></tbody></table>

Most of the state-of-the-art methods have been concentrating on the recognition of COVID-19 from the CT scans or segmentation of the infected regions. Despite the huge efforts that have been made, the state-of-the-art methods have not provided many helpful tools to monitor the patient state, the evolution of the infection, or the response of patient to the treatment, which can play a crucial role in saving the patient's life. In this paper, we propose a fully automatic approach to evaluate the evolution of COVID-19 infection from the CT scans as regression task which can provide a richer information about the COVID-19 infection evolution. The estimation of COVID-19 percentage can help intensive care workers to identify the patients that need urgent care, especially the critical and severe cases. With the extensive number of COVID-19 infections, estimating the COVID-19 percentage can help intensive care workers free up resuscitation beds for the critical cases and follow other protocol for less severe cases.

Unlike the mainstream that dealt with COVID-19 recognition and segmentation, this paper addresses the estimation of COVID-19 infection percentage. To this end, we constructed the Per-COVID-19 dataset, then we used it to evaluate the performance of three CNN architectures with two loss functions and two pretrained models scenarios. In summary, the main contributions of this paper are as follows:

* We introduce the Per-COVID-19 dataset for estimating the COVID-19 infection percentage for both slice-level and patient-level. The constructed dataset consists of 183 CT scans with the corresponding slice-level COVID-19 infection percentage which were estimated by two expert radiologists. To the best of our knowledge, our work
is the first to propose a finer granularity of COVID-19 virus presence and solve a challenging task related to exact estimation of COVID-19 infection percentage.

* In order to test some state-of-the-art methods, we evaluated the performance of three CNN architectures: ResneXt-50, Densenet-161, and Inception-v3. For the three CNN architectures, we use two loss functions: Mean Squared Error (MSE) and Dynamic Huber loss. In addition, two pretrained scenarios are investigated. In the first scenario, the pretrained models on ImageNet are used. To study the influence of using pretrained models on medical imaging task, we use the pretrained models on X-ray images.

* We make our database and codes publicly available to encourage other researchers to use it as a benchmark for their studies https://github.com/faresbougourzi/Per-COVID-19 (last accessed on 4 August 2021).

## 2. Related Works

The state-of-the-Art methods using CT scans can be classified into two main tasks: COVID-19 recognition [5,6,14–16] and COVID-19 segmentation [7,8,17,18]. In [19], Zheng, C. et al. proposed the DeCoVNet approach, which is based on 3D deep convolutional neural Network to Detect COVID-19 (DeCoVNet) from CT volumes. The input to DeCoVNet is CT volume, and its 3D lung mask was generated by using pretrained UNet [20]. Their proposed DeCoV-Net architecture has three parts: vanilla 3D convolution, 3D residual blocks (ResBlocks), and progressive classifier (ProClf). He, K. et al. proposed a multi-task multi-instance deep network (M²UNet) to assess the severity of COVID-19 patients [21]. Their proposed approach classifies the volumetric CT-scans into two classes of severity: severe or non-severe. Their M²UNet approach consists of a patch-level encoder, a segmentation subnetwork for lung lobe segmentation, and a classification subnetwork for severity assessment. In [22], Yao, Q. et al. proposed the NormNet architecture, which is a voxel-level anomaly modeling network, to distinguish healthy tissues from the COVID-19 lesion in the thorax area. Paulo. L. et al. investigated transfer learning and hyperparameter optimization techniques to improve the computer-aided diagnosis for COVID-19 recognition sensitivity [15]. To this end, they tested different data preprocessing and augmentation techniques. In addition, four CNN architectures were used and four hyperparameters were optimized for each CNN architecture.

Zhao, X.et al. proposed a dilated dual-attention U-Net (D2A U-Net) approach for COVID-19 lesion segmentation in CT slices based on dilated convolution and a novel dual-attention mechanism to address the issues above [7]. In [17], Alessandro. S. et al. proposed a customized ENET (C-ENET) approach for COVID-19 infection segmentation. Their proposed C-ENET approach proved its efficiency in public datasets compared with UNET [20] and ERFNET [23] segmentation architectures. To deal with the limitation of the training data for segmenting COVID-19 infection, Athanasios. V. et al. introduced the few-shot learning (FSL) concept of network model training using a very small number of samples [18]. They explored the efficiency of few-shot learning in U-Net architectures, allowing for a dynamic fine-tuning of the network weights as new few samples are being fed into the UNet. Experimental results indicate improvement in the segmentation accuracy of identifying COVID-19 infected regions.

The main limitation of the state-of-the-art works is that they have been concentrating on the recognition and segmentation of COVID-19 infection. However, more information about the disease evolution and severity could be inferred from the CT scans. On the other hand, the available datasets are very limited, especially for the segmentation and severity tasks. Table 2 shows some of the available Segmentation datasets. From this table, we notice that these datasets were contrasted with a small number of CT scans and slices. This is because the time and effort required for the labeling process is very large, with most radiologists having very little time, especially during this pandemic. In this work, we have created a dataset for estimating the percentage of COVID-19 infections that requires less time and effort for the labeling process.
is the first to propose a finer granularity of COVID-19 virus presence and solve a challenging task related to exact estimation of COVID-19 infection percentage.

* In order to test some state-of-the-art methods, we evaluated the performance of three CNN architectures: ResneXt-50, Densenet-161, and Inception-v3. For the three CNN architectures, we use two loss functions: Mean Squared Error (MSE) and Dynamic Huber loss. In addition, two pretrained scenarios are investigated. In the first scenario, the pretrained models on ImageNet are used. To study the influence of using pretrained models on medical imaging task, we use the pretrained models on X-ray images.

* We make our database and codes publicly available to encourage other researchers to use it as a benchmark for their studies https://github.com/faresbougourzi/Per-COVID-19 (last accessed on 4 August 2021).

## 2. Related Works

The state-of-the-Art methods using CT scans can be classified into two main tasks: COVID-19 recognition [5,6,14–16] and COVID-19 segmentation [7,8,17,18]. In [19], Zheng, C. et al. proposed the DeCoVNet approach, which is based on 3D deep convolutional neural Network to Detect COVID-19 (DeCoVNet) from CT volumes. The input to DeCoVNet is CT volume, and its 3D lung mask was generated by using pretrained UNet [20]. Their proposed DeCoV-Net architecture has three parts: vanilla 3D convolution, 3D residual blocks (ResBlocks), and progressive classifier (ProClf). He, K. et al. proposed a multi-task multi-instance deep network (M²UNet) to assess the severity of COVID-19 patients [21]. Their proposed approach classifies the volumetric CT-scans into two classes of severity: severe or non-severe. Their M²UNet approach consists of a patch-level encoder, a segmentation subnetwork for lung lobe segmentation, and a classification subnetwork for severity assessment. In [22], Yao, Q. et al. proposed the NormNet architecture, which is a voxel-level anomaly modeling network, to distinguish healthy tissues from the COVID-19 lesion in the thorax area. Paulo. L. et al. investigated transfer learning and hyperparameter optimization techniques to improve the computer-aided diagnosis for COVID-19 recognition sensitivity [15]. To this end, they tested different data preprocessing and augmentation techniques. In addition, four CNN architectures were used and four hyperparameters were optimized for each CNN architecture.

Zhao, X.et al. proposed a dilated dual-attention U-Net (D2A U-Net) approach for COVID-19 lesion segmentation in CT slices based on dilated convolution and a novel dual-attention mechanism to address the issues above [7]. In [17], Alessandro. S. et al. proposed a customized ENET (C-ENET) approach for COVID-19 infection segmentation. Their proposed C-ENET approach proved its efficiency in public datasets compared with UNET [20] and ERFNET [23] segmentation architectures. To deal with the limitation of the training data for segmenting COVID-19 infection, Athanasios. V. et al. introduced the few-shot learning (FSL) concept of network model training using a very small number of samples [18]. They explored the efficiency of few-shot learning in U-Net architectures, allowing for a dynamic fine-tuning of the network weights as new few samples are being fed into the UNet. Experimental results indicate improvement in the segmentation accuracy of identifying COVID-19 infected regions.

The main limitation of the state-of-the-art works is that they have been concentrating on the recognition and segmentation of COVID-19 infection. However, more information about the disease evolution and severity could be inferred from the CT scans. On the other hand, the available datasets are very limited, especially for the segmentation and severity tasks. Table 2 shows some of the available Segmentation datasets. From this table, we notice that these datasets were contrasted with a small number of CT scans and slices. This is because the time and effort required for the labeling process is very large, with most radiologists having very little time, especially during this pandemic. In this work, we have created a dataset for estimating the percentage of COVID-19 infections that requires less time and effort for the labeling process.
<table><caption>**Table 2.** COVID-19 segmentation databases.</caption><thead><tr><th>Database</th><th># CT-Scan</th><th># Slices</th></tr></thead><tbody><tr><td>COVID-19 CT segmentation dataset [24] March 2020</td><td>40 CT-scans</td><td>100 slices</td></tr><tr><td>Segmentation dataset nr. 2 [24] April 2020</td><td>9 CT-scans</td><td>Total of 829 slices from which 373 infected slices</td></tr><tr><td>MOSMEDDATA Segmentation [25] May 2020</td><td>50 (the 10th slice was kept from each CT-scan)</td><td>50</td></tr><tr><td>COVID-19-CT-Seg-Benchmark dataset [26] April 2021</td><td>20 COVID-19 CT-scans</td><td>Total: 1800 slices with 300 infected slices</td></tr><tr><td>The proposed dataset</td><td>183 CT-scans</td><td>3986 labelled slices, where COVID-19 Infection percentage were estimated by two expert Radiologists</td></tr></tbody></table>

# 3. Materials and Methods

## 3.1. Per-COVID-19 Dataset

Our Per-COVID-19 database consists of 183 CT scans that were confirmed to have COVID-19 infection. Figure 1 shows the histogram of the infection percentage per case. The patients were from both genders and aged between 27 to 70 years old. Each volumetric CT scan of the Per-COVID-19 database was taken from a different patient, so the number of CT scans equals to the number of patients. The diagnosis of COVID-19 infection is based on positive reverse transcription polymerase chain reaction (RT-PCR) and CT scan manifestations identified by two experienced thoracic radiologists. The CT scans were collected from two hospitals: Hakim Saidane Biskra and Ziouch Mohamed Tolga (Algeria) from June to December 2020. Each CT scan consists of 40–70 slices. Table 3 summarizes the number of CT-scans and the used recording settings and device for each hospital. The two Radiologists estimated COVID-19 infection percentage based on the area of infected lungs over the overall size of the lungs. From each CT scan, the radiologists picked the slices that contain signs of COVID-19 infection and the ones that do not contain any COVID-19 signs.

<table><caption>**Table 3.** Per-COVID-19 dataset construction.</caption><thead><tr><th>Hospital</th><th>Hakim Saidane Biskra</th><th>Ziouch Mohamed Tolga</th></tr></thead><tbody><tr><td>Number of CT-scans</td><td>150</td><td>33</td></tr><tr><td>Device Scanner</td><td>Hitachi ECLOS CT-Scanner</td><td>Toshiba Alexion CT-Scanner</td></tr><tr><td>Slice Thickness</td><td>5 mm</td><td>3 mm</td></tr></tbody></table>

In the proposed dataset, we kept the slices that have diagnosis agreement of the two radiologists. In summary, from each CT-scan, we can have between 2 and 45 typical slice images (these slices have close estimation of COVID-19 infection percentage from both radiologists). Figure 2 shows the histogram of number of CT-scans over the number of CT slices. Figure 3 shows examples of slices images with their corresponding COVID-19 infection percentage.

To evaluate different machine learning methods, we divided the database Per-COVID-19 into five patient-independent folds, with each patient slice contained in only one fold. Figure 4 shows the distribution of the slices of the dataset and the fold slices over the percentage of COVID-19 infections, where the dataset and the fold slices have an almost similar distribution. The goal of using the Patient-Independent evaluation protocol is to test the performance of the CNN architectures for slices from new patients not seen in the training phase. In total, there are 3986 labeled slices with the corresponding COVID-19 infection per-
<table><caption>**Table 2.** COVID-19 segmentation databases.</caption><thead><tr><th>Database</th><th># CT-Scan</th><th># Slices</th></tr></thead><tbody><tr><td>COVID-19 CT segmentation dataset [24] March 2020</td><td>40 CT-scans</td><td>100 slices</td></tr><tr><td>Segmentation dataset nr. 2 [24] April 2020</td><td>9 CT-scans</td><td>Total of 829 slices from which 373 infected slices</td></tr><tr><td>MOSMEDDATA Segmentation [25] May 2020</td><td>50 (the 10th slice was kept from each CT-scan)</td><td>50</td></tr><tr><td>COVID-19-CT-Seg-Benchmark dataset [26] April 2021</td><td>20 COVID-19 CT-scans</td><td>Total: 1800 slices with 300 infected slices</td></tr><tr><td>The proposed dataset</td><td>183 CT-scans</td><td>3986 labelled slices, where COVID-19 Infection percentage were estimated by two expert Radiologists</td></tr></tbody></table>

## 3. Materials and Methods

### 3.1. Per-COVID-19 Dataset

Our Per-COVID-19 database consists of 183 CT scans that were confirmed to have COVID-19 infection. Figure 1 shows the histogram of the infection percentage per case. The patients were from both genders and aged between 27 to 70 years old. Each volumetric CT scan of the Per-COVID-19 database was taken from a different patient, so the number of CT scans equals to the number of patients. The diagnosis of COVID-19 infection is based on positive reverse transcription polymerase chain reaction (RT-PCR) and CT scan manifestations identified by two experienced thoracic radiologists. The CT scans were collected from two hospitals: Hakim Saidane Biskra and Ziouch Mohamed Tolga (Algeria) from June to December 2020. Each CT scan consists of 40–70 slices. Table 3 summarizes the number of CT-scans and the used recording settings and device for each hospital. The two Radiologists estimated COVID-19 infection percentage based on the area of infected lungs over the overall size of the lungs. From each CT scan, the radiologists picked the slices that contain signs of COVID-19 infection and the ones that do not contain any COVID-19 signs.

<table><caption>**Table 3.** Per-COVID-19 dataset construction.</caption><thead><tr><th>Hospital</th><th>Hakim Saidane Biskra</th><th>Ziouch Mohamed Tolga</th></tr></thead><tbody><tr><td>Number of CT-scans</td><td>150</td><td>33</td></tr><tr><td>Device Scanner</td><td>Hitachi ECLOS CT-Scanner</td><td>Toshiba Alexion CT-Scanner</td></tr><tr><td>Slice Thickness</td><td>5 mm</td><td>3 mm</td></tr></tbody></table>

In the proposed dataset, we kept the slices that have diagnosis agreement of the two radiologists. In summary, from each CT-scan, we can have between 2 and 45 typical slice images (these slices have close estimation of COVID-19 infection percentage from both radiologists). Figure 2 shows the histogram of number of CT-scans over the number of CT slices. Figure 3 shows examples of slices images with their corresponding COVID-19 infection percentage.

To evaluate different machine learning methods, we divided the database Per-COVID-19 into five patient-independent folds, with each patient slice contained in only one fold. Figure 4 shows the distribution of the slices of the dataset and the fold slices over the percentage of COVID-19 infections, where the dataset and the fold slices have an almost similar distribution. The goal of using the Patient-Independent evaluation protocol is to test the performance of the CNN architectures for slices from new patients not seen in the training phase. In total, there are 3986 labeled slices with the corresponding COVID-19 infection per-
centage. These slices were converted to PNG format and the lung regions were then manually cropped. The dataset is available at https://github.com/faresbougourzi/Per-Covid-19 (accessed on 4 August 2021), with labels available in the file 'Database_new.xlsx'. The columns represent the image name, percentage of COVID-19 infection, fold split number, and subject ID.

<figure><img src="image_4.png" /><figcaption>**Figure 1.** Histogram of the slices average infection percentage per case in the proposed Per-COVID-19 database.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Figure 2.** Histogram of slices number per patient in the proposed Per-COVID-19 database.</figcaption></figure>
centage. These slices were converted to PNG format and the lung regions were then manually cropped. The dataset is available at https://github.com/faresbougourzi/Per-Covid-19 (accessed on 4 August 2021), with labels available in the file 'Database_new.xlsx'. The columns represent the image name, percentage of COVID-19 infection, fold split number, and subject ID.

<figure><img src="image_5.png" /><figcaption>**Figure 1.** Histogram of the slices average infection percentage per case in the proposed Per-COVID-19 database.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Figure 2.** Histogram of slices number per patient in the proposed Per-COVID-19 database.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Figure 3.** CT scan images examples which have the COVID-19 infection percentages of 0%, 5%, 18%, 40%, 70%, and 80%, respectively. These slices are classified as Normal, Minimal (<10%), Moderate (10–25%), Extent (25–50%), Severe (50–75%), and Critical (>75%), respectively.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**Figure 4.** *Cont.*</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>**Figure 3.** CT scan images examples which have the COVID-19 infection percentages of 0%, 5%, 18%, 40%, 70%, and 80%, respectively. These slices are classified as Normal, Minimal (<10%), Moderate (10–25%), Extent (25–50%), Severe (50–75%), and Critical (>75%), respectively.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Figure 4.** *Cont.*</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>Figure 4. *Cont.*</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Figure 4. *Cont.*</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>**Figure 4.** (a) Per-COVID-19 Dataset, (b) Fold1. (c) Fold2, (d) Fold3. (e) Fold4, and (f) Fold5. Histogram of COVID-19 infection percentage per slice for the dataset and the five folds.</figcaption></figure>

## 3.2. Methods

### Loss Functions

In our experiments, we used two loss functions: Mean Squared Error (MSE) and Dynamic Huber loss. The loss functions are defined for N batch size, and $X = (x_1, x_2, ..., x_N)$ are the ground-truth percentages and $\hat{X} = (\hat{x}_1, \hat{x}_2, ..., \hat{x}_N)$ are their corresponding estimated percentages.

MSE is sensitive towards outliers. For N predictions, MSE loss function is defined by

$$L_{MSE} = \frac{1}{N} \sum_{i=1}^{N} (x_i - \hat{x}_i)^2 \quad (1)$$

on the other hand, the Huber loss function is less sensitive to outliers in data than $L_2$ loss function. For N training batch size images, the Huber loss function is defined by [27]:

$$L_{Huber} = \frac{1}{N} \sum_{i=1}^{N} z_i \quad (2)$$

where N is the batch size and $z_i$ is defined by

$$z_i = \begin{cases} 0.5 (x_i - \hat{x}_i)^2, & \text{if } |x_i - \hat{x}_i| \le \beta \\ \beta |x_i - \hat{x}_i| - 0.5 \beta^2, & \text{otherwise} \end{cases} \quad (3)$$
<figure><img src="image_10.png" /><figcaption>**Figure 4.** (a) Per-COVID-19 Dataset, (b) Fold1. (c) Fold2, (d) Fold3. (e) Fold4, and (f) Fold5. Histogram of COVID-19 infection percentage per slice for the dataset and the five folds.</figcaption></figure>

## 3.2. Methods

### Loss Functions

In our experiments, we used two loss functions: Mean Squared Error (MSE) and Dynamic Huber loss. The loss functions are defined for N batch size, and $X = (x_1, x_2, \dots, x_N)$ are the ground-truth percentages and $\hat{X} = (\hat{x}_1, \hat{x}_2, \dots, \hat{x}_N)$ are their corresponding estimated percentages.

MSE is sensitive towards outliers. For N predictions, MSE loss function is defined by

$$L_{MSE} = \frac{1}{N} \sum_{i=1}^{N} (x_i - \hat{x}_i)^2 \quad (1)$$

on the other hand, the Huber loss function is less sensitive to outliers in data than $L_2$ loss function. For N training batch size images, the Huber loss function is defined by [27]:

$$L_{Huber} = \frac{1}{N} \sum_{i=1}^{N} z_i \quad (2)$$

where N is the batch size and $z_i$ is defined by

$$z_i = \begin{cases} 0.5 (x_i - \hat{x}_i)^2, & \text{if } |x_i - \hat{x}_i| \le \beta \\ \beta |x_i - \hat{x}_i| - 0.5 \beta^2, & \text{otherwise} \end{cases} \quad (3)$$
where $\beta$ is a controlling hyperparameter. In our experiments, $\beta$ decreases from 15 to 1 during the training.

### 3.3. Evaluation Metrics

To evaluate the performance of the state-of-the-art methods, we use slice-level metrics which are Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Pearson Correlation coefficient (PC), which are defined in Equations (4)–(6), respectively.

$$MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i| \qquad (4)$$

$$RMSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 \qquad (5)$$

$$PC = \frac{\sum_{i=1}^{n} (y_i - \bar{y}_i) (\hat{y}_i - \bar{\hat{y}}_i)}{\sqrt{\sum_{i=1}^{n} (y_i - \bar{y}_i)^2} \sqrt{\sum_{i=1}^{n} (\hat{y}_i - \bar{\hat{y}}_i)^2}} \qquad (6)$$

where $Y = (y_1, y_2, \dots, y_n)$ are the ground-truth COVID-19 percentages of the testing data which consists of $n$ slices and $\hat{Y} = (\hat{y}_1, \hat{y}_2, \dots, \hat{y}_n)$ are their corresponding estimated percentages. For Equation (6), $\bar{y}_i$ and $\hat{y}_i$ are the means of the ground-truth percentages and the estimated ones, respectively.

In addition, we used subject-level metrics $MAE_{subj}$, $RMSE_{subj}$, and $PC_{subj}$, which are defined in Equations (7)–(9), respectively.

$$MAE_{subj} = \frac{1}{s} \sum_{i=1}^{s} |y_{s_i} - \hat{y}_{s_i}| \qquad (7)$$

$$RMSE_{subj} = \frac{1}{s} \sum_{i=1}^{s} (y_{s_i} - \hat{y}_{s_i})^2 \qquad (8)$$

$$PC_{subj} = \frac{\sum_{i=1}^{s} (y_{s_i} - \bar{y}_{s_i}) (\hat{y}_{s_i} - \bar{\hat{y}}_{s_i})}{\sqrt{\sum_{i=1}^{s} (y_{s_i} - \bar{y}_{s_i})^2 \sum_{i=1}^{s} (\hat{y}_{s_i} - \bar{\hat{y}}_{s_i})^2}} \qquad (9)$$

where $Y_s = (y_{s_1}, y_{s_2}, \dots, y_{s_n})$ are the ground-truth means of COVID-19 percentages of each patient' slices from the testing data and $\hat{Y}_s = (\hat{y}_{s_1}, \hat{y}_{s_2}, \dots, \hat{y}_{s_n})$ are their corresponding estimated patient-level percentages (means of patient' slices percentages). For Equation (9), $\bar{y}_{s_i}$ and $\hat{y}_{s_i}$ are the means of the ground-truth patient percentages and the estimated ones, respectively.

$MAE$ and $RMSE$ are error indicators where the smaller values indicates better performance. From other hand, $PC$ is a statistic measurement of linear correlation between two variables $Y$ and $\hat{Y}$. A value of 1 means that there is a total positive linear correlation and 0 indicates no linear correlation.

## 4. Results

To train and test the CNN architectures (ResneXt-50, DenseNet-161, and Inception-v3), we used the Pytorch [28] library, and a SGD optimizer with momentum equal to 0.9 is used during the training phase. All experiments were carried out on PC with 64 GB Ram and NVIDIA GPU Device Geforce TITAN RTX 24 GB (National Research Council (CNR-ISASI) of Italy, 73100 Lecce, Italy). Each CNN architecture was trained for 30 epochs with initial learning rate of $10^{-4}$ with decays by 0.1 every 10 epochs and batch size equals 20. In addition, we used two active data augmentation techniques: we used random crop data augmentation followed by random rotation using an angle between -10 to 10 degrees. In summary, our experiments are divided into two scenarios: In the first scenario, we used retrained models of ImageNet, while in the second scenario, we used pretrained models that were trained on medical imaging task.
where $\beta$ is a controlling hyperparameter. In our experiments, $\beta$ decreases from 15 to 1 during the training.

### 3.3. Evaluation Metrics

To evaluate the performance of the state-of-the-art methods, we use slice-level metrics which are Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Pearson Correlation coefficient (PC), which are defined in Equations (4)–(6), respectively.

$$MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i| \qquad (4)$$

$$RMSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 \qquad (5)$$

$$PC = \frac{\sum_{i=1}^{n} (y_i - \bar{y}_i) (\hat{y}_i - \bar{\hat{y}}_i)}{\sqrt{\sum_{i=1}^{n} (y_i - \bar{y}_i)^2} \sqrt{\sum_{i=1}^{n} (\hat{y}_i - \bar{\hat{y}}_i)^2}} \qquad (6)$$

where $Y = (y_1, y_2, \dots, y_n)$ are the ground-truth COVID-19 percentages of the testing data which consists of $n$ slices and $\hat{Y} = (\hat{y}_1, \hat{y}_2, \dots, \hat{y}_n)$ are their corresponding estimated percentages. For Equation (6), $\bar{y}_i$ and $\hat{y}_i$ are the means of the ground-truth percentages and the estimated ones, respectively.

In addition, we used subject-level metrics $MAE_{subj}$, $RMSE_{subj}$, and $PC_{subj}$, which are defined in Equations (7)–(9), respectively.

$$MAE_{subj} = \frac{1}{s} \sum_{i=1}^{s} |y_{s_i} - \hat{y}_{s_i}| \qquad (7)$$

$$RMSE_{subj} = \frac{1}{s} \sum_{i=1}^{s} (y_{s_i} - \hat{y}_{s_i})^2 \qquad (8)$$

$$PC_{subj} = \frac{\sum_{i=1}^{s} (y_{s_i} - \bar{y}_{s_i}) (\hat{y}_{s_i} - \bar{\hat{y}}_{s_i})}{\sqrt{\sum_{i=1}^{s} (y_{s_i} - \bar{y}_{s_i})^2} \sqrt{\sum_{i=1}^{s} (\hat{y}_{s_i} - \bar{\hat{y}}_{s_i})^2}} \qquad (9)$$

where $Y_s = (y_{s_1}, y_{s_2}, \dots, y_{s_n})$ are the ground-truth means of COVID-19 percentages of each patient' slices from the testing data and $\hat{Y}_s = (\hat{y}_{s_1}, \hat{y}_{s_2}, \dots, \hat{y}_{s_n})$ are their corresponding estimated patient-level percentages (means of patient' slices percentages). For Equation (9), $\bar{y}_{s_i}$ and $\hat{y}_{s_i}$ are the means of the ground-truth patient percentages and the estimated ones, respectively.

MAE and RMSE are error indicators where the smaller values indicates better performance. From other hand, PC is a statistic measurement of linear correlation between two variables $Y$ and $\hat{Y}$. A value of 1 means that there is a total positive linear correlation and 0 indicates no linear correlation.

## 4. Results

To train and test the CNN architectures (ResneXt-50, DenseNet-161, and Inception-v3), we used the Pytorch [28] library, and a SGD optimizer with momentum equal to 0.9 is used during the training phase. All experiments were carried out on PC with 64 GB Ram and NVIDIA GPU Device Geforce TITAN RTX 24 GB (National Research Council (CNR-ISASI) of Italy, 73100 Lecce, Italy). Each CNN architecture was trained for 30 epochs with initial learning rate of $10^{-4}$ with decays by 0.1 every 10 epochs and batch size equals 20. In addition, we used two active data augmentation techniques: we used random crop data augmentation followed by random rotation using an angle between -10 to 10 degrees. In summary, our experiments are divided into two scenarios: In the first scenario, we used retrained models of ImageNet, while in the second scenario, we used pretrained models that were trained on medical imaging task.
4.1. First Scenario

In the first scenario, we used three pretrained CNN architectures on ImageNet [29] (ResneXt-50 [30], Inception-V3 [31], and Densenet-161 [32]). Moreover, we used two loss functions: MSE and Dynamic Huber. Figures 5–10 summarize the obtained results of the first scenario for PC, MAE, RMSE, *PC*<sub>subj</sub>, *MAE*<sub>subj</sub>, and *RMSE*<sub>subj</sub>, respectively. From the results, we notice that for all models almost the Dynamic Huber loss gives better results then MSE loss function. This proves the efficiency of using Dynamic Huber loss function for this regression task. On the other hand, we notice that the three trained models with Huber dynamic loss achieved close results. In details, ResneXt-50 achieved the best results performance in MAE, *PC*<sub>subj</sub>, *MAE*<sub>subj</sub>, and *RMSE*<sub>subj</sub>, while Densenet-161 achieved the best result for PC metric and Inception-v3 for RMSE metric. In addition, we notice that Folds 2 and 3 are more challenging compared with Folds 1, 4, and 5, this is probably because these two folds contain more challenging patients than the other folds.

In order to study the influence of changing the hyperparameters, we used Inception-v3
architecture, MSE loss function, different values of the batch size (64, 32, 20, and 16), and
two different initial learning rates (10⁻⁴ and 10⁻³). Figure 11 shows the PC and MAE
results of these experiments. From these results, we notice that changing the learning rate
or the batch size has no big influence on the results. On the other hand, using smaller batch
size (32–16) gave slightly better performance, as the dataset has a medium size.

<figure><img src="image_10.png" /><figcaption>**Figure 5.** PearsonCorrelation coefficient (PC) results of the first scenario.</figcaption></figure>
4.1. First Scenario

In the first scenario, we used three pretrained CNN architectures on ImageNet [29] (ResneXt-50 [30], Inception-V3 [31], and Densenet-161 [32]). Moreover, we used two loss functions: MSE and Dynamic Huber. Figures 5–10 summarize the obtained results of the first scenario for PC, MAE, RMSE, *PC*<sub>subj</sub>, *MAE*<sub>subj</sub>, and *RMSE*<sub>subj</sub>, respectively. From the results, we notice that for all models almost the Dynamic Huber loss gives better results then MSE loss function. This proves the efficiency of using Dynamic Huber loss function for this regression task. On the other hand, we notice that the three trained models with Huber dynamic loss achieved close results. In details, ResneXt-50 achieved the best results performance in MAE, *PC*<sub>subj</sub>, *MAE*<sub>subj</sub>, and *RMSE*<sub>subj</sub>, while Densenet-161 achieved the best result for PC metric and Inception-v3 for RMSE metric. In addition, we notice that Folds 2 and 3 are more challenging compared with Folds 1, 4, and 5, this is probably because these two folds contain more challenging patients than the other folds.

In order to study the influence of changing the hyperparameters, we used Inception-v3
architecture, MSE loss function, different values of the batch size (64, 32, 20, and 16), and
two different initial learning rates (10⁻⁴ and 10⁻³). Figure 11 shows the PC and MAE
results of these experiments. From these results, we notice that changing the learning rate
or the batch size has no big influence on the results. On the other hand, using smaller batch
size (32–16) gave slightly better performance, as the dataset has a medium size.

<figure><img src="image_11.png" /><figcaption>**Figure 5.** PearsonCorrelation coefficient (PC) results of the first scenario.</figcaption></figure>
<figure><img src="image_11.png" /><figcaption>**Figure 6.** Mean Absolute Error (MAE), results of the first scenario.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>**Figure 7.** Root Mean Square Error (RMSE), results of the first scenario.</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>**Figure 6.** Mean Absolute Error (MAE), results of the first scenario.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>**Figure 7.** Root Mean Square Error (RMSE), results of the first scenario.</figcaption></figure>
<figure><img src="image_13.png" /><figcaption>**Figure 8.** $PC_{subj}$ results of the first scenario.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>**Figure 9.** $MAE_{subj}$ results of the first scenario.</figcaption></figure>

## 4.2. Second Scenario

In the second scenario, we use the same models as the first scenario but this time they were trained on the recognition of COVID-19 from X-ray scans [2]. In more detail, four lung diseases plus neutral were used to train the CNN architectures [2]. The objective of this scenario is to study the influence of the pretrained model which was trained on medical imaging task. The experimental results are summarized in Table 4. From these results, we notice that Inception-v3 achieved the best performance. Similar to the results of the first scenario, the Dynamic Huber loss gives better results than MSE loss function for most of the evaluation metrics.
<figure><img src="image_14.png" /><figcaption>**Figure 8.** $PC_{subj}$ results of the first scenario.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>**Figure 9.** $MAE_{subj}$ results of the first scenario.</figcaption></figure>

## 4.2. Second Scenario

In the second scenario, we use the same models as the first scenario but this time they were trained on the recognition of COVID-19 from X-ray scans [2]. In more detail, four lung diseases plus neutral were used to train the CNN architectures [2]. The objective of this scenario is to study the influence of the pretrained model which was trained on medical imaging task. The experimental results are summarized in Table 4. From these results, we notice that Inception-v3 achieved the best performance. Similar to the results of the first scenario, the Dynamic Huber loss gives better results than MSE loss function for most of the evaluation metrics.
<figure><img src="image_15.png" /><figcaption>**Figure 10.** RMSE<sub>subj</sub> results of the first scenario.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>**Figure 11.** (a) PC, (b) MAE. Inception-v3 architecture and MSE loss function for studying the influence of two learning rates (LR1 = $10^{-4}$ and LR2 = $10^{-3}$) and four batch sizes (64, 32, 20, and 16).</figcaption></figure>

As the pretrained models using X-ray data showed performance improvement compared with ImageNet pretrained models, we investigate the converging speed of each training scenario. To this end, we compare the convergence of the three CNN architectures using Huber loss function and Fold 1 and 2 splits. From Figure 12, we notice that the pretrained models on X-ray data converge faster to the best performance than the ImageNet pretrained models in four out of six experiments. Consequently, using pretrained model trained on medical imaging not only improve the performance, but it can speed up the training process.
<figure><img src="image_16.png" /><figcaption>**Figure 10.** RMSE<sub>subj</sub> results of the first scenario.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>**Figure 11.** (a) PC, (b) MAE. Inception-v3 architecture and MSE loss function for studying the influence of two learning rates (LR1 = $10^{-4}$ and LR2 = $10^{-3}$) and four batch sizes (64, 32, 20, and 16).</figcaption></figure>

As the pretrained models using X-ray data showed performance improvement compared with ImageNet pretrained models, we investigate the converging speed of each training scenario. To this end, we compare the convergence of the three CNN architectures using Huber loss function and Fold 1 and 2 splits. From Figure 12, we notice that the pretrained models on X-ray data converge faster to the best performance than the ImageNet pretrained models in four out of six experiments. Consequently, using pretrained model trained on medical imaging not only improve the performance, but it can speed up the training process.
<table><caption>**Table 4.** Fivefold cross-validation results of the second scenario (pretrained X-ray models) using three CNN architectures—ResneXt-50, Densenet-161, and Inception-v3—and two loss functions (MSE, Huber). § Mean and STD were calculated using the average and the standard deviation of the five folds results, respectively. * the evaluation metrics were calculated using all five folds predictions. The bold is for the results of the five folds. The red, purple and blue colors are for the best performances for Mean (5 folds), STD (5 folds) and All Predictions, respectively.</caption><thead><tr><th>Architecture</th><th>Fold</th><th>PC ↑</th><th>MAE ↓</th><th>RMSE ↓</th><th>PC<sub>subj</sub> ↑</th><th>MAE<sub>subj</sub> ↓</th><th>RMSE<sub>subj</sub> ↓</th></tr></thead><tbody><tr><td rowspan="7">ResneXt-50<br>MSE</td><td>Fold 1</td><td>0.9453</td><td>5.24</td><td>8.77</td><td>0.9619</td><td>4.05</td><td>6.58</td></tr><tr><td>Fold 2</td><td>0.8569</td><td>6.68</td><td>13.39</td><td>0.9234</td><td>4.87</td><td>8.70</td></tr><tr><td>Fold 3</td><td>0.9083</td><td>6.17</td><td>12.53</td><td>0.9196</td><td>5.23</td><td>11.17</td></tr><tr><td>Fold 4</td><td>0.9589</td><td>3.97</td><td>6.32</td><td>0.9884</td><td>2.31</td><td>3.22</td></tr><tr><td>Fold 5</td><td>0.9457</td><td>5.99</td><td>9.05</td><td>0.9674</td><td>4.92</td><td>6.43</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9230</strong></td><td><strong>5.61</strong></td><td><strong>10.01</strong></td><td><strong>0.9521</strong></td><td><strong>4.28</strong></td><td><strong>7.22</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0371</strong></td><td><strong>0.9412</strong></td><td><strong>2.6019</strong></td><td><strong>0.0265</strong></td><td><strong>1.0577</strong></td><td><strong>2.6391</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9173</strong></td><td><strong>5.62</strong></td><td><strong>10.41</strong></td><td><strong>0.9431</strong></td><td><strong>4.29</strong></td><td><strong>7.91</strong></td></tr><tr><td rowspan="7">ResneXt-50<br>Huber</td><td>Fold 1</td><td>0.9487</td><td>4.96</td><td>8.48</td><td>0.9633</td><td>3.80</td><td>6.43</td></tr><tr><td>Fold 2</td><td>0.8227</td><td>6.93</td><td>15.29</td><td>0.9147</td><td>4.96</td><td>9.25</td></tr><tr><td>Fold 3</td><td>0.9191</td><td>5.35</td><td>11.84</td><td>0.9301</td><td>4.27</td><td>10.47</td></tr><tr><td>Fold 4</td><td>0.9622</td><td>3.70</td><td>6.09</td><td>0.9873</td><td>2.39</td><td>3.26</td></tr><tr><td>Fold 5</td><td>0.9507</td><td>5.50</td><td>8.80</td><td>0.9703</td><td>4.34</td><td>6.30</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9207</strong></td><td><strong>5.29</strong></td><td><strong>10.10</strong></td><td><strong>0.9532</strong></td><td><strong>3.95</strong></td><td><strong>7.14</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0510</strong></td><td><strong>1.0367</strong></td><td><strong>3.1735</strong></td><td><strong>0.0267</strong></td><td><strong>0.8637</strong></td><td><strong>2.5221</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9132</strong></td><td><strong>5.30</strong></td><td><strong>10.65</strong></td><td><strong>0.9427</strong></td><td><strong>4.02</strong></td><td><strong>7.87</strong></td></tr><tr><td rowspan="7">Densenet-161<br>MSE</td><td>Fold 1</td><td>0.9438</td><td>5.15</td><td>8.65</td><td>0.9611</td><td>3.79</td><td>6.36</td></tr><tr><td>Fold 2</td><td>0.8768</td><td>6.48</td><td>12.34</td><td>0.9354</td><td>4.62</td><td>8.04</td></tr><tr><td>Fold 3</td><td>0.9116</td><td>5.86</td><td>12.44</td><td>0.9259</td><td>4.73</td><td>10.89</td></tr><tr><td>Fold 4</td><td>0.9576</td><td>4.11</td><td>6.65</td><td>0.9861</td><td>2.97</td><td>4.14</td></tr><tr><td>Fold 5</td><td>0.9479</td><td>5.79</td><td>8.99</td><td>0.9691</td><td>4.82</td><td>6.38</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9275</strong></td><td><strong>5.48</strong></td><td><strong>9.81</strong></td><td><strong>0.9555</strong></td><td><strong>4.18</strong></td><td><strong>7.16</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0297</strong></td><td><strong>0.8032</strong></td><td><strong>2.2503</strong></td><td><strong>0.0220</strong></td><td><strong>0.7101</strong></td><td><strong>2.2385</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9234</strong></td><td><strong>5.49</strong></td><td><strong>10.12</strong></td><td><strong>0.9501</strong></td><td><strong>4.09</strong></td><td><strong>7.56</strong></td></tr><tr><td rowspan="7">Densenet-161<br>Huber</td><td>Fold 1</td><td>0.9440</td><td>4.97</td><td>8.78</td><td>0.9569</td><td>3.91</td><td>6.84</td></tr><tr><td>Fold 2</td><td>0.8938</td><td>6.14</td><td>11.36</td><td>0.9459</td><td>4.57</td><td>7.58</td></tr><tr><td>Fold 3</td><td>0.9158</td><td>5.89</td><td>12.33</td><td>0.9290</td><td>4.99</td><td>10.87</td></tr><tr><td>Fold 4</td><td>0.9643</td><td>3.67</td><td>5.93</td><td>0.9893</td><td>2.33</td><td>3.16</td></tr><tr><td>Fold 5</td><td>0.9526</td><td>5.48</td><td>8.71</td><td>0.9697</td><td>4.56</td><td>6.54</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9341</strong></td><td><strong>5.23</strong></td><td><strong>9.42</strong></td><td><strong>0.9582</strong></td><td><strong>4.07</strong></td><td><strong>7.00</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0257</strong></td><td><strong>0.8749</strong></td><td><strong>2.2505</strong></td><td><strong>0.0205</strong></td><td><strong>0.9369</strong></td><td><strong>2.4615</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9305</strong></td><td><strong>5.25</strong></td><td><strong>9.75</strong></td><td><strong>0.9523</strong></td><td><strong>4.06</strong></td><td><strong>7.51</strong></td></tr><tr><td rowspan="7">Inception-v3<br>MSE</td><td>Fold 1</td><td>0.9483</td><td>5.22</td><td>8.51</td><td>0.9630</td><td>3.95</td><td>6.53</td></tr><tr><td>Fold 2</td><td>0.8571</td><td>6.63</td><td>13.14</td><td>0.9258</td><td>5.01</td><td>8.57</td></tr><tr><td>Fold 3</td><td>0.9111</td><td>5.89</td><td>12.28</td><td>0.9240</td><td>5.22</td><td>10.84</td></tr><tr><td>Fold 4</td><td>0.9605</td><td>3.91</td><td>6.22</td><td>0.9878</td><td>2.52</td><td>3.27</td></tr><tr><td>Fold 5</td><td>0.9463</td><td>6.10</td><td>9.22</td><td>0.9667</td><td>5.02</td><td>6.74</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9247</strong></td><td><strong>5.55</strong></td><td><strong>9.87</strong></td><td><strong>0.9535</strong></td><td><strong>4.34</strong></td><td><strong>7.19</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0375</strong></td><td><strong>0.9362</strong></td><td><strong>2.5335</strong></td><td><strong>0.0248</strong></td><td><strong>1.0149</strong></td><td><strong>2.4992</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9163</strong></td><td><strong>5.90</strong></td><td><strong>10.57</strong></td><td><strong>0.9470</strong></td><td><strong>4.16</strong></td><td><strong>7.51</strong></td></tr><tr><td rowspan="7">Inception-v3<br>Huber</td><td>Fold 1</td><td>0.9526</td><td>4.81</td><td>8.43</td><td>0.9673</td><td>3.81</td><td>6.45</td></tr><tr><td>Fold 2</td><td>0.8905</td><td>6.17</td><td>11.64</td><td>0.9414</td><td>4.59</td><td>7.63</td></tr><tr><td>Fold 3</td><td>0.9230</td><td>5.47</td><td>11.69</td><td>0.9324</td><td>4.75</td><td>10.41</td></tr><tr><td>Fold 4</td><td>0.9607</td><td>3.79</td><td>6.33</td><td>0.9850</td><td>2.85</td><td>3.91</td></tr><tr><td>Fold 5</td><td>0.9556</td><td>5.29</td><td>8.14</td><td>0.9753</td><td>4.04</td><td>5.56</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9365</strong></td><td><strong>5.10</strong></td><td><strong>9.25</strong></td><td><strong>0.9603</strong></td><td><strong>4.01</strong></td><td><strong>6.79</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0264</strong></td><td><strong>0.7896</strong></td><td><strong>2.1022</strong></td><td><strong>0.0201</strong></td><td><strong>0.6738</strong></td><td><strong>2.1786</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9330</strong></td><td><strong>5.34</strong></td><td><strong>9.44</strong></td><td><strong>0.9558</strong></td><td><strong>3.98</strong></td><td><strong>6.87</strong></td></tr></tbody></table>
<table><caption>**Table 4.** Fivefold cross-validation results of the second scenario (pretrained X-ray models) using three CNN architectures—ResneXt-50, Densenet-161, and Inception-v3—and two loss functions (MSE, Huber). § Mean and STD were calculated using the average and the standard deviation of the five folds results, respectively. * the evaluation metrics were calculated using all five folds predictions. The bold is for the results of the five folds. The red, purple and blue colors are for the best performances for Mean (5 folds), STD (5 folds) and All Predictions, respectively.</caption><thead><tr><th>Architecture</th><th>Fold</th><th>PC ↑</th><th>MAE ↓</th><th>RMSE ↓</th><th>PC<sub>subj</sub> ↑</th><th>MAE<sub>subj</sub> ↓</th><th>RMSE<sub>subj</sub> ↓</th></tr></thead><tbody><tr><td rowspan="6">ResneXt-50<br>MSE</td><td>Fold 1</td><td>0.9453</td><td>5.24</td><td>8.77</td><td>0.9619</td><td>4.05</td><td>6.58</td></tr><tr><td>Fold 2</td><td>0.8569</td><td>6.68</td><td>13.39</td><td>0.9234</td><td>4.87</td><td>8.70</td></tr><tr><td>Fold 3</td><td>0.9083</td><td>6.17</td><td>12.53</td><td>0.9196</td><td>5.23</td><td>11.17</td></tr><tr><td>Fold 4</td><td>0.9589</td><td>3.97</td><td>6.32</td><td>0.9884</td><td>2.31</td><td>3.22</td></tr><tr><td>Fold 5</td><td>0.9457</td><td>5.99</td><td>9.05</td><td>0.9674</td><td>4.92</td><td>6.43</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9230</strong></td><td><strong>5.61</strong></td><td><strong>10.01</strong></td><td><strong>0.9521</strong></td><td><strong>4.28</strong></td><td><strong>7.22</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0371</strong></td><td><strong>0.9412</strong></td><td><strong>2.6019</strong></td><td><strong>0.0265</strong></td><td><strong>1.0577</strong></td><td><strong>2.6391</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9173</strong></td><td><strong>5.62</strong></td><td><strong>10.41</strong></td><td><strong>0.9431</strong></td><td><strong>4.29</strong></td><td><strong>7.91</strong></td></tr><tr><td rowspan="6">ResneXt-50<br>Huber</td><td>Fold 1</td><td>0.9487</td><td>4.96</td><td>8.48</td><td>0.9633</td><td>3.80</td><td>6.43</td></tr><tr><td>Fold 2</td><td>0.8227</td><td>6.93</td><td>15.29</td><td>0.9147</td><td>4.96</td><td>9.25</td></tr><tr><td>Fold 3</td><td>0.9191</td><td>5.35</td><td>11.84</td><td>0.9301</td><td>4.27</td><td>10.47</td></tr><tr><td>Fold 4</td><td>0.9622</td><td>3.70</td><td>6.09</td><td>0.9873</td><td>2.39</td><td>3.26</td></tr><tr><td>Fold 5</td><td>0.9507</td><td>5.50</td><td>8.80</td><td>0.9703</td><td>4.34</td><td>6.30</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9207</strong></td><td><strong>5.29</strong></td><td><strong>10.10</strong></td><td><strong>0.9532</strong></td><td><strong>3.95</strong></td><td><strong>7.14</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0510</strong></td><td><strong>1.0367</strong></td><td><strong>3.1735</strong></td><td><strong>0.0267</strong></td><td><strong>0.8637</strong></td><td><strong>2.5221</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9132</strong></td><td><strong>5.30</strong></td><td><strong>10.65</strong></td><td><strong>0.9427</strong></td><td><strong>4.02</strong></td><td><strong>7.87</strong></td></tr><tr><td rowspan="6">Densenet-161<br>MSE</td><td>Fold 1</td><td>0.9438</td><td>5.15</td><td>8.65</td><td>0.9611</td><td>3.79</td><td>6.36</td></tr><tr><td>Fold 2</td><td>0.8768</td><td>6.48</td><td>12.34</td><td>0.9354</td><td>4.62</td><td>8.04</td></tr><tr><td>Fold 3</td><td>0.9116</td><td>5.86</td><td>12.44</td><td>0.9259</td><td>4.73</td><td>10.89</td></tr><tr><td>Fold 4</td><td>0.9576</td><td>4.11</td><td>6.65</td><td>0.9861</td><td>2.97</td><td>4.14</td></tr><tr><td>Fold 5</td><td>0.9479</td><td>5.79</td><td>8.99</td><td>0.9691</td><td>4.82</td><td>6.38</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9275</strong></td><td><strong>5.48</strong></td><td><strong>9.81</strong></td><td><strong>0.9555</strong></td><td><strong>4.18</strong></td><td><strong>7.16</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0297</strong></td><td><strong>0.8032</strong></td><td><strong>2.2503</strong></td><td><strong>0.0220</strong></td><td><strong>0.7101</strong></td><td><strong>2.2385</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9234</strong></td><td><strong>5.49</strong></td><td><strong>10.12</strong></td><td><strong>0.9501</strong></td><td><strong>4.09</strong></td><td><strong>7.56</strong></td></tr><tr><td rowspan="6">Densenet-161<br>Huber</td><td>Fold 1</td><td>0.9440</td><td>4.97</td><td>8.78</td><td>0.9569</td><td>3.91</td><td>6.84</td></tr><tr><td>Fold 2</td><td>0.8938</td><td>6.14</td><td>11.36</td><td>0.9459</td><td>4.57</td><td>7.58</td></tr><tr><td>Fold 3</td><td>0.9158</td><td>5.89</td><td>12.33</td><td>0.9290</td><td>4.99</td><td>10.87</td></tr><tr><td>Fold 4</td><td>0.9643</td><td>3.67</td><td>5.93</td><td>0.9893</td><td>2.33</td><td>3.16</td></tr><tr><td>Fold 5</td><td>0.9526</td><td>5.48</td><td>8.71</td><td>0.9697</td><td>4.56</td><td>6.54</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9341</strong></td><td><strong>5.23</strong></td><td><strong>9.42</strong></td><td><strong>0.9582</strong></td><td><strong>4.07</strong></td><td><strong>7.00</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0257</strong></td><td><strong>0.8749</strong></td><td><strong>2.2505</strong></td><td><strong>0.0205</strong></td><td><strong>0.9369</strong></td><td><strong>2.4615</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9305</strong></td><td><strong>5.25</strong></td><td><strong>9.75</strong></td><td><strong>0.9523</strong></td><td><strong>4.06</strong></td><td><strong>7.51</strong></td></tr><tr><td rowspan="6">Inception-v3<br>MSE</td><td>Fold 1</td><td>0.9483</td><td>5.22</td><td>8.51</td><td>0.9630</td><td>3.95</td><td>6.53</td></tr><tr><td>Fold 2</td><td>0.8571</td><td>6.63</td><td>13.14</td><td>0.9258</td><td>5.01</td><td>8.57</td></tr><tr><td>Fold 3</td><td>0.9111</td><td>5.89</td><td>12.28</td><td>0.9240</td><td>5.22</td><td>10.84</td></tr><tr><td>Fold 4</td><td>0.9605</td><td>3.91</td><td>6.22</td><td>0.9878</td><td>2.52</td><td>3.27</td></tr><tr><td>Fold 5</td><td>0.9463</td><td>6.10</td><td>9.22</td><td>0.9667</td><td>5.02</td><td>6.74</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9247</strong></td><td><strong>5.55</strong></td><td><strong>9.87</strong></td><td><strong>0.9535</strong></td><td><strong>4.34</strong></td><td><strong>7.19</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0375</strong></td><td><strong>0.9362</strong></td><td><strong>2.5335</strong></td><td><strong>0.0248</strong></td><td><strong>1.0149</strong></td><td><strong>2.4992</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9163</strong></td><td><strong>5.90</strong></td><td><strong>10.57</strong></td><td><strong>0.9470</strong></td><td><strong>4.16</strong></td><td><strong>7.51</strong></td></tr><tr><td rowspan="6">Inception-v3<br>Huber</td><td>Fold 1</td><td>0.9526</td><td>4.81</td><td>8.43</td><td>0.9673</td><td>3.81</td><td>6.45</td></tr><tr><td>Fold 2</td><td>0.8905</td><td>6.17</td><td>11.64</td><td>0.9414</td><td>4.59</td><td>7.63</td></tr><tr><td>Fold 3</td><td>0.9230</td><td>5.47</td><td>11.69</td><td>0.9324</td><td>4.75</td><td>10.41</td></tr><tr><td>Fold 4</td><td>0.9607</td><td>3.79</td><td>6.33</td><td>0.9850</td><td>2.85</td><td>3.91</td></tr><tr><td>Fold 5</td><td>0.9556</td><td>5.29</td><td>8.14</td><td>0.9753</td><td>4.04</td><td>5.56</td></tr><tr><td>Mean (5 folds) §</td><td><strong>0.9365</strong></td><td><strong>5.10</strong></td><td><strong>9.25</strong></td><td><strong>0.9603</strong></td><td><strong>4.01</strong></td><td><strong>6.79</strong></td></tr><tr><td>STD (5 folds) §</td><td><strong>0.0264</strong></td><td><strong>0.7896</strong></td><td><strong>2.1022</strong></td><td><strong>0.0201</strong></td><td><strong>0.6738</strong></td><td><strong>2.1786</strong></td></tr><tr><td></td><td>All Predictions *</td><td><strong>0.9330</strong></td><td><strong>5.34</strong></td><td><strong>9.44</strong></td><td><strong>0.9558</strong></td><td><strong>3.98</strong></td><td><strong>6.87</strong></td></tr></tbody></table>
<figure><img src="image_17.png" /><figcaption>**Figure 12.** (a) Fold1 and (b) Fold2. Convergence comparison: X-ray pretrained models vs. ImageNet pretrained models using Huber loss function.</figcaption></figure>

## 5. Discussion

The comparison between the first and second scenario experiments shows that the pretrained models on medical imaging task give better result than the pretrained models of ImageNet. From all experiments, we conclude that the best scenario for COVID-19 infection percentage estimation is by using Inception-v3 architecture with an X-ray pretrained model and Dynamic Huber loss function.
<figure><img src="image_18.png" /><figcaption>**Figure 12.** (a) Fold1 and (b) Fold2. Convergence comparison: X-ray pretrained models vs. ImageNet pretrained models using Huber loss function.</figcaption></figure>

## 5. Discussion

The comparison between the first and second scenario experiments shows that the pretrained models on medical imaging task give better result than the pretrained models of ImageNet. From all experiments, we conclude that the best scenario for COVID-19 infection percentage estimation is by using Inception-v3 architecture with an X-ray pretrained model and Dynamic Huber loss function.
To calculate the required time to estimate the COVID-19 infection percentage, we used a PC with Intel i7-CPU (3.60GHz × 8), 64 GB Ram, and NVIDIA GPU Device Geforce TITAN RTX 24 GB. Table 5 shows the testing time of the three CNN architectures. In the second column, we calculated the required time for one slice. As the number of slices is different from one patient to another, we calculated the required time of batch size of 120 slices (third column of Table 5). The goal of calculating this time is to estimate the required time for volumetric CT-scan, where we calculate the testing time of 120 slices. From Table 5, we notice that the testing time is close for all CNN architectures with slightly lower required testing time for ResneXt-50 architecture for both 1 and 120 slices. Moreover, the required testing time of one volumetric CT-scan is very small (~0.2 s). This proves that estimating the COVID-19 infection percentage using CNN architectures can be applied as real-time applications.

<table><caption>**Table 5.** Average of five testing times for the evaluated CNN architectures (ResneXt-50, Densenet-161 and Inception-v3).</caption><thead><tr><td>Model</td><td>Testing Time of One Slice (s)</td><td>Testing Time of 120 Slices (s)</td></tr></thead><tbody><tr><td>ResneXt-50</td><td>0.012470</td><td>0.171227</td></tr><tr><td>DenseNet-161</td><td>0.023900</td><td>0.205754</td></tr><tr><td>Inception-v3</td><td>0.013844</td><td>0.185773</td></tr></tbody></table>

## 6. Conclusions

In this paper, we introduced the Per-COVID-19 dataset, which presents COVID-19 infection percentage estimation from CT scans. The estimation of COVID-19 infection percentage can help quantify the infection and monitor the evolution of the disease. In addition, the required time and efforts from the radiologists to estimate the COVID-19 infection are less compared with infection segmentation labeling. This can help in constructing a large dataset for COVID-19 severity tracking with reasonable time.

Moreover, we evaluated the performance of three CNN architectures: ResneXt-50, Densenet-161, and Inception-v3. For the three CNN architectures, we used two loss functions which are MSE and Dynamic Huber loss. In addition, we evaluate two pretrained models scenarios. In the first scenario, we used ImageNet pretrained models. In the second scenario, we used pretrained models that where trained on X-ray scans to investigate the influence of using pretrained models that were trained on medical imaging task.

The experimental results show that using the X-ray pretrained models improve the results. Moreover, the experiments using Dynamic Huber loss function achieved better performance than the ones used standard MSE loss function. From other hand, Inception-v3 outperformed ResneXt-50 and Densenet-161 architectures in both scenarios. The required time to estimate the COVID-19 infection from a slice is ~0.02 (s). On the other hand, the required time for a CT scan of 120 slices is approximately 0.2 s, which is very small compared with expert radiologists. Both results and testing time prove that is possible to implement real-time application for COVID-19 infection percentage estimation.

Despite the fact that the constructed database consists of 183 CT scans and 3986 slices, adding more CT scans and slices will provide more generalization ability to estimate the COVID-19 infection percentage. From the COVID-19 percentage slices histogram, it is clear that the dataset has less severe slices than other slices categories. This makes estimating the COVID-19 infection percentage for the sever cases more challenging than the other cases. As future future work, we propose to test other CNN architectures with using different data augmentation techniques. Including more labeled CT scans from different devices with different recording settings will help improve the results.

**Author Contributions:** F.B.: Data curation, Formal analysis, Investigation, Resources, Project administration, Software, Validation, Visualization Writing—original draft. C.D.: Data curation, Formal analysis, Investigation, Resources, Software, Project administration, Validation, Supervision, Visualization Writing—original draft. A.O.: Data curation, Formal analysis, Investigation, Methodology,
To calculate the required time to estimate the COVID-19 infection percentage, we used a PC with Intel i7-CPU (3.60GHz × 8), 64 GB Ram, and NVIDIA GPU Device Geforce TITAN RTX 24 GB. Table 5 shows the testing time of the three CNN architectures. In the second column, we calculated the required time for one slice. As the number of slices is different from one patient to another, we calculated the required time of batch size of 120 slices (third column of Table 5). The goal of calculating this time is to estimate the required time for volumetric CT-scan, where we calculate the testing time of 120 slices. From Table 5, we notice that the testing time is close for all CNN architectures with slightly lower required testing time for ResneXt-50 architecture for both 1 and 120 slices. Moreover, the required testing time of one volumetric CT-scan is very small (~0.2 s). This proves that estimating the COVID-19 infection percentage using CNN architectures can be applied as real-time applications.

<table><caption>**Table 5.** Average of five testing times for the evaluated CNN architectures (ResneXt-50, Densenet-161 and Inception-v3).</caption><thead><tr><td>Model</td><td>Testing Time of One Slice (s)</td><td>Testing Time of 120 Slices (s)</td></tr></thead><tbody><tr><td>ResneXt-50</td><td>0.012470</td><td>0.171227</td></tr><tr><td>DenseNet-161</td><td>0.023900</td><td>0.205754</td></tr><tr><td>Inception-v3</td><td>0.013844</td><td>0.185773</td></tr></tbody></table>

## 6. Conclusions

In this paper, we introduced the Per-COVID-19 dataset, which presents COVID-19 infection percentage estimation from CT scans. The estimation of COVID-19 infection percentage can help quantify the infection and monitor the evolution of the disease. In addition, the required time and efforts from the radiologists to estimate the COVID-19 infection are less compared with infection segmentation labeling. This can help in constructing a large dataset for COVID-19 severity tracking with reasonable time.

Moreover, we evaluated the performance of three CNN architectures: ResneXt-50, Densenet-161, and Inception-v3. For the three CNN architectures, we used two loss functions which are MSE and Dynamic Huber loss. In addition, we evaluate two pretrained models scenarios. In the first scenario, we used ImageNet pretrained models. In the second scenario, we used pretrained models that where trained on X-ray scans to investigate the influence of using pretrained models that were trained on medical imaging task.

The experimental results show that using the X-ray pretrained models improve the results. Moreover, the experiments using Dynamic Huber loss function achieved better performance than the ones used standard MSE loss function. From other hand, Inception-v3 outperformed ResneXt-50 and Densenet-161 architectures in both scenarios. The required time to estimate the COVID-19 infection from a slice is ~0.02 (s). On the other hand, the required time for a CT scan of 120 slices is approximately 0.2 s, which is very small compared with expert radiologists. Both results and testing time prove that is possible to implement real-time application for COVID-19 infection percentage estimation.

Despite the fact that the constructed database consists of 183 CT scans and 3986 slices, adding more CT scans and slices will provide more generalization ability to estimate the COVID-19 infection percentage. From the COVID-19 percentage slices histogram, it is clear that the dataset has less severe slices than other slices categories. This makes estimating the COVID-19 infection percentage for the sever cases more challenging than the other cases. As future future work, we propose to test other CNN architectures with using different data augmentation techniques. Including more labeled CT scans from different devices with different recording settings will help improve the results.

**Author Contributions:** F.B.: Data curation, Formal analysis, Investigation, Resources, Project administration, Software, Validation, Visualization Writing—original draft. C.D.: Data curation, Formal analysis, Investigation, Resources, Software, Project administration, Validation, Supervision, Visualization Writing—original draft. A.O.: Data curation, Formal analysis, Investigation, Methodology,
Resources, Software, Supervision, Validation, Visualization, Writing—original draft. F.D.: Formal analysis, Investigation, Methodology, Supervision, Writing—original draft. A.H.: Formal analysis, Investigation, Methodology, Supervision, Writing—original draft. A.T.-A.: Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Writing—original draft. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of Hospitals of Hakim Saidane Biskra and Ziouch Mohamed Tolga (Algeria), June and July 2020, respectively.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data were collected from the Hospitals of Hakim Saidane Biskra and Ziouch Mohamed Tolga (Algeria) and it was made available at https://github.com/faresbougourzi/Per-COVID-19 (Last accessed on 4 August 2021).

**Acknowledgments:** The authors would like to thank Arturo Argentieri from CNR-ISASI Italy for his support on the multi-GPU computing facilities.

**Conflicts of Interest:** The authors declare no conflict of interest.

**Abbreviations**

The following abbreviations are used in this manuscript:

CNN Convolutional Neural Network

RT-PCR Reverse Transcription Polymerase Chain Reaction

**References**

1. Kucirka, L.M.; Lauer, S.A.; Laeyendecker, O.; Boon, D.; Lessler, J. Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure. *ANNals Intern. Med.* **2020**, *173*, 262–267. [CrossRef] [PubMed]

2. Vantaggiato, E.; Paladini, E.; Bougourzi, F.; Distante, C.; Hadid, A.; Taleb-Ahmed, A. COVID-19 Recognition Using Ensemble-CNNs in Two New Chest X-ray Databases. *Sensors* **2021**, *21*, 1742. [CrossRef] [PubMed]

3. Wang, G.; Liu, X.; Shen, J.; Wang, C.; Li, Z.; Ye, L.; Wu, X.; Chen, T.; Wang, K.; Zhang, X. A deep-learning pipeline for the diagnosis and discrimination of viral, non-viral and COVID-19 pneumonia from chest X-ray images. *Nat. Biomed. Eng.* **2021**, *5*, 509–521. [CrossRef]

4. Wang, L.; Lin, Z.Q.; Wong, A. COVID-Net: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images. *Sci. Rep.* **2020**, *10*, 19549. [CrossRef]

5. Bougourzi, F.; Contino, R.; Distante, C.; Taleb-Ahmed, A. CNR-IEMN: A Deep Learning Based Approach To Recognize COVID-19 From CT-SCAN. In Proceedings of the 2021 IEEE International Conference on Autonomous Systems (IEEE ICAS), Toronto, ON, Canada, 6–11 June 2021.

6. Lassau, N.; Ammari, S.; Chouzenoux, E.; Gortais, H.; Herent, P.; Devilder, M.; Soliman, S.; Meyrignac, O.; Talabard, M.P.; Lamarque, J.P.; et al. Integrating deep learning CT-scan model, biological and clinical variables to predict severity of COVID-19 patients. *Nat. Commun.* **2021**, *12*, 634. [CrossRef]

7. Zhao, X.; Zhang, P.; Song, F.; Fan, G.; Sun, Y.; Wang, Y.; Tian, Z.; Zhang, L.; Zhang, G. D2A U-Net: Automatic Segmentation of COVID-19 Lesions from CT Slices with Dilated Convolution and Dual Attention Mechanism. *arXiv* **2021**, arXiv:2102.05210.

8. Fan, D.P.; Zhou, T.; Ji, G.P.; Zhou, Y.; Chen, G.; Fu, H.; Shen, J.; Shao, L. Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images. *IEEE Trans. Med Imaging* **2020**, *39*, 2626–2637. [CrossRef]

9. Bougourzi, F.; Dornaika, F.; Mokrani, K.; Taleb-Ahmed, A.; Ruichek, Y. Fusion Transformed Deep and Shallow features (FTDS) for Image-Based Facial Expression Recognition. *Expert Syst. Appl.* **2020**, *156*, 113459. [CrossRef]

10. Paladini, E.; Vantaggiato, E.; Bougourzi, F.; Distante, C.; Hadid, A.; Taleb-Ahmed, A. Two Ensemble-CNN Approaches for Colorectal Cancer Tissue Type Classification. *J. Imaging* **2021**, *7*, 51. [CrossRef] [PubMed]

11. Krittanawong, C.; Virk, H.U.H.; Kumar, A.; Aydar, M.; Wang, Z.; Stewart, M.P.; Halperin, J.L. Machine learning and deep learning to predict mortality in patients with spontaneous coronary artery dissection. *Sci. Rep.* **2021**, *11*, 8992. [CrossRef] [PubMed]

12. Lei, J.; Li, J.; Li, X.; Qi, X. CT Imaging of the 2019 Novel Coronavirus (2019-nCoV) Pneumonia. *Radiology* **2020**, *295*, 18–18. [CrossRef] [PubMed]
Resources, Software, Supervision, Validation, Visualization, Writing—original draft. F.D.: Formal analysis, Investigation, Methodology, Supervision, Writing—original draft. A.H.: Formal analysis, Investigation, Methodology, Supervision, Writing—original draft. A.T.-A.: Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Writing—original draft. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of Hospitals of Hakim Saidane Biskra and Ziouch Mohamed Tolga (Algeria), June and July 2020, respectively.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data were collected from the Hospitals of Hakim Saidane Biskra and Ziouch Mohamed Tolga (Algeria) and it was made available at https://github.com/faresbougourzi/Per-COVID-19 (Last accessed on 4 August 2021).

**Acknowledgments:** The authors would like to thank Arturo Argentieri from CNR-ISASI Italy for his support on the multi-GPU computing facilities.

**Conflicts of Interest:** The authors declare no conflict of interest.

**Abbreviations**

The following abbreviations are used in this manuscript:

CNN Convolutional Neural Network

RT-PCR Reverse Transcription Polymerase Chain Reaction

**References**

1. Kucirka, L.M.; Lauer, S.A.; Laeyendecker, O.; Boon, D.; Lessler, J. Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure. *ANNals Intern. Med.* **2020**, *173*, 262–267. [CrossRef] [PubMed]

2. Vantaggiato, E.; Paladini, E.; Bougourzi, F.; Distante, C.; Hadid, A.; Taleb-Ahmed, A. COVID-19 Recognition Using Ensemble-CNNs in Two New Chest X-ray Databases. *Sensors* **2021**, *21*, 1742. [CrossRef] [PubMed]

3. Wang, G.; Liu, X.; Shen, J.; Wang, C.; Li, Z.; Ye, L.; Wu, X.; Chen, T.; Wang, K.; Zhang, X. A deep-learning pipeline for the diagnosis and discrimination of viral, non-viral and COVID-19 pneumonia from chest X-ray images. *Nat. Biomed. Eng.* **2021**, *5*, 509–521. [CrossRef]

4. Wang, L.; Lin, Z.Q.; Wong, A. COVID-Net: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images. *Sci. Rep.* **2020**, *10*, 19549. [CrossRef]

5. Bougourzi, F.; Contino, R.; Distante, C.; Taleb-Ahmed, A. CNR-IEMN: A Deep Learning Based Approach To Recognize COVID-19 From CT-SCAN. In Proceedings of the 2021 IEEE International Conference on Autonomous Systems (IEEE ICAS), Toronto, ON, Canada, 6–11 June 2021.

6. Lassau, N.; Ammari, S.; Chouzenoux, E.; Gortais, H.; Herent, P.; Devilder, M.; Soliman, S.; Meyrignac, O.; Talabard, M.P.; Lamarque, J.P.; et al. Integrating deep learning CT-scan model, biological and clinical variables to predict severity of COVID-19 patients. *Nat. Commun.* **2021**, *12*, 634. [CrossRef]

7. Zhao, X.; Zhang, P.; Song, F.; Fan, G.; Sun, Y.; Wang, Y.; Tian, Z.; Zhang, L.; Zhang, G. D2A U-Net: Automatic Segmentation of COVID-19 Lesions from CT Slices with Dilated Convolution and Dual Attention Mechanism. *arXiv* **2021**, arXiv:2102.05210.

8. Fan, D.P.; Zhou, T.; Ji, G.P.; Zhou, Y.; Chen, G.; Fu, H.; Shen, J.; Shao, L. Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images. *IEEE Trans. Med Imaging* **2020**, *39*, 2626–2637. [CrossRef]

9. Bougourzi, F.; Dornaika, F.; Mokrani, K.; Taleb-Ahmed, A.; Ruichek, Y. Fusion Transformed Deep and Shallow features (FTDS) for Image-Based Facial Expression Recognition. *Expert Syst. Appl.* **2020**, *156*, 113459. [CrossRef]

10. Paladini, E.; Vantaggiato, E.; Bougourzi, F.; Distante, C.; Hadid, A.; Taleb-Ahmed, A. Two Ensemble-CNN Approaches for Colorectal Cancer Tissue Type Classification. *J. Imaging* **2021**, *7*, 51. [CrossRef] [PubMed]

11. Krittanawong, C.; Virk, H.U.H.; Kumar, A.; Aydar, M.; Wang, Z.; Stewart, M.P.; Halperin, J.L. Machine learning and deep learning to predict mortality in patients with spontaneous coronary artery dissection. *Sci. Rep.* **2021**, *11*, 8992. [CrossRef] [PubMed]

12. Lei, J.; Li, J.; Li, X.; Qi, X. CT Imaging of the 2019 Novel Coronavirus (2019-nCoV) Pneumonia. *Radiology* **2020**, *295*, 18–18. [CrossRef] [PubMed]
13. Jalaber, C.; Lapotre, T.; Morcet-Delattre, T.; Ribet, F.; Jouneau, S.; Lederlin, M. Chest CT in COVID-19 pneumonia: A review of current knowledge. *Diagn. Interv. Imaging* **2020**, *101*, 431-437. [CrossRef] [PubMed]

14. Wang, X.; Deng, X.; Fu, Q.; Zhou, Q.; Feng, J.; Ma, H.; Liu, W.; Zheng, C. A Weakly-Supervised Framework for COVID-19 Classification and Lesion Localization From Chest CT. *IEEE Trans. Med. Imaging* **2020**, *39*, 2615-2625. [CrossRef]

15. Lacerda, P.; Barros, B.; Albuquerque, C.; Conci, A. Hyperparameter Optimization for COVID-19 Pneumonia Diagnosis Based on Chest CT. *Sensors* **2021**, *21*, 2174. [CrossRef]

16. Bougourzi, F.; Contino, R.; Distante, C.; Taleb-Ahmed, A. Recognition of COVID-19 from CT Scans Using Two-Stage Deep-Learning-Based Approach: CNR-IEMN. *Sensors* **2021**, *21*, 5878. [CrossRef]

17. Stefano, A.; Comelli, A. Customized Efficient Neural Network for COVID-19 Infected Region Identification in CT Images. *J. Imaging* **2021**, *7*, 131. [CrossRef]

18. Voulodimos, A.; Protopapadakis, E.; Katsamenis, I.; Doulamis, A.; Doulamis, N. A Few-Shot U-Net Deep Learning Model for COVID-19 Infected Area Segmentation in CT Images. *Sensors* **2021**, *21*, 2215. [CrossRef]

19. Zheng, C.; Deng, X.; Fu, Q.; Zhou, Q.; Feng, J.; Ma, H.; Liu, W.; Wang, X. Deep Learning-based Detection for COVID-19 from Chest CT using Weak Label. *MedRxiv* **2020**.

20. Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In *Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015; Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2015; pp. 234–241*. [CrossRef]

21. He, K.; Zhao, W.; Xie, X.; Ji, W.; Liu, M.; Tang, Z.; Shi, Y.; Shi, F.; Gao, Y.; Liu, J.; et al. Synergistic learning of lung lobe segmentation and hierarchical multi-instance classification for automated severity assessment of COVID-19 in CT images. *Pattern Recognit.* **2021**, *113*, 107828. [CrossRef] [PubMed]

22. Yao, Q.; Xiao, L.; Liu, P.; Kevin Zhou, S. Label-Free Segmentation of COVID-19 Lesions in Lung CT. *IEEE Trans. Med. Imaging* **2021**, [CrossRef]

23. Romera, E.; Alvarez, J.M.; Bergasa, L.M.; Arroyo, R. Erfnet: Efficient residual factorized convnet for real-time semantic segmentation. *IEEE Trans. Intell. Transp. Syst.* **2017**, *19*, 263-272. [CrossRef]

24. COVID-19 CT-Scans Segmentation Datasets. Available online: http://medicalsegmentation.com/covid19/ (accessed on 18 August 2021).

25. Morozov, S.; Andreychenko, A.; Pavlov, N.; Vladzymyrskyy, A.; Ledikhova, N.; Gombolevskiy, V.; Blokhin, I.; Gelezhe, P.; Gonchar, A.; Chernina, V. MosMedData: Chest CT Scans with COVID-19 Related Findings Dataset. *arXiv* **2020**, arXiv:2005.06465. [CrossRef]

26. Ma, J.; Wang, Y.; An, X.; Ge, C.; Yu, Z.; Chen, J.; Zhu, Q.; Dong, G.; He, J.; He, Z.; et al. Toward data efficient learning: A benchmark for COVID 19 CT lung and infection segmentation. *Med. Phys.* **2021**, *48*, 1197-1210. [CrossRef] [PubMed]

27. Huber, P.J. Robust estimation of a location parameter. In *Breakthroughs in Statistics*; Springer: Berlin/Heidelberg, Germany, 1992; pp. 492-518.

28. Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L. Pytorch: An imperative style, high-performance deep learning library. *Adv. Neural Inf. Process. Syst.* **2019**, *32*, 8026-8037.

29. Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. ImageNet Large Scale Visual Recognition Challenge. *Int. J. Comput. Vis.* **2015**, *115*, 211-252. [CrossRef]

30. Xie, S.; Girshick, R.; Dollár, P.; Tu, Z.; He, K. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1492–1500.

31. Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2818–2826.

32. Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4700–4708.
13. Jalaber, C.; Lapotre, T.; Morcet-Delattre, T.; Ribet, F.; Jouneau, S.; Lederlin, M. Chest CT in COVID-19 pneumonia: A review of current knowledge. *Diagn. Interv. Imaging* **2020**, *101*, 431–437. [CrossRef] [PubMed]

14. Wang, X.; Deng, X.; Fu, Q.; Zhou, Q.; Feng, J.; Ma, H.; Liu, W.; Zheng, C. A Weakly-Supervised Framework for COVID-19 Classification and Lesion Localization From Chest CT. *IEEE Trans. Med. Imaging* **2020**, *39*, 2615–2625. [CrossRef]

15. Lacerda, P.; Barros, B.; Albuquerque, C.; Conci, A. Hyperparameter Optimization for COVID-19 Pneumonia Diagnosis Based on Chest CT. *Sensors* **2021**, *21*, 2174. [CrossRef]

16. Bougourzi, F.; Contino, R.; Distante, C.; Taleb-Ahmed, A. Recognition of COVID-19 from CT Scans Using Two-Stage Deep-Learning-Based Approach: CNR-IEMN. *Sensors* **2021**, *21*, 5878. [CrossRef]

17. Stefano, A.; Comelli, A. Customized Efficient Neural Network for COVID-19 Infected Region Identification in CT Images. *J. Imaging* **2021**, *7*, 131. [CrossRef]

18. Voulodimos, A.; Protopapadakis, E.; Katsamenis, I.; Doulamis, A.; Doulamis, N. A Few-Shot U-Net Deep Learning Model for COVID-19 Infected Area Segmentation in CT Images. *Sensors* **2021**, *21*, 2215. [CrossRef]

19. Zheng, C.; Deng, X.; Fu, Q.; Zhou, Q.; Feng, J.; Ma, H.; Liu, W.; Wang, X. Deep Learning-based Detection for COVID-19 from Chest CT using Weak Label. *MedRxiv* **2020**.

20. Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In *Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015; Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2015; pp. 234–241*. [CrossRef]

21. He, K.; Zhao, W.; Xie, X.; Ji, W.; Liu, M.; Tang, Z.; Shi, Y.; Shi, F.; Gao, Y.; Liu, J.; et al. Synergistic learning of lung lobe segmentation and hierarchical multi-instance classification for automated severity assessment of COVID-19 in CT images. *Pattern Recognit.* **2021**, *113*, 107828. [CrossRef] [PubMed]

22. Yao, Q.; Xiao, L.; Liu, P.; Kevin Zhou, S. Label-Free Segmentation of COVID-19 Lesions in Lung CT. *IEEE Trans. Med. Imaging* **2021**, [CrossRef]

23. Romera, E.; Alvarez, J.M.; Bergasa, L.M.; Arroyo, R. Erfnet: Efficient residual factorized convnet for real-time semantic segmentation. *IEEE Trans. Intell. Transp. Syst.* **2017**, *19*, 263–272. [CrossRef]

24. COVID-19 CT-Scans Segmentation Datasets. Available online: http://medicalsegmentation.com/covid19/ (accessed on 18 August 2021).

25. Morozov, S.; Andreychenko, A.; Pavlov, N.; Vladzymyrskyy, A.; Ledikhova, N.; Gombolevskiy, V.; Blokhin, I.; Gelezhe, P.; Gonchar, A.; Chernina, V. MosMedData: Chest CT Scans with COVID-19 Related Findings Dataset. *arXiv* **2020**, arXiv:2005.06465. [CrossRef]

26. Ma, J.; Wang, Y.; An, X.; Ge, C.; Yu, Z.; Chen, J.; Zhu, Q.; Dong, G.; He, J.; He, Z.; et al. Toward data efficient learning: A benchmark for COVID 19 CT lung and infection segmentation. *Med. Phys.* **2021**, *48*, 1197–1210. [CrossRef] [PubMed]

27. Huber, P.J. Robust estimation of a location parameter. In *Breakthroughs in Statistics*; Springer: Berlin/Heidelberg, Germany, 1992; pp. 492–518.

28. Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L. Pytorch: An imperative style, high-performance deep learning library. *Adv. Neural Inf. Process. Syst.* **2019**, *32*, 8026–8037.

29. Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. ImageNet Large Scale Visual Recognition Challenge. *Int. J. Comput. Vis.* **2015**, *115*, 211–252. [CrossRef]

30. Xie, S.; Girshick, R.; Dollár, P.; Tu, Z.; He, K. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1492–1500.

31. Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2818–2826.

32. Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4700–4708.