# Classification of Ultrasound Images with Convolutional Neural Networks in High-Performance Using IoT

Authors: Lamia Alhazmi
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

This paper contains the specification of functional and nonfunctional requirements of a CNN (Convolutional Neural Network) model to identify masses in liver ultrasounds. A neural network model based on CNN was implemented, which performs the analysis of ultrasound images of the liver, identifying the presence of masses or not in them. CNN can be executed either on a machine with a CPU (Central Processing Unit) or in a high processing capacity environment that uses GPU (Graphics Processing Unit). The validation of the accuracy of the obtained results happens through the statistical technique of k-fold cross-validation using IoT. During the neural network training process, dense data necessitates a lot of computational resources. Because the number of network entries is precisely proportional to the number of pixels in the image, parallel processing for image processing is justified. It is important to remember that medical photos cannot be saved in a lossy format and must be identified with exceptional accuracy using IoT.


---

## Body

## 1. Introduction

Images play an important role in society, regardless of how they are made, communicated, or accessed [1]. The transition from analog to digital photography and the increase in capture device storage capacity reinforced this significance. Medicine, particularly diagnostic imaging, makes extensive use of images [2]. Technical developments made ultrasound a viable diagnostic tool in this setting.Ultrasonography aims to provide real-time visualisation of any organ or tissue in the body [3]. Exam pictures must be processed, retrieved, and identified to aid in diagnosis. Specialists make these diagnoses visually, with a few exceptions. With the rapid multiplication of images, analysing them by specialists has become difficult due to professionals’ high costs and scarcity. Even with the necessary human resources and resources, a detailed examination of obtained pictures is time-consuming and exhausting, especially in large volumes, resulting in inaccuracy. Thus, technologies are required to effectively and reliably process, extract, and analyse images, replicating or supporting human professionals. These approaches can be automated using artificial intelligence tools, particularly Artificial Neural Networks (ANN) [4].CNNs (Convolutional Neural Networks), the study’s foundation, increase RNAS generalisation by being able to manage deformations and fluctuations in input data due to their architecture of receptive fields, weight sharing, and temporal or spatial subsampling [5]. CNNs can predict or extrapolate from established patterns to unknown data, allowing enormous image analysis and knowledge extraction to deliver consistent outputs for unpresented inputs. Because the training technique is computationally intensive, it takes time.The tool’s intensive processing pieces were relocated from the CPU to a GPU (Graphics Processing Unit) to reduce the computation time required for neural network training and image analysis during testing. This is due to its parallel design, consisting of hundreds or thousands of smaller and more efficient cores [6], which speeds up the integrated neural network’s training process.Marcomini et al. [7] created a breast ultrasound identification approach based on a neural classifier. The visualisations were created using a simulator and clinical photographs. The Wiener filter, equalisation, and median filter reduced noise in preprocessing. For the smallest representation, five segmentation approaches were tested. SOM was the most crucial. The discovery got more expressive as the object was specified. Training based on simulation was 94.2% correct. The system classified unknown photographs with 90% accuracy during generalisation testing. Finally, simulation tests were performed to validate the diagnosis in the liver ultrasound images. The accuracy and overall processing time for the specified analyses were tested by comparing the network’s CPU and GPU training and testing timings. The goal is to build a CNN-based computer model to detect liver masses in ultrasound pictures [8].


### 1.1. Ultrasound Imaging

The ability to collect data almost anywhere, at any time, regardless of weather or ambient illumination conditions, is the most impressive feature of picture collection using sound waves. This imaging technique is employed in many industries, including geological exploration, industry, and medicine. Millions of pulses and echoes are sent and received in an ultrasound image every second. The probe can be moved around the body’s surface and tilted to obtain diverse perspectives. Once the ideal view is identified, the image is saved and printed for expert review.


### 1.2. Steps of Digital Image Processing

After identifying the problem domain, image acquisition begins to build a database for processing. Images can be found on the internet or captured using a tool.Preprocessing increases image quality for subsequent steps. The digital image produced by this process is of higher quality than the original. A digital image is segmented into its principal units or objects of interest.The feature extraction stage uses descriptors to extract crucial properties from segmented images, allowing for exact object categorization. These descriptors must be represented with an algorithmically appropriate data structure. Although the input is still an image, the output is image data [9, 10].Identification and interpretation (phases of the classification stage). Recognition assigns labels to objects based on their descriptions. Interpretation is the process by which recognised objects are given meaning.These phases assume knowledge about the problem to be solved, which is stored in a database whose size and complexity vary substantially depending on the application. The Digital Image Processing (DIP) stages addressed here were provided by Filho and Neto [11].


### 1.3. Artificial Neural Networks

Lorrentzm [12] defined neural network, or ANN, as a parallel distributed processor made up of simple processing units, which tends to store experimental knowledge and make it available for use. It is inspired by the functioning of the brain in two ways:
(i)
knowledge is acquired by the network from its environment through a learning process(ii)
connection strengths between neurons, known as synaptic weights, store the acquired knowledgeAccording to Myo et al. [13], ANN is a mathematical model with computational power obtained through learning and generalisation and resembles organic brain structures. In a nutshell, learning ANN entails making systematic modifications to the synaptic weights to bring the ANN response to acceptable precision levels. As a result, the collection of connection weights can be viewed as a dynamic system that adapts to codify the knowledge that is desired to be learned. ANN’s generalisation is linked to its capacity to provide coherent replies to data that was not previously shown to it during the training stage.


#### 1.3.1. The Artificial Neuron (ANN)

Neurons are the smallest information processing unit in the ANN. According to Lorrentzm [12], Figure 1 depicts a neuron model with three parts:
(1)
a set of synapses characterized by their respective weights. Specifically, a signalxj at the input of synapse j connected to neuron k is multiplied by the synaptic weight wkj(2)
a propagation function, whose function is to sum the input signals weighted by the neuron’s respective synapses(3)
an activation function constrains the amplitude of a neuron’s outputFigure 1
Nonlinear model of an artificial neuron [12].Figure1 Neural model incorporates a bias bk to increase or decrease activation function input. Lorrentzm [12] describes a neuron k with two equations
(1)vk=∑j=1nwkj.xj,yk=φvk+bk,wherexj is the input signals, wkj is the synaptic weights of neuron k, vk is the sign resulting from the summation, bk is the bias, φ is the activation function, and finally, yk is the output signal of neuron k.


#### 1.3.2. Network Topology and Architecture

The network topology refers to the different structural compositions and numbers of neurons in the layers that make up the ANN. Usually, the ANN neurons are distributed in three layers, as explained in Figure2.Figure 2
(a) Illustration of the layering topology of a typical ANN and (b) Illustration of the ANN architecture.
(a)(b)The input layer receives the data to be analysed as well as the synaptic weights. The intermediate or hidden layer of neurons extracts and processes most of the data-related information (learning). The output layer receives data from the preceding layers and selects the best reaction.The architecture of ANN is the layout of its neurons in relation to synaptic connections. Artificial neurons constructed and connected in various ways generate neural networks with various properties and applications [12]. These network architectures are distinguished by the following characteristics: neuron-network connectivity and layer count. The ANN can be single-layer, with one neuron between each input and output, or multilayer, having more than one neuron between each input and output.The connection property between neurons can be of the feed-forward type(i)
the output of a neuron from a particular layer of the ANN can only be used as input from a neuron belonging to a layer posterior to it - or feedback(ii)
where the output of a neuron from a given layer of the ANN is used as the input of a neuron belonging to a layer before itANNs are either weakly or completely connected. Weakly linked networks do not connect all neurons in one layer to the next. When all neurons in one layer are connected to the next, the network is said to be fully interconnected. By integrating the above qualities, ANN designs tackle recognition and generalisation problems. An MLP, or multilayer, forward-fed, fully-connected ANN, is seen in Figure2(b).


### 1.4. Machine Learning

Mitchell [14] defined learning as improving an algorithm’s performance overtime. Machine learning is a branch of artificial intelligence that employs algorithms and tactics to learn data patterns, forecast behaviour, or recognise new data. This allows computers to self-learn. This section discusses the machine learning techniques and concepts used in this study.


#### 1.4.1. Learning Paradigms

Once specified, ANN requires training. ANN training necessitates altering synaptic weights iteratively. When a neural network learns, it discovers a general response. A network is trained by modifying its synaptic weight matrix so that the output vector matches each input vector. Lorrentzm [12] defined learning as the environment modifying the free parameters of a neural network (synaptic weights and bias). Input data influence the synaptic weights of a neural network, resulting in a varied response pattern. According to Lorrentzm [12], changing parameters determines the type of training (or learning paradigm). The primary paradigms are supervised, unsupervised, and reinforced learning. Unsupervised learning assists the ANN in detecting patterns in unlabelled data. According to Lorrentzm [12], reinforcement learning facilitates learning without supervision or a comprehensive model. In supervised learning, a supervisor provides examples of inputs and outputs. Discover an input-output rule, weighted ANNs are taught by iterative weight adjustments.


### 1.5. Convolutional Neural Networks

CNNs are MLP networks inspired by biological visual data processing. This variation is provided for developing a certain structure in MLP, according to Lecun and Bengio [15]. Two methods are required
(i)
restrict the choice of synaptic weights through the use of weight sharing(ii)
constrain the network architecture by using local connections, known as receptive fieldsAccording to Lorrentzm [12], using these tactics minimizes free network parameters, particularly the last one. This lowers computational expenses. A CNN serves several purposes. Figure 3 depicts the input, feature extraction, and classification processes.Figure 3
Example of CNN with two stages and its different stages [16].For the input step, image quality (no noise or shadows) and size (number of pixels) must be known. The number of feature extraction phases is affected by the number of pixels. The number of input layer neurons is affected by the number of categorization steps. Figure3 depicts a two-stage feature extraction process. Convolutional and pooling layers are present at each step.The classification step uses an MLP to recognise and understand incoming visual data; its output is the number of probable replies after processing.


## 1.1. Ultrasound Imaging

The ability to collect data almost anywhere, at any time, regardless of weather or ambient illumination conditions, is the most impressive feature of picture collection using sound waves. This imaging technique is employed in many industries, including geological exploration, industry, and medicine. Millions of pulses and echoes are sent and received in an ultrasound image every second. The probe can be moved around the body’s surface and tilted to obtain diverse perspectives. Once the ideal view is identified, the image is saved and printed for expert review.


## 1.2. Steps of Digital Image Processing

After identifying the problem domain, image acquisition begins to build a database for processing. Images can be found on the internet or captured using a tool.Preprocessing increases image quality for subsequent steps. The digital image produced by this process is of higher quality than the original. A digital image is segmented into its principal units or objects of interest.The feature extraction stage uses descriptors to extract crucial properties from segmented images, allowing for exact object categorization. These descriptors must be represented with an algorithmically appropriate data structure. Although the input is still an image, the output is image data [9, 10].Identification and interpretation (phases of the classification stage). Recognition assigns labels to objects based on their descriptions. Interpretation is the process by which recognised objects are given meaning.These phases assume knowledge about the problem to be solved, which is stored in a database whose size and complexity vary substantially depending on the application. The Digital Image Processing (DIP) stages addressed here were provided by Filho and Neto [11].


## 1.3. Artificial Neural Networks

Lorrentzm [12] defined neural network, or ANN, as a parallel distributed processor made up of simple processing units, which tends to store experimental knowledge and make it available for use. It is inspired by the functioning of the brain in two ways:
(i)
knowledge is acquired by the network from its environment through a learning process(ii)
connection strengths between neurons, known as synaptic weights, store the acquired knowledgeAccording to Myo et al. [13], ANN is a mathematical model with computational power obtained through learning and generalisation and resembles organic brain structures. In a nutshell, learning ANN entails making systematic modifications to the synaptic weights to bring the ANN response to acceptable precision levels. As a result, the collection of connection weights can be viewed as a dynamic system that adapts to codify the knowledge that is desired to be learned. ANN’s generalisation is linked to its capacity to provide coherent replies to data that was not previously shown to it during the training stage.


### 1.3.1. The Artificial Neuron (ANN)

Neurons are the smallest information processing unit in the ANN. According to Lorrentzm [12], Figure 1 depicts a neuron model with three parts:
(1)
a set of synapses characterized by their respective weights. Specifically, a signalxj at the input of synapse j connected to neuron k is multiplied by the synaptic weight wkj(2)
a propagation function, whose function is to sum the input signals weighted by the neuron’s respective synapses(3)
an activation function constrains the amplitude of a neuron’s outputFigure 1
Nonlinear model of an artificial neuron [12].Figure1 Neural model incorporates a bias bk to increase or decrease activation function input. Lorrentzm [12] describes a neuron k with two equations
(1)vk=∑j=1nwkj.xj,yk=φvk+bk,wherexj is the input signals, wkj is the synaptic weights of neuron k, vk is the sign resulting from the summation, bk is the bias, φ is the activation function, and finally, yk is the output signal of neuron k.


### 1.3.2. Network Topology and Architecture

The network topology refers to the different structural compositions and numbers of neurons in the layers that make up the ANN. Usually, the ANN neurons are distributed in three layers, as explained in Figure2.Figure 2
(a) Illustration of the layering topology of a typical ANN and (b) Illustration of the ANN architecture.
(a)(b)The input layer receives the data to be analysed as well as the synaptic weights. The intermediate or hidden layer of neurons extracts and processes most of the data-related information (learning). The output layer receives data from the preceding layers and selects the best reaction.The architecture of ANN is the layout of its neurons in relation to synaptic connections. Artificial neurons constructed and connected in various ways generate neural networks with various properties and applications [12]. These network architectures are distinguished by the following characteristics: neuron-network connectivity and layer count. The ANN can be single-layer, with one neuron between each input and output, or multilayer, having more than one neuron between each input and output.The connection property between neurons can be of the feed-forward type(i)
the output of a neuron from a particular layer of the ANN can only be used as input from a neuron belonging to a layer posterior to it - or feedback(ii)
where the output of a neuron from a given layer of the ANN is used as the input of a neuron belonging to a layer before itANNs are either weakly or completely connected. Weakly linked networks do not connect all neurons in one layer to the next. When all neurons in one layer are connected to the next, the network is said to be fully interconnected. By integrating the above qualities, ANN designs tackle recognition and generalisation problems. An MLP, or multilayer, forward-fed, fully-connected ANN, is seen in Figure2(b).


## 1.3.1. The Artificial Neuron (ANN)

Neurons are the smallest information processing unit in the ANN. According to Lorrentzm [12], Figure 1 depicts a neuron model with three parts:
(1)
a set of synapses characterized by their respective weights. Specifically, a signalxj at the input of synapse j connected to neuron k is multiplied by the synaptic weight wkj(2)
a propagation function, whose function is to sum the input signals weighted by the neuron’s respective synapses(3)
an activation function constrains the amplitude of a neuron’s outputFigure 1
Nonlinear model of an artificial neuron [12].Figure1 Neural model incorporates a bias bk to increase or decrease activation function input. Lorrentzm [12] describes a neuron k with two equations
(1)vk=∑j=1nwkj.xj,yk=φvk+bk,wherexj is the input signals, wkj is the synaptic weights of neuron k, vk is the sign resulting from the summation, bk is the bias, φ is the activation function, and finally, yk is the output signal of neuron k.


## 1.3.2. Network Topology and Architecture

The network topology refers to the different structural compositions and numbers of neurons in the layers that make up the ANN. Usually, the ANN neurons are distributed in three layers, as explained in Figure2.Figure 2
(a) Illustration of the layering topology of a typical ANN and (b) Illustration of the ANN architecture.
(a)(b)The input layer receives the data to be analysed as well as the synaptic weights. The intermediate or hidden layer of neurons extracts and processes most of the data-related information (learning). The output layer receives data from the preceding layers and selects the best reaction.The architecture of ANN is the layout of its neurons in relation to synaptic connections. Artificial neurons constructed and connected in various ways generate neural networks with various properties and applications [12]. These network architectures are distinguished by the following characteristics: neuron-network connectivity and layer count. The ANN can be single-layer, with one neuron between each input and output, or multilayer, having more than one neuron between each input and output.The connection property between neurons can be of the feed-forward type(i)
the output of a neuron from a particular layer of the ANN can only be used as input from a neuron belonging to a layer posterior to it - or feedback(ii)
where the output of a neuron from a given layer of the ANN is used as the input of a neuron belonging to a layer before itANNs are either weakly or completely connected. Weakly linked networks do not connect all neurons in one layer to the next. When all neurons in one layer are connected to the next, the network is said to be fully interconnected. By integrating the above qualities, ANN designs tackle recognition and generalisation problems. An MLP, or multilayer, forward-fed, fully-connected ANN, is seen in Figure2(b).


## 1.4. Machine Learning

Mitchell [14] defined learning as improving an algorithm’s performance overtime. Machine learning is a branch of artificial intelligence that employs algorithms and tactics to learn data patterns, forecast behaviour, or recognise new data. This allows computers to self-learn. This section discusses the machine learning techniques and concepts used in this study.


### 1.4.1. Learning Paradigms

Once specified, ANN requires training. ANN training necessitates altering synaptic weights iteratively. When a neural network learns, it discovers a general response. A network is trained by modifying its synaptic weight matrix so that the output vector matches each input vector. Lorrentzm [12] defined learning as the environment modifying the free parameters of a neural network (synaptic weights and bias). Input data influence the synaptic weights of a neural network, resulting in a varied response pattern. According to Lorrentzm [12], changing parameters determines the type of training (or learning paradigm). The primary paradigms are supervised, unsupervised, and reinforced learning. Unsupervised learning assists the ANN in detecting patterns in unlabelled data. According to Lorrentzm [12], reinforcement learning facilitates learning without supervision or a comprehensive model. In supervised learning, a supervisor provides examples of inputs and outputs. Discover an input-output rule, weighted ANNs are taught by iterative weight adjustments.


## 1.4.1. Learning Paradigms

Once specified, ANN requires training. ANN training necessitates altering synaptic weights iteratively. When a neural network learns, it discovers a general response. A network is trained by modifying its synaptic weight matrix so that the output vector matches each input vector. Lorrentzm [12] defined learning as the environment modifying the free parameters of a neural network (synaptic weights and bias). Input data influence the synaptic weights of a neural network, resulting in a varied response pattern. According to Lorrentzm [12], changing parameters determines the type of training (or learning paradigm). The primary paradigms are supervised, unsupervised, and reinforced learning. Unsupervised learning assists the ANN in detecting patterns in unlabelled data. According to Lorrentzm [12], reinforcement learning facilitates learning without supervision or a comprehensive model. In supervised learning, a supervisor provides examples of inputs and outputs. Discover an input-output rule, weighted ANNs are taught by iterative weight adjustments.


## 1.5. Convolutional Neural Networks

CNNs are MLP networks inspired by biological visual data processing. This variation is provided for developing a certain structure in MLP, according to Lecun and Bengio [15]. Two methods are required
(i)
restrict the choice of synaptic weights through the use of weight sharing(ii)
constrain the network architecture by using local connections, known as receptive fieldsAccording to Lorrentzm [12], using these tactics minimizes free network parameters, particularly the last one. This lowers computational expenses. A CNN serves several purposes. Figure 3 depicts the input, feature extraction, and classification processes.Figure 3
Example of CNN with two stages and its different stages [16].For the input step, image quality (no noise or shadows) and size (number of pixels) must be known. The number of feature extraction phases is affected by the number of pixels. The number of input layer neurons is affected by the number of categorization steps. Figure3 depicts a two-stage feature extraction process. Convolutional and pooling layers are present at each step.The classification step uses an MLP to recognise and understand incoming visual data; its output is the number of probable replies after processing.


## 2. Methodology

The scientific method is a collection of procedures, techniques, and processes for objectively and methodically acquiring knowledge. The technique encompasses both theoretical and practical processes employed in scientific research, ensuring that every researcher who does the investigation would reach the same conclusion. Chalakkal categorises scientific research according to its goal (basic or applied), technique (quantitative or qualitative), research objectives (exploratory, descriptive, or explanatory), and technical means (bibliographic, documentary, experimental, survey, case study, etc.).The current research is applied with the goal of developing a model to solve the research subject. The accuracy of the diagnoses and the overall processing time of the solution will be quantified. CNN is an exploring topic that provides additional information. The technical techniques of the research emphasize connected, experimental, and case study publishing.This study adheres to Gonzalez and Woods’ [17] approach, which represents the fundamental steps of the DIP. The next sections go through database procedures in depth. In addition, this chapter defines software engineering and model validation.


### 2.1. Software Engineering Process

The software engineering process used in its development will be a variant of the spiral model, evolutionary prototyping, to build a CNN model that meets the given criteria. According to Marcomini [7], the development spiral is used to build prototypes under this paradigm. Prototypes add features until the final product is ready. To reach the end-user, a radiologist, the suggested neural network model will need a user-friendly interface and the ability to identify masses. In this regard, a technical requirements document for model execution and modification was described.


### 2.2. Development

This subsection explains the implementation of the proposed CNN source code and the tools involved in its development. This subsection also details the design decisions made and emphasizes training-related methods.


#### 2.2.1. Development Environment

We used the Anaconda Navigator interface, which is a graphical user interface (GUI) that allows us to create programs and manage packages without using command lines. 2017-Anaconda, the CNTK (Microsoft Cognitive Toolkit) framework is used in the GUI, which supports C++, C#, Python, and BrainScript. According to NVIDIA [17–19], this toolkit efficiently implements CNN training for audio, image, and text data.We used Anaconda Navigator’s Spyder tool for source code development. Spyder is a Python IDE that allows you to edit, test, and debug source code [15, 20].


#### 2.2.2. Image Processing

This subsection describes the phases involved in processing the images that make up the database for this study. This step is performed before the classification of images by the neural network and according to the methodology proposed in Figure3.
(1)
Acquisition of images: Figure4 shows ultrasound pictures of livers taken from the Flickr platform, a photo and video management and sharing service. (a) These photographs are accessible for download in the albums of a computerised ultrasound clinic and form the database, where they were assigned, numbers ranging from 0 to 119. For convenience, it was decided that images with even names have masses, whereas those with odd names do not(2)
Preprocessing: The images were transformed to monochrome using the free and open-source GIMP software (GNU Image Manipulation Program). Remove logos and labels to improve results. Figure4(b) depicts this(3)
Segmentation: During the segmentation process, still with the help of the GIMP tool, the pixel size of the images was redefined, extracting only the area of interest from the images (subtitles and logos were removed). This redefinition occurred so that all images had the same size after the editing process (450×450), without any loss of pixels of interest for the application, generating the image of interest (Figure 4(c)).(4)
Feature extraction (representation and description): At this stage, the code provided was developed using the Python programming language. The code aims to extract pixel values from the images resulting from the segmentation process, accurately characterizing the shapes present in them. As these images have a size corresponding to450×450 pixels, a vector with 202500 positions is extracted for each image in the databaseFigure 4
Ultrasound image during its processing steps. (a), (b), and (c). [(a) Acquisition: Original image; (b) Preprocessing: Monochrome image; (c) Segmentation: Image of interest].
(a)(b)(c)The function that does this extraction, where label, path, and name are user-defined entries that correspond to the output predicted by CNN, the location where the photos are stored, and the name of the created file, respectively. The extracted pixel values are saved in a text document’s features field. The name field specifies whether the text file is a CNN test or training file, and it is supplied as a parameter to the load image (path) method. This function’s output corresponds to the neural network training file. This document’s label field corresponds to the output that the image must generate in the final layer of neurons of the constructed neural network model (where 10 presents the characteristic to be identified and 01 does not).The images were divided into 10 subgroups of 12 images, each for the use of the k-fold cross-validation statistical approach withk=10. (Totalling the 120 images in the database), nine of them were progressively written to the test file, while the remaining portion was written to the training file. This method was repeated ten times using a cyclic permutation of the subsets, resulting in ten separate training files and their associated test files. The number of photographs determines the value of the image amount variable. Gens that make up a subset multiplied by the number of training subsets, ie, 129=108, specifying which file the photos of a specific subset will be recorded in and directly dependent on the defined k. In this scenario, 108 images will be captured in the training file, and 12 will be recorded in the test file (Figure 5).Figure 5
Image represented by its pixel values and its respective label (a) and (b).
(a)
Generated training file(b)
Generated test file


#### 2.2.3. Classification: Recognition and Interpretation

This stage was carried out with the assistance of a CNN written in Python and deployed with the help of the CNTK framework. The developed model can be run on both a CPU and a GPU without compromising the ability to identify masses in the ultrasound images of livers used in the tests. It is worth noting that these are the most visible extracts from the implemented functions and for a better comprehension of the created code. The create reader function is in charge of deserializing the prior code’s training and test files and using them as standard CNTK input parameters, instantiated as C. This function takes as inputs path, is training, input dim, and number of label classes, which correspond to the locations of the training and test files, a flag indicating the training process, image pixel vector size (in this case 202500), and the number of possible outputs (whether it has mass or not).The CNN model is defined in the function create the model. The definition of a convolutional layer followed by max-pooling stands out in the snippets. Layers of C. convolution2D produce a function that takes an input and performs the convolution operation on it. This return is used as a parameter for the C.layers return. Max-pooling is a function that accepts an argument and performs the max-pooling operation on it. The result of the last max-pooling layer is defined as an attribute of the C. layers after the convolution and max-pooling layers are defined. Dense layers of C. dense generate an MPL with the number of output classes specified as a parameter (number of output classes).A loss function was designed to quantify how well the function inferred by the network fits the training instances. It measures the difference (or error) that the network makes during the recognition process. Using the softmax activation function, a cross entropy operation is done between the labels and the prediction probability of the instantiation network. Because we will be using the softmax operation in conjunction with the loss function during training, the final dense layer does not require an activation function. The loss calculation function is called during the training process, and the result is used as one of the arguments of C. trainer, a class built by CNTK that contains the neural network training algorithms. The neural network was trained using examples that were both positive and negative. Positive instances are associated with input training data that has the desired objective (masses). Negative examples (without mass) were added to the training data to instruct the network not to confuse changes with the target to reduce the number of false positives. The training file contains 108 photos, 54 of which are positive examples and 54 of which are negative ones. The code divides the files into nine batches of 12 photos each. The neural network input map is then defined, wherey represents the labels and x represents the training file features. At each iteration of the repetition structure, the data variable receives a fresh batch of images retrieved from the neural network’s input map. Data is then passed as a parameter to the training function implemented by CNTK within the trainer class.A training season concludes when the network has sorted all of the images in the training file. Each training file received ten epochs (number of sweeps to trainwith=10), and the test file received one epoch. It should be noticed that there are ten separate training files and test files. The testing process begins at the conclusion of the training phase. The test file contains 12 photos, six of which are positive examples and six negative examples, all of which are interleaved. The separate code specifies these photos as a batch, and then, identical to the training procedure, the test file is used to construct an input map for the neural network. The batch of images read from the neural network input map is stored in the data variable. Data, in turn, becomes a parameter of the test function created by CNTK within the trainer class, which returns the error value made by the network throughout the testing procedure for each batch. Meanwhile, test result is the sum of the partial mistakes of the training batch.Following the completion of the training process, the average error of evaluation of the CNN model is calculated and displayed to the user in the terminal. Finally, the do train test function invokes the functions for creating the CNN model (create a model) and reading (create reader) the training and test files (attributed to reader train and reader test, respectively), which are later used as input parameters for train test, initiating yet another process of training and testing the model. This execution process, dubbed as “step,” is repeated for each of the ten created training and test files. After this procedure, a text file is created in which the data obtained for each training phase is recorded and the errors connected with each training batch and the test batch. This file is depicted in Table1. The accuracy was then computed by taking the arithmetic mean of the mistakes found in each of the test batch executions.Table 1
Output file generated for each step.
MinibatchLossError (%)00.70358%10.61825%20.87750%30.70258%40.69133%50.69950%60.70975%70.692542%80.672433%∗Training time1415 sec; test time 467 sec and average error 50%.


## 2.1. Software Engineering Process

The software engineering process used in its development will be a variant of the spiral model, evolutionary prototyping, to build a CNN model that meets the given criteria. According to Marcomini [7], the development spiral is used to build prototypes under this paradigm. Prototypes add features until the final product is ready. To reach the end-user, a radiologist, the suggested neural network model will need a user-friendly interface and the ability to identify masses. In this regard, a technical requirements document for model execution and modification was described.


## 2.2. Development

This subsection explains the implementation of the proposed CNN source code and the tools involved in its development. This subsection also details the design decisions made and emphasizes training-related methods.


### 2.2.1. Development Environment

We used the Anaconda Navigator interface, which is a graphical user interface (GUI) that allows us to create programs and manage packages without using command lines. 2017-Anaconda, the CNTK (Microsoft Cognitive Toolkit) framework is used in the GUI, which supports C++, C#, Python, and BrainScript. According to NVIDIA [17–19], this toolkit efficiently implements CNN training for audio, image, and text data.We used Anaconda Navigator’s Spyder tool for source code development. Spyder is a Python IDE that allows you to edit, test, and debug source code [15, 20].


### 2.2.2. Image Processing

This subsection describes the phases involved in processing the images that make up the database for this study. This step is performed before the classification of images by the neural network and according to the methodology proposed in Figure3.
(1)
Acquisition of images: Figure4 shows ultrasound pictures of livers taken from the Flickr platform, a photo and video management and sharing service. (a) These photographs are accessible for download in the albums of a computerised ultrasound clinic and form the database, where they were assigned, numbers ranging from 0 to 119. For convenience, it was decided that images with even names have masses, whereas those with odd names do not(2)
Preprocessing: The images were transformed to monochrome using the free and open-source GIMP software (GNU Image Manipulation Program). Remove logos and labels to improve results. Figure4(b) depicts this(3)
Segmentation: During the segmentation process, still with the help of the GIMP tool, the pixel size of the images was redefined, extracting only the area of interest from the images (subtitles and logos were removed). This redefinition occurred so that all images had the same size after the editing process (450×450), without any loss of pixels of interest for the application, generating the image of interest (Figure 4(c)).(4)
Feature extraction (representation and description): At this stage, the code provided was developed using the Python programming language. The code aims to extract pixel values from the images resulting from the segmentation process, accurately characterizing the shapes present in them. As these images have a size corresponding to450×450 pixels, a vector with 202500 positions is extracted for each image in the databaseFigure 4
Ultrasound image during its processing steps. (a), (b), and (c). [(a) Acquisition: Original image; (b) Preprocessing: Monochrome image; (c) Segmentation: Image of interest].
(a)(b)(c)The function that does this extraction, where label, path, and name are user-defined entries that correspond to the output predicted by CNN, the location where the photos are stored, and the name of the created file, respectively. The extracted pixel values are saved in a text document’s features field. The name field specifies whether the text file is a CNN test or training file, and it is supplied as a parameter to the load image (path) method. This function’s output corresponds to the neural network training file. This document’s label field corresponds to the output that the image must generate in the final layer of neurons of the constructed neural network model (where 10 presents the characteristic to be identified and 01 does not).The images were divided into 10 subgroups of 12 images, each for the use of the k-fold cross-validation statistical approach withk=10. (Totalling the 120 images in the database), nine of them were progressively written to the test file, while the remaining portion was written to the training file. This method was repeated ten times using a cyclic permutation of the subsets, resulting in ten separate training files and their associated test files. The number of photographs determines the value of the image amount variable. Gens that make up a subset multiplied by the number of training subsets, ie, 129=108, specifying which file the photos of a specific subset will be recorded in and directly dependent on the defined k. In this scenario, 108 images will be captured in the training file, and 12 will be recorded in the test file (Figure 5).Figure 5
Image represented by its pixel values and its respective label (a) and (b).
(a)
Generated training file(b)
Generated test file


### 2.2.3. Classification: Recognition and Interpretation

This stage was carried out with the assistance of a CNN written in Python and deployed with the help of the CNTK framework. The developed model can be run on both a CPU and a GPU without compromising the ability to identify masses in the ultrasound images of livers used in the tests. It is worth noting that these are the most visible extracts from the implemented functions and for a better comprehension of the created code. The create reader function is in charge of deserializing the prior code’s training and test files and using them as standard CNTK input parameters, instantiated as C. This function takes as inputs path, is training, input dim, and number of label classes, which correspond to the locations of the training and test files, a flag indicating the training process, image pixel vector size (in this case 202500), and the number of possible outputs (whether it has mass or not).The CNN model is defined in the function create the model. The definition of a convolutional layer followed by max-pooling stands out in the snippets. Layers of C. convolution2D produce a function that takes an input and performs the convolution operation on it. This return is used as a parameter for the C.layers return. Max-pooling is a function that accepts an argument and performs the max-pooling operation on it. The result of the last max-pooling layer is defined as an attribute of the C. layers after the convolution and max-pooling layers are defined. Dense layers of C. dense generate an MPL with the number of output classes specified as a parameter (number of output classes).A loss function was designed to quantify how well the function inferred by the network fits the training instances. It measures the difference (or error) that the network makes during the recognition process. Using the softmax activation function, a cross entropy operation is done between the labels and the prediction probability of the instantiation network. Because we will be using the softmax operation in conjunction with the loss function during training, the final dense layer does not require an activation function. The loss calculation function is called during the training process, and the result is used as one of the arguments of C. trainer, a class built by CNTK that contains the neural network training algorithms. The neural network was trained using examples that were both positive and negative. Positive instances are associated with input training data that has the desired objective (masses). Negative examples (without mass) were added to the training data to instruct the network not to confuse changes with the target to reduce the number of false positives. The training file contains 108 photos, 54 of which are positive examples and 54 of which are negative ones. The code divides the files into nine batches of 12 photos each. The neural network input map is then defined, wherey represents the labels and x represents the training file features. At each iteration of the repetition structure, the data variable receives a fresh batch of images retrieved from the neural network’s input map. Data is then passed as a parameter to the training function implemented by CNTK within the trainer class.A training season concludes when the network has sorted all of the images in the training file. Each training file received ten epochs (number of sweeps to trainwith=10), and the test file received one epoch. It should be noticed that there are ten separate training files and test files. The testing process begins at the conclusion of the training phase. The test file contains 12 photos, six of which are positive examples and six negative examples, all of which are interleaved. The separate code specifies these photos as a batch, and then, identical to the training procedure, the test file is used to construct an input map for the neural network. The batch of images read from the neural network input map is stored in the data variable. Data, in turn, becomes a parameter of the test function created by CNTK within the trainer class, which returns the error value made by the network throughout the testing procedure for each batch. Meanwhile, test result is the sum of the partial mistakes of the training batch.Following the completion of the training process, the average error of evaluation of the CNN model is calculated and displayed to the user in the terminal. Finally, the do train test function invokes the functions for creating the CNN model (create a model) and reading (create reader) the training and test files (attributed to reader train and reader test, respectively), which are later used as input parameters for train test, initiating yet another process of training and testing the model. This execution process, dubbed as “step,” is repeated for each of the ten created training and test files. After this procedure, a text file is created in which the data obtained for each training phase is recorded and the errors connected with each training batch and the test batch. This file is depicted in Table1. The accuracy was then computed by taking the arithmetic mean of the mistakes found in each of the test batch executions.Table 1
Output file generated for each step.
MinibatchLossError (%)00.70358%10.61825%20.87750%30.70258%40.69133%50.69950%60.70975%70.692542%80.672433%∗Training time1415 sec; test time 467 sec and average error 50%.


## 2.2.1. Development Environment

We used the Anaconda Navigator interface, which is a graphical user interface (GUI) that allows us to create programs and manage packages without using command lines. 2017-Anaconda, the CNTK (Microsoft Cognitive Toolkit) framework is used in the GUI, which supports C++, C#, Python, and BrainScript. According to NVIDIA [17–19], this toolkit efficiently implements CNN training for audio, image, and text data.We used Anaconda Navigator’s Spyder tool for source code development. Spyder is a Python IDE that allows you to edit, test, and debug source code [15, 20].


## 2.2.2. Image Processing

This subsection describes the phases involved in processing the images that make up the database for this study. This step is performed before the classification of images by the neural network and according to the methodology proposed in Figure3.
(1)
Acquisition of images: Figure4 shows ultrasound pictures of livers taken from the Flickr platform, a photo and video management and sharing service. (a) These photographs are accessible for download in the albums of a computerised ultrasound clinic and form the database, where they were assigned, numbers ranging from 0 to 119. For convenience, it was decided that images with even names have masses, whereas those with odd names do not(2)
Preprocessing: The images were transformed to monochrome using the free and open-source GIMP software (GNU Image Manipulation Program). Remove logos and labels to improve results. Figure4(b) depicts this(3)
Segmentation: During the segmentation process, still with the help of the GIMP tool, the pixel size of the images was redefined, extracting only the area of interest from the images (subtitles and logos were removed). This redefinition occurred so that all images had the same size after the editing process (450×450), without any loss of pixels of interest for the application, generating the image of interest (Figure 4(c)).(4)
Feature extraction (representation and description): At this stage, the code provided was developed using the Python programming language. The code aims to extract pixel values from the images resulting from the segmentation process, accurately characterizing the shapes present in them. As these images have a size corresponding to450×450 pixels, a vector with 202500 positions is extracted for each image in the databaseFigure 4
Ultrasound image during its processing steps. (a), (b), and (c). [(a) Acquisition: Original image; (b) Preprocessing: Monochrome image; (c) Segmentation: Image of interest].
(a)(b)(c)The function that does this extraction, where label, path, and name are user-defined entries that correspond to the output predicted by CNN, the location where the photos are stored, and the name of the created file, respectively. The extracted pixel values are saved in a text document’s features field. The name field specifies whether the text file is a CNN test or training file, and it is supplied as a parameter to the load image (path) method. This function’s output corresponds to the neural network training file. This document’s label field corresponds to the output that the image must generate in the final layer of neurons of the constructed neural network model (where 10 presents the characteristic to be identified and 01 does not).The images were divided into 10 subgroups of 12 images, each for the use of the k-fold cross-validation statistical approach withk=10. (Totalling the 120 images in the database), nine of them were progressively written to the test file, while the remaining portion was written to the training file. This method was repeated ten times using a cyclic permutation of the subsets, resulting in ten separate training files and their associated test files. The number of photographs determines the value of the image amount variable. Gens that make up a subset multiplied by the number of training subsets, ie, 129=108, specifying which file the photos of a specific subset will be recorded in and directly dependent on the defined k. In this scenario, 108 images will be captured in the training file, and 12 will be recorded in the test file (Figure 5).Figure 5
Image represented by its pixel values and its respective label (a) and (b).
(a)
Generated training file(b)
Generated test file


## 2.2.3. Classification: Recognition and Interpretation

This stage was carried out with the assistance of a CNN written in Python and deployed with the help of the CNTK framework. The developed model can be run on both a CPU and a GPU without compromising the ability to identify masses in the ultrasound images of livers used in the tests. It is worth noting that these are the most visible extracts from the implemented functions and for a better comprehension of the created code. The create reader function is in charge of deserializing the prior code’s training and test files and using them as standard CNTK input parameters, instantiated as C. This function takes as inputs path, is training, input dim, and number of label classes, which correspond to the locations of the training and test files, a flag indicating the training process, image pixel vector size (in this case 202500), and the number of possible outputs (whether it has mass or not).The CNN model is defined in the function create the model. The definition of a convolutional layer followed by max-pooling stands out in the snippets. Layers of C. convolution2D produce a function that takes an input and performs the convolution operation on it. This return is used as a parameter for the C.layers return. Max-pooling is a function that accepts an argument and performs the max-pooling operation on it. The result of the last max-pooling layer is defined as an attribute of the C. layers after the convolution and max-pooling layers are defined. Dense layers of C. dense generate an MPL with the number of output classes specified as a parameter (number of output classes).A loss function was designed to quantify how well the function inferred by the network fits the training instances. It measures the difference (or error) that the network makes during the recognition process. Using the softmax activation function, a cross entropy operation is done between the labels and the prediction probability of the instantiation network. Because we will be using the softmax operation in conjunction with the loss function during training, the final dense layer does not require an activation function. The loss calculation function is called during the training process, and the result is used as one of the arguments of C. trainer, a class built by CNTK that contains the neural network training algorithms. The neural network was trained using examples that were both positive and negative. Positive instances are associated with input training data that has the desired objective (masses). Negative examples (without mass) were added to the training data to instruct the network not to confuse changes with the target to reduce the number of false positives. The training file contains 108 photos, 54 of which are positive examples and 54 of which are negative ones. The code divides the files into nine batches of 12 photos each. The neural network input map is then defined, wherey represents the labels and x represents the training file features. At each iteration of the repetition structure, the data variable receives a fresh batch of images retrieved from the neural network’s input map. Data is then passed as a parameter to the training function implemented by CNTK within the trainer class.A training season concludes when the network has sorted all of the images in the training file. Each training file received ten epochs (number of sweeps to trainwith=10), and the test file received one epoch. It should be noticed that there are ten separate training files and test files. The testing process begins at the conclusion of the training phase. The test file contains 12 photos, six of which are positive examples and six negative examples, all of which are interleaved. The separate code specifies these photos as a batch, and then, identical to the training procedure, the test file is used to construct an input map for the neural network. The batch of images read from the neural network input map is stored in the data variable. Data, in turn, becomes a parameter of the test function created by CNTK within the trainer class, which returns the error value made by the network throughout the testing procedure for each batch. Meanwhile, test result is the sum of the partial mistakes of the training batch.Following the completion of the training process, the average error of evaluation of the CNN model is calculated and displayed to the user in the terminal. Finally, the do train test function invokes the functions for creating the CNN model (create a model) and reading (create reader) the training and test files (attributed to reader train and reader test, respectively), which are later used as input parameters for train test, initiating yet another process of training and testing the model. This execution process, dubbed as “step,” is repeated for each of the ten created training and test files. After this procedure, a text file is created in which the data obtained for each training phase is recorded and the errors connected with each training batch and the test batch. This file is depicted in Table1. The accuracy was then computed by taking the arithmetic mean of the mistakes found in each of the test batch executions.Table 1
Output file generated for each step.
MinibatchLossError (%)00.70358%10.61825%20.87750%30.70258%40.69133%50.69950%60.70975%70.692542%80.672433%∗Training time1415 sec; test time 467 sec and average error 50%.


## 3. Results & Discussion

The pooling function decreases the network’s data dimensionality. This reduction is significant since it promotes agility in training while also creating spatial invariance. With a2×2 window, the max-pooling function was used to do this work, reducing the input of the next layer by half. This window size was chosen since the database photos are already very small, and a wider window would severely limit the number of layers that could be applied.The sizes3×3 and 5×5 were examined to define the size of the convolutional window, with both using two convolution layers followed by max-pooling and displacement step of 1. In addition to obtaining a lower average classification error, the 3×3 size window allows for the insertion of more convolution layers followed by max-pooling over the input image. It is vital to note that the images are only 450×450 pixels in size and that both the convolution and max-pooling processes generate large reductions. Thus, their physical limit must be considered. After that, a 3×3 convolutional window with a displacement step of 1 was utilised.After defining the convolutional and max-pooling windows, the number of layers applied to the pictures was changed before being used as input for the classificatory MLP. The training and testing process with the k-fold approach was repeated for each number of convolution layers followed by max-pooling, both on the CPU and the GPU. The results of these tests are depicted in the figures as follows.Figure6 Explains the results obtained for the CPU1C test. This test obtained an average error of 50%, that is, an accuracy of 50%. The test subset error was 41.666%.Figure 6
Results obtained for the CPU1C test, CPU2C test, and CPUC test, along with average results.The results of the CPU2C test are shown in Figure6. The average error decreases from 50% of the CPU1C test to 34.165% during test execution, yielding an average accuracy of 65.835%. The test subset error was 25.0%, corresponding to 75.0% accuracy. Figure 6 depicts the results of the CPU3C test, which revealed a 15% inaccuracy in the test subgroup. This results in an average accuracy of 85%. The error in the test subgroup was 8.333%, resulting in an accuracy of 91.666%. The averages for test execution in a CPU were then displayed in Figure 6, along with the averages of training and test time in seconds (s). Attempting to make data visualisation easier, Figure 7 illustrates the results of the GPU1C test, which, like the CPU1C test, yields an average predicted error of 50%. When the execution times were compared, it was discovered that the GPU had shorter average times for both the training process (293.3 s) and the test process (34.6 s) than the CPU (3126 s and 351.8 s, respectively).Figure 7
Results obtained for the GPU1C test, GPU2C test, and GPU3C test, along with average results.As in the previous test, the GPU has an advantage in training and test runtimes, with 496.7 s and 55.7 s (Figure7), whereas the CPU has an average of 5348.5 s and 7595.3 s. As in prior testing, the GPU outperformed the CPU in execution time. Figure 7 shows the times obtained, as well as their average. Finally, Figure 7 shows the average training and test execution times on the GPU and the average error for each test. The graph depicted in Figure 8 was created to improve the visualisation of the data received. When Figures 6 and 7 are compared, the percentage of inaccuracy in identifying masses and nonmasses is the same for each layer added. The data in these figures are shown in Figure 8, which compares the CPU and GPU training times, and Figure 8 compares their test times.Figure 8
Comparison graph of training times between CPU and GPU and comparison graph of test times between CPU and GPU.The CNN model with only one convolutional layer followed by max-pooling results in a neural network with 22500 input neurons and 2 output neurons (massless or massless), totaling 197346 neurons. This is because the input image is450×450 pixels in size, and at the end of the first convolution layer, we have a reduction to 150×150 pixels, which serves as input for the max-pooling, where the spatial reduction is yields an image of 75×75 pixels. The max-pooling function’s output is subsequently used as input to the neural network. The CNN model with only two convolutional layers and max-pooling has 5625 input neurons and 2 output neurons for 12634 neurons. The 75×75 pixels that served as input to the neural network in the previous model are now parameters for the following convolutional layer, whose output is 25×25 pixels.Similarly, to the last model, the25×25 pixels are fed into the max-pooling function, the result of which is fed into the MLP. The CNN with three convolutive layers followed by max-pooling goes through the same process as the previous networks, with a total of 10322 neurons. This development is depicted in Figure 9. Therefore, it is possible to construct a graph to observe the network’s learning for each k of the k-fold technique run.Figure 9
Learning graph of the neural networks proposed for the execution ofK=9.For the proposed CNN models, having one, two, and three convolutive layers followed by max-pooling, an average accuracy of 50%, 65, 835%, and 85%, respectively, was reached. As for the test subsets, the accuracies correspond to 58, 666%, 75%, and 91, 666%. Figure10 explains this difference in accuracy by comparing the average errors and the errors of the test subset for each number of layers.Figure 10
Comparative graph between the average errors and the errors obtained for the test subset and comparative graph of training and test times between CPU and GPU.A neural network with 58.666% accuracy is not practical because it behaves as if the output is generated at random rather than learning the properties that lead to image classification into “has mass” and “has no mass.” In the case of the network with a 75% accuracy, it is worth noting that the neural network’s learning was aided by the addition of layers. The resultant value was still not what was needed; therefore, the training procedure was repeated with another layer. The accuracy increased to 91.666% after that. Due to the database picture size constraints, adding another convolution layer and max-pooling is impossible. As a result, using the k-fold cross-validation method as a metric, it was determined that the third proposed model had the best generalisation capability for the database employed. When the execution times of the built neural networks were analysed, it was discovered that the training procedure took an average of 8.85 times the test time. Even with a relatively small database, consisting of 120 photos with 202500 pixels apiece, the GPU acquired a considerable advantage over the CPU (approximately 10.6 times faster) for both the training and the test cases. Figure10 shows how the GPU outperforms the CPU regarding training and testing times. It can be deduced from this that as the size or number of photos in the database grows, utilizing a high-performance environment such as the GPU becomes increasingly important in the neural network training process.


## 4. Conclusions

Hepatocarcinoma (also known as hepatocellular carcinoma), cholangiocarcinoma (which affects the bile ducts within the liver), angiosarcoma (blood vessel tumour), and hepatoblastoma (which affects children) are all examples of masses originating in the liver. The CNN model was able to identify these masses in ultrasound pictures of the liver with an average accuracy of 85% for the collection of photos investigated. This CNN model can be extended to distinguish between masses and nonmasses in addition to classifying them. To assist in identification, increase the number of photos in the database and identify each type of dough separately. A set of photographs of healthy livers with their labels should also be kept to avoid false recognition. As the database grows, so will CNN’s training and testing time, necessitating the usage of a high-processing-capacity environment. After completing training and testing, a graphical interface can be developed that allows users to submit liver ultrasounds for analysis, resulting in a diagnostic aid tool that can be used for educational purposes, collaboration with training, and qualifying medical students.It is also intended to investigate the performance of neural network training and testing processes using other CPU and GPU architectures in terms of execution time in the future.



---

*Source: 1688076-2022-09-17.xml*