# Intelligent Microservices-based Approach to Support Data Analytics for IoT Applications

Safa Ben Atitallah

<figure><img src="image_1.png" /></figure>

## To cite this version:

Safa Ben Atitallah. Intelligent Microservices-based Approach to Support Data Analytics for IoT Applications. Computer Science [cs]. University of Manouba, 2023. English. NNT: tel-04166458

HAL Id: tel-04166458

https://theses.hal.science/tel-04166458

Submitted on 19 Jul 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
UNIVERSITY OF MANOUBA
NATIONAL SCHOOL OF COMPUTER SCIENCE

PHD THESIS
For the degree of
Doctor in Computer Science

Intelligent Microservices-based Approach to
Support Data Analytics for IoT Applications

By
Safa Ben Atitallah

<figure><img src="image_2.png" /></figure>

Publicly defended on 10 July 2023, in the front of the jury composed of:

President: Imed Riadh FARAH, Professor, University of Manouba, Tunisia.

Examiner: Yassine JAMOUSI, Professor, Sultan Qaboos University, Oman.

Examiner: Faisal SAEED, Associate Professor, Birmingham City University, United Kingdom.

Examiner: Hanen IDOUDI, Associate Professor, University of Manouba, Tunisia.

Thesis Director: Henda BEN GHEZALA, Professor, University of Manouba, Tunisia.

Thesis Co-Director: Wadii BOULILA, Associate Professor, Prince Sultan University, Saudi Arabia.

Thesis Supervisor: Maha DRISS, Assistant Professor, Prince Sultan University, Saudi Arabia.
Abstract

The rapid development of the Internet of Things (IoT) has created a huge and complex network of interlinked equipment generating massive amounts of data. To efficiently analyze this data for a variety of applications, we propose an intelligent approach that utilizes multi-source and heterogeneous data from IoT devices.

Our approach incorporates centralized and distributed learning techniques and is implemented through a set of secure, flexible, and scalable microservices. We divide the data analytics functionality into containerized microservices and train analytics models using AI methods before deploying them on edge computing nodes. We also utilize transfer learning and ensemble learning techniques to enhance model generalization and prediction accuracy.

The adoption of a microservices-based architecture offers several advantages, including scalability, flexibility, reliability, modularity, and integration. This simplifies the development, implementation, and management of intricate machine learning and deep learning systems compared to conventional monolithic architectures.

Our approach has significant implications for the IoT industry, providing a powerful tool for unlocking valuable insights from IoT data. We validate our approach through a set of IoT case studies and comparative studies, demonstrating its effectiveness and practicality in real-world scenarios. Overall, our suggested approach offers a robust and effective solution for analyzing IoT data and deriving important insights that can be used in a wide range of use cases.

**Keywords:** Microservices architecture; Internet of Things; Data analytics; Deep learning; Federated learning; Transfer learning; Ensemble learning; IoT applications; Smart services; Usability; Flexibility.
First of all, I would like to thank God very much.
I dedicate my thesis work
to my dear parents Mehrez and Ahlem,
to my dear husband Omar,
to my daughters Nouha and Roua,
to my son Ibrahim,
to my sister Marwa,
to my brothers Jabeur and Ahmed,
to all my family,
to all my friends,
and to all people I love.
Acknowledgements

Above all else, I thank Allah for blessing me with good health and unwavering
patience. I was able to navigate the numerous challenges and obstacles that I en-
countered during my Ph.D. journey and throughout my life because of His divine
guidance and help.

I want to express my deepest sincerest gratitude to my doctoral research director, Pr. Henda HAJJAMI BEN GHZELA, Professor at the University of Manouba, for her exceptional guidance and unwavering support during my thesis.

I am deeply indebted to my thesis co-director, Dr. Wadii BOULILA, whose selfless guidance and mentorship have been instrumental in my academic journey. His valuable support, constructive feedback, and encouragement have inspired me to become a better researcher.

Furthermore, I would also like to extend my appreciation to my advisor, Dr. Maha DRISS, whose unwavering support and encouragement have been instrumental in my success. I am incredibly thankful for the priceless chance to pursue my Ph.D. under her tutelage and guidance.

I wish to thank Dr. Imed Riadh FARAH, Professor at the University of Manouba, for the honor he has bestowed upon me by presiding over my thesis defense jury.

I am also deeply thankful to Dr. Yassine JAMMOUSI and Dr. Faisal SAEED for taking the time to review this thesis. I thank them for the remarks and critiques they have provided me, which have helped improve this manuscript.

I also thank Dr. Hanen IDOUDI, a Professor at the University of Manouba, who kindly agreed to be part of this jury.

Finally, I wish to offer my heartfelt appreciation to my family for their ongoing encouragement and understanding throughout my research journey. Their unconditional love, encouragement, and understanding have been a constant source of motivation and strength for me all through the good and bad moments of this challenging path.
# Contents

## Abstract i

## 1 General Introduction 1

1.1 Context and Motivation 1

1.2 Problem Statement 2

1.3 Objectives 4

1.4 Contributions 4

1.5 Thesis Outline 6

## 2 Preliminary Concepts and Theoretical Background 7

2.1 Introduction 8

2.2 Internet of Things 9

2.2.1 Definition 9

2.2.2 IoT Architecture 11
<table>
  <tbody>
    <tr>
      <td>2.2.3</td>
      <td>IoT Big Data Characteristics</td>
      <td>12</td>
    </tr>
    <tr>
      <td>2.2.4</td>
      <td>Applications of IoT</td>
      <td>13</td>
    </tr>
    <tr>
      <td>2.3</td>
      <td>Microservices Paradigm: Overview</td>
      <td>15</td>
    </tr>
    <tr>
      <td>2.3.1</td>
      <td>Microservices' Definition</td>
      <td>16</td>
    </tr>
    <tr>
      <td>2.3.2</td>
      <td>Microservices Advantages for IoT Data Analytics</td>
      <td>17</td>
    </tr>
    <tr>
      <td>2.4</td>
      <td>Data Analytics</td>
      <td>18</td>
    </tr>
    <tr>
      <td>2.4.1</td>
      <td>Paradigms</td>
      <td>19</td>
    </tr>
    <tr>
      <td>2.4.2</td>
      <td>Types of Data Analytics</td>
      <td>20</td>
    </tr>
    <tr>
      <td>2.4.3</td>
      <td>Analytical Tasks</td>
      <td>21</td>
    </tr>
    <tr>
      <td>2.4.4</td>
      <td>Evaluation Metrics</td>
      <td>22</td>
    </tr>
    <tr>
      <td>2.4.5</td>
      <td>Computational Infrastructures for IoT Big Data Analytics</td>
      <td>24</td>
    </tr>
    <tr>
      <td>2.4.5.1</td>
      <td>Cloud Computing</td>
      <td>24</td>
    </tr>
    <tr>
      <td>2.4.5.2</td>
      <td>Fog Computing</td>
      <td>26</td>
    </tr>
    <tr>
      <td>2.4.5.3</td>
      <td>Edge Computing</td>
      <td>26</td>
    </tr>
    <tr>
      <td>2.4.6</td>
      <td>Data Analytics Techniques</td>
      <td>27</td>
    </tr>
    <tr>
      <td>2.4.7</td>
      <td>Deep Learning</td>
      <td>28</td>
    </tr>
    <tr>
      <td>2.4.7.1</td>
      <td>Convolutional Neural Network</td>
      <td>31</td>
    </tr>
    <tr>
      <td>2.5</td>
      <td>Transfer Learning</td>
      <td>34</td>
    </tr>
    <tr>
      <td>2.5.1</td>
      <td>MobileNetV2</td>
      <td>34</td>
    </tr>
    <tr>
      <td>2.5.2</td>
      <td>DenseNet201</td>
      <td>35</td>
    </tr>
  </tbody>
</table>
<table><tr><td>2.5.3</td><td>InceptionV3</td><td>35</td></tr><tr><td>2.5.4</td><td>ResNet50</td><td>35</td></tr><tr><td>2.5.5</td><td>Xception</td><td>36</td></tr><tr><td>2.6</td><td>Federated Learning</td><td>36</td></tr><tr><td>2.7</td><td>Ensemble Methods</td><td>37</td></tr><tr><td>2.8</td><td>Conclusion</td><td>37</td></tr></table>

## 3 Related Work: IoT Data Analytics Approaches Based on Microservices Architecture 38

<table><tr><td>3.1</td><td>Introduction</td><td>39</td></tr><tr><td>3.2</td><td>The Role of Data Analytics in IoT</td><td>39</td></tr><tr><td>3.3</td><td>IoT Data Analytics Approaches with Microservices Architecture</td><td>40</td></tr><tr><td>3.3.1</td><td>Analytics Techniques</td><td>40</td></tr><tr><td>3.3.1.1</td><td>Approaches Based on Statistical Models</td><td>41</td></tr><tr><td>3.3.1.2</td><td>Approaches Based on Typical ML Algorithms</td><td>42</td></tr><tr><td>3.3.1.3</td><td>Approaches Based on Deep Learning Models</td><td>44</td></tr><tr><td>3.3.2</td><td>Approaches Based on Federated Learning</td><td>46</td></tr><tr><td>3.3.3</td><td>Approaches Based on Transfer Learning</td><td>47</td></tr><tr><td>3.4</td><td>Comparison of The Microservices-based IoT Data Analytics Approaches</td><td>48</td></tr><tr><td>3.5</td><td>Conclusion</td><td>54</td></tr></table>
<table>
  <tbody>
    <tr>
      <td>4.4.1.3</td>
      <td>Interpretation</td>
      <td>76</td>
    </tr>
    <tr>
      <td>4.4.2</td>
      <td>Overview of the Proposed Approach</td>
      <td>81</td>
    </tr>
    <tr>
      <td>4.4.3</td>
      <td>Context-Aware Microservice Selection</td>
      <td>83</td>
    </tr>
    <tr>
      <td>4.5</td>
      <td>Conclusion</td>
      <td>84</td>
    </tr>
    <tr>
      <td><b>5</b></td>
      <td><b>Experimentation</b></td>
      <td><b>85</b></td>
    </tr>
    <tr>
      <td>5.1</td>
      <td>Introduction</td>
      <td>86</td>
    </tr>
    <tr>
      <td>5.2</td>
      <td>Environment Setup</td>
      <td>87</td>
    </tr>
    <tr>
      <td>5.3</td>
      <td>Case Study 1: Cybersecurity</td>
      <td>88</td>
    </tr>
    <tr>
      <td>5.3.1</td>
      <td>Experiment 1: Detection and Classification of IoT Malware</td>
      <td>89</td>
    </tr>
    <tr>
      <td>5.3.1.1</td>
      <td>Dataset</td>
      <td>89</td>
    </tr>
    <tr>
      <td>5.3.1.2</td>
      <td>Data Preprocessing</td>
      <td>91</td>
    </tr>
    <tr>
      <td>5.3.1.3</td>
      <td>Models Building</td>
      <td>91</td>
    </tr>
    <tr>
      <td>5.3.1.4</td>
      <td>Interpretation</td>
      <td>92</td>
    </tr>
    <tr>
      <td>5.3.1.5</td>
      <td>Experiments Evaluation</td>
      <td>92</td>
    </tr>
    <tr>
      <td>5.3.2</td>
      <td>Experiment 2: Detection and Classification of Denial of Service Attacks in IoT Networks</td>
      <td>97</td>
    </tr>
    <tr>
      <td>5.3.2.1</td>
      <td>Dataset</td>
      <td>97</td>
    </tr>
    <tr>
      <td>5.3.2.2</td>
      <td>Data Preprocessing</td>
      <td>98</td>
    </tr>
    <tr>
      <td>5.3.2.3</td>
      <td>Models Building</td>
      <td>99</td>
    </tr>
    <tr>
      <td>5.3.2.4</td>
      <td>Interpretation</td>
      <td>99</td>
    </tr>
  </tbody>
</table>
<table><tr><td>5.3.2.5</td><td>Comparison with Similar Studies</td><td>101</td></tr><tr><td>5.4</td><td>Case Study 2: Healthcare</td><td>101</td></tr><tr><td>5.4.1</td><td>Diagnosis of Pneumonia Using Chest X-Ray Imagery</td><td>102</td></tr><tr><td>5.4.1.1</td><td>Dataset</td><td>102</td></tr><tr><td>5.4.1.2</td><td>Data Preprocessing</td><td>104</td></tr><tr><td>5.4.1.3</td><td>Models Building</td><td>104</td></tr><tr><td>5.4.1.4</td><td>Interpretation Stage</td><td>104</td></tr><tr><td>5.4.1.5</td><td>Experimental Evaluation</td><td>105</td></tr><tr><td>5.5</td><td>Case Study 3: Environment</td><td>107</td></tr><tr><td>5.5.1</td><td>Smart Monitoring of Water Environments</td><td>107</td></tr><tr><td>5.5.1.1</td><td>Dataset</td><td>109</td></tr><tr><td>5.5.1.2</td><td>Data Preprocessing</td><td>109</td></tr><tr><td>5.5.1.3</td><td>Models Building</td><td>110</td></tr><tr><td>5.5.1.4</td><td>Interpretation</td><td>110</td></tr><tr><td>5.6</td><td>Limitations of The Proposed Approach</td><td>111</td></tr><tr><td>5.7</td><td>Conclusion</td><td>112</td></tr><tr><td>6</td><td>Conclusion and Future Lines of Research</td><td>113</td></tr><tr><td>6.1</td><td>Summary and Conclusions</td><td>113</td></tr><tr><td>6.2</td><td>Future Work and Research Directions</td><td>116</td></tr></table>
List of Figures

<table><tr><td>2.1</td><td>The design of IoT systems.</td><td>9</td></tr><tr><td>2.2</td><td>IoT architecture.</td><td>12</td></tr><tr><td>2.3</td><td>The different steps of the data analytics process.</td><td>19</td></tr><tr><td>2.4</td><td>Different types of learning techniques.</td><td>20</td></tr><tr><td>2.5</td><td>Data analytics categorization.</td><td>21</td></tr><tr><td>2.6</td><td>Fog nodes act as an intermediary between edge devices and the cloud.</td><td>27</td></tr><tr><td>2.7</td><td>A diagram that uses overlapping circles to show the distinctions among AI, ML, and DL.</td><td>28</td></tr><tr><td>2.8</td><td>The neuron structure.</td><td>29</td></tr><tr><td>2.9</td><td>Explanation of the operations carried out within the convolutional layer</td><td>32</td></tr><tr><td>2.10</td><td>Explanation of the operations carried out within the pooling layer.</td><td>32</td></tr><tr><td>4.1</td><td>Abstract architecture of the proposed approach design</td><td>59</td></tr><tr><td>4.2</td><td>The Edge Node architecture in the decentralized learning method.</td><td>64</td></tr></table>
4.3 The generalized workflow between the FL server and the client following the FL method. The communication between Step 3 and 4 are done for predefined rounds. 77

4.4 The generalized workflow between the cloud and edge following the CL method. 78

4.5 Overview of the proposed approach: processing, and analyzing IoT data using microservices, FL,TL, and Fusion Methods 82

5.1 Experimental setup of the proposed approach used for different IoT use case scenarios. 88

5.2 IoT malware detection and classification Scenario. 90

5.3 Performance results of the MobileNet CNN resulting from the adoption of FL and CL approaches. 93

5.4 The execution time of different microservices in the process of malware detection. 95

5.5 Execution time in ms for data analytics process with a different number of FL clients. 96

5.6 Visual representations of the dataset using IGTD technique. 98

5.7 Comparison of the proposed approach's performance with existing studies in the literature. 101

5.8 Pneumonia detection scenario following the proposed approach. 103

5.9 Microservices execution time for the pneumonia detection process. 106

5.10 The proposed smart monitoring of water environment scenario following the proposed approach (Centralized method). 108

5.11 Normalized confusion matrices for the water zones classification without embedding using: (a) SVM, (b) LR, (c) KNN 111
List of Tables

<table><tr><td>5.1</td><td>Hyperparameters' values of the considered CNNs.</td><td>91</td></tr><tr><td>5.2</td><td>Experimental results of the single CNN models, the centralized Dempster-based fusion classifier, and the proposed approach.</td><td>93</td></tr><tr><td>5.3</td><td>Evaluation of our approach against previous methods utilizing Male-Vis dataset</td><td>96</td></tr><tr><td>5.4</td><td>Specified hyper-parameters for optimal models performance</td><td>99</td></tr><tr><td>5.5</td><td>Evaluation results of TL-based classifiers on the WSN-DS dataset.</td><td>100</td></tr><tr><td>5.6</td><td>Performance results of the learned DL models and the proposed classifier.</td><td>105</td></tr><tr><td>5.7</td><td>Comparison of the proposed approach results with our previously published study</td><td>106</td></tr><tr><td>5.8</td><td>Performance evaluation of water zones classification using embedding techniques</td><td>111</td></tr></table>
Acronyms

**AI** Artificial Intelligence. 27

**ANN** Artificial Neural Network. 30

**ARIMA** Autoregressive Integrated Moving Average. 41

**CNN** Convolutional Neural Network. 30

**DBN** Deep Belief Network. 30

**DL** Deep Learning. 4

**DNN** Deep Neural Networks. 30

**DoS** Denial-of-Service. 97

**DRL** Deep Reinforcement Learning. 30

**DST** Dempster-Shafer Theory. 102

**DT** Decision Tree. 43

**FedAvg** Federated Averaging. 68

**FL** Federated Learning. 3

**FN** False Negative. 23

**FP** False Positive. 23
**GAN** Generative Adversarial Network. 30

**IoT** Internet of Things. 1

**KNN** K-Nearest Neighbours. 43

**LDA** Linear discriminant analysis. 43

**LEACH** Low Energy Aware Cluster Hierarchy. 97

**LISPS** Lightweight IoT Smart Public Safety. 44

**LR** Linear Regression. 43

**MCC** Matthews Correlation Coefficient. 24

**ML** Machine Learning. 10

**MLMA** Machine Learning in Microservices Architecture. 43

**MLP** MultiLayer Perceptron. 44

**PE** Portable Execution. 94

**QoS** Quality of service. 54

**RBM** Restricted Boltzmann Machines. 30

**RNN** Recurrent Neural Network. 30

**SAE** Stacked Autoencoder. 30

**SOA** Service Oriented Architecture. 6

**SVM** Support Vector Machine. 43

**TL** Transfer Learning. 8

**TN** True Negative. 23

**TP** True Positive. 23

**WKG** Water Knowledge Graph. 109
**WQI** Water Quality Index. 109

**WSNs** Wireless Sensor Networks. 97

**XDL** Explainable Deep Learning. 117
Chapter 1

# General Introduction

## Contents

<table><tr><td>1.1 Context and Motivation</td><td>1</td></tr><tr><td>1.2 Problem Statement</td><td>2</td></tr><tr><td>1.3 Objectives</td><td>4</td></tr><tr><td>1.4 Contributions</td><td>4</td></tr><tr><td>1.5 Thesis Outline</td><td>6</td></tr></table>

## 1.1 Context and Motivation

Nowadays, the Internet of Things (IoT) has emerged to be a disruptive Internet technology. It helps to propel the Internet to the next level by connecting enormous heterogeneous devices, sensors, and actuators [1]. These devices can communicate and engage with the actual world, gather environmental data, and support smart decision-making processes [2]. The incorporation of these intelligent equipment in different sectors of the environment has enabled the creation of intelligent and innovative services, providing valuable insights and actionable knowledge. The IoT has facilitated the establishment of applications for various sectors, including smart cities, home automation, industrial management, and many more [3]. An IoT application could be identified as a set of automated operations and data combined
with other objects, including hardware and software. All these entities interact
and exchange information with each other and the environment to achieve shared
objectives and goals.

Alongside the rise of IoT and cloud computing, the amount of data generated
has increased exponentially, leading to the need for efficient predictive analytics for
various IoT applications. Consequently, IoT big data analytics has emerged as a
key study field, as traditional methods of data processing are unable to deal with
huge quantities of data [4]. Predictive analytics is a crucial process that transforms
raw data into actionable insights and has been widely applied in diverse domains,
including healthcare, smart home, energy management, and social media analysis,
among others [5]. Therefore, it is imperative to develop novel data processing tech-
niques and predictive analytic models that can handle the scale and complexity of
IoT data for better decision-making and improved performance of IoT applications.

The growing volume of data produced through IoT devices and the need for
efficient, scalable, and reusable analytics solutions have brought about several chal-
lenges [6]. These challenges include the heterogeneity of data with various types and
forms, the growing volume of data, monolithic service development approaches, and
the efficiency of predictive analytics. To address these challenges, there is a need to
design a novel approach based on the microservices architecture that leverages the
capabilities of data analytics techniques. This approach will enable the development
of intelligent microservices that provide efficient predictive analytics, improving the
efficiency of IoT applications in different domains. By adopting a microservices-
based approach, it will be possible to overcome the limitations of monolithic archi-
tectures and enable the development of scalable, reusable, and efficient IoT-based
analytics solutions.

## 1.2 Problem Statement

IoT devices are generating huge quantities of data from everywhere, each and every
time, and in different formats. The collected data is moved to the cloud to be
monitored, processed, and investigated in order to assist smart IoT applications.
Employing data analytics on these huge data sets plays an essential role in various
IoT applications, such as energy management, business, security, etc. Indeed, big
data analytics has emerged as a major IT trend that involves academia, research
institutions, and industries [7]. However, the IoT analytics community is confronted
by new issues and challenges.

Managing data from multiple sources can be difficult because each device in
the IoT ecosystem may produce data in different formats and with varying levels
of accuracy. In addition, the data may be incomplete or contain errors that need
to be corrected before it can be analyzed effectively. This can lead to delays and
inaccuracies in data processing, which can negatively impact the performance of
IoT applications. Dealing with different data types and formats is also challenging,
as IoT devices generate data in a wide range of formats, such as text, images, and
sensor data. Different data types require different processing methods, and handling
them all is a complex and time-consuming procedure. Moreover, the heterogeneity
of data types in IoT can also result in interoperability issues, which further exac-
erbate the problem. In addition, managing the fast-growing data volumes can be a
significant challenge, as the volume of data produced by IoT equipment is increas-
ing exponentially. Traditional techniques of data storing and processing may not be
able to keep up with this growth, leading to scalability issues that can impact the
effectiveness in general and the re-usability of IoT applications.

In furtherance of the difficulties associated with processing large amounts of
data, transferring this data stored on the edge equipment to the centralized server
can also be costly and risky [8]. This is because IoT devices are often located in
dispersed locations, far from the central server. Transferring large volumes of data
from these devices to the centralized server can be time-consuming and expensive.

Furthermore, transferring data from IoT devices to the central server may also
expose data to various attacks and threats. These threats include eavesdropping,
data tampering, and denial-of-service assaults. Attackers can intercept data during
transmission, steal sensitive information, or modify the data for malicious purposes.
This can compromise the privacy and security of data and potentially cause harm
to individuals and organizations.

In summary, developing a scalable design of data analytics approach for IoT
applications that takes into account all these considerations is crucial to successfully
handling and investigating the huge volumes of data produced by IoT devices. The
efficiency of IoT data analytics may be optimized and improved by using technologies
such as edge computing and Federated Learning (FL).
## 1.3 Objectives

This work aims to propose a microservices-based architecture for the creation of IoT applications. The suggested design is made up of software components that have been designed to support centralized and distributed learning and satisfy the needs of IoT applications. We advocate an intelligent microservices-based approach to guarantee data privacy and deliver proper data analytics to the end users. We propose decomposing the IoT application into a linkable set of microservices running across different computing layers (cloud/edge). We focus on delivering an appropriate service computation distribution method that takes into account the layer's constraints and decreases data transmission cost. The suggested approach can aid in developing sustainable computing architectures by allowing an efficient distribution of data processing at the device's edge.

## 1.4 Contributions

The following are the primary contributions of this research study:

* A survey of leveraging Deep Learning (DL) and IoT big data analytics to aid in the development of smart cities [P12]
In this survey, we present an overview of the available research on the usage of IoT and DL to construct smart cities. To begin, we define the IoT and identify the features of IoT big data. Then, we discuss the various computing platforms employed for IoT big data analytics, such as cloud, fog, and edge computing. Following that, we report widely used models for DL and current studies using IoT and DL to create smart services and applications for smart cities. Lastly, we discuss the present challenges and concerns encountered in the development of smart city services.

* Comprehensive review of microservices for data analytics in IoT applications [P3]

The published review represents a pioneering effort to discuss the current state of solutions that employ microservices-based architecture to support data analytics in IoT applications. Our comprehensive analysis sheds light on the potential benefits and obstacles of using microservices in the context of IoT,
and we highlight the most promising directions for further studies in this field. Furthermore, we emphasize the importance of integrating cutting-edge microservices technologies into IoT systems design to enhance effectiveness, scalability, and adaptability.

* **The development and implementation of DL models to enable analytical processing of many kinds of data generated through IoT applications. [P1, P2, P3, P4, P10, P11]**

In this part of the contributions, we proposed different DL models in various IoT domains, including healthcare, security, and water management. This part allowed us to prepare the test-bed and design solutions for specific problems.

* **The suggestion and the implementation of a microservices-based approach that supports data analytics for IoT applications [P5]**

We proposed an intelligent microservices-based approach to ensure the appropriate privacy and provide adequate data analytics to the end users. The suggested design is made up of several software components that have been designed to support centralized and distributed learning and satisfy the needs of IoT applications. We suggested decomposing the IoT application into a linkable set of microservices that run across different computing layers (cloud/edge). We focused on delivering an appropriate service computation distribution method that takes into account the layer's constraints and decreases data transmission cost. The performance analysis proved that our proposal satisfies the objective of the development of sustainable computing architectures as well as the high efficiency of the results.

* **The demonstration of the viability and the validity of the suggested approach through a set of experimentation [P7, P8, P9]**

In this part, a series of experiments are conducted, and the findings are analyzed and contrasted with existing cutting-edge methodologies. This allowed for a thorough evaluation of the performance and productivity of the proposed solutions, as well as the identification of areas for further improvement.
1.5 Thesis Outline

The rest of this thesis is structured in the following order:

**Chapter 2** thoroughly examines the latest advancements in IoT, data analytics, and microservices paradigm. This chapter explains the learning techniques for developing valuable IoT application services. It discusses the essential benefits of microservices for IoT data analytics, challenges, and future directions.

**Chapter 3** is dedicated to the related works. In this chapter, various works in the field are reviewed, their limitations are outlined, and the gap in the existing research is identified.

**Chapter 4** presents the suggested approach in detail. This chapter elaborates on the different phases of the approach and how each phase contributes to the overall effectiveness of the proposed approach.

In **Chapter 5**, several analytics models have been proposed and designed, ranging from simple DL models to SOA designs. The proposed models have been evaluated and compared regarding their efficiency and effectiveness. Moreover, each model's strengths and weaknesses are identified, and recommendations for future improvement are given.

Finally, **Chapter 6** summarizes the different work phases and provides some concluding remarks, along with promising work that could be carried out in the future within this field of study to enhance the efficiency and effectiveness of IoT data analytics using the proposed microservices-based approach.
Chapter 2

# Preliminary Concepts and Theoretical Background

## Contents

<table><tr><td>2.1</td><td>Introduction</td><td>8</td></tr><tr><td>2.2</td><td>Internet of Things</td><td>9</td></tr><tr><td>2.2.1</td><td>Definition</td><td>9</td></tr><tr><td>2.2.2</td><td>IoT Architecture</td><td>11</td></tr><tr><td>2.2.3</td><td>IoT Big Data Characteristics</td><td>12</td></tr><tr><td>2.2.4</td><td>Applications of IoT</td><td>13</td></tr><tr><td>2.3</td><td>Microservices Paradigm: Overview</td><td>15</td></tr><tr><td>2.3.1</td><td>Microservices' Definition</td><td>16</td></tr><tr><td>2.3.2</td><td>Microservices Advantages for IoT Data Analytics</td><td>17</td></tr><tr><td>2.4</td><td>Data Analytics</td><td>18</td></tr><tr><td>2.4.1</td><td>Paradigms</td><td>19</td></tr><tr><td>2.4.2</td><td>Types of Data Analytics</td><td>20</td></tr><tr><td>2.4.3</td><td>Analytical Tasks</td><td>21</td></tr><tr><td>2.4.4</td><td>Evaluation Metrics</td><td>22</td></tr><tr><td>2.4.5</td><td>Computational Infrastructures for IoT Big Data Analytics</td><td>24</td></tr><tr><td>2.4.5.1</td><td>Cloud Computing</td><td>24</td></tr></table>
<table><tr><td>2.4.5.2</td><td>Fog Computing</td><td>26</td></tr><tr><td>2.4.5.3</td><td>Edge Computing</td><td>26</td></tr><tr><td>2.4.6</td><td>Data Analytics Techniques</td><td>27</td></tr><tr><td>2.4.7</td><td>Deep Learning</td><td>28</td></tr><tr><td>2.4.7.1</td><td>Convolutional Neural Network</td><td>31</td></tr><tr><td><b>2.5</b></td><td><b>Transfer Learning</b></td><td><b>34</b></td></tr><tr><td>2.5.1</td><td>MobileNetV2</td><td>34</td></tr><tr><td>2.5.2</td><td>DenseNet201</td><td>35</td></tr><tr><td>2.5.3</td><td>InceptionV3</td><td>35</td></tr><tr><td>2.5.4</td><td>ResNet50</td><td>35</td></tr><tr><td>2.5.5</td><td>Xception</td><td>36</td></tr><tr><td><b>2.6</b></td><td><b>Federated Learning</b></td><td><b>36</b></td></tr><tr><td><b>2.7</b></td><td><b>Ensemble Methods</b></td><td><b>37</b></td></tr><tr><td><b>2.8</b></td><td><b>Conclusion</b></td><td><b>37</b></td></tr></table>

## 2.1 Introduction

The convergence of IoT technologies and advanced data analytics has ushered in a new era of intelligent and connected systems. In this background chapter, we explore key components that form the foundation of efficient IoT data analytics including IoT, microservices paradigm, data analytics, Transfer Learning (TL), FL, and ensemble methods. We begin by delving into the IoT and its transformative role in collecting vast amounts of data from interconnected devices. Next, we explore the microservices paradigm, a powerful framework for building scalable and flexible IoT data analytics solutions. Data analytics techniques, including ML and DL, play a crucial role in extracting insights from vast amounts of IoT data. TL, FL, and ensemble methods contribute to the advancement of IoT data analytics by enhancing performance, addressing privacy concerns, and improving predictive accuracy and robustness.

By understanding IoT Microservices architecture, data analytics, TL, FL, and ensemble methods, we gain insights into the fundamental components that drive effective IoT data analytics.
## 2.2 Internet of Things

Before detailing the different data analytics in IoT applications, an overview of IoT is necessary. Therefore, we will provide the IoT definition, the IoT architecture, the properties of IoT big data, and the different applications of IoT architecture.

### 2.2.1 Definition

IoT is a concept that involves connecting numerous objects to create a smart environment [9]. This linkage is accomplished using standardized communication protocols that allow devices to exchange and share data through different frameworks [10]. Consequently, IoT significantly enhances the interaction and effectiveness of vital infrastructure in fields such as transportation, security, education, agriculture, and healthcare. The IoT design consists of four different stages, namely hardware, connectivity and communication middleware, big data storage and analytics, and IoT applications [11]. Each of these levels is briefly described in the following paragraphs, and the structure is shown in Figure 2.1.

<figure><img src="image_3.png" /><figcaption>FIGURE 2.1: The design of IoT systems.</figcaption></figure>

#### Hardware

At the hardware level of IoT design, smart devices, which consist of sensors and actuators, serve a crucial part in capturing and processing signals. Sensors are responsible for collecting data generated by the surrounding environment, whereas
actuators convert electrical signals into tangible movements or operations. By collecting real-time data, sensors enable the interconnectivity of physical devices and digital networks. Depending on the purpose of the IoT application, various types of sensors are utilized, such as wearable sensors for monitoring human activity and others for measuring factors like temperature, humidity, air quality, pressure, speed, movement, length, and even heart rate [12].

## Connectivity and communication middleware

In the IoT ecosystem, the collected data from the hardware level is usually stored in the cloud for further investigation and analysis. However, to achieve this, it is essential to have reliable and efficient data transmission from detectors to storage and analytics tools. This is where connectivity and communication middleware come into play. These middleware act as intermediaries between the hardware level and the storage and analytics tools. Their primary function is to make sure that data is transferred smoothly and securely from sensors to the cloud. Different types of middleware technologies such as WIFI, RFID, and Ethernet are used depending on the application requirements [13].

## Big data storage and analytics

To utilize the generated data from IoT devices, it is critical to save and examine it in order to extract meaningful insights that can inform decision-making [14]. Data analytics is the procedure of translating raw data into usable insights, and it includes the following kinds of analysis: descriptive, predictive, and prescriptive [14]. Descriptive analytics summarizes and describes past events and data, providing a historical perspective. Predictive analytics use models of statistics and Machine Learning (ML) techniques to identify patterns and trends in historical data to make informed predictions about future events. Prescriptive analytics takes the analysis a step further and provides recommendations on actions to achieve a desired outcome.

## IoT applications

To leverage the potential of IoT, the top-level IoT architecture is dedicated to IoT applications. IoT applications have been developed across various sectors, such as factory automation, predictive maintenance, waste management, and public safety, to enable real-time decision-making and actions in a smart environment. Through the use of IoT applications, the environment can be transformed into a dynamic and interactive ecosystem, enabling efficient and optimized workflows.
## 2.2.2 IoT Architecture

IoT architecture is the foundation that allows interconnected devices to communicate with each other and create an IoT system network. This architecture is typically structured in layers, enabling administrators to analyze, monitor, and maintain the system's integrity [15].

The IoT architectural stack is a commonly used framework that includes five layers, as depicted in Figure 2.2: the business layer, application layer, processing layer, and perception layer. Each layer is significant in the IoT system's overall functioning, from collecting and processing data to providing valuable insights and taking appropriate actions. The following points provide a basic overview of the responsibilities of these different layers:

* **Business layer:** This business layer serves as the foundation of the IoT architecture, as it oversees and manages the entire system's functionalities, applications, and business models. This layer acts as a bridge between the technical aspects of IoT and the business objectives of the organization. It provides a holistic view of the system and helps stakeholders to make informed decisions based on data-driven insights. Moreover, this layer ensures that the IoT system aligns with the business's overall strategy, goals, and vision, providing a sustainable growth and development framework.

* **Application layer:** This application layer of IoT architecture plays an important role in managing the processed data. It serves as a mediator between the processing layer and end-users or clients, allowing them to interact with the system and make decisions based on the generated insights. This layer transforms data into useful information that is tailored to specific user requirements. Furthermore, it enables the development of customized applications and services that cater to various domains and industries, such as healthcare, smart homes, and transportation.

* **Processing layer:** This processing layer performs data processing and analysis to extract meaningful insights and knowledge from the collected data. This layer employs various data analytics techniques, such as descriptive, predictive, and prescriptive analytics, to transform raw data into actionable insights. Additionally, this layer employs advanced ML techniques to uncover hidden trends and relationships in the data. The results obtained from this layer can
be used to optimize various aspects of IoT systems, including energy efficiency, performance, and security.

*   **Transport layer:** This transport layer establishes communication links among IoT devices and transfers collected data to upper layers. This layer uses various communication protocols and technologies such as Wi-Fi, Zigbee, Bluetooth, and cellular networks to ensure seamless and secure data transmission. Additionally, the transport layer manages the quality of service to prioritize critical data and ensure timely delivery.

*   **Perception layer:** The perception layer has the task of gathering information from the surroundings with the aid of a range of sensors and actuators. The sensors acquire data and transmit it to the transport layer for additional analysis and handling, while the actuators receive instructions from the processing layer and execute physical actions in the environment. This layer is vital for the entire efficiency and efficacy of the IoT ecosystem since it is the first step where data is gathered, making it critical to guarantee the sensors’ and actuators’ precision and dependability.

<figure><img src="image_4.png" /><figcaption>FIGURE 2.2: IoT architecture</figcaption></figure>

### 2.2.3 IoT Big Data Characteristics

IoT has gained popularity in various fields, resulting in the creation of numerous applications that create huge data volumes in various forms, such as images, videos,
and audio. To address the challenges of IoT big data, Mohammadi et al. [13] have identified six key features, known as the six V's. The initial characteristic is Volume, which pertains to the enormous quantity of data produced, necessitating new methods for processing and extracting valuable insights. The second characteristic is Velocity, which denotes the rapid generation of data by numerous IoT equipment that automatically gather, record, and transmits data throughout various platforms leveraging internet standards. The third feature is Variety, which includes unstructured, quasi-structured, semi-structured, and structured data. The fourth characteristic is Veracity, which relates to the correctness and reliability of the obtained data, affecting the quality of analytics results. Variability, the fifth characteristic, depicts the varied patterns of data collection and movement based on both time and information sources. Finally, the sixth feature is Value, which evaluates the significance of the gathered IoT data upon examination, providing useful insights for improving efficiency in production and gaining a competitive edge.

According to Chebbi et al., [16], the introduction of innovative software and hardware architectures has accelerated the expansion of big data created by IoT. These systems have been created to manage the particular features of IoT big data in a timely manner. Multiple research investigations have examined the currently accessible big data mediums, classifying them as horizontal or vertical scaling platforms [17]. Horizontal expansion frameworks are designed to offer users incremental improvements in program efficiency and performance. Apache Hadoop, Mahout, Spark, and H2O represent a few of the foremost prominent horizontal scaling systems. Vertical scaling platforms, in contrast, are made up of numerous parts that may add additional strength or capacities, such as boosting processing capacity by attaching additional processors and quicker memory. Graphics processing units, high-performance computing clusters, and field programmable gate arrays are examples of prominent vertical scaling platforms [3].

### 2.2.4 Applications of IoT

The Internet's swift growth, IoT, and other technological advancements are the major factors contributing to the continuous growth of smart services across various fields [18]. The creation of smart cities is a prominent field of IoT application, in which multiple devices and applications are built to function autonomously or adapt to operational or environmental changes without human interaction. These smart devices can improve their decision-making capabilities and actions by analyzing
previously gathered data. To facilitate this process, The acquired data is kept on
cloud or fog servers, and there it is analyzed using various data analytics techniques.

Various industries have recently been able to adopt smart technologies and prac-
tices. Some of the sectors that have benefited from these technologies include:

* **Smart homes:**
  Smart homes are equipped with internet-connected devices such as refrigerators, lights, and televisions that exchange data with each other and with users [19]. The major goal of smart home deployment is to improve the control of devices and optimum energy usage.

* **Smart healthcare:**
  Smart healthcare utilizes technology to provide continuous monitoring of patient's welfare and wellness, allowing for immediate information and enhancing care [20]. Wearable and non-wearable devices are used to control a user's daily activities and record various health metrics, such as blood pressure, heart rate, body temperature, oxygen saturation, respiratory rate, and glucose level. These gadgets are linked to the internet and provide information to doctors, allowing for remote monitoring and personalized care.

* **Smart transportation:**
  Smart transportation aims to improve transportation systems by utilizing advanced equipment and optimized services [21]. This includes providing convenient smart parking solutions, reducing population density through efficient public transportation, and managing mobility inside the city. The ultimate goal is to enhance traffic flow and provide a seamless and sustainable transportation experience for all users.

* **Smart surveillance:**
  Smart surveillance is the use of observation technologies, like advanced cameras, strategically positioned around a city to improve security and avoid incidents. These smart security systems gather and analyze data in real-time to spot potential catastrophes before they happen, which can help avoid harmful situations from occurring [22].

* **Smart agriculture:**
  Smart agriculture entails using IoT devices in farming activities in order to speed up and enhance standard processes [23]. This approach helps to ensure
healthy crop production and can enhance various processes such as collect-
ing, packing, and shipment via the use of services like real-time monitoring,
predictive analytics, and automated irrigation systems.

* **Smart environment:**
The use of advanced smart sensors distributed across various regions of the city to enhance infrastructure and monitor the surroundings is referred to as the smart environment [24]. By collecting and analyzing data on factors like air quality and water management, smart environment technologies aim to create a healthier and more sustainable environment [25].

The emergence and evolution of IoT data have given rise to various data analytics techniques. Big data analytics, specifically, involves analyzing vast and diverse data sets to find unnoticed relationships, behaviors, and customer needs, among other things [26].

## 2.3 Microservices Paradigm: Overview

Nowadays, IoT and big data analytics are popular areas of research. However, there is still a significant gap between the IoT's ability to gather environmental information as well as the analytical capabilities required to analyze such data. This creates various challenges, making it challenging to exploit data analytics in IoT systems. Among the most challenging challenges involve handling heterogeneous IoT data, quickly analyzing acquired data, and creating and maintaining flexible and adaptable functions that may be utilized in different applications and settings. To handle these challenges, an adaptable and scalable architecture, rather than a monolithic one, is the best approach. A number of recent research projects have chosen to employ the microservices-based architecture for IoT application development in this context.

The microservices architecture has transformed software application develop-
ment, becoming popular as a means of creating adaptable and distributed applica-
tions [27]. Each service is constructed separately in a microservices-based architec-
ture, with cheap communication techniques and standards adapted to meet specific
company requirements [28]. This method is especially advantageous for restricted
resource setting like IoT, where a modular structure is essential for enabling multiple
analytical functions. This architecture employs a collection of distinct services that are in charge of certain analytic features, allowing them to be established, expanded, and operated separately from other services. This leads to enhanced efficiency and resistance to faults, which is especially important in an IoT setting. Unlike monolithic solutions, this architecture considers each service's analytic process, resulting in more efficient utilization of resources [29].

### 2.3.1 Microservices' Definition

Microservices architecture is a kind of web-based service that enables the development of distributed software platforms by leveraging a number of fine-grained, loosely linked, and reusable elements of software which collaborate to fulfill the system's goals [27]. Every single service in a microservices-based architecture operates in its own unique operation and shares data with others. Unlike the monolithic development approach, which uses just one piece of coding to create and construct standard applications, a microservices architecture represents a program as an ensemble of distinct but interrelated components which are capable of being created, evaluated, and deployed separately, despite being distinct elements. APIs, containers, a service mesh, service-oriented architecture concepts, and the cloud are examples of these things, in addition to individual services. By decomposing an application into these entities, microservices architecture provides greater flexibility, scalability, and resilience than traditional approaches.

Microservices' scalable and flexibly linked architecture serves as an ideal fit for IoT software, allowing it to be easily available across several settings. By using this approach, IoT applications may be composed of smaller parts, leading to less dependency and greater flexibility for change. This approach is particularly useful in IoT systems because it helps to address the diversification that occurs through the usage of numerous hardware and standards. Using approaches like web-based semantic frameworks and tools, microservices may be leveraged to assure consistent information delivery. This approach provides a way to standardize data and enable various devices as well as protocols to interact with one another, thereby reducing the complexity of IoT systems. Additionally, microservices architecture provides a scalable solution to IoT software development, allowing individual services to be scaled up or down as required to meet the demands of changing environments.
2.3.2 Microservices Advantages for IoT Data Analytics

The microservices architecture is gaining popularity in IoT systems due to several
advantages. Here are some of the main benefits of this development style [30]:

* **Availability:** Because of the independence of the services, the microservices design provides the availability of software applications with less downtime, which substantially helps to enhance and boost overall performance. Actually, whenever a failure occurs, it doesn't impact the entire system; rather, it only impacts the affected entities, which could be substituted by utilizing devoted payment processes.

* **Dynamism:** Microservices design is a DevOps-based technique that employs parts that are modular as opposed to a monolithic structure to assure an enhanced flexible and efficient approach for creating, carrying out, and sustaining software systems.

* **Modularity:** Microservices give programs with robust encapsulation, well-defined interactions, and unambiguous relationships. A microservice can be constructed in any way which offers an established connection for additional services. Its implementation details are internal to the service, and they can change without affecting or synchronizing the rest of the system. Dependencies between microservices are explicit because they are defined by coordination and choreography guidelines.

* **Scalability:** The microservices architecture allows software development teams to use different technologies and programming languages for each service, without having to worry about compatibility issues. This means that teams can choose the best technology stack for each module and upgrade individual services without disrupting the entire system. Additionally, components can be introduced without requiring downtime or re-deployment. Furthermore, services can be deployed across multiple servers, reducing the impact of resource-intensive components on the overall system performance [31].

* **Functionalities' Optimization:** The microservices architecture enables easier customization of each service to enhance and optimize business functionality since the focus is on a single service rather than the entire application. This approach allows development teams to concentrate on business logic capabilities rather than technologies, as each service can be developed and upgraded
independently. Additionally, existing services can be modified and reused for different purposes, eliminating the need to reimplement new services from scratch.

*   **Resilience:** With a microservices architecture, it becomes easier to identify and fix performance issues as the independent service deployment provides better fault isolation. In case of a failure, only the specific service is affected, which makes it easier for developers to roll back an update or make changes to the service without affecting the overall application. This reduces the possibility of downtime, and complex applications remain unaffected by a single failure, which in turn simplifies the process of identifying and fixing the main cause of performance issues [32].

The data analytics results can be improved by utilizing various techniques, such as TL, FL, and ensemble learning, among others. These techniques will be discussed in the upcoming sections.

## 2.4 Data Analytics

Recently, IoT big data analytics domain has gained significant attention. This is mainly due to the limitations demonstrated by traditional data processing methods, especially when dealing with vast amounts of diverse data.

The process of data analytics involves five key stages [33], which are illustrated in Figure 2.3. The initial stage is requirements comprehension, which entails recognizing the issue, the companies’ demands, and the reason for performing data analytics. This stage is crucial, even though it may be time-consuming and challenging, as it sets the groundwork for the subsequent stages. Once the needs are established, the next stage is data collection. This involves gathering data, which must then be cleaned, filtered, and sorted to ensure the accuracy and effectiveness of the analysis. The next stage involves using appropriate analytics techniques to manipulate and analyze data. Finally, the outcomes and outputs provided by the analytics process are interpreted in the last stage.
<figure><img src="image_5.png" /><figcaption>FIGURE 2.3: The different steps of the data analytics process.</figcaption></figure>

In the parts that follow, we will present the categories of data analytics, outline the primary analytical tasks performed by researchers, examine the primary computational infrastructures employed in IoT big data analytics, and offer an outline of the major analytics approaches.

## 2.4.1 Paradigms

As indicated by Saheb et al. [34], data analytics are typically divided in four major groups: supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning [34]. This classification is depicted in Figure 2.4. Each of these categories is briefly described below:

* **Supervised learning:** This refers to the process of acquiring knowledge by using labeled data to predict outputs for new, unlabeled data. Classification and regression are among the most common applications of supervised learning.
* **Unsupervised learning:** This involves examining the trends and correlations that emerge organically among data. Unsupervised learning is employed to discover unseen and intriguing patterns within data.

* **Semi-supervised learning:** This occurs when the experimental data has a little bit of data with labels mixed together with a big amount of data that is unlabeled.

* **Reinforcement learning:** This is a hybrid of supervised and unsupervised learning. It enables an "agent" to gain knowledge from its surroundings, allowing it to act and behave intelligently [35]. To inspire and create a flawless behavior policy, the agent communicates in real-time with its surroundings through "actions" and obtains "rewards" from the surroundings. The most extensively used reinforcement learning approach is Q-learning.

<figure><img src="image_6.png" /><figcaption>FIGURE 2.4: Different types of learning techniques.</figcaption></figure>

### 2.4.2 Types of Data Analytics

As per the findings of Marjani et al. [26], there are four main categories of data analytics: descriptive, diagnostic, predictive, and prescriptive analytics, as depicted in Figure 2.5. Below are concise explanations of each of the four types of data analytics:
* **Descriptive analytics:** This refers to a broad type of analytics that seeks to improve taking decisions through analyzing and gaining knowledge from previous accomplishments in order to improve present performance.

* **Diagnostic analytics:** This sort of analytics aids in identifying and comprehending the main reasons for occurrences, as well as determining which elements impacted the final outcome.

* **Predictive analytics:** This type of analytics aims to gain knowledge and useful information regarding the future using unprocessed data, employing the both historic and present data to forecast behaviors and occurrences.

* **Prescriptive analytics:** This type of data analysis is the more powerful approach that assesses the best response from several options and suggests decision options to maximize potential benefits or minimize risks in the future.

<figure><img src="image_7.png" /><figcaption>FIGURE 2.5: Data analytics categorization.</figcaption></figure>

### 2.4.3 Analytical Tasks

Various analytical techniques have been suggested to accomplish different analytical tasks [26], which include:
* **Classification:** This technique involves using classification algorithms to distinguish the class of a new observation among a set of categories based on the learning gained from the input data. It is widely used in ML and can be applied in various fields such as image recognition, sentiment analysis, and spam filtering.

* **Clustering:** This technique involves using clustering methods to form and recognize groupings of comparable goods based on specific characteristic qualities. It is useful for discovering patterns in data and is commonly used in market segmentation, social network analysis, and image segmentation.

* **Recognition:** This technique involves teaching machines how to identify and categorize events, objects, places, or people from images or video data, mainly. It is commonly used in computer vision and can be utilized in a variety of areas, including recognition of faces and handwriting recognition.

* **Detection:** This technique involves analyzing photos and videos to detect items or situations. It is widely employed in surveillance systems, medical imaging, and quality control in manufacturing.

* **Prediction:** This technique is a more advanced sort of analytics that is utilized to uncover trends concerning occurrences in the future and indicate the necessary actions to be performed. It is commonly used in predictive maintenance, demand forecasting, and fraud detection.

* **Analysis:** This technique involves cleaning and processing data to extract relevant information that supports decision-making. It is a crucial step in data analytics and can be used in various fields such as business intelligence, customer profiling, and social media analysis.

* **Recommendation:** This technique uses data mining techniques to offer clients products, solutions, and recommendations on the most appropriate path of action. It is commonly used in recommendation systems, personalized marketing, and e-commerce.

### 2.4.4 Evaluation Metrics

Several metrics are employed to evaluate the performance of data analytics models, including accuracy, precision, recall, and F1-score. These statistical measures are represented mathematically in Equations 4.7 – 2.5, where:
* True Positive (TP): The prediction is positive, and it is accurate.

* True Negative (TN): The prediction is negative, and it is accurate.

* False Positive (FP): The prediction is positive, but it is inaccurate.

* False Negative (FN): The prediction is negative, but it is inaccurate.

**Accuracy:** is a measure applied for assessing a strategy's overall performance across every category. It can be described by the proportion of accurate forecasts compared to the entire forecasts generated through the algorithm. In simpler terms, accuracy assesses a model's ability to properly anticipate both positive and negative outputs.

$$ Accuracy = \frac{TP + TN}{TP + TN + FP + FN} \qquad (2.1) $$

**Precision:** is a measure applied for assessing the model accuracy in classifying a sample as positive or negative. It can be described by the proportion of the number of TP forecasts to the sum of TP and FP forecasts. Therefore, precision examines how well the model identifies positive samples without falsely classifying negative samples as positive.

$$ Precision = \frac{TP}{TP + FP} \qquad (2.2) $$

**Recall:** is a measure applied for assessing the capability of the model to recognize positive instances appropriately. It can be described by the proportion of the number of TP forecasts to the sum of TP and FN forecasts. Therefore, recall assesses how well the model detects all positive instances without missing any.

$$ Recall = \frac{TP}{TP + FN} \qquad (2.3) $$

**F1-score:** is a statistic that incorporates precision and recall measurements to provide an individual value-added rating for performance verification. The F1-score gives a more realistic assessment of the model's performance through the combination of accuracy and recall.

$$ F1\text{-}score = \frac{2 * Precision * Recall}{Precision + Recall} \qquad (2.4) $$

**Specificity:** is a measure applied for evaluating the model's capability to appropriately detect negative samples. It measures how well the model identifies negative samples without falsely classifying positive samples as negative.

$$ Specificity = \frac{TN}{TP + FN} \qquad (2.5) $$
**Matthews Correlation Coefficient (MCC):** is considered to be a balanced metric, as it takes into account both the number of right forecasts versus the number of incorrect forecasts. It is particularly useful in cases where the data is imbalanced, as it provides a more accurate assessment of the model's effectiveness compared to other measures that do not take into account the imbalance in the data.

$$MCC = \frac{(TP * TN) - (FP * FN)}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}} \quad (2.6)$$

**Loss:** is a function used to estimate the variation between a model's forecast and the real result for a given input. It is also known as the cost function or goal function. This function is used to assist the model during training by modifying its parameters so that the disparity between expected and actual outputs is as little as possible.

### 2.4.5 Computational Infrastructures for IoT Big Data Analytics

Data analytics for big IoT data sets has resulted in substantial improvements in accuracy and performance over the last several years [3]. These advancements, nevertheless, come with significant computing and memory needs, which may be met by utilizing modern computing platforms. In addition to cloud computing, new ideas such as fog and edge computing have evolved to enable data analytics to be performed as near to the data source as feasible. Both fog and edge computing expands the cloud network, although they operate in different locations. While fog computing is performed on local network servers, edge computing is performed directly on intelligent IoT devices. The next sections will go through the ideas of cloud, edge, and fog computing in further detail.

#### 2.4.5.1 Cloud Computing

According to Stergiou et al., [36], cloud computing is an approach that allows users to access data from any location and at any moment. It is composed of multiple server rooms and data centers which are available via the Internet to users worldwide. Cloud computing has several distinct features, including:
* **Internet-based storage:** Cloud computing is an infrastructure for connecting servers to various dispersed devices using TCP/IP networks. This allows for storage deployment and data movement to be carried out with ease.

* **Internet-based services:** The cloud provides a broad range of services, including connectivity and computational intelligence to assist people in various locations.

* **Internet-based applications:** Cloud applications are programs that are hosted on the cloud, and they carry out their functions via Internet connections rather than functioning locally on machines after deployment.

The application of data analytics techniques to IoT data is enabled through the use of cloud computing [37]. With this method, IoT data is transferred to the cloud infrastructure, where it is stored and analyzed using various analytics methods in real time. Despite its benefits in terms of environmental sustainability compatibility, and adaptability, cloud computing for IoT data analytics comes with certain limitations [38]. Among these constraints are the following:

* **High computation cost:** The transmission of large volumes of data during the analytics phases increases network bandwidth usage, resulting in high costs for cloud resource utilization.

* **Latency:** The response time for accessing cloud services is dependent on network strength. The cloud may sometimes fail to respond quickly, particularly for time-critical applications like financial trading applications.

* **Reliability:** Cloud applications and other services are delivered via the internet and wireless connectivity. If there are any problems with the network connection, it can have a direct impact on the dependability of the services provided.

* **Privacy:** Storing data collected from IoT devices in the cloud can raise privacy concerns due to the sensitive and private nature of the information. There is a risk that the data may be lost during transmission or accessed by unauthorized individuals.
### 2.4.5.2 Fog Computing

To avoid having all data analytics processes conducted solely on the cloud, fog computing has been proposed as an alternative [39]. Fog computing is an extension of cloud computing that puts computational power closer to the network's edge, specifically designed to sustain various IoT applications [40]. It involves moving computational nodes and data analytics nearer to end-user devices, making use of a large number of diverse ubiquitous and distributed units that interact and possibly collaborate with one other and the network to conduct data storage and processing operations without the participation of other entities. As stated by Vaquero et al., fog computing is the placement of processing nodes and data analytics close to ending devices [41].

Storage, deployment, and networking are all services provided by cloud and fog computing concepts. The usage of fog computing, on the other hand, is limited to certain geographical locations, whereas the cloud is not. Furthermore, fog computing is designed primarily for active IoT applications that require immediate reactions. Fog computing, in essence, acts as a bridge between cloud services and edge devices such as IoT devices, allowing computation, communication, and storage operations to take place close to the edge. This method addresses challenges including rapid response times, restricted network capacity, and security and privacy concerns. The connection between edge devices and the cloud is depicted in Figure 2.6, showing how fog nodes play a role in linking them together.

### 2.4.5.3 Edge Computing

Edge computing has been introduced as a response to the limitations of cloud computing, as explained in the study by Yu et al. [42]. Edge computing involves processing data as close as possible to where it was generated, thereby extending the capabilities of cloud computing. This computing model offers increased computational power, enabling big volumes of data to be analyzed on the network edge before being transmitted to central cloud servers. The rapid expansion in data creation has made data transmission to the cloud difficult. By analyzing data on the edge instead of transmitting it into the cloud for investigation, bandwidth efficiency is improved, response time is reduced, network pressure is decreased, and energy consumption is minimized.
<figure><img src="image_8.png" /><figcaption>FIGURE 2.6: Fog nodes act as an intermediary between edge devices and the cloud.</figcaption></figure>

Edge computing is particularly well-suited for IoT applications, according to Shi et al., [43]. An edge gateway may be employed to link every intelligent device and locally process data using on-demand services. This approach boosts reliability, safety, and privacy.

### 2.4.6 Data Analytics Techniques

The growth of ML and DL has brought Artificial Intelligence (AI) into various fields of our lives, making it an essential aspect. The capacity of computers to execute activities that normally require human intellect, such as picture identification, speech-to-text transformation, and translating languages, is referred to as AI. AI's capability has grown tremendously throughout the years.

ML is a subclass of AI that allows computer systems to acquire knowledge and enhance their behavior via training them on a specified dataset [44], [45]. Once the model is trained, it can make decisions without explicit programming.
DL, a more sophisticated ML approach, has sparked a lot of academic interest over the past decade [46]. According to Li et al., [47], DL is a potential strategy for retrieving reliable insights from raw data supplied by IoT devices in challenging situations.

Traditional ML systems rely on hand-engineered feature extractors to handle raw data, which often requires domain expertise to develop an efficient model. Additionally, feature selection must be adjusted and restarted for each new situation. In contrast, DL techniques are based on representation learning, which means they can directly detect and acquire meaningful characteristics or features from raw data, frequently outperforming standard ML methods, particularly when handling a huge quantity of data [48]. Figure 2.7 depicts the relationship among AI, ML, and DL.

<figure><img src="image_9.png" /><figcaption>FIGURE 2.7: A diagram that uses overlapping circles to show the distinctions among AI, ML, and DL.</figcaption></figure>

### 2.4.7 Deep Learning

DL is a category within the computing paradigm inspired by the human brain, composed of several neural networks [49]. A neural network is designed similarly to a neuron, with three layers: input, hidden, and output. These neurons process incoming data and execute computations to generate a weighted sum before producing an output through an activation function. Each neuron has weights as well as a bias.
The weights that are used in the initial layer are set up according to the given input size, and the bias is adjusted throughout training.

<figure><img src="image_10.png" /><figcaption>FIGURE 2.8: The neuron structure.</figcaption></figure>

Weight initializers are utilized to set the initial weights of neural network layers and prevent the activation outputs from suffering from gradient vanishing and exploding issues during the training phase [50]. The problem of vanishing gradients is caused by the output loss of feedback signals to previous layers during back-propagation, leading to low or missing signals, rendering the network untrainable [51]. Hence, setting the weight values carefully is crucial to obtaining better outcomes and superior performance.

According to Sun et al., [52], there are three types of weight initialization methods. The first type involves constant approaches that employ identical weights for all connections in the neural network. Examples of these methods include the zero and one initializers. However, this approach can cause the learning algorithm to be unable to adjust or update the network weights, leading to a "locked" model where all layers perform the same computations [53]. The second category includes distribution methods, which randomly assign values to input matrix numbers using the Gaussian or uniform distribution. However, selecting the appropriate distribution parameters, such as the mean and standard deviation, is crucial, as incorrect values can negatively impact the model's performance and cause the issue of vanishing gradients. Random initialization that utilizes heuristics and prior information is the third category. Heuristics, in addition to nonlinear activation functions, are utilized in this technique to give weights to layers. Although these heuristics can mitigate the problem of vanishing or exploding gradients, they are not foolproof and may not completely eliminate the issue.
In the field of DL, there are several commonly used and relevant models. Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Deep Belief Network (DBN), Stacked Autoencoder (SAE), Deep Reinforcement Learning (DRL), and Generative Adversarial Network (GAN) are examples of these models. DL algorithms are part of the larger category of Artificial Neural Network (ANN), which are distinguished by the presence of several hidden layers [54]. Because of the inclusion of these layered structures in DL approaches, the name Deep Neural Networks (DNN) is widely employed. CNNs are especially successful in image or video processing because they use filter operations to extract significant characteristics [55]. RNNs, on the other hand, are ideal for time-series predictions since they rely on loop connections between layers [56]. DBN is a DL model with stacked Restricted Boltzmann Machines (RBM) in its architecture [57], whereas SAEs are made up of several autoencoders [58]. DRL combines DNNs and reinforcement learning, allowing agents to acquire knowledge from their surroundings and set long-term reward plans [59]. GANs, on the other hand, are composite DL structures that combine a generative model and a discriminative model to achieve the intended outcomes [60].

In particular, CNNs are capable of autonomously learning hierarchical representations from the input. This hierarchical feature learning capability enables CNNs to automatically recognize detailed patterns and representations from raw data, making them particularly well-suited for picture classification, object recognition, and image segmentation applications. Furthermore, CNNs excel in detecting local patterns and structures in data. Additionally, CNNs may be efficiently pre-trained on large datasets such as ImageNet and then fine-tuned for specific tasks utilizing TL. This method overcomes the limits of training deep networks from scratch, especially when dealing with little labeled data. By leveraging the knowledge and representations learned from large-scale datasets, CNNs can significantly enhance their performance on targeted IoT tasks and facilitate efficient utilization of available data resources.

In the subsequent section, we will provide a comprehensive and detailed overview of the CNN architecture, elucidating its various components and their respective functions. By delving into the intricacies of CNNs, we aim to offer a comprehensive understanding of their structural elements and the roles they play in extracting meaningful features from input data.
### 2.4.7.1 Convolutional Neural Network

CNN is a DNN with numerous layers, each of which performs different filter orations [61]. It is commonly used for processing images and videos due to its effectiveness. CNN uses supervised learning, which means that it is capable of recognizing and categorizing patterns in data based on labeled examples provided during the training phase. This feature classifies CNN as a discriminative DL architecture according to reference [61].

The structure of this network is comprised of several layers, starting with an input layer and followed by a sequence of hidden layers before concluding with an output layer. The network is constructed with a sequence of convolutional and pooling layers, that are then succeeded by a fully connected layer. Within each convolutional layer, multiple filters are incorporated, generating distinct analyses of the input data to yield feature maps.

The CNN network comprises three fundamental layers: the convolutional layer, the pooling layer, and the fully connected layer. The following is a basic description of these layers:

* The CNN model begins with a convolutional layer that performs a filter to the input picture, which has been transformed into a matrix. The resulting output is called a feature map, which contains the acquired features. The filter used in this layer is essentially a small matrix with predetermined dimensions and randomly selected values. It traverses the input matrix from left to right and then proceeds downwards, moving in steps as determined via the stride parameter as long as it completes the entire matrix. The convolutional layer process can be represented by Equation 2.7.

$$y_j = f(\sum_i K_{ij} * x_i + b_j) \qquad (2.7)$$

where x presents an input picture, $y_j$ denotes the output of the $j$th convolutional layer, $f$ serves as the activation operation, $K_{ij}$ is the convolutional kernel times the $i$th input $x(i)$, and $b_j$ is the bias. Figure 2.9 depicts convolutional procedures.

* **Pooling layer:** is utilized to simplify the CNN model by extracting the most important features from the generated feature maps through either an average
<figure><img src="image_11.png" /><figcaption>FIGURE 2.9: Explanation of the operations carried out within the convolutional layer</figcaption></figure>

or max-pooling operation. To perform this operation, a kernel or window of a predetermined size is applied to the feature map. The kernel gathers either the average or maximum values of the matrix elements based on the operation being performed and slides across the entire feature map with a set stride. Figure 2.10 illustrates the max pooling operation in the pooling layer, where four slide positions are shown with different colors. The resulting pooled values illustrate how the computations of the model become less complex.

<figure><img src="image_12.png" /><figcaption>FIGURE 2.10: Explanation of the operations carried out within the pooling layer.</figcaption></figure>

* **Fully connected layer:** This layer is in charge of the CNN network's last phase by reconnecting the processed parts to reconstruct the entire image. This layer converts the two-dimensional array to just one list and uses activation algorithms, like Sigmoid and Tanh, to generate values for probabilities that reflect the likelihood of the processed picture belonging to a particular category. The output layer sends the picture to the output class with the biggest probability ratio.

Typically, the convolutional and pooling layers are often merged at the beginning of the network design, whereas the fully connected layers are piled together in the conclusion, unlike the former two.
## How Does CNN Work?

CNNs have proven to be effective in various domains, particularly in image classification and recognition. The capacity of CNNs to discover and acquire hidden features using massive datasets is one of its most significant advantages [62]. CNNs follow a series of steps during the training phase, which are outlined below.

During the training stage, the input layer allocates weights to the data being entered, which are subsequently transmitted to the following layer. Depending on the initializer type used, the weights may be set to either constant or randomized values [63]. The subsequent layers receive weights, apply different filter functions, and generate an output that becomes the input for the subsequent layer. Finally, the ultimate output is established in the end layer.

During the stage of training, a loss function is utilized to evaluate how well a prediction performs. The function assesses the rate of error by analyzing and contrasting both the predicted and the actual outcomes. There are different types of loss functions created for various intentions. In particular, the binary cross-entropy function is implemented for binary classification issues that deal with only two categories, the categorical cross-entropy function is utilized for multi-class classification issues, and the mean-squared error function is employed for regression issues.

To verify the weights of a neuron, various optimization algorithms can be employed, including Stochastic Gradient Descent and Adam. These algorithms investigate the gradient of the loss function and endeavor to modify and adjust the network's weights or learning rates to decrease the losses. This sequence of operations is reiterated during the training period until the weights are adjusted for each neuron in every layer, and the error rate value declines. The model can be used after the training is completed.

A CNN model's performance can be improved by optimizing the right parameters. These parameters are the internal model configuration variables computed via data, such as the weights linking neural network layers.
## 2.5 Transfer Learning

TL approach has obtained considerable adoption in various sectors and applications, and it aims to improve performance on new tasks by reusing previously trained models [53], [64], [65]. This approach can lead to significant acceleration in training and better performance on related issues [66]. Image classification is among the most extensive applications of DL and TL. By training a model on a wide and diverse set of data, it can become a generic model that can be used to learn new tasks. The learned feature maps can be utilized afterward for training a new model using different data without having to start from scratch. The fine-tuning technique is a widely used approach to tailor pre-trained models to new data. It involves unfreezing a subset of the initial layers of the pre-existing model and training the newly added layers concurrently with the final layers of the base model, for the new problem. Significant performance improvements can be achieved by fine-tuning the pre-trained features to adapt them to new data [67]. Fine-tuning is a powerful tool in the ML toolbox and has been shown to yield improved accuracy and generalization when adapting pre-trained models to new tasks or datasets.

In the field of TL, several CNN architectures can be found. MobileNetV2, DenseNet201, InceptionV3, ResNet50, and Xception are some models that will be discussed in the parts that follow:

### 2.5.1 MobileNetV2

The MobileNetV2 model, an improved version of the initial MobileNet architecture, uses a reversed residual design with bottleneck layers that include convolutional units [68]. Each convolutional block is supported by skip connections between the starting and ending points. By utilizing a thin skip connection technique, MobileNetV2 can retrieve earlier activations that were not modified for each convolutional block. This design improves upon the performance of previous versions while maintaining computational efficiency.
2.5.2 DenseNet201

In the realm of DL, a highly effective CNN model with a feed-forward structure, called DenseNet201, was introduced by Huang et al. [69]. In this model, each layer takes feature maps out of the previous layer and generates its own, which are subsequently passed over to the following layer. This approach drastically decreases the total amount of parameters and resolves the issue of vanishing gradients while also improving performance by repeatedly using feature maps. Nonetheless, due to its numerous layers, this model necessitates a longer training time than other DL models.

2.5.3 InceptionV3

The InceptionV3 architecture, introduced in Szegedy et al.'s work [70], is a DNN with 48 layers that improves on the original Inception model's processing capabilities by increasing its depth. InceptionV3 uses several convolutional blocks simultaneously, each one has varying filter sizes (1x1, 3x3, and 5x5), which enables it to capture both local and generalized characteristics due to its hybrid filter architecture. Additionally, the parallel computation employed by the model results in better performance than other existing CNN architectures in several applications.

2.5.4 ResNet50

The ResNet50 model was proposed to make deep network training easier by utilizing a residual learning framework [71]. This representation encompasses an identity mapping feature that helps to address problems related to vanishing and exploding gradients. The identity mapping serves as an instant link among the convolutional layers' input and output. The ResNet50 model is a popular architecture for TL because it allows for easier training than other CNN models and helps to prevent accuracy degradation across various applications, especially those related to image classification [72].
### 2.5.5 Xception

Google developed a CNN model called Xception, which is composed of 71 layers. It is a DL model that uses depthwise separable convolutional blocks, which are designed to extract features from input images with high efficiency and accuracy [73]. Xception uses a set of convolutional layers, inception modules, depth-wise and separable convolutional blocks, and residual links. Xception has been shown to perform well in various studies related to image classification, including remote sensing and construction applications [74], [75].

## 2.6 Federated Learning

FL methods adopt a distributed learning strategy where models are trained on local devices sans exposing their private data [76]. The training process is carried out using separate devices, such as smartphones or hospital computers, where the data remains confidential. FL has become an essential factor in the development of IoT applications, such as sentiment analysis, location tracking, and cyberattack detection while maintaining data privacy. These methods enable the creation of models that can efficiently adapt to changes while ensuring user privacy by keeping the training data locally and sharing only the results of analytics [77].

In this context, FL is a promising technique that has achieved great success in bringing intelligence to the edge layers of IoT devices. Involved clients individually train a global model that is initialized by a central authority using their own data. FL offers several advantages [78], including:

* FL enables edge devices to acquire knowledge via ML/DL models and preserve their training datasets without having to host them on a centralized server.

* Data is accessible on edge devices, allowing for real-time changes to AI models. This feature saves time and makes data available without requiring a connection to the centralized server.

* FL ensures data security by storing it locally on the device.

* Because of its modest complexity and decentralized structure, it is appropriate for deployment on resource-constrained devices.
## 2.7 Ensemble Methods

Ensembling is a strategy used to improve the performance of data analytics models. It involves merging multiple learning algorithms to leverage their combined performance and enhance the accuracy of the current model [79], [80]. The concept of ensemble learning revolves around combining various models in a way where the output model surpasses the individual models. The process of integrating decisions using multiple models has proven that it is an excellent approach to improving the overall efficiency of a model.

Ensembling techniques are a popular approach to improving model performance, with several methods proposed. Hard voting and soft voting are two common techniques where the predictions of multiple models are combined. In hard voting, the model with the highest number of votes is selected, while in soft voting, the probabilities of each model's predictions are taken into account. Another technique is stacking, where a meta-model is trained on the predictions of multiple base models. Bayesian inference is used in evidence-based theory to combine the outputs of different models. These techniques have been demonstrated to be highly effective in enhancing model accuracy and are widely used in a variety of applications.

## 2.8 Conclusion

This chapter provides an overview of IoT and its impact in various domains in addition to the microservices-based architecture. The fundamental data analytics concepts are discussed, along with techniques that can enhance learning, such as TL, FL, and ensemble methods. It is important to keep up with the advancements in these areas to address the challenges that arise in its development and deployment.

In the next chapter, we will delve into the suggested methods of data analytics in the IoT environment. We will explore the different techniques and algorithms that are commonly used in this domain, including but not limited to ML, DL, statistical analysis, and data visualization.
Chapter 3

# Related Work: IoT Data Analytics Approaches Based on Microservices Architecture

## Contents

<table><tr><td>3.1</td><td>Introduction</td><td>39</td></tr><tr><td>3.2</td><td>The Role of Data Analytics in IoT</td><td>39</td></tr><tr><td>3.3</td><td>IoT Data Analytics Approaches with Microservices Architecture</td><td>40</td></tr><tr><td>3.3.1</td><td>Analytics Techniques</td><td>40</td></tr><tr><td>3.3.1.1</td><td>Approaches Based on Statistical Models</td><td>41</td></tr><tr><td>3.3.1.2</td><td>Approaches Based on Typical ML Algorithms</td><td>42</td></tr><tr><td>3.3.1.3</td><td>Approaches Based on Deep Learning Models</td><td>44</td></tr><tr><td>3.3.2</td><td>Approaches Based on Federated Learning</td><td>46</td></tr><tr><td>3.3.3</td><td>Approaches Based on Transfer Learning</td><td>47</td></tr><tr><td>3.4</td><td>Comparison of The Microservices-based IoT Data Analytics Approaches</td><td>48</td></tr><tr><td>3.5</td><td>Conclusion</td><td>54</td></tr></table>
## 3.1 Introduction

With the increasing number of IoT devices, huge quantities of data are being gen-
erated. However, this data is useless if not properly analyzed. There are many
analytics solutions available that allow users to gain valuable insights from the mas-
sive IoT data. This chapter discusses the importance of data analytics in improving
IoT applications and explores the latest research efforts focused on using microser-
vices architecture for IoT data analytics. The proposed approaches are categorized
into five groups based on their underlying techniques. Lastly, a comparison is made
between these works.

## 3.2 The Role of Data Analytics in IoT

AI, a subfield of computer science, is concerned with the development of algorithms for processing data. Its classification has evolved into two main groups based on the extent of intelligence: weak AI and strong AI [81]. Weak AI, or narrow AI, involves categorizing data using a pre-existing statistical model that has been trained to perform specific tasks. In contrast, strong AI, also called general AI or AGI, has the ability to create a system that can work intelligently and independently by utilizing ML on any available standardized data.

ML is a powerful AI approach that gives a computer system the ability to learn and evolve from a data set while also being able to tackle problems without human intervention. This cutting-edge branch of AGI is known for its ability to utilize vast quantities of initial data, referred to as a training set, to generate statistical algorithms that can effectively analyze and act on new data. In recent years, several advanced techniques have emerged from ML, such as DL, TL, and FL, each with its unique set of benefits and applications. These innovative methods are transforming the field of AI and expanding its potential for solving complex problems in diverse domains.

The adoption of the microservices architecture for creating IoT applications has proven to be a game-changer, offering numerous benefits and advantages. By breaking down the development procedure into a set of steps, this approach enables adaptable application deployment and the development of extensible, reusable, and resilient infrastructures [82], [83]. In line with the microservices architecture, various
research endeavors have focused on integrating AI techniques into the exploration of accumulated IoT data to deliver intelligent solutions and services across diverse fields. With these cutting-edge techniques, IoT systems can analyze large quantities of data in real time, extract valuable insights, and produce informed decisions that enhance various processes' efficiency, reliability, and security. Overall, the integration of microservices and AI represents a promising pathway toward unlocking the full potential of IoT and achieving more significant breakthroughs in various domains.

In the upcoming subsection, we will delve into various proposed approaches that leverage microservices architecture and utilize a range of data analytics methods to propose intelligent IoT services.

## 3.3 IoT Data Analytics Approaches with Microservices Architecture

After conducting a review of relevant literature,

FL is a distributed method approach where models are trained collaboratively across multiple decentralized devices or nodes, preserving data privacy and security. TL refers to the process of leveraging knowledge learned from one task or domain and applying it to a different but related task or domain. These categories were selected to encompass a broad range of approaches in the field of IoT data analytics, representing distinct methodologies and techniques.

### 3.3.1 Analytics Techniques

This section explores works that employed various analytics techniques in IoT data analytics with microservices, including statistical models, ML, and DL.
3.3.1.1 Approaches Based on Statistical Models

Time series data is commonly modeled using statistical models, as noted in a study by Khedkar et al. (2021) [84]. Essentially, a statistical model is a mathematical representation that elucidates the relationship between one or more random variables and other non-random variables, according to Dai et al. (2020) [85]. By using statistical modeling on raw data, IoT data scientists can more easily analyze data and uncover correlations between variables, which in turn can be used to generate predictions. These models rely on various regression techniques and are often implemented as data analytics services utilizing a microservices-based architecture.

Ortiz et al. [86] developed a microservice-based architecture that employs a sta-
tistical model for predictive data processing. This architecture enables the system
to generate accurate and compelling predictions through predictive data analytics,
which can be used to take real-time actions. Specifically, the authors utilized an
Autoregressive Integrated Moving Average (ARIMA) algorithm to investigate time
series data, leading to improved comprehension and forecasting. Furthermore, the
implementation of a microservices-based architecture has resulted in significant im-
provements in terms of maintenance and scalability. The authors conducted experi-
ments on a case study related to air quality to assess the architecture's performance,
yielding promising results.

Mateo et al. [87] introduced a cloud-based software platform that utilizes microser-
vices to simplify the analytics application in a wide of IoT scenarios. To satisfy the
platform’s essential needs of scalability, portability, interoperability, and usability,
several technologies were integrated. The resulting platform has the capability to
incorporate new sensing equipment, execute data processing tasks, merge various
kinds of data, incorporate sophisticated statistical model improvements easily, and
give an intuitive visual display. To test the suggested software architecture, the
authors conducted a smart farming scenario, demonstrating the platform’s effec-
tiveness in real-world settings.

In their work [88], Vresk et al. proposed an architecture for a microservices-based
middleware that aims to connect various IoT devices that may have different prop-
erties. The proposed approach focuses on addressing the complexity resulting from
differences in end-device properties by using a consistent method for representing
IoT devices and applications. The approach encapsulates multiple communication
methods to enhance interoperability and extensibility to suitable microservice units.
This facilitates easy change of current system components and the addition of new
ones, thereby creating an open and dynamic IoT ecosystem. The authors validated
their approach with a use case involving the estimation of future energy usage, uti-
lizing a simple linear regression model to identify correlations between outcomes
and predictors. Once the regression model is trained, it can be applied to predict
outcomes for new data points.

The works mentioned above utilized statistical modeling for IoT data analytics. These types of algorithms are a mathematical approach that focuses on identifying correlations between variables to predict an output. However, statistical models typically rely on coefficient estimation and are often applied to smaller datasets with fewer features. Additionally, they require the human designer to clearly understand the connections between variables before inputting them into the model.

3.3.1.2 Approaches Based on Typical ML Algorithms

ML is a branch of AI that includes a set of models that can be trained using historical data to classify, identify, or uncover hidden patterns [89]. The primary advantage of using ML is that once an algorithm has learned how to handle data, it can automatically perform the same tasks. With a wealth of IoT datasets available, the need for ML is increasing. Many industries rely on ML to extract meaningful insights from data. In this context, several experiments have been conducted to teach machines to get knowledge autonomously, without specific programming, using microservices-based architecture.

In their work [90], Jamil et al. presented a secure microservice system that utilizes ML and blockchain technology to enhance the predictive modeling of IoT fitness data. The proposed system offers microservice-based analytics capabilities, which offer trusted and dependable IoT services. Data privacy, security, and identity are guaranteed by a permissioned blockchain network. In addition, the system utilizes a recommendation model that generates customized workout and diet plans for fitness users based on a set of microservices, including data preprocessing, feature extraction, model selection, and visualization microservices. This proposed design ensures good performance, accurate results, scalability, and reliability of the fitness application.
The authors of [91] suggested a microservices-based approach to provide modularized data analytics services. Data analytics is decomposed into distinct stages and created in microservices for each step. The proposed approach was implemented using a prototype and evaluated with an IoT use case scenario. They employed a set of ML models, including Support Vector Machine (SVM), Linear discriminant analysis (LDA), K-Nearest Neighbours (KNN), Decision Tree (DT), and Linear Regression (LR) to analyze and learn from the data. Results showed that the proposed approach significantly improved the efficiency of IoT applications in terms of robustness, scalability, and stability.

In their work, Ribeiro et al. [92] proposed a new architecture, Machine Learning in Microservices Architecture (MLMA), for implementing intelligent applications in smart cities. The proposed architecture is based on a set of independent microservices, and it aims to develop an ML-based process that provides high-performance analysis and prediction, efficient scalability and usability, and easy maintenance of code. Two smart city use cases, a recommendation smart tourism application and a predictive policing system, were used to validate the feasibility of the MLMA. The findings revealed that the suggested architecture could provide numerous advantages in developing ML models for intelligent applications in smart city environments.

Dineva et al. [93] proposed a sophisticated and versatile architectural framework based on microservices for the delivery of ML solutions as services. This framework’s primary goal is to support analytics using a collection of modular and scalable microservices, with each one providing a specific analytics function. The framework is composed of several key microservices, including the identity service, which manages user authentication and authorization, the data acquisition service, which retrieves data from IoT devices, the transformation service, which preprocesses and transforms raw data into usable formats, the ML service, which performs the actual analytics, the device management service, which monitors and manages the connected IoT devices, and the logging service, which logs and tracks the analytics process for future analysis and debugging.

In [94], a smart dairy farming platform was introduced to analyze animal behavior and monitor their health. The platform was designed using microservices and distributed computing to solve the challenge of limited Internet connectivity in remote farm locations. By using the RF and KNN algorithms, various analytics processes were conducted closer to the data source, enabling real-time processing and monitoring capabilities. These ML models provided farmers with accurate insights and effective recommendations.
In [95], a method for automatically configuring and evaluating ML algorithms for the quick identification of Android malware was presented. This method is built on DevOps concepts in addition to microservice architecture and utilizes ML models, including RF, SVM, KNN, and MultiLayer Perceptron (MLP), in the experiments. By following this method, the authors were able to systematically identify the optimal match to enhance the performance of Android malware detection algorithms.

The utilization of microservices and ML algorithms has shown that it is highly beneficial for improving the efficiency and scalability of IoT services. However, a major limitation of these approaches is their inability to process unstructured data effectively. This requires the creation of features through manual engineering, which can be a tedious and time-consuming task. Therefore, it is imperative to explore and develop innovative techniques that can automate feature extraction and effectively handle unstructured data to further improve the accuracy and effectiveness of ML-based microservices in the context of IoT applications.

### 3.3.1.3 Approaches Based on Deep Learning Models

DL models have demonstrated their effectiveness in modeling and analyzing spatial and temporal variations of IoT data in several disciplines. Unlike traditional ML algorithms, DL algorithms do not rely on hand-crafted feature engineering, but instead, they can learn feature representations directly from raw data. This allows DL algorithms to accurately capture spatial and temporal correlations in the data, leading to better performance and higher accuracy. Moreover, DL algorithms have been shown to be effective in handling unstructured data, like images, videos, and text, which are commonly found in IoT applications. Therefore, DL algorithms have become increasingly popular in the field of IoT, especially in applications like image and speech recognition and anomaly detection, where the ability to capture complex patterns and dependencies from data is essential.

In [96], a novel approach called Lightweight IoT Smart Public Safety (LISPS) was introduced, which employs a microservices architecture to support the safety and improve the security of public systems. The LISPS framework is designed as a distributed solution in which sub-tasks are hosted by different microservices. The
system consists of two key processes: feature extraction from videos and decision-making. To extract features from videos, a Lightweight CNN and a Kerman algorithm were employed. The authors also proposed a fuzzy logic-based decision-making technique for detecting aberrant activities. The LISPS approach demonstrated high accuracy and efficiency in detecting and reporting critical events and could be a promising solution for enhancing public safety and security.

In [97], Houmani et al. presented an innovative architecture that aims to optimize DL workflows, reduce implementation time, and improve analytics results. This architecture was evaluated using an object detection use case and deployed on edge and cloud resources. One of the key features of this approach is the data management method which limits the resolution of incoming data. By doing so, the architecture can reduce the volume of data sent across the edge and the cloud, leading to lower latencies and faster processing times.

In their paper [98], Roy et al. introduced Micro-safe, an innovative architecture that combines the power of microservices and DL to deliver security as a service in a 6G environment. The key feature of this architecture is its ability to generate personalized safety decisions and send them to registered end-users through a DNN. Through different experiments, the proposed architecture achieved improved latency, reduced energy consumption, and increased throughput, making it a promising solution for providing safety-related services in future 6G networks.

Utilizing DL models can enable the creation of IoT solutions that can efficiently tackle specific issues. Nevertheless, DL methods often require extensive datasets to deliver accurate outcomes, especially with supervised learning approaches that demand numerous images. This large dataset requirement can significantly increase the complexity and computational expense of the training process, leading to a considerable amount of time consumption [99].

Security and privacy are two significant challenges that face DL IoT applications. DL models are sensitive to a variety of challenges that might degrade the way they perform. For instance, false data injection is a growing threat that exploits IoT equipment in various locations sending misleading readings and data, compromising the accuracy of the models [100]. Another critical challenge is privacy, where sensors capturing data in public areas, such as images of people or their behavior, may raise privacy concerns when data is moved to the cloud for investigation and processing. In response to these issues, several privacy-preserving algorithms have been proposed that use the FL approach. These algorithms aim to protect users' privacy by keeping
their sensitive data local and sharing only model updates with the cloud.

This section focuses on different learning approaches used in IoT data analytics with microservices, specifically TL and FL.

### 3.3.2 Approaches Based on Federated Learning

Recent studies have highlighted the use of FL as an effective approach to address privacy and security concerns in DL IoT applications. FL has emerged as a promising solution to improve analytics performance while ensuring the privacy and security of the accumulated IoT data. Several research and development studies have been conducted in this direction to develop new techniques and frameworks for efficient and secure FL-based IoT analytics.

Abdel-Basset et al. provided a Fed-TH model that leverages both DL and FL to detect cyber attacks in industrial cyber-physical systems in [101]. The model was implemented as a microservice at the edge and validated on two public datasets, TON-IOT and LITNET-2020. An exploratory microservice was also included to handle latency issues. The proposed Fed-TH model obtained an accuracy of 92.97% and 92.84% using TON-IOT and LITNET-2020, respectively, demonstrating its effectiveness in detecting cyber attacks in industrial settings.

The authors of [102] improved the capabilities of IIoT applications by utilizing decentralized training and shared learning through container microservices. They employed various strategies to decrease energy consumption and application delay, including deadline-efficient task sequencing and scheduling, latency-efficient task planning, and energy-efficient task organization. The experimental results demonstrated that this approach improved energy efficiency by 30% and reduced training time by 50%.

Zhang et al. [103] presented an FL-based platform for cyber-physical systems. This platform is able to develop smart services with ML models trained through the FL paradigm.

There is a prevalent use of standard ML/DL techniques in proposed FL methods for IoT. However, a major concern associated with FL is the need for significant
learning approaches to achieve efficient outputs. To address this challenge, one
contemporary trend is adopting the TL paradigm that utilizes previously acquired
knowledge to solve new problems.

3.3.3 Approaches Based on Transfer Learning

Recently, the application of TL in IoT has received significant interest because of its particular characteristics and strengths. TL has the ability to leverage knowledge learned from a huge amount of data to address new challenges and problems with a small amount of data [104]. This has resulted in the proposal of several works that utilize TL to improve IoT applications.

In the context of smart surveillance, [105] proposed a microservices-based appli-
cation with AI to analyze massive quantities of images from criminal evidence. The
TL paradigm was used with pre-trained CNN architecture to classify those images
and deduce crucial information (such as weapons, ammo, identity cards, etc.).

The authors of [106] have proposed a framework for supporting data analytics
and maintenance procedures in industrial settings. Microservices and techniques for
visualization are used to offer great levels of customization. The framework’s design
is organized into a collection of microservices, each performing a specific function.
The platform can handle large amounts of data and provides adaptable data storage
and management capabilities. Additionally, trained models are made available as a
service to facilitate the use of predictive analytics.

In their work, Chen et al. [107] provided a re-identification system based on microservices, which performs identification calculations on edge computing to ensure efficiency and privacy protection. The system was designed to be flexible and accommodate various situations and demand loads. The experiments showed that the proposed design provided enough computing resources for Re-ID and ensured flexibility.

The application of TL has demonstrated significant improvements in delivering
smart IoT services. However, most of the proposed methods are built on central-
ized structures in which devices transfer their locally generated data to cloud-based
storage facilities or powerful servers for computing. Such centralized approaches
may raise concerns regarding data privacy and security. Alternately, leveraging TL
and FL approaches can efficiently support IoT data analytics while preserving data privacy by performing computations locally on edge devices or using distributed computing across multiple edge devices.

## 3.4 Comparison of The Microservices-based IoT Data Analytics Approaches

For the purpose of comparing microservices-based IoT data analytics approaches, we have established the following criteria: the domain of the IoT application, the computing paradigm employed, the types of the data analytics performed, the analytical task carried out, the data analytics techniques utilized, and the key contributions of each approach.

Table 3.4 provides an overview of the reviewed studies based on microservices-based architecture/data analytics in IoT. The chosen criteria for comparing the related works were selected to provide a comprehensive analysis of the approaches. Each criterion was specifically chosen to highlight different aspects of the approaches and facilitate a thorough comparison. We start by the domain of the IoT application. This criterion focuses on the specific application domain to which each approach is applied. It helps understand the breadth and diversity of IoT applications addressed by the approaches and highlight their applicability in different domains. The computing paradigm emphasizes the paradigm adopted by each approach. It could include cloud computing, edge computing, or a combination of both. Understanding the computing paradigm used provides insights into the scalability, latency, and resource utilization aspects of the approaches. The types of data analytics performed focuses on the specific types of analytics performed by each approach. It could include descriptive analytics, diagnostic analytics, predictive analytics, or prescriptive analytics. By analyzing the types of analytics employed, we can gain insights into the depth and sophistication of the approaches' analytical capabilities. The analytical task carried out delves into the specific analytical tasks accomplished by each approach. It could include anomaly detection, classification, clustering, regression, etc. Understanding the analytical tasks undertaken by the approaches helps evaluate their suitability for different data analysis objectives. The data analytics techniques criterion highlights the specific techniques employed by each approach. It could include statistical models, ML algorithms, DL architectures, or other specialized techniques. Assessing the techniques utilized sheds light on the sophistication
and potential effectiveness of the approaches. The last criterion, key contributions,
focuses on identifying the unique contributions of each approach. It helps to under-
stand the novel aspects, innovations, or advancements introduced by the approaches
in the field of IoT data analytics following the microservices-based approaches.
<table><thead><tr><th></th><th>Work</th><th>IoT Application domain</th><th>Computing paradigm</th><th>Type of data analytics</th><th>Analytical task</th><th>DA technique</th><th>Key contributions</th></tr></thead><tbody><tr><td rowspan="2">Statistical models</td><td>[86]</td><td>Air quality</td><td>Centralized computing</td><td>Predictive analytics</td><td>Event detection, real-time prediction, context awareness</td><td>ARIMA model</td><td></td></tr><tr><td>[87]</td><td>Smart Agriculture</td><td>Centralized computing</td><td>Descriptive/ Predictive analytics</td><td>Estimate the pigs fattening growth, Send alerts to farmers</td><td>Sigmoid function from the Gompertz family, A cubic polynomial function</td><td>+Facilitate digital transformation in agricultural fields.</td></tr><tr><td rowspan="6">ML</td><td>[88]</td><td>Smart environment</td><td>centralized computing</td><td>Descriptive analytics</td><td>Estimation of future energy consumption</td><td>Regression model</td><td>+An interoperability and extensibility architecture for IoT ecosystem</td></tr><tr><td>[90]</td><td>Healthcare</td><td>Centralized computing</td><td>Prescriptive analytics</td><td>Recommendation of diet and workout plan</td><td>SVM, KNN, LR</td><td></td></tr><tr><td>[91]</td><td>Healthcare</td><td>Centralized computing</td><td>Predictive analytics</td><td>Classification of positive or negative diabetes patients</td><td>SVM, KNN, DT, LR</td><td></td></tr><tr><td>[92]</td><td>Smart City</td><td>Centralized computing</td><td>Predictive analytics</td><td>Policing system</td><td>ML</td><td>+Reusability of code. +The ease of separate services maintenance.</td></tr><tr><td>[94]</td><td>Smart agriculture</td><td>Distributed computing (fog)</td><td>Perspective analytics</td><td>The analysis of animal behaviour and their health monitoring</td><td>KNN</td><td>+Precise insights and effective recommendations</td></tr><tr><td>[95]</td><td>Cyber security</td><td>Centralized computing</td><td>Predictive analytics</td><td>Android malware detection</td><td>RF SVM, KNN MLP</td><td>+Increase the detection rate. +Optimizing resource use.</td></tr></tbody></table>
<table><tr><td rowspan="4">DL</td><td>[96]</td><td>Smart surveillance</td><td>Distributed Computing (fog+edge)</td><td>Predictive analytics</td><td>detect, track objects and identify suspicious activities</td><td>L-CNN</td><td>+Minimum delay of video processing +Easy to deploy and configure.<br/>+Efficient way of data management.<br/>+ Simple workflow to implement a DL application</td></tr><tr><td>[97]</td><td>Smart city</td><td>Distributed Computing (cloud+ fog+edge)</td><td>Predictive analytics</td><td>Object detection in images and videos</td><td>YOLO algorithm</td><td>+Improved latency, +Reduced energy consumption, +Increased throughput<br/>+A low error rate, An excellent response time.</td></tr><tr><td>[98]</td><td>Smart transportation</td><td>Distributed computing</td><td>Predictive analytics</td><td>Address the safety problems in transportation services.</td><td>DNN</td><td>+Efficient cyber threat detection, +Preserve participants privacy</td></tr><tr><td>[108]</td><td>Smart city</td><td>Centralized computing</td><td>Descriptive analytics</td><td>Feature detection and image matching</td><td>CNN</td><td>+Produce the best results in terms of latency and power consumption</td></tr><tr><td rowspan="4">FL</td><td>[101]</td><td>Industrial threat hunting</td><td>Distributed computing</td><td>Predictive analytics</td><td>Cyber-threat detection</td><td>LSTM-AE</td><td>+Ensure the development of trustworthy Edge Computing</td></tr><tr><td>[102]</td><td>Healthcare, smart home, transportation</td><td>Distributed computing</td><td>Descriptive analytics</td><td>Provide different IoT services using ML</td><td>ML</td><td>+A flexible architecture, +Speed up the image classification process.</td></tr><tr><td>[103]</td><td>Cyber security</td><td>Distributed computing</td><td>Predictive analytics</td><td>Anomaly detection</td><td>ML algorithm</td><td>+A lightweight, resilient, and scalable architecture + Enable data analytics on the edge</td></tr><tr><td>[105]</td><td>Smart Surveillance</td><td>Distributed computing</td><td>Predictive analytics</td><td>The identification of crimes</td><td>YOLO, InceptionV3, Nudenet</td><td>Low-latency performance</td></tr><tr><td rowspan="4">TL</td><td>[106]</td><td>Smart manufacturing</td><td>Distributed computing (edge)</td><td>Predictive analytics</td><td>Support smart predictive maintenance operations</td><td>Pretrained analytical models</td><td>+A lightweight, resilient, and scalable architecture + Enable data analytics on the edge</td></tr><tr><td>[107]</td><td>Smart surveillance</td><td>Distributed computing (edge)</td><td>Predictive analytics</td><td>Person identification</td><td>OSNet, ResNet Inception, MobileNet SqueezeNet</td><td>Low-latency performance</td></tr></table>
By conducting a thorough analysis and comparison of the related works based on the defined criteria, we gain a comprehensive understanding of the various dimensions of these approaches, and we deduce the following limitations:

* Many of the examined works focus on specific types of data analytics, overlooking the potential benefits of integrating multiple data analytics techniques.

* Some approaches exhibit a lack of flexibility, particularly in terms of accommodating different types of IoT devices, data formats, and application domains. This rigidity can restrict the applicability and adaptability of analytics solutions in diverse IoT environments.

* Several approaches rely on a centralized architecture for data processing, which can introduce potential bottlenecks and single points of failure. The centralized nature may also lead to increased latency and network congestion in scenarios with large-scale deployments.

* While some works prioritize data protection, there is a need for a more comprehensive focus on privacy concerns in IoT applications.

* In some of the existing works, privacy concerns are addressed through the use of FL, which ensures data privacy by keeping sensitive information locally on edge devices. However, this focus on privacy often comes at the cost of sacrificing the performance of the analytical models.

* The reviewed works largely concentrate on a single type of data analytics or learning technique, failing to explore the advantages of hybrid architectures that combine various approaches. By adopting a more comprehensive and integrated approach, we can harness the strengths of different techniques and achieve enhanced performance in IoT data analytics.

At the end of this literature review on the utilization of microservices architecture for IoT data analytics, we can draw the following conclusions:

* ML and DL algorithms have demonstrated their effectiveness in modeling and analyzing the spatial and temporal fluctuations of IoT data in various domains. By learning feature representations, these algorithms can accurately capture spatial and temporal patterns. The model can distinguish between different features and provide precise results due to its ability to learn unique features
across multiple classes rapidly. Furthermore, these data analytics methods
can account for the spatial and temporal diversity of the environment and
efficiently classify different types of learned information related to the problem,
even when the surroundings or time changes.

* TL is becoming increasingly popular in IoT data analytics research due to its ability to efficiently leverage previously gained knowledge to address new problems. TL requires fewer data and resources during the model training phase than traditional learning methods such as supervised, unsupervised, or reinforcement learning. The table shows that TL gives several improvements, including enhanced service performance, reduced effort and cost, improved accuracy, and support for real-time data analytics. Being able to handle real-time data analytics is particularly important for time-sensitive applications in the IoT environment, such as smart parking scenarios and elderly care applications.

Experimental results of the reviewed approaches suggest that the features combination of multiple TL methods can achieve superior outcomes than using just one TL method for the same data. This is because different models may have different strengths and weaknesses, and by combining them, their strengths are utilized to enhance the system's overall performance.

* FL is a distributed approach to training ML models in which the training data is kept locally on the user's device, and the model is updated with the help of a central server without transferring the raw data. This approach is gaining popularity due to its ability to ensure data privacy and improve the performance of the analytical process. In the FL approach, the model is trained on the client side, and only updates are sent to the central server, where the updates from all clients are aggregated to generate a global model. FL allows for the efficient use of distributed resources and can handle huge quantities of data without the need for a centralized storage unit. This approach is particularly useful in scenarios involving sensitive data, such as healthcare or finance, where data privacy is crucial. Moreover, FL may be used for IoT data analytics to reduce latency and ensure data privacy while providing accurate results.

* Some of the studies analyzed aimed to improve data analytics capabilities by implementing microservices architecture. In one study [91], the researchers proposed breaking down the data analytics process into a set of steps that can be implemented using composable microservices. This approach improves the efficiency of smart data analytics services by allowing them to handle diverse
IoT data from various sources and process vast amounts of data. Additionally, these services are modular and flexible, enabling IoT systems to scale with analytic features. The other reviewed works aimed to improve specific IoT data analytics case studies in various application domains, such as smart surveillance, smart cities, manufacturing, and smart farming.

* The main objective of using microservices-based architecture is to suit the needs of end-users. This approach helps to improve the usability, personalization, and contextualization of IoT applications. The studies reviewed in this article have utilized microservices-based architecture to address the needs of end-users. Some of the research works, including [109], [110], have focused on Quality of service (QoS) requirements that microservices-based architecture must meet. As this architecture is dynamic and scalable, various QoS assurance mechanisms have been developed to ensure the performance, security, and privacy of the applications during their execution. Some studies, such as [101], [106], have investigated solution-oriented criteria to enable decentralized computing and stream processing and facilitate device management.

Based on current research on IoT, microservices, and data analytics, we discovered a research need in microservices-based architecture to support data analytics with effective performance and high privacy. Combining microservices-based architecture, TL, and FL with IoT applications boosts performance, privacy, speed, scalability, integrity, and throughput.

## 3.5 Conclusion

This chapter is devoted to the state-of-the-art on proposed approaches for data analytics in IoT environments. We introduced five categories of analytics methods developed through microservices architecture, including approaches based on statistical models, ML, DL, FL, and TL. We created a comparative analysis between the reported studies and several conclusions have been drawn: 1) the use of the TL paradigm improves the learning process benefiting from the accumulated knowledge, 2) the employment of FL has boosted the efficiency of the knowledge-sharing process and increased accuracy while preserving data privacy, 3) the adoption of microservices architecture improves the functioning of IoT applications, particularly in terms of availability and speed of response.
In the next chapter, we will propose a new approach for data analytics in an IoT
environment. This approach combines both the TL and FL paradigms and follows
the microservices-based architecture.
Chapter 4

# Proposed Approach: An Effective Analytical Solution Based On Microservices For IoT Applications

## Contents

<table><tr><td>4.1</td><td>Introduction</td><td>57</td></tr><tr><td>4.2</td><td>Proposed Approach Architecture</td><td>57</td></tr><tr><td>4.2.1</td><td>Centralized Learning Method</td><td>58</td></tr><tr><td>4.2.2</td><td>Distributed Learning Method</td><td>60</td></tr><tr><td>4.2.2.1</td><td>Edge Node</td><td>61</td></tr><tr><td>4.2.2.2</td><td>Cloud Node</td><td>64</td></tr><tr><td>4.2.2.3</td><td>Federated Learning Process</td><td>65</td></tr><tr><td>4.2.2.4</td><td>Models development</td><td>66</td></tr><tr><td>4.3</td><td>Improving The Generalization of The Analytical model</td><td>69</td></tr><tr><td>4.3.1</td><td>Decision Fusion Using Ensemble Strategies</td><td>70</td></tr><tr><td>4.3.1.1</td><td>Hard Voting</td><td>70</td></tr><tr><td>4.3.1.2</td><td>Soft Voting</td><td>71</td></tr><tr><td>4.3.1.3</td><td>Random Forests-Based Voting</td><td>71</td></tr><tr><td>4.3.1.4</td><td>Evidence Theory</td><td>72</td></tr></table>
<table><tr><td>4.4</td><td>Optimizing Data Analytics through Microservices</td><td>73</td></tr><tr><td>4.4.1</td><td>Analytical Process in Microservices</td><td>73</td></tr><tr><td>4.4.1.1</td><td>Data Preprocessing</td><td>74</td></tr><tr><td>4.4.1.2</td><td>Models Building</td><td>75</td></tr><tr><td>4.4.1.3</td><td>Interpretation</td><td>76</td></tr><tr><td>4.4.2</td><td>Overview of the Proposed Approach</td><td>81</td></tr><tr><td>4.4.3</td><td>Context-Aware Microservice Selection</td><td>83</td></tr><tr><td>4.5</td><td>Conclusion</td><td>84</td></tr></table>

## 4.1 Introduction

In this chapter, we introduce a novel approach to data analytics in IoT applications. Our approach is rooted in the microservices architecture and leverages a variety of advanced AI techniques to deliver great analytical performance, unparalleled privacy protection, and exceptional scalability.

We begin this chapter by introducing the architecture of the proposed approach. Next, we present how we have used different ensemble methods to improve the generalization of the analytical model. After that, we explain how we have optimizing the scalability of the analytical process using microservices, highlighting how it is designed to effectively process and analyze IoT data.

## 4.2 Proposed Approach Architecture

This work aims to develop an intelligent approach that facilitates IoT data analytics. The approach is implemented using a set of microservices that are designed to be secure, flexible, scalable, and highly responsive to user demands. Each microservice provides a functionality independently. The microservices are merged via a pipeline of multiple microservices to work together and achieve the overall service objectives. To do this, the data analytics functionality is divided into discrete containerized microservices, and the analytics models are trained using appropriate AI methods before being deployed on edge computing nodes.
The proposed approach is a novel solution that is divided into two main nodes: the cloud node and the edge node, as depicted in Figure 4.1.

This architecture has been developed with the flexibility to support both cen-
tralized and distributed learning, depending on the specific requirements of the
application. While the architecture is primarily designed for centralized learning on
large datasets, it has also been explored for distributed learning. Centralized learn-
ing involves training the model on a large dataset that is aggregated in a central
location. This approach is well-suited for scenarios where the dataset can be easily
collected and managed in a centralized manner, and where the computing resources
required for training the model are available in a single location. Distributed learn-
ing involves training the model on multiple machines, with each machine processing
a subset of training data. This method of learning is more suitable for scenarios
where the dataset is too large to be managed in a centralized manner, or where the
computing resources required for training the model are distributed across multiple
locations. In summary, the proposed architecture is versatile enough to support both
centralized and distributed learning, and the choice between the two approaches de-
pends on a number of parameters, such as the size and complexity of the dataset,
computing resources availability, and the specific requirements of the use case.

In order to achieve highly efficient IoT data analytics with enhanced perfor-
mance and increased scalability, the approach has been developed with a range of
functionalities that are summarized below. Each component of the approach has
been assigned specific responsibilities to ensure seamless integration and optimal
performance. This intelligent approach is proposed to enhance IoT data analytics
by providing highly efficient and scalable solutions. By leveraging the cloud and
edge nodes and supporting both centralized and distributed learning, the proposed
approach is well-suited for a large set of use cases.
The following subsections will outline the analytical process for both centralized and
distributed learning methods.

4.2.1 Centralized Learning Method

Centralized learning in data analytics refers to a system where data is gathered
from various sources and stored in a central location, such as a data warehouse,
before being analyzed. It has several benefits. First, it allows for the processing of
big amounts of data that may be difficult or impossible to analyze locally. Second,
<figure><img src="image_13.png" /><figcaption>FIGURE 4.1: Abstract architecture of the proposed approach design</figcaption></figure>

it can be more efficient and cost-effective than analyzing data locally, particularly when dealing with large datasets. Finally, centralized learning can lead to more accurate results, as the central location can leverage more powerful hardware and algorithms for data processing.

In this context, the data analytics process is performed through cloud nodes and edge nodes. The cloud node refers to a powerful remote server that is typically located in a data center or a cloud computing infrastructure. It is used for storing and processing big quantities of data and is often used for training ML/DL models. It is particularly useful when dealing with large datasets that require significant processing power to analyze. This node is responsible for data storage, preprocessing, model learning, and analytics.

On the other hand, an edge node refers to a local device that is connected to a network and is used to collect data from sensors or other devices. Edge nodes are often presented as IoT applications, where data is collected from multiple sensors located in various places. Edge nodes are typically less powerful than cloud nodes, but they are closer to the data source and can process data more quickly. In a centralized learning setup, data is gathered from edge nodes and transferred to a
cloud node for processing and analysis. The analytical findings are subsequently
relayed back to the edge nodes, which can utilize the information to make decisions
or take action.

In the proposed approach, to perform data analytics with a centralized learning
method, these steps are followed:

1. Data collection: data is collected from different sources, like databases, web-sites, and social media platforms.

2. Data storage: data is stored in a central location, like a data warehouse, where it can be accessed and analyzed easily.

3. Data preprocessing: data is preprocessed to ensure that it is clean, consistent, and formatted correctly for analysis.

4. Data analytics: data is analyzed using various data analytics methods, like descriptive analytics, predictive analytics, and prescriptive analytics.

5. Data Visualization: the results of the analysis are illustrated using graphs, charts, and other visualization tools in order to understand the insights that have been generated.

This method is particularly useful when dealing with real-time data that requires
quick analysis and decision-making. However, there are also some potential draw-
backs to centralized learning. One concern is that it can raise privacy and security
concerns, particularly when sensitive or personal data is being collected and ana-
lyzed. Additionally, centralized learning can be less resilient to system failures or
attacks, because one single point of malfunction may bring the whole system down.
The distributed learning method is proposed as a solution for some of the drawbacks
of the centralized learning method.

4.2.2 Distributed Learning Method

Distributed learning is an approach to ML that involves training models on multiple
devices or computers. This can be particularly useful for data analytics when dealing
with big quantities of data which can not be processed using a single machine.
There are several benefits to distributed learning for data analytics. By leveraging multiple machines, distributed learning can decrease the required time to analyze large amounts of data and improve the accuracy of models by leveraging more data. Additionally, distributed learning can improve fault tolerance and reduce the risk of data loss by replicating data across multiple machines.

There are several approaches to distributed learning for data analytics, including parallel computing, distributed computing, and FL. We employ FL which is a newer approach that involves training models on devices like smartphones or other edge devices. The data stays at devices, and model parameters are transmitted back to the cloud server for aggregation. This approach is particularly useful for privacy-sensitive applications where the data cannot be sent to a central location.

The proposed approach for distributed computing employs intelligence coordination through the edge and cloud nodes. The functions of these nodes are described in the following.

### 4.2.2.1 Edge Node

In FL, the edge nodes are machines such as smartphones, tablets, or IoT devices that have local data and computational power. The role of edge nodes in FL is to participate in the training of ML/DL models without needing to transfer data to a centralized cloud server. Rather than transmitting data to a centralized server for training, edge nodes utilize their local data for model training and then send the model's updates to a centralized server. This process allows for the ML/DL models to train on decentralized data, without exposing sensitive data to external entities.

Edge nodes play a critical role in FL as they enable the training of models on data that is placed at the edge, near where it is generated. This approach reduces network latency, saves bandwidth, and protects data privacy, making it ideal for use in various applications such as mobile devices, smart homes, and IoT devices. In our suggested approach, the edge layer is a complex yet dynamic system that consists of several layers of service-oriented microservices that work in tandem to form an intelligent IoT application. The microservices are linkable and capable of inter-communicating to execute specific tasks. Figure 4.2 illustrates the edge layer in the distributed learning method and its underlying structure.
To better understand the functionalities of the edge layer, it can be summarized as a layered service-oriented system that performs tasks ranging from data acquisition, preprocessing, model learning, and interpretation. These tasks are carried out through a series of interconnected microservices that work together to provide intelligent decision-making capabilities. The layers of the edge layer are presented below:

* **Data Collection**

Raw data with varying formats and types are generated from a multitude of smart devices like sensors and actuators. These objects collect data and information from their environment, with sensors providing real-time data that enables communication between physical objects and digital networks. The functions of various types of sensors vary depending on the goals of IoT applications. Wearable sensors provide precise information on human actions, whereas alternative sensors gauge various variables, including temperature, humidity, pressure, air quality, duration, time, velocity, motion, and pulse rate, among other factors. Despite the vast amounts of data collected, it is unstructured and comes in different formats and types. To make it usable, the data must be preprocessed, normalized, and filtered.

* **Data Preprocessing**

Data preprocessing is an important step in the procedure of preparing data for further processing with a learning model. It involves applying various techniques to the raw data collected by multiple sensors and objects in different formats and types. The main methods used in preprocessing include data integration, normalization, transformation, and augmentation. The data integration function is used to combine heterogeneous data into a standardized format, rectify outliers and inconsistencies in the data, and smooth noisy data. Normalization is used to turn data into dimensionless and similar distributions. The dataset features are usually normalized between 0.0 and 1.0. Data transformation is employed to convert accumulated data into a format that matches the input type of the ML/DL model. Data augmentation is used to increase the size of the training dataset and improve the generalization abilities of the training model [111]. Different augmentation strategies such as random cropping and resizing, horizontal and vertical flipping, random rotation, color jittering, and adding noise are used for data transformations.

Overall, these preprocessing techniques help to prepare the collected raw data to be used with ML and DL models.
* Model learning

The model learning layer is a critical component of ML systems that is responsible for creating and refining the model. In an FL setting, this layer communicates with the cloud node to initiate the training process based on the settings defined in the cloud server. The cloud node provides the initial model to the model learning layer, which then begins training the model using the data stored on the local device.

The model learning layer is in charge of implementing the appropriate learning algorithms to optimize the model for the specific use case. This includes selecting the appropriate loss function, defining the learning rate, and selecting the optimization algorithm. Additionally, the model learning layer may incorporate techniques such as regularization, early stopping, and hyperparameter tuning to improve model performance. It is essential to monitor the learning process and track the model's progress to ensure that it is converging toward the desired outcome. The model learning layer is a crucial component of ML systems and plays a significant role in developing accurate and reliable models that can be used to make informed decisions.

* Application

This phase presents the process of using a trained ML model to produce predictions on new or unseen data. In other words, it is the process of using a model to infer the output or outcome of a given input data. During the training phase, an ML model learns patterns and relationships in the training data, which are used to make predictions on the new data during inference. The trained model can be used to make various types of predictions, depending on the problem being solved. For example, in a classification problem, the model can be used to predict the class label of a new data point. In a regression problem, the model can be used to predict a continuous value.

During inference, the new data is fed into the trained model, which then processes the data and produces a prediction. The output of the model can then be used for various purposes, such as decision-making, reporting, or recommendations. It should be noted that the accuracy and effectiveness of the predictions made by a model during interpretation depend on the quality of training data, the model complexity, and the inference data quality. Therefore, it is important to continually track and assess the model performance during inference to ensure that it is producing accurate and reliable predictions.
<figure><img src="image_14.png" /><figcaption>FIGURE 4.2: The Edge Node architecture in the decentralized learning method.</figcaption></figure>

4.2.2.2 Cloud Node

The cloud node is located in the cloud and is responsible for processing functions from edge nodes. The model-building process occurs on edge and cloud nodes. In the centralized view, the entire model construction process occurs on the cloud, necessitating the movement of data from the edge to the cloud. This approach raises privacy concerns, as sensitive data could be compromised during data transmission.

However, by employing the FL paradigm, the proposed approach addresses these concerns. The cloud node aggregates the inference models from the edge nodes and performs evaluation before delegating the final models to the edge nodes. This approach ensures that model building occurs in a decentralized and secure manner, where data remains on the edge nodes and is not transmitted to the cloud. The proposed approach improves the coordination between edge and cloud nodes, increases efficiency, and ensures data privacy.

* Model Manager
The model manager is a key component in the FL pipeline that has an im-
portant role in the development, training, and deployment of ML/DL models.
It is responsible for overseeing the entire model lifecycle, from the initial cre-
ation and selection of the appropriate model architecture to its deployment
and evaluation. The model manager, in turn, is responsible for utilizing these
resources effectively by handling model initialization, training configuration,
and evaluation. In addition to model selection, the model manager is also
essential in the development of the initial model architecture and its compila-
tion. This process requires a deep understanding of the IoT use case and the
specific data that will be utilized for training.

Once the local models trained by the clients have been aggregated, the model
manager synthesizes them into a new global model that is more accurate and
efficient than any individual local model. It then deploys these models to pro-
duction and monitors their performance to ensure that it continues to perform
optimally. The model manager is a highly skilled and specialized component
that is essential to the success of any ML project. Its ability to handle the
complex tasks of model selection, development, and deployment is critical to
ensuring that ML models are accurate, efficient, and effective in solving real-
world problems.

4.2.2.3 Federated Learning Process

FL is an effective paradigm for training ML models in a distributed environment. It involves the use of multiple virtual instances, located at the edge, to train models using their respective training data. The updated parameters are then sent to a central server in the cloud for aggregation, which allows for the creation of a final global model that incorporates the insights gained from all of the individual instances.

This final global model is then deployed on the edge for use in analytics. En-
semble approaches can be used to increase the model’s reliability and accuracy even
more. By leveraging the power of distributed learning and ensemble techniques, we
can develop more accurate and reliable ML models that can drive better insights
and decision-making.

The working process of the proposed FL is described in the following steps:
1. Data preparation: Each client collects and prepares its local data for training.

2. Model initialization and distribution: The initial ML/DL models are initialized in the cloud server and shared with the edge IoT clients.

3. Local training: Upon receiving the models, each client trains the models locally on its own data for multiple epochs.

4. FL Aggregation: The new weights of the locally trained models are uploaded to the cloud server to be aggregated.

5. Global model construction: The weights of the models are aggregated in the FL server to construct new global models. The FL training process (Steps 3 and 4) is repeated for multiple communication rounds until the models reach an effective performance.

6. Global model distribution: The new global model weights are transferred back to the IoT devices, replacing the existing local models with the final global models.

7. Interpretation: Analytics operations are performed on the client's local data using the final global models.

8. Fusion ensembling: A fusion ensembler is developed to receive the predictions from the final global models and select the final output.

4.2.2.4 Models development

The proposed approach for distributed ML involves constructing the model using both cloud servers and edge devices. Firstly, the model is created and initialized by the model manager in the cloud. Then, the learning process is carried out collaboratively through the edge clients using their own data.

The model-building process follows a three-step method. Firstly, system initial-
ization and device selection take place, where the cloud server initializes the ML/DL
model and chooses the edge devices to collaborate in the training procedure. Sec-
ondly, decentralized learning and updates occur, where selected edge devices do local
training using their respective datasets, and transfer the updated model weights to
the cloud server. This step is repeated for multiple communication rounds until
the model performance is satisfactory. Finally, model aggregation and download
take place, where the cloud server aggregates the updated model weights received from edge devices to construct a novel global model. The new global model is then downloaded and distributed back to the edge devices for further use.

1. Model Selection and System Initialization

The model manager selects an IoT job, such as intrusion detection or disease classification, and creates the appropriate ML/DL models. Multiple models can be selected to be trained using the clients' data. Additionally, TL techniques can be employed to leverage previously learned knowledge and provide models with better performance.

Depending on the use case, the type of models to be used is identified, and the learning rates and communication rounds are selected. These parameters are crucial in ensuring that the models are trained effectively and efficiently. The IoT devices are also selected to participate in the FL process as clients.

By selecting the appropriate models and configuring the learning parameters, the model manager can ensure that the FL process is optimized for the specific application case. The use of TL can also help to enhance the models' performance, while the selection of the IoT devices as clients can ensure that data privacy is maintained.

2. Decentralized Learning and Updates

Once the model manager has configured the learning parameters and selected the appropriate models, the cloud server initializes the global models and transmits them to clients to begin the FL process. Every client, denoted as $k$, employs its own local models for training using its own dataset, denoted as $D_k$, and calculates a weight, denoted as $w_k$, through reducing the loss function $F(w_k)$:

$$w_k = \operatorname{argmin} F(w_k), k \in K \qquad (4.1)$$

Subsequently, every client $k$ transfers its calculated weight $w_k$ to the server to be aggregated. Through the training process, the clients just share model weight updates with the server, ensuring that data privacy is maintained.

3. Models Aggregation and Download

The cloud server combines locally trained models' weights from participating clients in each round and produces a new version of the global model. The communication rounds number in an FL process refers to how many times this aggregation technique is executed, i.e., how many times the server collects the
locally trained models, combines them, and refreshes the global model. Each round typically involves several communication iterations, during which the IoT devices send their locally learned weights, and the server sends back the updated global model. The communication rounds number in FL depends on various factors such as the convergence rate of the models, training data size, IoT devices number, and communication bandwidth.

In each training cycle, local weight updates provided by clients in FL are merged. Federated Averaging (FedAvg) is the most fundamental aggregation function in FL [112]. FedAvg updates the global model by averaging the weights provided by clients. Suppose that $W = (w_i)$ denotes the new aggregated weight, and $W^k = (w_i^k)$ denote learned weights by each client $k$, then:

$$w_i = \sum \frac{d_i}{D} w_i^k, \qquad (4.2)$$

where $D$ presents the whole data size and $d_i$ presents the data size for each client. Technically, the aggregation process is done according to equations (4.3) and (4.4).

$$F_k(w) = \frac{1}{d_k} \sum_{i \in P_k} f_i(w), \qquad (4.3)$$

$$f(w) = \sum_{k=1}^{K} \frac{d_k}{D} F_k(w). \qquad (4.4)$$

where $K$ denotes the number of included clients, the loss of prediction of sample $(x_i; y_i)$ is denoted as $f_i(w) = l(x_i; y_i w)$, and $P_k$ is a collection of data sample indices for client $k$. Equation 4.3 determines the weight parameters of all devices based on the loss values experienced across training data points. Equation 4.4 scales the parameters and adds them all up component-by-component.

By exchanging just trained models' parameters rather than local data, this strategy decreases the device's communication overhead and ensures privacy.

Overall, the FL process allows for collaborative learning across multiple IoT devices, without the need for centralized data storage. By allowing each client to train on its own local data, the privacy of the data is maintained, while still allowing the development of accurate and effective models for a wide range of IoT applications.
## 4.3 Improving The Generalization of The Analytical model

Ensemble learning is a powerful technique that can significantly enhance the performance and robustness of ML models [113]. To further enhance the proposed approach, we choose to explore different ensemble learning techniques of stacking. Stacking can improve performance by combining the predictions of multiple models [114]. Additionally, it is essential to carefully select the DL models and their architectures to ensure that they complement each other and provide diverse predictions.

Besides, we employ different TL models for ensemble learning. TL is a technique commonly used in DL to leverage the knowledge learned by pre-trained models on a different but related task. In the field of computer vision, CNNs are commonly employed, and there are several architectures available, including MobileNetV2, DenseNet201, and InceptionV3. The mentioned DL models are excellent choices and can be integrated into different forms to generate an ensemble. Different architectures and combinations can be performed to find the suitable ensemble for a given use case.

Finally, it is crucial to assess the efficiency of the ensemble learning model using appropriate measurements, like accuracy, precision, recall, and F1-score. This will aid to identify any weaknesses in the model and guide further improvements.

The steps for developing an ensemble learning model are summarised in the following points:

1. Data preprocessing: Prepare the data set by cleaning, pre-processing, and feature selection.

2. Model Selection: Choose a set of base models that are diverse in terms of their algorithms, hyperparameters, and training data.

3. Model Training: Train each of the selected models.

4. Model Evaluation: Evaluate the trained models on a validation set to determine their individual strengths and weaknesses.

5. Ensemble Method Selection: Select an appropriate ensemble method, such as majority voting.
6. Ensemble model inference: Use the ensemble model to combine predictions.

7. Model Evaluation: Evaluate the efficiency of the ensemble model to determine its overall accuracy and other performance metrics.

8. Deployment: Following the completion of training and fine-tuning, the ensemble model is deployable in a production environment for actual utilization.

4.3.1 Decision Fusion Using Ensemble Strategies

After completing the training process, the final models are ready to be utilized for
data analytics and interpretation. To validate their performance and accuracy, new
data is used to test these models. However, combining the outputs of multiple
models can result in a more accurate prediction. This is where ensemble learning
comes in.

Ensemble learning is an excellent strategy for improving performance by com-
bining the predictions of numerous analytical models [79], [80], [115]. This strategy
relies on the concept combining the outputs of multiple models can provide better
results than using a single model alone. To implement ensemble learning, different
methods such as majority voting, and evidence theory can be used to merge final
models' predictions.

4.3.1.1 Hard Voting

Hard voting is a simple and popular ensemble learning technique used for combin-
ing outputs generated by multiple classifiers. This method works by choosing the
plurality decision generated from different classifiers [115]. In this approach, each
classifier predicts the class of the input data, and the outcomes are stored in an
array as follows: [$O_1(x)$, $O_2(x)$, ..., $O_n(x)$], where n denotes the total number of
used classifiers.

To determine the final output class, hard voting applies a voting mechanism that
selects the class with the highest frequency of prediction in the vector. Specifically,
the category with the greatest number of voters is chosen to be the final output class
for a given data instance. Mathematically, the final determined class $y$ for input is
determined as follows:

$$y = \arg \max_{i \in 1, 2, \dots, C} \sum_{j=1}^{n} I(\mathcal{O}_j(x) = i), \quad (4.5)$$

where C presents the involved classes number, $\mathcal{O}_j(x)$ is the output predicted class label of the $j_1h$ classifier for the input data $x$, and $I$ stands for the indicator function that returns 1 if the prediction is correct and 0 otherwise.

### 4.3.1.2 Soft Voting

Soft voting is a classification technique that combines the predictions of multiple classifiers by averaging their probabilities for each class [115]. Given a set of classifiers $Cl = cl_1, cl_2, \dots, cl_j$ which work for a multi-classification problem, soft voting obtains the mean probability for every class following the Equation 4.6.

$$\mathcal{P}_{mean}(i_j|x) = \frac{1}{n} \sum_{z=1}^{n} P_{Cl_z}(i_j|x) \quad (4.6)$$

In this equation, $\mathcal{P}_{mean}(i_j|x)$ represents the average probability of class $i_j$ for a given input $x$. It is calculated as the sum of the probabilities $P_{m_z}(i_j|x)$ produced by each classifier $m_z$ for class $i_j$, divided by the total number of classifiers $n$.

Finally, the output class $\mathcal{Y}$ of an input $x$ is selected by Equation 4.7. This equation selects the category that has the greatest average probability among the available classes. The argmax function returns the index of the class that obtains the highest probability value, which corresponds to the final selected class for classification.

$$\mathcal{Y} = \operatorname{argmax}[P_{mean}(i_0|x), \dots, P_{mean}(i_j|x)] \quad (4.7)$$

### 4.3.1.3 Random Forests-Based Voting

RF classifier is a powerful tool for combining multiple ML/DL models ($M_j$) to improve predictive accuracy [116]. Given an input network data $x$ with $n$ columns, each model $m_j$ predicts a set of classes probabilities $Pr = pr_1, pr_2, \dots, pr_n$. By employing the RF algorithm, the probabilities of the $M_j$ models are integrated
through an ensemble prediction function $f(x)$. Equation 4.8 illustrates this process, where each model's likelihood is viewed as a vote. The label with the highest level of certainty is produced as the output.

$$f(x) = \operatorname{argmax}_{y \in Y} \sum_{j=1}^{J} I(y = p_i(x)) \qquad (4.8)$$

where $Y$ denotes the total number of classes and $y$ a single class.

The benefit of using the RF classifier is that it can combine the strengths of multiple models, thereby improving overall predictive accuracy. Additionally, it is able to handle large datasets and avoid overfitting, making it a robust tool for data analysis. By utilizing the RF classifier to combine probability values, it is possible to create an accurate and reliable prediction function that could be employed in a wide range of data types and use cases.

### 4.3.1.4 Evidence Theory

The Dempster-Shafer fusion theory is a widely utilized technique for enhancing decisions obtained from different classifiers by generating stronger and more accurate conclusions. The evidence theory is a useful decision-making technique, which handles the uncertainty associated with data, allowing for the modeling of both aleatory and epistemic uncertainty. Mass (m), belief (Bel), and plausibility (Pl) functions are utilized in this theory.

As defined by Equation 4.9, the mass function is a crucial element of this evidence theory. For any $S \subset D$, $m(S)$ represents the mass function of the proposition that precisely expresses the fundamental level of belief and support attributed to suggestion $S$. $D = (P_1, P_2, \dots, P_n)$ represents the sets of predictions from the final global models.

$$2^D \rightarrow [0.1], m(\emptyset) = 0, \text{ and } \sum_{S \subset D} m(S) = 1, \qquad (4.9)$$

The Bel function exhibits a constant rise in value on $2^D \rightarrow [0.1]$, with $Bel(\emptyset)=0$ and $Bel(D)=1$, as defined by Equation 4.13. Plausibility functions are then defined as $Pl(S) = 1 - Bel(S)$, where $Pl(S)$ and $Bel(S)$ are the highest and lowest boundary
functions of S, respectively.

$$ \forall A \in 2^D, Bel(S) = \sum_{B \subseteq S, B \neq \emptyset} m(B) \quad (4.10) $$

To merge features and decisions from different DL models, the DST orthogonal method is employed, as seen in Equation 4.11. The maximum of belief rule is used in this study for decision-making, as presented in Equation 4.13.

$$ m(S) = (m_1 \oplus m_2 \oplus \dots \oplus m_l)(S) = \frac{\sum_{B_1 \cap \dots \cap B_l} m_1(B_1) m_2(B_2) \dots m_l(B_l)}{1 - K} \quad (4.11) $$

where

$$ K = \sum_{B_1 \cap \dots \cap B_l =} m_1(B_1) m_2(B_2) \dots m_l(B_l), \quad (4.12) $$

K reflects the degree of disagreement among the l distinct DL algorithms. In the present research, the decision-making strategy used is to select the greatest belief value as stated in Equation 4.13.

$$ x \in C_i, \text{ if } Bel(C_i)(x) = \max[Bel(C_k)(x), 1 \le k \le n]. \quad (4.13) $$

# 4.4 Optimizing Data Analytics through Microservices

## 4.4.1 Analytical Process in Microservices

This section describes the analytical process involved in microservices, which analyzes and processes data collected from IoT devices.

The data analytics process is developed using a set of microservices that work together to analyze data and generate insights. These microservices are designed to perform specific tasks within the data analytics process, such as data preprocessing, analysis, and visualization.
The first step in this process is data enhancement, which involves improving the quality of the collected data by identifying missing or incorrect values and converting it into a consistent format. To achieve this, data preprocessing microservices are employed to examine the data and identify any missing or inconsistent information. The data is then normalized, transformed, or augmented to ensure better accuracy. Once the data has been processed, it is used as input for the model learning microservices, which apply ML/DL techniques to analyze the data and develop intelligent IoT applications.

The microservices involved in this process include preprocessing, model manager, model learning, interpretation, and results fusion. Once the model configuration is defined in the cloud microservices, the preprocessed data is then utilized for the learning process. This involves the communication among edge and cloud nodes, where microservices are responsible for selecting the appropriate learning models to be trained. The functionalities of these microservices are described in the following subsections.

### 4.4.1.1 Data Preprocessing

Data preprocessing serves as the primary stage in data analysis, where raw data is transformed into a format suitable for further analysis. We design a set of services for this step using the service-oriented computing paradigm. For this stage, the following microservices are designed:

*   **Data cleaning:** This component defines the data filtering process, where the noise is eliminated, the missing values are resolved, and unnecessary data is removed. It ensures that the data is of high quality and ready for further analysis.

*   **Data integration:** This component integrates and structures data from different sources with the same format. Since the data is gathered from various sources, this step ensures that the data is consistent and can be used for further analysis.

*   **Data scaling/transformation:** This component normalizes and transforms the gathered data to a defined range to enhance the consistency of the data. This ensures that the data is consistent and can be used for further analysis.
4.4.1.2 Models Building

After the data preprocessing, the next stage is model building. This stage uses var-
ious analytical methods, such as ML, DL, and statistical analysis, to build models
that can accurately predict future outcomes based on historical data. This stage
consists of a set of microservices, each of which is responsible for a predefined func-
tion, as follows:

*   Model creator: The model creator microservice is responsible for selecting the appropriate ML models based on the specific IoT use case and creating the initial models. It also handles the compilation of the models before training. The selection of the right models is crucial to ensure that the models are well-suited to the IoT use case and can achieve high accuracy with low computation costs. The model creator microservice can take into consideration a range of factors, including the type of data, the expected output, and the available computational resources to identify the suitable model for the task.

*   Model training: Using the preprocessed data, the model begins the training process according to the configuration of the global model. The model training microservice is responsible for selecting the appropriate ML algorithm and tuning the hyperparameters to optimize the performance. In the centralized learning method, this microservice is deployed in the cloud, while in the distributed learning method, it is deployed on the edge nodes.

*   Model evaluator: The model evaluator microservice is responsible for examining the effectiveness of the global model. This microservice evaluates the model's accuracy, precision, recall, and F1-score and provides feedback on the model's performance to the model manager. According to the evaluation findings, the model manager may initiate a new round of training or adjust the model's hyperparameters to improve its performance.

*   Model uploader: In the FL setting, the model uploader microservice transfers the learned weights from edge nodes to the central server. This microservice ensures that the trained model parameters are transferred securely and efficiently to the central server.

*   Model aggregator: The model aggregator microservice in the cloud is a key component in the FL setting. It receives the updated model parameters from the edge devices and aggregates them to create a new global model.
4.4.1.3 Interpretation

In our approach, the final stage of data analytics is to use the learned models to come up with predictions using new data. The predictions or decisions are then fused using a fusion method to obtain a final output. This output can be used to inform decision-making or provide insights into the data. It should be emphasized that the quality of the data preprocessing, analysis, and model training stages have a significant impact on the accuracy and reliability of the final output. Therefore, these stages should be carefully designed and executed to ensure the best possible results. This stage consists of the following microservices:

* Interpretation microservice: This microservice is responsible for making predictions or classifications based on the trained model, given new input data. This microservice takes in the input data and passes it through the model to generate prediction/ classification, which can then be used for decision-making or other downstream processes.

* The results fusion microservice: This microservice combines the results obtained from multiple learned models. It employs a fusion technique to merge the data analysis findings from models to enhance the accuracy and comprehensiveness of the results. This microservice can use a variety of fusion methods, such as majority voting, weighted averaging, and fuzzy logic, according to the use case to aggregate the results from different models. The fusion of results can help to overcome the limitations of individual models and provide more reliable and accurate insights for decision-making.

In Figure 4.3, the procedure flow between the FL server and edge client is illustrated in detail through a set of microservices to describe the analytical process following the decentralized learning method. Figure 4.4 describes the microservices connections among cloud and edge nodes to perform the analytical process through the centralized learning method.

Algorithm 1 outlines the entire process for the fusion-based FL approach. On the other hand, Algorithm 2 explains the analytical process of the proposed approach associated with the centralized learning method.
<figure><img src="image_15.png" /><figcaption>FIGURE 4.3: The generalized workflow between the FL server and the client following the FL method. The communication between Step 3 and 4 are done for predefined rounds.</figcaption></figure>
<figure><img src="image_16.png" /><figcaption>FIGURE 4.4: The generalized workflow between the cloud and edge following the CL method.</figcaption></figure>
<caption>**Algorithm 1** : Analytical process following the distributed learning method</caption>

1: **Input:** N different datasets $DS_1, DS_2, .., DS_n$.
2: **Output:** Prediction/detection.
3: **Begin**
4: Choose the appropriate models. (three models used in this example)
5: Set the models' initial parameters for all the clients
6: $mw_1, mw_2, mw_3$ /* The local models' weights */
7: C= c1, c2, ..., cn /* Initializing the FL clients */
8: Reading input data
9: **function** F(L) training (CommunicationRrounds):
10:     **while** $c_i$ in C do:
11:         /* train the models with local data of each client */
12:         $mw_1$ = train(data in $c_i$)
13:         $mw_2$ = train(data in $c_i$)
14:         $mw_3$ = train(data in $c_i$)
15:         return $mw_1, mw_2, mw_3$
16:     **end while**
17: **end function**
18: **function** F(L)aggregation:
19:     /* Averaging parameters of each model from all devices using Eq(4.3) and(4.4) */
20:     $GMw_1$ = FLaverage($c_1(mw_1), c_2(mw_1), .., c_n(mw_1)$)
21:     $GMw_2$ = FLaverage($c_1(mw_2), c_2(mw_2), .., c_n(mw_2)$)
22:     $GMw_3$ = FLaverage($c_1(mw_3), c_2(mw_3), .., c_n(mw_3)$)
23:     Replace local models with global models
24:     return $GMw_1, GMw_2, GMw_3$
25: **end function**
26: **function** E(n)sembler ($GMw_1, GMw_2, GMw_3$):
27:     **for** each client do
28:         newdata /*to be used for data analytics*/
29:         predictions[] = $Mw_1(newdata), Mw_2(newdata), Mw_3(newdata)$
30:         final prediction = ensemble(predictions)
31:     **end for**
32: **end function**
**Algorithm 2**: Analytical process following the centralized learning method

1: /*Step 1: Data collection*/

2: Collect the dataset

3: Upload the dataset to the cloud server.

4: /*Step 2: Data preprocessing*/

5: Clean data, remove redundant data, and solve the missing data problem.

6: Data scaling and integration.

7: /*Step 3: Model creation */

8: Initialize the model parameters.

9: /*Step 4: Model learning */

10: Loop through the training data:

11: a. Forward propagate the data through the model

12: b. Calculate the loss

13: c. Backward propagate the loss through the model

14: d. Update the model parameters using an optimization algorithm

15: /*Step 5: Model evaluation */

16: Loop through the testing data:

17: a. Forward propagate the data through the trained model

18: b. Calculate the loss and accuracy of the model with testing data.

19: /*Step 6: Model上传器*/

20: Save the trained model for later use

21: /*Step 7: Interpretation*/

22: Load the saved models

23: Process new data using the models for predictions

24: Combine the predictions using a fusion technique
4.4.2 Overview of the Proposed Approach

Figure 4.5 provides an overview of the operations involved in the proposed approach. The process starts with the collection of data by different IoT devices. This data is then gathered and stored for further processing.

The next step is the data processing phase, where the collected data undergoes
various preprocessing and transformation steps to make it suitable for analysis.
This processed data is then transmitted to the TL models generated by the model
manager. The TL models utilize the FL methodology to train and update the
parameters using the distributed data available on the edge. FL ensures that the
training process preserves data privacy by keeping sensitive information on edge
devices and only exchanging model updates.

Once the TL models are trained, they are being used to assess new data and
provide valuable insights. The model's performance and accuracy are continuously
monitored and evaluated to ensure its effectiveness in analyzing IoT data.

In the final phase, the results obtained from the TL model are combined using
a suitable fusion method. This fusion method, chosen based on the specific use
case, integrates the individual model outputs to provide more effective and reliable
results.

Overall, the proposed approach encompasses data processing, TL model training using FL, result evaluation, and fusion of the obtained outputs. This comprehensive process aims to leverage the benefits of microservices, TL, FL, and fusion methods to enhance the performance of IoT data analytics and provide valuable insights for decision-making.
<figure><img src="image_17.png" /><figcaption>FIGURE 4.5: Overview of the proposed approach: processing, and analyzing IoT data using microservices, FL,TL, and Fusion Methods</figcaption></figure>
4.4.3 Context-Aware Microservice Selection

In the proposed approach, the utilization of a variation of microservices for different
steps, including data preprocessing, models building, and data fusion, enables a
modular and flexible architecture for IoT data analytics. This modular approach
allows for the customization and adaptation of the analytical pipeline based on the
specific characteristics and requirements of the IoT system at hand.

Each type of input data, such as sensor data, images, or textual information, may have distinct processing needs. Similarly, different analytical models require specific algorithms and techniques to be applied effectively. The specific IoT context and requirements heavily influence the selection of microservices. Factors such as the nature of the data, the desired level of accuracy, and the available computational resources, play a role in determining the suitable microservices for each step of the analytics pipeline.

In addition, the fusion method employed in our approach is carefully selected based on the specific use case being considered. It primarily depends on the performance achieved during the model learning phase. In cases where the individual models exhibit high performance and satisfactory results, a simple ensemble method can be employed for fusion. This involves combining the predictions of multiple models and generating an aggregated output. However, in scenarios where the individual models' performance is not as impressive and there is a need for significant improvement, we adopt a more advanced fusion method. One such method is the Dempster-Shafer theory of evidence, which allows for the combination of uncertain and conflicting information from different sources. This method incorporates a more profound level of fusion, taking into account the reliability and credibility of each model's predictions to make a more informed decision.

By aligning the chosen microservices with the requirements of the IoT use case, the proposed approach can effectively address the challenges of data preprocessing, model learning, and data fusion in a way that meets the specific needs and objectives. Therefore, the approach can leverage the power of microservices to handle diverse data types, perform complex analysis tasks, and ultimately drive valuable insights for informed decision-making in the IoT ecosystem.
## 4.5 Conclusion

In summary, the chapter presents a novel approach to data analytics that addresses the challenges of privacy and scalability in IoT applications. The suggested solution employs the microservices architecture, that enables the creation of fine-grained and reusable entities that can be easily updated and combined. The process involves several stages, including data preprocessing, data analysis, model building and training, and ultimately, using learned models and fusing decisions to generate insights. The various microservices developed for each stage are responsible for carrying out specific functions, such as data cleaning, data integration, and model evaluation. The final stage emphasizes the importance of using the learned models and fusing decisions to acquire more precise and comprehensive outputs.

In the upcoming chapter, we will delve into the practical application and evaluation of the proposed approach. By showcasing how the microservices architecture can be implemented in real-world IoT scenarios, we intend to exhibit the efficiency and scalability of the proposed solution. Furthermore, the performance of our approach will be evaluated using appropriate metrics to measure the quality of the data analytics results.
Chapter 5

# Experimentation

## Contents

<table><tr><td>5.1</td><td>Introduction</td><td>86</td></tr><tr><td>5.2</td><td>Environment Setup</td><td>87</td></tr><tr><td>5.3</td><td>Case Study 1: Cybersecurity</td><td>88</td></tr><tr><td>5.3.1</td><td>Experiment 1: Detection and Classification of IoT Malware</td><td>89</td></tr><tr><td>5.3.1.1</td><td>Dataset</td><td>89</td></tr><tr><td>5.3.1.2</td><td>Data Preprocessing</td><td>91</td></tr><tr><td>5.3.1.3</td><td>Models Building</td><td>91</td></tr><tr><td>5.3.1.4</td><td>Interpretation</td><td>92</td></tr><tr><td>5.3.1.5</td><td>Experiments Evaluation</td><td>92</td></tr><tr><td>5.3.2</td><td>Experiment 2: Detection and Classification of Denial of Service Attacks in IoT Networks</td><td>97</td></tr><tr><td>5.3.2.1</td><td>Dataset</td><td>97</td></tr><tr><td>5.3.2.2</td><td>Data Preprocessing</td><td>98</td></tr><tr><td>5.3.2.3</td><td>Models Building</td><td>99</td></tr><tr><td>5.3.2.4</td><td>Interpretation</td><td>99</td></tr><tr><td>5.3.2.5</td><td>Comparison with Similar Studies</td><td>101</td></tr><tr><td>5.4</td><td>Case Study 2: Healthcare</td><td>101</td></tr><tr><td>5.4.1</td><td>Diagnosis of Pneumonia Using Chest X-Ray Imagery</td><td>102</td></tr></table>
<table><tr><td>5.4.1.1</td><td>Dataset</td><td>102</td></tr><tr><td>5.4.1.2</td><td>Data Preprocessing</td><td>104</td></tr><tr><td>5.4.1.3</td><td>Models Building</td><td>104</td></tr><tr><td>5.4.1.4</td><td>Interpretation Stage</td><td>104</td></tr><tr><td>5.4.1.5</td><td>Experimental Evaluation</td><td>105</td></tr><tr><td><b>5.5</b></td><td><b>Case Study 3: Environment</b></td><td><b>107</b></td></tr><tr><td>5.5.1</td><td>Smart Monitoring of Water Environments</td><td>107</td></tr><tr><td>5.5.1.1</td><td>Dataset</td><td>109</td></tr><tr><td>5.5.1.2</td><td>Data Preprocessing</td><td>109</td></tr><tr><td>5.5.1.3</td><td>Models Building</td><td>110</td></tr><tr><td>5.5.1.4</td><td>Interpretation</td><td>110</td></tr><tr><td><b>5.6</b></td><td><b>Limitations of The Proposed Approach</b></td><td><b>111</b></td></tr><tr><td><b>5.7</b></td><td><b>Conclusion</b></td><td><b>112</b></td></tr></table>

## 5.1 Introduction

In the previous chapter, we introduced our proposed approach to IoT data analytics based on microservices architecture. Our approach is centered around creating a collection of fine-grained, reusable entities that are loosely linked together to form a flexible and adaptable IoT application. We have designed this architecture to support both centralized and distributed learning, in accordance with the use case’s particular demands. By leveraging FL techniques for model learning, our approach delivers intelligent microservices that can effectively analyze data while maintaining high levels of privacy.

In this chapter, we will dive deeper into the capabilities of our approach and evaluate its performance in a variety of different use cases and scenarios. Through the presentation and assessment of various IoT scenarios, we aim to show how versatile and adaptable our suggested strategy is.
## 5.2 Environment Setup

For the implementation of the experiments, we utilized a PC having these specifications: an Intel(R) Core(TM) i7-8565U CPU @ 1.80 GHz 1.99 GHz processor, 16 GB RAM, and an NVIDIA GeForce MX graphics card running on the latest version of Windows 11. We deployed the developed microservices using Docker images [117], which were orchestrated into a cluster of containers using swarm orchestration [118] for efficient management and execution.

To implement and test the various ML/DL models, we utilized the power of Python 3.8 [119] along with the versatile Jupyter notebook [120] provided by the Anaconda package [121]. This combination allowed us to leverage the benefits of Python’s vast ecosystem of libraries and tools, while also providing an interactive and intuitive environment for model development and testing. Through the use of these technologies, we were able to achieve efficient and reliable experimentation, leading to robust and accurate results.

In Figure 5.1, we illustrate our simulation experiments intended for achieving the primary objective of our proposed approach - ensuring predictive analytics in IoT environments through the adoption of microservices and AI techniques. To design and validate the different scenarios using our approach, we utilized open-source components that were readily accessible to the public. Through the adoption of a microservices-based architectural style and FL/TL approaches, we were able to ensure the scalability and reliability of our suggested solution.

To facilitate the implementation of our proposed approach, we constructed three virtual machines using distributed Docker images [117]. Each virtual machine hosted a microservice, with the Docker images installed on each machine forming a cluster of containers. Swarm orchestration [122] was employed to manage and orchestrate the different components of this cluster, providing a high degree of availability for our applications. The first virtual machine was dedicated to supporting the model manager processes, which made use of microservices such as the model creator, aggregator, and deployer. The second virtual machine was set up for model-learning processes, which comprised microservices for training, evaluating, and uploading models. Finally, the third virtual machine was responsible for executing the data analytics algorithms, which were implemented using microservices for data preprocessing and results fusion.
<figure><img src="image_18.png" /><figcaption>FIGURE 5.1: Experimental setup of the proposed approach used for different IoT use case scenarios.</figcaption></figure>

This setup allowed us to achieve efficient and reliable execution of our proposed approach, with each virtual machine handling specific tasks in a distributed and scalable manner. By leveraging the power of Docker swarm and microservices, we were able to create a highly available and adaptable system capable of handling complex and diverse data sets.

## 5.3 Case Study 1: Cybersecurity

The proliferation of IoT devices has led to increased vulnerability to cyber-attacks. Weak passwords, unsecured network connections, and a shortfall in secure update methods are just some of the well-known vulnerabilities that make IoT devices attractive targets for malware. To combat these threats, data analytics models have emerged as powerful tools for cybersecurity. Data analytics models are able to
mine, investigate, and get knowledge from big volumes of data, making them well-
suited for analyzing traffic flows and detecting attacks. By leveraging these models,
cybersecurity professionals can ease the burden of manual analysis and more quickly
identify and respond to threats. In this case study, we employ our proposed approach
to enhance cybersecurity. Specifically, we will use data analytics models to analyze
traffic flows, detect attacks, and classify them using different scenarios.

5.3.1 Experiment 1: Detection and Classification of IoT Mal-
ware

We implement an IoT-malware detection and classification use case to validate and assess our proposed approach through an FL setting. The different stages of the use case scenario are depicted in Figure 5.2. It comprises IoT sensors and devices, malware detection entities (clients), and FL servers as its high-level components. The data traffic is collected, preprocessed, and analyzed for each client using our proposed malware detection solution. The solution is composed of microservices that support data preprocessing, model building, and interpretation. These microservices encapsulate distinct capabilities, such as data transformation, model training, and model uploader. With the help of an FL server, the flow of data analytics is carried out to detect and classify various malware effectively.

5.3.1.1 Dataset

To evaluate and validate our proposed approach, we utilized the MaleVis dataset [123], which is a comprehensive dataset for evaluating IoT malware using image-based analysis. The dataset consists of 14,226 RGB-converted images from 26 distinct IoT malware families, covering 25 types of malware divided into six categories: adware, trojans, viruses, worms, backdoors, and one benign class. To ensure the robustness of our approach, we partitioned the dataset into three subgroups for training, validation, and testing. This allowed us to train and fine-tune our model on a large portion of the dataset, while still having enough data for validation and testing to accurately assess the effectiveness of our approach.

Below, we demonstrate the outcomes of implementing our suggested method to
identify and categorize IoT malware, utilizing the MaleVis dataset.
<figure><img src="image_19.png" /><figcaption>FIGURE 5.2: IoT malware detection and classification Scenario.</figcaption></figure>
### 5.3.1.2 Data Preprocessing

During this stage, we utilized the bin2png algorithm to transform the PE files acquired from malfunctioning software into RGB images. This step was essential to preprocess the data and make it compatible with the pre-trained CNN models, which require images as inputs. In order to convert binary files into images, we used a three-byte encoding scheme, where each pixel in the output image represents three bytes. Specifically, the initial byte is encoded and displayed in the red channel of the resulting image, while the second byte is shown in the green channel and the third byte in the blue channel. Using this encoding scheme, we generated RGB images for all the collected PE files. These images were then utilized as input for the models to perform further analysis and detection of malware in the collected data. This approach allowed us to leverage the power of pre-trained CNN models, which have been successfully demonstrated to be efficient in detecting and classifying malware.

### 5.3.1.3 Models Building

To evaluate our proposed approach, we conducted experiments in a federated setting with 10 virtual IoT devices. We selected three pre-trained CNN architectures, including MobileNetV2, DensNet201, and InceptionV3, and used TL to detect and efficiently classify IoT malware. To train the CNN models, we selected hyperparameters based on prior research and experimentation. Table 5.1 gives details about the hyperparameters chosen for the training of the considered CNNs, including learning rate, batch size, and epochs number. We conducted several experiments with different combinations of hyperparameters to ensure the best performance of the models.

<table><caption>TABLE 5.1: Hyperparameters' values of the considered CNNs.</caption><thead><tr><td>Hyperparameter</td><td>Value</td></tr></thead><tbody><tr><td>Batch size</td><td>32</td></tr><tr><td>Number of clients</td><td>10</td></tr><tr><td>Number of epochs</td><td>10</td></tr><tr><td>Number of communication rounds</td><td>35</td></tr><tr><td>Optimizer</td><td>Adam</td></tr><tr><td>Learning rate</td><td>1e-3</td></tr></tbody></table>
### 5.3.1.4 Interpretation

Table 5.2 summarizes the findings of fine-tuned CNNs trained using FL. The results show that the FL paradigm yields excellent model performance. Figure 5.3 illustrates that the MobileNetV2 model achieved the best performance by applying the FL, with accuracy, precision, recall, and F1-score metrics surpassing those obtained with the CL approach.

After completing this experiment, the evidence theory was used for decision fusion, and the results of the generated classifiers are presented in Table 5.2. In terms of efficiency, the combined classifier beat individual models, earning 99.24% accuracy, 99.12% precision, 99.36% recall, and 99.45% F1-score.

### 5.3.1.5 Experiments Evaluation

#### 1. Performance Against Centralized Learning:

To conduct a comparative analysis, we also implemented our proposed approach's Centralized Learning model using the TensorFlow DL framework [124]. The MobileNetV2, DenseNet201, and InceptionV3 were fine-tuned and trained using MaleVis data. In contrast to the FL implementation, the knowledge produced by the centralized approach is shared in the cloud, and the amount of data utilized for training is enormous compared to the amount of data used in the FL approach. The CNNs were trained over 35 epochs. Comparing the CNN models' performance results emphasizes that the FL implementation outperforms the CL implementation.

Table 5.2 includes the experimental results of the fine-tuned CNNs trained using the CL and FL in addition to the fusion-based classifiers of these models. The results presented by Table 5.2 indicate that the performance of models trained in FL is better than those trained in CL approach. It is clear from Figure 5.3 that the accuracy, precision, recall, and F1-score metrics of the MobileNetV2 model, which provided the highest performance results by applying the FL, surpass the performance results of the same model by using the CL approach.
<figure><img src="image_20.png" /><figcaption>FIGURE 5.3: Performance results of the MobileNet CNN resulting from the adoption of FL and CL approaches.</figcaption></figure>

<table><caption>TABLE 5.2: Experimental results of the single CNN models, the centralized Dempster-based fusion classifier, and the proposed approach.</caption><thead><tr><th rowspan="2">Model \ Evaluation metric</th><th colspan="2">MobileNetV2</th><th colspan="2">DenseNet201</th><th colspan="2">InceptionV3</th><th rowspan="2">CL Dempster-based fusion classifier</th><th rowspan="2">Proposed approach</th></tr><tr><th>FL</th><th>CL</th><th>FL</th><th>CL</th><th>FL</th><th>CL</th></tr></thead><tbody><tr><td>Accuracy(%)</td><td>98.56</td><td>93.95</td><td>98.27</td><td>95.12</td><td>97.98</td><td>93.49</td><td>97.15</td><td>99.24</td></tr><tr><td>Precision(%)</td><td>98.57</td><td>93.98</td><td>98.35</td><td>95.14</td><td>97.87</td><td>93.65</td><td>96.87</td><td>99.12</td></tr><tr><td>Recall(%)</td><td>98.68</td><td>93.84</td><td>98.12</td><td>95.18</td><td>97.78</td><td>93.23</td><td>97.02</td><td>99.36</td></tr><tr><td>F1-score(%)</td><td>98.52</td><td>93.81</td><td>98.24</td><td>95.21</td><td>98.94</td><td>93.21</td><td>97.48</td><td>99.45</td></tr><tr><td>Loss</td><td>0.1072</td><td>0.2939</td><td>0.1251</td><td>0.2483</td><td>0.1538</td><td>0.2462</td><td>0.1541</td><td>0.08471</td></tr><tr><td>Specificity(%)</td><td>98.78</td><td>93.25</td><td>98.15</td><td>95.09</td><td>97.74</td><td>93.15</td><td>97.44</td><td>99.31</td></tr><tr><td>MCC(%)</td><td>98.48</td><td>93.89</td><td>98.18</td><td>95.23</td><td>97.57</td><td>93.41</td><td>97.27</td><td>99.18</td></tr></tbody></table>

## 2. Performance Analysis of Microservices:

The suggested framework has been evaluated in terms of microservices performance. For this purpose, we looked at the execution time of microservices based on their functionality. The end-to-end response time of the data analytics function (f) refers to the entire procedure from start to finish. It is defined by Equation 5.6:

$$T_f = T_{Pre} + T_{Pro} + T_{Fus} \qquad (5.1)$$

The time spent in data pre-processing, data analytics, and results fusion stages are denoted as $T_{Pre}$, $T_{Pro}$, $T_{Fus}$, respectively.

The total time cost ($T_{total}$) of the data analytics function corresponds to the response time of its composed microservices and is calculated using Equations 5.2 and 5.3 [97]:

$$T_{total} = \sum_{i=1}^{n} RT_{\mu}, \qquad (5.2)$$
Where

$$RT_{\mu} = Trans(M_j, M_i) + T_{Exec} \qquad (5.3)$$

$T_{Exec}$ presents the time needed for a microservice execution. $Trans(m_j, m_i)$ represents the duration of data transfer from microservice $M_j$ to microservice $M_i$ and it is computed using Equation 5.4:

$$Trans(M_j, M_i) = \begin{cases} \frac{size}{bw(m_j, m_n)} + \frac{size}{bw(m_n, m_i)} & \text{if } r_j \in \text{edge}, r_i \in \text{cloud} \\ 0 & r_i = r_j \\ \frac{size}{bw(m_j, m_i)} & \text{otherwise} \end{cases} \qquad (5.4)$$

Where size is the data size transferred in Mbits, and bw(r_j, r_n) is the bandwidth of the link between the data creator and data consumer in Mbits.

Figure 5.4 depicts the execution time for each microservice in the malware detection/classification process, starting with data preparation and finishing with the fusion of the results. The data preprocessing microservice converts the recorded traffic from Portable Execution (PE) file to an RGB image. This process takes only approximately three seconds to complete. Because the data is processed through three different CNN models, the data analytics function is the most time-consuming. These models analyze the transformed image, and the classification results, expressed as probabilities for each class, are the provided outputs.

Because it includes the selection of the best output based on the collected results, the fusion microservice operates significantly faster than the preceding microservices. In a monolithic architecture, all the components of an application are tightly coupled and must be deployed together, which can make it difficult to scale individual components to handle the increased load. This can result in longer response times and decreased performance. In contrast, a microservices-based architecture allows for the development of smaller, more specialized services that can be scaled independently. This allows for faster response times and improved performance, as the application can better adapt to changing loads and resource demands.

### 3. Impact of Number of FL Clients:

Let us now replicate the previous learning process with a larger number of data owners. Instead of decomposing the train set into 10 clients, we will breakdown it down into 15, 20, 25, and 30 clients. Figure 5.5 depicts the average
<figure><img src="image_21.png" /><figcaption>FIGURE 5.4: The execution time of different microservices in the process of malware detection.</figcaption></figure>

calculation time of the learning process for the number of clients considered. As a result, varying the number of clients in the FL process doesn't significantly worsen the overall execution time. Here are the main reasons for this conclusion:

* **Asynchronous Communication:** clients communicated with the server independently, without waiting for other clients. This can help reduce the overall execution time, as clients do not have to wait for slow or unresponsive peers. As a result, adding more clients may not necessarily increase the overall execution time since each client can operate independently;

* **Parallel Processing:** the local computations for multiple clients were performed in parallel, which helped reduce the overall time required to complete a training round;

* **Sample Size:** each client trained the models on a subset of its local data. Here the sample size is relatively small; adding more clients may not significantly impact the overall execution time since each client's contribution to the final model update was relatively small;

* **Efficient Aggregation:** The server aggregated the model weights obtained by clients to update the global model. Here an efficient aggregation strategy, which is federated averaging, was used, so the overhead of aggregating updates from a large number of clients was minimized.

## 4. Comparison with Existing Research:

We compare the acquired findings with those of previous studies published that worked on the same use case scenario and used the same dataset in
<figure><img src="image_22.png" /><figcaption>FIGURE 5.5: Execution time in ms for data analytics process with a different number of FL clients.</figcaption></figure>

order to verify the validation of the proposed approach. As seen from the accomplishments and outcomes in Table 5.7, our approach achieves the highest performance results compared to the related studies that rely on centralized ML/DL models and use monolithic architectures.

<table><caption>TABLE 5.3: Evaluation of our approach against previous methods utilizing MaleVis dataset</caption><thead><tr><td>Ref</td><td>Method</td><td>Precision(%)</td><td>Recall(%)</td><td>F1-score(%)</td><td>Accuracy (%)</td></tr></thead><tbody><tr><td>[125]</td><td>TL (DenseNet201)</td><td>not stated</td><td>not stated</td><td>not stated</td><td>97.48</td></tr><tr><td>[126]</td><td>Deep RF approach</td><td>97.43</td><td>97.32</td><td>97.42</td><td>97.43</td></tr><tr><td>[127]</td><td>Hybrid CNN approach using AlexNet and ResNet152</td><td>97.1</td><td>94.9</td><td>94.5</td><td>96.6</td></tr><tr><td>[128]</td><td>TL with ShuffleNet and DenseNet201)</td><td>99.80</td><td>95.61</td><td>95.37</td><td>95.01</td></tr><tr><td>[129]</td><td>Pretrained DenseNet model</td><td>98.56</td><td>97.74</td><td>98.15</td><td>98.21</td></tr><tr><td>[130]</td><td>RF-based voting classifier</td><td>98.74</td><td>98.67</td><td>98.70</td><td>98.98</td></tr><tr><td>Proposed approach</td><td>FL and microservices based framework for IoT data analytics</td><td>99.12</td><td>99.36</td><td>99.45</td><td>99.24</td></tr></tbody></table>