## 6 Design of an Intelligent Middleware Platform
### Machine Learning-Based Resource Management Layer  
Existing resource management platforms are divided into two categories. (1) Resource management platforms for IoT systems provide access to and control of physical devices. They also support data collection, data analysis, and application composition services. However, they do not provide computing services and are not designed for complex multilayer network infrastructure. (2) Resource management platforms for distributed computing systems provide computing and storage services. However, they do not focus on data collection and actuation services. Moreover, they are not designed for heterogeneous multilayer network infrastructure, and therefore, are not aware of static and dynamic routes available in each network layer. Consequently, they are unable to fulfill the low latency and high data rate requirements of several smart city and 5G network applications. Existing resource management schemes also use conventional approximate or heuristic algorithms, which are computationally expensive, incur significant overhead, and do not perform well with an increasing number of parameters. Recently, several machine learning techniques, such as those presented in [59; 60], were used to address resource management problems. However, these techniques use only a few system-level parameters and they also do not utilize a vast amount of data generated by multi-network infrastructure. A detailed analysis of resource management schemes was provided in [2; 31]. A new resource management layer aims to leverage regression analysis and reinforcement learning methods to allocate and manage heterogeneous computing, sensing, actuating, and network resources efficiently. This layer comprises six services and storage units. _Efficient Discovery and Monitoring Service:_ This service is responsible for discovering and monitoring a wide range of devices and services across multiple heterogeneous networks and for performing the registration and de-registration of devices. The information collected by this service should be used to train machine learning algorithms and make resource management decisions. A similar discovery and monitoring service was developed in [2] for a homogenous network environment. _System Data Store:_ This service is responsible for storing a large amount of historical application and system-level data, such as application submission time, application completion time, task completion success rate, device failure rate, task completion rate of devices, and energy consumption profile of devices. The data should be used by machine learning algorithms to analyze system performance and predict device and task failures [15]. _Task Queue:_ This service is responsible for maintaining a list of tasks submitted by a user and storing task-related information, such as task size, input data size, output data size, task deadline, and data latency requirements. Once a task is completely executed, it is removed from the queue and its information is stored in the system data store. _Virtual Device Registry:_ Virtual device registry is responsible for storing a list of available virtual devices. For each device, device-related information, such as speed, energy consumption rate, list and description of services, and communication technologies, is listed. This service is also responsible for storing information required to access the other services. _Task Dispatch Service:_ This service is responsible for sending tasks and required data to a node selected by a resource allocation service. _Resource Allocation Service:_ The allocation of resources is a complex and difficult task [2]. A multi-network environment and varied QoS requirements make the process even more difficult [1]. Existing approximate or heuristic schemes are designed to utilize single wireless communication technology or rely on an eclectic system that exclusively selects one distinct communication technology. They also do not efficiently utilize network-level parameters, such as link lifetime and energy consumption.