## 2 Related Work
modeled daily patient arrivals at ED via ANN. [PERSON] et al. developed an artificial intelligence model based on back-propagation Neural Network for predicting patient inflow [47]. Clearly, more research is needed to demonstrate and harness the power of machine learning to improve the provision of healthy services, which can lead to improved quality of care and also efficient utilization of resources. The patient flow data is basically time series data. In [48], a time series is defined as a vector \(X=\{\mathbf{x}^{(1)},\mathbf{x}^{(2)},\cdots,\mathbf{x}^{(n)}\}\), where each element \(\mathbf{x}(t)\in\mathbb{R}^{m}\) pertaining to \(X\) is an array of \(m\) values \(\{\mathbf{x}_{1}^{(t)},\mathbf{x}_{2}^{(t)},\cdots,\mathbf{x}_{m}^{(t)}\}\), with each of the \(m\) values corresponding to the input variables measured in the time series. Several traditional techniques of manipulating time series data including traditional ANNs have been exploited in the literature for applications in economics, engineering and medicine [49; 50]. Deep learning, however, has shown an exceptional performance in both classification and forecasting applications [51; 52; 53]. For classification, the existing methods generally relied on the usage of domain specific features normally crafted manually by human experts. However, getting best features was a daunting task and the performance of the classifier was heavily dependent on their quality. The advantage of deep learning is in its ability to learn such features by itself, reducing the need for human experts [54; 55]. Convolutional neural network (CNN) models have continued to gain popularity among the deep learning community in various domains [56]. Due to their efficiency with data that has a topological structure in the features space, CNN models have achieved the best results in computer vision [57]. They have also been applied successfully to sequential data such as sentences, time series and speech [58; 59; 60; 61; 62; 63]. CNNs employ sets of shared weights across the whole input, which is efficient both statistically and computationally. CNNs are particularly interesting in domains associated with the processing of large amounts of data [64]. Since patient load data and traffic data can be categorized as big data, the application of deep learning-based solutions is suited to solve the combined problem of patient load predictions and intelligent inter-hospital patient transfers. We point out that in recent times deep reinforcement learning (Deep-RL) has gained popularity. Deep learning basically models a scenario in which no direct interaction exist between the algorithm and the environment it is operating in. Basically, RL enables a feedback loop between the algorithm and the environment. It allows the algorithm to experience a dataset that varies with time as a result of the interaction with the surrounding environment. RL is applicable to scenarios that can be modeled by a Markov decision process (MDP). Recent works based on RL techniques have shown comparable performance against NNs. In [65], [PERSON] et al. applied an adaptive neural fuzzy inference system (ANFIS) to study the problem of load forecasting in power systems. They further pointed out the difficulty of load forecasting in power systems resulting from the complexity of the power load series as it exhibits seasonality levels and the fact that power load series have various weather related exogenous variables. In [66], the authors applied principles of RL and game theory to develop an autonomous evacuation process to support distributed and efficient evacuation planning. Deep-RL methods results when deep NNs are used to approximate any of the components of RL, e.g., action-value and/or the policy functions [67]. A combination of NN with RL will enable solving even more complex problems. Despite the wide range of successes, current state-of-the-art Deep-RL methods still face a number of significant drawbacks [68].