# Deep-Learning Model Selection and Parameter Estimation from a Wind Power Farm in Taiwan +
## 2 Relate Work
### Reinforcement Learning Algorithm  
To solve the problem of expensive cost in grid search, a dynamic programming-based reinforcement learning algorithm has been proposed to assist engineers in finding the optimal performance for specific problems by adjusting the definite hyperparameters for the developed model.  
Notably, the reinforcement learning-based algorithm has shown great success in many real-world hyperparameter optimization problems, including hyperparameter tuning in controlled environments for deep learning network modelling [6; 15; 16; 26; 27; 28]. Many researchers have focused on neural architecture search (NAS) and hyperparameter optimization using reinforcement learning approaches. For example, [PERSON] and [PERSON] addressed hyperparameter optimization of RNN and LSTM using reinforcement learning to design automation of deep learning networks [26]. Later, [PERSON] et al. used reinforcement learning to tune the hyperparameters for a multi-layer perceptron (MLP) and CNN [27]. [PERSON] et al. used an ensemble learning scheme with reinforcement learning strategy to increase the prediction accuracy of wind power forecasting [28].  
In the following, we review the basic principle of reinforcement learning algorithm.  
In real applications, decisions seldom observe the complete state of the system; it is more common to receive observations of the state that are noisy or incomplete. Markov decision processes (MDPs) with stochastic theory have been proposed to solve these model decision problems in incomplete states. The MDP focuses on finding a good policy that will yield the maximum cumulative rewards by taking a series of actions with respect to specific states of the observed system.  
In the problem model for the MDP, an agent (decision maker) may interact with its environment and change its behavior using an action in available states \(S\) from the consequences of actions \(A\). The dynamic process responds at the next time step by randomly moving into a new state and giving the decision maker a corresponding reward \(R\)[6].  
A major approach in reinforcement learning, namely the \(Q\)-value approximation, is to select an action \(a_{t}\) that maximises the expected reward at any state \(s_{t}\). The goal of the agent in \(Q\)-value approximation is to find an optimal policy in the sense of maximizing the expected value of the total reward over any and all successive steps, starting from the current state. Therefore, this algorithm calculates the quantity of a state-action pair as [26].  
\[Q:\ S\times\ A\to R \tag{10}\]  
The update rule \(Q\)-value approximation is shown as  
\[Q(s,a)=(1-\alpha)Q(s_{t},a_{t})+\alpha\Big{(}r_{t+1}+\gamma\ \underset{a}{\max}Q(s_{t+1},a)\Big{)} \tag{11}\]  
Here, \(\alpha\in\) (0, 1) is the learning rate. A factor of value '0' lets the agent not learn anything, while a factor of value '1' lets the agent consider only the most recent information. Equation (11) indicates that the \(Q\)-value approximation is a multi-decision process based on the MDP framework to provide a generalised framework for optimal behaviour based on actions of the agent to maximise the cumulative rewards of a series of decisions.  
Recently, [PERSON] et al. addressed hyperparameter optimization of CNNs through a Q-learning-based approach with a multi-agent system [27]. In the proposed multi-agent Q-learning-based approach, agents are able to communicate through our novel definition of state-action pairs, Q-tables, and a Q-table update rule. Thus, a Q-learning scheme was adopted for hyperparameter optimization and to examine the predicted accuracy of the model for overcoming an expensive cost problem scheme in grid search.