# A Modified Quad Q Network Algorithm for Predicting Resource Management
## 3 Description of the Quad Q Network Algorithms and Functionalities  
In the Deep Q Network, the idea of target network is the key. If one looks at the expression that updates the Q value in Q-Learning, it's the same as \(Q(s,a)=Q(s,a)+\alpha(R+\gamma maxQ(s^{\prime},a^{\prime})-Q(s,a))\). In the Deep Q Network, the target is \((R+\gamma maxQ(s^{\prime},a^{\prime})-Q(s,a))\). The idea of Dueling DQN is to divide the result value of the network by V and A before obtaining the Q value and then merge it again. Figure 1 shows the combination of V (value function) and A (advantage) at the bottom. The concept of Dueling DQN starts with splitting Q value into two. The existing Q value refers to the value when an action is taken at a given state, which is divided into two terms. One may wonder why we first divide then merge later. In order to see where the agent is interested in obtaining future rewards from a value stream perspective, to maximize future rewards, the agent ends up on the way to score and resource forecasting and resource forecasting. Based on the advantage stream, if no action is taken right away, one's reward is affected. Thus it is necessary to pay attention to the preceding value. If you dualize Q-learning into two streams, one can establish which state is valuable without having to learn the results by trying to act on each state. In a normal DQN, the agent must calculate the Q value for each action in a state when the value function is bad. Calculation of the Q value is done even in situations where the state is different or not best to get a reward. Despite performing all actions possible in the states, a bad results can be obtained. The reason for dueling is because normal DQN calculates Q-learning with state and action pairs, If either of these values is bad, then calculating the Q value with them will only prevent us from learning the optimal policy, thus, the priorities are re-arranged priorities. This will reduce unnecessary computational time and allow learning with slightly higher quality samples. Figure 2 below shows a schematic diagram of the Dueling DQN algorithm written as a control in this study. In the Figure 2, \(q_{\theta S^{\prime}}\) assumes value as the following array (1), calculated from Value stream and Advantage stream. Figure 1: A figure showing the DQN network structure in Grid World. Below shows Dueling DQN. Figure 2: A figure showing Dueling DQN into two terms from the point of view of a Value stream and an Advantage stream. \[q_{\theta s^{\prime}}=\left[ \begin{array}{ccccc}4.1&12.4&3.1&2.1\\ 3.7&3.2&14.2&1.0\\ 10.2&2.1&1.2&7.2\end{array} \right] \tag{1}\]  
In addition, in Figure 2, the \(argmax(q_{\theta s^{\prime}})\) value can be presented as in the following array (2) values in terms of the weight for each row. \[argmax(q_{\theta s^{\prime}})=\left[ \begin{array}{ccccc}1&2&0\end{array} \right] \tag{2}\]  
These Algorithms can fully play a role in increasing the accuracy of the existing Deep Q Network process. In addition, existing algorithms have the advantage of being easy to apply to Double Deep Q Network algorithms. The process is shown in Figure 3 and \(q_{\theta s^{\prime}}\) was assumed to be array (3). \[q_{\theta s^{\prime}}=\left[ \begin{array}{ccccc}4.1&2.4&5.1&2.1\\ 3.7&4.2&1.2&1.0\\ 1.2&2.1&1.2&8.2\end{array} \right] \tag{3}\]  
\(argmax(q_{\theta s^{\prime}})\) through \(q_{\theta s^{\prime}}\) is applied to \(q_{\theta s^{\prime}}\), and \(Double\)\(q_{s^{\prime}}\) is derived using Formula (4). \[Double\)\(q_{s^{\prime}}=q_{\theta s^{\prime}}[range(batch_{size}),argmax(q_{\theta s ^{\prime}})] \tag{4}\]  
The above \(Double\)\(q_{s^{\prime}}\) calculated using Formula (4) gives the following array (5) when it is calculated based on the example. \[Double\)\(q_{s^{\prime}}=\left[ \begin{array}{ccccc}2.4&1.2&1.2\end{array} \right] \tag{5}\]  
The calculated \(Double\)\(q_{s^{\prime}}\) is substituted in Formula (6), and the calculated Formula (6) is substituted in Formula (7) in order to calculate the final \(loss_{s^{\prime}}\).