# Quantum Machine Learning for Security Assessment in the Internet of Medical Things (IoMT)
## 3 Proposed Model and Research Framework
### Deep Q Neural Network (DQNN)  
Q-learning [33] can be used as an off-policy RL technique in temporal-difference learning for prediction in IoMT-based applications [34]. Temporal difference learning techniques can be used to compare the accuracy of the predictions made at different times. The value function Q (\(S\), \(a\)) is then learned. This function shows the value of performing a specific action (a) in a specific state (S) (s). State action reward state action (SARSA) is an example of an on-policy temporal difference learning system for vulnerability prediction in IoMT-based applications. The on-policy control technique [35] selects the best way to implement a policy in each state while the policy is being learned. The goal of the SARSA program is to calculate Q (s, a) for the selected policy and for all the permutations of s and a. (s-a) (s-a). The most significant difference between the SARSA and Q-learning algorithms is that in SARSA, the Q-value in the table does not need to be changed by knowing what the highest reward will be for the next state. SARSA decides what the next step and reward will be based on the same policy that determines the first action. This maintains consistency. The quintuple Q (s, a, r, s\({}^{\prime}\), a\({}^{\prime}\)) used by the algorithm is, where the term SARSA is derived from. Where  
s: the starting point a: The initial action taken r: what people hope to get out of following the rules The letters s and a represent a new combination of a state and an action. DQN describes \"Q-learning through Neural networks\". Defining and maintaining a Q-table in an environment with a vast state space is a difficult and time-consuming task owing to the complexity of the domain. When we use the DQN method, we can find an answer to this problem. In this example, the neural network approximatively calculates the Q-values for each action and state, rather than specifying a Q-table. Q-learning is a popular model-free reinforcement-learning algorithm based on the Bellman equation. The goal of Q-major learning is to learn the policy that tells the agent what to do and when to do it to obtain the highest reward. This is an unofficial RL that tries to determine the best thing to do right now. In Q-learning, the agent's goal is to make Q's value as high as possible. With the help of the Bellman equation, you can figure out the value of the Q-learning algorithm. The Bellman equation is as follows:  
\[V(s)=\max\{R(s,a)+\gamma\Sigma_{s^{\prime}}P)V\big{(}s^{\prime}\big{)}\} \tag{1}\]  
Equation (1) has several variables, such as the reward, discount factor (), probability, and end states (s\({}^{\prime}\)). However, it does not consider the Q-value; therefore, we start with the image above. The letter V represents a variable with values s1, s2, or V. (s3). Because this is a Markov Decision Process, the agent only cares about the current state and what will happen in the future. The agent is free to go wherever he wants, so he has to choose between three possible routes to reach his destination as quickly as possible. Here, the agent acts based on the odds, which causes the system's state to change. However, we will have to make some changes to the Q-value if we want our maneuvers to be accurate. Q is a way to determine how well the steps taken in the situation work. Therefore, instead of using a value at each stage, we used the letter Q to represent a set of states and actions (s, a). The agent decides what to do next by looking at the Q-value, which shows which action is more lubricating than the others. The Bellman equation was used to determine the Q-value. When the agent performs an action, he either receives a reward of type R(s, a) or moves to a different state.