# Dementia Classification Using Deep Reinforcement Learning for Early Diagnosis
## 3 Proposed System
### Extreme Gradient (XG) BOOST Learning with Deep Reinforcement  
Figure 1, step 2, depicts a novel approach for developing a policy that leads the actor-network in the selection of samples for annotation. The next sections go over the suggested approach in depth. \[XG_{ij}=Pr(y_{i}^{u}=j|n_{i}^{u};\ \theta_{d}), \tag{2}\]  
Here \(y_{i}^{u}\) represents the uncertain label, and \(n_{i}^{u}\) represents an unlabeled training sample. Action: Define \(\theta\)a as the actor network's parameters. Since the objective of an actor-network would be to annotate training samples out of unlabeled training data, we specify the action in terms of \(\in(0,1)^{N_{u}}\), where every other component refers to an unlabeled training instance. Also, as an activation function for every element, the sigmoid function is used to produce a value amid 0 and 1. A policy \(\pi(S,\theta_{a})\) is learnt to produce action in response to a state S. After collecting the action vector; one can rate all candidate image samples, excluding those that have already been picked, and pick the very first specimens with the greatest values for annotations. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Layer** & **Output** & **Kernel Size** & **Activation Layer** \\ \hline Conv2D-1 & 128 & 3\(\times\) 3 & RELU \\ \hline Conv2D-2 & 64 & 3\(\times\) 3 & Sigmoid \\ \hline Conv2D-3 & 32 & 3\(\times\) 3 & RELU \\ \hline Conv2D-4 & 16 & 2\(\times\) 2 & \\ \hline \multicolumn{4}{c}{Pool size(2 \(\times\) 2)} \\ \hline Dropout & & & 0.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Details of CNN Layers. The annotator's several sorts of samples and labels are described by \(\{n_{i}^{s},l_{i}^{s}\}_{i=1}^{n_{s}}\) whereas the enhanced labeled training data are represented by  
\[(X_{l},Y_{l}):=(L_{l},Y_{l})\ \cup\ \{x_{l}^{s},y_{l}^{s}\}_{i=1}^{n_{s}}.\]  
State Transition. One can upgrade the classifier using the enriched training data \((X_{l},Y_{l})\) by minimizing the cross-entropy loss in Equation (1). Once the selected samples have been annotated and incorporated to the labeled training data, the modified classifier is then used to generate the new state matrix \(\mathcal{S}\#\) depending on Equation (2). Reward. We recommend that the actor emphasize more on those samples that are very likely to be misidentified by the classifier in terms of improving the classifier's performance. To do this, we devise a novel incentive mechanism based mostly on the annotator's true labels. We define \(\hat{k}_{i}\) as the predicted label received from the classifier, i.e., \(\hat{k}_{i}=max_{j}Pr\left(y_{i}^{s}=\hat{k}_{i}|x_{i}^{s};\theta_{d}\right)\) for the sample selection \(x_{i}^{s}\) and \(k_{i}\) as the true label gained from the annotator. The reward is as follows:  
\[r(S,a)=\frac{1}{n_{s}}\sum_{i=1}^{n_{s}}Pr\left(y_{i}^{s}=\hat{k}_{i}|x_{i}^{s };\theta_{d}\right)-Pr(y_{i}^{s}=k_{i}|x_{i}^{s};\theta_{d}) \tag{3}\]  
If a sample \(x_{i}^{s}\) is correctly classified, then \(k_{i}=\hat{k}_{i}\), and  
\[Pr\left(y_{i}^{s}=\hat{k}_{i}|x_{i}^{s};\theta_{d}\right)-Pr(y_{i}^{s}=k_{i}|x _{i}^{s};\theta_{d}).\]  
A big reward, on the other hand, suggests that the selected samples have been erroneously categorized. This suggests that the classifier should pay more attention to the samples with incorrect predictions. As a result, the actor-network is incentivized to choose these image samples. Reinforcement learning in state \(S\) tries to maximize the anticipated future reward, which would be represented like a Q-value function. The Q-value function is being used to assess the state-action pair \((S,a)\), analogous to Q-learning in conventional reinforcement learning, and is described by the Bellman equation as  
\[Q(S,a;\theta_{c})=E\left[\gamma Q(S^{\prime},\pi(S^{\prime};\theta_{a}); \theta_{c})+r(S,a)\right] \tag{4}\]  
To estimate the Q-value function, we use a critic network with parameter \(\theta_{c}\). By considering the following problem, we want to establish a greedy policy for actors via deep Q-Learning.