## 2 Materials and Method
### Method Phase 1--Pre-Processing of Input Data
Before the agent begins to search through the features, it needs to be trained on how to recognise the actions that provide the maximum reward and others that take away the reward (penalise) when bad action is taken [10; 11; 12; 24]. To build the algorithm for feature selection, we have considered the following:  
**Environment**--It is the set of features ([PT112], [TT105], [V], [PT112 TT105], [PT112 V], [TT105 V], [PT112 TT105 V]) in a sample of data as shown in Figure 9.  
Figure 8: Schematics of DRL showing its agent. Figure 6.0 Plot of input features pre-processed using (**a**) normalisation and (**b**) z-scoring. **Action--**Selection of features to find out which is best to predict [TT121]. As there are 7 subsets of features, there will be 7 possible actions the agent can choose from in the space. **Observation--** Each time an action is taken to select a feature subset. The observation generated from the environment is a 3 \(\times\) 1 matrix consisting of the error of the previous action, current action, and integral of the current error (RMSE). **Type of Agent based on Policy--**A Deep Q-Network (DQN) agent is used for taking action. It is a value-based reinforcement learning agent that trains a critic to estimate the return or future rewards [18]. The observations are continuous signals of errors while the actions are discrete (7 possible actions). During training, the agent does the following:  
* Explores the action space of the set of features;
* During each iteration interval, the agent selects a random action for which the value function is the greatest;
* The agent updates a critic based on a mini-batch of experiences. **Reward--**To determine the best next action in the environment consisting of a feature set, a measure is necessary to compare feature selection. [PERSON] et al. [10] introduced a criterion named AOR (average of rewards), which is the average evaluation of each feature whenever it is selected in the environment, and it is determined by (9). Figure 6.1  
where:  
\(v(feature_{t})\) is the assessment value function of the current state. \(v(feature_{t+1})\) is the assessment value function of the successor state. We will use this approach to develop a unique evaluation system in our novel algorithm. Method Phase 3--Authors' Approach--DRL-Based Algorithm for Feature Selection Using Novel Evaluation Criteria  
Based on phase 1 and phase 2, the authors' approach consists of an algorithm that implements a DRL environment and a DQN agent with training parameters optimised based on Figure 6.2 The reward for each action that the agent takes is determined by evaluating the minimum of an error difference (RMSE), as shown in expression (10). Figure 6.3 Schematics of methodology in the DRL algorithm for evaluating features to predict TT121. Figure 6.4  
Expression (10) gives the minimum difference between the error (RMSE) arising from predicting TT121 by an initial LSTM neural network and another prediction of TT121 using the same (initial) LSTM when fed with each feature subset selected by the DRL agent. The methodology of the authors' approach is shown in Figure 6.5  
For the developed DRL algorithm to select features, only a small sample of the dataset is required after pre-processing which reduces computation time as well. Also, it is important to note from Figure 9 that for feature subsets consisting of more than one sensor data, an average value that gives a single representation of the combination is used for evaluation. Figure 10 shows an illustrative example of the methodology followed in the proposed DRL algorithm. The single representation has inherited characteristics from each sensor data such as contribution to the RMSE. This approach makes the proposed DRL algorithm unique because it can handle complicated features subset of tens or hundreds of sensor data by reducing them to a single representation, which further reduces computation time.