## 2 Materials and Method
### Method Phase 1--Pre-Processing of Input Data
Before the agent begins to take actions to select features, the algorithm checks that an appropriate weight has been computed in an initial LSTM neural network which has a simple architecture of 1 input and 1 output. It uses the matrix of TT121 which is our interested output (\(y\)) to train the LSTM and adjust its weights. Then, it is tested to obtain a reference RMSE\({}_{y}\), which is stored in memory. Thereafter, during iteration, the agent in the DRL algorithm (Algorithm 1) selects each feature subset from the matrix of features which is the environment. The pre-trained (initial) LSTM is then used again to compute another RMSE\({}_{t}\). The feature that gives the minimum difference in RMSE based on expression (10) is selected by the agent. Figure 6.6 Illustrative example of feature selection by the developed DRL algorithm. ```
Step 1: Input data consisting of features: pressure [PT112], cooling water temperature [TT105], stack voltage [V], and hydrogen temperature [TT121]. Step 2: Normalise each feature using: z-scoring \(\rightarrow\) (data-mean)/(standard_deviation). Step 3: Create and train an initial deep neural network (LTSM) with [TT121], as both input and output. Obtain the matrix consisting of the subset of features. Step 4: Subset 1 = [PT112]; Subset 2 = [TT105], Subset 3 = [V]; Subset 4 = [PT112, TT105], Subset 5 = [PT112, V]; Subset 6 = [TT105, V]; Subset 7 = [PT112, V, TT105]. Step 5: For feature subsets having two or more sensor data, obtain a single representation by computing the average. Step 6: Create DRL environment model with observations and action settings based on Figure 6.7 Step 7: Create a DRL agent based on DQN policy. Step 8: Define discrete actions for the DRL agent as scalar vector: Action = selection of features from the set in Step 4  Define the reward for the agent based on expression Equation (10): If action results in minimum RMSE difference Step 9: Then reward = 2  Else Reward = -1  End if Step 10: Train the RL agent  For: each stochastic action taken by the agent  Receive observations from the environment model: error (RMSE)  Calculate the reward of the selected action. Step 11: If reward > 0, then  Store feature selected by the agent  Else  Take another action to select another feature from the subset  End
```  
**Algorithm 1** Novel DRL algorithm proposed by authors  
Table 3 shows the training parameters of the DRL algorithm implemented. The DRL algorithm is implemented in MATLAB Simulink as shown in Figure 6.8 The model consists of a trained DRL agent that is created as a block subsystem. The DRL model is trained with the parameters stated in Table 3 as required by the DQN agent and environment. The agent takes actions which is a scalar input from 1 to 7, based on training conducted earlier. The action is used as input to a MATLAB function which contains \begin{table}
\begin{tabular}{l l l l} \hline \hline
**DRL Model** & \multirow{2}{*}{**Type**} & **Training Parameter** & **Value** \\
**Component** & & Learning rate & 0.01 \\  & & Number of hidden layers & 128 \\  & & Gradient threshold & 1 \\  & & Discount factor & 0.99 \\ DRL agent policy & DQN & Batch size & 64 \\  & & Initial epsilon & 1 \\  & & Epsilon decay & 0.005 \\  & & Epsilon min & 0.01 \\  & & Number of training episodes & 50 \\ \hline  & & Observation type & Continuous \\  & & Observation dimension & [3, 1] \\ Environment & & Action type & Discrete \\  & & Actions & [1, 2, 3, 4, 5, 6, 7] \\  & & Observation lower limit & [\(-\inf\), \(-\inf\), 0] \\  & & Observation upper lower limit & [\(\inf\), \(\inf\), \(\inf\)] \\ \hline \hline \end{tabular}
\end{table} 
Table 3: DRL model information used in MATLAB. the code developed by authors for interactions between the blocks. Another subsystem computes the observation matrix. There is also a subsystem to compute the reward based on expression (10). The reward is set up to have a scalar value of 2 when the agent takes an action that reduces the RMSE, while it is penalised by a scalar value of -1 when it takes an action that increases the RMSE.