## 2 Materials and Methods
### The Design of a Restoration Task Method for Finger Vein Image Enhancement and Denoising
#### 2.4.1 The Design Idea of the DRL-FVRestore Method  
The architecture figure of the DRL-FVRestore at the step \(t\) is shown in Figure 6. As can be seen from the figure, state is the input to the agent and when \(step=t\) the agent \(f_{agent}\) receives the current input of the finger vein image \(I_{t}\) and its input value vector \(V_{t}\), which is obtained from the network output of the previous agent at \(step=t-1\). Based on the maximum value of the output \(V_{t}\), the behaviors \(a_{t}\) is selected and the corresponding method is chosen to restore the current finger vein image in the image restoration task based on \(a_{t}\). After the restoration process, the restored image \(I_{t+1}\) and the value vector \(V_{t+1}\) are obtained. Based on the maximum value of the output \(V_{t+1}\), the action \(a_{t+1}\) is selected and the restoration process continues iteratively for the image \(I_{t+1}\) until the desired value is reached. At this point the agent chooses to stop the operation and ends the restoration process. Therefore, to design an agent that can select a restoration behavior based on the state of the image, it is first necessary to determine the'state', 'action', and'reward' of the agent. **The state of agent. ** The Agent needs an 'Observation' to observe the input finger vein image information, and in reinforcement learning,'state' is this 'Observation'. As shown in Figure 6, the input to the'state' of the intelligence is \(f_{agent}(I_{t},V_{t})\). 'State' is provided to the agent for the current finger vein image \(I_{t}\) to be processed and the vector of values \(V_{t}\) obtained by the previous agent. Knowledge of the previous decision can help in the selection of the restoration action for the current step. **The action of agent. ** In this work, we define the behaviors space as an 8-dimensional vector \(a_{i}\{a_{1},a_{2},a_{3},a_{4},a_{5},a_{6},a_{7},a_{8}\}\), as shown in Figure 6. Among them, the behaviors vectors \(\{a_{1},a_{2},a_{3},a_{4},a_{5},a_{6},a_{7}\}\) each correspond to a finger vein image restoration method. \(a_{8}\) is the restoration task stopping behaviors. The restoration task for the current image will end when the agent selects the stop behavior. **The reward of agent. ** The reward drives the training of the agent when it maximizes the cumulative reward. 'Reward' can determine the result of the image quality of the input finger vein image after restoration. To achieve a final restored finger vein image with clear details and satisfactory quality, we introduce the \(\mathcal{L}_{2}\) loss of the vein feature as a texture constraint in the reward function, and the perceptual loss \(\mathcal{L}_{perc}\) of the original image to jointly constrain the training of the agent, where \(\mathcal{L}_{perc}\) is the same as the perceptual loss for the motion blur removal task. Assuming that the current agent is at the step t stage, the specific equation is shown in Equations (20) and (21):  
\[\mathcal{L}_{2,t+1}=\sum_{i}\frac{1}{N_{i}}\Big{(}f_{feature}(I_{t+1})-f_{ feature}(I_{t})\Big{)}^{2} \tag{20}\]  
\[\mathcal{L}_{perc\_{t+1}}=\mathbb{E}\Bigg{\{}\sum_{i}\frac{1}{N_{i}}\|\phi_{i }(I_{t+1})-\phi_{i}(I_{t})\|\Bigg{\}} \tag{21}\]  
where \(I_{t}\) is the image restored from the previous step of the agent and \(I_{t+1}\) is the finger vein image obtained by the agent at the end of the current step. The \(f_{feature}\) is feature extraction based on direction valley method [33]. To ensure that the image quality is improved at each step, our stepwise reward design is shown in Equation (22):  
\[r_{t}=1\times(\mathcal{L}_{2,t+1}-\mathcal{L}_{2,t})+0.6\times\big{(}\mathcal{ L}_{perc\_{t+1}}-\mathcal{L}_{perc_{t}}\big{)} \tag{22}\]  
In this study, we designed reasonable reward and punishment mechanisms in Equations (20) and (21) to constrain the agent's behavior. In order to improve the quality of the finger vein image after each restoration, we designed a progressive reward \(r_{t}\), as shown in Equation (22).