## 3 Experimental Results  
### The Results of Reconstruction  
We have trained the datasets in the network to prove the effectiveness of the proposed SRN. The reconstructed handwritten digits and characters of the proposed SRN are shown in Figures 4 and 5, respectively, where the first row denotes the presented images and the second row denotes the reconstructed images of SRN. Overall, the images that were reconstructed by SRN have shown fine reconstruction effects for handwritten digits and characters, especially in capturing the essential features of the presented images. Although there are subtle differences in the strokes, the obtained reconstructions of handwritten digits and characters shared certain characteristics of their corresponding original images. In addition, some original images that have complex shapes and architectures are difficult to be reconstructed, but they are more recognizable to the correct class that was reconstructed by the proposed SRN. We attribute this phenomenon to the fact that, when we train the SRN for visual reconstruction, we take the relationship of different batches into consideration, which takes advantage of data in the limitation of sample number. After the training process, the feature vectors of same classes are clustered, while the feature vectors of different classes are dispersed. Therefore, the features that were extracted by well-trained SRN from the same classes are similar. The images reconstructed based on these features have similar appearance. It is an open and difficult problem to evaluate the  
Figure 4: Examples of reconstructed 10 distinct handwritten digits obtained from Dataset 1. Figure 5: Examples of reconstructed 12 distinct handwritten characters obtained from Dataset 2.  
quality of reconstructed images [36]. To better evaluate the reconstruction performance quantitatively, we used several standard image similarity metrics, including Pearson's correlation coefficient (PCC), mean squared error (MSE), and structural similarity index (SSIM) [20]. However, traditional metrics, such as per-pixel MSE, not evaluating the joint statistics of the results is because they do not measure the very structure that the structured losses aim to capture. Therefore, as the auxiliary metrics, MSE and PCC do not well indicate the similarity. By contrast, SSIM can address this shortcoming by taking texture into account, so it is a more convincing measure of structural similarity. In the experiment, we employed 10-fold cross validation to test the proposed Siamese method. We listed the PCC, MSE, and SSIM of the reconstructed images in the SRN. The results of Dataset 1 and Dataset 2 are shown in Tables 2 and 3, respectively. Next, we presented the quantitative results of the two test datasets based on 10-fold cross validation as compared with the several state-of-the-art methods. As shown in Table 4, in both Dataset 1 and Dataset 2 the PCC and MSE of our proposed Siamese method were slightly weaker than those of the DGMM [20]. When compared with the latest methods, the Siamese reconstruction net provided about 8% higher accuracy on the most important SSIM metric in Dataset 1 and 10% higher accuracy in Dataset 2. The better performance is caused by the fact that converging or dispersing input pairs in feature space according to whether they belong to the same or different classes and making full use of limited training samples by learning to compare. The high values of MSE and PCC should not be a cause of concern, because it is not so important for pixel-level comparison due to the complex noise in the fMRI data and limited samples. Moreover, humans do not care much on detailed information at the pixel level and the structure according to attention mechanism. This is the reason why we should disperse the characteristics of extra-classes to distinguish the class. Consequently, the two metrics only serve as auxiliary measures and our proposed method performed better overall.