Venue: NeurIPS 2020 Workshop SVRHM

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper shows that the disentangled variational autoencoder models for faces learn a set of latent dimensions which are shared with the representation in the human brain. Their main results are 1) the average encoding performance for dVAE is comparable to that for regular VAE. 2) Latent dimensions in their model correlate with non-overlapping regions in the brain. 

Overall, I believe this paper demonstrates interesting findings which suggest that compared to standard deep learning models, dVAE models can potentially provide additional interpretation of the representation in the human brain.

Pros:
- Using VAEs for modeling visual processing in the human brain is interesting.
- Some latent variables in dVAE are easily interpretable. 
- Clear delivery

Cons:
- It is not clear why there is a distinction between training and test face images in line 80.
- I see that only two of the latent variables were included in analysis for the results in section 4.2. For the other latent variables, do they map to voxels that spatially overlap with others? Those latent variables may be less human interpretable, but it would still be helpful to understand whether the human brain and dVAE have additional shared disentangled features.
- A quantification on the overlap of voxels correlated with the latent variables will be helpful.
- How does the number of latent variables in dVAE affect your results?

review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary of contributions: 
The authors propose a disentangled VAE as a more interpretable encoding model of human fMRI activity. They show that dVAE shows better encoding than a simple VAE and demonstrate that it can be used to disentangle regions in the brain corresponding to meaningful interpretable dimensions such as 3D rotation and gender  

Significance:
The approach is relevant to understanding how the human brain performs face perception. However, the results do not provide sufficient evidence and might need some improvement.

Pros:
1. dVAE seems to find interpretable latent such as Gender and 3D rotation
2. The paper is written clearly 

Cons:
1. In Table 1, the Spearman correlation is quite low and might even not be significant
	1. I would encourage authors to perform some statistical test to find whether the correlations are significant or not
	2. The authors might consider finding out data quality using noise ceiling or split-half reliability for the 20 test faces to estimate data quality and compare the correlation with noise ceiling
2. In Figure 4, the results can be visualized in a better way showing both the beta-weights in a single brain map to find out whether there is an overlap or not. One possibility is showing beta weights corresponding to gender and rotation in a single brain map using different colormaps. Further, the authors might consider performing statistical tests to validate their claim  

Suggestion to authors:
Please read cons
review_title: Results need more work to validate the claims, a good step towards interpretable encoding models

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper is investigating VAEs and disentangled VAEs as potential representation of faces in the human brain. It claims that disentangled VAEs similarly well match fmri data as VAEs but have the benefit of additional control and disentanglement. They further claim that gender and 3D rotation (yaw) is represented disentangled in the brain.

I'm not an expert of evaluation of fmri data, but I have seen such evaluations before. And for me those results are just not convincing to me. The statistical evidence is rather limited and the effect is also not really consistend among the 3 subjects. It is also unclear why not a VAE-GAN as proposed in the paper the provides the dataset was used for comparison?

I got confused with the wording of dVAE and FactorVAE, that is refeRring to the same, right? In the discussion you then refer to [18] which uses a beta-VAE, which is different (?) but say they use a dVAE?

Overall the paper is interesting and might hold interesting results, but for me it is not conclusive. I think that is fine for a workshop paper, but I think the limited results are overstated in this work
review_title: interesting idea with limited results

Decision: Accept (Poster)