Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the encoding and decoding of faces in the (macaque) brain. To study decoding, they generate faces by sampling a random vector from the latent space of a GAN. They show the faces to a macaque monkey and record the neural response. A linear model is trained to map the neural response to the latent vector that produced the face image. On test data, they find that the recovered vector can be used to produce a high quality image of a face that matches the original face qualitatively and quantitatively. To study encoding, they map the activations of a trained vision model (AlexNet, VGG16) to the neural response. They find that correlation with the neural response increases with depth in the machine model.
strength_and_weaknesses: # strengths
- New macaque monkey data.
- This seems like the first attempt at reconstructing faces from intracranial data. (Is it?) If so, that should be mentioned somewhere
- The reconstructed images are impressive

# weaknesses
- I have concerns about the originality of this work. Decoding high quality faces from brain signals has been previously accomplished (Dado et al. 2021, VanRullen and Reddy 2019, Güçlütürk and Güçlü 2017). The decoding and encoding techniques, namely the use of GAN latent spaces, have already been discussed in previous works. In this work, the authors use a slightly different model (StyleGAN3 vs StyleGAN), but the overall methods remain the same.
- The analysis of reconstructed faces in terms of attributes is also introduced in these previous works. (The authors do acknowledge this)
- I think the paper could be much improved if the relationship with previous works were more clearly explained.
- The novel contribution of this work seems mainly to be the newly collected data. But even this is only done for one subject.
- The difference between the $w$ latent space and $z$ latent space could be explained more clearly. It's mentioned that the $w$ latent space is obtained by passing the $z$ latent space through an MLP, but since it results in much better reproductions (relative to $z$), could the authors explain a little bit more about what makes the two spaces different?

## references
- Dado et al. 2021 Hyperrealistic neural decoding: Reconstructing faces from fMRI activations via the GAN latent space (https://www.biorxiv.org/content/10.1101/2020.07.01.168849v3.full)
- VanRullen and Reddy 2019 Reconstructing faces from fMRI patterns using deep generative neural networks https://www.nature.com/articles/s42003-019-0438-y
- Güçlütürk and Güçlü et al. 2017 Reconstructing perceived faces from brain activations with deep adversarial neural decoding
clarity,_quality,_novelty_and_reproducibility: - originality: many previous works should be mentioned (see above section). As it stands, I don't think this paper adds a lot of novelty.
- clarity: Clarity is good. Some figures (figure 7) have hard to read portions.
- reproducability: The authors have said they will release the data, including the macaque data and code
- quality: The reconstructed images are impressive. But confidence in the significance of these results could be improved by a more complete discussion of the permutation test results. (See below) 

# questions/minor comments
- figure 8B: typo - "Brunet" --> "Brunette man"
- Section 2.3.2 $Y_i$ is defined, but I can't see where it is ever used?
- figure 7A: Hard to see the texture of VGG16
- If it's simple to produce, could we see a plot of all electrode locations?
- The authors mention a permutation test in section 3.1? Could we see the full results of the permutation test? What is the average closeness of the random latent vector to the ground truth? 
summary_of_the_review: The reconstruction results are impressive, but have already been accomplished in many previous works. The decoding techniques are not significantly different than what was introduced in prior work, although the data is new. But even then, the data consists only of neural recordings for a single subject.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper is about "Neural coding", that is to characterize the biological brain recognization. They used a StyleGAN3 to synthesis several high solution images, and show those images to a 7 years old macaque , and recorded its multi-unit brain activity. Finally, they use several latent similarity metric to compute distant z and w space. Finally, they conclude that StyleGAN3 W space is better than Z space in decoding performance. 
strength_and_weaknesses: Pro: 
The idea to evaluate StyleGAN3 W space and Z space by using a macaque is interesting. 
The experiments are extensive and solid. 
Con:
I believe this paper is more experimental. There is not much novelty. 

clarity,_quality,_novelty_and_reproducibility: The experiments and writing are clear and sound. It might be hard to reproduce the result, since it needs a macaque.

summary_of_the_review: It is my first time to  know "neural coding" field. 
The experiments that to explore brain actively with neural representation is interesting. 
Although the there is a face that StyleGAN3 W space is more disentangled than Z space, the experimental shows its closer relationship to brain perceived stimuli is exciting.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The author found that the neural responses of macaques to facial visual stimuli are very similar to the w-space of styleGAN3. Specifically, the neural response of the macaques' brain can be decoded into w-space using linear mapping, and the corresponding generation faces are similar to the perceived stimuli. Also, the reconstructing looks similar when applying the same linear operations on the brain activity and the latent.
strength_and_weaknesses: Strength:The author finds evidence that the neural face manifold and the disentangled w-latent space conditioned on StyleGAN3 are similar in how they represent the high-level semantics of the high-dimensional space of faces. Especially, the StyleGAN3 has never been optimized on neural data.

Weaknesses: 
1) Why the author use a AlexNet pretrained on object recognition task rather than on the face recognition task? What is the different between the cosine similarity from the AlexNet and the VGGFace?
2) For the conclusions in the summary, “This provides strong evidence that the neural face manifold and the disentangled w-latent space conditioned on StyleGAN3 (rather than the z-latent space of arbitrary GANs or other feature representations we encountered so far) share how they represent the high-level semantics of the high-dimensional space of faces”, the author should add some qualitative comparison to make a visual support. Also, the qualitative comparison can give the reader a visual feeling of the difference in image space caused by the difference in six decoding performance evaluation metrics.
3) The gap between the z-latent and w-latent in the “Alexnet sim” and “VGGFace sim” are much more lower than in the “Lat. sim.” and the “Lat. corr.”, how does the author explain this phenomenon? How much does the difference in “Alexnet sim” and “VGGFace sim” affect the reconstruction image quality? If the visualized difference is not obvious, it will weaken the conclustion that “This provides strong evidence that the neural face manifold and the disentangled w-latent space conditioned on StyleGAN3 (rather than the z-latent space of arbitrary GANs or other feature representations we encountered so far) share how they represent the high-level semantics of the high-dimensional space of faces” in the abstract. 

clarity,_quality,_novelty_and_reproducibility: The experiments are not based on a public dataset and no implementation was released, which makes difficulty of reproduction.
summary_of_the_review: This paper provides evidence that the neural face manifold and the disentangled w-latent space conditioned on StyleGAN3 are similar in how they represent the high-level semantics of the high-dimensional space of faces. But more comparison are suggest to be added between the reconstruction quality of z-latent and w-latent.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary: Decoding neural activity while subjects are presented with images into the w-latent space of StyleGAN3.

Strengths: A new dataset. The first reconstruction of faces from intracranial data.

Weaknesses: Limited novelty and a lack of clarity about what the ICLR community can learn from this work. It's unclear how the dataset could be reused, it's fairly small (2 subjects), and designed very specifically for this experiment. An audience that would appreciate the value of decoding faces from intracranial data on its own would be more appropriate and receptive.
justification_for_why_not_higher_score: The method is of limited interest to the ICLR community. A venue where the key contribution is not the idea, but any improvements in performance, would be a much better fit.
Decision: Reject