From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: The paper proposes an approach for 3D-aware video generation, based on implicit representations and adversarial training. The method is evaluated on several datasets, where it approaches or beats relevant baselines (2D video or 3D static image)

The reviewers' evaluation of the paper is mixed, leaning towards negative. The main points are as follows:

Pros:
1. A new task of 3D-aware video synthesis
2. A reasonable solution that works quite well in practice
3. Good presentation, extensive details, and extra experiments in the appendix

Cons:
1. Limited technical novelty: mostly a combination of known components
2. Not super impressive results, the method does not always beat the baselines, 
3. Some missing baselines (D-NeRF, 3D static methods beyond StyleNeRF) and analysis experiments (effect of joint/sequential training, motion vector design)
4. Code not released

Some of the concerns (3, 4) have been addressed in the rebuttal. However, some others (1, 2) haven't quite been addressed (reviewers in a private discussion pointed out that they aren't changing their ratings) 

Overall, it is an interesting paper that has its merit; however, the technical contribution is limited and the results are somewhat mixed. Based on this, at this point, it falls slightly below the acceptance threshold, and I recommend rejection, but I encourage the authors to polish the paper and resubmit to a different venue.


justification_for_why_not_higher_score: 1. Limited technical novelty: mostly a combination of known components
2. Not super impressive results, the method does not always beat the baselines, 
justification_for_why_not_lower_score: N/A

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: Further Discussions with Reviewer dFag
comment: We have provided an answer to all of your concerns and would like to ask if there are any remaining questions? As the discussion period is coming to an end, we would be happy if you could reassess our work based on the provided rebuttal.

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: Further Discussions with Reviewer jBEi
comment: As the discussion period is coming to an end, we would like to ask if there are any remaining questions?

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: Further Discussions with Reviewer PyKL
comment: We have provided an answer to all of your concerns and would like to ask if there are any remaining questions? As the discussion period is coming to an end, we would be happy if you could reassess our work based on the provided rebuttal.

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: Further Discussions with Reviewer rreh
comment: We have provided an answer to all of your concerns and would like to ask if there are any remaining questions? As the discussion period is coming to an end, we would be happy if you could reassess our work based on the provided rebuttal.

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: Summary of Changes
comment: Dear reviewers,

Thank you for your valuable feedback. We believe that we have addressed the concerns of all reviewers with our rebuttal and updated documents. If you think that any additional clarifications are helpful to reassess our work, we are happy to further clarify your concerns. 
Below, we summarize our updates on draft and supplementary materials:
- Contributions stated at the end of the introduction (Sec. 1)
- Consistent notation for equations (Sec. 3.1)
- Extensive investigation of pre-training and simultaneous training on an image dataset with qualitative and quantitative comparisons (Sec. E and supp.html)
- Ablation experiment to justify design choice of time incorporation in motion generator (Sec. F.1)
- Ablation experiment using deterioration of video-based discriminator to replace image discriminator (Sec. F.2)
- Related work for dynamic view synthesis approaches (Sec. G)
- Further qualitative results for motion generator (Ours with Random Motions in supp.html)
- Code with README in supplementary materials

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: Response to Reviewer rreh
comment: Thank you for your valuable feedback.

> **Q1: Technical contribution at the end of introduction**

Thank you for this suggestion. We revised our introduction to clearly state the contributions of our work.

> **Q2: Limited length of videos**

We would like to clarify that the main focus of our work is to perform 3D-aware video generation. Generating 3D-aware videos with longer duration is a great direction for future research that goes beyond the scope of this work. In this work, we only focused on shorter sequences since most of the existing 2D video generation pipelines [MoCoGAN-HD, DIGAN, VideoGPT, etc], besides StyleGAN-V, can only generate videos of a couple of seconds. Extending our approach to videos of arbitrary length is a great direction for future research. One possible approach to address this could be to leverage the acyclic positional encoding introduced in StyleGAN-V and incorporate it into our motion generator to enable 3D-aware video generation of arbitrary length. We mention this limitation in Sec. 5.

> **Q3: Background quality and consistency**

Modeling the background generation for datasets as TaiChi poses an extreme challenge due to its complex and compositional appearance and a wide range of camera viewpoints, which might require larger network capacity or compositionality modeling. As such, state-of-the-art 2D video GAN methods like DIGAN struggle to model realistic backgrounds. Note that since one of the main goals of our work was to demonstrate 3D camera control of a generated scene, while focusing primarily on the foreground moving objects, we used a much smaller network to predict the background in comparison to the foreground (see Appendix B.1) in order to reduce computational costs. However, we agree that for future work exploring the 3D video generation of more complex scenes with realistic backgrounds is an exciting direction to pursue.

> **Q4: Comparison to dynamic view synthesis approaches**

Thank you for this suggestion. We added a comparison to dynamic view synthesis methods, like D-NeRF, in our Sec. G in the appendix. In contrast to our method, this line of work requires videos with different viewpoints to train a single network per scene, hence are not able to generate novel scenes. Instead, we learn a generative model from unstructured single-view video collections.

> **Q5: Limited novelty**

We kindly refer to the general comment above for our response regarding novelty.

> **Q6: Missing code**

We updated our supplementary material and added our code as well as a README with instructions.

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: Response to Reviewer PyKL [1/2]
comment: Thank you for your valuable feedback.

> **Q1: Clarification about time-aware video discriminator**

We would like to clarify that introducing the time-aware video discriminator is not one of our core contributions. We fully acknowledge that we adopted the discriminator from DIGAN, as explained in Sec. 3.2. We do however claim that we are the first video generation work that combines a NeRF-based generator with a time-aware discriminator. All previous 3D-aware image generation models are only able to learn from static images and therefore only synthesize static images. We show that introducing a 4D generator by adding a motion generator to our neural implicit representation and combining it with a time-aware video discriminator enables novel controllability of the video generation process. We kindly refer to the general comment above for a more detailed response regarding novelty. However, upon PyKL’s suggestion, we will rephrase our text to make sure that it properly reflects our contributions.

> **Q2: Design of motion vector**

We would like to clarify that our MLP layers include bias terms, which leads to non-zero feature vectors for time step 0. This design choice is mainly based on empirical observations, namely comparing multiplication, concatenation, and positional time encoding. In Sec 4.3 we report a worse FVD score (175.4) with positional time encoding. Furthermore, we experimented with concatenation, which also results in a slightly worse FVD score (164.7) compared to simple multiplication (158.3). However, we think further extending the motion generator and investigating its incorporation into the 3D content part of the network is an interesting direction for future work. One example could be to try acyclic positional encoding, which was introduced in StyleGAN-V. We updated our draft and included an ablation study on the motion vector as part of our appendix in Sec. F, including the previously not included concatenation result.

Ablations on TaiChi:

| Method |    FVD     |
|--------|:--------:|
|**w/ Positional Time Encoding**          | 175.4  | 
|**w/ Concatenation**          | 164.7  | 
|**w/ Multiplication**          |  158.3  | 

> **Q3: Inconsistent notation in equations**

Thank you for the suggestion. We updated the notation to make it more consistent.

> **Q4: Evaluation of pre-training and simultaneous training**

We kindly refer to the general comment above for our response regarding pre-training and simultaneous training.

> **Q5: Low visual quality**

We believe that one of the main bottlenecks in video generation is the datasets with limited size, diversity, and resolution. For example, while the FFHQ dataset, which is widely used for training image generation pipelines, contains approximately 70k high quality images (1024x1024 resolution) with large variations in terms of age, ethnicity, background and identities, FaceForensics contains video sequences with only 704 identities at 256x256 image resolution with limited viewpoint diversity. While MEAD provides higher quality videos, they contain videos with only 48 speakers in a controlled setup. We believe that using the recently introduced CelebV-HQ dataset [1] that contains diverse and high quality videos would significantly improve the quality of our generated videos. As most of the 3D-aware image GANs report their results on the high quality and diverse datasets FFHQ and CelebA-HQ, their visual quality is not directly comparable to the results provided by video generation works using FaceForensics or MEAD. TaiChi is generally a very challenging dataset, as it is not aligned and includes full body motion with varying backgrounds. We would like to emphasize that even 2D video generation methods are struggling with this dataset, as shown in the visual comparisons to DIGAN. Furthermore, applying more recent advancements in 3D-aware GANs (e.g. EG3D [2] or EpiGRAF [3]) could improve the efficiency and quality of our approach. Lastly, we believe that using the acyclic positional encoding proposed in StyleGAN-V could improve the video quality.

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: Response to Reviewer PyKL [2/2]
comment: > **Q6: Quantitative results sometimes worse than 2D methods**

We kindly refer to the general comment above for our response regarding quantitative comparisons to previous video generation methods.

> **Q7: Comparison to 3D image generators**

We chose StyleNeRF as the comparison to a 3D image generation method as we built our model on top of their approach. The main purpose of this comparison is to show that we can achieve similar quality as a 3D image generator in terms of FID while modeling the additional temporal dimension. We believe that we already provide an extensive comparison in Tab. 1 of our work to other image and video generation methods. Generally, we introduce our work as a video generation method and hence evaluate on video generation datasets. Consequently, our main experiments are in comparison to previous SOTA video generation approaches, where we provide multiple previous approaches as a comparison, namely: VideoGPT, MoCoGAN, MoCoGAN-HD, DIGAN, and StyleGAN-V. Furthermore, to the best of our knowledge, only EG3D [2] outperforms StyleNeRF in terms of FID on FFHQ, while EpiGRAF [3] shows lower visual quality with improved geometry. We believe that using their representations and training techniques for video generation is an exciting direction for future work.

> **Q8: Visualization of geometry**

We want to refer to Fig. 8 in our supplementary material, which includes visualizations of rendered depth maps from our model. This is generally not possible for previous video generation models. However, we agree that further improving geometry is an important topic for future 3D-aware video generation methods. Solutions could be to use more advanced feature representations like the tri-plane representation introduced in EG3D [2] and to leverage the patch-based learning technique of EpiGRAF [3].

> **Q9: Sparse Table 2 and 3**

We use Tab. 1 with the FaceForensics dataset to position our work in comparison to previous video generation and image generation methods. FaceForensics has been the main benchmark for existing video generation datasets, hence we chose this dataset. Moreover, we used the TaiChi dataset to show that our model does not only work on human faces, but is generally applicable. TaiChi was not used by StyleGAN-V or MoCoGAN-HD. Furthermore, StyleGAN-V only used front views from MEAD as part of their experiments, hence we further re-trained StyleGAN-V on the MEAD dataset with different viewpoints for a comparison to our approach. We beat StyleGAN-V by a significant margin on MEAD. As a consequence, we did not further compare to other models which generally show lower quality than the SOTA video generation method StyleGAN-V. We generally want to note that training these generative models requires substantial computational resources and training time. Therefore, we focused on providing comparisons to at least one of the very recent works DIGAN and StyleGAN-V and a more extensive comparison to different types of image and video generation methods on the main benchmark FaceForensics.

> **Q10: Missing code**

We updated our supplementary material and added our code as well as a README with instructions.

[1] Zhu et al., "CelebV-HQ: A Large-scale Video Facial Attributes Dataset", ECCV 2022
[2] Chan et al., "Efficient Geometry-aware 3D Generative Adversarial Networks", CVPR 2022
[3] Skorokhodov et al., "EpiGRAF: Rethinking training of 3D GANs", NeurIPS 2022

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: Response to Reviewer jBEi
comment: Thank you for your valuable and very positive feedback.

> **Q1: Simple datasets and short videos**

We agree with these two limitations and we plan to tackle them as part of future works.

> **Q2: Missing code**

We updated our supplementary material and added our code as well as a README with instructions.

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: Response to Reviewer dFag
comment: Thank you for your valuable feedback.

> **Q1: Evaluation of pre-training and simultaneous training**

We kindly refer to the general comment above for our response regarding pre-training and simultaneous training.

> **Q2: Deterioration of video-based discriminator to replace image discriminator**

Thank you for this idea. For our ablation study in Sec. 4.3 we actually did exactly this when using no image discriminator. We repeat every image along the channel dimension to form a 6 channel tensor and set the time difference to zero, which we further concatenate to obtain a 7 channel tensor. We then use a deterioration of our video discriminator to replace the image discriminator. We also tried simply removing the image discriminator, which results in an even worse FVD score (234.3) compared to using a deterioration of our video discriminator (190.9). Consequently, we observe that using discriminators with disentangled weights significantly boosts the quality. We updated our draft to include the actual ablation without an image discriminator in Sec. 4.3 and moved the deterioration of the video discriminator to the appendix to explain it in more detail.

Ablations on TaiChi:

|  Method |    FVD     |
|--------|:--------:|
|**w/o Image Discriminator**          | 234.3  | 
|**w/ Deterioration of Video Discriminator**          | 190.9  | 
|**w/ Separate Image Discriminator**          |  158.3 | 

> **Q3: Further qualitative results for the motion generator**

Thank you for the suggestion. We added further qualitative results for our motion generator to our supplementary website (Ours with Random Motions).

> **Q4: StyleGAN-V outperforming our approach in terms of FVD and FID**

We kindly refer to the general comment above for our response regarding quantitative comparisons to previous video generation methods.

> **Q5: Comparison to DIGAN and novelty**

We agree that our model architecture may be conceptually similar to DIGAN from the video modeling perspective, as both models use neural implicit representations. However, we are tackling a substantially more difficult task, namely 3D-aware video generation. The main contribution of our work is that we introduce the first approach to explore this new task. As you mentioned, no previous video generation model has investigated NeRFs and how to use them in order to learn from monocular 2D video data. In comparison to DIGAN, we are able to synthesize videos from different viewpoints without a loss in quality, which makes a significant difference in terms of model controllability. We kindly refer to the general comment above for a more detailed response regarding novelty.

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: General Response [2/2]
comment: > **Q3: Quantitative comparisons to previous video generation methods (Reviewers dFag and PyKL)**

We believe that our benchmark numbers are competitive against previous works when you look at the overall picture, even though we model a more complex task with additional dimensions. While our method shows worse FVD score (lower the better) on FaceForensics compared to StyleGAN-V, the difference against DIGAN is only around 10%, and we even outperform DIGAN in terms of FID. On the MEAD dataset, our FVD (55.4) is only half of FVD (109.3) of StyleGAN-V, showing competitive visual quality of our method against the current best video GAN. Besides the two most recent methods, we outperform all other previous 2D video generation methods on FaceForensics by a significant margin, including MoCoGAN-HD (ICLR 2021). We expect that the quality difference between 2D and 3D video generators will further decrease with future works, similarly to 3D image generators currently closing the gap to 2D image generators like StyleGAN2. We want to emphasize that our work is the first approach we know on 3D-aware video generation, so it is difficult to expect our method to outperform all the previous works that tackle simpler tasks.

From: ICLR.cc/2023/Conference/Paper1419/Authors
title: General Response [1/2]
comment: We thank all reviewers for the thoughtful and constructive feedback. Below we address the common questions raised by the reviewers.

> **Q1: Novelty**

We would like to clarify that one of the key contributions of our work is that we introduce the first 4D GAN model that learns to generate multi-view consistent video data from single-view videos. While we agree that our model is inspired by previously introduced architectures (StyleNeRF and DIGAN), our main focus is on 3D-aware video generation in order to enable spatial and temporal control. To the best of our knowledge, this 4D GAN task has not been investigated before, and thus, our work is novel, which is also acknowledged by all reviewers: “first work to generate 4D videos with GAN” (Reviewer dFag), “This work is original and unique” (Reviewer jBEi), “first solution to this problem” (Reviewer PyKL), “first paper to generate a 3D-aware video” (Reviewer rreh). Moreover, we believe that our design choices for modeling 4D implicit fields and enabling motion / content decomposition, our technique for combining images and videos during training, and our extensive analysis and experiments would facilitate future research in this new direction.

> **Q2: Evaluation of pre-training and simultaneous training (Reviewers dFag and PyKL)**

We conducted a more extensive comparison between i) pre-training on FFHQ before training on FaceForensics, ii) simultaneously training on FFHQ and FaceForensics, and iii) only training on FaceForensics. We summarized the  new quantitative and qualitative results in Sec. E in the appendix of our draft. First, we want to clarify that to ensure fair comparison to previous works, in our main submission,  we only used our vanilla approach without FFHQ for both qualitative and quantitative comparisons. The main idea behind using FFHQ is to compensate for the narrow camera pose range of FaceForensics, leading to flat renderings especially in the pitch angle variations, as already discussed in Section Training on Image and Video Datasets (page 5 of our main submission). For the case of simultaneous image and video training, the resulting model fails to output view-consistent videos as shown in Fig. 11 and via the following anonymous link: https://imgur.com/a/Jb1MMQN. We hypothesize that training the generators to fit two different data distributions does not lead to consistent 4D models. However, only pre-training on FFHQ leads to view-consistent renderings while maintaining high quality as shown in Fig. 12 and via the following anonymous link: https://imgur.com/a/NWnPN2h 
 
Furthermore, we provide quantitative comparisons between all three training schemes. Generally, it is difficult to interpret FVD and FID when using two training datasets as these scores compare real and fake data distributions. Due to the usage of the FFHQ dataset the latent space of the model is manipulated with a dataset that is not used as part of the evaluation process. We still provide these two scores for completeness. For our pre-trained model, we observe that the FVD gap is significantly higher than the FID gap in comparison to our vanilla approach. We hypothesize that due to the model converging to a static dataset in the first training phase, namely FFHQ, the model has difficulties in learning motion in the second training phase with FaceForensics. However, we observe a significantly improved ID, which confirms the better multi-view consistency in our videos. Simultaneous training results in similar scores as our vanilla approach, however the multi-view consistency is significantly worse than using pre-training, which aligns with the lower ID score.

| Method |    FVD     |     FID    |       ACD    |  CPBD    |       ID    | 
|--------|:--------:|:--------:|:--------:|:--------:|:--------:|
|**Pre-trained**          | 127.2  | 17.2  | 0.819  | 0.2084  | 0.982 |
|**Simultaneously**          | 62.1  | 18.2  | 0.945  | 0.2077  | 0.893 |
|**Vanilla**          |  68.7  | 13.7  | 0.965  | 0.196  | 0.861 |

From: ICLR.cc/2023/Conference/Paper1419/Reviewer_dFag
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper proposes a 3D-Aware Video Generation pipeline. It integrates a GAN model and a time-conditioned implicit field to enable the pipeline to generate 4D videos. The proposed method encodes each video into a 3D content code and a motion code. These two codes, together with the camera pose, determine the video content. In discriminator, the proposed pipeline separately evaluates the quality of each frame and the rationality of the co-occurrence of two given frames in the same video.  
strength_and_weaknesses: Strengths:

1. The proposed architecture successfully combines GAN and Implicit field and is the first work to generate 4D videos with GAN.

2. The proposed method outperforms existing works remarkably in terms of ACD and ID. It suggests that this paper significantly improves the temporal consistency, which can also be concluded from qualitative comparison.

3. The paper is clearly written and easy to follow.

Weaknesses:

1. Some tricks are interesting, but the related experiments are insufficient.

    - Image pre-training seems to contribute a lot to the performance. However, there are few ablation studies about this option. In Supplementary Section E, the authors only provide two groups of figures to compare pre-training and joint training. Authors should provide quantitative results about it.

    - Authors claim that one of the key components of the proposed framework is an image-based discriminator. Can we use a deterioration of video-based discriminator (e.g., let $t_k=t_l$) to replace the image discriminator?

    - More qualitative results about the motion generator should be provided. 

2. In Table 1, StyleGAN-V outperforms the proposed method in aspects of FVD and FID. Does it mean that the implicit field causes many difficulties for the model to fit the original distribution?

3. Authors should clarify more about the differences between DIGAN and the proposed method. In my opinion, the only significant difference is that DIGAN uses INR directly, while this paper exploits Nerf to equip the model with more 3D information. In the architecture design, the split of FG&BG and the motion generator are the new components compared to DIGAN. However, the former component has been applied in the related works \[1-3\]. It is hard to identify the contribution of this paper in its current form.


\[1\] FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modelling

\[2\] LOLNeRF: Learn from One Look

\[3\] NeuralDiff: Segmenting 3D objects that move in egocentric videos
clarity,_quality,_novelty_and_reproducibility: Clarity: well, easy to read and understand

Quality: well

Novelty: not very

Reproducibility: looks OK
summary_of_the_review: An interesting study with some novel designs and extensive experiments. But the novelty should be further justified.

correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper1419/Reviewer_jBEi
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: This works presents approach to multi-view-consistent video generation, whereby the camera viewpoint in a generated video can be adjusted (or the same video can be generated from multiple camera viewpoints/perspectives). Furthermore, it is able to achieve this using 2D image/video streams. By using time-conditioned implicit fields, this work extends methods based on neural implicit respesentation to the video domain, crucially without the requirement for explicit 3D data/supervision. They classify their model as a 4D generative model, i.e. a 3-D aware video generator.
strength_and_weaknesses: **Strengths**:

- Clear and well written paper, with a well motivated approach to incorporating 3D structure in video generation models and tasks.
- Great treatise of the capabilities and limitations of the proposed approach.
- Deep consideration of the ethical challenges and threats of this research (and a strong rejection and condemnation of misuse of such work)
- Exhaustive consideration of prior and related works.
- The method appears to be computationally efficient and quite performant when compared to prior art (e.g. Table 1.)
- Extends the use of INRs to 3D video generation using only 2D image/video data. Whereby the viewpoint in the generated video is controllable parameter.

**Weaknesses**:

- The proposed approach is limited to simple datasets with easy foreground/background separation and limited objects in the foreground. This weakness is acknowledged in the paper and is in general a challenge for the broader field.
- The approach is limited to generating 16 frames of videos. Again, this weakness is acknowledged in the paper and is in general a challenge for the broader field.
- Code to reproduce the main experiments is missing.
clarity,_quality,_novelty_and_reproducibility: - **Quality**: This is a high quality article detailing a concrete and unique contribution to the video generation literation.
- **Clarity**: This article is well articulated and clear. It exhuastively covers the proposed approach in a manner that makes it straightforward to understand and replicate the contribution.
- **Originality**: This work is original and unique. It extends the capabilities of current approaches to facilitate a new capability not encountered before in this line of research.
summary_of_the_review: Overall, this is a great and significant contribution to the video generation literature. It is well-polished and provides for a concrete and unique contribution to the field. I support this papers publication to ICLR. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
flag_for_ethics_review: NO.
recommendation: 8: accept, good paper

From: ICLR.cc/2023/Conference/Paper1419/Reviewer_PyKL
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper presents a system to generate multi-view consistent videos. We have only seen 3D GANs on image domain so far and this paper extends it to video domain. There are two core components: a time-conditioned 4D generator and a time-aware video discriminator. Experiments are conducted on 3 datasets ranging from talking faces to moving people. The results are promising and encouraging.
strength_and_weaknesses: **Strength**

- Extending the 3D-aware GAN to video domain is an interesting direction and this paper proposes a legitimate first solution to this problem.
- The proposed two modules time-conditioned 4D generator and time-aware video discriminator make a lot of sense and work well in practice.
- The results are encouraging and the generated videos are view-consistent. 

**Weakness**

- Claiming the time-aware video discriminator to be one of two core contributions seems inappropriate. This module is introduced in DIGAN (Yu et al., 2022) and no further changes are made in this work. The original module is also introduced for video GAN.
- It's a bit strange to me that in Eq.1 the time is used as a multiplier. What's the motivation behind this? When time step is 0, this will make entire feature to be 0-vector. Are the other options such as concatenation or positional encoding tested?
- The writing is messy and inconsistent. For example,  Eq.1 use $\psi^3\circ\psi^2\circ\psi^1$ for MLPs, the mapping network is denoted as $w=MLP(z)$, and Eq.3 use $\phi_{\sigma}$ to denote 2-layer MLP. Another example, there is an extra comma in eq.4 after $\gamma(d)$. Another example, $n$ is defined as a vector above Eq.1 but it's used as a function in eq.1.  These are just few examples. Overall, there is lots of room for writing improvement.
- In the approach it's mentioned there are two training options (section "Training on Image and Video Datasets"). Which performs better? I don't find further discussion or ablation study in sec. 4.3. 
- The visual quality isn't very satisfying. I can see artifacts like extra arms Fig.3. Also, these results are of low-resolution and the local fine details are not recovered well. The quality of the generated faces also falls far behind the image 3D GAN. 
- Even though the evaluation metrics are not perfect for evaluating 3D GANs, the quantitative results are just competitive to 2D methods and sometimes worse (Tab.1).
- Only one 3D static baseline (StyleNeRF) is tested. It's not enough and also StyleNeRF itself isn't the state-of-the-art method.
- No geometry is visualized or shown in the paper. It's important to demonstrate the learned 3D for 3D GANs in my opinion. Does the network learn reasonable structure from the input data?
- I don't understand why Tab.2 and Tab.3 are so incomplete compared to Tab.1. 
clarity,_quality,_novelty_and_reproducibility: **Clarity**

As pointed out in the above section, some of the writing isn't very clear. There remains lots of improvement space especially for the approach section.

**Novelty**

One of the claimed contribution (time-aware video discriminator) isn't novel. For the other contribution (time-conditioned 4D generator), I have some doubts about the way of time conditioning as stated above.

**Quality**

The visual quality isn't very satisfying and I think the experiments are incomplete and needs some further efforts. 

**Reproducibility**

Implementation details are provided in Appendix but no code is submitted. 
summary_of_the_review: I have concerns in multiple aspects of this work including writing, experiments, and technical contribution. It's unlikely that these weaknesses can be properly addressed during rebuttal period. Therefore, I recommend rejection. However, this paper explores an interesting yet challenging direction and I do encourage the authors keeping improving the quality of this work.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
details_of_ethics_concerns: I don't have any ethics concerns. I didn't check the research integrity issues (e.g., plagiarism, dual submission) though.

recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper1419/Reviewer_rreh
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The authors propose 4D generative adversarial networks (GANs) to learn the unconditional generation of 3D-aware videos in neural implicit representations. The experimental results show that the proposed method learns a rich embedding of decomposable 3D structures and motions whose quality is comparable to that of existing 3D or video GANs.
strength_and_weaknesses: **Strengths**

- The paper is clearly written.
- It is the first paper to generate a 3D-aware video in neural implicit representations.

**Weaknesses*

- The technical contribution should be clearly demonstrated at the end of the Introduction.
- According to the results in the supplementary material, the generated videos are quite short and the background images are not consistent among frames.
- The proposed method should be compared to the NeRF-based methods for dynamic scenes, such as D-NeRF.
clarity,_quality,_novelty_and_reproducibility: - The paper is clearly written.
- The contributions are only marginally significant or novel.
- The authors have not provided the code and enough details for reproduction yet.
summary_of_the_review: Please refer to the strengths and weaknesses.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: Not applicable
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold