Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes a method of discretizing continuous nodes in a neural network, turning them into several discrete tokens. The method, DVNC, has theoretical and practical generalization improvements.
main_review: _Originality_: The ideas proposed in this paper are novel and are centred on a generalization of the discretization node of VQ-VAE. The inspiration of VQ-VAE is clear in the paper, but oftentimes the differences are not made very clear in this work (some suggestions on _Clarity_ section).

_Quality_: The method proposed is technically sound. The authors provide a thorough theoretical analysis of the improved noise robustness of DVNC when compared to a continuous representation. The experimental procedure supports these theoretical claims. Experiments are performed in three different architectures/tasks and the results improve significantly when compared to the baseline.

_Clarity_: The paper is well-written. As briefly mentioned above, I think the authors should have used some space to better distinguish their method and VQ-VAE at the beginning of §2. A background section on that architecture would benefit the clarity of the paper.

_Significance_: The analysis and method proposed in this paper are relevant to the community and this is a line of research that warrants more future work.

__Comments__

- I think the use of $m$ as the embedding size is not consistent. It is defined as the embedding size of the discrete latent space, but then the segments $s_i$ are of size $\frac{m}{G}$ but it seems they should be of size $m$, given Eq. 1.

- In Figure 3 (right) it seems like 3 lines are missing

- Typos:

    - Line 64, 74: analyses --> analysis
    - Line 94: "selected" is repeated
    - Line 98: decent --> descent
    - Lines 194, 197, 201, 207, 210, 215, 274, 304: no space between period and next sentence, or comma has extra space/needs a space
    - Line 198: that a learned --> that are learned
    - Line 263: position --> positions
limitations_and_societal_impact: The authors have adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 2: Strong rejection
review_content: summary: The submission discusses discrete communication between neural network components and proposes 'discrete-valued neural communication (DVNC)', i.e. discretization of continuous representations in neural networks. The key contribution is a segmented discretization of vectors (see Eq. 1), in contrast to a joint discretization in a prior work (Oord et al., 2017).
main_review: The submission addresses an important research questions, namely how different neural submodules can communicate effectively. This is relevant since future neural networks may have several specialized components that need to interact in an efficient and effective way. However, the submission does not discuss clearly related prior work. Furthermore, is remains unclear what the paper actually claims to contribute. Hence, I believe that the submission cannot be accepted to this conference in its current form.

More details:

Missing/vague contribution:
One of the main reasons for my reject recommendation is the vagueness of the submission. The title 'Discrete-Valued Neural Communication' and the abstract/introduction suggest that this paper proposes discrete-valued neural communication for the first time. However, this is clearly not the case. There are several works that have been working on this area (see details below). Hence, after reading the first parts of the submission, it is unclear what its actual contribution is. The actual contribution becomes more clear in section 2, and specifically in Equation 1, which is very similar to Equation 3 in the VQ-VAE paper (Oord et al., 2017). The key difference is the segmentation and the separate discretization of h. However, this does not become clear at all in the introduction of the submission. I would like to recommend to clearly state early on what the actual contribution of the paper is instead of vaguely talking about discrete communication.

Missing related work:
Discrete-valued neural communication has already been addressed by several lines of work. Unfortunately, non of these areas is discussed. Examples are discrete communication between different agents in reinforcement learning (e.g. [1]), networks with discrete-continuous computation graphs (e.g. [2]), and memory networks with discrete states (e.g. [3]). Hence, I would like to recommend to discuss these works and how the current submission differs.

[1] Jakob N. Foerster et al. Learning to Communicate with Deep Multi-Agent Reinforcement Learning. In: CoRR abs/1605.06676 (2016). arXiv: 1605.06676. url: http://arxiv.org/abs/1605.06676.
[2] Schulman, J., Heess, N., Weber, T., & Abbeel, P. (2015). Gradient Estimation Using Stochastic Computation Graphs. Advances in Neural Information Processing Systems, 3528–3536.
[3] Wang, C., & Niepert, M. (2019). State-Regularized Recurrent Neural Networks. Proceedings of the 36th International Conference on Machine Learning, 6596–6606. http://proceedings.mlr.press/v97/wang19j.html

Minor important comments/suggestions:
- The submission states that 'graph neural networks are composed of distinct nodes'. I think this description is rather confusing. A GNN is not composed of distinct nodes. It operates over nodes as underlying structure. Similarly, an RNN for NLP is not composed of tokens, but operates on a sequence of tokens. Similarly, transformers are not composed of positional elements.

- The submission argues that discrete encoding have an evolutionary advantage since different areas of the human brain are tuned to discrete variables. However, making strong connections between human brains and neural networks is far fetched. It remains an hypothesis if computers may benefit from similarities with the human brain.

- There are several typos and missing spaces in the 'Communication among Specialists' section.

- The text in Figure 4 should be larger. It is impossible to read it without zooming in.
limitations_and_societal_impact: n/a
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a method of discretizing communication between different modules in a network of specialists in order to improve generalization. They propose using a common codebook of discrete symbols shared by all network components that can be used to share information between modules. They propose a method of discretization based on the VQVAE architecture called Discrete-Valued Neural Communication (DVNC), in which a continuous vector is partitioned into "discretization heads", each of which is then mapped to its nearest neighbour among a set of learned latent vectors that are shared between modules. Gradients are propagated through the discretization using a straight-through estimator.
The authors provide theoretical analysis to argue that DVNC improves sample efficiency and robustness to noise, while decreasing intrinsic dimensionality. They also provide experiments with a number of different tasks and architectures to show that their method of discretization improves performance on out-of-distribution samples at inference time.
main_review: This paper’s analysis of the benefits of a discrete latent space for model generalization is novel and valuable, and highlights an interesting avenue for future research. The mathematics and theoretical results presented are thorough and rigorous, and provide a very interesting look at the benefits of discretization for generalization in general terms. The authors provide evidence for their claims through a variety of experimental settings, backed up by theoretical analysis and examples using Gaussian-distributed vectors. The work is clearly novel and provides a new and relevant framework for analysis that will hopefully be built upon by future research.

My main criticism of the paper would be that I think it needs some additional work in editing and improving the clarity of some sections, particularly the theoretical analysis in Section 2. From reading the paper it is not entirely clear to me how the results stated in Theorems 1 & 2  generalize to the claims about noise robustness and intrinsic dimensionality made in the text. First of all, in looking at the theorems, my impression is that they are discussing only functions restricted to act on a small specific subdomain of the input space, and I don't see from the content of the text or the theorems any justification for the more general claims you make. From your response I gather this does generalize, but I would make the generalization more clear. Second, I am unclear on what the paper means by 'noise robustness'. As far as I can tell, there is no explicit noise model mentioned in the paper or the appendices. The theorems rather appear to be analyzing the error of approximating the output of functions by Monte Carlo sampling. Finally, the paper’s discussion of ‘intrinsic dimensionality’ could benefit from some additional detail and discussion, as the definition or significance of this was never made clear in the paper (is this a reference to the Minkowski dimension, or maybe the packing dimension proposed by Kegl?). It would be helpful to have a more concrete discussion of what the intended meaning of these terms is in the context of the two theorems.

I also think that the experiments may be overly limited. While the paper shows experiments on a number of tasks and architectures, many seem to be toy tasks rather than more realistic applications. In addition, aside from Figure 4, the reported results (e.g. Table 2, Table 3, Figure 5) show only the effect of the method on the out of distribution performance, and do not show the effect of the method on in-distribution performance, which I think would be beneficial to include for completeness’s sake.

There are also some clear editing mistakes in the paper that need to be fixed (e.g. ‘selected’ repeated on page 3, appendix B missing, etc…). Many of the diagrams are also far too small, making them very difficult to read – particularly figure 4. 

Overall I think that this paper is a worthwhile and relevant work, as long as the issues with clarity and editing are fixed.

limitations_and_societal_impact: As this is a purely technical and theoretical work, the direct negative social impacts are not a major concern. The authors do address the limitations of their work, though I think it would be beneficial to more thoroughly address the concerns that discretization could harm expressivity or reduce in-distribution performance (perhaps with more robust experiments to show that this is empirically not the case).
review_title: null

Meta-review: This paper discusses neural network architectures that exchange discrete messages, trained using a strategy similar to the successful VQ-VAE. The paper includes both theoretical results about such architectures, as well as encouraging empirical results.

The largest concerns have to do with the discussion of the scope, as the setup discussed here has large overlap with existing but different literature. (Reviewers point out examples from reinforcement learning or on RNN hidden state discretization.) Overall, some comparison to those strategies seems possible and would improve the work, but, at the same time, the work here seems like a good step. At the very least, the connections to this others applications of discrete variables should be discussed as clearly as possible and in a positive way. (Connections here are likely to be productive.)

The paper could be strengthened with results on more realistic applications, possibly where the discrete messages can be nicely interpretable. 

The authors have been very active in the discussion and have provided many clarification as well as promises for improvements of the presentation, including agreeing to a change of title that seems like it would improve clarity.
Decision: Accept (Poster)