Paper: Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments (ICLR-2024)

Reviewer comment (Novelty):
The authors do a good job noting related prior literature, but the reviewer remains somewhat confused about the position of this paper relative to such literature. If prior art has used the same formulation and considered a generalization of the problem this paper is considering, the specific contributions of this paper are unclear. The reviewer suspects there are many unique contributions, but the contrast relative to prior art should be made much more obvious, for example by explicitly stating differences at the end of each related works section and ideally by running experiments comparing to Resnick's method.

Author response:
Thank you very much for your detailed and thoughtful reviews.
We have extensively revised our paper in light of the feedback received.
These revisions aim to clarify the paper's positioning and enhance the readability of our experimental results.

## Reviewer's Comment 1
> The authors do a good job noting related prior literature, but I remain somewhat confused by the position of this paper relative to such literature. ...

Thank you for point it out.
We added the positioning of our paper, compared to the related studies Resnick et al. (2020); Chaabouni et al. (2021); Tucker et al. (2022).

Perhaps another problem was the beginning of Sec 3: `The main proposal of this paper is redefine the objective of signaling game as (beta-)VAE's`.
We modified the beginning of Sec 3 too.

Moreover, we made the table to clarify our positioning, which is put to Appendix F in the revised version.

## Revision for Comment 1
At Paragraph "EC as Representation Learning" in Sec 6 "Related Work", we added:
> (***Revision***) Resnick et al. (2020) formulated the objective as ELBO. However, they defined messages as fixed-length binaries and agents as non-autoregressive models, which is not applicable to our variable-length setting. Moreover, they do not discuss the prior distribution choice; they seem to have adopted Bernoulli purely for computational convenience. In contrast, we indicated that their choice influences the structure of emergent languages.

At Pragraph "VIB" in Sec 6 "Related Work", we added:
> (***Revision***) Note that messages are defined as single symbols in their settings. In contrast, messages are of variable length in our setting to discuss the structure of emergent languages such as ZLA and HAS.

In Sec 3, we modified the first few sentences:
> (***Revision***) ~~The main proposal of this paper is to redefine the objective of signaling games as (beta-)VAE's, i.e., the evidence lower bound (ELBO) with an additional KL weighting hyperparameter $\beta$.~~
> In this section, we first redefine the objective of signaling games as (beta-)VAE's, i.e., the evidence lower bound (ELBO) with a KL weighting hyperparameter $\beta$. Next, we show several reasons supporting our ELBO-based formulation.

In Appendix F, we added the table to clarify our positioning.
The table shows the characteristics of the studies in terms of:
- whether their formulation is based on a generative perspective such VAE and VIB,
- whether message length in their settings is variable,
- whether they study the compositionality of emergent languages,
- whether they investigate if emergent languages follow ZLA,
- whether they investigate if emergent languages follow HAS, and
- whether they are aware of the existence of prior distributions and carefully choose an appropriate one.

Each related study stasfies some of them, while it does not satisfy the others.
In contrast, our paper satisfies all of them.

Reviewer comment (Presentation):
The results are somewhat difficult to read, especially Figure 2, which contains the main results but requires flipping between pages to interpret. Figures should be made more self-contained, and baselines should be labeled with descriptive names rather than generic identifiers like "BL1."

Author response:
Thank you for this feedback.

## Revision for Comment 2
To make Figure 2; 3; 4 more readable and self-contained, we added more information to the legend and included the explanation on the evaluation metrics and the baseline methods in the captions.
As a result, the caption (in Figure 2) became as follows:
> (***Revision***) Results for $n_{\textrm{bou}}$ (C1), $n_{\textrm{seg}}$ (C2), $\Delta_{w,c}$ (C3), C-TopSim (C3), and W-TopSim (C3) are shown in order from the left. The x-axis represents $(n_{\textrm{att}},n_{\textrm{val}})$ while the y-axis represents the values of each metric. The shaded regions and error bars represent the standard error of mean. The threshold parameter is set to $0$. The blue plots represent the results for our ELBO-based objective $\mathop{\mathcal{J}}\nolimits_{\textrm{ours}}$, the orange ones for (BL1) the conventional objective $\mathop{\mathcal{J}}\nolimits_{\textrm{conv}}$ plus the entropy regularizer, and the grey ones for (BL2) the ELBO-based objective whose prior is $P_{\alpha}^{\textrm{prior}}$.
> ...... (*For the rest part of the revised caption, see our response to Comment 6*)

Reviewer comment (Writing):
In the section discussing why segments become more meaningful, parts of the writing are too casual (e.g., "The receiver must be surprised several times..."), and the concept of "being surprised" is unclear as a binary term. The receiver simply needs to receive enough bits over multiple timesteps to reconstruct the input.

Author response:
Thank you for pointing that out.

## Revision for Comment 3
To make the discussion more mathematically grounded, we added some mathematical terms, after casually used "surprised", as follows:
> (***Revision***) a receiver has to be "surprised," i.e., to receive symbols $m_{t}$ with relatively high $-\log P_{\theta}(m_{t}|m_{1:t-1})$.

Reviewer comment (Theory):
The reviewer questions the fundamental conclusion that competing entropy and reconstruction terms produce word boundaries by making communication predictable in some cases, creating word-like clumps. It is unclear why information should be concentrated in a few timesteps instead of being evenly distributed across time. There is a need for mathematical justification for why, for example, [2 bits, 0 bits, 2 bits, 0 bits] would be preferable to [1 bit, 1 bit, 1 bit, 1 bit] in terms of loss reduction.

Author response:
Thank you for your insightful comment, which has provided valuable perspectives to our discussion.
Your feedback prompts us to think more deeply about the issue.

Currently we have the following hypothesis, though it's more like an intuition, rather than a mathematical basis:
- Agents themselves do not necessarily have any strong motivation to make bit pattern zigzag or flat.
- However, as a matter of probability, it is simply difficult to converge to a flat one among infinitely many possible bit patterns (almost all of them would zigzag).
- The bit patterns are optimized under the trade-off between reconstruction and surprisal, while they (almost surely) fail to find a flat pattern.

## Revision for Comment 4
To avoid overstatement, we slightly weakened our statement in discussion, as follows:
> (***Revision***) Though it is not straightforward to give a clear mathematical basis, we currently hypothesize that the improvement of segments' meaningfulness is due to the competing pressure of the reconstruction and the surprisal.

Also, to clarify the limitation of our discussion, we added a footnote to mention that our current hypothesis does not fully explain why the bit pattern does not converges to a flat one, as follows:
> (***Revision***) Our current hypothesis does not fully explain the reason why the branching entropy does not converge to a flat one instead of a jittering one. At the moment, we conjecture that, as a matter of probability, it is simply difficult to converge to a flat one among infinitely many possible bit patterns (almost all of them would jitter).

Reviewer comment (Presentation):
Appendix D would benefit from more explanatory text describing what the graphs present, and some sentences need editing (e.g., "threshold is set to 0.25").

Author response:
Thank you for this feedback.

## Revision for Comment 2
To make Figure 2; 3; 4 more readable and self-contained, we added more information to the legend and included the explanation on the evaluation metrics and the baseline methods in the captions.
As a result, the caption (in Figure 2) became as follows:
> (***Revision***) Results for $n_{\textrm{bou}}$ (C1), $n_{\textrm{seg}}$ (C2), $\Delta_{w,c}$ (C3), C-TopSim (C3), and W-TopSim (C3) are shown in order from the left. The x-axis represents $(n_{\textrm{att}},n_{\textrm{val}})$ while the y-axis represents the values of each metric. The shaded regions and error bars represent the standard error of mean. The threshold parameter is set to $0$. The blue plots represent the results for our ELBO-based objective $\mathop{\mathcal{J}}\nolimits_{\textrm{ours}}$, the orange ones for (BL1) the conventional objective $\mathop{\mathcal{J}}\nolimits_{\textrm{conv}}$ plus the entropy regularizer, and the grey ones for (BL2) the ELBO-based objective whose prior is $P_{\alpha}^{\textrm{prior}}$.
> ...... (*For the rest part of the revised caption, see our response to Comment 6*)

Reviewer comment (Theory):
The reviewer asks whether there is any mathematical reason that the training losses should concentrate surprisal in just a few timesteps, which would create word-like clumps, rather than distributing surprisal more evenly across time.

Author response:
Thank you for your insightful comment, which has provided valuable perspectives to our discussion.
Your feedback prompts us to think more deeply about the issue.

Currently we have the following hypothesis, though it's more like an intuition, rather than a mathematical basis:
- Agents themselves do not necessarily have any strong motivation to make bit pattern zigzag or flat.
- However, as a matter of probability, it is simply difficult to converge to a flat one among infinitely many possible bit patterns (almost all of them would zigzag).
- The bit patterns are optimized under the trade-off between reconstruction and surprisal, while they (almost surely) fail to find a flat pattern.

## Revision for Comment 4
To avoid overstatement, we slightly weakened our statement in discussion, as follows:
> (***Revision***) Though it is not straightforward to give a clear mathematical basis, we currently hypothesize that the improvement of segments' meaningfulness is due to the competing pressure of the reconstruction and the surprisal.

Also, to clarify the limitation of our discussion, we added a footnote to mention that our current hypothesis does not fully explain why the bit pattern does not converges to a flat one, as follows:
> (***Revision***) Our current hypothesis does not fully explain the reason why the branching entropy does not converge to a flat one instead of a jittering one. At the moment, we conjecture that, as a matter of probability, it is simply difficult to converge to a flat one among infinitely many possible bit patterns (almost all of them would jitter).

Reviewer comment (Presentation):
The reviewer struggled to understand Figure 5 and requests clarification on what it depicts and what the legend entries and different lines correspond to.

Author response:
We appreciate your feedback regarding the lack of legend descriptions in Figure 5.

## Revision for Comment  5
To make them understandable, we explictly describe what the legends mean, e.g., "Alphabet size $|\mathcal{A}|=5$".

Reviewer comment (Evaluation):
The reviewer seeks clarification on the results for Criterion C3, specifically whether the proposed method is worse than the baselines for this metric, noting that while topsim values generally improved, the difference between topsim values appeared to decrease.

Author response:
Thank you for another very insightful comment.

We understand your concern, but we would like to claim that it is safer not to explicitly state that it is better/worse than baselines.

This is because the following interpretation is also possible: "Relatively large (resp. small) Score X minus relatively large (resp. small) Score Y would simply yield a relatively large (resp. small) difference (X=W-TopSim and Y=C-TopSim in our case).
In other words,  it may simply be a matter of scale.

In our opinion, what we can take from the result of $\Delta_{w,c}$ is only whether it is positive (W-TopSim>C-TopSim), negative (W-TopSim<C-TopSim), or zero (W-TopSim=C-TopSim).

## Revision for Comment 6
Nevertheless, future readers may have the same impression.
For clarity, we briefly mention that point in the captions of Figure 2; 3; 4, as follows:
> (***Revision***) ...... The apparent inferior performance of $\Delta_{w,c}$ for $\mathop{\mathcal{J}}\nolimits_{\textrm{ours}}$ compared to the baselines might be misleading. It is because $\mathop{\mathcal{J}}\nolimits_{\textrm{ours}}$ greatly improves both C-TomSim and W-TopSim. The larger scale of their improvements could result in a seemingly worse $\Delta_{w,c}$, but this does not necessarily indicate poorer performance.

Reviewer comment (Theory):
(major) The paper, I think, tried to do too much, and ends up not spending enough time on the core claims, namely, the signalling game can be re-analyzed as a VAE. I think the paper would benefit greatly from cutting away all but the essential claims and going through those more slowly and thoroughly.

Author response:
We value your suggestion to focus more on the core claims of our paper.
In the revised version, we have taken the reviewers' comments into consideration and made efforts to improve the quality of the paper.
We have removed unnecessary text and moved some parts to the appendix to focus more on the essential aspects.
We hope you will find the revised version more satisfactory.

Roughly speaking, our core claims are made of two components, which are (somewhat philosophically) of mutual dependence.

Our core claims are:
1. The conventional EC signaling game is analogous to (beta-)VAE.
2. By explicitly re-defining EC signaling game as (beta-)VAE, the structure of emergent languages improves.

Thanks to the 2nd claim,
- the 1st claim becomes more than a mere theoretical result.

Thanks to the 1st claim,
- the 2nd claim becomes more than an arbitrary "hack" of the evaluation criteria (The arbitrary hack might be a questionable practice as a simulation of language emergence, in a scientific sense).

For a more detailed decomposition of our claims, please refer to our response to Comment 4.

Reviewer comment (Theory):
What exactly is the "prior" in the emergent language game? I understand that it is implicit, but does that mean that is embedded in the objective function (i.e., the $D_\text{KL}$ term is constant)? Or is it instead the case that the sender's architectural biases represent the prior?

Author response:
We appreciate your question regarding what exactly the prior is.
This is an important point to clarify our storytelling.

In terms of the conventional EC signaling game, yes, the prior is constant.
Specifically, it is a uniform distribution over a message space.

(In contrast, we regard it as a language model)
(See also our response to Comment 9; 10; 11; 12).

Reviewer comment (Experiments):
In addition to the theoretical analysis, what else can the authors point to to support the claim that an EC signalling game is analogous to a VAE?

Author response:
Thank you for questioning that point.
Our contribution is indeed more than the theoretical analysis of the game, including e.g.,
- a new idea for EC setting (i.e., prior as a language model),
- empirical results (for ZLA and HAS), and
- relationship to the linguistic concept (i.e., surprisal theory).

Our overall claims are as follows:

First, as you mentioned,
- ***As an interesting math problem***, EC signaling game is similar to (beta-)VAE (Sec 3.1; 3.2).

Importantly,
- By interpreting EC signaling game as (beta-)VAE, ***we can reveal the existence of implicit prior*** (Sec 3.1).

Furthermore, by having revealed the existence of implicit prior,
- We can appropriately re-define the prior distribution to have a ***good influence on the structure of emergent languages***, namely ZLA/HAS (Sec 3.3; 3.4; Sec 4).
- We can naturally introduce a ***language model (=prior distribution) explicitly in the simulation of language emergence*** (Sec 3.4).

Moreover, by regarding the prior as a language model,
- We'll be somewhat ***satisfied in a scientific sense*** to notice that ***the prior coincides with surprisal, a concept from psycho-linguistics*** (Sec 3.5).

They are also shown in "Recipe for Supporting Our ELBO-based Formulations" in Sec 3.

Reviewer comment (Presentation):
If the authors are assuming a REINFORCE objective for the signalling game, that should be mentioned earlier than Sec 4.1.

Author response:
Thank you for pointing it out.
We agree with your point.

## Revision for Comment 5
In Sec 2, we added the following explanation on the gradient of the (conventional) objective function, which naturally indicates we consider REINFORCE-like optimization in this paper.
> (***Revision***) The gradient of $\mathop{\mathcal{J}}\nolimits_{\textrm{conv}}(\phi,\theta)$ is obtained as (Chaabouni et al. 2019): $$\nabla_{\phi,\theta}\mathop{\mathcal{J}}\nolimits_{\textrm{conv}}(\phi,\theta)=\mathop{\mathbb{E}}\nolimits_{x\sim P_{\textrm{obj}}(\cdot),m\sim S_{\phi}(\cdot|x)}\left[\nabla_{\phi,\theta}\log R_{\theta}(x|m)+(\log R_{\theta}(x|m)-B(x))\nabla_{\phi,\theta}\log S_{\phi}(m|x)\right],$$ where $B:\mathcal{X}\to\mathbb{R}$ is a baseline. Intuitively, $\phi$ is updated via REINFORCE (Williams, 1992) and $\theta$ via the standard backpropagation.

Reviewer comment (Writing):
What is $A_t$ in Eq 3?

Author response:
We meant by $A_{t}$ a variable of a finite alphabet $\mathcal{A}$.

## Revision for Comment 6
To make it more direct and natural, we rewrote $A_{t}$ to $M_{t}$ ($t$-th message symbol variable).
> (***Revision***) $$\nabla_{\phi,\theta}^{\textrm{entreg}}:=\frac{\lambda^{\text{entreg}}}{|m|}\sum\nolimits_{t=1}^{|m|}\nabla_{\phi,\theta}\mathcal{H}(S_{\phi}(M_{t}|x,m_{1:t-1})),\tag{3}$$

Reviewer comment (Presentation):
In Sec 2.2, I do not think the section compositionality is relevant or important; it should be removed, in that case.

Author response:
Thank you for the feedback.
We agree with your point.

## Revision for Comment 7
We moved the section "Compositionality" to the footnote.

Reviewer comment (Writing):
Before Eq 5, what is $\mathcal A^*$? Is it supposed to be a Kleene star?

Author response:
Yes, we meant by $\mathcal{A}^{*}$ a Kleene star.

## Revision for Comment 8
To make it more intuitive, we use a more direct notation instead of the Kleene star.
> (***Revision***) $s=a_{1}\cdots a_{n}$ ($a_{i}\in\mathcal{A}$)

Reviewer comment (Theory):
What exactly is the uniform prior? Is it just a constant probability mass over every possible sequence? If so, how do we know that is the "implicit prior" and not something like a uniform unigram prior instead, for example.

Author response:
Yes, the former is the case.
The implicit prior $P_{\textrm{unif}}^{\textrm{prior}}$ is a uniform distribution over a finite message space (constant w.r.t messages $m$).
$P_{\textrm{unif}}^{\textrm{prior}}(m)$ is not a unigram (see the example below).

**Example:**
Let a finite alphabet be $\mathcal{A}:=\\{\texttt{eos},0,1\\}$ and a message space be $\mathcal{M}$.
Consider two messages: $m^{(1)}=0101\texttt{eos}$ and $m^{(2)}=01010101\texttt{eos}$.
If we consider a (uniform) unigram model, then $P(m^{(1)})=(\frac{1}{3})^{5}$ and $P(m^{(2)})=(\frac{1}{3})^{9}$, which are not the same.
In contrast, if we consider literary a uniform distribution, then $P(m^{(1)})=P(m^{(2)})=\frac{1}{|\mathcal{M}|}$.

## Revision for Comment 9
To make it more direct and clear, we rewote $P_{\textrm{unif}}^{\textrm{prior}}(m)\propto\textrm{const}$ to $P_{\textrm{unif}}^{\textrm{prior}}(m)=\frac{1}{|\mathcal{M}|}$.
> (***Revision***) $P_{\textrm{unif}}^{\textrm{prior}}(m)=\frac{1}{|\mathcal{M}|}$

Reviewer comment (Theory):
Sec 3.1: It is not clear to me how (9) is derived from (2). I looked at Section B.1, but it was very unclear what was happening because rather than starting with (2) and going to (9), it talks about "transforming" different sides of the equation.

Author response:
As you point out, our proof is unclear.
We made the following revision.
We hope that it contributes to the interpretability and readability of our proof.

## Revision for Comment 10
- To clarify the proof, we briefly recall the definition of the target equation and relevant terms.
- To clarify the proof, we briefly explain what we will transform.

Reviewer comment (Theory):
As a result, I'm not convinced that the reconstruction game, absent modifications to the traditional object (e.g., length penalty), assumes a uniform prior of messages.

Author response:
Thank you for pointing it out.

Apart from the standard VAE, we adopted the REINFORCE-like optimization but the "reward" (reconstruction term) is differentiable (w.r.t the receiver $\theta$).
The differentiability of the "reward" hinders the direct application of REINFORCE.
That is why, instead of Williams (1992), we mentioned Schulman et al. (2015), who proposed some sort of the generalization of REINFORCE.
As you pointed out, our proof would become clearer by explicitly explaining how to utilize the result of Schulman et al. (2015).

## Revision for Comment 11
To clarify the proof, we added two lemmas (Lemma 1; 2) in Appendix B.1 to support the proof description (The lemmas can be seen as a special case of Schulman (2015) and/or REINFORCE).

Reviewer comment (Writing):
It seems like the $P(m) = \mathbb E_{x\sim{}P_\text{obj}}[S(m|x)]$ should be the prior over messages. I very well might be misunderstanding something here due to terminology. Am I conflating here that "prior" as the distribution of messages the receiver produces given the distribution over inputs with "prior" in the sense of our objective function which we are optimizing against (in which case "prior" does not refer to anything concrete in the EC environment but rather only to the optimization process by analogy to a VAE's optimization)?

Author response:
We understand your concern, but this is not the case.
We hope that the above revision made our proof persuasive enough to regard the uniform distribution as the prior.

Let us explain why the average sender is not the prior in the conventional EC signaling game, using some equations.
First, let us decompose Eq. 9 as follows:

$$
\begin{aligned}
\nabla_{\phi,\theta}\mathop{\underbrace{\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log R_{\theta}(x|m)]}}\limits_{\text{conventional objective}}
&\mathop{\underbrace{=}}\limits_{\text{proof target}}
\nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log R_{\theta}(x|m)+\log P_{\textrm{unif}}^{\textrm{prior}}(m)]
\\\\
&=
\mathop{\underbrace{\nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log R_{\theta}(x|m)]}}\limits_{\text{same as the left-hand side.}}
+\mathop{\underbrace{\nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log P_{\textrm{unif}}^{\textrm{prior}}(m)].}}\limits_{\text{should be proven $=0.$}}
\end{aligned}
$$

Based on this decomposition, what we have to prove essentially is: $\nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log P_{\textrm{unif}}^{\textrm{prior}}(m)]=0$.
According to Lemma 1 (which is added in the revised version), it can be further transformed as follows:

$$
\begin{aligned}
\nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log P_{\textrm{unif}}^{\textrm{prior}}(m)]
&\mathop{\underbrace{=}}\limits_{\text{From Lemma 1.}}
\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\nabla_{\phi,\theta}\log P_{\textrm{unif}}^{\textrm{prior}}(m)+(\log P_{\textrm{unif}}^{\textrm{prior}}(m))\nabla_{\phi,\theta}\log S_{\phi}(m|x)].
\\\\
&=
\mathop{\underbrace{\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\nabla_{\phi,\theta}\log P_{\textrm{unif}}^{\textrm{prior}}(m)]}}\limits_{(\dagger)}
+\mathop{\underbrace{\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[(\log P_{\textrm{unif}}^{\textrm{prior}}(m))\nabla_{\phi,\theta}\log S_{\phi}(m|x)]}}\limits_{(\ddagger)}.
\end{aligned}
$$

Here,
- Term $(\dagger)$ would be $0$ if $\nabla_{\phi,\theta}\log P_{\textrm{unif}}^{\textrm{prior}}(m)=0$.
- Term $(\ddagger)$ would be $0$ if $\log P_{\textrm{unif}}^{\textrm{prior}}(m)$ is constant w.r.t $m$ (See Lemma 2, which is added in the revised version).

Thus, we should define $P_{\textrm{unif}}^{\textrm{prior}}(m)$ as a uniform distribution, i.e., $P_{\textrm{unif}}^{\textrm{prior}}(m)=\frac{1}{|\mathcal{M}|}$.

Now, let us assume a prior distribution to be $S_{\phi}(m):=\mathop{\mathbb{E}}\nolimits_{x\sim P_{\textrm{obj}}}[S_{\phi}(m|x)]$ as you suggested.
Then, what we have to check (prove/disprove) is whether $\nabla_{\phi,\theta}\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log S_{\phi}(m)]=0$.
But the gradient is not zero in general, since
$$\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x),S_{\phi}(m|x)}[\log S_{\phi}(m)]=-\mathop{\mathbb{E}}\nolimits_{S_{\phi}(m)}[-\log S_{\phi}(m)]=-\mathcal{H}(S_{\phi}(M)).$$

Reviewer comment (Theory):
Sec 3.4: The very first paragraph of this section, I think, is glossing over a critical question in the paper: what is the connection between the "prior" and the actual EC setup. I understand that the EC setup is analogous to a VAE, but what exactly is the analog of the VAE's prior?

Author response:
We appreciate your comment regarding what role the prior should play in EC setup.
For clafication, let us contrast the pov of previous work and the pov of ours.

The pov of previous work:
- In the first place, they are not aware of the implicit prior distribution.
- Thus, the prior distribution is just an unknown, unexpected artefact.

The pov of ours:
- In this paper, we consider the prior as the receiver's language model (LM) $P_{\theta}^{\textrm{prior}}(m)$.
- (Somewhat surprisingly, the idea of introducing LM into the EC signaling game has not been done before to the best of our knowledge, even though the idea per se seems quite natural.)

## Revision for Comment 14
We largely wrote the beginning of Sec 3.4 to clarify our position, as follows:
> (***Revision***) ... However, the signaling-game counterpart of the VAE's prior distribution has not been clear yet. In other words, as a sub-domain of linguistics, it is important to take a clear stance on which part of real-world communication it models. In this paper, we consider the prior distribution as the receiver's neural language model (LM) parametrized by $\theta$: ...

Also, to make our positioning clearer, we added the following phrase in Sec 1 (Introduction):
> (***Revision***) Interestingly, the prior distribution can be seen as a language model, and thus the corresponding term can be seen as...

Reviewer comment (Theory):
I think the use of "approximately" is dangerous when trying to make theoretical claims; I understand that it is unavoidable in something as messy as EC, but it still needs to be accompanied by some justification in order to keep the theoretical claims strong.

Author response:
We agree with your point.

## Revision for Comment 15
Just after Eq 15 in Sec 3.4, we added some more equations to imply a prior distribution "approximates" the average behavior of a sender, as follows:
> (***Revision***) ... It can be seen as an LM because it approximates the average behavior of a sender, i.e., the KL term in Eq 7, $\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x)}[D_{KL}(S_{\phi}(M|x)||P_{\theta}(M))]=-\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x)}[\mathcal{H}(S_{\phi}(M|x))]-\mathop{\mathbb{E}}\nolimits_{S_{\phi}(m)}[\log P_{\theta}(m)]$, is minimized w.r.t $\theta$ when $P_{\theta}(m)=S_{\phi}(m):=\mathop{\mathbb{E}}\nolimits_{P_{\textrm{obj}}(x)}[S_{\phi}(m|x)]$. ...

Reviewer comment (Writing):
"they are not" reproduced emergent lanuaguages has awkward phrasing.

Author response:
We rephrased it as follows:
> (***Revision***) It has been reported that emergent languages do not give rise to these properties ...

Reviewer comment (Theory):
Right after Eq. (1), it should be $\log(|\mathcal A| - 1)\ge0$ in the case that $|\mathcal A| = 2$.

Author response:
In fact we meant $|\mathcal{A}|\geq 3$  by `We assume $\mathcal{A}$ contains at least 2 symbols other than $\texttt{eos}$`.
But it was indeed an awkward phrasing.

## Revision for Comment 17
We rephrased it as a more direct one:
> (***Revision***) We assume $|\mathcal{A}|\geq 3$

Reviewer comment (Writing):
Sec 3.3: what is $\mathcal M$ -- the set of all messages?

Author response:
Yes, their notations are the same as Sec 2.

## Revision for Comment 18
To clarify common mathematical notation throughout this paper, we made `Mathematical Notation` section at the beginning of Sec 2.

Reviewer comment (Writing):
Sec 3.3: What is $\mathcal A$, again?

Author response:
Yes, their notations are the same as Sec 2.

## Revision for Comment 18
To clarify common mathematical notation throughout this paper, we made `Mathematical Notation` section at the beginning of Sec 2.

Reviewer comment (Writing):
Sec 3.3: Eq 15 limit notation here would be more appropriate.

Author response:
Thank you for pointing it out.

## Revision for Comment 19
- We wrote Eq 15, using limit notation.
- (But note that the corresponding paragraph moved to Appendix B.3)

Reviewer comment (Theory):
Based on the current formulation, it seems that the distractors on the receiver’s side are not considered. How would you incorporate the context of the distractors and their corresponding influences [1,2] into the prior design?  

[1] Lazaridou, Angeliki, Alexander Peysakhovich, and Marco Baroni. "Multi-agent cooperation and the emergence of (natural) language." arXiv preprint arXiv:1612.07182 (2016).  

[2] Evtimova, Katrina, et al. "Emergent communication in a multi-modal, multi-step referential game." arXiv preprint arXiv:1705.10369 (2017).

Author response:
This is an important point. Thank you for highlighting it.

Similar to Lewis's signaling (reconstruction) game, Lewis's discrimination (referential) game has also been a significant model in EC.
Due to the generative nature of our storytelling, we have focused on the reconstruction game in this study.
We plan to extend our study to include the discrimination game in our future work.

Reviewer comment (Writing):
The paper is hard to read. The theoretical section includes symbols and equations without full explanation.

Author response:
Thank you for pointing out the need for clearer explanations in the theoretical section.
Taking all the reviewers' comments into consideration, we have now added explanations for symbols and equations.

If you have any other specific problems or comments, please do not hesitate to mention them.

Reviewer comment (Experiments):
The experiments are weak. The compared methods lack descriptions, and the performance improvement is not well explained.

Author response:
We appreciate your feedback regarding the experimental section.

We believe we have adequately addressed the important aspects of the theoretical contribution through mathematical proof. Regarding word length and segmentation, we are confident that the experimental results we have obtained support the main claims of our contribution.

However, it is possible that we may have overlooked some points that the reviewer has noticed. If you believe there is a lack of experiments in terms of supporting specific hypotheses, we would greatly appreciate your feedback.

Reviewer comment (Novelty):
The studied problem lacks audience in the community.

Author response:
We are afraid that you may be underestimating the growing interest in Emergent Communication (EC) and Representation Learning.
Both of them are increasingly being recognized as important in recent years, as evidenced by the significant number of papers published in major machine leaning conferences.
Our paper bridges these two areas by demonstrating how EC signaling game can be re-interpreted as representation learning in generative models, namely beta-VAE.