Paper: Approaching an unknown communication system by latent space exploration and causal inference (ICLR-2024)

Reviewer comment (Evaluation):
How do the authors know that the latent generated by the proposed method are the actual latent structures of the data, not some artifacts of the model? How do you validate your results?

Author response:
## How to validate?

Without the ground truth, validation in this sense is impossible by definition. As restated in the general
remarks, the aim of this work is to provide a method that can guide research by providing independent
feedback for hypotheses.

That said, the model architecture used as the learning mechanism is structured to encode actually
meaningful properties of the data, which has been extensively shown on the closest dataset possible where
ground truth is available - raw human speech. Please note that in that case, the learning was done in exactly
the same way as here - raw audio, without any domain knowledge provided whatsoever. Furthermore, the
model is shown to uncover structural relations that are present in whale communication in general, but not
in the training dataset (Section 4, Paragraph 3). The odds of this happening due to a model artifact are extremely
low.

Reviewer comment (Experiments):
There are other disentanglement methods using VAE (e.g. betaVAE). The paper lacks comparisons with the existing approaches.

Author response:
## Comparison with the existing approaches

The relations to other disentangled representation learning methods are discussed in Section 1.2, with $\beta$-VAE specifically being cited as reference number 17.

We feel that the pros and cons of GAN vs VAE-based disentangled representation learning have been discussed in enough detail elsewhere (e.g. reference number 35). What is pertinent to our case is that we have a relatively small training dataset (Section 1.1., Paragraph 2). Additionally, we do not require the posterior latent distribution, nor do we necessarily wish to enforce strict independence between the factors. To illustrate, our model encodes both the mean frequency as well as the acoustic regularity (Section 5), which might not be independent depending on the true ground truth distribution (which we, of course, do not know). However, both properties are salient for e.g. human speech  - eg. general pitch vs. dynamics of a phrase, and it has been shown that whales have the capacity to deliberately control both of them.

Reviewer comment (Evaluation):
In figure 4 and 6, it seems that only 1 bit is different while other bits are in general similar to each other.

Author response:
## In figure 4 and 6, it seems that only 1 bit is different 
Disentanglement in this work refers to a process observable in the causal effect, as defined in Section 3: _"which
lends credence to interpreting this as a process of disentanglement: with higher values, the primary encoded
effect in each bit starts to dominate, leading the other bits to lose their secondary effects on the number of
clicks. Thus, we propose that bit 1 primarily encodes the number of clicks"_

As stated in another response: within the training range, the bits remain entangled in their effect; moving
outside the training range, the primary encoded effect begins to dominate, singling out one bit as the carrier
of the encoding, while the others return to a baseline value in unison. This phenomenon is replicated across
different observable quantities and estimators and is further examined in detail via an unrelated method in
the Appendix, section A.6, which shows that the effect of the bits not encoding the property becomes close
to random (in any particular observation) once sufficiently disentangled.

Hence the bit that is "different" is the one that disentangles and whose primary effect is the property
tested, while the others return to a baseline.

Reviewer comment (Experiments):
Can the proposed method be generalized to other animals' communication systems or other fields? The paper could be more solid by including results on another dataset.

Author response:
## Limitations, future research directions, and generalizations

The limitations are discussed in the third paragraph of Section 2.

As mentioned in another response and throughout the paper: the overall approach - using a generative
model to independently encode properties salient to believable data generation, then treating it as a black
box and uncovering these encodings in a causal experimental setup is completely model-agnostic. There is, therefore, no general impediment to applying the technique to other datasets, be it animal communication or not. In the latter case, one might want to replace the architecture used as the "learner" with another that has been tailored to the nature of the data (and verified on a close dataset with ground truth, as is the case here with human speech). This is one area of future research, with a way to test whether combinations of encodings carry meaning being another area.

In the specific case of animal communication, there is some impediment in the form of the availability of
good and open datasets, which should be understandable given the quite frequent difficulty in collecting such data.

Reviewer comment (Experiments):
The proposed CDEV method is applied to a single model trained on a small amount of data from a single unknown process. The model is fairly unconventional, with (i) modeling assumptions (the specific use of 5 discrete inputs) rooted in rudimentary understanding of the data process (5 general classes of data), and (ii) an unconventional GAN setup called fiwGAN. Although the authors claim that “the proposed approach can be extended to other architectures and datasets,” it would be more convincing if this extension were demonstrated, for example by corroborating the CDEV method on a known process such as human speech. Related work on human speech inspires confidence in GANs in general but does not evaluate the CDEV algorithm specifically. Moreover, domain knowledge appears to be a substantive input to the CDEV algorithm at multiple levels: (i) the model itself, (ii) the data selection process, (iii) the determining of appropriate extreme values, and (iv) the outcome functions used to measure average treatment effect. This raises doubts about the generality of the method.

Author response:
Regarding the remark about the model being closed: while the original training set is, unfortunately, not yet available, the model, the experiment data, and reproduction code are all provided in full - see Appendix, section A.1 for details

# Weaknesses

##  Narrow application of proposed methods

### (i) modeling assumptions ...
Please refer to the overall comment for clarification regarding the choice of the dimension of t.

### (ii) an unconventional GAN setup called fiwGAN ... / domain knowledge appears to be a substantive input ... (i) the model itself
Regarding the generality/conventionality of the model: as outlined in the general remarks, the main reason
for choosing this setup is that it provides the following: the synthetic data is independently verified, the
encodings $t$ are consistent and completely unsupervised, and the space available to them is limited, allowing
us to, together with the preceding two properties, argue for their saliency for believable generation.

The fact that similar models uncover known rules on human speech is used as support of the relevancy of
the encodings _after_ they’ve been identified via CDEV; in other words, the procedure itself does not require domain knowledge.

CDEV is, indeed, a novel contribution of this work; regarding its generality - the reason for using causal
inference methods is precisely to establish relationships between the encodings and observable properties
without additional assumptions or need for external knowledge: e.g., the value of bit 1 _has_ an effect on the
number of clicks. What that can suggest, however, is reasoned about following the argument in the preceding
paragraphs.


### (ii) the data selection process,
The data selection was driven by the fact that whale codas (naturally) come in the context of dialogue, thus
a significant part of the data featured interleaving codas. Thus, the reason for keeping only the top five
"classes" is simply that after removing the codas with dialogue, the remaining classes had too few examples
left - often one or none. As such, this process was not really driven by any goal obtained from domain
knowledge. Conversely, we posit that a more diverse dataset would be even more beneficial to the generator’s
ability to uncover salient properties.

### (iii) the determining of appropriate extreme values
As outlined in Appendix, Section A.4., the limits of the range of the values used as treatment are determined
by the quality of output of the generator. This is pointed out as a potential shortcoming in the case that
these are too narrow, however, there is no domain knowledge involved in that determination - noise is reasonably close to an universal phenomenon.  All values within this range are tested; additionally, the ICE estimator is "location-invariant" as it estimates the effect of an infinitesimal perturbation.

### (iv) the outcome functions used to measure average treatment effect.
Please refer to the general comments.

Reviewer comment (Evaluation):
There is insufficient evidence to support the application-specific claims made by the authors, such as the claim that mean spectral frequency is a salient property of sperm whale communication. It is unclear whether all confounders have been ruled out. For example, fluctuations in mean spectral frequency could be caused by unrelated factors such as water temperature changes, rather than communication. If this were the case, the GAN would still reproduce such fluctuations to fit the data distribution despite their irrelevance. Another potential confounder is that the GAN may have simply failed to fit the underlying data distribution, and any observed interventional effects could be artifacts of a poorly fit model.

Author response:
## Insufficient evidence to support claims

To continue on the points made in the general remarks: the purpose of this work is to present a procedure
that can _aid_ the scientific process in difficult problems without a ground truth. We do not claim to have
_proven_ any of the properties identified; however, the agreement of the method with the consensus on the
more obvious properties, such as the number of clicks, can be used to bolster its credibility for more novel
suggestions. Keeping this in mind, random sources of noise should even out with enough data, making the
uncovered encodings more likely to be deliberately controlled by the whales. 

Convergence of GANs is, as you’re most likely aware, a complex problem. We’d like to restate that we don’t need perfect convergence
for this use case; as is illustrated in Figure 3 for an almost worst-case example, the output still looks like a
whale coda, meaning that the GAN did not fail to fit the data to the extent that would make such analysis
worthless. As stated in Section 2: _"[noise] might, in part, be due to the network training not having converged
completely; since the goal is to discover what observable properties are encoded and not high-fidelity whale
communication generation, this is not a significant impediment."_

In the hypothetical case where one would not be even able to tell when the output is noise (we previously argue that this is hard to believe), one  could simply employ a fail-safe on the overall deviation from the data, for example using something in the vein of the Wasserstein estimators presented in Section 5.

Reviewer comment (Presentation):
In Figure 1, the t codes are shown as 0, 1 instead of -1, 1 as in the rest of the paper. This causes confusion, especially regarding the choice of low extreme values of -1 rather than 0.

Author response:
##  Figure 1 shows 0, 1 for $t$ codes ...
 Thank you for pointing this out, the typo in Section 2 has been fixed. More explanation is given in the general remarks.

Reviewer comment (Novelty):
I have a general concern. What is the advantage of learning a representation (t,x) using a deep generative model in the hope that $t$ will represent some intuitive features (like number of clicks and click regularity), if these very features can be extracted from the raw data algorithmically (as is done in this work, for evaluation)? I understand that this is only to evaluate the disentanglement of the method, but if these intuitive features can be computed from the raw data, what’s the point of trying to extract them in an unsupervised manner here in this application?

Author response:
Restating the clarification of our motivation from the general comment:  we’re interested in what an independent learner under the constraint of believability considers most salient to an unknown communication system. K.P. Murphy's Probabilistic Machine Learning: Advanced Topics describes it thus in Section (32.3.2.1): _"One criterion for learning a good representation of the world is that it is useful for synthesizing observed data. If we can build a model that can create new observations and has a simple set of latent variables, then hopefully this model will learn variables that are related to the underlying physical process that created the observations."_

To answer the question directly regarding extracting the features from the data directly: we are naturally biased towards assigning meaning to features that appear meaningful **to us** in truly unknown data such as sperm whale communication. For example, we can count the number of clicks in a vocalization "by hand". Can we then simply pronounce this as a carrier of meaning? How are we sure it's not simply a side-effect of something else? To provide an analogy, surely bats hear "features" of our speech that carry absolutely no meaning to us since we cannot hear those frequencies - they are merely high-frequency artefacts.

As outlined in the general comment and the previous responses, the architecture gives us a way of letting an **independent, unbiased** learner suggest what it considers most salient.
The learner has the following properties: (1) it has to be believable to an independent observer with access to real data, (2) the special encoding $t$ has to be consistent (3) it has to be salient else the model is wasting a precious encoding spot.

Thus, to answer your first question, this is in essence *not* about ensuring that $t$ is interpretable, us having already decided what it should be. We aim to answer **what** gets encoded in an unsupervised learning, no ground truth situation. Therefore, we believe there has been a misunderstanding about the basic aim of the paper.

Reviewer comment (Novelty):
As I said above, I believe this work present an original application of disentangled representation learning. However, I don't see novel methodological contributions. The authors essentially trained an existing method and applied fairly standard qualitative evaluation strategies to assess disentanglement.

Author response:
Novelty & Causal terminology

As a reminder, for GANs, the "incompressible noise" $X$ also encodes information, that is, its values have direct effects on the outcome. Thus, "ordinary" GANs have just $X$ as their feature space. However, it does not have the above-mentioned properties (2) & (3). Thus, the goal here is to identify **what** *gets encoded in $t$ as opposed to merely $X$*.

The novelty compared with [1] is described in paragraph 2 of Section 2. To expand on that: [1] is fundamentally concerned with examining how neural networks learn features of data with ground truth - human speech. The forcing of extreme values is employed in a latent space _interpolation_ problem setting (cf. chapter 20.3.5 in KP Murphy's book cited previously), with the architecture being an "ordinary" GAN. Thus specific, **pre-determined**  phonetic features are focused on. Extreme values are employed in the context of manipulating single bits in $X$ while the rest are set at _constant_ (random) values _for a given, single, example in which the desired phonetic property has already been identified_. This enables the author to examine, for example, how that specific phonetic feature morphs when one traverses the latent space outside the bounds of the training range. While the regression (not testing) methods employed are appropriate for an interpolation setting, there is no experimental setup with large samples of experimental _units_ being tested.

Conversely, this work is in almost all aspects an inverse problem to the above. We are dealing with a _structure discovery_ problem (Chapter 20.3.4 in the same book), where we have no ground truth. Thus we situate our methodology within an _experimental setup_ and wish to **test** for **what** properties are encoded. The **criterion** of something being encoded in a bit of $t$ is it undergoing the _process of disentanglement_ in the causal effect space with sufficiently large "treatment" values in the same bit. The extreme values technique is used here (in contrast to [1]) to **force** this process of disentanglement, which is not studied in [1].

Reviewer comment (Evaluation):
Figure 4: This does not correspond to the usual notion of “disentanglement”. Here, all the dimensions of t have an impact on the number of clicks. The interesting thing is that it seems only one dimension of $t$ has a positive impact, while all the others have a negative impact.

Author response:
Regarding Figure 5, the plots presented are for a single bit, stratified by the number of clicks. The bit proposed has a consistent
effect no matter the number of clicks in the generated coda, while other bits (such as the one presented) do not present any consistency in their encoding. This is in line with the definition presented above.

Reviewer comment (Evaluation):
Figure 5: I still believe this does not correspond to the usual notion of disentanglement, since, again, every dimension of t_i have an impact on the “inter-click” interval and the “coda regularity”, it’s just that one of the bits has a monotonic influence while the others are non-monotonic.

Author response:
Regarding Figure 5, the plots presented are for a single bit, stratified by the number of clicks. The bit proposed has a consistent
effect no matter the number of clicks in the generated coda, while other bits (such as the one presented) do not present any consistency in their encoding. This is in line with the definition presented above.

Reviewer comment (Experiments):
Are the findings robust to reinitialization? I.e., if you rerun this experiment with multiple seeds, are you always finding similar patterns? I don’t think the experiments show reruns. This is very important for such analysis, to avoid cherry-picking.

Author response:
We're not entirely clear what specific aspect is meant by "re-run" here. Below, we assume that the argument is that there is a significant probability of the results presented being due largely to chance. We'll address all interpretations that come to mind:

- **re-run of experiments given a trained model**:  
Appendix A.1.2. shows _almost identical_ effect values for independent experiments, each with a differently randomly sampled (i.e. different random seeds) samples of 2500 units. It is almost a tautology that if two random sequences converge to the same value, the limit must exist. To put it in a different way: the odds of a statistical estimator giving two nearly identical effect curves with two large, independent samples, _yet_ the true effect curve not existing are extremely unlikely. 

- **re-training of the model**:  
Appendix A.1.1 shows that the same property is encoded when a separate model with a different encoding space size is trained, with the same _process of disentanglement_.
Since GANs are trained via a minimax game, which almost surely has no equilibrium in cases such as here where we've added loss terms, the odds of two such games for different models  (1) converging to good output generation (2) encoding the same property in a completely unsupervised fashion  with (3) the same disentanglement process _purely by chance_ are, again, astronomically unlikely. This is the flip-side of the difficulty of training GANs.

- **replicability of the  disentanglement process:**  
 The same disentanglement process, which is used to identify encodings, is observed across completely unrelated properties and estimators, including a non-causal inference method presented in the Appendix, section A.7.

Reviewer comment (Presentation):
In Figure 4: the curves that go down seem to go up again after t = 10, do they end up being positive if we extend the range of t even further?

Author response:
As mentioned in Section 2 and the Appendix, Section A.2., the upper range was selected on the basis of
the generator still outputting samples with low enough noise so that our algorithmic measurement of the
observables was possible. To answer the question - extending the range would give us data that makes no
sense, hence speaking of an effect above (roughly) that value is not possible.

Reviewer comment (Evaluation):
Surprising findings, in Figure 1: “The 9R type was never part of the training data, yet our network learns to generate codas that resemble this type (Sec. 4).” I would like to see stronger evidence that the model can generate reasonable samples never seen during training. Was this sample cherry-picked?

Author response:
The evidence can be inferred from the relationships displayed in Figure 5. We see the presence of codas with
click numbers up to 10, while the model infers the connection between the number of clicks and the regularity
on its own: _"Furthermore, [real] codas with more clicks are often more regular in their ICIs, regardless of
their duration. The generator has thus inferred this connection without the codas with a high number of clicks
even being present in the dataset."_

Thus, regular 9R-like (the R denoting "regular") codas are quite common in the generated samples.

Reviewer comment (Theory):
The framing in terms of causal inference feels a bit unjustified, since one has full control over both x and t, so that there is not direct causal effect from x to t, making causal inference trivial.

Author response:
While this is true to an extent (and argued for in the Appendix, Section A.8), we believe the work shows that
causal inference-derived estimators are a natural fit for this problem setup. This is the reason phrases like
"methods from causal inference" are used in the work instead of a full causal argument. As an example of the
fit, the ICE estimator neatly sidesteps most dilemmas with regard to the choice of the baseline and the range
of "treatment", which are not self-evident in this case.

Reviewer comment (Writing):
In intro: “Networks trained on raw speech data are shown to learn to associate lexical items with code values and sublexical structure with individual bits in $t$ …” the variable $t$ was not defined, so it’s hard to follow what is meant here.

Author response:
###  variable $t$ was not defined
Thank you for pointing this out, this has been fixed.

Reviewer comment (Writing):
I found the description of fiwGAN a bit imprecise at the beginning of Section 2. I think it would help to write down explicitly the loss that is optimized, even if this is not part of your contribution.

Author response:
###  it would help to write down explicitly the loss that is optimized
The exact loss has been added in the Appendix, Section A.2.

Reviewer comment (Presentation):
Section 3: “We observe a high degree of entanglement (4) of the learned encodings within the range seen in training ([−1, 1]).” I don’t understand the meaning of “entanglement” here. I now understand that this paragraph refers to experiments in Figure 4. I feel like the separation between theory/method and experiments is not clear enough here.

Author response:
###  the meaning of "entanglement"
Please refer to the discussion regarding the use of the phrase "entanglement" in the previous comment.

Reviewer comment (Experiments):
$t$ is a vector of 5 bits, to fit the number of coda types in the dataset. Is this for the approach to work? You say other works have shown that having a mismatch doesn’t matter. But does it matter in your specific application?

Author response:
###  Do we need 5 bits for the approach to work?
Please refer to the general remarks.

Reviewer comment (Evaluation):
The contribution of the explanation to the original data of the latent dimension specified in the 5 dimensions being extracted by fiwGAN. Is it known how much the ATE of the number of clicks contributes to the representation of the observed data? More specifically, does the fact that the bit with index 1 in the latent variable space (5 dimensions) responds well to ATE mean that ATE is the second principal component (in the sense of, say, principal component analysis)? If so, what elements are responding to the first principal component (i.e., the bit with index 0)? Is the index of these bits tied to foresight knowledge such as the index of these 5 coder types, or does the index of bits reflect some ranking of its expressive power, as in classical principal component partial analysis? This may be a simple question that arises because I missed the description of the details.

Author response:
Dear reviewer, 

To follow up on your interesting question (1): we've added a section in the Appendix (A1.1.) that demonstrates our answer in the previous comment: in a new model, the number of clicks is encoded in bit0, while in the main model presented it was in bit1.

Please let us know if you have any further questions or comments!

Reviewer comment (Experiments):
Does the method of identifying causal structures analogous to dose-response, which the current proposed method does, also work for factor combinations? The current causal structure appears to be investigated separately for each factor, such as number of clicks, tones, etc. Is it not possible for a combination of those factors to have a new semantic meaning? For example, could "a constant rhythm of low tones" and "a constant rhythm of high tones" have different meanings? It seems that it would be difficult to distinguish between the two with only one factor, "constant rhythm," and that distinguishing them might require combining factors.

Author response:
Factor combinations, or "bit interactions", is indeed an area we’re examining for the extension of this method, in addition to adopting it to other architectures. As your motivating example shows, the presence of these is eminently plausible; additionally, the specific architecture has been shown to learn with compositionality on human speech, where the ground truth is available.