Unsupervised Learning of Neurosymbolic Encoders
Authors: Eric Zhan
Jennifer J. Sun
Ann Kennedy
Yisong Yue
Swarat Chaudhuri
Venue: ICLR 2022 Submitted (2022)
Keywords: unsupervised learning
representation learning
neurosymbolic program synthesis

Abstract: We present a framework for the unsupervised learning of neurosymbolic encoders, i.e., encoders obtained by composing neural networks with symbolic programs from a domain-specific language. Such a framework can naturally incorporate symbolic expert knowledge into the learning process and lead to more interpretable and factorized latent representations than fully neural encoders. Also, models learned this way can have downstream impact, as many analysis workflows can benefit from having clean programmatic descriptions. We ground our learning algorithm in the variational autoencoding (VAE) framework, where we aim to learn a neurosymbolic encoder in conjunction with a standard decoder. Our algorithm integrates standard VAE-style training with modern program synthesis techniques. We evaluate our method on learning latent representations for real-world trajectory data from animal biology and sports analytics. We show that our approach offers significantly better separation than standard VAEs and leads to practical gains on downstream tasks.

Under review as a conference paper at ICLR 2022
UNSUPERVISED LEARNING OF NEUROSYMBOLIC ENCODERS
Anonymous authors Paper under double-bling review
ABSTRACT
We present framework for the unsupervised learning of neurosymbolic encoders, i.e., encoders obtained by composing neural networks with symbolic programs from domain-specific language. Such framework can naturally incorporate symbolic expert knowledge into the learning process and lead to more inter- pretable and factorized latent representations than fully neural encoders. Also, models learned this way can have downstream impact, as many analysis work- flows can benefit from having clean programmatic descriptions. We ground our learning algorithm in the variational autoencoding (VAE) framework, where we aim to learn neurosymbolic encoder in conjunction with a standard decoder. Our algorithm integrates standard VAE-style training with modern program synthesis techniques. We evaluate our method on learning latent representations for real- world trajectory data from animal biology and sports analytics. We show that our approach offers significantly better separation than standard VAEs and leads to practical gains on downstream tasks.
INTRODUCTION
Advances in unsupervised learning have enabled the discovery of latent structures in data from variety of domains, such as image data (Dupont, 2018), sound recordings (Calhoun et al. 2019 and tracking data (Luxem et al. 2020 For instance, common approach is to use encoder-decoder frameworks, such variational autoencoders (VAE) (Kingma & WWelingg 2014), to identify a low- dimensional latent representation from the raw data that could contain disentangled factors of vari- ation (Dupont 2018) or semantically meaningful clusters (Luxem et al.] 20200. Such approaches typically employ complex mappings based on neural networks, which make it difficult to ex- plain how the model assigns inputs to latent representations (Zhang et al.) 2020).
To address this issue, introduce unsupervised neurosymbolic representation learning, where the goal is to find a programmatically interpretable representation (as part of a larger neurosymbolic representation) of the raw data. We consider programs to be differentiable, symbolic models in- stantiated using a domain-specific language (DSL), and use neurosymbolic to refer to blendings of neural and symbolic. Neurosymbolic encoders can offer few key benefits. First, since the DSL. reflects structured domain knowledge, they can often be numan-interpretabb (Verma et al. 2018 Shah et al. 2020). Second, by leveraging the inductive bias of the DSL, neurosymbolic encoders can potentially offer more factorized or well-separated representations of the raw data (i.c., the representations are more semantically meaningful), which has been observed in studies that used hand-crafted programmatic encoders (Zhan et al. 2020).
Our learning algorithm is grounded in the VAE framework (Kingma Welling 2014 Mnih & Gregor 2014). where the goal is to learn neurosymbolic encoder coupled with a standard neural decoder. key challenge is that the space of programs is combinatorial, which we tackle via a tight integration between standard VAE training with modern program synthesis methods (Shah
ctt al., 2020). We further show how to incorporate ideas from adversarial information factorization (Creswell et al. 2017) and enforcing capacity constraints (Burgess al., 2017 Dupont 2018) in
order to mitigate issues such as posterior and index collapse in the learned representation.
We evaluate neurosymbolic encoding approach multiple behavior analysis domains, where the data are from challenging real-world settings and cluster interpretability is important for domain experts. Our contributions

---

Under review as a conference paper at ICLR 2022
We propose novel unsupervised approach train neurosymbolic encoders, result in pro- grammatically interpretable representation of data (as part of a neurosymbolic representation).
We show that our approach can significantly outperform purely neural encoders in extracting se- mantically meaningful representations of behavior, as measured by standard unsupervised metrics.
We further explore the flexibility of our approach, by showing that performance can be robust across different DSL designs by domain experts.
We showcase the practicality of our approach downstream tasks, by incorporating our approach into a state-of-the-art self-supervised learning approach for behavior analysis (Sun et al.. 2021b).
2
BACKGROUND
2.1 VARIATIONAL AUTOENCODERS
We build on VAEs (Knggma & Welling 2014 Mnih & Gregor 2014), a latent variable modeling
framework shown to learn effective latent representations (also called encodings/emeedddnng (Hig-
gins al. 2 Zhao al. 2017 Yingzhen & Mandi 2018) and can capture the generative process (Oord et al. 2017 Vahdat & Kautz, 2020 Zhan et al. 2020 VAEs introduce a latent variable z, an encoder gú. decoder pe. and prior distribution p on z. Φ and 0 are the parameters of the q and p respectively, often instantiated with neural networks. The learning objective is to maximize the evidence lower bound (ELBO) of the data log- likelihood:::
(1) The first term in Eq. [ is the log-density assigned to the data, while the second term is the KL- divergence between the prior and approximate posterior of z. Latent representations z are often continuous and modeled with a Gaussian prior, but z can be modeled to contain discrete dimensions as well (Kingma et al. 2014 Hu et al. 2017: Dupont 2018 Our experiments are focused on behavioral tracking data in the form of trajectories, and so practice we utilize a trajectory variant of VAEs (eo-Reyes et al. 2018 Zhan et al. 2020 Sun et al. 2021b), described in Section 3.4
ELBO: := E [log
-DL (1o(2/x)(lp(z) S logg(x).
One challenge with VAEs (and deep encoder-decoder models in general) is that while the model is expressive, is often difficult to interpret what is encoded in the latent representation z. Common approaches include taking traversals in the latent space and visualizing the resulting generations (Burgess et al. 2017). post-processing the latent variables using techniques such clustering (Luxem et al.] 20200. Such techniques are post-hoc and thus cannot guide (in an interpretable way) the encoder to be biased towards a family of structures. Some recent work have studied how to impose structure in the form of graphical models or dynamics in the latent space (Johnson et al., 2016 Deng et al. 2017). and our work can be thought of as first step towards imposing structure in the form of symbolic knowledge encoded in a domain specific programming language.
2.2 SYNTHESIS DIFFERENTIALLE PROGRAMS
Our approach utilizes recent work on the synthesis of differentiable programs (Shah al. 2020 Valkov et aaall|20133 where one learns both the discrete structure of the symbolic program (analogous to the architecture of a neural network) as well as differentiable parameters within that structure. Our formulation of this problem closely follows that of Shah et al. 2020). We use domain-specific functional programming language (DSL), generated with a context-free grammar (see Figure 2 for an example). Programs are represented as pair (a, v), where is a discrete program architecture and s are its real-valued valued parameters. We denote as the space of symbolic programs (i.e. programs with complete architectures). The semantics of program (a,, 1b) are given by a function [a]] (z, v)), which is guaranteed by the semantics of the DSL to be differentiable in both x and v/.
Like Shah et al. (2020), we pose the problem of learning differentiable programs as search through directed program graph G. The graph 9 models the top-down construction of program architectures through the repeated firing of rules of the DSL grammar, starting with empty architecture (represented by the "start" nonterminal of the grammar). The leaf nodes of G represent programs with complete architectures (no nonterminals). Thus, P is the set of programs in the leaf nodes of g. The other nodes in G contain programs with partial architectures (has at least one nonterminal). We interpret program in nonfleaf node being neurosymbolic, by viewing its nonterminals as
2

---

Under review as a conference paper at ICLR 2022
Optimize fixed
Update symbolic encorer (a, af - (ol, of), (f,
stepa Optimize with complete
Muwd) fully symbolic,
Symboois Fullymuraaton
Reconstrucedd
Input
Symbool: Encocir: Progrom depch
Input
symboll:
Reconstruuted Inpul
R(aup]
R(o.8)
Rio.s
Po
Rut
P0
96
96
Noural Decoder
Z
Neural Decoder
Noural
Fornder
The symbolic encoder is initially fully neural. We alternate between VAE training with the program architecture fixed (Step 1), and supervised program learning to increase the depth of the program by (Step 2). Once we reach symbolic program, we train the model one last time to learn all the parameters. The color (in terms of lightness) of the symbolic encoder corresponds to the encoder becoming symbolic over time.
Figure 1: Sketch of Algorithm (Section 3.1
representing neural networks with free parameters. The root node in G is the empty architecture coo interpreted as a fully neural program. An edge (a,a') exists in G if one can obtain a' from a by applying rule in the DSL that replaces nonterminal in
Program synthesis in this problem setting equates to searching through G to find the optimal com- plete program architecture, and then learning corresponding parameters v. i... find the optimal (a, w) that minimizes a combination of standard training loss (e.g., classification error) and struc- tural loss (preferring "simpler" a's). [hah et al. (2020) evaluate multiple strategies for solving this problem and finds informed search using admissible neural heuristics to be the most efficient strat- egy (see appendix). Consequently, we adopt this algorithm for our program synthesis task.
3 NEUROSYMBOLLIC ENCODERS
The structure of neurosymbolic encoder is shown in the right diagram of Figure 1 The la- tent representation = [Zg. [()))) is partitioned into neurally encoded 2g and programmatically encoded (a,,)) This approach boasts several advantages:
The symbolic component of the latent representation is programmatically interpretable.
The neural component encode any residual information not captured by the program, which maintains the model's capacity compared to standard deep encoders.
By incorporating a modular design, we can leverage state-of-the-ar learning algorithms for both differentiable encoder-decoder training and program synthesis.
We denote Ya and ((,)) the neural and symbolic encoders respectively (see Figure D where
Zg qo(-(x) and (a,,)) ((,,) [x). 9 is instantiated with a neural network, but ((a.v) is differentiable program with architecture a and parameters vi in some program space P defined by
DSL. Given unlabeled training of x's, the VAE learning objective in Eq. [ then becomes:
max (a(,,)))
E 9e 5o(2,l10cca. (R(a.))))))
log Px Z(a
-D DEL (3) (Z) ())
regularization for neural latent
regularization for symbolic latent
Compared to the standard VAE objective in Eq. for single neural encoder, Eq. has separate KL-divergence terms for the neural and programmatic encoders.
3.1 LEARNING ALGORITHM
The challenge with solving for Eq. 2 is that while (6, 0) can be optimized via back-propagation with fixed, optimizing for is discrete optimization problem. Since it is difficult jointly optimize over both continuous and discrete spaces, we take an iterative, alternating optimization approach. We start with fully neural program (one with empty architecture ag) trained using standard differentiable optimization (Figure I Step 1). We then gradually make it more symbolic (Figure Step 2) by finding program that child of the current program in g (more symbolic
3

---

Under review as a conference paper at ICLR 2022
by construction of () that outputs as similar to the current latent representations possible:
min
Lsapervised (M(a (
(x), Maar (x));
(3)
o' gM
which can be viewed as a form of distillation (from less symbolic to more symbolic programs) via matching the input/output behavior. We solve Eq, by enumerating over all child programs and se- lecting the best one, which is similar to iteratively-deepened depth-first search in [Shah et al. (2020) (see appendix). We alternate between optimizing Eq. 2 and Eq. until we obtain complete pro- gram. Algorithm ! outllnns this procedure and is guaranteed to terminate if G is finite by specifying maximum program depth.
We chose this optimization procedure for two reasons. First, it maximally leverages state-of-the- thete-rr tools in both differentiable latent variable modeling (VAB-style training) and supervised program synthesis, leading to tractable algorithm design. Second, this procedure never makes a drastic change to the program architecture, leading relatively stable learning behavior across iterations.
Algorithm 1 Learning neurosymbolic encoder
Algorithm 2 Learning a neurosymbolic encoder with k programs
I: Input: program space P. program graph G
2: initialize 6 vt, 0, <xg (empty architecture)
1: Input: program space P. program graph G,
3: while a is not complete do
2: for i = L..k do
4: 6, tb, + optimize Eq. 2 with a fixed
3: fix programs (h .... 1o.-1.91-1 1.::
5: (a, eb) +- optimize Eq.
4: execute Algorithm to learn ((,
6: end while
5: remove ((a,))))
from P avoid redundancies
7: o, 0,00 t optimize Eq 0 with complete a
end for
8: Return: encoder (())))
7: Return: encoder (90. (or.)1))-- Homaoa)
3.2 LEARNING MULTIPLE PROGRAMS
The interpretability of latent representations induced by symbolic encoders ((,,) ultimately de- pends the DSL. For instance, program that encodes one of ten classes may not be very inter- pretable if it involves a matrix multiplication within the program. Instead, we learn binary programs that encode sequences into one of two classes (using binary cross-entropy for Lsupervised: uniform prior on (().v)) and Gumbel-Softtax (Jang et al. (017) to sample from the posterior). Figures 4a & 4b depict learned binary programs that encode mice trajectories and their interpretations.
To encode more than two classes, we can simply learn multiple binary programs by extending Eq. to sum over Lsupervised for k symbolic programs {K (()...))....... ((a) and corresponding latent representations (Z(a), ...., -x)) This results in 2* classes and solution space that now scales exponentially (e.g. |p|] if using exhaustive enumeration). Algorithm 2 outlines our greedy solution that reuses Algorithm by by iteratively learning one symbolic program at a time. We leave the exploration of more sophisticated search methods as future work.
3.3 DEALING WITH POSTERIOR AND INDEX COLLAPSE
Deep latent variable models, especially those with discrete latent variables, are notoriously prone to both posterior (Bowman et al., 2015 Chen et al. 20166 Oord et al.] 2017) and index (Kaiser et al. 2018) collapse. Since our algorithms optimize for such models repeatedly, we can be more susceptible to these failure modes. There are many approaches available for tackling both these issues, but we emphasize that these contributions are orthogonal to ours; as techniques for preventing posterior and index collapse improve, so will the robustness of our algorithm. Below, we summarize two strategies that found to work well in our setting.
Ad Adversarial information factorization Creswell et al. (017)) introduces an adversarial network A. that aims to predict (a)v) from zz. Maximizing this adversarial loss can prevent index collapse, where all data is encoded into the same class, as doing so would would fail to fool the adversary.
max
E
[log pa (x Maaw)) + min Ladv (Aw (Zo), ((a,w))) )
(0,0)
(4)
adversary
- -D (go ())))(())) DKL (Hoo)) (2 ())l)(((a))))
4

---

Under review as a conference paper at ICLR 2022
Channel capacity constraint (Burgess al. 2017 Dupont 2018) forces the KL-divergence terms
to match capacities Co and (a Since the KL-divergence is an upper bound on the mutual infor-
mation between latent variables and the data (Kim & Mnih 2018 Dupont 2018), this encourages the latent variables to encode information and aims to prevent posterior collapse.
max
E
log Po (x|z [o Yo D (90(2,
Col
(.(0,,)),0
(5)
(o,,)) | (Ma.w) (
(ax,)) v) xx p (o.
Cla,wo)
In our algorithms,, we augment our initial objective in Eq. with Eq. and Eq. 5
3.4 INSTANTIATION FOR SEQUENTIAL. DOMAINS
The objective in Eq. describes general problem that applicable any domain. In our ex- periments, we focus on the sequential domain of trajectory data. Trajectory data is often used in scientific applications where interpretability is desirable, such as behavior discovery (Luxem al.) 2020 Hsu & Yitri 2020). The ability to easily explain the learned latent representation using pro- grams can help domain experts better understand the structure their data. Additionally, trajectory data is often low dimensional for each timestamp, which helps experts encode domain knowledge
into the DSL more easily (Shah et al. 2020 Sun et al. 2021b Zhan et al. 2020).
In this domain, is trajectory of length T: (x1...., 2T). ( in Eq. as product of conditional probabilities:
We then factorize the log-density
log po (x/Z (a.v))
log pa (ze Tett Bay M(a,,))-
When 4s and pa instantiated with recurrent neural networks (RNN). the model is more commonly
known as a trajectory- VAE (TVAE) Co-Reyes et al. (2018): Zhan al. (020): Sun et al. (2021b).
As symbolic encoder ((a,v) maps sequences to vectors, adopt DSL (Figure previously used for sequence classification (Shah et al. 2020). Our DSL is purely functional and contains both basic algebraic operations and parameterized library functions. Domain experts easily augment the DSL with their own functions, such as selection functions that select subsets features that they deem potentially important. We ensure that all programs in our DSL. are differentiable, utilizing smooth approximation of the nondifferentiable if-then-else construct (Shah et al. 2020). Figures 4a i3 depict example programs in our DSL (full details in the appendix).
: x ((a)...., ax) (Bea....... .ak)
if a1 then 02 elsc 0:3 sels x mapaverage (fun x1.01) ) ar
Our DSL sequential domains, similar the one used in Shah et inputs, basic algehraic operations, and parameterized library functions, respectively. fun xre(x) represents a function that evaluates an expression e(r) over the input x. sely selects subset $ of the dimensions of the input mapaverage
Figure 2:
2020 x, @. and DD represent
applies the function to every element of the sequence and returns the average of the results. We employ different approximation of the (f-then-chl construct.
4 EXPERIMENTS
We study our proposed approach on sequential trajectory data from a synthetic dataset to first provide intuition for our algorithm, and then on real-world datasets in neuroscience and sports analytics.
4.1 EXPERIMENTS WITH SYNTHETIC DATASET
We generate a synthetic dataset of trajectories and run our algorithm to demonstrate the following:
Programs can capture factors of variation in the data (in our case, discrete factors).
Information pertaining captured factors of variation are extracted out of the latent space.
We generate synthetic trajectories by sampling initial positions and velocities from a Gaussian dis- tribution and introducing 2 ground-truth factors of variation large external forces in the posi-
5

---

Under review as a conference paper at ICLR 2022
tive/negative x/y directions that affect velocity, totalling to 4 discrete classes. Velocities are fixed for the entire trajectory, but we also sample small Gaussian noise at each timestep. We generate 10k/2k/2k trajectories of length 25 for train/ validationn// Figure shows 50 trajectories from the training set. Full details of the synthetic dataset are included in the appendix.
We visualize the neural latent space in 2 dimensions of TVAE with 0, 1, and 2 learned programs in Figure ped. The initial TVAE latent space contains 4 clusters corresponding to the 4 ground- truth classes in Figure 3c After our algorithm learns the first program that thresholds the final captures the other factor of variation as 2 clusters corresponding to the final y-positions. Lastly, when our algorithm learns second program that thresholds the final y-position, the resulting latent space in Figure Be no longer contains any clear clustering, as we've successfully extracted the ground-truth classes with our 2 programs.
x-position, the resulting latent space in Figure 3d
|28.31] selpmaaXXovttion
1 28 |2809 sel PinalPPeccsso
x
(a) 50 synthetic trajectories (b) learned programs (c) zg. 0 programs (d) Zz. program (c) Z,. programs
Figure 3: (a) Trajectories synthetic training set. Initial/final positions are indicated in green/blue. Red lines delineate ground-truth classes, based on final positions. (b) & = 2 learned binary programs using our algorithm. The first program (top) thresholds the final x-position while the second program (bottom) thresholds the final y-position. (c, d, Neural latent variables reduced 2 dimensions. Top/bottom by final dy-possitions respectively (green/yellow is positive/negatiie). (c) Clusters in TVAE neural latent space correspond ground-truth classes. (d) After learning the first program, the neural latent space contains clusters only based on the final y-position. (e) After learning the second program, all 4 ground-truth classes have heen extracted as programs and the remaining neural latent space contains no clear clustering.
are colored
4.2 EXPERIMENTS WITH REAL- WORLD DATASETS
4.2.1 DATASETS
CaIMS21. Our primary weal-world dataset is the CalMS21 dataset (Sun et al. 2021a), containing trajectories of socially interacting mice captured for neuroscience experiments. Each frame contains 7 tracked keypoints for each of two mice. The dataset has one set of unlabeled tracking data, which we use to train our neurosymbolic encoder, and another set annotated for 4 behaviors, which we use to evaluate our programs. Specifically, our evaluation uses labels from the test split of the CalMS21 classification task. We have 231k/52k/262k trajectories of length 21 for train/val/test. The features in our DSL are selected by a domain expert based on the attributes from (Segalin et al. 2020).
Basketball. We use the same basketball dataset as in (Shah et al. 2020 Zhan al. 2020 that tracks professional basketball players. Each trajectory is of length 25 over 8 seconds and contains the ry- positions of 10 players. We split trajectories into two by grouping offensive and defensive players (5 each), effectively doubling the dataset size. We evaluate our algorithm and the baselines with respect the labels of offensiie/defeesiiee players. Our DSL includes additional domain features like player speed and distance-to tortieee----oeee basket. In total, we have 1772/31k/27k trajectories for train/val/test.
4.2.2 QUANTITATIVE EVALUATION SETUP
Baselines. We compare our model containing a neurosymbolic encoder against other approaches based on VAES and its variations. particular, we compare against VAE, VAE with K-means loss used in (Ma et al. 2019: Luxem al. 2020), and Beta-VAE (Burgess et al., 2017). These models have a fully neural encoder and learn continuous latent representations, which we can then use to produce clusters with K-means clustering (Lloyd 1982). Additionally, compare against models which produce discrete latent clusterings, such as Joint VAES (Dupont) 2018) and VQ-VAEs (Oord et al. 2017). We use the TVAE version of all baselines (details included in the appendix).

---

Under review as a conference paper at ICLR 2022
mapaverage (fun It.
multiply (ResidentSpedd [- (-).)) 8.28 (Tt),
[27000
Nose TailDDit (aaa 9.06
(a) Program learned using CalMS21 DSL I, resulting NMI 0.428. Since speed is positive, the first term is always negative. One cluster thus generally consists of trajectories where the mice are further apart, such that the second term is positive, and the negative product is less than the threshold. The other cluster generally occurs when the mice are close logether, the second term is negative, and the product will be positive.
mapaverage (fun 21. add [ResidentAuis (fesidenttAiissttiiffffifiee -
1p -..0
BoundingBoz!OUU -
(b) Program learned using CalMS21 DSL 2. resulting 0.300 The axis ratio is the ratio of major axis length and minor axis length of an ellipse fitted to the mouse keypoints. The second term measures the bounding box overlap between mice, and is zero when the mice are far apart. It follows that cluster generally contains trajectories when the mice has larger bounding box overlaps cluster thus contains trajectories where the mice bounding boxes not overlap, and resident body is compact.
the resident axis ratio is large. The other
Figure 4: Learned programs Ca The subscripts represents the learned weights and biases, in partic- ular, the brackets contain the weights for the affine transformation followed by the bias.
CalMS21
Basketball
Model
Purity
NMI
RI
Purity
NMI
RI
Random assignment
.397
.000
.536
.500
.000
.500
TVAE
.598
.089
.564
.501
.001
.500
TVAE+KMeans loss
.605
.118
.573
.501
.001
.500
JointVAE
.597
.019
.537
.560
.034
.507
VQ-TVAE
.601
.124
.588
.572
.016
.511
Reta-TVAA
.616
.115
.589
.565
.013
.509
Ours (1 program)
.706
.423
.694
.566
.027
.188
Ours (2 programs)
.725
.320
.648
.561
.033
.507
Ours (3 programs)
.756
.314
.633
.844
.022
.514
Table 1: Median purity, NMI, and RI on CaIMS21 and Baskethall compared to human-annotated labels (3 runs). Experiment hyperparameters are included in the appendix.
Metrics. Unlike in the synthetic setting, we do not have ground truth programs in the real-world datasets. We thus evaluate our programs quantitatively using standard cluster metrics relative to human-defined labels. In particular, we use Purity (Schütze et al., 2008), Normalized Mutual Infor- mation (NMI) (Zhng et al. 20060, and Rand Index (RI) (and 1971). Purity measures the extent to which clusters contain a single human-defined class. NMI scales with the mutual information between two cluster assignments. RI measures similarity between our clusters and human labels. We report the median of three runs, and include a random baseline that assigns a class randomly to each sequence. More details, including the standard deviation and the ELBO, are in the appendix.
4.2.3 RESEARCH QUESTIONS
Are the clusters created with our programs meaningful? We compare clusters produced by our neurosymbolic encoder with fully neural autoencoding baselines (Table D. measured against human-annotated behaviors. For CaIMS21, we observe that our method consistently outperforms the baselines in all three cluster metrics. Our method is able to do this by leveraging a DSL which encodes domain knowledge such as behavior attributes that are useful for identifying behaviors. We note that the purity increases the number of programs (thus clusters) increase, while NMI and RI decreases. This implies our method with two clusters best correspond to CalMS21 behaviors, but the other clusters found by our method may still be useful for domain experts. For Basketball, our method improves slightly with respect to purity, but is overall comparable with the baselines.
We further study the programs and clusters produced by our algorithm for the CalMS21 dataset, through a qualitative study with a domain expert in behavioral neuroscience. In the single program case, the domain expert classified the discovered clusters as when the mice are interacting and when there interaction. They noted that this is based distance between the mice, which is sistent with our program (Figure Pa using distance between nose of resident and tail of intruder. For two programs, there are total of four clusters, with two clusters each corresponding
in-

---

Under review as a conference paper at ICLR 2022
CaIMS21 (DSL.
1)
CalMS21 (DSL 2)
CaIMS21 (DSL3)
Model
Purity
NMI
RI
Purity
NMI
RI
Purity
NMI
Ours (1 program)
.706
423
.694
.689
.364
.681
.449
.325
.616
Ours (2 programs)
.725
.200
.648
.715
.359
.673
.664
.324
.634
Table 2:
Median purity, NMI, and RI on CaIMS21 of our algorithms with DSLs selected by three domain
experts compared to human-annotaled labels (3 runs). DSLI corresponds to Table
CalMS21
Basketball
Model
Purity
NMI
RI
Purity
NMI
RI
TVAE
.598
.099
.564
.501
.001
.500
TVAE (w/ features)
.597
.103
.570
.565
.012
.508
VQ-TVAE
.601
.124
.588
.571
.016
.511
VQ-TVAE (w/ features)
.608
.114
.601
.525
.002
.501
Beta-T TVAE
.616
.115
.589
.566
.013
.509
Beta-TVAE (w/ features)
.612
.096
.571
.563
.011
.508
Table 3: Median purity, NMI, and RI on CaIMS21 and Basketball compared to human-annotated labels (3 runs) for baseline with trajectory inputs only, and baseline with trajectory features added.
teraction and interaction. For the interaction clusters, the domain expert was further able identify sniff tail behavior as one of the clusters. In this case, the programs found were based on intruder head body angle, resident nose and intruder tail distance, and resident nose and intruder nose dis- tance. The domain expert found the three program case to be more difficult to interpret, but was able to identify clusters corresponding to sniff tail, resident exploration. interaction facing the same direction (ex: mounting), and interaction facing opposite directions (ex: face-to-face sniffing).
How sensitive our approach to different DSL choices? To study the effect of domain expert variation on our approach, we worked with different domain experts to construct alternate DSLs for studying mouse social behavior on CalMS2 While there is some variation in median cluster metrics, our approach consistently outperforms other baseline approaches that contain fully neural encoders for all three DSLs (Table D. Comparing some learned programs from two DSLs (Fig- ures 4a 4b.. both contain term that is correlated with whether the mice is interacting (distance and bounding box overlap), and another term on resident speed (mouse tends to be more stretched when they are moving quickly). A full list of features selected by domain experts is in the appendix.
What happens if we simply encode DSL features as part of input trajectories? Since our method uses features from domain experts part of our DSL, additionally experiment with providing the same features as input to the baseline models during training (Table B For both CalMS21 and Basketball, providing the features as additional inputs baseline has comparable performance to using input trajectory data alone. In contrast, by using the features more explicitly as part of the DSL in our neurosymbolic encoders, we are able to produce clusters with a better separation between behavior classes based on cluster metrics (as was seen in Table II.
Are the programs useful for downstream tasks? We integrate the learned programs from our neurosymbolic encoder into the task programming framework (Sun et al., 202Tb), a state-of-the- art self- and oogrrammtticall-ssperiii programs to train a trajectory representation, which can be applied to behavior analysis. Here, rather than using hand-crafted programs, we instead use the learned programs from our approach, so that experts would only need to provide the DSL. We evaluate the same mouse dataset (Segalin al., 2020) as task programming. Using only one program found using our approach, we are able to achieve comparable performance 10 expert-writte programs on the behavior classification task
learning approach. This framework uses expert provided
Task Programming Program Variations
Figure 5: Applying symbolic encoders for self-supervision. "Features" baseline w/o self sulervuuervisio. "TREBA" is a self- supervised approach, using either expert- crafted programs or our symbolic encoders as the weak-supervision rules. The shaded region is std dev over 9 repeats. The dev for our approach (not shown) is comparable.
24-1
2w10-1
TRESA
(LO expert progsi
18x10
1,4x10
(ours, 1 pmm))
TRESA
14410
Features
(eurs, progs)
12-
10-2
30 Traning Data Fraction (Leg Scale)
8

---

Under review as a conference paper at ICLR 2022
studied in Sun al. (2021b) (Figure 5) This demonstrates that programs found by approach
can be applied effectively to downstream behavior analysis tasks such as task programming.
OTHER RELATED WORK
Interpretable latent variable models. Latent representations, especially those that human- interpretable, can help us understand the structure of data. These models may learn disentangled factors Qiggins et al. 2016 Chen et al. 2016a) semantically meaningful clusters (Ma et al. 2019) using unsupervised learning approaches. These approaches are often grounded in the VAE framework (Kingma & Welling 2014p. Similar to our programs which produce discrete code, there are other VAE variations which also learns a discrete latent representation, such as Joint- VAE (Dupont 2018). Discrete VAE (Rlte 2016), and VQ-VAE (Oord et al. 2017). In particular, Joint models also learns discrete and continuous representation. However, these approaches use fully neural encoders. To the best of our knowledge, our work is the first to propose neurosym- bolic encoders, where the symbolic component produces an interpretable program.
Neurosybbolicciiffeeenniia program synthesis. Existing program synthesis approaches are of- ten trained in supervised fashion (Gulwami 2011 Wang et al. 2017 [Sahh et al. 2020), or within
(generative) policy learning context with an explicit reward function (Chen et al. 2018 Verma et al, 2018 Inala et al. 2020; Feinman & Lake 2020). In terms of unsupervised program synthesis, the closest related area is generative modeling, as the goal there is to discover programs that can gener- ate the training data (Ellis al. 2018 8 Feinman & Lake 2020 which can be viewed as analogous to learning a symbolic decoder rather than a symbolic encoder. Studying how to incorporate such methods into our framework can be interesting future direction.
Representation learning for behavior analysis. Representation learning has been applied to a variety of downstream tasks for behavior analysis, such as discovering behavior motif's (Berman et al. 2014 Singh et al.] 202Ip, identifying internal states (Calhoun et al. 2019). and improving sample-efficiene (Sun et al.) (021b). Studies in this area have used methods such as VAE (Kingma & Welling, 2014), AR-HMM (Wiltschko et al.j (013), and Umap (Melnnes et al. 2018 to better understand the latent structure of behavior. Similar to a few other representation learning meth- ods (Luxem et al., 2020) Sun et al. 2021b), we also use encoder-decoder setup on trajectory data. However, our work learns a neurosymbolic encoder whereas existing works in this area have fully neural encoders. Our work can aid behavior analysis by learning more interpretable latent representations and can be applied to existing frameworks, such as task programming.
CONCLUSION
We present a novel approach for unsupervised learning of neurosymbolic encoders. Our approach in- tegrates the VAE framework with program synthesis and results in a learned representation with both neural and symbolic components. Experiments on trajectory data from behavior analysis demon- strate that our programmatic descriptions of the latent space result in more meaningful clusters rel- ative to human-defined behaviors, compared purely neural encoders. Additionally, we show the practicality of our approach by applying our learned programs to achieve comparable performance to expert-constructed tasks in a self-supervised learning approach for behavior classification.
Based on our work, there are many future directions to explore for neurosymbolic encoders. One direction is based on the observation that interpreting multiple programs simultaneously can still be difficult for domain experts. Combining our approach with other clustering methods, such as hierarchical clustering (Rokach Maimon 2005). or working with domain experts detail more expressive DSLs can help with interpretabilitt. Another direction is to extend this work to other domains such as image and text data, to study both discrete and continuous symbolic latent repre- sentations in more naturally interpretable way. A third direction is to improve upon our greedy approach in Algorithm g for finding the optimal set of symbolic programs, e.g. by performing lo- cal coordinate ascent in program space, similar to algorithms for large-scale neighborhood search extensions of VAEs such as our own have
(Ahuja et al. 2002). Lastly, while practically-oriema yielded great practical benefit, they often lead sub-optimal results from pure likelihood (or ELBO) perspective. One final direction is to rigorously formulate a learning objective from the ground up that formally encapsulates practically-oriene extensions of VAES.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Ravindra K Ahuja, Özlem Ergun, James B Orlin, and Abraham P Punnen. A survey of very large- scale neighborhood search techniques. Discrete Applied Mathematics, 123(1-3):75-102, 2002.
Gordon Berman, Daniel M Choi, William Bialek, and Joshua W Shaevitz. Mapping the stereotyped behaviour of freely moving fruit flies. Journal of The Royal Society Interface, 1 11(99):2014061) 2014.
Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, and Samy Bengio. Generating sentences from a continuous space. arXiv preprint arXiv:1511.06344, 2015.
Christopher Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins, and Alexander Lerchner. Understanding disentangling in 8-vae. In Neural Information Process- ing Systems Disentanglement Workshop, 2017.
Adam J Calhoun, Jonathan W Pillow, and Mala Murthy. Unsupervised identification of the internal states that shape natural behavior. Nature neuroscience, 22(12))00002049, 2019.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Info- gan: Interpretable representation learning by information maximizing generative adversarial nets. arXiv preprint arXiv: w60.. 03667, 2016a.
Xi Chen, Diederik P Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, and Pieter Abbeel. Variational lossy autoencoder. arXiv preprint (r.iv:/611.00231 2016b.
Xinyun Chen, Chang Liu, and Dawn Song. Towards synthesizing complex programs from input- output examples. In International Conference on Learning Representations, 2018
John Co-Reyes, YuXuan Liu, Abhishek Gupta, Benjamin Eysenbach, Picter Abbeel, and Sergey Levine. Self-consistent trajectory autoencoder: Hierarchical reinforcement learning with trajec- tory embeddings. In International Conference on Machine Learning, 2018.
Antonia Creswell, Yumnah Mohamied, Biswa Sengupta, and Anil A Bharath. Adversarial informa- tion factorization, arXiv preprint arXiv: 1.05 2017.
Zhiwei Deng, Rajitha Navarathna, Peter Carr, Stephan Mandt, Yisong Yue, lain Matthews, and Greg Mori. Factorized variational autoencoders for modeling audience reactions to movies. In IEEE conference on computer vision and pattern recognition, 2017.
Emilien Dupont. Learning disentangled joint continuous and discrete representations. In Neural Information Processing Systems, 2018.
Kevin Ellis, Daniel Ritchie, Armando Solar-Lezama, and Joshua B Tenenbaum. Learning to infer graphics programs from hand-drawn images. In Advances in Neural Information Processing Systems, 2018.
Reuben Feinman and Brenden M Lake. Learning task-general representations with generative neuro- symbolic modeling. In International Conference on Learning Representations, 2020.
Sumit Gulwani. Automating string processing in spreadsheets using input-output examples. ACM
Sigplan Notices, 46(1):317-330, 2011.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations, 2016.
Alexander Hsu and Eric A Yttri. B-soid: An open source unsupervised algorithm for discovery of spontaneous behaviors. bioRxiv, pp. 770271, 2020.
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. Toward controlled generation of text. In International Conference on Machine Learning, 2017.
10

---

Under review as a conference paper at ICLR 2022
Jeevana Priya Inala, Osbert Bastani, Zenna Tavares, and Armando Solar- Synthesizing programmatic policies that inductively generalize. International Conference on Learning Rep- resentations, 2020.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterizztioo with gumbel-softmax. arXiv preprint arXiv:1611.01 144, 2017.
Matthew J Johnson, David Duvenaud, Alexander Wiltschko, Sandeep Datta, and Ryan P Adams. Composing graphical models with neural networks for structured representations and fast infer- ence. In Advances Neural Information Processing Systems, 2016.
Lukasz Kaiser, Samy Bengio, Aurko Roy, Ashish Vaswani, Niki Parmar, Jakob Uszkoreit, and Noam Shazeer. Fast decoding sequence models using discrete latent variables. In International Con- ference Machine Leurning. 2018.
Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In International Conference on Machine Learning. 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Diederik P Kingma and Max Welling, Auto-encoding variational bayes. In International Conference on Learning Representations, 2014.
Diederik P Kingma, Danilo J Rezende, Shakir Mohamed, and Max Welling. Semi-supervised learn- ing with deep generative models. arXiv preprint arXiv:1406.5298, 2014.
Stuart Lloyd. Least squares quantization in pem. IEEE transactions information theory, 28(2): 129-137, 1982.
Kevin Luxem, Falko Fuhrmann, Johannes Kürsch, Stefan Remy, and Pavol Bauer. Identifying be- havioral structure from deep variational embeddings of animal motion. bioRxiv, 2020.
Qianli Ma, Jiawei Zheng, Sen Li, and Gary W Cottrell. Learning representations for time series
clustering. Advances neural information processing systems, 32:3781-3777 13791, 2019.
Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint 1r02vv1802.0030 2018.
Andriy Mnih and Karol Gregor. Neural variational inference and learning belief networks. In International Conference Machine Learning, 2014.
Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learn- ing. In Advances in Neural Information Processing Systems, 2017.
William M Rand. Objective criteria for the evaluation of clustering methods. Journal of the Ameri- can Statistical association, 666336);86-850, 1971.
Lior Rokach and Oded Maimon. Clustering methods. In Data mining and knowledge discovery handbook, pp. 321-352. Springer, 2005.
Jason Tyler Rolfe. Discrete variational autoencoders. arXiv preprint arXiv:1609.02200 2016.
Hinrich Schütze, Christopher D Manning, and Prabhakar Raghavan. Introduction to information retrieval, volume 39. Cambridge University Press Cambridge, 2008
Cristina Segalin, Jalani Williams, Tomomi Karigo, May Hui, Moriel Zelikowsky, Jennifer J Sun, Pietro Perona, David J Anderson, and Ann Kennedy. The mouse action recognition system (mars): a software pipeline for automated analysis of social behaviors in mice. bioRxiv, 2020.
Ameesh Shah. Eric Zhan, Jennifer Sun, Abhinav Verma, Yisong Yue, and Swarat Chaudhuri. Learning differeeniaalee programs with admissible neural heuristics. In Neural Information Pro- cessing Systems, 2020.

---

Under review as a conference paper at ICLR 2022
Satpreet H. Singh, Steven M Peterson, Rajesh PN Rao, and Bingni W Brunton. Mining naturalistic human behaviors in long-term video and neural recordings. Journal of Neuroscience Methods, 2021.
Jennifer Sun, Tomomi Karigo, Dipam Chakraborty, Sharada Mohanty, David Anderson, Pietro Perona, Yisong Yue, and Ann Kennedy. The multi-agent behavior dataset: Mouse dyadic social interactions. arXiv preprint arXiv:2104.0277 2021a.
Jennifer Sun, Ann Kennedy, Eric Zhan, David Anderson, Yisong Yue, and Pietro Perona. Task programming: Learning data efficient behavior representations. In Conference Computer Vi- sion and Patten Recognition, 2021b.
Arash Vahdat and Jan Kautz. Nvae: deep hierarchical variational autoencoder. In Advances in Neural Information Processing Systems, 2020.
Lazar Valkov, Dipak Chaudhari, Akash Srivastava, Charles Sutton, and Swarat Chaudhuri. Houdini: Lifelong learning as program synthesis. In Advances in neural information processing systems, 2018.
Abhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, and Swarat Chaudhuri. Programmatically interpretable reinforcement learning. In International Conference Machine Learning. 2018.
Xinyu Wang, Isil Dillig, and Rishabh Singh. Program synthesis using abstraction refinement. Pro- ceedings of the ACM on Programming Languages, 2017.
Alexander B Wiltschko, Matthew Johnson, Giuliano Iurilli, Ralph Peterson, Jesse M Katon, Stan L Pashkovski, Victoria E Abraira, Ryan P Adams, and Sandeep Robert Datta. Mapping sub-second structure in mouse behavior. Neuron, ((()):121-1135, 2015.
Li Yingzhen and Stephan Mandt. Disentangled sequential autoencoder. In International Conference
Machine Learning. 2018.
Eric Zhan, Albert Tseng, Yisong Yue, Adith Swaminathan, and Matthew Hausknecht. Learning cal- ibratable policies using programmatic style-consistency. In International Conference on Machine Learning, 2020.
Hui Zhang, Tu Bao Ho, Yang Zhang, and M-S Lin. Unsupervised feature extraction for time series clustering using orthogonal wavelet transform. Informatica, 30(3), 2006.
Yu Zhang, Peter Tiño, Aleš Leonardis, and Ke Tang. A survey neural network interpretability. arXiv preprint arXiv:2012. 14266, 2020.
Shengjia Zhao, Jiaming Song, and Stefano Ermon. Learning hierarchical features from generative models. In International Conference Machine Learning, 2017.
12


Reviews:
[{"review_id": "zHslqxab8Jc", "reviewer": "Reviewer_cxLd", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positives\n+ The general problem of learning programs to represent data in unsupervised learning is novel and potentially fundamental to machine learning and AI\n+ The proposed method is simple and interesting, and seems to achieve nice gains over purely neural approaches\n+ The paper is generally well written\n\nNegatives\n- My main issue is that the neuroscience data that the paper currently mainly evaluates on is not something a general ML audience might be very familiar with, and thus, it is hard to estimate what makes the task hard or easy. Could the authors give intuitions on why regular neural encodings fail to cluster in a better manner? Is it because of low SNR in the sequence data, or that RNN encoders pick up on spurious noise signals and thus do not yield a sufficiently discriminative embedding? If that is the case how would a convolutional encoder perform? Are the gains from the symbolic approach something that could just be achieved by a convolutional encoder? How much do we really need the symbolic approach and what are the inductive biases that we gain from them? I think these are important questions to answer in order to judge the efficacy of the proposed approach. (*)\n\n- In terms of the methodology, I do not understand why increasing the complexity of the programs gradually also leads to the purely neural part becoming less and less informative in the toy-dataset. Could it not have been the case that somehow the more complex program just learnt the features that the less complex program (with say length 2) learnt, and that in going from Fig. 3 d) to 3 e) we just learnt the same embedding space for the neural part? Is it encouraged that we learn more complex programs when we increase the length of the programs by tuning the corresponding gamma weights in Eqn. 5, or the corresponding channel capacities? (*)\n\n- I generally think such an encoding is potentially very useful for image data, and might be a big step towards better image representations and advances towards human-like intelligence. It would be great if the authors could comment on the potential for extending the approach to such applications and how far we might be from such use-cases. \n\nMinor Points:\n1) [A] is a closely related work that would be useful to cite in a neural-symbolic + amortized inference / VAE context.\n\n[A]: Vedantam, Ramakrishna, Karan Desai, Stefan Lee, Marcus Rohrbach, Dhruv Batra, and Devi Parikh. 2019. “Probabilistic Neural-Symbolic Models for Interpretable Visual Question Answering.” arXiv [cs.LG]. arXiv. http://arxiv.org/abs/1902.07864.", "summary_of_the_paper": "The paper proposes to learn a novel, interpretable neuro-symbolic encoder for sequences in an autoencoding framework. The key idea is that one learns both a symbolic as well as a neural encoder, and gradually makes the symbolic encoder more and more structured progressively increasing the complexity of it. Given such an encoder, the paper proceeds to show that the encoding from it is useful for clustering sequence data and demonstrates gains over other “unstructured” purely neural approaches.\n", "summary_of_the_review": "My main concerns with the paper are around whether the RNN encodings are overfitting and picking up on spurious correlations in sequence data that say a convolutional encoder might already fix, which will mean that one does not really need symbolic encoders for the current task. This is an even larger concern in light of the fact that a general machine learning researcher might not have a lot of great intuitions about the neuroscience sequence data and that makes it hard to assess the impact of the work. For the rebuttal I would encourage the authors to address concerns marked with (*) in the main review.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "XPeyY1pU_w3", "reviewer": "Reviewer_zt4F", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is clearly written and the idea was easy to understand and\nfollow. The experiments are always going to be a challenge in a\nspace like this as how can we say if the symbolic latent\nrepresentation meaningfully captures something in the data.\nThe synthetic result while a little contrived is convincing that\nat least in a controlled setting a reasonable program is produced.\nI wish there was more exploration of how often are sensible\nprograms learned and how sensitive this is to the choice of DSLs.\nMaybe something comparing the programs of experts to what\nthe latent representation learned? Basically the program equivlaent\nof showing a grid of faces.\n\nThis would also benefit from comparisons that use purely the neural component\n\nI am also curious empirically how it's decided how many symbolic\nprograms should be in the latent representation? Would a similar\nprocess be used for real-world data?\n\nMy main concern is how heavily this paper relies on the NEAR\nwork particularly for learning the symbolic encoder and even\nthe experimental data used. As the VAE bits are fairly standard\nit makes the work feel fairly incremental.\n", "summary_of_the_paper": "This paper is a clean extension of the supervised neurosymbolic approach\nof Shah et al. to the unsupervised setting where a VAE is used\nto encode input to the system as an interpretable program.\nThese latent representations are shown to be interpretable for\na synthetic experiment and useful for downstream tasks.\n\t", "summary_of_the_review": "This is an interesting and novel way to learn programs as symbolic encoders of the input. I have some nagging concerns about how incremental the contribution given how much this work relies on Shah et al.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "MyK3_gcPvNF", "reviewer": "Reviewer_a6JR", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think the idea of utilizing self-supervised training task to generate symbolic programs is very interesting and worth exploring.\n\nMy major concerns lie in the approach and evaluation.\n\nFor approach:\n1) Though I read the paper carefully, I still didn't fully get how the authors represent the program via z(alpha, fi). It seems that the output of VAE should be a vector, and how did the authors decode z(alpha, fi) to alpha and fi, especially when alpha should be a \"discrete program architecture\". Is alpha a program or just a single operator? Also how did you encode q(alpha, fi) when alpha could be a complex tree-structured program?\n2) I'm also not fully understand equation 3. How to you search the child program of alpha in a differentiable manner? Why this is a supervised task (from my understanding it's more like finding a child program that outputs similar encoding vector as parent program).\n3) You mention you \"repeat steps 1&2 until q(apha, fi) is fully symbolic\". The meaning of fully symbolic is really confusing to me. What is the stopping criterion? Also, what's the intuition that you need to update the parameter agrain after fixing program alpha?\n4) The authors didn't mention how each function in the DSL is implemented in paper and appendix. I think it's very important to understand the whole architecture. Also, it's important to know how you handle the non-differentiable operators, such as if-then, and others.\n\n\nFor evaluation:\n1) Current evaluation seems not very convincing to me. The authors only show that with the help of symbolic program, the method could get representations with better cluster quality (program helps representation learning). But I think a more intersting perspective is to see whether the learned program itself is helpful. For example, whether it could be used to predict future trajectory (such as 3-body problem), or even help solving some high-level reasoning tasks.\n2) Lack of the ablation study of the proposed framework. For example, why it's important to update the parameter again. Is the design of DSL influences the final representation learning, etc.", "summary_of_the_paper": "This paper tries to use VAE-based generative task to generate a latent representation that could be encoded as symbolic program. The authors show that in this way it could achieve better cluster results and also generate interpret-able and reasonable programs for each data.", "summary_of_the_review": "This paper proposes a very interesting research direction, but the writing and organization of the proposed method make it hard to understand it. In addition, the current evaluation is way too simple and not interesting enough. I highly recommend the authors to add some experiments to show that the learned programs can help some down-stream tasks.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "I_DmoVDizn", "reviewer": "Reviewer_G4ix", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The strength of this paper is described above. Although the concept and method are simple and new, and the paper itself is well-written, the significance in the problem setting and experimental results were unclear to me at this stage. The specific comments are as follows. \n\nMajor concerns: \n1. Although the unsupervised neurosymbolic “encoder” is new (as described in Sec. 5, but the decoders existed in [Feinman & Lake, 2020] and also the following [1]?), but I did not understand why the authors considered that creating the encoder is important.\n2. Although the concept of the proposed method is general and interesting, the applied domains were limited to animal/human trajectories. That is, it is unclear to me why the authors selected these domains to verify the proposed method. Or verification using other domain data may be also required.\n3. The significance of the experimental results was unclear to me. The task to extract distinctive information between the data w/ and w/o interaction in mice may be considered less challenging for me. The basketball task including offensive/defensive players is more challenging, but the result of the proposed method was overall comparable with the baselines (whereas it improved slightly with respect to purity). In the basketball task, I am also concerned about the importance of offensive/defensive players detection (this is usually given). The final downstream task (not tasks) is a classification task, and the results were comparable with the previous work (Sun et al. 2021b). \n\nMinor comments:\n4. What is x in Sec. 2.2? If it is some inputs, what types of inputs?\n5. Which is correct, .428 in NMI of Figure 4 or .423 of Table 1? Or What DSL was used in Table 1 results? \n\n[1] Halley Young, Osbert Bastani, Mayur Naik, Learning Neurosymbolic Generative Models via Program Synthesis, ICML, 97:7144-7153, 2019.", "summary_of_the_paper": "The authors propose an unsupervised approach to train neurosymbolic encoders to obtain a programmatically interpretable representation using the dictionary of a domain-speciﬁc language (DSL). The experimental results show that the proposed method can outperform baseline neural encoders in extracting semantically meaningful representations of behavior in CalMS21 (mice) dataset. The results also show that the performance can be robust across different DSL designs by domain experts.\n\n\n\n\n", "summary_of_the_review": "Although the concept and method are simple and new, and the paper itself is well-written, the significance in the problem setting and experimental results were unclear for me. Therefore, it is difficult for me to provide a higher rating at this stage. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors develop a technique for unsupervised learning of neurosymbolic encoders. Some of the difficulty with the paper came from the accessibility to a broader machine learning audience, though there is related work such as Shah 2020 in machine learning. The other difficulty came from the experiments: there was both a question about the metrics and the task. Quoting a reviewer

"Current evaluation seems not very convincing to me. The authors only show that with the help of symbolic program, the method could get representations with better cluster quality (program helps representation learning). But I think a more intersting perspective is to see whether the learned program itself is helpful. For example, whether it could be used to predict future trajectory (such as 3-body problem), or even help solving some high-level reasoning tasks."

and another reviewer 

"Maybe something comparing the programs of experts to what the latent representation learned?"

Making the paper more accessible and improving the experiments will improve its quality.
Decision: Reject