Unsupervised Hierarchical Concept Learning
Authors: Sumegh Roychowdhury
Sumedh Anand Sontakke
Mausoom Sarkar
Nikaash Puri
Milan Aggarwal
Pinkesh Badjatiya
Balaji Krishnamurthy
Laurent Itti
Venue:  (2021)
Keywords: hierarchical learning
unsupervised learning
unsupervised hierarchical learning
video representation learning
learning from demonstrations

Abstract: Discovering concepts (or temporal abstractions) in an unsupervised manner from demonstration data in the absence of an environment is an important problem. Organizing these discovered concepts hierarchically at different levels of abstraction is useful in discovering patterns, building ontologies, and generating tutorials from demonstration data. However, recent work to discover such concepts without access to any environment does not discover relationships (or a hierarchy) between these discovered concepts.  In this paper, we present a Transformer-based concept abstraction architecture UNHCLE (pronounced uncle) that extracts a hierarchy of concepts in an unsupervised way from demonstration data. We empirically demonstrate how UNHCLE discovers meaningful hierarchies using datasets from Chess and Cooking domains. Finally, we show how UNHCLE learns meaningful language labels for concepts by using demonstration data augmented with natural language for cooking and chess.  All of our code is available at https://github.com/UNHCLE/UNHCLE


Under review as a conference paper at ICLR 2021
UNSUPERVISED HIERARCHICAL CONCEPT LEARNING
Anonymous authors
Paper under double-blind review
ABSTRACT
Concepts or temporal abstractions are an essential aspect of learning among humans. They allow for succinct representations of the experiences we have through a variety of sensory inputs. Also, these concepts are arranged hierarchically, allowing for efficient representation of complex long- song-horizon experiences. Analogously, here we propose model that learns temporal representations from long-horizon visual demonstration data and associated textual descriptions without explicit temporal supervision. Additionally, our method produces hierarchy of concepts that align more closely with ground-truth human-annotated events than several state-of-the- supervised and unsupervised baselines in complex visual domains such as chess and cooking demonstrations. We illustrate the utility of the abstracted concepts in downstream tasks, such as captioning and reasoning. Finally, we perform several ablation studies illustrating the robustness of our approach to data-scarcity.
INTRODUCTION
Consider video (Figure that demonstrates how to cook egg. Humans subconsciously learn con- cepts (such as boiling water) that describe different concepts (or skills) in such demonstrations Pammi ct al. (2004). These learned skills can be composed and reused in different ways to learn new concepts. Discovering such concepts automatically from demonstration data is a non-trivial problem. architecture that clusters long-horizon action trajectories into shorter temporal skills. However, their approach treats skills as independent concepts. In contrast, humans organize these concepts hierarchies where lower-level concepts can be grouped to define higher-level concepts Naim et al. (2019).
Shankar al. (2019) introduces
We extend the architecture in Shankar al. 2019) simultaneously discover concepts along with their hierarchical organization without any supervision.
We propose an end-to-end trainable architecture UNHCLE for hierarchical representation learning from demonstrations. UNHCLE takes as input a long horizon trajectory of high- ligg--immnssonal images demonstrating a complex task (in our case, chess and cooking) and the associated textual commentary and isolates semantically meaningful subsequences in input trajectories. We emphasize that it does not require temporal annotations which link subsequences in the trajectories of images to the free- flowing commentary, but instead, autonomously discovers this mapping. Therefore, this work takes step towards unsupervised video understanding of high-dimensional data. Our contributions can be summarized follows:
We introduce a transformer- architecture to learn a multi-modal hierarchical latent embedding space to encode the various concepts in long-horizon demonstration trajectories. UNHCLE abstracts these concepts (shown through visual qualitative analysis) without requiring any temporal supervision, i.e., it divides long-horizon trajectories into semantically meaningful subsequences, without access to any temporal annotations that split these trajectories optimally.
We show the quantitative effectiveness of learning high-level concepts in a hierarchical manner compared to learning them in isolation while outperforming several baselines on YouCook2 (Zhou et al. (2017)) and Chess Opening datase
We further introduce a mechanism to incorporate commentary accompanying demonstrations in UNHCLE and show improvements hierarchical concepts discovered.

---

Under review as a conference paper at ICLR 2021
Making pizza
Poaching eggs
UNHCLE
Kneading dough
Cracking eggs
Adding
Builing water
Figure 1: Overview of approach. UNHCLE learns hierarchical latent space of concepts describing long horizon tasks like cooking and chess gameplay.
We introduce Wimeearped IoU (TW-IoU), evaluation metric that compare the alignment of our discovered concepts and ground-truth events.
Existing approaches to representation learning for demonstrations or videos typically require signifi- cant supervision. Typically, loquence-to-sqquene architectures are trained on datasets segmented by humans. During inference, these architectures generate proposals for timestamps that segment the input trajectory into semantically meaningful sequences. These complex sowuencc-to-equeeee models require significant amounts of annotated data, making them costly to train them.
More generally, video and scene understanding is important research area with wide-ranging applications. Most recently, Chen et al. (2019) utilize semantic awareness to perform complex depth estimation tasks to acquire the geometric properties of 3-dimensional space from 2-dimensional images. Tosi et al. (2020) utilize similar semantic information for depth estimation, optical flow and motion segmentation. Boggust et al. (2019) attempt to ground words in the video, but apply significant supervision to synchronize them, requiring human intervention. We attempt to learn similar embeddings but do in completely unsupervised manner, not utilizing any of the temporal labels available.
The field of learning from demonstrations (Nicolescu & Mataric 2003) seeks to learn to perform tasks from a set of demonstrated behaviors. Behavioral Cloning is one popular scheme (Esmaili al. (1955)) AAtesson & Schaal (1997) and [Schaal (1997) show how agents can learn simple tasks like cartpole simply from demonstrations. Pastor et al. (2009) also study how robots can learn from human demonstrations of tasks. Peters et al. (2013) and Kober & Peters (2009) fit a parametric model to the demonstrations, NNckum et al. (2012). Murali al. (2016), and Meier al. (2011) 1) first segment trajectories into subsequence and then apply parametric models to each subsequence, More recently, Schmeckpeper et al. 2019) shows that agents learn to maximize external reward using large corpus of observation data, i.e., trajectories of states on a relatively smaller corpus of interaction data, i.c., trajectories of state-action pairs. Hierarchical task representations have been studied as well. Instead of treating demonstrations in flat manner, one may also infer the hierarchical structure. A few recent works attempt do so (Xu al. (2018); Sun al. (208)). or as task graphs (Huang et al., 2019). Both Xu et al. (2018) and Huang et al. (2019) address generalizing to new instances of manipulation tasks in the few-shot regime by abstracting away low-level controls. However, all of these approaches require environment i.e., a transition and reward function to learn from. On the contrary, humans show an ability to learn by watching demonstrations, which we attempt to replicate.
Temporal abstractions of action sequences skill/primitive learning is also related field. Eysenbach cet al. (2018), learn large number of low-level sequences of actions by forcing the agent to produce skills that are different from those previously acquired. However, due to the diversity bias, the agent results in learning many useless skills that cannot be used for any semantically meaningful task. Similarly, Sharma et al. (2019) attempt to learn skills such that their transitions almost deterministic in a given environment. These approaches also require access to an environment, whereas try to learn without environment.
2

---

Under review as a conference paper at ICLR 2021
CIReA
Observation Regeneration
OCAM
2
CIR
Transfermer
Encodiing Observation
CR
Traversing across concept hlerarchy
CR
Encoding Instructions
IC
ICAP Transfoemer
Trandormer
COR
Regeneration
Figure Overview of our approach. UNHCLE learns hierarchical semantically mmmaticalll-mmaannngfuu embedding space which allows to perform complex downstream tasks such as temporal concept segmentation and label prediction. Refer to Section 2.1 for details about the abbreviations used in the figure.
2 APPROACH
2.1 UNHCLE: UNSUPERVISED HIERARCHICAL CONCEPT LEARNING
Intuitively, we define concept as a short sequence of states which repeatedly occur across several demonstration trajectories. Concepts have upper limit on their length in time-steps. These concepts can be obtained from the images of demonstrations, denoted by 2, and from the associated textual description, represented by We also refer to them as Observational Concept (OC) and Instructional Concept (IC) respectively and the module that encodes these concepts is referred to as Observational Concept Abstraction (OCA) and Instructional Concept Abstraction (ICA) modules. Additionally, concepts are hierarchical nature- thus, lower-level and higher-level observational or image concepts denoted by = and ² respectivelyy Analogously, lower-level, and higher-level textual concepts are represented by aº and et respectively. Similarly, the higher-level and lower-level modules are denoted by low high for the corresponding level.
Once we obtain these concept abstractions across levels, we can also transform them across the levels using a Concept Regeneration (CR) Module transform low-level concepts high-level concepts and vice-versa. Instead of just traversing in the concept level hierarchy, we can also use the Concept Instruction Regeneration Module CIR to obtain the original instructions that map that concept in its respective concept modality.
Subsequently, we provide details about the different stages in our proposed technique.
Encoding Observation: Given a long horizon trajectory of demonstration images along with its associated textual description. UNHCLE is able to abstract a hierarchy of concepts from demonstration images. We first pass these input images through ResNet-32 (He et al. ((016)) to get sequence of image vectors S $ $1:m and the associated text is converted into word vectors W Wi.n using BERT-base (Devlin et al. (2018)). Observations combine to produce lower-level concepts whereas higher-level concepts are simply aggregations of such lower-level concepts. Thus, the Lower-level Observation Concept Abstraction module (OCAlow) is trained embed sequence of image vectors of video (81, 82...8m) into sequence of concept vectors (=f, ... ...+)) where <<< m such that =f.. OCAAoo low ($1:m). Subsequently. lower-level concepts combine together to form higher level of concepts using the Higher-level Observation Concept Abstraction module (OCA high) such that H OCA high (2f..)
Fhe

---

Under review as a conference paper at ICLR 2021
Encoding Instructions: We also endeavour to discover these higher-level and lower-level concepts through natural language instructions. The Lower-level Instruction Concept Abstraction module (ICA (ow) and Higher-level Instruction Concept Abstraction module (ICA h are responsible for this functionality. From a corpus of words, (Wi, W2)...wn), the ICA low module generates concept i ......,,] d.), u << n. diuu = ICA low (WIn).. Subsequently the ICA high encodes the lower-level language concepts c/a, into higher level concepts as cH = ICA high (cf:u.)
Traversing across the concept hierarchy:
Learning concepts at different levels has added advantage of traversing these concepts in the concept hierarchy. This additionally allows us to utilize these hierarchical traversals obtain coarse or fine-grained concepts at any level. We thus regenerate the lower concepts from higher-level instruction concepts using the Lower-level Concept Regeneration Module (CR (ow) such that = low (cE.e). We can then later utilize this to obtain lower-level concepts from the higher-level concepts regenerate the demonstration images S = s in cross- cross-modal modal fashion.
Observation and Instruction Regeneration: Under a concept, the sequence of frames is nearly deterministic i.c. the knowledge of a concept uniquely identifies the accompanying sequence of images in the demonstration trajectory. Subsequently, we regenerate the demonstration image vectors s = s from lower-level concepts using Lower-level Concept Observation Regeneration Module (COR!o") such that s COR (z We also regenerate the demonstration image vectors S" fi.m from higher-level concepts abstracted from language using Higher-Ievel Concept Observation Regeneration Module (COR high) such that sY.m" = COR high (cf.v) in similar
cross-modal manner.
Finally, inspired by humans who can easily describe concept representations using free-flowing natural language, we first regenerate lower-level concepts from higher-level observation concepts using the Lower-level Concept Regeneration Module (CR such that o CR Low (>f.x) and subsequently, regenerate the word vectors WL = wfi from lower-level concepts using Lower-level
Concept Instruction Regeneration Module (CIR (o") such that wi:n = CIRloW (c(t. ). Additionally,
the higher-level concepts identified by the OCA high module from demonstration frames are also
described using a meaningful free-flowing commentary by the Higher-level Concept Instruction Regeneration module or high Thus, we regenerate the word vectors W = w from higher- level concepts using CIR high such that wf = CIR high (z1..).
CiRhigh
2.2 SOFT-DYNAMIC TIME WARPING (SOFT-DTW)
The most significant challenge that our work overcomes that we do not require supervision for hierarchical temporal segmentation i.c. we do not require annotations which demarcate the beginning and ending of a concept, both in language and the space of frame's s timestamps. Currently, most approaches rely on annotation describing the frame start and end for an event and the associated text for this event. Our architecture is trained using free-flowing commentary without event demarcations. This is achieved using the loss function we implement. We utilize Soft-DTW (Cuturi & Blondel (2017) loss to align the reconstructed trajectories with the input long-horizon trajectory. So given (2/, ... Vmm, the soft-D (x, y) is computed as, two trajectories z (I1, T2, ...Zn) and y (y1, 3/2
Soft-DTW(zz (z. y) = min? { (AA, A A(x, v)), A}
where A E An,m is the alignment matrix, A v) [6(2;, villss E Raxm and d being the cost function. min" operator is then computed as,
min isn ai, min {a . = [ log Σi= -a:/7,
γ 0.
(2)
y 0.
For our experiments, we use L2 distance as 5 and = 1.
2.3 LEARNING OBJECTIVE
Our approach that has been outlined in Figure 2 uses several loss terms between different network outputs to achieve our objective. The soft-DTW is used between several sequences as follows.
Layn = soft- soft-DTW(esd
(2f., c + Soff-D JL + soft-DTW(ssmm Hi.m.) + soft-DTW (81:mm $1mm"
4

---

Under review as a conference paper at ICLR 2021
We the Negative Log-Likelihood (NLL) loss (Cnit) between the generated comment vectors and the BERT vectors and use the Lg loss between concepts from different modalities as follows,
Estatic = -Lnl( win) + Lnu( (w (H)
We then define our final loss as, Leoral = a Ldyn + B Latatic
2.4 EVALUATION METRICS
The ground-truth events in the dataset and the concepts generated by UNHCLE may differ number, duration, and start-time. To evaluate the efficacy of UNHCLE in generating concepts that align with the annaaaadd events in our dataset, it is imperative that we utilize metric that measures the overlap between generated concepts and ground truths and also accounts for this possible temporal mismatch. To this end, we introduce time-warp IoU. Note that we do not utilize these the temporal fround-truth event annotations during training, but only to compare the abstraction generated by architecture to the human-annotated events.
Consider the search series S (81, $2, 83)..8M) E S and target series T (41, l2, lg...lx) where S corresponds to the end-of-concept time stamp for each concept as generated by UNHCLE for single long-horizes demonstration trajectory. Thus, the ith concept abstracted from UNHCLE starts at time 5;-1 and end at time = Similarly. I corresponds to the end-of-event time stamp for each ground-truth event the demonstration trajectory, where the jth ground truth event starts at time f--1 and ends at time t.. Note that both sy and to are equal to zero i.e. we measure time starting at zero for all demonstration trajectories.
2.4.1 TIME-WAR WARPED IOU
We caleulate the warped alignment between between the event annotations and the concepts from UNHCLE. This implies calculating A(S. solving the following optimization problem (Berndt Clifford (1994)) ASS, T) minpep Em.ep (m tn). where the S and T correspond the search target series respectively and 6 corresponds distance metric (in our case the L2 norm), measuring time mismatch.
(S, therefore corresponds the trajectory discrepancy measure defined as the matching cost for the optimal matching path P among all possible valid matching paths P (i.e. paths satisfying monotonicity, continuity, and boundary conditions). From this optimal trajectory obtained we can also get the warping function W such that W (s;)) = t5. i.e. we find the optimal mapping between the 5th concept ending at time = S; and the jth event ending at time = ty. To quantify the quality of this mapping we introduce Time- Warp Intersection over Union for a single long- morizon trajectory S.
Enj:W(,))= min(ks, 85) - TW - IS) = Σ max (+))==t (max((;, 8;)} min,, W (s,)=t, (min(li-1,8) 18
(3)
Intuitively, this corresponds to the overlap between the jth concept ending at time s; and the jth event ending at time = tj.
3 EXPERIMENTS
3.1 DATASETS
In order to find conceptual temporal patterns from videos in a hierarchical manner, we experiment with two datasets from different domains.
YouCook2 (Zhou et al. (2017) dataset comprises of instructional videos for 89 unique recipes (~22 videos per recipe) containing labels that separate the long horizon trajectories of demonstrations into events with explicit time stamps for the beginning and end of each event along with the associated commentary. It contains 1,333 videos for training and 457 vidcos for testing. The average number of segments per video is 7.7 and the average duration the video 5.27 minutes.
Recommending Chess Openings dataset consists of opening moves in the game of Chess. An
5

---

Under review as a conference paper at ICLR 2021
Opening in Chess is fixed sequence of moves which when performed leads to final board state putting the player in a strategic position in the game. Commonly used chess openings are each labeled with name (Figure B shows examples). The dataset contains 20,058 openings with each containing a sequence of chess moves and it's corresponding opening and variation labels. The train-test split used for our experiments is 80-20.
3.2 IMPLEMENTATION DETAILS
The Observational Concept Abstraction (OCA) and Instructional Concept Abstraction (ICA) modules consist of the Transformer (Vaswani et al. 2017) Encoder with 8 hidden layers and 8-Head Attention which takes as input, sequence of frames positionally encoded and outputs a hidden vector. It is then passed through Transformer Decoder again having 8 hidden layers and encoder-decoder cross- attention with masking which finally outputs the concepts which are latent variables having dimension conceptlength 768. We use a 1-layer GRU (Chung et al. (2014) for our Concept Instruction Regeneration (CIR), Concept Observation Regeneration (COR), and the Concept Regeneration (CR) module which converts the high-level concepts to low-level concepts. For encoding the image frames, down-samplc all videos to 500 frames and use ResNet-32 (pretrained on MSCOCO dataset) (He) ict al. (2016) for dense embedding of dimension 512 x 1. We further down-sample to 200 frames per trajectory for all experiments. Comments encoded using BERT-base pre-trained embeddings with a 768 hidden dimension.
For our low-level concepts, keep the maximum number of concepts discovered 16 and for high-level concepts as 4. These assumptions are based on the YouCook2 dataset statistics wherein ground-truth annotations (not used during training) the minimum number of segments were 5 and the maximum as 16. We keep a = and / = for all our experiments along with a batch-size of 128 and use Adam optimizer for training with a learning rate of le and train the network for 100 epochs.
3.3 RESULTS AND ANALYSIS
3.3.1 VISUALIZING HIERARCHY
Here we analyze whether the discovered concepts are human interpretable i.c. the temporal clusters within a single demonstration semantically meaningful?. We find that our architecture abstracts several useful human interpretable concepts without any supervision. Figure 3 and 4 shows the results obtained. These abstracted high-level concepts align well with the ground truth event labels provided in YouCook2. We also additionally find that our model able to split each ground truth event into lower-level concepts. For instance, in a pasta-making demonstration in YouCook2, single event corresponding the description "heat a pan add spoon oil and prosciutto it". UNHCLE is divided into low level concepts corresponding to "heat pan", " "add oil" and "prosciuto". Note that no explicit event time labels were provided to UNHCLE, indicating that our model is able to abstract such coherent sub-sequences, thus taking the first step towards video understanding.
We also visualize the concepts abstracted by UNHCLE when trained on chess opening data, The concepts learnt here also produce coherent, human- intmann-tterpretable results.
Blackmar-Diemer Gambit
Pletrowsky Defense
Low-level skill 1
Low-level skill 2
Low-level skill
Figure 3: Hicrarchy of concepts discovered by UNHCLE using openings data in Chess. At high level, UNHCLE correctly identifies concepts corresponding to the Blackmar-Diemer Gambit and the Pictrowsky Defense, At low level, identifies concepts such "d4 d5., and e4 d3 that are used across several openings

---

Under review as a conference paper at ICLR 2021
heat a pan add I spoon oil and prosciatto to it
add marinara sauce and some cream to the pan and cook for seconds
pull the pan off fire and add some vodka
strain some boiled pasta the sauce
Figure 4: Example Hierarchy of concepts discovered by UNHCLE on the YouCook2 dataset
3.3.2 COMPARISON WITH BASELINES
In this section, we evaluate the performance of UNHCLE quantitatively on YouCook and quantify its ability to generate coherent concepts that align with the human annotated ground truths using approach with 4 baselines.
(1) Random baseline predicts segment randomly on the basis of uniformly sampled timestamps for a given video on the basis of its duration. (2) EQUALDIV consists of dividing the video into conceptual segments of equal duration (3) GRU-supervised: Further we also consider a naive supervised baseline comprising of a GRU Cho et al. (2014) based encoder that sequentially processes the Resnet features corresponding to frames in a video followed by a decoder GRU that attends BBahdanau et al. (2014) on encoder outputs and is trained sequentially predict end timestamp of each meaningful segment (variable in number) in the video. (4) FLAT w/o comment: We implement the Shankar et al. (2019) approach which takes as input, a sequence of video frames and discovers single level of concepts without any hierarchy. (5) FLAT w/ comment: This is a modification of the above baseline where we also decode comments from the single-level concept discovered and use that to give the model an extra signal which will help it in discovering better aligned concepts. See Figure 6(C) (in Appendix) (6) Clustering: Given an input sequence of frames, we define the weight function based on their temporal position in the sequence and also the L2 distance between the frame embeddings. Then we use standard K-means algorithm (K=4) to cluster the frames based on the weighting function defined and use the clusters formed to predict the temporal boundaries. (7) GRU_Segg Instead of predicting end time stamps of each segment (as in GRU-superiised), the decoder is trained to predict/assign identical ids to frames which are part of the same segment. Further, the model's decoder is trained to assign different ids to frames part of different segments while frames not part of any meaningful segment in the ground truth are trained have default null id O. During inference, continuous subsequence of frames predicted to be having same id are considered as part of one segment and different predicted segments are extracted accordingly (frames predicted be having null ids are ignored).
the TW-IoU metric. We compare
For baselines, we predict only one level of concepts and hence refer to them as high-level concepts. Table Ta summarises and compares the TW-IoU computed between ground truth time stamp anno- tations and predicted/discoverr compared to all other baselines. It can be noted that discovering only one (high) level of concepts (FLAT) results in low TW-IoU (for High-Level Concepts). However, UNHCLE discovers concepts that align better with the ground truth events (UNHCLE performs ~ 23% better) compared to FLAT. Further, the GRU-supervised baseline performs better than FLAT but UNHCLE outperforms (~ 15%) the GRU-supervised baseline which shows that predicting such concepts is difficult even for supervised baseline. Even though supervised GRU_See achieves better TW-IoU of 53.1 compared to high level concepts discovered by UNHCLE w/o comment as well as UNHCLE with comment,
segments. As can be seen, UNHCLE achieves the best To-
UNHCLE with comment (which is unsupervised) performance (TW-IoU 47.4) still comparable with GRU_Sec However, TW-IoU corresponding to low level concepts discovered by UNHCLE is better than GRU_See
We use YouCook2 since each recipe comprises of multiple conceptual segments unlike Chess dataset (each game just an opening variation) which instead use for evaluating label prediction ability later.
7

---

Under review as a conference paper at ICLR 2021
Method
TW-IoU
Random
28.2
EQUALDIV
GRU-supervised
22.8
TW-IoU
Frames Used
FLAT wfo comment
14.4
high
low
FLAT w/ comment
14.8
200
37.4
58.7
Clustering GRU_Ses
32.2
33.4
34.0
53.1
32
13.1
12.7
high-level 37.4
low-level
(b)
UNHCLE w/o comment
58.7
UNHCLE w/ comment
47.4
59.7
Table I: (a) TW-IoU scores for single level concepts predicted by the baselines along with the TW-IoU scores for both the high-level and low-level concepts for predictions from proposed technique UNHCLE (b) Comparison showing the trade-off between number of down samm-dd frames and TW-IoU UNHCLE
3.3.3 EFFECT OF GUIDANCE THROUGH COMMENTARY
In this section, we discuss the improvement in the quality and precision of conceptual segments discovered by UNHCLE using video commentary to guide our model training YouCook2 dataset. To study this, we compare UNHCLE without comment which discovers concept hierarchy using just frame observations in video against UNHCLE with comment which additionally uses commentary as guide (as in Figure 2) Table La compares the TW-IoU of segments obtained corresponding to High Level Concepts for each of these two models. As can be seen, using commentary significantly improves the TW-IoU by ~ 10% which establishes that using commentary enables UNHCLE to detect precise boundaries for segments corresponding to various concepts. However, we find only a marginal improvement of ~ 1% in TW-IoU corresponding to low level concepts. Further, it can be seen in Figure that using commentary allows the model to better understand how to merge lower-level concepts appropriately into higher-level concepts "heat a pan, add spoon oil and prosciutto it" separately without merging with subsequent segments.
Heat pan add I spoon ell and prosclatto Add onlons batter chopped parsley and salt and stirrins, Pull add
Wait sauce some cream the pan
Garnish with parmesan 24
some secund..
Boil
Pour
2!
pas spoon prosciatto to it. Add onions butter chopped parsley and salt to and keep stirring.
Pull the pan off fire and add
Add marinars sauce and soms cream to the and cook for some seconds..
strain some bolled pasta pour the souce on it and garnish with parmesan
Figure In this figure, we show that using commentary as guide during training. UNHCLE Iearns to better combine low-level concepts to form high-level concepts which are better aligned towards our ground- annotations. A refers the concept segments discovered w/o comments and B refers to the one with comment.
3.3.4 LABEL PREDICTION TASK FOR CHESS OPENINGS
We further evaluate UNHCLE on the task label (name of opening + variation) prediction corre- sponding to hierarchical concepts discovered. Since recommending Chess Openings comprises of label for each sequence of moves in an opening. we train UNHCLE w/ commentary (Figure 2}. Since there max only two segments corresponding to opening move and variation move in the dataset we use label prediction accuracy as our metric here instead of TW-JoU. There are 300 distinct labels and our model achieves 78.2 % accuracy which shows that it is able to correctly discover the correct
8

---

Under review as a conference paper at ICLR 2021
segmentation between openings with different variations hierarchical concepts and associate them with the correct label in most of the test samples.
3.3.5 EFFECT OF SAMPLING RATE ON THE QUALITY OF HIERARCHY
For YouCook2 we down-sample 200 frames from the original 500 frames provided in the dataset and use that as our input to UNHCLE. We analyze the trade-off between time and performance if we further nown-sample the frames. This also provides us a better insight into how much granular information regarding the video do we need to be able to discover better hierarchies. Table Ib shows the results obtained for our experiment. Interestingly, we don't It observe a linear drop in pertormance by reducing the number of frames. Even with just 64 frames, the results beat the other baselines.
3.4 ABLATION EXPERIMENTS & VISUAL ORDERING TASK
We performed more ablation experiments to show the need for the modules and the losses used in our model. We removed the soft-DT (ea loss from our UNHCLE (w/o comment) model in Figure 6(A) (in Appendix) to highlight the importance of this loss that guides traversing back and forth across the concept hierarchy. This loss guides the alignment between the initially generated low-level concepts (=²) and the low-level concepts (c'*) extracted back from high-level concepts (a (z#). Removing this loss reduces the TW-IoU scores drastically thus proving it's necessity .See UNHCLE w/o comment w/o low-align loss in Table 2a
We also choose a simplified version of the UNHCLE w/o commentary model shown in Fig. 6(A) (in Appendix), where we remove the OCA modules and directly output z" which looks simpler, but this removes the extra alignment signal as mentioned in the above point. We see this results the drop of TW-IoU (Table 2a), thus confirming need for the modules used. We call this the Direct Hierarchy baseline for which the system diagram is also provided in Figure 6(B) (in Appendix).
To show the effectiveness of the concepts discovered by UNHCLE Ordering Task. The task is that given a sequence of frames as input, our trained model (with frozen weights) should discover the high-level concepts associated with those frames and can be used to predict whether or not the given sequence of frames are in correct/meaningguu order (binary classification), We use simple 1-layer GRU to do so. To create the training data for this we take examples from the YouCook2 (Zhou et al. (017)) dataset and randomly shuffle the sequence of frames creating 10 negative examples for each positive sample the dataset. We finally evaluate this on our testing data after performing the same steps mentioned above and show our results in Table 2b We report both the Accuracy and the F1 scores. As we can see that there a significant gain of 9% in the F1 score and 2% in Accuracy using our model UNHCLE over the FLAT baseline.
them to perform Visual
Model Variant
Metrics
high
low
Model Used
Accuracy
Fl
UNHCLE w/o comment
37.4
58.7
FLAT UNHCLE
88.9
2.13
Direct Hierarchy
20.3
28.6
90.9
14.9
UNHCLE w/o comment w/o low-align loss
18.9
28.7
(b)
Table 2: (a) TW-IoU scores for ablation experiments for both high-level and low-level concepts prediction (b) Comparison hetween the Shaanhaar et al. (2020) FLAT haseline and our proposed UNHCLE on the designed Visual Ordering Task.
4 CONCLUSION
We show that our approach (UNHCLE) discovers concepts and organizes them in meaningful hierarchy using only demonstration data from chess openings and cooking. We also show that this discovered hierarchy of concepts is useful in predicting textual labels and temporal concept segmentations for the associated demonstrations. would be interesting to use the concept hierarchy discovered by UNHCLE to generate a curriculum where lower-level concepts would be taught first followed by higher-level concepts. Another good direction would be to extend UNHCLE to learn different types of relationships between concepts without supervision.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Christopher G Atkeson and Stefan Schaal. Robot learning from demonstration. In ICML,, volume 97, pp. 12-20. Citeseer, 1997.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv: 409.0433 2014.
Donald J Berndt and James Clifford. Using dynamic time warping to find patterns in time series, In KDD workshop, volume 10, pp. 359-370. Seattle, WA, USA: 1994.
Angie W Boggust, Kartik Audhkhasi, Dhiraj Joshi, David Harwath, Samuel Thomas, Rogério Schmidt Feris, Danny Gutfreund, Yang Zhang. Antonio Torralba, Michael Picheny, et al. Grounding spoken words in unlabeled video. In CVPR Workshops, pp. 29-32, 2019.
Po-Yi Chen, Alexander H Liu, Yen-Cheng Liu, and Yu-Chiang Frank Wang. Towards scene un- derstanding: Unsupervised monocular depth estimation with semantic-aware representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2624-2632, 2019.
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724-177 1734, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.1115//1/D14-1117 URL https: www. aclweb
Junyoung Chung. Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
Marco Cuturi and Mathieu Blondel. Soft-DTW: a differentiable loss function for time-series. volume 70 of Proceedings Machine Learning Research, pp. 894-903 International Convention Centre, Sydney, Australia, 06-11 Aug 2017. PMLR.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.
Nasser Esmaili, Claude Sammut, and GM Shirazi. Behavioural cloning in control of dynamic system. In 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21 Century, volume 3, pp. 2904-2909. IEEE, 1995.
Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine Diversity is you need: Learning skills without a reward function. arXiv preprint arXiv: / Ne0:.060700 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
De-An Huang, Suraj Nair, Danfei Xu, Yuke Zhu, Animesh Garg, Li Fei-Fei, Silvio Savarese, and Juan Carlos Niebles. Neural task graphs: Generalizing to unseen tasks from a single video demon- stration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8565-8574, 2019.
Jens Kober and Jan Peters. Learning motor primitives for robotics. In 2009 IEEE International Conference on Robotics and Automation, pp- 21 18. IEBE, 2009.
Franziska Meier, Evangelos Theodorou, Freek Stulp, and Stefan Schaal. Movement segmentation using primitive library. In 201 IEFF/RSJ International Conference on Intelligent Robots and Systems, pp. 3407-3412. IEEE, 2011.
Adithyavairavan Murali, Animesh Garg, Sanjay Krishnan, Florian T Pokorny, Pieter Abbeel, Trevor Darrell, and Ken Goldberg. Tsc-dl: Unsupervised trajectory segmentation of multi-moda surgical demonstrations with deep learning. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 4150-4157. IEFE, 2016.
10

---

Under review as a conference paper at ICLR 2021
Michelangelo Naim, Mikhail Katkov, Stefano Recanatesi, and Misha Tsodyks. Emergence of hierarchical organization in memory for random material. Scientific Reports, 91):1-110, 2019.
Monica N Nicolescu and Maja Mataric. Natural methods for robot task learning: Instructive demonstrations, generalization and practice. In Proceedings of the second international joint conference on Autonomous agents and multiagent systems, pp. 241-248, 2003.
Scott Nickum, Sarah Osentoski, George Konidaris, and Andrew G Barto. Learning and generalization of complex tasks from unstructured demonstrations. In 20/2 JEEE/RSJ International Conference Intelligent Robots and Systems, pp- 5239-5246. IEEE, 2012.
VS Chandrasekhar Pammi, Krishna P Miyapuram, Raju S Bapi, and Kenji Doya. Chunking phe- nomenon complex sequential skill learning humans. In International Conference on Neural Information Processing, pp. 294-299. Springer, 2004.
Peter Pastor, Heiko Hoffmann, Tamim Asfour, and Stefan Schaal. Learning and generalization of motor skills by learning from demonstration. In 2009 IEEE International Conference on Robotics and Automation, pp. 763-768. IEEE, 2009.
Jan Peters, Jens Kober, Katharina Mülling, Oliver Krämer, and Gerhard Neumann. Towards robot skill learning: From simple skills to table tennis. In Joint European Conference Machine Learning and Knowledge Discovery in Databases, pp. 627-631 Springer, 2013.
Stefan Schaal. Learning from demonstration. Advances in neural information processing systems, pp. 1040-1046, 1997.
Karl Schmeckpeper, Annie Xie, Oleh Rybkin, Stephen Tian, Kostas Daniilidis, Sergey Levine, and Chelsea Finn. Learning predictive models from observation and interaction. arXiv preprint 1912 2019.
Tanmay Shankar, Shubham Tulsiani, Lerrel Pinto, and Abhinav Gupta. Discovering motor programs by recomposing demonstrations. In International Conference on Learning Representations, 2019.
Tanmay Shankar, Shubham Tulsiani, Lerrel Pinto, and Abhinav Gupta. Discovering motor programs by recomposing demonstrations. In International Conference Learning Representations, 2020. URL htt.hhtpps///ppeneeeieeee wtrprlermeeeeeeorueemmmm ld=rkgHYONYKKW
Archit Sharma, Shixiang Gu, Sergey Levine, Vikash Kumar, and Karol Hausman, Dynamics- unsupervised discovery of skills. arXiv preprint arXiv: /90700167, 2019.
Shao-Hua Sun, Hyeonwoo Noh, Sriram Somasundaram, and Joseph Lim. Neural program synthesis from diverse demonstration videos. In International Conference Machine Learning, pp. 4790- 4799, 2018
Fabio Tosi, Filippo Aleotti, Pierluigi Zama Ramirez, Matteo Poggi, Samuele Salti, Luigi Di Stefano, and Stefano Mattoccia. Distilled semantics for comprehensive scene understanding from videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
4654-4665, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Danfei Xu, Suraj Nair, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, and Silvio Savarese. Neural task programming: Learning to generalize across hierarchical tasks. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1-8. IEEE, 2018.
Luowei Zhou, Chenliang Xu, and Jason Corso. Towards automatic learning of procedures from web instructional videos. arXiv preprint arXiv: 1703.09988 2017.
A MORE DETAILS
We provide all the modified system diagrams for all the ablation experiments mentioned Section 3.4 for clarity.
11

---

Under review as a conference paper at ICLR 2021
Observation Regeneration
Urundeesner
OCANO Transferterr
hn
CHI
A
Encodiigg Observation
Traversing across the concept hierarchy
Observation Regeneration
Encoding Observation
e
Traversing across the concept hierarchy
Instruction Regeneration
DC Aft Treesformner nner
Encoding
Observation Rageneration
Figure 6: Three model variants are shown here. (A) refers to UNHCLE w/o comment. (B) refers to the Direct Hierarchy variant where we directly predict the high-level skill using OCA transformer. (C) refers to FLAT baseline with comment.
French Defense: Advance Variation
Paulsen Attack
Low-level skill
Low-level skill 2
Low-level skill
Low-Jevel skill
Figure 7: Skills discovered in the French defence.
12

---

Under review as a conference paper at ICLR 2021
Sicilian Defense: Mongoose Variation
Low-level skill
Low-level skill
Low-level skill
Figure 8: Skills discovered in the Sicilian opening.
Nimzowitsch Defense: Kennedy Variation
Linksspringer Variation
Low-level skill
Low-level skill
Low-level skill 3
Figure 9: Skills discovered in the Nimzowitsch Defense: Kennedy Variation Linksspringer Variation opening.
Slav Defense: Exchange Variation
Low-level skill
Low-level skill 2
Figure 10: Skills discovered in the Slav Defense: Exchange Variation opening.
13

---

Under review as a conference paper at ICLR 2021
Philidor Defense
Low-level skill 1
Low-level skill 2
Low-level skill 3
Figure 11: Skills discovered in the Philidor Defense opening.
14


Reviews:
[{"review_id": "w77fzbg5y1", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper addresses the problem of extracting a hierarchy of concepts in an unsupervised way from demonstration data. The authors present a Transformer-based concept abstraction architecture called UNHCLE and show how it discovers meaningful hierarchies using datasets from Chess and Cooking domains. In particular, the model is designed to function without specific temporal supervision, which makes it potentially practical for real-world applications. \n\nPros.\n+ problem clear stated with potential practical implications\n+ experiments on real-world video datasets\n\nCons.\n- The results are evaluated using a time-warping based metric. I am not convinced this metric provides a faithful assessment of the capability of the model. The exact temporal boundaries are also important in discovering the specific concept and ground them back to the visual observations. TW-IoU will not take this into consideration. \n- The novelty of the proposed model is limited. The two-level hierarchy has been seen a lot in relevant fields such as video recognition. So the concept by itself is not new. Introducing a two-level hierarchy into concept learning is also not new. In fact, I find it somewhat arbitrary that the authors decide to set the number of hierarchies to two. The cooking video dataset does not have such annotations so hypothetically the one can set an arbitrary number for the levels of hierarchy. Can the authors explain why this two-level hierarchy is chosen and how it can generalize to other tasks?\n- Comparison to works in unsupervised clustering, video segmentation, and change point detection. While the authors form this work as concept learning, the problem nevertheless relates to these research topics. The authors may need to look upon those for a better variety of baselines and also evaluation metrics. \n- Experiment results are weak. The proposed model's performance does not exceed the simple equal division baseline a lot when 64 frames are sampled for each video.", "extra_scores": {}, "extra_text": {}}, {"review_id": "IvbTvxmrZLQ", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper addresses a relatively new topic to learn the hierarchical concepts in videos and commentary in an unsupervised manner. The authors proposed a hierarchical and transversing encoder-decoder network architecture to tackle the problem, where the pre-trained ResNet32 and BERT are used as feature extractors, transformers and GRUs serve as the concept encoders and decoders. As the network is unsupervised, the starting time and ending time of each concept is quite crucial and the problem was tackled by training using a soft-DTW loss. A metric (time-wrapped IoU) is also proposed for quantitative evaluation. The experiments indicate the effectiveness of the network and the network was tested in different datasets and scenarios.\n\nI rate the paper as weakly accept and here are some questions.\n1. The lower-level concepts and upper-level concepts in both visual and language are a little vague without clear definitions. How do you define the level of concepts and how did you balance the levels in video and language? For example in Figure 5, the results without commentary (A) look more reasonable for z^H_1 to contain all the steps for a higher-level concept (say, pan heating) but the model with commentary (B) split the concepts mainly according to the language descriptions. This may also influence the evaluation scores as different work may have different \"flavors\" to select concepts and the starting/ending time could be different. \n2. Besides, since your network is trained across the video and language, how did you train the network without commentary part?\n3. You used two different evaluation metrics in two different datasets, which seems that you tend to focus on different aspects in different datasets. This is a bit of bias. Besides, the comparison with other work in the second dataset (Chess Openings) is not provided. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "DCaZznoJhL2", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper introduces the solution of an important task: hierarchical concept learning(or temporal abstractions) from demonstration data. Specifically, this paper considers 1) unsupervised setting; 2) the hierarchy of concepts, and conducts experiments in two datasets. However, there are some points in the experiment section to be discussed. \n\nThe pros of this paper include:\n- The task setting of this paper is important and applicable. Especially jointly learning concepts from frames and instructions while considering the hierarchy.\n- The proposed model utilizes multiple regeneration module for the concept learning, which may also help for other hierarchical-aware tasks.\n\nThe cons include (In my opinion, the main weakness is the experiments):\n- Some design of model (e.g., Traversing across the concept hierarchy, Observation, and Instruction Regeneration, etc) are not fully estimated in this section: are they really useful? which cases did they corrected? The experiments make the previous model design somewhat difficult to evaluate.\n- The selected baselines are also weak. Even the **Random**(Random seg) and **EQUALDIV**(Equal Divided) perform much better than **GRU-supervised**. I believe more powerful baselines should be proposed, otherwise, the contribution of the paper is hard to be recognized.", "extra_scores": {}, "extra_text": {}}, {"review_id": "7pVvjXOHa9", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a method to unsupervisedly discover structure in unlabeled videos, by finding events at different temporal resolutions.\n\n### Strengths:\n- The paper focuses on the important problem of exploiting weakly labeled video data, by exploiting its structure, for example by recovering temporal structure in an autoencoder fashion.\n- Use of multiple modalities to cross-supervise each other.\n- Code is available.\n\n### Weaknesses:\n- The concept of hierarchy is not well defined or well motivated. While most hierarchical papers refer to hierarhies of concepts, the hierarchy considered in this paper is much weaker as a hierarchy, and it refers to subactions within longer actions (not actions that are specific instances of more abstract actions). While this way of understanding hierarchy can be valid, it is never explained or motivated in the paper, or even compared to the standard way of understanding it. \n- The overall motivation for the method has a lot of gaps. For example:\n\t- Regeneration of low level concepts from high level concepts: what are we expecting from a network that moves from a \"high in the hierarchy\" concept to a \"low in the hierarchy\" concept? Should we expect the network to randomly select one subconcept? The specific information is not there, the problem is ill-defined (for example, we can go from \"cat\" to \"animal\", but not from \"animal\" to \"cat\"). How are we expecting any reconstruction? The paper does not provide any justification or intuition.\n\t- Why are the authors using those specific pairs in the L_dyn term? (last line of page 4 -- I suggest adding equation numbers). Apart from no motivation, there are no ablations showing that those are the correct pairs.\n\t- Why is the \"low\" case even necessary? Wouldn't it be possible to train only with the \"high\" one? This would probably imply rethinking some of the losses, but overall the method would look very similar. This is, the idea of hierarchy would disappear, but this idea is not used in the experiments anyway.\n\t- What is the motivation for the two modalities? I can understand it can help, but it is not central to the method. This is not necessarily bad, but it requires some explanation.\n- The explanation revolves around demonstration data. It is unclear why demonstration data is important for this method, and why it is not general for any human action. The introduction explaining demonstrations in a robotics scenario does not feel related to the content of the paper. For example, a lot of stress is given to \"agents\" interacting in \"environments\".\n- Some terms introduced in the paper would benefit from a change. For example, a \"concept\" in the paper is actually an \"event\", not a \"concept\". This is more in line with the literature, for example the dataset they use labels that as \"event\".\n- Results on chess are hard to believe. Do the authors think that the system has really learned (unsupervisedly) to identify interesting openings? It could instead be learning strong biases like length of openings in the dataset.\n- Quantitative results are not convincing. How does FLAT and even the supervised method perform much worse than the two trivial baselines (random and equal division)? Does FLAT use text data?\n- Conceptual assertions that I do not believe to be true: \"under a concept, the sequence of frames is nearly deterministic\". This is not true, there are nearly infinite ways of having a sequence of frames (video clip) depicting how to crack an egg, for example. Different background, different way of performing the action, different elements in the scene, speed of the action, point of view, etc. This is related to the \"regeneration\" point above.\n\n### Additional comments and questions:\n- Figure 2 is hard to understand. What is it exactly representing? What should we learn from it?\n- Does the algorithm have any \"motivation\" to not predict always a single concept per sequence?\n- What is the relationship between u and v?\n- Have you tried smaller networks? 8 layers and 8 heads just for the Encoder seem like a very big model for such a relatively simple text.\n- In the first paragraph of page 8 the paper mentions that there is only marginal improvement in low level concepts. How are these evaluated? As far as I could understand, there was only GT available for the high level ones.\n\n### Final recommendation\nOverall, I believe the paper as it stands is not ready to be presented to ICLR and I recommend a rejection.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Although the paper studies a relevant and important topic, which is about learning of hierarchy of concepts in an unsupervised manner, the reviewers raised several critical concerns. In particular, although the hierarchical structure of concepts is the key idea in this paper, the concept of hierarchy itself is not well explained. How to define the hierarchical level of concepts should be carefully and mathematically discussed. In addition, empirical evaluation is not thorough as reviewers pointed out. Although we acknowledge that the authors addressed concerns by the author response, newly added results are still confusing and more careful treatment is needed before publication. I will therefore reject the paper. 

This work reminds me the the topic called "formal concept analysis" (e.g. see [1]), which mathematically defines concepts as closed sets and constructs a hierarchy of concepts in an unsupervised manner. This method can be viewed as co-clustering and also has a close relationship to closed itemset mining. This approach is used in machine learning (e.g. [2]). I think it is beneficial for the authors to refer such existing and well-established approaches to elaborate this work further.

[1] Davey, B.A., Priestley, H.A.: Introduction to Lattices and Order, Cambridge Univ. Press (2002)  
[2] Yoneda, et al., Learning Graph Representation via Formal Concept Analysis, 	arXiv:1812.03395 
Decision: Reject