TYPE Original Research
PUBLISHED 30 September 2022
DOI 10.3389/frobt.2022.903450

OPEN ACCESSEDITED BY

Michael Spranger,
Sony Computer Science Laboratories,
Japan
REVIEWED BY

Wataru Noguchi,
Hokkaido University, Japan
Sina Ardabili,
University of Mohaghegh Ardabili, Iran*CORRESPONDENCE

Masatoshi Nagano,m_nagano@radish.ee.uec.ac.jp
SPECIALTY SECTION

This article was submitted toComputational Intelligence in Robotics,
a section of the journal
Frontiers in Robotics and AI

Spatio-temporal categorization
for ﬁrst-person-view videos
using a convolutional variational
autoencoder and Gaussian
processes
Masatoshi Nagano 1*, Tomoaki Nakamura 1, Takayuki Nagai 2,3,
Daichi Mochihashi 4 and Ichiro Kobayashi 5
1
Department of Mechanical Engineering and Intelligent Systems, The University of ElectroCommunications, Tokyo, Japan, 2Department of Systems Science, Osaka University, Osaka, Japan,
3
Artiﬁcial Intelligence eXploration Research Center, The University of Electro-Communications,Tokyo, Japan, 4Department of Statistical Inference and Mathematics, The Institute of Statistical
Mathematics, Tokyo, Japan, 5Department of Information Sciences, Ochanomizu University, Tokyo,
Japan

RECEIVED 24 March 2022
ACCEPTED 22 August 2022
PUBLISHED 30 September 2022
CITATION

Nagano M, Nakamura T, Nagai T,
Mochihashi D and Kobayashi I (2022),
Spatio-temporal categorization for ﬁrstperson-view videos using a
convolutional variational autoencoder
and Gaussian processes.
Front. Robot. AI 9:903450.
doi: 10.3389/frobt.2022.903450
COPYRIGHT© 2022 Nagano, Nakamura, Nagai,
Mochihashi and Kobayashi. This is an
open-access article distributed under
the terms of the Creative Commons
Attribution License (CC BY). The use,
distribution or reproduction in otherforums is permitted, provided theoriginal author(s) and the copyrightowner(s) are credited and that theoriginal publication in this journal iscited, in accordance with accepted
academic practice. No use, distribution
or reproduction is permitted which does
not comply with these terms.

In this study, HcVGH, a method that learns spatio-temporal categories by segmenting
ﬁrst-person-view (FPV) videos captured by mobile robots, is proposed. Humans perceive
continuous high-dimensional information by dividing and categorizing it into signiﬁcantsegments. This unsupervised segmentation capability is considered important for mobile
robots to learn spatial knowledge. The proposed HcVGH combines a convolutionalvariational autoencoder (cVAE) with HVGH, a past method, which follows the hierarchical
Dirichlet process-variational autoencoder-Gaussian process-hidden semi-Markov
model comprising deep generative and statistical models. In the experiment, FPV
videos of an agent were used in a simulated maze environment. FPV videos containspatial information, and spatial knowledge can be learned by segmenting them. Using theFPV-video dataset, the segmentation performance of the proposed model wascompared with previous models: HVGH and hierarchical recurrent state spacemodel. The average segmentation F-measure achieved by HcVGH was 0.77;
therefore, HcVGH outperformed the baseline methods. Furthermore, theexperimental results showed that the parameters that represent the movability of themaze environment can be learned.
KEYWORDS

convolutional variational autoencoder, Gaussian process, hidden semi-Markov model,
spatio-temporal categorization, segmentation, unsupervised learning

1 Introduction
Humans recognize continuous high-dimensional information by dividing andcategorizing it into signiﬁcant segments without explicit segmentation points. This
unsupervised method has high generalizability and can be extended to mobile robots
to help them adapt to various environments and contexts. Similarly, words or phonemes

Frontiers in Robotics and AI

01

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

feature extraction from videos while dividing and classifying theminto signiﬁcant segments. Furthermore, FPV videos obtained by amobile agent in a simulated maze are used in this study. The
images in such videos have spatio-temporal structure because theimages temporally change under the spatial continuousness of theagent moving. Therefore, by segmenting such a video, HcVGH
learns spatial categories and their transitions that represent both
spatial and temporal changes. In this paper, we deﬁne such
categorization as spatio-temporal categorization. Figure 1
presents an overview of the proposed model. Video data arecompressed and converted into a latent variable sequence by
the cVAE, and the latent variable sequence is divided andclassiﬁed into segments using the same HDP-GP-HSMM
process as before.
Herein, it is shown that the proposed method successfully
segments and classiﬁes variable sequences using a small numberof video data. Following Kim et al. (2019), an experiment isconducted using FPV video data obtained by a mobile agent in a
maze to demonstrate the superior stability and performance of
HcVGH. By comparing HcVGH with HVGH, which uses only
fully connected layers, it is demonstrated that capturing spatial
characteristics in the image by convolution is effective for theestimation of the maze structure. Moreover, it is found that the
proposed method has higher explainability regarding the spatial
structure and segment classes than the end-to-end hierarchicalrecurrent state space model (HRSSM).

can be learned by segmenting speech, and unit motions can belearned by segmenting continuous motion data. Furthermore,
spatio-temporal categories that are symbolized representations ofa particular space can be learned via the segmentation of timeseries visual information obtained as the agent moves around.
This ability is also important for the spatial cognition of robots.
Furthermore, it has been suggested that visual information
contributes to spatial cognition in animals and humans. It has
been reported that the brains of rats have place cells in theirhippocampus (O’Keefe and Recce, 1993; Kitanishi et al., 2021)
that are activated when the rat ﬁnds itself in a speciﬁc locationand context; hence, the cells are considered to play an importantrole in navigation. It has also been reported that the
hippocampus of bats plays an important role in the spatial
navigation or recognition of their current position (Dotson
and Yartsev, 2021). Moreover, Rolls and O’Mara (1995) andRolls (1999) reported that view cells of a monkey, which areactivated from visual information regardless of their location,affect spatial cognitive processing and help in spatial navigation.
Spatial cognition is considered important not only incomputational neuroscience but also in machine learning androbotics, and spatial cognition studies with robots have been
conducted (Milford et al., 2004; Madl et al., 2015; Schapiro et al.,2017; Banino et al., 2018; Kowadlo et al., 2019; Scleidorovich
et al., 2020). For advanced intelligent mobile robots, using
information obtained by their own sensors to learn spatial
knowledge is necessary (Taniguchi et al., 2018). Based on thisbackground, this paper presents a stochastic model that divides
and categorizes ﬁrst-person-view (FPV) videos obtained by amobile agent in a simulated maze. FPV videos contain spatial
information, and the parameters representing the spatiotemporal structure can be learned by segmenting them.
Previously, HVGH1 was proposed; it is an unsupervised
segmentation method for time-series data that divides andclassiﬁes information using the hierarchical Dirichlet processGaussian process-hidden semi-Markov model (HDP-GPHSMM). HVGH includes a variational autoencoder (VAE) thatcan be used as a feature extractor (Kingma and Welling, 2013). The
parameters learned by the HDP-GP-HSMM (Nagano et al., 2018)
are used as hyperparameters for the VAE, and parameters for
HVGH are learned through the interaction between the VAE andthe HDP-GP-HSMM process. It was conﬁrmed that HVGH canestimate segments of motions more accurately than hidden
Markov model (HMM)-based methods (Nagano et al., 2019).
However, it was difﬁcult for HVGH to segment videos inwhich signiﬁcant features appeared among channels or pixels
because HVGH extracts features using only fully connected
layers. To overcome this limitation, a combined convolutional
VAE (cVAE) and HVGH (HcVGH) are proposed; they enable

1 HVGH: Hierarchical Dirichlet process-Variational
Gaussian process-Hidden semi-Markov model

Frontiers in Robotics and AI

2 Related work
Several unsupervised time-series data changepoint detection
methods, which assess ﬂuctuation differences and repeatedtemporal similarities to identify potential change points, havebeen proposed (Yamanishi and Takeuchi, 2002; Lund et al., 2007;
Liu et al., 2013; Haber et al., 2014). However, these methods do
not necessarily indicate segment boundaries.
Many time-series data segmentation methods have been
proposed (Lioutikov et al., 2015; Wächter and Asfour, 2015;
Takano and Nakamura, 2016; Deldari et al., 2020). However, they
make heuristic assumptions. For example, Wächter and Asfour(2015) proposed a method that uses contacts between an endeffector and an encountered object to segment objectmanipulation motions. A method proposed by Lioutikov et al.
(2015) requires segmentation candidate points in advance.
Moreover, Takano and Nakamura (2016) proposed a methodthat leverages errors between predicted and actual values for
robot observation segmentation. In another example, Deldari
et al. (2020) developed an entropy and shape-aware time-series
segmentation method that used segment similarities based ondata mining without stochastic modeling. However, this method
uses a threshold for computing segment length and similarity.
Notably, most proposed methods are probabilisticallyformulated using HMMs to segment time-series data (Bealautoencoder-02

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

FIGURE 1
Generative process of the proposed method.

et al., 2002; Fox et al., 2011; Taniguchi and Nagasaka, 2011;
Matsubara et al., 2014). However, HMMs have difﬁculty
handling complicated time-series patterns. By contrast, ourmodel uses a GP, which nonparametrically represents
complicated time-series data more appropriately than HMMs
(Nakamura et al., 2017; Nagano et al., 2018).
Several studies have used deep neural networks to extract
signiﬁcant patterns from time-series data (Fraccaro et al., 2017;
Rangapuram et al., 2018; Kurle et al., 2020). They proposed
combination state-space models and deep neural networks to
extract meaningful patterns; however, they focused not on
segmentation but on prediction from observed time-series
data considering their dynamics. By contrast, Liu et al. (2018)
and Ansari et al. (2021) estimated the boundary points ofsegments and their classes by combining a recurrent neuralnetwork with a hidden semi-Markov model (Yu, 2010). In allcases, auxiliary variables are computed to represent the duration
and boundaries of a state to segment time-series data. However,the number of classes is ﬁxed, and only simple or periodic timeseries data are used. Therefore, it is difﬁcult for these methods toappropriately handle complex high-dimensional time-series
video data acquired by a mobile robot.
Related to the proposed approach, high-dimensional timeseries video data segmentation studies have been conducted (Kimet al., 2019; Tanwani et al., 2020). Tanwani et al. (2020) usedlabeled video data of robot-guided surgical operations to learn
the latent space of suitable primitive motions. Semi-supervised
learning for video segmentation was achieved by segmenting avideo and relearning the latent space with a small number ofmanually labeled video segments. By contrast, the HRSSM wasproposed to divide video data into primitive segments in an
unsupervised, end-to-end manner using deep learning (Kim

Frontiers in Robotics and AI

FIGURE 2
HcVGH model: White and gray nodes respectively represent
unobserved variables and the high-dimensional observed
sequence obtained by concatenating segments.

et al., 2019), and applied for a navigation task using divided
segments in reinforcement learning. However, it is difﬁcult for
HRSSM to estimate classes. Furthermore, the method requires aninordinate amount of training data, and 1M frames of videos
were used in the experiment. However, our proposed method canperform accurate segmentation using only about 1,000 frames.
For application to real robots, the proposed method is considered03

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

more feasible from the viewpoint of cost for data collection than
HRSSM.
Studies have investigated simultaneous mapping andcategorization of the environment by mobile robots(Taniguchi et al., 2017; Chaplot et al., 2020). Chaplot et al.
(2020) built a semantic map of object categories detected by
pre-trained object detection and generated a path to the speciﬁed
goal based on reinforcement learning. However, it required pretrained object detection, and had difﬁculties in environmentswith objects whose categories could not be recognized by the pretrained model or without characteristic objects. Taniguchi et al.
(2017) proposed a method for a mobile robot to divide the space
into place categories based on the position, visual features, andplace-related utterances provided by the user. However, the place
categories could not be learned without user utterances.
Moreover, they used a pre-trained convolutional neuralnetwork to extract visual features, which is not a fully
unsupervised method. By contrast, the aim of our proposed
method is to segment space using only visual information andlearn features extracted from images in an unsupervised manner.

Z j ~ GPZ|ϕcj .

(4)Segments Xj are generated from latent variables Zj:
X j ~ pdec X|Z j .

(5)Here, we assume that video data (i.e., a time series of images)
comprise segment Xj. Hence, the cVAE’s decoder for pdec isutilized to generate images Xj from their low-dimensional latentvariables, Zj. The observation sequence, s = X1, X2, . . . , XJ, isobtained by combining Xj, based on these generative processes.
Moreover, the sequence of latent variables, s  Z 1 , Z 2 , . . . , Z J , isgenerated by connecting the segments of latent variables, Zj = zj1,
zj2, . . . , zji, /. Segment Xj = xj1, xj2, . . . , xji, / is comprised ofdata points xji. The subscripts are omitted if the characters in thedata point indicate the content.
The generative process of HcVGH is summarized in
Algorithm 1, and the observed sequence, s, is generated fromthis process. In the algorithm, the number of classes is assumed to
be inﬁnite, and there are inﬁnitely possible class transitions.
Notably, it is very difﬁcult to directly implement this algorithm.
To overcome this problem, a slice sampler (Van Gael et al., 2008)is used to produce a ﬁnite number of classes. In the slice sampler,
an auxiliary variable, uj, is computed to truncate transitions withprobability π cj−1 ,cj < uj .

3 Proposed Method
3.1 HcVGH
Figure 2 provides a graphical model of the proposed HcVGH,
which is a generative model for segmenting time-series data. In
HcVGH, it is assumed that the time-series data are generated
based on the following process.
β represents an inﬁnite-dimensional multinomial
distribution and is generated by a Grifﬁths, Engen, andMcCloskey (GEM) distribution (i.e., a stick-breaking process(Sethuraman, 1994; Pitman, 2002)) parameterized by γ. In
Figure 2, cj (j = 1, 2, . . . , ∞) denotes segment classes.
Moreover, πc represents the transition probability based on
Dirichlet processes (Teh et al., 2006) parameterized by η andβ as follows:
β ~ GEMγ,(1)

π c ~ DPη, β,(2)Algorithm 1. HcVGH s-Generation Process

where γ and η represent the concentration parameters of the
Dirichlet processes controlling the sparseness of the generated
distribution. The two-phase Dirichlet process in Eqs 1, 2 is ahierarchical Dirichlet process (Teh et al., 2006).
In Figure 2, class cj of the jth segment is generated by the (j −1)th class, cj−1, and the transition probability, πc. Moreover, latentvariable Zj represents the jth segment generated by a GP
(MacKay et al., 1998) based on the parameter, ϕc,corresponding to class c as follows:

To obtain a suitable latent variable, z, of observation x, acVAE whose prior distribution is a GP was utilized. Figure 3A
illustrates the encoder, and Figure 3B illustrates the decoder. In
this ﬁgure, observed data point x is compressed into lowdimensional latent variable z through the encoder network,μenc(x), Σenc(x):cj ~ Pc|cj−1 , π c ,z ~ qenc (z )  N z|μenc (x), Σenc (x),Frontiers in Robotics and AI

3.2 cVAE with GP prior

(3)

04

(6)

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

FIGURE 3
Convolutional variational autoencoder network architecture: (A) encoder = six convolutional layers (conv) and two fully connected layers (FC);(B) decoder = one fully connected layer and seven deconvolutional layers (conv_T).

FIGURE 4
Overview of HcVGH parameter estimation. The parameters are learned using a mutual cVAE and HDP-GP-HSMM learning loop.

3.3 Parameter inference

where qenc(z) is a probability that approximates the posterior
distribution, p (z|x). As a prior of z, a Gaussian distribution withthe mean vector, μc, and variance–covariance matrix Σccomputed by GP(z|ϕc ) is used:

The log-likelihood of HcVGH is as follows:
log pX 1 , . . . , X J , c1 , . . . , cJ   log jp(z )  N z|μc , Σc .

 log (7)

j

Frontiers in Robotics and AI

ZjpZ j , cj pX j |Z j dZ j
GP Z j |ϕc Pcj |cj−1 , π c  pX j |Z j  dZ j .

HDP−GP−HSMM

cVAE

(9)

Using this prior, latent variables reﬂecting the characteristics ofclass c can be obtained. Moreover, the middle layer of the cVAE
network is the convolution layer. Therefore, HcVGH efﬁciently
compresses the time series of three-dimensional tensors(i.e., images, each of which is composed of height, width andchannel).
The decoder network reconstructs observation x^ from latentvariable z through the decoder network:x^ ~ pdec (x|z ).

Zj

The factors, GP(Z j |ϕc )P(cj |cj−1 , π c ), are computed using
HDP-GP-HSMM, and p (Xj|Zj) are computed with cVAE in
Eq. 9. However, it is difﬁcult to maximize Eq. 9 directly. To
overcome this problem, the parameters are approximately
maximized by alternately optimizing HDP-GP-HSMM
and cVAE.
Figure 4 presents an overview of HcVGH’s parameter
estimation process. First, the cVAE converts a sequence ofobservations, s = X1, X2, . . . , XJ, into a sequence of latent

(8)

05

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

variables, s  Z 1 , Z 2 , . . . , Z J . The cVAE parameters are
estimated by maximizing the following variational lower
bound:
Lxji , z ji  qenc z ji |xji logpdec xji |z ji dz ji− wDKL qenc z ji |xji pz ji |μc (i), Σc (i),(10)

where w represents the parameter used to weight the
Kullback–Leibler divergence. If w > 1, it becomes possible tolearn the disentangled latent variables that are suitable forsegmentation (Higgins et al., 2017).
Then, the latent variable sequence, s, is divided and classiﬁed
into segments Z1, Z2, . . . , ZJ using HDP-GP-HSMM as follows:Z n,1 , . . . , Z n,Jn , cn,1 , . . . , cn,Jn  ~ pZ 1 , Z 2 , . . . , Z J , c1 , c2 , . . . , cJ |sn ,(11)where μc(i) and Σc(i) are parameters of the predictive distribution
computed by HDP-GP-HSMM and are used as parameters in thecVAE’s prior distribution in Eq. 10. Moreover, in the latent spacelearned by HcVGH, each latent variable reﬂects thecharacteristics of the time-series data and those of each classbecause the GP parameters differ for each.
In the proposed method, parameters of cVAE and HDP-GPHSMM are optimized by repeating the above computations until
the likelihoods converge.
As the proposed method is an improved version of HVGH,
several detailed sections are omitted in this paper. Please refer to(Nagano et al., 2019).

FIGURE 5(A) Upper view of the maze. The white block indicates theagent, which can move along 6 paths indicated by the arrows. (B)Example of the ﬁrst-person-view video data surrounded by theblack circle in (A).

4 Experimentscomplete correspondence of an estimated boundary point and theground truth; therefore, the estimated boundary was consideredcorrect when it was within a tolerance of the ground truth. In thisstudy, the tolerance was set to ±5% of the sequence length. Details ofthese metrics are described in (Nagano et al., 2019).

In this experiment, a small number of video data weredivided and classiﬁed into signiﬁcant segments using
HcVGH to demonstrate that the estimated parameters
express spatial structures. To evaluate the proposed
HcVGH, it was applied to time-series data of FPV videos
of an agent in a maze. For comparison, HRSSM (Kim et al.,2019) and HVGH (Nagano et al., 2019) were used asbaselines.

4.1.2 Dataset
Figure 5A presents the maze, and Figure 5B depicts FPV data of
the agent in the maze. In this experiment, the agent moved along the
6 paths indicated by the arrows in Figure 5A. Each FPV data frame
comprised red–green–blue image: xi ∈ R32×32×3 . To train HVGH,
ﬂattened vectors of the images were used. The maze consisted of
26 × 18 blocks, and the colored ones indicate areas where the agent
could not traverse. In Figure 5A, the white block represents theagent, who can move “straight ahead,” “turn left,” and “turn right.”
The agent moved straight ahead one block in ﬁve frames and rotated
90° to the left or right in three frames. The ground truth of thesegment boundaries is the corners and T-junctions in the maze, andeach hallway between boundaries is one segment.
To evaluate HRSSM, a 1M-frame dataset was constructedfrom the agent randomly selecting its action at the corner andT-junction with uniform distribution. The FPV data and maze

4.1 Experimental setup
4.1.1 Evaluation metrics
Four measures were used to evaluate segmentation accuracy:
normalized Hamming distance, precision, recall, and F-measure. The
normalized Hamming distance, an evaluation metric for clustering,
ranges from zero to one, and a value closer to zero indicatesapproximation to ground truth. The remaining metrics range fromzero to one as well, and larger values indicate that the estimatedboundary points of segments are more similar to ground truth. With
regard to the boundary point evaluation, it is very difﬁcult to achieve a

Frontiers in Robotics and AI

06

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

TABLE 1 Baselines and HcVGH segmentation results.

HcVGH

HVGH

HRSSM (6 paths)

HRSSM (1M)

Hyperparameter

Hamming Distance

Precision

RecallF-measure

λ = 200.33 ± 0.05

0.84 ± 0.06

0.91 ± 0.06

0.87 ± 0.06

λ = 100.19 ± 0.02

0.68 ± 0.05

0.96 ± 0.01

0.79 ± 0.03

λ=70.18 ± 0.01

0.61 ± 0.03

1.0 ± 0.0

0.75 ± 0.02

λ=50.19 ± 0.01

0.56 ± 0.02

0.99 ± 0.01

0.72 ± 0.01

λ=40.19 ± 0.01

0.55 ± 0.02

1.0 ± 0.0

0.71 ± 0.02

Average

0.22 ± 0.06

0.65 ± 0.12

0.97 ± 0.04

0.77 ± 0.07

λ = 200.78 ± 0.18

0.54 ± 0.33

0.49 ± 0.35

0.50 ± 0.33

λ = 100.66 ± 0.19

0.58 ± 0.33

0.56 ± 0.33

0.55 ± 0.32

λ=70.60 ± 0.26

0.34 ± 0.31

0.45 ± 0.42

0.39 ± 0.35

λ=50.68 ± 0.20

0.55 ± 0.34

0.55 ± 0.34

0.51 ± 0.29

λ=40.80 ± 0.21

0.20 ± 0.27

0.29 ± 0.39

0.23 ± 0.31

Average0.70 ± 0.20

0.44 ± 0.33

0.47 ± 0.35

0.43 ± 0.32

Nmax = 1

0.40

0.95

0.56

0.70

Nmax = 2

0.41

0.72

0.23

0.34

Nmax = 3

0.41

0.79

1.0

0.88

Nmax = 4

0.40

0.62

0.96

0.76

Nmax = 5

0.40

0.39

1.0

0.55

Average0.40 ± 0.01

0.69 ± 0.21

0.76 ± 0.35

0.65 ± 0.21

Nmax = 1

0.35

1.0

0.69

0.80

Nmax = 2

0.35

1.0

0.48

0.64

Nmax = 3

0.34

0.64

0.65

0.63

Nmax = 4

0.39

0.51

0.73

0.60

Nmax = 5

0.39

0.44

0.92

0.59

Average

0.36 ± 0.03

0.72 ± 0.26

0.70 ± 0.16

0.65 ± 0.09

arrows in Figure 5 and the context of ﬁve frames of data before andafter the path data frames were used. Furthermore, the learning
iterations were repeated until the training loss converged.
HRSSM does not estimate classes but estimates the boundary
points of the segments in an unsupervised manner. To evaluate
the categorization capability of HRSSM, Gaussian mixture model
was applied to the estimated latent variable and computed
normalized Hamming distance. Moreover, to observe theinﬂuence of the training data size, HRSSM was evaluated in
the following two cases.

used in this experiment are published at https://github.com/
nagano28/color_maze.git.

4.1.3 HRSSM
HRSSM requires many hyperparameters; it estimates whether
there is a boundary in each subsequence, and 20 frames were set as
the length of a subsequence. This length was used as the default
value in the study. Lmax is the maximum length of a segment, andLmax = 20 was set because a boundary may not exist in a
subsequence. Nmax is the maximum number of segments in a
subsequence, and this parameter is inﬂuential to the segmentation.
Therefore, HRSSM was trained by varying Nmax = 1, 2, /, 5. The
number of dimensions of latent variables was set to 128, which was
also set as the default value. For all other hyperparameters, thedefault values were used. With HRSSM, the paths indicated by the

Frontiers in Robotics and AI• HRSSM (6 paths): training and testing on the 6-pathdataset• HRSSM (1M): training on the 1M dataset, testing on the 6path dataset

07

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

FIGURE 6
Segmentation results of a ﬁrst-person-view video data.

FIGURE 7
Transition probabilities of the estimated classes: (A) represents a transition matrix, and (B) represents the locations that correspond to each classin the maze.

Frontiers in Robotics and AI

08

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

TABLE 2 Evaluation of spatial movability of paths. “T” in the “Type” column shows their class sequences were included in the training data. “G” in the“Type” column shows generated class sequences that were not included in the training data. Underlined numbers in “class sequence” represent
spatially impossible transitions.

Type

Class sequence

L

1

T

20, 11, 10, 21, 17, 15, 8, 16, 9, 4, 12, 23, 26, 2, 22−0.2282

T20, 6, 3, 7, 24, 14, 3, 25, 22, 26, 2, 22−0.3903

T

20, 6, 3, 7, 24, 14, 5, 12, 23, 26, 2, 22−0.3274

T

20, 11, 10, 21, 17, 15, 8, 16, 2, 13−0.432

5

T20, 11, 10, 21, 17, 15, 0, 20, 27, 2, 4, 12, 23, 26, 2, 22−0.4266

T

20, 11, 10, 19, 9, 18, 7, 24, 14, 3, 2, 1, 22−0.5317G

20, 11, 10, 21, 17, 15, 0, 20, 11, 10, 21, 17, 15, 8, 16, 9, 4, 12, 23, 26, 2, 22−0.2448

G

20, 11, 10, 21, 17, 15, 0, 20, 11, 10, 19, 9, 18, 7, 24, 14, 5, 12, 23, 26, 2, 22−0.2979

G

20, 11, 10, 21, 17, 15, 0, 20, 11, 10, 21, 17, 15, 8, 16, 2, 13−0.36410

G

20, 11, 10, 21, 17, 15, 8, 16, 9, 4, 20, 11, 10, 21, 17, 15, 8, 16, 9, 4, 12, 23, 26, 2, 22−4.39711

G

20, 11, 10, 21, 17, 15, 0, 20, 11, 10, 20, 11, 10, 21, 17, 15, 8, 16, 9, 4, 12, 23, 26, 2, 22−4.420FIGURE 8
Estimated results by HRSSM: (A–D) respectively represent an example of predicted images by using the same contexts.

block Gibbs sampler was iterated eight times. Additionally,cVAE and HDP-GP-HSMM loops were repeated until thevariational lower bound of the cVAE converged.

4.1.4 HcVGH
To compare HcVGH and HRSSM, the value of the required
HcVGH parameter, λ, was changed to 20, 10, 7, 5, and 4, whichcorresponds to approximately Nmax of HRSSM. λ is a mean
parameter of the Poisson distribution, Plen (k|λ)2, that
determines segment lengths. The parameters of the GP
kernel function were the same as those used in the past work
(Nagano et al., 2019). The weight of the regularization term of
the cVAE was set to w = 5, and the number of dimensions of thelatent variable was set to 16. When training the cVAE, 16 of the
input data points were used as a mini-batch, and Adam
(Kingma and Ba, 2014) was used for optimization with
100 iterations of updates. To train the HDP-GP-HSMM, the

4.2 Results
4.2.1 Segmentation results
Table 1 shows the results of segmentation using HRSSM,
HVGH, and HcVGH. In this result, HRSSM’s estimation accuracy
represents the most accurate result of learning iterations because in
the preliminary experiment, we conﬁrmed that the initial values
did not signiﬁcantly affect segmentation. In contrast, for HcVGH,
we used the average value of the results of ﬁve executions withdifferent initial values; this is because it has been empiricallyconﬁrmed that the GP-HSMM-based model could sometimes
not be able to get past the local optima.

k −λ

2 Plen (k|λ)  λ k!e

Frontiers in Robotics and AI

09

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

Table 1 shows that the F-measure of HcVGH and HRSSM(6 paths, Nmax = 3) were both high, and the correct boundary
points of the segments were estimated. However, this tablealso shows that the F-measure of HRSSM (6 paths) wasstrongly affected by the hyperparameter, Nmax. The
normalized Hamming distance of HRSSM (1M) was shorter
than that of HRSSM (6 paths). By contrast, regardless of thehyperparameter settings, the F-measure of HcVGH was stable,
indicating high accuracy, and the Hamming distance of
HcVGH was stably small. Finally, the accuracy ofsegmentation of HVGH was lower than that of the othermethods.
From this result, it is considered difﬁcult for HVGH whose
VAE is composed of only fully connected layers to extracteffective features to represent the maze. In HRSSM (6 paths),
F-measure was not stable, and the maximum value was 0.88.
This may be because the training dataset was too small fortraining HRSSM, and the parameters went into different local
optima in each training. By contrast, the F-measure of HRSSM(1M) was stable although it was not higher than 0.88. The
normalized Hamming distance of HRSSM (1M) became
shorter than that of HRSSM (6 paths), and the latentvariable capturing the characteristics of each category of
FPV images could be learned through increasing thetraining data. However, in HcVGH, the average normalizedHamming distance was the smallest, the average F-measure
was the highest, and their standard deviations were thesmallest. This result shows that the dependence of HcVGH
on a hyperparameter is less strong than that of HRSSM. The
recall of HcVGH tended to be larger than its precision, andthis was because it classiﬁed the corner and T-junction into a
different category. Although this was judged as incorrect
based on the deﬁnition of ground truth in this experiment,
this estimation was also considered reasonable.
Figure 6 shows the qualitative results of HcVGH and HVGH
segmentation. In this ﬁgure, the horizontal axis represents thetime step, and the color of the horizontal bar graph represents theclass of the segments. The bar graph at the top indicates theboundary points and classes of the ground truth, and an example
of an image observed by the agent is shown to correspond to thecorrect class. This ﬁgure demonstrates that the segments andclasses estimated by HcVGH are approximate to the groundtruth. However, the ﬁnal segment of Sequence 3 was estimated to
be a different class from the one obtained from the othersequences, even though the agents traversed the same
corridor. This is because, depending on the direction of theagent, the image features were slightly different, even in the same
corridor. Furthermore, some corners were classiﬁed as individualclasses, which does not correspond with the ground truth.
However, this estimation is reasonable and is not a problem.
In HVGH, there were many misclassiﬁcations, and it can be seenthat it is difﬁcult to capture the characteristics of the images using
only fully connected layers.

Frontiers in Robotics and AI

TABLE 3 Number of the predicted paths at the T-junction.

HRSSM (1M)

left

right

else

52

23

25

4.2.2 Evaluation for spatial movability usingtransition probability
Figure 7A presents a transition matrix that shows the transition
probability of estimated classes. In this ﬁgure, the intensity of eachcell represents log P (cj|cj−1), and lighter values indicate higher
probabilities. The white numbers in Figure 7B show the estimatedclasses, c, of HcVGH. As shown in Figure 7A, probabilities that
represent movability from one place to another were explicitly
obtained. For example, as seen in the red dashed rectangle of
Figure 7A, the transition probabilities, log P (cj = 19|cj−1 = 10)
and log P (cj = 21|cj−1 = 10), are high, and it was conﬁrmed that theyreﬂect actual transitions in the area enclosed by the black dashed lineof Figure 7B.
Table 2 shows the class sequences that are paths in the mazeand their normalized accumulated transition probabilities L
computed as follows:L

1 J^cj |cj−1 , logPJ j1(12)^ j |cj−1 ) is the transition probability without a priorwhere P(cdistribution to prevent overestimation by GEM distribution, J is
the length of the class sequence, and L is normalized by dividing
by J. “T” in the “Type” column of the table shows the classsequences are included in the training data. “G” in the “Type”
column of the table shows ﬁve paths with the highest
probabilities in the randomly generated 100 paths byrandomly dividing and connecting the class sequences of six
paths in the training data. From this table, L of the six paths(paths 1–6) in the training data is higher. Furthermore, L of path
7, which circulates around the green right bottom block in
Figure 7B, and paths 8 and 9, which combine paths from thetraining data, are also higher although the paths were notincluded in the training data. By contrast, L of paths 10 and
11, which contain spatially impossible transitions (underlined inTable 2), is lower. From this result, transition probabilities canrepresent spatial movability, and explicitly obtaining theseprobabilities is an advantage of HcVGH. However, spatial
movability can sometimes be inaccurately estimated owing tomisclassiﬁcation. In our experiment, different corners weremisclassiﬁed and estimated to belong to the same class; thiscaused the movability at these positions to be incorrectly
estimated. To solve this problem, the number of
misclassiﬁcations must be reduced.
However, in HRSSM, subsequent states are generated fromthe current state by a recurrent neural network, and it is difﬁcult
to explicitly obtain movability between states. To evaluate

10

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

FIGURE 9
Latent variables learned by HcVGH: (A–C) respectively represent the ﬁrst and second, ﬁrst and third, and second and third dimension of theprincipal component of the latent variables. The color of each point reﬂects the correct corridor class.

FIGURE 10
Latent variables learned by HRSSM (6 paths): (A–C) respectively represent the ﬁrst and second, ﬁrst and third, and second and third dimension of
the principal component of the latent variables. The color of each point reﬂects the correct corridor class.

FIGURE 11
Latent variables learned by HRSSM (1M): (A–C) respectively represent the ﬁrst and second, ﬁrst and third, and second and third dimension of theprincipal component of the latent variables. The color of each point reﬂects the correct corridor class.

shows the agent prediction that it is possible to turn left, andFigure 8B shows the agent prediction that it is possible to turnright. The agent actions are determined by uniform distribution
in the 1M dataset; however, the prediction was biased as shown in
Table 3. Moreover, HRSSM predicted images that mixed both the

movability estimated by HRSSM, the number of transitions at
T-junction in the blacked dashed rectangle in Figure 7B ofpredicted 100 paths by the most accurate model HRSSM (1M,
Nmax = 1) were manually counted. Figure 8 shows examples ofprediction and Table 3 shows the counting result. Figure 8A

Frontiers in Robotics and AI

11

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

FIGURE 12
Latent variables learned by HVGH: (A–C) respectively represent the ﬁrst and second, ﬁrst and third, and second and third dimension of theprincipal component of the latent variables. The color of each point reﬂects the correct corridor class.

FIGURE 13
Latent variables learned by only cVAE: (A–C) respectively represent the ﬁrst and second, ﬁrst and third, and second and third dimension of theprincipal component of the latent variables. The color of each point reﬂects the correct corridor class.

overlapped and their separation for each class is inadequate
for clustering them. Therefore, it is difﬁcult to classify thelatent variables into place categories in an unsupervisedmanner.
By contrast, in HcVGH (Figure 9), the latent variables of thesame class have more similar values, and the latent variables ofdifferent classes are well separated. Compared with those of only
cVAE (Figure 13), the latent variables of HcVGH (Figure 9) are
better separated. This is because μ and σ computed by HDP-GPHSMM are used as the prior of cVAE in the HcVGH, andtherefore, the latent variables that are classiﬁed into the same
categories get closer.

left- and the right-side paths (Figure 8C) and mixed the left/right-side and other paths (Figure 8D). From this result, we ﬁndthat in HRSSM, rough transition probabilities can be computed
manually or by any other additional means, but they cannot be
obtained explicitly. Therefore, it is difﬁcult to evaluate spatial
movability with HRSSM in an unsupervised manner.

4.2.3 Comparison of latent variables
Figures 9–13 show the latent variables of 6 paths estimated by
HcVGH, HRSSM (6 paths), HRSSM (1M), HVGH, and only
cVAE. In these ﬁgures, panels (a), (b), and (c) respectively
represent the ﬁrst and second, ﬁrst and third, and second andthird dimensions of the latent variables, which were compressed
via principal component analysis (Pearson, 1901). The color ofeach point reﬂects the correct corridor class.
In Figure 12, the latent variable of HVGH is not well
separated for each class. Similarly, in Figure 10, the latentvariables of HRSSM (6 paths) are not separated for each class,and compared with HRSSM (6 paths), the latent variables of
HRSSM (1M) seem to improve slightly in Figure 11. However,the latent variables of HRSSM (6 paths) and HRSSM (1M) are

Frontiers in Robotics and AI

4.3 Discussion
From the results, it can be seen that HcVGH is accurate
and stable regardless of the hyperparameters. By contrast,
HRSSM tends to be affected by hyperparameters,
and parameter tuning is required depending on the
training data.

12

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

plan its actions based on probabilistic inference (Levine, 2018)using HcVGH will be formulated.
However, one of the limitations of HcVGH is the
misclassiﬁcation caused by using unimodal information. In
the future, the cVAE of HcVGH will be extended to a joint
multimodal VAE to divide and classify multimodal information
to overcome this problem. Another limitation of HcVGH is its
scalability. Therefore, it will be necessary to verify the scalabilityof HcVGH by performing segmentation on a larger dataset, andmore realistic dataset.

Furthermore, HcVGH has high explainability because thetransition probabilities that are considered movabilities can beobtained explicitly. These transition probabilities can be used for
global path planning, and various paths can be planned
according to a purpose. For instance, shortest path, longest
path, paths going through a particular location, or paths thatmaximize a particular objective function can be planned
considering the movability. However, HRSSM does not havesuch explicit parameters, and therefore, it is difﬁcult to plan paths
according to a purpose. One solution is generating many paths
and selecting one path that matches the purpose; however, theoptimal path is not always generated. Another solution iscomputing transition probabilities as in Section 4.2.2;
however, generated samples require manual classiﬁcation. For
this reason, HcVGH is suitable for application to mobile robots.
However, HcVGH has limitations. It depends on visual
information only, and locations with similar appearances canbe misclassiﬁed. To overcome this limitation, integration of themethod that can deal with multimodal information such as a
joint multimodal VAE (Suzuki et al., 2016) should be considered.
Moreover, by integrating the slam-based method such as(Taniguchi et al., 2017), the robot can learn the place concept
in a fully unsupervised manner avoiding misclassiﬁcation.
Another limitation of the model is its scalability. The size ofthe dataset used for HcVGH training was not very large because
the proposed method uses a Gaussian process whose
computational cost to train N data is O(N3). Therefore, itwould be difﬁcult to apply this to a huge dataset. In the
future, a veriﬁcation of the scalability of the proposed method
will be conducted by using realistic huge data such as car-camera
videos (Geiger et al., 2013).

Data availability statement
The datasets presented in this study can be found in online
repositories. The names of the repository/repositories andaccession number(s) can be found in the article/
Supplementary Material.

Author contributions
MN, TNak, TNag, and DM conceived, designed, anddeveloped the research. MN and TNak performed theexperiment and analyzed the data. MN wrote the manuscriptwith support from TNak, TNag, DM, and IK. All authors
discussed the results and contributed to the ﬁnal manuscript.

Funding
This work was supported by JSPS KAKENHI Grant
Number 21J11346 and JST Moonshot R&D Grant Number
JPMJMS 2011.

5 Conclusion
In this article, a cVAE was integrated into HVGH, a model
developed in a past work, and HcVGH, which divides andclassiﬁes video time-series data into segments, wasproposed. The experimental results show that HcVGH
achieved more accurate FPV video data segmentation thanthe baseline methods. Moreover, the results showed that
HcVGH has high explainability and a high segmentation
accuracy when compared with HRSSM, which segments
video data in an end-to-end manner. HcVGH estimatesboundary points and classes of segments more stably than
HRSSM.
Furthermore, in HcVGH, the parameters that represent
spatio-temporal structure of the maze can be obtained
explicitly. Using these parameters, spatial movability can beevaluated, which is useful for navigation planning. In the
future, the agent’s actions will be introduced, and a method to

Frontiers in Robotics and AI

Conﬂict of interest
The authors declare that the research was conducted in theabsence of any commercial or ﬁnancial relationships that could
be construed as a potential conﬂict of interest.

Publisher’s noteAll claims expressed in this article are solely those of theauthors and do not necessarily represent those of their afﬁliatedorganizations, or those of the publisher, the editors and thereviewers. Any product that may be evaluated in this article, or
claim that may be made by its manufacturer, is not guaranteed or
endorsed by the publisher.

13

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

References
Ansari, A. F., Benidis, K., Kurle, R., Turkmen, A. C., Soh, H., Smola, A. J., et al.
(2021). Deep explicit duration switching models for time series. Adv. Neural Inf.
Process. Syst. 34, 29949–29961.

Matsubara, Y., Sakurai, Y., and Faloutsos, C. (2014). Autoplait: Automatic mining of
co-evolving time sequences,” in Proceedings of the 2014 ACM SIGMOD international
conference on Management of data, Snowbird, UT, June 22–27, 2014, 193–204.

Banino, A., Barry, C., Uria, B., Blundell, C., Lillicrap, T., Mirowski, P., et al. (2018).
Vector-based navigation using grid-like representations in artiﬁcial agents. Nature
557, 429–433. doi:10.1038/s41586-018-0102-6

Milford, M. J., Wyeth, G. F., and Prasser, D. (2004). Ratslam: A hippocampal
model for simultaneous localization and mapping,” in IEEE International
Conference on Robotics and Automation, 2004 Proceedings ICRA’04. 2004
(IEEE), New Orleans, LA, April 26–May 01, 2004 1, 403–408.

Beal, M. J., Ghahramani, Z., and Rasmussen, C. E. (2002). The inﬁnite hidden
markov model. Adv. neural Inf. Process. Syst. 1, 577. doi:10.7551/mitpress/1120.003.
0079

Nagano, M., Nakamura, T., Nagai, T., Mochihashi, D., Kobayashi, I., and Kaneko,
M. (2018). Sequence pattern extraction by segmenting time series data using gphsmm with hierarchical Dirichlet process,” in 2018 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, October
1–5, 2018 (IEEE), 4067–4074.

Chaplot, D. S., Gandhi, D. P., Gupta, A., and Salakhutdinov, R. R. (2020). Object
goal navigation using goal-oriented semantic exploration. Adv. Neural Inf. Process.
Syst. 33, 4247–4258.

Nagano, M., Nakamura, T., Nagai, T., Mochihashi, D., Kobayashi, I., and Takano,W. (2019). Hvgh: Unsupervised segmentation for high-dimensional time series
using deep neural compression and statistical generative model. Front. Robot. AI 6,
115. doi:10.3389/frobt.2019.00115

Deldari, S., Smith, D. V., Sadri, A., and Salim, F. (2020). Espresso: Entropy andshape aware time-series segmentation for processing heterogeneous sensor data.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 1–24. doi:10.1145/
3411832

Nakamura, T., Nagai, T., Mochihashi, D., Kobayashi, I., Asoh, H., and Kaneko, M.
(2017). Segmenting continuous motions with hidden semi-markov models andGaussian processes. Front. Neurorobot. 11, 67. doi:10.3389/fnbot.2017.00067

Dotson, N. M., and Yartsev, M. M. (2021). Nonlocal spatiotemporal
representation in the hippocampus of freely ﬂying bats. Science 373, 242–247.
doi:10.1126/science.abg1278

O’Keefe, J., and Recce, M. L. (1993). Phase relationship between hippocampal place
units and the eeg theta rhythm. Hippocampus 3, 317–330. doi:10.1002/hipo.450030307

Fox, E. B., Sudderth, E. B., Jordan, M. I., and Willsky, A. S. (2011). Joint modelingof multiple related time series via the beta process. arXiv preprint arXiv:1111.4226.

Pearson, K. (1901). Liii. on lines and planes of closest ﬁt to systems of points inspace. Lond. Edinb. Dublin philosophical Mag. J. Sci. 2, 559–572. doi:10.1080/
14786440109462720

Fraccaro, M., Kamronn, S., Paquet, U., and Winther, O. (2017). A disentangled
recognition and nonlinear dynamics model for unsupervised learning. Adv. Neural.
Inf. Process Syst. 30.

Pitman, J. (2002). Poisson–Dirichlet and gem invariant distributions for splitand-merge transformations of an interval partition. Comb. Probab. Comput. 11,
501–514. doi:10.1017/s0963548302005163

Geiger, A., Lenz, P., Stiller, C., and Urtasun, R. (2013). Vision meets robotics: The
kitti dataset. Int. J. Robotics Res. 32, 1231–1237. doi:10.1177/0278364913491297
Haber, D., Thomik, A. A., and Faisal, A. A. (2014). “Unsupervised time series
segmentation for high-dimensional body sensor network data streams,” in2014
11th international conference on wearable and implantable body sensor networks(IEEE), 121–126.

Rangapuram, S. S., Seeger, M. W., Gasthaus, J., Stella, L., Wang, Y., andJanuschowski, T. (2018). Deep state space models for time series forecasting.
Adv. neural Inf. Process. Syst. 31.
Rolls, E. T., and O’Mara, S. M. (1995). View-responsive neurons in the primate
hippocampal complex. Hippocampus 5, 409–424. doi:10.1002/hipo.450050504

Higgins, I., Matthey, L., Pal, A., Burgess, C. P., Glorot, X., Botvinick, M. M., et al.
(2017). beta-vae: Learning basic visual concepts with a constrained variationalframework. Toulon, France: ICLR.

Rolls, E. T. (1999). Spatial view cells and the representation of place in the primate
hippocampus. Hippocampus 9, 467–480. doi:10.1002/(sici)1098-1063(1999)9:4<467::aid-hipo13>3.0.co;2-fKim, T., Ahn, S., and Bengio, Y. (2019). Variational temporal abstraction. Adv.
Neural Inf. Process. Syst. 32, 11570–11579.

Schapiro, A. C., Turk-Browne, N. B., Botvinick, M. M., and Norman, K. A. (2017).
Complementary learning systems within the hippocampus: A neural networkmodelling approach to reconciling episodic memory with statistical learning.
Phil. Trans. R. Soc. B 372, 20160049. doi:10.1098/rstb.2016.0049

Kingma, D. P., and Ba, J. (2014). Adam: A method for stochastic optimization.
arXiv preprint arXiv:1412.6980.
Kingma, D. P., and Welling, M. (2014). “Auto-encoding variational bayes,” in 2nd
International Conference on Learning Representations, Banff, AB, April 14–16,
2014 (Conference Track Proceedings).

Scleidorovich, P., Llofriu, M., Fellous, J.-M., and Weitzenfeld, A. (2020). A
computational model for latent learning based on hippocampal replay,” inInternational Joint Conference on Neural Networks(IJCNN), Glasgow, UK, July
19-24, 2020 (IEEE), 1–8.

Kitanishi, T., Umaba, R., and Mizuseki, K. (2021). Robust information routing by
dorsal subiculum neurons. Sci. Adv. 7, eabf1913. doi:10.1126/sciadv.abf1913
Kowadlo, G., Ahmed, A., and Rawlinson, D. (2019). Aha! an’artiﬁcial
hippocampal algorithm’for episodic machine learning. arXiv preprint arXiv:
1909.10340.

Sethuraman, J. (1994). A constructive deﬁnition of Dirichlet priors. Stat. Sin.,
639–650.
Suzuki, M., Nakayama, K., and Matsuo, Y. (2016). Joint multimodal learning withdeep generative models. arXiv preprint arXiv:1611.01891.

Kurle, R., Rangapuram, S. S., de Bézenac, E., Günnemann, S., and Gasthaus, J.
(2020). Deep rao-blackwellised particle ﬁlters for time series forecasting. Adv.
Neural Inf. Process. Syst. 33.
Levine, S. (2018). Reinforcement learning and control as probabilistic inference:
Tutorial and review. arXiv:1805.00909.

Takano, W., and Nakamura, Y. (2016). Real-time unsupervised segmentationof human whole-body motion and its application to humanoid robot acquisitionof motion symbols. Robotics Aut. Syst. 75, 260–272. doi:10.1016/j.robot.2015.
09.021

Lioutikov, R., Neumann, G., Maeda, G., and Peters, J. (2015).
Probabilistic
segmentation
appliedtoanassemblytask,”
in2015 IEEE-RAS 15th International Conference on Humanoid Robots(Humanoids) (IEEE), Seoul, South Korea, November 3–5, 2015, 533–540.

Taniguchi, A., Hagiwara, Y., Taniguchi, T., and Inamura, T. (2017). Online spatialconcept and lexical acquisition with simultaneous localization and mapping,” in2017 ieee/rsj international conference on intelligent robots and systems (iros),
Vancouver, BC, September 24–28, 2017 (IEEE), 811–818.

Liu, H., He, L., Bai, H., Dai, B., Bai, K., and Xu, Z. (2018). Structured inference forrecurrent hidden semi-markov model. IJCAI, 2447–2453.

Taniguchi, T., and Nagasaka, S. (2011). Double articulation analyzer for
unsegmented human motion using pitman-yor language model and inﬁnite
hidden markov model,” in 2011 IEEE/SICE International Symposium on System
Integration (SII), Kyoto, Japan, December 20–22, 2011 (IEEE), 250–255.

Liu, S., Yamada, M., Collier, N., and Sugiyama, M. (2013). Change-point
detection in time-series data by relative density-ratio estimation. Neural Netw.
43, 72–83. doi:10.1016/j.neunet.2013.01.012

Taniguchi, T., Ugur, E., Hoffmann, M., Jamone, L., Nagai, T., Rosman, B., et al.
(2018). Symbol emergence in cognitive developmental systems: A survey. IEEE
Trans. Cogn. Dev. Syst. 11, 494–516. doi:10.1109/tcds.2018.2867772

Lund, R., Wang, X. L., Lu, Q. Q., Reeves, J., Gallagher, C., and Feng, Y. (2007).
Changepoint detection in periodic and autocorrelated time series. J. Clim. 20,
5178–5190. doi:10.1175/jcli4291.1

Tanwani, A. K., Sermanet, P., Yan, A., Anand, R., Phielipp, M., and Goldberg,
K. (2020). Motion2vec: Semi-supervised representation learning from surgicalvideos,” in 2020 IEEE International Conference on Robotics and Automation(ICRA), May 31–August 31, 2020 (IEEE), 2174–2181.

MacKay, D. J., et al. (1998). Introduction to Gaussian processes. NATO ASI Ser. F
Comput. Syst. Sci. 168, 133–166.
Madl, T., Chen, K., Montaldi, D., and Trappl, R. (2015). Computational cognitivemodels of spatial memory in navigation space: A review. Neural Netw. 65, 18–43.
doi:10.1016/j.neunet.2015.01.002

Frontiers in Robotics and AI

Teh, Y. W., Jordan, M. I., Beal, M. J., and Blei, D. M. (2006). Hierarchical Dirichlet
processes. J. Am. Stat. Assoc. 101, 1566–1581. doi:10.1198/016214506000000302

14

frontiersin.org

Nagano et al.

10.3389/frobt.2022.903450

Yamanishi, K., and Takeuchi, J.-i. (2002). “A unifying framework for detecting
outliers and change points from non-stationary time series data,” in Proceedings ofthe eighth ACM SIGKDD international conference on Knowledge discovery anddata mining, Edmonton, AB, July 23–26, 2002, 676–681.

Van Gael, J., Saatci, Y., Teh, Y. W., and Ghahramani, Z. (2008). Beam sampling
for the inﬁnite hidden markov model,” in Proceedings of the 25th international
conference on Machine learning, Helsinki, Finland, July 5–9, 2008, 1088–1095.
Wächter, M., and Asfour, T. (2015). Hierarchical segmentation of manipulation
actions based on object relations and motion characteristics,” in 2015 International
Conference on Advanced Robotics (ICAR) IEEE, 549–556.

Frontiers in Robotics and AI

Yu, S.-Z. (2010). Hidden semi-markov models. Artif. Intell. 174, 215–243. doi:10.
1016/j.artint.2009.11.011

15

frontiersin.org