Object Level Visual Reasoning in Videos

Summary:
Human activity recognition is typically addressed by detecting key concepts
like global and local motion, features related to object classes present in the
scene, as well as features related to the global context. The next open
challenges in activity recognition require a level of understanding that pushes
beyond this and call for models with capabilities for fine distinction and
detailed comprehension of interactions between actors and objects in a scene.
We propose a model capable of learning to reason about semantically meaningful
spatiotemporal interactions in videos. The key to our approach is a choice of
performing this reasoning at the object level through the integration of state
of the art object detection networks. This allows the model to learn detailed
spatial interactions that exist at a semantic, object-interaction relevant
level. We evaluate our method on three standard datasets (Twenty-BN
Something-Something, VLOG and EPIC Kitchens) and achieve state of the art
results on all of them. Finally, we show visualizations of the interactions
learned by the model, which illustrate object classes and their interactions
corresponding to different activity classes.

Fabien Baradel
Natalia Neverova
Christian Wolf
Julien Mille
Greg Mori

Accepted at ECCV 2018 - long version (16 pages + ref)

Object Level Visual Reasoning in Videos
Fabien Baradel1, Natalia Neverova2, Christian Wolf1;3,
Julien Mille4, and Greg Mori5
1Universit e Lyon, INSA Lyon, CNRS, LIRIS, F-69621, Villeurbanne, France,
firstname.lastname@liris.cnrs.fr
2Facebook AI Research, Paris, France, nneverova@fb.com
3INRIA, CITI Laboratory, Villeurbanne, France
4Laboratoire d'Informatique de l'Univ. de Tours, INSA Centre Val de Loire,
41034, Blois, France, julien.mille@insa-cvl.fr
5Simon Fraser University, Vancouver, Canada, mori@cs.sfu.ca
https://fabienbaradel.github.io/eccv18_object_level_visual_reasoning/
Abstract. Human activity recognition is typically addressed by detect-
ing key concepts like global and local motion, features related to object
classes present in the scene, as well as features related to the global con-
text. The next open challenges in activity recognition require a level of
understanding that pushes beyond this and call for models with capa-
bilities for ne distinction and detailed comprehension of interactions
between actors and objects in a scene. We propose a model capable of
learning to reason about semantically meaningful spatio-temporal inter-
actions in videos. The key to our approach is a choice of performing
this reasoning at the object level through the integration of state of the
art object detection networks. This allows the model to learn detailed
spatial interactions that exist at a semantic, object-interaction relevant
level. We evaluate our method on three standard datasets (Twenty-BN
Something-Something, VLOG and EPIC Kitchens) and achieve state of
the art results on all of them. Finally, we show visualizations of the in-
teractions learned by the model, which illustrate object classes and their
interactions corresponding to dierent activity classes.
Keywords: Video understanding Human-object interaction
1 Introduction
The eld of video understanding is extremely diverse, ranging from extract-
ing highly detailed information captured by specically designed motion cap-
ture systems [33] to making general sense of videos from the Web [1]. As in
the domain of image recognition, there exist a number of large-scale video
datasets [6,26,13,12,23,14], which allow the training of high-capacity deep learn-
ing models from massive amounts of data. These models enable detection of key
cues present in videos, such as global and local motion, various object categories
and global scene-level information, and often achieve impressive performance in
recognizing high-level, abstract concepts in the wild.arXiv:1806.06157v3  [cs.CV]  20 Sep 2018
2 Baradel et al.
person 0.86person 0.87knife 0.98carrot 0.99carrot 0.72carrot 0.98
person 0.65knife 0.99carrot 0.97carrot 0.97carrot 0.93carrot 0.86carrot 0.95
Fig. 1. Humans can understand what happened in a video (\the leftmost carrot was
chopped by the person") given only a pair of frames. Along these lines, the goal of this
work is to explore the capabilities of higher-level reasoning in neural models operating
at the semantic level of objects and interactions.
However, recent attention has been directed toward a more thorough un-
derstanding of human-focused activity in diverse internet videos. These eorts
range from atomic human actions [14] to ne-grained object interactions [13]
to everyday, commonly occurring human-object interactions [12]. This returns
us to a human-centric viewpoint of activity recognition where it is not only the
presence of certain objects / scenes that dictate the activity present, but the
manner, order, and eects of human interaction with these scene elements that
are necessary for understanding. In a sense, this is akin to the problems in current
3D human activity recognition datasets [33], but requires the more challenging
reasoning and understanding of diverse environments common to internet video
collections.
Humans are able to infer what happened in a video given only a few sam-
ple frames. In particular, they can infer complex activities happening between
pairs of frames. This faculty is called reasoning and is a key component of hu-
man intelligence. As an example we can consider the pair of images in Figure 1,
which shows a complex situation involving articulated objects (human, carrots
and knife), the change of location and composition of objects. For humans it is
straightforward to draw a conclusion on what happened (a carrot was chopped
by the human). Humans have this extraordinary ability of performing visual rea-
soning on very complicated tasks while it remains unattainable for contemporary
computer vision algorithms [37,11].
The ability to perform visual reasoning in computer vision algorithms is still
an open problem. Attempts have been made for learning interactions between
dierent entities in images with promising results on Visual Question Answering.
There have been a number of attempts to equip neural models with reasoning
abilities by training them to solve Visual Question Answering (VQA) problems.
Among proposed solutions are prior-less data normalization [27], structuring
networks to model relationships [32,43] as well as more complex attention based
Object Level Visual Reasoning in Videos 3
mechanisms [18]. At the same time, it was shown that high performance on exist-
ing VQA datasets can be achieved by simply discovering biases in the data [21].
We extend these eorts to object level reasoning in videos . Since a video is
a temporal sequence, we leverage time as an explicit causal signal to identify
causal object relations. Our approach is related to the concept of the \arrow of
the time" [28] involving the \one-way direction" or \asymmetry" of time. Causal
event occurs before the event it aects ( A!B). In Figure 1 the knife was used
before the carrot switched over to the chopped-up state on the right side. For
a video classication problem, we want to identify a causal event Ahappening
in a video that aects its label B. But instead of identifying this causal event
directly from pixels we want to identify it from an object level perspective. We
believe that such an approach would be able to learn causal signals.
Following this hypothesis we propose to make a bridge between object de-
tection and activity recognition. Object detection allows us to extract low-level
information from a scene with all the present object instances and their semantic
meanings. However, detailed activity understanding requires reasoning over these
semantic structures, determining which objects were involved in interactions, of
what nature, and what were the results of these. To compound problems, the
semantic structure of a scene may change during a video (e.g. a new object can
appear, a person may make a move from one point to another one of the scene).
We propose an Object Relation Network (ORN), a neural network mod-
ule for reasoning between detected semantic object instances through space and
time. The ORN has potential to address these issues and conduct relational
reasoning over object interactions for the purpose of activity recognition. A set
of object detection masks ranging over dierent object categories and temporal
occurrences is input to the ORN. The ORN is able to infer pairwise relationships
between objects detected at varying dierent moments in time.
Code and object masks predictions will be publicly available6.
2 Related work
Action Recognition. Action recognition has a long history in computer vi-
sion. Pre-deep learning approaches in action recognition focused on handcrafted
spatio-temporal features including space-time interest points like SIFT-3D, HOG3D,
IDT and aggregated them using bag-of-words techniques. Some hand-crafted rep-
resentations, like dense trajectories [42], still give competitive performance and
are frequently combined with deep learning.
In the recent past, work has shifted to deep learning. Early attempts adapt
2D convolutional networks to videos through temporal pooling and 3D convolu-
tions [2,40]. 3D convolutions are now widely adopted for activity recognition with
the introduction of feature transfer by in
ating pre-trained 2D convolutional ker-
nels from image classication models trained on ImageNet/ILSVRC [31] through
3D kernels [6]. The downside of 3D kernels is their computational complexity
6https://github.com/fabienbaradel/object_level_visual_reasoning
4 Baradel et al.
and the large number of learnable parameters, leading to the introduction of
2.5D kernels, i.e. separable lters in the form of a 2D spatial kernel followed by
a temporal kernel [44]. An alternative to temporal convolutions are Recurrent
Neural Networks (RNNs) in their various gated forms (GRUs, LSTMs) [17,9].
Karpathy et al. [20] presented a wide study on dierent ways of connecting
information in spatial and temporal dimensions through convolutions and pool-
ing. On very general datasets with coarse activity classes they have showed that
there was a small margin between classifying individual frames and classifying
videos with more sophisticated temporal aggregation.
Simoyan et al. [35] proposed a widely adopted two-stream architecture for
action recognition which extracts two dierent streams, one processing raw RGB
input and one processing pre-computed optical
ow images. The method out-
performed the state of the art, but relies on rather small scale optical
ow com-
putations.
In slightly narrower settings, prior information on the video content can allow
more ne-grained models. Articulated pose is widely used in cases where humans
are guaranteed to be present [33]. Pose estimation and activity recognition as a
joint (multi-task) problem has recently shown to improve both tasks [25]. Some-
what related to our work, Structural RNNs [19] perform activity recognition
by integrating features from semantic objects and their relationships. However,
they handle the temporal evolution of tracked objects in videos with a set of
RNNs, each of which corresponds to cliques in a graph which models the spatio-
temporal relationships between these objects. This graph is hand-crafted man-
ually for each application, though related work provides learnable connections
via gating functions [8].
Attention models are a way to structure deep networks in an often generic
way. They are able to iteratively focus attention to specic parts in the data with-
out requiring prior knowledge about part or object positions. In activity recog-
nition, they have gained some traction in recent years, either as soft-attention
on articulated pose (joints) [36], on feature map cells [34,39], on time [45] or on
parts in raw RGB input through dierentiable crops [3].
When raw video data is globally fed into deep neural networks, they focus
on extracting spatio-temporal features and perform aggregations. It has been
shown that these techniques fail on challenging ne-grained datasets, which re-
quire learning long temporal dependencies and human-object interactions. A
concentrated eort has been made to create large scale datasets to overcome
these issues [13,12,23,14].
Relational Reasoning. Relational reasoning is a well studied eld for many
applications ranging from visual reasoning [32] to reasoning about physical
systems [4]. Battaglia et al. [4] introduce a fully-dierentiable network physics
engine called Interaction Network (IN). IN learns to predict several physical
systems such as gravitational systems, rigid body dynamics, and mass-spring
systems. It shows impressive results; however, it learns from a virtual environ-
ment, which provides access to virtually unlimited training examples. Following
the same perspective, Santoro et al. [32] introduced Relation Network (RN),
Object Level Visual Reasoning in Videos 5
a plug-in module for reasoning in deep networks. RN shows human-level per-
formance in Visual Question Answering (VQA) by inferring pairwise \object"
relations. However, in contrast to our work, the term \object" in [32] does not
refer to semantically meaningful entities, but to discrete cells in feature maps.
The number of interactions therefore grows with feature map resolutions, which
makes it dicult to scale. Furthermore, a recent study [21] has shown that some
of these results are subject to dataset bias and do not generalize well to small
changes in the settings of the dataset.
In the same line, a recent work [38] has shown promising results on discov-
ering objects and their interactions in an unsupervised manner using training
examples from virtual environments. In [41], attention and relational modules
are combined on a graph structure. From a dierent perspective, [27] show that
relational reasoning can be learned for visual reasoning in a data driven way with-
out any prior using conditional batch normalization with a feature-wise ane
transformation based on conditioning information. In an opposite approach, a
strong structural prior is learned in the form of a complex attention mecha-
nism: in [18], an external memory module combined with attention processes
over input images and text questions, performing iterative reasoning for VQA.
While most of the discussed work has been designed for VQA and for pre-
dictions on physical systems and environments, extensions have been proposed
for video understanding. Reasoning in videos on a mask or segmentation level
has been attempted for video prediction [24], where the goal was to leverage se-
mantic information to be able predict further into the future. Zhou et al [5] have
recently shown state-of-the-art performance on challenging datasets by extend-
ing Relation Network to video classication. Their chosen entities are frames, on
which they employ RN to reason on a temporal level only through pairwise frame
relations. The approach is promising, but restricted to temporal contextual in-
formation without an understanding on a local object level, which is provided
by our approach.
Reasoning over sets of objects is somewhat related to reasoning from unstruc-
tured data points, as done in PointNet [29], designed to learn from unordered sets
of points. PointNet shares many properties with DeepSet [46] which is a more
general framework for extracting information from sets of objects. To some ex-
tent, our work is related to PointNet, as we handle unordered sets of objects in a
permutation invariant way. However, we have an object relation viewpoint that
directly reasons over relationships between these semantic entities.
3 Object-level Visual Reasoning in Space and Time
Our goal is to extract multiple types of cues from a video sequence: interactions
between predicted objects and their semantic classes, as well as local and global
motion in the scene. We formulate this objective as a neural architecture with
two heads: an activity head and an object head . Figure 2 gives a functional
overview of the model. Both heads share common features up to a certain layer
shown in red in the gure. The activity head , shown in orange in the gure,
6 Baradel et al.
spatio-temporalblock
<latexit sha1_base64="5+bMsd7t45s92IqShGCyFnPZmYE=">AAACG3icbVBNS8NAFNzUrxq/qh69BIvgxZLUg/VW8OKxgrGFJpTN5rVdutkNuxuhhP4QL/4VLx5UPAke/Ddu2graOrAwzMzj7ZsoZVRp1/2ySiura+sb5U17a3tnd6+yf3CnRCYJ+EQwITsRVsAoB19TzaCTSsBJxKAdja4Kv30PUlHBb/U4hTDBA077lGBtpF7lPIhgQHlOgGuQE1ulhXGmIUmFxCwI7IgJMrID4PFPqFepujV3CmeZeHNSRXO0epWPIBYkS8w4YVipruemOsyx1JQwmNhBpiDFZIQH0DWU4wRUmE+PmzgnRomdvpDmce1M1d8TOU6UGieRSSZYD9WiV4j/ed1M9xthTnmaaeBktqifMUcLp2jKiakEotnYEEwkNX91yBBLTEwHyjYleIsnLxO/XruseTf1arMxb6OMjtAxOkUeukBNdI1ayEcEPaAn9IJerUfr2Xqz3mfRkjWfOUR/YH1+A2JZolk=</latexit><latexit sha1_base64="5+bMsd7t45s92IqShGCyFnPZmYE=">AAACG3icbVBNS8NAFNzUrxq/qh69BIvgxZLUg/VW8OKxgrGFJpTN5rVdutkNuxuhhP4QL/4VLx5UPAke/Ddu2graOrAwzMzj7ZsoZVRp1/2ySiura+sb5U17a3tnd6+yf3CnRCYJ+EQwITsRVsAoB19TzaCTSsBJxKAdja4Kv30PUlHBb/U4hTDBA077lGBtpF7lPIhgQHlOgGuQE1ulhXGmIUmFxCwI7IgJMrID4PFPqFepujV3CmeZeHNSRXO0epWPIBYkS8w4YVipruemOsyx1JQwmNhBpiDFZIQH0DWU4wRUmE+PmzgnRomdvpDmce1M1d8TOU6UGieRSSZYD9WiV4j/ed1M9xthTnmaaeBktqifMUcLp2jKiakEotnYEEwkNX91yBBLTEwHyjYleIsnLxO/XruseTf1arMxb6OMjtAxOkUeukBNdI1ayEcEPaAn9IJerUfr2Xqz3mfRkjWfOUR/YH1+A2JZolk=</latexit><latexit sha1_base64="5+bMsd7t45s92IqShGCyFnPZmYE=">AAACG3icbVBNS8NAFNzUrxq/qh69BIvgxZLUg/VW8OKxgrGFJpTN5rVdutkNuxuhhP4QL/4VLx5UPAke/Ddu2graOrAwzMzj7ZsoZVRp1/2ySiura+sb5U17a3tnd6+yf3CnRCYJ+EQwITsRVsAoB19TzaCTSsBJxKAdja4Kv30PUlHBb/U4hTDBA077lGBtpF7lPIhgQHlOgGuQE1ulhXGmIUmFxCwI7IgJMrID4PFPqFepujV3CmeZeHNSRXO0epWPIBYkS8w4YVipruemOsyx1JQwmNhBpiDFZIQH0DWU4wRUmE+PmzgnRomdvpDmce1M1d8TOU6UGieRSSZYD9WiV4j/ed1M9xthTnmaaeBktqifMUcLp2jKiakEotnYEEwkNX91yBBLTEwHyjYleIsnLxO/XruseTf1arMxb6OMjtAxOkUeukBNdI1ayEcEPaAn9IJerUfr2Xqz3mfRkjWfOUR/YH1+A2JZolk=</latexit><latexit sha1_base64="5+bMsd7t45s92IqShGCyFnPZmYE=">AAACG3icbVBNS8NAFNzUrxq/qh69BIvgxZLUg/VW8OKxgrGFJpTN5rVdutkNuxuhhP4QL/4VLx5UPAke/Ddu2graOrAwzMzj7ZsoZVRp1/2ySiura+sb5U17a3tnd6+yf3CnRCYJ+EQwITsRVsAoB19TzaCTSsBJxKAdja4Kv30PUlHBb/U4hTDBA077lGBtpF7lPIhgQHlOgGuQE1ulhXGmIUmFxCwI7IgJMrID4PFPqFepujV3CmeZeHNSRXO0epWPIBYkS8w4YVipruemOsyx1JQwmNhBpiDFZIQH0DWU4wRUmE+PmzgnRomdvpDmce1M1d8TOU6UGieRSSZYD9WiV4j/ed1M9xthTnmaaeBktqifMUcLp2jKiakEotnYEEwkNX91yBBLTEwHyjYleIsnLxO/XruseTf1arMxb6OMjtAxOkUeukBNdI1ayEcEPaAn9IJerUfr2Xqz3mfRkjWfOUR/YH1+A2JZolk=</latexit>base network<latexit sha1_base64="voWnq9SNdsZ2QgQqJJC9i25ut4M=">AAACEHicbVC9TsMwGHTKXwl/AUYWiwqpU5V0oWyVWBiLRGmlNqoc90tr1XEi2wFVUV+BhVdhYQDEysjG2+C0QYKWkyyd7r6z/V2QcKa0635ZpbX1jc2t8ra9s7u3f+AcHt2qOJUU2jTmsewGRAFnAtqaaQ7dRAKJAg6dYHKZ+507kIrF4kZPE/AjMhIsZJRoIw2caj+AERMZBaFBzuz8LixA38dyYvdBDH+cgVNxa+4ceJV4BamgAq2B89kfxjSNTJxyolTPcxPtZ0RqRjnM7H6qICF0QkbQM1SQCJSfzTea4TOjDHEYS3OExnP1dyIjkVLTKDCTEdFjtezl4n9eL9Vhw8+YSFINgi4eClOOdYzzevCQSaCaTw0hVDLzV0zHRBJqOlC2KcFbXnmVtOu1i5p3Xa80G0UbZXSCTlEVeegcNdEVaqE2ougBPaEX9Go9Ws/Wm/W+GC1ZReYY/YH18Q26Yp3C</latexit><latexit sha1_base64="voWnq9SNdsZ2QgQqJJC9i25ut4M=">AAACEHicbVC9TsMwGHTKXwl/AUYWiwqpU5V0oWyVWBiLRGmlNqoc90tr1XEi2wFVUV+BhVdhYQDEysjG2+C0QYKWkyyd7r6z/V2QcKa0635ZpbX1jc2t8ra9s7u3f+AcHt2qOJUU2jTmsewGRAFnAtqaaQ7dRAKJAg6dYHKZ+507kIrF4kZPE/AjMhIsZJRoIw2caj+AERMZBaFBzuz8LixA38dyYvdBDH+cgVNxa+4ceJV4BamgAq2B89kfxjSNTJxyolTPcxPtZ0RqRjnM7H6qICF0QkbQM1SQCJSfzTea4TOjDHEYS3OExnP1dyIjkVLTKDCTEdFjtezl4n9eL9Vhw8+YSFINgi4eClOOdYzzevCQSaCaTw0hVDLzV0zHRBJqOlC2KcFbXnmVtOu1i5p3Xa80G0UbZXSCTlEVeegcNdEVaqE2ougBPaEX9Go9Ws/Wm/W+GC1ZReYY/YH18Q26Yp3C</latexit><latexit sha1_base64="voWnq9SNdsZ2QgQqJJC9i25ut4M=">AAACEHicbVC9TsMwGHTKXwl/AUYWiwqpU5V0oWyVWBiLRGmlNqoc90tr1XEi2wFVUV+BhVdhYQDEysjG2+C0QYKWkyyd7r6z/V2QcKa0635ZpbX1jc2t8ra9s7u3f+AcHt2qOJUU2jTmsewGRAFnAtqaaQ7dRAKJAg6dYHKZ+507kIrF4kZPE/AjMhIsZJRoIw2caj+AERMZBaFBzuz8LixA38dyYvdBDH+cgVNxa+4ceJV4BamgAq2B89kfxjSNTJxyolTPcxPtZ0RqRjnM7H6qICF0QkbQM1SQCJSfzTea4TOjDHEYS3OExnP1dyIjkVLTKDCTEdFjtezl4n9eL9Vhw8+YSFINgi4eClOOdYzzevCQSaCaTw0hVDLzV0zHRBJqOlC2KcFbXnmVtOu1i5p3Xa80G0UbZXSCTlEVeegcNdEVaqE2ougBPaEX9Go9Ws/Wm/W+GC1ZReYY/YH18Q26Yp3C</latexit><latexit sha1_base64="voWnq9SNdsZ2QgQqJJC9i25ut4M=">AAACEHicbVC9TsMwGHTKXwl/AUYWiwqpU5V0oWyVWBiLRGmlNqoc90tr1XEi2wFVUV+BhVdhYQDEysjG2+C0QYKWkyyd7r6z/V2QcKa0635ZpbX1jc2t8ra9s7u3f+AcHt2qOJUU2jTmsewGRAFnAtqaaQ7dRAKJAg6dYHKZ+507kIrF4kZPE/AjMhIsZJRoIw2caj+AERMZBaFBzuz8LixA38dyYvdBDH+cgVNxa+4ceJV4BamgAq2B89kfxjSNTJxyolTPcxPtZ0RqRjnM7H6qICF0QkbQM1SQCJSfzTea4TOjDHEYS3OExnP1dyIjkVLTKDCTEdFjtezl4n9eL9Vhw8+YSFINgi4eClOOdYzzevCQSaCaTw0hVDLzV0zHRBJqOlC2KcFbXnmVtOu1i5p3Xa80G0UbZXSCTlEVeegcNdEVaqE2ougBPaEX9Go9Ws/Wm/W+GC1ZReYY/YH18Q26Yp3C</latexit>
object mask detector<latexit sha1_base64="iBk9qvVCo5x+LLjPnshRIAowXE4=">AAACGHicbVDLSgMxFM34rONr1KWbYBFclZlurLuCG5cVHFtoh5LJ3LaxmWRIMkIZ+htu/BU3LlTcduffmD4EbT0QOJxzLzfnxBln2vj+l7O2vrG5tV3acXf39g8OvaPjey1zRSGkkkvViokGzgSEhhkOrUwBSWMOzXh4PfWbj6A0k+LOjDKIUtIXrMcoMVbqen4nhj4TBQVhQI1dGT8ANTgleogTMJZL5XZAJD8TXa/sV/wZ8CoJFqSMFmh0vUknkTRP7TrlROt24GcmKogyjHIYu51cQ0bokPShbakgKeiomCUb43OrJLgnlX3C4Jn6e6MgqdajNLaTKTEDvexNxf+8dm56tahgIssNCDo/1Ms5NhJPa8IJUzY7H1lCqGL2r5gOiCLUdqBdW0KwHHmVhNXKVSW4rZbrtUUbJXSKztAFCtAlqqMb1EAhougJvaA39O48O6/Oh/M5H11zFjsn6A+cyTfnKaEK</latexit><latexit sha1_base64="iBk9qvVCo5x+LLjPnshRIAowXE4=">AAACGHicbVDLSgMxFM34rONr1KWbYBFclZlurLuCG5cVHFtoh5LJ3LaxmWRIMkIZ+htu/BU3LlTcduffmD4EbT0QOJxzLzfnxBln2vj+l7O2vrG5tV3acXf39g8OvaPjey1zRSGkkkvViokGzgSEhhkOrUwBSWMOzXh4PfWbj6A0k+LOjDKIUtIXrMcoMVbqen4nhj4TBQVhQI1dGT8ANTgleogTMJZL5XZAJD8TXa/sV/wZ8CoJFqSMFmh0vUknkTRP7TrlROt24GcmKogyjHIYu51cQ0bokPShbakgKeiomCUb43OrJLgnlX3C4Jn6e6MgqdajNLaTKTEDvexNxf+8dm56tahgIssNCDo/1Ms5NhJPa8IJUzY7H1lCqGL2r5gOiCLUdqBdW0KwHHmVhNXKVSW4rZbrtUUbJXSKztAFCtAlqqMb1EAhougJvaA39O48O6/Oh/M5H11zFjsn6A+cyTfnKaEK</latexit><latexit sha1_base64="iBk9qvVCo5x+LLjPnshRIAowXE4=">AAACGHicbVDLSgMxFM34rONr1KWbYBFclZlurLuCG5cVHFtoh5LJ3LaxmWRIMkIZ+htu/BU3LlTcduffmD4EbT0QOJxzLzfnxBln2vj+l7O2vrG5tV3acXf39g8OvaPjey1zRSGkkkvViokGzgSEhhkOrUwBSWMOzXh4PfWbj6A0k+LOjDKIUtIXrMcoMVbqen4nhj4TBQVhQI1dGT8ANTgleogTMJZL5XZAJD8TXa/sV/wZ8CoJFqSMFmh0vUknkTRP7TrlROt24GcmKogyjHIYu51cQ0bokPShbakgKeiomCUb43OrJLgnlX3C4Jn6e6MgqdajNLaTKTEDvexNxf+8dm56tahgIssNCDo/1Ms5NhJPa8IJUzY7H1lCqGL2r5gOiCLUdqBdW0KwHHmVhNXKVSW4rZbrtUUbJXSKztAFCtAlqqMb1EAhougJvaA39O48O6/Oh/M5H11zFjsn6A+cyTfnKaEK</latexit><latexit sha1_base64="iBk9qvVCo5x+LLjPnshRIAowXE4=">AAACGHicbVDLSgMxFM34rONr1KWbYBFclZlurLuCG5cVHFtoh5LJ3LaxmWRIMkIZ+htu/BU3LlTcduffmD4EbT0QOJxzLzfnxBln2vj+l7O2vrG5tV3acXf39g8OvaPjey1zRSGkkkvViokGzgSEhhkOrUwBSWMOzXh4PfWbj6A0k+LOjDKIUtIXrMcoMVbqen4nhj4TBQVhQI1dGT8ANTgleogTMJZL5XZAJD8TXa/sV/wZ8CoJFqSMFmh0vUknkTRP7TrlROt24GcmKogyjHIYu51cQ0bokPShbakgKeiomCUb43OrJLgnlX3C4Jn6e6MgqdajNLaTKTEDvexNxf+8dm56tahgIssNCDo/1Ms5NhJPa8IJUzY7H1lCqGL2r5gOiCLUdqBdW0KwHHmVhNXKVSW4rZbrtUUbJXSKztAFCtAlqqMb1EAhougJvaA39O48O6/Oh/M5H11zFjsn6A+cyTfnKaEK</latexit>activity head<latexit sha1_base64="tSmUcjZoatQbRNwVotvgoNbiChk=">AAACEXicbVBNS8NAFNzUrxq/qh69LBZBLyXpxXorePFYwdhCE8pm89ou3WzC7qZQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2/ClDOlHefLKq2tb2xulbftnd29/YPK4dG9SjJJwaMJT2QnJAo4E+Bppjl0UgkkDjm0w9H1zG+PQSqWiDs9SSGIyUCwPqNEG6lXufBDGDCRUxAa5NQmVLMx0xM8BBLZPojox+pVqk7NmQOvErcgVVSg1at8+lFCs9jEKSdKdV0n1UFOpGaUw9T2MwUpoSMygK6hgsSggnx+0hSfGSXC/USaJzSeq78TOYmVmsShmYyJHqplbyb+53Uz3W8EORNppkHQxaJ+xrFO8KwfHDEJVPOJIYRKZv6K6ZBI04tp0TYluMsnrxKvXruqubf1arNRtFFGJ+gUnSMXXaImukEt5CGKHtATekGv1qP1bL1Z74vRklVkjtEfWB/feK2eJg==</latexit><latexit sha1_base64="tSmUcjZoatQbRNwVotvgoNbiChk=">AAACEXicbVBNS8NAFNzUrxq/qh69LBZBLyXpxXorePFYwdhCE8pm89ou3WzC7qZQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2/ClDOlHefLKq2tb2xulbftnd29/YPK4dG9SjJJwaMJT2QnJAo4E+Bppjl0UgkkDjm0w9H1zG+PQSqWiDs9SSGIyUCwPqNEG6lXufBDGDCRUxAa5NQmVLMx0xM8BBLZPojox+pVqk7NmQOvErcgVVSg1at8+lFCs9jEKSdKdV0n1UFOpGaUw9T2MwUpoSMygK6hgsSggnx+0hSfGSXC/USaJzSeq78TOYmVmsShmYyJHqplbyb+53Uz3W8EORNppkHQxaJ+xrFO8KwfHDEJVPOJIYRKZv6K6ZBI04tp0TYluMsnrxKvXruqubf1arNRtFFGJ+gUnSMXXaImukEt5CGKHtATekGv1qP1bL1Z74vRklVkjtEfWB/feK2eJg==</latexit><latexit sha1_base64="tSmUcjZoatQbRNwVotvgoNbiChk=">AAACEXicbVBNS8NAFNzUrxq/qh69LBZBLyXpxXorePFYwdhCE8pm89ou3WzC7qZQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2/ClDOlHefLKq2tb2xulbftnd29/YPK4dG9SjJJwaMJT2QnJAo4E+Bppjl0UgkkDjm0w9H1zG+PQSqWiDs9SSGIyUCwPqNEG6lXufBDGDCRUxAa5NQmVLMx0xM8BBLZPojox+pVqk7NmQOvErcgVVSg1at8+lFCs9jEKSdKdV0n1UFOpGaUw9T2MwUpoSMygK6hgsSggnx+0hSfGSXC/USaJzSeq78TOYmVmsShmYyJHqplbyb+53Uz3W8EORNppkHQxaJ+xrFO8KwfHDEJVPOJIYRKZv6K6ZBI04tp0TYluMsnrxKvXruqubf1arNRtFFGJ+gUnSMXXaImukEt5CGKHtATekGv1qP1bL1Z74vRklVkjtEfWB/feK2eJg==</latexit><latexit sha1_base64="tSmUcjZoatQbRNwVotvgoNbiChk=">AAACEXicbVBNS8NAFNzUrxq/qh69LBZBLyXpxXorePFYwdhCE8pm89ou3WzC7qZQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2/ClDOlHefLKq2tb2xulbftnd29/YPK4dG9SjJJwaMJT2QnJAo4E+Bppjl0UgkkDjm0w9H1zG+PQSqWiDs9SSGIyUCwPqNEG6lXufBDGDCRUxAa5NQmVLMx0xM8BBLZPojox+pVqk7NmQOvErcgVVSg1at8+lFCs9jEKSdKdV0n1UFOpGaUw9T2MwUpoSMygK6hgsSggnx+0hSfGSXC/USaJzSeq78TOYmVmsShmYyJHqplbyb+53Uz3W8EORNppkHQxaJ+xrFO8KwfHDEJVPOJIYRKZv6K6ZBI04tp0TYluMsnrxKvXruqubf1arNRtFFGJ+gUnSMXXaImukEt5CGKHtATekGv1qP1bL1Z74vRklVkjtEfWB/feK2eJg==</latexit>object head<latexit sha1_base64="I0wT7S5QKSPfCmhqWZ9Vmd66HwA=">AAACD3icbVDLSgMxFM3UVx1foy7dBIvoqsx0Y90V3Lis4NhCW0omc9vGZpIhyQhl6Ce48VfcuFBx69adf2P6ELT1QOBwzr3cnBOlnGnj+19OYWV1bX2juOlube/s7nn7B7daZopCSCWXqhkRDZwJCA0zHJqpApJEHBrR8HLiN+5BaSbFjRml0ElIX7Aeo8RYqeudtiPoM5FTEAbU2JXRHVCDB0Bitw0i/jG6Xskv+1PgZRLMSQnNUe96n+1Y0iyx65QTrVuBn5pOTpRhlMPYbWcaUkKHpA8tSwVJQHfyaaAxPrFKjHtS2ScMnqq/N3KSaD1KIjuZEDPQi95E/M9rZaZX7eRMpJkBQWeHehnHRuJJOzhmysbnI0sIVcz+FdMBUYTaDrRrSwgWIy+TsFK+KAfXlVKtOm+jiI7QMTpDATpHNXSF6ihEFD2gJ/SCXp1H59l5c95nowVnvnOI/sD5+AaXkJ0c</latexit><latexit sha1_base64="I0wT7S5QKSPfCmhqWZ9Vmd66HwA=">AAACD3icbVDLSgMxFM3UVx1foy7dBIvoqsx0Y90V3Lis4NhCW0omc9vGZpIhyQhl6Ce48VfcuFBx69adf2P6ELT1QOBwzr3cnBOlnGnj+19OYWV1bX2juOlube/s7nn7B7daZopCSCWXqhkRDZwJCA0zHJqpApJEHBrR8HLiN+5BaSbFjRml0ElIX7Aeo8RYqeudtiPoM5FTEAbU2JXRHVCDB0Bitw0i/jG6Xskv+1PgZRLMSQnNUe96n+1Y0iyx65QTrVuBn5pOTpRhlMPYbWcaUkKHpA8tSwVJQHfyaaAxPrFKjHtS2ScMnqq/N3KSaD1KIjuZEDPQi95E/M9rZaZX7eRMpJkBQWeHehnHRuJJOzhmysbnI0sIVcz+FdMBUYTaDrRrSwgWIy+TsFK+KAfXlVKtOm+jiI7QMTpDATpHNXSF6ihEFD2gJ/SCXp1H59l5c95nowVnvnOI/sD5+AaXkJ0c</latexit><latexit sha1_base64="I0wT7S5QKSPfCmhqWZ9Vmd66HwA=">AAACD3icbVDLSgMxFM3UVx1foy7dBIvoqsx0Y90V3Lis4NhCW0omc9vGZpIhyQhl6Ce48VfcuFBx69adf2P6ELT1QOBwzr3cnBOlnGnj+19OYWV1bX2juOlube/s7nn7B7daZopCSCWXqhkRDZwJCA0zHJqpApJEHBrR8HLiN+5BaSbFjRml0ElIX7Aeo8RYqeudtiPoM5FTEAbU2JXRHVCDB0Bitw0i/jG6Xskv+1PgZRLMSQnNUe96n+1Y0iyx65QTrVuBn5pOTpRhlMPYbWcaUkKHpA8tSwVJQHfyaaAxPrFKjHtS2ScMnqq/N3KSaD1KIjuZEDPQi95E/M9rZaZX7eRMpJkBQWeHehnHRuJJOzhmysbnI0sIVcz+FdMBUYTaDrRrSwgWIy+TsFK+KAfXlVKtOm+jiI7QMTpDATpHNXSF6ihEFD2gJ/SCXp1H59l5c95nowVnvnOI/sD5+AaXkJ0c</latexit><latexit sha1_base64="I0wT7S5QKSPfCmhqWZ9Vmd66HwA=">AAACD3icbVDLSgMxFM3UVx1foy7dBIvoqsx0Y90V3Lis4NhCW0omc9vGZpIhyQhl6Ce48VfcuFBx69adf2P6ELT1QOBwzr3cnBOlnGnj+19OYWV1bX2juOlube/s7nn7B7daZopCSCWXqhkRDZwJCA0zHJqpApJEHBrR8HLiN+5BaSbFjRml0ElIX7Aeo8RYqeudtiPoM5FTEAbU2JXRHVCDB0Bitw0i/jG6Xskv+1PgZRLMSQnNUe96n+1Y0iyx65QTrVuBn5pOTpRhlMPYbWcaUkKHpA8tSwVJQHfyaaAxPrFKjHtS2ScMnqq/N3KSaD1KIjuZEDPQi95E/M9rZaZX7eRMpJkBQWeHehnHRuJJOzhmysbnI0sIVcz+FdMBUYTaDrRrSwgWIy+TsFK+KAfXlVKtOm+jiI7QMTpDATpHNXSF6ihEFD2gJ/SCXp1H59l5c95nowVnvnOI/sD5+AaXkJ0c</latexit>visual reasoningmodule
<latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit><latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit><latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit><latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit>RoI pool<latexit sha1_base64="4Rn5dqov7d08Zkstz1UFiuPjd8o=">AAACDHicbVDLSgMxFM34rOOr6tJNsAquykw31l3Bje6qOLbQDiWTuW1DM8mQZIQy9Afc+CtuXKi49QPc+TemD0FbDwQO59zDzT1Rypk2nvflLC2vrK6tFzbcza3tnd3i3v6dlpmiEFDJpWpGRANnAgLDDIdmqoAkEYdGNLgY+417UJpJcWuGKYQJ6QnWZZQYK3WKx+0IekzkFIQBNXJv5BVOpeRuG0T8o3aKJa/sTYAXiT8jJTRDvVP8bMeSZomNU060bvleasKcKMMoh5HbzjSkhA5ID1qWCpKADvPJNSN8YpUYd6WyTxg8UX8ncpJoPUwiO5kQ09fz3lj8z2tlplsNcybSzICg00XdjGMj8bgaHDMF1PChJYQqZv+KaZ8oQm0H2rUl+PMnL5KgUj4v+9eVUq06a6OADtEROkU+OkM1dInqKEAUPaAn9IJenUfn2Xlz3qejS84sc4D+wPn4Bg4mm7k=</latexit><latexit sha1_base64="4Rn5dqov7d08Zkstz1UFiuPjd8o=">AAACDHicbVDLSgMxFM34rOOr6tJNsAquykw31l3Bje6qOLbQDiWTuW1DM8mQZIQy9Afc+CtuXKi49QPc+TemD0FbDwQO59zDzT1Rypk2nvflLC2vrK6tFzbcza3tnd3i3v6dlpmiEFDJpWpGRANnAgLDDIdmqoAkEYdGNLgY+417UJpJcWuGKYQJ6QnWZZQYK3WKx+0IekzkFIQBNXJv5BVOpeRuG0T8o3aKJa/sTYAXiT8jJTRDvVP8bMeSZomNU060bvleasKcKMMoh5HbzjSkhA5ID1qWCpKADvPJNSN8YpUYd6WyTxg8UX8ncpJoPUwiO5kQ09fz3lj8z2tlplsNcybSzICg00XdjGMj8bgaHDMF1PChJYQqZv+KaZ8oQm0H2rUl+PMnL5KgUj4v+9eVUq06a6OADtEROkU+OkM1dInqKEAUPaAn9IJenUfn2Xlz3qejS84sc4D+wPn4Bg4mm7k=</latexit><latexit sha1_base64="4Rn5dqov7d08Zkstz1UFiuPjd8o=">AAACDHicbVDLSgMxFM34rOOr6tJNsAquykw31l3Bje6qOLbQDiWTuW1DM8mQZIQy9Afc+CtuXKi49QPc+TemD0FbDwQO59zDzT1Rypk2nvflLC2vrK6tFzbcza3tnd3i3v6dlpmiEFDJpWpGRANnAgLDDIdmqoAkEYdGNLgY+417UJpJcWuGKYQJ6QnWZZQYK3WKx+0IekzkFIQBNXJv5BVOpeRuG0T8o3aKJa/sTYAXiT8jJTRDvVP8bMeSZomNU060bvleasKcKMMoh5HbzjSkhA5ID1qWCpKADvPJNSN8YpUYd6WyTxg8UX8ncpJoPUwiO5kQ09fz3lj8z2tlplsNcybSzICg00XdjGMj8bgaHDMF1PChJYQqZv+KaZ8oQm0H2rUl+PMnL5KgUj4v+9eVUq06a6OADtEROkU+OkM1dInqKEAUPaAn9IJenUfn2Xlz3qejS84sc4D+wPn4Bg4mm7k=</latexit><latexit sha1_base64="4Rn5dqov7d08Zkstz1UFiuPjd8o=">AAACDHicbVDLSgMxFM34rOOr6tJNsAquykw31l3Bje6qOLbQDiWTuW1DM8mQZIQy9Afc+CtuXKi49QPc+TemD0FbDwQO59zDzT1Rypk2nvflLC2vrK6tFzbcza3tnd3i3v6dlpmiEFDJpWpGRANnAgLDDIdmqoAkEYdGNLgY+417UJpJcWuGKYQJ6QnWZZQYK3WKx+0IekzkFIQBNXJv5BVOpeRuG0T8o3aKJa/sTYAXiT8jJTRDvVP8bMeSZomNU060bvleasKcKMMoh5HbzjSkhA5ID1qWCpKADvPJNSN8YpUYd6WyTxg8UX8ncpJoPUwiO5kQ09fz3lj8z2tlplsNcybSzICg00XdjGMj8bgaHDMF1PChJYQqZv+KaZ8oQm0H2rUl+PMnL5KgUj4v+9eVUq06a6OADtEROkU+OkM1dInqKEAUPaAn9IJenUfn2Xlz3qejS84sc4D+wPn4Bg4mm7k=</latexit>activity loss<latexit sha1_base64="BcPLS//JtZk28cnHPQ8Mz/Eh5ig=">AAACEXicbVBNS8NAFNzUrxq/oh69LBZBLyXpxXorePFYwdpCG8pm89ou3WzC7qYQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2+ChDOlXffLKq2tb2xulbftnd29/QPn8Ohexamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Prmd+egFQsFnc6S8CPyFCwAaNEG6nvXPQCGDKRUxAa5NQmVLMJ0xnmsVJ2D0T4Y/Wdilt158CrxCtIBRVo9p3PXhjTNDJxyolSXc9NtJ8TqRnlMLV7qYKE0DEZQtdQQSJQfj4/aYrPjBLiQSzNExrP1d+JnERKZVFgJiOiR2rZm4n/ed1UD+p+zkSSahB0sWiQcqxjPOsHh0wC1TwzhFDJzF8xHRFpejEt2qYEb/nkVdKqVa+q3m2t0qgXbZTRCTpF58hDl6iBblATtRBFD+gJvaBX69F6tt6s98VoySoyx+gPrI9vwk+eVQ==</latexit><latexit sha1_base64="BcPLS//JtZk28cnHPQ8Mz/Eh5ig=">AAACEXicbVBNS8NAFNzUrxq/oh69LBZBLyXpxXorePFYwdpCG8pm89ou3WzC7qYQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2+ChDOlXffLKq2tb2xulbftnd29/QPn8Ohexamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Prmd+egFQsFnc6S8CPyFCwAaNEG6nvXPQCGDKRUxAa5NQmVLMJ0xnmsVJ2D0T4Y/Wdilt158CrxCtIBRVo9p3PXhjTNDJxyolSXc9NtJ8TqRnlMLV7qYKE0DEZQtdQQSJQfj4/aYrPjBLiQSzNExrP1d+JnERKZVFgJiOiR2rZm4n/ed1UD+p+zkSSahB0sWiQcqxjPOsHh0wC1TwzhFDJzF8xHRFpejEt2qYEb/nkVdKqVa+q3m2t0qgXbZTRCTpF58hDl6iBblATtRBFD+gJvaBX69F6tt6s98VoySoyx+gPrI9vwk+eVQ==</latexit><latexit sha1_base64="BcPLS//JtZk28cnHPQ8Mz/Eh5ig=">AAACEXicbVBNS8NAFNzUrxq/oh69LBZBLyXpxXorePFYwdpCG8pm89ou3WzC7qYQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2+ChDOlXffLKq2tb2xulbftnd29/QPn8Ohexamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Prmd+egFQsFnc6S8CPyFCwAaNEG6nvXPQCGDKRUxAa5NQmVLMJ0xnmsVJ2D0T4Y/Wdilt158CrxCtIBRVo9p3PXhjTNDJxyolSXc9NtJ8TqRnlMLV7qYKE0DEZQtdQQSJQfj4/aYrPjBLiQSzNExrP1d+JnERKZVFgJiOiR2rZm4n/ed1UD+p+zkSSahB0sWiQcqxjPOsHh0wC1TwzhFDJzF8xHRFpejEt2qYEb/nkVdKqVa+q3m2t0qgXbZTRCTpF58hDl6iBblATtRBFD+gJvaBX69F6tt6s98VoySoyx+gPrI9vwk+eVQ==</latexit><latexit sha1_base64="BcPLS//JtZk28cnHPQ8Mz/Eh5ig=">AAACEXicbVBNS8NAFNzUrxq/oh69LBZBLyXpxXorePFYwdpCG8pm89ou3WzC7qYQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2+ChDOlXffLKq2tb2xulbftnd29/QPn8Ohexamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Prmd+egFQsFnc6S8CPyFCwAaNEG6nvXPQCGDKRUxAa5NQmVLMJ0xnmsVJ2D0T4Y/Wdilt158CrxCtIBRVo9p3PXhjTNDJxyolSXc9NtJ8TqRnlMLV7qYKE0DEZQtdQQSJQfj4/aYrPjBLiQSzNExrP1d+JnERKZVFgJiOiR2rZm4n/ed1UD+p+zkSSahB0sWiQcqxjPOsHh0wC1TwzhFDJzF8xHRFpejEt2qYEb/nkVdKqVa+q3m2t0qgXbZTRCTpF58hDl6iBblATtRBFD+gJvaBX69F6tt6s98VoySoyx+gPrI9vwk+eVQ==</latexit>object class loss
<latexit sha1_base64="52a627od74BeVjKBj74VpiEI2Rg=">AAACFXicbVBNSwMxEM3Wr7p+rXr0EiyCF8tuL9ZbwYvHCq4ttKVk02kbm02WJCuUpb/Ci3/FiwcVr4I3/41pu4K2Pgg83puZzLwo4Uwb3/9yCiura+sbxU13a3tnd8/bP7jVMlUUQiq5VM2IaOBMQGiY4dBMFJA44tCIRpdTv3EPSjMpbsw4gU5MBoL1GSXGSl3vrB3BgImMgjCgJq6M7oAaTDnRGnOptdsG0fuxu17JL/sz4GUS5KSEctS73me7J2ka2/bZyFbgJ6aTEWUY5TBx26mGhNARGUDLUkFi0J1sdtYEn1ilh/tS2SfsTlP1d0dGYq3HcWQrY2KGetGbiv95rdT0q52MiSQ1IOj8o37KsZF4mhHuMWVD4GNLCFXM7orpkChCbQbatSEEiycvk7BSvigH15VSrZqnUURH6BidogCdoxq6QnUUIooe0BN6Qa/Oo/PsvDnv89KCk/ccoj9wPr4Bd6SfvQ==</latexit><latexit sha1_base64="52a627od74BeVjKBj74VpiEI2Rg=">AAACFXicbVBNSwMxEM3Wr7p+rXr0EiyCF8tuL9ZbwYvHCq4ttKVk02kbm02WJCuUpb/Ci3/FiwcVr4I3/41pu4K2Pgg83puZzLwo4Uwb3/9yCiura+sbxU13a3tnd8/bP7jVMlUUQiq5VM2IaOBMQGiY4dBMFJA44tCIRpdTv3EPSjMpbsw4gU5MBoL1GSXGSl3vrB3BgImMgjCgJq6M7oAaTDnRGnOptdsG0fuxu17JL/sz4GUS5KSEctS73me7J2ka2/bZyFbgJ6aTEWUY5TBx26mGhNARGUDLUkFi0J1sdtYEn1ilh/tS2SfsTlP1d0dGYq3HcWQrY2KGetGbiv95rdT0q52MiSQ1IOj8o37KsZF4mhHuMWVD4GNLCFXM7orpkChCbQbatSEEiycvk7BSvigH15VSrZqnUURH6BidogCdoxq6QnUUIooe0BN6Qa/Oo/PsvDnv89KCk/ccoj9wPr4Bd6SfvQ==</latexit><latexit sha1_base64="52a627od74BeVjKBj74VpiEI2Rg=">AAACFXicbVBNSwMxEM3Wr7p+rXr0EiyCF8tuL9ZbwYvHCq4ttKVk02kbm02WJCuUpb/Ci3/FiwcVr4I3/41pu4K2Pgg83puZzLwo4Uwb3/9yCiura+sbxU13a3tnd8/bP7jVMlUUQiq5VM2IaOBMQGiY4dBMFJA44tCIRpdTv3EPSjMpbsw4gU5MBoL1GSXGSl3vrB3BgImMgjCgJq6M7oAaTDnRGnOptdsG0fuxu17JL/sz4GUS5KSEctS73me7J2ka2/bZyFbgJ6aTEWUY5TBx26mGhNARGUDLUkFi0J1sdtYEn1ilh/tS2SfsTlP1d0dGYq3HcWQrY2KGetGbiv95rdT0q52MiSQ1IOj8o37KsZF4mhHuMWVD4GNLCFXM7orpkChCbQbatSEEiycvk7BSvigH15VSrZqnUURH6BidogCdoxq6QnUUIooe0BN6Qa/Oo/PsvDnv89KCk/ccoj9wPr4Bd6SfvQ==</latexit><latexit sha1_base64="52a627od74BeVjKBj74VpiEI2Rg=">AAACFXicbVBNSwMxEM3Wr7p+rXr0EiyCF8tuL9ZbwYvHCq4ttKVk02kbm02WJCuUpb/Ci3/FiwcVr4I3/41pu4K2Pgg83puZzLwo4Uwb3/9yCiura+sbxU13a3tnd8/bP7jVMlUUQiq5VM2IaOBMQGiY4dBMFJA44tCIRpdTv3EPSjMpbsw4gU5MBoL1GSXGSl3vrB3BgImMgjCgJq6M7oAaTDnRGnOptdsG0fuxu17JL/sz4GUS5KSEctS73me7J2ka2/bZyFbgJ6aTEWUY5TBx26mGhNARGUDLUkFi0J1sdtYEn1ilh/tS2SfsTlP1d0dGYq3HcWQrY2KGetGbiv95rdT0q52MiSQ1IOj8o37KsZF4mhHuMWVD4GNLCFXM7orpkChCbQbatSEEiycvk7BSvigH15VSrZqnUURH6BidogCdoxq6QnUUIooe0BN6Qa/Oo/PsvDnv89KCk/ccoj9wPr4Bd6SfvQ==</latexit>
pairwisetemporal sampling
<latexit sha1_base64="IMkmLAb5bdMJnxwm+q7K1alA/fA=">AAACIHicbVBNSwMxFMz6WdevqkcvwSJ4Kru9WG8FLx4ruFrolvI2fa2hSXZJskpZ+le8+Fe8eFDRm/4a01pBqwOBYWYeeW+STHBjg+DdW1hcWl5ZLa356xubW9vlnd1Lk+aaYcRSkepWAgYFVxhZbgW2Mo0gE4FXyfB04l/doDY8VRd2lGFHwkDxPmdgndQt1+MEB1wVDJVFPfYz4PqWG4xjalFmqQZBDUi3ihr4Mared7JbrgTVYAr6l4QzUiEzNLvlt7iXsly6cSbAmHYYZLZTgLacCRz7cW4wAzaEAbYdVSDRdIrphWN66JQe7afaPWXpVP05UYA0ZiQTl5Rgr828NxH/89q57dc7BVdZblGxr4/6uaA2pZO6aI9rZFaMHAGmuduVsmvQwFwHxnclhPMn/yVRrXpSDc9rlUZ91kaJ7JMDckRCckwa5Iw0SUQYuSMP5Ik8e/feo/fivX5FF7zZzB75Be/jE9LypLg=</latexit><latexit sha1_base64="IMkmLAb5bdMJnxwm+q7K1alA/fA=">AAACIHicbVBNSwMxFMz6WdevqkcvwSJ4Kru9WG8FLx4ruFrolvI2fa2hSXZJskpZ+le8+Fe8eFDRm/4a01pBqwOBYWYeeW+STHBjg+DdW1hcWl5ZLa356xubW9vlnd1Lk+aaYcRSkepWAgYFVxhZbgW2Mo0gE4FXyfB04l/doDY8VRd2lGFHwkDxPmdgndQt1+MEB1wVDJVFPfYz4PqWG4xjalFmqQZBDUi3ihr4Mared7JbrgTVYAr6l4QzUiEzNLvlt7iXsly6cSbAmHYYZLZTgLacCRz7cW4wAzaEAbYdVSDRdIrphWN66JQe7afaPWXpVP05UYA0ZiQTl5Rgr828NxH/89q57dc7BVdZblGxr4/6uaA2pZO6aI9rZFaMHAGmuduVsmvQwFwHxnclhPMn/yVRrXpSDc9rlUZ91kaJ7JMDckRCckwa5Iw0SUQYuSMP5Ik8e/feo/fivX5FF7zZzB75Be/jE9LypLg=</latexit><latexit sha1_base64="IMkmLAb5bdMJnxwm+q7K1alA/fA=">AAACIHicbVBNSwMxFMz6WdevqkcvwSJ4Kru9WG8FLx4ruFrolvI2fa2hSXZJskpZ+le8+Fe8eFDRm/4a01pBqwOBYWYeeW+STHBjg+DdW1hcWl5ZLa356xubW9vlnd1Lk+aaYcRSkepWAgYFVxhZbgW2Mo0gE4FXyfB04l/doDY8VRd2lGFHwkDxPmdgndQt1+MEB1wVDJVFPfYz4PqWG4xjalFmqQZBDUi3ihr4Mared7JbrgTVYAr6l4QzUiEzNLvlt7iXsly6cSbAmHYYZLZTgLacCRz7cW4wAzaEAbYdVSDRdIrphWN66JQe7afaPWXpVP05UYA0ZiQTl5Rgr828NxH/89q57dc7BVdZblGxr4/6uaA2pZO6aI9rZFaMHAGmuduVsmvQwFwHxnclhPMn/yVRrXpSDc9rlUZ91kaJ7JMDckRCckwa5Iw0SUQYuSMP5Ik8e/feo/fivX5FF7zZzB75Be/jE9LypLg=</latexit><latexit sha1_base64="IMkmLAb5bdMJnxwm+q7K1alA/fA=">AAACIHicbVBNSwMxFMz6WdevqkcvwSJ4Kru9WG8FLx4ruFrolvI2fa2hSXZJskpZ+le8+Fe8eFDRm/4a01pBqwOBYWYeeW+STHBjg+DdW1hcWl5ZLa356xubW9vlnd1Lk+aaYcRSkepWAgYFVxhZbgW2Mo0gE4FXyfB04l/doDY8VRd2lGFHwkDxPmdgndQt1+MEB1wVDJVFPfYz4PqWG4xjalFmqQZBDUi3ihr4Mared7JbrgTVYAr6l4QzUiEzNLvlt7iXsly6cSbAmHYYZLZTgLacCRz7cW4wAzaEAbYdVSDRdIrphWN66JQe7afaPWXpVP05UYA0ZiQTl5Rgr828NxH/89q57dc7BVdZblGxr4/6uaA2pZO6aI9rZFaMHAGmuduVsmvQwFwHxnclhPMn/yVRrXpSDc9rlUZ91kaJ7JMDckRCckwa5Iw0SUQYuSMP5Ik8e/feo/fivX5FF7zZzB75Be/jE9LypLg=</latexit>
activity features<latexit sha1_base64="r+7eRHfo+xMX1EIiEEcQFlIGw6w=">AAACFXicbVC7SgNBFJ2Nr7i+opY2g0GwMeymMXYBG8sIxgSSEGYnd5Mhs7PLzN1AWPIVNv6KjYWKrWDn3zh5CJp4YOBwzj3cuSdIpDDoeV9Obm19Y3Mrv+3u7O7tHxQOj+5NnGoOdR7LWDcDZkAKBXUUKKGZaGBRIKERDK+nfmME2ohY3eE4gU7E+kqEgjO0Urdw0Q6gL1TGQSHoics4ipHAMQ2BYarBuG1QvR+7Wyh6JW8Gukr8BSmSBWrdwme7F/M0snEumTEt30uwkzGNgkuYuO3UQML4kPWhZaliEZhONjtrQs+s0qNhrO1TSGfq70TGImPGUWAnI4YDs+xNxf+8VophpZMJlaQIis8XhamkGNNpR7QnNHCUY0sY18L+lfIB07Yb26RrS/CXT14l9XLpquTflovVyqKNPDkhp+Sc+OSSVMkNqZE64eSBPJEX8uo8Os/Om/M+H805i8wx+QPn4xsNc6Ab</latexit><latexit sha1_base64="r+7eRHfo+xMX1EIiEEcQFlIGw6w=">AAACFXicbVC7SgNBFJ2Nr7i+opY2g0GwMeymMXYBG8sIxgSSEGYnd5Mhs7PLzN1AWPIVNv6KjYWKrWDn3zh5CJp4YOBwzj3cuSdIpDDoeV9Obm19Y3Mrv+3u7O7tHxQOj+5NnGoOdR7LWDcDZkAKBXUUKKGZaGBRIKERDK+nfmME2ohY3eE4gU7E+kqEgjO0Urdw0Q6gL1TGQSHoics4ipHAMQ2BYarBuG1QvR+7Wyh6JW8Gukr8BSmSBWrdwme7F/M0snEumTEt30uwkzGNgkuYuO3UQML4kPWhZaliEZhONjtrQs+s0qNhrO1TSGfq70TGImPGUWAnI4YDs+xNxf+8VophpZMJlaQIis8XhamkGNNpR7QnNHCUY0sY18L+lfIB07Yb26RrS/CXT14l9XLpquTflovVyqKNPDkhp+Sc+OSSVMkNqZE64eSBPJEX8uo8Os/Om/M+H805i8wx+QPn4xsNc6Ab</latexit><latexit sha1_base64="r+7eRHfo+xMX1EIiEEcQFlIGw6w=">AAACFXicbVC7SgNBFJ2Nr7i+opY2g0GwMeymMXYBG8sIxgSSEGYnd5Mhs7PLzN1AWPIVNv6KjYWKrWDn3zh5CJp4YOBwzj3cuSdIpDDoeV9Obm19Y3Mrv+3u7O7tHxQOj+5NnGoOdR7LWDcDZkAKBXUUKKGZaGBRIKERDK+nfmME2ohY3eE4gU7E+kqEgjO0Urdw0Q6gL1TGQSHoics4ipHAMQ2BYarBuG1QvR+7Wyh6JW8Gukr8BSmSBWrdwme7F/M0snEumTEt30uwkzGNgkuYuO3UQML4kPWhZaliEZhONjtrQs+s0qNhrO1TSGfq70TGImPGUWAnI4YDs+xNxf+8VophpZMJlaQIis8XhamkGNNpR7QnNHCUY0sY18L+lfIB07Yb26RrS/CXT14l9XLpquTflovVyqKNPDkhp+Sc+OSSVMkNqZE64eSBPJEX8uo8Os/Om/M+H805i8wx+QPn4xsNc6Ab</latexit><latexit sha1_base64="r+7eRHfo+xMX1EIiEEcQFlIGw6w=">AAACFXicbVC7SgNBFJ2Nr7i+opY2g0GwMeymMXYBG8sIxgSSEGYnd5Mhs7PLzN1AWPIVNv6KjYWKrWDn3zh5CJp4YOBwzj3cuSdIpDDoeV9Obm19Y3Mrv+3u7O7tHxQOj+5NnGoOdR7LWDcDZkAKBXUUKKGZaGBRIKERDK+nfmME2ohY3eE4gU7E+kqEgjO0Urdw0Q6gL1TGQSHoics4ipHAMQ2BYarBuG1QvR+7Wyh6JW8Gukr8BSmSBWrdwme7F/M0snEumTEt30uwkzGNgkuYuO3UQML4kPWhZaliEZhONjtrQs+s0qNhrO1TSGfq70TGImPGUWAnI4YDs+xNxf+8VophpZMJlaQIis8XhamkGNNpR7QnNHCUY0sY18L+lfIB07Yb26RrS/CXT14l9XLpquTflovVyqKNPDkhp+Sc+OSSVMkNqZE64eSBPJEX8uo8Os/Om/M+H805i8wx+QPn4xsNc6Ab</latexit>n⇥2Dsets ofobject features
<latexit sha1_base64="viP+YHsQOd91KFvZysvFMJg/WIY=">AAACK3icbVBNSwMxFMz67fpV9egl2Aqeym4v6k3Qg8cKVgvdUrLp2xrNJkvyVihLf5AX/4ogHqx49X+Y1graOhAYZt6Q9ybOpLAYBENvbn5hcWl5ZdVfW9/Y3Cpt71xbnRsODa6lNs2YWZBCQQMFSmhmBlgaS7iJ789G/s0DGCu0usJ+Bu2U9ZRIBGfopE7pLIqhJ1TBQSGYgV9RRYQiBTuonVeoBbRUJ1Hk6/gOONIEGOYGrB+B6v6EOqVyUA3GoLMknJAymaDeKb1EXc3z1MW5ZNa2wiDDdsEMCi5h4Ee5hYzxe9aDlqOKuX3axfjYAT1wSpcm2rinkI7V34mCpdb209hNpgxv7bQ3Ev/zWjkmx+1CqCxHUPz7oySXFDUdNUe7wrgKZN8Rxo1wu1J+ywzjrgPruxLC6ZNnSaNWPamGl7Xy6fGkjRWyR/bJIQnJETklF6ROGoSTR/JM3sjQe/JevXfv43t0zptkdskfeJ9f7xuoOQ==</latexit><latexit sha1_base64="viP+YHsQOd91KFvZysvFMJg/WIY=">AAACK3icbVBNSwMxFMz67fpV9egl2Aqeym4v6k3Qg8cKVgvdUrLp2xrNJkvyVihLf5AX/4ogHqx49X+Y1graOhAYZt6Q9ybOpLAYBENvbn5hcWl5ZdVfW9/Y3Cpt71xbnRsODa6lNs2YWZBCQQMFSmhmBlgaS7iJ789G/s0DGCu0usJ+Bu2U9ZRIBGfopE7pLIqhJ1TBQSGYgV9RRYQiBTuonVeoBbRUJ1Hk6/gOONIEGOYGrB+B6v6EOqVyUA3GoLMknJAymaDeKb1EXc3z1MW5ZNa2wiDDdsEMCi5h4Ee5hYzxe9aDlqOKuX3axfjYAT1wSpcm2rinkI7V34mCpdb209hNpgxv7bQ3Ev/zWjkmx+1CqCxHUPz7oySXFDUdNUe7wrgKZN8Rxo1wu1J+ywzjrgPruxLC6ZNnSaNWPamGl7Xy6fGkjRWyR/bJIQnJETklF6ROGoSTR/JM3sjQe/JevXfv43t0zptkdskfeJ9f7xuoOQ==</latexit><latexit sha1_base64="viP+YHsQOd91KFvZysvFMJg/WIY=">AAACK3icbVBNSwMxFMz67fpV9egl2Aqeym4v6k3Qg8cKVgvdUrLp2xrNJkvyVihLf5AX/4ogHqx49X+Y1graOhAYZt6Q9ybOpLAYBENvbn5hcWl5ZdVfW9/Y3Cpt71xbnRsODa6lNs2YWZBCQQMFSmhmBlgaS7iJ789G/s0DGCu0usJ+Bu2U9ZRIBGfopE7pLIqhJ1TBQSGYgV9RRYQiBTuonVeoBbRUJ1Hk6/gOONIEGOYGrB+B6v6EOqVyUA3GoLMknJAymaDeKb1EXc3z1MW5ZNa2wiDDdsEMCi5h4Ee5hYzxe9aDlqOKuX3axfjYAT1wSpcm2rinkI7V34mCpdb209hNpgxv7bQ3Ev/zWjkmx+1CqCxHUPz7oySXFDUdNUe7wrgKZN8Rxo1wu1J+ywzjrgPruxLC6ZNnSaNWPamGl7Xy6fGkjRWyR/bJIQnJETklF6ROGoSTR/JM3sjQe/JevXfv43t0zptkdskfeJ9f7xuoOQ==</latexit><latexit sha1_base64="viP+YHsQOd91KFvZysvFMJg/WIY=">AAACK3icbVBNSwMxFMz67fpV9egl2Aqeym4v6k3Qg8cKVgvdUrLp2xrNJkvyVihLf5AX/4ogHqx49X+Y1graOhAYZt6Q9ybOpLAYBENvbn5hcWl5ZdVfW9/Y3Cpt71xbnRsODa6lNs2YWZBCQQMFSmhmBlgaS7iJ789G/s0DGCu0usJ+Bu2U9ZRIBGfopE7pLIqhJ1TBQSGYgV9RRYQiBTuonVeoBbRUJ1Hk6/gOONIEGOYGrB+B6v6EOqVyUA3GoLMknJAymaDeKb1EXc3z1MW5ZNa2wiDDdsEMCi5h4Ee5hYzxe9aDlqOKuX3axfjYAT1wSpcm2rinkI7V34mCpdb209hNpgxv7bQ3Ev/zWjkmx+1CqCxHUPz7oySXFDUdNUe7wrgKZN8Rxo1wu1J+ywzjrgPruxLC6ZNnSaNWPamGl7Xy6fGkjRWyR/bJIQnJETklF6ROGoSTR/JM3sjQe/JevXfv43t0zptkdskfeJ9f7xuoOQ==</latexit>
n⇥2Dsets ofobject masks
<latexit sha1_base64="wsaTEZjazwjgqpKPonLLv8b2h38=">AAACKHicbVDLSgMxFM3UVx1fVZdugq3gqsx0Y90VdOGygrWFTimZ9E4bm0mGJCOUob/jxl9xo6DSrV9i+hC09UDgcM495N4TJpxp43kTJ7e2vrG5ld92d3b39g8Kh0f3WqaKQoNKLlUrJBo4E9AwzHBoJQpIHHJohsOrqd98BKWZFHdmlEAnJn3BIkaJsVK3UAtC6DORURAG1NgtiSwwLAY9rlyXsAajsYyCwJXhA1CDY6KH2g1A9H4S3ULRK3sz4FXiL0gRLVDvFt6CnqRpbOOUE63bvpeYTkaUYZTD2A1SDQmhQ9KHtqWC2GU62ezSMT6zSg9HUtknDJ6pvxMZibUexaGdjIkZ6GVvKv7ntVMTVTsZE0lqQND5R1HKsZF4WhvuMWXv5yNLCFXM7orpgChCbQfatSX4yyevkkalfFn2byvFWnXRRh6doFN0jnx0gWroBtVRA1H0hF7QO/pwnp1X59OZzEdzziJzjP7A+foGUQGm2w==</latexit><latexit sha1_base64="wsaTEZjazwjgqpKPonLLv8b2h38=">AAACKHicbVDLSgMxFM3UVx1fVZdugq3gqsx0Y90VdOGygrWFTimZ9E4bm0mGJCOUob/jxl9xo6DSrV9i+hC09UDgcM495N4TJpxp43kTJ7e2vrG5ld92d3b39g8Kh0f3WqaKQoNKLlUrJBo4E9AwzHBoJQpIHHJohsOrqd98BKWZFHdmlEAnJn3BIkaJsVK3UAtC6DORURAG1NgtiSwwLAY9rlyXsAajsYyCwJXhA1CDY6KH2g1A9H4S3ULRK3sz4FXiL0gRLVDvFt6CnqRpbOOUE63bvpeYTkaUYZTD2A1SDQmhQ9KHtqWC2GU62ezSMT6zSg9HUtknDJ6pvxMZibUexaGdjIkZ6GVvKv7ntVMTVTsZE0lqQND5R1HKsZF4WhvuMWXv5yNLCFXM7orpgChCbQfatSX4yyevkkalfFn2byvFWnXRRh6doFN0jnx0gWroBtVRA1H0hF7QO/pwnp1X59OZzEdzziJzjP7A+foGUQGm2w==</latexit><latexit sha1_base64="wsaTEZjazwjgqpKPonLLv8b2h38=">AAACKHicbVDLSgMxFM3UVx1fVZdugq3gqsx0Y90VdOGygrWFTimZ9E4bm0mGJCOUob/jxl9xo6DSrV9i+hC09UDgcM495N4TJpxp43kTJ7e2vrG5ld92d3b39g8Kh0f3WqaKQoNKLlUrJBo4E9AwzHBoJQpIHHJohsOrqd98BKWZFHdmlEAnJn3BIkaJsVK3UAtC6DORURAG1NgtiSwwLAY9rlyXsAajsYyCwJXhA1CDY6KH2g1A9H4S3ULRK3sz4FXiL0gRLVDvFt6CnqRpbOOUE63bvpeYTkaUYZTD2A1SDQmhQ9KHtqWC2GU62ezSMT6zSg9HUtknDJ6pvxMZibUexaGdjIkZ6GVvKv7ntVMTVTsZE0lqQND5R1HKsZF4WhvuMWXv5yNLCFXM7orpgChCbQfatSX4yyevkkalfFn2byvFWnXRRh6doFN0jnx0gWroBtVRA1H0hF7QO/pwnp1X59OZzEdzziJzjP7A+foGUQGm2w==</latexit><latexit sha1_base64="wsaTEZjazwjgqpKPonLLv8b2h38=">AAACKHicbVDLSgMxFM3UVx1fVZdugq3gqsx0Y90VdOGygrWFTimZ9E4bm0mGJCOUob/jxl9xo6DSrV9i+hC09UDgcM495N4TJpxp43kTJ7e2vrG5ld92d3b39g8Kh0f3WqaKQoNKLlUrJBo4E9AwzHBoJQpIHHJohsOrqd98BKWZFHdmlEAnJn3BIkaJsVK3UAtC6DORURAG1NgtiSwwLAY9rlyXsAajsYyCwJXhA1CDY6KH2g1A9H4S3ULRK3sz4FXiL0gRLVDvFt6CnqRpbOOUE63bvpeYTkaUYZTD2A1SDQmhQ9KHtqWC2GU62ezSMT6zSg9HUtknDJ6pvxMZibUexaGdjIkZ6GVvKv7ntVMTVTsZE0lqQND5R1HKsZF4WhvuMWXv5yNLCFXM7orpgChCbQfatSX4yyevkkalfFn2byvFWnXRRh6doFN0jnx0gWroBtVRA1H0hF7QO/pwnp1X59OZzEdzziJzjP7A+foGUQGm2w==</latexit>activity loss<latexit sha1_base64="BcPLS//JtZk28cnHPQ8Mz/Eh5ig=">AAACEXicbVBNS8NAFNzUrxq/oh69LBZBLyXpxXorePFYwdpCG8pm89ou3WzC7qYQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2+ChDOlXffLKq2tb2xulbftnd29/QPn8Ohexamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Prmd+egFQsFnc6S8CPyFCwAaNEG6nvXPQCGDKRUxAa5NQmVLMJ0xnmsVJ2D0T4Y/Wdilt158CrxCtIBRVo9p3PXhjTNDJxyolSXc9NtJ8TqRnlMLV7qYKE0DEZQtdQQSJQfj4/aYrPjBLiQSzNExrP1d+JnERKZVFgJiOiR2rZm4n/ed1UD+p+zkSSahB0sWiQcqxjPOsHh0wC1TwzhFDJzF8xHRFpejEt2qYEb/nkVdKqVa+q3m2t0qgXbZTRCTpF58hDl6iBblATtRBFD+gJvaBX69F6tt6s98VoySoyx+gPrI9vwk+eVQ==</latexit><latexit sha1_base64="BcPLS//JtZk28cnHPQ8Mz/Eh5ig=">AAACEXicbVBNS8NAFNzUrxq/oh69LBZBLyXpxXorePFYwdpCG8pm89ou3WzC7qYQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2+ChDOlXffLKq2tb2xulbftnd29/QPn8Ohexamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Prmd+egFQsFnc6S8CPyFCwAaNEG6nvXPQCGDKRUxAa5NQmVLMJ0xnmsVJ2D0T4Y/Wdilt158CrxCtIBRVo9p3PXhjTNDJxyolSXc9NtJ8TqRnlMLV7qYKE0DEZQtdQQSJQfj4/aYrPjBLiQSzNExrP1d+JnERKZVFgJiOiR2rZm4n/ed1UD+p+zkSSahB0sWiQcqxjPOsHh0wC1TwzhFDJzF8xHRFpejEt2qYEb/nkVdKqVa+q3m2t0qgXbZTRCTpF58hDl6iBblATtRBFD+gJvaBX69F6tt6s98VoySoyx+gPrI9vwk+eVQ==</latexit><latexit sha1_base64="BcPLS//JtZk28cnHPQ8Mz/Eh5ig=">AAACEXicbVBNS8NAFNzUrxq/oh69LBZBLyXpxXorePFYwdpCG8pm89ou3WzC7qYQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2+ChDOlXffLKq2tb2xulbftnd29/QPn8Ohexamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Prmd+egFQsFnc6S8CPyFCwAaNEG6nvXPQCGDKRUxAa5NQmVLMJ0xnmsVJ2D0T4Y/Wdilt158CrxCtIBRVo9p3PXhjTNDJxyolSXc9NtJ8TqRnlMLV7qYKE0DEZQtdQQSJQfj4/aYrPjBLiQSzNExrP1d+JnERKZVFgJiOiR2rZm4n/ed1UD+p+zkSSahB0sWiQcqxjPOsHh0wC1TwzhFDJzF8xHRFpejEt2qYEb/nkVdKqVa+q3m2t0qgXbZTRCTpF58hDl6iBblATtRBFD+gJvaBX69F6tt6s98VoySoyx+gPrI9vwk+eVQ==</latexit><latexit sha1_base64="BcPLS//JtZk28cnHPQ8Mz/Eh5ig=">AAACEXicbVBNS8NAFNzUrxq/oh69LBZBLyXpxXorePFYwdpCG8pm89ou3WzC7qYQQn+DF/+KFw8qXr1589+4bSNo68DCMPOGt2+ChDOlXffLKq2tb2xulbftnd29/QPn8Ohexamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Prmd+egFQsFnc6S8CPyFCwAaNEG6nvXPQCGDKRUxAa5NQmVLMJ0xnmsVJ2D0T4Y/Wdilt158CrxCtIBRVo9p3PXhjTNDJxyolSXc9NtJ8TqRnlMLV7qYKE0DEZQtdQQSJQfj4/aYrPjBLiQSzNExrP1d+JnERKZVFgJiOiR2rZm4n/ed1UD+p+zkSSahB0sWiQcqxjPOsHh0wC1TwzhFDJzF8xHRFpejEt2qYEb/nkVdKqVa+q3m2t0qgXbZTRCTpF58hDl6iBblATtRBFD+gJvaBX69F6tt6s98VoySoyx+gPrI9vwk+eVQ==</latexit>U<latexit sha1_base64="7MGsePtR5b4Fn9+IJvBQ7gwy4pE=">AAACFHicbVBNS8NAEN34WeNX1KOXYCt4kJL0ot6KXjxWMLbQhLLZTNqlm03Y3Qgl9E948a948aDi1YM3/42btoK2Diz7eG8eM/PCjFGpHOfLWFpeWV1br2yYm1vbO7vW3v6dTHNBwCMpS0UnxBIY5eApqhh0MgE4CRm0w+FVqbfvQUia8ls1yiBIcJ/TmBKsNNWzTv0Q+pQXBLgCMTZrfpiySI4S/RXeuGb6wKMftWdVnbozKXsRuDNQRbNq9axPP0pJnmg7YVjKrutkKiiwUJQwGJt+LiHDZIj70NWQ4wRkUEyuGtvHmonsOBX6cWVP2N+OAieyXFR3JlgN5LxWkv9p3VzF50FBeZYr4GQ6KM6ZrVK7jMiOqACi2EgDTATVu9pkgAUmOgNp6hDc+ZMXgdeoX9Tdm0a1eTlLo4IO0RE6QS46Q010jVrIQwQ9oCf0gl6NR+PZeDPep61LxsxzgP6U8fENqWOfVw==</latexit><latexit sha1_base64="7MGsePtR5b4Fn9+IJvBQ7gwy4pE=">AAACFHicbVBNS8NAEN34WeNX1KOXYCt4kJL0ot6KXjxWMLbQhLLZTNqlm03Y3Qgl9E948a948aDi1YM3/42btoK2Diz7eG8eM/PCjFGpHOfLWFpeWV1br2yYm1vbO7vW3v6dTHNBwCMpS0UnxBIY5eApqhh0MgE4CRm0w+FVqbfvQUia8ls1yiBIcJ/TmBKsNNWzTv0Q+pQXBLgCMTZrfpiySI4S/RXeuGb6wKMftWdVnbozKXsRuDNQRbNq9axPP0pJnmg7YVjKrutkKiiwUJQwGJt+LiHDZIj70NWQ4wRkUEyuGtvHmonsOBX6cWVP2N+OAieyXFR3JlgN5LxWkv9p3VzF50FBeZYr4GQ6KM6ZrVK7jMiOqACi2EgDTATVu9pkgAUmOgNp6hDc+ZMXgdeoX9Tdm0a1eTlLo4IO0RE6QS46Q010jVrIQwQ9oCf0gl6NR+PZeDPep61LxsxzgP6U8fENqWOfVw==</latexit><latexit sha1_base64="7MGsePtR5b4Fn9+IJvBQ7gwy4pE=">AAACFHicbVBNS8NAEN34WeNX1KOXYCt4kJL0ot6KXjxWMLbQhLLZTNqlm03Y3Qgl9E948a948aDi1YM3/42btoK2Diz7eG8eM/PCjFGpHOfLWFpeWV1br2yYm1vbO7vW3v6dTHNBwCMpS0UnxBIY5eApqhh0MgE4CRm0w+FVqbfvQUia8ls1yiBIcJ/TmBKsNNWzTv0Q+pQXBLgCMTZrfpiySI4S/RXeuGb6wKMftWdVnbozKXsRuDNQRbNq9axPP0pJnmg7YVjKrutkKiiwUJQwGJt+LiHDZIj70NWQ4wRkUEyuGtvHmonsOBX6cWVP2N+OAieyXFR3JlgN5LxWkv9p3VzF50FBeZYr4GQ6KM6ZrVK7jMiOqACi2EgDTATVu9pkgAUmOgNp6hDc+ZMXgdeoX9Tdm0a1eTlLo4IO0RE6QS46Q010jVrIQwQ9oCf0gl6NR+PZeDPep61LxsxzgP6U8fENqWOfVw==</latexit><latexit sha1_base64="7MGsePtR5b4Fn9+IJvBQ7gwy4pE=">AAACFHicbVBNS8NAEN34WeNX1KOXYCt4kJL0ot6KXjxWMLbQhLLZTNqlm03Y3Qgl9E948a948aDi1YM3/42btoK2Diz7eG8eM/PCjFGpHOfLWFpeWV1br2yYm1vbO7vW3v6dTHNBwCMpS0UnxBIY5eApqhh0MgE4CRm0w+FVqbfvQUia8ls1yiBIcJ/TmBKsNNWzTv0Q+pQXBLgCMTZrfpiySI4S/RXeuGb6wKMftWdVnbozKXsRuDNQRbNq9axPP0pJnmg7YVjKrutkKiiwUJQwGJt+LiHDZIj70NWQ4wRkUEyuGtvHmonsOBX6cWVP2N+OAieyXFR3JlgN5LxWkv9p3VzF50FBeZYr4GQ6KM6ZrVK7jMiOqACi2EgDTATVu9pkgAUmOgNp6hDc+ZMXgdeoX9Tdm0a1eTlLo4IO0RE6QS46Q010jVrIQwQ9oCf0gl6NR+PZeDPep61LxsxzgP6U8fENqWOfVw==</latexit>V<latexit sha1_base64="OvpqHdnJr07GAq4B5qFqQBwG38Y=">AAACFHicbVA9T8MwEHXKVwlfBUYWixaJAVVJF2CrYGEsEmkrNVXlONfWquNEtoNURf0TLPwVFgZArAxs/BuctkjQcpLlp/fu6e5ekHCmtON8WYWV1bX1jeKmvbW9s7tX2j9oqjiVFDwa81i2A6KAMwGeZppDO5FAooBDKxhd53rrHqRisbjT4wS6ERkI1meUaEP1Smd+AAMmMgpCg5zYFT+IeajGkfmy5qRi+yDCH7VXKjtVZ1p4GbhzUEbzavRKn34Y0zQydsqJUh3XSXQ3I1IzymFi+6mChNARGUDHQEEiUN1setUEnxgmxP1Ymic0nrK/HRmJVL6o6YyIHqpFLSf/0zqp7l90MyaSVIOgs0H9lGMd4zwiHDIJVPOxAYRKZnbFdEgkoSYDZZsQ3MWTl4FXq15W3dtauX41T6OIjtAxOkUuOkd1dIMayEMUPaAn9IJerUfr2Xqz3metBWvuOUR/yvr4Bqr1n1g=</latexit><latexit sha1_base64="OvpqHdnJr07GAq4B5qFqQBwG38Y=">AAACFHicbVA9T8MwEHXKVwlfBUYWixaJAVVJF2CrYGEsEmkrNVXlONfWquNEtoNURf0TLPwVFgZArAxs/BuctkjQcpLlp/fu6e5ekHCmtON8WYWV1bX1jeKmvbW9s7tX2j9oqjiVFDwa81i2A6KAMwGeZppDO5FAooBDKxhd53rrHqRisbjT4wS6ERkI1meUaEP1Smd+AAMmMgpCg5zYFT+IeajGkfmy5qRi+yDCH7VXKjtVZ1p4GbhzUEbzavRKn34Y0zQydsqJUh3XSXQ3I1IzymFi+6mChNARGUDHQEEiUN1setUEnxgmxP1Ymic0nrK/HRmJVL6o6YyIHqpFLSf/0zqp7l90MyaSVIOgs0H9lGMd4zwiHDIJVPOxAYRKZnbFdEgkoSYDZZsQ3MWTl4FXq15W3dtauX41T6OIjtAxOkUuOkd1dIMayEMUPaAn9IJerUfr2Xqz3metBWvuOUR/yvr4Bqr1n1g=</latexit><latexit sha1_base64="OvpqHdnJr07GAq4B5qFqQBwG38Y=">AAACFHicbVA9T8MwEHXKVwlfBUYWixaJAVVJF2CrYGEsEmkrNVXlONfWquNEtoNURf0TLPwVFgZArAxs/BuctkjQcpLlp/fu6e5ekHCmtON8WYWV1bX1jeKmvbW9s7tX2j9oqjiVFDwa81i2A6KAMwGeZppDO5FAooBDKxhd53rrHqRisbjT4wS6ERkI1meUaEP1Smd+AAMmMgpCg5zYFT+IeajGkfmy5qRi+yDCH7VXKjtVZ1p4GbhzUEbzavRKn34Y0zQydsqJUh3XSXQ3I1IzymFi+6mChNARGUDHQEEiUN1setUEnxgmxP1Ymic0nrK/HRmJVL6o6YyIHqpFLSf/0zqp7l90MyaSVIOgs0H9lGMd4zwiHDIJVPOxAYRKZnbFdEgkoSYDZZsQ3MWTl4FXq15W3dtauX41T6OIjtAxOkUuOkd1dIMayEMUPaAn9IJerUfr2Xqz3metBWvuOUR/yvr4Bqr1n1g=</latexit><latexit sha1_base64="OvpqHdnJr07GAq4B5qFqQBwG38Y=">AAACFHicbVA9T8MwEHXKVwlfBUYWixaJAVVJF2CrYGEsEmkrNVXlONfWquNEtoNURf0TLPwVFgZArAxs/BuctkjQcpLlp/fu6e5ekHCmtON8WYWV1bX1jeKmvbW9s7tX2j9oqjiVFDwa81i2A6KAMwGeZppDO5FAooBDKxhd53rrHqRisbjT4wS6ERkI1meUaEP1Smd+AAMmMgpCg5zYFT+IeajGkfmy5qRi+yDCH7VXKjtVZ1p4GbhzUEbzavRKn34Y0zQydsqJUh3XSXQ3I1IzymFi+6mChNARGUDHQEEiUN1setUEnxgmxP1Ymic0nrK/HRmJVL6o6YyIHqpFLSf/0zqp7l90MyaSVIOgs0H9lGMd4zwiHDIJVPOxAYRKZnbFdEgkoSYDZZsQ3MWTl4FXq15W3dtauX41T6OIjtAxOkUuOkd1dIMayEMUPaAn9IJerUfr2Xqz3metBWvuOUR/yvr4Bqr1n1g=</latexit>
B<latexit sha1_base64="iJdXLwSzYTGEjcqRLfw6lR+3IoY=">AAACFHicbVA9T8MwEHXKVwlfAUaWiBaJAVVJF2CrysJYJEorNVXlONfWqmNHtoNURf0TLPwVFgZArAxs/BuctkjQcpLlp/fu6e5emDCqtOd9WYWV1bX1jeKmvbW9s7vn7B/cKZFKAk0imJDtECtglENTU82gnUjAccigFY6ucr11D1JRwW/1OIFujAec9inB2lA95ywIYUB5RoBrkBO7HISCRWocmy+rT8p2ADz6UXtOyat403KXgT8HJTSvRs/5DCJB0tjYCcNKdXwv0d0MS00Jg4kdpAoSTEZ4AB0DOY5BdbPpVRP3xDCR2xfSPK7dKfvbkeFY5YuazhjroVrUcvI/rZPq/kU3ozxJNXAyG9RPmauFm0fkRlQC0WxsACaSml1dMsQSE5OBsk0I/uLJy6BZrVxW/JtqqVafp1FER+gYnSIfnaMaukYN1EQEPaAn9IJerUfr2Xqz3metBWvuOUR/yvr4BouNn0Q=</latexit><latexit sha1_base64="iJdXLwSzYTGEjcqRLfw6lR+3IoY=">AAACFHicbVA9T8MwEHXKVwlfAUaWiBaJAVVJF2CrysJYJEorNVXlONfWqmNHtoNURf0TLPwVFgZArAxs/BuctkjQcpLlp/fu6e5emDCqtOd9WYWV1bX1jeKmvbW9s7vn7B/cKZFKAk0imJDtECtglENTU82gnUjAccigFY6ucr11D1JRwW/1OIFujAec9inB2lA95ywIYUB5RoBrkBO7HISCRWocmy+rT8p2ADz6UXtOyat403KXgT8HJTSvRs/5DCJB0tjYCcNKdXwv0d0MS00Jg4kdpAoSTEZ4AB0DOY5BdbPpVRP3xDCR2xfSPK7dKfvbkeFY5YuazhjroVrUcvI/rZPq/kU3ozxJNXAyG9RPmauFm0fkRlQC0WxsACaSml1dMsQSE5OBsk0I/uLJy6BZrVxW/JtqqVafp1FER+gYnSIfnaMaukYN1EQEPaAn9IJerUfr2Xqz3metBWvuOUR/yvr4BouNn0Q=</latexit><latexit sha1_base64="iJdXLwSzYTGEjcqRLfw6lR+3IoY=">AAACFHicbVA9T8MwEHXKVwlfAUaWiBaJAVVJF2CrysJYJEorNVXlONfWqmNHtoNURf0TLPwVFgZArAxs/BuctkjQcpLlp/fu6e5emDCqtOd9WYWV1bX1jeKmvbW9s7vn7B/cKZFKAk0imJDtECtglENTU82gnUjAccigFY6ucr11D1JRwW/1OIFujAec9inB2lA95ywIYUB5RoBrkBO7HISCRWocmy+rT8p2ADz6UXtOyat403KXgT8HJTSvRs/5DCJB0tjYCcNKdXwv0d0MS00Jg4kdpAoSTEZ4AB0DOY5BdbPpVRP3xDCR2xfSPK7dKfvbkeFY5YuazhjroVrUcvI/rZPq/kU3ozxJNXAyG9RPmauFm0fkRlQC0WxsACaSml1dMsQSE5OBsk0I/uLJy6BZrVxW/JtqqVafp1FER+gYnSIfnaMaukYN1EQEPaAn9IJerUfr2Xqz3metBWvuOUR/yvr4BouNn0Q=</latexit><latexit sha1_base64="iJdXLwSzYTGEjcqRLfw6lR+3IoY=">AAACFHicbVA9T8MwEHXKVwlfAUaWiBaJAVVJF2CrysJYJEorNVXlONfWqmNHtoNURf0TLPwVFgZArAxs/BuctkjQcpLlp/fu6e5emDCqtOd9WYWV1bX1jeKmvbW9s7vn7B/cKZFKAk0imJDtECtglENTU82gnUjAccigFY6ucr11D1JRwW/1OIFujAec9inB2lA95ywIYUB5RoBrkBO7HISCRWocmy+rT8p2ADz6UXtOyat403KXgT8HJTSvRs/5DCJB0tjYCcNKdXwv0d0MS00Jg4kdpAoSTEZ4AB0DOY5BdbPpVRP3xDCR2xfSPK7dKfvbkeFY5YuazhjroVrUcvI/rZPq/kU3ozxJNXAyG9RPmauFm0fkRlQC0WxsACaSml1dMsQSE5OBsk0I/uLJy6BZrVxW/JtqqVafp1FER+gYnSIfnaMaukYN1EQEPaAn9IJerUfr2Xqz3metBWvuOUR/yvr4BouNn0Q=</latexit>RNN
<latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit>RNN
<latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit>u<latexit sha1_base64="LFW6wPaDvB3y/CVRazOyp2YvgOQ=">AAACB3icbVBNS8NAFNz4WeNX1KMHg63gqSS9qLeiF48VjC00oWw2L+3SzSbsboQSevTiX/HiQcWrf8Gb/8ZtG0FbBxaGmTe8fRNmjErlOF/G0vLK6tp6ZcPc3Nre2bX29u9kmgsCHklZKjohlsAoB09RxaCTCcBJyKAdDq8mfvsehKQpv1WjDIIE9zmNKcFKSz3ryA+hT3lBgCsQY7OW10wfePQj9KyqU3emsBeJW5IqKtHqWZ9+lJI80XHCsJRd18lUUGChKGEwNv1cQobJEPehqynHCcigmB4ytk+0EtlxKvTjyp6qvxMFTqQcJaGeTLAayHlvIv7ndXMVnwcF5VmugJPZojhntkrtSSt2RAUQxUaaYCKo/qtNBlhgojuQpi7BnT95kXiN+kXdvWlUm5dlGxV0iI7RKXLRGWqia9RCHiLoAT2hF/RqPBrPxpvxPhtdMsrMAfoD4+Mb04qZag==</latexit><latexit sha1_base64="LFW6wPaDvB3y/CVRazOyp2YvgOQ=">AAACB3icbVBNS8NAFNz4WeNX1KMHg63gqSS9qLeiF48VjC00oWw2L+3SzSbsboQSevTiX/HiQcWrf8Gb/8ZtG0FbBxaGmTe8fRNmjErlOF/G0vLK6tp6ZcPc3Nre2bX29u9kmgsCHklZKjohlsAoB09RxaCTCcBJyKAdDq8mfvsehKQpv1WjDIIE9zmNKcFKSz3ryA+hT3lBgCsQY7OW10wfePQj9KyqU3emsBeJW5IqKtHqWZ9+lJI80XHCsJRd18lUUGChKGEwNv1cQobJEPehqynHCcigmB4ytk+0EtlxKvTjyp6qvxMFTqQcJaGeTLAayHlvIv7ndXMVnwcF5VmugJPZojhntkrtSSt2RAUQxUaaYCKo/qtNBlhgojuQpi7BnT95kXiN+kXdvWlUm5dlGxV0iI7RKXLRGWqia9RCHiLoAT2hF/RqPBrPxpvxPhtdMsrMAfoD4+Mb04qZag==</latexit><latexit sha1_base64="LFW6wPaDvB3y/CVRazOyp2YvgOQ=">AAACB3icbVBNS8NAFNz4WeNX1KMHg63gqSS9qLeiF48VjC00oWw2L+3SzSbsboQSevTiX/HiQcWrf8Gb/8ZtG0FbBxaGmTe8fRNmjErlOF/G0vLK6tp6ZcPc3Nre2bX29u9kmgsCHklZKjohlsAoB09RxaCTCcBJyKAdDq8mfvsehKQpv1WjDIIE9zmNKcFKSz3ryA+hT3lBgCsQY7OW10wfePQj9KyqU3emsBeJW5IqKtHqWZ9+lJI80XHCsJRd18lUUGChKGEwNv1cQobJEPehqynHCcigmB4ytk+0EtlxKvTjyp6qvxMFTqQcJaGeTLAayHlvIv7ndXMVnwcF5VmugJPZojhntkrtSSt2RAUQxUaaYCKo/qtNBlhgojuQpi7BnT95kXiN+kXdvWlUm5dlGxV0iI7RKXLRGWqia9RCHiLoAT2hF/RqPBrPxpvxPhtdMsrMAfoD4+Mb04qZag==</latexit><latexit sha1_base64="LFW6wPaDvB3y/CVRazOyp2YvgOQ=">AAACB3icbVBNS8NAFNz4WeNX1KMHg63gqSS9qLeiF48VjC00oWw2L+3SzSbsboQSevTiX/HiQcWrf8Gb/8ZtG0FbBxaGmTe8fRNmjErlOF/G0vLK6tp6ZcPc3Nre2bX29u9kmgsCHklZKjohlsAoB09RxaCTCcBJyKAdDq8mfvsehKQpv1WjDIIE9zmNKcFKSz3ryA+hT3lBgCsQY7OW10wfePQj9KyqU3emsBeJW5IqKtHqWZ9+lJI80XHCsJRd18lUUGChKGEwNv1cQobJEPehqynHCcigmB4ytk+0EtlxKvTjyp6qvxMFTqQcJaGeTLAayHlvIv7ndXMVnwcF5VmugJPZojhntkrtSSt2RAUQxUaaYCKo/qtNBlhgojuQpi7BnT95kXiN+kXdvWlUm5dlGxV0iI7RKXLRGWqia9RCHiLoAT2hF/RqPBrPxpvxPhtdMsrMAfoD4+Mb04qZag==</latexit>v<latexit sha1_base64="z8Uo196wu96EVWWhjXXVmngSm98=">AAACB3icbVBNS8NAFNzUrxq/oh49GGwFTyXpRb0VvXisYGyhDWWzeWmXbjZhd1MooUcv/hUvHlS8+he8+W/cthG0dWBhmHnD2zdByqhUjvNllFZW19Y3ypvm1vbO7p61f3Avk0wQ8EjCEtEOsARGOXiKKgbtVACOAwatYHg99VsjEJIm/E6NU/Bj3Oc0ogQrLfWs424AfcpzAlyBmJjVUdXsAg9/hJ5VcWrODPYycQtSQQWaPeuzGyYki3WcMCxlx3VS5edYKEoYTMxuJiHFZIj70NGU4xikn88OmdinWgntKBH6cWXP1N+JHMdSjuNAT8ZYDeSiNxX/8zqZii78nPI0U8DJfFGUMVsl9rQVO6QCiGJjTTARVP/VJgMsMNEdSFOX4C6evEy8eu2y5t7WK42roo0yOkIn6Ay56Bw10A1qIg8R9ICe0At6NR6NZ+PNeJ+Plowic4j+wPj4BtUbmWs=</latexit><latexit sha1_base64="z8Uo196wu96EVWWhjXXVmngSm98=">AAACB3icbVBNS8NAFNzUrxq/oh49GGwFTyXpRb0VvXisYGyhDWWzeWmXbjZhd1MooUcv/hUvHlS8+he8+W/cthG0dWBhmHnD2zdByqhUjvNllFZW19Y3ypvm1vbO7p61f3Avk0wQ8EjCEtEOsARGOXiKKgbtVACOAwatYHg99VsjEJIm/E6NU/Bj3Oc0ogQrLfWs424AfcpzAlyBmJjVUdXsAg9/hJ5VcWrODPYycQtSQQWaPeuzGyYki3WcMCxlx3VS5edYKEoYTMxuJiHFZIj70NGU4xikn88OmdinWgntKBH6cWXP1N+JHMdSjuNAT8ZYDeSiNxX/8zqZii78nPI0U8DJfFGUMVsl9rQVO6QCiGJjTTARVP/VJgMsMNEdSFOX4C6evEy8eu2y5t7WK42roo0yOkIn6Ay56Bw10A1qIg8R9ICe0At6NR6NZ+PNeJ+Plowic4j+wPj4BtUbmWs=</latexit><latexit sha1_base64="z8Uo196wu96EVWWhjXXVmngSm98=">AAACB3icbVBNS8NAFNzUrxq/oh49GGwFTyXpRb0VvXisYGyhDWWzeWmXbjZhd1MooUcv/hUvHlS8+he8+W/cthG0dWBhmHnD2zdByqhUjvNllFZW19Y3ypvm1vbO7p61f3Avk0wQ8EjCEtEOsARGOXiKKgbtVACOAwatYHg99VsjEJIm/E6NU/Bj3Oc0ogQrLfWs424AfcpzAlyBmJjVUdXsAg9/hJ5VcWrODPYycQtSQQWaPeuzGyYki3WcMCxlx3VS5edYKEoYTMxuJiHFZIj70NGU4xikn88OmdinWgntKBH6cWXP1N+JHMdSjuNAT8ZYDeSiNxX/8zqZii78nPI0U8DJfFGUMVsl9rQVO6QCiGJjTTARVP/VJgMsMNEdSFOX4C6evEy8eu2y5t7WK42roo0yOkIn6Ay56Bw10A1qIg8R9ICe0At6NR6NZ+PNeJ+Plowic4j+wPj4BtUbmWs=</latexit><latexit sha1_base64="z8Uo196wu96EVWWhjXXVmngSm98=">AAACB3icbVBNS8NAFNzUrxq/oh49GGwFTyXpRb0VvXisYGyhDWWzeWmXbjZhd1MooUcv/hUvHlS8+he8+W/cthG0dWBhmHnD2zdByqhUjvNllFZW19Y3ypvm1vbO7p61f3Avk0wQ8EjCEtEOsARGOXiKKgbtVACOAwatYHg99VsjEJIm/E6NU/Bj3Oc0ogQrLfWs424AfcpzAlyBmJjVUdXsAg9/hJ5VcWrODPYycQtSQQWaPeuzGyYki3WcMCxlx3VS5edYKEoYTMxuJiHFZIj70NGU4xikn88OmdinWgntKBH6cWXP1N+JHMdSjuNAT8ZYDeSiNxX/8zqZii78nPI0U8DJfFGUMVsl9rQVO6QCiGJjTTARVP/VJgMsMNEdSFOX4C6evEy8eu2y5t7WK42roo0yOkIn6Ay56Bw10A1qIg8R9ICe0At6NR6NZ+PNeJ+Plowic4j+wPj4BtUbmWs=</latexit>global spatialpooling
<latexit sha1_base64="3W6yy+2ai0f4QLKqE0g7VDHQpds=">AAACHHicbVDLSgMxFM34rOOr6tJNsAiuykwRrLuCG5cVrC10hpLJ3E5DM8mQZIQy9Efc+CtuXKi4cSH4N6YPQVsPBA7nnJvknijjTBvP+3JWVtfWNzZLW+72zu7efvng8E7LXFFoUcml6kREA2cCWoYZDp1MAUkjDu1oeDXx2/egNJPi1owyCFOSCNZnlBgr9crnQQQJEwUFYUCN3YTLiHCsM+sTHgRuJqW9O3EDEPFPqleueFVvCrxM/DmpoDmavfJHEEuap3accqJ11/cyExZEGUY5jN0g15AROiQJdC0VJAUdFtPtxvjUKjHuS2WPMHiq/p4oSKr1KI1sMiVmoBe9ifif181Nvx4WTGS5AUFnD/Vzjo3Ek6pwzBRQw0eWEKqY/SumA6IItR1o15bgL668TFq16mXVv6lVGvV5GyV0jE7QGfLRBWqga9RELUTRA3pCL+jVeXSenTfnfRZdceYzR+gPnM9vB+yirg==</latexit><latexit sha1_base64="3W6yy+2ai0f4QLKqE0g7VDHQpds=">AAACHHicbVDLSgMxFM34rOOr6tJNsAiuykwRrLuCG5cVrC10hpLJ3E5DM8mQZIQy9Efc+CtuXKi4cSH4N6YPQVsPBA7nnJvknijjTBvP+3JWVtfWNzZLW+72zu7efvng8E7LXFFoUcml6kREA2cCWoYZDp1MAUkjDu1oeDXx2/egNJPi1owyCFOSCNZnlBgr9crnQQQJEwUFYUCN3YTLiHCsM+sTHgRuJqW9O3EDEPFPqleueFVvCrxM/DmpoDmavfJHEEuap3accqJ11/cyExZEGUY5jN0g15AROiQJdC0VJAUdFtPtxvjUKjHuS2WPMHiq/p4oSKr1KI1sMiVmoBe9ifif181Nvx4WTGS5AUFnD/Vzjo3Ek6pwzBRQw0eWEKqY/SumA6IItR1o15bgL668TFq16mXVv6lVGvV5GyV0jE7QGfLRBWqga9RELUTRA3pCL+jVeXSenTfnfRZdceYzR+gPnM9vB+yirg==</latexit><latexit sha1_base64="3W6yy+2ai0f4QLKqE0g7VDHQpds=">AAACHHicbVDLSgMxFM34rOOr6tJNsAiuykwRrLuCG5cVrC10hpLJ3E5DM8mQZIQy9Efc+CtuXKi4cSH4N6YPQVsPBA7nnJvknijjTBvP+3JWVtfWNzZLW+72zu7efvng8E7LXFFoUcml6kREA2cCWoYZDp1MAUkjDu1oeDXx2/egNJPi1owyCFOSCNZnlBgr9crnQQQJEwUFYUCN3YTLiHCsM+sTHgRuJqW9O3EDEPFPqleueFVvCrxM/DmpoDmavfJHEEuap3accqJ11/cyExZEGUY5jN0g15AROiQJdC0VJAUdFtPtxvjUKjHuS2WPMHiq/p4oSKr1KI1sMiVmoBe9ifif181Nvx4WTGS5AUFnD/Vzjo3Ek6pwzBRQw0eWEKqY/SumA6IItR1o15bgL668TFq16mXVv6lVGvV5GyV0jE7QGfLRBWqga9RELUTRA3pCL+jVeXSenTfnfRZdceYzR+gPnM9vB+yirg==</latexit><latexit sha1_base64="3W6yy+2ai0f4QLKqE0g7VDHQpds=">AAACHHicbVDLSgMxFM34rOOr6tJNsAiuykwRrLuCG5cVrC10hpLJ3E5DM8mQZIQy9Efc+CtuXKi4cSH4N6YPQVsPBA7nnJvknijjTBvP+3JWVtfWNzZLW+72zu7efvng8E7LXFFoUcml6kREA2cCWoYZDp1MAUkjDu1oeDXx2/egNJPi1owyCFOSCNZnlBgr9crnQQQJEwUFYUCN3YTLiHCsM+sTHgRuJqW9O3EDEPFPqleueFVvCrxM/DmpoDmavfJHEEuap3accqJ11/cyExZEGUY5jN0g15AROiQJdC0VJAUdFtPtxvjUKjHuS2WPMHiq/p4oSKr1KI1sMiVmoBe9ifif181Nvx4WTGS5AUFnD/Vzjo3Ek6pwzBRQw0eWEKqY/SumA6IItR1o15bgL668TFq16mXVv6lVGvV5GyV0jE7QGfLRBWqga9RELUTRA3pCL+jVeXSenTfnfRZdceYzR+gPnM9vB+yirg==</latexit>r<latexit sha1_base64="Jctlm+cBm4menoiGMTyltpddS+U=">AAACBXicbVBPS8MwHE39O+u/qkcRgkPwNNpd1NvQi8cJ1g3WMtL0ty0sTUuSCqPs5MWv4sWDile/gze/jdlWQTcfBF7e+z2S34syzpR23S9raXlldW29smFvbm3v7Dp7+3cqzSUFn6Y8le2IKOBMgK+Z5tDOJJAk4tCKhlcTv3UPUrFU3OpRBmFC+oL1GCXaSF3nKIigz0RBQWiQY1vaAYj459p1qm7NnQIvEq8kVVSi2XU+gzileWLilBOlOp6b6bAgUjPKYWwHuYKM0CHpQ8dQQRJQYTFdY4xPjBLjXirNERpP1d+JgiRKjZLITCZED9S8NxH/8zq57p2HBRNZrkHQ2UO9nGOd4kknOGYSqOYjQwiVzPwV0wGRhJoOlG1K8OZXXiR+vXZR827q1cZl2UYFHaJjdIo8dIYa6Bo1kY8oekBP6AW9Wo/Ws/Vmvc9Gl6wyc4D+wPr4BgdfmQs=</latexit><latexit sha1_base64="Jctlm+cBm4menoiGMTyltpddS+U=">AAACBXicbVBPS8MwHE39O+u/qkcRgkPwNNpd1NvQi8cJ1g3WMtL0ty0sTUuSCqPs5MWv4sWDile/gze/jdlWQTcfBF7e+z2S34syzpR23S9raXlldW29smFvbm3v7Dp7+3cqzSUFn6Y8le2IKOBMgK+Z5tDOJJAk4tCKhlcTv3UPUrFU3OpRBmFC+oL1GCXaSF3nKIigz0RBQWiQY1vaAYj459p1qm7NnQIvEq8kVVSi2XU+gzileWLilBOlOp6b6bAgUjPKYWwHuYKM0CHpQ8dQQRJQYTFdY4xPjBLjXirNERpP1d+JgiRKjZLITCZED9S8NxH/8zq57p2HBRNZrkHQ2UO9nGOd4kknOGYSqOYjQwiVzPwV0wGRhJoOlG1K8OZXXiR+vXZR827q1cZl2UYFHaJjdIo8dIYa6Bo1kY8oekBP6AW9Wo/Ws/Vmvc9Gl6wyc4D+wPr4BgdfmQs=</latexit><latexit sha1_base64="Jctlm+cBm4menoiGMTyltpddS+U=">AAACBXicbVBPS8MwHE39O+u/qkcRgkPwNNpd1NvQi8cJ1g3WMtL0ty0sTUuSCqPs5MWv4sWDile/gze/jdlWQTcfBF7e+z2S34syzpR23S9raXlldW29smFvbm3v7Dp7+3cqzSUFn6Y8le2IKOBMgK+Z5tDOJJAk4tCKhlcTv3UPUrFU3OpRBmFC+oL1GCXaSF3nKIigz0RBQWiQY1vaAYj459p1qm7NnQIvEq8kVVSi2XU+gzileWLilBOlOp6b6bAgUjPKYWwHuYKM0CHpQ8dQQRJQYTFdY4xPjBLjXirNERpP1d+JgiRKjZLITCZED9S8NxH/8zq57p2HBRNZrkHQ2UO9nGOd4kknOGYSqOYjQwiVzPwV0wGRhJoOlG1K8OZXXiR+vXZR827q1cZl2UYFHaJjdIo8dIYa6Bo1kY8oekBP6AW9Wo/Ws/Vmvc9Gl6wyc4D+wPr4BgdfmQs=</latexit><latexit sha1_base64="Jctlm+cBm4menoiGMTyltpddS+U=">AAACBXicbVBPS8MwHE39O+u/qkcRgkPwNNpd1NvQi8cJ1g3WMtL0ty0sTUuSCqPs5MWv4sWDile/gze/jdlWQTcfBF7e+z2S34syzpR23S9raXlldW29smFvbm3v7Dp7+3cqzSUFn6Y8le2IKOBMgK+Z5tDOJJAk4tCKhlcTv3UPUrFU3OpRBmFC+oL1GCXaSF3nKIigz0RBQWiQY1vaAYj459p1qm7NnQIvEq8kVVSi2XU+gzileWLilBOlOp6b6bAgUjPKYWwHuYKM0CHpQ8dQQRJQYTFdY4xPjBLjXirNERpP1d+JgiRKjZLITCZED9S8NxH/8zq57p2HBRNZrkHQ2UO9nGOd4kknOGYSqOYjQwiVzPwV0wGRhJoOlG1K8OZXXiR+vXZR827q1cZl2UYFHaJjdIo8dIYa6Bo1kY8oekBP6AW9Wo/Ws/Vmvc9Gl6wyc4D+wPr4BgdfmQs=</latexit>s<latexit sha1_base64="Hhn0o7t1HhojgAQ33p1M3+chUB8=">AAACBXicbVBPS8MwHE39O+u/qkcRgkPwNNpd1NvQi8cJ1g3WMtL0ty0sTUuSCqPs5MWv4sWDile/gze/jdlWQTcfBF7e+z2S34syzpR23S9raXlldW29smFvbm3v7Dp7+3cqzSUFn6Y8le2IKOBMgK+Z5tDOJJAk4tCKhlcTv3UPUrFU3OpRBmFC+oL1GCXaSF3nKIigz0RBQWiQY1vZAYj459p1qm7NnQIvEq8kVVSi2XU+gzileWLilBOlOp6b6bAgUjPKYWwHuYKM0CHpQ8dQQRJQYTFdY4xPjBLjXirNERpP1d+JgiRKjZLITCZED9S8NxH/8zq57p2HBRNZrkHQ2UO9nGOd4kknOGYSqOYjQwiVzPwV0wGRhJoOlG1K8OZXXiR+vXZR827q1cZl2UYFHaJjdIo8dIYa6Bo1kY8oekBP6AW9Wo/Ws/Vmvc9Gl6wyc4D+wPr4BgjvmQw=</latexit><latexit sha1_base64="Hhn0o7t1HhojgAQ33p1M3+chUB8=">AAACBXicbVBPS8MwHE39O+u/qkcRgkPwNNpd1NvQi8cJ1g3WMtL0ty0sTUuSCqPs5MWv4sWDile/gze/jdlWQTcfBF7e+z2S34syzpR23S9raXlldW29smFvbm3v7Dp7+3cqzSUFn6Y8le2IKOBMgK+Z5tDOJJAk4tCKhlcTv3UPUrFU3OpRBmFC+oL1GCXaSF3nKIigz0RBQWiQY1vZAYj459p1qm7NnQIvEq8kVVSi2XU+gzileWLilBOlOp6b6bAgUjPKYWwHuYKM0CHpQ8dQQRJQYTFdY4xPjBLjXirNERpP1d+JgiRKjZLITCZED9S8NxH/8zq57p2HBRNZrkHQ2UO9nGOd4kknOGYSqOYjQwiVzPwV0wGRhJoOlG1K8OZXXiR+vXZR827q1cZl2UYFHaJjdIo8dIYa6Bo1kY8oekBP6AW9Wo/Ws/Vmvc9Gl6wyc4D+wPr4BgjvmQw=</latexit><latexit sha1_base64="Hhn0o7t1HhojgAQ33p1M3+chUB8=">AAACBXicbVBPS8MwHE39O+u/qkcRgkPwNNpd1NvQi8cJ1g3WMtL0ty0sTUuSCqPs5MWv4sWDile/gze/jdlWQTcfBF7e+z2S34syzpR23S9raXlldW29smFvbm3v7Dp7+3cqzSUFn6Y8le2IKOBMgK+Z5tDOJJAk4tCKhlcTv3UPUrFU3OpRBmFC+oL1GCXaSF3nKIigz0RBQWiQY1vZAYj459p1qm7NnQIvEq8kVVSi2XU+gzileWLilBOlOp6b6bAgUjPKYWwHuYKM0CHpQ8dQQRJQYTFdY4xPjBLjXirNERpP1d+JgiRKjZLITCZED9S8NxH/8zq57p2HBRNZrkHQ2UO9nGOd4kknOGYSqOYjQwiVzPwV0wGRhJoOlG1K8OZXXiR+vXZR827q1cZl2UYFHaJjdIo8dIYa6Bo1kY8oekBP6AW9Wo/Ws/Vmvc9Gl6wyc4D+wPr4BgjvmQw=</latexit><latexit sha1_base64="Hhn0o7t1HhojgAQ33p1M3+chUB8=">AAACBXicbVBPS8MwHE39O+u/qkcRgkPwNNpd1NvQi8cJ1g3WMtL0ty0sTUuSCqPs5MWv4sWDile/gze/jdlWQTcfBF7e+z2S34syzpR23S9raXlldW29smFvbm3v7Dp7+3cqzSUFn6Y8le2IKOBMgK+Z5tDOJJAk4tCKhlcTv3UPUrFU3OpRBmFC+oL1GCXaSF3nKIigz0RBQWiQY1vZAYj459p1qm7NnQIvEq8kVVSi2XU+gzileWLilBOlOp6b6bAgUjPKYWwHuYKM0CHpQ8dQQRJQYTFdY4xPjBLjXirNERpP1d+JgiRKjZLITCZED9S8NxH/8zq57p2HBRNZrkHQ2UO9nGOd4kknOGYSqOYjQwiVzPwV0wGRhJoOlG1K8OZXXiR+vXZR827q1cZl2UYFHaJjdIo8dIYa6Bo1kY8oekBP6AW9Wo/Ws/Vmvc9Gl6wyc4D+wPr4BgjvmQw=</latexit>
Fig. 2. A functional overview of the model. A global convolutional model extracts
features and splits into two heads trained to predict, respectively activity classes and
object classes. The latter are predicted by pooling over object instance masks, which
are predicted by an additional convolutional model. The object instances are passed
through a visual reasoning module.
is a CNN-based architecture employing convolutional layers, including spatio-
temporal convolutions, able to extract global motion features. However, it is not
able to extract information from an object level perspective. We leverage the
object head to perform reasoning on the relationships between predicted object
instances.
Our main contribution is a new structured module called Object Relation
Network (ORN), which is able to perform spatio-temporal reasoning between
detected object instances in the video. ORN is able to reason by modeling how
objects move, appear and disappear and how they interact between two frames.
In this section, we will rst describe our main contribution, the ORN network.
We then provide details about object instance features, about the activity head,
and nally about the nal recognition task. In what follows, lowercase letters
denote 1D vectors while uppercase letters are used for 2D and 3D matrices or
higher order tensors. We assume that the input of our system is a video of T
frames denoted by X1:T= (Xt)T
t=1where Xtis the RGB image at timestep t.
The goal is to learn a mapping from X1:Tto activity classes y.
3.1 Object Relation Network
ORN (Object Relation Network) is a module for reasoning between semantic
objects through space and time. It captures object moves, arrivals and interac-
tions in an ecient manner. We suppose that for each frame t, we have a set
of objects kwith associated features ok
t. Objects and features are detected and
computed by the object head described in Section 3.2.
Reasoning about activities in videos is inherently temporal, as activities fol-
low the arrow of time [28], i.e. the causality of the time dimension imposes that
past actions have consequences in the future but notvice-versa. We handle this
by sampling: running a process over time t, and for each instant t, sampling a
Object Level Visual Reasoning in Videos 7
second frame t0with t0<t. Our network reasons on objects which interact be-
tween pairs of frames and their corresponding sets of objects Ot0=
ok
t0	K0
k=1
andOt=
ok
t	K
k=1. The goal is to learn a general function dened on the set of
all input objects from the combined set of both frames:
gt=g(o1
t0; : : : ;oK0
t0;o1
t; : : : ;oK
t): (1)
The objects in this set are unordered, aside for the frame they belong to.
This task is related to a problem raised in the PointNet algorithm [29] dis-
cussed in Section 2. PointNet approximates a general function gover an item
setS=fx1; x2; : : : ; xNgas a symmetric function g0on transformed elements of
the set
g(x1; x2; : : : ; xN)g0(h(x1); h(x2); : : : ; h (xN)): (2)
In [29], g0is a max pooling operation. The authors show that it allows universal
approximation of continuous sets of functions given that the hidden representa-
tion (the output of the mapping h()) is of suciently high dimension.
We argue that the approximation in (2) can be extended as follows:
g(x1; x2; : : : ; xN)f X
c2Chc([i2cxi)!
(3)
whereCis the set of cliques of a graph dened over the item set S,[is the
concatenation operator and we chose the sum operator as symmetric function.
The input dimension of the non-linearity hc() depends on the size of clique c
but maps to a xed output dimension H. In the case where Cis composed of
unary cliques only, form (3) decomposes like (2) with the exception of a dierent
symmetry operator (sum instead of max pooling). Choosing dierent graphical
structures through Swill lead to dierent terms in the summation and allows
modeling dierent types of interactions between items in the item set.
Note that the interactions between items in (3) are not exclusively modeled
through hc(). Indeed, it is interesting to note, that the graphical decomposi-
tion provided byCleads to interactions which are dierent from the interactions
the same decomposition would provide when used in a probabilistic graphical
model, like for instance a Markov Random Field (MRF). In particular, a de-
composition into unary terms only, as given in equation (2), does notlead to
independence between items, whereas an MRF with unary terms only is equiva-
lent to a distribution over independent random variables. This is a consequence
of the global mapping f(), which is dened on the sum over all direct inter-
actions. Higher-order interactions between several items not directly modeled
through a non-linearity hc() can eventually be learned by the model through
the joint output space of all hc(), provided that the dimensionality Hof this
space is high enough to incorporate all interactions. However, whereas the map-
pinghc() provides a direct model of interactions between pairs of items, learning
interactions between two items ( j; k), which are not directly captured through a
8 Baradel et al.
visual reasoningmodule
<latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit><latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit><latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit><latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit>visual reasoningmodule
<latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit><latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit><latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit><latexit sha1_base64="fBWZg+aPOFAImknIghA1Bu9U+88=">AAACHXicbVDLSgMxFM3UVx1foy7dBIvgqswUxLoruHFZwdpCp5RM5rYNzSRDkimUoV/ixl9x40LFhRvxb0wfgrYeCBzOOTfJPVHKmTa+/+UU1tY3NreK2+7O7t7+gXd4dK9lpig0qORStSKigTMBDcMMh1aqgCQRh2Y0vJ76zREozaS4M+MUOgnpC9ZjlBgrdb2LMII+EzkFYUBN3BHTGeHYXqGlYKIfhjiRccbBDUHEP7GuV/LL/gx4lQQLUkIL1LveRxhLmiV2nHKidTvwU9PJiTKMcpi4YaYhJXRI+tC2VJAEdCefrTfBZ1aJcU8qe4TBM/X3RE4SrcdJZJMJMQO97E3F/7x2ZnrVTs5EmhkQdP5QL+PYSDztCsdMATV8bAmhitm/YjogilDbgXZtCcHyyqukUSlflYPbSqlWXbRRRCfoFJ2jAF2iGrpBddRAFD2gJ/SCXp1H59l5c97n0YKzmDlGf+B8fgM5O6NX</latexit>random framet02[0...t2]
<latexit sha1_base64="3hmJJGXFri/4Fn9gRqnNsEIPzF4=">AAACJ3icbVDLSsQwFE19W1+jLt0ER9GNQzsbdaXgxqWCo8K0DGl6OwbTpCS3wlDmc9z4K25EVHTpn5gZR/B1IHA45x5y70kKKSwGwZs3Nj4xOTU9M+vPzS8sLtWWV86tLg2HFtdSm8uEWZBCQQsFSrgsDLA8kXCRXB8N/IsbMFZodYa9AuKcdZXIBGfopE7tIEqgK1TFQSGYvm+YSnVOM8NyoBu4FQnVDiKZarQUq51+M97wI1DpV6BTqweNYAj6l4QjUicjnHRqj1GqeZm7OJfM2nYYFBhXzKDgEvp+VFooGL9mXWg7qtweNq6Gh/bpplNSmmnjnkI6VL8nKpZb28sTN5kzvLK/vYH4n9cuMduLK6GKEkHxz4+yUlLUdNAaTYUBjrLnCONGuF0pv2KGcdeB9V0J4e+T/5JWs7HfCE+b9cPtURszZI2sk20Skl1ySI7JCWkRTm7JPXkiz96d9+C9eK+fo2PeKLNKfsB7/wD5faYK</latexit><latexit sha1_base64="3hmJJGXFri/4Fn9gRqnNsEIPzF4=">AAACJ3icbVDLSsQwFE19W1+jLt0ER9GNQzsbdaXgxqWCo8K0DGl6OwbTpCS3wlDmc9z4K25EVHTpn5gZR/B1IHA45x5y70kKKSwGwZs3Nj4xOTU9M+vPzS8sLtWWV86tLg2HFtdSm8uEWZBCQQsFSrgsDLA8kXCRXB8N/IsbMFZodYa9AuKcdZXIBGfopE7tIEqgK1TFQSGYvm+YSnVOM8NyoBu4FQnVDiKZarQUq51+M97wI1DpV6BTqweNYAj6l4QjUicjnHRqj1GqeZm7OJfM2nYYFBhXzKDgEvp+VFooGL9mXWg7qtweNq6Gh/bpplNSmmnjnkI6VL8nKpZb28sTN5kzvLK/vYH4n9cuMduLK6GKEkHxz4+yUlLUdNAaTYUBjrLnCONGuF0pv2KGcdeB9V0J4e+T/5JWs7HfCE+b9cPtURszZI2sk20Skl1ySI7JCWkRTm7JPXkiz96d9+C9eK+fo2PeKLNKfsB7/wD5faYK</latexit><latexit sha1_base64="3hmJJGXFri/4Fn9gRqnNsEIPzF4=">AAACJ3icbVDLSsQwFE19W1+jLt0ER9GNQzsbdaXgxqWCo8K0DGl6OwbTpCS3wlDmc9z4K25EVHTpn5gZR/B1IHA45x5y70kKKSwGwZs3Nj4xOTU9M+vPzS8sLtWWV86tLg2HFtdSm8uEWZBCQQsFSrgsDLA8kXCRXB8N/IsbMFZodYa9AuKcdZXIBGfopE7tIEqgK1TFQSGYvm+YSnVOM8NyoBu4FQnVDiKZarQUq51+M97wI1DpV6BTqweNYAj6l4QjUicjnHRqj1GqeZm7OJfM2nYYFBhXzKDgEvp+VFooGL9mXWg7qtweNq6Gh/bpplNSmmnjnkI6VL8nKpZb28sTN5kzvLK/vYH4n9cuMduLK6GKEkHxz4+yUlLUdNAaTYUBjrLnCONGuF0pv2KGcdeB9V0J4e+T/5JWs7HfCE+b9cPtURszZI2sk20Skl1ySI7JCWkRTm7JPXkiz96d9+C9eK+fo2PeKLNKfsB7/wD5faYK</latexit><latexit sha1_base64="3hmJJGXFri/4Fn9gRqnNsEIPzF4=">AAACJ3icbVDLSsQwFE19W1+jLt0ER9GNQzsbdaXgxqWCo8K0DGl6OwbTpCS3wlDmc9z4K25EVHTpn5gZR/B1IHA45x5y70kKKSwGwZs3Nj4xOTU9M+vPzS8sLtWWV86tLg2HFtdSm8uEWZBCQQsFSrgsDLA8kXCRXB8N/IsbMFZodYa9AuKcdZXIBGfopE7tIEqgK1TFQSGYvm+YSnVOM8NyoBu4FQnVDiKZarQUq51+M97wI1DpV6BTqweNYAj6l4QjUicjnHRqj1GqeZm7OJfM2nYYFBhXzKDgEvp+VFooGL9mXWg7qtweNq6Gh/bpplNSmmnjnkI6VL8nKpZb28sTN5kzvLK/vYH4n9cuMduLK6GKEkHxz4+yUlLUdNAaTYUBjrLnCONGuF0pv2KGcdeB9V0J4e+T/5JWs7HfCE+b9cPtURszZI2sk20Skl1ySI7JCWkRTm7JPXkiz96d9+C9eK+fo2PeKLNKfsB7/wD5faYK</latexit>random framet002[0...t1]
<latexit sha1_base64="A/qj6iEXV6zX9h9ZfaxCC8IJQkw=">AAACKHicbVDLSgMxFM34dnxVXboJVtGNZcaNuiu4cVnBqtAZSiZzpwYzyZDcEcrQ33Hjr7hRUOnWLzGtFXwdCBzOuYfce5JCCotBMPSmpmdm5+YXFv2l5ZXVtdr6xqXVpeHQ5lpqc50wC1IoaKNACdeFAZYnEq6S29ORf3UHxgqtLrBfQJyznhKZ4Ayd1K01owR6QlUcFIIZ+IapVOc0MywHuoN7e5FQnSCSqUZLsToYhPGOH4FKvxLdWj1oBGPQvySckDqZoNWtPUep5mXu4lwyazthUGBcMYOCSxj4UWmhYPyW9aDjqHKL2LgaXzqgu05JaaaNewrpWP2eqFhubT9P3GTO8Mb+9kbif16nxOw4roQqSgTFPz/KSklR01FtNBUGOMq+I4wb4Xal/IYZxl0H1nclhL9P/kvah42TRnh+WG/uT9pYIFtkm+yTkByRJjkjLdImnNyTR/JCXr0H78l784afo1PeJLNJfsB7/wBovKY6</latexit><latexit sha1_base64="A/qj6iEXV6zX9h9ZfaxCC8IJQkw=">AAACKHicbVDLSgMxFM34dnxVXboJVtGNZcaNuiu4cVnBqtAZSiZzpwYzyZDcEcrQ33Hjr7hRUOnWLzGtFXwdCBzOuYfce5JCCotBMPSmpmdm5+YXFv2l5ZXVtdr6xqXVpeHQ5lpqc50wC1IoaKNACdeFAZYnEq6S29ORf3UHxgqtLrBfQJyznhKZ4Ayd1K01owR6QlUcFIIZ+IapVOc0MywHuoN7e5FQnSCSqUZLsToYhPGOH4FKvxLdWj1oBGPQvySckDqZoNWtPUep5mXu4lwyazthUGBcMYOCSxj4UWmhYPyW9aDjqHKL2LgaXzqgu05JaaaNewrpWP2eqFhubT9P3GTO8Mb+9kbif16nxOw4roQqSgTFPz/KSklR01FtNBUGOMq+I4wb4Xal/IYZxl0H1nclhL9P/kvah42TRnh+WG/uT9pYIFtkm+yTkByRJjkjLdImnNyTR/JCXr0H78l784afo1PeJLNJfsB7/wBovKY6</latexit><latexit sha1_base64="A/qj6iEXV6zX9h9ZfaxCC8IJQkw=">AAACKHicbVDLSgMxFM34dnxVXboJVtGNZcaNuiu4cVnBqtAZSiZzpwYzyZDcEcrQ33Hjr7hRUOnWLzGtFXwdCBzOuYfce5JCCotBMPSmpmdm5+YXFv2l5ZXVtdr6xqXVpeHQ5lpqc50wC1IoaKNACdeFAZYnEq6S29ORf3UHxgqtLrBfQJyznhKZ4Ayd1K01owR6QlUcFIIZ+IapVOc0MywHuoN7e5FQnSCSqUZLsToYhPGOH4FKvxLdWj1oBGPQvySckDqZoNWtPUep5mXu4lwyazthUGBcMYOCSxj4UWmhYPyW9aDjqHKL2LgaXzqgu05JaaaNewrpWP2eqFhubT9P3GTO8Mb+9kbif16nxOw4roQqSgTFPz/KSklR01FtNBUGOMq+I4wb4Xal/IYZxl0H1nclhL9P/kvah42TRnh+WG/uT9pYIFtkm+yTkByRJjkjLdImnNyTR/JCXr0H78l784afo1PeJLNJfsB7/wBovKY6</latexit><latexit sha1_base64="A/qj6iEXV6zX9h9ZfaxCC8IJQkw=">AAACKHicbVDLSgMxFM34dnxVXboJVtGNZcaNuiu4cVnBqtAZSiZzpwYzyZDcEcrQ33Hjr7hRUOnWLzGtFXwdCBzOuYfce5JCCotBMPSmpmdm5+YXFv2l5ZXVtdr6xqXVpeHQ5lpqc50wC1IoaKNACdeFAZYnEq6S29ORf3UHxgqtLrBfQJyznhKZ4Ayd1K01owR6QlUcFIIZ+IapVOc0MywHuoN7e5FQnSCSqUZLsToYhPGOH4FKvxLdWj1oBGPQvySckDqZoNWtPUep5mXu4lwyazthUGBcMYOCSxj4UWmhYPyW9aDjqHKL2LgaXzqgu05JaaaNewrpWP2eqFhubT9P3GTO8Mb+9kbif16nxOw4roQqSgTFPz/KSklR01FtNBUGOMq+I4wb4Xal/IYZxl0H1nclhL9P/kvah42TRnh+WG/uT9pYIFtkm+yTkByRJjkjLdImnNyTR/JCXr0H78l784afo1PeJLNJfsB7/wBovKY6</latexit>framet1
<latexit sha1_base64="QdD1TILqtKPiRr85PfCrEm51r9o=">AAACEXicbVDLSsNAFJ3UV42vqEs3g61QF5akG3VXcOOygrGFppTJ9KYdOpmEmYlQQr/Bjb/ixoWKW3fu/BunD0FbDwwczrmHO/eEKWdKu+6XVVhZXVvfKG7aW9s7u3vO/sGdSjJJwacJT2QrJAo4E+Brpjm0UgkkDjk0w+HVxG/eg1QsEbd6lEInJn3BIkaJNlLXOQ1C6DORUxAa5NiOJIkBl3V+NvbKdgCi92N1nZJbdafAy8SbkxKao9F1PoNeQrPYxCknSrU9N9WdnEjNKIexHWQKUkKHpA9tQ4VZrDr59KQxPjFKD0eJNE9oPFV/J3ISKzWKQzMZEz1Qi95E/M9rZzq66ORMpJkGQWeLooxjneBJP7jHJFDNR4YQKpn5K6YDIgk1HSjblOAtnrxM/Fr1surd1Er1yryNIjpCx6iCPHSO6ugaNZCPKHpAT+gFvVqP1rP1Zr3PRgvWPHOI/sD6+AYPMJ00</latexit><latexit sha1_base64="QdD1TILqtKPiRr85PfCrEm51r9o=">AAACEXicbVDLSsNAFJ3UV42vqEs3g61QF5akG3VXcOOygrGFppTJ9KYdOpmEmYlQQr/Bjb/ixoWKW3fu/BunD0FbDwwczrmHO/eEKWdKu+6XVVhZXVvfKG7aW9s7u3vO/sGdSjJJwacJT2QrJAo4E+Brpjm0UgkkDjk0w+HVxG/eg1QsEbd6lEInJn3BIkaJNlLXOQ1C6DORUxAa5NiOJIkBl3V+NvbKdgCi92N1nZJbdafAy8SbkxKao9F1PoNeQrPYxCknSrU9N9WdnEjNKIexHWQKUkKHpA9tQ4VZrDr59KQxPjFKD0eJNE9oPFV/J3ISKzWKQzMZEz1Qi95E/M9rZzq66ORMpJkGQWeLooxjneBJP7jHJFDNR4YQKpn5K6YDIgk1HSjblOAtnrxM/Fr1surd1Er1yryNIjpCx6iCPHSO6ugaNZCPKHpAT+gFvVqP1rP1Zr3PRgvWPHOI/sD6+AYPMJ00</latexit><latexit sha1_base64="QdD1TILqtKPiRr85PfCrEm51r9o=">AAACEXicbVDLSsNAFJ3UV42vqEs3g61QF5akG3VXcOOygrGFppTJ9KYdOpmEmYlQQr/Bjb/ixoWKW3fu/BunD0FbDwwczrmHO/eEKWdKu+6XVVhZXVvfKG7aW9s7u3vO/sGdSjJJwacJT2QrJAo4E+Brpjm0UgkkDjk0w+HVxG/eg1QsEbd6lEInJn3BIkaJNlLXOQ1C6DORUxAa5NiOJIkBl3V+NvbKdgCi92N1nZJbdafAy8SbkxKao9F1PoNeQrPYxCknSrU9N9WdnEjNKIexHWQKUkKHpA9tQ4VZrDr59KQxPjFKD0eJNE9oPFV/J3ISKzWKQzMZEz1Qi95E/M9rZzq66ORMpJkGQWeLooxjneBJP7jHJFDNR4YQKpn5K6YDIgk1HSjblOAtnrxM/Fr1surd1Er1yryNIjpCx6iCPHSO6ugaNZCPKHpAT+gFvVqP1rP1Zr3PRgvWPHOI/sD6+AYPMJ00</latexit><latexit sha1_base64="QdD1TILqtKPiRr85PfCrEm51r9o=">AAACEXicbVDLSsNAFJ3UV42vqEs3g61QF5akG3VXcOOygrGFppTJ9KYdOpmEmYlQQr/Bjb/ixoWKW3fu/BunD0FbDwwczrmHO/eEKWdKu+6XVVhZXVvfKG7aW9s7u3vO/sGdSjJJwacJT2QrJAo4E+Brpjm0UgkkDjk0w+HVxG/eg1QsEbd6lEInJn3BIkaJNlLXOQ1C6DORUxAa5NiOJIkBl3V+NvbKdgCi92N1nZJbdafAy8SbkxKao9F1PoNeQrPYxCknSrU9N9WdnEjNKIexHWQKUkKHpA9tQ4VZrDr59KQxPjFKD0eJNE9oPFV/J3ISKzWKQzMZEz1Qi95E/M9rZzq66ORMpJkGQWeLooxjneBJP7jHJFDNR4YQKpn5K6YDIgk1HSjblOAtnrxM/Fr1surd1Er1yryNIjpCx6iCPHSO6ugaNZCPKHpAT+gFvVqP1rP1Zr3PRgvWPHOI/sD6+AYPMJ00</latexit>framet
<latexit sha1_base64="BD8grwOxxSmQYxdsSCDxptyziy4=">AAACDXicbVBNS8NAFNz4WeNX1aOXxbbQU0l6UW8FLx4rWFtoQtlsXtqlm03Y3Qgl9Bd48a948aDi1bs3/43bNoK2DiwMM294+yZIOVPacb6stfWNza3t0o69u7d/cFg+Or5TSSYpdGjCE9kLiALOBHQ00xx6qQQSBxy6wfhq5nfvQSqWiFs9ScGPyVCwiFGijTQo17wAhkzkFIQGObUjSWLAVV21PRDhjzwoV5yGMwdeJW5BKqhAe1D+9MKEZrGJU06U6rtOqv2cSM0oh6ntZQpSQsdkCH1DhVmq/Hx+zhTXjBLiKJHmCY3n6u9ETmKlJnFgJmOiR2rZm4n/ef1MRxd+zkSaaRB0sSjKONYJnnWDQyaBaj4xhFDJzF8xHRFJqOlA2aYEd/nkVdJpNi4b7k2z0qoXbZTQKTpDdeSic9RC16iNOoiiB/SEXtCr9Wg9W2/W+2J0zSoyJ+gPrI9vPTWbtg==</latexit><latexit sha1_base64="BD8grwOxxSmQYxdsSCDxptyziy4=">AAACDXicbVBNS8NAFNz4WeNX1aOXxbbQU0l6UW8FLx4rWFtoQtlsXtqlm03Y3Qgl9Bd48a948aDi1bs3/43bNoK2DiwMM294+yZIOVPacb6stfWNza3t0o69u7d/cFg+Or5TSSYpdGjCE9kLiALOBHQ00xx6qQQSBxy6wfhq5nfvQSqWiFs9ScGPyVCwiFGijTQo17wAhkzkFIQGObUjSWLAVV21PRDhjzwoV5yGMwdeJW5BKqhAe1D+9MKEZrGJU06U6rtOqv2cSM0oh6ntZQpSQsdkCH1DhVmq/Hx+zhTXjBLiKJHmCY3n6u9ETmKlJnFgJmOiR2rZm4n/ef1MRxd+zkSaaRB0sSjKONYJnnWDQyaBaj4xhFDJzF8xHRFJqOlA2aYEd/nkVdJpNi4b7k2z0qoXbZTQKTpDdeSic9RC16iNOoiiB/SEXtCr9Wg9W2/W+2J0zSoyJ+gPrI9vPTWbtg==</latexit><latexit sha1_base64="BD8grwOxxSmQYxdsSCDxptyziy4=">AAACDXicbVBNS8NAFNz4WeNX1aOXxbbQU0l6UW8FLx4rWFtoQtlsXtqlm03Y3Qgl9Bd48a948aDi1bs3/43bNoK2DiwMM294+yZIOVPacb6stfWNza3t0o69u7d/cFg+Or5TSSYpdGjCE9kLiALOBHQ00xx6qQQSBxy6wfhq5nfvQSqWiFs9ScGPyVCwiFGijTQo17wAhkzkFIQGObUjSWLAVV21PRDhjzwoV5yGMwdeJW5BKqhAe1D+9MKEZrGJU06U6rtOqv2cSM0oh6ntZQpSQsdkCH1DhVmq/Hx+zhTXjBLiKJHmCY3n6u9ETmKlJnFgJmOiR2rZm4n/ef1MRxd+zkSaaRB0sSjKONYJnnWDQyaBaj4xhFDJzF8xHRFJqOlA2aYEd/nkVdJpNi4b7k2z0qoXbZTQKTpDdeSic9RC16iNOoiiB/SEXtCr9Wg9W2/W+2J0zSoyJ+gPrI9vPTWbtg==</latexit><latexit sha1_base64="BD8grwOxxSmQYxdsSCDxptyziy4=">AAACDXicbVBNS8NAFNz4WeNX1aOXxbbQU0l6UW8FLx4rWFtoQtlsXtqlm03Y3Qgl9Bd48a948aDi1bs3/43bNoK2DiwMM294+yZIOVPacb6stfWNza3t0o69u7d/cFg+Or5TSSYpdGjCE9kLiALOBHQ00xx6qQQSBxy6wfhq5nfvQSqWiFs9ScGPyVCwiFGijTQo17wAhkzkFIQGObUjSWLAVV21PRDhjzwoV5yGMwdeJW5BKqhAe1D+9MKEZrGJU06U6rtOqv2cSM0oh6ntZQpSQsdkCH1DhVmq/Hx+zhTXjBLiKJHmCY3n6u9ETmKlJnFgJmOiR2rZm4n/ef1MRxd+zkSaaRB0sSjKONYJnnWDQyaBaj4xhFDJzF8xHRFJqOlA2aYEd/nkVdJpNi4b7k2z0qoXbZTQKTpDdeSic9RC16iNOoiiB/SEXtCr9Wg9W2/W+2J0zSoyJ+gPrI9vPTWbtg==</latexit>video stream<latexit sha1_base64="tv8v5jh4gPs3lF/UcaOyaYFtsvY=">AAACEHicbVDLSgMxFM34rOOr6tJNsAhdlZlu1F3BjcsKji10Sslk7rShmWRIMoUy9Bfc+CtuXKi4denOvzF9CNp6IHA4554k90QZZ9p43peztr6xubVd2nF39/YPDstHx/da5opCQCWXqh0RDZwJCAwzHNqZApJGHFrR8Hrqt0agNJPizowz6KakL1jCKDFW6pWrYQR9JgoKwoCauCMWg8TaTO9wQxDxj9MrV7yaNwNeJf6CVNACzV75M4wlzVMbp5xo3fG9zHQLogyjHCZumGvICB2SPnQsFSQF3S1mG03wuVVinEhljzB4pv5OFCTVepxGdjIlZqCXvan4n9fJTXLZLZjIcgOCzh9Kco6NxNN6cMwUUMPHlhCqmP0rpgOiCLUdaNeW4C+vvEqCeu2q5t/WK43qoo0SOkVnqIp8dIEa6AY1UYAoekBP6AW9Oo/Os/PmvM9H15xF5gT9gfPxDbN+nbA=</latexit><latexit sha1_base64="tv8v5jh4gPs3lF/UcaOyaYFtsvY=">AAACEHicbVDLSgMxFM34rOOr6tJNsAhdlZlu1F3BjcsKji10Sslk7rShmWRIMoUy9Bfc+CtuXKi4denOvzF9CNp6IHA4554k90QZZ9p43peztr6xubVd2nF39/YPDstHx/da5opCQCWXqh0RDZwJCAwzHNqZApJGHFrR8Hrqt0agNJPizowz6KakL1jCKDFW6pWrYQR9JgoKwoCauCMWg8TaTO9wQxDxj9MrV7yaNwNeJf6CVNACzV75M4wlzVMbp5xo3fG9zHQLogyjHCZumGvICB2SPnQsFSQF3S1mG03wuVVinEhljzB4pv5OFCTVepxGdjIlZqCXvan4n9fJTXLZLZjIcgOCzh9Kco6NxNN6cMwUUMPHlhCqmP0rpgOiCLUdaNeW4C+vvEqCeu2q5t/WK43qoo0SOkVnqIp8dIEa6AY1UYAoekBP6AW9Oo/Os/PmvM9H15xF5gT9gfPxDbN+nbA=</latexit><latexit sha1_base64="tv8v5jh4gPs3lF/UcaOyaYFtsvY=">AAACEHicbVDLSgMxFM34rOOr6tJNsAhdlZlu1F3BjcsKji10Sslk7rShmWRIMoUy9Bfc+CtuXKi4denOvzF9CNp6IHA4554k90QZZ9p43peztr6xubVd2nF39/YPDstHx/da5opCQCWXqh0RDZwJCAwzHNqZApJGHFrR8Hrqt0agNJPizowz6KakL1jCKDFW6pWrYQR9JgoKwoCauCMWg8TaTO9wQxDxj9MrV7yaNwNeJf6CVNACzV75M4wlzVMbp5xo3fG9zHQLogyjHCZumGvICB2SPnQsFSQF3S1mG03wuVVinEhljzB4pv5OFCTVepxGdjIlZqCXvan4n9fJTXLZLZjIcgOCzh9Kco6NxNN6cMwUUMPHlhCqmP0rpgOiCLUdaNeW4C+vvEqCeu2q5t/WK43qoo0SOkVnqIp8dIEa6AY1UYAoekBP6AW9Oo/Os/PmvM9H15xF5gT9gfPxDbN+nbA=</latexit><latexit sha1_base64="tv8v5jh4gPs3lF/UcaOyaYFtsvY=">AAACEHicbVDLSgMxFM34rOOr6tJNsAhdlZlu1F3BjcsKji10Sslk7rShmWRIMoUy9Bfc+CtuXKi4denOvzF9CNp6IHA4554k90QZZ9p43peztr6xubVd2nF39/YPDstHx/da5opCQCWXqh0RDZwJCAwzHNqZApJGHFrR8Hrqt0agNJPizowz6KakL1jCKDFW6pWrYQR9JgoKwoCauCMWg8TaTO9wQxDxj9MrV7yaNwNeJf6CVNACzV75M4wlzVMbp5xo3fG9zHQLogyjHCZumGvICB2SPnQsFSQF3S1mG03wuVVinEhljzB4pv5OFCTVepxGdjIlZqCXvan4n9fJTXLZLZjIcgOCzh9Kco6NxNN6cMwUUMPHlhCqmP0rpgOiCLUdaNeW4C+vvEqCeu2q5t/WK43qoo0SOkVnqIp8dIEa6AY1UYAoekBP6AW9Oo/Os/PmvM9H15xF5gT9gfPxDbN+nbA=</latexit>
RNN
<latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit>RNN
<latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit><latexit sha1_base64="WNqRVHooK3MRbsx3jlRlp53Ez9s=">AAACB3icbVBNS8NAFNzUrxq/oh49GCyCp5L0Yr0VvHgqVYwttKFsNi/t0s0m7G6EEnr04l/x4kHFq3/Bm//GbRtBWwcWhpk3vH0TpIxK5ThfRmlldW19o7xpbm3v7O5Z+wd3MskEAY8kLBGdAEtglIOnqGLQSQXgOGDQDkaXU799D0LShN+qcQp+jAecRpRgpaW+ddwLYEB5ToArEBPzptk0e8DDH6FvVZyqM4O9TNyCVFCBVt/67IUJyWIdJwxL2XWdVPk5FooSBhOzl0lIMRnhAXQ15TgG6eezQyb2qVZCO0qEflzZM/V3IsexlOM40JMxVkO56E3F/7xupqK6n1OeZgo4mS+KMmarxJ62YodUAFFsrAkmguq/2mSIBSa6A2nqEtzFk5eJV6teVN3rWqVRL9oooyN0gs6Qi85RA12hFvIQQQ/oCb2gV+PReDbejPf5aMkoMofoD4yPbx1cmZE=</latexit>rt1
<latexit sha1_base64="1rbc+gZHaqLmGJLkenaEBa002fg=">AAACDXicbVDLSsNAFJ3UV42vqEs3g22hG0vSjboruHFZwdpCG8pketMOnUzCzEQoIV/gxl9x40LFrXt3/o3Th6CtBwYO59zDnXuChDOlXffLKqytb2xuFbftnd29/QPn8OhOxamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Orqd++B6lYLG71JAE/IkPBQkaJNlLfqfQCGDKRURAaZG6XZT/TZ15etnsgBj9y3ym5NXcGvEq8BSmhBZp957M3iGkamTjlRKmu5ybaz4jUjHLI7V6qICF0TIbQNVSQCJSfzc7JccUoAxzG0jyh8Uz9nchIpNQkCsxkRPRILXtT8T+vm+rwws+YSFINgs4XhSnHOsbTbvCASaCaTwwhVDLzV0xHRBJqOlC2KcFbPnmVtOq1y5p3Uy81qos2iugEnaIq8tA5aqBr1EQtRNEDekIv6NV6tJ6tN+t9PlqwFplj9AfWxzc1pZuy</latexit><latexit sha1_base64="1rbc+gZHaqLmGJLkenaEBa002fg=">AAACDXicbVDLSsNAFJ3UV42vqEs3g22hG0vSjboruHFZwdpCG8pketMOnUzCzEQoIV/gxl9x40LFrXt3/o3Th6CtBwYO59zDnXuChDOlXffLKqytb2xuFbftnd29/QPn8OhOxamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Orqd++B6lYLG71JAE/IkPBQkaJNlLfqfQCGDKRURAaZG6XZT/TZ15etnsgBj9y3ym5NXcGvEq8BSmhBZp957M3iGkamTjlRKmu5ybaz4jUjHLI7V6qICF0TIbQNVSQCJSfzc7JccUoAxzG0jyh8Uz9nchIpNQkCsxkRPRILXtT8T+vm+rwws+YSFINgs4XhSnHOsbTbvCASaCaTwwhVDLzV0xHRBJqOlC2KcFbPnmVtOq1y5p3Uy81qos2iugEnaIq8tA5aqBr1EQtRNEDekIv6NV6tJ6tN+t9PlqwFplj9AfWxzc1pZuy</latexit><latexit sha1_base64="1rbc+gZHaqLmGJLkenaEBa002fg=">AAACDXicbVDLSsNAFJ3UV42vqEs3g22hG0vSjboruHFZwdpCG8pketMOnUzCzEQoIV/gxl9x40LFrXt3/o3Th6CtBwYO59zDnXuChDOlXffLKqytb2xuFbftnd29/QPn8OhOxamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Orqd++B6lYLG71JAE/IkPBQkaJNlLfqfQCGDKRURAaZG6XZT/TZ15etnsgBj9y3ym5NXcGvEq8BSmhBZp957M3iGkamTjlRKmu5ybaz4jUjHLI7V6qICF0TIbQNVSQCJSfzc7JccUoAxzG0jyh8Uz9nchIpNQkCsxkRPRILXtT8T+vm+rwws+YSFINgs4XhSnHOsbTbvCASaCaTwwhVDLzV0xHRBJqOlC2KcFbPnmVtOq1y5p3Uy81qos2iugEnaIq8tA5aqBr1EQtRNEDekIv6NV6tJ6tN+t9PlqwFplj9AfWxzc1pZuy</latexit><latexit sha1_base64="1rbc+gZHaqLmGJLkenaEBa002fg=">AAACDXicbVDLSsNAFJ3UV42vqEs3g22hG0vSjboruHFZwdpCG8pketMOnUzCzEQoIV/gxl9x40LFrXt3/o3Th6CtBwYO59zDnXuChDOlXffLKqytb2xuFbftnd29/QPn8OhOxamk0KIxj2UnIAo4E9DSTHPoJBJIFHBoB+Orqd++B6lYLG71JAE/IkPBQkaJNlLfqfQCGDKRURAaZG6XZT/TZ15etnsgBj9y3ym5NXcGvEq8BSmhBZp957M3iGkamTjlRKmu5ybaz4jUjHLI7V6qICF0TIbQNVSQCJSfzc7JccUoAxzG0jyh8Uz9nchIpNQkCsxkRPRILXtT8T+vm+rwws+YSFINgs4XhSnHOsbTbvCASaCaTwwhVDLzV0xHRBJqOlC2KcFbPnmVtOq1y5p3Uy81qos2iugEnaIq8tA5aqBr1EQtRNEDekIv6NV6tJ6tN+t9PlqwFplj9AfWxzc1pZuy</latexit>gt1
<latexit sha1_base64="yzX8Dk93KrfNdGHgx6JSbbSoHms=">AAACDXicbVDLSsNAFJ3UV42vqEs3wbbQjSXpRt0V3LisYG2hDWEyuUmHTiZhZiKU0C9w46+4caHi1r07/8bpQ9DWAwOHc+7hzj1BxqhUjvNllNbWNza3ytvmzu7e/oF1eHQn01wQ6JCUpaIXYAmMcugoqhj0MgE4CRh0g9HV1O/eg5A05bdqnIGX4JjTiBKstORbtUEAMeUFAa5ATMxq7BfqzJ1UzQHw8Ef2rYrTcGawV4m7IBW0QNu3PgdhSvJExwnDUvZdJ1NegYWihMHEHOQSMkxGOIa+phwnIL1ids7ErmkltKNU6MeVPVN/JwqcSDlOAj2ZYDWUy95U/M/r5yq68ArKs1wBJ/NFUc5sldrTbuyQCiCKjTXBRFD9V5sMscBEdyBNXYK7fPIq6TQblw33pllp1RdtlNEJOkV15KJz1ELXqI06iKAH9IRe0KvxaDwbb8b7fLRkLDLH6A+Mj28kKJun</latexit><latexit sha1_base64="yzX8Dk93KrfNdGHgx6JSbbSoHms=">AAACDXicbVDLSsNAFJ3UV42vqEs3wbbQjSXpRt0V3LisYG2hDWEyuUmHTiZhZiKU0C9w46+4caHi1r07/8bpQ9DWAwOHc+7hzj1BxqhUjvNllNbWNza3ytvmzu7e/oF1eHQn01wQ6JCUpaIXYAmMcugoqhj0MgE4CRh0g9HV1O/eg5A05bdqnIGX4JjTiBKstORbtUEAMeUFAa5ATMxq7BfqzJ1UzQHw8Ef2rYrTcGawV4m7IBW0QNu3PgdhSvJExwnDUvZdJ1NegYWihMHEHOQSMkxGOIa+phwnIL1ids7ErmkltKNU6MeVPVN/JwqcSDlOAj2ZYDWUy95U/M/r5yq68ArKs1wBJ/NFUc5sldrTbuyQCiCKjTXBRFD9V5sMscBEdyBNXYK7fPIq6TQblw33pllp1RdtlNEJOkV15KJz1ELXqI06iKAH9IRe0KvxaDwbb8b7fLRkLDLH6A+Mj28kKJun</latexit><latexit sha1_base64="yzX8Dk93KrfNdGHgx6JSbbSoHms=">AAACDXicbVDLSsNAFJ3UV42vqEs3wbbQjSXpRt0V3LisYG2hDWEyuUmHTiZhZiKU0C9w46+4caHi1r07/8bpQ9DWAwOHc+7hzj1BxqhUjvNllNbWNza3ytvmzu7e/oF1eHQn01wQ6JCUpaIXYAmMcugoqhj0MgE4CRh0g9HV1O/eg5A05bdqnIGX4JjTiBKstORbtUEAMeUFAa5ATMxq7BfqzJ1UzQHw8Ef2rYrTcGawV4m7IBW0QNu3PgdhSvJExwnDUvZdJ1NegYWihMHEHOQSMkxGOIa+phwnIL1ids7ErmkltKNU6MeVPVN/JwqcSDlOAj2ZYDWUy95U/M/r5yq68ArKs1wBJ/NFUc5sldrTbuyQCiCKjTXBRFD9V5sMscBEdyBNXYK7fPIq6TQblw33pllp1RdtlNEJOkV15KJz1ELXqI06iKAH9IRe0KvxaDwbb8b7fLRkLDLH6A+Mj28kKJun</latexit><latexit sha1_base64="yzX8Dk93KrfNdGHgx6JSbbSoHms=">AAACDXicbVDLSsNAFJ3UV42vqEs3wbbQjSXpRt0V3LisYG2hDWEyuUmHTiZhZiKU0C9w46+4caHi1r07/8bpQ9DWAwOHc+7hzj1BxqhUjvNllNbWNza3ytvmzu7e/oF1eHQn01wQ6JCUpaIXYAmMcugoqhj0MgE4CRh0g9HV1O/eg5A05bdqnIGX4JjTiBKstORbtUEAMeUFAa5ATMxq7BfqzJ1UzQHw8Ef2rYrTcGawV4m7IBW0QNu3PgdhSvJExwnDUvZdJ1NegYWihMHEHOQSMkxGOIa+phwnIL1ids7ErmkltKNU6MeVPVN/JwqcSDlOAj2ZYDWUy95U/M/r5yq68ArKs1wBJ/NFUc5sldrTbuyQCiCKjTXBRFD9V5sMscBEdyBNXYK7fPIq6TQblw33pllp1RdtlNEJOkV15KJz1ELXqI06iKAH9IRe0KvxaDwbb8b7fLRkLDLH6A+Mj28kKJun</latexit>gt
<latexit sha1_base64="uu12jd0B8LZaVIoujm9hG9bt468=">AAACC3icbVDLSsNAFJ3UV42vqks3wVboqiTdqLuCG5cVjC00oUwmN+nQySTMTIQS8gFu/BU3LlTc+gPu/BunD0FbDwwczjmXO/cEGaNS2faXUVlb39jcqm6bO7t7+we1w6M7meaCgEtSlop+gCUwysFVVDHoZwJwEjDoBeOrqd+7ByFpym/VJAM/wTGnESVYaWlYq3sBxJQXBLgCUZqNeFiosmF6wMMfUafslj2DtUqcBamjBbrD2qcXpiRP9DhhWMqBY2fKL7BQlDAoTS+XkGEyxjEMNOU4AekXs2NK60wroRWlQj+urJn6e6LAiZSTJNDJBKuRXPam4n/eIFfRhV9QnuUKOJkvinJmqdSaNmOFVABRbKIJJoLqv1pkhAUmugNp6hKc5ZNXidtuXbacm3a901y0UUUn6BQ1kYPOUQddoy5yEUEP6Am9oFfj0Xg23oz3ebRiLGaO0R8YH982SJs1</latexit><latexit sha1_base64="uu12jd0B8LZaVIoujm9hG9bt468=">AAACC3icbVDLSsNAFJ3UV42vqks3wVboqiTdqLuCG5cVjC00oUwmN+nQySTMTIQS8gFu/BU3LlTc+gPu/BunD0FbDwwczjmXO/cEGaNS2faXUVlb39jcqm6bO7t7+we1w6M7meaCgEtSlop+gCUwysFVVDHoZwJwEjDoBeOrqd+7ByFpym/VJAM/wTGnESVYaWlYq3sBxJQXBLgCUZqNeFiosmF6wMMfUafslj2DtUqcBamjBbrD2qcXpiRP9DhhWMqBY2fKL7BQlDAoTS+XkGEyxjEMNOU4AekXs2NK60wroRWlQj+urJn6e6LAiZSTJNDJBKuRXPam4n/eIFfRhV9QnuUKOJkvinJmqdSaNmOFVABRbKIJJoLqv1pkhAUmugNp6hKc5ZNXidtuXbacm3a901y0UUUn6BQ1kYPOUQddoy5yEUEP6Am9oFfj0Xg23oz3ebRiLGaO0R8YH982SJs1</latexit><latexit sha1_base64="uu12jd0B8LZaVIoujm9hG9bt468=">AAACC3icbVDLSsNAFJ3UV42vqks3wVboqiTdqLuCG5cVjC00oUwmN+nQySTMTIQS8gFu/BU3LlTc+gPu/BunD0FbDwwczjmXO/cEGaNS2faXUVlb39jcqm6bO7t7+we1w6M7meaCgEtSlop+gCUwysFVVDHoZwJwEjDoBeOrqd+7ByFpym/VJAM/wTGnESVYaWlYq3sBxJQXBLgCUZqNeFiosmF6wMMfUafslj2DtUqcBamjBbrD2qcXpiRP9DhhWMqBY2fKL7BQlDAoTS+XkGEyxjEMNOU4AekXs2NK60wroRWlQj+urJn6e6LAiZSTJNDJBKuRXPam4n/eIFfRhV9QnuUKOJkvinJmqdSaNmOFVABRbKIJJoLqv1pkhAUmugNp6hKc5ZNXidtuXbacm3a901y0UUUn6BQ1kYPOUQddoy5yEUEP6Am9oFfj0Xg23oz3ebRiLGaO0R8YH982SJs1</latexit><latexit sha1_base64="uu12jd0B8LZaVIoujm9hG9bt468=">AAACC3icbVDLSsNAFJ3UV42vqks3wVboqiTdqLuCG5cVjC00oUwmN+nQySTMTIQS8gFu/BU3LlTc+gPu/BunD0FbDwwczjmXO/cEGaNS2faXUVlb39jcqm6bO7t7+we1w6M7meaCgEtSlop+gCUwysFVVDHoZwJwEjDoBeOrqd+7ByFpym/VJAM/wTGnESVYaWlYq3sBxJQXBLgCUZqNeFiosmF6wMMfUafslj2DtUqcBamjBbrD2qcXpiRP9DhhWMqBY2fKL7BQlDAoTS+XkGEyxjEMNOU4AekXs2NK60wroRWlQj+urJn6e6LAiZSTJNDJBKuRXPam4n/eIFfRhV9QnuUKOJkvinJmqdSaNmOFVABRbKIJJoLqv1pkhAUmugNp6hKc5ZNXidtuXbacm3a901y0UUUn6BQ1kYPOUQddoy5yEUEP6Am9oFfj0Xg23oz3ebRiLGaO0R8YH982SJs1</latexit>rt
<latexit sha1_base64="Clq5Cfm2vgFbNe3s9vp/CSS8cH0=">AAACC3icbVDLSsNAFJ3UV42vqks3wVboqiTdqLuCG5cVjC00oUwmN+3QySTMTIQS8gFu/BU3LlTc+gPu/BunbQRtPTBwOOdc7twTpIxKZdtfRmVtfWNzq7pt7uzu7R/UDo/uZJIJAi5JWCL6AZbAKAdXUcWgnwrAccCgF0yuZn7vHoSkCb9V0xT8GI84jSjBSkvDWt0LYER5ToArEIXZEMNcFQ3TAx7+iDplt+w5rFXilKSOSnSHtU8vTEgW63HCsJQDx06Vn2OhKGFQmF4mIcVkgkcw0JTjGKSfz48prDOthFaUCP24subq74kcx1JO40AnY6zGctmbif95g0xFF35OeZop4GSxKMqYpRJr1owVUgFEsakmmAiq/2qRMRaY6A6kqUtwlk9eJW67ddlybtr1TrNso4pO0ClqIgedow66Rl3kIoIe0BN6Qa/Go/FsvBnvi2jFKGeO0R8YH99Hr5tA</latexit><latexit sha1_base64="Clq5Cfm2vgFbNe3s9vp/CSS8cH0=">AAACC3icbVDLSsNAFJ3UV42vqks3wVboqiTdqLuCG5cVjC00oUwmN+3QySTMTIQS8gFu/BU3LlTc+gPu/BunbQRtPTBwOOdc7twTpIxKZdtfRmVtfWNzq7pt7uzu7R/UDo/uZJIJAi5JWCL6AZbAKAdXUcWgnwrAccCgF0yuZn7vHoSkCb9V0xT8GI84jSjBSkvDWt0LYER5ToArEIXZEMNcFQ3TAx7+iDplt+w5rFXilKSOSnSHtU8vTEgW63HCsJQDx06Vn2OhKGFQmF4mIcVkgkcw0JTjGKSfz48prDOthFaUCP24subq74kcx1JO40AnY6zGctmbif95g0xFF35OeZop4GSxKMqYpRJr1owVUgFEsakmmAiq/2qRMRaY6A6kqUtwlk9eJW67ddlybtr1TrNso4pO0ClqIgedow66Rl3kIoIe0BN6Qa/Go/FsvBnvi2jFKGeO0R8YH99Hr5tA</latexit><latexit sha1_base64="Clq5Cfm2vgFbNe3s9vp/CSS8cH0=">AAACC3icbVDLSsNAFJ3UV42vqks3wVboqiTdqLuCG5cVjC00oUwmN+3QySTMTIQS8gFu/BU3LlTc+gPu/BunbQRtPTBwOOdc7twTpIxKZdtfRmVtfWNzq7pt7uzu7R/UDo/uZJIJAi5JWCL6AZbAKAdXUcWgnwrAccCgF0yuZn7vHoSkCb9V0xT8GI84jSjBSkvDWt0LYER5ToArEIXZEMNcFQ3TAx7+iDplt+w5rFXilKSOSnSHtU8vTEgW63HCsJQDx06Vn2OhKGFQmF4mIcVkgkcw0JTjGKSfz48prDOthFaUCP24subq74kcx1JO40AnY6zGctmbif95g0xFF35OeZop4GSxKMqYpRJr1owVUgFEsakmmAiq/2qRMRaY6A6kqUtwlk9eJW67ddlybtr1TrNso4pO0ClqIgedow66Rl3kIoIe0BN6Qa/Go/FsvBnvi2jFKGeO0R8YH99Hr5tA</latexit><latexit sha1_base64="Clq5Cfm2vgFbNe3s9vp/CSS8cH0=">AAACC3icbVDLSsNAFJ3UV42vqks3wVboqiTdqLuCG5cVjC00oUwmN+3QySTMTIQS8gFu/BU3LlTc+gPu/BunbQRtPTBwOOdc7twTpIxKZdtfRmVtfWNzq7pt7uzu7R/UDo/uZJIJAi5JWCL6AZbAKAdXUcWgnwrAccCgF0yuZn7vHoSkCb9V0xT8GI84jSjBSkvDWt0LYER5ToArEIXZEMNcFQ3TAx7+iDplt+w5rFXilKSOSnSHtU8vTEgW63HCsJQDx06Vn2OhKGFQmF4mIcVkgkcw0JTjGKSfz48prDOthFaUCP24subq74kcx1JO40AnY6zGctmbif95g0xFF35OeZop4GSxKMqYpRJr1owVUgFEsakmmAiq/2qRMRaY6A6kqUtwlk9eJW67ddlybtr1TrNso4pO0ClqIgedow66Rl3kIoIe0BN6Qa/Go/FsvBnvi2jFKGeO0R8YH99Hr5tA</latexit>...<latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit>...<latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit>
...<latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit>...<latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit><latexit sha1_base64="E21iIXh2YHf14GxWPuZ9SSzttl8=">AAACDHicbVDLSsNAFJ34rPEVdekm2AquStKNuiu4cVnB2kITymRy0w6dzISZiVBCf8CNv+LGhYpbP8Cdf+O0jaCtBwYO59zDnXuijFGlPe/LWlldW9/YrGzZ2zu7e/vOweGdErkk0CaCCdmNsAJGObQ11Qy6mQScRgw60ehq6nfuQSoq+K0eZxCmeMBpQgnWRuo7tSCCAeUFAa5BTuxawGKhVc0OgMc/at+penVvBneZ+CWpohKtvvMZxILkqYkThpXq+V6mwwJLTQmDiR3kCjJMRngAPUM5TkGFxeyaiXtqlNhNhDSPa3em/k4UOFVqnEZmMsV6qBa9qfif18t1chEWlGe5Bk7mi5KcuVq402rcmEogmo0NwURS81eXDLHExHSgbFOCv3jyMmk36pd1/6ZRbTbKNiroGJ2gM+Sjc9RE16iF2oigB/SEXtCr9Wg9W2/W+3x0xSozR+gPrI9v47+bmQ==</latexit>
Fig. 3. ORN in the object head operating on detected instances of objects.
clique cand its corresponding hc(), requires learning a corresponding subspace
in the common output space spanned by all hc().
This leads to the question of how to dene the trade-o between the com-
plexity of the decomposition Cand the output dimensionality Hof the mapping
hc(), both of which will determine the complexity of the modeled interactions.
Increasing the size of cliques in Cwill increase the input dimension (and there-
fore the capacity) of the mapping hc() as well as the computational complexity
of the sum operation.
Inspired by relational networks [32], we chose to directly model inter-frame
interactions between pairs of objects ( j; k) and leave modeling of higher-order
interactions to the output space of the mappings hand the global mapping f:
gt=X
j;kh(oj
t0;ok
t) (4)
In order to better directly model long-range interactions, we make the global
mapping f(;) recurrent, which leads to the following form:
rt=f(gt;rt1) (5)
where rtrepresents the recurrent object reasoning state at time tandgtis the
global inter-frame interaction inferred at time tsuch as described in Equation 4.
In practice, this is implemented as a GRU, but for simplicity we omitted the
gates in Equation (5). The pairwise mappings h(;) are implemented as an
MLP. Figure 3 provides a visual explanation of the object head's operating
through time.
Our proposed ORN diers from [32] in three main points:
Objects have a semantic denition | we model relationships with respect
to semantically meaningful entities (object instances) instead of feature map
cells which do not have a semantically meaningful spatial extent. We will show
in the experimental section that this is a key dierence.
Object Level Visual Reasoning in Videos 9
Objects are selected from dierent frames | we infer object pairwise
relations only between objects present in two dierent sets. This is a key design
choice which allows our model to reason about changes in object relationships
over time.
Long range reasoning | integration of the object relations over time is re-
current by using a RNN for f(). Since reasoning from a full sequence cannot
be done by inferring the relations between two frames, f() allows long range
reasoning on sequences of variable length.
3.2 Object instance features
The object features Ot=
ok
t	K
k=1for each frame tused for the ORN module
described above are computed and collected from local regions predicted by
a mask predictor. Independently for each frame Xtof the input data block,
we predict object instances as binary masks Bk
tand associated object class
predictions ck
t, a distribution over Cclasses. We use Mask-RCNN [15], which
is able to detect objects in a frame using region proposal networks [30] and
produces a high quality segmentation mask for each object instance.
The objective is to collect features for each object instance, which jointly
describe its appearance, the change in its appearance over time, and its shape,
i.e. the shape of the binary mask. In theory, appearance could also be described
by pooling the feature representation learned by the mask predictor (Mask R-
CNN). However, in practice we choose to pool features from the dedicated object
head such as shown in Figure 2, which also include motion through the spatio-
temporal convolutions shared with the activity head:
uk
t= ROI-Pooling( Ut;Bk
t) (6)
where Utis the feature map output by the object head ,uk
tis aD-dimensional
vector of appearance and appearance change of object k.
Shape information from the binary mask Bk
tis extracted through the follow-
ing mapping function: bk
t=g(Bk
t);where g() is a MLP. Information about
object kin image Xtis given by a concatenation of appearance, shape, and
object class: ok
t= [bk
tuk
tck
t].
3.3 Global Motion and Context
Current approaches in video understanding focus on modeling the video from
a high-level perspective. By a stack of spatio-temporal convolution and pooling
they focus on learning global scene context information. Eective activity recog-
nition requires integration of both of these sources: global information about the
entire video content in addition to relational reasoning for making ne distinc-
tions regarding object interactions and properties.
In our method, local low-level reasoning is provided through object head
and the ORN module such as described above in Section 3.1. We complement
10 Baradel et al.
this representation by high-level context information described by Vtwhich are
feature outputs from the activity head (orange block in Figure 2).
We use spatial global average pooling over Vtto output T D-dimensional
feature vectors denoted by vt, where vtcorresponds to the context information
of the video at timestep t.
We model the dynamics of the context information through time by employ-
ing a RNN f
() given by:
st=f
(vt;st1) (7)
where sis the hidden state of f
() and gives cues about the evolution of the
context though time.
3.4 Recognition
Given an input video sequence X1:T, the two dierent streams corresponding to
the activity head and the object head result in the two representations handr,
respectively where h=P
thtandr=P
trt. Each representation is the hidden
state of the respective GRU, which were described in the preceding subsections.
Recall that hprovides the global motion context while rprovides the object
reasoning state output by the ORN module. We perform independent linear
classication for each representation:
y1=W h (8)
y2=Z r (9)
where y1;y2correspond to the logits from the activity head and the object head ,
respectively, and WandZare trainable weights (including biases). The nal
prediction is done by averaging logits y1andy2followed by softmax activation.
4 Network Architectures and feature dimensions
The input RGB images Xtare of size R3WHwhere WandHcorrespond to
the width and height and are of size 224 each. The object and activity heads
(orange and green in Figure 2) are a joint convolutional neural network with
Resnet50 architecture pre-trained on ImageNet/ILSVRC [31], with Conv1 and
Conv5 blocks being in
ated to 2.5D convolutions [44] (3D convolutions with a
separable temporal dimension). This choice has been optimized on the validation
set, as explained in Section 6 and shown in Table 5.
The last conv5 layers have been split into two dierent heads (activity head
and object head). The intermediate feature representations UtandVtare of di-
mensions 2048T1414 and 2048T77, respectively. We provide a higher
spatial resolution for the feature maps Utof the object head to get more precise
local descriptors. This can be done by changing the stride of the initial conv5
layers from 2 to 1. Temporal convolutions have been congured to keep the same
time temporal dimension through the network.
Object Level Visual Reasoning in Videos 11
Global spatial pooling of activity features results in a 2048 dimensional fea-
ture vector fed into a GRU with 512 dimensional hidden state st. ROI-Pooling
of object features results in 2048 dimensional feature vectors uk
t. The encoder of
the binary mask is a MLP with one hidden layer of size 100 and outputs a mask
embedding bk
tof dimension 100. The number of object classes is 80, which leads
in total to a 2229 dimensional object feature vector ok
t.
The non-linearity h() is implemented as an MLP with 2 hidden layers each
with 512 units and produces an 512 dimensional output space. f() is imple-
mented as a GRU with a 256 dimension hidden state rt. We use ReLU as the
activation function after each layer for each network.
5 Training
We train the model with a loss split into two terms:
Ltot=L^y1+^y2
2;y
+X
tX
kL(^ck
t;ck
t): (10)
whereLis the cross-entropy loss. The rst term corresponds to supervised ac-
tivity class losses comparing two dierent activity class predictions to the class
ground truth: ^y1is the prediction of the activity head, whereas ^y2is the pre-
diction of the object head, as given by Equations (8) and (9), respectively.
The second term is a loss which pushes the features Uof the object towards
representations of the semantic object classes. The goal is to obtain features
related to, both, motion (through the layers shared with the activity head), as
well as as object classes. As ground-truth object classes are not available, we
dene the loss as the cross-entropy between the class label ck
tpredicted by the
mask predictor and a dedicated linear class prediction ^ck
tbased on features uk
t,
which, as we recall, are RoI-pooled from U:
ck
t=R uk
t (11)
where Rtrainable parameters (biases integrated) learned end-to-end together
with the other parameters of the model.
We found that rst training the object head only and then the full network
was performing better. A ResNet50 network pretrained on ImageNet is modied
by in
ating some of its lters to 2.5 convolutions (3D convolutions with the time
dimension separated), as described in Section 4; then by ne-tuning.
We train the model using the Adam optimizer [22] with an initial learning
rate of 104on 30 epochs and use early-stopping criterion on the validation set
for hyper-parameter optimization. Training takes 50 minutes per epoch on 4
Titan XP GPUs with clips of 8 frames.
12 Baradel et al.
6 Experimental results
We evaluated the method on three standard datasets, which represent dicult
ne-grained activity recognition tasks: the Something-Something dataset, the
VLOG dataset and the recently released EPIC Kitchens dataset.
Something-Something (SS) is a recent video classication dataset with
108,000 example videos and 157 classes [13]. It shows humans performing dier-
ent actions with dierent objects, actions and objects being combined in dierent
ways. Solving SS requires common sense reasoning and the state-of-the-art meth-
ods in activity recognition tend to fail, which makes this dataset challenging.
VLOG is a multi-label binary classication of human-object interactions
recently released with 114,000 videos and 30 classes [12]. Classes correspond
to objects, and labels of a class are 1 if a person has touched a certain object
during the video, otherwise they are 0. It has recently been shown, that state-
of-the-art video based methods [6] are outperformed on VLOG by image based
methods like ResNet-50 [16], although these video methods outperform image
based ResNet-50 on large-scale video datasets like the Kinetics dataset [6]. This
suggests a gap between traditional datasets like Kinetics and the ne-grained
dataset VLOG, making it particularly dicult.
EPIC Kitchens (EPIC) is an egocentric video dataset recently released
containing 55 hours recording of daily activities [7]. This is the largest in rst-
person vision and the activities performed are non-scripted, which makes the
dataset very challenging and close to real world data. The dataset is densely
annotated and several tasks exist such as object detection, action recognition and
action prediction. We focus on action recognition with 39'594 action segments
in total and 125 actions classes (i.e verbs). Since the test set is not available yet
we conducted our experiments on the training set (28'561 videos). We use the
videos recorded by person 01 to person 25 for training (22'675 videos) and dene
the validation set as the remaining videos (5'886 videos).
For all datasets we rescale the input video resolution to 256 256. While
training, we crop space-time blocks of 224 224 spatial resolution and Lframes,
withL=8 for the SS dataset and L=4 for VLOG and EPIC. We do not perform
any other data augmentation. While training we extract Lframes from the entire
video by splitting the video into Lsub-sequences and randomly sampling one
frame per sub-sequence. The output sequence of size Lis called a clip. A clip
aims to represent the full video with less frames. For testing we aggregate results
of 10 clips. We use lintel [10] for decoding video on the
y.
The ablation study is done by using the train set as training data and we
report the result on the validation set. We compare against other state-of-the-
art approaches on the test set. For the ablation studies, we slightly decreased
the computational complexity of the model: the base network (including activity
and object heads) is a ResNet-18 instead of ResNet-50, a single clip of 4 frames
is extracted from a video at test time.
Comparison with other approaches. Table 1 shows the performance of
the proposed approach on the VLOG dataset. We outperform the state of the art
Object Level Visual Reasoning in Videos 13
Table 1. Results on Hand/Semantic Object Interaction Classication (Average pre-
cision in % on the test set) on VLOG dataset. R50 and I3D implemented by [12].
mAP
bag
bed
bedding
book/papers
bottle/tube
bowl
box
brush
cabinet
cell-phone
clothing
cup
door
drawers
food
fork
knife
laptop
microwave
oven
pen/pencil
pillow
plate
refrigerator
sink
spoon
stued animal
table
toothbrush
towel
R50 [16] 40.5 29.7 68.9 65.8 64.5 58.2 33.1 22.1 19.0 23.9 54.0 45.5 28.6 49.2 28.7 49.6 19.4 37.5 62.9 48.8 23.0 36.9 39.2 12.5 55.9 58.8 31.1 57.4 26.8 39.6 22.9
I3D [6] 39.7 24.9 71.7 71.4 62.5 57.1 27.1 19.2 33.9 20.7 50.6 45.8 24.7 54.7 19.1 50.8 19.3 41.9 54.0 27.5 21.4 37.4 42.9 12.6 42.5 60.4 33.9 46.0 23.5 59.6 34.7
Ours 44.7 30.2 72.3 70.7 64.9 59.8 38.2 24.6 26.3 22.4 64.5 47.2 35.4 57.9 25.2 48.5 24.5 40.2 72.0 54.1 26.5 39.9 48.6 15.2 53.5 60.7 36.8 52.8 27.9 64.0 37.6
on this challenging dataset by a margin of 4.2 points (44.7% accuracy against
40.5% by [16]). As mentioned above, traditional video approaches tend to fail
on this challenging ne-grained dataset, providing inferior results. Table 3 shows
performance on SS where we outperform the state of the art given by very recent
methods (+2.3 points). On EPIC we re-implement standard baselines and report
results on the validation set (Table 4) since the test set is not available. Our full
method reports an accuracy of 40.89 and outperforms baselines by a large margin
(+6.4 and+7.9 points respectively for against CNN-2D and I3D based on a
ResNet-18).
Eect of object-level reasoning. Table 2 shows the importance of rea-
soning on the performance of the method. The baseline corresponds to the per-
formance obtained by the activity head trained alone (in
ated ResNet, in the
ResNet-18 version for this table). No object level reasoning is present in this
baseline. The proposed approach (third line) including an object head and the
ORN module gains 0.8, 2.5 and 2.4 points compared to our baseline respectively
on SS, on EPIC and on VLOG. This indicates that the reasoning module is able
to extract complementary features compared to the activity head.
Using semantically dened objects proved to be important and led to a gain of
2 points on EPIC and 2.3 points on VLOG for the full model (6/12.7 points using
the object head only) compared to an extension of Santoro et al. [32] operating
on pixel level. This indicates importance of object level reasoning. The gain on
SS is smaller (0.7 point with the full model and 7.8 points with the object head
only) and can be explained by the dierence in spatial resolution of the videos.
Object detections and predictions of the binary masks are done using the initial
video resolution. The mean video resolution for VLOG is 660 1183 and for EPIC
is 640480 against 100157 for SS. Mask-RCNN has been trained on images of
resolution 800800 and thus performs best on higher resolutions. The quality of
the object detector is important for leveraging object level understanding then
for the rest of the ablation study we focus on EPIC and VLOG datasets.
The function fin Equation (5) is an important design choice in our model.
In our proposed model, fis recurrent over time to ensure that the ORN module
captures long range reasoning over time, as shown in Equation (5). Removing
the recurrence in this equation leads to an MLP instead of a (gated) RNN, as
evaluated in row 4 of Table 2. Performance decreases by 1.1 point on VLOG
and 1.4 points on EPIC. The larger gap for EPIC compared to VLOG and
can arguably be explained by the fact that in SS actions cover the whole video,
14 Baradel et al.
Table 2. Ablation study with ResNet-18 backbone. Results in %: Top-1 accuracy for
EPIC and SS datasets, and mAP for VLOG dataset.
Method Object typeEPIC VLOG SS
obj. head 2 heads obj. head 2 heads obj. head 2 heads
Baseline - - 38.33 - 35.03 - 31.31
ORN pixel 23.71 38.83 14.40 35.18 2.51 31.43
ORN COCO 29.94 40.89 27.14 37.49 10.26 32.12
ORN-mlp COCO 28.15 39.41 25.40 36.35 - -
ORN COCO-visual 28.45 38.92 22.92 35.49 - -
ORN COCO-shape 21.92 37.16 7.18 35.39 - -
ORN COCO-class 21.96 37.75 13.40 35.94 - -
ORN COCO-intra 29.25 38.10 26.78 36.28 - -
ORN clique-1 COCO 28.25 40.18 26.48 36.71 - -
ORN clique-3 COCO 22.61 37.67 27.05 36.04 - -
Table 3. Experimental results on the
Something-Something dataset (classi-
cation accuracy in % on the test set).
Methods Top1
C3D + Avg [13] 21.50
I3D [13] 27.63
MultiScale TRN [5] 33.60
Ours 35.97Table 4. Experimental results on the
EPIC Kitchens dataset (accuracy in %
on the validation set { methods with
have been re-implemented).
Methods Top1
R18 [16]32.05
I3D-18 [6]34.20
Ours 40.89
while solving VLOG requires detecting the right moment when the human-object
interaction occurs and thus long range reasoning plays a less important role.
Visual features extracted from object regions are the most discriminative,
however object shapes and labels also provide complementary information. Fi-
nally, the last part of Table 2 evaluates the eect of the cliques size for model-
ing the interactions between objects and show that pairwise cliques outperform
cliques of size 1 and 3. We would like to recall, that even with unary cliques
only, interactions between objects are still modeled. However, the model needs
to nd subspaces in the hidden representations associated to each interaction.
CNN architecture and kernel in
ations. The convolutional architecture
of the model was optimized over the validation set of the SS dataset, as shown
in Table 5. The architecture itself (in terms of numbers of layers, lters etc.)
is determined by pre-training on image classication. We optimized the choice
of lter in
ations from 2D to 2.5D or 3D for several convolutional blocks. This
has been optimized for the single head model and using a ResNet-18 variant
to speed up computation. Adding temporal convolutions increases performance
up to 100% w.r.t. to pure 2D baselines. This indicates, without surprise, that
motion is a strong cue. In
ating kernels to 2.5D on the input side and on the
Object Level Visual Reasoning in Videos 15
Table 5. Eect of the CNN architecture (choice of kernel in
ations) on a single head
ResNet-18 network. Accuracy in % on the validation set of Something-Something is
shown. 2.5D kernels are separable kernels: 2D followed by a 1D temporal.
Conv1 Conv2 Conv3 Conv4 Conv5 AggregSS2D 3D 2.5D 2D 3D 2.5D 2D 3D 2.5D 2D 3D 2.5D 2D 3D 2.5D GAP RNN
X- - X- - X- - X- - X- - X -15.73
X- - X- - X- - X- - X- - -X 15.88
-X - -X - -X - -X - -X -X -31.42
- - X - - X - - X - - X - - XX -27.58
X- - X- - X- - X- - -X -X -31.28
X- - X- - X- - -X - -X -X -32.06
X- - X- - -X - -X - -X -X -32.25
X- - X- - X- - X- - - - XX -31.31
X- - X- - X- - - - X - - XX -32.79
X- - X- - - - X - - X - - XX -33.77
-X -X- - X- - X- - X- - X -28.71
-X - -X -X- - X- - X- - X -31.42
- - XX- - X- - X- - X- - X -20.05
- - X - - XX- - X- - X- - X -22.52
output side provided best performances, suggesting that temporal integration
is required at a very low level (motion estimation) as well as on a very high
level, close to reasoning. Our study also corroborates recent research in activity
recognition, indicating that 2.5D kernels provide a good trade-o between high-
capacity and learnable numbers of parameters. Finally temporal integration via
RNN outperforms global average pooling over space and time. The choice of
a (gated) RNN for temporal integration of the activity head features proved
important (see Table 5) compared to global average pooling (GAP) over space
and time.
Visualizing the learned object interactions. Figure 4 shows visualiza-
tions of the pairwise object relationships the model learned from data, in partic-
ular from the VLOG dataset. Each graph is computed for a given activity class,
and strong arcs between two nodes in the graph indicate strong relationships
between the object classes, i.e. the model detects a high correlation between
these relationships and the activity. The graphs were obtained by thresholding
the summed activations of each pairwise relationship ( j; k) in equation (4). Each
pair ( j; k) can be assigned a pair of object classes cj
tandck
tthrough the predic-
tions of the object instance mask predictor. Integrating over all samples of the
dataset for a given class leads to the visualizations in Figure 4. We can see that
the object interactions are highly relevant to the detected activities: the person-
touches-bed activity is correlated to interactions between relevant object classes
person and bed. Similarly, activities human-bowl interaction and human-cup in-
teraction show interactions with the respective objects bowl and cup. Moreover,
other recovered relationships are highly correlated to the scene (for example,
dining-table and bowl for activity human-bowl interaction ).
16 Baradel et al.
Fig. 4. Example of object pairwise interactions learned by our model on VLOG for
four dierent classes. Objects co-occurrences are at the top and learned pairwise objects
interactions are at the bottom. Line thickness indicates learned importance of a given
relation. Interactions have been normalized by the object co-occurrences.
teddy bearteddy bearteddy bearbedcell phonepersonpersonbedteddy bear
bottlewine glasspersonpersonpersonpersonbottle
Fig. 5. Examples of failure cases { a) small sized objects (on the left). Our model
detects a cell phone and a person but fails to detect hand-cell-phone contact ; b) con-
fusion between semantically similar objects (on the right). The model falsly predicts
hand-cup contact instead of hand-glass-contact even though the wine glass is detected.
Finally, Figure 5 shows some failure cases, which are either due to errors
done by the object mask prediction (Mask R-CNN) or by the ORN itself.
7 Conclusion
We presented a method for activity recognition in videos which leverages ob-
ject instance detections for visual reasoning on object interactions over time.
The choice of reasoning over semantically well-dened objects is key to our ap-
proach and outperforms state of the art methods which reason on grid-levels,
such as cells of convolutional feature maps. Temporal dependencies and causal
relationships are dealt with by integrating relationships between dierent time
instants. We evaluated the method on three dicult datasets, on which standard
approaches do not perform well, and report state-of-the-art results.
Acknowledgements. This work was funded by grant Deepvision (ANR-15-
CE23-0029, STPGP-479356-15), a joint French/Canadian call by ANR & NSERC.
Object Level Visual Reasoning in Videos 17
References
1. Abu-El-Haija, S., Kothari, N., Lee, J., Natsev, P., Toderici, G., Varadarajan, B.,
Vijayanarasimhan, S.: Youtube-8m: A large-scale video classication benchmark.
arXiv preprint arxiv:1609.08675 (2016)
2. Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., Baskurt, A.: Sequential deep
learning for human action recognition. In: HBU (2011)
3. Baradel, F., Wolf, C., Mille, J., Taylor, G.: Glimpse clouds: Human activity recog-
nition from unstructured feature points. In: CVPR (2018)
4. Battaglia, P.W., Pascanu, R., Lai, M., Rezende, D.J., Kavukcuoglu, K.: Interaction
networks for learning about objects, relations and physics. In: NIPS (2016)
5. Bolei, Z., Zhang, A.A., Torralba, A.: Temporal relational reasoning in videos. In:
ECCV (2018)
6. Carreira, J., Zisserman, A.: Quo vadis, action recognition? a new model and the
kinetics dataset. In: CVPR (2017)
7. Damen, D., Doughty, H., Farinella, G.M., Fidler, S., Furnari, A., Kazakos, E.,
Moltisanti, D., Munro, J., Perrett, T., Price, W., Wray, M.: Scaling egocentric
vision: The epic-kitchens dataset. In: ECCV (2018)
8. Deng, Z., Vahdat, A., Hu, H., Mori, G.: Structure inference machines: Recurrent
neural networks for analyzing relations in group activity recognition. In: CVPR
(2016)
9. Donahue, J., Hendricks, L.A., Guadarrama, S., Rohrbach, M., Venugopalan, S.,
Saenko, K., Darrell, T.: Long-term recurrent convolutional networks for visual
recognition and description. In: CVPR (2015)
10. Duke, B.: Lintel: Python video decoding. https://github.com/dukebw/lintel
(2018)
11. Fleuret, F., Li, T., Dubout, C., Wampler, E.K., Yantis, S., Geman, D.: Comparing
machines and humans on a visual categorization test. Proceedings of the National
Academy of Sciences of the United States of America 108 43 , 17621{5 (2011)
12. Fouhey, D.F., Kuo, W., Efros, A.A., Malik, J.: From lifestyle vlogs to everyday
interactions. In: CVPR (2018)
13. Goyal, R., Ebrahimi Kahou, S., Michalski, V., Materzynska, J., Westphal, S., Kim,
H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., Hoppe, F., Thurau,
C., Bax, I., Memisevic, R.: The "something something" video database for learning
and evaluating visual common sense. In: ICCV (2017)
14. Gu, C., Sun, C., Ross, D.A., Vondrick, C., Pantofaru, C., Li, Y., Vijayanarasimhan,
S., Toderici, G., Ricco, S., Sukthankar, R., Schmid, C., Malik, J.: Ava: A
video dataset of spatio-temporally localized atomic visual actions. arXiv preprint
arXiv:1705.08421 (2017)
15. He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask r-cnn. In: ICCV (2017)
16. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition.
In: CVPR (2016)
17. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation
9(8), 1735{1780 (1997)
18. Hudson, D., Manning, C.: Compositional attention networks for machine reasoning.
In: ICLR (2018)
19. Jain, A., Zamir, A.R., Savarese, S., Saxena, A.: Structural-RNN: Deep Learning
on Spatio-Temporal Graphs. In: CVPR (2016)
20. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-
scale video classication with convolutional neural networks. In: CVPR (2014)
18 Baradel et al.
21. Kim, J., Ricci, M., Serre, T.: Not-so-CLEVR: Visual relations strain feedforward
neural networks. arXiv preprint arXiv:1802.03390 (2018)
22. Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: ICML (2015)
23. Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Ka-
landitis, Y., Li, L.J., Shamma, D.A., Bernstein, M., Fei-Fei, L.: Visual genome:
Connecting language and vision using crowdsourced dense image annotations. In-
ternational Journal of Computer Vision (IJCV) 123, 32{73 (2017)
24. Luc, P., , Neverova, N., Couprie, C., Verbeek, J., LeCun, Y.: Predicting deeper
into the future of semantic segmentation. In: ICCV (2017)
25. Luvizon, D., Picard, D., Tabia, H.: 2d/3d pose estimation and action recognition
using multitask deep learning. In: CVPR (2018)
26. Monfort, M., Zhou, B., Bargal, S.A., Andonian, A., Yan, T., Ramakrishnan,
K., Brown, L., Fan, Q., Gutfruend, D., Vondrick, C., Oliva, A.: Moments
in time dataset: one million videos for event understanding. arXiv preprint
arXiv:1801.03150 (2018)
27. Perez, E., Vries, H.D., Strub, F., Dumoulin, V., Courville, A.: Learning visual rea-
soning without strong priors. In: ICML Machine Learning in Speech and Language
Processing Workshop (2017)
28. Pickup, L.C., Pan, Z., Wei, D., Shih, Y., Zhang, C., Zisserman, A., Scholkopf, B.,
Freeman, W.T.: Seeing the arrow of time. In: CVPR (2014)
29. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for
3d classication and segmentation. In: CVPR (2017)
30. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards real-time object
detection with region proposal networks. In: NIPS (2015)
31. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z.,
Karpathy, A., Khosla, A., Bernstein, M., Berg, A., Fei-Fei, L.: Imagenet large scale
visual recognition challenge. IJCV 115(3), 211{252 (2015)
32. Santoro, A., Raposo, D., Barrett, D.G., Malinowski, M., Pascanu, R., Battaglia,
P., Lillicrap, T.: A simple neural network module for relational reasoning. In: NIPS
(2017)
33. Shahroudy, A., Liu, J., Ng, T.T., Wang, G.: NTU RGB+D: A Large Scale Dataset
for 3D Human Activity Analysis. In: CVPR (2016)
34. Sharma, S., Kiros, R., Salakhutdinov, R.: Action recognition using visual attention.
In: ICLR Workshop (2016)
35. Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recog-
nition in videos. In: NIPS (2014)
36. Song, S., Lan, C., Xing, J., Zeng, W., Liu, J.: An End-to-End Spatio-Temporal
Attention Model for Human Action Recognition from Skeleton Data. In: AAAI
(2016)
37. Stabinger, S., Rodr guez-S anchez, A., Piater, J.: 25 years of CNNs: Can we compare
to human abstraction capabilities? In: ICANN (2016)
38. van Steenkiste, S., Chang, M., Gre, K., Schmidhuber, J.: Relational neural ex-
pectation maximization: Unsupervised discovery of objects and their interactions.
In: ICLR (2018)
39. Sun, L., Jia, K., Chen, K., Yeung, D., Shi, B.E., Savarese, S.: Lattice long short-
term memory for human action recognition. In: ICCV (2017)
40. Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotem-
poral features with 3d convolutional networks. In: ICCV (2015)
41. Velikovi, P., Cucurull, G., Casanova, A., Romero, A., Li, P., Bengio, Y.: Graph
attention networks. In: ICLR (2018)
Object Level Visual Reasoning in Videos 19
42. Wang, H., Kl aser, A., Schmid, C., Liu, C.L.: Action Recognition by Dense Trajec-
tories. In: CVPR (2011)
43. Watters, N., Zoran, D., Weber, T., Battaglia, P., Pascanu, R., Tacchetti, A.: Visual
interaction networks: Learning a physics simulator from video. In: NIPS (2017)
44. Xie, S., Sun, C., Huang, J., Tu, Z., Murphy, K.: Rethinking spatiotemporal feature
learning for video understanding. arXiv prepring arxiv:1712.04851 (2017)
45. Yeung, S., Russakovsky, O., Jin, N., Andriluka, M., Mori, G., Fei-Fei, L.: Every mo-
ment counts: Dense detailed labeling of actions in complex videos. arXiv preprint
arXiv:1507.05738 (2015)
46. Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R.R., Smola,
A.J.: Deep sets. In: NIPS. pp. 3391{3401 (2017), http://papers.nips.cc/paper/
6931-deep-sets.pdf