Year: 2019

Abstract:
We present trellis networks, a new architecture for sequence modeling. On the one hand, a trellis network is a temporal convolutional network with special structure, characterized by weight tying across depth and direct injection of the input into deep layers. On the other hand, we show that truncated recurrent networks are equivalent to trellis networks with special sparsity structure in their weight matrices. Thus trellis networks with general weight matrices generalize truncated recurrent networks. We leverage these connections to design high-performing trellis networks that absorb structural and algorithmic elements from both recurrent and convolutional models. Experiments demonstrate that trellis networks outperform the current state of the art methods on a variety of challenging benchmarks, including word-level language modeling and character-level language modeling tasks, and stress tests designed to evaluate long-term memory retention. The code is available here1.

heading: 1 INTRODUCTION
text: What is the best architecture for sequence modeling? Recent research has produced significant progress on multiple fronts. Recurrent networks, such as LSTMs, continue to be optimized and extended (Merity et al., 2018b; Melis et al., 2018; Yang et al., 2018; Trinh et al., 2018). Temporal convolutional networks have demonstrated impressive performance, particularly in modeling longrange context (van den Oord et al., 2016; Dauphin et al., 2017; Bai et al., 2018). And architectures based on self-attention are gaining ground (Vaswani et al., 2017; Santoro et al., 2018).
In this paper, we introduce a new architecture for sequence modeling, the Trellis Network. We aim to both improve empirical performance on sequence modeling benchmarks and shed light on the relationship between two existing model families: recurrent and convolutional networks.
On the one hand, a trellis network is a special temporal convolutional network, distinguished by two unusual characteristics. First, the weights are tied across layers. That is, weights are shared not only by all time steps but also by all network layers, tying them into a regular trellis pattern. Second, the input is injected into all network layers. That is, the input at a given time-step is provided not only to the first layer, but directly to all layers in the network. So far, this may seem merely as a peculiar convolutional network for processing sequences, and not one that would be expected to perform particularly well.
Yet on the other hand, we show that trellis networks generalize truncated recurrent networks (recurrent networks with bounded memory horizon). The precise derivation of this connection is one of the key contributions of our work. It allows trellis networks to serve as bridge between recurrent and convolutional architectures, benefitting from algorithmic and architectural techniques developed in either context. We leverage these relationships to design high-performing trellis networks that absorb ideas from both architectural families. Beyond immediate empirical gains, these connections may serve as a step towards unification in sequence modeling.
We evaluate trellis networks on challenging benchmarks, including word-level language modeling on the standard Penn Treebank (PTB) and the much larger WikiText-103 (WT103) datasets; character-level language modeling on Penn Treebank; and standard stress tests (e.g. sequential MNIST, permuted MNIST, etc.) designed to evaluate long-term memory retention. On word-level
1https://github.com/locuslab/trellisnet
Penn Treebank, a trellis network outperforms by more than a unit of perplexity the recent architecture search work of Pham et al. (2018), as well as the recent results of Melis et al. (2018), which leveraged the Google Vizier service for exhaustive hyperparameter search. On character-level Penn Treebank, a trellis network outperforms the thorough optimization work of Merity et al. (2018a). On word-level WikiText-103, a trellis network outperforms by 7.6% in perplexity the contemporaneous self-attention-based Relational Memory Core (Santoro et al., 2018), and by 11.5% the work of Merity et al. (2018a). (Concurrently with our work, Dai et al. (2019) employ a transformer and achieve even better results on WikiText-103.) On stress tests, trellis networks outperform recent results achieved by recurrent networks and self-attention (Trinh et al., 2018). It is notable that the prior state of the art across these benchmarks was held by models with sometimes dramatic mutual differences.

heading: 2 BACKGROUND
text: Recurrent networks (Elman, 1990; Werbos, 1990; Graves, 2012), particularly with gated cells such as LSTMs (Hochreiter & Schmidhuber, 1997) and GRUs (Cho et al., 2014), are perhaps the most popular architecture for modeling temporal sequences. Recurrent architectures have been used to achieve breakthrough results in natural language processing and other domains (Sutskever et al., 2011; Graves, 2013; Sutskever et al., 2014; Bahdanau et al., 2015; Vinyals et al., 2015; Karpathy & Li, 2015). Convolutional networks have also been widely used for sequence processing (Waibel et al., 1989; Collobert et al., 2011). Recent work indicates that convolutional networks are effective on a variety of sequence modeling tasks, particularly ones that demand long-range information propagation (van den Oord et al., 2016; Kalchbrenner et al., 2016; Dauphin et al., 2017; Gehring et al., 2017; Bai et al., 2018). A third notable approach to sequence processing that has recently gained ground is based on self-attention (Vaswani et al., 2017; Santoro et al., 2018; Chen et al., 2018). Our work is most closely related to the first two approaches. In particular, we establish a strong connection between recurrent and convolutional networks and introduce a model that serves as a bridge between the two. A related recent theoretical investigation showed that under a certain stability condition, recurrent networks can be well-approximated by feed-forward models (Miller & Hardt, 2018).
There have been many combinations of convolutional and recurrent networks (Sainath et al., 2015). For example, convolutional LSTMs combine convolutional and recurrent units (Donahue et al., 2015; Venugopalan et al., 2015; Shi et al., 2015). Quasi-recurrent neural networks interleave convolutional and recurrent layers (Bradbury et al., 2017). Techniques introduced for convolutional networks, such as dilation, have been applied to RNNs (Chang et al., 2017). Our work establishes a deeper connection, deriving a direct mapping across the two architectural families and providing a structural bridge that can incorporate techniques from both sides.

heading: 3 SEQUENCE MODELING AND TRELLIS NETWORKS
text: Sequence modeling. Given an input x1:T = x1, . . . , xT with sequence length T , a sequence model is any function G : X T → YT such that
y1:T = y1, . . . , yT = G(x1, . . . , xT ), (1)
where yt should only depend on x1:t and not on xt+1:T (i.e. no leakage of information from the future). This causality constraint is essential for autoregressive modeling.
In this section, we describe a new architecture for sequence modeling, referred to as a trellis network or TrellisNet. In particular, we provide an atomic view of TrellisNet, present its fundamental features, and highlight the relationship to convolutional networks. Section 4 will then elaborate on the relationship of trellis networks to convolutional and recurrent models.
Notation. We use x1:T = (x1, . . . , xT ) to denote a length-T input sequence, where vector xt ∈ Rp is the input at time step t. Thus x1:T ∈ RT×p. We use z(i)t ∈ Rq to represent the hidden unit at time t in layer i of the network. We use Conv1D(x;W ) to denote a 1D convolution with a kernel W applied to input x = x1:T .
A basic trellis network. At the most basic level, a feature vector z(i+1)t+1 at time step t+ 1 and level i+ 1 of TrellisNet is computed via three steps, illustrated in Figure 1a:
1. The hidden input comprises the hidden outputs z(i)t , z (i) t+1 ∈ Rq from the previous layer i, as well
as an injection of the input vectors xt, xt+1. At level 0, we initialize to z (0) t = 0.
2. A pre-activation output ẑ(i+1)t+1 ∈ Rr is produced by a feed-forward linear transformation:
ẑ (i+1) t+1 =W1 [ xt z (i) t ] +W2 [ xt+1 z (i) t+1 ] , (2)
where W1,W2 ∈ Rr×(p+q) are weights, and r is the size of the pre-activation output ẑ(i+1)t+1 . (Here and throughout the paper, all linear transformations can include additive biases. We omit these for clarity.)
3. The output z(i+1)t+1 is produced by a nonlinear activation function f : Rr × Rq → Rq applied to the pre-activation output ẑ(i+1)t+1 and the output z (i) t from the previous layer. More formally,
z (i+1) t+1 = f ( ẑ (i+1) t+1 , z (i) t ) .

heading: A full trellis network can be built by tiling this elementary procedure across time and depth. Given
text: an input sequence x1:T , we apply the same production procedure across all time steps and all layers, using the same weights. The transformation is the same for all elements in the temporal dimension and in the depth dimension. This is illustrated in Figure 1b. Note that since we inject the same input sequence at every layer of the TrellisNet, we can precompute the linear transformation x̃t+1 = W x 1 xt +W x 2 xt+1 for all layers i. This identical linear combination of the input can then be added in each layer i to the appropriate linear combination of the hidden units, W z1 z (i) t + W z 2 z (i) t+1, where W xj ∈ Rr×p,W zj ∈ Rr×q.
Now observe that in each level of the network, we are in effect performing a 1D convolution over the hidden units z(i)1:T . The output of this convolution is then passed through the activation function f . Formally, withW ∈ Rr×q as the kernel weight matrix, the computation in layer i can be summarized as follows (Figure 1b):
ẑ (i+1) 1:T = Conv1D ( z (i) 1:T ;W ) + x̃1:T , z (i+1) 1:T = f ( ẑ (i+1) 1:T , z (i) 1:T−1 ) . (3)
The resulting network operates in feed-forward fashion, with deeper elements having progressively larger receptive fields. There are, however, important differences from typical (temporal) convolutional networks. Notably, the filter matrix is shared across all layers. That is, the weights are tied not only across time but also across depth. (Vogel & Pock (2017) have previously tied weights across depth in image processing.) Another difference is that the transformed input sequence x̃1:T is directly injected into each hidden layer. These differences and their importance will be analyzed further in Section 4.
The activation function f in Equation (3) can be any nonlinearity that processes the pre-activation output ẑ(i+1)1:T and the output from the previous layer z (i) 1:T−1. We will later describe an activation function based on the LSTM cell. The rationale for its use will become clearer in light of the analysis presented in the next section.

heading: 4 TRELLISNET, TCN, AND RNN
text: In this section, we analyze the relationships between trellis networks, convolutional networks, and recurrent networks. In particular, we show that trellis networks can serve as a bridge between convolutional and recurrent networks. On the one hand, TrellisNet is a special form of temporal convolutional networks (TCN); this has already been clear in Section 3 and will be discussed further in Section 4.1. On the other hand, any truncated RNN can be represented as a TrellisNet with special structure in the interlayer transformations; this will be the subject of Section 4.2. These connections allow TrellisNet to harness architectural elements and regularization techniques from both TCNs and RNNs; this will be summarized in Section 4.3.

heading: 4.1 TRELLISNET AND TCN
text: We briefly introduce TCNs here, and refer the readers to Bai et al. (2018) for a more thorough discussion. Briefly, a temporal convolutional network (TCN) is a ConvNet that uses one-dimensional convolutions over the sequence. The convolutions are causal, meaning that, at each layer, the transformation at time t can only depend on previous layer units at times t or earlier, not from later points in time. Such approaches were used going back to the late 1980s, under the name of “time-delay neural networks” (Waibel et al., 1989), and have received significant interest in recent years due to their application in architectures such as WaveNet (van den Oord et al., 2016).
In essence, TrellisNet is a special kind of temporal convolutional network. TCNs have two distinctive characteristics: 1) causal convolution in each layer to satisfy the causality constraint and 2) deep stacking of layers to increase the effective history length (i.e. receptive field). Trellis networks have both of these characteristics. The basic model presented in Section 3 can easily be elaborated with larger kernel sizes, dilated convolutions, and other architectural elements used in TCNs; some of these are reviewed further in Section 4.3.
However, TrellisNet is not a general TCN. As mentioned in Section 3, two important differences are: 1) the weights are tied across layers and 2) the linearly transformed input x̃1:T is injected into each layer. Weight tying can be viewed as a form of regularization that can stabilize training, support generalization, and significantly reduce the size of the model. Input injection mixes deep features with the original sequence. These structural characteristics will be further illuminated by the connection between trellis networks and recurrent networks, presented next.

heading: 4.2 TRELLISNET AND RNN
text: Recurrent networks appear fundamentally different from convolutional networks. Instead of operating on all elements of a sequence in parallel in each layer, an RNN processes one input element at a time and unrolls in the time dimension. Given a non-linearity g (which could be a sigmoid or a more elaborate cell), we can summarize the transformations in an L-layer RNN at time-step t as follows:
h (i) t = g
( W
(i) hx h (i−1) t +W (i) hhh (i) t−1 ) for 1 ≤ i ≤ L, h(0)t = xt. (4)
Despite the apparent differences, we will now show that any RNN unrolled to a finite length is equivalent to a TrellisNet with special sparsity structure in the kernel matrix W . We begin by formally defining the notion of a truncated (i.e. finite-horizon) RNN. Definition 1. Given an RNN ρ, a corresponding M-truncated RNN ρM , applied to the sequence x1:T , produces at time step t the output yt by applying ρ to the sequence xt−M+1:t (here x<0 = 0).
Theorem 1. Let ρM be an M -truncated RNN with L layers and hidden unit dimensionality d. Then there exists an equivalent TrellisNet τ with depth (M + L − 1) and layer width (i.e. number of channels in each hidden layer) Ld. Specifically, for any x1:T , ρM (x1:T ) = τL(d−1)+1:Ld(x1:T ) (i.e. the TrellisNet outputs contain the RNN outputs).
Theorem 1 states that any M -truncated RNN can be represented as a TrellisNet. How severe of a restriction is M -truncation? Note that M -truncation is intimately related to truncated backpropagation-through-time (BPTT), used pervasively in training recurrent networks on long sequences. While RNNs can in principle retain unlimited history, there is both empirical and theoretical evidence that the memory horizon of RNNs is bounded (Bai et al., 2018; Khandelwal et al., 2018;
Miller & Hardt, 2018). Furthermore, if desired, TrellisNets can recover exactly a common method of applying RNNs to long sequences – hidden state repackaging, i.e. copying the hidden state across subsequences. This is accomplished using an analogous form of hidden state repackaging, detailed in Appendix B.
Proof of Theorem 1. Let h(i)t,t′ ∈ Rd be the hidden state at time t and layer i of the truncated RNN ρt−t
′+1 (i.e., the RNN begun at time t′ and run until time t). Note that without truncation, history starts at time t′ = 1, so the hidden state h(i)t of ρ can be equivalently expressed as h (i) t,1. When t
′ > t, we define ht,t′ = 0 (i.e. no history information if the clock starts in the future).
By assumption, ρM is an RNN defined by the following parameters: {W (i)hx ,W (i) hh , g,M}, where W (i) hh ∈ Rw×d for all i, W (1) hx ∈ Rw×p, and W (i) hx ∈ Rw×d for all i = 2, . . . , L are the weight matrices at each layer (w is the dimension of pre-activation output). We now construct a TrellisNet τ according to the exact definition in Section 3, with parameters {W1,W2, f}, where
W1 =  0 W (1) hh 0 . . . 0 0 0 W (2) hh . . . 0 ... ... ... . . .
... 0 0 0 . . . W
(L) hh
 ,W2 =  W (1) hx 0 . . . 0 0 0 W (2) hx . . . 0 0 ... ... . . . ...
... 0 0 . . . W
(L) hx 0  , (5) such that W1,W2 ∈ RLw×(p+Ld). We define a nonlinearity f by f(α, β) = g(α) (i.e. applying g only on the first entry).
Let t ∈ [T ] , j ≥ 0 be arbitrary and fixed. We now claim that the hidden unit at time t and layer j of TrellisNet τ can be expressed in terms of hidden units at time t in truncated forms of ρ:
z (j) t = [ h (1) t,t−j+1 h (2) t,t−j+2 . . . h (L) t,t−j+L ]> ∈ RLd, (6)
where z(j)t is the time-t hidden state at layer j of τ and h (i) t,t′ is the time-t hidden state at layer i of ρt−t ′+1.
We prove Eq. (6) by induction on j. As a base case, consider j = 0; i.e. the input layer of τ . Since ht,t′ = 0 when t′ > t, we have that z (0) j = [0 0 . . . 0]
>. (Recall that in the input layer of TrellisNet we initialize z(0)t = 0.) For the inductive step, suppose Eq. (6) holds for layer j, and consider layer j + 1. By the feed-forward transformation of TrellisNet defined in Eq. (2) and the nonlinearity f we defined above, we have:
ẑ (j+1) t =W1 [ xt−1 z (j) t−1 ] +W2 [ xt z (j) t ] (7)
=  0 W (1) hh 0 . . . 0 0 0 W (2) hh . . . 0 ... ... ... . . .
... 0 0 0 . . . W
(L) hh
  xt−1 h (1) t−1,t−j
... h (L) t−1,t−j+L−1
+  W (1) hx 0 . . . 0 0 0 W (2) hx . . . 0 0 ... ... . . . ...
... 0 0 . . . W
(L) hx 0
  xt h (1) t,t−j+1
... h (L) t,t−j+L  (8)
=  W (1) hh h (1) t−1,t−j +W (1) hx xt
... W
(L) hh h (L) t−1,t−j+L−1 +W (L) hx h (L−1) t,t−j+L−1  (9) z (j+1) t = f(ẑ (j+1) t , z (j) t−1) = g(ẑ (j+1) t ) = [ h (1) t,t−j h (2) t,t−j+1 . . . h (L) t,t−j+L−1 ]> (10)
where in Eq. (10) we apply the RNN non-linearity g following Eq. (4). Therefore, by induction, we have shown that Eq. (6) holds for all j ≥ 0. If TrellisNet τ hasM+L−1 layers, then at the final layer we have z(M+L−1)t = [. . . . . . h(L)t,t+1−M ]>. Since ρM is an L-layer M -truncated RNN, this (taking the last d channels of z(M+L−1)t ) is exactly the output of ρM at time t.
In other words, we have shown that ρM is equivalent to a TrellisNet with sparse kernel matrices W1,W2. This completes the proof.
Note that the convolutions in the TrellisNet τ constructed in Theorem 1 are sparse, as shown in Eq. (5). They are related to group convolutions (Krizhevsky et al., 2012), but have an unusual form because group k at time t is convolved with group k − 1 at time t + 1. We refer to these as mixed group convolutions. Moreover, while Theorem 1 assumes that all layers of ρM have the same dimensionality d for clarity, the proof easily generalizes to cases where each layer has different widths.
For didactic purposes, we recap and illustrate the construction in the case of a 2-layer RNN. The key challenge is that a naı̈ve unrolling of the RNN into a feed-forward network does not produce a convolutional network, since the linear transformation weights are not constant across a layer. The solution, illustrated in Figure 2a, is to organize each hidden unit into groups of channels, such that each TrellisNet unit represents 3 RNN units simultaneously (for xt, h(1)t , h (2) t ). Each TrellisNet unit thus has (p + 2d) channels. The interlayer transformation can then be expressed as a mixed group convolution, illustrated in Figure 2b. This can be represented as a sparse convolution with the structure given in Eq. (5) (with L = 2). Applying the nonlinearity g on the pre-activation output, this exactly reproduces the transformations in the original 2-layer RNN.
The TrellisNet that emerges from this construction has special sparsity structure in the weight matrix.

heading: It stands to reason that a general TrellisNet with an unconstrained (dense) weight matrix W may have greater expressive power: it can model a broader class of transformations than the original RNN ρM . Note that while the hidden channels of the TrellisNet τ constructed in the proof of
text: Theorem 1 are naturally arranged into groups that represent different layers of the RNN ρM (Eq. (6)), an unconstrained dense weight matrix W no longer admits such an interpretation. A model defined by a dense weight matrix is fundamentally distinct from the RNN ρM that served as our point of departure. We take advantage of this expressivity and use general weight matrices W , as presented in Section 3, in our experiments. Our ablation analysis will show that such generalized dense transformations are beneficial, even when model capacity is controlled for.
The proof of Theorem 1 did not delve into the inner structure of the nonlinear transformation g in RNN (or f in the constructed TrellisNet). For a vanilla RNN, for instance, f is usually an elementwise sigmoid or tanh function. But the construction in Theorem 1 applies just as well to RNNs with structured cells, such as LSTMs and GRUs. We adopt LSTM cells for the TrellisNets in our experiments and provide a detailed treatment of this nonlinearity in Section 5.1 and Appendix A.

heading: 4.3 TRELLISNET AS A BRIDGE BETWEEN RECURRENT AND CONVOLUTIONAL MODELS
text: In Section 4.1 we concluded that TrellisNet is a special kind of TCN, characterized by weight tying and input injection. In Section 4.2 we established that TrellisNet is a generalization of truncated
RNNs. These connections along with the construction in our proof of Theorem 1 allow TrellisNets to benefit significantly from techniques developed originally for RNNs, while also incorporating architectural and algorithmic motifs developed for convolutional networks. We summarize a number of techniques here. From recurrent networks, we can integrate 1) structured nonlinear activations (e.g. LSTM and GRU gates); 2) variational RNN dropout (Gal & Ghahramani, 2016); 3) recurrent DropConnect (Merity et al., 2018b); and 4) history compression and repackaging. From convolutional networks, we can adapt 1) larger kernels and dilated convolutions (Yu & Koltun, 2016); 2) auxiliary losses at intermediate layers (Lee et al., 2015; Xie & Tu, 2015); 3) weight normalization (Salimans & Kingma, 2016); and 4) parallel convolutional processing. Being able to directly incorporate techniques from both streams of research is one of the benefits of trellis networks. We leverage this in our experiments and provide a more comprehensive treatment of these adaptations in Appendix B.

heading: 5 EXPERIMENTS
text: 5.1 A TRELLISNET WITH GATED ACTIVATION
xt <latexit sha1_base64="eEPz6pgZfvj6K8i8F78InKVPN1o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwMYyovmA5Ah7m71kyd7esTsnhiM/wcZCEVt/kZ3/xk1yhSY+GHi8N8PMvCCRwqDrfjuFtfWNza3idmlnd2//oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9mfvuRayNi9YCThPsRHSoRCkbRSvdPfeyXK27VnYOsEi8nFcjR6Je/eoOYpRFXyCQ1puu5CfoZ1SiY5NNSLzU8oWxMh7xrqaIRN342P3VKzqwyIGGsbSkkc/X3REYjYyZRYDsjiiOz7M3E/7xuiuGVnwmVpMgVWywKU0kwJrO/yUBozlBOLKFMC3srYSOqKUObTsmG4C2/vEpaF1XPrXp3l5X6dR5HEU7gFM7BgxrU4RYa0AQGQ3iGV3hzpPPivDsfi9aCk88cwx84nz9vbo3d</latexit><latexit sha1_base64="eEPz6pgZfvj6K8i8F78InKVPN1o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwMYyovmA5Ah7m71kyd7esTsnhiM/wcZCEVt/kZ3/xk1yhSY+GHi8N8PMvCCRwqDrfjuFtfWNza3idmlnd2//oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9mfvuRayNi9YCThPsRHSoRCkbRSvdPfeyXK27VnYOsEi8nFcjR6Je/eoOYpRFXyCQ1puu5CfoZ1SiY5NNSLzU8oWxMh7xrqaIRN342P3VKzqwyIGGsbSkkc/X3REYjYyZRYDsjiiOz7M3E/7xuiuGVnwmVpMgVWywKU0kwJrO/yUBozlBOLKFMC3srYSOqKUObTsmG4C2/vEpaF1XPrXp3l5X6dR5HEU7gFM7BgxrU4RYa0AQGQ3iGV3hzpPPivDsfi9aCk88cwx84nz9vbo3d</latexit><latexit sha1_base64="eEPz6pgZfvj6K8i8F78InKVPN1o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwMYyovmA5Ah7m71kyd7esTsnhiM/wcZCEVt/kZ3/xk1yhSY+GHi8N8PMvCCRwqDrfjuFtfWNza3idmlnd2//oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9mfvuRayNi9YCThPsRHSoRCkbRSvdPfeyXK27VnYOsEi8nFcjR6Je/eoOYpRFXyCQ1puu5CfoZ1SiY5NNSLzU8oWxMh7xrqaIRN342P3VKzqwyIGGsbSkkc/X3REYjYyZRYDsjiiOz7M3E/7xuiuGVnwmVpMgVWywKU0kwJrO/yUBozlBOLKFMC3srYSOqKUObTsmG4C2/vEpaF1XPrXp3l5X6dR5HEU7gFM7BgxrU4RYa0AQGQ3iGV3hzpPPivDsfi9aCk88cwx84nz9vbo3d</latexit><latexit sha1_base64="eEPz6pgZfvj6K8i8F78InKVPN1o=">AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EiIVFwMYyovmA5Ah7m71kyd7esTsnhiM/wcZCEVt/kZ3/xk1yhSY+GHi8N8PMvCCRwqDrfjuFtfWNza3idmlnd2//oHx41DJxqhlvsljGuhNQw6VQvIkCJe8kmtMokLwdjG9mfvuRayNi9YCThPsRHSoRCkbRSvdPfeyXK27VnYOsEi8nFcjR6Je/eoOYpRFXyCQ1puu5CfoZ1SiY5NNSLzU8oWxMh7xrqaIRN342P3VKzqwyIGGsbSkkc/X3REYjYyZRYDsjiiOz7M3E/7xuiuGVnwmVpMgVWywKU0kwJrO/yUBozlBOLKFMC3srYSOqKUObTsmG4C2/vEpaF1XPrXp3l5X6dR5HEU7gFM7BgxrU4RYa0AQGQ3iGV3hzpPPivDsfi9aCk88cwx84nz9vbo3d</latexit>
xt+1 <latexit sha1_base64="nkvIntHmKtHH1HtvVwUH5BQy280=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoiBT14KHjxWMF+QBvKZrtpl242YXciltAf4cWDIl79Pd78N27bHLT1wcDjvRlm5gWJFAZd99sprK1vbG4Vt0s7u3v7B+XDo5aJU814k8Uy1p2AGi6F4k0UKHkn0ZxGgeTtYHw789uPXBsRqwecJNyP6FCJUDCKVmo/9TO88Kb9csWtunOQVeLlpAI5Gv3yV28QszTiCpmkxnQ9N0E/oxoFk3xa6qWGJ5SN6ZB3LVU04sbP5udOyZlVBiSMtS2FZK7+nshoZMwkCmxnRHFklr2Z+J/XTTG89jOhkhS5YotFYSoJxmT2OxkIzRnKiSWUaWFvJWxENWVoEyrZELzll1dJ67LquVXvvlap3+RxFOEETuEcPLiCOtxBA5rAYAzP8ApvTuK8OO/Ox6K14OQzx/AHzucPDcKPWQ==</latexit><latexit sha1_base64="nkvIntHmKtHH1HtvVwUH5BQy280=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoiBT14KHjxWMF+QBvKZrtpl242YXciltAf4cWDIl79Pd78N27bHLT1wcDjvRlm5gWJFAZd99sprK1vbG4Vt0s7u3v7B+XDo5aJU814k8Uy1p2AGi6F4k0UKHkn0ZxGgeTtYHw789uPXBsRqwecJNyP6FCJUDCKVmo/9TO88Kb9csWtunOQVeLlpAI5Gv3yV28QszTiCpmkxnQ9N0E/oxoFk3xa6qWGJ5SN6ZB3LVU04sbP5udOyZlVBiSMtS2FZK7+nshoZMwkCmxnRHFklr2Z+J/XTTG89jOhkhS5YotFYSoJxmT2OxkIzRnKiSWUaWFvJWxENWVoEyrZELzll1dJ67LquVXvvlap3+RxFOEETuEcPLiCOtxBA5rAYAzP8ApvTuK8OO/Ox6K14OQzx/AHzucPDcKPWQ==</latexit><latexit sha1_base64="nkvIntHmKtHH1HtvVwUH5BQy280=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoiBT14KHjxWMF+QBvKZrtpl242YXciltAf4cWDIl79Pd78N27bHLT1wcDjvRlm5gWJFAZd99sprK1vbG4Vt0s7u3v7B+XDo5aJU814k8Uy1p2AGi6F4k0UKHkn0ZxGgeTtYHw789uPXBsRqwecJNyP6FCJUDCKVmo/9TO88Kb9csWtunOQVeLlpAI5Gv3yV28QszTiCpmkxnQ9N0E/oxoFk3xa6qWGJ5SN6ZB3LVU04sbP5udOyZlVBiSMtS2FZK7+nshoZMwkCmxnRHFklr2Z+J/XTTG89jOhkhS5YotFYSoJxmT2OxkIzRnKiSWUaWFvJWxENWVoEyrZELzll1dJ67LquVXvvlap3+RxFOEETuEcPLiCOtxBA5rAYAzP8ApvTuK8OO/Ox6K14OQzx/AHzucPDcKPWQ==</latexit><latexit sha1_base64="nkvIntHmKtHH1HtvVwUH5BQy280=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoiBT14KHjxWMF+QBvKZrtpl242YXciltAf4cWDIl79Pd78N27bHLT1wcDjvRlm5gWJFAZd99sprK1vbG4Vt0s7u3v7B+XDo5aJU814k8Uy1p2AGi6F4k0UKHkn0ZxGgeTtYHw789uPXBsRqwecJNyP6FCJUDCKVmo/9TO88Kb9csWtunOQVeLlpAI5Gv3yV28QszTiCpmkxnQ9N0E/oxoFk3xa6qWGJ5SN6ZB3LVU04sbP5udOyZlVBiSMtS2FZK7+nshoZMwkCmxnRHFklr2Z+J/XTTG89jOhkhS5YotFYSoJxmT2OxkIzRnKiSWUaWFvJWxENWVoEyrZELzll1dJ67LquVXvvlap3+RxFOEETuEcPLiCOtxBA5rAYAzP8ApvTuK8OO/Ox6K14OQzx/AHzucPDcKPWQ==</latexit>
W1<latexit sha1_base64="lXMe2VVJVimguU04W2823Zlih1M=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt2kO2vpg4PHeDDPzgkQKg6777ZTW1jc2t8rblZ3dvf2D6uFR28SpZrzFYhnrbkANl0LxFgqUvJtoTqNA8k4wuZ37nSeujYjVI04T7kd0pEQoGEUrPXQG3qBac+tuDrJKvILUoEBzUP3qD2OWRlwhk9SYnucm6GdUo2CSzyr91PCEsgkd8Z6likbc+Fl+6oycWWVIwljbUkhy9fdERiNjplFgOyOKY7PszcX/vF6K4bWfCZWkyBVbLApTSTAm87/JUGjOUE4toUwLeythY6opQ5tOxYbgLb+8StoXdc+te/eXtcZNEUcZTuAUzsGDK2jAHTShBQxG8Ayv8OZI58V5dz4WrSWnmDmGP3A+fwDW8413</latexit><latexit sha1_base64="lXMe2VVJVimguU04W2823Zlih1M=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt2kO2vpg4PHeDDPzgkQKg6777ZTW1jc2t8rblZ3dvf2D6uFR28SpZrzFYhnrbkANl0LxFgqUvJtoTqNA8k4wuZ37nSeujYjVI04T7kd0pEQoGEUrPXQG3qBac+tuDrJKvILUoEBzUP3qD2OWRlwhk9SYnucm6GdUo2CSzyr91PCEsgkd8Z6likbc+Fl+6oycWWVIwljbUkhy9fdERiNjplFgOyOKY7PszcX/vF6K4bWfCZWkyBVbLApTSTAm87/JUGjOUE4toUwLeythY6opQ5tOxYbgLb+8StoXdc+te/eXtcZNEUcZTuAUzsGDK2jAHTShBQxG8Ayv8OZI58V5dz4WrSWnmDmGP3A+fwDW8413</latexit><latexit sha1_base64="lXMe2VVJVimguU04W2823Zlih1M=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt2kO2vpg4PHeDDPzgkQKg6777ZTW1jc2t8rblZ3dvf2D6uFR28SpZrzFYhnrbkANl0LxFgqUvJtoTqNA8k4wuZ37nSeujYjVI04T7kd0pEQoGEUrPXQG3qBac+tuDrJKvILUoEBzUP3qD2OWRlwhk9SYnucm6GdUo2CSzyr91PCEsgkd8Z6likbc+Fl+6oycWWVIwljbUkhy9fdERiNjplFgOyOKY7PszcX/vF6K4bWfCZWkyBVbLApTSTAm87/JUGjOUE4toUwLeythY6opQ5tOxYbgLb+8StoXdc+te/eXtcZNEUcZTuAUzsGDK2jAHTShBQxG8Ayv8OZI58V5dz4WrSWnmDmGP3A+fwDW8413</latexit><latexit sha1_base64="lXMe2VVJVimguU04W2823Zlih1M=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHisaD+gDWWz3bRLN5uwOxFK6E/w4kERr/4ib/4bt2kO2vpg4PHeDDPzgkQKg6777ZTW1jc2t8rblZ3dvf2D6uFR28SpZrzFYhnrbkANl0LxFgqUvJtoTqNA8k4wuZ37nSeujYjVI04T7kd0pEQoGEUrPXQG3qBac+tuDrJKvILUoEBzUP3qD2OWRlwhk9SYnucm6GdUo2CSzyr91PCEsgkd8Z6likbc+Fl+6oycWWVIwljbUkhy9fdERiNjplFgOyOKY7PszcX/vF6K4bWfCZWkyBVbLApTSTAm87/JUGjOUE4toUwLeythY6opQ5tOxYbgLb+8StoXdc+te/eXtcZNEUcZTuAUzsGDK2jAHTShBQxG8Ayv8OZI58V5dz4WrSWnmDmGP3A+fwDW8413</latexit> W2<latexit sha1_base64="uQ/4VJ3D9Rsf5zu3tZUkUWQBkuo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUPFU8OKxov2ANpTNdtIu3WzC7kYooT/BiwdFvPqLvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M79zhMqzWP5aKYJ+hEdSR5yRo2VHjqD2qBccavuAmSdeDmpQI7moPzVH8YsjVAaJqjWPc9NjJ9RZTgTOCv1U40JZRM6wp6lkkao/Wxx6oxcWGVIwljZkoYs1N8TGY20nkaB7YyoGetVby7+5/VSE177GZdJalCy5aIwFcTEZP43GXKFzIipJZQpbm8lbEwVZcamU7IheKsvr5N2req5Ve/+qtK4yeMowhmcwyV4UIcG3EETWsBgBM/wCm+OcF6cd+dj2Vpw8plT+APn8wfYd414</latexit><latexit sha1_base64="uQ/4VJ3D9Rsf5zu3tZUkUWQBkuo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUPFU8OKxov2ANpTNdtIu3WzC7kYooT/BiwdFvPqLvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M79zhMqzWP5aKYJ+hEdSR5yRo2VHjqD2qBccavuAmSdeDmpQI7moPzVH8YsjVAaJqjWPc9NjJ9RZTgTOCv1U40JZRM6wp6lkkao/Wxx6oxcWGVIwljZkoYs1N8TGY20nkaB7YyoGetVby7+5/VSE177GZdJalCy5aIwFcTEZP43GXKFzIipJZQpbm8lbEwVZcamU7IheKsvr5N2req5Ve/+qtK4yeMowhmcwyV4UIcG3EETWsBgBM/wCm+OcF6cd+dj2Vpw8plT+APn8wfYd414</latexit><latexit sha1_base64="uQ/4VJ3D9Rsf5zu3tZUkUWQBkuo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUPFU8OKxov2ANpTNdtIu3WzC7kYooT/BiwdFvPqLvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M79zhMqzWP5aKYJ+hEdSR5yRo2VHjqD2qBccavuAmSdeDmpQI7moPzVH8YsjVAaJqjWPc9NjJ9RZTgTOCv1U40JZRM6wp6lkkao/Wxx6oxcWGVIwljZkoYs1N8TGY20nkaB7YyoGetVby7+5/VSE177GZdJalCy5aIwFcTEZP43GXKFzIipJZQpbm8lbEwVZcamU7IheKsvr5N2req5Ve/+qtK4yeMowhmcwyV4UIcG3EETWsBgBM/wCm+OcF6cd+dj2Vpw8plT+APn8wfYd414</latexit><latexit sha1_base64="uQ/4VJ3D9Rsf5zu3tZUkUWQBkuo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUPFU8OKxov2ANpTNdtIu3WzC7kYooT/BiwdFvPqLvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M79zhMqzWP5aKYJ+hEdSR5yRo2VHjqD2qBccavuAmSdeDmpQI7moPzVH8YsjVAaJqjWPc9NjJ9RZTgTOCv1U40JZRM6wp6lkkao/Wxx6oxcWGVIwljZkoYs1N8TGY20nkaB7YyoGetVby7+5/VSE177GZdJalCy5aIwFcTEZP43GXKFzIipJZQpbm8lbEwVZcamU7IheKsvr5N2req5Ve/+qtK4yeMowhmcwyV4UIcG3EETWsBgBM/wCm+OcF6cd+dj2Vpw8plT+APn8wfYd414</latexit>
L ay
er i !
i + 1 <latexit sha1_base64="Omx146Fv69pKxCTzcUXdA+GkZwY=">AAACAnicbVBNS8NAEN3Ur1q/op7Ey2IrCEJJelE8Fbx48FDBfkAbyma7aZdusmF3ooRQvPhXvHhQxKu/wpv/xm2bg1YfDDzem2Fmnh8LrsFxvqzC0vLK6lpxvbSxubW9Y+/utbRMFGVNKoVUHZ9oJnjEmsBBsE6sGAl9wdr++HLqt++Y0lxGt5DGzAvJMOIBpwSM1LcPrknKFK5w3FN8OAKilLzH/NSt9O2yU3VmwH+Jm5MyytHo25+9gaRJyCKggmjddZ0YvIwo4FSwSamXaBYTOiZD1jU0IiHTXjZ7YYKPjTLAgVSmIsAz9edERkKt09A3nSGBkV70puJ/XjeB4NzLeBQnwCI6XxQkAoPE0zzwgCtGQaSGEKq4uRXTEVGEgkmtZEJwF1/+S1q1qutU3ZtauX6Rx1FEh+gInSAXnaE6ukIN1EQUPaAn9IJerUfr2Xqz3uetBSuf2Ue/YH18A14BlhM=</latexit><latexit sha1_base64="Omx146Fv69pKxCTzcUXdA+GkZwY=">AAACAnicbVBNS8NAEN3Ur1q/op7Ey2IrCEJJelE8Fbx48FDBfkAbyma7aZdusmF3ooRQvPhXvHhQxKu/wpv/xm2bg1YfDDzem2Fmnh8LrsFxvqzC0vLK6lpxvbSxubW9Y+/utbRMFGVNKoVUHZ9oJnjEmsBBsE6sGAl9wdr++HLqt++Y0lxGt5DGzAvJMOIBpwSM1LcPrknKFK5w3FN8OAKilLzH/NSt9O2yU3VmwH+Jm5MyytHo25+9gaRJyCKggmjddZ0YvIwo4FSwSamXaBYTOiZD1jU0IiHTXjZ7YYKPjTLAgVSmIsAz9edERkKt09A3nSGBkV70puJ/XjeB4NzLeBQnwCI6XxQkAoPE0zzwgCtGQaSGEKq4uRXTEVGEgkmtZEJwF1/+S1q1qutU3ZtauX6Rx1FEh+gInSAXnaE6ukIN1EQUPaAn9IJerUfr2Xqz3uetBSuf2Ue/YH18A14BlhM=</latexit><latexit sha1_base64="Omx146Fv69pKxCTzcUXdA+GkZwY=">AAACAnicbVBNS8NAEN3Ur1q/op7Ey2IrCEJJelE8Fbx48FDBfkAbyma7aZdusmF3ooRQvPhXvHhQxKu/wpv/xm2bg1YfDDzem2Fmnh8LrsFxvqzC0vLK6lpxvbSxubW9Y+/utbRMFGVNKoVUHZ9oJnjEmsBBsE6sGAl9wdr++HLqt++Y0lxGt5DGzAvJMOIBpwSM1LcPrknKFK5w3FN8OAKilLzH/NSt9O2yU3VmwH+Jm5MyytHo25+9gaRJyCKggmjddZ0YvIwo4FSwSamXaBYTOiZD1jU0IiHTXjZ7YYKPjTLAgVSmIsAz9edERkKt09A3nSGBkV70puJ/XjeB4NzLeBQnwCI6XxQkAoPE0zzwgCtGQaSGEKq4uRXTEVGEgkmtZEJwF1/+S1q1qutU3ZtauX6Rx1FEh+gInSAXnaE6ukIN1EQUPaAn9IJerUfr2Xqz3uetBSuf2Ue/YH18A14BlhM=</latexit><latexit sha1_base64="Omx146Fv69pKxCTzcUXdA+GkZwY=">AAACAnicbVBNS8NAEN3Ur1q/op7Ey2IrCEJJelE8Fbx48FDBfkAbyma7aZdusmF3ooRQvPhXvHhQxKu/wpv/xm2bg1YfDDzem2Fmnh8LrsFxvqzC0vLK6lpxvbSxubW9Y+/utbRMFGVNKoVUHZ9oJnjEmsBBsE6sGAl9wdr++HLqt++Y0lxGt5DGzAvJMOIBpwSM1LcPrknKFK5w3FN8OAKilLzH/NSt9O2yU3VmwH+Jm5MyytHo25+9gaRJyCKggmjddZ0YvIwo4FSwSamXaBYTOiZD1jU0IiHTXjZ7YYKPjTLAgVSmIsAz9edERkKt09A3nSGBkV70puJ/XjeB4NzLeBQnwCI6XxQkAoPE0zzwgCtGQaSGEKq4uRXTEVGEgkmtZEJwF1/+S1q1qutU3ZtauX6Rx1FEh+gInSAXnaE6ukIN1EQUPaAn9IJerUfr2Xqz3uetBSuf2Ue/YH18A14BlhM=</latexit>
ẑ (i+1) t+1 <latexit sha1_base64="Am4axvpT/RbzZdU3VQKPoUt9hZE=">AAAB/nicbVDLSgNBEJz1GeNrVTx5WQxCJBB2RFA8Bbx4jGAekKxhdjKbDJl9MNMrxGHBX/HiQRGvfoc3/8ZJsgdNLGgoqrrp7vITwRW47re1tLyyurZe2Chubm3v7Np7+00Vp5KyBo1FLNs+UUzwiDWAg2DtRDIS+oK1/NH1xG89MKl4HN3BOGFeSAYRDzglYKSefdgdEtCPWU9DBWf3uswr+DTr2SW36k7hLBKckxLKUe/ZX91+TNOQRUAFUaqD3QQ8TSRwKlhW7KaKJYSOyIB1DI1IyJSnp+dnzolR+k4QS1MROFP194QmoVLj0DedIYGhmvcm4n9eJ4Xg0tM8SlJgEZ0tClLhQOxMsnD6XDIKYmwIoZKbWx06JJJQMIkVTQh4/uVF0jyrYreKb89Ltas8jgI6QseojDC6QDV0g+qogSjS6Bm9ojfryXqx3q2PWeuSlc8coD+wPn8AaO2VEw==</latexit><latexit sha1_base64="Am4axvpT/RbzZdU3VQKPoUt9hZE=">AAAB/nicbVDLSgNBEJz1GeNrVTx5WQxCJBB2RFA8Bbx4jGAekKxhdjKbDJl9MNMrxGHBX/HiQRGvfoc3/8ZJsgdNLGgoqrrp7vITwRW47re1tLyyurZe2Chubm3v7Np7+00Vp5KyBo1FLNs+UUzwiDWAg2DtRDIS+oK1/NH1xG89MKl4HN3BOGFeSAYRDzglYKSefdgdEtCPWU9DBWf3uswr+DTr2SW36k7hLBKckxLKUe/ZX91+TNOQRUAFUaqD3QQ8TSRwKlhW7KaKJYSOyIB1DI1IyJSnp+dnzolR+k4QS1MROFP194QmoVLj0DedIYGhmvcm4n9eJ4Xg0tM8SlJgEZ0tClLhQOxMsnD6XDIKYmwIoZKbWx06JJJQMIkVTQh4/uVF0jyrYreKb89Ltas8jgI6QseojDC6QDV0g+qogSjS6Bm9ojfryXqx3q2PWeuSlc8coD+wPn8AaO2VEw==</latexit><latexit sha1_base64="Am4axvpT/RbzZdU3VQKPoUt9hZE=">AAAB/nicbVDLSgNBEJz1GeNrVTx5WQxCJBB2RFA8Bbx4jGAekKxhdjKbDJl9MNMrxGHBX/HiQRGvfoc3/8ZJsgdNLGgoqrrp7vITwRW47re1tLyyurZe2Chubm3v7Np7+00Vp5KyBo1FLNs+UUzwiDWAg2DtRDIS+oK1/NH1xG89MKl4HN3BOGFeSAYRDzglYKSefdgdEtCPWU9DBWf3uswr+DTr2SW36k7hLBKckxLKUe/ZX91+TNOQRUAFUaqD3QQ8TSRwKlhW7KaKJYSOyIB1DI1IyJSnp+dnzolR+k4QS1MROFP194QmoVLj0DedIYGhmvcm4n9eJ4Xg0tM8SlJgEZ0tClLhQOxMsnD6XDIKYmwIoZKbWx06JJJQMIkVTQh4/uVF0jyrYreKb89Ltas8jgI6QseojDC6QDV0g+qogSjS6Bm9ojfryXqx3q2PWeuSlc8coD+wPn8AaO2VEw==</latexit><latexit sha1_base64="Am4axvpT/RbzZdU3VQKPoUt9hZE=">AAAB/nicbVDLSgNBEJz1GeNrVTx5WQxCJBB2RFA8Bbx4jGAekKxhdjKbDJl9MNMrxGHBX/HiQRGvfoc3/8ZJsgdNLGgoqrrp7vITwRW47re1tLyyurZe2Chubm3v7Np7+00Vp5KyBo1FLNs+UUzwiDWAg2DtRDIS+oK1/NH1xG89MKl4HN3BOGFeSAYRDzglYKSefdgdEtCPWU9DBWf3uswr+DTr2SW36k7hLBKckxLKUe/ZX91+TNOQRUAFUaqD3QQ8TSRwKlhW7KaKJYSOyIB1DI1IyJSnp+dnzolR+k4QS1MROFP194QmoVLj0DedIYGhmvcm4n9eJ4Xg0tM8SlJgEZ0tClLhQOxMsnD6XDIKYmwIoZKbWx06JJJQMIkVTQh4/uVF0jyrYreKb89Ltas8jgI6QseojDC6QDV0g+qogSjS6Bm9ojfryXqx3q2PWeuSlc8coD+wPn8AaO2VEw==</latexit>
z (i) t <latexit sha1_base64="2svhF9CpH3GHMAPuUCL8JxYJANA=">AAAB8nicbVBNS8NAEN34WetX1aOXxSLUS0lEUDwVvHisYD8gjWWz3bRLN7thdyLUkJ/hxYMiXv013vw3btsctPXBwOO9GWbmhYngBlz321lZXVvf2Cxtlbd3dvf2KweHbaNSTVmLKqF0NySGCS5ZCzgI1k00I3EoWCcc30z9ziPThit5D5OEBTEZSh5xSsBK/lM/g/whq/GzvF+punV3BrxMvIJUUYFmv/LVGyiaxkwCFcQY33MTCDKigVPB8nIvNSwhdEyGzLdUkpiZIJudnONTqwxwpLQtCXim/p7ISGzMJA5tZ0xgZBa9qfif56cQXQUZl0kKTNL5oigVGBSe/o8HXDMKYmIJoZrbWzEdEU0o2JTKNgRv8eVl0j6ve27du7uoNq6LOEroGJ2gGvLQJWqgW9RELUSRQs/oFb054Lw4787HvHXFKWaO0B84nz9HwZE1</latexit><latexit sha1_base64="2svhF9CpH3GHMAPuUCL8JxYJANA=">AAAB8nicbVBNS8NAEN34WetX1aOXxSLUS0lEUDwVvHisYD8gjWWz3bRLN7thdyLUkJ/hxYMiXv013vw3btsctPXBwOO9GWbmhYngBlz321lZXVvf2Cxtlbd3dvf2KweHbaNSTVmLKqF0NySGCS5ZCzgI1k00I3EoWCcc30z9ziPThit5D5OEBTEZSh5xSsBK/lM/g/whq/GzvF+punV3BrxMvIJUUYFmv/LVGyiaxkwCFcQY33MTCDKigVPB8nIvNSwhdEyGzLdUkpiZIJudnONTqwxwpLQtCXim/p7ISGzMJA5tZ0xgZBa9qfif56cQXQUZl0kKTNL5oigVGBSe/o8HXDMKYmIJoZrbWzEdEU0o2JTKNgRv8eVl0j6ve27du7uoNq6LOEroGJ2gGvLQJWqgW9RELUSRQs/oFb054Lw4787HvHXFKWaO0B84nz9HwZE1</latexit><latexit sha1_base64="2svhF9CpH3GHMAPuUCL8JxYJANA=">AAAB8nicbVBNS8NAEN34WetX1aOXxSLUS0lEUDwVvHisYD8gjWWz3bRLN7thdyLUkJ/hxYMiXv013vw3btsctPXBwOO9GWbmhYngBlz321lZXVvf2Cxtlbd3dvf2KweHbaNSTVmLKqF0NySGCS5ZCzgI1k00I3EoWCcc30z9ziPThit5D5OEBTEZSh5xSsBK/lM/g/whq/GzvF+punV3BrxMvIJUUYFmv/LVGyiaxkwCFcQY33MTCDKigVPB8nIvNSwhdEyGzLdUkpiZIJudnONTqwxwpLQtCXim/p7ISGzMJA5tZ0xgZBa9qfif56cQXQUZl0kKTNL5oigVGBSe/o8HXDMKYmIJoZrbWzEdEU0o2JTKNgRv8eVl0j6ve27du7uoNq6LOEroGJ2gGvLQJWqgW9RELUSRQs/oFb054Lw4787HvHXFKWaO0B84nz9HwZE1</latexit><latexit sha1_base64="2svhF9CpH3GHMAPuUCL8JxYJANA=">AAAB8nicbVBNS8NAEN34WetX1aOXxSLUS0lEUDwVvHisYD8gjWWz3bRLN7thdyLUkJ/hxYMiXv013vw3btsctPXBwOO9GWbmhYngBlz321lZXVvf2Cxtlbd3dvf2KweHbaNSTVmLKqF0NySGCS5ZCzgI1k00I3EoWCcc30z9ziPThit5D5OEBTEZSh5xSsBK/lM/g/whq/GzvF+punV3BrxMvIJUUYFmv/LVGyiaxkwCFcQY33MTCDKigVPB8nIvNSwhdEyGzLdUkpiZIJudnONTqwxwpLQtCXim/p7ISGzMJA5tZ0xgZBa9qfif56cQXQUZl0kKTNL5oigVGBSe/o8HXDMKYmIJoZrbWzEdEU0o2JTKNgRv8eVl0j6ve27du7uoNq6LOEroGJ2gGvLQJWqgW9RELUSRQs/oFb054Lw4787HvHXFKWaO0B84nz9HwZE1</latexit>
z (i) t+1 <latexit sha1_base64="LuZCTtj0QRFI4tTxozbliK7Tzjg=">AAAB9HicbVDLSgNBEOyNrxhfUY9eBoMQEcKuCIqngBePEcwDkjXMTibJkNmHM72BuOx3ePGgiFc/xpt/4yTZgyYWNBRV3XR3eZEUGm3728qtrK6tb+Q3C1vbO7t7xf2Dhg5jxXidhTJULY9qLkXA6yhQ8lakOPU9yZve6GbqN8dcaREG9ziJuOvTQSD6glE0kvvUTfDMSR+SsjhNu8WSXbFnIMvEyUgJMtS6xa9OL2SxzwNkkmrdduwI3YQqFEzytNCJNY8oG9EBbxsaUJ9rN5kdnZITo/RIP1SmAiQz9fdEQn2tJ75nOn2KQ73oTcX/vHaM/Ss3EUEUIw/YfFE/lgRDMk2A9ITiDOXEEMqUMLcSNqSKMjQ5FUwIzuLLy6RxXnHsinN3UapeZ3Hk4QiOoQwOXEIVbqEGdWDwCM/wCm/W2Hqx3q2PeWvOymYO4Q+szx8jcZGl</latexit><latexit sha1_base64="LuZCTtj0QRFI4tTxozbliK7Tzjg=">AAAB9HicbVDLSgNBEOyNrxhfUY9eBoMQEcKuCIqngBePEcwDkjXMTibJkNmHM72BuOx3ePGgiFc/xpt/4yTZgyYWNBRV3XR3eZEUGm3728qtrK6tb+Q3C1vbO7t7xf2Dhg5jxXidhTJULY9qLkXA6yhQ8lakOPU9yZve6GbqN8dcaREG9ziJuOvTQSD6glE0kvvUTfDMSR+SsjhNu8WSXbFnIMvEyUgJMtS6xa9OL2SxzwNkkmrdduwI3YQqFEzytNCJNY8oG9EBbxsaUJ9rN5kdnZITo/RIP1SmAiQz9fdEQn2tJ75nOn2KQ73oTcX/vHaM/Ss3EUEUIw/YfFE/lgRDMk2A9ITiDOXEEMqUMLcSNqSKMjQ5FUwIzuLLy6RxXnHsinN3UapeZ3Hk4QiOoQwOXEIVbqEGdWDwCM/wCm/W2Hqx3q2PeWvOymYO4Q+szx8jcZGl</latexit><latexit sha1_base64="LuZCTtj0QRFI4tTxozbliK7Tzjg=">AAAB9HicbVDLSgNBEOyNrxhfUY9eBoMQEcKuCIqngBePEcwDkjXMTibJkNmHM72BuOx3ePGgiFc/xpt/4yTZgyYWNBRV3XR3eZEUGm3728qtrK6tb+Q3C1vbO7t7xf2Dhg5jxXidhTJULY9qLkXA6yhQ8lakOPU9yZve6GbqN8dcaREG9ziJuOvTQSD6glE0kvvUTfDMSR+SsjhNu8WSXbFnIMvEyUgJMtS6xa9OL2SxzwNkkmrdduwI3YQqFEzytNCJNY8oG9EBbxsaUJ9rN5kdnZITo/RIP1SmAiQz9fdEQn2tJ75nOn2KQ73oTcX/vHaM/Ss3EUEUIw/YfFE/lgRDMk2A9ITiDOXEEMqUMLcSNqSKMjQ5FUwIzuLLy6RxXnHsinN3UapeZ3Hk4QiOoQwOXEIVbqEGdWDwCM/wCm/W2Hqx3q2PeWvOymYO4Q+szx8jcZGl</latexit><latexit sha1_base64="LuZCTtj0QRFI4tTxozbliK7Tzjg=">AAAB9HicbVDLSgNBEOyNrxhfUY9eBoMQEcKuCIqngBePEcwDkjXMTibJkNmHM72BuOx3ePGgiFc/xpt/4yTZgyYWNBRV3XR3eZEUGm3728qtrK6tb+Q3C1vbO7t7xf2Dhg5jxXidhTJULY9qLkXA6yhQ8lakOPU9yZve6GbqN8dcaREG9ziJuOvTQSD6glE0kvvUTfDMSR+SsjhNu8WSXbFnIMvEyUgJMtS6xa9OL2SxzwNkkmrdduwI3YQqFEzytNCJNY8oG9EBbxsaUJ9rN5kdnZITo/RIP1SmAiQz9fdEQn2tJ75nOn2KQ73oTcX/vHaM/Ss3EUEUIw/YfFE/lgRDMk2A9ITiDOXEEMqUMLcSNqSKMjQ5FUwIzuLLy6RxXnHsinN3UapeZ3Hk4QiOoQwOXEIVbqEGdWDwCM/wCm/W2Hqx3q2PeWvOymYO4Q+szx8jcZGl</latexit>
z (i+1) t+1 <latexit sha1_base64="+VtluC+UvZ7AXfnV6T+0WUjJZNw=">AAAB+HicbVDLSgNBEOz1GeMjqx69DAYhEgg7IiieAl48RjAPSNZldjJJhsw+mJkVkmW/xIsHRbz6Kd78GyfJHjSxoKGo6qa7y48FV9pxvq219Y3Nre3CTnF3b/+gZB8etVSUSMqaNBKR7PhEMcFD1tRcC9aJJSOBL1jbH9/O/PYTk4pH4YOexMwNyDDkA06JNpJnl6Zeqqs4e0wrvIrPM88uOzVnDrRKcE7KkKPh2V+9fkSTgIWaCqJUFzuxdlMiNaeCZcVeolhM6JgMWdfQkARMuen88AydGaWPBpE0FWo0V39PpCRQahL4pjMgeqSWvZn4n9dN9ODaTXkYJ5qFdLFokAikIzRLAfW5ZFSLiSGESm5uRXREJKHaZFU0IeDll1dJ66KGnRq+vyzXb/I4CnACp1ABDFdQhztoQBMoJPAMr/BmTa0X6936WLSuWfnMMfyB9fkDeZiSRg==</latexit><latexit sha1_base64="+VtluC+UvZ7AXfnV6T+0WUjJZNw=">AAAB+HicbVDLSgNBEOz1GeMjqx69DAYhEgg7IiieAl48RjAPSNZldjJJhsw+mJkVkmW/xIsHRbz6Kd78GyfJHjSxoKGo6qa7y48FV9pxvq219Y3Nre3CTnF3b/+gZB8etVSUSMqaNBKR7PhEMcFD1tRcC9aJJSOBL1jbH9/O/PYTk4pH4YOexMwNyDDkA06JNpJnl6Zeqqs4e0wrvIrPM88uOzVnDrRKcE7KkKPh2V+9fkSTgIWaCqJUFzuxdlMiNaeCZcVeolhM6JgMWdfQkARMuen88AydGaWPBpE0FWo0V39PpCRQahL4pjMgeqSWvZn4n9dN9ODaTXkYJ5qFdLFokAikIzRLAfW5ZFSLiSGESm5uRXREJKHaZFU0IeDll1dJ66KGnRq+vyzXb/I4CnACp1ABDFdQhztoQBMoJPAMr/BmTa0X6936WLSuWfnMMfyB9fkDeZiSRg==</latexit><latexit sha1_base64="+VtluC+UvZ7AXfnV6T+0WUjJZNw=">AAAB+HicbVDLSgNBEOz1GeMjqx69DAYhEgg7IiieAl48RjAPSNZldjJJhsw+mJkVkmW/xIsHRbz6Kd78GyfJHjSxoKGo6qa7y48FV9pxvq219Y3Nre3CTnF3b/+gZB8etVSUSMqaNBKR7PhEMcFD1tRcC9aJJSOBL1jbH9/O/PYTk4pH4YOexMwNyDDkA06JNpJnl6Zeqqs4e0wrvIrPM88uOzVnDrRKcE7KkKPh2V+9fkSTgIWaCqJUFzuxdlMiNaeCZcVeolhM6JgMWdfQkARMuen88AydGaWPBpE0FWo0V39PpCRQahL4pjMgeqSWvZn4n9dN9ODaTXkYJ5qFdLFokAikIzRLAfW5ZFSLiSGESm5uRXREJKHaZFU0IeDll1dJ66KGnRq+vyzXb/I4CnACp1ABDFdQhztoQBMoJPAMr/BmTa0X6936WLSuWfnMMfyB9fkDeZiSRg==</latexit><latexit sha1_base64="+VtluC+UvZ7AXfnV6T+0WUjJZNw=">AAAB+HicbVDLSgNBEOz1GeMjqx69DAYhEgg7IiieAl48RjAPSNZldjJJhsw+mJkVkmW/xIsHRbz6Kd78GyfJHjSxoKGo6qa7y48FV9pxvq219Y3Nre3CTnF3b/+gZB8etVSUSMqaNBKR7PhEMcFD1tRcC9aJJSOBL1jbH9/O/PYTk4pH4YOexMwNyDDkA06JNpJnl6Zeqqs4e0wrvIrPM88uOzVnDrRKcE7KkKPh2V+9fkSTgIWaCqJUFzuxdlMiNaeCZcVeolhM6JgMWdfQkARMuen88AydGaWPBpE0FWo0V39PpCRQahL4pjMgeqSWvZn4n9dN9ODaTXkYJ5qFdLFokAikIzRLAfW5ZFSLiSGESm5uRXREJKHaZFU0IeDll1dJ66KGnRq+vyzXb/I4CnACp1ABDFdQhztoQBMoJPAMr/BmTa0X6936WLSuWfnMMfyB9fkDeZiSRg==</latexit>
z (i) t+1,1 <latexit sha1_base64="3QxhVQRV4MPfAQCGJeDKkKtO4Q8=">AAAB+HicbVDLSsNAFL2pr1ofjbp0M1iEilISEdRdwY3LCvYBbQyT6aQdOpmEmYnQhnyJGxeKuPVT3Pk3Th8LrR64cDjnXu69J0g4U9pxvqzCyura+kZxs7S1vbNbtvf2WypOJaFNEvNYdgKsKGeCNjXTnHYSSXEUcNoORjdTv/1IpWKxuNfjhHoRHggWMoK1kXy7PPEzfeqeuflDVmUnuW9XnJozA/pL3AWpwAIN3/7s9WOSRlRowrFSXddJtJdhqRnhNC/1UkUTTEZ4QLuGChxR5WWzw3N0bJQ+CmNpSmg0U39OZDhSahwFpjPCeqiWvan4n9dNdXjlZUwkqaaCzBeFKUc6RtMUUJ9JSjQfG4KJZOZWRIZYYqJNViUTgrv88l/SOq+5Ts29u6jUrxdxFOEQjqAKLlxCHW6hAU0gkMITvMCrNbGerTfrfd5asBYzB/AL1sc3eNWSRg==</latexit><latexit sha1_base64="3QxhVQRV4MPfAQCGJeDKkKtO4Q8=">AAAB+HicbVDLSsNAFL2pr1ofjbp0M1iEilISEdRdwY3LCvYBbQyT6aQdOpmEmYnQhnyJGxeKuPVT3Pk3Th8LrR64cDjnXu69J0g4U9pxvqzCyura+kZxs7S1vbNbtvf2WypOJaFNEvNYdgKsKGeCNjXTnHYSSXEUcNoORjdTv/1IpWKxuNfjhHoRHggWMoK1kXy7PPEzfeqeuflDVmUnuW9XnJozA/pL3AWpwAIN3/7s9WOSRlRowrFSXddJtJdhqRnhNC/1UkUTTEZ4QLuGChxR5WWzw3N0bJQ+CmNpSmg0U39OZDhSahwFpjPCeqiWvan4n9dNdXjlZUwkqaaCzBeFKUc6RtMUUJ9JSjQfG4KJZOZWRIZYYqJNViUTgrv88l/SOq+5Ts29u6jUrxdxFOEQjqAKLlxCHW6hAU0gkMITvMCrNbGerTfrfd5asBYzB/AL1sc3eNWSRg==</latexit><latexit sha1_base64="3QxhVQRV4MPfAQCGJeDKkKtO4Q8=">AAAB+HicbVDLSsNAFL2pr1ofjbp0M1iEilISEdRdwY3LCvYBbQyT6aQdOpmEmYnQhnyJGxeKuPVT3Pk3Th8LrR64cDjnXu69J0g4U9pxvqzCyura+kZxs7S1vbNbtvf2WypOJaFNEvNYdgKsKGeCNjXTnHYSSXEUcNoORjdTv/1IpWKxuNfjhHoRHggWMoK1kXy7PPEzfeqeuflDVmUnuW9XnJozA/pL3AWpwAIN3/7s9WOSRlRowrFSXddJtJdhqRnhNC/1UkUTTEZ4QLuGChxR5WWzw3N0bJQ+CmNpSmg0U39OZDhSahwFpjPCeqiWvan4n9dNdXjlZUwkqaaCzBeFKUc6RtMUUJ9JSjQfG4KJZOZWRIZYYqJNViUTgrv88l/SOq+5Ts29u6jUrxdxFOEQjqAKLlxCHW6hAU0gkMITvMCrNbGerTfrfd5asBYzB/AL1sc3eNWSRg==</latexit><latexit sha1_base64="3QxhVQRV4MPfAQCGJeDKkKtO4Q8=">AAAB+HicbVDLSsNAFL2pr1ofjbp0M1iEilISEdRdwY3LCvYBbQyT6aQdOpmEmYnQhnyJGxeKuPVT3Pk3Th8LrR64cDjnXu69J0g4U9pxvqzCyura+kZxs7S1vbNbtvf2WypOJaFNEvNYdgKsKGeCNjXTnHYSSXEUcNoORjdTv/1IpWKxuNfjhHoRHggWMoK1kXy7PPEzfeqeuflDVmUnuW9XnJozA/pL3AWpwAIN3/7s9WOSRlRowrFSXddJtJdhqRnhNC/1UkUTTEZ4QLuGChxR5WWzw3N0bJQ+CmNpSmg0U39OZDhSahwFpjPCeqiWvan4n9dNdXjlZUwkqaaCzBeFKUc6RtMUUJ9JSjQfG4KJZOZWRIZYYqJNViUTgrv88l/SOq+5Ts29u6jUrxdxFOEQjqAKLlxCHW6hAU0gkMITvMCrNbGerTfrfd5asBYzB/AL1sc3eNWSRg==</latexit> z (i) t+1,2 <latexit sha1_base64="jJAvqfFQwy/CDu8cCn5IwVKXhnU=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSlJEdRdwY3LCvYBbQyT6aQdOnkwcyO0IV/ixoUibv0Ud/6N0zYLbT1w4XDOvdx7jxcLrsCyvo3C2vrG5lZxu7Szu7dfNg8O2ypKJGUtGolIdj2imOAhawEHwbqxZCTwBOt449uZ33liUvEofIBJzJyADEPuc0pAS65ZnropnNsX9ewxrfKzzDUrVs2aA68SOycVlKPpml/9QUSTgIVABVGqZ1sxOCmRwKlgWamfKBYTOiZD1tM0JAFTTjo/PMOnWhlgP5K6QsBz9fdESgKlJoGnOwMCI7XszcT/vF4C/rWT8jBOgIV0schPBIYIz1LAAy4ZBTHRhFDJ9a2YjogkFHRWJR2CvfzyKmnXa7ZVs+8vK42bPI4iOkYnqIpsdIUa6A41UQtRlKBn9IrejKnxYrwbH4vWgpHPHKE/MD5/AHpgkkc=</latexit><latexit sha1_base64="jJAvqfFQwy/CDu8cCn5IwVKXhnU=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSlJEdRdwY3LCvYBbQyT6aQdOnkwcyO0IV/ixoUibv0Ud/6N0zYLbT1w4XDOvdx7jxcLrsCyvo3C2vrG5lZxu7Szu7dfNg8O2ypKJGUtGolIdj2imOAhawEHwbqxZCTwBOt449uZ33liUvEofIBJzJyADEPuc0pAS65ZnropnNsX9ewxrfKzzDUrVs2aA68SOycVlKPpml/9QUSTgIVABVGqZ1sxOCmRwKlgWamfKBYTOiZD1tM0JAFTTjo/PMOnWhlgP5K6QsBz9fdESgKlJoGnOwMCI7XszcT/vF4C/rWT8jBOgIV0schPBIYIz1LAAy4ZBTHRhFDJ9a2YjogkFHRWJR2CvfzyKmnXa7ZVs+8vK42bPI4iOkYnqIpsdIUa6A41UQtRlKBn9IrejKnxYrwbH4vWgpHPHKE/MD5/AHpgkkc=</latexit><latexit sha1_base64="jJAvqfFQwy/CDu8cCn5IwVKXhnU=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSlJEdRdwY3LCvYBbQyT6aQdOnkwcyO0IV/ixoUibv0Ud/6N0zYLbT1w4XDOvdx7jxcLrsCyvo3C2vrG5lZxu7Szu7dfNg8O2ypKJGUtGolIdj2imOAhawEHwbqxZCTwBOt449uZ33liUvEofIBJzJyADEPuc0pAS65ZnropnNsX9ewxrfKzzDUrVs2aA68SOycVlKPpml/9QUSTgIVABVGqZ1sxOCmRwKlgWamfKBYTOiZD1tM0JAFTTjo/PMOnWhlgP5K6QsBz9fdESgKlJoGnOwMCI7XszcT/vF4C/rWT8jBOgIV0schPBIYIz1LAAy4ZBTHRhFDJ9a2YjogkFHRWJR2CvfzyKmnXa7ZVs+8vK42bPI4iOkYnqIpsdIUa6A41UQtRlKBn9IrejKnxYrwbH4vWgpHPHKE/MD5/AHpgkkc=</latexit><latexit sha1_base64="jJAvqfFQwy/CDu8cCn5IwVKXhnU=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSlJEdRdwY3LCvYBbQyT6aQdOnkwcyO0IV/ixoUibv0Ud/6N0zYLbT1w4XDOvdx7jxcLrsCyvo3C2vrG5lZxu7Szu7dfNg8O2ypKJGUtGolIdj2imOAhawEHwbqxZCTwBOt449uZ33liUvEofIBJzJyADEPuc0pAS65ZnropnNsX9ewxrfKzzDUrVs2aA68SOycVlKPpml/9QUSTgIVABVGqZ1sxOCmRwKlgWamfKBYTOiZD1tM0JAFTTjo/PMOnWhlgP5K6QsBz9fdESgKlJoGnOwMCI7XszcT/vF4C/rWT8jBOgIV0schPBIYIz1LAAy4ZBTHRhFDJ9a2YjogkFHRWJR2CvfzyKmnXa7ZVs+8vK42bPI4iOkYnqIpsdIUa6A41UQtRlKBn9IrejKnxYrwbH4vWgpHPHKE/MD5/AHpgkkc=</latexit> z (i) t,2 <latexit sha1_base64="CM90o1WRrsRKWE4t3BC62PomYXs=">AAAB9HicbVDLSgNBEOz1GeMr6tHLYBAiSNgNgnoLePEYwTwgWcPsZDYZMvtwpjcQl3yHFw+KePVjvPk3TpI9aGJBQ1HVTXeXF0uh0ba/rZXVtfWNzdxWfntnd2+/cHDY0FGiGK+zSEaq5VHNpQh5HQVK3ooVp4EnedMb3kz95ogrLaLwHscxdwPaD4UvGEUjuU/dFM8rk4e0JM4m3ULRLtszkGXiZKQIGWrdwlenF7Ek4CEySbVuO3aMbkoVCib5JN9JNI8pG9I+bxsa0oBrN50dPSGnRukRP1KmQiQz9fdESgOtx4FnOgOKA73oTcX/vHaC/pWbijBOkIdsvshPJMGITBMgPaE4Qzk2hDIlzK2EDaiiDE1OeROCs/jyMmlUyo5ddu4uitXrLI4cHMMJlMCBS6jCLdSgDgwe4Rle4c0aWS/Wu/Uxb12xspkj+APr8wcmO5Gm</latexit><latexit sha1_base64="CM90o1WRrsRKWE4t3BC62PomYXs=">AAAB9HicbVDLSgNBEOz1GeMr6tHLYBAiSNgNgnoLePEYwTwgWcPsZDYZMvtwpjcQl3yHFw+KePVjvPk3TpI9aGJBQ1HVTXeXF0uh0ba/rZXVtfWNzdxWfntnd2+/cHDY0FGiGK+zSEaq5VHNpQh5HQVK3ooVp4EnedMb3kz95ogrLaLwHscxdwPaD4UvGEUjuU/dFM8rk4e0JM4m3ULRLtszkGXiZKQIGWrdwlenF7Ek4CEySbVuO3aMbkoVCib5JN9JNI8pG9I+bxsa0oBrN50dPSGnRukRP1KmQiQz9fdESgOtx4FnOgOKA73oTcX/vHaC/pWbijBOkIdsvshPJMGITBMgPaE4Qzk2hDIlzK2EDaiiDE1OeROCs/jyMmlUyo5ddu4uitXrLI4cHMMJlMCBS6jCLdSgDgwe4Rle4c0aWS/Wu/Uxb12xspkj+APr8wcmO5Gm</latexit><latexit sha1_base64="CM90o1WRrsRKWE4t3BC62PomYXs=">AAAB9HicbVDLSgNBEOz1GeMr6tHLYBAiSNgNgnoLePEYwTwgWcPsZDYZMvtwpjcQl3yHFw+KePVjvPk3TpI9aGJBQ1HVTXeXF0uh0ba/rZXVtfWNzdxWfntnd2+/cHDY0FGiGK+zSEaq5VHNpQh5HQVK3ooVp4EnedMb3kz95ogrLaLwHscxdwPaD4UvGEUjuU/dFM8rk4e0JM4m3ULRLtszkGXiZKQIGWrdwlenF7Ek4CEySbVuO3aMbkoVCib5JN9JNI8pG9I+bxsa0oBrN50dPSGnRukRP1KmQiQz9fdESgOtx4FnOgOKA73oTcX/vHaC/pWbijBOkIdsvshPJMGITBMgPaE4Qzk2hDIlzK2EDaiiDE1OeROCs/jyMmlUyo5ddu4uitXrLI4cHMMJlMCBS6jCLdSgDgwe4Rle4c0aWS/Wu/Uxb12xspkj+APr8wcmO5Gm</latexit><latexit sha1_base64="CM90o1WRrsRKWE4t3BC62PomYXs=">AAAB9HicbVDLSgNBEOz1GeMr6tHLYBAiSNgNgnoLePEYwTwgWcPsZDYZMvtwpjcQl3yHFw+KePVjvPk3TpI9aGJBQ1HVTXeXF0uh0ba/rZXVtfWNzdxWfntnd2+/cHDY0FGiGK+zSEaq5VHNpQh5HQVK3ooVp4EnedMb3kz95ogrLaLwHscxdwPaD4UvGEUjuU/dFM8rk4e0JM4m3ULRLtszkGXiZKQIGWrdwlenF7Ek4CEySbVuO3aMbkoVCib5JN9JNI8pG9I+bxsa0oBrN50dPSGnRukRP1KmQiQz9fdESgOtx4FnOgOKA73oTcX/vHaC/pWbijBOkIdsvshPJMGITBMgPaE4Qzk2hDIlzK2EDaiiDE1OeROCs/jyMmlUyo5ddu4uitXrLI4cHMMJlMCBS6jCLdSgDgwe4Rle4c0aWS/Wu/Uxb12xspkj+APr8wcmO5Gm</latexit> z (i) t,1
<latexit sha1_base64="hEqsC+awpPdZyeIQrbA/Py+DhL4=">AAAB9HicbVDLSgNBEOyNrxhfUY9eBoMQQcKuCOot4MVjBPOAZA2zk0kyZPbhTG8gLvsdXjwo4tWP8ebfOEn2oIkFDUVVN91dXiSFRtv+tnIrq2vrG/nNwtb2zu5ecf+gocNYMV5noQxVy6OaSxHwOgqUvBUpTn1P8qY3upn6zTFXWoTBPU4i7vp0EIi+YBSN5D51Ezxz0oekLE7TbrFkV+wZyDJxMlKCDLVu8avTC1ns8wCZpFq3HTtCN6EKBZM8LXRizSPKRnTA24YG1OfaTWZHp+TEKD3SD5WpAMlM/T2RUF/rie+ZTp/iUC96U/E/rx1j/8pNRBDFyAM2X9SPJcGQTBMgPaE4QzkxhDIlzK2EDamiDE1OBROCs/jyMmmcVxy74txdlKrXWRx5OIJjKIMDl1CFW6hBHRg8wjO8wps1tl6sd+tj3pqzsplD+APr8wcksJGl</latexit><latexit sha1_base64="hEqsC+awpPdZyeIQrbA/Py+DhL4=">AAAB9HicbVDLSgNBEOyNrxhfUY9eBoMQQcKuCOot4MVjBPOAZA2zk0kyZPbhTG8gLvsdXjwo4tWP8ebfOEn2oIkFDUVVN91dXiSFRtv+tnIrq2vrG/nNwtb2zu5ecf+gocNYMV5noQxVy6OaSxHwOgqUvBUpTn1P8qY3upn6zTFXWoTBPU4i7vp0EIi+YBSN5D51Ezxz0oekLE7TbrFkV+wZyDJxMlKCDLVu8avTC1ns8wCZpFq3HTtCN6EKBZM8LXRizSPKRnTA24YG1OfaTWZHp+TEKD3SD5WpAMlM/T2RUF/rie+ZTp/iUC96U/E/rx1j/8pNRBDFyAM2X9SPJcGQTBMgPaE4QzkxhDIlzK2EDamiDE1OBROCs/jyMmmcVxy74txdlKrXWRx5OIJjKIMDl1CFW6hBHRg8wjO8wps1tl6sd+tj3pqzsplD+APr8wcksJGl</latexit><latexit sha1_base64="hEqsC+awpPdZyeIQrbA/Py+DhL4=">AAAB9HicbVDLSgNBEOyNrxhfUY9eBoMQQcKuCOot4MVjBPOAZA2zk0kyZPbhTG8gLvsdXjwo4tWP8ebfOEn2oIkFDUVVN91dXiSFRtv+tnIrq2vrG/nNwtb2zu5ecf+gocNYMV5noQxVy6OaSxHwOgqUvBUpTn1P8qY3upn6zTFXWoTBPU4i7vp0EIi+YBSN5D51Ezxz0oekLE7TbrFkV+wZyDJxMlKCDLVu8avTC1ns8wCZpFq3HTtCN6EKBZM8LXRizSPKRnTA24YG1OfaTWZHp+TEKD3SD5WpAMlM/T2RUF/rie+ZTp/iUC96U/E/rx1j/8pNRBDFyAM2X9SPJcGQTBMgPaE4QzkxhDIlzK2EDamiDE1OBROCs/jyMmmcVxy74txdlKrXWRx5OIJjKIMDl1CFW6hBHRg8wjO8wps1tl6sd+tj3pqzsplD+APr8wcksJGl</latexit><latexit sha1_base64="hEqsC+awpPdZyeIQrbA/Py+DhL4=">AAAB9HicbVDLSgNBEOyNrxhfUY9eBoMQQcKuCOot4MVjBPOAZA2zk0kyZPbhTG8gLvsdXjwo4tWP8ebfOEn2oIkFDUVVN91dXiSFRtv+tnIrq2vrG/nNwtb2zu5ecf+gocNYMV5noQxVy6OaSxHwOgqUvBUpTn1P8qY3upn6zTFXWoTBPU4i7vp0EIi+YBSN5D51Ezxz0oekLE7TbrFkV+wZyDJxMlKCDLVu8avTC1ns8wCZpFq3HTtCN6EKBZM8LXRizSPKRnTA24YG1OfaTWZHp+TEKD3SD5WpAMlM/T2RUF/rie+ZTp/iUC96U/E/rx1j/8pNRBDFyAM2X9SPJcGQTBMgPaE4QzkxhDIlzK2EDamiDE1OBROCs/jyMmmcVxy74txdlKrXWRx5OIJjKIMDl1CFW6hBHRg8wjO8wps1tl6sd+tj3pqzsplD+APr8wcksJGl</latexit>
z (i+1) t+1,1 <latexit sha1_base64="HwsUkBgByuOgfWaANPnhipbeukE=">AAAB+nicbVDLSsNAFJ34rPWV6tLNYBEqlZIRQd0V3LisYB/QxjCZTtqhk0mYmSg15lPcuFDErV/izr9x2mahrQcuHM65l3vv8WPOlHacb2tpeWV1bb2wUdzc2t7ZtUt7LRUlktAmiXgkOz5WlDNBm5ppTjuxpDj0OW37o6uJ376nUrFI3OpxTN0QDwQLGMHaSJ5devRSXUUnKLtLK6yKjjPPLjs1Zwq4SFBOyiBHw7O/ev2IJCEVmnCsVBc5sXZTLDUjnGbFXqJojMkID2jXUIFDqtx0enoGj4zSh0EkTQkNp+rviRSHSo1D33SGWA/VvDcR//O6iQ4u3JSJONFUkNmiIOFQR3CSA+wzSYnmY0MwkczcCskQS0y0SatoQkDzLy+S1mkNOTV0c1auX+ZxFMABOAQVgMA5qINr0ABNQMADeAav4M16sl6sd+tj1rpk5TP74A+szx9YR5K2</latexit><latexit sha1_base64="HwsUkBgByuOgfWaANPnhipbeukE=">AAAB+nicbVDLSsNAFJ34rPWV6tLNYBEqlZIRQd0V3LisYB/QxjCZTtqhk0mYmSg15lPcuFDErV/izr9x2mahrQcuHM65l3vv8WPOlHacb2tpeWV1bb2wUdzc2t7ZtUt7LRUlktAmiXgkOz5WlDNBm5ppTjuxpDj0OW37o6uJ376nUrFI3OpxTN0QDwQLGMHaSJ5devRSXUUnKLtLK6yKjjPPLjs1Zwq4SFBOyiBHw7O/ev2IJCEVmnCsVBc5sXZTLDUjnGbFXqJojMkID2jXUIFDqtx0enoGj4zSh0EkTQkNp+rviRSHSo1D33SGWA/VvDcR//O6iQ4u3JSJONFUkNmiIOFQR3CSA+wzSYnmY0MwkczcCskQS0y0SatoQkDzLy+S1mkNOTV0c1auX+ZxFMABOAQVgMA5qINr0ABNQMADeAav4M16sl6sd+tj1rpk5TP74A+szx9YR5K2</latexit><latexit sha1_base64="HwsUkBgByuOgfWaANPnhipbeukE=">AAAB+nicbVDLSsNAFJ34rPWV6tLNYBEqlZIRQd0V3LisYB/QxjCZTtqhk0mYmSg15lPcuFDErV/izr9x2mahrQcuHM65l3vv8WPOlHacb2tpeWV1bb2wUdzc2t7ZtUt7LRUlktAmiXgkOz5WlDNBm5ppTjuxpDj0OW37o6uJ376nUrFI3OpxTN0QDwQLGMHaSJ5devRSXUUnKLtLK6yKjjPPLjs1Zwq4SFBOyiBHw7O/ev2IJCEVmnCsVBc5sXZTLDUjnGbFXqJojMkID2jXUIFDqtx0enoGj4zSh0EkTQkNp+rviRSHSo1D33SGWA/VvDcR//O6iQ4u3JSJONFUkNmiIOFQR3CSA+wzSYnmY0MwkczcCskQS0y0SatoQkDzLy+S1mkNOTV0c1auX+ZxFMABOAQVgMA5qINr0ABNQMADeAav4M16sl6sd+tj1rpk5TP74A+szx9YR5K2</latexit><latexit sha1_base64="HwsUkBgByuOgfWaANPnhipbeukE=">AAAB+nicbVDLSsNAFJ34rPWV6tLNYBEqlZIRQd0V3LisYB/QxjCZTtqhk0mYmSg15lPcuFDErV/izr9x2mahrQcuHM65l3vv8WPOlHacb2tpeWV1bb2wUdzc2t7ZtUt7LRUlktAmiXgkOz5WlDNBm5ppTjuxpDj0OW37o6uJ376nUrFI3OpxTN0QDwQLGMHaSJ5devRSXUUnKLtLK6yKjjPPLjs1Zwq4SFBOyiBHw7O/ev2IJCEVmnCsVBc5sXZTLDUjnGbFXqJojMkID2jXUIFDqtx0enoGj4zSh0EkTQkNp+rviRSHSo1D33SGWA/VvDcR//O6iQ4u3JSJONFUkNmiIOFQR3CSA+wzSYnmY0MwkczcCskQS0y0SatoQkDzLy+S1mkNOTV0c1auX+ZxFMABOAQVgMA5qINr0ABNQMADeAav4M16sl6sd+tj1rpk5TP74A+szx9YR5K2</latexit> z (i+1) t+1,2 <latexit sha1_base64="G0XcPH55M3XTZdbxur3zMiK0otc=">AAAB+nicbVDLSgMxFM3UV62vqS7dBItQqZRJEdRdwY3LCvYB7Thk0rQNzWSGJKPUcT7FjQtF3Pol7vwb03YW2nrgwuGce7n3Hj/iTGnH+bZyK6tr6xv5zcLW9s7unl3cb6kwloQ2SchD2fGxopwJ2tRMc9qJJMWBz2nbH19N/fY9lYqF4lZPIuoGeCjYgBGsjeTZxUcv0RV0WkvvkjKroJPUs0tO1ZkBLhOUkRLI0PDsr14/JHFAhSYcK9VFTqTdBEvNCKdpoRcrGmEyxkPaNVTggCo3mZ2ewmOj9OEglKaEhjP190SCA6UmgW86A6xHatGbiv953VgPLtyEiSjWVJD5okHMoQ7hNAfYZ5ISzSeGYCKZuRWSEZaYaJNWwYSAFl9eJq1aFTlVdHNWql9mceTBITgCZYDAOaiDa9AATUDAA3gGr+DNerJerHfrY96as7KZA/AH1ucPWdSStw==</latexit><latexit sha1_base64="G0XcPH55M3XTZdbxur3zMiK0otc=">AAAB+nicbVDLSgMxFM3UV62vqS7dBItQqZRJEdRdwY3LCvYB7Thk0rQNzWSGJKPUcT7FjQtF3Pol7vwb03YW2nrgwuGce7n3Hj/iTGnH+bZyK6tr6xv5zcLW9s7unl3cb6kwloQ2SchD2fGxopwJ2tRMc9qJJMWBz2nbH19N/fY9lYqF4lZPIuoGeCjYgBGsjeTZxUcv0RV0WkvvkjKroJPUs0tO1ZkBLhOUkRLI0PDsr14/JHFAhSYcK9VFTqTdBEvNCKdpoRcrGmEyxkPaNVTggCo3mZ2ewmOj9OEglKaEhjP190SCA6UmgW86A6xHatGbiv953VgPLtyEiSjWVJD5okHMoQ7hNAfYZ5ISzSeGYCKZuRWSEZaYaJNWwYSAFl9eJq1aFTlVdHNWql9mceTBITgCZYDAOaiDa9AATUDAA3gGr+DNerJerHfrY96as7KZA/AH1ucPWdSStw==</latexit><latexit sha1_base64="G0XcPH55M3XTZdbxur3zMiK0otc=">AAAB+nicbVDLSgMxFM3UV62vqS7dBItQqZRJEdRdwY3LCvYB7Thk0rQNzWSGJKPUcT7FjQtF3Pol7vwb03YW2nrgwuGce7n3Hj/iTGnH+bZyK6tr6xv5zcLW9s7unl3cb6kwloQ2SchD2fGxopwJ2tRMc9qJJMWBz2nbH19N/fY9lYqF4lZPIuoGeCjYgBGsjeTZxUcv0RV0WkvvkjKroJPUs0tO1ZkBLhOUkRLI0PDsr14/JHFAhSYcK9VFTqTdBEvNCKdpoRcrGmEyxkPaNVTggCo3mZ2ewmOj9OEglKaEhjP190SCA6UmgW86A6xHatGbiv953VgPLtyEiSjWVJD5okHMoQ7hNAfYZ5ISzSeGYCKZuRWSEZaYaJNWwYSAFl9eJq1aFTlVdHNWql9mceTBITgCZYDAOaiDa9AATUDAA3gGr+DNerJerHfrY96as7KZA/AH1ucPWdSStw==</latexit><latexit sha1_base64="G0XcPH55M3XTZdbxur3zMiK0otc=">AAAB+nicbVDLSgMxFM3UV62vqS7dBItQqZRJEdRdwY3LCvYB7Thk0rQNzWSGJKPUcT7FjQtF3Pol7vwb03YW2nrgwuGce7n3Hj/iTGnH+bZyK6tr6xv5zcLW9s7unl3cb6kwloQ2SchD2fGxopwJ2tRMc9qJJMWBz2nbH19N/fY9lYqF4lZPIuoGeCjYgBGsjeTZxUcv0RV0WkvvkjKroJPUs0tO1ZkBLhOUkRLI0PDsr14/JHFAhSYcK9VFTqTdBEvNCKdpoRcrGmEyxkPaNVTggCo3mZ2ewmOj9OEglKaEhjP190SCA6UmgW86A6xHatGbiv953VgPLtyEiSjWVJD5okHMoQ7hNAfYZ5ISzSeGYCKZuRWSEZaYaJNWwYSAFl9eJq1aFTlVdHNWql9mceTBITgCZYDAOaiDa9AATUDAA3gGr+DNerJerHfrY96as7KZA/AH1ucPWdSStw==</latexit>
z(i)t,1 + tanh <latexit sha1_base64="lSOnCFWQCuvxbU0oQl4fw4kzbuc=">AAACGHicbVDLSsNAFJ34rPUVdelmsAgVpSYiqLuCG5cV7AOaWCbTaTt0MgkzN0IN+Qw3/oobF4q47c6/cfpY2NYDFw7n3Mu99wSx4Boc58daWl5ZXVvPbeQ3t7Z3du29/ZqOEkVZlUYiUo2AaCa4ZFXgIFgjVoyEgWD1oH878utPTGkeyQcYxMwPSVfyDqcEjNSyzz3NuyHBHuWK4udWCmdu9pgW+UmGT/GM6QGRvZZdcErOGHiRuFNSQFNUWvbQa0c0CZkEKojWTdeJwU+JAk4Fy/JeollMaJ90WdNQSUKm/XT8WIaPjdLGnUiZkoDH6t+JlIRaD8LAdIYEenreG4n/ec0EOtd+ymWcAJN0sqiTCAwRHqWE21wxCmJgCKGKm1sx7RFFKJgs8yYEd/7lRVK7KLlOyb2/LJRvpnHk0CE6QkXkoitURneogqqIohf0hj7Qp/VqvVtf1vekdcmazhygGVjDX8dSnvA=</latexit><latexit sha1_base64="lSOnCFWQCuvxbU0oQl4fw4kzbuc=">AAACGHicbVDLSsNAFJ34rPUVdelmsAgVpSYiqLuCG5cV7AOaWCbTaTt0MgkzN0IN+Qw3/oobF4q47c6/cfpY2NYDFw7n3Mu99wSx4Boc58daWl5ZXVvPbeQ3t7Z3du29/ZqOEkVZlUYiUo2AaCa4ZFXgIFgjVoyEgWD1oH878utPTGkeyQcYxMwPSVfyDqcEjNSyzz3NuyHBHuWK4udWCmdu9pgW+UmGT/GM6QGRvZZdcErOGHiRuFNSQFNUWvbQa0c0CZkEKojWTdeJwU+JAk4Fy/JeollMaJ90WdNQSUKm/XT8WIaPjdLGnUiZkoDH6t+JlIRaD8LAdIYEenreG4n/ec0EOtd+ymWcAJN0sqiTCAwRHqWE21wxCmJgCKGKm1sx7RFFKJgs8yYEd/7lRVK7KLlOyb2/LJRvpnHk0CE6QkXkoitURneogqqIohf0hj7Qp/VqvVtf1vekdcmazhygGVjDX8dSnvA=</latexit><latexit sha1_base64="lSOnCFWQCuvxbU0oQl4fw4kzbuc=">AAACGHicbVDLSsNAFJ34rPUVdelmsAgVpSYiqLuCG5cV7AOaWCbTaTt0MgkzN0IN+Qw3/oobF4q47c6/cfpY2NYDFw7n3Mu99wSx4Boc58daWl5ZXVvPbeQ3t7Z3du29/ZqOEkVZlUYiUo2AaCa4ZFXgIFgjVoyEgWD1oH878utPTGkeyQcYxMwPSVfyDqcEjNSyzz3NuyHBHuWK4udWCmdu9pgW+UmGT/GM6QGRvZZdcErOGHiRuFNSQFNUWvbQa0c0CZkEKojWTdeJwU+JAk4Fy/JeollMaJ90WdNQSUKm/XT8WIaPjdLGnUiZkoDH6t+JlIRaD8LAdIYEenreG4n/ec0EOtd+ymWcAJN0sqiTCAwRHqWE21wxCmJgCKGKm1sx7RFFKJgs8yYEd/7lRVK7KLlOyb2/LJRvpnHk0CE6QkXkoitURneogqqIohf0hj7Qp/VqvVtf1vekdcmazhygGVjDX8dSnvA=</latexit><latexit sha1_base64="lSOnCFWQCuvxbU0oQl4fw4kzbuc=">AAACGHicbVDLSsNAFJ34rPUVdelmsAgVpSYiqLuCG5cV7AOaWCbTaTt0MgkzN0IN+Qw3/oobF4q47c6/cfpY2NYDFw7n3Mu99wSx4Boc58daWl5ZXVvPbeQ3t7Z3du29/ZqOEkVZlUYiUo2AaCa4ZFXgIFgjVoyEgWD1oH878utPTGkeyQcYxMwPSVfyDqcEjNSyzz3NuyHBHuWK4udWCmdu9pgW+UmGT/GM6QGRvZZdcErOGHiRuFNSQFNUWvbQa0c0CZkEKojWTdeJwU+JAk4Fy/JeollMaJ90WdNQSUKm/XT8WIaPjdLGnUiZkoDH6t+JlIRaD8LAdIYEenreG4n/ec0EOtd+ymWcAJN0sqiTCAwRHqWE21wxCmJgCKGKm1sx7RFFKJgs8yYEd/7lRVK7KLlOyb2/LJRvpnHk0CE6QkXkoitURneogqqIohf0hj7Qp/VqvVtf1vekdcmazhygGVjDX8dSnvA=</latexit>
every step using the result from the gated activations. We integrate the LSTM cell into the TrellisNet as follows (Figure 3):
ẑ (i+1) t+1 =W1 [ xt z (i) t,2 ] +W2 [ xt+1 z (i) t+1,2 ] = [ẑt+1,1 ẑt+1,2 ẑt+1,3 ẑt+1,4] > (11)
z (i+1) t+1,1 = σ(ẑt+1,1) ◦ z (i) t,1 + σ(ẑt+1,2) ◦ tanh(ẑt+1,3)
z (i+1) t+1,2 = σ(ẑt+1,4) ◦ tanh(z (i+1) t+1,1)
(12; Gated activation f )
Thus the linear transformation in each layer of the TrellisNet produces a pre-activation feature ẑt+1 with r = 4q feature channels, which are then processed by elementwise transformations and Hadamard products to yield the final output z(i+1)t+1 = ( z (i+1) t+1,1, z (i+1) t+1,2 ) of the layer.

heading: 5.2 RESULTS
text: We evaluate trellis networks on word-level and character-level language modeling on the standard Penn Treebank (PTB) dataset (Marcus et al., 1993; Mikolov et al., 2010), large-scale word-level modeling on WikiText-103 (WT103) (Merity et al., 2017), and standard stress tests used to study long-range information propagation in sequence models: sequential MNIST, permuted MNIST (PMNIST), and sequential CIFAR-10 (Chang et al., 2017; Bai et al., 2018; Trinh et al., 2018). Note that these tasks are on very different scales, with unique properties that challenge sequence models in different ways. For example, word-level PTB is a small dataset that a typical model easily overfits, so judicious regularization is essential. WT103 is a hundred times larger, with less danger of overfitting, but with a vocabulary size of 268K that makes training more challenging (and precludes the application of techniques such as mixture of softmaxes (Yang et al., 2018)). A more complete description of these tasks and their characteristics can be found in Appendix C.

heading: Word-level WikiText-103 (WT103)
text: The prior state of the art on these tasks was set by completely different models, such as AWD-LSTM on character-level PTB (Merity et al., 2018a), neural architecture search on word-level PTB (Pham et al., 2018), and the self-attention-based Relational Memory Core on WikiText-103 (Santoro et al., 2018). We use trellis networks on all tasks and outperform the respective state-of-the-art models on each. For example, on word-level Penn Treebank, TrellisNet outperforms by a good margin the recent results of Melis et al. (2018), which used the Google Vizier service for exhaustive hyperparameter tuning, as well as the recent neural architecture search work of Pham et al. (2018). On WikiText-103, a trellis network outperforms by 7.6% the Relational Memory Core (Santoro et al.,

heading: 2018) and by 11.5% the thorough optimization work of Merity et al. (2018a).
text: Many hyperparameters we use are adapted directly from prior work on recurrent networks. (As highlighted in Section 4.3, many techniques can be carried over directly from RNNs.) For others, we perform a basic grid search. We decay the learning rate by a fixed factor once validation error plateaus. All hyperparameters are reported in Appendix D, along with an ablation study.
Word-level language modeling. For word-level language modeling, we use PTB and WT103. The results on PTB are listed in Table 1. TrellisNet sets a new state of the art on PTB, both with and without mixture of softmaxes (Yang et al., 2018), outperforming all previously published results by more than one unit of perplexity.
WT103 is 110 times larger than PTB, with vocabulary size 268K. We follow prior work and use the adaptive softmax (Grave et al., 2017a), which improves memory efficiency by assigning higher capacity to more frequent words. The results are listed in Table 2. TrellisNet sets a new state of the art on this dataset as well, with perplexity 29.19: about 7.6% better than the contemporaneous
Generic TCN (Bai et al., 2018) 3.0M 1.31 Independently RNN (Li et al., 2018) 12.0M 1.23
Hyper LSTM (Ha et al., 2017) 14.4M 1.219 NAS Cell (Zoph & Le, 2017) 16.3M 1.214
Fast-Slow-LSTM-2 (Mujika et al., 2017) 7.2M 1.19 Quasi-RNN (Merity et al., 2018a) 13.8M 1.187
AWD-LSTM (Merity et al., 2018a) 13.8M 1.175 Ours - TrellisNet 13.4M 1.158
self-attention-based Relational Memory Core (RMC) (Santoro et al., 2018). TrellisNet achieves this better accuracy with much faster convergence: 25 epochs, versus 90 for RMC.
Character-level language modeling. When used for character-level modeling, PTB is a mediumscale dataset with stronger long-term dependencies between characters. We thus use a deeper network as well as techniques such as weight normalization (Salimans & Kingma, 2016) and deep supervision (Lee et al., 2015; Xie & Tu, 2015). The results are listed in Table 3. TrellisNet sets a new state of the art with 1.158 bpc, outperforming the recent results of Merity et al. (2018a) by a comfortable margin.
Long-range modeling with Sequential MNIST, PMNIST, and CIFAR-10. We also evaluate the TrellisNet for ability to model long-term dependencies. In the Sequential MNIST, PMNIST, and CIFAR-10 tasks, images are processed as long sequences, one pixel at a time (Chang et al., 2017; Bai et al., 2018; Trinh et al., 2018). Our model has 8M parameters, in alignment with prior work. To cover the larger context, we use dilated convolutions in intermediate layers, adopting a common architectural element from TCNs (Yu & Koltun, 2016; van den Oord et al., 2016; Bai et al., 2018). The results are listed in Table 4. Note that the performance of prior models is inconsistent. The Transformer works well on MNIST but fairs poorly on CIFAR-10, while r-LSTM with unsupervised auxiliary losses achieves good results on CIFAR-10 but underperforms on Permuted MNIST. TrellisNet outperforms all these models on all three tasks.

heading: 6 DISCUSSION
text: We presented trellis networks, a new architecture for sequence modeling. Trellis networks form a structural bridge between convolutional and recurrent models. This enables direct assimilation of many techniques designed for either of these two architectural families. We leverage these connections to train high-performing trellis networks that set a new state of the art on highly competitive language modeling benchmarks. Beyond the empirical gains, we hope that trellis networks will serve as a step towards deeper and more unified understanding of sequence modeling.
There are many exciting opportunities for future work. First, we have not conducted thorough performance optimizations on trellis networks. For example, architecture search on the structure of the gated activation f may yield a higher-performing activation function than the classic LSTM cell
we used (Zoph & Le, 2017; Pham et al., 2018). Likewise, principled hyperparameter tuning will likely improve modeling accuracy beyond the levels we have observed (Melis et al., 2018). Future work can also explore acceleration schemes that speed up training and inference.
Another significant opportunity is to establish connections between trellis networks and selfattention-based architectures (Transformers) (Vaswani et al., 2017; Santoro et al., 2018; Chen et al., 2018), thus unifying all three major contemporary approaches to sequence modeling. Finally, we look forward to seeing applications of trellis networks to industrial-scale challenges such as machine translation.

heading: Shiyu Chang, Yang Zhang, Wei Han, Mo Yu, Xiaoxiao Guo, Wei Tan, Xiaodong Cui, Michael
text: Witbrock, Mark Hasegawa-Johnson, and Thomas Huang. Dilated recurrent neural networks. In Neural Information Processing Systems (NIPS), 2017.
Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Niki Parmar, Noam Shazeer, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Mike Schuster, Zhifeng Chen, Yonghui Wu, and Macduff Hughes. The best of both worlds: Combining recent advances in neural machine translation. In Annual Meeting of the Association for Computational Linguistics (ACL), 2018.

heading: Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties
text: of neural machine translation: Encoder-decoder approaches. arXiv:1409.1259, 2014.

heading: Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel P.
text: Kuksa. Natural language processing (almost) from scratch. Journal of Machine Learning Research (JMLR), 12, 2011.

heading: Zihang Dai, Zhilin Yang, Yiming Yang, William W Cohen, Jaime Carbonell, Quoc V Le, and
text: Ruslan Salakhutdinov. Transformer-XL: Language modeling with longer-term dependency. arXiv:1901.02860, 2019.
Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In International Conference on Machine Learning (ICML), 2017.
Jeff Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Trevor Darrell, and Kate Saenko. Long-term recurrent convolutional networks for visual recognition and description. In Computer Vision and Pattern Recognition (CVPR), 2015.
Jeffrey L Elman. Finding structure in time. Cognitive Science, 14(2), 1990.
Yarin Gal and Zoubin Ghahramani. A theoretically grounded application of dropout in recurrent neural networks. In Neural Information Processing Systems (NIPS), 2016.

heading: Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. Convolutional
text: sequence to sequence learning. In International Conference on Machine Learning (ICML), 2017.

heading: Edouard Grave, Armand Joulin, Moustapha Cissé, David Grangier, and Hervé Jégou. Efficient
text: softmax approximation for GPUs. In International Conference on Machine Learning (ICML), 2017a.
Edouard Grave, Armand Joulin, and Nicolas Usunier. Improving neural language models with a continuous cache. In International Conference on Learning Representations (ICLR), 2017b.
Alex Graves. Supervised Sequence Labelling with Recurrent Neural Networks. Springer, 2012.
Alex Graves. Generating sequences with recurrent neural networks. arXiv:1308.0850, 2013.

heading: Klaus Greff, Rupesh Kumar Srivastava, Jan Koutnı́k, Bas R. Steunebrink, and Jürgen Schmidhuber.
text: LSTM: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2017.
David Ha, Andrew Dai, and Quoc V Le. HyperNetworks. In International Conference on Learning Representations (ICLR), 2017.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8), 1997.
Rafal Jozefowicz, Wojciech Zaremba, and Ilya Sutskever. An empirical exploration of recurrent network architectures. In International Conference on Machine Learning (ICML), 2015.

heading: Nal Kalchbrenner, Lasse Espeholt, Karen Simonyan, Aäron van den Oord, Alex Graves, and Koray
text: Kavukcuoglu. Neural machine translation in linear time. arXiv:1610.10099, 2016.
Andrej Karpathy and Fei-Fei Li. Deep visual-semantic alignments for generating image descriptions. In Computer Vision and Pattern Recognition (CVPR), 2015.
Urvashi Khandelwal, He He, Peng Qi, and Dan Jurafsky. Sharp nearby, fuzzy far away: How neural language models use context. In Annual Meeting of the Association for Computational Linguistics (ACL), 2018.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. ImageNet classification with deep convolutional neural networks. In Neural Information Processing Systems (NIPS), 2012.

heading: Yann LeCun, Bernhard Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne
text: Hubbard, and Lawrence D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4), 1989.
Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu. Deeplysupervised nets. In AISTATS, 2015.
Shuai Li, Wanqing Li, Chris Cook, Ce Zhu, and Yanbo Gao. Independently recurrent neural network (IndRNN): Building a longer and deeper RNN. In Computer Vision and Pattern Recognition (CVPR), 2018.
Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: Differentiable architecture search. arXiv:1806.09055, 2018.
Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. Building a large annotated corpus of English: The Penn treebank. Computational Linguistics, 19(2), 1993.
Gábor Melis, Chris Dyer, and Phil Blunsom. On the state of the art of evaluation in neural language models. In International Conference on Learning Representations (ICLR), 2018.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations (ICLR), 2017.
Stephen Merity, Nitish Shirish Keskar, and Richard Socher. An analysis of neural language modeling at multiple scales. arXiv:1803.08240, 2018a.

heading: Stephen Merity, Nitish Shirish Keskar, and Richard Socher. Regularizing and optimizing LSTM
text: language models. In International Conference on Learning Representations (ICLR), 2018b.

heading: Tomas Mikolov, Martin Karafiát, Lukás Burget, Jan Cernocký, and Sanjeev Khudanpur. Recurrent
text: neural network based language model. In Interspeech, 2010.
John Miller and Moritz Hardt. When recurrent models don’t need to be recurrent. arXiv:1805.10369, 2018.
Yasumasa Miyamoto and Kyunghyun Cho. Gated word-character recurrent language model. arXiv:1606.01700, 2016.
Asier Mujika, Florian Meier, and Angelika Steger. Fast-slow recurrent neural networks. In Neural Information Processing Systems (NIPS), 2017.
Hieu Pham, Melody Y Guan, Barret Zoph, Quoc V Le, and Jeff Dean. Efficient neural architecture search via parameters sharing. In International Conference on Machine Learning (ICML), 2018.
Tara N. Sainath, Oriol Vinyals, Andrew W. Senior, and Hasim Sak. Convolutional, long short-term memory, fully connected deep neural networks. In International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015.
Tim Salimans and Diederik P Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In Neural Information Processing Systems (NIPS), 2016.

heading: Adam Santoro, Ryan Faulkner, David Raposo, Jack Rae, Mike Chrzanowski, Theophane Weber,
text: Daan Wierstra, Oriol Vinyals, Razvan Pascanu, and Timothy Lillicrap. Relational recurrent neural networks. In Neural Information Processing Systems (NIPS), 2018.

heading: Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo.
text: Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Neural Information Processing Systems (NIPS), 2015.
Ilya Sutskever, James Martens, and Geoffrey E. Hinton. Generating text with recurrent neural networks. In International Conference on Machine Learning (ICML), 2011.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Neural Information Processing Systems (NIPS), 2014.
Trieu H Trinh, Andrew M Dai, Thang Luong, and Quoc V Le. Learning longer-term dependencies in RNNs with auxiliary losses. In International Conference on Machine Learning (ICML), 2018.

heading: Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves,
text: Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu. WaveNet: A generative model for raw audio. arXiv:1609.03499, 2016.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neural Information Processing Systems (NIPS), 2017.
Subhashini Venugopalan, Marcus Rohrbach, Jeffrey Donahue, Raymond J. Mooney, Trevor Darrell, and Kate Saenko. Sequence to sequence – video to text. In International Conference on Computer Vision (ICCV), 2015.
Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neural image caption generator. In Computer Vision and Pattern Recognition (CVPR), 2015.
Christoph Vogel and Thomas Pock. A primal dual network for low-level vision problems. In German Conference on Pattern Recognition, 2017.

heading: Alex Waibel, Toshiyuki Hanazawa, Geoffrey Hinton, Kiyohiro Shikano, and Kevin J Lang. Phoneme
text: recognition using time-delay neural networks. IEEE Transactions on Acoustics, Speech, and Signal Processing, 37(3), 1989.
Paul J Werbos. Backpropagation through time: What it does and how to do it. Proceedings of the IEEE, 78(10), 1990.
Saining Xie and Zhuowen Tu. Holistically-nested edge detection. In International Conference on Computer Vision (ICCV), 2015.

heading: Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W. Cohen. Breaking the softmax
text: bottleneck: A high-rank RNN language model. International Conference on Learning Representations (ICLR), 2018.
Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. In International Conference on Learning Representations (ICLR), 2016.

heading: Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutnı́k, and Jürgen Schmidhuber. Recurrent
text: highway networks. In International Conference on Machine Learning (ICML), 2017.
Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning. In International Conference on Learning Representations (ICLR), 2017.
A EXPRESSING AN LSTM AS A TRELLISNET
Here we trace in more detail the transformation of an LSTM into a TrellisNet. This is an application of Theorem 1. The nonlinear activation has been examined in Section 5.1. We will walk through the construction again here.
In each time step, an LSTM cell computes the following:
f (`) t = σ(Wfh (`−1) t + Ufh (`) t−1) i (`) t = σ(Wih (`−1) t + Uih (`) t−1) g (`) t = tanh(Wgh (`−1) t + Ugh (`) t−1)
o (`) t = σ(Woh (`−1) t + Uoh (`) t−1) c (`) t = f (`) t ◦ c(`)t−1 + i (`) t ◦ g(`)t h(`)t = o(`)t ◦ tanh(c(`)t )
(13)
where h(0)t = xt, and ft, it, ot are typically called the forget, input, and output gates. By a similar construction to how we defined τ in Theorem 1, to recover an LSTM the mixed group convolution needs to produce 3q more channels for these gated outputs, which have the form ft,t′ , it,t′ and gt,t′ (see Figure 5 for an example). In addition, at each layer of the mixed group convolution, the network also needs to maintain a group of channels for cell states ct,t′ . Note that in an LSTM network, ct is updated “synchronously” with ht, so we can similarly write
c (1) t,t′ = f (1) t,t′ ◦ c (1) t−1,t′ + i (1) t,t′ ◦ g (1) t,t′ h (1) t,t′ = o (1) t,t′ ◦ tanh(c (1) t,t′) (14)
Based on these changes, we show in Figure 4 an atomic and a sequence view of TrellisNet with the LSTM activation. The hidden units z1:T consist of two parts: z1:T,1, which gets updated directly via the gated activations (akin to LSTM cell states), and z1:T,2, which is processed by parameterized convolutions (akin to LSTM hidden states). Formally, in layer i:
ẑ (i+1) 1:T = Conv1D(z (i) 1:T,2;W ) + x̃1:T = [ẑ1:T,1 ẑ1:T,2 ẑ1:T,3 ẑ1:T,4]
>
z (i+1) 1:T,1 = σ(ẑ1:T,1) ◦ z (i) 0:T−1,1 + σ(ẑ1:T,2) ◦ tanh(ẑ1:T,3)
z (i+1) 1:T,2 = σ(ẑ1:T,4) ◦ tanh(z (i+1) 1:T,1 )
B OPTIMIZING AND REGULARIZING TRELLISNET WITH RNN AND TCN METHODS
xT <latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit><latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit><latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit><latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit> xT 1 <latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit><latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit><latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit><latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit> xT 2 <latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit><latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit><latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit><latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit>
h (1) T,1 <latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit> h (1) T 1,1 <latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit><latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit><latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit><latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit>
h (2) T 1,1 <latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit><latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit><latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit><latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit> h (2) T,1
<latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit>
h (1) T+1,1 <latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit><latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit><latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit><latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit>
h (2) T+1,1 <latexit sha1_base64="kKYH7p8a+9A6lDPD12H+MQBbr8Y=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSmZIthlwY3LCn1BG8NkOmmHTiZhZiLUkC9x40IRt36KO//GaZuFth64cDjnXu69x485U9pxvq3CxubW9k5xt7S3f3BYto+OuypKJKEdEvFI9n2sKGeCdjTTnPZjSXHoc9rzp7dzv/dIpWKRaOtZTN0QjwULGMHaSJ5dnnhp+xJdoewhrdYvMs+uODVnAbhOUE4qIEfLs7+Go4gkIRWacKzUADmxdlMsNSOcZqVhomiMyRSP6cBQgUOq3HRxeAbPjTKCQSRNCQ0X6u+JFIdKzULfdIZYT9SqNxf/8waJDhpuykScaCrIclGQcKgjOE8BjpikRPOZIZhIZm6FZIIlJtpkVTIhoNWX10m3XkNODd1fV5qNPI4iOAVnoAoQuAFNcAdaoAMISMAzeAVv1pP1Yr1bH8vWgpXPnIA/sD5/ANaLkdw=</latexit><latexit sha1_base64="kKYH7p8a+9A6lDPD12H+MQBbr8Y=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSmZIthlwY3LCn1BG8NkOmmHTiZhZiLUkC9x40IRt36KO//GaZuFth64cDjnXu69x485U9pxvq3CxubW9k5xt7S3f3BYto+OuypKJKEdEvFI9n2sKGeCdjTTnPZjSXHoc9rzp7dzv/dIpWKRaOtZTN0QjwULGMHaSJ5dnnhp+xJdoewhrdYvMs+uODVnAbhOUE4qIEfLs7+Go4gkIRWacKzUADmxdlMsNSOcZqVhomiMyRSP6cBQgUOq3HRxeAbPjTKCQSRNCQ0X6u+JFIdKzULfdIZYT9SqNxf/8waJDhpuykScaCrIclGQcKgjOE8BjpikRPOZIZhIZm6FZIIlJtpkVTIhoNWX10m3XkNODd1fV5qNPI4iOAVnoAoQuAFNcAdaoAMISMAzeAVv1pP1Yr1bH8vWgpXPnIA/sD5/ANaLkdw=</latexit><latexit sha1_base64="kKYH7p8a+9A6lDPD12H+MQBbr8Y=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSmZIthlwY3LCn1BG8NkOmmHTiZhZiLUkC9x40IRt36KO//GaZuFth64cDjnXu69x485U9pxvq3CxubW9k5xt7S3f3BYto+OuypKJKEdEvFI9n2sKGeCdjTTnPZjSXHoc9rzp7dzv/dIpWKRaOtZTN0QjwULGMHaSJ5dnnhp+xJdoewhrdYvMs+uODVnAbhOUE4qIEfLs7+Go4gkIRWacKzUADmxdlMsNSOcZqVhomiMyRSP6cBQgUOq3HRxeAbPjTKCQSRNCQ0X6u+JFIdKzULfdIZYT9SqNxf/8waJDhpuykScaCrIclGQcKgjOE8BjpikRPOZIZhIZm6FZIIlJtpkVTIhoNWX10m3XkNODd1fV5qNPI4iOAVnoAoQuAFNcAdaoAMISMAzeAVv1pP1Yr1bH8vWgpXPnIA/sD5/ANaLkdw=</latexit><latexit sha1_base64="kKYH7p8a+9A6lDPD12H+MQBbr8Y=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSmZIthlwY3LCn1BG8NkOmmHTiZhZiLUkC9x40IRt36KO//GaZuFth64cDjnXu69x485U9pxvq3CxubW9k5xt7S3f3BYto+OuypKJKEdEvFI9n2sKGeCdjTTnPZjSXHoc9rzp7dzv/dIpWKRaOtZTN0QjwULGMHaSJ5dnnhp+xJdoewhrdYvMs+uODVnAbhOUE4qIEfLs7+Go4gkIRWacKzUADmxdlMsNSOcZqVhomiMyRSP6cBQgUOq3HRxeAbPjTKCQSRNCQ0X6u+JFIdKzULfdIZYT9SqNxf/8waJDhpuykScaCrIclGQcKgjOE8BjpikRPOZIZhIZm6FZIIlJtpkVTIhoNWX10m3XkNODd1fV5qNPI4iOAVnoAoQuAFNcAdaoAMISMAzeAVv1pP1Yr1bH8vWgpXPnIA/sD5/ANaLkdw=</latexit>
xT+1 <latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit>
h (1) T,1 <latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit>
h (2) T,1 <latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit>
Layer 1 <latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit><latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit><latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit><latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit>
Layer 2 <latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit><latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit><latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit><latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit>
Layer 1 <latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit><latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit><latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit><latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit>
Layer 2 <latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit><latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit><latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit><latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit>
…
…
Truncated at time T<latexit sha1_base64="Mk2iSt4BcKdyVsVjsGdq5sFJR0s=">AAAB/nicbVDLSgNBEJyNrxhfUfHkZTARPIXdXPQY9OIxQl6QLGF2tpMMmZ1dZnqFsAT8FS8eFPHqd3jzb5w8DppY0FBT1c10V5BIYdB1v53cxubW9k5+t7C3f3B4VDw+aZk41RyaPJax7gTMgBQKmihQQifRwKJAQjsY38389iNoI2LVwEkCfsSGSgwEZ2ilfvGsoVNlHxBShhRFBLTcKPeLJbfizkHXibckJbJEvV/86oUxTyNQyCUzpuu5CfoZ0yi4hGmhlxpIGB+zIXQtVSwC42fz9af00iohHcTalkI6V39PZCwyZhIFtjNiODKr3kz8z+umOLjxM6GSFEHxxUeDVFKM6SwLGgoNHOXEEsa1sLtSPmKacbSJFWwI3urJ66RVrXhuxXuolmq3yzjy5JxckCvikWtSI/ekTpqEk4w8k1fy5jw5L86787FozTnLmVPyB87nD7cVlKU=</latexit><latexit sha1_base64="Mk2iSt4BcKdyVsVjsGdq5sFJR0s=">AAAB/nicbVDLSgNBEJyNrxhfUfHkZTARPIXdXPQY9OIxQl6QLGF2tpMMmZ1dZnqFsAT8FS8eFPHqd3jzb5w8DppY0FBT1c10V5BIYdB1v53cxubW9k5+t7C3f3B4VDw+aZk41RyaPJax7gTMgBQKmihQQifRwKJAQjsY38389iNoI2LVwEkCfsSGSgwEZ2ilfvGsoVNlHxBShhRFBLTcKPeLJbfizkHXibckJbJEvV/86oUxTyNQyCUzpuu5CfoZ0yi4hGmhlxpIGB+zIXQtVSwC42fz9af00iohHcTalkI6V39PZCwyZhIFtjNiODKr3kz8z+umOLjxM6GSFEHxxUeDVFKM6SwLGgoNHOXEEsa1sLtSPmKacbSJFWwI3urJ66RVrXhuxXuolmq3yzjy5JxckCvikWtSI/ekTpqEk4w8k1fy5jw5L86787FozTnLmVPyB87nD7cVlKU=</latexit><latexit sha1_base64="Mk2iSt4BcKdyVsVjsGdq5sFJR0s=">AAAB/nicbVDLSgNBEJyNrxhfUfHkZTARPIXdXPQY9OIxQl6QLGF2tpMMmZ1dZnqFsAT8FS8eFPHqd3jzb5w8DppY0FBT1c10V5BIYdB1v53cxubW9k5+t7C3f3B4VDw+aZk41RyaPJax7gTMgBQKmihQQifRwKJAQjsY38389iNoI2LVwEkCfsSGSgwEZ2ilfvGsoVNlHxBShhRFBLTcKPeLJbfizkHXibckJbJEvV/86oUxTyNQyCUzpuu5CfoZ0yi4hGmhlxpIGB+zIXQtVSwC42fz9af00iohHcTalkI6V39PZCwyZhIFtjNiODKr3kz8z+umOLjxM6GSFEHxxUeDVFKM6SwLGgoNHOXEEsa1sLtSPmKacbSJFWwI3urJ66RVrXhuxXuolmq3yzjy5JxckCvikWtSI/ekTpqEk4w8k1fy5jw5L86787FozTnLmVPyB87nD7cVlKU=</latexit><latexit sha1_base64="Mk2iSt4BcKdyVsVjsGdq5sFJR0s=">AAAB/nicbVDLSgNBEJyNrxhfUfHkZTARPIXdXPQY9OIxQl6QLGF2tpMMmZ1dZnqFsAT8FS8eFPHqd3jzb5w8DppY0FBT1c10V5BIYdB1v53cxubW9k5+t7C3f3B4VDw+aZk41RyaPJax7gTMgBQKmihQQifRwKJAQjsY38389iNoI2LVwEkCfsSGSgwEZ2ilfvGsoVNlHxBShhRFBLTcKPeLJbfizkHXibckJbJEvV/86oUxTyNQyCUzpuu5CfoZ0yi4hGmhlxpIGB+zIXQtVSwC42fz9af00iohHcTalkI6V39PZCwyZhIFtjNiODKr3kz8z+umOLjxM6GSFEHxxUeDVFKM6SwLGgoNHOXEEsa1sLtSPmKacbSJFWwI3urJ66RVrXhuxXuolmq3yzjy5JxckCvikWtSI/ekTpqEk4w8k1fy5jw5L86787FozTnLmVPyB87nD7cVlKU=</latexit>
Backpropagation <latexit sha1_base64="LncFmNq2rqbPQkuMtnveWZabwyc=">AAAB+HicbVBNTwIxEJ3FL8QPUI9eGomJJ7LLRY8ELx4xkY8ENqRbutDQbZu2a4IbfokXDxrj1Z/izX9jgT0o+JJJXt6bycy8SHFmrO9/e4Wt7Z3dveJ+6eDw6LhcOTntGJlqQttEcql7ETaUM0HblllOe0pTnEScdqPp7cLvPlJtmBQPdqZomOCxYDEj2DppWCk3MZkqLRUe50rVr/lLoE0S5KQKOVrDytdgJEmaUGEJx8b0A1/ZMMPaMsLpvDRIDVVuBx7TvqMCJ9SE2fLwObp0ygjFUrsSFi3V3xMZToyZJZHrTLCdmHVvIf7n9VMb34QZEyq1VJDVojjlyEq0SAGNmKbE8pkjmGjmbkVkgjUm1mVVciEE6y9vkk69Fvi14L5ebTTzOIpwDhdwBQFcQwPuoAVtIJDCM7zCm/fkvXjv3seqteDlM2fwB97nDw4+k1Q=</latexit><latexit sha1_base64="LncFmNq2rqbPQkuMtnveWZabwyc=">AAAB+HicbVBNTwIxEJ3FL8QPUI9eGomJJ7LLRY8ELx4xkY8ENqRbutDQbZu2a4IbfokXDxrj1Z/izX9jgT0o+JJJXt6bycy8SHFmrO9/e4Wt7Z3dveJ+6eDw6LhcOTntGJlqQttEcql7ETaUM0HblllOe0pTnEScdqPp7cLvPlJtmBQPdqZomOCxYDEj2DppWCk3MZkqLRUe50rVr/lLoE0S5KQKOVrDytdgJEmaUGEJx8b0A1/ZMMPaMsLpvDRIDVVuBx7TvqMCJ9SE2fLwObp0ygjFUrsSFi3V3xMZToyZJZHrTLCdmHVvIf7n9VMb34QZEyq1VJDVojjlyEq0SAGNmKbE8pkjmGjmbkVkgjUm1mVVciEE6y9vkk69Fvi14L5ebTTzOIpwDhdwBQFcQwPuoAVtIJDCM7zCm/fkvXjv3seqteDlM2fwB97nDw4+k1Q=</latexit><latexit sha1_base64="LncFmNq2rqbPQkuMtnveWZabwyc=">AAAB+HicbVBNTwIxEJ3FL8QPUI9eGomJJ7LLRY8ELx4xkY8ENqRbutDQbZu2a4IbfokXDxrj1Z/izX9jgT0o+JJJXt6bycy8SHFmrO9/e4Wt7Z3dveJ+6eDw6LhcOTntGJlqQttEcql7ETaUM0HblllOe0pTnEScdqPp7cLvPlJtmBQPdqZomOCxYDEj2DppWCk3MZkqLRUe50rVr/lLoE0S5KQKOVrDytdgJEmaUGEJx8b0A1/ZMMPaMsLpvDRIDVVuBx7TvqMCJ9SE2fLwObp0ygjFUrsSFi3V3xMZToyZJZHrTLCdmHVvIf7n9VMb34QZEyq1VJDVojjlyEq0SAGNmKbE8pkjmGjmbkVkgjUm1mVVciEE6y9vkk69Fvi14L5ebTTzOIpwDhdwBQFcQwPuoAVtIJDCM7zCm/fkvXjv3seqteDlM2fwB97nDw4+k1Q=</latexit><latexit sha1_base64="LncFmNq2rqbPQkuMtnveWZabwyc=">AAAB+HicbVBNTwIxEJ3FL8QPUI9eGomJJ7LLRY8ELx4xkY8ENqRbutDQbZu2a4IbfokXDxrj1Z/izX9jgT0o+JJJXt6bycy8SHFmrO9/e4Wt7Z3dveJ+6eDw6LhcOTntGJlqQttEcql7ETaUM0HblllOe0pTnEScdqPp7cLvPlJtmBQPdqZomOCxYDEj2DppWCk3MZkqLRUe50rVr/lLoE0S5KQKOVrDytdgJEmaUGEJx8b0A1/ZMMPaMsLpvDRIDVVuBx7TvqMCJ9SE2fLwObp0ygjFUrsSFi3V3xMZToyZJZHrTLCdmHVvIf7n9VMb34QZEyq1VJDVojjlyEq0SAGNmKbE8pkjmGjmbkVkgjUm1mVVciEE6y9vkk69Fvi14L5ebTTzOIpwDhdwBQFcQwPuoAVtIJDCM7zCm/fkvXjv3seqteDlM2fwB97nDw4+k1Q=</latexit>
Backpropagation <latexit sha1_base64="LncFmNq2rqbPQkuMtnveWZabwyc=">AAAB+HicbVBNTwIxEJ3FL8QPUI9eGomJJ7LLRY8ELx4xkY8ENqRbutDQbZu2a4IbfokXDxrj1Z/izX9jgT0o+JJJXt6bycy8SHFmrO9/e4Wt7Z3dveJ+6eDw6LhcOTntGJlqQttEcql7ETaUM0HblllOe0pTnEScdqPp7cLvPlJtmBQPdqZomOCxYDEj2DppWCk3MZkqLRUe50rVr/lLoE0S5KQKOVrDytdgJEmaUGEJx8b0A1/ZMMPaMsLpvDRIDVVuBx7TvqMCJ9SE2fLwObp0ygjFUrsSFi3V3xMZToyZJZHrTLCdmHVvIf7n9VMb34QZEyq1VJDVojjlyEq0SAGNmKbE8pkjmGjmbkVkgjUm1mVVciEE6y9vkk69Fvi14L5ebTTzOIpwDhdwBQFcQwPuoAVtIJDCM7zCm/fkvXjv3seqteDlM2fwB97nDw4+k1Q=</latexit><latexit sha1_base64="LncFmNq2rqbPQkuMtnveWZabwyc=">AAAB+HicbVBNTwIxEJ3FL8QPUI9eGomJJ7LLRY8ELx4xkY8ENqRbutDQbZu2a4IbfokXDxrj1Z/izX9jgT0o+JJJXt6bycy8SHFmrO9/e4Wt7Z3dveJ+6eDw6LhcOTntGJlqQttEcql7ETaUM0HblllOe0pTnEScdqPp7cLvPlJtmBQPdqZomOCxYDEj2DppWCk3MZkqLRUe50rVr/lLoE0S5KQKOVrDytdgJEmaUGEJx8b0A1/ZMMPaMsLpvDRIDVVuBx7TvqMCJ9SE2fLwObp0ygjFUrsSFi3V3xMZToyZJZHrTLCdmHVvIf7n9VMb34QZEyq1VJDVojjlyEq0SAGNmKbE8pkjmGjmbkVkgjUm1mVVciEE6y9vkk69Fvi14L5ebTTzOIpwDhdwBQFcQwPuoAVtIJDCM7zCm/fkvXjv3seqteDlM2fwB97nDw4+k1Q=</latexit><latexit sha1_base64="LncFmNq2rqbPQkuMtnveWZabwyc=">AAAB+HicbVBNTwIxEJ3FL8QPUI9eGomJJ7LLRY8ELx4xkY8ENqRbutDQbZu2a4IbfokXDxrj1Z/izX9jgT0o+JJJXt6bycy8SHFmrO9/e4Wt7Z3dveJ+6eDw6LhcOTntGJlqQttEcql7ETaUM0HblllOe0pTnEScdqPp7cLvPlJtmBQPdqZomOCxYDEj2DppWCk3MZkqLRUe50rVr/lLoE0S5KQKOVrDytdgJEmaUGEJx8b0A1/ZMMPaMsLpvDRIDVVuBx7TvqMCJ9SE2fLwObp0ygjFUrsSFi3V3xMZToyZJZHrTLCdmHVvIf7n9VMb34QZEyq1VJDVojjlyEq0SAGNmKbE8pkjmGjmbkVkgjUm1mVVciEE6y9vkk69Fvi14L5ebTTzOIpwDhdwBQFcQwPuoAVtIJDCM7zCm/fkvXjv3seqteDlM2fwB97nDw4+k1Q=</latexit><latexit sha1_base64="LncFmNq2rqbPQkuMtnveWZabwyc=">AAAB+HicbVBNTwIxEJ3FL8QPUI9eGomJJ7LLRY8ELx4xkY8ENqRbutDQbZu2a4IbfokXDxrj1Z/izX9jgT0o+JJJXt6bycy8SHFmrO9/e4Wt7Z3dveJ+6eDw6LhcOTntGJlqQttEcql7ETaUM0HblllOe0pTnEScdqPp7cLvPlJtmBQPdqZomOCxYDEj2DppWCk3MZkqLRUe50rVr/lLoE0S5KQKOVrDytdgJEmaUGEJx8b0A1/ZMMPaMsLpvDRIDVVuBx7TvqMCJ9SE2fLwObp0ygjFUrsSFi3V3xMZToyZJZHrTLCdmHVvIf7n9VMb34QZEyq1VJDVojjlyEq0SAGNmKbE8pkjmGjmbkVkgjUm1mVVciEE6y9vkk69Fvi14L5ebTTzOIpwDhdwBQFcQwPuoAVtIJDCM7zCm/fkvXjv3seqteDlM2fwB97nDw4+k1Q=</latexit>
(a) History repackaging between truncated sequences in recurrent networks.
…
…
…
xT <latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit><latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit><latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit><latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit> xT 1 <latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit><latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit><latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit><latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit> xT 2 <latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit><latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit><latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit><latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit>
h (1) T,1 <latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit> h (2) T,2 <latexit sha1_base64="dBbrzC8c00u+Haqy68kHIHlR20M=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCIqnghePFfoFbSyb7aZdutmku5tCCfkdXjwo4tUf481/47bNQVsfDDzem2FmnhdxprRtf1u5jc2t7Z38bmFv/+DwqHh80lJhLAltkpCHsuNhRTkTtKmZ5rQTSYoDj9O2N76f++0plYqFoqFnEXUDPBTMZwRrI7mjftK4qqZPSbl6mfaLJbtiL4DWiZOREmSo94tfvUFI4oAKTThWquvYkXYTLDUjnKaFXqxohMkYD2nXUIEDqtxkcXSKLowyQH4oTQmNFurviQQHSs0Cz3QGWI/UqjcX//O6sfZv3YSJKNZUkOUiP+ZIh2ieABowSYnmM0MwkczcisgIS0y0yalgQnBWX14nrWrFsSvO43WpdpfFkYczOIcyOHADNXiAOjSBwASe4RXerKn1Yr1bH8vWnJXNnMIfWJ8/hO+RPg==</latexit><latexit sha1_base64="dBbrzC8c00u+Haqy68kHIHlR20M=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCIqnghePFfoFbSyb7aZdutmku5tCCfkdXjwo4tUf481/47bNQVsfDDzem2FmnhdxprRtf1u5jc2t7Z38bmFv/+DwqHh80lJhLAltkpCHsuNhRTkTtKmZ5rQTSYoDj9O2N76f++0plYqFoqFnEXUDPBTMZwRrI7mjftK4qqZPSbl6mfaLJbtiL4DWiZOREmSo94tfvUFI4oAKTThWquvYkXYTLDUjnKaFXqxohMkYD2nXUIEDqtxkcXSKLowyQH4oTQmNFurviQQHSs0Cz3QGWI/UqjcX//O6sfZv3YSJKNZUkOUiP+ZIh2ieABowSYnmM0MwkczcisgIS0y0yalgQnBWX14nrWrFsSvO43WpdpfFkYczOIcyOHADNXiAOjSBwASe4RXerKn1Yr1bH8vWnJXNnMIfWJ8/hO+RPg==</latexit><latexit sha1_base64="dBbrzC8c00u+Haqy68kHIHlR20M=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCIqnghePFfoFbSyb7aZdutmku5tCCfkdXjwo4tUf481/47bNQVsfDDzem2FmnhdxprRtf1u5jc2t7Z38bmFv/+DwqHh80lJhLAltkpCHsuNhRTkTtKmZ5rQTSYoDj9O2N76f++0plYqFoqFnEXUDPBTMZwRrI7mjftK4qqZPSbl6mfaLJbtiL4DWiZOREmSo94tfvUFI4oAKTThWquvYkXYTLDUjnKaFXqxohMkYD2nXUIEDqtxkcXSKLowyQH4oTQmNFurviQQHSs0Cz3QGWI/UqjcX//O6sfZv3YSJKNZUkOUiP+ZIh2ieABowSYnmM0MwkczcisgIS0y0yalgQnBWX14nrWrFsSvO43WpdpfFkYczOIcyOHADNXiAOjSBwASe4RXerKn1Yr1bH8vWnJXNnMIfWJ8/hO+RPg==</latexit><latexit sha1_base64="dBbrzC8c00u+Haqy68kHIHlR20M=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCIqnghePFfoFbSyb7aZdutmku5tCCfkdXjwo4tUf481/47bNQVsfDDzem2FmnhdxprRtf1u5jc2t7Z38bmFv/+DwqHh80lJhLAltkpCHsuNhRTkTtKmZ5rQTSYoDj9O2N76f++0plYqFoqFnEXUDPBTMZwRrI7mjftK4qqZPSbl6mfaLJbtiL4DWiZOREmSo94tfvUFI4oAKTThWquvYkXYTLDUjnKaFXqxohMkYD2nXUIEDqtxkcXSKLowyQH4oTQmNFurviQQHSs0Cz3QGWI/UqjcX//O6sfZv3YSJKNZUkOUiP+ZIh2ieABowSYnmM0MwkczcisgIS0y0yalgQnBWX14nrWrFsSvO43WpdpfFkYczOIcyOHADNXiAOjSBwASe4RXerKn1Yr1bH8vWnJXNnMIfWJ8/hO+RPg==</latexit> h (1) T 1,1
<latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit><latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit><latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit><latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit>
h (2) T 1,1 <latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit><latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit><latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit><latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit>
h (2) T,1 <latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit>
h (1) T,1 <latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit> h (1) T 1,1 <latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit><latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit><latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit><latexit sha1_base64="y/FVYb6QZ9a7R1R98R13vjdYFc0=">AAAB+HicbVDJSgNBEK2JW4xLRj16aQxCBA3TIiieAl48RsgGyTj0dHqSJj0L3T1CHOZLvHhQxKuf4s2/sbMcNPqg4PFeFVX1/ERwpR3nyyqsrK6tbxQ3S1vbO7tle2+/reJUUtaisYhl1yeKCR6xluZasG4iGQl9wTr++Gbqdx6YVDyOmnqSMDckw4gHnBJtJM8uj7yseYZPcX6fVfFJ7tkVp+bMgP4SvCAVWKDh2Z/9QUzTkEWaCqJUDzuJdjMiNaeC5aV+qlhC6JgMWc/QiIRMudns8BwdG2WAgliaijSaqT8nMhIqNQl90xkSPVLL3lT8z+ulOrhyMx4lqWYRnS8KUoF0jKYpoAGXjGoxMYRQyc2tiI6IJFSbrEomBLz88l/SPq9hp4bvLir160UcRTiEI6gChkuowy00oAUUUniCF3i1Hq1n6816n7cWrMXMAfyC9fEN2LuR3w==</latexit> h (2) T 1,1 <latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit><latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit><latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit><latexit sha1_base64="fE1cCTapsK3zIqw8Pg24HnxpwMw=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCBS2ZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLWxfoHGUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9pBkeA=</latexit>
xT <latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit><latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit><latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit><latexit sha1_base64="mxhW5YZNSmp2iLlswqU2446rSoM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUDwVvHis2C9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fCopeNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6LSPJYNM0nQj+hQ8pAzaqz08NRv9MsVt+rOQVaJl5MK5Kj3y1+9QczSCKVhgmrd9dzE+BlVhjOB01Iv1ZhQNqZD7FoqaYTaz+anTsmZVQYkjJUtachc/T2R0UjrSRTYzoiakV72ZuJ/Xjc14bWfcZmkBiVbLApTQUxMZn+TAVfIjJhYQpni9lbCRlRRZmw6JRuCt/zyKmldVD236t1fVmo3eRxFOIFTOAcPrqAGd1CHJjAYwjO8wpsjnBfn3flYtBacfOYY/sD5/AE+VI27</latexit> xT 1 <latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit><latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit><latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit><latexit sha1_base64="Cbw1qmWaO0p0CMB0l93d9rvur2U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyURQfFU8OKxQr+gDWWz3bRLN5uwOxFL6I/w4kERr/4eb/4bt20O2vpg4PHeDDPzgkQKg6777RTW1jc2t4rbpZ3dvf2D8uFRy8SpZrzJYhnrTkANl0LxJgqUvJNoTqNA8nYwvpv57UeujYhVAycJ9yM6VCIUjKKV2k/9rHHhTfvlilt15yCrxMtJBXLU++Wv3iBmacQVMkmN6Xpugn5GNQom+bTUSw1PKBvTIe9aqmjEjZ/Nz52SM6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLIheMsvr5LWZdVzq97DVaV2m8dRhBM4hXPw4BpqcA91aAKDMTzDK7w5ifPivDsfi9aCk88cwx84nz/fRY85</latexit> xT 2 <latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit><latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit><latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit><latexit sha1_base64="dXjy5/4DK4Wimh7Sn1xvl8IKfMc=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4sSRFUDwVvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O2vrG5tb24Wd4u7e/sFh6ei4ZeJUM95ksYx1J6CGS6F4EwVK3kk0p1EgeTsY38389iPXRsSqgZOE+xEdKhEKRtFK7ad+1risTvulsltx5yCrxMtJGXLU+6Wv3iBmacQVMkmN6Xpugn5GNQom+bTYSw1PKBvTIe9aqmjEjZ/Nz52Sc6sMSBhrWwrJXP09kdHImEkU2M6I4sgsezPxP6+bYnjjZ0IlKXLFFovCVBKMyex3MhCaM5QTSyjTwt5K2IhqytAmVLQheMsvr5JWteK5Fe/hqly7zeMowCmcwQV4cA01uIc6NIHBGJ7hFd6cxHlx3p2PReuak8+cwB84nz/gyo86</latexit>
Layer T + 1 <latexit sha1_base64="SAl8ZIDAGqF6+OQtJeM24sLD3zo=">AAAB8nicbVA9SwNBEN2LXzF+RS1tFhNBEMJdCrUM2FhYRMgXXI6wt5kkS/Z2j9094TjyM2wsFLH119j5b9wkV2jig4HHezPMzAtjzrRx3W+nsLG5tb1T3C3t7R8cHpWPTzpaJopCm0ouVS8kGjgT0DbMcOjFCkgUcuiG07u5330CpZkULZPGEERkLNiIUWKs5D+QFBSutq686qBccWvuAnideDmpoBzNQfmrP5Q0iUAYyonWvufGJsiIMoxymJX6iYaY0CkZg2+pIBHoIFucPMMXVhnikVS2hMEL9fdERiKt0yi0nRExE73qzcX/PD8xo9sgYyJODAi6XDRKODYSz//HQ6aAGp5aQqhi9lZMJ0QRamxKJRuCt/ryOunUa55b8x7rlcZ1HkcRnaFzdIk8dIMa6B41URtRJNEzekVvjnFenHfnY9lacPKZU/QHzucPW9iP7w==</latexit><latexit sha1_base64="SAl8ZIDAGqF6+OQtJeM24sLD3zo=">AAAB8nicbVA9SwNBEN2LXzF+RS1tFhNBEMJdCrUM2FhYRMgXXI6wt5kkS/Z2j9094TjyM2wsFLH119j5b9wkV2jig4HHezPMzAtjzrRx3W+nsLG5tb1T3C3t7R8cHpWPTzpaJopCm0ouVS8kGjgT0DbMcOjFCkgUcuiG07u5330CpZkULZPGEERkLNiIUWKs5D+QFBSutq686qBccWvuAnideDmpoBzNQfmrP5Q0iUAYyonWvufGJsiIMoxymJX6iYaY0CkZg2+pIBHoIFucPMMXVhnikVS2hMEL9fdERiKt0yi0nRExE73qzcX/PD8xo9sgYyJODAi6XDRKODYSz//HQ6aAGp5aQqhi9lZMJ0QRamxKJRuCt/ryOunUa55b8x7rlcZ1HkcRnaFzdIk8dIMa6B41URtRJNEzekVvjnFenHfnY9lacPKZU/QHzucPW9iP7w==</latexit><latexit sha1_base64="SAl8ZIDAGqF6+OQtJeM24sLD3zo=">AAAB8nicbVA9SwNBEN2LXzF+RS1tFhNBEMJdCrUM2FhYRMgXXI6wt5kkS/Z2j9094TjyM2wsFLH119j5b9wkV2jig4HHezPMzAtjzrRx3W+nsLG5tb1T3C3t7R8cHpWPTzpaJopCm0ouVS8kGjgT0DbMcOjFCkgUcuiG07u5330CpZkULZPGEERkLNiIUWKs5D+QFBSutq686qBccWvuAnideDmpoBzNQfmrP5Q0iUAYyonWvufGJsiIMoxymJX6iYaY0CkZg2+pIBHoIFucPMMXVhnikVS2hMEL9fdERiKt0yi0nRExE73qzcX/PD8xo9sgYyJODAi6XDRKODYSz//HQ6aAGp5aQqhi9lZMJ0QRamxKJRuCt/ryOunUa55b8x7rlcZ1HkcRnaFzdIk8dIMa6B41URtRJNEzekVvjnFenHfnY9lacPKZU/QHzucPW9iP7w==</latexit><latexit sha1_base64="SAl8ZIDAGqF6+OQtJeM24sLD3zo=">AAAB8nicbVA9SwNBEN2LXzF+RS1tFhNBEMJdCrUM2FhYRMgXXI6wt5kkS/Z2j9094TjyM2wsFLH119j5b9wkV2jig4HHezPMzAtjzrRx3W+nsLG5tb1T3C3t7R8cHpWPTzpaJopCm0ouVS8kGjgT0DbMcOjFCkgUcuiG07u5330CpZkULZPGEERkLNiIUWKs5D+QFBSutq686qBccWvuAnideDmpoBzNQfmrP5Q0iUAYyonWvufGJsiIMoxymJX6iYaY0CkZg2+pIBHoIFucPMMXVhnikVS2hMEL9fdERiKt0yi0nRExE73qzcX/PD8xo9sgYyJODAi6XDRKODYSz//HQ6aAGp5aQqhi9lZMJ0QRamxKJRuCt/ryOunUa55b8x7rlcZ1HkcRnaFzdIk8dIMa6B41URtRJNEzekVvjnFenHfnY9lacPKZU/QHzucPW9iP7w==</latexit>
Layer T <latexit sha1_base64="95kS2GDLkzDmHwiYUt/qqictcx8=">AAAB8HicbVA9SwNBEN2LXzF+RS1tFhPBKtylUMuAjYVFhHxJcoS9zVyyZHfv2N0TjiO/wsZCEVt/jp3/xk1yhSY+GHi8N8PMvCDmTBvX/XYKG5tb2zvF3dLe/sHhUfn4pKOjRFFo04hHqhcQDZxJaBtmOPRiBUQEHLrB9Hbud59AaRbJlklj8AUZSxYySoyVHu9JCgpXW9VhueLW3AXwOvFyUkE5msPy12AU0USANJQTrfueGxs/I8owymFWGiQaYkKnZAx9SyURoP1scfAMX1hlhMNI2ZIGL9TfExkRWqcisJ2CmIle9ebif14/MeGNnzEZJwYkXS4KE45NhOff4xFTQA1PLSFUMXsrphOiCDU2o5INwVt9eZ106jXPrXkP9UrjKo+jiM7QObpEHrpGDXSHmqiNKBLoGb2iN0c5L86787FsLTj5zCn6A+fzB4C+j38=</latexit><latexit sha1_base64="95kS2GDLkzDmHwiYUt/qqictcx8=">AAAB8HicbVA9SwNBEN2LXzF+RS1tFhPBKtylUMuAjYVFhHxJcoS9zVyyZHfv2N0TjiO/wsZCEVt/jp3/xk1yhSY+GHi8N8PMvCDmTBvX/XYKG5tb2zvF3dLe/sHhUfn4pKOjRFFo04hHqhcQDZxJaBtmOPRiBUQEHLrB9Hbud59AaRbJlklj8AUZSxYySoyVHu9JCgpXW9VhueLW3AXwOvFyUkE5msPy12AU0USANJQTrfueGxs/I8owymFWGiQaYkKnZAx9SyURoP1scfAMX1hlhMNI2ZIGL9TfExkRWqcisJ2CmIle9ebif14/MeGNnzEZJwYkXS4KE45NhOff4xFTQA1PLSFUMXsrphOiCDU2o5INwVt9eZ106jXPrXkP9UrjKo+jiM7QObpEHrpGDXSHmqiNKBLoGb2iN0c5L86787FsLTj5zCn6A+fzB4C+j38=</latexit><latexit sha1_base64="95kS2GDLkzDmHwiYUt/qqictcx8=">AAAB8HicbVA9SwNBEN2LXzF+RS1tFhPBKtylUMuAjYVFhHxJcoS9zVyyZHfv2N0TjiO/wsZCEVt/jp3/xk1yhSY+GHi8N8PMvCDmTBvX/XYKG5tb2zvF3dLe/sHhUfn4pKOjRFFo04hHqhcQDZxJaBtmOPRiBUQEHLrB9Hbud59AaRbJlklj8AUZSxYySoyVHu9JCgpXW9VhueLW3AXwOvFyUkE5msPy12AU0USANJQTrfueGxs/I8owymFWGiQaYkKnZAx9SyURoP1scfAMX1hlhMNI2ZIGL9TfExkRWqcisJ2CmIle9ebif14/MeGNnzEZJwYkXS4KE45NhOff4xFTQA1PLSFUMXsrphOiCDU2o5INwVt9eZ106jXPrXkP9UrjKo+jiM7QObpEHrpGDXSHmqiNKBLoGb2iN0c5L86787FsLTj5zCn6A+fzB4C+j38=</latexit><latexit sha1_base64="95kS2GDLkzDmHwiYUt/qqictcx8=">AAAB8HicbVA9SwNBEN2LXzF+RS1tFhPBKtylUMuAjYVFhHxJcoS9zVyyZHfv2N0TjiO/wsZCEVt/jp3/xk1yhSY+GHi8N8PMvCDmTBvX/XYKG5tb2zvF3dLe/sHhUfn4pKOjRFFo04hHqhcQDZxJaBtmOPRiBUQEHLrB9Hbud59AaRbJlklj8AUZSxYySoyVHu9JCgpXW9VhueLW3AXwOvFyUkE5msPy12AU0USANJQTrfueGxs/I8owymFWGiQaYkKnZAx9SyURoP1scfAMX1hlhMNI2ZIGL9TfExkRWqcisJ2CmIle9ebif14/MeGNnzEZJwYkXS4KE45NhOff4xFTQA1PLSFUMXsrphOiCDU2o5INwVt9eZ106jXPrXkP9UrjKo+jiM7QObpEHrpGDXSHmqiNKBLoGb2iN0c5L86787FsLTj5zCn6A+fzB4C+j38=</latexit>
0 <latexit sha1_base64="SnGAWWDvcKOm5XCgld8TjcUaeUk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RS2tnd294r7pYPDo+OT8ulZR8epYthmsYhVL6AaBZfYNtwI7CUKaRQI7AbTu4XffUKleSwfzCxBP6JjyUPOqLFSyx2WK27VXYJsEi8nFcjRHJa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tD52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asK6n3GZpAYlWy0KU0FMTBZfkxFXyIyYWUKZ4vZWwiZUUWZsNiUbgrf+8ibp3FQ9t+q1biuNeh5HES7gEq7Bgxo04B6a0AYGCM/wCm/Oo/PivDsfq9aCk8+cwx84nz92F4yq</latexit><latexit sha1_base64="SnGAWWDvcKOm5XCgld8TjcUaeUk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RS2tnd294r7pYPDo+OT8ulZR8epYthmsYhVL6AaBZfYNtwI7CUKaRQI7AbTu4XffUKleSwfzCxBP6JjyUPOqLFSyx2WK27VXYJsEi8nFcjRHJa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tD52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asK6n3GZpAYlWy0KU0FMTBZfkxFXyIyYWUKZ4vZWwiZUUWZsNiUbgrf+8ibp3FQ9t+q1biuNeh5HES7gEq7Bgxo04B6a0AYGCM/wCm/Oo/PivDsfq9aCk8+cwx84nz92F4yq</latexit><latexit sha1_base64="SnGAWWDvcKOm5XCgld8TjcUaeUk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RS2tnd294r7pYPDo+OT8ulZR8epYthmsYhVL6AaBZfYNtwI7CUKaRQI7AbTu4XffUKleSwfzCxBP6JjyUPOqLFSyx2WK27VXYJsEi8nFcjRHJa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tD52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asK6n3GZpAYlWy0KU0FMTBZfkxFXyIyYWUKZ4vZWwiZUUWZsNiUbgrf+8ibp3FQ9t+q1biuNeh5HES7gEq7Bgxo04B6a0AYGCM/wCm/Oo/PivDsfq9aCk8+cwx84nz92F4yq</latexit><latexit sha1_base64="SnGAWWDvcKOm5XCgld8TjcUaeUk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RS2tnd294r7pYPDo+OT8ulZR8epYthmsYhVL6AaBZfYNtwI7CUKaRQI7AbTu4XffUKleSwfzCxBP6JjyUPOqLFSyx2WK27VXYJsEi8nFcjRHJa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tD52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asK6n3GZpAYlWy0KU0FMTBZfkxFXyIyYWUKZ4vZWwiZUUWZsNiUbgrf+8ibp3FQ9t+q1biuNeh5HES7gEq7Bgxo04B6a0AYGCM/wCm/Oo/PivDsfq9aCk8+cwx84nz92F4yq</latexit> xT+1 <latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit>
xT+1 <latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit> 0 <latexit sha1_base64="SnGAWWDvcKOm5XCgld8TjcUaeUk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RS2tnd294r7pYPDo+OT8ulZR8epYthmsYhVL6AaBZfYNtwI7CUKaRQI7AbTu4XffUKleSwfzCxBP6JjyUPOqLFSyx2WK27VXYJsEi8nFcjRHJa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tD52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asK6n3GZpAYlWy0KU0FMTBZfkxFXyIyYWUKZ4vZWwiZUUWZsNiUbgrf+8ibp3FQ9t+q1biuNeh5HES7gEq7Bgxo04B6a0AYGCM/wCm/Oo/PivDsfq9aCk8+cwx84nz92F4yq</latexit><latexit sha1_base64="SnGAWWDvcKOm5XCgld8TjcUaeUk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RS2tnd294r7pYPDo+OT8ulZR8epYthmsYhVL6AaBZfYNtwI7CUKaRQI7AbTu4XffUKleSwfzCxBP6JjyUPOqLFSyx2WK27VXYJsEi8nFcjRHJa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tD52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asK6n3GZpAYlWy0KU0FMTBZfkxFXyIyYWUKZ4vZWwiZUUWZsNiUbgrf+8ibp3FQ9t+q1biuNeh5HES7gEq7Bgxo04B6a0AYGCM/wCm/Oo/PivDsfq9aCk8+cwx84nz92F4yq</latexit><latexit sha1_base64="SnGAWWDvcKOm5XCgld8TjcUaeUk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RS2tnd294r7pYPDo+OT8ulZR8epYthmsYhVL6AaBZfYNtwI7CUKaRQI7AbTu4XffUKleSwfzCxBP6JjyUPOqLFSyx2WK27VXYJsEi8nFcjRHJa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tD52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asK6n3GZpAYlWy0KU0FMTBZfkxFXyIyYWUKZ4vZWwiZUUWZsNiUbgrf+8ibp3FQ9t+q1biuNeh5HES7gEq7Bgxo04B6a0AYGCM/wCm/Oo/PivDsfq9aCk8+cwx84nz92F4yq</latexit><latexit sha1_base64="SnGAWWDvcKOm5XCgld8TjcUaeUk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RS2tnd294r7pYPDo+OT8ulZR8epYthmsYhVL6AaBZfYNtwI7CUKaRQI7AbTu4XffUKleSwfzCxBP6JjyUPOqLFSyx2WK27VXYJsEi8nFcjRHJa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tD52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asK6n3GZpAYlWy0KU0FMTBZfkxFXyIyYWUKZ4vZWwiZUUWZsNiUbgrf+8ibp3FQ9t+q1biuNeh5HES7gEq7Bgxo04B6a0AYGCM/wCm/Oo/PivDsfq9aCk8+cwx84nz92F4yq</latexit>
Layer 1 <latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit><latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit><latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit><latexit sha1_base64="ZxQuDh5uth0sILyD6mLZELll5sw=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1TEG1Zrbt1dgKwTryA1KNAaVr8Go5ilEUrDBNW677mJ8XOqDGcCZ5VBqjGhbErH2LdU0gi1ny/OnZELq4xIGCtb0pCF+nsip5HWWRTYzoiaiV715uJ/Xj814Y2fc5mkBiVbLgpTQUxM5r+TEVfIjMgsoUxxeythE6ooMzahig3BW315nXQadc+tew+NWvOqiKMMZ3AOl+DBNTThDlrQBgZTeIZXeHMS58V5dz6WrSWnmDmFP3A+fwCPy48A</latexit>
Layer 2 <latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit><latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit><latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit><latexit sha1_base64="JU08TK1DtIt6F5DuEM9JfoFGuoU=">AAAB7nicbVA9SwNBEJ2LXzF+RS1tFoNgFe5SqGXAxsIigvmA5Ah7m7lkyd7esbsnHEd+hI2FIrb+Hjv/jZvkCk18MPB4b4aZeUEiuDau++2UNja3tnfKu5W9/YPDo+rxSUfHqWLYZrGIVS+gGgWX2DbcCOwlCmkUCOwG09u5331CpXksH02WoB/RseQhZ9RYqXtPM1SkMazW3Lq7AFknXkFqUKA1rH4NRjFLI5SGCap133MT4+dUGc4EziqDVGNC2ZSOsW+ppBFqP1+cOyMXVhmRMFa2pCEL9fdETiOtsyiwnRE1E73qzcX/vH5qwhs/5zJJDUq2XBSmgpiYzH8nI66QGZFZQpni9lbCJlRRZmxCFRuCt/ryOuk06p5b9x4ateZVEUcZzuAcLsGDa2jCHbSgDQym8Ayv8OYkzovz7nwsW0tOMXMKf+B8/gCRT48B</latexit>
h (2) T,1 <latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit>
h (1) T,1 <latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit>
h (2) T,1 <latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit><latexit sha1_base64="EV47VmZDPBs9/1DEoVX6lpLCP3c=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBahgpSkCPZY8OKxQr+gjWWz3bRLN5t0d1MoIb/DiwdFvPpjvPlv3LY5aOuDgcd7M8zM8yLOlLbtbyu3tb2zu5ffLxwcHh2fFE/P2iqMJaEtEvJQdj2sKGeCtjTTnHYjSXHgcdrxJvcLvzOjUrFQNPU8om6AR4L5jGBtJHc8SJo3TvqUlKvX6aBYsiv2EmiTOBkpQYbGoPjVH4YkDqjQhGOleo4daTfBUjPCaVrox4pGmEzwiPYMFTigyk2WR6foyihD5IfSlNBoqf6eSHCg1DzwTGeA9VitewvxP68Xa7/mJkxEsaaCrBb5MUc6RIsE0JBJSjSfG4KJZOZWRMZYYqJNTgUTgrP+8iZpVyuOXXEeb0v1WhZHHi7gEsrgwB3U4QEa0AICU3iGV3izZtaL9W59rFpzVjZzDn9gff4AgsqROw==</latexit>
h (1) T,1 <latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit><latexit sha1_base64="h4/JN0FCwd/16T8uYmGW0hiOxLk=">AAAB9HicbVBNSwMxEJ2tX7V+VT16CRahgpSNCIqnghePFfoF7VqyadqGZrNrki2UZX+HFw+KePXHePPfmLZ70NYHA4/3ZpiZ50eCa+O6305ubX1jcyu/XdjZ3ds/KB4eNXUYK8oaNBShavtEM8ElaxhuBGtHipHAF6zlj+9mfmvClOahrJtpxLyADCUfcEqMlbxRL6lf4PQxKePztFcsuRV3DrRKcEZKkKHWK351+yGNAyYNFUTrDnYj4yVEGU4FSwvdWLOI0DEZso6lkgRMe8n86BSdWaWPBqGyJQ2aq78nEhJoPQ182xkQM9LL3kz8z+vEZnDjJVxGsWGSLhYNYoFMiGYJoD5XjBoxtYRQxe2tiI6IItTYnAo2BLz88ippXlawW8EPV6XqbRZHHk7gFMqA4RqqcA81aACFJ3iGV3hzJs6L8+58LFpzTjZzDH/gfP4Agd6RPA==</latexit>
h (1) T+1,1 <latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit><latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit><latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit><latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit>
xT+1 <latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit><latexit sha1_base64="qetRKztGPtBFR5l7XpLVxqsh6Nc=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBZBEEoigj0WvHis0C9oQ9lsN+3SzSbsTsQS+iO8eFDEq7/Hm//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNM7uZ+55FrI2LVxGnC/YiOlAgFo2ilztMga155s0G54lbdBcg68XJSgRyNQfmrP4xZGnGFTFJjep6boJ9RjYJJPiv1U8MTyiZ0xHuWKhpx42eLc2fkwipDEsbalkKyUH9PZDQyZhoFtjOiODar3lz8z+ulGNb8TKgkRa7YclGYSoIxmf9OhkJzhnJqCWVa2FsJG1NNGdqESjYEb/XlddK+rnpu1Xu4qdRreRxFOINzuAQPbqEO99CAFjCYwDO8wpuTOC/Ou/OxbC04+cwp/IHz+QPbn481</latexit>
h (1) T+1,1 <latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit><latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit><latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit><latexit sha1_base64="hNFfSyxk3GWlKtd2UXv7dwutHjA=">AAAB+HicbVBNS8NAEJ34WetHox69BItQUUpWBHssePFYoV/QxrDZbtqlm03Y3Qg19Jd48aCIV3+KN/+N2zYHbX0w8Hhvhpl5QcKZ0q77ba2tb2xubRd2irt7+wcl+/CoreJUEtoiMY9lN8CKciZoSzPNaTeRFEcBp51gfDvzO49UKhaLpp4k1IvwULCQEayN5NulkZ81L9Almj5kFXQ+9e2yW3XncFYJykkZcjR8+6s/iEkaUaEJx0r1kJtoL8NSM8LptNhPFU0wGeMh7RkqcESVl80PnzpnRhk4YSxNCe3M1d8TGY6UmkSB6YywHqllbyb+5/VSHda8jIkk1VSQxaIw5Y6OnVkKzoBJSjSfGIKJZOZWh4ywxESbrIomBLT88ippX1WRW0X31+V6LY+jACdwChVAcAN1uIMGtIBACs/wCm/Wk/VivVsfi9Y1K585hj+wPn8A1QWR2w==</latexit>
h (2) T+1,1 <latexit sha1_base64="nU9WoWCYWa7Q0ZNrnOd1M/6u7oc=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSmZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLW+foAmUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9clkd4=</latexit><latexit sha1_base64="nU9WoWCYWa7Q0ZNrnOd1M/6u7oc=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSmZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLW+foAmUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9clkd4=</latexit><latexit sha1_base64="nU9WoWCYWa7Q0ZNrnOd1M/6u7oc=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSmZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLW+foAmUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9clkd4=</latexit><latexit sha1_base64="nU9WoWCYWa7Q0ZNrnOd1M/6u7oc=">AAAB+HicbVDLSsNAFJ3UV62PRl26GSxCRSmZIiiuCm5cVugL2hgm00k7dDIJMxOhhnyJGxeKuPVT3Pk3TtsstPXAhcM593LvPX7MmdKO820V1tY3NreK26Wd3b39sn1w2FFRIgltk4hHsudjRTkTtK2Z5rQXS4pDn9OuP7md+d1HKhWLREtPY+qGeCRYwAjWRvLs8thLW+foAmUPabV+lnl2xak5c8BVgnJSATmanv01GEYkCanQhGOl+siJtZtiqRnhNCsNEkVjTCZ4RPuGChxS5abzwzN4apQhDCJpSmg4V39PpDhUahr6pjPEeqyWvZn4n9dPdHDtpkzEiaaCLBYFCYc6grMU4JBJSjSfGoKJZOZWSMZYYqJNViUTAlp+eZV06jXk1ND9ZaVxk8dRBMfgBFQBAlegAe5AE7QBAQl4Bq/gzXqyXqx362PRWrDymSPwB9bnD9clkd4=</latexit>
History repackaging corresponds to a kind of non-zero history padding in TrellisNet (and TCNs in general). <latexit sha1_base64="7bB0qr+84xms1mTu0nN0C+zf8sE=">AAACZHicdVHPS9xAFJ6kttqtrbHSU0EeXQr20JCsuvEo9eJJFHZV2CzLZPKyO+xkJsxMhG3wn+ytx178O5ysKbSlPhh4fD94732TVYIbG0U/Pf/FxstXm1uve2+2377bCXbfXxtVa4ZjpoTStxk1KLjEseVW4G2lkZaZwJtsedbyN3eoDVdyZFcVTks6l7zgjFoHzYImlYrLHKWFczdN6RVorChb0jmXc2BKazSVkrkBqyBNexSWTg+qAKnk1++oFSw6Y0XzvDVx2QpHGoU74AItHFDnGJ1dmJaao0RNxZcQZkE/CpPk6Cg5hCgcDI+Hw8Q18clhEg8gDqN19UlXl7PgR5orVpduWyaoMZM4quy0odpyJvC+l9am2x0nrpW0RDNt1iHdw2eH5FAo7Z67do3+6WhoacyqzJyypHZh/uVa8H/cpLbFybThsqotSvY0qKhFm1ebOORcI7Ni5RrKNHe7AltQTZl1/9JzIfy+FJ5vrgdh7JK5GvRPv3VxbJGP5BM5IDFJyCk5J5dkTBj55W16gbfrPfjb/p7/4Unqe51nj/xV/v4jo3a2zw==</latexit><latexit sha1_base64="7bB0qr+84xms1mTu0nN0C+zf8sE=">AAACZHicdVHPS9xAFJ6kttqtrbHSU0EeXQr20JCsuvEo9eJJFHZV2CzLZPKyO+xkJsxMhG3wn+ytx178O5ysKbSlPhh4fD94732TVYIbG0U/Pf/FxstXm1uve2+2377bCXbfXxtVa4ZjpoTStxk1KLjEseVW4G2lkZaZwJtsedbyN3eoDVdyZFcVTks6l7zgjFoHzYImlYrLHKWFczdN6RVorChb0jmXc2BKazSVkrkBqyBNexSWTg+qAKnk1++oFSw6Y0XzvDVx2QpHGoU74AItHFDnGJ1dmJaao0RNxZcQZkE/CpPk6Cg5hCgcDI+Hw8Q18clhEg8gDqN19UlXl7PgR5orVpduWyaoMZM4quy0odpyJvC+l9am2x0nrpW0RDNt1iHdw2eH5FAo7Z67do3+6WhoacyqzJyypHZh/uVa8H/cpLbFybThsqotSvY0qKhFm1ebOORcI7Ni5RrKNHe7AltQTZl1/9JzIfy+FJ5vrgdh7JK5GvRPv3VxbJGP5BM5IDFJyCk5J5dkTBj55W16gbfrPfjb/p7/4Unqe51nj/xV/v4jo3a2zw==</latexit><latexit sha1_base64="7bB0qr+84xms1mTu0nN0C+zf8sE=">AAACZHicdVHPS9xAFJ6kttqtrbHSU0EeXQr20JCsuvEo9eJJFHZV2CzLZPKyO+xkJsxMhG3wn+ytx178O5ysKbSlPhh4fD94732TVYIbG0U/Pf/FxstXm1uve2+2377bCXbfXxtVa4ZjpoTStxk1KLjEseVW4G2lkZaZwJtsedbyN3eoDVdyZFcVTks6l7zgjFoHzYImlYrLHKWFczdN6RVorChb0jmXc2BKazSVkrkBqyBNexSWTg+qAKnk1++oFSw6Y0XzvDVx2QpHGoU74AItHFDnGJ1dmJaao0RNxZcQZkE/CpPk6Cg5hCgcDI+Hw8Q18clhEg8gDqN19UlXl7PgR5orVpduWyaoMZM4quy0odpyJvC+l9am2x0nrpW0RDNt1iHdw2eH5FAo7Z67do3+6WhoacyqzJyypHZh/uVa8H/cpLbFybThsqotSvY0qKhFm1ebOORcI7Ni5RrKNHe7AltQTZl1/9JzIfy+FJ5vrgdh7JK5GvRPv3VxbJGP5BM5IDFJyCk5J5dkTBj55W16gbfrPfjb/p7/4Unqe51nj/xV/v4jo3a2zw==</latexit><latexit sha1_base64="7bB0qr+84xms1mTu0nN0C+zf8sE=">AAACZHicdVHPS9xAFJ6kttqtrbHSU0EeXQr20JCsuvEo9eJJFHZV2CzLZPKyO+xkJsxMhG3wn+ytx178O5ysKbSlPhh4fD94732TVYIbG0U/Pf/FxstXm1uve2+2377bCXbfXxtVa4ZjpoTStxk1KLjEseVW4G2lkZaZwJtsedbyN3eoDVdyZFcVTks6l7zgjFoHzYImlYrLHKWFczdN6RVorChb0jmXc2BKazSVkrkBqyBNexSWTg+qAKnk1++oFSw6Y0XzvDVx2QpHGoU74AItHFDnGJ1dmJaao0RNxZcQZkE/CpPk6Cg5hCgcDI+Hw8Q18clhEg8gDqN19UlXl7PgR5orVpduWyaoMZM4quy0odpyJvC+l9am2x0nrpW0RDNt1iHdw2eH5FAo7Z67do3+6WhoacyqzJyypHZh/uVa8H/cpLbFybThsqotSvY0qKhFm1ebOORcI7Ni5RrKNHe7AltQTZl1/9JzIfy+FJ5vrgdh7JK5GvRPv3VxbJGP5BM5IDFJyCk5J5dkTBj55W16gbfrPfjb/p7/4Unqe51nj/xV/v4jo3a2zw==</latexit>
(b) History repackaging in mixed group convolutions, where we write out zt explicitly by Eq. (6).
Figure 6: Using the equivalence established by Theorem 1, we can transfer the notion of history repackaging in recurrent networks to trellis networks.
In Section 4, we formally described the relationship between TrellisNets, RNNs, and temporal convolutional networks (TCN). On the one hand, TrellisNet is a special TCN (with weight-tying and input injection), while on the other hand it can also express any structured RNN via a sparse convolutional kernel. These relationships open clear paths for applying techniques developed for either recurrent or convolutional networks. We summarize below some of the techniques that can be applied in this way to TrellisNet, categorizing them as either inspired by RNNs or TCNs.

heading: B.1 FROM RECURRENT NETWORKS
text: History repackaging. One theoretical advantage of RNNs is their ability to represent a history of infinite length. However, in many applications, sequence lengths are too long for infinite backpropagation during training. A typical solution is to partition the sequence into smaller subsequences and perform truncated backpropagation through time (BPTT) on each. At sequence boundaries, the hidden state ht is “repackaged” and passed onto the next RNN sequence. Thus gradient flow stops at sequence boundaries (see Figure 6a). Such repackaging is also sometimes used at test time.
We can now map this repackaging procedure to trellis networks. As shown in Figure 6, the notion of passing the compressed history vector ht in an RNN corresponds to specific non-zero padding in the mixed group convolution of the corresponding TrellisNet. The padding is simply the channels from the last step of the final layer applied on the previous sequence (see Figure 6b, where without the repackaging padding, at layer 2 we will have h(1)T+1,T+1 instead of h (1) T+1,1). We illustrate this in Figure 6b, where we have written out z(i)t in TrellisNet explicitly in the form of ht,t′ according to Eq. (6). This suggests that instead of storing all effective history in memory, we can compress history in a feed-forward network to extend its history as well. For a general TrellisNet that employs a dense kernel, similarly, we can pass the hidden channels of the last step of the final layer in the previous sequence as the “history” padding for the next TrellisNet sequence (this works in both training and testing).
Gated activations. In general, the structured gates in RNN cells can be translated to gated activations in temporal convolutions, as we did in Appendix A in the case of an LSTM. While in the experiments we adopted the LSTM gating, other activations (e.g. GRUs (Cho et al., 2014) or activations found via architecture search (Zoph & Le, 2017)) can also be applied in trellis networks via the equivalence established in Theorem 1.
RNN variational dropout. Variational dropout (VD) for RNNs (Gal & Ghahramani, 2016) is a useful regularization scheme that applies the same mask at every time step within a layer (see Figure 7a). A direct translation of this technique from RNN to the group temporal convolution implies that we need to create a different mask for each diagonal of the network (i.e. each history starting point), as well as for each group of the mixed group convolution. We propose an alternative (and extremely simple) dropout scheme for TrellisNet, which is inspired by VD in RNNs as well as Theo-
Published as a conference paper at ICLR 2019
x1 <latexit sha1_base64="Gx1rB9K2pA68P1aOYP+U250HTz0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK9099r1+uuFV3DrJKvJxUIEejX/7qDWKWRigNE1Trrucmxs+oMpwJnJZ6qcaEsjEdYtdSSSPUfjY/dUrOrDIgYaxsSUPm6u+JjEZaT6LAdkbUjPSyNxP/87qpCa/8jMskNSjZYlGYCmJiMvubDLhCZsTEEsoUt7cSNqKKMmPTKdkQvOWXV0nrouq5Ve/uslK/zuMowgmcwjl4UIM63EIDmsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wcJSI2Y</latexit><latexit sha1_base64="Gx1rB9K2pA68P1aOYP+U250HTz0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK9099r1+uuFV3DrJKvJxUIEejX/7qDWKWRigNE1Trrucmxs+oMpwJnJZ6qcaEsjEdYtdSSSPUfjY/dUrOrDIgYaxsSUPm6u+JjEZaT6LAdkbUjPSyNxP/87qpCa/8jMskNSjZYlGYCmJiMvubDLhCZsTEEsoUt7cSNqKKMmPTKdkQvOWXV0nrouq5Ve/uslK/zuMowgmcwjl4UIM63EIDmsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wcJSI2Y</latexit><latexit sha1_base64="Gx1rB9K2pA68P1aOYP+U250HTz0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK9099r1+uuFV3DrJKvJxUIEejX/7qDWKWRigNE1Trrucmxs+oMpwJnJZ6qcaEsjEdYtdSSSPUfjY/dUrOrDIgYaxsSUPm6u+JjEZaT6LAdkbUjPSyNxP/87qpCa/8jMskNSjZYlGYCmJiMvubDLhCZsTEEsoUt7cSNqKKMmPTKdkQvOWXV0nrouq5Ve/uslK/zuMowgmcwjl4UIM63EIDmsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wcJSI2Y</latexit><latexit sha1_base64="Gx1rB9K2pA68P1aOYP+U250HTz0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK9099r1+uuFV3DrJKvJxUIEejX/7qDWKWRigNE1Trrucmxs+oMpwJnJZ6qcaEsjEdYtdSSSPUfjY/dUrOrDIgYaxsSUPm6u+JjEZaT6LAdkbUjPSyNxP/87qpCa/8jMskNSjZYlGYCmJiMvubDLhCZsTEEsoUt7cSNqKKMmPTKdkQvOWXV0nrouq5Ve/uslK/zuMowgmcwjl4UIM63EIDmsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wcJSI2Y</latexit> x2 <latexit sha1_base64="958LDsa5w4EedPbIxPcFgVddcGo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5JWreq5Ve/uolK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8KzI2Z</latexit><latexit sha1_base64="958LDsa5w4EedPbIxPcFgVddcGo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5JWreq5Ve/uolK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8KzI2Z</latexit><latexit sha1_base64="958LDsa5w4EedPbIxPcFgVddcGo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5JWreq5Ve/uolK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8KzI2Z</latexit><latexit sha1_base64="958LDsa5w4EedPbIxPcFgVddcGo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5JWreq5Ve/uolK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8KzI2Z</latexit> x3 <latexit sha1_base64="FsFp5SDaWDjvwmxpdt2PBoWBq3Q=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lUUDwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1znvlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97dRaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8MUI2a</latexit><latexit sha1_base64="FsFp5SDaWDjvwmxpdt2PBoWBq3Q=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lUUDwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1znvlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97dRaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8MUI2a</latexit><latexit sha1_base64="FsFp5SDaWDjvwmxpdt2PBoWBq3Q=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lUUDwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1znvlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97dRaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8MUI2a</latexit><latexit sha1_base64="FsFp5SDaWDjvwmxpdt2PBoWBq3Q=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lUUDwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1znvlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97dRaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8MUI2a</latexit>
0 <latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit>
0 <latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit> x1 <latexit sha1_base64="Gx1rB9K2pA68P1aOYP+U250HTz0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK9099r1+uuFV3DrJKvJxUIEejX/7qDWKWRigNE1Trrucmxs+oMpwJnJZ6qcaEsjEdYtdSSSPUfjY/dUrOrDIgYaxsSUPm6u+JjEZaT6LAdkbUjPSyNxP/87qpCa/8jMskNSjZYlGYCmJiMvubDLhCZsTEEsoUt7cSNqKKMmPTKdkQvOWXV0nrouq5Ve/uslK/zuMowgmcwjl4UIM63EIDmsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wcJSI2Y</latexit><latexit sha1_base64="Gx1rB9K2pA68P1aOYP+U250HTz0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK9099r1+uuFV3DrJKvJxUIEejX/7qDWKWRigNE1Trrucmxs+oMpwJnJZ6qcaEsjEdYtdSSSPUfjY/dUrOrDIgYaxsSUPm6u+JjEZaT6LAdkbUjPSyNxP/87qpCa/8jMskNSjZYlGYCmJiMvubDLhCZsTEEsoUt7cSNqKKMmPTKdkQvOWXV0nrouq5Ve/uslK/zuMowgmcwjl4UIM63EIDmsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wcJSI2Y</latexit><latexit sha1_base64="Gx1rB9K2pA68P1aOYP+U250HTz0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK9099r1+uuFV3DrJKvJxUIEejX/7qDWKWRigNE1Trrucmxs+oMpwJnJZ6qcaEsjEdYtdSSSPUfjY/dUrOrDIgYaxsSUPm6u+JjEZaT6LAdkbUjPSyNxP/87qpCa/8jMskNSjZYlGYCmJiMvubDLhCZsTEEsoUt7cSNqKKMmPTKdkQvOWXV0nrouq5Ve/uslK/zuMowgmcwjl4UIM63EIDmsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wcJSI2Y</latexit><latexit sha1_base64="Gx1rB9K2pA68P1aOYP+U250HTz0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK9099r1+uuFV3DrJKvJxUIEejX/7qDWKWRigNE1Trrucmxs+oMpwJnJZ6qcaEsjEdYtdSSSPUfjY/dUrOrDIgYaxsSUPm6u+JjEZaT6LAdkbUjPSyNxP/87qpCa/8jMskNSjZYlGYCmJiMvubDLhCZsTEEsoUt7cSNqKKMmPTKdkQvOWXV0nrouq5Ve/uslK/zuMowgmcwjl4UIM63EIDmsBgCM/wCm+OcF6cd+dj0Vpw8plj+APn8wcJSI2Y</latexit> x2 <latexit sha1_base64="958LDsa5w4EedPbIxPcFgVddcGo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5JWreq5Ve/uolK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8KzI2Z</latexit><latexit sha1_base64="958LDsa5w4EedPbIxPcFgVddcGo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5JWreq5Ve/uolK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8KzI2Z</latexit><latexit sha1_base64="958LDsa5w4EedPbIxPcFgVddcGo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5JWreq5Ve/uolK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8KzI2Z</latexit><latexit sha1_base64="958LDsa5w4EedPbIxPcFgVddcGo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5JWreq5Ve/uolK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8KzI2Z</latexit> x3 <latexit sha1_base64="FsFp5SDaWDjvwmxpdt2PBoWBq3Q=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lUUDwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1znvlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97dRaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8MUI2a</latexit><latexit sha1_base64="FsFp5SDaWDjvwmxpdt2PBoWBq3Q=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lUUDwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1znvlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97dRaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8MUI2a</latexit><latexit sha1_base64="FsFp5SDaWDjvwmxpdt2PBoWBq3Q=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lUUDwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1znvlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97dRaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8MUI2a</latexit><latexit sha1_base64="FsFp5SDaWDjvwmxpdt2PBoWBq3Q=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lUUDwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1znvlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97dRaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8MUI2a</latexit>
0 <latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit>
0 <latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit><latexit sha1_base64="+kYPchScNyik1n1ulpTEzE1fdeA=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePHYgv2ANpTNdtKu3WzC7kYoob/AiwdFvPqTvPlv3LY5aOuDgcd7M8zMCxLBtXHdb6ewsbm1vVPcLe3tHxwelY9P2jpOFcMWi0WsugHVKLjEluFGYDdRSKNAYCeY3M39zhMqzWP5YKYJ+hEdSR5yRo2Vmu6gXHGr7gJknXg5qUCOxqD81R/GLI1QGiao1j3PTYyfUWU4Ezgr9VONCWUTOsKepZJGqP1sceiMXFhlSMJY2ZKGLNTfExmNtJ5Gge2MqBnrVW8u/uf1UhPe+BmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te87pSv83jKMIZnMMleFCDOtxDA1rAAOEZXuHNeXRenHfnY9lacPKZU/gD5/MHdrGMrA==</latexit>
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
Mask 1 <latexit sha1_base64="4ZHyCx04sxNSGjgTq8r6BUxDtgU=">AAAB7XicbVA9SwNBEJ2LXzF+RS1tFoNgFe7SmDJgYyNEMImQHGFvM5es2ds9dveEEPIfbCwUsfX/2Plv3CRXaOKDgcd7M8zMi1LBjfX9b6+wsbm1vVPcLe3tHxwelY9P2kZlmmGLKaH0Q0QNCi6xZbkV+JBqpEkksBONr+d+5wm14Ure20mKYUKHksecUeuk9i01YxL0yxW/6i9A1kmQkwrkaPbLX72BYlmC0jJBjekGfmrDKdWWM4GzUi8zmFI2pkPsOippgiacLq6dkQunDEistCtpyUL9PTGliTGTJHKdCbUjs+rNxf+8bmbjejjlMs0sSrZcFGeCWEXmr5MB18ismDhCmebuVsJGVFNmXUAlF0Kw+vI6adeqgV8N7mqVRj2PowhncA6XEMAVNOAGmtACBo/wDK/w5invxXv3PpatBS+fOYU/8D5/AL20joc=</latexit><latexit sha1_base64="4ZHyCx04sxNSGjgTq8r6BUxDtgU=">AAAB7XicbVA9SwNBEJ2LXzF+RS1tFoNgFe7SmDJgYyNEMImQHGFvM5es2ds9dveEEPIfbCwUsfX/2Plv3CRXaOKDgcd7M8zMi1LBjfX9b6+wsbm1vVPcLe3tHxwelY9P2kZlmmGLKaH0Q0QNCi6xZbkV+JBqpEkksBONr+d+5wm14Ure20mKYUKHksecUeuk9i01YxL0yxW/6i9A1kmQkwrkaPbLX72BYlmC0jJBjekGfmrDKdWWM4GzUi8zmFI2pkPsOippgiacLq6dkQunDEistCtpyUL9PTGliTGTJHKdCbUjs+rNxf+8bmbjejjlMs0sSrZcFGeCWEXmr5MB18ismDhCmebuVsJGVFNmXUAlF0Kw+vI6adeqgV8N7mqVRj2PowhncA6XEMAVNOAGmtACBo/wDK/w5invxXv3PpatBS+fOYU/8D5/AL20joc=</latexit><latexit sha1_base64="4ZHyCx04sxNSGjgTq8r6BUxDtgU=">AAAB7XicbVA9SwNBEJ2LXzF+RS1tFoNgFe7SmDJgYyNEMImQHGFvM5es2ds9dveEEPIfbCwUsfX/2Plv3CRXaOKDgcd7M8zMi1LBjfX9b6+wsbm1vVPcLe3tHxwelY9P2kZlmmGLKaH0Q0QNCi6xZbkV+JBqpEkksBONr+d+5wm14Ure20mKYUKHksecUeuk9i01YxL0yxW/6i9A1kmQkwrkaPbLX72BYlmC0jJBjekGfmrDKdWWM4GzUi8zmFI2pkPsOippgiacLq6dkQunDEistCtpyUL9PTGliTGTJHKdCbUjs+rNxf+8bmbjejjlMs0sSrZcFGeCWEXmr5MB18ismDhCmebuVsJGVFNmXUAlF0Kw+vI6adeqgV8N7mqVRj2PowhncA6XEMAVNOAGmtACBo/wDK/w5invxXv3PpatBS+fOYU/8D5/AL20joc=</latexit><latexit sha1_base64="4ZHyCx04sxNSGjgTq8r6BUxDtgU=">AAAB7XicbVA9SwNBEJ2LXzF+RS1tFoNgFe7SmDJgYyNEMImQHGFvM5es2ds9dveEEPIfbCwUsfX/2Plv3CRXaOKDgcd7M8zMi1LBjfX9b6+wsbm1vVPcLe3tHxwelY9P2kZlmmGLKaH0Q0QNCi6xZbkV+JBqpEkksBONr+d+5wm14Ure20mKYUKHksecUeuk9i01YxL0yxW/6i9A1kmQkwrkaPbLX72BYlmC0jJBjekGfmrDKdWWM4GzUi8zmFI2pkPsOippgiacLq6dkQunDEistCtpyUL9PTGliTGTJHKdCbUjs+rNxf+8bmbjejjlMs0sSrZcFGeCWEXmr5MB18ismDhCmebuVsJGVFNmXUAlF0Kw+vI6adeqgV8N7mqVRj2PowhncA6XEMAVNOAGmtACBo/wDK/w5invxXv3PpatBS+fOYU/8D5/AL20joc=</latexit>
Mask 2 <latexit sha1_base64="OPSDj8Q5IVTfQC+O+npTy+6yjVo=">AAAB7XicbVA9SwNBEJ2LXzF+RS1tFoNgFe7SmDJgYyNEMImQHGFvM5es2ds9dveEEPIfbCwUsfX/2Plv3CRXaOKDgcd7M8zMi1LBjfX9b6+wsbm1vVPcLe3tHxwelY9P2kZlmmGLKaH0Q0QNCi6xZbkV+JBqpEkksBONr+d+5wm14Ure20mKYUKHksecUeuk9i01Y1Lrlyt+1V+ArJMgJxXI0eyXv3oDxbIEpWWCGtMN/NSGU6otZwJnpV5mMKVsTIfYdVTSBE04XVw7IxdOGZBYaVfSkoX6e2JKE2MmSeQ6E2pHZtWbi/953czG9XDKZZpZlGy5KM4EsYrMXycDrpFZMXGEMs3drYSNqKbMuoBKLoRg9eV10q5VA78a3NUqjXoeRxHO4BwuIYAraMANNKEFDB7hGV7hzVPei/fufSxbC14+cwp/4H3+AL84jog=</latexit><latexit sha1_base64="OPSDj8Q5IVTfQC+O+npTy+6yjVo=">AAAB7XicbVA9SwNBEJ2LXzF+RS1tFoNgFe7SmDJgYyNEMImQHGFvM5es2ds9dveEEPIfbCwUsfX/2Plv3CRXaOKDgcd7M8zMi1LBjfX9b6+wsbm1vVPcLe3tHxwelY9P2kZlmmGLKaH0Q0QNCi6xZbkV+JBqpEkksBONr+d+5wm14Ure20mKYUKHksecUeuk9i01Y1Lrlyt+1V+ArJMgJxXI0eyXv3oDxbIEpWWCGtMN/NSGU6otZwJnpV5mMKVsTIfYdVTSBE04XVw7IxdOGZBYaVfSkoX6e2JKE2MmSeQ6E2pHZtWbi/953czG9XDKZZpZlGy5KM4EsYrMXycDrpFZMXGEMs3drYSNqKbMuoBKLoRg9eV10q5VA78a3NUqjXoeRxHO4BwuIYAraMANNKEFDB7hGV7hzVPei/fufSxbC14+cwp/4H3+AL84jog=</latexit><latexit sha1_base64="OPSDj8Q5IVTfQC+O+npTy+6yjVo=">AAAB7XicbVA9SwNBEJ2LXzF+RS1tFoNgFe7SmDJgYyNEMImQHGFvM5es2ds9dveEEPIfbCwUsfX/2Plv3CRXaOKDgcd7M8zMi1LBjfX9b6+wsbm1vVPcLe3tHxwelY9P2kZlmmGLKaH0Q0QNCi6xZbkV+JBqpEkksBONr+d+5wm14Ure20mKYUKHksecUeuk9i01Y1Lrlyt+1V+ArJMgJxXI0eyXv3oDxbIEpWWCGtMN/NSGU6otZwJnpV5mMKVsTIfYdVTSBE04XVw7IxdOGZBYaVfSkoX6e2JKE2MmSeQ6E2pHZtWbi/953czG9XDKZZpZlGy5KM4EsYrMXycDrpFZMXGEMs3drYSNqKbMuoBKLoRg9eV10q5VA78a3NUqjXoeRxHO4BwuIYAraMANNKEFDB7hGV7hzVPei/fufSxbC14+cwp/4H3+AL84jog=</latexit><latexit sha1_base64="OPSDj8Q5IVTfQC+O+npTy+6yjVo=">AAAB7XicbVA9SwNBEJ2LXzF+RS1tFoNgFe7SmDJgYyNEMImQHGFvM5es2ds9dveEEPIfbCwUsfX/2Plv3CRXaOKDgcd7M8zMi1LBjfX9b6+wsbm1vVPcLe3tHxwelY9P2kZlmmGLKaH0Q0QNCi6xZbkV+JBqpEkksBONr+d+5wm14Ure20mKYUKHksecUeuk9i01Y1Lrlyt+1V+ArJMgJxXI0eyXv3oDxbIEpWWCGtMN/NSGU6otZwJnpV5mMKVsTIfYdVTSBE04XVw7IxdOGZBYaVfSkoX6e2JKE2MmSeQ6E2pHZtWbi/953czG9XDKZZpZlGy5KM4EsYrMXycDrpFZMXGEMs3drYSNqKbMuoBKLoRg9eV10q5VA78a3NUqjXoeRxHO4BwuIYAraMANNKEFDB7hGV7hzVPei/fufSxbC14+cwp/4H3+AL84jog=</latexit>
Mask 3 <latexit sha1_base64="72z96cMDFAHDlZhDIp7SM9bZr0o=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqezWgz0WvHgRKtgPaJeSTbNtbDZZkqxQlv4HLx4U8er/8ea/MW33oK0PBh7vzTAzL0wEN9bzvlFhY3Nre6e4W9rbPzg8Kh+ftI1KNWUtqoTS3ZAYJrhkLcutYN1EMxKHgnXCyc3c7zwxbbiSD3aasCAmI8kjTol1UvuOmAm+GpQrXtVbAK8TPycVyNEclL/6Q0XTmElLBTGm53uJDTKiLaeCzUr91LCE0AkZsZ6jksTMBNni2hm+cMoQR0q7khYv1N8TGYmNmcah64yJHZtVby7+5/VSG9WDjMsktUzS5aIoFdgqPH8dD7lm1IqpI4Rq7m7FdEw0odYFVHIh+Ksvr5N2rep7Vf++VmnU8ziKcAbncAk+XEMDbqEJLaDwCM/wCm9IoRf0jj6WrQWUz5zCH6DPH8C8jok=</latexit><latexit sha1_base64="72z96cMDFAHDlZhDIp7SM9bZr0o=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqezWgz0WvHgRKtgPaJeSTbNtbDZZkqxQlv4HLx4U8er/8ea/MW33oK0PBh7vzTAzL0wEN9bzvlFhY3Nre6e4W9rbPzg8Kh+ftI1KNWUtqoTS3ZAYJrhkLcutYN1EMxKHgnXCyc3c7zwxbbiSD3aasCAmI8kjTol1UvuOmAm+GpQrXtVbAK8TPycVyNEclL/6Q0XTmElLBTGm53uJDTKiLaeCzUr91LCE0AkZsZ6jksTMBNni2hm+cMoQR0q7khYv1N8TGYmNmcah64yJHZtVby7+5/VSG9WDjMsktUzS5aIoFdgqPH8dD7lm1IqpI4Rq7m7FdEw0odYFVHIh+Ksvr5N2rep7Vf++VmnU8ziKcAbncAk+XEMDbqEJLaDwCM/wCm9IoRf0jj6WrQWUz5zCH6DPH8C8jok=</latexit><latexit sha1_base64="72z96cMDFAHDlZhDIp7SM9bZr0o=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqezWgz0WvHgRKtgPaJeSTbNtbDZZkqxQlv4HLx4U8er/8ea/MW33oK0PBh7vzTAzL0wEN9bzvlFhY3Nre6e4W9rbPzg8Kh+ftI1KNWUtqoTS3ZAYJrhkLcutYN1EMxKHgnXCyc3c7zwxbbiSD3aasCAmI8kjTol1UvuOmAm+GpQrXtVbAK8TPycVyNEclL/6Q0XTmElLBTGm53uJDTKiLaeCzUr91LCE0AkZsZ6jksTMBNni2hm+cMoQR0q7khYv1N8TGYmNmcah64yJHZtVby7+5/VSG9WDjMsktUzS5aIoFdgqPH8dD7lm1IqpI4Rq7m7FdEw0odYFVHIh+Ksvr5N2rep7Vf++VmnU8ziKcAbncAk+XEMDbqEJLaDwCM/wCm9IoRf0jj6WrQWUz5zCH6DPH8C8jok=</latexit><latexit sha1_base64="72z96cMDFAHDlZhDIp7SM9bZr0o=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqezWgz0WvHgRKtgPaJeSTbNtbDZZkqxQlv4HLx4U8er/8ea/MW33oK0PBh7vzTAzL0wEN9bzvlFhY3Nre6e4W9rbPzg8Kh+ftI1KNWUtqoTS3ZAYJrhkLcutYN1EMxKHgnXCyc3c7zwxbbiSD3aasCAmI8kjTol1UvuOmAm+GpQrXtVbAK8TPycVyNEclL/6Q0XTmElLBTGm53uJDTKiLaeCzUr91LCE0AkZsZ6jksTMBNni2hm+cMoQR0q7khYv1N8TGYmNmcah64yJHZtVby7+5/VSG9WDjMsktUzS5aIoFdgqPH8dD7lm1IqpI4Rq7m7FdEw0odYFVHIh+Ksvr5N2rep7Vf++VmnU8ziKcAbncAk+XEMDbqEJLaDwCM/wCm9IoRf0jj6WrQWUz5zCH6DPH8C8jok=</latexit>
Mask 4 <latexit sha1_base64="DVx06ueRSIXHgwswu8ctZcXDE/k=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqewWwR4LXrwIFewHtEvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZFyaCG+t536iwsbm1vVPcLe3tHxwelY9P2kalmrIWVULpbkgME1yyluVWsG6iGYlDwTrh5Gbud56YNlzJBztNWBCTkeQRp8Q6qX1HzARfDcoVr+otgNeJn5MK5GgOyl/9oaJpzKSlghjT873EBhnRllPBZqV+alhC6ISMWM9RSWJmgmxx7QxfOGWII6VdSYsX6u+JjMTGTOPQdcbEjs2qNxf/83qpjepBxmWSWibpclGUCmwVnr+Oh1wzasXUEUI1d7diOiaaUOsCKrkQ/NWX10m7VvW9qn9fqzTqeRxFOINzuAQfrqEBt9CEFlB4hGd4hTek0At6Rx/L1gLKZ07hD9DnD8JAjoo=</latexit><latexit sha1_base64="DVx06ueRSIXHgwswu8ctZcXDE/k=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqewWwR4LXrwIFewHtEvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZFyaCG+t536iwsbm1vVPcLe3tHxwelY9P2kalmrIWVULpbkgME1yyluVWsG6iGYlDwTrh5Gbud56YNlzJBztNWBCTkeQRp8Q6qX1HzARfDcoVr+otgNeJn5MK5GgOyl/9oaJpzKSlghjT873EBhnRllPBZqV+alhC6ISMWM9RSWJmgmxx7QxfOGWII6VdSYsX6u+JjMTGTOPQdcbEjs2qNxf/83qpjepBxmWSWibpclGUCmwVnr+Oh1wzasXUEUI1d7diOiaaUOsCKrkQ/NWX10m7VvW9qn9fqzTqeRxFOINzuAQfrqEBt9CEFlB4hGd4hTek0At6Rx/L1gLKZ07hD9DnD8JAjoo=</latexit><latexit sha1_base64="DVx06ueRSIXHgwswu8ctZcXDE/k=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqewWwR4LXrwIFewHtEvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZFyaCG+t536iwsbm1vVPcLe3tHxwelY9P2kalmrIWVULpbkgME1yyluVWsG6iGYlDwTrh5Gbud56YNlzJBztNWBCTkeQRp8Q6qX1HzARfDcoVr+otgNeJn5MK5GgOyl/9oaJpzKSlghjT873EBhnRllPBZqV+alhC6ISMWM9RSWJmgmxx7QxfOGWII6VdSYsX6u+JjMTGTOPQdcbEjs2qNxf/83qpjepBxmWSWibpclGUCmwVnr+Oh1wzasXUEUI1d7diOiaaUOsCKrkQ/NWX10m7VvW9qn9fqzTqeRxFOINzuAQfrqEBt9CEFlB4hGd4hTek0At6Rx/L1gLKZ07hD9DnD8JAjoo=</latexit><latexit sha1_base64="DVx06ueRSIXHgwswu8ctZcXDE/k=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqewWwR4LXrwIFewHtEvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZFyaCG+t536iwsbm1vVPcLe3tHxwelY9P2kalmrIWVULpbkgME1yyluVWsG6iGYlDwTrh5Gbud56YNlzJBztNWBCTkeQRp8Q6qX1HzARfDcoVr+otgNeJn5MK5GgOyl/9oaJpzKSlghjT873EBhnRllPBZqV+alhC6ISMWM9RSWJmgmxx7QxfOGWII6VdSYsX6u+JjMTGTOPQdcbEjs2qNxf/83qpjepBxmWSWibpclGUCmwVnr+Oh1wzasXUEUI1d7diOiaaUOsCKrkQ/NWX10m7VvW9qn9fqzTqeRxFOINzuAQfrqEBt9CEFlB4hGd4hTek0At6Rx/L1gLKZ07hD9DnD8JAjoo=</latexit>
Mask 5 <latexit sha1_base64="ocIS2tcIMuYbgr/+l7ECrgU4J+Y=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqewWxB4LXrwIFewHtEvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZFyaCG+t536iwsbm1vVPcLe3tHxwelY9P2kalmrIWVULpbkgME1yyluVWsG6iGYlDwTrh5Gbud56YNlzJBztNWBCTkeQRp8Q6qX1HzARfDcoVr+otgNeJn5MK5GgOyl/9oaJpzKSlghjT873EBhnRllPBZqV+alhC6ISMWM9RSWJmgmxx7QxfOGWII6VdSYsX6u+JjMTGTOPQdcbEjs2qNxf/83qpjepBxmWSWibpclGUCmwVnr+Oh1wzasXUEUI1d7diOiaaUOsCKrkQ/NWX10m7VvW9qn9fqzTqeRxFOINzuAQfrqEBt9CEFlB4hGd4hTek0At6Rx/L1gLKZ07hD9DnD8PEjos=</latexit><latexit sha1_base64="ocIS2tcIMuYbgr/+l7ECrgU4J+Y=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqewWxB4LXrwIFewHtEvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZFyaCG+t536iwsbm1vVPcLe3tHxwelY9P2kalmrIWVULpbkgME1yyluVWsG6iGYlDwTrh5Gbud56YNlzJBztNWBCTkeQRp8Q6qX1HzARfDcoVr+otgNeJn5MK5GgOyl/9oaJpzKSlghjT873EBhnRllPBZqV+alhC6ISMWM9RSWJmgmxx7QxfOGWII6VdSYsX6u+JjMTGTOPQdcbEjs2qNxf/83qpjepBxmWSWibpclGUCmwVnr+Oh1wzasXUEUI1d7diOiaaUOsCKrkQ/NWX10m7VvW9qn9fqzTqeRxFOINzuAQfrqEBt9CEFlB4hGd4hTek0At6Rx/L1gLKZ07hD9DnD8PEjos=</latexit><latexit sha1_base64="ocIS2tcIMuYbgr/+l7ECrgU4J+Y=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqewWxB4LXrwIFewHtEvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZFyaCG+t536iwsbm1vVPcLe3tHxwelY9P2kalmrIWVULpbkgME1yyluVWsG6iGYlDwTrh5Gbud56YNlzJBztNWBCTkeQRp8Q6qX1HzARfDcoVr+otgNeJn5MK5GgOyl/9oaJpzKSlghjT873EBhnRllPBZqV+alhC6ISMWM9RSWJmgmxx7QxfOGWII6VdSYsX6u+JjMTGTOPQdcbEjs2qNxf/83qpjepBxmWSWibpclGUCmwVnr+Oh1wzasXUEUI1d7diOiaaUOsCKrkQ/NWX10m7VvW9qn9fqzTqeRxFOINzuAQfrqEBt9CEFlB4hGd4hTek0At6Rx/L1gLKZ07hD9DnD8PEjos=</latexit><latexit sha1_base64="ocIS2tcIMuYbgr/+l7ECrgU4J+Y=">AAAB7XicbVBNSwMxEJ3Ur1q/qh69BIvgqewWxB4LXrwIFewHtEvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZFyaCG+t536iwsbm1vVPcLe3tHxwelY9P2kalmrIWVULpbkgME1yyluVWsG6iGYlDwTrh5Gbud56YNlzJBztNWBCTkeQRp8Q6qX1HzARfDcoVr+otgNeJn5MK5GgOyl/9oaJpzKSlghjT873EBhnRllPBZqV+alhC6ISMWM9RSWJmgmxx7QxfOGWII6VdSYsX6u+JjMTGTOPQdcbEjs2qNxf/83qpjepBxmWSWibpclGUCmwVnr+Oh1wzasXUEUI1d7diOiaaUOsCKrkQ/NWX10m7VvW9qn9fqzTqeRxFOINzuAQfrqEBt9CEFlB4hGd4hTek0At6Rx/L1gLKZ07hD9DnD8PEjos=</latexit>
x4 <latexit sha1_base64="S4f+bq/VrUW0JYj64fGswbvelkw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5LWRdVzq95drVK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8N1I2b</latexit><latexit sha1_base64="S4f+bq/VrUW0JYj64fGswbvelkw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5LWRdVzq95drVK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8N1I2b</latexit><latexit sha1_base64="S4f+bq/VrUW0JYj64fGswbvelkw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5LWRdVzq95drVK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8N1I2b</latexit><latexit sha1_base64="S4f+bq/VrUW0JYj64fGswbvelkw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoHgqePFY0X5AG8pmO2mXbjZhdyOW0J/gxYMiXv1F3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2Z++xGV5rF8MJME/YgOJQ85o8ZK90/9Wr9ccavuHGSVeDmpQI5Gv/zVG8QsjVAaJqjWXc9NjJ9RZTgTOC31Uo0JZWM6xK6lkkao/Wx+6pScWWVAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE175GZdJalCyxaIwFcTEZPY3GXCFzIiJJZQpbm8lbEQVZcamU7IheMsvr5LWRdVzq95drVK/zuMowgmcwjl4cAl1uIUGNIHBEJ7hFd4c4bw4787HorXg5DPH8AfO5w8N1I2b</latexit> x5 <latexit sha1_base64="dPT//6RAyPZ9Ol0ZAjwg7n/hgME=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUTwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1Lnrlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97deaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8PWI2c</latexit><latexit sha1_base64="dPT//6RAyPZ9Ol0ZAjwg7n/hgME=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUTwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1Lnrlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97deaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8PWI2c</latexit><latexit sha1_base64="dPT//6RAyPZ9Ol0ZAjwg7n/hgME=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUTwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1Lnrlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97deaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8PWI2c</latexit><latexit sha1_base64="dPT//6RAyPZ9Ol0ZAjwg7n/hgME=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUTwVvHisaD+gDWWz3bRLN5uwOxFL6E/w4kERr/4ib/4bt20O2vpg4PHeDDPzgkQKg6777RRWVtfWN4qbpa3tnd298v5B08SpZrzBYhnrdkANl0LxBgqUvJ1oTqNA8lYwupn6rUeujYjVA44T7kd0oEQoGEUr3T/1Lnrlilt1ZyDLxMtJBXLUe+Wvbj9macQVMkmN6Xhugn5GNQom+aTUTQ1PKBvRAe9YqmjEjZ/NTp2QE6v0SRhrWwrJTP09kdHImHEU2M6I4tAselPxP6+TYnjlZ0IlKXLF5ovCVBKMyfRv0heaM5RjSyjTwt5K2JBqytCmU7IheIsvL5PmWdVzq97deaV2ncdRhCM4hlPw4BJqcAt1aACDATzDK7w50nlx3p2PeWvByWcO4Q+czx8PWI2c</latexit>
(a) Left: variational dropout (VD) in an RNN. Right: VD in a TrellisNet. Each color indicates a different dropout mask.
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
Auxiliary Loss Laux <latexit sha1_base64="WreFjTnxmoUXC5AhPcp14C4Aeo4=">AAACEHicdVC7TgJBFJ3FF+ILtbSZCEYrskujscLYWFBgIo8ENpu7wwATZh+ZuWsgGz7Bxl+xsdAYW0s7/8ZdwARfJ7nJyTn35t573FAKjab5YWSWlldW17LruY3Nre2d/O5eQweRYrzOAhmolguaS+HzOgqUvBUqDp4redMdXqZ+85YrLQL/Bschtz3o+6InGGAiOfnji2gkpAA1ptVAa1rseIADBjKuTpwO8hHGEI0mRSdfsErmFNT8Rb6sApmj5uTfO92ARR73kUnQum2ZIdoxKBRM8kmuE2keAhtCn7cT6oPHtR1PH5rQo0Tp0l6gkvKRTtXFiRg8rceem3Sm5+qfXir+5bUj7J3ZsfDDCLnPZot6kaQY0DQd2hWKM5TjhABTIrmVsgEoYJhkmFsM4X/SKJcss2RdlwuV83kcWXJADskJscgpqZArUiN1wsgdeSBP5Nm4Nx6NF+N11pox5jP75BuMt08yfZ1J</latexit><latexit sha1_base64="WreFjTnxmoUXC5AhPcp14C4Aeo4=">AAACEHicdVC7TgJBFJ3FF+ILtbSZCEYrskujscLYWFBgIo8ENpu7wwATZh+ZuWsgGz7Bxl+xsdAYW0s7/8ZdwARfJ7nJyTn35t573FAKjab5YWSWlldW17LruY3Nre2d/O5eQweRYrzOAhmolguaS+HzOgqUvBUqDp4redMdXqZ+85YrLQL/Bschtz3o+6InGGAiOfnji2gkpAA1ptVAa1rseIADBjKuTpwO8hHGEI0mRSdfsErmFNT8Rb6sApmj5uTfO92ARR73kUnQum2ZIdoxKBRM8kmuE2keAhtCn7cT6oPHtR1PH5rQo0Tp0l6gkvKRTtXFiRg8rceem3Sm5+qfXir+5bUj7J3ZsfDDCLnPZot6kaQY0DQd2hWKM5TjhABTIrmVsgEoYJhkmFsM4X/SKJcss2RdlwuV83kcWXJADskJscgpqZArUiN1wsgdeSBP5Nm4Nx6NF+N11pox5jP75BuMt08yfZ1J</latexit><latexit sha1_base64="WreFjTnxmoUXC5AhPcp14C4Aeo4=">AAACEHicdVC7TgJBFJ3FF+ILtbSZCEYrskujscLYWFBgIo8ENpu7wwATZh+ZuWsgGz7Bxl+xsdAYW0s7/8ZdwARfJ7nJyTn35t573FAKjab5YWSWlldW17LruY3Nre2d/O5eQweRYrzOAhmolguaS+HzOgqUvBUqDp4redMdXqZ+85YrLQL/Bschtz3o+6InGGAiOfnji2gkpAA1ptVAa1rseIADBjKuTpwO8hHGEI0mRSdfsErmFNT8Rb6sApmj5uTfO92ARR73kUnQum2ZIdoxKBRM8kmuE2keAhtCn7cT6oPHtR1PH5rQo0Tp0l6gkvKRTtXFiRg8rceem3Sm5+qfXir+5bUj7J3ZsfDDCLnPZot6kaQY0DQd2hWKM5TjhABTIrmVsgEoYJhkmFsM4X/SKJcss2RdlwuV83kcWXJADskJscgpqZArUiN1wsgdeSBP5Nm4Nx6NF+N11pox5jP75BuMt08yfZ1J</latexit><latexit sha1_base64="WreFjTnxmoUXC5AhPcp14C4Aeo4=">AAACEHicdVC7TgJBFJ3FF+ILtbSZCEYrskujscLYWFBgIo8ENpu7wwATZh+ZuWsgGz7Bxl+xsdAYW0s7/8ZdwARfJ7nJyTn35t573FAKjab5YWSWlldW17LruY3Nre2d/O5eQweRYrzOAhmolguaS+HzOgqUvBUqDp4redMdXqZ+85YrLQL/Bschtz3o+6InGGAiOfnji2gkpAA1ptVAa1rseIADBjKuTpwO8hHGEI0mRSdfsErmFNT8Rb6sApmj5uTfO92ARR73kUnQum2ZIdoxKBRM8kmuE2keAhtCn7cT6oPHtR1PH5rQo0Tp0l6gkvKRTtXFiRg8rceem3Sm5+qfXir+5bUj7J3ZsfDDCLnPZot6kaQY0DQd2hWKM5TjhABTIrmVsgEoYJhkmFsM4X/SKJcss2RdlwuV83kcWXJADskJscgpqZArUiN1wsgdeSBP5Nm4Nx6NF+N11pox5jP75BuMt08yfZ1J</latexit>
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
. . . <latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit><latexit sha1_base64="RdkBBJ5iF/V21Ir3JttKZEnQ2EE=">AAAB7HicbVBNS8NAFHypX7V+VT16WSyCp5KIUPFU8OKxgmkLbSib7aZdutmE3RehhP4GLx4U8eoP8ua/cdvmoK0DC8PMG/a9CVMpDLrut1Pa2Nza3invVvb2Dw6PqscnbZNkmnGfJTLR3ZAaLoXiPgqUvJtqTuNQ8k44uZv7nSeujUjUI05THsR0pEQkGEUr+f1hgmZQrbl1dwGyTryC1KBAa1D9sjmWxVwhk9SYnuemGORUo2CSzyr9zPCUsgkd8Z6lisbcBPli2Rm5sMqQRIm2TyFZqL8TOY2NmcahnYwpjs2qNxf/83oZRjdBLlSaIVds+VGUSYIJmV9OhkJzhnJqCWVa2F0JG1NNGdp+KrYEb/XkddK+qntu3Xu4rjVvizrKcAbncAkeNKAJ99ACHxgIeIZXeHOU8+K8Ox/L0ZJTZE7hD5zPH+8Ajro=</latexit>
Original Loss Lorig <latexit sha1_base64="r0j8rNB/sM95q79vYwH7pdQr+wU=">AAACEHicdVDLSgMxFM34rPU16tJNsBVdlZluFFcFNy4KVrAPaIeSSdM2NJMMyR2xDP0EN/6KGxeKuHXpzr8xfQj1dSBwOOdc7s0JY8ENeN6Hs7C4tLyymlnLrm9sbm27O7s1oxJNWZUqoXQjJIYJLlkVOAjWiDUjUShYPRycj/36DdOGK3kNw5gFEelJ3uWUgJXa7tGl5j0uicBlZQzOtyICfUpEWh61W8BuIVU2MMq33Zxf8CbA3i/yZeXQDJW2+97qKJpETAIVxJim78UQpEQDp4KNsq3EsJjQAemxpqWSRMwE6eRDI3xolQ7uKm2fBDxR5ydSEhkzjEKbHN9rfnpj8S+vmUD3NEi5jBNgkk4XdROBQeFxO7jDNaMghpYQqrm9FdM+0YSC7TA7X8L/pFYs+F7BvyrmSmezOjJoHx2gY+SjE1RCF6iCqoiiO/SAntCzc+88Oi/O6zS64Mxm9tA3OG+f/iudKQ==</latexit><latexit sha1_base64="r0j8rNB/sM95q79vYwH7pdQr+wU=">AAACEHicdVDLSgMxFM34rPU16tJNsBVdlZluFFcFNy4KVrAPaIeSSdM2NJMMyR2xDP0EN/6KGxeKuHXpzr8xfQj1dSBwOOdc7s0JY8ENeN6Hs7C4tLyymlnLrm9sbm27O7s1oxJNWZUqoXQjJIYJLlkVOAjWiDUjUShYPRycj/36DdOGK3kNw5gFEelJ3uWUgJXa7tGl5j0uicBlZQzOtyICfUpEWh61W8BuIVU2MMq33Zxf8CbA3i/yZeXQDJW2+97qKJpETAIVxJim78UQpEQDp4KNsq3EsJjQAemxpqWSRMwE6eRDI3xolQ7uKm2fBDxR5ydSEhkzjEKbHN9rfnpj8S+vmUD3NEi5jBNgkk4XdROBQeFxO7jDNaMghpYQqrm9FdM+0YSC7TA7X8L/pFYs+F7BvyrmSmezOjJoHx2gY+SjE1RCF6iCqoiiO/SAntCzc+88Oi/O6zS64Mxm9tA3OG+f/iudKQ==</latexit><latexit sha1_base64="r0j8rNB/sM95q79vYwH7pdQr+wU=">AAACEHicdVDLSgMxFM34rPU16tJNsBVdlZluFFcFNy4KVrAPaIeSSdM2NJMMyR2xDP0EN/6KGxeKuHXpzr8xfQj1dSBwOOdc7s0JY8ENeN6Hs7C4tLyymlnLrm9sbm27O7s1oxJNWZUqoXQjJIYJLlkVOAjWiDUjUShYPRycj/36DdOGK3kNw5gFEelJ3uWUgJXa7tGl5j0uicBlZQzOtyICfUpEWh61W8BuIVU2MMq33Zxf8CbA3i/yZeXQDJW2+97qKJpETAIVxJim78UQpEQDp4KNsq3EsJjQAemxpqWSRMwE6eRDI3xolQ7uKm2fBDxR5ydSEhkzjEKbHN9rfnpj8S+vmUD3NEi5jBNgkk4XdROBQeFxO7jDNaMghpYQqrm9FdM+0YSC7TA7X8L/pFYs+F7BvyrmSmezOjJoHx2gY+SjE1RCF6iCqoiiO/SAntCzc+88Oi/O6zS64Mxm9tA3OG+f/iudKQ==</latexit><latexit sha1_base64="r0j8rNB/sM95q79vYwH7pdQr+wU=">AAACEHicdVDLSgMxFM34rPU16tJNsBVdlZluFFcFNy4KVrAPaIeSSdM2NJMMyR2xDP0EN/6KGxeKuHXpzr8xfQj1dSBwOOdc7s0JY8ENeN6Hs7C4tLyymlnLrm9sbm27O7s1oxJNWZUqoXQjJIYJLlkVOAjWiDUjUShYPRycj/36DdOGK3kNw5gFEelJ3uWUgJXa7tGl5j0uicBlZQzOtyICfUpEWh61W8BuIVU2MMq33Zxf8CbA3i/yZeXQDJW2+97qKJpETAIVxJim78UQpEQDp4KNsq3EsJjQAemxpqWSRMwE6eRDI3xolQ7uKm2fBDxR5ydSEhkzjEKbHN9rfnpj8S+vmUD3NEi5jBNgkk4XdROBQeFxO7jDNaMghpYQqrm9FdM+0YSC7TA7X8L/pFYs+F7BvyrmSmezOjJoHx2gY+SjE1RCF6iCqoiiO/SAntCzc+88Oi/O6zS64Mxm9tA3OG+f/iudKQ==</latexit>
{ht,t k | t = 1, . . . , T} <latexit sha1_base64="9VCBo+1x7H1sq/4u1Lxpq28SfU4=">AAACCnicbVC7SgNBFJ2Nrxhfq5Y2o0GwiGFXhNgIARvLCHlBdllmJ5NkyOyDmbtCWLe28VdsLBSx9Qvs/BsnyRaaeODC4Zx7mLnHjwVXYFnfRmFldW19o7hZ2tre2d0z9w/aKkokZS0aiUh2faKY4CFrAQfBurFkJPAF6/jjm6nfuWdS8ShswiRmbkCGIR9wSkBLnnnspCMvhQqcjzPs4Ac9cG1XsNOPQFVw08k8s2xVrRnwMrFzUkY5Gp75pcM0CVgIVBClerYVg5sSCZwKlpWcRLGY0DEZsp6mIQmYctPZKRk+1UofDyKpJwQ8U38nUhIoNQl8vRkQGKlFbyr+5/USGFy5KQ/jBFhI5w8NEoEhwtNecJ9LRkFMNCFUcv1XTEdEEgq6vZIuwV48eZm0L6q2VbXvLsv1Wl5HER2hE3SGbFRDdXSLGqiFKHpEz+gVvRlPxovxbnzMVwtGnjlEf2B8/gAxQ5id</latexit><latexit sha1_base64="9VCBo+1x7H1sq/4u1Lxpq28SfU4=">AAACCnicbVC7SgNBFJ2Nrxhfq5Y2o0GwiGFXhNgIARvLCHlBdllmJ5NkyOyDmbtCWLe28VdsLBSx9Qvs/BsnyRaaeODC4Zx7mLnHjwVXYFnfRmFldW19o7hZ2tre2d0z9w/aKkokZS0aiUh2faKY4CFrAQfBurFkJPAF6/jjm6nfuWdS8ShswiRmbkCGIR9wSkBLnnnspCMvhQqcjzPs4Ac9cG1XsNOPQFVw08k8s2xVrRnwMrFzUkY5Gp75pcM0CVgIVBClerYVg5sSCZwKlpWcRLGY0DEZsp6mIQmYctPZKRk+1UofDyKpJwQ8U38nUhIoNQl8vRkQGKlFbyr+5/USGFy5KQ/jBFhI5w8NEoEhwtNecJ9LRkFMNCFUcv1XTEdEEgq6vZIuwV48eZm0L6q2VbXvLsv1Wl5HER2hE3SGbFRDdXSLGqiFKHpEz+gVvRlPxovxbnzMVwtGnjlEf2B8/gAxQ5id</latexit><latexit sha1_base64="9VCBo+1x7H1sq/4u1Lxpq28SfU4=">AAACCnicbVC7SgNBFJ2Nrxhfq5Y2o0GwiGFXhNgIARvLCHlBdllmJ5NkyOyDmbtCWLe28VdsLBSx9Qvs/BsnyRaaeODC4Zx7mLnHjwVXYFnfRmFldW19o7hZ2tre2d0z9w/aKkokZS0aiUh2faKY4CFrAQfBurFkJPAF6/jjm6nfuWdS8ShswiRmbkCGIR9wSkBLnnnspCMvhQqcjzPs4Ac9cG1XsNOPQFVw08k8s2xVrRnwMrFzUkY5Gp75pcM0CVgIVBClerYVg5sSCZwKlpWcRLGY0DEZsp6mIQmYctPZKRk+1UofDyKpJwQ8U38nUhIoNQl8vRkQGKlFbyr+5/USGFy5KQ/jBFhI5w8NEoEhwtNecJ9LRkFMNCFUcv1XTEdEEgq6vZIuwV48eZm0L6q2VbXvLsv1Wl5HER2hE3SGbFRDdXSLGqiFKHpEz+gVvRlPxovxbnzMVwtGnjlEf2B8/gAxQ5id</latexit><latexit sha1_base64="9VCBo+1x7H1sq/4u1Lxpq28SfU4=">AAACCnicbVC7SgNBFJ2Nrxhfq5Y2o0GwiGFXhNgIARvLCHlBdllmJ5NkyOyDmbtCWLe28VdsLBSx9Qvs/BsnyRaaeODC4Zx7mLnHjwVXYFnfRmFldW19o7hZ2tre2d0z9w/aKkokZS0aiUh2faKY4CFrAQfBurFkJPAF6/jjm6nfuWdS8ShswiRmbkCGIR9wSkBLnnnspCMvhQqcjzPs4Ac9cG1XsNOPQFVw08k8s2xVrRnwMrFzUkY5Gp75pcM0CVgIVBClerYVg5sSCZwKlpWcRLGY0DEZsp6mIQmYctPZKRk+1UofDyKpJwQ8U38nUhIoNQl8vRkQGKlFbyr+5/USGFy5KQ/jBFhI5w8NEoEhwtNecJ9LRkFMNCFUcv1XTEdEEgq6vZIuwV48eZm0L6q2VbXvLsv1Wl5HER2hE3SGbFRDdXSLGqiFKHpEz+gVvRlPxovxbnzMVwtGnjlEf2B8/gAxQ5id</latexit>
{ht,t ` | t = 1, . . . , T} <latexit sha1_base64="xOwikC/OvDelYgqWWxJ/0hJ1fJ0=">AAACDXicbVC7SgNBFJ2Nrxhfq5Y2g1GwiGFXhNgIARvLCHlBdllmJ7PJkNkHM3eFsOYHbPwVGwtFbO3t/BsnyRaaeODC4Zx7mLnHTwRXYFnfRmFldW19o7hZ2tre2d0z9w/aKk4lZS0ai1h2faKY4BFrAQfBuolkJPQF6/ijm6nfuWdS8ThqwjhhbkgGEQ84JaAlzzxxsqGXQQXOHSbEBDv4QQ9c2xXs9GNQFdx0Jp5ZtqrWDHiZ2DkpoxwNz/zSYZqGLAIqiFI920rAzYgETgWblJxUsYTQERmwnqYRCZlys9k1E3yqlT4OYqknAjxTfycyEio1Dn29GRIYqkVvKv7n9VIIrtyMR0kKLKLzh4JUYIjxtBrc55JREGNNCJVc/xXTIZGEgi6wpEuwF09eJu2Lqm1V7bvLcr2W11FER+gYnSEb1VAd3aIGaiGKHtEzekVvxpPxYrwbH/PVgpFnDtEfGJ8/kmKZ6Q==</latexit><latexit sha1_base64="xOwikC/OvDelYgqWWxJ/0hJ1fJ0=">AAACDXicbVC7SgNBFJ2Nrxhfq5Y2g1GwiGFXhNgIARvLCHlBdllmJ7PJkNkHM3eFsOYHbPwVGwtFbO3t/BsnyRaaeODC4Zx7mLnHTwRXYFnfRmFldW19o7hZ2tre2d0z9w/aKk4lZS0ai1h2faKY4BFrAQfBuolkJPQF6/ijm6nfuWdS8ThqwjhhbkgGEQ84JaAlzzxxsqGXQQXOHSbEBDv4QQ9c2xXs9GNQFdx0Jp5ZtqrWDHiZ2DkpoxwNz/zSYZqGLAIqiFI920rAzYgETgWblJxUsYTQERmwnqYRCZlys9k1E3yqlT4OYqknAjxTfycyEio1Dn29GRIYqkVvKv7n9VIIrtyMR0kKLKLzh4JUYIjxtBrc55JREGNNCJVc/xXTIZGEgi6wpEuwF09eJu2Lqm1V7bvLcr2W11FER+gYnSEb1VAd3aIGaiGKHtEzekVvxpPxYrwbH/PVgpFnDtEfGJ8/kmKZ6Q==</latexit><latexit sha1_base64="xOwikC/OvDelYgqWWxJ/0hJ1fJ0=">AAACDXicbVC7SgNBFJ2Nrxhfq5Y2g1GwiGFXhNgIARvLCHlBdllmJ7PJkNkHM3eFsOYHbPwVGwtFbO3t/BsnyRaaeODC4Zx7mLnHTwRXYFnfRmFldW19o7hZ2tre2d0z9w/aKk4lZS0ai1h2faKY4BFrAQfBuolkJPQF6/ijm6nfuWdS8ThqwjhhbkgGEQ84JaAlzzxxsqGXQQXOHSbEBDv4QQ9c2xXs9GNQFdx0Jp5ZtqrWDHiZ2DkpoxwNz/zSYZqGLAIqiFI920rAzYgETgWblJxUsYTQERmwnqYRCZlys9k1E3yqlT4OYqknAjxTfycyEio1Dn29GRIYqkVvKv7n9VIIrtyMR0kKLKLzh4JUYIjxtBrc55JREGNNCJVc/xXTIZGEgi6wpEuwF09eJu2Lqm1V7bvLcr2W11FER+gYnSEb1VAd3aIGaiGKHtEzekVvxpPxYrwbH/PVgpFnDtEfGJ8/kmKZ6Q==</latexit><latexit sha1_base64="xOwikC/OvDelYgqWWxJ/0hJ1fJ0=">AAACDXicbVC7SgNBFJ2Nrxhfq5Y2g1GwiGFXhNgIARvLCHlBdllmJ7PJkNkHM3eFsOYHbPwVGwtFbO3t/BsnyRaaeODC4Zx7mLnHTwRXYFnfRmFldW19o7hZ2tre2d0z9w/aKk4lZS0ai1h2faKY4BFrAQfBuolkJPQF6/ijm6nfuWdS8ThqwjhhbkgGEQ84JaAlzzxxsqGXQQXOHSbEBDv4QQ9c2xXs9GNQFdx0Jp5ZtqrWDHiZ2DkpoxwNz/zSYZqGLAIqiFI920rAzYgETgWblJxUsYTQERmwnqYRCZlys9k1E3yqlT4OYqknAjxTfycyEio1Dn29GRIYqkVvKv7n9VIIrtyMR0kKLKLzh4JUYIjxtBrc55JREGNNCJVc/xXTIZGEgi6wpEuwF09eJu2Lqm1V7bvLcr2W11FER+gYnSEb1VAd3aIGaiGKHtEzekVvxpPxYrwbH/PVgpFnDtEfGJ8/kmKZ6Q==</latexit>
{ht,t L | t = 1, . . . , T} <latexit sha1_base64="6w8lQX6t/2b6qRcXuEQdV3549jw=">AAACCnicbVC7SgNBFJ31GeNr1dJmNAgWMeyKEBshYGNhESEvyC7L7GQ2GTL7YOauENbUNv6KjYUitn6BnX/jJNlCEw9cOJxzDzP3+IngCizr21haXlldWy9sFDe3tnd2zb39lopTSVmTxiKWHZ8oJnjEmsBBsE4iGQl9wdr+8Hrit++ZVDyOGjBKmBuSfsQDTgloyTOPnGzgZVCGs9sxdvCDHriyy9jpxaDKuOGMPbNkVawp8CKxc1JCOeqe+aXDNA1ZBFQQpbq2lYCbEQmcCjYuOqliCaFD0mddTSMSMuVm01PG+EQrPRzEUk8EeKr+TmQkVGoU+nozJDBQ895E/M/rphBcuhmPkhRYRGcPBanAEONJL7jHJaMgRpoQKrn+K6YDIgkF3V5Rl2DPn7xIWucV26rYdxelWjWvo4AO0TE6RTaqohq6QXXURBQ9omf0it6MJ+PFeDc+ZqtLRp45QH9gfP4A/2+Yfg==</latexit><latexit sha1_base64="6w8lQX6t/2b6qRcXuEQdV3549jw=">AAACCnicbVC7SgNBFJ31GeNr1dJmNAgWMeyKEBshYGNhESEvyC7L7GQ2GTL7YOauENbUNv6KjYUitn6BnX/jJNlCEw9cOJxzDzP3+IngCizr21haXlldWy9sFDe3tnd2zb39lopTSVmTxiKWHZ8oJnjEmsBBsE4iGQl9wdr+8Hrit++ZVDyOGjBKmBuSfsQDTgloyTOPnGzgZVCGs9sxdvCDHriyy9jpxaDKuOGMPbNkVawp8CKxc1JCOeqe+aXDNA1ZBFQQpbq2lYCbEQmcCjYuOqliCaFD0mddTSMSMuVm01PG+EQrPRzEUk8EeKr+TmQkVGoU+nozJDBQ895E/M/rphBcuhmPkhRYRGcPBanAEONJL7jHJaMgRpoQKrn+K6YDIgkF3V5Rl2DPn7xIWucV26rYdxelWjWvo4AO0TE6RTaqohq6QXXURBQ9omf0it6MJ+PFeDc+ZqtLRp45QH9gfP4A/2+Yfg==</latexit><latexit sha1_base64="6w8lQX6t/2b6qRcXuEQdV3549jw=">AAACCnicbVC7SgNBFJ31GeNr1dJmNAgWMeyKEBshYGNhESEvyC7L7GQ2GTL7YOauENbUNv6KjYUitn6BnX/jJNlCEw9cOJxzDzP3+IngCizr21haXlldWy9sFDe3tnd2zb39lopTSVmTxiKWHZ8oJnjEmsBBsE4iGQl9wdr+8Hrit++ZVDyOGjBKmBuSfsQDTgloyTOPnGzgZVCGs9sxdvCDHriyy9jpxaDKuOGMPbNkVawp8CKxc1JCOeqe+aXDNA1ZBFQQpbq2lYCbEQmcCjYuOqliCaFD0mddTSMSMuVm01PG+EQrPRzEUk8EeKr+TmQkVGoU+nozJDBQ895E/M/rphBcuhmPkhRYRGcPBanAEONJL7jHJaMgRpoQKrn+K6YDIgkF3V5Rl2DPn7xIWucV26rYdxelWjWvo4AO0TE6RTaqohq6QXXURBQ9omf0it6MJ+PFeDc+ZqtLRp45QH9gfP4A/2+Yfg==</latexit><latexit sha1_base64="6w8lQX6t/2b6qRcXuEQdV3549jw=">AAACCnicbVC7SgNBFJ31GeNr1dJmNAgWMeyKEBshYGNhESEvyC7L7GQ2GTL7YOauENbUNv6KjYUitn6BnX/jJNlCEw9cOJxzDzP3+IngCizr21haXlldWy9sFDe3tnd2zb39lopTSVmTxiKWHZ8oJnjEmsBBsE4iGQl9wdr+8Hrit++ZVDyOGjBKmBuSfsQDTgloyTOPnGzgZVCGs9sxdvCDHriyy9jpxaDKuOGMPbNkVawp8CKxc1JCOeqe+aXDNA1ZBFQQpbq2lYCbEQmcCjYuOqliCaFD0mddTSMSMuVm01PG+EQrPRzEUk8EeKr+TmQkVGoU+nozJDBQ895E/M/rphBcuhmPkhRYRGcPBanAEONJL7jHJaMgRpoQKrn+K6YDIgkF3V5Rl2DPn7xIWucV26rYdxelWjWvo4AO0TE6RTaqohq6QXXURBQ9omf0it6MJ+PFeDc+ZqtLRp45QH9gfP4A/2+Yfg==</latexit>
x1:T <latexit sha1_base64="L7Pcqd/GJ1LwHaj9CnB7qcjLJvg=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFYoV/QhrLZTtqlm03Y3Ygl9Ed48aCIV3+PN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RQ2Nre2d4q7pb39g8Oj8vFJW8epYthisYhVN6AaBZfYMtwI7CYKaRQI7ASTu7nfeUSleSybZpqgH9GR5CFn1Fip8zTIvNvmbFCuuFV3AbJOvJxUIEdjUP7qD2OWRigNE1Trnucmxs+oMpwJnJX6qcaEsgkdYc9SSSPUfrY4d0YurDIkYaxsSUMW6u+JjEZaT6PAdkbUjPWqNxf/83qpCW/8jMskNSjZclGYCmJiMv+dDLlCZsTUEsoUt7cSNqaKMmMTKtkQvNWX10n7quq5Ve/hulKv5XEU4QzO4RI8qEEd7qEBLWAwgWd4hTcncV6cd+dj2Vpw8plT+APn8wfx5o9D</latexit><latexit sha1_base64="L7Pcqd/GJ1LwHaj9CnB7qcjLJvg=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFYoV/QhrLZTtqlm03Y3Ygl9Ed48aCIV3+PN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RQ2Nre2d4q7pb39g8Oj8vFJW8epYthisYhVN6AaBZfYMtwI7CYKaRQI7ASTu7nfeUSleSybZpqgH9GR5CFn1Fip8zTIvNvmbFCuuFV3AbJOvJxUIEdjUP7qD2OWRigNE1Trnucmxs+oMpwJnJX6qcaEsgkdYc9SSSPUfrY4d0YurDIkYaxsSUMW6u+JjEZaT6PAdkbUjPWqNxf/83qpCW/8jMskNSjZclGYCmJiMv+dDLlCZsTUEsoUt7cSNqaKMmMTKtkQvNWX10n7quq5Ve/hulKv5XEU4QzO4RI8qEEd7qEBLWAwgWd4hTcncV6cd+dj2Vpw8plT+APn8wfx5o9D</latexit><latexit sha1_base64="L7Pcqd/GJ1LwHaj9CnB7qcjLJvg=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFYoV/QhrLZTtqlm03Y3Ygl9Ed48aCIV3+PN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RQ2Nre2d4q7pb39g8Oj8vFJW8epYthisYhVN6AaBZfYMtwI7CYKaRQI7ASTu7nfeUSleSybZpqgH9GR5CFn1Fip8zTIvNvmbFCuuFV3AbJOvJxUIEdjUP7qD2OWRigNE1Trnucmxs+oMpwJnJX6qcaEsgkdYc9SSSPUfrY4d0YurDIkYaxsSUMW6u+JjEZaT6PAdkbUjPWqNxf/83qpCW/8jMskNSjZclGYCmJiMv+dDLlCZsTUEsoUt7cSNqaKMmMTKtkQvNWX10n7quq5Ve/hulKv5XEU4QzO4RI8qEEd7qEBLWAwgWd4hTcncV6cd+dj2Vpw8plT+APn8wfx5o9D</latexit><latexit sha1_base64="L7Pcqd/GJ1LwHaj9CnB7qcjLJvg=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqHgqePFYoV/QhrLZTtqlm03Y3Ygl9Ed48aCIV3+PN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RQ2Nre2d4q7pb39g8Oj8vFJW8epYthisYhVN6AaBZfYMtwI7CYKaRQI7ASTu7nfeUSleSybZpqgH9GR5CFn1Fip8zTIvNvmbFCuuFV3AbJOvJxUIEdjUP7qD2OWRigNE1Trnucmxs+oMpwJnJX6qcaEsgkdYc9SSSPUfrY4d0YurDIkYaxsSUMW6u+JjEZaT6PAdkbUjPWqNxf/83qpCW/8jMskNSjZclGYCmJiMv+dDLlCZsTUEsoUt7cSNqaKMmMTKtkQvNWX10n7quq5Ve/hulKv5XEU4QzO4RI8qEEd7qEBLWAwgWd4hTcncV6cd+dj2Vpw8plT+APn8wfx5o9D</latexit>
(b) Auxiliary loss on intermediate layers in a TrellisNet.
Figure 7: (a) RNN-inspired variational dropout. (b) ConvNet-inspired auxiliary losses.
rem 1. In each iteration, we apply the same mask on the post-activation outputs, at every time step in both the temporal dimension and depth dimension. That is, based on Eq. (6) in Theorem 1, we adapt VD to the TrellisNet setting by assuming ht,t′±δ ≈ ht,t′ ; see Figure 7a. Empirically, we found this dropout to work significantly better than other dropout schemes (e.g. drop certain channels entirely).
Recurrent weight dropout/DropConnect. We apply DropConnect on the TrellisNet kernel. Merity et al. (2018b) showed that regularizing hidden-to-hidden weights Whh can be useful in optimizing LSTM language models, and we carry this scheme over to trellis networks.

heading: B.2 FROM CONVOLUTIONAL NETWORKS
text: Dense convolutional kernel. Generalizing the convolution from a mixed group (sparse) convolution to a general (dense) one means the connections are no longer recurrent and we are computing directly on the hidden units with a large kernel, just like any temporal ConvNet.
Deep supervision. Recall that for sparse TrellisNet to recover truncated RNN, at each level the hidden units are of the form ht,t′ , representing the state at time t if we assume that history started at time t′ (Eq. (6)). We propose to inject the loss function at intermediate layers of the convolutional network (e.g. after every ` layers of transformations, where we call ` the auxiliary loss frequency). For example, during training, to predict an output at time t with a L-layer TrellisNet, besides z(L)t in the last layer, we can also apply the loss function on z(L−`)t , z (L−2`) t , etc. – where hidden units will predict with a shorter history because they are at lower levels of the network. This had been introduced for convolutional models in computer vision (Lee et al., 2015; Xie & Tu, 2015). The eventual loss of the network will be
Ltotal = Lorig + λ · Laux, (15)
where λ is a fixed scaling factor that controls the weight of the auxiliary loss.
Note that this technique is not directly transferable (or applicable) to RNNs.
Larger kernel and dilations (Yu & Koltun, 2016). These techniques have been used in convolutional networks to more quickly increase the receptive field. They can be immediately applied to trellis networks. Note that the activation function f of TrellisNet may need to change if we change the kernel size or dilation settings (e.g. with dilation d and kernel size 2, the activation will be f(ẑ
(i) 1:T , z (i) 1:T−d)).
Weight normalization (Salimans & Kingma, 2016). Weight normalization (WN) is a technique that learns the direction and the magnitude of the weight matrix independently. Applying WN on the convolutional kernel was used in some prior works on temporal convolutional architectures (Dauphin et al., 2017; Bai et al., 2018), and have been found useful in regularizing the convolutional filters and boosting convergence.
Parallelism. Because TrellisNet is convolutional in nature, it can easily leverage the parallel processing in the convolution operation (which slides the kernel across the input features). We note that when the input sequence is relatively long, the predictions of the first few time steps will have insufficient history context compared to the predictions later in the sequence. This can be addressed by either history padding (mentioned in Appendix B.1) or chopping off the loss incurred by the first few time steps.

heading: C BENCHMARK TASKS
text: Word-level language modeling on Penn Treebank (PTB). The original Penn Treebank (PTB) dataset selected 2,499 stories from a collection of almost 100K stories published in Wall Street Journal (WSJ) (Marcus et al., 1993). After Mikolov et al. (2010) processed the corpus, the PTB dataset contains 888K words for training, 70K for validation and 79K for testing, where each sentence is marked with an <eos> tag at its end. All of the numbers (e.g. in financial news) were replaced with a ? symbol with many punctuations removed. Though small, PTB has been a highly studied dataset in the domain of language modeling (Miyamoto & Cho, 2016; Zilly et al., 2017; Merity et al., 2018b; Melis et al., 2018; Yang et al., 2018). Due to its relatively small size, many computational models can easily overfit on word-level PTB. Therefore, good regularization methods and optimization techniques designed for sequence models are especially important on this benchmark task (Merity et al., 2018b).
Word-level language modeling on WikiText-103. WikiText-103 (WT103) is 110 times larger than PTB, containing a training corpus from 28K lightly processed Wikipedia articles (Merity et al., 2017). In total, WT103 features a vocabulary size of about 268K2, with 103M words for training, 218K words for validation, and 246K words for testing/evaluation. The WT103 corpus also retains the original case, punctuation and numbers in the raw data, all of which were removed from the PTB corpus. Moreover, since WT103 is composed of full articles (whereas PTB is sentence-based), it is better suited for testing long-term context retention. For these reasons, WT103 is typically considered much more representative and realistic than PTB (Merity et al., 2018a).
Character-level language modeling on Penn Treebank (PTB). When used for character-level language modeling, PTB is a medium size dataset that contains 5M chracters for training, 396K for validation, and 446K for testing, with an alphabet size of 50 (note: the <eos> tag that marks the end of a sentence in word-level tasks is now considered one character). While the alphabet size of char-level PTB is much smaller compared to the word-level vocabulary size (10K), there is much longer sequential token dependency because a sentence contains many more characters than words.
Sequential and permuted MNIST classification. The MNIST handwritten digits dataset (LeCun et al., 1989) contains 60K normalized training images and 10K testing images, all of size 28 × 28. In the sequential MNIST task, MNIST images are presented to the sequence model as a flattened 784 × 1 sequence for digit classification. Accurate predictions therefore require good long-term memory of the flattened pixels – longer than in most language modeling tasks. In the setting of permuted MNIST (PMNIST), the order of the sequence is permuted at random, so the network can no longer rely on local pixel features for classification.
Sequential CIFAR-10 classification. The CIFAR-10 dataset (Krizhevsky & Hinton, 2009) contains 50K images for training and 10K for testing, all of size 32 × 32. In the sequential CIFAR-10 task, these images are passed into the model one at each time step, flattended as in the MNIST tasks. Compared to sequential MNIST, this task is more challenging. For instance, CIFAR-10 contains more complex image structures and intra-class variations, and there are 3 channels to the input. Moreover, as the images are larger, a sequence model needs to have even longer memory than in sequential MNIST or PMNIST (Trinh et al., 2018).

heading: D HYPERPARAMETERS AND ABLATION STUDY
text: Table 5 specifies the trellis networks used for the various tasks. There are a few things to note while reading the table. First, in training, we decay the learning rate once the validation error plateaus for a while (or according to some fixed schedule, such as after 100 epochs). Second, for auxiliary loss (see Appendix B for more details), we insert the loss function after every fixed number of layers in the network. This “frequency” is included below under the “Auxiliary Frequency” entry. Finally, the hidden dropout in the Table refers to the variational dropout we translated from RNNs (see Appendix B), which is applied at all hidden layers of the TrellisNet. Due to the insight from Theorem 1, many techniques in TrellisNet were translated directly from RNNs or TCNs. Thus, most of the hyperparameters were based on the numbers reported in prior works (e.g. embedding size, embedding dropout, hidden dropout, output dropout, optimizer, weight-decay, etc.) with minor
2As a reference, Oxford English Dictionary only contains less than 220K unique English words.
adjustments (Merity et al., 2018b; Yang et al., 2018; Bradbury et al., 2017; Merity et al., 2018a; Trinh et al., 2018; Bai et al., 2018; Santoro et al., 2018). For factors such as auxiliary loss weight and frequency, we perform a basic grid search.
We have also performed an ablation study on TrellisNet to study the influence of various ingredients and techniques on performance. The results are reported in Table 6. We conduct the study on wordlevel PTB using a TrellisNet with 24M parameters. When we study one factor (e.g. removing hidden dropout), all hyperparameters and settings remain the same as in column 1 of Table 5 (except for “Dense Kernel”, where we adjust the number of hidden units so that the model size remains the same).

Review summaries:
The authors propose a new type of neural network architecture for sequence modelling : Trellis Networks. A trellis network is a special case of temporal convolutional network with shared weights across time and layers and  with input at each layer. As stated by the authors, this architecture does not seem really interesting. The authors show that there exists  an equivalent Trellis Network to any truncated RNN and therefore that truncated RNN can be represented by temporal convolutional network. This result is not surprising since  truncated RNN can be  unrolled and that their time dependency is bounded.  The construction of the Trellis Network equivalent to a truncated RNN involves sparse weight matrices, therefore using full weight matrices provides a greater expressive power. One can regret that the authors do not explain what kind of modelling power one can gain with full weight matrices. 

This paper introduces a novel architecture for sequence modeling, called the trellis network. The trellis network is in a sense a combination of RNNs and CNNs. The authors give a constructive proof that the trellis network is a special case of a truncated RNN. It also resembles CNNs since the neurons at higher levels have bigger receptive fields. As a result, techniques from RNN and CNN literature can be conveniently brought in and adapted to trellis network. The proposed method is evaluated on benchmark tasks and shows performance gain over existing methods.