Title: DEEP PROBABILISTIC VIDEO COMPRESSION

Abstract: We propose a variational inference approach to deep probabilistic video compression. Our model uses advances in variational autoencoders (VAEs) for sequential data and combines it with recent work on neural image compression. The approach jointly learns to transform the original video into a lower-dimensional representation as well as to entropy code this representation according to a temporally-conditioned probabilistic model. We split the latent space into local (per frame) and global (per segment) variables, and show that training the VAE to utilize both representations leads to an improved rate-distortion performance. Evaluation on small videos from public data sets with varying complexity and diversity show that our model yields competitive results when trained on generic video content. Extreme compression performance is achieved for videos with specialized content if the model is trained on similar videos.

1. INTRODUCTION:

The transmission of video content is responsible for up to 80% of the consumer internet traffic, and both the overall internet traffic as well as the share of video data is expected to increase even further in the future (Cisco, 2017) . Improving compression efficiency is more crucial than ever.

Today, a variety of video codecs exists that have reached an impressive performance. The most commonly used standard is H.264 (Wiegand et al., 2003) , also known as Advanced Video Coding (AVC). More recent codecs include H.265 (Sullivan et al., 2012) , also known as High Efficiency Video Coding (HEVC), and VP9 (Mukherjee et al., 2015) . All of these existing codecs follow the same block based hybrid structure (Musmann et al., 1985) which essentially emerged from engineering out and refining this concept over decades. From a high level perspective, they differ in a huge number of smaller design choices and have grown to become more and more complex systems.

While there is room for improving the block based hybrid approach even further (Fraunhofer, 2018) , the question is how much longer significant improvements can be obtained when following the same concept. Interestingly, in the context of image compression, deep learning approaches that are fundamentally different to existing codecs have already shown promising results (Ballé et al., 2018; 2016; Theis et al., 2017; Agustsson et al., 2017; Minnen et al., 2018) , mostly within the past year.

Motivated by these successes for images, we propose a first step towards innovating beyond blockbased hybrid codecs by framing video compression in a deep probabilistic context. To this end, we propose an unsupervised deep learning approach to encoding video. The approach simultaneously learns the optimal transformation of the video to a low-dimensional representation and a powerful predictive model that assigns probabilities to video segments, allowing us to efficiently entropy-code the discretized latent representation into a short code length.

Our end-to-end neural video compression scheme is based on sequential variational autoencoders (Bayer & Osendorfer, 2014; Chung et al., 2015; Li & Mandt, 2018) and the approach of Ballé et al. (2016) for discretizing and entropy coding a continuous latent representation. The transformations to and from the latent representation, known as the encoder and decoder, are parameterized by deep neural networks and are learned by unsupervised training on videos. We introduce both local latent variables, which are inferred from a single frame, and a global state, inferred from an entire segment, to efficiently store a video sequence. Furthermore, the trajectory of the latent variables is modeled stochastically by a deep probabilistic model. After training, the context-dependent predictive model is used to entropy code the latent variables into binary with an arithmetic coder. As the first step towards a new approach, we focus on small resolution video (64 × 64) and aim to efficiently capture temporal correlations. Figure 1 shows a test example of the possible performance improvements using our approach if the model is trained on similar content. The plots show two frames of a video, compressed and reconstructed by our approach and classical video codecs. One sees that fine granular details, such as the hands of the cartoon character, are lost in the classical approach due to artifacts from block motion estimation (low bitrate regime), whereas our deep learning approach successfully captures these details with less than 10% of the file length.

Our contributions can be understood as follows:

1) Deep probabilistic video compression. To the best of our knowledge, this is the first work to employ a variational autoencoder (VAE) in conjuction with discretization and entropy coding to build an end-to-end trainable video codec.

2) Global inference. Temporal redundancy in a video can be taken into account by a temporal prior on each frame or through an architectural design that encodes an entire video segment into a global state. We propose a model which incorporates both global and local latent variables and show that this produces a shorter code length for a given image quality.

3) Small bit rates. We perform experiments on three large public data sets of varying complexity and diversity. Performance is evaluated by rate-distortion curves. Our method is competitive with traditional codecs on small videos after training and testing on a diverse set of videos. Extreme compression performance can be achieved for videos with specialized content if the model is trained on similar videos.

Paper Organization. In Section 2, we summarize important related works before describing our method for deep video compression in Section 3. Section 4 discusses our experimental results, including quantitative results and a qualitative discussion.

2. RELATED WORK:

The approaches related to our method fall into three categories: deep generative video models, neural image compression, and neural video compression.

Deep Generative Video Models. Several works have applied the variational autoencoder (VAE) (Kingma & Welling, 2014; Rezende et al., 2014) to stochastically model sequences (Bayer & Osendorfer, 2014; Chung et al., 2015) . Babaeizadeh et al. (2018) ; Xu et al. (2018) use a VAE for stochastic video generation. He et al. (2018) and Denton & Fergus (2018) apply a long short term memory (LSTM) in conjunction with a sequential VAE to model the evolution of the latent space across many video frames. Li & Mandt (2018) separate latent variables of a sequential VAE into local and global variables in order to learn a disentangled representation for video generation. Vondrick et al. (2016) generate realistic videos by using a GAN (Goodfellow et al., 2014) to learn to separate foreground and background, and Lee et al. (2018) combine variational and adversarial methods to generate realistic videos. This paper also employs a deep generative model to model the sequential probability distribution of frames from a video source. In contrast to other work, our work learns a continuous latent representation that can be discretized with minimal information loss, required for further compression. Furthermore, our objective is to convert the original video into a short binary description rather than to generate new videos.

Neural Image Compression. There has been significant work on applying deep learning to image compression. In Toderici et al. (2016; 2017) , a LSTM based codec is used to capture spatial correlations of pixel values and can achieve different bit-rates without having to retrain the model. Ballé et al. (2016) perform image compression with a VAE and demonstrate how to approximately discretize the VAE latent space by introducing noise during training. This work is refined by (Ballé et al., 2018) which improves the prior model (used for entropy coding) beyond the mean-field approximation by transmitting side information in the form of a hierarchical model. Minnen et al. (2018) consider an autoregressive model to achieve a similar effect. These image codecs encode each image independently and therefore their probabilistic models are stationary with respect to time. In contrast, our method performs compression according to a non-stationary, time-dependent probability model which has much lower entropy per pixel.

Neural Video Compression. The use of deep neural networks for video compression is relatively new. Wu et al. (2018) perform video compression through image interpolation between reference frames using a predictive model based on a deep neural network. Chen et al. (2017) and Chen et al. (2018) use a deep neural architecture to predict the most likely frame with a modified form of block motion prediction and store residuals in a lossy representation. Since these works are based on block prediction, they are similar in function and in performance to existing codecs. Our method is not based on block motion estimation, and the full inferred probability distribution over the space of plausible subsequent frames (rather than residuals) is used for entropy coding.

3. DEEP PROBABILISTIC VIDEO COMPRESSION:

The objective of lossy video compression can be defined as finding the shortest description of a video while tolerating a certain level of information loss. Classical video codecs try to predict intermediate frames based on block motion estimates, since information which can be predicted does not need to be stored. The residual error is then stored in a lossy transform representation. An end-to-end machine learning approach to encoding video, however, should simultaneously learn the appropriate predictive model and the optimal lossy transformation. This allows both to transform the video into a low dimensional latent representation and to then use the jointly learned predictive model to remove the remaining redundancy in the latents by entropy coding them to a short binary representation (Huffman, 1952; Langdon, 1984) .

The challenge over image compression is that the data is now sequences of images. These exhibit strong temporal correlations in addition to the spatial correlations already present in images. A naive approach to neural video compression would be to encode the video frame-by-frame, using the marginal distribution of images as done in VAE image compression. This distribution does not capture the temporal correlations and therefore tends to have high entropy, leading to long code lengths. On the other hand, treating an entire video segment as an independent data point in the latent representation leads to data sparseness and poor generalization performance.

Therefore, we propose to use a temporally-conditioned prior distribution parameterized by a deep generative model to efficiently code the latent variables associated with each frame. By conditioning on nearby frames in the sequence, the predictive model can be more certain about the next frame, thus achieving a smaller entropy and code length. As detailed below, in addition to using a deep sequential probabilistic model, we propose an architecture that combines local and global information in the video. A global variable stores information that is common to the sequence of frames, while a local variable stores additional dynamical content.

In the following paragraphs, we describe our approach (see Fig. 3 ) in more detail. We describe the encoder and decoder models, the objective function, and the interplay between our deep probabilistic sequential model and entropy coding scheme.

Decoder. We propose a stochastic recurrent variational autoencoder to transform a sequence of frames

x 1:T = (x 1 , • • • , x T ) into a compressed representation of local latent variables z 1:T = (z 1 , • • • , z T )

, where each z i only depends on x i . This model is refined to additionally include a global state f similar to Li & Mandt (2018) , resulting in the following probabilistic deep generative

x 1:T < l a t e x i t s h a 1 _ b a s e 6 4 = " Y q Q C X M o 5 A 7 S q X D z J K m d t Z 2 M 1 A r 8 = " > A A A B / X i c b V D N S 8 M w H E 3 9 n P O r f t y 8 B I f g a b Q i K J 6 G X j x O 2 B d s p a R p u o W l S U l S c Z b i v + L F g y J e / T + 8 + d + Y b j 3 o 5 o O Q x 3 u / H 3 l 5 Q c K o 0 o 7 z b S 0 t r 6 y u r V c 2 q p t b 2 z u 7 9 t 5 + R 4 l U Y t L G g g n Z C 5 A i j H L S 1 l Q z 0 k s k Q X H A S D c Y 3 x R + 9 5 5 I R Q V v 6 U l C v B g N O Y 0 o R t p I v n 0 4 C A Q L 1 S Q 2 V / a Q + 5 l 7 1 c p 9 u + b U n S n g I n F L U g M l m r 7 9 N Q g F T m P C N W Z I q b 7 r J N r L k N Q U M 5 J X B 6 k i C c J j N C R 9 Q z m K i f K y a f o c n h g l h J G Q 5 n A N p + r v j Q z F q g h o J m O k R 2 r e K 8 T / v H 6 q o 0 s v o z x J N e F 4 9 l C U M q g F L K q A I Z U E a z Y x B G F J T V a I R 0 g i r E 1 h V V O C O / / l R d I 5 q 7 t O 3 b 0 7 r z W u y z o q 4 A g c g 1 P g g g v Q A L e g C d o A g 0 f w D F 7 B m / V k v V j v 1 s d s d M k q d w 7 A H 1 i f P / x 2 l Y w = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Y q Q C X M o 5 A 7 S q X D z J K m d t Z 2 M 1 A r 8 = " > A A A B / X i c b V D N S 8 M w H E 3 9 n P O r f t y 8 B I f g a b Q i K J 6 G X j x O 2 B d s p a R p u o W l S U l S c Z b i v + L F g y J e / T + 8 + d + Y b j 3 o 5 o O Q x 3 u / H 3 l 5 Q c K o 0 o 7 z b S 0 t r 6 y u r V c 2 q p t b 2 z u 7 9 t 5 + R 4 l U Y t L G g g n Z C 5 A i j H L S 1 l Q z 0 k s k Q X H A S D c Y 3 x R + 9 5 5 I R Q V v 6 U l C v B g N O Y 0 o R t p I v n 0 4 C A Q L 1 S Q 2 V / a Q + 5 l 7 1 c p 9 u + b U n S n g I n F L U g M l m r 7 9 N Q g F T m P C N W Z I q b 7 r J N r L k N Q U M 5 J X B 6 k i C c J j N C R 9 Q z m K i f K y a f o c n h g l h J G Q 5 n A N p + r v j Q z F q g h o J m O k R 2 r e K 8 T / v H 6 q o 0 s v o z x J N e F 4 9 l C U M q g F L K q A I Z U E a z Y x B G F J T V a I R 0 g i r E 1 h V V O C O / / l R d I 5 q 7 t O 3 b 0 7 r z W u y z o q 4 A g c g 1 P g g g v Q A L e g C d o A g 0 f w D F 7 B m / V k v V j v 1 s d s d M k q d w 7 A H 1 i f P / x 2 l Y w = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Y q Q C X M o 5 A 7 S q X D z J K m d t Z 2 M 1 A r 8 = " > A A A B / X i c b V D N S 8 M w H E 3 9 n P O r f t y 8 B I f g a b Q i K J 6 G X j x O 2 B d s p a R p u o W l S U l S c Z b i v + L F g y J e / T + 8 + d + Y b j 3 o 5 o O Q x 3 u / H 3 l 5 Q c K o 0 o 7 z b S 0 t r 6 y u r V c 2 q p t b 2 z u 7 9 t 5 + R 4 l U Y t L G g g n Z C 5 A i j H L S 1 l Q z 0 k s k Q X H A S D c Y 3 x R + 9 5 5 I R Q V v 6 U l C v B g N O Y 0 o R t p I v n 0 4 C A Q L 1 S Q 2 V / a Q + 5 l 7 1 c p 9 u + b U n S n g I n F L U g M l m r 7 9 N Q g F T m P C N W Z I q b 7 r J N r L k N Q U M 5 J X B 6 k i C c J j N C R 9 Q z m K i f K y a f o c n h g l h J G Q 5 n A N p + r v j Q z F q g h o J m O k R 2 r e K 8 T / v H 6 q o 0 s v o z x J N e F 4 9 l C U M q g F L K q A I Z U E a z Y x B G F J T V a I R 0 g i r E 1 h V V O C O / / l R d I 5 q 7 t O 3 b 0 7 r z W u y z o q 4 A g c g 1 P g g g v Q A L e g C d o A g 0 f w D F 7 B m / V k v V j v 1 s d s d M k q d w 7 A H 1 i f P / x 2 l Y w = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Y q Q C X M o 5 A 7 S q X D z J K m d t Z 2 M 1 A r 8 = " > A A A B / X i c b V D N S 8 M w H E 3 9 n P O r f t y 8 B I f g a b Q i K J 6 G X j x O 2 B d s p a R p u o W l S U l S c Z b i v + L F g y J e / T + 8 + d + Y b j 3 o 5 o O Q x 3 u / H 3 l 5 Q c K o 0 o 7 z b S 0 t r 6 y u r V c 2 q p t b 2 z u 7 9 t 5 + R 4 l U Y t L G g g n Z C 5 A i j H L S 1 l Q z 0 k s k Q X H A S D c Y 3 x R + 9 5 5 I R Q V v 6 U l C v B g N O Y 0 o R t p I v n 0 4 C A Q L 1 S Q 2 V / a Q + 5 l 7 1 c p 9 u + b U n S n g I n F L U g M l m r 7 9 N Q g F T m P C N W Z I q b 7 r J N r L k N Q U M 5 J X B 6 k i C c J j N C R 9 Q z m K i f K y a f o c n h g l h J G Q 5 n A N p + r v j Q z F q g h o J m O k R 2 r e K 8 T / v H 6 q o 0 s v o z x J N e F 4 9 l C U M q g F L K q A I Z U E a z Y x B G F J T V a I R 0 g i r E 1 h V V O C O / / l R d I 5 q 7 t O 3 b 0 7 r z W u y z o q 4 A g c g 1 P g g g v Q A L e g C d o A g 0 f w D F 7 B m / V k v V j v 1 s d s d M k q d w 7 A H 1 i f P / x 2 l Y w = < / l a t e x i t > x t < l a t e x i t s h a 1 _ b a s e 6 4 = " G j N W D X V Z + X h H z G 1 c n k 8 M H n N / J K 8 = " > A A A B + X i c b V D L S g M x F L 3 j s 9 b X q E s 3 w S K 4 K j M i 6 L L o x m U F + 4 B 2 G D K Z t A 3 N J E O S K Z a h f + L G h S J u / R N 3 / o 2 Z d h b a e i D k c M 6 9 5 O R E K W f a e N 6 3 s 7 a + s b m 1 X d m p 7 u 7 t H x y 6 R 8 d t L T N F a I t I L l U 3 w p p y J m j L M M N p N 1 U U J x G n n W h 8 V / i d C V W a S f F o p i k N E j w U b M A I N l Y K X b c f S R 7 r a W K v / G k W m t C t e X V v D r R K / J L U o E Q z d L / 6 s S R Z Q o U h H G v d 8 7 3 U B D l W h h F O Z 9 V + p m m K y R g P a c 9 S g R O q g 3 y e f I b O r R K j g V T 2 C I P m 6 u + N H C e 6 C G c n E 2 x G e t k r x P + 8 X m Y G N 0 H O R J o Z K s j i o U H G k Z G o q A H F T F F i + N Q S T B S z W R E Z Y Y W J s W V V b Q n + 8 p d X S f u y 7 n t 1 / + G q 1 r g t 6 6 j A K Z z B B f h w D Q 2 4 h y a 0 g M A E n u E V 3 p z c e X H e n Y / F 6 J p T 7 p z A H z i f P 2 F t l C E = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " G j N W D X V Z + X h H z G 1 c n k 8 M H n N / J K 8 = " > A A A B + X i c b V D L S g M x F L 3 j s 9 b X q E s 3 w S K 4 K j M i 6 L L o x m U F + 4 B 2 G D K Z t A 3 N J E O S K Z a h f + L G h S J u / R N 3 / o 2 Z d h b a e i D k c M 6 9 5 O R E K W f a e N 6 3 s 7 a + s b m 1 X d m p 7 u 7 t H x y 6 R 8 d t L T N F a I t I L l U 3 w p p y J m j L M M N p N 1 U U J x G n n W h 8 V / i d C V W a S f F o p i k N E j w U b M A I N l Y K X b c f S R 7 r a W K v / G k W m t C t e X V v D r R K / J L U o E Q z d L / 6 s S R Z Q o U h H G v d 8 7 3 U B D l W h h F O Z 9 V + p m m K y R g P a c 9 S g R O q g 3 y e f I b O r R K j g V T 2 C I P m 6 u + N H C e 6 C G c n E 2 x G e t k r x P + 8 X m Y G N 0 H O R J o Z K s j i o U H G k Z G o q A H F T F F i + N Q S T B S z W R E Z Y Y W J s W V V b Q n + 8 p d X S f u y 7 n t 1 / + G q 1 r g t 6 6 j A K Z z B B f h w D Q 2 4 h y a 0 g M A E n u E V 3 p z c e X H e n Y / F 6 J p T 7 p z A H z i f P 2 F t l C E = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " G j N W D X V Z + X h H z G 1 c n k 8 M H n N / J K 8 = " > A A A B + X i c b V D L S g M x F L 3 j s 9 b X q E s 3 w S K 4 K j M i 6 L L o x m U F + 4 B 2 G D K Z t A 3 N J E O S K Z a h f + L G h S J u / R N 3 / o 2 Z d h b a e i D k c M 6 9 5 O R E K W f a e N 6 3 s 7 a + s b m 1 X d m p 7 u 7 t H x y 6 R 8 d t L T N F a I t I L l U 3 w p p y J m j L M M N p N 1 U U J x G n n W h 8 V / i d C V W a S f F o p i k N E j w U b M A I N l Y K X b c f S R 7 r a W K v / G k W m t C t e X V v D r R K / J L U o E Q z d L / 6 s S R Z Q o U h H G v d 8 7 3 U B D l W h h F O Z 9 V + p m m K y R g P a c 9 S g R O q g 3 y e f I b O r R K j g V T 2 C I P m 6 u + N H C e 6 C G c n E 2 x G e t k r x P + 8 X m Y G N 0 H O R J o Z K s j i o U H G k Z G o q A H F T F F i + N Q S T B S z W R E Z Y Y W J s W V V b Q n + 8 p d X S f u y 7 n t 1 / + G q 1 r g t 6 6 j A K Z z B B f h w D Q 2 4 h y a 0 g M A E n u E V 3 p z c e X H e n Y / F 6 J p T 7 p z A H z i f P 2 F t l C E = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " G j N W D X V Z + X h H z G 1 c n k 8 M H n N / J K 8 = " > A A A B + X i c b V D L S g M x F L 3 j s 9 b X q E s 3 w S K 4 K j M i 6 L L o x m U F + 4 B 2 G D K Z t A 3 N J E O S K Z a h f + L G h S J u / R N 3 / o 2 Z d h b a e i D k c M 6 9 5 O R E K W f a e N 6 3 s 7 a + s b m 1 X d m p 7 u 7 t H x y 6 R 8 d t L T N F a I t I L l U 3 w p p y J m j L M M N p N 1 U U J x G n n W h 8 V / i d C V W a S f F o p i k N E j w U b M A I N l Y K X b c f S R 7 r a W K v / G k W m t C t e X V v D r R K / J L U o E Q z d L / 6 s S R Z Q o U h H G v d 8 7 3 U B D l W h h F O Z 9 V + p m m K y R g P a c 9 S g R O q g 3 y e f I b O r R K j g V T 2 C I P m 6 u + N H C e 6 C G c n E 2 x G e t k r x P + 8 X m Y G N 0 H O R J o Z K s j i o U H G k Z G o q A H F T F F i + N Q S T B S z W R E Z Y Y W J s W V V b Q n + 8 p d X S f u y 7 n t 1 / + G q 1 r g t 6 6 j A K Z z B B f h w D Q 2 4 h y a 0 g M A E n u E V 3 p z c e X H e n Y / F 6 J p T 7 p z A H z i f P 2 F t l C E = < / l a t e x i t > µ (x 1:T ) < l a t e x i t s h a 1 _ b a s e 6 4 = " V r v t K 3 D 1 q y m n q a W G d n 2 Z x r Y w Y f I = " > A A A C J H i c b V D L S s N A F J 3 U V 6 2 v q E s 3 g 0 W o m 5 K I o O i m 6 M Z l h b 6 g C W E y m b Z D J 5 M w M x F L y M e 4 8 V f c u P C B C z d + i 5 M 2 C 9 t 6 Y Z j D O e d y 7 z 1 + z K h U l v V t l F Z W 1 9 Y 3 y p u V r e 2 d 3 T 1 z / 6 A j o 0 R g 0 s Y R i 0 T P R 5 I w y k l b U c V I L x Y E h T 4 j X X 9 8 m + v d B y I k j X h L T W L i h m j I 6 Y B i p D T l m d e O H 7 F A T k L 9 p U 6 Y Z F 4 6 x 8 Q j m m W w 9 p d 7 1 B 7 7 q p W d e m b V q l v T g s v A L k A V F N X 0 z A 8 n i H A S E q 4 w Q 1 L 2 b S t W b o q E o p i R r O I k k s Q I j 9 G Q 9 D X k K C T S T a d H Z v B E M w E c R E I / r u C U / d u R o l D m G 2 p n i N R I L m o 5 + Z / W T 9 T g 0 k 0 p j x N F O J 4 N G i Q M q g j m i c G A C o I V m 2 i A s K B 6 V 4 h H S C C s d K 4 V H Y K 9 e P I y 6 J z V b a t u 3 5 9 X G z d F H G V w B I 5 B D d j g A j T A H W i C N s D g C b y A N / B u P B u v x q f x N b O W j K L n E M y V 8 f M L 9 8 G m y w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " V r v t K 3 D 1 q y m n q a W G d n 2 Z x r Y w Y f I = " > A A A C J H i c b V D L S s N A F J 3 U V 6 2 v q E s 3 g 0 W o m 5 K I o O i m 6 M Z l h b 6 g C W E y m b Z D J 5 M w M x F L y M e 4 8 V f c u P C B C z d + i 5 M 2 C 9 t 6 Y Z j D O e d y 7 z 1 + z K h U l v V t l F Z W 1 9 Y 3 y p u V r e 2 d 3 T 1 z / 6 A j o 0 R g 0 s Y R i 0 T P R 5 I w y k l b U c V I L x Y E h T 4 j X X 9 8 m + v d B y I k j X h L T W L i h m j I 6 Y B i p D T l m d e O H 7 F A T k L 9 p U 6 Y Z F 4 6 x 8 Q j m m W w 9 p d 7 1 B 7 7 q p W d e m b V q l v T g s v A L k A V F N X 0 z A 8 n i H A S E q 4 w Q 1 L 2 b S t W b o q E o p i R r O I k k s Q I j 9 G Q 9 D X k K C T S T a d H Z v B E M w E c R E I / r u C U / d u R o l D m G 2 p n i N R I L m o 5 + Z / W T 9 T g 0 k 0 p j x N F O J 4 N G i Q M q g j m i c G A C o I V m 2 i A s K B 6 V 4 h H S C C s d K 4 V H Y K 9 e P I y 6 J z V b a t u 3 5 9 X G z d F H G V w B I 5 B D d j g A j T A H W i C N s D g C b y A N / B u P B u v x q f x N b O W j K L n E M y V 8 f M L 9 8 G m y w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " V r v t K 3 D 1 q y m n q a W G d n 2 Z x r Y w Y f I = " > A A A C J H i c b V D L S s N A F J 3 U V 6 2 v q E s 3 g 0 W o m 5 K I o O i m 6 M Z l h b 6 g C W E y m b Z D J 5 M w M x F L y M e 4 8 V f c u P C B C z d + i 5 M 2 C 9 t 6 Y Z j D O e d y 7 z 1 + z K h U l v V t l F Z W 1 9 Y 3 y p u V r e 2 d 3 T 1 z / 6 A j o 0 R g 0 s Y R i 0 T P R 5 I w y k l b U c V I L x Y E h T 4 j X X 9 8 m + v d B y I k j X h L T W L i h m j I 6 Y B i p D T l m d e O H 7 F A T k L 9 p U 6 Y Z F 4 6 x 8 Q j m m W w 9 p d 7 1 B 7 7 q p W d e m b V q l v T g s v A L k A V F N X 0 z A 8 n i H A S E q 4 w Q 1 L 2 b S t W b o q E o p i R r O I k k s Q I j 9 G Q 9 D X k K C T S T a d H Z v B E M w E c R E I / r u C U / d u R o l D m G 2 p n i N R I L m o 5 + Z / W T 9 T g 0 k 0 p j x N F O J 4 N G i Q M q g j m i c G A C o I V m 2 i A s K B 6 V 4 h H S C C s d K 4 V H Y K 9 e P I y 6 J z V b a t u 3 5 9 X G z d F H G V w B I 5 B D d j g A j T A H W i C N s D g C b y A N / B u P B u v x q f x N b O W j K L n E M y V 8 f M L 9 8 G m y w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " V r v t K 3 D 1 q y m n q a W G d n 2 Z x r Y w Y f I = " > A A A C J H i c b V D L S s N A F J 3 U V 6 2 v q E s 3 g 0 W o m 5 K I o O i m 6 M Z l h b 6 g C W E y m b Z D J 5 M w M x F L y M e 4 8 V f c u P C B C z d + i 5 M 2 C 9 t 6 Y Z j D O e d y 7 z 1 + z K h U l v V t l F Z W 1 9 Y 3 y p u V r e 2 d 3 T 1 z / 6 A j o 0 R g 0 s Y R i 0 T P R 5 I w y k l b U c V I L x Y E h T 4 j X X 9 8 m + v d B y I k j X h L T W L i h m j I 6 Y B i p D T l m d e O H 7 F A T k L 9 p U 6 Y Z F 4 6 x 8 Q j m m W w 9 p d 7 1 B 7 7 q p W d e m b V q l v T g s v A L k A V F N X 0 z A 8 n i H A S E q 4 w Q 1 L 2 b S t W b o q E o p i R r O I k k s Q I j 9 G Q 9 D X k K C T S T a d H Z v B E M w E c R E I / r u C U / d u R o l D m G 2 p n i N R I L m o 5 + Z / W T 9 T g 0 k 0 p j x N F O J 4 N G i Q M q g j m i c G A C o I V m 2 i A s K B 6 V 4 h H S C C s d K 4 V H Y K 9 e P I y 6 J z V b a t u 3 5 9 X G z d F H G V w B I 5 B D d j g A j T A H W i C N s D g C b y A N / B u P B u v x q f x N b O W j K L n E M y V 8 f M L 9 8 G m y w = = < / l a t e x i t > µ (x t ) < l a t e x i t s h a 1 _ b a s e 6 4 = " c / c N S D Y k t E l G k 5 W 1 Y W l Z e 5 s y 2 F M = " > A A A C I H i c b V D L S s N A F J 3 4 r P U V d e l m s A h 1 U x I R 6 r L o x m U F + 4 A m h M l k 0 g 6 d T M L M R C w h n + L G X 3 H j Q h H d 6 d c 4 a b P o w w P D H M 6 5 l 3 v v 8 R N G p b K s H 2 N t f W N z a 7 u y U 9 3 d 2 z 8 4 N I + O u z J O B S Y d H L N Y 9 H 0 k C a O c d B R V j P Q T Q V D k M 9 L z x 7 e F 3 3 s k Q t K Y P 6 h J Q t w I D T k N K U Z K S 5 7 Z d P y Y B X I S 6 S 9 z o j T 3 s g U l G d E 8 h / V 5 7 S n 3 1 I V n 1 q y G N Q V c J X Z J a q B E 2 z O / n S D G a U S 4 w g x J O b C t R L k Z E o p i R v K q k 0 q S I D x G Q z L Q l K O I S D e b H p j D c 6 0 E M I y F f l z B q T r f k a F I F t v p y g i p k V z 2 C v E / b 5 C q 8 N r N K E 9 S R T i e D Q p T B l U M i 7 R g Q A X B i k 0 0 Q V h Q v S v E I y Q Q V j r T q g 7 B X j 5 5 l X Q v G 7 b V s O + v a q 2 b M o 4 K O A V n o A 5 s 0 A Q t c A f a o A M w e A a v 4 B 1 8 G C / G m / F p f M 1 K 1 4 y y 5 w Q s w P j 9 A x v 9 p W A = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " c / c N S D Y k t E l G k 5 W 1 Y W l Z e 5 s y 2 F M = " > A A A C I H i c b V D L S s N A F J 3 4 r P U V d e l m s A h 1 U x I R 6 r L o x m U F + 4 A m h M l k 0 g 6 d T M L M R C w h n + L G X 3 H j Q h H d 6 d c 4 a b P o w w P D H M 6 5 l 3 v v 8 R N G p b K s H 2 N t f W N z a 7 u y U 9 3 d 2 z 8 4 N I + O u z J O B S Y d H L N Y 9 H 0 k C a O c d B R V j P Q T Q V D k M 9 L z x 7 e F 3 3 s k Q t K Y P 6 h J Q t w I D T k N K U Z K S 5 7 Z d P y Y B X I S 6 S 9 z o j T 3 s g U l G d E 8 h / V 5 7 S n 3 1 I V n 1 q y G N Q V c J X Z J a q B E 2 z O / n S D G a U S 4 w g x J O b C t R L k Z E o p i R v K q k 0 q S I D x G Q z L Q l K O I S D e b H p j D c 6 0 E M I y F f l z B q T r f k a F I F t v p y g i p k V z 2 C v E / b 5 C q 8 N r N K E 9 S R T i e D Q p T B l U M i 7 R g Q A X B i k 0 0 Q V h Q v S v E I y Q Q V j r T q g 7 B X j 5 5 l X Q v G 7 b V s O + v a q 2 b M o 4 K O A V n o A 5 s 0 A Q t c A f a o A M w e A a v 4 B 1 8 G C / G m / F p f M 1 K 1 4 y y 5 w Q s w P j 9 A x v 9 p W A = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " c / c N S D Y k t E l G k 5 W 1 Y W l Z e 5 s y 2 F M = " > A A A C I H i c b V D L S s N A F J 3 4 r P U V d e l m s A h 1 U x I R 6 r L o x m U F + 4 A m h M l k 0 g 6 d T M L M R C w h n + L G X 3 H j Q h H d 6 d c 4 a b P o w w P D H M 6 5 l 3 v v 8 R N G p b K s H 2 N t f W N z a 7 u y U 9 3 d 2 z 8 4 N I + O u z J O B S Y d H L N Y 9 H 0 k C a O c d B R V j P Q T Q V D k M 9 L z x 7 e F 3 3 s k Q t K Y P 6 h J Q t w I D T k N K U Z K S 5 7 Z d P y Y B X I S 6 S 9 z o j T 3 s g U l G d E 8 h / V 5 7 S n 3 1 I V n 1 q y G N Q V c J X Z J a q B E 2 z O / n S D G a U S 4 w g x J O b C t R L k Z E o p i R v K q k 0 q S I D x G Q z L Q l K O I S D e b H p j D c 6 0 E M I y F f l z B q T r f k a F I F t v p y g i p k V z 2 C v E / b 5 C q 8 N r N K E 9 S R T i e D Q p T B l U M i 7 R g Q A X B i k 0 0 Q V h Q v S v E I y Q Q V j r T q g 7 B X j 5 5 l X Q v G 7 b V s O + v a q 2 b M o 4 K O A V n o A 5 s 0 A Q t c A f a o A M w e A a v 4 B 1 8 G C / G m / F p f M 1 K 1 4 y y 5 w Q s w P j 9 A x v 9 p W A = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " c / c N S D Y k t E l G k 5 W 1 Y W l Z e 5 s y 2 F M = " > A A A C I H i c b V D L S s N A F J 3 4 r P U V d e l m s A h 1 U x I R 6 r L o x m U F + 4 A m h M l k 0 g 6 d T M L M R C w h n + L G X 3 H j Q h H d 6 d c 4 a b P o w w P D H M 6 5 l 3 v v 8 R N G p b K s H 2 N t f W N z a 7 u y U 9 3 d 2 z 8 4 N I + O u z J O B S Y d H L N Y 9 H 0 k C a O c d B R V j P Q T Q V D k M 9 L z x 7 e F 3 3 s k Q t K Y P 6 h J Q t w I D T k N K U Z K S 5 7 Z d P y Y B X I S 6 S 9 z o j T 3 s g U l G d E 8 h / V 5 7 S n 3 1 I V n 1 q y G N Q V c J X Z J a q B E 2 z O / n S D G a U S 4 w g x J O b C t R L k Z E o p i R v K q k 0 q S I D x G Q z L Q l K O I S D e b H p j D c 6 0 E M I y F f l z B q T r f k a F I F t v p y g i p k V z 2 C v E / b 5 C q 8 N r N K E 9 S R T i e D Q p T B l U M i 7 R g Q A X B i k 0 0 Q V h Q v S v E I y Q Q V j r T q g 7 B X j 5 5 l X Q v G 7 b V s O + v a q 2 b M o 4 K O A V n o A 5 s 0 A Q t c A f a o A M w e A a v 4 B 1 8 G C / G m / F p f M 1 K 1 4 y y 5 w Q s w P j 9 A x v 9 p W A = < / l a t e x i t > f < l a t e x i t s h a 1 _ b a s e 6 4 = " K T 3 g y m 1 1 X A j K B t m Q l x p g o E 5 y 3 + 0 = " > A A A B 9 X i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I v g q s y I U J d F N y 4 r 2 A e 0 Y 8 l k M m 1 o J h m S j F K G / o c b F 4 q 4 9 V / c + T d m 2 l l o 6 4 G Q w z n 3 k p M T J J x p 4 7 r f T m l t f W N z q 7 x d 2 d n d 2 z + o H h 5 1 t E w V o W 0 i u V S 9 A G v K m a B t w w y n v U R R H A e c d o P J T e 5 3 H 6 n S T I p 7 M 0 2 o H + O R Y B E j 2 F j p Y R B I H u p p b K 8 s m g 2 r N b f u z o F W i V e Q G h R o D a t f g 1 C S N K b C E I 6 1 7 n t u Y v w M K 8 M I p 7 P K I N U 0 w W S C R 7 R v q c A x 1 X 4 2 T z 1 D Z 1 Y J U S S V P c K g u f p 7 I 8 O x z q P Z y R i b s V 7 2 c v E / r 5 + a 6 M r P m E h S Q w V Z P B S l H B m J 8 g p Q y B Q l h k 8 t w U Q x m x W R M V a Y G F t U x Z b g L X 9 5 l X Q u 6 p 5 b 9 + 4 u a 8 3 r o o 4 y n M A p n I M H D W j C L b S g D Q Q U P M M r v D l P z o v z 7 n w s R k t O s X M M f + B 8 / g A 3 8 p L 3 < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " K T 3 g y m 1 1 X A j K B t m Q l x p g o E 5 y 3 + 0 = " > A A A B 9 X i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I v g q s y I U J d F N y 4 r 2 A e 0 Y 8 l k M m 1 o J h m S j F K G / o c b F 4 q 4 9 V / c + T d m 2 l l o 6 4 G Q w z n 3 k p M T J J x p 4 7 r f T m l t f W N z q 7 x d 2 d n d 2 z + o H h 5 1 t E w V o W 0 i u V S 9 A G v K m a B t w w y n v U R R H A e c d o P J T e 5 3 H 6 n S T I p 7 M 0 2 o H + O R Y B E j 2 F j p Y R B I H u p p b K 8 s m g 2 r N b f u z o F W i V e Q G h R o D a t f g 1 C S N K b C E I 6 1 7 n t u Y v w M K 8 M I p 7 P K I N U 0 w W S C R 7 R v q c A x 1 X 4 2 T z 1 D Z 1 Y J U S S V P c K g u f p 7 I 8 O x z q P Z y R i b s V 7 2 c v E / r 5 + a 6 M r P m E h S Q w V Z P B S l H B m J 8 g p Q y B Q l h k 8 t w U Q x m x W R M V a Y G F t U x Z b g L X 9 5 l X Q u 6 p 5 b 9 + 4 u a 8 3 r o o 4 y n M A p n I M H D W j C L b S g D Q Q U P M M r v D l P z o v z 7 n w s R k t O s X M M f + B 8 / g A 3 8 p L 3 < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " K T 3 g y m 1 1 X A j K B t m Q l x p g o E 5 y 3 + 0 = " > A A A B 9 X i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I v g q s y I U J d F N y 4 r 2 A e 0 Y 8 l k M m 1 o J h m S j F K G / o c b F 4 q 4 9 V / c + T d m 2 l l o 6 4 G Q w z n 3 k p M T J J x p 4 7 r f T m l t f W N z q 7 x d 2 d n d 2 z + o H h 5 1 t E w V o W 0 i u V S 9 A G v K m a B t w w y n v U R R H A e c d o P J T e 5 3 H 6 n S T I p 7 M 0 2 o H + O R Y B E j 2 F j p Y R B I H u p p b K 8 s m g 2 r N b f u z o F W i V e Q G h R o D a t f g 1 C S N K b C E I 6 1 7 n t u Y v w M K 8 M I p 7 P K I N U 0 w W S C R 7 R v q c A x 1 X 4 2 T z 1 D Z 1 Y J U S S V P c K g u f p 7 I 8 O x z q P Z y R i b s V 7 2 c v E / r 5 + a 6 M r P m E h S Q w V Z P B S l H B m J 8 g p Q y B Q l h k 8 t w U Q x m x W R M V a Y G F t U x Z b g L X 9 5 l X Q u 6 p 5 b 9 + 4 u a 8 3 r o o 4 y n M A p n I M H D W j C L b S g D Q Q U P M M r v D l P z o v z 7 n w s R k t O s X M M f + B 8 / g A 3 8 p L 3 < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " K T 3 g y m 1 1 X A j K B t m Q l x p g o E 5 y 3 + 0 = " > A A A B 9 X i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I v g q s y I U J d F N y 4 r 2 A e 0 Y 8 l k M m 1 o J h m S j F K G / o c b F 4 q 4 9 V / c + T d m 2 l l o 6 4 G Q w z n 3 k p M T J J x p 4 7 r f T m l t f W N z q 7 x d 2 d n d 2 z + o H h 5 1 t E w V o W 0 i u V S 9 A G v K m a B t w w y n v U R R H A e c d o P J T e 5 3 H 6 n S T I p 7 M 0 2 o H + O R Y B E j 2 F j p Y R B I H u p p b K 8 s m g 2 r N b f u z o F W i V e Q G h R o D a t f g 1 C S N K b C E I 6 1 7 n t u Y v w M K 8 M I p 7 P K I N U 0 w W S C R 7 R v q c A x 1 X 4 2 T z 1 D Z 1 Y J U S S V P c K g u f p 7 I 8 O x z q P Z y R i b s V 7 2 c v E / r 5 + a 6 M r P m E h S Q w V Z P B S l H B m J 8 g p Q y B Q l h k 8 t w U Q x m x W R M V a Y G F t U x Z b g L X 9 5 l X Q u 6 p 5 b 9 + 4 u a 8 3 r o o 4 y n M A p n I M H D W j C L b S g D Q Q U P M M r v D l P z o v z 7 n w s R k t O s X M M f + B 8 / g A 3 8 p L 3 < / l a t e x i t > z t < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 + b y p 7 0 t W v s x b o f l 6 u Q H E 4 o v R B w = " > A A A B + X i c b V D L S g M x F L 3 j s 9 b X q E s 3 w S K 4 K j M i 6 L L o x m U F + 4 B 2 G D K Z t A 3 N J E O S K d S h f + L G h S J u / R N 3 / o 2 Z d h b a e i D k c M 6 9 5 O R E K W f a e N 6 3 s 7 a + s b m 1 X d m p 7 u 7 t H x y 6 R 8 d t L T N F a I t I L l U 3 w p p y J m j L M M N p N 1 U U J x G n n W h 8 V / i d C V W a S f F o p i k N E j w U b M A I N l Y K X b c f S R 7 r a W K v / G k W m t C t e X V v D r R K / J L U o E Q z d L / 6 s S R Z Q o U h H G v d 8 7 3 U B D l W h h F O Z 9 V + p m m K y R g P a c 9 S g R O q g 3 y e f I b O r R K j g V T 2 C I P m 6 u + N H C e 6 C G c n E 2 x G e t k r x P + 8 X m Y G N 0 H O R J o Z K s j i o U H G k Z G o q A H F T F F i + N Q S T B S z W R E Z Y Y W J s W V V b Q n + 8 p d X S f u y 7 n t 1 / + G q 1 r g t 6 6 j A K Z z B B f h w D Q 2 4 h y a 0 g M A E n u E V 3 p z c e X H e n Y / F 6 J p T 7 p z A H z i f P 2 R 7 l C M = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 + b y p 7 0 t W v s x b o f l 6 u Q H E 4 o v R B w = " > A A A B + X i c b V D L S g M x F L 3 j s 9 b X q E s 3 w S K 4 K j M i 6 L L o x m U F + 4 B 2 G D K Z t A 3 N J E O S K d S h f + L G h S J u / R N 3 / o 2 Z d h b a e i D k c M 6 9 5 O R E K W f a e N 6 3 s 7 a + s b m 1 X d m p 7 u 7 t H x y 6 R 8 d t L T N F a I t I L l U 3 w p p y J m j L M M N p N 1 U U J x G n n W h 8 V / i d C V W a S f F o p i k N E j w U b M A I N l Y K X b c f S R 7 r a W K v / G k W m t C t e X V v D r R K / J L U o E Q z d L / 6 s S R Z Q o U h H G v d 8 7 3 U B D l W h h F O Z 9 V + p m m K y R g P a c 9 S g R O q g 3 y e f I b O r R K j g V T 2 C I P m 6 u + N H C e 6 C G c n E 2 x G e t k r x P + 8 X m Y G N 0 H O R J o Z K s j i o U H G k Z G o q A H F T F F i + N Q S T B S z W R E Z Y Y W J s W V V b Q n + 8 p d X S f u y 7 n t 1 / + G q 1 r g t 6 6 j A K Z z B B f h w D Q 2 4 h y a 0 g M A E n u E V 3 p z c e X H e n Y / F 6 J p T 7 p z A H z i f P 2 R 7 l C M = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 + b y p 7 0 t W v s x b o f l 6 u Q H E 4 o v R B w = " > A A A B + X i c b V D L S g M x F L 3 j s 9 b X q E s 3 w S K 4 K j M i 6 L L o x m U F + 4 B 2 G D K Z t A 3 N J E O S K d S h f + L G h S J u / R N 3 / o 2 Z d h b a e i D k c M 6 9 5 O R E K W f a e N 6 3 s 7 a + s b m 1 X d m p 7 u 7 t H x y 6 R 8 d t L T N F a I t I L l U 3 w p p y J m j L M M N p N 1 U U J x G n n W h 8 V / i d C V W a S f F o p i k N E j w U b M A I N l Y K X b c f S R 7 r a W K v / G k W m t C t e X V v D r R K / J L U o E Q z d L / 6 s S R Z Q o U h H G v d 8 7 3 U B D l W h h F O Z 9 V + p m m K y R g P a c 9 S g R O q g 3 y e f I b O r R K j g V T 2 C I P m 6 u + N H C e 6 C G c n E 2 x G e t k r x P + 8 X m Y G N 0 H O R J o Z K s j i o U H G k Z G o q A H F T F F i + N Q S T B S z W R E Z Y Y W J s W V V b Q n + 8 p d X S f u y 7 n t 1 / + G q 1 r g t 6 6 j A K Z z B B f h w D Q 2 4 h y a 0 g M A E n u E V 3 p z c e X H e n Y / F 6 J p T 7 p z A H z i f P 2 R 7 l C M = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 + b y p 7 0 t W v s x b o f l 6 u Q H E 4 o v R B w = " > A A A B + X i c b V D L S g M x F L 3 j s 9 b X q E s 3 w S K 4 K j M i 6 L L o x m U F + 4 B 2 G D K Z t A 3 N J E O S K d S h f + L G h S J u / R N 3 / o 2 Z d h b a e i D k c M 6 9 5 O R E K W f a e N 6 3 s 7 a + s b m 1 X d m p 7 u 7 t H x y 6 R 8 d t L T N F a I t I L l U 3 w p p y J m j L M M N p N 1 U U J x G n n W h 8 V / i d C V W a S f F o p i k N E j w U b M A I N l Y K X b c f S R 7 r a W K v / G k W m t C t e X V v D r R K / J L U o E Q z d L / 6 s S R Z Q o U h H G v d 8 7 3 U B D l W h h F O Z 9 V + p m m K y R g P a c 9 S g R O q g 3 y e f I b O r R K j g V T 2 C I P m 6 u + N H C e 6 C G c n E 2 x G e t k r x P + 8 X m Y G N 0 H O R J o Z K s j i o U H G k Z G o q A H F T F F i + N Q S T B S z W R E Z Y Y W J s W V V b Q n + 8 p d X S f u y 7 n t 1 / + G q 1 r g t 6 6 j A K Z z B B f h w D Q 2 4 h y a 0 g M A E n u E V 3 p z c e X H e n Y / F 6 J p T 7 p z A H z i f P 2 R 7 l C M = < / l a t e x i t > Quantization zt < l a t e x i t s h a 1 _ b a s e 6 4 = " Y t 2 K 9 k 0 M R g U K a C E U N N m L v H Q Q c U M = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y L U E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E x K 6 0 Z V c 8 + + g P j 8 w f 1 p p f Z < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Y t 2 K 9 k 0 M R g U K a C E U N N m L v H Q Q c U M = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y L U E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E x K 6 0 Z V c 8 + + g P j 8 w f 1 p p f Z < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Y t 2 K 9 k 0 M R g U K a C E U N N m L v H Q Q c U M = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y L U E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E x K 6 0 Z V c 8 + + g P j 8 w f 1 p p f Z < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Y t 2 K 9 k 0 M R g U K a C E U N N m L v H Q Q c U M = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y L U E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E x K 6 0 Z V c 8 + + g P j 8 w f 1 p p f Z < / l a t e x i t > Encoders AE AE Binary AD AD zt < l a t e x i t s h a 1 _ b a s e 6 4 = " Y t 2 K 9 k 0 M R g U K a C E U N N m L v H Q Q c U M = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y L U E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E x K 6 0 Z V c 8 + + g P j 8 w f 1 p p f Z < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Y t 2 K 9 k 0 M R g U K a C E U N N m L v H Q Q c U M = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y L U E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E x K 6 0 Z V c 8 + + g P j 8 w f 1 p p f Z < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Y t 2 K 9 k 0 M R g U K a C E U N N m L v H Q Q c U M = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y L U E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E x K 6 0 Z V c 8 + + g P j 8 w f 1 p p f Z < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Y t 2 K 9 k 0 M R g U K a C E U N N m L v H Q Q c U M = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y L U E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E x K 6 0 Z V c 8 + + g P j 8 w f 1 p p f Z < / l a t e x i t > Decoder xt < l a t e x i t s h a 1 _ b a s e 6 4 = " X 4 Q 1 a F m 5 X r H W D o + I K Y E V u O 4 2 e N 8 = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y K W E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E

x K 6 0 Z V c 8 + + g P j 8 w f y l p f X < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = "

X 4 Q 1 a F m 5 X r H W D o + I K Y E V u O 4 2 e N 8 = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y K W E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E

x K 6 0 Z V c 8 + + g P j 8 w f y l p f X < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = "

X 4 Q 1 a F m 5 X r H W D o + I K Y E V u O 4 2 e N 8 = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y K W E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E

x K 6 0 Z V c 8 + + g P j 8 w f y l p f X < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = "

X 4 Q 1 a F m 5 X r H W D o + I K Y E V u O 4 2 e N 8 = " > A A A C A X i c b V B N S 8 N A E N 3 U r 1 q / o l 4 E L 8 E i e C q J C H o s e v F Y w b Z C E 8 J m s 2 m X b j Z h d y K W E C / + F S 8 e F P H q v / D m v 3 H T 5 q C t D 5 Z 9 v D f D z L w g 5 U y B b X 8 b t a X l l d W 1 + n p j Y 3 N r e 8 f c 3 e u p J J O E d k n C E 3 k X Y E U 5 E 7 Q L D D i 9 S y X F c c B p P x h f l X 7 / n k r F E n E L k 5 R 6 M R 4 K F j G C Q U u + e e A C 4 y H N 3 S D h o Z r E + s s f i s I H 3 2 z a L X s K a 5 E 4 F W m i C h 3 f / H L D h G Q x F U A 4 V m r g 2 C l 4 O Z b A C K d F w 8 0 U T T E Z 4 y E d a C p w T J W X T y 8 o r G O t h F a U S P 0 E W F P 1 d 0 e O Y 1 V u p y t j D C M 1 7 5 X i f 9 4 g g + j C y 5 l I M 6 C C z A Z F G b c g s c o 4 r J B J S o B P N M F E M r 2 r R U Z Y Y g I 6 t I Y O w Z k / e Z

H 0 T l u O 3 X J u z p r t y y q O O j p E R + g E O e g c t d E 1 6 q A u I u g R P a N X 9 G Y 8 G S / G u / E x K 6 0 Z V c 8 + + g P j 8 w f y l p f X < / l a t e x i t >

Input Video Reconstructed Video N o is e R o u n d:



Training:

After training p ✓ (z t | z<t )

< l a t e x i t s h a 1 _ b a s e 6 4 = " L b Q g D 7 k q K z y 9 e S / a 3 c S 6 e P E a Y Z w = " >

A A A C P H i c d V A 9 S w N B E N 3 z 2 / g V t b R Z D E J s w p 0 I W l i I N p Y R T S L k j m N v b 2 I W 9 z 7 Y n R P i c T / M x h 9 h Z 2 V j o Y i t t X t J C j X 6 Y N n H e z P M z A t S K T T a 9 p M 1 N T 0 z O z e / s F h Z W l 5 Z X a u u b 7 R 1 k i k O L Z 7 I R F 0 F T I M U M b R Q o I S r V A G L A g m d 4 O a 0 9 D u 3 o L R I 4 k s c p O B F 7 D o W P c E Z G s m v X q R + 7 g a J D P U g M l / u Y h + Q F Q W t u y h k C D / M u 6 L w k b q R C O k / b n 6 E x a 5 f r d k N e w g 6 S Z w x q Z E x m n 7 1 0 Q 0 T n k U Q I 5 d M 6 6 5 j p + j l T K H g E o q K m 2 l I G b 9 h 1 9 A 1 N G Y R a C 8 f H l / Q H a O E t J c o 8 2 K k Q / V 7 R 8 4 i X W 5 o K i O G f f 3 b K 8 W / v G 6 G v U M v F 3 G a I c R 8 N K i X S Y o J L Z O k o V D A U Q 4 M Y V w J s y v l f a Y Y R 5 N 3 x Y T g / D 5 5 k r T 3 G o 7 d c M 7 3 a 8 c n 4 z g W y B b Z J n X i k A N y T M 5 I k 7 Q I J / f k m b y S N + v B e r H e r Y 9 R 6 Z Q 1 7 t k k P 2 B 9 f g E o T b G k < /

l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " L b Q g D 7 k q K z y 9 e S / a 3 c S 6 e P E a Y Z w = " >

A A A C P H i c d V A 9 S w N B E N 3 z 2 / g V t b R Z D E J s w p 0 I W l i I N p Y R T S L k j m N v b 2 I W 9 z 7 Y n R P i c T / M x h 9 h Z 2 V j o Y i t t X t J C j X 6 Y N n H e z P M z A t S K T T a 9 p M 1 N T 0 z O z e / s F h Z W l 5 Z X a u u b 7 R 1 k i k O L Z 7 I R F 0 F T I M U M b R Q o I S r V A G L A g m d 4 O a 0 9 D u 3 o L R I 4 k s c p O B F 7 D o W P c E Z G s m v X q R + 7 g a J D P U g M l / u Y h + Q F Q W t u y h k C D / M u 6 L w k b q R C O k / b n 6 E x a 5 f r d k N e w g 6 S Z w x q Z E x m n 7 1 0 Q 0 T n k U Q I 5 d M 6 6 5 j p + j l T K H g E o q K m 2 l I G b 9 h 1 9 A 1 N G Y R a C 8 f H l / Q H a O E t J c o 8 2 K k Q / V 7 R 8 4 i X W 5 o K i O G f f 3 b K 8 W / v G 6 G v U M v F 3 G a I c R 8 N K i X S Y o J L Z O k o V D A U Q 4 M Y V w J s y v l f a Y Y R 5 N 3 x Y T g / D 5 5 k r T 3 G o 7 d c M 7 3 a 8 c n 4 z g W y B b Z J n X i k A N y T M 5 I k 7 Q I J / f k m b y S N + v B e r H e r Y 9 R 6 Z Q 1 7 t k k P 2 B 9 f g E o T b G k < /

l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " L b Q g D 7 k q K z y 9 e S / a 3 c S 6 e P E a Y Z w = " >

A A A C P H i c d V A 9 S w N B E N 3 z 2 / g V t b R Z D E J s w p 0 I W l i I N p Y R T S L k j m N v b 2 I W 9 z 7 Y n R P i c T / M x h 9 h Z 2 V j o Y i t t X t J C j X 6 Y N n H e z P M z A t S K T T a 9 p M 1 N T 0 z O z e / s F h Z W l 5 Z X a u u b 7 R 1 k i k O L Z 7 I R F 0 F T I M U M b R Q o I S r V A G L A g m d 4 O a 0 9 D u 3 o L R I 4 k s c p O B F 7 D o W P c E Z G s m v X q R + 7 g a J D P U g M l / u Y h + Q F Q W t u y h k C D / M u 6 L w k b q R C O k / b n 6 E x a 5 f r d k N e w g 6 S Z w x q Z E x m n 7 1 0 Q 0 T n k U Q I 5 d M 6 6 5 j p + j l T K H g E o q K m 2 l I G b 9 h 1 9 A 1 N G Y R a C 8 f H l / Q H a O E t J c o 8 2 K k Q / V 7 R 8 4 i X W 5 o K i O G f f 3 b K 8 W / v G 6 G v U M v F 3 G a I c R 8 N K i X S Y o J L Z O k o V D A U Q 4 M Y V w J s y v l f a Y Y R 5 N 3 x Y T g / D 5 5 k r T 3 G o 7 d c M 7 3 a 8 c n 4 z g W y B b Z J n X i k A N y T M 5 I k 7 Q I J / f k m b y S N + v B e r H e r Y 9 R 6 Z Q 1 7 t k k P 2 B 9 f g E o T b G k < /

l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " L b Q g D 7 k q K z y 9 e S / a 3 c S 6 e P E a Y Z w = " >

A A A C P H i c d V A 9 S w N B E N 3 z 2 / g V t b R Z D E J s w p 0 I W l i I N p Y R T S L k j m N v b 2 I W 9 z 7 Y n R P i c T / M x h 9 h Z 2 V j o Y i t t X t J C j X 6 Y N n H e z P M z A t S K T T a 9 p M 1 N T 0 z O z e / s F h Z W l 5 Z X a u u b 7 R 1 k i k O L Z 7 I R F 0 F T I M U M b R Q o I S r V A G L A g m d 4 O a 0 9 D u 3 o L R I 4 k s c p O B F 7 D o W P c E Z G s m v X q R + 7 g a J D P U g M l / u Y h + Q F Q W t u y h k C D / M u 6 L w k b q R C O k / b n 6 E x a 5 f r d k N e w g 6 S Z w x q Z E x m n 7 1 0 Q 0 T n k U Q I 5 d M 6 6 5 j p + j l T K H g E o q K m 2 l I G b 9 h 1 9 A 1 N G Y R a C 8 f H l / Q H a O E t J c o 8 2 K k Q / V 7 R 8 4 i X W 5 o K i O G f f 3 b K 8 W / v G 6 G v U M v F 3 G a I c R 8 N K i X S Y o J L Z O k o V D A U Q 4 M Y V w J s y v l f a Y Y R 5 N 3 x Y T g / D 5 5 k r T 3 G o 7 d c M 7 3 a 8 c n 4 z g W y B b Z J n X i k A N y T M 5 I k 7 Q I J / f k m b y S N + v B e r H e r Y 9 R 6 Z Q 1 7 t k k P 2 B 9 f g E o T b G k < / l a t e x i t > f < l a t e x i t s h a 1 _ b a s e 6 4 = " a o b g V s h m D 4 k + W f M e r 3 P z P p R O s l E = " > A A A B / 3 i c b V D L S s N A F J 3 U V 6 2 v q O D G z W A R X J V E B F 0 W 3 b i s Y B / Q l D K Z 3 L R D J 5 M w M x F K z M J f c e N C E b f + h j v / x k m b h b Y e G O Z w z r 3 M m e M n n C n t O N 9 W Z W V 1 b X 2 j u l n b 2 t 7 Z 3 b P 3 D z o q T i W F N o 1 5 L H s + U c C Z g L Z m m k M v k U A i n 0 P X n 9 w U f v c B p G K x u N f T B A Y R G Q k W M k q 0 k Y b 2 k a c Z D y D z / J g H a h q Z K w v z f G j X n Y Y z A 1 4 m b k n q q E R r a H 9 5 Q U z T C I S m n C j V d 5 1 E D z I i N a M c 8 p q X K k g I n Z A R 9 A 0 V J A I 1 y G b 5 c 3 x q l A C H s T R H a D x T f 2 9 k J F J F N j M Z E T 1 W i 1 4 h / u f 1 U x 1 e D T I m k l S D o P O H w p R j H e O i D B w w C V T z q S G E S m a y Y j o m k l B t K q u Z E t z F L y + T z n n D d R r u 3 U W 9 e V 3 W U U X H 6 A S d I R d d o i a 6 R S 3 U R h Q 9 o

m f 0 i t 6 s J + v F e r c + 5 q M V q 9 w 5 R H 9 g f f 4 A O c K W 3 g = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " a o b g V s h m D 4 k + W f M e r 3 P z

P p R O s l E = " > A A A B / 3 i c b V D L S s N A F J 3 U V 6 2 v q O D G z W A R X J V E B F 0 W 3 b i s Y B / Q l D K Z 3 L R D J 5 M w M x F K z M J f c e N C E b f + h j v / x k m b h b Y e G O Z w z r 3 M m e M n n C n t O N 9 W Z W V 1 b X 2 j u l n b 2 t 7 Z 3 b P 3 D z o q T i W F N o 1 5 L H s + U c C Z g L Z m m k M v k U A i n 0 P X n 9 w U f v c B p G K x u N f T B A Y R G Q k W M k q 0 k Y b 2 k a c Z D y D z / J g H a h q Z K w v z f G j X n Y Y z A 1 4 m b k n q q E R r a H 9 5 Q U z T C I S m n C j V d 5 1 E D z I i N a M c 8 p q X K k g I n Z A R 9 A 0 V J A I 1 y G b 5 c 3 x q l A C H s T R H a D x T f 2 9 k J F J F N j M Z E T 1 W i 1 4 h / u f 1 U x 1 e D T I m k l S D o P O H w p R j H e O i D B w w C V T z q S G E S m a y Y j o m k l B t K q u Z E t z F L y + T z n n D d R r u 3 U W 9 e V 3 W U U X H 6 A S d I R d d o i a 6 R S 3 U R h Q 9 o

m f 0 i t 6 s J + v F e r c + 5 q M V q 9 w 5 R H 9 g f f 4 A O c K W 3 g = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " a o b g V s h m D 4 k + W f M e r 3 P z

P p R O s l E = " > A A A B / 3 i c b V D L S s N A F J 3 U V 6 2 v q O D G z W A R X J V E B F 0 W 3 b i s Y B / Q l D K Z 3 L R D J 5 M w M x F K z M J f c e N C E b f + h j v / x k m b h b Y e G O Z w z r 3 M m e M n n C n t O N 9 W Z W V 1 b X 2 j u l n b 2 t 7 Z 3 b P 3 D z o q T i W F N o 1 5 L H s + U c C Z g L Z m m k M v k U A i n 0 P X n 9 w U f v c B p G K x u N f T B A Y R G Q k W M k q 0 k Y b 2 k a c Z D y D z / J g H a h q Z K w v z f G j X n Y Y z A 1 4 m b k n q q E R r a H 9 5 Q U z T C I S m n C j V d 5 1 E D z I i N a M c 8 p q X K k g I n Z A R 9 A 0 V J A I 1 y G b 5 c 3 x q l A C H s T R H a D x T f 2 9 k J F J F N j M Z E T 1 W i 1 4 h / u f 1 U x 1 e D T I m k l S D o P O H w p R j H e O i D B w w C V T z q S G E S m a y Y j o m k l B t K q u Z E t z F L y + T z n n D d R r u 3 U W 9 e V 3 W U U X H 6 A S d I R d d o i a 6 R S 3 U R h Q 9 o

m f 0 i t 6 s J + v F e r c + 5 q M V q 9 w 5 R H 9 g f f 4 A O c K W 3 g = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " a o b g V s h m D 4 k + W f M e r 3 P z

P p R O s l E = " > A A A B / 3 i c b V D L S s N A F J 3 U V 6 2 v q O D G z W A R X J V E B F 0 W 3 b i s Y B / Q l D K Z 3 L R D J 5 M w M x F K z M J f c e N C E b f + h j v / x k m b h b Y e G O Z w z r 3 M m e M n n C n t O N 9 W Z W V 1 b X 2 j u l n b 2 t 7 Z 3 b P 3 D z o q T i W F N o 1 5 L H s + U c C Z g L Z m m k M v k U A i n 0 P X n 9 w U f v c B p G K x u N f T B A Y R G Q k W M k q 0 k Y b 2 k a c Z D y D z / J g H a h q Z K w v z f G j X n Y Y z A 1 4 m b k n q q E R r a H 9 5 Q U z T C I S m n C j V d 5 1 E D z I i N a M c 8 p q X K k g I n Z A R 9 A 0 V J A I 1 y G b 5 c 3 x q l A C H s T R H a D x T f 2 9 k J F J F N j M Z E T 1 W i 1 4 h / u f 1 U x 1 e D T I m k l S D o P O H w p R j H e O i D B w w C V T z q S G E S m a y Y j o m k l B t K q u Z E t z F L y + T z n n D d R r u 3 U W 9 e V 3 W U U X H 6 A S d I R d d o i a 6 R S 3 U R h Q 9 o m f 0 i t 6 s J + v F e r c + 5 q M V q 9 w 5 R H 9 g f f 4 A O c K W 3 g = = < / l a t e x i t > p ✓ ( f )

< l a t e x i t s h a 1 _ b a s e 6 4 = " X n r C 0 n + r 9

Q q + o X 8 I W E E E U c L D R z w = " > A A A C G X i c b V D J S g N B E O 2 J W 4 x b 1 K O X x i D E S 5 g R Q Y 9 B L x 4 j m A U y I f T 0 1 C R N e h a 6 a 4 Q w z G 9 4 8 V e 8 e F D E o 5 7 8 G z v L w S Q + a P r x X h V V 9 b x E C o 2 2 / W M V 1 t Y 3 N r e K 2 6 W d 3 b 3 9 g / L h U U v H q e L Q 5 L G M V c d j G q S I o I k C J X Q S B S z 0 J L S 9 0 e 3 E b z + C 0 i K O H n C c Q C 9 k g 0 g E g j M 0 U r 9 s J / 3 M 9 W L p 6 3 F o v s z F I S D L c 1 p 1 U U g f F s w g z 8 / 7 5 Y p d s 6 e g q 8 S Z k w q Z o 9 E v f 7 l + z N M Q I u S S a d 1 1 7 A R 7 G V M o u I S 8 5 K Y a E s Z H b A B d Q y M W g u 5 l 0 8 t y e m Y U n w a x M i 9 C O l X / d m Q s 1 J P l T G X I c K i X v Y n 4 n 9 d N M b j u Z S J K U o S I z w Y F q a Q Y 0 0 l M 1 B c K O M q x I Y w r Y X a l f M g U 4 2 j C L J k Q n O W T V 0 n r o u b Y N e f + s l K / m c d R J C f k l F S J Q 6 5 I n d y R B m

k S T p 7 I C 3 k j 7 9 a z 9 W p 9 W J + z 0 o I 1 7 z k m C 7 C + f w H q W a I X < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " X n r C 0 n + r 9

Q q + o X 8 I W E E E U c L D R z w = " > A A A C G X i c b V D J S g N B E O 2 J W 4 x b 1 K O X x i D E S 5 g R Q Y 9 B L x 4 j m A U y I f T 0 1 C R N e h a 6 a 4 Q w z G 9 4 8 V e 8 e F D E o 5 7 8 G z v L w S Q + a P r x X h V V 9 b x E C o 2 2 / W M V 1 t Y 3 N r e K 2 6 W d 3 b 3 9 g / L h U U v H q e L Q 5 L G M V c d j G q S I o I k C J X Q S B S z 0 J L S 9 0 e 3 E b z + C 0 i K O H n C c Q C 9 k g 0 g E g j M 0 U r 9 s J / 3 M 9 W L p 6 3 F o v s z F I S D L c 1 p 1 U U g f F s w g z 8 / 7 5 Y p d s 6 e g q 8 S Z k w q Z o 9 E v f 7 l + z N M Q I u S S a d 1 1 7 A R 7 G V M o u I S 8 5 K Y a E s Z H b A B d Q y M W g u 5 l 0 8 t y e m Y U n w a x M i 9 C O l X / d m Q s 1 J P l T G X I c K i X v Y n 4 n 9 d N M b j u Z S J K U o S I z w Y F q a Q Y 0 0 l M 1 B c K O M q x I Y w r Y X a l f M g U 4 2 j C L J k Q n O W T V 0 n r o u b Y N e f + s l K / m c d R J C f k l F S J Q 6 5 I n d y R B m

k S T p 7 I C 3 k j 7 9 a z 9 W p 9 W J + z 0 o I 1 7 z k m C 7 C + f w H q W a I X < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " X n r C 0 n + r 9

Q q + o X 8 I W E E E U c L D R z w = " > A A A C G X i c b V D J S g N B E O 2 J W 4 x b 1 K O X x i D E S 5 g R Q Y 9 B L x 4 j m A U y I f T 0 1 C R N e h a 6 a 4 Q w z G 9 4 8 V e 8 e F D E o 5 7 8 G z v L w S Q + a P r x X h V V 9 b x E C o 2 2 / W M V 1 t Y 3 N r e K 2 6 W d 3 b 3 9 g / L h U U v H q e L Q 5 L G M V c d j G q S I o I k C J X Q S B S z 0 J L S 9 0 e 3 E b z + C 0 i K O H n C c Q C 9 k g 0 g E g j M 0 U r 9 s J / 3 M 9 W L p 6 3 F o v s z F I S D L c 1 p 1 U U g f F s w g z 8 / 7 5 Y p d s 6 e g q 8 S Z k w q Z o 9 E v f 7 l + z N M Q I u S S a d 1 1 7 A R 7 G V M o u I S 8 5 K Y a E s Z H b A B d Q y M W g u 5 l 0 8 t y e m Y U n w a x M i 9 C O l X / d m Q s 1 J P l T G X I c K i X v Y n 4 n 9 d N M b j u Z S J K U o S I z w Y F q a Q Y 0 0 l M 1 B c K O M q x I Y w r Y X a l f M g U 4 2 j C L J k Q n O W T V 0 n r o u b Y N e f + s l K / m c d R J C f k l F S J Q 6 5 I n d y R B m

k S T p 7 I C 3 k j 7 9 a z 9 W p 9 W J + z 0 o I 1 7 z k m C 7 C + f w H q W a I X < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " X n r C 0 n + r 9

Q q + o X 8 I W E E E U c L D R z w = " > A A A C G X i c b V D J S g N B E O 2 J W 4 x b 1 K O X x i D E S 5 g R Q Y 9 B L x 4 j m A U y I f T 0 1 C R N e h a 6 a 4 Q w z G 9 4 8 V e 8 e F D E o 5 7 8 G z v L w S Q + a P r x X h V V 9 b x E C o 2 2 / W M V 1 t Y 3 N r e K 2 6 W d 3 b 3 9 g / L h U U v H q e L Q 5 L G M V c d j G q S I o I k C J X Q S B S z 0 J L S 9 0 e 3 E b z + C 0 i K O H n C c Q C 9 k g 0 g E g j M 0 U r 9 s J / 3 M 9 W L p 6 3 F o v s z F I S D L c 1 p 1 U U g f F s w g z 8 / 7 5 Y p d s 6 e g q 8 S Z k w q Z o 9 E v f 7 l + z N M Q I u S S a d 1 1 7 A R 7 G V M o u I S 8 5 K Y a E s Z H b A B d Q y M W g u 5 l 0 8 t y e m Y U n w a x M i 9 C O l X / d m Q s 1 J P l T G X I c K i X v Y n 4 n 9 d N M b j u Z S J K U o S I z w Y F q a Q Y 0 0 l M 1 B c K O M q x I Y w r Y X a l f M g U 4 2 j C L J k Q n O W T V 0 n r o u b Y N e f + s l K / m c d R J C f k l F S J Q 6 5 I n d y R B m k S T p 7 I C 3 k j 7 9 a z 9 W p 9 W J + z 0 o I 1 7 z k m C 7 C + f w H q W a I X < / l a t e x i t > µ ✓ (z t , f ) < l a t e x i t s h a 1 _ b a s e 6 4 = " P k l n U v n l z d i z q g V A O X g X 0 D b 4 c M I = " > A A A C Q n i c b V D L S g N B E J y N 7 / i K e v Q y G A Q F C b s i 6 F H 0 4 l H B G C U b l t n Z X j N k 9 s F M r x C X + T Y v f o E 3 P 8 C L B 0 W 8 e n D y O B i 1 o O m i q p v p q T C X Q q P r P j u V q e m Z 2 b n 5 h e r i 0 v L K a m 1 t / U p n h e L Q 5 J n M 1 H X I N E i R Q h M F S r j O F b A k l N A K e 6 c D v 3 U H S o s s v c R + D p 2 E 3 a Y i F p y h l Y L a j R 9 m M t L 9 x L b S T w o T l B M K d g G Z M X T H R y E j m D D v j Q l w j / 7 j x M b s B r W 6 2 3 C H o H + J N y Z 1 M s Z 5 U H v y o 4 w X C a T I J d O 6 7 b k 5 d k q m U H A J p u o X G n L G e + w W 2 p a m L A H d K Y c R G L p t l Y j G m b K V I h 2 q P z d K l u j B c X Y y Y d j V v 7 2 B + J / X L j A + 6 p Q i z Q u E l I 8 e i g t J M a O D P G k k F H C U f U s Y V 8 L e S n m X K c b R p l 6 1 I X i / v / y X X O 0 3 P L f h X R z U j 0 / G c c y T T b J F d o h H D s k

x O S P n p E k 4 e S A v 5 I 2 8 O 4 / O q / P h f I 5 G K 8 5 4 Z 4 N M w P n 6 B g e 3 t J Q = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " P k l n U v n l z d i z q

g V A O X g X 0 D b 4 c M I = " > A A A C Q n i c b V D L S g N B E J y N 7 / i K e v Q y G A Q F C b s i 6 F H 0 4 l H B G C U b l t n Z X j N k 9 s F M r x C X + T Y v f o E 3 P 8 C L B 0 W 8 e n D y O B i 1 o O m i q p v p q T C X Q q P r P j u V q e m Z 2 b n 5 h e r i 0 v L K a m 1 t / U p n h e L Q 5 J n M 1 H X I N E i R Q h M F S r j O F b A k l N A K e 6 c D v 3 U H S o s s v c R + D p 2 E 3 a Y i F p y h l Y L a j R 9 m M t L 9 x L b S T w o T l B M K d g G Z M X T H R y E j m D D v j Q l w j / 7 j x M b s B r W 6 2 3 C H o H + J N y Z 1 M s Z 5 U H v y o 4 w X C a T I J d O 6 7 b k 5 d k q m U H A J p u o X G n L G e + w W 2 p a m L A H d K Y c R G L p t l Y j G m b K V I h 2 q P z d K l u j B c X Y y Y d j V v 7 2 B + J / X L j A + 6 p Q i z Q u E l I 8 e i g t J M a O D P G k k F H C U f U s Y V 8 L e S n m X K c b R p l 6 1 I X i / v / y X X O 0 3 P L f h X R z U j 0 / G c c y T T b J F d o h H D s k

x O S P n p E k 4 e S A v 5 I 2 8 O 4 / O q / P h f I 5 G K 8 5 4 Z 4 N M w P n 6 B g e 3 t J Q = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " P k l n U v n l z d i z q

g V A O X g X 0 D b 4 c M I = " > A A A C Q n i c b V D L S g N B E J y N 7 / i K e v Q y G A Q F C b s i 6 F H 0 4 l H B G C U b l t n Z X j N k 9 s F M r x C X + T Y v f o E 3 P 8 C L B 0 W 8 e n D y O B i 1 o O m i q p v p q T C X Q q P r P j u V q e m Z 2 b n 5 h e r i 0 v L K a m 1 t / U p n h e L Q 5 J n M 1 H X I N E i R Q h M F S r j O F b A k l N A K e 6 c D v 3 U H S o s s v c R + D p 2 E 3 a Y i F p y h l Y L a j R 9 m M t L 9 x L b S T w o T l B M K d g G Z M X T H R y E j m D D v j Q l w j / 7 j x M b s B r W 6 2 3 C H o H + J N y Z 1 M s Z 5 U H v y o 4 w X C a T I J d O 6 7 b k 5 d k q m U H A J p u o X G n L G e + w W 2 p a m L A H d K Y c R G L p t l Y j G m b K V I h 2 q P z d K l u j B c X Y y Y d j V v 7 2 B + J / X L j A + 6 p Q i z Q u E l I 8 e i g t J M a O D P G k k F H C U f U s Y V 8 L e S n m X K c b R p l 6 1 I X i / v / y X X O 0 3 P L f h X R z U j 0 / G c c y T T b J F d o h H D s k

x O S P n p E k 4 e S A v 5 I 2 8 O 4 / O q / P h f I 5 G K 8 5 4 Z 4 N M w P n 6 B g e 3 t J Q = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " P k model:

l n U v n l z d i z q g V A O X g X 0 D b 4 c M I = " > A A A C Q n i c b V D L S g N B E J y N 7 / i K e v Q y G A Q F C b s i 6 F H 0 4 l H B G C U b l t n Z X j N k 9 s F M r x C X + T Y v f o E 3 P 8 C L B 0 W 8 e n D y O B i 1 o O m i q p v p q T C X Q q P r P j u V q e m Z 2 b n 5 h e r i 0 v L K a m 1 t / U p n h e L Q 5 J n M 1 H X I N E i R Q h M F S r j O F b A k l N A K e 6 c D v 3 U H S o s s v c R + D p 2 E 3 a Y i F p y h l Y L a j R 9 m M t L 9 x L b S T w o T l B M K d g G Z M X T H R y E j m D D v j Q l w j / 7 j x M b s B r W 6 2 3 C H o H + J N y Z 1 M s Z 5 U H v y o 4 w X C a T I J d O 6 7 b k 5 d k q m U H A J p u o X G n L G e + w W 2 p a m L A H d K Y c R G L p t l Y j G m b K V I h 2 q P z d K l u j B c X Y y Y d j V v 7 2 B + J / X L j A + 6 p Q i z Q u E l I 8 e i g t J M a O D P G k k F H C U f U s Y V 8 L e S n m X K c b R p l 6 1 I X i / v / y X X O 0 3 P L f h X R z U j 0 / G c c y T T b J F d o h H D s k x O S P n p E k 4 e S A v 5 I 2 8 O 4 / O q / P h f I 5 G K 8 5 4 Z 4 N M w P n 6 B g e 3 t J Q = < / l a t e x i t > f < l a t e x i t s h a 1 _ b a s e 6 4 = " a o b g V s h m D 4 k + W f M e r 3 P z P p R O s l E = " > A A A B / 3 i c b V D L S s N A F J 3 U V 6 2 v q O D G z W A R X J V E B F 0 W 3 b i s Y B / Q l D K Z 3 L R D J 5 M w M x F K z M J f c e N C E b f + h j v / x k m b h b Y e G O Z w z r 3 M m e M n n C n t O N 9 W Z W V 1 b X 2 j u l n b 2 t 7 Z 3 b P 3 D z o q T i W F N o 1 5 L H s + U c C Z g L Z m m k M v k U A i n 0 P X n 9 w U f v c B p G K x u N f T B A Y R G Q k W M k q 0 k Y b 2 k a c Z D y D z / J g H a h q Z K w v z f G j X n Y Y z A 1 4 m b k n q q E R r a H 9 5 Q U z T C I S m n C j V d 5 1 E D z I i N a M c 8 p q X K k g I n Z A R 9 A 0 V J A I 1 y G b 5 c 3 x q l A C H s T R H a D x T f 2 9 k J F J F N j M Z E T 1 W i 1 4 h / u f 1 U x 1 e D T I m k l S D o P O H w p R j H e O i D B w w C V T z q S G E S m a y Y j o m k l B t K q u Z E t z F L y + T z n n D d R r u 3 U W 9 e V 3 W U U X H 6 A S d I R d d o i a 6 R S 3 U R h Q 9 o m f 0 i t 6 s J + v F e r c + 5 q M V q 9 w 5 R H 9 g f f 4 A O c K W 3 g = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " a o b g V s h m D 4 k + W f M e r 3 P z P p R O s l E = " > A A A B / 3 i c b V D L S s N A F J 3 U V 6 2 v q O D G z W A R X J V E B F 0 W 3 b i s Y B / Q l D K Z 3 L R D J 5 M w M x F K z M J f c e N C E b f + h j v / x k m b h b Y e G O Z w z r 3 M m e M n n C n t O N 9 W Z W V 1 b X 2 j u l n b 2 t 7 Z 3 b P 3 D z o q T i W F N o 1 5 L H s + U c C Z g L Z m m k M v k U A i n 0 P X n 9 w U f v c B p G K x u N f T B A Y R G Q k W M k q 0 k Y b 2 k a c Z D y D z / J g H a h q Z K w v z f G j X n Y Y z A 1 4 m b k n q q E R r a H 9 5 Q U z T C I S m n C j V d 5 1 E D z I i N a M c 8 p q X K k g I n Z A R 9 A 0 V J A I 1 y G b 5 c 3 x q l A C H s T R H a D x T f 2 9 k J F J F N j M Z E T 1 W i 1 4 h / u f 1 U x 1 e D T I m k l S D o P O H w p R j H e O i D B w w C V T z q S G E S m a y Y j o m k l B t K q u Z E t z F L y + T z n n D d R r u 3 U W 9 e V 3 W U U X H 6 A S d I R d d o i a 6 R S 3 U R h Q 9 o m f 0 i t 6 s J + v F e r c + 5 q M V q 9 w 5 R H 9 g f f 4 A O c K W 3 g = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " a o b g V s h m D 4 k + W f M e r 3 P z P p R O s l E = " > A A A B / 3 i c b V D L S s N A F J 3 U V 6 2 v q O D G z W A R X J V E B F 0 W 3 b i s Y B / Q l D K Z 3 L R D J 5 M w M x F K z M J f c e N C E b f + h j v / x k m b h b Y e G O Z w z r 3 M m e M n n C n t O N 9 W Z W V 1 b X 2 j u l n b 2 t 7 Z 3 b P 3 D z o q T i W F N o 1 5 L H s + U c C Z g L Z m m k M v k U A i n 0 P X n 9 w U f v c B p G K x u N f T B A Y R G Q k W M k q 0 k Y b 2 k a c Z D y D z / J g H a h q Z K w v z f G j X n Y Y z A 1 4 m b k n q q E R r a H 9 5 Q U z T C I S m n C j V d 5 1 E D z I i N a M c 8 p q X K k g I n Z A R 9 A 0 V J A I 1 y G b 5 c 3 x q l A C H s T R H a D x T f 2 9 k J F J F N j M Z E T 1 W i 1 4 h / u f 1 U x 1 e D T I m k l S D o P O H w p R j H e O i D B w w C V T z q S G E S m a y Y j o m k l B t K q u Z E t z F L y + T z n n D d R r u 3 U W 9 e V 3 W U U X H 6 A S d I R d d o i a 6 R S 3 U R h Q 9 o m f 0 i t 6 s J + v F e r c + 5 q M V q 9 w 5 R H 9 g f f 4 A O c K W 3 g = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " a o b g V s h m D 4 k + W f M e r 3 P z P p R O s l E = " > A A A B / 3 i c b V D L S s N A F J 3 U V 6 2 v q O D G z W A R X J V E B F 0 W 3 b i s Y B / Q l D K Z 3 L R D J 5 M w M x F K z M J f c e N C E b f + h j v / x k m b h b Y e G O Z w z r 3 M m e M n n C n t O N 9 W Z W V 1 b X 2 j u l n b 2 t 7 Z 3 b P 3 D z o q T i W F N o 1 5 L H s + U c C Z g L Z m m k M v k U A i n 0 P X n 9 w U f v c B p G K x u N f T B A Y R G Q k W M k q 0 k Y b 2 k a c Z D y D z / J g H a h q Z K w v z f G j X n Y Y z A 1 4 m b k n q q E R r a H 9 5 Q U z T C I S m n C j V d 5 1 E D z I i N a M c 8 p q X K k g I n Z A R 9 A 0 V J A I 1 y G b 5 c 3 x q l A C H s T R H a D x T f 2 9 k J F J F N j M Z E T 1 W i 1 4 h / u f 1 U x 1 e D T I m k l S D o P O H w p R j H e O i D B w w C V T z q S G E S m a y Y j o m k l B t K q u Z E t z F L y + T z n n D d R r u 3 U W 9 e V 3 W U U X H 6 A S d I R d d o i a 6 R S 3 U R h Q 9 o m f 0 i t 6 s J + v F e r c + 5 q M V q 9 w 5 R H 9 g f f 4 A O c K W 3 g = = < / l a t e x i t > N o is e R o u n d

EQUATION

Above, θ is shorthand for the parameters. Each frame x t at time t depends on the corresponding latent variables z t and global variables f . The frame likelihood p θ (x t |f , z t ) for reconstruction is the Laplace distribution, Laplace µ θ (z t , f ), λ -1 1 . The reason for its choice is that its log likelihood results in an 1 loss, which typically produces sharper images than the 2 loss for autoencoding images (Isola et al., 2016; Zhao et al., 2015) . The decoder µ θ (•) is a function parameterized by neural networks. The prior distributions p θ (f ) and p θ (z 1:T ) will be discussed separately below.

After training, the reconstructed frame in image space is obtained from taking the most likely frame

xt = argmax p θ (x t |f , z t ) = µ θ (z t , f ).

Crucially, the decoder is conditioned both on global code f and time-local code z t .

Encoder. We employ amortized variational inference (Blei et al., 2017; Zhang et al., 2017; Marino et al., 2018) to predict a distribution over latent codes given the input video,

EQUATION

The global variables f are inferred from all video frames in a sequence and may thus contain static information, while z t is only inferred from a single frame x t .

As will be explained in the paragraph on model-based entropy coding below, modifications to standard variational inference are required for compression. Instead of sampling from Gaussian distributions with learned variances, here we employ uniform distributions centered at their means:

EQUATION

)

The means are predicted by additional encoder neural networks f = µ φ (x 1:T ), ẑt = µ φ (x t ) with parameters φ. This corresponds to adding random noise i ∼ U(-1 2 , 1 2 ) to f and μ in the inference process. The encoder and decoder neural networks are described in more detail in Appendix B.

Prior Models. The models that parameterize the learned prior distributions are ultimately used for entropy coding. Each dimension of the latent space has its own density model:

EQUATION

Above, indices refer to the dimension index of the latent variable and c t is a time-dependent context. The convolution with uniform noise is to allow the priors to better match the true marginal distribution when working with the box-shaped approximate posterior in Eq. 3 (see Ballé et al. (2018) Appendix 6.2). This convolution has an analytic form in terms of the cumulative probability density.

The stationary density p θ (f i ) is parameterized by a flexible non-parametric, fully-factorized model from (Ballé et al., 2018) . The density is defined by its cumulative and is built out of compositions of nonlinear probability densities, similar to the construction of a normalizing flow (Rezende & Mohamed, 2015) .

Two dynamical models are considered to model the sequence z 1:T . We propose a LSTM prior architecture which conditions on all previous frames in a segment:

p θ (z i t | c t ) ≡ p θ (z i t | z <t ).

We also considered a simpler model, which we compare against, with a single frame context: p θ (z i t | z t-1 ) which is essentially a deep Kalman filter (Krishnan et al., 2015) . A discussion of the connection between the amount of conditional context and the entropy is given in appendix A.

Variational Objective. The encoder (variational model) and decoder (generative model) can be learned jointly by minimizing the VAE loss function which consists of the KL divergence between the approximate and true posterior. In compression applications, however, one needs to adjust the trade-off between the bit rate and distortion. This is achieved by the β-VAE loss (Higgins et al., 2016; Mandt et al., 2016) , which takes the form (up to constant terms)

EQUATION

where the reconstructed frame xt = µ θ ( zt , f ) and we have introduced a parameter β to control the rate-distortion trade-off (Alemi et al., 2018) . The Laplace parameter λ was set to one.

The first term corresponds to the distortion and the second term is the cross entropy between the approximate posterior and the prior. The latter has the interpretation of the expected code length when using the prior distribution p(f , z 1:T ) to entropy code the latent variables. This term is minimized for p(f , z 1:T ) = E x 1:T [q(f , z 1:T |x 1:T )], that is, when the empirical distribution of codes matches the prior model. For our choice of generative model, the cross entropy separates into two terms H q φ (f |x 1:T ), p θ (f ) and H q φ (z 1:T |x 1:T ), p θ (z 1:T ) .

Model-based Entropy Coding. Sequential VAEs can be used to reduce the dimensionality of a video by performing a lossy transformation. However, the reduction in dimensionality does not mean the video is optimally compressed since there still exists redundancy in the form of temporal correlations in the sequence of the latent space variables. Sequential VAEs may include a temporal prior to model this redundancy, but to actually remove this redundancy and achieve a compact binary representation, the latent space has to be entropy coded. This is the distinguishing element between variational autoencoders and compression algorithms.

Crucially, the entropy coder needs a discrete vocabulary, which is obtained by rounding the latent state ( f and ẑ1:T ) after training. Care must be taken such that the quantization is approximated in a differentiable way during training. Ballé et al. (2016) address the problem of discretization of a continuous latent space in the context of VAE image compression by introducing noise in the inference process. By adding uniform noise to the most-likely inferred latent variables, the VAE is prevented from storing information in the latent space on length scales smaller than the discretization bin size. As such, rounding (after training) does not significantly affect the image reconstruction.

Our VAE framework exactly leads to such injection of noise with width one; it corresponds to our choice of a box-shaped approximate posterior distribution, centered at the maximally-likely values for the latent variables. Besides dealing with quantization, for efficient entropy coding we also need an estimate of the frequency of atoms in order to obtain short file sizes. We can obtain this from our prior model.

4. EXPERIMENTS:

In this section, we present the experimental results of our work. We first describe the video datasets, performance metrics, and competing methods in Section 4.1. This is followed by a quantitative analysis of our performance in terms of rate-distortion curves in Section 4.2 and then finally qualitative results in Section 4.3. We report that our method can achieve extreme compression ratios on videos with specialized content if it is trained on similar videos and that the performance is content-dependent. Our method is also comparable with modern codecs when trained on videos with diverse content. We find that the inclusion of the global state is more efficient than using solely local variables. Implementation details of our proposed models are provided in Appendix B.

4.1. DATASETS, METRICS, AND METHODS:

In this work, we train separately on three video datasets of increasing complexity with frame size 64×64. 1) Sprites. The simplest dataset consists of videos of Sprites characters from an open-source video game project, which is used in (Reed et al., 2015; Mathieu et al., 2016; Li & Mandt, 2018) .

The videos are generated from a script that samples the character action, skin color, clothing, and eyes from a collection of choices and have an inherently low-dimensional description (i.e. the script that generated it). 2) BAIR. BAIR action-free robot pushing dataset (Ebert et al., 2017) consists of a robot pushing objects on a table, which is also used in (Babaeizadeh et al., 2018; Denton & Fergus, 2018; Lee et al., 2018) . The video is more realistic and less sparse, but the content is specialized since all scenes contain the same background and robot, and the depicted action is simple since the motion is described by a limited set of commands sent to the robot. The first two datasets are uncompressed and no preprocessing is performed. 3) Kinetics600. The last dataset is the Kinetics600 dataset (Kay et al., 2017) which is a diverse set of YouTube videos depicting human actions. The dataset is cropped and downsampled, which removes compression artifacts, to 64 × 64.

Metrics. We evaluate our method based on bit rate in bits per pixel (bpp), and distortion measured in average frame peak signal-to-noise ratio (PSNR), which is related to the frame mean square error.

In the appendix, we also report on multi-scale structural similarity (MS-SSIM) (Wang et al., 2004) which is a perception-based metric that approximates the perceived change in structural information.

Comparisons. We wish to study the performance of our proposed local-global architecture with LSTM prior (LSTMP-LG) by comparing to other approaches. et al., 2015) . We also provide the performance of H.264, H.265, and VP9 codecs. Traditional codecs are not optimized for small resolution videos. However, their performance is far superior to neural or classical image compression methods (applied to compress video frame by frame), so their performance is presented for comparison. Codec performance is evaluated using the open source FFMPEG implementation in constant rate mode and distortion is varied by adjusting the constant rate factor. Unless otherwise stated, performance is tested on videos with 4:4:4 chroma sampling and on test videos with T = 10 frames. Quantitative compression performance is evaluated in terms of rate-distortion theory. This characterizes the trade-off between the binary representation size and average distortion. For a fixed image quality setting, a video codec produces an average bit rate on a given dataset. By varying the image quality setting, a curve is traced out in the rate-distortion plane. The curves for our method are generated by varying β (Equation 5).

The rate-distortion curves for our method, trained on three datasets and measured in PSNR, are shown in Fig. 3 . Higher curves indicate better performance. From the Sprites and BAIR results, one sees that our method has the ability to drastically outperform traditional codecs when focusing on specialized content. By training on videos with a fixed content, the model is able to learn an efficient representation for such content and the learned priors capture the empirical data distribution well (Appendix C). The results from training on the more diverse Kinetics videos also outperform or are competitive with standard codecs and better demonstrate the performance of our method on general content videos. Results measured in MS-SSIM (Appendix D) show similar behavior.

The first observation is that the LSTM prior outperforms the deep Kalman filter prior in all cases. This is because the LSTM model has a longer memory, allowing the predictive model to be more certain about the trajectory of the local latent variables. This, in turn, results in shorter code lengths.

Global Variables. The VAE encoder has the option to store information in local or global variables. The local variables are modeled by a temporal prior and can be efficiently stored in binary if the sequence z 1:T can be sequentially predicted with relative certainty from the context. The global variables, on the other hand, provide an architectural approach to removing temporal redundancy since the entire segment is stored in one global state without temporal structure. We find that the local-global architecture (LSTMP-LG) outperforms the local architecture (LSTMP-L) on all datasets, demonstrating the usefulness of a hybrid approach which partially encodes the entire video segment in a global state along with extra frame-by-frame information stored as a sequence.

During training, the VAE learns to utilize the global and local information in the optimal way. The utilization of each variable can be visualized by plotting the average code length of each latent state, which is shown in Fig. 4 . The VAE learns to significantly utilize the global variables even though dim(z) is sufficiently large to store the entire content of each individual frame. This provides further evidence that it is more efficient to incorporate global inference over several frames. The entropy in the local variables initially tends to decrease as a function of time, which supports the benefits from our predictive models. Note that our approach relies on sequential decoding, prohibiting a bi-directional LSTM for the local state.

f z1 z2 z3 z4 z5 z6 z7 z8 c ). Entropy drops with the frame index as the models adapt to the video sequence.

4.3. QUALITATIVE RESULTS:

Now we discuss the qualitative performance of our method. We have shown that a deep neural approach to encode video (LSTMP-LG architecture) can achieve competitive results with traditional codecs with respect to PSNR or MS-SSIM (Appendix D) metrics overall on low-resolution videos.

Test videos from the Sprites and BAIR datasets after compression with our method are shown in Fig. 1 and Fig. 5 (left), respectively, and compared to modern codec performance. Our method achieves a superior image quality at a significantly lower bit rate than H.264/H.265 and VP9 on these specialized content datasets. This is perhaps expected since traditional codecs cannot learn efficient representations for specialized content. Furthermore, fine-grained motion is not accurately predicted with block motion estimation. The artifacts from our method are more clearly displayed in Fig. 5 (right). Our method tends to produce blurry video in the low bit-rate regime but does not suffer from the block artifacts present in the H.265/VP9 compressed video. Limitations. When training on specialized content videos, our method tends to fit to the type of content, leading to artifacts which can be more abstract and perhaps undesirable for a general purpose codec. For example, at a low bit-rate setting the codec tends to leave out new kinds of objects which were not present in the training data, or to misinterpret content. This is a big departure from the behavior of traditional codecs. Such undesirable defects can be avoided, however, by training on a general content training set or keeping a larger local latent state.

Our current paper focused on small scale videos. One future avenue is to extrapolate our method to full resolution videos, where the dimension of the latent representation must scale with the resolution of the video in order to achieve good reconstruction preformance. Currently, the GPU memory limits the maximum size of the latent dimension for the local/global architecture due to the presence of fully-connected layers to infer global and local states. This effectively limits the maximum achievable image quality of our method as well as restricts our method to short, low-resolution input video. While we showed that this architecture was very efficient for small videos in the strongly compressed regime, a future architecture may focus more on convolutional structures and avoidance of fully connected networks.

5. CONCLUSIONS:

We have proposed a deep probabilistic modeling approach to video compression. Our method simultaneously learns to transform the original video into a lower-dimensional representation as well as the temporally-conditioned probabilistic model for entropy coding. The best performing proposed architecture splits up the latent code into global and local variables and yields competitive results on low resolution videos. For video sources with specialized content, deep probabilistic video coding allows for a significant increase coding performance. This could be interesting for transmitting specialized content such as teleconferencing or sports broadcasting.

We have shown viability on low resolution videos, and in future work, it is interesting to see how our LSTMP-L will work with convolutional LSTM as the predictive model in full-resolution videos and longer video sequences. Furthermore, there is much potential in investigating how additional "side information" could aid the predictive model. Thus we think that our work is a first step into a new direction for video coding which opens up several exciting avenues for future work.

A ENTROPY CODING:

Predictive modeling is important in the entropy coding stage. Theoretical compression ratios can be computed from the cross entropy between the true probability distribution (measured empirically) and a model for the probabilities. A better model that more accurately captures the true certainty about the next symbol can achieve a lower entropy and thus a smaller bit rate. Entropy coding is a solved problem since there exist coding schemes, e.g. arithmetic coding, which approximately achieve the the entropy, up to extra information that is sent through the stream for practical reasons, and this bit rate is optimal for long messages (Shannon, 2001) . Modeling, however, is difficult and leaves much room for improvement.

In this work, after the latent representation has been quantized to N finite symbols, the variables are losslessly compressed into binary. The global latent variables f are static and are entropy coded according to a stationary distribution. The optimal lossless compression algorithm encodes the ith symbol f i with a number of bits set by log p(f i ) on average, where p is the true probability for f i . The theoretical code length in bits after entropy coding the global variables with the prior distribution p θ (f ) is computed from the cross entropy:

EQUATION

where index i refers to the dimension index of latent variable f .

The cross entropy is minimized on average in the limit that p θ (f ) = E x 1:T [q φ (f |x 1:T )] ≡ p(f ), in which case the cross entropy becomes the Shannon entropy. In this work, we quote theoretical compression rates by computing the cross entropy between the variational distribution and the prior. However, we have explicitly checked that this performance is achieved by an arithmetic coder implementation which uses the probabilities from our learned prior.

For the case of the sequence z 1:T , the theoretical description length of the sequence is given by the following cross entropy:

H q φ (z 1:T |x 1:T ), p θ (z 1:T ) = -E z 1:T ∼q

EQUATION

where c t is the context which may in principle depend on the previous appearing symbols in the sequence, the future symbols, or the global variables.

Consider the true probability distribution p(z t |¬z t ), conditioned on all other z t in the sequence both forward and backward in time. Averaging over all future elements of the sequence leads to a simplified distribution which only captures the conditional distribution based on a backwards in time context p(z t |z <t ). One can average over long-distance context to arrive at an even simpler condition disribution p(z t |z t-1 ), or average out all context to obtain the marginal distribution p(z t ). We are guaranteed that the Shannon entropies are progressively larger since 

Equality holds when the mutual information between z 1:T and the additional context vanishes. 1 The left-most entropy corresponds to treating each frame independently as in image compression. By improving the amount of context in the probabilistic model, the entropy and code length can be potentially reduced. We illustrate this point in the experiment section by demonstrating that the LSTM prior model, which has a longer contextual scope, produces a shorter code length than the deep Kalman filter prior.

B MODEL ARCHITECTURE:

The specific implementation details of our model are now described. We describe the two baseline models, LSTMP-L and KFP-LG, and the best-performing LSTMP-LG model. In this appendix, we visualize the distribution of our prior model and compare to the empirical distribution of the posterior of the inference model estimated from data. In Fig. 7 , we show the learned priors and the empirically observed posterior over two dimensions of the global latent state f and local latent state z in order to demonstrate that the prior is capturing the correct empirical distribution in low-bit rate of our model. From Fig. 7 , we can see that the learned priors p θ (f ) and p θ (z 1 ) match the empirical data distributions well, which leads to low-bit rate encoding of the latent variables. As the conditional probability model p θ (z t | z <t ) is high dimensional, we do not display this distribution. We also plot the average frame MS-SSIM with respect to the bit rate to quantitatively compare our models with traditional codecs. From Fig. 8 , we can see that our LSTMP-LG method saves significantly more bits when trained on specialized content dataset (Sprites and BAIR) and achieves competitive result with respect to MS-SSIM when trained on general content dataset.

D ADDITIONAL PERFORMANCE EVALUATION:

In Section 4, we trained our method on short video segments of T = 10 frames and evaluated classical codec performance on the same segments. For typical videos, somewhat longer segments tend to have less information per pixel than T = 10 segments, and standard video codecs are designed to take advantage of this fact. For this reason, we have presented codec performance, evaluated on T = 10, 30, and 100 frame segments for the Kinetics data in Fig. 9 . While codec performance improves for longer segments, we note that our method (trained and evaluated on 10 frames) is still comparable to modern codec performance evaluated on longer segments. Additionally, with proper design, our method could potentially be improved to achieve similar temporal performance scaling since longer segments usually have less information per pixel. 

:

This is equivalent to the fact from statistical physics that entropy of a system cannot decrease when averaging over a subset of the degrees of freedom.

Figures and Tables:

Figure 1: Reconstructed Sprites test video (bpp=0.06, PSNR=44.6 dB), H.265 (bpp=0.86, PSNR = 21.1 dB), and VP9 (bpp=0.57, PSNR = 26.0 dB), see Section 4. In contrast to our method, H.265 and VP9 show artifacts of block motion prediction. Our method uses a fraction of the bit rate.

Figure 2: Operational diagram of our compression codec. A video segment is encoded into perframe latent variables z t and per-segment global state f , which are then quantized and arithmetically encoded into binary according to the prior model. To recover an approximation to the original video, the latent variables are arithmetically decoded from the binary and passed through the decoder.

Figure3: Rate-distortion curves on three datasets measured in PSNR (higher corresponds to lower distortion). Legend shared. Solid lines correspond to our models, with LSTMP-LG proposed.

Figure4: Average bits of information stored in f and z 1:T with PSNR 43.2, 37.1, 30.3 for different models in (a, b, c). Entropy drops with the frame index as the models adapt to the video sequence.

Figure 5: Compressed videos by our LSTMP-LG model and VP9 in the low bit rate regime. Our approach achieves better quality on specialized content (BAIR, left) and comparable visual quality on generic video content (Kinetics, right) compared to VP9.

Figure 7: Empirical distributions of the posterior of inference model and ground truth prior model in one specific rate-distortion BAIR example.

Figure8: Rate-distortion curves on three datasets measured in MS-SSIM (higher corresponds to lower distortion). Legend shared. Solid lines correspond to our models, with LSTMP-LG proposed.

Figure 9: Rate-distortion curves on the Kinetics dataset measured in PSNR. Codec performance is evaluated on video segments of T = 10, 30, and 100 frames. Our best performing method (trained and evaluated on T = 10 frames) is shown in red for comparison.

To study the effectiveness of predictive model for entropy coding, we introduce our baseline model LSTMP-L which has only local states with LSTM prior p θ (z t | z <t ). To study the efficiency of incorporating global and local states, we shows our baseline model KFP-LG which has both global and local states but with a weak predictive model p θ (z t | z t-1 ), a deep Kalman filter (Krishnan

Table content: <table/>

H [p(z t ) t=1:T ] ≥ H [p(z t |z t-1 ) t=1:T ] ≥ H [p(z t |z <t ) t=1:T ] ≥ H [p(z t |¬z t ) t=1:T ] .

Table content: <table/>

Peer review (iclr2019):
 The paper is well written and the basic ideas are reasonably well explained and supported. However, several aspects are insufficiently explained. Several examples follow. In Figure 1, it is not clear at all how the bitstream is formed; frames 1 to T are compressed jointly with frame t; but frame t is part of the set of frames from 1 to T. How the global state updated when compressing frame t+1? Using frames 2 to T+1? Writing that you use a Laplacian distribution because l1 regularized loss typically outperforms the l`2 loss for autoencoding images is clearly an insufficient justification, if not backed by experiments or references. Moreover, the authors seem to confuse regularization with loss; by using a Laplace density for the generative model, they are using a l1 loss, not an l1 regularizer. There is absolutely no information about implementation details. The video sequences used in the experiments are extremely small, both in spatial and temporal terms. A collection of 10 64*64 frames has fewer pixels than even a moderately sized still image. As the authors acknowledge, standard video codecs are far from optimized for video sequences of this size, making the comparisons unfair. The extreme compression results on the sprites and BAIR datasets may be quite misleading, since the data lives in a very low dimensional manifold, due to the simplicity of the scenes. For the more realistic Kinetics dataset, the proposed method is competitive with H264 and H265, but only in a very limited range of bit rates. In fact, the authors do not explain why they have not show results for wider ranges of bitrates.