Year: 2018

Abstract:
Recurrent Neural Networks (RNNs) have been widely applied to sequential data analysis. Due to their complicated modeling structures, however, the theory behind is still largely missing. To connect theory and practice, we study the generalization properties of vanilla RNNs as well as their variants, including Minimal Gated Unit (MGU) and Long Short Term Memory (LSTM) RNNs. Specifically, our theory is established under the PAC-Learning framework. The generalization bound is presented in terms of the spectral norms of the weight matrices and the total number of parameters. We also establish refined generalization bounds with additional norm assumptions, and draw a comparison among these bounds. We remark: (1) Our generalization bound for vanilla RNNs is significantly tighter than the best of existing results; (2) We are not aware of any other generalization bounds for MGU and LSTM in the exiting literature; (3) We demonstrate the advantages of these variants in generalization.

heading: 1 INTRODUCTION
text: Recurrent Neural Networks (RNNs) have successfully revolutionized sequential data analysis, and been widely applied to many real world problems, such as natural language processing (Cho et al., 2014; Bahdanau et al., 2014; Sutskever et al., 2014), speech recognition (Graves et al., 2006; Mikolov et al., 2010; Graves, 2012; Graves et al., 2013), computer vision (Gregor et al., 2015; Xu et al., 2015; Donahue et al., 2015; Karpathy and Fei-Fei, 2015), healthcare (Lipton et al., 2015; Choi et al., 2016a;b), and robot control (Ku and Lee, 1995; Lee and Teng, 2000; Yoo et al., 2006). Quite a few of these applications can be approached easily in our daily life, such as Google Translate, Google Now, Apple Siri, etc.
The sequential modeling nature of RNNs is significantly different from feedforward neural networks, though they both have neurons as the basic components. RNNs exploit the internal state (also known as hidden unit) to process the sequence of inputs, which naturally captures the dependence of the sequence. RNNs can also be viewed as nonlinear dynamical systems, and reduced to linear dynamical systems given identity activation operators. Besides the vanilla version, RNNs have many other variants. A large class of variants incorporate the so-called “gated” units to trim RNNs for different tasks. Typical examples include Long Short-Term Memory (LSTM, Hochreiter and Schmidhuber (1997)), Gated Recurrent Unit (GRU, Jozefowicz et al. (2015)) and Minimal Gated Unit (MGU, Zhou et al. (2016)).
The success of RNNs owes not only to their special network structures and the ability to fit training data, but also to their good generalization property: They can provide accurate predictions on unseen data. For example, Graves et al. (2013) report that after training with merely 462 speech samples, deep LSTM RNNs achieve a test set error of 17.7% on TIMIT phoneme recognition benchmark, which is the best recorded score. Mikolov et al. (2010) also show that RNNs outperform significantly state-of-the-art backoff models for speech recognition. When using RNNs in Wall Street Journal task to predict the next word in textual data given the context, word error rate is reduced around 18% compared to backoff models trained on the same amount of data, and 12% when backoff model is trained on 5 times more data. Despite of the popularity of RNNs in applications, their theory is less studied than other feedforward neural networks (Bartlett et al., 2017; Neyshabur et al., 2017; Golowich et al., 2017; Li et al., 2018). There are still several long lasting fundamental questions regarding the approximation, trainability, and generalization of RNNs.
In this paper, we propose to understand the generalization ability of RNNs and their variants. We aim to answer two questions from a theoretical perspective:
Q.1) Do RNNs suffer from significant curse of dimensionality?
Q.2) What are the advantages of MGU and LSTM over vanilla RNNs?
The investigation of generalization properties of neural networks including RNNs has a long history. Most of these works adopt a layer wise analysis and establish their results by induction. For example, Haussler (1992) establishes a complexity bound for feedforward neural networks. The result assumes average Lipschitz constant of each layer to be greater than 1 and involves the product of these Lipschitz constants. Thus, the resulting bound is inevitably exponential in the depth of the network. Later, Dasgupta and Sontag (1996) and Koiran (1998) adopt a VC-dimension argument to show complexity bounds of RNNs that are polynomial in the size of the network. Both of the works, however, are based on oversimplified assumptions: Dasgupta and Sontag (1996) only consider linear RNNs for binary classification tasks; Koiran (1998) assumes RNNs take the first coordinate of their hidden states as outputs. More recently, Bartlett et al. (2017) propose a new technique for developing generalization bounds for feedforward neural networks based on empirical Rademacher complexity under the PAC-Learning framework. Neyshabur et al. (2017) further adapt the technique to establish their generalization bound using the PAC-Bayes approach. Then the follow-up work Zhang et al. (2018) use the PAC-Bayes approach to establish a generalization bound for vanilla RNNs.
Our theory is partially motivated by Bartlett et al. (2017), but is quite different from Bartlett et al. (2017) and Zhang et al. (2018). In particular, our analysis exploits the compositional nature of RNNs, and decouples the spectral norms of weight matrices and the number of weight parameters. This makes our analysis conceptually much simpler (e.g. avoid layer wise analysis), and also yields better generalization bound than Zhang et al. (2018).
<latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit>
<latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit>
<latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit>
<latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit>
rt <latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit>
1 rt <latexit sha1_base64="wQAvzFe7l0eLgjHF3NV0yfiWEZQ=">AAAB73icbVBNS8NAEJ34WetX1aOXxVbwYkmKoMeCF48V7Ae0oWy2m3bpZhN3J0IJ/RNePCji1b/jzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1gJOE+xEdKhEKRtFKnYp3SXQfK/1S2a26c5BV4uWkDDka/dJXbxCzNOIKmaTGdD03QT+jGgWTfFrspYYnlI3pkHctVTTixs/m907JuVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbzxM6GSFLlii0VhKgnGZPY8GQjNGcqJJZRpYW8lbEQ1ZWgjKtoQvOWXV0mrVvXcqndfK9ev8jgKcApncAEeXEMd7qABTWAg4Rle4c15dF6cd+dj0brm5DMn8AfO5w9M3I7F</latexit><latexit sha1_base64="wQAvzFe7l0eLgjHF3NV0yfiWEZQ=">AAAB73icbVBNS8NAEJ34WetX1aOXxVbwYkmKoMeCF48V7Ae0oWy2m3bpZhN3J0IJ/RNePCji1b/jzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1gJOE+xEdKhEKRtFKnYp3SXQfK/1S2a26c5BV4uWkDDka/dJXbxCzNOIKmaTGdD03QT+jGgWTfFrspYYnlI3pkHctVTTixs/m907JuVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbzxM6GSFLlii0VhKgnGZPY8GQjNGcqJJZRpYW8lbEQ1ZWgjKtoQvOWXV0mrVvXcqndfK9ev8jgKcApncAEeXEMd7qABTWAg4Rle4c15dF6cd+dj0brm5DMn8AfO5w9M3I7F</latexit><latexit sha1_base64="wQAvzFe7l0eLgjHF3NV0yfiWEZQ=">AAAB73icbVBNS8NAEJ34WetX1aOXxVbwYkmKoMeCF48V7Ae0oWy2m3bpZhN3J0IJ/RNePCji1b/jzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1gJOE+xEdKhEKRtFKnYp3SXQfK/1S2a26c5BV4uWkDDka/dJXbxCzNOIKmaTGdD03QT+jGgWTfFrspYYnlI3pkHctVTTixs/m907JuVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbzxM6GSFLlii0VhKgnGZPY8GQjNGcqJJZRpYW8lbEQ1ZWgjKtoQvOWXV0mrVvXcqndfK9ev8jgKcApncAEeXEMd7qABTWAg4Rle4c15dF6cd+dj0brm5DMn8AfO5w9M3I7F</latexit><latexit sha1_base64="wQAvzFe7l0eLgjHF3NV0yfiWEZQ=">AAAB73icbVBNS8NAEJ34WetX1aOXxVbwYkmKoMeCF48V7Ae0oWy2m3bpZhN3J0IJ/RNePCji1b/jzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1gJOE+xEdKhEKRtFKnYp3SXQfK/1S2a26c5BV4uWkDDka/dJXbxCzNOIKmaTGdD03QT+jGgWTfFrspYYnlI3pkHctVTTixs/m907JuVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbzxM6GSFLlii0VhKgnGZPY8GQjNGcqJJZRpYW8lbEQ1ZWgjKtoQvOWXV0mrVvXcqndfK9ev8jgKcApncAEeXEMd7qABTWAg4Rle4c15dF6cd+dj0brm5DMn8AfO5w9M3I7F</latexit>
h̃t <latexit sha1_base64="S3CdFfDx9HsepZJgWjBttnLljms=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2FZoQ9lsNu3SzSbuTgol9Hd48aCIV3+MN/+N24+Dtj4YeLw3w8y8IJXCoOt+O4WNza3tneJuaW//4PCofHzSNkmmGW+xRCb6MaCGS6F4CwVK/phqTuNA8k4wup35nTHXRiTqAScp92M6UCISjKKV/GoPhQx5Ppz2sdovV9yaOwdZJ96SVGCJZr/81QsTlsVcIZPUmK7npujnVKNgkk9LvczwlLIRHfCupYrG3Pj5/OgpubBKSKJE21JI5urviZzGxkziwHbGFIdm1ZuJ/3ndDKMbPxcqzZArtlgUZZJgQmYJkFBozlBOLKFMC3srYUOqKUObU8mG4K2+vE7a9Zrn1rz7eqVxtYyjCGdwDpfgwTU04A6a0AIGT/AMr/DmjJ0X5935WLQWnOXMKfyB8/kDdWmR1Q==</latexit><latexit sha1_base64="S3CdFfDx9HsepZJgWjBttnLljms=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2FZoQ9lsNu3SzSbuTgol9Hd48aCIV3+MN/+N24+Dtj4YeLw3w8y8IJXCoOt+O4WNza3tneJuaW//4PCofHzSNkmmGW+xRCb6MaCGS6F4CwVK/phqTuNA8k4wup35nTHXRiTqAScp92M6UCISjKKV/GoPhQx5Ppz2sdovV9yaOwdZJ96SVGCJZr/81QsTlsVcIZPUmK7npujnVKNgkk9LvczwlLIRHfCupYrG3Pj5/OgpubBKSKJE21JI5urviZzGxkziwHbGFIdm1ZuJ/3ndDKMbPxcqzZArtlgUZZJgQmYJkFBozlBOLKFMC3srYUOqKUObU8mG4K2+vE7a9Zrn1rz7eqVxtYyjCGdwDpfgwTU04A6a0AIGT/AMr/DmjJ0X5935WLQWnOXMKfyB8/kDdWmR1Q==</latexit><latexit sha1_base64="S3CdFfDx9HsepZJgWjBttnLljms=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2FZoQ9lsNu3SzSbuTgol9Hd48aCIV3+MN/+N24+Dtj4YeLw3w8y8IJXCoOt+O4WNza3tneJuaW//4PCofHzSNkmmGW+xRCb6MaCGS6F4CwVK/phqTuNA8k4wup35nTHXRiTqAScp92M6UCISjKKV/GoPhQx5Ppz2sdovV9yaOwdZJ96SVGCJZr/81QsTlsVcIZPUmK7npujnVKNgkk9LvczwlLIRHfCupYrG3Pj5/OgpubBKSKJE21JI5urviZzGxkziwHbGFIdm1ZuJ/3ndDKMbPxcqzZArtlgUZZJgQmYJkFBozlBOLKFMC3srYUOqKUObU8mG4K2+vE7a9Zrn1rz7eqVxtYyjCGdwDpfgwTU04A6a0AIGT/AMr/DmjJ0X5935WLQWnOXMKfyB8/kDdWmR1Q==</latexit><latexit sha1_base64="S3CdFfDx9HsepZJgWjBttnLljms=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2FZoQ9lsNu3SzSbuTgol9Hd48aCIV3+MN/+N24+Dtj4YeLw3w8y8IJXCoOt+O4WNza3tneJuaW//4PCofHzSNkmmGW+xRCb6MaCGS6F4CwVK/phqTuNA8k4wup35nTHXRiTqAScp92M6UCISjKKV/GoPhQx5Ppz2sdovV9yaOwdZJ96SVGCJZr/81QsTlsVcIZPUmK7npujnVKNgkk9LvczwlLIRHfCupYrG3Pj5/OgpubBKSKJE21JI5urviZzGxkziwHbGFIdm1ZuJ/3ndDKMbPxcqzZArtlgUZZJgQmYJkFBozlBOLKFMC3srYUOqKUObU8mG4K2+vE7a9Zrn1rz7eqVxtYyjCGdwDpfgwTU04A6a0AIGT/AMr/DmjJ0X5935WLQWnOXMKfyB8/kDdWmR1Q==</latexit>
U <latexit sha1_base64="VLV1WfLffWbJvU05bja4sMBfihk=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmrbQhrLZbtqlm03YnQil9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQdf9dgobm1vbO8Xd0t7+weFR+fikZZJMM+6zRCa6E1LDpVDcR4GSd1LNaRxK3g7Ht3O//cS1EYl6xEnKg5gOlYgEo2ilh6pf7Zcrbs1dgKwTLycVyNHsl796g4RlMVfIJDWm67kpBlOqUTDJZ6VeZnhK2ZgOeddSRWNuguni1Bm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+53UzjG6CqVBphlyx5aIokwQTMv+bDITmDOXEEsq0sLcSNqKaMrTplGwI3urL66RVr3luzbuvVxpXeRxFOINzuAQPrqEBd9AEHxgM4Rle4c2Rzovz7nwsWwtOPnMKf+B8/gBj7Y0l</latexit><latexit sha1_base64="VLV1WfLffWbJvU05bja4sMBfihk=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmrbQhrLZbtqlm03YnQil9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQdf9dgobm1vbO8Xd0t7+weFR+fikZZJMM+6zRCa6E1LDpVDcR4GSd1LNaRxK3g7Ht3O//cS1EYl6xEnKg5gOlYgEo2ilh6pf7Zcrbs1dgKwTLycVyNHsl796g4RlMVfIJDWm67kpBlOqUTDJZ6VeZnhK2ZgOeddSRWNuguni1Bm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+53UzjG6CqVBphlyx5aIokwQTMv+bDITmDOXEEsq0sLcSNqKaMrTplGwI3urL66RVr3luzbuvVxpXeRxFOINzuAQPrqEBd9AEHxgM4Rle4c2Rzovz7nwsWwtOPnMKf+B8/gBj7Y0l</latexit><latexit sha1_base64="VLV1WfLffWbJvU05bja4sMBfihk=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmrbQhrLZbtqlm03YnQil9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQdf9dgobm1vbO8Xd0t7+weFR+fikZZJMM+6zRCa6E1LDpVDcR4GSd1LNaRxK3g7Ht3O//cS1EYl6xEnKg5gOlYgEo2ilh6pf7Zcrbs1dgKwTLycVyNHsl796g4RlMVfIJDWm67kpBlOqUTDJZ6VeZnhK2ZgOeddSRWNuguni1Bm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+53UzjG6CqVBphlyx5aIokwQTMv+bDITmDOXEEsq0sLcSNqKaMrTplGwI3urL66RVr3luzbuvVxpXeRxFOINzuAQPrqEBd9AEHxgM4Rle4c2Rzovz7nwsWwtOPnMKf+B8/gBj7Y0l</latexit><latexit sha1_base64="VLV1WfLffWbJvU05bja4sMBfihk=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmrbQhrLZbtqlm03YnQil9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQdf9dgobm1vbO8Xd0t7+weFR+fikZZJMM+6zRCa6E1LDpVDcR4GSd1LNaRxK3g7Ht3O//cS1EYl6xEnKg5gOlYgEo2ilh6pf7Zcrbs1dgKwTLycVyNHsl796g4RlMVfIJDWm67kpBlOqUTDJZ6VeZnhK2ZgOeddSRWNuguni1Bm5sMqARIm2pZAs1N8TUxobM4lD2xlTHJlVby7+53UzjG6CqVBphlyx5aIokwQTMv+bDITmDOXEEsq0sLcSNqKaMrTplGwI3urL66RVr3luzbuvVxpXeRxFOINzuAQPrqEBd9AEHxgM4Rle4c2Rzovz7nwsWwtOPnMKf+B8/gBj7Y0l</latexit>
W <latexit sha1_base64="2vPE/2toIup2Z/Lenq7o8M88OA0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD9VOdVCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophjd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZVHkcRzuAcLsGDa2jAHTShBQxG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwBm940n</latexit><latexit sha1_base64="2vPE/2toIup2Z/Lenq7o8M88OA0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD9VOdVCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophjd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZVHkcRzuAcLsGDa2jAHTShBQxG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwBm940n</latexit><latexit sha1_base64="2vPE/2toIup2Z/Lenq7o8M88OA0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD9VOdVCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophjd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZVHkcRzuAcLsGDa2jAHTShBQxG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwBm940n</latexit><latexit sha1_base64="2vPE/2toIup2Z/Lenq7o8M88OA0=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2g9oQ9lsN+3SzSbsToQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbud+54lrI2L1iNOE+xEdKREKRtFKD9VOdVCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophjd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZVHkcRzuAcLsGDa2jAHTShBQxG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwBm940n</latexit>
Wr <latexit sha1_base64="YH7coM1v251MW/+4mqAFEDNLMyM=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr070PVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRjdBLlQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaV0UcZTiDc7gED66hBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz/vwo4M</latexit><latexit sha1_base64="YH7coM1v251MW/+4mqAFEDNLMyM=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr070PVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRjdBLlQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaV0UcZTiDc7gED66hBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz/vwo4M</latexit><latexit sha1_base64="YH7coM1v251MW/+4mqAFEDNLMyM=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr070PVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRjdBLlQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaV0UcZTiDc7gED66hBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz/vwo4M</latexit><latexit sha1_base64="YH7coM1v251MW/+4mqAFEDNLMyM=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr070PVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRjdBLlQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaV0UcZTiDc7gED66hBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz/vwo4M</latexit>
Ur <latexit sha1_base64="BW2T5sAmwcKzhZ8g1qGxQ2wjKug=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKft0f6PqgWnMb7gJknXgFqUGB9qD61R8mLIu5QiapMT3PTTHIqUbBJJ9V+pnhKWUTOuI9SxWNuQnyxbEzcmGVIYkSbUshWai/J3IaGzONQ9sZUxybVW8u/uf1MoxuglyoNEOu2HJRlEmCCZl/ToZCc4ZyagllWthbCRtTTRnafCo2BG/15XXSaTY8t+HdN2utqyKOMpzBOVyCB9fQgjtogw8MBDzDK7w5ynlx3p2PZWvJKWZO4Q+czx/stI4K</latexit><latexit sha1_base64="BW2T5sAmwcKzhZ8g1qGxQ2wjKug=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKft0f6PqgWnMb7gJknXgFqUGB9qD61R8mLIu5QiapMT3PTTHIqUbBJJ9V+pnhKWUTOuI9SxWNuQnyxbEzcmGVIYkSbUshWai/J3IaGzONQ9sZUxybVW8u/uf1MoxuglyoNEOu2HJRlEmCCZl/ToZCc4ZyagllWthbCRtTTRnafCo2BG/15XXSaTY8t+HdN2utqyKOMpzBOVyCB9fQgjtogw8MBDzDK7w5ynlx3p2PZWvJKWZO4Q+czx/stI4K</latexit><latexit sha1_base64="BW2T5sAmwcKzhZ8g1qGxQ2wjKug=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKft0f6PqgWnMb7gJknXgFqUGB9qD61R8mLIu5QiapMT3PTTHIqUbBJJ9V+pnhKWUTOuI9SxWNuQnyxbEzcmGVIYkSbUshWai/J3IaGzONQ9sZUxybVW8u/uf1MoxuglyoNEOu2HJRlEmCCZl/ToZCc4ZyagllWthbCRtTTRnafCo2BG/15XXSaTY8t+HdN2utqyKOMpzBOVyCB9fQgjtogw8MBDzDK7w5ynlx3p2PZWvJKWZO4Q+czx/stI4K</latexit><latexit sha1_base64="BW2T5sAmwcKzhZ8g1qGxQ2wjKug=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKft0f6PqgWnMb7gJknXgFqUGB9qD61R8mLIu5QiapMT3PTTHIqUbBJJ9V+pnhKWUTOuI9SxWNuQnyxbEzcmGVIYkSbUshWai/J3IaGzONQ9sZUxybVW8u/uf1MoxuglyoNEOu2HJRlEmCCZl/ToZCc4ZyagllWthbCRtTTRnafCo2BG/15XXSaTY8t+HdN2utqyKOMpzBOVyCB9fQgjtogw8MBDzDK7w5ynlx3p2PZWvJKWZO4Q+czx/stI4K</latexit>
Uh <latexit sha1_base64="WwGo1Rp5Q2OaeI9VfuNwpkrOtzo=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsJ+3SzSbsboQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dkobm1vbO+Xdyt7+weFR9fiko5NMMfRZIhL1GFKNgkv0DTcCH1OFNA4FdsPJ7dzvPqHSPJEPZppiENOR5BFn1FjJr/uDcX1QrbkNdwGyTryC1KBAe1D96g8TlsUoDRNU657npibIqTKcCZxV+pnGlLIJHWHPUklj1EG+OHZGLqwyJFGibElDFurviZzGWk/j0HbG1Iz1qjcX//N6mYlugpzLNDMo2XJRlAliEjL/nAy5QmbE1BLKFLe3EjamijJj86nYELzVl9dJp9nw3IZ336y1roo4ynAG53AJHlxDC+6gDT4w4PAMr/DmSOfFeXc+lq0lp5g5hT9wPn8A3YKOAA==</latexit><latexit sha1_base64="WwGo1Rp5Q2OaeI9VfuNwpkrOtzo=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsJ+3SzSbsboQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dkobm1vbO+Xdyt7+weFR9fiko5NMMfRZIhL1GFKNgkv0DTcCH1OFNA4FdsPJ7dzvPqHSPJEPZppiENOR5BFn1FjJr/uDcX1QrbkNdwGyTryC1KBAe1D96g8TlsUoDRNU657npibIqTKcCZxV+pnGlLIJHWHPUklj1EG+OHZGLqwyJFGibElDFurviZzGWk/j0HbG1Iz1qjcX//N6mYlugpzLNDMo2XJRlAliEjL/nAy5QmbE1BLKFLe3EjamijJj86nYELzVl9dJp9nw3IZ336y1roo4ynAG53AJHlxDC+6gDT4w4PAMr/DmSOfFeXc+lq0lp5g5hT9wPn8A3YKOAA==</latexit><latexit sha1_base64="WwGo1Rp5Q2OaeI9VfuNwpkrOtzo=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsJ+3SzSbsboQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dkobm1vbO+Xdyt7+weFR9fiko5NMMfRZIhL1GFKNgkv0DTcCH1OFNA4FdsPJ7dzvPqHSPJEPZppiENOR5BFn1FjJr/uDcX1QrbkNdwGyTryC1KBAe1D96g8TlsUoDRNU657npibIqTKcCZxV+pnGlLIJHWHPUklj1EG+OHZGLqwyJFGibElDFurviZzGWk/j0HbG1Iz1qjcX//N6mYlugpzLNDMo2XJRlAliEjL/nAy5QmbE1BLKFLe3EjamijJj86nYELzVl9dJp9nw3IZ336y1roo4ynAG53AJHlxDC+6gDT4w4PAMr/DmSOfFeXc+lq0lp5g5hT9wPn8A3YKOAA==</latexit><latexit sha1_base64="WwGo1Rp5Q2OaeI9VfuNwpkrOtzo=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsJ+3SzSbsboQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dkobm1vbO+Xdyt7+weFR9fiko5NMMfRZIhL1GFKNgkv0DTcCH1OFNA4FdsPJ7dzvPqHSPJEPZppiENOR5BFn1FjJr/uDcX1QrbkNdwGyTryC1KBAe1D96g8TlsUoDRNU657npibIqTKcCZxV+pnGlLIJHWHPUklj1EG+OHZGLqwyJFGibElDFurviZzGWk/j0HbG1Iz1qjcX//N6mYlugpzLNDMo2XJRlAliEjL/nAy5QmbE1BLKFLe3EjamijJj86nYELzVl9dJp9nw3IZ336y1roo4ynAG53AJHlxDC+6gDT4w4PAMr/DmSOfFeXc+lq0lp5g5hT9wPn8A3YKOAA==</latexit>
Wh <latexit sha1_base64="3UtSFnvG9s5LzXvAasw77bxdiww=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr07GNcH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0E+RCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllrXRVxlOEMzuESPLiGFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4A4JCOAg==</latexit><latexit sha1_base64="3UtSFnvG9s5LzXvAasw77bxdiww=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr07GNcH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0E+RCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllrXRVxlOEMzuESPLiGFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4A4JCOAg==</latexit><latexit sha1_base64="3UtSFnvG9s5LzXvAasw77bxdiww=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr07GNcH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0E+RCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllrXRVxlOEMzuESPLiGFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4A4JCOAg==</latexit><latexit sha1_base64="3UtSFnvG9s5LzXvAasw77bxdiww=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr07GNcH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0E+RCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllrXRVxlOEMzuESPLiGFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4A4JCOAg==</latexit>
h <latexit sha1_base64="D0ousYKRBvlWYFmcQmQNhtQFSy0=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaT9UaVfKvtVfwGyToKclCFHo1/66g0SlklUlglqTDfwUxtOqbacCZwVe5nBlLIxHWLXUUUlmnC6uHhGLp0yIHGiXSlLFurviSmVxkxk5DoltSOz6s3F/7xuZuObcMpVmllUbLkozgSxCZm/TwZcI7Ni4ghlmrtbCRtRTZl1IRVdCMHqy+ukVasGfjW4r5Xr9TyOApzDBVxBANdQhztoQBMYKHiGV3jzjPfivXsfy9YNL585gz/wPn8A03SQVg==</latexit><latexit sha1_base64="D0ousYKRBvlWYFmcQmQNhtQFSy0=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaT9UaVfKvtVfwGyToKclCFHo1/66g0SlklUlglqTDfwUxtOqbacCZwVe5nBlLIxHWLXUUUlmnC6uHhGLp0yIHGiXSlLFurviSmVxkxk5DoltSOz6s3F/7xuZuObcMpVmllUbLkozgSxCZm/TwZcI7Ni4ghlmrtbCRtRTZl1IRVdCMHqy+ukVasGfjW4r5Xr9TyOApzDBVxBANdQhztoQBMYKHiGV3jzjPfivXsfy9YNL585gz/wPn8A03SQVg==</latexit><latexit sha1_base64="D0ousYKRBvlWYFmcQmQNhtQFSy0=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaT9UaVfKvtVfwGyToKclCFHo1/66g0SlklUlglqTDfwUxtOqbacCZwVe5nBlLIxHWLXUUUlmnC6uHhGLp0yIHGiXSlLFurviSmVxkxk5DoltSOz6s3F/7xuZuObcMpVmllUbLkozgSxCZm/TwZcI7Ni4ghlmrtbCRtRTZl1IRVdCMHqy+ukVasGfjW4r5Xr9TyOApzDBVxBANdQhztoQBMYKHiGV3jzjPfivXsfy9YNL585gz/wPn8A03SQVg==</latexit><latexit sha1_base64="D0ousYKRBvlWYFmcQmQNhtQFSy0=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaT9UaVfKvtVfwGyToKclCFHo1/66g0SlklUlglqTDfwUxtOqbacCZwVe5nBlLIxHWLXUUUlmnC6uHhGLp0yIHGiXSlLFurviSmVxkxk5DoltSOz6s3F/7xuZuObcMpVmllUbLkozgSxCZm/TwZcI7Ni4ghlmrtbCRtRTZl1IRVdCMHqy+ukVasGfjW4r5Xr9TyOApzDBVxBANdQhztoQBMYKHiGV3jzjPfivXsfy9YNL585gz/wPn8A03SQVg==</latexit>rt <latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit>
ht<latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit>ht 1<latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit> xt<latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit>
ht<latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit> ht 1<latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit> xt<latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit> h<latexit sha1_base64="69P0MQb1+ipK1vhUuUqNxbRCjpA=">AAAB8XicbVDLSgNBEOyNrxhfUY9eBhPBU9jNRU8S8OIxgnlgsoTZyWwyZB7LzKwQlvyFFw+KePVvvPk3TpI9aGJBQ1HVTXdXlHBmrO9/e4WNza3tneJuaW//4PCofHzSNirVhLaI4kp3I2woZ5K2LLOcdhNNsYg47UST27nfeaLaMCUf7DShocAjyWJGsHXSY7Vv2Ejgwbg6KFf8mr8AWidBTiqQozkof/WHiqSCSks4NqYX+IkNM6wtI5zOSv3U0ASTCR7RnqMSC2rCbHHxDF04ZYhipV1Jixbq74kMC2OmInKdAtuxWfXm4n9eL7XxdZgxmaSWSrJcFKccWYXm76Mh05RYPnUEE83crYiMscbEupBKLoRg9eV10q7XAr8W3NcrjZs8jiKcwTlcQgBX0IA7aEILCEh4hld484z34r17H8vWgpfPnMIfeJ8/0tqQVA==</latexit><latexit sha1_base64="69P0MQb1+ipK1vhUuUqNxbRCjpA=">AAAB8XicbVDLSgNBEOyNrxhfUY9eBhPBU9jNRU8S8OIxgnlgsoTZyWwyZB7LzKwQlvyFFw+KePVvvPk3TpI9aGJBQ1HVTXdXlHBmrO9/e4WNza3tneJuaW//4PCofHzSNirVhLaI4kp3I2woZ5K2LLOcdhNNsYg47UST27nfeaLaMCUf7DShocAjyWJGsHXSY7Vv2Ejgwbg6KFf8mr8AWidBTiqQozkof/WHiqSCSks4NqYX+IkNM6wtI5zOSv3U0ASTCR7RnqMSC2rCbHHxDF04ZYhipV1Jixbq74kMC2OmInKdAtuxWfXm4n9eL7XxdZgxmaSWSrJcFKccWYXm76Mh05RYPnUEE83crYiMscbEupBKLoRg9eV10q7XAr8W3NcrjZs8jiKcwTlcQgBX0IA7aEILCEh4hld484z34r17H8vWgpfPnMIfeJ8/0tqQVA==</latexit><latexit sha1_base64="69P0MQb1+ipK1vhUuUqNxbRCjpA=">AAAB8XicbVDLSgNBEOyNrxhfUY9eBhPBU9jNRU8S8OIxgnlgsoTZyWwyZB7LzKwQlvyFFw+KePVvvPk3TpI9aGJBQ1HVTXdXlHBmrO9/e4WNza3tneJuaW//4PCofHzSNirVhLaI4kp3I2woZ5K2LLOcdhNNsYg47UST27nfeaLaMCUf7DShocAjyWJGsHXSY7Vv2Ejgwbg6KFf8mr8AWidBTiqQozkof/WHiqSCSks4NqYX+IkNM6wtI5zOSv3U0ASTCR7RnqMSC2rCbHHxDF04ZYhipV1Jixbq74kMC2OmInKdAtuxWfXm4n9eL7XxdZgxmaSWSrJcFKccWYXm76Mh05RYPnUEE83crYiMscbEupBKLoRg9eV10q7XAr8W3NcrjZs8jiKcwTlcQgBX0IA7aEILCEh4hld484z34r17H8vWgpfPnMIfeJ8/0tqQVA==</latexit><latexit sha1_base64="69P0MQb1+ipK1vhUuUqNxbRCjpA=">AAAB8XicbVDLSgNBEOyNrxhfUY9eBhPBU9jNRU8S8OIxgnlgsoTZyWwyZB7LzKwQlvyFFw+KePVvvPk3TpI9aGJBQ1HVTXdXlHBmrO9/e4WNza3tneJuaW//4PCofHzSNirVhLaI4kp3I2woZ5K2LLOcdhNNsYg47UST27nfeaLaMCUf7DShocAjyWJGsHXSY7Vv2Ejgwbg6KFf8mr8AWidBTiqQozkof/WHiqSCSks4NqYX+IkNM6wtI5zOSv3U0ASTCR7RnqMSC2rCbHHxDF04ZYhipV1Jixbq74kMC2OmInKdAtuxWfXm4n9eL7XxdZgxmaSWSrJcFKccWYXm76Mh05RYPnUEE83crYiMscbEupBKLoRg9eV10q7XAr8W3NcrjZs8jiKcwTlcQgBX0IA7aEILCEh4hld484z34r17H8vWgpfPnMIfeJ8/0tqQVA==</latexit>
+ <latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit><latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit><latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit><latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit>
Taking a sequence to sequence multiclass classification problem as an example, we observe m sequences of data points (xi,t, zi,t) T t=1, where xi,t ∈ Rdx and the class label zi,t ∈ {1, . . . ,K} for all t = 1, ..., T and i = 1, ...,m. Each sequence is drawn independently from some underlying distribution over Rdx×T × {1, . . . ,K}. The vanilla RNNs compute hi,t and yi,t iteratively as follows,
hi,t = σh (Uhi,t−1 +Wxi,t) , and yi,t = σy (V hi,t) ,
where σy and σh are activation operators, hi,t ∈ Rdh is the hidden state with hi,0 = 0, and U ∈ Rdh×dh , V ∈ Rdy×dh , and W ∈ Rdh×dx are weight matrices. The activation operators σh and σy are entrywise, i.e., σh([v1, . . . , vd] > ) = [σh(v1), . . . , σh(vd)]
>, and Lipschitz with parameters ρh and ρy respectively. For simplicity, we assume σh(·) = tanh(·), σy(0) = 0, and ρy = 1. Extensions to general activation operators are given in Section 2. For a new testing sequence (xt, zt) T t=1, we predict the label sequence using
z̃t = argmaxj [yt]j , for all t = 1, . . . , T.
To establish the generalization bound, we need to define the “model complexity” of vanilla RNNs. In this paper, we adopt the empirical Rademacher complexity (ERC, see more details in Section 2), which has been widely used in the existing literature on PAC-Learning. For many nonparametric function classes, we often need complicated argument to upper bound their ERC. Our analysis, however, shows that we can upper bound the ERC of vanilla RNNs in a very simple manner by exploiting their Lipschitz continuity with respect to (w.r.t) the model parameters, since they are essentially in parametric forms. More specifically, denote Ft = {ft : {x1, ..., xt} 7→ yt} as the class of mappings from the first t inputs to the t-th output computed by vanilla RNNs. For a matrix A, ‖A‖2 denotes the spectral norm, and for a vector v, ‖v‖2 denotes the Euclidean norm. Define xt−1 x−1 = t for x = 1. Then, informally speaking, the “model complexity” of vanilla RNNs satisfies
Complexity = O ( dmin {√ d, ‖W‖2
‖U‖t2 − 1 ‖U‖2 − 1
} ‖V ‖2 √ log ( t √ d ‖U‖t2 − 1 ‖U‖2 − 1 )) ,
where d = √ dxdh + d2h + dhdy . We then give the generalization bound in the following statement.
Theorem 1 (informal). Given a collection of samples S = { (xi,t, zi,t) T t=1, i = 1, ...,m } with ‖xi,t‖2 ≤ 1 and a new testing sequence (xt, zt)Tt=1, with probability at least 1 − δ over S, for every margin value γ > 0 and ft ∈ F for integer t ≤ T , we have,
P(z̃t 6= zt) ≤ R̂γ,t +O Complexity√ mγ + √ log 1δ m  , (1) where R̂γ,t = 1m ∑m i=1 1([yi,t]zi,t ≤ maxj 6=zi,t [yi,t]j + γ).
Please refer to Section 2 for a complete statement. The generalization bound in Theorem 1 can be interpreted under three different scenarios1: (I) When ‖U‖2 < 1, the generalization bound is Õ (
d√ mγ
) , which only has a logarithmic dependence on t; (II) When ‖U‖2 = 1, the generaliza-
tion bound is Õ (
dt√ mγ
) , which has a linear dependence on d and t; (III) When ‖U‖2 > 1, the
generalization bound is Õ (√
d3t√ mγ
) , which has a polynomial dependence on d and t.
We theoretically justify that vanilla RNNs do not suffer from significant curse of dimensionality. Because they compute outputs yt recursively using the same weight matrices, and their hidden states ht are entrywise bounded.
Compared with the generalization bound in Zhang et al. (2018), which is of the order
Õ ( dt2‖W‖2‖V ‖2 max{1, ‖U‖t2}√
mγ
) ,
our bound is tighter by a factor of t2 for case (I), a factor of t for case (II). Additionally, Zhang et al. (2018) fail to incorporate the boundedness condition of hidden state into their analysis, thus the generalization bound is exponential in t for case (III). Our generalization bound, however, is still polynomial in d and t for case (III).
Moreover, (II) is closely related to a few recent results on imposing orthogonal constraints on weight matrices to stabilize the training of RNNs (Saxe et al., 2013; Le et al., 2015; Arjovsky et al., 2016; Vorontsov et al., 2017; Zhang et al., 2018). We remark that from a learning theory perspective, (II) also implies that the orthogonal constraints can potentially help generalization.
We also present refined generalization bounds with additional matrix norm assumptions. These assumptions allow us to derive norm-based generalization bounds. We draw a comparison among these bounds and highlight their advantage under different scenarios.
Our theory can be further extended to several variants, including MGU and LSTM RNNs. Specifically, we show that the gated units in MGU and LSTM RNNs can introduce extra decaying factors to further reduce the dependence on d and t in generalization. Such an advantage in generalization make these RNNs do not suffer from significant curse of dimensionality either. To the best of our knowledge, these are the first results on generalization guarantees for these neural networks.
The rest of the paper is organized as follows: Section 2 presents the generalization bound of vanilla RNNs; Section 3 presents the proof outline of the generalization bound; Section 4 presents refined generalization bounds and their comparison; Section 5 presents the generalization bound of MGU and LSTM RNNs; Section 7 discusses related works and collects open problems.
Notations: Given a vector v ∈ Rd, we denote the vector Euclidean norm by ‖v‖22 = ∑d i=1 |vi|2, and the infinity norm by ‖v‖∞ = maxj |vj |. Given a matrix M ∈ Rm×n, we denote the spectral norm by ‖M‖2 as the largest singular value ofM , the Frobenius norm by ‖M‖2F = trace(MM>), and the (2, 1) norm by ‖M‖2,1 = ∑n i=1 ‖M:,i‖2. Given a function f , we denote the function infinity norm by ‖f‖∞ = sup |f |. We adopt the standard O(·) notation, which is defined as f(x) = O (g(x)) for x → ∞ if and only if there exists M > 0 and x0, such that |f(x)| ≤ Mg(x) for x ≥ x0. We use Õ(·) to denote O(·) with hidden log factors.
1To ease the discussion, we assume ‖U‖2 does not scale with t. Therefore, ‖U‖2 < 1 is equivalent to ‖U‖2 ≤ 1−∆ for a constant ∆ > 0. A precise statement can be found following Theorem 2.

heading: 2 GENERALIZATION OF VANILLA RNNS
text: To establish the generalization bound, we start with imposing some mild assumptions.
Assumption 1. Input data are bounded, i.e., ‖xi,t‖2 ≤ Bx for all i = 1, . . . ,m and t = 1, . . . , T . Assumption 2. The spectral norms of weight matrices are bounded respectively, i.e., ‖U‖2 ≤ BU , ‖V ‖2 ≤ BV , and ‖W‖2 ≤ BW . Assumption 3. Activation operators σh and σy are Lipschitz with parameters ρh and ρy respectively, and σh(0) = σy(0) = 0. Additionally, σh is entrywise bounded by b.
Assumptions 1 and 2 are moderate assumptions. Moreover, Assumption 3 holds for most commonly used activation operators, such as σh(·) = tanh(·) and σy(·) = ReLU(·) = max{·, 0} (1-Lipschitz). Recall that vanilla RNNs compute hi,t and yi,t as follows,
hi,t = σh (Uhi,t−1 +Wxi,t) and yi,t = σy (V hi,t) ,
where U ∈ Rdh×dh , V ∈ Rdy×dh , andW ∈ Rdh×dx . Given a sequence (xt, zt)Tt=1, we defineXt ∈ Rdx×t by concatenating x1, . . . , xt as columns of Xt. Recall that we denote Ft = {ft : Xt 7→ yt} as the class of mappings from the first t inputs to the t-th output computed by vanilla RNNs. Then we define the functional margin for the t-th output in vanilla RNNs as
M(ft(Xt), zt) = [ft(Xt)]zt −maxj 6=zt [ft(Xt)]j .
We further define a ramp loss `γ (−M(ft(Xt), zt)) : R 7→ R+ to each margin, where `γ is a piecewise linear function defined as
`γ(a) = 1{a > 0}+ (1 + a
γ )1{−γ ≤ a ≤ 0},
where 1{A} denotes the indicator function of a set A. Accordingly, the ramp risk is defined as Rγ(ft) = E [`γ (−M(ft(Xt), zt))] , and its empirical counterpart is defined as R̂γ(ft) = 1 m ∑m i=1 `γ (−M(ft(Xi,t), zi,t)) . We then present the formal statement of Theorem 1.
Theorem 2. Let activation operators σh and σy be given, and Assumptions 1–3 hold. Then for (xt, zt) T t=1 and S = { (xi,t, zi,t) T t=1, i = 1, . . . ,m } drawn i.i.d. from any underlying distribution over Rdx×T × {1, . . . ,K}, with probability at least 1− δ over S, for every margin value γ > 0 and every ft ∈ Ft for integer t ≤ T , we have
P (z̃t 6= zt) ≤ R̂γ(ft) +O ( dρyBV min { b √ d,ρhBxBW βt−1 β−1 }√ log ( t √ dm β t−1 β−1 ) √ mγ + √ log 1δ m ) ,
where d = √ dxdh + d2h + dhdy and β = ρhBU .
We remark that the generalization bound depends on the total number of weights, and the range of ρhBU in three cases as indicated in Section 1. More precisely, if ρhBU . (1 + 1tα ) for constant α > 0 bounded away from zero, the generalization bound is of the order Õ ( dtα√ mγ ) , which has a polynomial dependence on d and t. As can be seen, with proper normalization on model parameters, vanilla RNNs do not suffer from significant curse of dimensionality.
We also highlight a tradeoff between generalization and representation of vanilla RNNs. As can be seen, when ρhBU is strictly smaller than 1, the generalization bound is nearly independent on t. The hidden state, however, only has limited representation ability, since its magnitude diminishes as t grows large. On the contrary, when ρhBU is strictly greater than 1, the representation ability of hidden state is amplified but the generalization becomes worse. As a consequence, recent empirical results show that imposing extra constraints or regularization, such as U>U = I or ‖U‖2 ≤ 1 (Saxe et al., 2013; Le et al., 2015; Arjovsky et al., 2016; Vorontsov et al., 2017; Zhang et al., 2018), helps balance the generalization and representation of RNNs.

heading: 3 PROOF OF MAIN RESULTS
text: Our analysis is based on the PAC-learning framework. Due to space limit, we only present an outline of our proof. More technical details are deferred to Appendix A. Before we proceed, we first define the empirical Rademacher complexity as follows. Definition 1 (Empirical Rademacher Complexity). LetH be a function class and S = {s1, . . . , sm} be a collection of samples. The empirical Rademacher complexity ofH given S is defined as
Empirical Rademacher Complexity: RS(H) = E [ sup h∈H 1 m m∑ i=1 ih(si) ] ,
where i’s are i.i.d. Rademacher random variables, i.e., P( i = 1) = P( i = −1) = 0.5.
We then proceed with our analysis. Recall that Mohri et al. (2012) give an empirical Rademacher complexity (ERC)-based generalization bound, which is restated in the following lemma with Fγ,t = {(Xt, zt) 7→ `γ(−M(ft(Xt), zt)) : ft ∈ Ft} . Lemma 1. Given a testing sequence (xt, zt)Tt=1, with probability at least 1 − δ over samples S ={ (xi,t, zi,t) T t=1, i = 1, . . . ,m } , for every margin value γ > 0 and any ft ∈ Ft, we have
P(z̃t 6= zt) ≤ Rγ(ft) ≤ R̂γ(ft) + 2RS(Fγ,t) + 3 √ log 2δ 2m . (2)
Note that Lemma 1 adapts the original version (Theorem 3.1, Chapter 3.1, Mohri et al. (2012)) for the multiclass ramp loss, and we have P(z̃t 6= zt) ≤ Rγ(ft) by definition. Now we only need to bound the ERC RS(Fγ,t). Our analysis consists of three steps. First, we characterize the Lipschitz continuity of vanilla RNNs w.r.t model parameters. Next, we bound the covering number of function class Ft. At last, we derive an upper bound on RS(Fγ,t) via the standard machinery in the PAC-learning framework. Specifically, consider two different sets of weight matrices (U, V,W ) and (U ′, V ′,W ′). Given the same activation operators and input data, denote the t-th output as yt and y′t respectively. We characterize the Lipschitz property of ‖yt‖2 w.r.t model parameters in the following lemma. Lemma 2. Under Assumptions 1–3, given input (xt)Tt=1 and for any integer t ≤ T , ‖yt‖2 is Lipschitz in U , V and W , i.e., ‖yt − y′t‖2 ≤ LU,t ‖U − U ′‖F + LV,t ‖V − V ′‖F + LW,t ‖W −W ′‖F , where LU,t = ρhBVBW tat, LV,t = BWat, and LW,t = BV at with at = ρyρhBx (ρhBU ) t−1
ρhBU−1 .
The detailed proof is provided in Appendix A.2. We give a simple example to illustrate the proof technique. Specifically, we consider a single layer network that outputs y = σ(Wx), where x is the input, σ is an activation operator with Lipschitz parameter ρ, and W is a weight matrix. Such a network is Lipschitz in both x and W as follows. Given weight matrices W and W ′, we have ‖y − y′‖2 = ‖σ(Wx)− σ(W ′x)‖2 ≤ ρ‖x‖2‖W −W ′‖F. Additionally, given inputs x and x′, we have ‖y − y′‖2 = ‖σ(Wx)− σ(Wx′)‖2 ≤ ρ‖W‖2‖x− x′‖2. Since vanilla RNNs are multilayer networks, Lemma 2 can be obtained by telescoping.
We remark that Lemma 2 is the key to the proof of our generalization bound, which separates the spectral norms of weight matrices and the total number of parameters.
Next, we bound the covering number of Ft. Denote by N (Ft, , dist(·, ·)) the minimal cardinality of a subset C ⊂ Ft that covers Ft at scale w.r.t the metric dist(·, ·), such that for any ft ∈ Ft, there exists f̂t ∈ C satisfying dist(ft, f̂t) = supXt‖ft(Xt) − f̂t(Xt)‖2 ≤ . The following lemma gives an upper bound on N (Ft, , dist(·, ·)). Lemma 3. Under Assumptions 1–3, given any > 0, the covering number of Ft satisfies
N (Ft, , dist(·, ·)) ≤ ( 1 + 6c √ dt ((ρhBU )
t − 1) (ρhBU − 1)
)3d2 ,
where c = ρyρhBVBWBxmax {1, ρhBU}.
The detailed proof is provided in Appendix A.3. We briefly explain the proof technique. Given activation operators, since vanilla RNNs are in parametric forms, ft has a one-to-one correspondence to its weight matrices U, V , and W . Lemma 2 implies that dist(·, ·) is controlled by the Frobenius norms of the difference of weight matrices. Thus, it suffices to bound the covering numbers of three weight matrices, which can be obtained by the standard machinery. The product of covering numbers of three weight matrices gives us Lemma 3.
Lastly, we give an upper bound on RS(Fγ,t) in the following lemma. Lemma 4. Under Assumptions 1–3, given activation operators and samples S = {(xi,t, zi,t)Tt=1, i = 1, . . . ,m}, the empirical Rademacher complexity RS(Fγ,t) satisfies
RS(Fγ,t) = O ( dρyBV min { b √ d, ρhBxBW (ρhBU ) t − 1
ρhBU − 1
} √ log ( t √ dm (ρhBU ) t−1
ρhBU−1 ) √ mγ ) .
The detailed proof is provided in Appendix A.4. Our proof exploits the Lipschitz continuity of M and `γ , and uses Dudley’s entropy integral as the standard machinery to establish Lemma 4. Combining Lemma 1 and Lemma 4, we complete the proof.

heading: 4 REFINED GENERALIZATION BOUNDS
text: When additional norm constraints on weight matrices U, V and W are available, we can further refine generalization bounds. Specifically, we consider the following assumptions. Assumption 4. The weight matrices satisfy ‖U‖2,1 ≤MU , ‖V ‖2,1 ≤MV , and ‖W‖2,1 ≤MW . Assumption 5. The weight matrices satisfy ‖U‖F ≤ BU,F, ‖V ‖F ≤ BV,F, and ‖W‖F ≤ BW,F. Note that Assumption 4 appears in Bartlett et al. (2017) and Assumption 5 appears in Neyshabur et al. (2017). We have an equivalent relation between matrix norms, i.e., ‖·‖2 ≤ ‖·‖2,1 ≤ √ d‖·‖F ≤ d‖ · ‖2. Comparing to Assumption 2, Assumptions 4 and 5 further restrict the model class. We then establish the following generalization bounds. Theorem 3. Let activation operators σh and σy be given, and Assumptions 1–3 hold. Then for (xt, zt) T t=1 and S = { (xi,t, zi,t) T t=1, i = 1, . . . ,m } drawn i.i.d. from any underlying distribution over Rdx×T × {1, . . . ,K}, with probability at least 1− δ over S, for every margin value γ > 0 and every ft ∈ Ft for integer t ≤ T , the following two bounds hold: • Suppose Assumption 4 also holds. We have
P (z̃t 6= zt) ≤ R̂γ(ft) +O ( α(MU+MV +MW )t βt−1 β−1 √ log d log( √ dm)√
mγ + √ log 1δ m ) , (3)
where α = ρ2hρyBVBWBx, d = √ dxdh + d2h + dhdy , and β = ρhBU .
• Suppose Assumption 5 also holds. We have P (z̃t 6= zt) ≤ R̂γ(ft) +O ( ρyρh(λhtBU+BxBW ) βt−1 β−1 √ d ln(d)(B2U,F+B2W,F+B2V,F)√
mγ
) , (4)
where λt = min { b √ d, ρhBxBW βt−1 β−1 } , d = √ dxdh + d2h + dhdy , and β = ρhBU .
The detailed proof is provided in Appendix B.1. The first bound (3) adapts the matrix covering lemma in Bartlett et al. (2017). The second bound (4) adapts the PAC-Bayes approach (Neyshabur et al., 2017) by analyzing the divergence when imposing small perturbations on the weight matrices.
We compare generalization bounds in Table 1 by differentiating ranges2 of β. It can be seen that when β > 1, both bounds (3) and (4) involve an exponential term in β. Thus, Theorem 2 yields a better result. When β ≤ 1, we distinguish two cases: the extreme case and the approximately low rank case. Specifically, remember that in the extreme case, we have ‖ · ‖2,1 = d‖ · ‖2 and
2For simplicity, we assume again that β does not scale with t. By β < 1, we mean β ≤ 1−∆ for a constant ∆ > 1. A similar characterization applies to β > 1.
‖ · ‖F = √ d‖ · ‖2, e.g., orthogonal weight matrices. Therefore, bound (4) meets Theorem 2 for β < 1 and is worse for β = 1. Bound (3) is worse than Theorem 2 for β ≤ 1. On the other hand, if the weight matrices are approximately low rank, we have ‖ · ‖2,1 d‖ · ‖2 and ‖ · ‖F √ d‖ · ‖2. In this case, bound (4) improves Theorem 2 for β < 1 by reducing dependence on d. Additionally, if t (MU +MV +MW ) < d, bound (3) yields tighter results both for β < 1 and β = 1. Note that t (MU +MV +MW ) < d also implies that the input sequence is relatively short.

heading: 5 EXTENSIONS TO MGU AND LSTM RNNS
text: We extend our analysis to Minimal Gated Unit (MRU) and Long Short-Term Memory (LSTM) RNNs. We show that these RNNs introduce extra decaying factors to further reduce the dependence on d and t in generalization.
The MGU RNNs are the simplest GRU RNNs, which compute output yt as follows,
rt = σ(Wrxt + Urht−1),
h̃t = σh (Whxt + Uh(rt ht−1)) , ht = (1− rt) ht−1 + rt h̃t, yt = σy(V ht),
where Wr,Wh ∈ Rdh×dx , Ur, Uh ∈ Rdh×dh , V ∈ Rdy×dh , and rt ∈ Rdh . The notation denotes the Hadamard product (entrywise product) of vectors. Denote by Fg,t the class of mappings from the first t inputs to the t-th output computed by gated (MGU or LSTM) RNNs. For simplicity, we consider σ being the sigmoid function, i.e., σ(x) = (1 + exp(−x))−1, σh(·) = tanh(·), and σy being ρy-Lipschitz with σy(0) = 0. Extensions to general Lipschitz activation operators as in Assumption 3 are straightforward. Suppose we have h0 = 0 and the following assumption. Assumption 6. All the weight matrices have bounded spectral norms respectively, i.e. ‖Wr‖2 ≤ BWr , ‖Wh‖2 ≤ BWh , ‖Ur‖2 ≤ BUr , ‖Uh‖2 ≤ BUh , and ‖V ‖2 ≤ BV .
Using a similar argument for vanilla RNNs yields a generalization bound of MGU RNNs as follows. Theorem 4. Let the activation operator σy be given and Assumptions 1 and 6 hold. Then for (xt, zt) T t=1 and S = { (xi,t, zi,t) T t=1, i = 1, . . . ,m } drawn i.i.d. from any underlying distribution over Rdx×T × {1, . . . ,K}, with probability at least 1− δ over S, for every margin value γ > 0 and every ft ∈ Fg,t for integer t ≤ T , we have
P (z̃t 6= zt) ≤ R̂γ(ft) +O ( dρyBV min {√ d,BWhBx βt−1 β−1 }√ log ( θt−1 θ−1 d √ m )
√ mγ
+ √ log 1δ m ) ,
where d = √ 2dxdh + 2d2h + dhdy , β = maxj≤t { ‖1− rj‖∞ +BUh ‖rj‖ 2 ∞ } , and θ = β + 2BUr +BUrBUh .
The detailed proof is provided in Appendix C.1. As can be seen, rt shrinks the magnitude of hidden state to reduce the dependence on d and t in generalization. Specifically, the hidden state ht is entrywise bounded by 1. Then for any integer t ≤ T , we have
‖rt‖∞ ≤ 1
1 + exp (−BWrBx − ‖Ur‖1) and ‖1− rt‖∞ ≤
1
1 + exp (−BWrBx − ‖Ur‖1) ,
where ‖Ur‖1 denotes the maximal absolute row sum. By restricting BWr and ‖Ur‖1 sufficiently small, we can guarantee that β and θ are strictly smaller than 1 when BUh = 1 or even BUh > 1. As a result, with proper normalization of weight matrices, the generalization bound of MGU RNNs is less dependent on d and t than that of vanilla RNNs.
The LSTM RNNs are more complicated than MGU RNNs, which introduce more gates to control the information flow in RNNs. LSTM RNNs have two hidden states, and compute them as,
<latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit>
<latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit>
<latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit>
<latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit>
<latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit>
<latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit>
gt <latexit sha1_base64="ofJKBrf7yurm8y6Rs4AdIQ4IycY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glvz4eYn1YrbkNdwmySbyC1KBAZ1j9GowSlsVcIZPUmL7nphjkVKNgks8rg8zwlLIpHfO+pYrG3AT58tg5ubLKiESJtqWQLNXfEzmNjZnFoe2MKU7MurcQ//P6GUatIBcqzZArtloUZZJgQhafk5HQnKGcWUKZFvZWwiZUU4Y2n4oNwVt/eZN0mw3PbXj3zVq7VcRRhgu4hGvw4AbacAcd8IGBgGd4hTdHOS/Ou/Oxai05xcw5/IHz+QMMf44i</latexit><latexit sha1_base64="ofJKBrf7yurm8y6Rs4AdIQ4IycY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glvz4eYn1YrbkNdwmySbyC1KBAZ1j9GowSlsVcIZPUmL7nphjkVKNgks8rg8zwlLIpHfO+pYrG3AT58tg5ubLKiESJtqWQLNXfEzmNjZnFoe2MKU7MurcQ//P6GUatIBcqzZArtloUZZJgQhafk5HQnKGcWUKZFvZWwiZUU4Y2n4oNwVt/eZN0mw3PbXj3zVq7VcRRhgu4hGvw4AbacAcd8IGBgGd4hTdHOS/Ou/Oxai05xcw5/IHz+QMMf44i</latexit><latexit sha1_base64="ofJKBrf7yurm8y6Rs4AdIQ4IycY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glvz4eYn1YrbkNdwmySbyC1KBAZ1j9GowSlsVcIZPUmL7nphjkVKNgks8rg8zwlLIpHfO+pYrG3AT58tg5ubLKiESJtqWQLNXfEzmNjZnFoe2MKU7MurcQ//P6GUatIBcqzZArtloUZZJgQhafk5HQnKGcWUKZFvZWwiZUU4Y2n4oNwVt/eZN0mw3PbXj3zVq7VcRRhgu4hGvw4AbacAcd8IGBgGd4hTdHOS/Ou/Oxai05xcw5/IHz+QMMf44i</latexit><latexit sha1_base64="ofJKBrf7yurm8y6Rs4AdIQ4IycY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glvz4eYn1YrbkNdwmySbyC1KBAZ1j9GowSlsVcIZPUmL7nphjkVKNgks8rg8zwlLIpHfO+pYrG3AT58tg5ubLKiESJtqWQLNXfEzmNjZnFoe2MKU7MurcQ//P6GUatIBcqzZArtloUZZJgQhafk5HQnKGcWUKZFvZWwiZUU4Y2n4oNwVt/eZN0mw3PbXj3zVq7VcRRhgu4hGvw4AbacAcd8IGBgGd4hTdHOS/Ou/Oxai05xcw5/IHz+QMMf44i</latexit>
rt <latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit> ot
<latexit sha1_base64="mpFsBHdFvf28lLVlN5O4kpHBGFw=">AAAB7HicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XOZJ48YiJBRNoyHbZwobtbrM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZelApu0PO+ndLW9s7uXnm/cnB4dHxSPT3rGpVpygKqhNKPETFMcMkC5CjYY6oZSSLBetH0duH3npg2XMkHnKUsTMhY8phTglYK6mqI9WG15jW8JdxN4hekBgU6w+rXYKRoljCJVBBj+r6XYpgTjZwKNq8MMsNSQqdkzPqWSpIwE+bLY+fulVVGbqy0LYnuUv09kZPEmFkS2c6E4MSsewvxP6+fYdwKcy7TDJmkq0VxJlxU7uJzd8Q1oyhmlhCqub3VpROiCUWbT8WG4K+/vEm6zYbvNfz7Zq3dKuIowwVcwjX4cANtuIMOBECBwzO8wpsjnRfn3flYtZacYuYc/sD5/AEYt44q</latexit><latexit sha1_base64="mpFsBHdFvf28lLVlN5O4kpHBGFw=">AAAB7HicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XOZJ48YiJBRNoyHbZwobtbrM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZelApu0PO+ndLW9s7uXnm/cnB4dHxSPT3rGpVpygKqhNKPETFMcMkC5CjYY6oZSSLBetH0duH3npg2XMkHnKUsTMhY8phTglYK6mqI9WG15jW8JdxN4hekBgU6w+rXYKRoljCJVBBj+r6XYpgTjZwKNq8MMsNSQqdkzPqWSpIwE+bLY+fulVVGbqy0LYnuUv09kZPEmFkS2c6E4MSsewvxP6+fYdwKcy7TDJmkq0VxJlxU7uJzd8Q1oyhmlhCqub3VpROiCUWbT8WG4K+/vEm6zYbvNfz7Zq3dKuIowwVcwjX4cANtuIMOBECBwzO8wpsjnRfn3flYtZacYuYc/sD5/AEYt44q</latexit><latexit sha1_base64="mpFsBHdFvf28lLVlN5O4kpHBGFw=">AAAB7HicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XOZJ48YiJBRNoyHbZwobtbrM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZelApu0PO+ndLW9s7uXnm/cnB4dHxSPT3rGpVpygKqhNKPETFMcMkC5CjYY6oZSSLBetH0duH3npg2XMkHnKUsTMhY8phTglYK6mqI9WG15jW8JdxN4hekBgU6w+rXYKRoljCJVBBj+r6XYpgTjZwKNq8MMsNSQqdkzPqWSpIwE+bLY+fulVVGbqy0LYnuUv09kZPEmFkS2c6E4MSsewvxP6+fYdwKcy7TDJmkq0VxJlxU7uJzd8Q1oyhmlhCqub3VpROiCUWbT8WG4K+/vEm6zYbvNfz7Zq3dKuIowwVcwjX4cANtuIMOBECBwzO8wpsjnRfn3flYtZacYuYc/sD5/AEYt44q</latexit><latexit sha1_base64="mpFsBHdFvf28lLVlN5O4kpHBGFw=">AAAB7HicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XOZJ48YiJBRNoyHbZwobtbrM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZelApu0PO+ndLW9s7uXnm/cnB4dHxSPT3rGpVpygKqhNKPETFMcMkC5CjYY6oZSSLBetH0duH3npg2XMkHnKUsTMhY8phTglYK6mqI9WG15jW8JdxN4hekBgU6w+rXYKRoljCJVBBj+r6XYpgTjZwKNq8MMsNSQqdkzPqWSpIwE+bLY+fulVVGbqy0LYnuUv09kZPEmFkS2c6E4MSsewvxP6+fYdwKcy7TDJmkq0VxJlxU7uJzd8Q1oyhmlhCqub3VpROiCUWbT8WG4K+/vEm6zYbvNfz7Zq3dKuIowwVcwjX4cANtuIMOBECBwzO8wpsjnRfn3flYtZacYuYc/sD5/AEYt44q</latexit>
c̃t <latexit sha1_base64="7IYKm8XwRp93GEETYH7gLw+FdAI=">AAAB9HicbVA9TwJBEJ3zE/ELtbTZCCZW5I5GShIbS0zkI4EL2dtbYMPe3rk7R0Iu/A4bC42x9cfY+W9c4AoFXzLJy3szmZkXJFIYdN1vZ2t7Z3dvv3BQPDw6PjktnZ23TZxqxlsslrHuBtRwKRRvoUDJu4nmNAok7wSTu4XfmXJtRKwecZZwP6IjJYaCUbSSX+mjkCHP2HyAlUGp7FbdJcgm8XJShhzNQemrH8YsjbhCJqkxPc9N0M+oRsEknxf7qeEJZRM64j1LFY248bPl0XNybZWQDGNtSyFZqr8nMhoZM4sC2xlRHJt1byH+5/VSHNb9TKgkRa7YatEwlQRjskiAhEJzhnJmCWVa2FsJG1NNGdqcijYEb/3lTdKuVT236j3Uyo16HkcBLuEKbsCDW2jAPTShBQye4Ble4c2ZOi/Ou/Oxat1y8pkL+APn8wdu9ZHU</latexit><latexit sha1_base64="7IYKm8XwRp93GEETYH7gLw+FdAI=">AAAB9HicbVA9TwJBEJ3zE/ELtbTZCCZW5I5GShIbS0zkI4EL2dtbYMPe3rk7R0Iu/A4bC42x9cfY+W9c4AoFXzLJy3szmZkXJFIYdN1vZ2t7Z3dvv3BQPDw6PjktnZ23TZxqxlsslrHuBtRwKRRvoUDJu4nmNAok7wSTu4XfmXJtRKwecZZwP6IjJYaCUbSSX+mjkCHP2HyAlUGp7FbdJcgm8XJShhzNQemrH8YsjbhCJqkxPc9N0M+oRsEknxf7qeEJZRM64j1LFY248bPl0XNybZWQDGNtSyFZqr8nMhoZM4sC2xlRHJt1byH+5/VSHNb9TKgkRa7YatEwlQRjskiAhEJzhnJmCWVa2FsJG1NNGdqcijYEb/3lTdKuVT236j3Uyo16HkcBLuEKbsCDW2jAPTShBQye4Ble4c2ZOi/Ou/Oxat1y8pkL+APn8wdu9ZHU</latexit><latexit sha1_base64="7IYKm8XwRp93GEETYH7gLw+FdAI=">AAAB9HicbVA9TwJBEJ3zE/ELtbTZCCZW5I5GShIbS0zkI4EL2dtbYMPe3rk7R0Iu/A4bC42x9cfY+W9c4AoFXzLJy3szmZkXJFIYdN1vZ2t7Z3dvv3BQPDw6PjktnZ23TZxqxlsslrHuBtRwKRRvoUDJu4nmNAok7wSTu4XfmXJtRKwecZZwP6IjJYaCUbSSX+mjkCHP2HyAlUGp7FbdJcgm8XJShhzNQemrH8YsjbhCJqkxPc9N0M+oRsEknxf7qeEJZRM64j1LFY248bPl0XNybZWQDGNtSyFZqr8nMhoZM4sC2xlRHJt1byH+5/VSHNb9TKgkRa7YatEwlQRjskiAhEJzhnJmCWVa2FsJG1NNGdqcijYEb/3lTdKuVT236j3Uyo16HkcBLuEKbsCDW2jAPTShBQye4Ble4c2ZOi/Ou/Oxat1y8pkL+APn8wdu9ZHU</latexit><latexit sha1_base64="7IYKm8XwRp93GEETYH7gLw+FdAI=">AAAB9HicbVA9TwJBEJ3zE/ELtbTZCCZW5I5GShIbS0zkI4EL2dtbYMPe3rk7R0Iu/A4bC42x9cfY+W9c4AoFXzLJy3szmZkXJFIYdN1vZ2t7Z3dvv3BQPDw6PjktnZ23TZxqxlsslrHuBtRwKRRvoUDJu4nmNAok7wSTu4XfmXJtRKwecZZwP6IjJYaCUbSSX+mjkCHP2HyAlUGp7FbdJcgm8XJShhzNQemrH8YsjbhCJqkxPc9N0M+oRsEknxf7qeEJZRM64j1LFY248bPl0XNybZWQDGNtSyFZqr8nMhoZM4sC2xlRHJt1byH+5/VSHNb9TKgkRa7YatEwlQRjskiAhEJzhnJmCWVa2FsJG1NNGdqcijYEb/3lTdKuVT236j3Uyo16HkcBLuEKbsCDW2jAPTShBQye4Ble4c2ZOi/Ou/Oxat1y8pkL+APn8wdu9ZHU</latexit>
<latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit>
<latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit>
<latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit><latexit sha1_base64="AWIQUmfbHSBXDwD7ImzIoMzPydA=">AAAB7nicbVBNT8JAEJ3iF+IX6tFLI5h4Ii0XPRK9eMREwAQast1uYcN2t9mdmpCGH+HFg8Z49fd489+4QA8KvmSSl/dmMjMvTAU36HnfTmljc2t7p7xb2ds/ODyqHp90jco0ZR2qhNKPITFMcMk6yFGwx1QzkoSC9cLJ7dzvPTFtuJIPOE1ZkJCR5DGnBK3Uqw9UpLA+rNa8hreAu078gtSgQHtY/RpEimYJk0gFMabveykGOdHIqWCzyiAzLCV0Qkasb6kkCTNBvjh35l5YJXJjpW1JdBfq74mcJMZMk9B2JgTHZtWbi/95/Qzj6yDnMs2QSbpcFGfCReXOf3cjrhlFMbWEUM3trS4dE00o2oQqNgR/9eV10m02fK/h3zdrrZsijjKcwTlcgg9X0II7aEMHKEzgGV7hzUmdF+fd+Vi2lpxi5hT+wPn8AaSxjxg=</latexit>
<latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit><latexit sha1_base64="+wPnq5mfo5j9NMc0te1uWUaQEUA=">AAAB73icbVDLTgJBEOzFF+IL9ehlIph4Irtc9Ej04hETeSSwIbPDLEyYxzoza0I2/IQXDxrj1d/x5t84wB4UrKSTSlV3uruihDNjff/bK2xsbm3vFHdLe/sHh0fl45O2UakmtEUUV7obYUM5k7RlmeW0m2iKRcRpJ5rczv3OE9WGKflgpwkNBR5JFjOCrZO61b5hI4Grg3LFr/kLoHUS5KQCOZqD8ld/qEgqqLSEY2N6gZ/YMMPaMsLprNRPDU0wmeAR7TkqsaAmzBb3ztCFU4YoVtqVtGih/p7IsDBmKiLXKbAdm1VvLv7n9VIbX4cZk0lqqSTLRXHKkVVo/jwaMk2J5VNHMNHM3YrIGGtMrIuo5EIIVl9eJ+16LfBrwX290rjJ4yjCGZzDJQRwBQ24gya0gACHZ3iFN+/Re/HevY9la8HLZ07hD7zPH1Y1j30=</latexit>
rt <latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit>
1 rt <latexit sha1_base64="wQAvzFe7l0eLgjHF3NV0yfiWEZQ=">AAAB73icbVBNS8NAEJ34WetX1aOXxVbwYkmKoMeCF48V7Ae0oWy2m3bpZhN3J0IJ/RNePCji1b/jzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1gJOE+xEdKhEKRtFKnYp3SXQfK/1S2a26c5BV4uWkDDka/dJXbxCzNOIKmaTGdD03QT+jGgWTfFrspYYnlI3pkHctVTTixs/m907JuVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbzxM6GSFLlii0VhKgnGZPY8GQjNGcqJJZRpYW8lbEQ1ZWgjKtoQvOWXV0mrVvXcqndfK9ev8jgKcApncAEeXEMd7qABTWAg4Rle4c15dF6cd+dj0brm5DMn8AfO5w9M3I7F</latexit><latexit sha1_base64="wQAvzFe7l0eLgjHF3NV0yfiWEZQ=">AAAB73icbVBNS8NAEJ34WetX1aOXxVbwYkmKoMeCF48V7Ae0oWy2m3bpZhN3J0IJ/RNePCji1b/jzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1gJOE+xEdKhEKRtFKnYp3SXQfK/1S2a26c5BV4uWkDDka/dJXbxCzNOIKmaTGdD03QT+jGgWTfFrspYYnlI3pkHctVTTixs/m907JuVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbzxM6GSFLlii0VhKgnGZPY8GQjNGcqJJZRpYW8lbEQ1ZWgjKtoQvOWXV0mrVvXcqndfK9ev8jgKcApncAEeXEMd7qABTWAg4Rle4c15dF6cd+dj0brm5DMn8AfO5w9M3I7F</latexit><latexit sha1_base64="wQAvzFe7l0eLgjHF3NV0yfiWEZQ=">AAAB73icbVBNS8NAEJ34WetX1aOXxVbwYkmKoMeCF48V7Ae0oWy2m3bpZhN3J0IJ/RNePCji1b/jzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1gJOE+xEdKhEKRtFKnYp3SXQfK/1S2a26c5BV4uWkDDka/dJXbxCzNOIKmaTGdD03QT+jGgWTfFrspYYnlI3pkHctVTTixs/m907JuVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbzxM6GSFLlii0VhKgnGZPY8GQjNGcqJJZRpYW8lbEQ1ZWgjKtoQvOWXV0mrVvXcqndfK9ev8jgKcApncAEeXEMd7qABTWAg4Rle4c15dF6cd+dj0brm5DMn8AfO5w9M3I7F</latexit><latexit sha1_base64="wQAvzFe7l0eLgjHF3NV0yfiWEZQ=">AAAB73icbVBNS8NAEJ34WetX1aOXxVbwYkmKoMeCF48V7Ae0oWy2m3bpZhN3J0IJ/RNePCji1b/jzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1gJOE+xEdKhEKRtFKnYp3SXQfK/1S2a26c5BV4uWkDDka/dJXbxCzNOIKmaTGdD03QT+jGgWTfFrspYYnlI3pkHctVTTixs/m907JuVUGJIy1LYVkrv6eyGhkzCQKbGdEcWSWvZn4n9dNMbzxM6GSFLlii0VhKgnGZPY8GQjNGcqJJZRpYW8lbEQ1ZWgjKtoQvOWXV0mrVvXcqndfK9ev8jgKcApncAEeXEMd7qABTWAg4Rle4c15dF6cd+dj0brm5DMn8AfO5w9M3I7F</latexit>
h̃t <latexit sha1_base64="S3CdFfDx9HsepZJgWjBttnLljms=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2FZoQ9lsNu3SzSbuTgol9Hd48aCIV3+MN/+N24+Dtj4YeLw3w8y8IJXCoOt+O4WNza3tneJuaW//4PCofHzSNkmmGW+xRCb6MaCGS6F4CwVK/phqTuNA8k4wup35nTHXRiTqAScp92M6UCISjKKV/GoPhQx5Ppz2sdovV9yaOwdZJ96SVGCJZr/81QsTlsVcIZPUmK7npujnVKNgkk9LvczwlLIRHfCupYrG3Pj5/OgpubBKSKJE21JI5urviZzGxkziwHbGFIdm1ZuJ/3ndDKMbPxcqzZArtlgUZZJgQmYJkFBozlBOLKFMC3srYUOqKUObU8mG4K2+vE7a9Zrn1rz7eqVxtYyjCGdwDpfgwTU04A6a0AIGT/AMr/DmjJ0X5935WLQWnOXMKfyB8/kDdWmR1Q==</latexit><latexit sha1_base64="S3CdFfDx9HsepZJgWjBttnLljms=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2FZoQ9lsNu3SzSbuTgol9Hd48aCIV3+MN/+N24+Dtj4YeLw3w8y8IJXCoOt+O4WNza3tneJuaW//4PCofHzSNkmmGW+xRCb6MaCGS6F4CwVK/phqTuNA8k4wup35nTHXRiTqAScp92M6UCISjKKV/GoPhQx5Ppz2sdovV9yaOwdZJ96SVGCJZr/81QsTlsVcIZPUmK7npujnVKNgkk9LvczwlLIRHfCupYrG3Pj5/OgpubBKSKJE21JI5urviZzGxkziwHbGFIdm1ZuJ/3ndDKMbPxcqzZArtlgUZZJgQmYJkFBozlBOLKFMC3srYUOqKUObU8mG4K2+vE7a9Zrn1rz7eqVxtYyjCGdwDpfgwTU04A6a0AIGT/AMr/DmjJ0X5935WLQWnOXMKfyB8/kDdWmR1Q==</latexit><latexit sha1_base64="S3CdFfDx9HsepZJgWjBttnLljms=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2FZoQ9lsNu3SzSbuTgol9Hd48aCIV3+MN/+N24+Dtj4YeLw3w8y8IJXCoOt+O4WNza3tneJuaW//4PCofHzSNkmmGW+xRCb6MaCGS6F4CwVK/phqTuNA8k4wup35nTHXRiTqAScp92M6UCISjKKV/GoPhQx5Ppz2sdovV9yaOwdZJ96SVGCJZr/81QsTlsVcIZPUmK7npujnVKNgkk9LvczwlLIRHfCupYrG3Pj5/OgpubBKSKJE21JI5urviZzGxkziwHbGFIdm1ZuJ/3ndDKMbPxcqzZArtlgUZZJgQmYJkFBozlBOLKFMC3srYUOqKUObU8mG4K2+vE7a9Zrn1rz7eqVxtYyjCGdwDpfgwTU04A6a0AIGT/AMr/DmjJ0X5935WLQWnOXMKfyB8/kDdWmR1Q==</latexit><latexit sha1_base64="S3CdFfDx9HsepZJgWjBttnLljms=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4r2FZoQ9lsNu3SzSbuTgol9Hd48aCIV3+MN/+N24+Dtj4YeLw3w8y8IJXCoOt+O4WNza3tneJuaW//4PCofHzSNkmmGW+xRCb6MaCGS6F4CwVK/phqTuNA8k4wup35nTHXRiTqAScp92M6UCISjKKV/GoPhQx5Ppz2sdovV9yaOwdZJ96SVGCJZr/81QsTlsVcIZPUmK7npujnVKNgkk9LvczwlLIRHfCupYrG3Pj5/OgpubBKSKJE21JI5urviZzGxkziwHbGFIdm1ZuJ/3ndDKMbPxcqzZArtlgUZZJgQmYJkFBozlBOLKFMC3srYUOqKUObU8mG4K2+vE7a9Zrn1rz7eqVxtYyjCGdwDpfgwTU04A6a0AIGT/AMr/DmjJ0X5935WLQWnOXMKfyB8/kDdWmR1Q==</latexit>
Wg <latexit sha1_base64="CXq0WSt6oXSKT6+bmuVa67gCis0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmCTTjPsskYnuhdRwKRT3UaDkvVRzGoeSd8Pp3cLvPnFtRKIecZbyIKZjJSLBKFrJr3eH4/qwWnMb7hJkk3gFqUGB9rD6NRglLIu5QiapMX3PTTHIqUbBJJ9XBpnhKWVTOuZ9SxWNuQny5bFzcmWVEYkSbUshWaq/J3IaGzOLQ9sZU5yYdW8h/uf1M4xug1yoNEOu2GpRlEmCCVl8TkZCc4ZyZgllWthbCZtQTRnafCo2BG/95U3SaTY8t+E9NGutZhFHGS7gEq7BgxtowT20wQcGAp7hFd4c5bw4787HqrXkFDPn8AfO5w/ecY3/</latexit><latexit sha1_base64="CXq0WSt6oXSKT6+bmuVa67gCis0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmCTTjPsskYnuhdRwKRT3UaDkvVRzGoeSd8Pp3cLvPnFtRKIecZbyIKZjJSLBKFrJr3eH4/qwWnMb7hJkk3gFqUGB9rD6NRglLIu5QiapMX3PTTHIqUbBJJ9XBpnhKWVTOuZ9SxWNuQny5bFzcmWVEYkSbUshWaq/J3IaGzOLQ9sZU5yYdW8h/uf1M4xug1yoNEOu2GpRlEmCCVl8TkZCc4ZyZgllWthbCZtQTRnafCo2BG/95U3SaTY8t+E9NGutZhFHGS7gEq7BgxtowT20wQcGAp7hFd4c5bw4787HqrXkFDPn8AfO5w/ecY3/</latexit><latexit sha1_base64="CXq0WSt6oXSKT6+bmuVa67gCis0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmCTTjPsskYnuhdRwKRT3UaDkvVRzGoeSd8Pp3cLvPnFtRKIecZbyIKZjJSLBKFrJr3eH4/qwWnMb7hJkk3gFqUGB9rD6NRglLIu5QiapMX3PTTHIqUbBJJ9XBpnhKWVTOuZ9SxWNuQny5bFzcmWVEYkSbUshWaq/J3IaGzOLQ9sZU5yYdW8h/uf1M4xug1yoNEOu2GpRlEmCCVl8TkZCc4ZyZgllWthbCZtQTRnafCo2BG/95U3SaTY8t+E9NGutZhFHGS7gEq7BgxtowT20wQcGAp7hFd4c5bw4787HqrXkFDPn8AfO5w/ecY3/</latexit><latexit sha1_base64="CXq0WSt6oXSKT6+bmuVa67gCis0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmCTTjPsskYnuhdRwKRT3UaDkvVRzGoeSd8Pp3cLvPnFtRKIecZbyIKZjJSLBKFrJr3eH4/qwWnMb7hJkk3gFqUGB9rD6NRglLIu5QiapMX3PTTHIqUbBJJ9XBpnhKWVTOuZ9SxWNuQny5bFzcmWVEYkSbUshWaq/J3IaGzOLQ9sZU5yYdW8h/uf1M4xug1yoNEOu2GpRlEmCCVl8TkZCc4ZyZgllWthbCZtQTRnafCo2BG/95U3SaTY8t+E9NGutZhFHGS7gEq7BgxtowT20wQcGAp7hFd4c5bw4787HqrXkFDPn8AfO5w/ecY3/</latexit>
Ug <latexit sha1_base64="hIJlFHarrACsHKkWbPIAATdJL/4=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmCTTjPsskYnuhdRwKRT3UaDkvVRzGoeSd8Pp3cLvPnFtRKIecZbyIKZjJSLBKFrJr/vDcX1YrbkNdwmySbyC1KBAe1j9GowSlsVcIZPUmL7nphjkVKNgks8rg8zwlLIpHfO+pYrG3AT58tg5ubLKiESJtqWQLNXfEzmNjZnFoe2MKU7MurcQ//P6GUa3QS5UmiFXbLUoyiTBhCw+JyOhOUM5s4QyLeythE2opgxtPhUbgrf+8ibpNBue2/AemrVWs4ijDBdwCdfgwQ204B7a4AMDAc/wCm+Ocl6cd+dj1Vpyiplz+APn8wfbY439</latexit><latexit sha1_base64="hIJlFHarrACsHKkWbPIAATdJL/4=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmCTTjPsskYnuhdRwKRT3UaDkvVRzGoeSd8Pp3cLvPnFtRKIecZbyIKZjJSLBKFrJr/vDcX1YrbkNdwmySbyC1KBAe1j9GowSlsVcIZPUmL7nphjkVKNgks8rg8zwlLIpHfO+pYrG3AT58tg5ubLKiESJtqWQLNXfEzmNjZnFoe2MKU7MurcQ//P6GUa3QS5UmiFXbLUoyiTBhCw+JyOhOUM5s4QyLeythE2opgxtPhUbgrf+8ibpNBue2/AemrVWs4ijDBdwCdfgwQ204B7a4AMDAc/wCm+Ocl6cd+dj1Vpyiplz+APn8wfbY439</latexit><latexit sha1_base64="hIJlFHarrACsHKkWbPIAATdJL/4=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmCTTjPsskYnuhdRwKRT3UaDkvVRzGoeSd8Pp3cLvPnFtRKIecZbyIKZjJSLBKFrJr/vDcX1YrbkNdwmySbyC1KBAe1j9GowSlsVcIZPUmL7nphjkVKNgks8rg8zwlLIpHfO+pYrG3AT58tg5ubLKiESJtqWQLNXfEzmNjZnFoe2MKU7MurcQ//P6GUa3QS5UmiFXbLUoyiTBhCw+JyOhOUM5s4QyLeythE2opgxtPhUbgrf+8ibpNBue2/AemrVWs4ijDBdwCdfgwQ204B7a4AMDAc/wCm+Ocl6cd+dj1Vpyiplz+APn8wfbY439</latexit><latexit sha1_base64="hIJlFHarrACsHKkWbPIAATdJL/4=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmCTTjPsskYnuhdRwKRT3UaDkvVRzGoeSd8Pp3cLvPnFtRKIecZbyIKZjJSLBKFrJr/vDcX1YrbkNdwmySbyC1KBAe1j9GowSlsVcIZPUmL7nphjkVKNgks8rg8zwlLIpHfO+pYrG3AT58tg5ubLKiESJtqWQLNXfEzmNjZnFoe2MKU7MurcQ//P6GUa3QS5UmiFXbLUoyiTBhCw+JyOhOUM5s4QyLeythE2opgxtPhUbgrf+8ibpNBue2/AemrVWs4ijDBdwCdfgwQ204B7a4AMDAc/wCm+Ocl6cd+dj1Vpyiplz+APn8wfbY439</latexit>
Wr<latexit sha1_base64="vz/mx7uwdW27AGvZy2UCSZZe0Fc=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv59e5Q1YfVmttwlyCbxCtIDQq0h9WvwShhWYzSMEG17ntuaoKcKsOZwHllkGlMKZvSMfYtlTRGHeTLY+fkyiojEiXKljRkqf6eyGms9SwObWdMzUSvewvxP6+fmeg2yLlMM4OSrRZFmSAmIYvPyYgrZEbMLKFMcXsrYROqKDM2n4oNwVt/eZN0mg3PbXgPzVqrWcRRhgu4hGvw4AZacA9t8IEBh2d4hTdHOi/Ou/Oxai05xcw5/IHz+QPvKI4K</latexit><latexit sha1_base64="vz/mx7uwdW27AGvZy2UCSZZe0Fc=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv59e5Q1YfVmttwlyCbxCtIDQq0h9WvwShhWYzSMEG17ntuaoKcKsOZwHllkGlMKZvSMfYtlTRGHeTLY+fkyiojEiXKljRkqf6eyGms9SwObWdMzUSvewvxP6+fmeg2yLlMM4OSrRZFmSAmIYvPyYgrZEbMLKFMcXsrYROqKDM2n4oNwVt/eZN0mg3PbXgPzVqrWcRRhgu4hGvw4AZacA9t8IEBh2d4hTdHOi/Ou/Oxai05xcw5/IHz+QPvKI4K</latexit><latexit sha1_base64="vz/mx7uwdW27AGvZy2UCSZZe0Fc=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv59e5Q1YfVmttwlyCbxCtIDQq0h9WvwShhWYzSMEG17ntuaoKcKsOZwHllkGlMKZvSMfYtlTRGHeTLY+fkyiojEiXKljRkqf6eyGms9SwObWdMzUSvewvxP6+fmeg2yLlMM4OSrRZFmSAmIYvPyYgrZEbMLKFMcXsrYROqKDM2n4oNwVt/eZN0mg3PbXgPzVqrWcRRhgu4hGvw4AZacA9t8IEBh2d4hTdHOi/Ou/Oxai05xcw5/IHz+QPvKI4K</latexit><latexit sha1_base64="vz/mx7uwdW27AGvZy2UCSZZe0Fc=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv59e5Q1YfVmttwlyCbxCtIDQq0h9WvwShhWYzSMEG17ntuaoKcKsOZwHllkGlMKZvSMfYtlTRGHeTLY+fkyiojEiXKljRkqf6eyGms9SwObWdMzUSvewvxP6+fmeg2yLlMM4OSrRZFmSAmIYvPyYgrZEbMLKFMcXsrYROqKDM2n4oNwVt/eZN0mg3PbXgPzVqrWcRRhgu4hGvw4AZacA9t8IEBh2d4hTdHOi/Ou/Oxai05xcw5/IHz+QPvKI4K</latexit>
Ur<latexit sha1_base64="bUgvZyrQPSR6O7BkYzmRNV/Fvuc=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv5dX+o6sNqzW24S5BN4hWkBgXaw+rXYJSwLEZpmKBa9z03NUFOleFM4LwyyDSmlE3pGPuWShqjDvLlsXNyZZURiRJlSxqyVH9P5DTWehaHtjOmZqLXvYX4n9fPTHQb5FymmUHJVouiTBCTkMXnZMQVMiNmllCmuL2VsAlVlBmbT8WG4K2/vEk6zYbnNryHZq3VLOIowwVcwjV4cAMtuIc2+MCAwzO8wpsjnRfn3flYtZacYuYc/sD5/AHsGo4I</latexit><latexit sha1_base64="bUgvZyrQPSR6O7BkYzmRNV/Fvuc=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv5dX+o6sNqzW24S5BN4hWkBgXaw+rXYJSwLEZpmKBa9z03NUFOleFM4LwyyDSmlE3pGPuWShqjDvLlsXNyZZURiRJlSxqyVH9P5DTWehaHtjOmZqLXvYX4n9fPTHQb5FymmUHJVouiTBCTkMXnZMQVMiNmllCmuL2VsAlVlBmbT8WG4K2/vEk6zYbnNryHZq3VLOIowwVcwjV4cAMtuIc2+MCAwzO8wpsjnRfn3flYtZacYuYc/sD5/AHsGo4I</latexit><latexit sha1_base64="bUgvZyrQPSR6O7BkYzmRNV/Fvuc=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv5dX+o6sNqzW24S5BN4hWkBgXaw+rXYJSwLEZpmKBa9z03NUFOleFM4LwyyDSmlE3pGPuWShqjDvLlsXNyZZURiRJlSxqyVH9P5DTWehaHtjOmZqLXvYX4n9fPTHQb5FymmUHJVouiTBCTkMXnZMQVMiNmllCmuL2VsAlVlBmbT8WG4K2/vEk6zYbnNryHZq3VLOIowwVcwjV4cAMtuIc2+MCAwzO8wpsjnRfn3flYtZacYuYc/sD5/AHsGo4I</latexit><latexit sha1_base64="bUgvZyrQPSR6O7BkYzmRNV/Fvuc=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv5dX+o6sNqzW24S5BN4hWkBgXaw+rXYJSwLEZpmKBa9z03NUFOleFM4LwyyDSmlE3pGPuWShqjDvLlsXNyZZURiRJlSxqyVH9P5DTWehaHtjOmZqLXvYX4n9fPTHQb5FymmUHJVouiTBCTkMXnZMQVMiNmllCmuL2VsAlVlBmbT8WG4K2/vEk6zYbnNryHZq3VLOIowwVcwjV4cAMtuIc2+MCAwzO8wpsjnRfn3flYtZacYuYc/sD5/AHsGo4I</latexit> Uc<latexit sha1_base64="nGb0ziqbMdgWBw2LRepOfBGsyT0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv5dX/I6sNqzW24S5BN4hWkBgXaw+rXYJSwLEZpmKBa9z03NUFOleFM4LwyyDSmlE3pGPuWShqjDvLlsXNyZZURiRJlSxqyVH9P5DTWehaHtjOmZqLXvYX4n9fPTHQb5FymmUHJVouiTBCTkMXnZMQVMiNmllCmuL2VsAlVlBmbT8WG4K2/vEk6zYbnNryHZq3VLOIowwVcwjV4cAMtuIc2+MCAwzO8wpsjnRfn3flYtZacYuYc/sD5/AHVT435</latexit><latexit sha1_base64="nGb0ziqbMdgWBw2LRepOfBGsyT0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv5dX/I6sNqzW24S5BN4hWkBgXaw+rXYJSwLEZpmKBa9z03NUFOleFM4LwyyDSmlE3pGPuWShqjDvLlsXNyZZURiRJlSxqyVH9P5DTWehaHtjOmZqLXvYX4n9fPTHQb5FymmUHJVouiTBCTkMXnZMQVMiNmllCmuL2VsAlVlBmbT8WG4K2/vEk6zYbnNryHZq3VLOIowwVcwjV4cAMtuIc2+MCAwzO8wpsjnRfn3flYtZacYuYc/sD5/AHVT435</latexit><latexit sha1_base64="nGb0ziqbMdgWBw2LRepOfBGsyT0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv5dX/I6sNqzW24S5BN4hWkBgXaw+rXYJSwLEZpmKBa9z03NUFOleFM4LwyyDSmlE3pGPuWShqjDvLlsXNyZZURiRJlSxqyVH9P5DTWehaHtjOmZqLXvYX4n9fPTHQb5FymmUHJVouiTBCTkMXnZMQVMiNmllCmuL2VsAlVlBmbT8WG4K2/vEk6zYbnNryHZq3VLOIowwVcwjV4cAMtuIc2+MCAwzO8wpsjnRfn3flYtZacYuYc/sD5/AHVT435</latexit><latexit sha1_base64="nGb0ziqbMdgWBw2LRepOfBGsyT0=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv5dX/I6sNqzW24S5BN4hWkBgXaw+rXYJSwLEZpmKBa9z03NUFOleFM4LwyyDSmlE3pGPuWShqjDvLlsXNyZZURiRJlSxqyVH9P5DTWehaHtjOmZqLXvYX4n9fPTHQb5FymmUHJVouiTBCTkMXnZMQVMiNmllCmuL2VsAlVlBmbT8WG4K2/vEk6zYbnNryHZq3VLOIowwVcwjV4cAMtuIc2+MCAwzO8wpsjnRfn3flYtZacYuYc/sD5/AHVT435</latexit>
Wc<latexit sha1_base64="ICxsCyzHFX1UVaYy4kFhbj2nPGQ=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv59e6Q1YfVmttwlyCbxCtIDQq0h9WvwShhWYzSMEG17ntuaoKcKsOZwHllkGlMKZvSMfYtlTRGHeTLY+fkyiojEiXKljRkqf6eyGms9SwObWdMzUSvewvxP6+fmeg2yLlMM4OSrRZFmSAmIYvPyYgrZEbMLKFMcXsrYROqKDM2n4oNwVt/eZN0mg3PbXgPzVqrWcRRhgu4hGvw4AZacA9t8IEBh2d4hTdHOi/Ou/Oxai05xcw5/IHz+QPYXY37</latexit><latexit sha1_base64="ICxsCyzHFX1UVaYy4kFhbj2nPGQ=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv59e6Q1YfVmttwlyCbxCtIDQq0h9WvwShhWYzSMEG17ntuaoKcKsOZwHllkGlMKZvSMfYtlTRGHeTLY+fkyiojEiXKljRkqf6eyGms9SwObWdMzUSvewvxP6+fmeg2yLlMM4OSrRZFmSAmIYvPyYgrZEbMLKFMcXsrYROqKDM2n4oNwVt/eZN0mg3PbXgPzVqrWcRRhgu4hGvw4AZacA9t8IEBh2d4hTdHOi/Ou/Oxai05xcw5/IHz+QPYXY37</latexit><latexit sha1_base64="ICxsCyzHFX1UVaYy4kFhbj2nPGQ=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv59e6Q1YfVmttwlyCbxCtIDQq0h9WvwShhWYzSMEG17ntuaoKcKsOZwHllkGlMKZvSMfYtlTRGHeTLY+fkyiojEiXKljRkqf6eyGms9SwObWdMzUSvewvxP6+fmeg2yLlMM4OSrRZFmSAmIYvPyYgrZEbMLKFMcXsrYROqKDM2n4oNwVt/eZN0mg3PbXgPzVqrWcRRhgu4hGvw4AZacA9t8IEBh2d4hTdHOi/Ou/Oxai05xcw5/IHz+QPYXY37</latexit><latexit sha1_base64="ICxsCyzHFX1UVaYy4kFhbj2nPGQ=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFoueiTx4hETCyTQkO0yhQ3bbbO7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemAqujet+O6Wt7Z3dvfJ+5eDw6PikenrW0UmmGPosEYnqhVSj4BJ9w43AXqqQxqHAbji9W/jdJ1SaJ/LRzFIMYjqWPOKMGiv59e6Q1YfVmttwlyCbxCtIDQq0h9WvwShhWYzSMEG17ntuaoKcKsOZwHllkGlMKZvSMfYtlTRGHeTLY+fkyiojEiXKljRkqf6eyGms9SwObWdMzUSvewvxP6+fmeg2yLlMM4OSrRZFmSAmIYvPyYgrZEbMLKFMcXsrYROqKDM2n4oNwVt/eZN0mg3PbXgPzVqrWcRRhgu4hGvw4AZacA9t8IEBh2d4hTdHOi/Ou/Oxai05xcw5/IHz+QPYXY37</latexit> Wo<latexit sha1_base64="8dlnrEaEbzsGhOr739+jqrRqLDQ=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLUlsLDHxgAQuZG/Zgw37cdndMyEXfoONhcbY+oPs/DcucIWCL5nk5b2ZzMyLU86M9f1vr7S1vbO7V96vHBweHZ9UT886RmWa0JAornQvxoZyJmlomeW0l2qKRcxpN57eLfzuE9WGKfloZymNBB5LljCCrZPCeneo6sNqzW/4S6BNEhSkBgXaw+rXYKRIJqi0hGNj+oGf2ijH2jLC6bwyyAxNMZniMe07KrGgJsqXx87RlVNGKFHalbRoqf6eyLEwZiZi1ymwnZh1byH+5/Uzm9xGOZNpZqkkq0VJxpFVaPE5GjFNieUzRzDRzN2KyARrTKzLp+JCCNZf3iSdZiPwG8FDs9ZqFnGU4QIu4RoCuIEW3EMbQiDA4Ble4c2T3ov37n2sWkteMXMOf+B9/gDqmY4H</latexit><latexit sha1_base64="8dlnrEaEbzsGhOr739+jqrRqLDQ=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLUlsLDHxgAQuZG/Zgw37cdndMyEXfoONhcbY+oPs/DcucIWCL5nk5b2ZzMyLU86M9f1vr7S1vbO7V96vHBweHZ9UT886RmWa0JAornQvxoZyJmlomeW0l2qKRcxpN57eLfzuE9WGKfloZymNBB5LljCCrZPCeneo6sNqzW/4S6BNEhSkBgXaw+rXYKRIJqi0hGNj+oGf2ijH2jLC6bwyyAxNMZniMe07KrGgJsqXx87RlVNGKFHalbRoqf6eyLEwZiZi1ymwnZh1byH+5/Uzm9xGOZNpZqkkq0VJxpFVaPE5GjFNieUzRzDRzN2KyARrTKzLp+JCCNZf3iSdZiPwG8FDs9ZqFnGU4QIu4RoCuIEW3EMbQiDA4Ble4c2T3ov37n2sWkteMXMOf+B9/gDqmY4H</latexit><latexit sha1_base64="8dlnrEaEbzsGhOr739+jqrRqLDQ=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLUlsLDHxgAQuZG/Zgw37cdndMyEXfoONhcbY+oPs/DcucIWCL5nk5b2ZzMyLU86M9f1vr7S1vbO7V96vHBweHZ9UT886RmWa0JAornQvxoZyJmlomeW0l2qKRcxpN57eLfzuE9WGKfloZymNBB5LljCCrZPCeneo6sNqzW/4S6BNEhSkBgXaw+rXYKRIJqi0hGNj+oGf2ijH2jLC6bwyyAxNMZniMe07KrGgJsqXx87RlVNGKFHalbRoqf6eyLEwZiZi1ymwnZh1byH+5/Uzm9xGOZNpZqkkq0VJxpFVaPE5GjFNieUzRzDRzN2KyARrTKzLp+JCCNZf3iSdZiPwG8FDs9ZqFnGU4QIu4RoCuIEW3EMbQiDA4Ble4c2T3ov37n2sWkteMXMOf+B9/gDqmY4H</latexit><latexit sha1_base64="8dlnrEaEbzsGhOr739+jqrRqLDQ=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLUlsLDHxgAQuZG/Zgw37cdndMyEXfoONhcbY+oPs/DcucIWCL5nk5b2ZzMyLU86M9f1vr7S1vbO7V96vHBweHZ9UT886RmWa0JAornQvxoZyJmlomeW0l2qKRcxpN57eLfzuE9WGKfloZymNBB5LljCCrZPCeneo6sNqzW/4S6BNEhSkBgXaw+rXYKRIJqi0hGNj+oGf2ijH2jLC6bwyyAxNMZniMe07KrGgJsqXx87RlVNGKFHalbRoqf6eyLEwZiZi1ymwnZh1byH+5/Uzm9xGOZNpZqkkq0VJxpFVaPE5GjFNieUzRzDRzN2KyARrTKzLp+JCCNZf3iSdZiPwG8FDs9ZqFnGU4QIu4RoCuIEW3EMbQiDA4Ble4c2T3ov37n2sWkteMXMOf+B9/gDqmY4H</latexit>
Uo<latexit sha1_base64="Kb0zcT93dOpcc1myT8ZFtY8k+Jc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLUlsLDHxgAQuZG/Zgw37cdndMyEXfoONhcbY+oPs/DcucIWCL5nk5b2ZzMyLU86M9f1vr7S1vbO7V96vHBweHZ9UT886RmWa0JAornQvxoZyJmlomeW0l2qKRcxpN57eLfzuE9WGKfloZymNBB5LljCCrZPCejhU9WG15jf8JdAmCQpSgwLtYfVrMFIkE1RawrEx/cBPbZRjbRnhdF4ZZIammEzxmPYdlVhQE+XLY+foyikjlCjtSlq0VH9P5FgYMxOx6xTYTsy6txD/8/qZTW6jnMk0s1SS1aIk48gqtPgcjZimxPKZI5ho5m5FZII1JtblU3EhBOsvb5JOsxH4jeChWWs1izjKcAGXcA0B3EAL7qENIRBg8Ayv8OZJ78V79z5WrSWvmDmHP/A+fwDni44F</latexit><latexit sha1_base64="Kb0zcT93dOpcc1myT8ZFtY8k+Jc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLUlsLDHxgAQuZG/Zgw37cdndMyEXfoONhcbY+oPs/DcucIWCL5nk5b2ZzMyLU86M9f1vr7S1vbO7V96vHBweHZ9UT886RmWa0JAornQvxoZyJmlomeW0l2qKRcxpN57eLfzuE9WGKfloZymNBB5LljCCrZPCejhU9WG15jf8JdAmCQpSgwLtYfVrMFIkE1RawrEx/cBPbZRjbRnhdF4ZZIammEzxmPYdlVhQE+XLY+foyikjlCjtSlq0VH9P5FgYMxOx6xTYTsy6txD/8/qZTW6jnMk0s1SS1aIk48gqtPgcjZimxPKZI5ho5m5FZII1JtblU3EhBOsvb5JOsxH4jeChWWs1izjKcAGXcA0B3EAL7qENIRBg8Ayv8OZJ78V79z5WrSWvmDmHP/A+fwDni44F</latexit><latexit sha1_base64="Kb0zcT93dOpcc1myT8ZFtY8k+Jc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLUlsLDHxgAQuZG/Zgw37cdndMyEXfoONhcbY+oPs/DcucIWCL5nk5b2ZzMyLU86M9f1vr7S1vbO7V96vHBweHZ9UT886RmWa0JAornQvxoZyJmlomeW0l2qKRcxpN57eLfzuE9WGKfloZymNBB5LljCCrZPCejhU9WG15jf8JdAmCQpSgwLtYfVrMFIkE1RawrEx/cBPbZRjbRnhdF4ZZIammEzxmPYdlVhQE+XLY+foyikjlCjtSlq0VH9P5FgYMxOx6xTYTsy6txD/8/qZTW6jnMk0s1SS1aIk48gqtPgcjZimxPKZI5ho5m5FZII1JtblU3EhBOsvb5JOsxH4jeChWWs1izjKcAGXcA0B3EAL7qENIRBg8Ayv8OZJ78V79z5WrSWvmDmHP/A+fwDni44F</latexit><latexit sha1_base64="Kb0zcT93dOpcc1myT8ZFtY8k+Jc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsaLUlsLDHxgAQuZG/Zgw37cdndMyEXfoONhcbY+oPs/DcucIWCL5nk5b2ZzMyLU86M9f1vr7S1vbO7V96vHBweHZ9UT886RmWa0JAornQvxoZyJmlomeW0l2qKRcxpN57eLfzuE9WGKfloZymNBB5LljCCrZPCejhU9WG15jf8JdAmCQpSgwLtYfVrMFIkE1RawrEx/cBPbZRjbRnhdF4ZZIammEzxmPYdlVhQE+XLY+foyikjlCjtSlq0VH9P5FgYMxOx6xTYTsy6txD/8/qZTW6jnMk0s1SS1aIk48gqtPgcjZimxPKZI5ho5m5FZII1JtblU3EhBOsvb5JOsxH4jeChWWs1izjKcAGXcA0B3EAL7qENIRBg8Ayv8OZJ78V79z5WrSWvmDmHP/A+fwDni44F</latexit>
Wr <latexit sha1_base64="YH7coM1v251MW/+4mqAFEDNLMyM=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr070PVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRjdBLlQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaV0UcZTiDc7gED66hBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz/vwo4M</latexit><latexit sha1_base64="YH7coM1v251MW/+4mqAFEDNLMyM=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr070PVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRjdBLlQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaV0UcZTiDc7gED66hBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz/vwo4M</latexit><latexit sha1_base64="YH7coM1v251MW/+4mqAFEDNLMyM=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr070PVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRjdBLlQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaV0UcZTiDc7gED66hBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz/vwo4M</latexit><latexit sha1_base64="YH7coM1v251MW/+4mqAFEDNLMyM=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr070PVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRjdBLlQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaV0UcZTiDc7gED66hBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz/vwo4M</latexit>
Ur <latexit sha1_base64="BW2T5sAmwcKzhZ8g1qGxQ2wjKug=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKft0f6PqgWnMb7gJknXgFqUGB9qD61R8mLIu5QiapMT3PTTHIqUbBJJ9V+pnhKWUTOuI9SxWNuQnyxbEzcmGVIYkSbUshWai/J3IaGzONQ9sZUxybVW8u/uf1MoxuglyoNEOu2HJRlEmCCZl/ToZCc4ZyagllWthbCRtTTRnafCo2BG/15XXSaTY8t+HdN2utqyKOMpzBOVyCB9fQgjtogw8MBDzDK7w5ynlx3p2PZWvJKWZO4Q+czx/stI4K</latexit><latexit sha1_base64="BW2T5sAmwcKzhZ8g1qGxQ2wjKug=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKft0f6PqgWnMb7gJknXgFqUGB9qD61R8mLIu5QiapMT3PTTHIqUbBJJ9V+pnhKWUTOuI9SxWNuQnyxbEzcmGVIYkSbUshWai/J3IaGzONQ9sZUxybVW8u/uf1MoxuglyoNEOu2HJRlEmCCZl/ToZCc4ZyagllWthbCRtTTRnafCo2BG/15XXSaTY8t+HdN2utqyKOMpzBOVyCB9fQgjtogw8MBDzDK7w5ynlx3p2PZWvJKWZO4Q+czx/stI4K</latexit><latexit sha1_base64="BW2T5sAmwcKzhZ8g1qGxQ2wjKug=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKft0f6PqgWnMb7gJknXgFqUGB9qD61R8mLIu5QiapMT3PTTHIqUbBJJ9V+pnhKWUTOuI9SxWNuQnyxbEzcmGVIYkSbUshWai/J3IaGzONQ9sZUxybVW8u/uf1MoxuglyoNEOu2HJRlEmCCZl/ToZCc4ZyagllWthbCRtTTRnafCo2BG/15XXSaTY8t+HdN2utqyKOMpzBOVyCB9fQgjtogw8MBDzDK7w5ynlx3p2PZWvJKWZO4Q+czx/stI4K</latexit><latexit sha1_base64="BW2T5sAmwcKzhZ8g1qGxQ2wjKug=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKft0f6PqgWnMb7gJknXgFqUGB9qD61R8mLIu5QiapMT3PTTHIqUbBJJ9V+pnhKWUTOuI9SxWNuQnyxbEzcmGVIYkSbUshWai/J3IaGzONQ9sZUxybVW8u/uf1MoxuglyoNEOu2HJRlEmCCZl/ToZCc4ZyagllWthbCRtTTRnafCo2BG/15XXSaTY8t+HdN2utqyKOMpzBOVyCB9fQgjtogw8MBDzDK7w5ynlx3p2PZWvJKWZO4Q+czx/stI4K</latexit>
Uh <latexit sha1_base64="WwGo1Rp5Q2OaeI9VfuNwpkrOtzo=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsJ+3SzSbsboQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dkobm1vbO+Xdyt7+weFR9fiko5NMMfRZIhL1GFKNgkv0DTcCH1OFNA4FdsPJ7dzvPqHSPJEPZppiENOR5BFn1FjJr/uDcX1QrbkNdwGyTryC1KBAe1D96g8TlsUoDRNU657npibIqTKcCZxV+pnGlLIJHWHPUklj1EG+OHZGLqwyJFGibElDFurviZzGWk/j0HbG1Iz1qjcX//N6mYlugpzLNDMo2XJRlAliEjL/nAy5QmbE1BLKFLe3EjamijJj86nYELzVl9dJp9nw3IZ336y1roo4ynAG53AJHlxDC+6gDT4w4PAMr/DmSOfFeXc+lq0lp5g5hT9wPn8A3YKOAA==</latexit><latexit sha1_base64="WwGo1Rp5Q2OaeI9VfuNwpkrOtzo=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsJ+3SzSbsboQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dkobm1vbO+Xdyt7+weFR9fiko5NMMfRZIhL1GFKNgkv0DTcCH1OFNA4FdsPJ7dzvPqHSPJEPZppiENOR5BFn1FjJr/uDcX1QrbkNdwGyTryC1KBAe1D96g8TlsUoDRNU657npibIqTKcCZxV+pnGlLIJHWHPUklj1EG+OHZGLqwyJFGibElDFurviZzGWk/j0HbG1Iz1qjcX//N6mYlugpzLNDMo2XJRlAliEjL/nAy5QmbE1BLKFLe3EjamijJj86nYELzVl9dJp9nw3IZ336y1roo4ynAG53AJHlxDC+6gDT4w4PAMr/DmSOfFeXc+lq0lp5g5hT9wPn8A3YKOAA==</latexit><latexit sha1_base64="WwGo1Rp5Q2OaeI9VfuNwpkrOtzo=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsJ+3SzSbsboQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dkobm1vbO+Xdyt7+weFR9fiko5NMMfRZIhL1GFKNgkv0DTcCH1OFNA4FdsPJ7dzvPqHSPJEPZppiENOR5BFn1FjJr/uDcX1QrbkNdwGyTryC1KBAe1D96g8TlsUoDRNU657npibIqTKcCZxV+pnGlLIJHWHPUklj1EG+OHZGLqwyJFGibElDFurviZzGWk/j0HbG1Iz1qjcX//N6mYlugpzLNDMo2XJRlAliEjL/nAy5QmbE1BLKFLe3EjamijJj86nYELzVl9dJp9nw3IZ336y1roo4ynAG53AJHlxDC+6gDT4w4PAMr/DmSOfFeXc+lq0lp5g5hT9wPn8A3YKOAA==</latexit><latexit sha1_base64="WwGo1Rp5Q2OaeI9VfuNwpkrOtzo=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsJ+3SzSbsboQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dkobm1vbO+Xdyt7+weFR9fiko5NMMfRZIhL1GFKNgkv0DTcCH1OFNA4FdsPJ7dzvPqHSPJEPZppiENOR5BFn1FjJr/uDcX1QrbkNdwGyTryC1KBAe1D96g8TlsUoDRNU657npibIqTKcCZxV+pnGlLIJHWHPUklj1EG+OHZGLqwyJFGibElDFurviZzGWk/j0HbG1Iz1qjcX//N6mYlugpzLNDMo2XJRlAliEjL/nAy5QmbE1BLKFLe3EjamijJj86nYELzVl9dJp9nw3IZ336y1roo4ynAG53AJHlxDC+6gDT4w4PAMr/DmSOfFeXc+lq0lp5g5hT9wPn8A3YKOAA==</latexit>
Wh <latexit sha1_base64="3UtSFnvG9s5LzXvAasw77bxdiww=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr07GNcH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0E+RCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllrXRVxlOEMzuESPLiGFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4A4JCOAg==</latexit><latexit sha1_base64="3UtSFnvG9s5LzXvAasw77bxdiww=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr07GNcH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0E+RCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllrXRVxlOEMzuESPLiGFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4A4JCOAg==</latexit><latexit sha1_base64="3UtSFnvG9s5LzXvAasw77bxdiww=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr07GNcH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0E+RCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllrXRVxlOEMzuESPLiGFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4A4JCOAg==</latexit><latexit sha1_base64="3UtSFnvG9s5LzXvAasw77bxdiww=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5IUQY8FLx4rmFZoQ9lsN+3SzSbsToQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzSMUmmGfdZIhP9GFLDpVDcR4GSP6aa0ziUvBtObud+94lrIxL1gNOUBzEdKREJRtFKfr07GNcH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0E+RCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllrXRVxlOEMzuESPLiGFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4A4JCOAg==</latexit>
c <latexit sha1_base64="dB+er7u/0aPiyGEI3sSMpHypazw=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaR9VumXyn7VX4CskyAnZcjR6Je+eoOEZRKVZYIa0w381IZTqi1nAmfFXmYwpWxMh9h1VFGJJpwuLp6RS6cMSJxoV8qShfp7YkqlMRMZuU5J7cisenPxP6+b2fgmnHKVZhYVWy6KM0FsQubvkwHXyKyYOEKZ5u5WwkZUU2ZdSEUXQrD68jpp1aqBXw3ua+V6PY+jAOdwAVcQwDXU4Q4a0AQGCp7hFd484714797HsnXDy2fO4A+8zx/L25BR</latexit><latexit sha1_base64="dB+er7u/0aPiyGEI3sSMpHypazw=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaR9VumXyn7VX4CskyAnZcjR6Je+eoOEZRKVZYIa0w381IZTqi1nAmfFXmYwpWxMh9h1VFGJJpwuLp6RS6cMSJxoV8qShfp7YkqlMRMZuU5J7cisenPxP6+b2fgmnHKVZhYVWy6KM0FsQubvkwHXyKyYOEKZ5u5WwkZUU2ZdSEUXQrD68jpp1aqBXw3ua+V6PY+jAOdwAVcQwDXU4Q4a0AQGCp7hFd484714797HsnXDy2fO4A+8zx/L25BR</latexit><latexit sha1_base64="dB+er7u/0aPiyGEI3sSMpHypazw=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaR9VumXyn7VX4CskyAnZcjR6Je+eoOEZRKVZYIa0w381IZTqi1nAmfFXmYwpWxMh9h1VFGJJpwuLp6RS6cMSJxoV8qShfp7YkqlMRMZuU5J7cisenPxP6+b2fgmnHKVZhYVWy6KM0FsQubvkwHXyKyYOEKZ5u5WwkZUU2ZdSEUXQrD68jpp1aqBXw3ua+V6PY+jAOdwAVcQwDXU4Q4a0AQGCp7hFd484714797HsnXDy2fO4A+8zx/L25BR</latexit><latexit sha1_base64="dB+er7u/0aPiyGEI3sSMpHypazw=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaR9VumXyn7VX4CskyAnZcjR6Je+eoOEZRKVZYIa0w381IZTqi1nAmfFXmYwpWxMh9h1VFGJJpwuLp6RS6cMSJxoV8qShfp7YkqlMRMZuU5J7cisenPxP6+b2fgmnHKVZhYVWy6KM0FsQubvkwHXyKyYOEKZ5u5WwkZUU2ZdSEUXQrD68jpp1aqBXw3ua+V6PY+jAOdwAVcQwDXU4Q4a0AQGCp7hFd484714797HsnXDy2fO4A+8zx/L25BR</latexit>
tanh <latexit sha1_base64="6gN6AKKwFjC/Dz27jWrOkvIQpPI=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LLaCp5KUgh4LXjxWsB/QhrLZbtqlm03YnQgl9Ed48aCIV3+PN/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpb2zu7e8X90sHh0fFJ+fSsY+JUM95msYx1L6CGS6F4GwVK3ks0p1EgeTeY3i387hPXRsTqEWcJ9yM6ViIUjKKVutUBUjWpDssVt+YuQTaJl5MK5GgNy1+DUczSiCtkkhrT99wE/YxqFEzyeWmQGp5QNqVj3rdU0YgbP1ueOydXVhmRMNa2FJKl+nsio5ExsyiwnRHFiVn3FuJ/Xj/F8NbPhEpS5IqtFoWpJBiTxe9kJDRnKGeWUKaFvZWwCdWUoU2oZEPw1l/eJJ16zXNr3kO90mzkcRThAi7hGjy4gSbcQwvawGAKz/AKb07ivDjvzseqteDkM+fwB87nD4/Mjv8=</latexit><latexit sha1_base64="6gN6AKKwFjC/Dz27jWrOkvIQpPI=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LLaCp5KUgh4LXjxWsB/QhrLZbtqlm03YnQgl9Ed48aCIV3+PN/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpb2zu7e8X90sHh0fFJ+fSsY+JUM95msYx1L6CGS6F4GwVK3ks0p1EgeTeY3i387hPXRsTqEWcJ9yM6ViIUjKKVutUBUjWpDssVt+YuQTaJl5MK5GgNy1+DUczSiCtkkhrT99wE/YxqFEzyeWmQGp5QNqVj3rdU0YgbP1ueOydXVhmRMNa2FJKl+nsio5ExsyiwnRHFiVn3FuJ/Xj/F8NbPhEpS5IqtFoWpJBiTxe9kJDRnKGeWUKaFvZWwCdWUoU2oZEPw1l/eJJ16zXNr3kO90mzkcRThAi7hGjy4gSbcQwvawGAKz/AKb07ivDjvzseqteDkM+fwB87nD4/Mjv8=</latexit><latexit sha1_base64="6gN6AKKwFjC/Dz27jWrOkvIQpPI=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LLaCp5KUgh4LXjxWsB/QhrLZbtqlm03YnQgl9Ed48aCIV3+PN/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpb2zu7e8X90sHh0fFJ+fSsY+JUM95msYx1L6CGS6F4GwVK3ks0p1EgeTeY3i387hPXRsTqEWcJ9yM6ViIUjKKVutUBUjWpDssVt+YuQTaJl5MK5GgNy1+DUczSiCtkkhrT99wE/YxqFEzyeWmQGp5QNqVj3rdU0YgbP1ueOydXVhmRMNa2FJKl+nsio5ExsyiwnRHFiVn3FuJ/Xj/F8NbPhEpS5IqtFoWpJBiTxe9kJDRnKGeWUKaFvZWwCdWUoU2oZEPw1l/eJJ16zXNr3kO90mzkcRThAi7hGjy4gSbcQwvawGAKz/AKb07ivDjvzseqteDkM+fwB87nD4/Mjv8=</latexit><latexit sha1_base64="6gN6AKKwFjC/Dz27jWrOkvIQpPI=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LLaCp5KUgh4LXjxWsB/QhrLZbtqlm03YnQgl9Ed48aCIV3+PN/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpb2zu7e8X90sHh0fFJ+fSsY+JUM95msYx1L6CGS6F4GwVK3ks0p1EgeTeY3i387hPXRsTqEWcJ9yM6ViIUjKKVutUBUjWpDssVt+YuQTaJl5MK5GgNy1+DUczSiCtkkhrT99wE/YxqFEzyeWmQGp5QNqVj3rdU0YgbP1ueOydXVhmRMNa2FJKl+nsio5ExsyiwnRHFiVn3FuJ/Xj/F8NbPhEpS5IqtFoWpJBiTxe9kJDRnKGeWUKaFvZWwCdWUoU2oZEPw1l/eJJ16zXNr3kO90mzkcRThAi7hGjy4gSbcQwvawGAKz/AKb07ivDjvzseqteDkM+fwB87nD4/Mjv8=</latexit>
h <latexit sha1_base64="D0ousYKRBvlWYFmcQmQNhtQFSy0=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaT9UaVfKvtVfwGyToKclCFHo1/66g0SlklUlglqTDfwUxtOqbacCZwVe5nBlLIxHWLXUUUlmnC6uHhGLp0yIHGiXSlLFurviSmVxkxk5DoltSOz6s3F/7xuZuObcMpVmllUbLkozgSxCZm/TwZcI7Ni4ghlmrtbCRtRTZl1IRVdCMHqy+ukVasGfjW4r5Xr9TyOApzDBVxBANdQhztoQBMYKHiGV3jzjPfivXsfy9YNL585gz/wPn8A03SQVg==</latexit><latexit sha1_base64="D0ousYKRBvlWYFmcQmQNhtQFSy0=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaT9UaVfKvtVfwGyToKclCFHo1/66g0SlklUlglqTDfwUxtOqbacCZwVe5nBlLIxHWLXUUUlmnC6uHhGLp0yIHGiXSlLFurviSmVxkxk5DoltSOz6s3F/7xuZuObcMpVmllUbLkozgSxCZm/TwZcI7Ni4ghlmrtbCRtRTZl1IRVdCMHqy+ukVasGfjW4r5Xr9TyOApzDBVxBANdQhztoQBMYKHiGV3jzjPfivXsfy9YNL585gz/wPn8A03SQVg==</latexit><latexit sha1_base64="D0ousYKRBvlWYFmcQmQNhtQFSy0=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaT9UaVfKvtVfwGyToKclCFHo1/66g0SlklUlglqTDfwUxtOqbacCZwVe5nBlLIxHWLXUUUlmnC6uHhGLp0yIHGiXSlLFurviSmVxkxk5DoltSOz6s3F/7xuZuObcMpVmllUbLkozgSxCZm/TwZcI7Ni4ghlmrtbCRtRTZl1IRVdCMHqy+ukVasGfjW4r5Xr9TyOApzDBVxBANdQhztoQBMYKHiGV3jzjPfivXsfy9YNL585gz/wPn8A03SQVg==</latexit><latexit sha1_base64="D0ousYKRBvlWYFmcQmQNhtQFSy0=">AAAB8XicbVA9SwNBEJ3zM8avqKXNYiJYhbs0WgZsLCOYD0yOsLeZS5bs7h27e0II+Rc2ForY+m/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9gr7h8cHh2XTk5bJsk0wyZLRKI7ETUouMKm5VZgJ9VIZSSwHY1v5377CbXhiXqwkxRDSYeKx5xR66THSs/woaT9UaVfKvtVfwGyToKclCFHo1/66g0SlklUlglqTDfwUxtOqbacCZwVe5nBlLIxHWLXUUUlmnC6uHhGLp0yIHGiXSlLFurviSmVxkxk5DoltSOz6s3F/7xuZuObcMpVmllUbLkozgSxCZm/TwZcI7Ni4ghlmrtbCRtRTZl1IRVdCMHqy+ukVasGfjW4r5Xr9TyOApzDBVxBANdQhztoQBMYKHiGV3jzjPfivXsfy9YNL585gz/wPn8A03SQVg==</latexit>rt <latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit><latexit sha1_base64="oH0n51gWY1w44tvOQOSpqHm4/NY=">AAAB7HicbVBNT8JAEJ3iF+IX6tHLRjDxRFouciTx4hETCybQkO2yhQ3bbbM7NSENv8GLB43x6g/y5r9xgR4UfMkkL+/NZGZemEph0HW/ndLW9s7uXnm/cnB4dHxSPT3rmiTTjPsskYl+DKnhUijuo0DJH1PNaRxK3guntwu/98S1EYl6wFnKg5iOlYgEo2glv66HWB9Wa27DXYJsEq8gNSjQGVa/BqOEZTFXyCQ1pu+5KQY51SiY5PPKIDM8pWxKx7xvqaIxN0G+PHZOrqwyIlGibSkkS/X3RE5jY2ZxaDtjihOz7i3E/7x+hlEryIVKM+SKrRZFmSSYkMXnZCQ0ZyhnllCmhb2VsAnVlKHNp2JD8NZf3iTdZsNzG959s9ZuFXGU4QIu4Ro8uIE23EEHfGAg4Ble4c1Rzovz7nysWktOMXMOf+B8/gAdTI4t</latexit>
ht<latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit>ht 1<latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit> xt<latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit>
ct 1<latexit sha1_base64="qx063OU/We40do9XK5orrzTupSw=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYSLYGO7SaCUBG8sI5kOSI+xt9pIlu3vH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzwkRwg5737aytb2xubRd2irt7+weHpaPjlolTTVmTxiLWnZAYJrhiTeQoWCfRjMhQsHY4vp357SemDY/VA04SFkgyVDzilKCVHiu0n+GlP630S2Wv6s3hrhI/J2XI0eiXvnqDmKaSKaSCGNP1vQSDjGjkVLBpsZcalhA6JkPWtVQRyUyQzQ+euudWGbhRrG0pdOfq74mMSGMmMrSdkuDILHsz8T+vm2J0HWRcJSkyRReLolS4GLuz790B14yimFhCqOb2VpeOiCYUbUZFG4K//PIqadWqvlf172vl+k0eRwFO4QwuwIcrqMMdNKAJFCQ8wyu8Odp5cd6dj0XrmpPPnMAfOJ8/qySPog==</latexit><latexit sha1_base64="qx063OU/We40do9XK5orrzTupSw=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYSLYGO7SaCUBG8sI5kOSI+xt9pIlu3vH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzwkRwg5737aytb2xubRd2irt7+weHpaPjlolTTVmTxiLWnZAYJrhiTeQoWCfRjMhQsHY4vp357SemDY/VA04SFkgyVDzilKCVHiu0n+GlP630S2Wv6s3hrhI/J2XI0eiXvnqDmKaSKaSCGNP1vQSDjGjkVLBpsZcalhA6JkPWtVQRyUyQzQ+euudWGbhRrG0pdOfq74mMSGMmMrSdkuDILHsz8T+vm2J0HWRcJSkyRReLolS4GLuz790B14yimFhCqOb2VpeOiCYUbUZFG4K//PIqadWqvlf172vl+k0eRwFO4QwuwIcrqMMdNKAJFCQ8wyu8Odp5cd6dj0XrmpPPnMAfOJ8/qySPog==</latexit><latexit sha1_base64="qx063OU/We40do9XK5orrzTupSw=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYSLYGO7SaCUBG8sI5kOSI+xt9pIlu3vH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzwkRwg5737aytb2xubRd2irt7+weHpaPjlolTTVmTxiLWnZAYJrhiTeQoWCfRjMhQsHY4vp357SemDY/VA04SFkgyVDzilKCVHiu0n+GlP630S2Wv6s3hrhI/J2XI0eiXvnqDmKaSKaSCGNP1vQSDjGjkVLBpsZcalhA6JkPWtVQRyUyQzQ+euudWGbhRrG0pdOfq74mMSGMmMrSdkuDILHsz8T+vm2J0HWRcJSkyRReLolS4GLuz790B14yimFhCqOb2VpeOiCYUbUZFG4K//PIqadWqvlf172vl+k0eRwFO4QwuwIcrqMMdNKAJFCQ8wyu8Odp5cd6dj0XrmpPPnMAfOJ8/qySPog==</latexit><latexit sha1_base64="qx063OU/We40do9XK5orrzTupSw=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYSLYGO7SaCUBG8sI5kOSI+xt9pIlu3vH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzwkRwg5737aytb2xubRd2irt7+weHpaPjlolTTVmTxiLWnZAYJrhiTeQoWCfRjMhQsHY4vp357SemDY/VA04SFkgyVDzilKCVHiu0n+GlP630S2Wv6s3hrhI/J2XI0eiXvnqDmKaSKaSCGNP1vQSDjGjkVLBpsZcalhA6JkPWtVQRyUyQzQ+euudWGbhRrG0pdOfq74mMSGMmMrSdkuDILHsz8T+vm2J0HWRcJSkyRReLolS4GLuz790B14yimFhCqOb2VpeOiCYUbUZFG4K//PIqadWqvlf172vl+k0eRwFO4QwuwIcrqMMdNKAJFCQ8wyu8Odp5cd6dj0XrmpPPnMAfOJ8/qySPog==</latexit> ct<latexit sha1_base64="pirW76MgnYII8nutbf2Q83ixmBI=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8utsgPVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRhdB7lQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaN0UcZTiDc7gED66gBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz8IMY4k</latexit><latexit sha1_base64="pirW76MgnYII8nutbf2Q83ixmBI=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8utsgPVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRhdB7lQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaN0UcZTiDc7gED66gBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz8IMY4k</latexit><latexit sha1_base64="pirW76MgnYII8nutbf2Q83ixmBI=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8utsgPVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRhdB7lQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaN0UcZTiDc7gED66gBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz8IMY4k</latexit><latexit sha1_base64="pirW76MgnYII8nutbf2Q83ixmBI=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8utsgPVBteY23AXIOvEKUoMC7UH1qz9MWBZzhUxSY3qem2KQU42CST6r9DPDU8omdMR7lioacxPki2Nn5MIqQxIl2pZCslB/T+Q0NmYah7Yzpjg2q95c/M/rZRhdB7lQaYZcseWiKJMEEzL/nAyF5gzl1BLKtLC3EjammjK0+VRsCN7qy+uk02x4bsO7b9ZaN0UcZTiDc7gED66gBXfQBh8YCHiGV3hzlPPivDsfy9aSU8ycwh84nz8IMY4k</latexit>
ht<latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit><latexit sha1_base64="3p2Sf4EV1fRYc738z6OONc502Do=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LLaCp5L0oicpePFYwbRCG8pmu2mXbjZhdyKU0N/gxYMiXv1B3vw3btsctPXBwOO9GWbmhakUBl332yltbG5t75R3K3v7B4dH1eOTjkkyzbjPEpnox5AaLoXiPgqU/DHVnMah5N1wcjv3u09cG5GoB5ymPIjpSIlIMIpW8uvjAdYH1ZrbcBcg68QrSA0KtAfVr/4wYVnMFTJJjel5bopBTjUKJvms0s8MTymb0BHvWapozE2QL46dkQurDEmUaFsKyUL9PZHT2JhpHNrOmOLYrHpz8T+vl2F0HeRCpRlyxZaLokwSTMj8czIUmjOUU0so08LeStiYasrQ5lOxIXirL6+TTrPhuQ3vvllr3RRxlOEMzuESPLiCFtxBG3xgIOAZXuHNUc6L8+58LFtLTjFzCn/gfP4AD9SOKQ==</latexit>ht 1<latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit><latexit sha1_base64="v4ZZ+3n27QM0FDeMDBfxAnlvQnY=">AAAB8HicbVA9SwNBEJ3zM8avqKXNYiLYGO7SaCUBG8sI5kOSI+xt9pIlu3fH7pwQjvwKGwtFbP05dv4bN8kVmvhg4PHeDDPzgkQKg6777aytb2xubRd2irt7+weHpaPjlolTzXiTxTLWnYAaLkXEmyhQ8k6iOVWB5O1gfDvz209cGxFHDzhJuK/oMBKhYBSt9FgZ9TO89KaVfqnsVt05yCrxclKGHI1+6as3iFmqeIRMUmO6npugn1GNgkk+LfZSwxPKxnTIu5ZGVHHjZ/ODp+TcKgMSxtpWhGSu/p7IqDJmogLbqSiOzLI3E//zuimG134moiRFHrHFojCVBGMy+54MhOYM5cQSyrSwtxI2opoytBkVbQje8surpFWrem7Vu6+V6zd5HAU4hTO4AA+uoA530IAmMFDwDK/w5mjnxXl3Phata04+cwJ/4Hz+ALLbj6c=</latexit> xt<latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit><latexit sha1_base64="wC5NfZJuEuzbwi8IACSSOv3lOAc=">AAAB7HicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsarQyJjSUmHpDAhewte7Bhb++yO2ckhN9gY6Extv4gO/+NC1yh4EsmeXlvJjPzwlQKg6777RQ2Nre2d4q7pb39g8Oj8vFJyySZZtxniUx0J6SGS6G4jwIl76Sa0ziUvB2Ob+d++5FrIxL1gJOUBzEdKhEJRtFKfvWpj9V+ueLW3AXIOvFyUoEczX75qzdIWBZzhUxSY7qem2IwpRoFk3xW6mWGp5SN6ZB3LVU05iaYLo6dkQurDEiUaFsKyUL9PTGlsTGTOLSdMcWRWfXm4n9eN8PoOpgKlWbIFVsuijJJMCHzz8lAaM5QTiyhTAt7K2EjqilDm0/JhuCtvrxOWvWa59a8+3qlcZPHUYQzOIdL8OAKGnAHTfCBgYBneIU3RzkvzrvzsWwtOPnMKfyB8/kDKESOOQ==</latexit>
+ <latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit><latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit><latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit><latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit>
+ <latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit><latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit><latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit><latexit sha1_base64="RgQFrgPP09G9wiNkCBnzA2EsptM=">AAAB6nicbVBNS8NAEJ34WetX1aOXxVYQhJL0oseiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/nbX1jc2t7cJOcXdv/+CwdHTcMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2Mb2d++4lrI2L1iJOE+xEdKhEKRtFKD5XLSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjbhCJqkxXc9N0M+oRsEknxZ7qeEJZWM65F1LFY248bP5qVNybpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDK/9TKgkRa7YYlGYSoIxmf1NBkJzhnJiCWVa2FsJG1FNGdp0ijYEb/nlVdKqVT236t3XyvWbPI4CnMIZXIAHV1CHO2hAExgM4Rle4c2Rzovz7nwsWtecfOYE/sD5/AEoUY0J</latexit>
where Wg,Wr,Wo,Wc ∈ Rdh×dx , Ug, Ur, Uo, Uc ∈ Rdh×dh , and gt, rt, ot ∈ Rdh . For simplicity, we also consider σ being the sigmoid function, and σc(·) = tan (·). The t-th output is yt = σy(V ht), where V ∈ Rdy×dh , and σy is ρy-Lipschitz w h σy(0) = 0. Suppose we have h0 = c0 = 0 and the following assumption. Assumption 7. The spectral norms of weight matrices are bounded respectively, i.e. ‖Wg‖2 ≤ BWg , ‖Wr‖2 ≤ BWr , ‖Wo‖2 ≤ BWo , ‖Wc‖2 ≤ BWc , ‖Ug‖2 ≤ BUg , ‖Ur‖2 ≤ BUr , ‖Uo‖2 ≤ BUo , ‖Uh‖2 ≤ BUh , and ‖V ‖2 ≤ BV .
For properly normalized weight matrices Wo and Uo, the generalization bound of LSTM RNNs is given in the following theorem. Theorem 5. Let the activation operator σy be given and Assumptions 1 and 7 hold. Then for (xt, zt) T t=1 and S = { (xi,t, zi,t) T t=1, i = 1, . . . ,m } drawn i.i.d. from any underlying distribution over Rdx×T × {1, . . . ,K}, with probability at least 1− δ over S, for every margin value γ > 0 and every ft ∈ Fg,t for integer t ≤ T , we have
P (z̃t 6= zt) ≤ R̂γ(ft) +O ( dρyBV min {√ d,BWcBx βt−1 β−1 }√ log ( θt−1 θ−1 d √ m )
√ mγ
+ √ log 1δ m ) ,
where d = √ 4dxdh + 4d2h + dhdy , β = maxj≤t { ‖gj‖∞ +BUc ‖rj‖∞ ‖oj‖∞ } and θ = β + BUg +BUr +BUo .
The detailed proof is provided in Appendix C.2. Similar to MGU RNNs, LSTM RNNs also introduce extra decaying factors to reduce the dependence on d and t in generalization. However, LSTM RNNs are more complicated, but more flexible than MGU RNNs, since three factors, rt, ot and gt are used to jointly control the spectrum of Uc. We further remark that LSTM RNNs also need the spectral norms of weight matrices, Wg,Wr,Wo, Ug, Ur, and Uo, to be properly controlled such that better generalization bounds can be obtained. Remark 1. We also extend our analysis to convolutional RNNs (Conv RNNs, Pinheiro and Collobert (2014); Liang and Hu (2015); Xingjian et al. (2015)), and show that the convolutional filters in Conv RNNs can reduce the dependence on d through parameter sharing in generalization. Due to space limit, the detailed discussion is provided in Appendix D.

heading: 6 NUMERICAL EVALUATION
text: We demonstrate a comparison among our obtained generalization bound with Bartlett et al. (2017), Neyshabur et al. (2017), and Zhang et al. (2018). Specifically, we train a vanilla RNN on the wikitext long term dependency language modeling dataset. We take σh = tanh and set the hidden state h ∈ R128 and the input x ∈ [0, 1]14. Accordingly, we have d = 128 and take the sequence length t = 56. We list the complexity bounds for vanilla RNNs in Theorem 2 (Ours), Zhang et al. (2018) (Bound 1), (3) of Theorem 3 (Bound 2), and (4) of Theorem 3 (Bound 3) in the following by neglecting common log factors in d and t,
• Ours: dBV min {√ d,BW BtU−1 BU−1 }√ log ( BtU−1 BU−1 ) ;
• Bound 1: dt2BVBW max{1, BtU};
• Bound 2: BVBW (MU +MV +MW ) tB t U−1
BU−1 ;
• Bound 3: ( min {√ d,BW BtU−1 BU−1 } BU +BW ) BtU−1 BU−1 √ d ( B2U,F +B 2 W,F +B 2 V,F ) .
The corresponding complexity bounds are shown in Figure 4 in the logarithmic scale. As can be seen, our complexity bound in Theorem 2 is much smaller than Bounds 1-3. In more detail, the trained vanilla RNN takes BU = 2.6801 > 1. As discussed earlier, for BU > 1, only our bound in Theorem 2 is polynomial in the size of the network, while Bounds 1-3 are all exponential in t. The resulting complexity bounds corroborate such a conclusion.
We also observe that Bound 3 is smaller than Bound 2. The reason behind is that the weight matrices in the trained vanilla RNN have relatively small Frobenius norms but large (2, 1) norms. For example, we have BU,F = 13.6823 and MU = 154.5439. Then, we can calculate the stable rank BU,FBU = 5.1 < √ d 2 , and MU BU,F = 11.3 ≈ √
d, which implies the singular values of U are not evenly distributed, while the norms of row vectors in U are mostly approximately equal.

heading: 7 DISCUSSIONS AND OPEN PROBLEMS
text: (I) Tighter bounds: Our obtained generalization bounds depend on the spectral norms of weight matrices and the network size (the total number of parameters). Can we exploit other modeling structures to further reduce the dependence on the network size? Or can we find better choices of norms of weight matrices that yield better bounds?
(II) Margin value: Our generalization bounds depend on the margin value of the predictors. As can be seen, a larger margin value yields a better generalization bound. However, establishing a sharp characterization of the margin value is technically very challenging, because of its complicated dependence on the underlying data distribution and the training algorithm.
(III) Implicit bias of SGD: Numerous empirical evidences have already shown that RNNs trained by stochastic gradient descent (SGD) algorithms have superior generalization performance. There have been a few theoretical results showing that SGD tends to yield low complexity models, which can generalize (Neyshabur et al., 2014; 2015; Zhang et al., 2016; Soudry et al., 2017). Can we extend their argument to RNNs? For example, can SGD always yield weight matrices with well controlled spectra? As mentioned, this is crucial to the generalization of MGU and LSTM RNNs, since not well controlled spectra may even hurt generalization.
(IV) Adaptivity to the underlying distribution: The current PAC-Learning framework focuses on the worst case. Taking classification as an example, the theoretical analysis holds even when the input features and labels are completely independent. Therefore, this often yields very pessimistic results. For many real applications, however, data are not obtained adversarially. Some recent empirical evidences suggest that the generalization of neural networks seems very adaptive to the underlying distribution: Easier tasks lead to low complexity neural networks, while harder ones lead to highly complex neural networks. Unfortunately, none of the existing analysis can take the underlying distribution into consideration.

heading: BARTLETT, P. L., FOSTER, D. J. and TELGARSKY, M. J. (2017). Spectrally-normalized margin
text: bounds for neural networks. In Advances in Neural Information Processing Systems.

heading: CHO, K., VAN MERRIËNBOER, B., GULCEHRE, C., BAHDANAU, D., BOUGARES, F., SCHWENK,
text: H. and BENGIO, Y. (2014). Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.

heading: CHOI, E., BAHADORI, M. T., SCHUETZ, A., STEWART, W. F. and SUN, J. (2016a). Doctor ai:
text: Predicting clinical events via recurrent neural networks. In Machine Learning for Healthcare Conference.
CHOI, E., SCHUETZ, A., STEWART, W. F. and SUN, J. (2016b). Using recurrent neural network models for early detection of heart failure onset. Journal of the American Medical Informatics Association, 24 361–370.
DASGUPTA, B. and SONTAG, E. D. (1996). Sample complexity for learning recurrent perceptron mappings. In Advances in Neural Information Processing Systems.

heading: DONAHUE, J., ANNE HENDRICKS, L., GUADARRAMA, S., ROHRBACH, M., VENUGOPALAN,
text: S., SAENKO, K. and DARRELL, T. (2015). Long-term recurrent convolutional networks for visual recognition and description. In Proceedings of the IEEE conference on computer vision and pattern recognition.
GOLOWICH, N., RAKHLIN, A. and SHAMIR, O. (2017). Size-independent sample complexity of neural networks. arXiv preprint arXiv:1712.06541.
GRAVES, A. (2012). Sequence transduction with recurrent neural networks. arXiv preprint arXiv:1211.3711.

heading: GRAVES, A., FERNÁNDEZ, S., GOMEZ, F. and SCHMIDHUBER, J. (2006). Connectionist temporal
text: classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning. ACM.
GRAVES, A., MOHAMED, A.-R. and HINTON, G. (2013). Speech recognition with deep recurrent neural networks. In Acoustics, speech and signal processing (icassp), 2013 ieee international conference on. IEEE.

heading: GREGOR, K., DANIHELKA, I., GRAVES, A., REZENDE, D. J. and WIERSTRA, D. (2015). Draw:
text: A recurrent neural network for image generation. arXiv preprint arXiv:1502.04623.
HAUSSLER, D. (1992). Decision theoretic generalizations of the pac model for neural net and other learning applications. Information and Computation, 100 78–150.
HOCHREITER, S. and SCHMIDHUBER, J. (1997). Long short-term memory. Neural computation, 9 1735–1780.
HUANG, L., LIU, X., LANG, B., YU, A. W. and LI, B. (2017). Orthogonal weight normalization: Solution to optimization over multiple dependent stiefel manifolds in deep neural networks. arXiv preprint arXiv:1709.06079.
JOZEFOWICZ, R., ZAREMBA, W. and SUTSKEVER, I. (2015). An empirical exploration of recurrent network architectures. In International Conference on Machine Learning.
KARPATHY, A. and FEI-FEI, L. (2015). Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition.
KOIRAN, P. (1998). Vapnik-chervonenkis dimension of recurrent neural networks. Discrete Applied Mathematics, 86 63–79.
KRIZHEVSKY, A., SUTSKEVER, I. and HINTON, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems.
KU, C.-C. and LEE, K. Y. (1995). Diagonal recurrent neural networks for dynamic systems control. IEEE transactions on neural networks, 6 144–156.
LE, Q. V., JAITLY, N. and HINTON, G. E. (2015). A simple way to initialize recurrent networks of rectified linear units. arXiv preprint arXiv:1504.00941.
LEE, C.-H. and TENG, C.-C. (2000). Identification and control of dynamic systems using recurrent fuzzy neural networks. IEEE Transactions on fuzzy systems, 8 349–366.
LI, X., LU, J., WANG, Z., HAUPT, J. and ZHAO, T. (2018). On tighter generalization bound for deep neural networks: Cnns, resnets, and beyond. arXiv preprint arXiv:1806.05159.
LIANG, M. and HU, X. (2015). Recurrent convolutional neural network for object recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
LIPTON, Z. C., KALE, D. C., ELKAN, C. and WETZEL, R. (2015). Learning to diagnose with lstm recurrent neural networks. arXiv preprint arXiv:1511.03677.
MIKOLOV, T., KARAFIÁT, M., BURGET, L., ČERNOCKỲ, J. and KHUDANPUR, S. (2010). Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association.
MOHRI, M., ROSTAMIZADEH, A. and TALWALKAR, A. (2012). Foundations of machine learning. MIT press.
NEYSHABUR, B., BHOJANAPALLI, S., MCALLESTER, D. and SREBRO, N. (2017). A pacbayesian approach to spectrally-normalized margin bounds for neural networks. arXiv preprint arXiv:1707.09564.

heading: NEYSHABUR, B., SALAKHUTDINOV, R. R. and SREBRO, N. (2015). Path-sgd: Path-normalized
text: optimization in deep neural networks. In Advances in Neural Information Processing Systems.
NEYSHABUR, B., TOMIOKA, R. and SREBRO, N. (2014). In search of the real inductive bias: On the role of implicit regularization in deep learning. arXiv preprint arXiv:1412.6614.
PINHEIRO, P. H. and COLLOBERT, R. (2014). Recurrent convolutional neural networks for scene labeling. In 31st International Conference on Machine Learning (ICML). EPFL-CONF-199822.
SAXE, A. M., MCCLELLAND, J. L. and GANGULI, S. (2013). Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120.
SOUDRY, D., HOFFER, E. and SREBRO, N. (2017). The implicit bias of gradient descent on separable data. arXiv preprint arXiv:1710.10345.
SUTSKEVER, I., VINYALS, O. and LE, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems.
VORONTSOV, E., TRABELSI, C., KADOURY, S. and PAL, C. (2017). On orthogonality and learning recurrent networks with long term dependencies. arXiv preprint arXiv:1702.00071.
XIE, D., XIONG, J. and PU, S. (2017). All you need is beyond a good init: Exploring better solution for training extremely deep convolutional neural networks with orthonormality and modulation. arXiv preprint arXiv:1703.01827.
XINGJIAN, S., CHEN, Z., WANG, H., YEUNG, D.-Y., WONG, W.-K. and WOO, W.-C. (2015). Convolutional lstm network: A machine learning approach for precipitation nowcasting. In Advances in neural information processing systems.

heading: XU, K., BA, J., KIROS, R., CHO, K., COURVILLE, A., SALAKHUDINOV, R., ZEMEL, R. and
text: BENGIO, Y. (2015). Show, attend and tell: Neural image caption generation with visual attention. In International Conference on Machine Learning.
YOO, S. J., PARK, J. B. and CHOI, Y. H. (2006). Adaptive dynamic surface control of flexiblejoint robots using self-recurrent wavelet neural networks. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 36 1342–1355.

heading: ZHANG, C., BENGIO, S., HARDT, M., RECHT, B. and VINYALS, O. (2016). Understanding deep
text: learning requires rethinking generalization. arXiv preprint arXiv:1611.03530.
ZHANG, J., LEI, Q. and DHILLON, I. S. (2018). Stabilizing gradients for deep neural networks via efficient svd parameterization. arXiv preprint arXiv:1803.09327.
ZHOU, G.-B., WU, J., ZHANG, C.-L. and ZHOU, Z.-H. (2016). Minimal gated unit for recurrent neural networks. International Journal of Automation and Computing, 13 226–234.

heading: A PROOFS IN SECTION 2
text: A.1 LIPSCHITZ CONTINUITY OFM AND `γ We show the Lipschitz continuity of the margin operatorM and the loss function `γ in the following lemma. Lemma 5. The margin operator M is 2-Lipschitz in its first argument with respect to vector Euclidean norm, and `γ is 1γ -Lipschitz.
Proof. Let y, y′ and z be given, then∣∣∣∣M(y, z)−M (y′, z) ∣∣∣∣ = ∣∣∣∣yz − y′z + (maxj 6=z y′j −maxj 6=z yj ) ∣∣∣∣
≤ ∣∣∣∣yz − y′z∣∣∣∣+ ∣∣∣∣maxj 6=z y′j − yj ∣∣∣∣ ≤ 2 ‖y − y′‖∞ ≤ ‖y − y′‖2 .
For function `γ , it is a piecewise linear function. Thus, it is straightforward to see that `γ is 1γ - Lipschitz.

heading: A.2 PROOF OF LEMMA 2
text: Proof. The Lemma is stated with matrix Frobenius norms. However, we can show a tighter bound only involving the spectral norms of weight matrices. Given weight matrices U, V,W and U ′, V ′,W ′, consider the t-th outputs yt and y′t of vanilla RNNs,
‖yt − y′t‖2 = ‖σy(V ht)− σy(V ′h′t)‖2 ≤ ρy ‖V ht − V ′ht + V ′ht − V ′h′t‖2 ≤ ρy (‖(V − V ′)ht‖2 + ‖V ′(ht − h′t)‖2) ≤ ρy (‖ht‖2 ‖V − V ′‖2 +BV ‖ht − h′t‖2) . (5)
We have to bound the norm of ht as in the following lemma. Lemma 6. Under Assumptions 1 to 3, for t ≥ 0, the norm of ht is bounded by
‖ht‖2 ≤ min { b √ d, ρhBWBx (ρhBU ) t − 1
ρhBU − 1
} . (6)
Proof. We prove by induction. Observe that for t ≥ 1, we have ‖ht‖2 = ‖σh(Wxt + Uht−1)‖2
≤ ρh ‖Wxt + Uht−1‖2 ≤ ρh (‖Wxt‖2 + ‖Uht−1‖2) ≤ ρh (BWBx +BU ‖ht−1‖2) . (7)
Applying equation (7) recursively with h0 = 0, we arrive at,
‖ht‖2 ≤ ρhBWBx t−1∑ j=0 (ρhBU ) j = ρhBWBx (ρhBU ) t − 1 ρhBU − 1 ,
We also have ‖ht‖∞ ≤ b. Thus, combining with the above upper bound, we get ‖ht‖2 ≤ min { b √ d, ρhBWBx (ρhBU ) t−1
ρhBU−1
} . Clearly, ‖h0‖2 = 0 satisfies the upper bound.
When ρhBU = 1, the ratio is defined, by L’Hospital’s rule, to be the limit,
lim ρhBU→1
(ρhBU ) t − 1
ρhBU − 1 = t.
With Lemma 6 in hand, we plug the bound (6) into equation (5) and end up with
‖yt − y′t‖2 ≤ ρyρhBWBx (ρhBU ) t − 1 ρhBU − 1 ‖V − V ′‖2 + ρyBV ‖ht − h′t‖2 . (8)
The remaining task is to bound ‖ht − h′t‖2 in terms of the spectral norms of the difference of weight matrices, ‖W −W ′‖2 and ‖U − U ′‖2. Lemma 7. Under Assumptions 1 to 3, for t ≥ 1, the difference of hidden states ht and h′t satisfies
‖ht − h′t‖2 ≤ LW,t ‖W −W ′‖2 + LU,t ‖U − U ′‖2 ,
where LW,t = ρhBx (ρhBU ) t−1 ρhBU−1 and LU,t = ρ 2 hBWBxt (ρhBU ) t 2−1 (ρhBU )−1 .
Proof. Similar to the proof of Lemma 6, we use induction. ‖ht − h′t‖2 = ∥∥σh (Wxt + Uht−1)− σh (W ′xt + U ′h′t−1)∥∥2
≤ ρh ∥∥(W −W ′)xt + Uht−1 − U ′h′t−1∥∥2
≤ ρh ( ‖(W −W ′)xt‖2 + ∥∥Uht−1 − U ′h′t−1∥∥2) ≤ ρh ( Bx ‖W −W ′‖2 +
∥∥Uht−1 − U ′ht−1 + U ′ht−1 − U ′h′t−1∥∥2) ≤ ρhBx ‖W −W ′‖2 + ρh ( ‖ht−1‖2 ‖U − U ′‖2 +BU
∥∥ht−1 − h′t−1∥∥2) . Repeat this derivation recursively, we have
‖ht − h′t‖2 ≤ ρhBx ‖W −W ′‖2 + ρh ‖ht−1‖2 ‖U − U ′‖2 + ρhBU ∥∥ht−1 − h′t−1∥∥2
≤ ρhBx (1 + ρhBU ) ‖W −W ′‖2 + ρh (‖ht−1‖2 + ρhBU ‖ht−2‖2) ‖U − U ′‖2 + (ρhBU ) 2 ∥∥ht−2 − h′t−2∥∥2 ≤ . . . . . . ≤ ρhBx t−1∑ j=0 (ρhBU ) j ‖W −W ′‖2 + ρh t−1∑ j=0 ( (ρhBU ) t−1−j ‖hj‖2 ) ‖U − U ′‖2
+ (ρhBU ) t ‖h0 − h′0‖2
≤ ρhBx (ρhBU ) t − 1 ρhBU − 1 ‖W −W ′‖2 + ρh t−1∑ j=0 ( (ρhBU ) t−1−j ‖hj‖2 ) ‖U − U ′‖2 . (9)
We now plug in the upper bound (6) to calculate the summation involving the Euclidean norms of the hidden state ht.
t−1∑ j=0 (ρhBU ) t−1−j ‖hj‖2 ≤ t−1∑ j=0 (j + 1)(ρhBU ) jρhBWBx ≤ t t−1∑ j=0 (ρhBU ) jρhBWBx
≤ ρhBWBxt (ρhBU ) t − 1 ρhBU − 1 .
Plugging back into equation (9), we have as desired,
‖ht − h′t‖2 ≤ ρhBx (ρhBU ) t − 1 ρhBU − 1 ‖W −W ′‖2 + ρ2hBWBxt (ρhBU ) t − 1 ρhBU − 1 ‖U − U ′‖2 .
Combining equation (8) and Lemma 7, and ‖W‖F ≥ ‖W‖2, we immediately get Lemma 2.

heading: A.3 PROOF OF LEMMA 3
text: Proof. Our goal is to construct a covering C(Ft, , dist(·, ·)), i.e., for any ft ∈ Ft, there exists f̂t ∈ Ft, for any input data (xt)Tt=1, satisfying
sup Xt ∥∥∥ft(Xt)− f̂t(Xt)∥∥∥ 2 ≤ .
Note that f is determined by weight matrices U, V and W . By Lemma 2, we have
sup Xt ∥∥∥f(Xt)− f̂t(Xt)∥∥∥ 2 ≤ LV,t ∥∥∥V − V̂ ∥∥∥ F + LW,t ∥∥∥W − Ŵ∥∥∥ F + LU,t ∥∥∥U − Û∥∥∥ F .
Then it is enough to construct three matrix coverings, C ( U, 3LU,t , ‖·‖F ) , C ( V, 3LV,t , ‖·‖F ) and
C ( W, 3LW,t , ‖·‖F ) . Their Cartesian product gives us the covering C(Ft, , dist(·, ·)). The following lemma gives an upper bound on the covering number of matrices with a bounded Frobenius norm. Lemma 8. Let G = { A ∈ Rd1×d2 : ‖A‖2 ≤ λ } be the set of matrices with bounded spectral norm and > 0 be given. The covering number N (G, , ‖·‖F) is upper bounded by
N (G, , ‖·‖F) ≤ ( 1 + 2 min {√ d1, √ d2 } λ )d1d2 .
Proof. For any matrix A ∈ G, we define a mapping φ : Rd1×d2 7→ Rd1d2 , such that φ(A) = [A>:,1, A > :,2, . . . , A > :,h] >, where A:,i denotes the i-th column of matrix A. Denote the vector space
induced by the mapping φ by V(G) = {φ(A) : A ∈ G}. Note that we have ‖A‖2F = ∑h i=1A > :,iA:,i = ‖φ(A)‖22 and the mapping φ is one-to-one and onto. By definition, the square of Frobenius norm equals the square of sum of singular values and the spectral norm is the largest singular value. Hence, the equivalence of Frobenius norm and spectral norm is given by the following inequalities,
‖A‖2 ≤ ‖A‖F ≤ min {√ d1, √ d2 } ‖A‖2.
Now, we see that if we construct a covering C(V(G), , ‖·‖2), then φ−1C(V(G), , ‖·‖2) ={ φ−1(v) : v ∈ C(V(G), , ‖·‖2) } is a covering of G at scale with respect to the matrix Frobenius norm. Therefore, we get N (G, , ‖·‖F) ≤ N (V(G), , ‖·‖2). As a consequence, it is suffices to upper bound the covering number of V(G). In order to do so, we need another closely related concept, packing number. Definition 2 (Packing). Let G be an arbitrary set and > 0 be given. We say P(G, , ‖·‖) is a packing of G at scale with respect to the norm ‖·‖, if for any two elements A,B ∈ P , we have
‖A−B‖ > . Denote byM(G, , ‖·‖) the maximal cardinality of P(G, , ‖·‖). By the maximality, we can check that N (C, , ‖·‖) ≤ M(C, , ‖·‖). Indeed, let P∗(G, , ‖·‖) be a maximal packing. Suppose there exists A ∈ G such that for any B ∈ P∗(G, , ‖·‖), the inequality ‖A−B‖ > holds. Then we can add A to P∗(G, , ‖·‖), while still keeping it being a packing, which contradicts the maximality of P∗(G, , ‖·‖). Thus, we have N (G, , ‖·‖) ≤M(G, , ‖·‖). Observe that V(G) is contained in an Euclidean ball B(0;R) ∈ Rd1d2 of radius at most
R = max A∈G ‖φ(A)‖2 ≤ min
{√ d1, √ d2 } ‖A‖2 ≤ min {√ d1, √ d2 } λ.
Additionally, the union of Euclidean balls B(v; /2) ⊂ Rd1d2 with radius /2 and center v ∈ P(V(G), , ‖·‖2) is further contained in an Euclidean ball B(0;R ) of slightly enlarged radius R = min {√ d1, √ d2 } λ + /2. Those balls B(v; /2) are disjoint by the definition of packing, thus we have
N (V(C), , ‖·‖2) ≤ P(V(C), , ‖·‖2) ≤ vol(B(0, R )) vol(B(v; /2)) = ( R /2 )d1d2 = ( 1 + 2 min{ √ d1, √ d2}λ )d1d2 ,
where vol(·) denotes the volume.
By Lemma 8, we can directly write out the upper bounds on the covering numbers of weight matrices,
N ( U,
3LU,t , ‖·‖F
) ≤ ( 1 + 6 √ dhBULU,t )d2h ,
N ( V,
3LV,t , ‖·‖F
) ≤ ( 1 + 6 min{ √ dy, √ dh}BV LV,t )dydh , and
N ( W,
3LW,t , ‖·‖F
) ≤ ( 1 + 6 min{ √ dx, √ dh}BWLW,t )dxdh .
Then we immediately have, N (Ft, , dist(·, ·)) ≤ N ( U,
3LU,t , ‖·‖F
) ×N ( V,
3LV,t , ‖·‖F
) ×N ( W,
3LW,t , ‖·‖F
)
≤ ( 1 + 6 √ dhBULU,t )d2h ( 1 + min{6 √ dy, √ dh}BV LV,t )dydh
× ( 1 + 6min{ √ dx, √ dh}BWLW,t )dxdh .
Substituting the coefficients LU,t, LV,t and LW,t from Lemma 2, we get
N (Ft, , dist(·, ·))
≤ 1 + 6√dρyρhBVBWBx (ρhBU )t−1ρhBU−1 2d 2 1 + 6√dρyρ2hBUBVBWBxt (ρhBU )t−1ρhBU−1 d 2
≤ 1 + 6c√dt (ρhBU )t−1ρhBU−1 3d 2 ,
where c = ρyρhBVBWBxmax {1, ρhBU}. For future usage, we also write down for small > 0, such that 6c √ dt (ρhBU ) t−1 ρhBU−1 > 1, the logarithm of covering number satisfies,
logN (Ft, , dist(·, ·)) ≤ 3d2 log
12c√dt (ρhBU )t−1ρhBU−1  .

heading: A.4 PROOF OF LEMMA 4
text: Proof. Define FM,t = {(Xt, zt) 7→ M(ft(Xt), zt) : ft ∈ Ft}. By Lemma 5, we see that M is 2-Lipschitz in its first argument. In order to cover FM,t at scale , it suffices to cover Ft at scale 2 . This immediately gives us the covering number N (FM,t, , ‖·‖∞) ≤ N (Ft, /2, dist(·, ·)). We then give the statement of Dudley’s entropy integral. Lemma 9. Let H be a real-valued function class taking values in [−r, r] for some constant r, and assume that 0 ∈ H. Let S = (s1, . . . , sm) be given points, then
RS(H) ≤ inf α>0 ( 4α√ m + 12 m ∫ 2r√m α √ logN (H, , ‖·‖)d ) .
The proof can be found in Bartlett et al. (2017). Taking H = FM,t, we can easily verify that FM,t takes values in [−r, r] with r = ρyBV ‖ht‖2 ≤ ρyBV min { b √ d, ρhBWBx (ρhBU ) t−1
ρhBU−1
} and
0 ∈ FM. Thus, directly applying Lemma 9 yields the following bound,
RS(FM,t) ≤ inf α>0 ( 4α√ m + 12 m ∫ 2r√m α √ logN (FM,t, , ‖·‖∞)d ) .
We bound the integral as follows,∫ 2r√m α √ logN (FM,t, , ‖·‖∞)d ≤ ∫ 2r√m α √√√√3d2 log(24c√dt (ρhBU )t−1ρhBU−1 ) d
≤ 2r√m √√√√3d2 log(24c√dt (ρhBU )t−1ρhBU−1 α ) .
Picking α = 1√ m is enough to give us an upper bound on RS(FM,t),
RS(FM) ≤ 4
m + 24√ m
√ 3d2r2 log ( 24c √ dmt
(ρhBU )t − 1 ρhBU − 1
) .
Finally, by Talagrand’s lemma (Mohri et al., 2012) and `γ being 1γ -Lipschitz, we have
RS(Fγ,t) ≤ 1
γ RS(FM,t) ≤
4
mγ + 24√ mγ
√ 3d2r2 log ( 24c √ dmt
(ρhBU )t − 1 ρhBU − 1
) .

heading: B PROOF IN SECTION 4

heading: B.1 PROOF OF THEOREM 3
text: Proof. Under additional Assumption 4, we only need to show that, with the additional matrix induced norm bound, we have a refined upper bound on the matrix covering number. The proof relies on the following lemma adapted from Bartlett et al. (2017) Lemma 3.2. Lemma 10. Let G = { A ∈ Rd1×d2 : ‖A‖2,1 ≤ λ } . We have the following matrix covering upper bound
logN (G, , ‖ · ‖2) ≤ λ2
2 log(2d1d2).
The above Lemma is a direct consequence of Lemma 3.2 in Bartlett et al. (2017) with X being identity, a = λ, b = 1, and m = d1, d = d2. We apply the same trick to split the overall covering accuracy into 3 parts, 3LU,t , 3LV,t
, and 3LW,t , corresponding to U, V,W respectively. Then we derive a refined bound on the covering number of Ft:
logN (Ft, , dist(·, ·)) ≤ 9 ( MUL 2 U,t +MV L 2 V,t +MWL 2 W,t ) 2 log(2d2), (10)
where d = max {dx, dy, dh}. Substituting (10) into the Dudley integral as in the proof of Lemma 4 yields
RS(FM,t) ≤ inf α>0 ( 4α√ m + 12 m ∫ 2r√m α √ logN (Ft, /2, ‖·‖∞)d ) .
We bound the integral as follows,∫ 2r√m α √ logN (Ft, /2, ‖·‖∞)d ≤ ∫ 2r√m α 36 √ MUL2U,t +MV L 2 V,t +MWL 2 W,t √ log(2d2)d
= 36 √ MUL2U,t +MV L 2 V,t +MWL 2 W,t √ log(2d2) log 2r √ m
α .
Choosing α = 1√ m yields
RS(FM) ≤ 4
m + 432√ m
√ MUL2U,t +MV L 2 V,t +MWL 2 W,t √ log(2d2) log ( 2m √ d ) .
Finally, substituting the Lipschitz constant LU,t, LV,t, LW,t into the expression, we have
RS(Fγ,t) ≤ 1
γ RS(FM,t) ≤
4
mγ +
432
γ √ m
√ MUL2U,t +MV L 2 V,t +MWL 2 W,t √ log(2d2) log ( 2m √ d )
≤ O αmax{MU ,MV ,MW }tβt−1β−1 γ √ m √ log d log ( m √ d ) .
Combining with Lemma 1 completes the proof.
Under additional Assumption 5, our proof is based on the following result from Lemma 1 in Neyshabur et al. (2017). Lemma 11. Let fα (x) : X → Rd be any predictor with parameter α, and P be any distribution on the parameter that is independent of training data. Then, for any γ, δ > 0, with probability at least 1 − δ over the training set of size m, for any α and any random perturbation β s.t. Pβ [ maxx∈X |fα+β (x)− fα (x)|∞ < γ 4 ] ≥ 12 , we have
R0 (fα)− R̂γ (fα) ≤ 4
√ KL (α+ β‖P) + log ( 6m δ ) m− 1 ,
where KL (α+ β‖P) is KL divergence of distributions α+ β and P .
For convenience, we omit the superscript for sample index. Denote ht (α) and ht (α+ β) as the hidden variables with parameters α and α+ β respectively. Then we provide an upper bound of the gap of hidden layers before and after the perturbation. Denote the parameters α = vec ({W,U, V }) and the perturbation β = vec ({δW, δU, δV }). For any t ∈ {1, 2, . . . , T}, we have ‖ht (α+ β)− ht (α)‖2 (i)
≤ ρh ‖(U + δU)ht−1 (α+ β) + (W + δW )xt − Uht−1 (α)−Wxt‖2 (ii)
≤ ρhBU ‖ht−1 (α+ β)− ht−1 (α)‖2 + δρhBU ‖ht−1 (α+ β)‖2 + δρhBxBW
≤ (ρhBU )t ‖h0 (α+ β)− h0 (α)‖2 + δ t∑ i=1 (ρhBU ) i ∥∥h(t−i) (α+ β)∥∥2 + δρhBxBW t−1∑ i=0 (ρhBU ) i,
(11) By Lemma 6, we have that for any t ≤ T ,
‖ht (α)‖2 ≤ min { b √ p, ρhBxBW (ρhBU ) t − 1
ρhBU − 1
} = λht . (12)
Combining (11), (12), and h(0) = 0, we have∥∥h(t) (α+ β)− h(t) (α)∥∥2 ≤ δλht t∑ i=1 (ρhBU ) i + δρhBxBW t−1∑ i=0 (ρhBU ) i
≤ δ (λhtρhBU + ρhBxBW ) (ρhBU ) t − 1 ρhBU − 1 . (13)
Denote yt (α) and yt (α+ β) as the out with parameters α and α+ β respectively. Then we have∥∥y(t) (α+ β)− y(t) (α)∥∥2 (i)≤ ρy ‖(1 + δ)V ht (α+ β)− V ht (α)‖2 ≤ ρyBV ‖ht (α+ β)− ht (α)‖2 + δρyBV ‖ht (α+ β)‖2 (ii)
≤ δρyBV (λhtρhBU + ρhBxBW ) (ρhBU ) t − 1 ρhBU − 1 + δρyBV λht , (14)
where (i) is from Lipschitz continuity of σy and (ii) is from (12) and (13). Then choosing the prior distribution and the perturbation distribution as N ( 0, σ2I ) , and from the concentration result for the spectral norm bounds, we have
PA∼N (0,σ2Id×d) [‖A‖2 > ξ] ≤ 2p exp ( −ξ2 2dσ2 ) .
This implies with probability at least 1/2, we have max {δBU , δBW , δBV } ≤ σ √ 2d ln (12d).
Taking σ = ( γ/4ρy ( (λhtρhBU + ρhBxBW ) (ρhBU ) t−1 ρhBU−1 + λht )√ 2d ln (12d) )
and combining with (14), with probability at least 1/2, we have
max x∈Xm
‖yt (α+ β)− yt (α)‖2 ≤ ( (λhtρhBU + ρhBxBW ) (ρhBU ) t − 1
ρhBU − 1 + λht
) · σ √
2d ln (12d) ≤ γ 4 .
Finally, we calculate the KL divergence of P and α+ β with respect to this choice of σ,
KL (α+ β‖P) ≤ ‖α‖ 2 2
2σ2
= O ( ρ2y γ2 · ( (λhtρhBU + ρhBxBW ) (ρhBU ) t − 1 ρhBU − 1 + λht )2 d ln (d) ( B2U,F +B 2 W,F +B 2 V,F ))
= O
( ρ2y (λhtρhBU + ρhBxBW ) 2 (βt − 1)2 p ln (p) ( B2U,F +B 2 W,F +B 2 V,F ) γ2 (β − 1)2 ) .
We complete the proof by applying Lemma 11.

heading: C PROOFS IN SECTION 5

heading: C.1 PROOF OF THEOREM 4
text: Proof. We use the same argument from the analysis of vanilla RNNs to investigate the Lipschitz continuity of MGU RNNs. Consider ht and h′t computed by different sets of weight matrices.
‖ht − h′t‖2 = ∥∥∥(1− rt) ht−1 + rt h̃t − (1− r′t) h′t−1 − r′t h̃′t∥∥∥
2 ≤ ∥∥(r′t − rt) h′t−1∥∥2 + ∥∥(1− rt) (ht−1 − h′t−1)∥∥2 + ∥∥∥(rt − r′t) h̃′t∥∥∥2 + ∥∥∥rt (h̃t − h̃′t)∥∥∥2
≤ ‖r′t − rt‖2 ∥∥h′t−1∥∥∞ + ‖1− rt‖∞ ∥∥ht−1 − h′t−1∥∥2 + ‖rt − r′t‖2 ∥∥∥h̃′t∥∥∥∞ + ‖rt‖∞ ∥∥∥h̃t − h̃′t∥∥∥2
Expand the expression of h̃t. Note that rt is nonnegative, and ‖rt‖∞ ≤ 1. Then we have ‖ht‖∞ ≤ 1. Additionally tanh(·) is 1-Lipschitz. Thus we get∥∥∥h̃t − h̃′t∥∥∥ 2 ≤ ∥∥Uh(ht−1 rt) +Whxt − U ′h(h′t−1 r′t)−W ′hxt∥∥2
≤ ∥∥Uh(ht−1 rt)− U ′h(h′t−1 r′t)∥∥2 +Bx ‖Wh −W ′h‖2
≤ ‖Uh − U ′h‖2 ‖ht−1 rt‖2 +BUh ‖rt‖∞ ∥∥ht−1 − h′t−1∥∥2 +BUh ∥∥h′t−1∥∥∞ ‖rt − r′t‖2
+Bx ‖Wh −W ′h‖2 ≤ ‖ht‖2 ‖Uh − U ′h‖2 +BUh ‖rt‖∞ ∥∥ht−1 − h′t−1∥∥2 +BUh ‖rt − r′t‖2 +Bx ‖Wh −W ′h‖2 . We have to expand rt − r′t as follows,
‖rt − r′t‖2 = ∥∥Wrxt + Urht−1 −W ′rxt − U ′rh′t−1∥∥2
≤ Bx ‖Wr −W ′r‖2 +BUr ∥∥ht−1 − h′t−1∥∥2 + ‖h′t−1‖2‖Ur − U ′r‖2.
We also need to bound ‖ht‖2, ‖ht‖2 ≤ ‖1− rt‖∞ ‖ht−1‖2 + ‖rt‖∞ ∥∥∥h̃t∥∥∥
2
≤ ‖1− rt‖∞ ‖ht−1‖2 + ‖rt‖∞ (BWhBx +BUh ‖rt‖∞ ‖ht−1‖2) = ( ‖1− rt‖∞ +BUh ‖rt‖ 2 ∞ ) ‖ht−1‖2 +BWhBx,
≤ max j≤t
{ ‖1− rj‖∞ +BUh ‖rj‖ 2 ∞ } ‖ht−1‖2 +BWhBx.
Applying the above inequality recursively and remember ‖ht‖∞ ≤ 1, we get ‖ht‖2 ≤ min {√ d, β
t−1 β−1 BWhBx } with β = maxj≤t { ‖1− rj‖∞ +BUh ‖rj‖ 2 ∞ } . Put all the above in-
gredients together, we have ‖ht − h′t‖2 ≤ (β + 2BUr +BUrBUh) ∥∥ht−1 − h′t−1∥∥2
+ √ d ‖Uh − U ′h‖2 +Bx ‖Wh −W ′h‖2 + (2 +BUh) √ d ‖Ur − U ′r‖2 + (2Bx +BUhBx) ‖Wr −W ′r‖2 .
Apply the above inequality recursively, denote by θ = β + 2BUr +BUrBUh , we have
‖ht − h′t‖2 ≤ √ d t∑ j=1 θj ‖Uh − U ′h‖2 +Bx t∑ j=1 θj ‖Wh −W ′h‖2
+ ( 2 √ d+BUh √ d ) t∑ j=1 θj ‖Ur − U ′r‖2 + (2Bx +BUhBx) t∑ j=1 θj ‖Wr −W ′r‖2 .
We then derive the Lipschitz continuity of ‖yt‖2,
‖yt − y′t‖2 ≤ ρyBV ‖ht − h′t‖2 + ρy √ d‖V − V ′‖2 ≤ ρyBV √ d θt − 1 θ − 1 ‖Uh − U ′ h‖2 + ρyBVBx θt − 1 θ − 1 ‖Wh −W ′ h‖2 + ρy √ d‖V − V ′‖2
+ ρyBV ( 2 √ d+BUh √ d ) θt − 1 θ − 1 ‖Ur − U ′ r‖2 + ρyBV (2Bx +BUhBx) θt − 1 θ − 1 ‖Wr −W ′ r‖2 .
Following the same argument for proving the generalization bound of vanilla RNNs, we can get the generalization bound for MGU RNNs as
P (z̃t 6= zt) ≤ R̂γ(ft) +O dρyBV min {√ d,BWhBx βt−1 β−1 }√ log ( d √ m θ t−1 θ−1 ) √ mγ + √ log 1δ m  .

heading: C.2 PROOF OF THEOREM 5
text: Proof. We first bound the norm of ht as follows,
‖ht‖2 ≤ ‖ot‖∞‖tanh(ct)‖2 ≤ ‖ot‖∞‖ct‖2 ≤ ‖gt‖∞‖ct−1‖2 + ‖rt‖∞‖c̃t‖2 ≤ ‖gt‖∞‖ct−1‖2 + ‖rt‖∞ (BWcBx +BUc‖ht−1‖2) ≤ ‖gt‖∞‖ct−1‖2 + ‖rt‖∞ (BWcBx +BUc‖ot‖∞‖ct−1‖2) ≤ (‖gt‖∞ + ‖rt‖∞‖ot‖∞BUc) ‖ct−1‖2 +BWcBx.
By applying the above inequality recursively, we have ‖ht‖2 ≤ ‖ct‖2 ≤ BWcBx β t−1 βt−1 , where
β = maxj≤t {‖gj‖∞ + ‖rj‖∞‖oj‖∞BUc}. We also have ‖ht‖2 ≤ √ d. Thus, put together, we
have ‖ht‖2 ≤ min {√ d,BWcBx βt−1 βt−1 } . Next, we investigate the Lipschitz continuity of ht.
‖ht − h′t‖2 ≤ ‖ot tanh(ct)− o′t tanh(c′t)‖2 ≤ ‖ot − o′t‖2‖tanh(ct)‖∞ + ‖o′t‖∞‖ct − c′t‖2
We have to expand ot − o′t, ‖ot − o′t‖2 ≤ Bx‖Wo −W ′o‖2 +BUo‖ht−1 − ht−1‖2 + ‖ht−1‖2‖Uo − U ′o‖2. Note that ‖BUo‖2 is usually small, ot and o′t are close, and we have ‖ht−1−h′t−1‖2 ≤ ‖ot‖∞‖ct−1− c′t−1‖2 ≤ ‖ct−1 − c′t−1‖2. Thus, we can derive
‖ot − o′t‖2 ≤ Bx‖Wo −W ′o‖2 +BUo‖ct−1 − ct−1‖2 + √ d‖Uo − U ′o‖2.
We also expand ct − c′t to get, ‖ct − c′t‖2 ≤ ‖ct−1‖∞‖gt − g′t‖2 + ‖r′t‖∞‖ct−1 − c′t−1‖2 + ‖c̃t‖∞‖rt − r′t‖2 + ‖r′t‖∞‖c̃t − c̃′t‖2. We also have,
‖c̃t − c̃′t‖2 ≤ BUc‖ht−1 − h′t−1‖2 + ‖ht−1‖2‖Uc − U ′c‖2 +Bx‖Wc −W ′c‖2, ‖gt − g′t‖2 ≤ Bx‖Wg −W ′g‖2 +BWg‖ht−1 − h′t−1‖2 + √ d‖Ug − U ′g‖2, and ‖rt − r′t‖2 ≤ Bx‖Wr −W ′r‖2 +BWr‖ht−1 − h′t−1‖2 + √ d‖Ur − U ′r‖2.
Putting together, we get
‖ct − c′t‖2 ≤ Bx ( ‖Wc −W ′c‖2 + ‖Wg −W ′g‖2 + ‖Wr −W ′r‖2 ) + √ d ( ‖Uc − U ′c‖2 + ‖Ug − U ′g‖2 + ‖Ur − U ′r‖2
) + ‖gt‖∞‖ct−1 − c′t−1‖2 + ( ‖rt‖∞BUc +BUg +BUr ) ‖ht−1 − h′t−1‖2
≤ Bx ( ‖Wc −W ′c‖2 + ‖Wg −W ′g‖2 + ‖Wr −W ′r‖2 + (BUc +BUg +BUr )‖Wo −W ′o‖2 ) + √ d ( ‖Uc − U ′c‖2 + ‖Ug − U ′g‖2 + ‖Ur − U ′r‖2 + (BUc +BUg +BUr )‖Uo − U ′o‖2
) + ( ‖ot‖∞‖rt‖∞BUc +BUg +BUr +BUo ) ‖ct−1 − c′t−1‖2.
By induction, we have
‖ct − c′t‖2
≤ Bx θt − 1 θ − 1
( ‖Wc −W ′c‖2 + ‖Wg −W ′g‖2 + ‖Wr −W ′r‖2 + (BUc +BUg +BUr )‖Wo −W ′o‖2 ) + √ d θt − 1 θ − 1 ( ‖Uc − U ′c‖2 + ‖Ug − U ′g‖2 + ‖Ur − U ′r‖2 + (BUc +BUg +BUr )‖Uo − U ′o‖2 ) ,
where θ = β +BUg +BUr +BUo . Now we immediately have ‖ht − h′t‖2 ≤ Bx
θt − 1 θ − 1
( ‖Wc −W ′c‖2 + ‖Wg −W ′g‖2 + ‖Wr −W ′r‖2 + (BUc +BUg +BUr )‖Wo −W ′o‖2 ) + √ d θt − 1 θ − 1 ( ‖Uc − U ′c‖2 + ‖Ug − U ′g‖2 + ‖Ur − U ′r‖2 + (BUc +BUg +BUr )‖Uo − U ′o‖2 ) .
Then the Lipschitz continuity of yt can be written as
‖yt − y′t‖2 ≤ ρyBV ‖ht − h′t‖2 + ρy √ d‖V − V ′‖2.
Following the same argument for proving the generalization bound of vanilla RNNs, we can get the generalization bound for LSTM RNNs as
P (z̃t 6= zt) ≤ R̂γ(ft) +O dρyBV min {√ d,BWcBx βt−1 β−1 }√ log ( d √ m θ t−1 θ−1 ) √ mγ + √ log 1δ m  .

heading: D EXTENSION TO CONVOLUTIONAL RNNS
text: We further extend our analysis to Convolutional RNNs (Conv RNNs). Conv RNNs integrate convolutional filters and recurrent neural networks. Specifically, we consider input x ∈ Rd and k-channel k-dimensional convolutional filters I1, . . . , Ik ∈ Rk followed by an average pooling layer over the k channels for reducing dimensionality. Extensions to convolution with strides and other kinds of average pooling layers (e.g., blockwise pooling) are straightforward.
Here we denote the circulant-like matrix generated by Ii as
Ci =  I>i 0 . . . . . . . . . 0︸ ︷︷ ︸ d−k 0 I>i 0 . . . . . . 0︸ ︷︷ ︸ d−k−1 ... . . . ... 0 . . . . . . 0︸ ︷︷ ︸ d−k−1 I>i 0
0 . . . . . . . . . 0︸ ︷︷ ︸ d−k I>i
 ∈ R(d−k+1)×d,
and write WI = [C>1 , . . . , C > k ] >. We further denote P = 1k [Id−k+1 Id−k+1 · · · Id−k+1]︸ ︷︷ ︸
totally k identity matrices
, where
Id denotes the d-dimensional identity matrix. Define I = [I1, . . . , Ik], and I ∗ x = PWIx. Given a sample (xt, zt)Tt=1, the Conv RNNs compute ht and yt as follows, ht = σh (U ∗ ht−1 +W ∗ xt) , and yt = σy (V ∗ ht) , where ht, xt ∈ Rd, and U ,V,W ∈ Rk×k are matrices with column vectors being k-dimensional convolutional filters. We use zero-padding to ensure the output dimension of convolutional filters matches the input (Krizhevsky et al., 2012). To get yt, we convolve ht with V followed by an average pooling to reduce the dimension to K. Since we aim to show that Conv RNNs reduce the dependence on d in generalization through parameter sharing, we simplify the notations to assume h0 = 0, and impose the following assumption. Extensions to general settings are straightforward. Assumption 8. The activation operators σh and σy are 1-Lipschitz with σh(0) = σy(0) = 0. σh is entrywise bounded by 1. The convolutional filters U , V , andW are orthogonal with normalized columns, i.e., U>U = UU> = 1k Ik,V>V = VV> = 1k Ik, andW>W =WW> = 1k Ik.
We remark that the orthogonality constraints enhance the diversity among convolutional filters (Xie et al., 2017; Huang et al., 2017). Additionally, the normalization factor 1k is to control the spectral norms ofWU , WV , andWW , which prevents the blowup of hidden state. Denote by Fc,t the class of mappings from the first t inputs to the t-th output computed by Conv RNNs. Then the generalization bound is given in the following theorem. Theorem 6. Let activation operators σh and σy be given, and Assumptions 1 and 8 hold. Then for (xt, zt) T t=1 and S = { (xi,t, zi,t) T t=1, i = 1, . . . ,m } drawn i.i.d. from any underlying distribution over Rd×T × {1, . . . ,K}, with probability at least 1− δ over S, for every margin value γ > 0 and every ft ∈ Fc,t for integer t ≤ T , we have
P (z̃t 6= zt) ≤ R̂γ(ft) +O Bxkt√log (dt√m)√ mγ + √ log 1δ m  . The detailed proof is provided in D.1. Similar to the analysis of vanilla RNNs, our proof is based on the Lipschitz continuity of Conv RNNs with respect to its model parameters in the convolutional filters. Specifically, by Assumption 8, the spectral norms of WU , WV , and WW are all bounded by 1. Combining with the inequality, ‖WU‖F ≤ √ d‖U‖F, we have ‖yt − y′t‖2 ≤ LV,t‖V − V ′‖F + LU,t‖U − U ′‖F + LW,t‖W − W ′‖F, where LU,t, LV,t, and LW,t are polynomials in d and t. Additionally, observe that the total number of parameters in a Conv RNN is at most 3k2, which is independent of input dimension d. As a consequence, the generalization bound of Conv RNNs only has a lieanr dependence on k and t.

heading: D.1 PROOF OF THEOREM 6
text: Proof. We first characterize the Lipschitz continuity of ‖yt‖2 with respect to model parameters U , W and V . We have
‖yt − y′t‖2 ≤ ρy‖ht‖2‖WV −WV′‖2 + ρy‖WV‖2‖ht − h′t‖2.
Since ‖ht‖∞ ≤ 1, we have ‖ht‖2 ≤ √ d. Then we expand ht − h′t,
‖ht − h′t‖2 ≤ ρh‖U ′ ∗ ht−1 +W ∗ xt − U ′ ∗ h′t−1 −W ′ ∗ xt‖2 = ρh‖PWUht−1 + PWWxt − PW ′Uh′t−1 − PWWxt‖2 ≤ ρh‖P‖2‖WUht−1 +WWxt −WU ′h′t−1 −WW′xt‖2 ≤ ρh‖P‖2 [ Bx‖WW −WW′‖2 + √ d‖WU −WU ′‖2 + ‖WU‖2‖ht−1 − h′t−1‖2
] Observe that we have by the definition of circulant matrix,
‖WU −WU ′‖22 ≤ ‖WU −WU ′‖2F = (d− k)‖U − U ′‖2F ≤ d‖U − U ′‖2F. The same holds for WW −WW′ and WV −WV′ . We also have ‖P‖2 = 1. The remaining task is to bound the spectral norm of WU and WV . Consider the matrix product W>U WU . We claim that the diagonal elements of W>U WU is bounded by ∑k i=1‖Ui‖22, and the off-diagonal elements are zero. To see this, denote by CUi the circulant like matrix generated by Ui. Then we have WU = [C>U1 , . . . , C > Uk ] >. The diagonal elements of W>U WU are
( W>U WU ) ii = k∑ j=1 ( C>UjCUj ) ii ≤ k∑ i=1 ‖Ui‖22.
By the orthogonality of U , the off-diagonal elements are
( W>U WU ) pq = k∑ j=1 ( C>UjCUj ) pq = k∑ j=1 ( CUj )> :p ( CUj ) :q = 0.
Thus, the spectral norm ‖WU‖2 ≤ √∑k
i=1 ‖Ui‖22 ≤ 1, and ‖WV‖2, ‖WW‖2 ≤ 1 also hold. Then we can derive
‖ht − h′t‖2 ≤ ρhBx √ d‖W −W ′‖F + ρhd‖U − U ′‖F + ρh‖ht−1 − h′t−1‖2.
Apply the above inequality recursively, we get
‖ht − h′t‖2 ≤ ρhBx √ d ρth − 1 ρh − 1 ‖W −W ′‖F + ρhd ρth − 1 ρh − 1 ‖U − U ′‖F
≤ Bx √ dt‖W −W ′‖F + dt‖U − U ′‖F.
Thus, we have the following Lipschitz continuity of ‖yt‖2,
‖yt − y′t‖2 ≤ d‖V − V ′‖F +Bx √ dt‖W −W ′‖F + dt‖U − U ′‖F.
We also bound the norm of ht by induction. Specifically, we have
‖ht‖2 ≤ ρh‖PWUht−1 + PWWxt‖2 ≤ ρh‖WUht−1‖2 + ρh‖WWxt‖2 ≤ ‖ht−1‖2 +Bx.
Applying the above expression recursively, we have ‖ht‖2 ≤ min{ √ d,Bxt} ≤ Bxt. Then following the same argument for proving the generalization bound of vanilla RNNs, we can get the generalization bound for Conv RNNs as
P (z̃t 6= zt) ≤ R̂γ(ft) +O Bxkt√log (dt√m)√ mγ + √ log 1δ m  .

Review summaries:
The paper focuses on the generalization performance of RNNs and its variant in a theoretical perspective. Compared to the previous result (Zhang et al., 2018) for RNNs, this paper refines the generalization bounds for vanilla RNNs in all cases and fills the blank for RNN variants like MGU and LSTM. Specifically, in the work of Zhang et al. (2018), the complexity term quadratically depends on the layer (or say, current sequence length, denoted by t in original paper), making it less instructive. This paper improves it to at most linear dependence and can achieve at logarithmic dependence in some cases, which should be accredited.