Tensor Network Representation for Machine
Learning - Recent Advances and Perspectives

Qibin ZHAO
Tensor Learning Unit
RIKEN AIPAIP Symposium(Mar. 19, 2019)Tensor Learning Unit - Members
Postdoctoral Researchers (2)‣ Ming Hou, Chao Li
Part-timer (2)‣ Longhao Yuan (PhD student), Xuyang Zhao (PhD student)Interns (4)‣ Canada, Japan, China
Visitors (9)‣ Andrzej Cichocki, Toshihisa Tanaka, Jianting Cao‣ Guillaume Rabusseau, Justin Dauwels, Danilo Mandic, Brahim Chibdraa, Cesar F. Caiafa, Jordi Sole Casals

2

Background and Problems

Tensor diagrams of the approach
Neural Nets

Kernel learning

f (x)

= W·

(x)

=

‣ Problems become easier when mapping to higher
dimensional space.

⇡s1 s2 ···sN
Learning(Ms1 Ms2 · · · MsKernel)(x)N‣ Curse of dimensionality, grows exponentially‣ Weights can be exponentially big⇡‣ “kernelization” scales quadratically with training set

UnsL

tensorsize. In the era of big data, this issue is cited as one Rank-1
Supervised
reason why neural nets have overtaken kernel
Learning
methods.
Tensor‣ Low generalization due to representer theorem
X
Perfect Problem form says exact W =↵j (xj )Tensor Networks to
solve

j

prox. can violate this condition

3

Background and Problems

MachineNeural Networks‣ Weight matrix is huge but highly redundant.
rate‣ Low-rank compression: limited compression
ML Overview‣ Computational inefficient due to huge parameters‣ Not applicable for small devices Popular approaches
Neural Nets

Multi-modal deep learning, multi-task
Neuraldeep
Networks
learning
f (x) =
Tensor Networks is a natural
tool to solve these problems

4

2

⇣

M2

1

M1 x

⌘

KernelLearnin
Non-Linear Kernel LearningNeural Network (NN) vs. Tensor Network (TN)
Similarity‣ Assembling simple units (neurons or tensors) into complicatedfunctionsDifference‣ Decision functions in ML vs. wavefunctions in quantum mechanics‣ Nonlinear in NN vs. linear in TN‣ NN do non-linear things to low-dimensional space vs. TN do linear
things in high-dimensional space5(a)What Are Tensor Networks(TNs) ?
ScalarMatrix

VectorJ

A J
IA
powerfultool
todescribe
stronglyentangled
quantum‣I
I A
many-body systems in physics
a

12
(a)

a I

3rd-order tensor
3rd-order diagonal tensor
Decomposeahigh-ordertensorintoacollection
of
low‣
R
I3
I1 A I3
R Λ R
order tensors connected according
to a network
patternR
I1
I2

Introduction and Motivation
I2‣ Tensor network diagram(b)

ScalarVectoraa I

IMatrix
J
I

A

I

I
A J

I3

I1

I1

A

I3

I2

I2

3rd-order diagonal tensor
R
R Λ R
R
R
R

=

J
B

A
I

3rd-order tensor

x

A

A

I

K

B

P
M

K

J

=

K

J

R

R

L

=

b=Ax
I
C=AB
K

I

P

C

I

M
J

L

a i,j,k b k,l,m,p = c i,j,l,m,p

k =1

(b)

x

A
I

J

=

b=Ax
I

Figure 1.5:6(a) Basic building blocks for tensor network diagrams. (b) Tensor nsha1_base64="4Xp/3dBHBfhT8YVNP+RwTz8l4vk=">AAACD3icbVDLSgMxFM3UV62vsS7dBItQN2WmCLosutGNVrEPaMeSSdM2NMkMSUYsw3yEP+BW/8CduPUT/AG/w0w7C209cOFwzr3cw/FDRpV2nC8rt7S8srqWXy9sbG5t79i7xaYKIolJAwcskG0fKcKoIA1NNSPtUBLEfUZa/vg89VsPRCoaiDs9CYnH0VDQAcVIG6lnF7sc6RFGLL5OyleXt/fVo55dcirOFHCRuBkpgQz1nv3d7Qc44kRozJBSHdcJtRcjqSlmJCl0I0VChMdoSDqGCsSJ8uJp9gQeGqUPB4E0IzScqr8vYsSVmnDfbKZJ1byXiv95nUgPTr2YijDSRODZo0HEoA5gWgTsU0mwZhNDEJbUZIV4hCTC2tQ1/0WPeFIwxbjzNSySZrXiOhX35rhUO8sqyoN9cADKwAUnoAYuQB00AAaP4Bm8gFfryXqz3q2P2WrOym72wB9Ynz9zcJwf</latexit>

sha1_base64="4Xp/3dBHBfhT8YVNP+RwTz8l4vk=">AAACD3icbVDLSgMxFM3UV62vsS7dBItQN2WmCLosutGNVrEPaMeSSdM2NMkMSUYsw3yEP+BW/8CduPUT/AG/w0w7C209cOFwzr3cw/FDRpV2nC8rt7S8srqWXy9sbG5t79i7xaYKIolJAwcskG0fKcKoIA1NNSPtUBLEfUZa/vg89VsPRCoaiDs9CYnH0VDQAcVIG6lnF7sc6RFGLL5OyleXt/fVo55dcirOFHCRuBkpgQz1nv3d7Qc44kRozJBSHdcJtRcjqSlmJCl0I0VChMdoSDqGCsSJ8uJp9gQeGqUPB4E0IzScqr8vYsSVmnDfbKZJ1byXiv95nUgPTr2YijDSRODZo0HEoA5gWgTsU0mwZhNDEJbUZIV4hCTC2tQ1/0WPeFIwxbjzNSySZrXiOhX35rhUO8sqyoN9cADKwAUnoAYuQB00AAaP4Bm8gFfryXqz3q2P2WrOym72wB9Ynz9zcJwf</latexit><latexit

sha1_base64="4Xp/3dBHBfhT8YVNP+RwTz8l4vk=">AAACD3icbVDLSgMxFM3UV62vsS7dBItQN2WmCLosutGNVrEPaMeSSdM2NMkMSUYsw3yEP+BW/8CduPUT/AG/w0w7C209cOFwzr3cw/FDRpV2nC8rt7S8srqWXy9sbG5t79i7xaYKIolJAwcskG0fKcKoIA1NNSPtUBLEfUZa/vg89VsPRCoaiDs9CYnH0VDQAcVIG6lnF7sc6RFGLL5OyleXt/fVo55dcirOFHCRuBkpgQz1nv3d7Qc44kRozJBSHdcJtRcjqSlmJCl0I0VChMdoSDqGCsSJ8uJp9gQeGqUPB4E0IzScqr8vYsSVmnDfbKZJ1byXiv95nUgPTr2YijDSRODZo0HEoA5gWgTsU0mwZhNDEJbUZIV4hCTC2tQ1/0WPeFIwxbjzNSySZrXiOhX35rhUO8sqyoN9cADKwAUnoAYuQB00AAaP4Bm8gFfryXqz3q2P2WrOym72wB9Ynz9zcJwf</latexit><latexit

sha1_base64="4Xp/3dBHBfhT8YVNP+RwTz8l4vk=">AAACD3icbVDLSgMxFM3UV62vsS7dBItQN2WmCLosutGNVrEPaMeSSdM2NMkMSUYsw3yEP+BW/8CduPUT/AG/w0w7C209cOFwzr3cw/FDRpV2nC8rt7S8srqWXy9sbG5t79i7xaYKIolJAwcskG0fKcKoIA1NNSPtUBLEfUZa/vg89VsPRCoaiDs9CYnH0VDQAcVIG6lnF7sc6RFGLL5OyleXt/fVo55dcirOFHCRuBkpgQz1nv3d7Qc44kRozJBSHdcJtRcjqSlmJCl0I0VChMdoSDqGCsSJ8uJp9gQeGqUPB4E0IzScqr8vYsSVmnDfbKZJ1byXiv95nUgPTr2YijDSRODZo0HEoA5gWgTsU0mwZhNDEJbUZIV4hCTC2tQ1/0WPeFIwxbjzNSySZrXiOhX35rhUO8sqyoN9cADKwAUnoAYuQB00AAaP4Bm8gFfryXqz3q2P2WrOym72wB9Ynz9zcJwf</latexit><latexit

(1)

...

1

R1

(1)

R1

1

(2)

Gi

R1

G

2

...

R2

In

R2

(2)

N

n

n-1

...

R2

7
...

1

...

R1
2

...

...

N

I2
R2

I2
Gi

R n-1

G (n)
R n-1
Rn

2

Rn

...

<latexit

I1

(2)

G (n)
R n-1
Rn
In
(n)
n

Gi
N

...

R N-1

n

N

N-1

RN

IN

G

...

sha1_base64="xq6SZyclRHkhMFODjB+IR1qIPAE=">AAAB/nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF71IRPOAZA2zk9lkyMzuMtMrhCXgD3jVP/AmXv0Vf8DvcPI4aGJBQ1HVTXdXkEhh0HW/nNzS8srqWn69sLG5tb1T3N2rmzjVjNdYLGPdDKjhUkS8hgIlbyaaUxVI3ggGl2O/8ci1EXF0j8OE+4r2IhEKRtFKd9cPN51iyS27E5BF4s1ICWaodorf7W7MUsUjZJIa0/LcBP2MahRM8lGhnRqeUDagPd6yNKKKGz+bnDoiR1bpkjDWtiIkE/X3REaVMUMV2E5FsW/mvbH4n9dKMTz3MxElKfKITReFqSQYk/HfpCs0ZyiHllCmhb2VsD7VlKFNZ34L9tWoYIPx5mNYJPWTsueWvdvTUuViFlEeDuAQjsGDM6jAFVShBgx68Awv8Oo8OW/Ou/Mxbc05s5l9+APn8wcYAZYG</latexit>

R1
G

...

sha1_base64="xq6SZyclRHkhMFODjB+IR1qIPAE=">AAAB/nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF71IRPOAZA2zk9lkyMzuMtMrhCXgD3jVP/AmXv0Vf8DvcPI4aGJBQ1HVTXdXkEhh0HW/nNzS8srqWn69sLG5tb1T3N2rmzjVjNdYLGPdDKjhUkS8hgIlbyaaUxVI3ggGl2O/8ci1EXF0j8OE+4r2IhEKRtFKd9cPN51iyS27E5BF4s1ICWaodorf7W7MUsUjZJIa0/LcBP2MahRM8lGhnRqeUDagPd6yNKKKGz+bnDoiR1bpkjDWtiIkE/X3REaVMUMV2E5FsW/mvbH4n9dKMTz3MxElKfKITReFqSQYk/HfpCs0ZyiHllCmhb2VsD7VlKFNZ34L9tWoYIPx5mNYJPWTsueWvdvTUuViFlEeDuAQjsGDM6jAFVShBgx68Awv8Oo8OW/Ou/Mxbc05s5l9+APn8wcYAZYG</latexit><latexit

(1)

...

G
1

...

sha1_base64="xq6SZyclRHkhMFODjB+IR1qIPAE=">AAAB/nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF71IRPOAZA2zk9lkyMzuMtMrhCXgD3jVP/AmXv0Vf8DvcPI4aGJBQ1HVTXdXkEhh0HW/nNzS8srqWn69sLG5tb1T3N2rmzjVjNdYLGPdDKjhUkS8hgIlbyaaUxVI3ggGl2O/8ci1EXF0j8OE+4r2IhEKRtFKd9cPN51iyS27E5BF4s1ICWaodorf7W7MUsUjZJIa0/LcBP2MahRM8lGhnRqeUDagPd6yNKKKGz+bnDoiR1bpkjDWtiIkE/X3REaVMUMV2E5FsW/mvbH4n9dKMTz3MxElKfKITReFqSQYk/HfpCs0ZyiHllCmhb2VsD7VlKFNZ34L9tWoYIPx5mNYJPWTsueWvdvTUuViFlEeDuAQjsGDM6jAFVShBgx68Awv8Oo8OW/Ou/Mxbc05s5l9+APn8wcYAZYG</latexit><latexit

Gi

...

2
O(N
IR
)
I
to
represent
data
valuesby‣ Efficient
I
I
I
I
parametersG i
Gi
Gi
Gi
I ...
or optimize
TT/MPS by DMRG
I
I
‣ EfficientI to compute
R
R
R
algorithm
R...
sha1_base64="xq6SZyclRHkhMFODjB+IR1qIPAE=">AAAB/nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF71IRPOAZA2zk9lkyMzuMtMrhCXgD3jVP/AmXv0Vf8DvcPI4aGJBQ1HVTXdXkEhh0HW/nNzS8srqWn69sLG5tb1T3N2rmzjVjNdYLGPdDKjhUkS8hgIlbyaaUxVI3ggGl2O/8ci1EXF0j8OE+4r2IhEKRtFKd9cPN51iyS27E5BF4s1ICWaodorf7W7MUsUjZJIa0/LcBP2MahRM8lGhnRqeUDagPd6yNKKKGz+bnDoiR1bpkjDWtiIkE/X3REaVMUMV2E5FsW/mvbH4n9dKMTz3MxElKfKITReFqSQYk/HfpCs0ZyiHllCmhb2VsD7VlKFNZ34L9tWoYIPx5mNYJPWTsueWvdvTUuViFlEeDuAQjsGDM6jAFVShBgx68Awv8Oo8OW/Ou/Mxbc05s5l9+APn8wcYAZYG</latexit><latexit

G

...

...

I1

<latexit

2.2. Tensor Train network

51

TT/MPS Representation and Properties(a)[V. Oseledets, SIAM J. Sci. Comput., 2011]
R N-1
G

N

(1)

(2)

(n)

(N)

1

2

n

N

(N)

IN

(N )

R N-1

Rn

(b)
TT: tensor
train decomposition; MPS: matrix product state
RN(N)sha1_base64="gSgoEtUkP9pOonaz4QPqjkJYlw4=">AAAB/HicbVDLSgNBEJz1GeMr6tHLYBA8hV0R9Bj04jEB84BkCbOT3mTIzOwy0yuEEH/Aq/6BN/Hqv/gDfoeTZA+aWNBQVHXT3RWlUlj0/S9vbX1jc2u7sFPc3ds/OCwdHTdtkhkODZ7IxLQjZkEKDQ0UKKGdGmAqktCKRnczv/UIxopEP+A4hVCxgRax4AydVG/1SmW/4s9BV0mQkzLJUeuVvrv9hGcKNHLJrO0EforhhBkUXMK02M0spIyP2AA6jmqmwIaT+aFTeu6UPo0T40ojnau/JyZMWTtWketUDId22ZuJ/3mdDOObcCJ0miFovlgUZ5JiQmdf074wwFGOHWHcCHcr5UNmGEeXzfIWHKpp0QUTLMewSpqXlcCvBPWrcvU2j6hATskZuSABuSZVck9qpEE4AfJMXsir9+S9ee/ex6J1zctnTsgfeJ8/0TGVVA==</latexit>

sha1_base64="gSgoEtUkP9pOonaz4QPqjkJYlw4=">AAAB/HicbVDLSgNBEJz1GeMr6tHLYBA8hV0R9Bj04jEB84BkCbOT3mTIzOwy0yuEEH/Aq/6BN/Hqv/gDfoeTZA+aWNBQVHXT3RWlUlj0/S9vbX1jc2u7sFPc3ds/OCwdHTdtkhkODZ7IxLQjZkEKDQ0UKKGdGmAqktCKRnczv/UIxopEP+A4hVCxgRax4AydVG/1SmW/4s9BV0mQkzLJUeuVvrv9hGcKNHLJrO0EforhhBkUXMK02M0spIyP2AA6jmqmwIaT+aFTeu6UPo0T40ojnau/JyZMWTtWketUDId22ZuJ/3mdDOObcCJ0miFovlgUZ5JiQmdf074wwFGOHWHcCHcr5UNmGEeXzfIWHKpp0QUTLMewSpqXlcCvBPWrcvU2j6hATskZuSABuSZVck9qpEE4AfJMXsir9+S9ee/ex6J1zctnTsgfeJ8/0TGVVA==</latexit><latexit

sha1_base64="gSgoEtUkP9pOonaz4QPqjkJYlw4=">AAAB/HicbVDLSgNBEJz1GeMr6tHLYBA8hV0R9Bj04jEB84BkCbOT3mTIzOwy0yuEEH/Aq/6BN/Hqv/gDfoeTZA+aWNBQVHXT3RWlUlj0/S9vbX1jc2u7sFPc3ds/OCwdHTdtkhkODZ7IxLQjZkEKDQ0UKKGdGmAqktCKRnczv/UIxopEP+A4hVCxgRax4AydVG/1SmW/4s9BV0mQkzLJUeuVvrv9hGcKNHLJrO0EforhhBkUXMK02M0spIyP2AA6jmqmwIaT+aFTeu6UPo0T40ojnau/JyZMWTtWketUDId22ZuJ/3mdDOObcCJ0miFovlgUZ5JiQmdf074wwFGOHWHcCHcr5UNmGEeXzfIWHKpp0QUTLMewSpqXlcCvBPWrcvU2j6hATskZuSABuSZVck9qpEE4AfJMXsir9+S9ee/ex6J1zctnTsgfeJ8/0TGVVA==</latexit><latexit

sha1_base64="gSgoEtUkP9pOonaz4QPqjkJYlw4=">AAAB/HicbVDLSgNBEJz1GeMr6tHLYBA8hV0R9Bj04jEB84BkCbOT3mTIzOwy0yuEEH/Aq/6BN/Hqv/gDfoeTZA+aWNBQVHXT3RWlUlj0/S9vbX1jc2u7sFPc3ds/OCwdHTdtkhkODZ7IxLQjZkEKDQ0UKKGdGmAqktCKRnczv/UIxopEP+A4hVCxgRax4AydVG/1SmW/4s9BV0mQkzLJUeuVvrv9hGcKNHLJrO0EforhhBkUXMK02M0spIyP2AA6jmqmwIaT+aFTeu6UPo0T40ojnau/JyZMWTtWketUDId22ZuJ/3mdDOObcCJ0miFovlgUZ5JiQmdf074wwFGOHWHcCHcr5UNmGEeXzfIWHKpp0QUTLMewSpqXlcCvBPWrcvU2j6hATskZuSABuSZVck9qpEE4AfJMXsir9+S9ee/ex6J1zctnTsgfeJ8/0TGVVA==</latexit><latexit

latexit

⇣⇡
2

xj
⌘i

(x) =
2 (x1)

Tensor diagrams of the approach

f (x)

W =

⇡

[ [ [ [[ [[ [ [[ [[1 (x1)‣ TT representation of weight parameter[A. Novikov, NIPS 2015]
⇡

⇡
sj

⌦

is an
Nth-order
tensoruse
algorithm
similar to DMRG to optimize‣ Decision function - W Can= W·
1⌦ 1 (x3) ⌦ . . . ⌦xj · · ·
cos
=
(x)
2
=(x2) xLinearxs12 (jxNs)2 or
s3 vectors4 s⇣5 ⇡ s6⌘
j2
2 (x3) scaling
xj
sin
x2j
2

1 (x2)(x)

Scaling is=

Main approximation(Ms1 Ms2 · · · MsN )s1 s2 ···sN

f (x)

=

matrixproductstate (MPS)

8

4

5

N · N T · m3

6

s1 s2 s3 s4 s5 ⇣ s6 ⌘
⇡
1
1(xN)
feature

[

[

W

sN
sha1_base64="KAHO3Iu4A+QHb5MKwUP/buVJ414=">AAAB/nicbVDLSgNBEOyNrxhfUY9eBoPgKewGQY9BL54konlAsobZyWwyZHZ2mekVQgj4A171D7yJV3/FH/A7nCR70MSChqKqm+6uIJHCoOt+ObmV1bX1jfxmYWt7Z3evuH/QMHGqGa+zWMa6FVDDpVC8jgIlbyWa0yiQvBkMr6Z+85FrI2J1j6OE+xHtKxEKRtFKd5WHm26x5JbdGcgy8TJSggy1bvG704tZGnGFTFJj2p6boD+mGgWTfFLopIYnlA1pn7ctVTTixh/PTp2QE6v0SBhrWwrJTP09MaaRMaMosJ0RxYFZ9Kbif147xfDCHwuVpMgVmy8KU0kwJtO/SU9ozlCOLKFMC3srYQOqKUObzuIWHESTgg3GW4xhmTQqZc8te7dnpeplFlEejuAYTsGDc6jCNdSgDgz68Awv8Oo8OW/Ou/Mxb8052cwh/IHz+QPzHJXv</latexit>

que idea of pixels as "spins" or "qubits"‣ Input:(Hilbert space)‣ Nonlinear mapping by tensor product
Other choices include:2

xj , sin⌘3

sha1_base64="KAHO3Iu4A+QHb5MKwUP/buVJ414=">AAAB/nicbVDLSgNBEOyNrxhfUY9eBoPgKewGQY9BL54konlAsobZyWwyZHZ2mekVQgj4A171D7yJV3/FH/A7nCR70MSChqKqm+6uIJHCoOt+ObmV1bX1jfxmYWt7Z3evuH/QMHGqGa+zWMa6FVDDpVC8jgIlbyWa0yiQvBkMr6Z+85FrI2J1j6OE+xHtKxEKRtFKd5WHm26x5JbdGcgy8TJSggy1bvG704tZGnGFTFJj2p6boD+mGgWTfFLopIYnlA1pn7ctVTTixh/PTp2QE6v0SBhrWwrJTP09MaaRMaMosJ0RxYFZ9Kbif147xfDCHwuVpMgVmy8KU0kwJtO/SU9ozlCOLKFMC3srYQOqKUObzuIWHESTgg3GW4xhmTQqZc8te7dnpeplFlEejuAYTsGDc6jCNdSgDgz68Awv8Oo8OW/Ou/Mxb8052cwh/IHz+QPzHJXv</latexit><latexit

cos

⇣⇡

2

sha1_base64="KAHO3Iu4A+QHb5MKwUP/buVJ414=">AAAB/nicbVDLSgNBEOyNrxhfUY9eBoPgKewGQY9BL54konlAsobZyWwyZHZ2mekVQgj4A171D7yJV3/FH/A7nCR70MSChqKqm+6uIJHCoOt+ObmV1bX1jfxmYWt7Z3evuH/QMHGqGa+zWMa6FVDDpVC8jgIlbyWa0yiQvBkMr6Z+85FrI2J1j6OE+xHtKxEKRtFKd5WHm26x5JbdGcgy8TJSggy1bvG704tZGnGFTFJj2p6boD+mGgWTfFLopIYnlA1pn7ctVTTixh/PTp2QE6v0SBhrWwrJTP09MaaRMaMosJ0RxYFZ9Kbif147xfDCHwuVpMgVmy8KU0kwJtO/SU9ozlCOLKFMC3srYQOqKUObzuIWHESTgg3GW4xhmTQqZc8te7dnpeplFlEejuAYTsGDc6jCNdSgDgz68Awv8Oo8OW/Ou/Mxb8052cwh/IHz+QPzHJXv</latexit><latexits sss s s& sKernel
TNs for Weight Compression
Learning
1

sha1_base64="KAHO3Iu4A+QHb5MKwUP/buVJ414=">AAAB/nicbVDLSgNBEOyNrxhfUY9eBoPgKewGQY9BL54konlAsobZyWwyZHZ2mekVQgj4A171D7yJV3/FH/A7nCR70MSChqKqm+6uIJHCoOt+ObmV1bX1jfxmYWt7Z3evuH/QMHGqGa+zWMa6FVDDpVC8jgIlbyWa0yiQvBkMr6Z+85FrI2J1j6OE+xHtKxEKRtFKd5WHm26x5JbdGcgy8TJSggy1bvG704tZGnGFTFJj2p6boD+mGgWTfFLopIYnlA1pn7ctVTTixh/PTp2QE6v0SBhrWwrJTP09MaaRMaMosJ0RxYFZ9Kbif147xfDCHwuVpMgVmy8KU0kwJtO/SU9ozlCOLKFMC3srYQOqKUObzuIWHESTgg3GW4xhmTQqZc8te7dnpeplFlEejuAYTsGDc6jCNdSgDgz68Awv8Oo8OW/Ou/Mxb8052cwh/IHz+QPzHJXv</latexit><latexit

x = input<latexith<mple, following local feature map

More detailed notation
Tensor diagram notationN

xj 2 [0, 1]

···
x
(x) =
x3 , inputs. . . [E.
, Stoudenmire,
xN ]
NIPS
raw2016]
inputsx = [x1 , x2 , x3 , . . . , xxN=] [x1 , x2 , rawsNN2
featureSpace
vectorN = size of input(x) NT = size of training se
m = MPS bond dimen(x)order-N tensor

W
(x)

x)
sha1_base64="dX9Tt5f7DNWc2/FaYMSrWmnKuNk=">AAACAHicbVDLSgNBEOyNrxhfUY9eBoPgKewGQY9BL54kQjYJJGuYncwmQ2Zml5lZISy5+ANe9Q+8iVf/xB/wO5wke9DEgoaiqpvurjDhTBvX/XIKa+sbm1vF7dLO7t7+QfnwqKXjVBHqk5jHqhNiTTmT1DfMcNpJFMUi5LQdjm9mfvuRKs1i2TSThAYCDyWLGMHGSv5dv/lQ65crbtWdA60SLycVyNHol797g5ikgkpDONa667mJCTKsDCOcTku9VNMEkzEe0q6lEguqg2x+7BSdWWWAoljZkgbN1d8TGRZaT0RoOwU2I73szcT/vG5qoqsgYzJJDZVksShKOTIxmn2OBkxRYvjEEkwUs7ciMsIKE2PzWd5iRmJassF4yzGsklat6rlV7/6iUr/OIyrCCZzCOXhwCXW4hQb4QIDBM7zAq/PkvDnvzseiteDkM8fwB87nD10VlrY=</latexit>

sha1_base64="dX9Tt5f7DNWc2/FaYMSrWmnKuNk=">AAACAHicbVDLSgNBEOyNrxhfUY9eBoPgKewGQY9BL54kQjYJJGuYncwmQ2Zml5lZISy5+ANe9Q+8iVf/xB/wO5wke9DEgoaiqpvurjDhTBvX/XIKa+sbm1vF7dLO7t7+QfnwqKXjVBHqk5jHqhNiTTmT1DfMcNpJFMUi5LQdjm9mfvuRKs1i2TSThAYCDyWLGMHGSv5dv/lQ65crbtWdA60SLycVyNHol797g5ikgkpDONa667mJCTKsDCOcTku9VNMEkzEe0q6lEguqg2x+7BSdWWWAoljZkgbN1d8TGRZaT0RoOwU2I73szcT/vG5qoqsgYzJJDZVksShKOTIxmn2OBkxRYvjEEkwUs7ciMsIKE2PzWd5iRmJassF4yzGsklat6rlV7/6iUr/OIyrCCZzCOXhwCXW4hQb4QIDBM7zAq/PkvDnvzseiteDkM8fwB87nD10VlrY=</latexit><latexit

sha1_base64="dX9Tt5f7DNWc2/FaYMSrWmnKuNk=">AAACAHicbVDLSgNBEOyNrxhfUY9eBoPgKewGQY9BL54kQjYJJGuYncwmQ2Zml5lZISy5+ANe9Q+8iVf/xB/wO5wke9DEgoaiqpvurjDhTBvX/XIKa+sbm1vF7dLO7t7+QfnwqKXjVBHqk5jHqhNiTTmT1DfMcNpJFMUi5LQdjm9mfvuRKs1i2TSThAYCDyWLGMHGSv5dv/lQ65crbtWdA60SLycVyNHol797g5ikgkpDONa667mJCTKsDCOcTku9VNMEkzEe0q6lEguqg2x+7BSdWWWAoljZkgbN1d8TGRZaT0RoOwU2I73szcT/vG5qoqsgYzJJDZVksShKOTIxmn2OBkxRYvjEEkwUs7ciMsIKE2PzWd5iRmJassF4yzGsklat6rlV7/6iUr/OIyrCCZzCOXhwCXW4hQb4QIDBM7zAq/PkvDnvzseiteDkM8fwB87nD10VlrY=</latexit><latexit‣
sha1_base64="dX9Tt5f7DNWc2/FaYMSrWmnKuNk=">AAACAHicbVDLSgNBEOyNrxhfUY9eBoPgKewGQY9BL54kQjYJJGuYncwmQ2Zml5lZISy5+ANe9Q+8iVf/xB/wO5wke9DEgoaiqpvurjDhTBvX/XIKa+sbm1vF7dLO7t7+QfnwqKXjVBHqk5jHqhNiTTmT1DfMcNpJFMUi5LQdjm9mfvuRKs1i2TSThAYCDyWLGMHGSv5dv/lQ65crbtWdA60SLycVyNHol797g5ikgkpDONa667mJCTKsDCOcTku9VNMEkzEe0q6lEguqg2x+7BSdWWWAoljZkgbN1d8TGRZaT0RoOwU2I73szcT/vG5qoqsgYzJJDZVksShKOTIxmn2OBkxRYvjEEkwUs7ciMsIKE2PzWd5iRmJassF4yzGsklat6rlV7/6iUr/OIyrCCZzCOXhwCXW4hQb4QIDBM7zAq/PkvDnvzseiteDkM8fwB87nD10VlrY=</latexit><latexit

sha1_base64="epX9R5sB3B60ED0MpZatSqgu43c=">AAACBXicbVDLTgJBEJz1ifhCPXqZSEzwQnbVRI9EL54QE14RVjI7zMKEmdnNTK8JIZz9Aa/6B96MV7/DH/A7HGAPClbSSaWqO91dQSy4Adf9cpaWV1bX1jMb2c2t7Z3d3N5+3USJpqxGIxHpZkAME1yxGnAQrBlrRmQgWCMYXE/8xiPThkeqCsOY+ZL0FA85JWCl+9tCudypyoezk04u7xbdKfAi8VKSRykqndx3uxvRRDIFVBBjWp4bgz8iGjgVbJxtJ4bFhA5Ij7UsVUQy44+mF4/xsVW6OIy0LQV4qv6eGBFpzFAGtlMS6Jt5byL+57USCC/9EVdxAkzR2aIwERgiPHkfd7lmFMTQEkI1t7di2ieaULAhzW+BvhxnbTDefAyLpH5a9Nyid3eeL12lEWXQITpCBeShC1RCN6iCaogihZ7RC3p1npw35935mLUuOenMAfoD5/MHSvmYRA==</latexit>

sha1_base64="epX9R5sB3B60ED0MpZatSqgu43c=">AAACBXicbVDLTgJBEJz1ifhCPXqZSEzwQnbVRI9EL54QE14RVjI7zMKEmdnNTK8JIZz9Aa/6B96MV7/DH/A7HGAPClbSSaWqO91dQSy4Adf9cpaWV1bX1jMb2c2t7Z3d3N5+3USJpqxGIxHpZkAME1yxGnAQrBlrRmQgWCMYXE/8xiPThkeqCsOY+ZL0FA85JWCl+9tCudypyoezk04u7xbdKfAi8VKSRykqndx3uxvRRDIFVBBjWp4bgz8iGjgVbJxtJ4bFhA5Ij7UsVUQy44+mF4/xsVW6OIy0LQV4qv6eGBFpzFAGtlMS6Jt5byL+57USCC/9EVdxAkzR2aIwERgiPHkfd7lmFMTQEkI1t7di2ieaULAhzW+BvhxnbTDefAyLpH5a9Nyid3eeL12lEWXQITpCBeShC1RCN6iCaogihZ7RC3p1npw35935mLUuOenMAfoD5/MHSvmYRA==</latexit><latexit

sha1_base64="epX9R5sB3B60ED0MpZatSqgu43c=">AAACBXicbVDLTgJBEJz1ifhCPXqZSEzwQnbVRI9EL54QE14RVjI7zMKEmdnNTK8JIZz9Aa/6B96MV7/DH/A7HGAPClbSSaWqO91dQSy4Adf9cpaWV1bX1jMb2c2t7Z3d3N5+3USJpqxGIxHpZkAME1yxGnAQrBlrRmQgWCMYXE/8xiPThkeqCsOY+ZL0FA85JWCl+9tCudypyoezk04u7xbdKfAi8VKSRykqndx3uxvRRDIFVBBjWp4bgz8iGjgVbJxtJ4bFhA5Ij7UsVUQy44+mF4/xsVW6OIy0LQV4qv6eGBFpzFAGtlMS6Jt5byL+57USCC/9EVdxAkzR2aIwERgiPHkfd7lmFMTQEkI1t7di2ieaULAhzW+BvhxnbTDefAyLpH5a9Nyid3eeL12lEWXQITpCBeShC1RCN6iCaogihZ7RC3p1npw35935mLUuOenMAfoD5/MHSvmYRA==</latexit><latexit

sha1_base64="epX9R5sB3B60ED0MpZatSqgu43c=">AAACBXicbVDLTgJBEJz1ifhCPXqZSEzwQnbVRI9EL54QE14RVjI7zMKEmdnNTK8JIZz9Aa/6B96MV7/DH/A7HGAPClbSSaWqO91dQSy4Adf9cpaWV1bX1jMb2c2t7Z3d3N5+3USJpqxGIxHpZkAME1yxGnAQrBlrRmQgWCMYXE/8xiPThkeqCsOY+ZL0FA85JWCl+9tCudypyoezk04u7xbdKfAi8VKSRykqndx3uxvRRDIFVBBjWp4bgz8iGjgVbJxtJ4bFhA5Ij7UsVUQy44+mF4/xsVW6OIy0LQV4qv6eGBFpzFAGtlMS6Jt5byL+57USCC/9EVdxAkzR2aIwERgiPHkfd7lmFMTQEkI1t7di2ieaULAhzW+BvhxnbTDefAyLpH5a9Nyid3eeL12lEWXQITpCBeShC1RCN6iCaogihZ7RC3p1npw35935mLUuOenMAfoD5/MHSvmYRA==</latexit><latexit<latexit‣<latexit

sha1_base64="ZzJKbG4SbA3w1kA8LmututhP7u4=">AAAB/nicbVDLSgNBEOz1GeMr6tHLYhA8hV0R9Bj04kki5gXJEmYns8mQmdllplcIS8Af8Kp/4E28+iv+gN/hJNmDJhY0FFXddHeFieAGPe/LWVldW9/YLGwVt3d29/ZLB4dNE6easgaNRazbITFMcMUayFGwdqIZkaFgrXB0M/Vbj0wbHqs6jhMWSDJQPOKUoJUe7nr1XqnsVbwZ3GXi56QMOWq90ne3H9NUMoVUEGM6vpdgkBGNnAo2KXZTwxJCR2TAOpYqIpkJstmpE/fUKn03irUthe5M/T2REWnMWIa2UxIcmkVvKv7ndVKMroKMqyRFpuh8UZQKF2N3+rfb55pRFGNLCNXc3urSIdGEok1ncQsO5aRog/EXY1gmzfOK71X8+4ty9TqPqADHcAJn4MMlVOEWatAACgN4hhd4dZ6cN+fd+Zi3rjj5zBH8gfP5AysslhI=</latexit>

sha1_base64="ZzJKbG4SbA3w1kA8LmututhP7u4=">AAAB/nicbVDLSgNBEOz1GeMr6tHLYhA8hV0R9Bj04kki5gXJEmYns8mQmdllplcIS8Af8Kp/4E28+iv+gN/hJNmDJhY0FFXddHeFieAGPe/LWVldW9/YLGwVt3d29/ZLB4dNE6easgaNRazbITFMcMUayFGwdqIZkaFgrXB0M/Vbj0wbHqs6jhMWSDJQPOKUoJUe7nr1XqnsVbwZ3GXi56QMOWq90ne3H9NUMoVUEGM6vpdgkBGNnAo2KXZTwxJCR2TAOpYqIpkJstmpE/fUKn03irUthe5M/T2REWnMWIa2UxIcmkVvKv7ndVKMroKMqyRFpuh8UZQKF2N3+rfb55pRFGNLCNXc3urSIdGEok1ncQsO5aRog/EXY1gmzfOK71X8+4ty9TqPqADHcAJn4MMlVOEWatAACgN4hhd4dZ6cN+fd+Zi3rjj5zBH8gfP5AysslhI=</latexit><latexit

sha1_base64="ZzJKbG4SbA3w1kA8LmututhP7u4=">AAAB/nicbVDLSgNBEOz1GeMr6tHLYhA8hV0R9Bj04kki5gXJEmYns8mQmdllplcIS8Af8Kp/4E28+iv+gN/hJNmDJhY0FFXddHeFieAGPe/LWVldW9/YLGwVt3d29/ZLB4dNE6easgaNRazbITFMcMUayFGwdqIZkaFgrXB0M/Vbj0wbHqs6jhMWSDJQPOKUoJUe7nr1XqnsVbwZ3GXi56QMOWq90ne3H9NUMoVUEGM6vpdgkBGNnAo2KXZTwxJCR2TAOpYqIpkJstmpE/fUKn03irUthe5M/T2REWnMWIa2UxIcmkVvKv7ndVKMroKMqyRFpuh8UZQKF2N3+rfb55pRFGNLCNXc3urSIdGEok1ncQsO5aRog/EXY1gmzfOK71X8+4ty9TqPqADHcAJn4MMlVOEWatAACgN4hhd4dZ6cN+fd+Zi3rjj5zBH8gfP5AysslhI=</latexit><latexit

sha1_base64="ZzJKbG4SbA3w1kA8LmututhP7u4=">AAAB/nicbVDLSgNBEOz1GeMr6tHLYhA8hV0R9Bj04kki5gXJEmYns8mQmdllplcIS8Af8Kp/4E28+iv+gN/hJNmDJhY0FFXddHeFieAGPe/LWVldW9/YLGwVt3d29/ZLB4dNE6easgaNRazbITFMcMUayFGwdqIZkaFgrXB0M/Vbj0wbHqs6jhMWSDJQPOKUoJUe7nr1XqnsVbwZ3GXi56QMOWq90ne3H9NUMoVUEGM6vpdgkBGNnAo2KXZTwxJCR2TAOpYqIpkJstmpE/fUKn03irUthe5M/T2REWnMWIa2UxIcmkVvKv7ndVKMroKMqyRFpuh8UZQKF2N3+rfb55pRFGNLCNXc3urSIdGEok1ncQsO5aRog/EXY1gmzfOK71X8+4ty9TqPqADHcAJn4MMlVOEWatAACgN4hhd4dZ6cN+fd+Zi3rjj5zBH8gfP5AysslhI=</latexit><latexit<latexit

TNs for Weight Compression & Kernel Learning
3
O(N
N
m
)
Optimization algorithm scaling:Tnction f (x) = W · (x)`=`Adaptivelearning
ns ‣overpossiblelabels`f ` (x){W
9(x)[E. Stoudenmire, NIPS 2016]

m: TT rank, NT : Sample size

2
Without “kernel trick”, avoiding NT scaling problem

2. Feature sharing
for multi-class function
extensionof model‣ Feature sharing‣ Without deep layers transformation

boundary pixels not`=
1. Tensor networks are adaptive

Progressively learn shared features`grayscaleTheoretical Analysis of ConvNets

Expressiveness of Convolutional Networks – Questions

Expressiveness of Bias
Convolutional– Questions
Inductiveof Networks
Convolution/Pooling
Geometry

Inductive
Bias ofemploy
Convolution/Pooling
ConvNets typicallysquare conv/pool windows Geometry

Fundamental theoretical questions:

ConvNets typically employ square conv/pool windows

Recently, dilated windows have also become popularAre
deep
networks
efficientw.r.t.
shallowone
for‣
Recently, dilated windows have also become popular
ConvNets?
Expressiveness of Convolutional Networks – Questions‣

EfficiencyofConnectivity
Schemes
What kind of func can different network arch

Expressiveness of Convolutional Networks – Questions

Q: What is the inductive bias of conv/pool window geometry?

represent?

Q: Can the geometries
be tailored for a given task?
Efficiency of Connectivity
Schemes

Nearly all state of the art ConvNets employ elaborate connectivity sche
Nadav Cohen (Hebrew U.)

Expressiveness of Convolutional Networks

Science of Intelligence, 2017

Q: What is the inductive bias of conv/pool window geometry?

Nearly all state of the art ConvNets employ elaborate connectivity‣ What is the inductive bias of conv/pool window

Nadav Cohen (Hebrew U.)

geometry?

‣ Do overlapping operations introduce efficiency?
justifiedin terms of‣ Can connectivity scheme beInception(GoogLeNet)

Inception (GoogLeNet)

efficiency?

Expressiveness of Convolutional Networks

ResNet ResNet

DenseNet

10

DenseNet

Science of Intellige

h1

h2

h3

h4

A(2)

A(3)

A(4)

A(5)

involve exponentially large resources, and it cannot be used
as a guiding principle to solve practical physical or industrial
problems.
In fact, the quest for more expressive wave function is central to quantum many-body physics. An ideal parametrizationof wave function should accurately describe a quantum statewith exponentially large degree of freedoms with polynomial
resources. Tensor network states (TNS) [25] are promising
candidates to meet this demand. Figure 1(b) shows one of
the simplest TNS, the matrix product state (MPS) [26], as an
example. The MPS parametrizes a wave function of nv physicalConvolutional Arithmetic Circuits
variables as!
Convolutional
Circuits(v) = TrA(i) [v(3)!MPSArithmetici ],Relations Between TNs and DNNs
(b)

A(1)

A(6)

‣ Equivalence of Restricted Boltzmann Machines and Tensor Networks(c)

RBM
Convolutional Arithmetic Circuits

Sec IISec IIITNS

Example 2: Deep Network Ωæ HT Decomposition[Chen et al, Physical Review B, 2018][Carleo et al, Science, 2017]iΩæ Hierarchical Tensor
Decompositions

FIG. 1. (a) Graphical representation of the RBM defined by
Published as a conference
at ICLRtensor
2018
where A(i) paper
is a three-index
represented by a red dot in
Eq. (2). The blue dots represent visible units v and the magenta dots
represent the hidden units h. They are coupled through the connections
Fig. 1(b). For a given value of vi , which is represented by aObservation
indicatedby thepooling:solid lines. (b) MPS defined by Eq. (3). Each red dot
Deep network withsize-2
dangling vertical bond, A(i) [vi ] is a matrix. The dimension ofdenotes a three-index tensor A(i) . Throughout this paper, we use dots
this matrix
is commonly
called theby
virtualbond dimension
of by hierarchical tensor
Gridtensorsof func realized
ConvACs
are given
hidden layer 0
hidden layer L-1
to represent units
RBM, and ball to represent tensors. Lines in thethe MPS, indicated by the thickness of the horizontal bonds
input X
representation
1x1of
conv
(L=log2N)
pooling
decompositions:
RBM denote connection weights while lines in the tensor network
in Fig. 1(b). Connecting these horizontal bonds is to take
dense
conv
denote tensor indices. (c) RBM and TNS are two1x1
ways
toparametrize
pooling
Our main theoreticalresultsare relatedcomparison
the expressive power of th
tensorcontraction
over all tothe avirtualdegree of of
freedoms.
(output)
xi
network
structuredecomposition
type
multivariable functions. With unlimited resources (number of hidden
By
increasing
thethatbondwedimension,
MPS
can represent
with that we are
networks.
Namely,thequestionask
isasfollows.
Supposegiven a TΩær0
r0
rL1 any functionrL1
Yunits or bonddimensions)both of themcan representM(depth,width,
poolingetc)(dim tree,internal ranks etc)
increasing
accuracy
anycomplexmultivariable
functions[26].
complexwould be a CP- or HT-Network realizing the same score function? A natu
arbitrary
However,with limited
theypoolL1 resources
J How
j ', J
conv0 j , Jaccuracy.
a0, j ,J , repj ,:rep i, dfT xto conv
L 1 representi
MPS representation is equivalent to the tensor train decomj '^1,2`
Published as a conference
paper at ICLR
2018
two independent
sets
with
overlapping
region.
Detailed
discussions
L ,1, yofcomplexity,in
thiscase,innetwork
would
be
the rankof thecorresponding
decompositio
pool
j
,
J
conv
j
',
J

0
0
weightsΩæ
decomposition
parametersout ya , poolL 1 :positions
the appliedmathcommunity[27].
Similarly,
one tensor
j '^2 j 1,2 j`on their relationship are given in Secs. II and III.
transitioning between tensor decompositions and deep learning vocabulary easier, we in‣ Equivalence of Deep Convolutional Network and Hierarchical Tucker
Decomposition[N. Cohen & A. Shashua, ICML 2016]d

can connect higher-order tensors to represent a physical statecorresponds to Hierarchical Tucker (HT) decomposition:
following table. in a two-dimensional network. This kind of TNS is named
One can train
anrRBMby specifying its parameters such thatprojected entangled pair state (PEPS) [28]. A generalization
ÿ
0
1,j,“
1,j,“
0,2j≠1,–0,2j,–
We canstudythenetworks
throughdecompositions!
3. Combining theresults,
powerthatof recurrent
theprobability
distribution
visibletheunits
reproducesof PEPS
to include
entanglementof all corresponding
particles in a
„ obtained=and known
a– ofwe· theacompare¢aexpressive
Table
1:
Correspondence
betweenlanguagesofTensorAnalysisand Deep Learn–=1 (CP) networks with each other (see Table 2).
(TT), convolutional
andshallowof the (HT),
input
data
[12,13].
The
hiddenunits
ofatrainedRBM
larger
unit
cell
is
call
projected
entangled
simplex
state(PESS)···
mayalso
revealÿ
correlationsin thedata with appealing
physical[29]. In the past decades, solid theoretical understanding and
rl≠1
l,j,“
l,j,“
l≠1,2j≠1,–l≠1,2j,–
„
=
a
·
„
¢
„
meanings. For example,
efficient numerical techniques for TNS have been established.
–in an RBM trained with a data set of–=1
Decompositions
Deep
images containing
handwritten digits, the connection weight
SeeTensor[25,30] forpedagogical reviews on TNS. Moreover,
the Learning
···
l
(X)
zd 1
ÿrL≠1z2
contains
information
about
the
pen stokesy [14]. These
applicationof TNS to classical systems also has a long history
CP-decomposition
shallow network
GA1y z1theG3= G2
a–L,1,y· „L≠1,1,– ¢G„dL≠1,2,–
learning features can(seeNadav[31–33] forexample).
–=1be used either for discriminative tasks,
Cohen(Hebrew
U.)
Expressiveness of Convolutional Networks
ScienceTT-decomposition
RNNof Intelligence, 2017 21 /suchf aspatternrecognition,orforgenerative
tasks,
suchasThe
physics
community
also
has
an
answertothe“unreaf✓ (x2 )f✓ (x3 )f✓ (xd )✓ (x1 )
HT-decomposition
CNN
generating
moresamplesto the learned
Nadav Cohen (Hebrew
U.)
Expressivenessof according
Convolutional Networks
Sciencedistribution.
of Intelligence, 2017 sonable
23 / 44 effectiveness” of TNS. It relies on the entanglementrank
the whichdecomposition
RBM has played an important role in the recent renaissance ofarea
lawof[34],states that the entanglementwidth
entropyof the network
Figure 1: Recurrent-typearchitecturethat corresponds
to the Tensor
Traindecomposition.
deepneurallearning[15,16] becauseof its versatileabilities
in featurescales just linearly with the boundary size separating any two
Gray circles are bilinear maps (for details see Section 4).
extraction and dimensionality reduction of complex data sets.
subsystems. The entanglement entropy [35] is a measure of
Recently, RBM has attracted great attention in the field of
the information content between these two subsystems. Many
quantum
physics.
Carleo and Troyer [17] proposed
physical states of practical interests fulfill this area law [34].
2 D EEP L EARNING
ANDmany-body
T ENSOR N
ETWORKSan RBM inspired variational wave function to study quanIt indicates that the degrees of freedom needed to describe
5equilibrium.
T HEORETICALA NALYSIS
tum themany-bodysystems atbetween
or awayfromDeng
a quantum
state of physical interest is generally much lessIn this section, we review
known connections
tensordecompositions
anddeep learn11
al. [18]
constructed
exact
RBM
representations
than the total degrees of freedom of the whole system. TNS
ing and then show theetnew
connection
betweenTensorTraindecomposition forandseveral
recurrent neural‣ Recurrent Neural Networks and Tensor Train [Khrulkov, ICLR 2018]‣ Powerful tools to study theory behind DNN

Tensor Networks for Large-Scale Optimization Problems
166

TT Networks for Optimization Problems

166

TT Networks for Optimization Problems

T
max
x
Ax
Eigenvalue
problem:LRn‣ TT format of a large vector L.2. Graphical Representation of Fundamental Tensor Networksa

I = I1 I2 … IN
J = J1 J2 … JN

I3

I2

n

J1

}

xT J3

J2

I = I1 I2 …INI1

K= K1K2 …KN
I = I1 I2 … IN

K
I11

I3

I2

K3
K
I22
A I3
J2
J
I3 3
I2
J3
J2

J
I1 J1
I
…
J = J1 J2 JN
J1

X(1)

JN

301

IN

I1

X(n-1)

Rn -1A

I n-1

}

(1)

…K
K= KtoTT-cores needstoredprocessed,
whichof
K3 makes theKnumber
Kand
K2
N
N
1K2be
1
parameters
to scale linearly in the tensor order, N , of a data tensor and
A
J2
J
JN
J operations
ll mathematical
1 thenI1 JareI3 3 only on Ithe low-orderI
I2 performedNnd relatively small size core tensors.
igure
2.20:
Formsisofdefinedtensor train
for ofa vector,a RI , matrix,
The
TT rank
as andecompositions
N 1 -tuple
the form
I J
I J K
R

, and 3rd-order tensor, A

R

x

JN

1

(1)

P n-1

(n-1)

N

IN
JN

X

X

(n-1)

TT

1

}

(n+1)

P 1R n X

(1)

A
I1

}

R

X

In
(n)

A

Pn+1

(n+2)
A(n+1) (n-1) RAn-1

P n+2I n

X

X

X

(n+1)

R n+1

X

n

R n+2

‣ Applicable to large-scale
SVD/PCA/CCA and etc

R n+1

(N)

Pn X

A(n+1)

Pn+1

P n+2

In

A(N)

(N)

RAn

X

(n+1)

R n+1

IN

I n+2

X(n+2)

R n+2

X(N)

X(N)

In

x (n)
Rn

In

_

R n

A( n )

X(n)R

A(n+2)

X(N)
IN

X(n)

A(n)
)
Xn(n-1

R n+2

I n+2

I n+1

Rn -1

Ln

X(n+2)

I n+1

X(n)

n

I1 stored
In In andT-corestoRbe
makesmatricization
the number of
1 IN processed,
is an nthwhichcanonicalofwhere
X neednarameters
linearlyin thetensororder, Nmemory, of a data
tensor and(n)he tensor to
X.scaleSince
the TTrank
determines
requirementsX
llf mathematical
are thenperformed
on the low-ordera tensor train,operations
it has a strong
impacton theonlycomplexity,
i.e., thendrelatively
small size
tensors.
uitabilityof tensor
traincore
representation
for a given raw data tensor.
The
TT
rank
is
definedasanN
1-tupleof linearly
the form
The number of data samples to be stored
scalesin the tensor
nrder, N , and the size, I, and quadratically in the
maximum TT rank
A(n)
rT T
R1 , . . . , RN 1 , Rn Lrank X n , (2.21)
rankTT X
bound, R, that is
RI1 In In 1 IN is an nth canonical matricization of
here
N X n
he tensor
X.
Since
requirements
Rn 1 RO the
N R2TTI , rank
R : determines
max Rn , memory
I : max
In . (2.22)
n In
n
n
(n)
f na 1tensor train, it has a strong impact on the complexity, i.e., X
theuitabilityof ittensor
train representation
for a low-rank
given rawTT
data
tensor.
This
is whyis crucially
important to haveapproxima7
Thenumberof datasamples
to be storedlinearly
the tensor
ions.A
drawbackof theTT format
is thatscales
the ranksof aintensor
train
rder,
N
,
and
the
size,
I,
and
quadraticallyinthemaximumTTrank
7In the worst case scenario the TT ranks can grow up to I N 2 for an N th-orderound, R, that isensor.

X(n+1)

IN(by applying a suitable tensorization).

N 1

(n)

I n+2
(n+2)

Rn

IN

I n+2

I n+1
Rn

R n+2(n)n+1A(n-1)

I n-1

Pn

In

(n+2)

PX
n-1

I n+1

A(n) (1) R 1

X

X(n)

Rn -1

I n-1

X(n)

R n-1

r
R ,...,R, R
rank X, (2.21)
ALS/DMRG
algorithm‣ FastrankTT X

X(n-1)

R1

In

K

N
IP 1

I 1 a RI , matrix,
I n-1
Figure A2.20: Forms of tensor train decompositions for a vector,
I J
I J K
A R
, andI 3rd-orderR
I3 a suitable tensorization).
IA
I2 (by applyingI= I1 I2 …Itensor,
1
N
RN
x

X(1)
I1

‣ TT format of a large} Amatrix A
A

R n

}

IN

A
R1
.2. GraphicalRepresentation of Fundamental Tensor Networks

a

301

xT
I1

n

Rn

Rn x-1( n )

In

In

X(n)

_

R n

Rn

In
x (n)

A( n )
Rn -1

In

Rn

In

(n)
Figure 6.7: The conceptual TT X
network
for the computationof a single extremex (n)Ieigenvalue, l, and the corresponding eigenvector, x
R , for a symmetric matrix
A R I I . The frame matrix maps the TT core into a large vector. The tensor network corresponds to the Rayleigh quotient, where the matrix A and vectors x R I

12Research Scheme‣ Study the fundamental principle of tensor networks‣ Investigate tensor networks for data representation‣ Investigate tensor networks for model representation‣ Explore the potential applications of tensor methods13G (n)
R n-1
Rn

(N)

Fundamental Tensor Network
Model
R N-1

R2

G

In

TT representation
(n)

(2)

Gi

IN
[Zhao
et al, ICLR workshop 2018, ICASSP 2019]2

2.2. Graphical(NRepresentationof Fundamental Tensor Networks)
Gi
Gi
Tensor TrainnN
IN . . .
Powerful but still some limitations
aQ. ZHAO et al. TENSOR RING DECOMPOSITION
In
I

...

‣
‣ TT-ranks of middle coresRare large I = I I … I
1 2

I

...

from (2), Z 1 and Z 2 is multiplied along one dimension indexed by ↵2 , which is thus denoted by a connection together.
.
(i.e., r2J) 3in the graph. It should
J.= J1 J2 … JNwith theJ1 size of thatJmodeJN be
2
noted that Z d is connected to Z 1 by the summation over theindex ↵1 , which corresponding to the trace operation. Fromthe graphical representation and mathematic expression in
I2 that TR IrepresentationIN
I = I1 I2 …IN(1), weI1can easily derive3
is a circularmultilinear products of a sequence of 3rd-order tensors,
resulting in that the sequence can be shifted circularly
which corresponds
KN
K= K1K2 …KNwithout
K3
K1changing theK2result essentially,
to a circular shift of tensor modes. Since our model graphically looks like a ring and its multilinear operations canJ2 thus call itJ3naturally as tensor ring
be circularly
JN
J1 shifted, we
J
I
I
I
I
1
3 denote TR decomposition
decomposition. For2 simplicity, weN
by T = <(Z 1 , Z 2 , . . . , Z d ).

N-1

...

R n-1

I

INsee
3 mode. As we can
1 over the2indices of that
Nsummation

A

Tensor ringR n representation R N‣ Generalized TT without constraints on boundary cores
he tensor train (TT) and tensor chain (TC) deA
TT with sharedcoretensors‣ Sum
C and
PBC,of respectively)
for anN th-orderdata
I) Tensor Train (TT) can be mathematicallydeEfficient computation for multilinear operations‣
G , where (bottom panel)
the Forms
slice of tensor train decompositions for a vector, a R ,G
Figure 2.20:
TensorRing(by applying a suitable tensori
A
R
,
and
3rd-order
tensor,AR
R ‣ Highly expressive
are defined model
as G
G:, i , :
2

N

i2

iN

Rn 1

In

I

I

n

Rn

J

I

n

J

K

nd

rd

n

in

···

Zd

···

rk+1
1. (b) For the Tensor Chain (TC), the enandprocessed,
makes Zthenum
1
2 TT-cores need
N to be storedn1
nk = n1 which
nk
Z1
T
k
d as xi1 ,i2 ,...,iN
tr Gi1 Gi2GiN

i2 , r2

nd

N
g rN

···

r1

parameters to scale linearlytensor order,
N , of a datatens
n2 in the
r2
rk
···
· ·on
·
all mathematical operations are then performedZ 2only
the lowr3, where (bottom
panel) the latn2
1 , iN , rN···
and relatively small size core tensors.
The TTrank is defined
as an N 1 -tuple of the form
14
Fig. 1. A graphical representation of the tensor ring decomposition
n
n

R

R

Tensor Networks for Data Representation
Real data is often high-dimensional‣ Recommender system (user x item x time)‣ Gene expression, remote sensing, fMRI
Real data is often incomplete

Multilinear multitask
learning(high computationcost)‣ Low-rank approximation via convex optimization
Tasks are associatedwith multiple indices, e.g. predict a
problem)‣ Decomposition based approach (model selection
given to di↵erent aspects of a restaurant by di↵erent cri‣ How much structure information can be used?

15

Tasks’ regression vectors are “vertical” fibers of the tens

n

7: 7:
End
End

3.4. Computational (n)Complexity

Computational
Complexity
8:3.4.
Updatecorresponding
GimG(n)bydescent
method.
8:
Updatecorrespondingby gradientdescent
method.
m gradient
n

@G

=

,n

n

m m
i@G
n in

@f

n

n
)
7: End core tensors
2: 2:
Initialization:
G (1)G, (1)G (2)· · ·, ·, ·G· (Nof )approximatedtensorX. X.
Initialization:
core tensors, G, (2), G (Nof approximatedtensor
7:
End
(n)
8:
UpdatecorrespondingGbygradient
descent
method.
m
3: 3:
While
the theoptimization
stopping
condition
is notnot
satisfied
(n)
in
While
optimization
stopping
conditionsatisfied8:
Updatecorresponding
Gim bygradientisdescentmethod.
n
9: sample
End
while
4: 4:
Randomly
oneoneobserved
entry
from
Y. Y.
Randomly
sampleobserved
entry
from
9:
End while
(1)
(2)
10:
Output:(1)G (2), G , · · (N· , )G (N ) .
5: 5:
ForFori=1:N
i=1:N
10: Output: G , G , · · · , G
.
@f @f
>n >n
T
6: 6: Compute= (y=m(ymxm )(GG<nComputexm )(GG)<n. )T .
(n) (n)
im
im
n im
n im
@G

in

2
N
For
tensorR 21⇥···⇥I
numberof observed
entries
M assume, we assume
9: 9:
End
while
N
End
while
For
tensor
X 2XR2I1 ⇥I
withwith
numberof observed
entries
M , weI ⇥I ⇥···⇥I

N

n

@Gim

So
j t) =t
he h
,8
th a
x
e s cci1
amm:imm
···
iN
n
ei
,n
=
@

nd

Tensor Networks for Data Imputationei

3:

Tensor completion based on TT/TR decomposition
(1)
(2)
(N )
10: 10:
Output:G·is· ·, equal,· G· · , G .(NallOutput:
I , I G, · ·G·, (1), I, G, (2)to) . I, and r

2

N N

1

1 2

)
1

increase
by bydata
dimensions,
STTOPT
andandTTSGD
is freefromdimensionality
increasedata
dimensions,
STTOPT
TTSGD
is freefromdimensionality

k=

Table
1: Computational
complexityof TTWOPT,STTOPT,TTSGD
for every
iteration
Table
1: Computational
complexityof TTWOPT,STTOPT,TTSGD
for every
iteration

so

1

nQ

for forevery
iteration
in table
1. 1.
Thoughthethetime
complexity
willwill
exponentiallyevery
iteration
in tableThoughtime
complexity
exponentially
Algorithm
Timecomplexity
Space
complexity
Algorithm
Timecomplexity
Space
complexity
so they
cancandealdeal
withlarge-scale
data.
Besides,
TTSGD
uses
thetheleast
time
so they
withlarge-scale
data.
Besides,
TTSGD
uses
least
time
N

STTOPTSTTOPT

1)

1)

O(I
O(I N
+ rN2 I+Nr 21I)N

1)

O(MO(MIr) Ir)TTSGD
O(r1 N
O(Ir)
TTSGD
O(r N
) 1)
O(Ir)
Table
1: Computational
complexityof TTWOPT,STTOPT,
TTSGD
for forevery
iteration
Table
1: Computational
complexityof TTWOPT,STTOPT,
TTSGD
every
iteration

Approximated entries

Algorithm
complexity
complexity
AlgorithmTime
Time
complexitySpaceSpace
complexity

N 1
N +2 I N2 1
NI N1 I 1
Nr+r I N) 1 )i2 =2i==34O(ri21i=2TTWOPT
1=iTTWOPT
2 3i
i2 2=O(ri2 =4 N) 1 ) O(IO(I
2 =
2

TTSGD
TTSGD

4. 4.
Experiments
ExperimentsO(Mr N r1N) 1 )O(MN 1N) 1 )
O(rO(rO(MIr) Ir)
O(M
O(Ir)
O(Ir)

)

STTOPTSTTOPT(1

High-ordertensorization

O(MO(Mr N 1r)N)N
1 NN 1 I
1)TTWOPTO(r I
TTWOPT
complexity
andandspace
complexity.
complexity
space
complexity. O(r

ik

Te
n

2

!

N N
1 1

2

ik

2
1

G (k

1

for
every
iteration
in table
1. Thoughcomplexity
exponentially
for
every
iteration
in table
1. Thoughthe thetimetime
complexity
will will
exponentially3.4.3.4.
Computational
Complexity
Computational
Complexity
increasebydata
dimensions,
STTOPT
and
TTSGD
is
free
from
dimensionality
I⇥I⇥···⇥I
1
2
N
I⇥I⇥···⇥I
1
2
N STTOPT
andisentries
freeM
from
Forincrease
tensorXby2
R2 Rdimensions,withnumberof observed
entries, we,dimensionality
assumeFortensorXdata
withnumberofTTSGD
observedM
weassume
can
deal
with
large-scale
data.
Besides,
TTSGD
uses
the
least
they
deal
with
large-scale
data.
Besides,
TTSGD
uses
the
timetime
all all
I , Iso, I·so
· ,· ·they,·I·can
is
equal
to
I,
and
r
=
r
=
·
·
·
=
r
=
r.
Accordingequa, I is equal to I, and r = r = · · · = r= r. Accordingleast
equaandspace
complexity.
and
space
complexity.
tiontion
10,complexity
20complexity
and
15,we
list
the
computational
complexityof ourthree
algorithms
10,
20 and
15,we
list
the
computational
complexityof ourthree
algorithms

Observed entries

T· · r· = rN= r.
r. According
1 =According
all I1 , I21, · ·2· , IN isNequal to I, and r1 = 1r2==r2· ·=·=equa-equaN 1
tion
10,
20
and
15,we
list
the
computational
complexity
of
our
three
algorithms
tion 10, 20 and 15,we list the computational complexity of our three algorithms

1

(n

G
ra
Lo
4
r
d
W
ng
ua
L
S
c
e
rit
te
ha
ho 3S arehool ch nin Sc
e
Y
re
h
o
o
g
@G @f
oo
:
f C ol U(n1, in ) =
lo
n
o
o
i
m f A t,,8 ···
2
f
(
E
p
x
R
x ,N
T
1
A
u
u
n
e
i1
i1
I
t
B
t
G
g
K
n
o
···
···
O
e
.
S
s
m
r
ra S a LEN i
iN
iN
TR or
O (N
4
o
d
,
t
c
n
W
ngC
u
(
L
A
S
yi
i
·· I N r N
i
o
a
e
C
c
=
e
rit
tenc n, hean
1··
ho T 3S ar
2
··i
)
e
1
ni
G ot
,·
S
e
)
he
o
N
c
c
)
yi
l
ho ng
an ua
··
r
@f
h
o
e
@
1·
ood ng,N :f C ol U
G(
QN
··i
n
Toe
1, in ) =
k=
l
N
n
.
o
o
f A it,n+mcEh,8 ···f(G
pu u R
x
1
x ,N
(k
n
A
n
i1
i1
I
)
ik B
t
te om K
···
···
O
.
S
n
r S a EN
iN
iN
TRQ 1
O (N
,n
ci tio C
k= A
(
r
y
N
· · I Nx
e
C
=
G
i1
1
!
n, e
(
·
n
T
k
···
)i1 · 2)
1,
c
)
ik
T
iN
e
SG
f y
··i
·
)
atnh o tuha··
= i1 1
N
= YN
,N
···
,8
QN
ed e
t
i
h
aTce
k=
2 kyN
x
.
e
cuc
i
n+
s
n=
1·
i1
a
G (n
··i
m h
···
1 G (k1

Initialization: core tensors G (1) , G (2) , · · · , G (N ) of approximated tensor X .
3: While the optimization stopping condition is not satisfied
While the optimization stopping condition is not satisfied
4: Randomly sample one observed entry from Y.
4: Randomly sample one observed entry from Y.
5: For i=1:N
5: Fori=1:N
Algorithm
3 Tensor-train
Stochastic
Gradient Descent(TTSGD)@fStochastic
Algorithm
Tensor-train
Descent<n
T(TTSGD)
6: 3Compute= (ym Gradient
xm>n
)(G>n
m Gi m ) .
i<n@f @G(n)
Tn
n
m(y
6:
Compute
=
G
m and
iY
1: 1:
Input:
Incomplete
tensorT xTmT)(Gr.imr.) .
(n)
im
n and
Input:
Incomplete
tensor
Y
Trank
rank
2:

OneOneadvantageof gradient-based
optimizationis that
we do
too tooadvantageof gradient-based
optimizationis that
we notdo neednot need
itune
i
=
2
i
=
3
i
=
4
i2 =
=
2
i
=
3
i
=
4
2 1=i21222
2
2
so many
hyper
parameters,
we caneasilyget any
wantedaccuracy
withintune
so many
hyper
parameters,
we caneasilyget any
wantedaccuracy
withinExperiments
4. 4.
Experiments

(2

)

11 11

Higher-order tensor

advantageof gradient-based
optimizationis thatneed
OneOneadvantageof gradient-based
optimizationis thatwe we
do donotnotneedtootootune
so many
hyper
parameters,
easilywantedaccuracy
withintune
so many
hyper
parameters,
we wecancaneasilygetgetanyanywantedaccuracy
within

or

ion

ct
i
d
e
Pr

Recovered data

···
i1

16

=

x

(x

m

ym

iN

TT/TR decompositionx

Low-rank TT/TR approximation

iN

.

in )

11 11

i1
k2
)
( G ···if= 1 i >n N F .
m
= YN
nQ
n
G<
1 2 k
yi
n=
inm n nG1
) T=1 G (n
1
···
(n
iN
in )
,
r, = inm )on(x . I.
xal lymth f w
i1
l
e
··

Observed datasha1_base64="eAY1r2BrOalstHNSXdkAZw2s+Yk=">AAAB/XicbVDLSsNAFL2pr1pf9bFzM1gEVyURQZdFNy5cVLEPaEKYTCbt0MmDmYlQQ/BX3LhQxK3/4c6/cdJmoa0HBg7n3sM9c7yEM6lM89uoLC2vrK5V12sbm1vbO/Xdva6MU0Foh8Q8Fn0PS8pZRDuKKU77iaA49DjteeOrYt57oEKyOLpXk4Q6IR5GLGAEKy259QM7xGrkBdmdm9k32ufjPHfrDbNpToEWiVWSBpRou/Uv249JGtJIEY6lHFhmopwMC8UIp3nNTiVNMBnjIR1oGuGQSiebps/RsVZ8FMRCv0ihqfrbkeFQykno6c0iq5yfFeJ/s0GqggsnY1GSKhqR2aEg5UjFqKgC+UxQovhEE0wE01kRGWGBidKF1XQJ1vyXF0n3tGlpfnvWaF2WdVThEI7gBCw4hxZcQxs6QOARnuEV3own48V4Nz5mqxWj9OzDHxifP+VwlX8=</latexit>

sha1_base64="eAY1r2BrOalstHNSXdkAZw2s+Yk=">AAAB/XicbVDLSsNAFL2pr1pf9bFzM1gEVyURQZdFNy5cVLEPaEKYTCbt0MmDmYlQQ/BX3LhQxK3/4c6/cdJmoa0HBg7n3sM9c7yEM6lM89uoLC2vrK5V12sbm1vbO/Xdva6MU0Foh8Q8Fn0PS8pZRDuKKU77iaA49DjteeOrYt57oEKyOLpXk4Q6IR5GLGAEKy259QM7xGrkBdmdm9k32ufjPHfrDbNpToEWiVWSBpRou/Uv249JGtJIEY6lHFhmopwMC8UIp3nNTiVNMBnjIR1oGuGQSiebps/RsVZ8FMRCv0ihqfrbkeFQykno6c0iq5yfFeJ/s0GqggsnY1GSKhqR2aEg5UjFqKgC+UxQovhEE0wE01kRGWGBidKF1XQJ1vyXF0n3tGlpfnvWaF2WdVThEI7gBCw4hxZcQxs6QOARnuEV3own48V4Nz5mqxWj9OzDHxifP+VwlX8=</latexit><latexit

sha1_base64="eAY1r2BrOalstHNSXdkAZw2s+Yk=">AAAB/XicbVDLSsNAFL2pr1pf9bFzM1gEVyURQZdFNy5cVLEPaEKYTCbt0MmDmYlQQ/BX3LhQxK3/4c6/cdJmoa0HBg7n3sM9c7yEM6lM89uoLC2vrK5V12sbm1vbO/Xdva6MU0Foh8Q8Fn0PS8pZRDuKKU77iaA49DjteeOrYt57oEKyOLpXk4Q6IR5GLGAEKy259QM7xGrkBdmdm9k32ufjPHfrDbNpToEWiVWSBpRou/Uv249JGtJIEY6lHFhmopwMC8UIp3nNTiVNMBnjIR1oGuGQSiebps/RsVZ8FMRCv0ihqfrbkeFQykno6c0iq5yfFeJ/s0GqggsnY1GSKhqR2aEg5UjFqKgC+UxQovhEE0wE01kRGWGBidKF1XQJ1vyXF0n3tGlpfnvWaF2WdVThEI7gBCw4hxZcQxs6QOARnuEV3own48V4Nz5mqxWj9OzDHxifP+VwlX8=</latexit><latexit

matically390
tion of (5) as follow:
R⇤
A matrix
related to dual certificate
315
The
proof
is
given
in
the
supplemental
material.
391
he imagem1 ⇥m2
316
Definition2(Dualcertificate).
Amatrix⇤2
R
isFirst,
We
considerthefourassumptions
given
in
Theo392
2 [K] in17
sha1_base64="eAY1r2BrOalstHNSXdkAZw2s+Yk=">AAAB/XicbVDLSsNAFL2pr1pf9bFzM1gEVyURQZdFNy5cVLEPaEKYTCbt0MmDmYlQQ/BX3LhQxK3/4c6/cdJmoa0HBg7n3sM9c7yEM6lM89uoLC2vrK5V12sbm1vbO/Xdva6MU0Foh8Q8Fn0PS8pZRDuKKU77iaA49DjteeOrYt57oEKyOLpXk4Q6IR5GLGAEKy259QM7xGrkBdmdm9k32ufjPHfrDbNpToEWiVWSBpRou/Uv249JGtJIEY6lHFhmopwMC8UIp3nNTiVNMBnjIR1oGuGQSiebps/RsVZ8FMRCv0ihqfrbkeFQykno6c0iq5yfFeJ/s0GqggsnY1GSKhqR2aEg5UjFqKgC+UxQovhEE0wE01kRGWGBidKF1XQJ1vyXF0n3tGlpfnvWaF2WdVThEI7gBCw4hxZcQxs6QOARnuEV3own48V4Nz5mqxWj9OzDHxifP+VwlX8=</latexit><latexit

sha1_base64="jD1aSF7rfvNDImFxhC+TySckS/I=">AAAB8nicbZBNS8NAEIYnftb6VfXoJViEeimJCHosevFYwX5AG8pms2mXbnbD7kQooT/DiwdFvPprvPlv3LY5aOsLCw/vzLAzb5gKbtDzvp219Y3Nre3STnl3b//gsHJ03DYq05S1qBJKd0NimOCStZCjYN1UM5KEgnXC8d2s3nli2nAlH3GSsiAhQ8ljTglaq0eVjGp9Gim8GFSqXt2by10Fv4AqFGoOKl/9SNEsYRKpIMb0fC/FICcaORVsWu5nhqWEjsmQ9SxKkjAT5POVp+65dSI3Vto+ie7c/T2Rk8SYSRLazoTgyCzXZuZ/tV6G8U2Qc5lmyCRdfBRnwkXlzu53I64ZRTGxQKjmdleXjogmFG1KZRuCv3zyKrQv677lh6tq47aIowSncAY18OEaGnAPTWgBBQXP8ApvDjovzrvzsWhdc4qZE/gj5/MHvymQ4w==</latexit>

sha1_base64="jD1aSF7rfvNDImFxhC+TySckS/I=">AAAB8nicbZBNS8NAEIYnftb6VfXoJViEeimJCHosevFYwX5AG8pms2mXbnbD7kQooT/DiwdFvPprvPlv3LY5aOsLCw/vzLAzb5gKbtDzvp219Y3Nre3STnl3b//gsHJ03DYq05S1qBJKd0NimOCStZCjYN1UM5KEgnXC8d2s3nli2nAlH3GSsiAhQ8ljTglaq0eVjGp9Gim8GFSqXt2by10Fv4AqFGoOKl/9SNEsYRKpIMb0fC/FICcaORVsWu5nhqWEjsmQ9SxKkjAT5POVp+65dSI3Vto+ie7c/T2Rk8SYSRLazoTgyCzXZuZ/tV6G8U2Qc5lmyCRdfBRnwkXlzu53I64ZRTGxQKjmdleXjogmFG1KZRuCv3zyKrQv677lh6tq47aIowSncAY18OEaGnAPTWgBBQXP8ApvDjovzrvzsWhdc4qZE/gj5/MHvymQ4w==</latexit><latexit

sha1_base64="jD1aSF7rfvNDImFxhC+TySckS/I=">AAAB8nicbZBNS8NAEIYnftb6VfXoJViEeimJCHosevFYwX5AG8pms2mXbnbD7kQooT/DiwdFvPprvPlv3LY5aOsLCw/vzLAzb5gKbtDzvp219Y3Nre3STnl3b//gsHJ03DYq05S1qBJKd0NimOCStZCjYN1UM5KEgnXC8d2s3nli2nAlH3GSsiAhQ8ljTglaq0eVjGp9Gim8GFSqXt2by10Fv4AqFGoOKl/9SNEsYRKpIMb0fC/FICcaORVsWu5nhqWEjsmQ9SxKkjAT5POVp+65dSI3Vto+ie7c/T2Rk8SYSRLazoTgyCzXZuZ/tV6G8U2Qc5lmyCRdfBRnwkXlzu53I64ZRTGxQKjmdleXjogmFG1KZRuCv3zyKrQv677lh6tq47aIowSncAY18OEaGnAPTWgBBQXP8ApvDjovzrvzsWhdc4qZE/gj5/MHvymQ4w==</latexit><latexit

sha1_base64="jD1aSF7rfvNDImFxhC+TySckS/I=">AAAB8nicbZBNS8NAEIYnftb6VfXoJViEeimJCHosevFYwX5AG8pms2mXbnbD7kQooT/DiwdFvPprvPlv3LY5aOsLCw/vzLAzb5gKbtDzvp219Y3Nre3STnl3b//gsHJ03DYq05S1qBJKd0NimOCStZCjYN1UM5KEgnXC8d2s3nli2nAlH3GSsiAhQ8ljTglaq0eVjGp9Gim8GFSqXt2by10Fv4AqFGoOKl/9SNEsYRKpIMb0fC/FICcaORVsWu5nhqWEjsmQ9SxKkjAT5POVp+65dSI3Vto+ie7c/T2Rk8SYSRLazoTgyCzXZuZ/tV6G8U2Qc5lmyCRdfBRnwkXlzu53I64ZRTGxQKjmdleXjogmFG1KZRuCv3zyKrQv677lh6tq47aIowSncAY18OEaGnAPTWgBBQXP8ApvDjovzrvzsWhdc4qZE/gj5/MHvymQ4w==</latexit><latexit

sha1_base64="CB1/02oqqrRe3v5WoGMH/HzTIbQ=">AAAB83icbVDLSsNAFL2pr1pfVZduBovgqiQi6LLoxo1QwT6gCWUyvWmHTiZhZiKU0N9w40IRt/6MO//GaZuFth4YOJxzL/fMCVPBtXHdb6e0tr6xuVXeruzs7u0fVA+P2jrJFMMWS0SiuiHVKLjEluFGYDdVSONQYCcc3878zhMqzRP5aCYpBjEdSh5xRo2VfD+mZhRG+f207/arNbfuzkFWiVeQGhRo9qtf/iBhWYzSMEG17nluaoKcKsOZwGnFzzSmlI3pEHuWShqjDvJ55ik5s8qARImyTxoyV39v5DTWehKHdnKWUS97M/E/r5eZ6DrIuUwzg5ItDkWZICYhswLIgCtkRkwsoUxxm5WwEVWUGVtTxZbgLX95lbQv6p7lD5e1xk1RRxlO4BTOwYMraMAdNKEFDFJ4hld4czLnxXl3PhajJafYOYY/cD5/AOcikZQ=</latexit>

sha1_base64="CB1/02oqqrRe3v5WoGMH/HzTIbQ=">AAAB83icbVDLSsNAFL2pr1pfVZduBovgqiQi6LLoxo1QwT6gCWUyvWmHTiZhZiKU0N9w40IRt/6MO//GaZuFth4YOJxzL/fMCVPBtXHdb6e0tr6xuVXeruzs7u0fVA+P2jrJFMMWS0SiuiHVKLjEluFGYDdVSONQYCcc3878zhMqzRP5aCYpBjEdSh5xRo2VfD+mZhRG+f207/arNbfuzkFWiVeQGhRo9qtf/iBhWYzSMEG17nluaoKcKsOZwGnFzzSmlI3pEHuWShqjDvJ55ik5s8qARImyTxoyV39v5DTWehKHdnKWUS97M/E/r5eZ6DrIuUwzg5ItDkWZICYhswLIgCtkRkwsoUxxm5WwEVWUGVtTxZbgLX95lbQv6p7lD5e1xk1RRxlO4BTOwYMraMAdNKEFDFJ4hld4czLnxXl3PhajJafYOYY/cD5/AOcikZQ=</latexit><latexit

sha1_base64="CB1/02oqqrRe3v5WoGMH/HzTIbQ=">AAAB83icbVDLSsNAFL2pr1pfVZduBovgqiQi6LLoxo1QwT6gCWUyvWmHTiZhZiKU0N9w40IRt/6MO//GaZuFth4YOJxzL/fMCVPBtXHdb6e0tr6xuVXeruzs7u0fVA+P2jrJFMMWS0SiuiHVKLjEluFGYDdVSONQYCcc3878zhMqzRP5aCYpBjEdSh5xRo2VfD+mZhRG+f207/arNbfuzkFWiVeQGhRo9qtf/iBhWYzSMEG17nluaoKcKsOZwGnFzzSmlI3pEHuWShqjDvJ55ik5s8qARImyTxoyV39v5DTWehKHdnKWUS97M/E/r5eZ6DrIuUwzg5ItDkWZICYhswLIgCtkRkwsoUxxm5WwEVWUGVtTxZbgLX95lbQv6p7lD5e1xk1RRxlO4BTOwYMraMAdNKEFDFJ4hld4czLnxXl3PhajJafYOYY/cD5/AOcikZQ=</latexit><latexit

sha1_base64="CB1/02oqqrRe3v5WoGMH/HzTIbQ=">AAAB83icbVDLSsNAFL2pr1pfVZduBovgqiQi6LLoxo1QwT6gCWUyvWmHTiZhZiKU0N9w40IRt/6MO//GaZuFth4YOJxzL/fMCVPBtXHdb6e0tr6xuVXeruzs7u0fVA+P2jrJFMMWS0SiuiHVKLjEluFGYDdVSONQYCcc3878zhMqzRP5aCYpBjEdSh5xRo2VfD+mZhRG+f207/arNbfuzkFWiVeQGhRo9qtf/iBhWYzSMEG17nluaoKcKsOZwGnFzzSmlI3pEHuWShqjDvJ55ik5s8qARImyTxoyV39v5DTWehKHdnKWUS97M/E/r5eZ6DrIuUwzg5ItDkWZICYhswLIgCtkRkwsoUxxm5WwEVWUGVtTxZbgLX95lbQv6p7lD5e1xk1RRxlO4BTOwYMraMAdNKEFDFJ4hld4czLnxXl3PhajJafYOYY/cD5/AOcikZQ=</latexit><latexit<latexit

sha1_base64="GxB/yK6EXN3hM9kq9tF0jxtsIi4=">AAAB+nicbVDLSsNAFL3xWesr1aWbwSK4KokIuiy6cSNUsA9oQplMJ+3QySTMTJQS8yluXCji1i9x5984abPQ1gMDh3Pu5Z45QcKZ0o7zba2srq1vbFa2qts7u3v7du2go+JUEtomMY9lL8CKciZoWzPNaS+RFEcBp91gcl343QcqFYvFvZ4m1I/wSLCQEayNNLBryBtjnXkR1uMgzG7zfGDXnYYzA1ombknqUKI1sL+8YUzSiApNOFaq7zqJ9jMsNSOc5lUvVTTBZIJHtG+owBFVfjaLnqMTowxRGEvzhEYz9fdGhiOlplFgJouIatErxP+8fqrDSz9jIkk1FWR+KEw50jEqekBDJinRfGoIJpKZrIiMscREm7aqpgR38cvLpHPWcA2/O683r8o6KnAEx3AKLlxAE26gBW0g8AjP8Apv1pP1Yr1bH/PRFavcOYQ/sD5/AGwmlBk=</latexit>

sha1_base64="GxB/yK6EXN3hM9kq9tF0jxtsIi4=">AAAB+nicbVDLSsNAFL3xWesr1aWbwSK4KokIuiy6cSNUsA9oQplMJ+3QySTMTJQS8yluXCji1i9x5984abPQ1gMDh3Pu5Z45QcKZ0o7zba2srq1vbFa2qts7u3v7du2go+JUEtomMY9lL8CKciZoWzPNaS+RFEcBp91gcl343QcqFYvFvZ4m1I/wSLCQEayNNLBryBtjnXkR1uMgzG7zfGDXnYYzA1ombknqUKI1sL+8YUzSiApNOFaq7zqJ9jMsNSOc5lUvVTTBZIJHtG+owBFVfjaLnqMTowxRGEvzhEYz9fdGhiOlplFgJouIatErxP+8fqrDSz9jIkk1FWR+KEw50jEqekBDJinRfGoIJpKZrIiMscREm7aqpgR38cvLpHPWcA2/O683r8o6KnAEx3AKLlxAE26gBW0g8AjP8Apv1pP1Yr1bH/PRFavcOYQ/sD5/AGwmlBk=</latexit><latexit

sha1_base64="GxB/yK6EXN3hM9kq9tF0jxtsIi4=">AAAB+nicbVDLSsNAFL3xWesr1aWbwSK4KokIuiy6cSNUsA9oQplMJ+3QySTMTJQS8yluXCji1i9x5984abPQ1gMDh3Pu5Z45QcKZ0o7zba2srq1vbFa2qts7u3v7du2go+JUEtomMY9lL8CKciZoWzPNaS+RFEcBp91gcl343QcqFYvFvZ4m1I/wSLCQEayNNLBryBtjnXkR1uMgzG7zfGDXnYYzA1ombknqUKI1sL+8YUzSiApNOFaq7zqJ9jMsNSOc5lUvVTTBZIJHtG+owBFVfjaLnqMTowxRGEvzhEYz9fdGhiOlplFgJouIatErxP+8fqrDSz9jIkk1FWR+KEw50jEqekBDJinRfGoIJpKZrIiMscREm7aqpgR38cvLpHPWcA2/O683r8o6KnAEx3AKLlxAE26gBW0g8AjP8Apv1pP1Yr1bH/PRFavcOYQ/sD5/AGwmlBk=</latexit><latexit

sha1_base64="GxB/yK6EXN3hM9kq9tF0jxtsIi4=">AAAB+nicbVDLSsNAFL3xWesr1aWbwSK4KokIuiy6cSNUsA9oQplMJ+3QySTMTJQS8yluXCji1i9x5984abPQ1gMDh3Pu5Z45QcKZ0o7zba2srq1vbFa2qts7u3v7du2go+JUEtomMY9lL8CKciZoWzPNaS+RFEcBp91gcl343QcqFYvFvZ4m1I/wSLCQEayNNLBryBtjnXkR1uMgzG7zfGDXnYYzA1ombknqUKI1sL+8YUzSiApNOFaq7zqJ9jMsNSOc5lUvVTTBZIJHtG+owBFVfjaLnqMTowxRGEvzhEYz9fdGhiOlplFgJouIatErxP+8fqrDSz9jIkk1FWR+KEw50jEqekBDJinRfGoIJpKZrIiMscREm7aqpgR38cvLpHPWcA2/O683r8o6KnAEx3AKLlxAE26gBW0g8AjP8Apv1pP1Yr1bH/PRFavcOYQ/sD5/AGwmlBk=</latexit><latexit<latexit

383
309
Then the solution(5)kept.
M̂ obeysFurthermore, we define a linear spaceues of
are
384 o
n
310
>
>
n1 ⇥R
n2385
⇥REstimationM̂
ploit kthe
, which,Y2RM̂ M0 kTF = UX + YV |X 2 R
311
386
s
ally split
reflects themin{nproperties
of+ theneighborhood
around
M0 . Let2 ) (9)
312
,
n
}(p
k
[Q]
k
387
Groundtruth
M
1
2
cond(Q)20h2i
and each.
2 ·to T. 388
Based on the
T? denote the orthogonal complement313
1 kR⇤ k2pcond(·)
Conditione image.
dual theory, we define the dual certificate
for389uniquesolu- number314<latexits likeREVIEW
re- COPY. DOminkQ(X)kNTIAL
NOT DISTRIBUTE.
⇤
m
⇥m
1
2
X2R
ce to dig<latexit

Assume that M0 2 R
denotes the “true” ma296
trix
that
we
want
to
recover,
and
its
rank
equalsR.
The
From
TensorizationtoLinearTransformation
ong i-th297
isgenerated
by
Y
=
M
+
H
where
noisedvariantofM
00perations298
the entries of H obey the i.i.d. Gaussian distribution, i.e.
MT In
when
299
the simplest
case,
the
completion
problemcanbesolved2
H(i, j) ⇠ N (0, ) for all i 2 [m1 ], j 2 [m2 ]. With theat tensor
300
by the following
optimization
problem:singlelineartransformation,we
simplify
(2)
as
CVPR
shapes,m1 ⇥m2295

s.t. kP⌦ (X)#****P⌦ (Y)kF  ,304
378
Lineartransformationi) there exists a dual
certificate
obeying kR⇤ k2 m
<11;305
2 [Chao et al, CVPR’19]
where the subscript of Q 2 R ⇥m2 ⇥n1 ⇥n379
is removed?
306
380 truncatedii) 9p > 0,for
s.t.Pbrevity.
T QP⌦ Q PLetT ⌫ pI
Q(M0 ) = UDV> be then be used381
307
singular value? decomposition (SVD), in which
only the

(5)

301
302
303

382 problem obeys
ications.
308
With mildsingularconditions,thesolution
of
the
above
vectors with respect to non-zero singular val-iii) The product [Q]h2i · [Q]h2i is a diagonal matrix.

Beyond Unfolding: Reshuffling Operation
mode-3mode-2mode-2Mode-1 Unfolding

mode-1mode-2high rank(b) The unfolded matrix X along the frist mode(a) a 3rd-order tensor with the size 3x3x3
Re

The correspondence

g

lin

uff

sh

“Restore the Rubix Cube”

mode-2mode-1mode-3Mode-3 Unfolding

low rank(c) The matrix Y by the propsoed tensor reshuffling

Fig. Difference between tensor unfolding and reshuffling.

18

Main Results

Low-rank image steganography
Reshuffled
Tensor Decomposition

In our study, we’ like to
Additional
hide a full-size RGB images (e.g. 2K ◊ 2K ) into a grayscale
image;Assumption
choose the images from different fields, e.g. natural, cartoon;
We further assume that each component Ai has the low-rank s
try to give an invisible change on the cover images.

Image steganography is to hide a secret image into cover image
cover

The latent components can be exactly recovered by solving th
problem:container

strength para.
secretmin

our method
reconstruction

Random permutation

Ai , iœ[N]

N
ÿ
i=1

ÎAi Îú ,s.t., X =Nÿ
i=1

Ri (Ai ),where we employ the matrix nuclear norm Î · Îú in the model asof the matrix rank.

Fig. 14 Block diagram of the system scheme used in the experiment.
Chao Li (RIKEN-AIP)

Tensor Learning Unit

February 23, 201921 / 30

Chao Li (RIKEN-AIP)

19

Tensor Learning Unit

February

Tensor Networks for Model Representation

Under review
as a review
conference
paper at ICLR
2019
Under
as a conferencepaper
at ICLR 2019

Deep Multi-task Learning[Long et al. NIPS 2017]

MRN‣ Cannot handle data from

[Yang et al, ICLR 2017]

MRN
DMTRL-Tucker
DMTRL-TTDMTRL-Tucker
DMTRL-TT

TN
Prior

TN
Prior

TN
Prior

TN
Prior

multiple sources/modalities‣ Cannot deal withheterogeneous network forindividual task

‣ Lack flexibility in knowledgesharing mechanism

Task
B A Task
B A TaskTask
Task
B A Task BT
Task
B A Task
Task A TaskTask
B A Task

Figure1: Theoverallsharing mechanismsof MRN,
two
Figure 1: The
overallsharingmechanismsof MRN, twovariants
of variants
DMTRLo(and our
TRMTL
w.r.t. The
two sharedtasks. The
sharedportion isindepictediand our TRMTL
w.r.t.
two tasks.
portionis depictedyellow. T
thinrepresent
rectanglestensor
represent
tensorcores,
matrices
vectors, respec
thin rectangles
cores,
matrices
andvectors,and
respectively.
MR
totally
sharedthe lower
layersand the relatedness
between
totally sharedat theloweratlayers
and therelatedness
between tasks
at thetas
totensornormalpriors.
DMTRL
(TT
or
Tucker):alllayer-wise
wetensor normal priors. DMTRL (TT or Tucker): all layer-wise weights must
be stacked
and decomposed
For almost
each task,
almostall
be stacked anddecomposed
into factors.into
Forfactors.
each task,
all thefactors
exceptlast Such
1D vector.
Suchpattern isofidenticalsharing isatidentical
except the very
lastthe1Dveryvector.
patternof sharing
all layeweightsareseparately
encodedinto
TR-formats
fordifferent
20
weights are separately
encoded into TR-formats for different tasks, and taska su

conference paper at ICLR 2019

Tensor Ring Multi-task Learning
DNN for eachtask
‣ Heterogeneous
DMTRL-Tucker
DMTRL-TT‣ Subset of TR-cores are shared among

TRMTL

tasks

‣ Flexibility in knowledge-sharing pattern‣ High efficiency by sharing information inlatent space‣ Disadvantages: choosing the best cores

sk B

for sharing is difficult.

Task A Task B

Task A Task B
21

Task A

Task BAI Support for Epileptic Diagnosis

Brain signal (EEG)
examination(surface)

Drug therapy tocontrol seizures

Need to record
data for one week

Not effective
for all patients

Brain signal(iEEG)
examination(inside)Surgical removal

Need Craniotomy
Difficult to determine
and long time record the location of remove

Challenging problems‣ Need special doctors, only about 600 eligible doctors in Japan.
‣ Need several weeks high-quality iEEG data.
‣ Time-consuming by several doctors’ visual judgment.
‣ Focal detection is not reliable.
順天堂
22

AI Support for Epileptic Diagnosis‣ Mission: Automatic localization of epileptic focal from iEEG
signals as a support technology for doctors‣ High accuracy

Entropies of different frequency bands forfeature extraction and CNN for classification

Healthzone

Epileptic
zone
Focal

‣ End to end model

Discovery of iEEG focal without handcraftfeature extraction

PU learning‣ Less labels

Only need a few labelled data by PU learning[Prof. Sugiyama’s PU algorithms]

Positive

Unlabeled順天堂
23• X1 , X2 , X3 , X4 independent

Future Work: Tensor Network for Graphical Model•H

10 variables, 10 states each −→ 1010 entries‣ CP decomposition
Tensors and graphical models
X1

X2

X3

X4

and graphical models!
P(x1 , x2 , x3 , x4 ) =P(x1 |h)P(x2 |h)P(x3 |h)P(x4 |h)P(h)

h

H

H

ANDECOMP / PARAFACCP / CANDECOMP / PARAFAC
X1
X2 · · · Xn
• more variables
Markov random field models as tensor train‣

X1

X2 · · ·

H1

H2

H3

X2

X3

Xn

• more hidden variables[Novikov et al., ICML 2014]

H1

train

Tensor train
X1

···

H3

H2

Hn

12

X3

X2

Xn

HMM

X1

‣ Undirected graphical model Hrepresented as a TT model
2[Glasser et al, 2018]

rk our algorithms for several generalizedarchitectures on di↵erent datasets[47–49].
sification, we find that generalized tensor
erform previously introduced tensor netms based on MPS or trees while keeping an of the tensors. In the context of envind recognition, we find that MPS and SBS
able performance. This shows that SBS
considered along with MPS when considnsional data, especially in the presence of
elations, and may be applied in other setatural language processing.
architectures we consider can be realized
omputer, and serve as a reminder that theal input data may also be useful in quan-hical Tucker

MLSVD

X1

Besides MPS, other succes
PEPS and MERA
G. EVENBLY AND G. VIDAL

HR. Orús / Annals of Physics 349 (2014) 117–158

X1

X

Hierarchical Tucker(a)··

X1 (b) X1

X1

X1

Latent tree model

X1

X

X1

X1

X1

Fig. 2. (Color online) Two examples of tensor network diagrams: (a) Matrix Product State (MPS) for 4 sites withconditions; (b) Projected Entangled Pair State (PEPS) for a 3 ⇥ 3 lattice with open boundary conditions.

(c)

(d)

24
FIG. 1. (a) Undirected graphical model with three maximal

MERA

PEPS

FIG. 1. % Color online! Quantum circuit C correspondi
specific realization of the MERA, namely, the binary 1D M
Fig. 2. In this particular example, circuit C is made of gatesing two incoming wires and two outgoing wires, p = pin =Some of the unitary gates in this circuit have one incoming
the fixed state #0$ and can be replaced with an isometry w(2D systems)

states is radically different from the usual approach, where one just gives the coefficienfunction in some given basis. When dealing with a TN state we will see that, insteadsha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit>

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit<latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexit<sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexit<N
sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV+

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexitNsha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexitN<

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit>

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit<latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexitN<

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexit<N

latexitsha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexit<sha1_base64="NQekAT9OTGnhOyKsKD9yfhsG+ug=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tFEA8lEUGPRS8eW7Af0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/N6u0nVJrH8sFMEvQjOpQ85IwaazUu+uWKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8MbPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVqXVc9y46pSu83jKMIJnMI5eHANNbiHOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPcAOMrg==</latexit>

sha1_base64="NQekAT9OTGnhOyKsKD9yfhsG+ug=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tFEA8lEUGPRS8eW7Af0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/N6u0nVJrH8sFMEvQjOpQ85IwaazUu+uWKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8MbPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVqXVc9y46pSu83jKMIJnMI5eHANNbiHOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPcAOMrg==</latexit><latexit

sha1_base64="NQekAT9OTGnhOyKsKD9yfhsG+ug=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tFEA8lEUGPRS8eW7Af0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/N6u0nVJrH8sFMEvQjOpQ85IwaazUu+uWKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8MbPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVqXVc9y46pSu83jKMIJnMI5eHANNbiHOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPcAOMrg==</latexit><latexit

sha1_base64="NQekAT9OTGnhOyKsKD9yfhsG+ug=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tFEA8lEUGPRS8eW7Af0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/N6u0nVJrH8sFMEvQjOpQ85IwaazUu+uWKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8MbPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVqXVc9y46pSu83jKMIJnMI5eHANNbiHOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPcAOMrg==</latexit><latexit

latexit<⇤⇤sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexit<sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexit

N
<

r

r

r
N

N
sha1_base64="8uDF8DVeQnsAQmi3YnN5BydvDus=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQiFL14bMF+QBvKZjtp1242YXcjlNBf4MWDIl79Sd78N27bHLT1hYWHd2bYmTdIBNfGdb+dwtr6xuZWcbu0s7u3f1A+PGrpOFUMmywWseoEVKPgEpuGG4GdRCGNAoHtYHw3q7efUGkeywczSdCP6FDykDNqrNW46ZcrbtWdi6yCl0MFctX75a/eIGZphNIwQbXuem5i/Iwqw5nAaamXakwoG9Mhdi1KGqH2s/miU3JmnQEJY2WfNGTu/p7IaKT1JApsZ0TNSC/XZuZ/tW5qwms/4zJJDUq2+ChMBTExmV1NBlwhM2JigTLF7a6EjaiizNhsSjYEb/nkVWhdVD3LjctK7TaPowgncArn4MEV1OAe6tAEBgjP8ApvzqPz4rw7H4vWgpPPHMMfOZ8/jM+MwQ==</latexit>

sha1_base64="8uDF8DVeQnsAQmi3YnN5BydvDus=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQiFL14bMF+QBvKZjtp1242YXcjlNBf4MWDIl79Sd78N27bHLT1hYWHd2bYmTdIBNfGdb+dwtr6xuZWcbu0s7u3f1A+PGrpOFUMmywWseoEVKPgEpuGG4GdRCGNAoHtYHw3q7efUGkeywczSdCP6FDykDNqrNW46ZcrbtWdi6yCl0MFctX75a/eIGZphNIwQbXuem5i/Iwqw5nAaamXakwoG9Mhdi1KGqH2s/miU3JmnQEJY2WfNGTu/p7IaKT1JApsZ0TNSC/XZuZ/tW5qwms/4zJJDUq2+ChMBTExmV1NBlwhM2JigTLF7a6EjaiizNhsSjYEb/nkVWhdVD3LjctK7TaPowgncArn4MEV1OAe6tAEBgjP8ApvzqPz4rw7H4vWgpPPHMMfOZ8/jM+MwQ==</latexit><latexit

sha1_base64="8uDF8DVeQnsAQmi3YnN5BydvDus=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQiFL14bMF+QBvKZjtp1242YXcjlNBf4MWDIl79Sd78N27bHLT1hYWHd2bYmTdIBNfGdb+dwtr6xuZWcbu0s7u3f1A+PGrpOFUMmywWseoEVKPgEpuGG4GdRCGNAoHtYHw3q7efUGkeywczSdCP6FDykDNqrNW46ZcrbtWdi6yCl0MFctX75a/eIGZphNIwQbXuem5i/Iwqw5nAaamXakwoG9Mhdi1KGqH2s/miU3JmnQEJY2WfNGTu/p7IaKT1JApsZ0TNSC/XZuZ/tW5qwms/4zJJDUq2+ChMBTExmV1NBlwhM2JigTLF7a6EjaiizNhsSjYEb/nkVWhdVD3LjctK7TaPowgncArn4MEV1OAe6tAEBgjP8ApvzqPz4rw7H4vWgpPPHMMfOZ8/jM+MwQ==</latexit><latexitNsha1_base64="8uDF8DVeQnsAQmi3YnN5BydvDus=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfQiFL14bMF+QBvKZjtp1242YXcjlNBf4MWDIl79Sd78N27bHLT1hYWHd2bYmTdIBNfGdb+dwtr6xuZWcbu0s7u3f1A+PGrpOFUMmywWseoEVKPgEpuGG4GdRCGNAoHtYHw3q7efUGkeywczSdCP6FDykDNqrNW46ZcrbtWdi6yCl0MFctX75a/eIGZphNIwQbXuem5i/Iwqw5nAaamXakwoG9Mhdi1KGqH2s/miU3JmnQEJY2WfNGTu/p7IaKT1JApsZ0TNSC/XZuZ/tW5qwms/4zJJDUq2+ChMBTExmV1NBlwhM2JigTLF7a6EjaiizNhsSjYEb/nkVWhdVD3LjctK7TaPowgncArn4MEV1OAe6tAEBgjP8ApvzqPz4rw7H4vWgpPPHMMfOZ8/jM+MwQ==</latexit><latexit

=

N

=

N

N

r

r

r

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit>

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit<latexit

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit>

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit

2N + 1<latexit·

2N + 1

2N + 1

sha1_base64="xMLzw93mWo6TZNmMgjx1vetRL80=">AAAB/HicbVDLSsNAFL3xWesr2qWbwSLUTUmKoMuiG1dawT6gjWUymbRDJw9mJkIJ8VfcuFDErR/izr9x0mahrQcGDufcyz1z3JgzqSzr21hZXVvf2Cxtlbd3dvf2zYPDjowSQWibRDwSPRdLyllI24opTnuxoDhwOe26k6vc7z5SIVkU3qtpTJ0Aj0LmM4KVloZmZRBgNSaYp7dZ7eYhbXjZ6dCsWnVrBrRM7IJUoUBraH4NvIgkAQ0V4VjKvm3FykmxUIxwmpUHiaQxJhM8on1NQxxQ6aSz8Bk60YqH/EjoFyo0U39vpDiQchq4ejKPKhe9XPzP6yfKv3BSFsaJoiGZH/ITjlSE8iaQxwQlik81wUQwnRWRMRaYKN1XWZdgL355mXQadVvzu7Nq87KoowRHcAw1sOEcmnANLWgDgSk8wyu8GU/Gi/FufMxHV4xipwJ/YHz+AC02lHE=</latexit>

sha1_base64="xMLzw93mWo6TZNmMgjx1vetRL80=">AAAB/HicbVDLSsNAFL3xWesr2qWbwSLUTUmKoMuiG1dawT6gjWUymbRDJw9mJkIJ8VfcuFDErR/izr9x0mahrQcGDufcyz1z3JgzqSzr21hZXVvf2Cxtlbd3dvf2zYPDjowSQWibRDwSPRdLyllI24opTnuxoDhwOe26k6vc7z5SIVkU3qtpTJ0Aj0LmM4KVloZmZRBgNSaYp7dZ7eYhbXjZ6dCsWnVrBrRM7IJUoUBraH4NvIgkAQ0V4VjKvm3FykmxUIxwmpUHiaQxJhM8on1NQxxQ6aSz8Bk60YqH/EjoFyo0U39vpDiQchq4ejKPKhe9XPzP6yfKv3BSFsaJoiGZH/ITjlSE8iaQxwQlik81wUQwnRWRMRaYKN1XWZdgL355mXQadVvzu7Nq87KoowRHcAw1sOEcmnANLWgDgSk8wyu8GU/Gi/FufMxHV4xipwJ/YHz+AC02lHE=</latexit><latexit

2N + 1
sha1_base64="xMLzw93mWo6TZNmMgjx1vetRL80=">AAAB/HicbVDLSsNAFL3xWesr2qWbwSLUTUmKoMuiG1dawT6gjWUymbRDJw9mJkIJ8VfcuFDErR/izr9x0mahrQcGDufcyz1z3JgzqSzr21hZXVvf2Cxtlbd3dvf2zYPDjowSQWibRDwSPRdLyllI24opTnuxoDhwOe26k6vc7z5SIVkU3qtpTJ0Aj0LmM4KVloZmZRBgNSaYp7dZ7eYhbXjZ6dCsWnVrBrRM7IJUoUBraH4NvIgkAQ0V4VjKvm3FykmxUIxwmpUHiaQxJhM8on1NQxxQ6aSz8Bk60YqH/EjoFyo0U39vpDiQchq4ejKPKhe9XPzP6yfKv3BSFsaJoiGZH/ITjlSE8iaQxwQlik81wUQwnRWRMRaYKN1XWZdgL355mXQadVvzu7Nq87KoowRHcAw1sOEcmnANLWgDgSk8wyu8GU/Gi/FufMxHV4xipwJ/YHz+AC02lHE=</latexit><latexit··
sha1_base64="xMLzw93mWo6TZNmMgjx1vetRL80=">AAAB/HicbVDLSsNAFL3xWesr2qWbwSLUTUmKoMuiG1dawT6gjWUymbRDJw9mJkIJ8VfcuFDErR/izr9x0mahrQcGDufcyz1z3JgzqSzr21hZXVvf2Cxtlbd3dvf2zYPDjowSQWibRDwSPRdLyllI24opTnuxoDhwOe26k6vc7z5SIVkU3qtpTJ0Aj0LmM4KVloZmZRBgNSaYp7dZ7eYhbXjZ6dCsWnVrBrRM7IJUoUBraH4NvIgkAQ0V4VjKvm3FykmxUIxwmpUHiaQxJhM8on1NQxxQ6aSz8Bk60YqH/EjoFyo0U39vpDiQchq4ejKPKhe9XPzP6yfKv3BSFsaJoiGZH/ITjlSE8iaQxwQlik81wUQwnRWRMRaYKN1XWZdgL355mXQadVvzu7Nq87KoowRHcAw1sOEcmnANLWgDgSk8wyu8GU/Gi/FufMxHV4xipwJ/YHz+AC02lHE=</latexit><latexit

2N + 1<latexit

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit>

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit<latexit

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit>

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit

sha1_base64="IRApvYN/Tp115ojiVyarUtmD03Q=">AAAB63icbZDLSgMxFIbP1Futt6pLN8EiCEKZKYIui25cSQV7gXYomTTThiaZIckIZegruHGhiFtfyJ1vY2Y6C239IfDxn3PIOX8Qc6aN6347pbX1jc2t8nZlZ3dv/6B6eNTRUaIIbZOIR6oXYE05k7RtmOG0FyuKRcBpN5jeZvXuE1WaRfLRzGLqCzyWLGQEm8xq3F94w2rNrbu50Cp4BdSgUGtY/RqMIpIIKg3hWOu+58bGT7EyjHA6rwwSTWNMpnhM+xYlFlT7ab7rHJ1ZZ4TCSNknDcrd3xMpFlrPRGA7BTYTvVzLzP9q/cSE137KZJwYKsniozDhyEQoOxyNmKLE8JkFTBSzuyIywQoTY+Op2BC85ZNXodOoe5YfLmvNmyKOMpzAKZyDB1fQhDtoQRsITOAZXuHNEc6L8+58LFpLTjFzDH/kfP4A7emNfg==</latexit><latexit<latexit·
latexitsha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit>

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit<latexit··<

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit>

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit

sha1_base64="MSuyM8BrIDrkDaYJacJW3w6Ycgo=">AAAB7nicbZBNS8NAEIYnftb6VfXoJVgETyURQY9FLx4r2A9oQ9lsNu3SzW7YnQgl9Ed48aCIV3+PN/+NmzYHbX1h4eGdGXbmDVPBDXret7O2vrG5tV3Zqe7u7R8c1o6OO0ZlmrI2VULpXkgME1yyNnIUrJdqRpJQsG44uSvq3SemDVfyEacpCxIykjzmlKC1ugMaKTTVYa3uNby53FXwS6hDqdaw9jWIFM0SJpEKYkzf91IMcqKRU8Fm1UFmWErohIxY36IkCTNBPl935p5bJ3Jjpe2T6M7d3xM5SYyZJqHtTAiOzXKtMP+r9TOMb4KcyzRDJuniozgTLiq3uN2NuGYUxdQCoZrbXV06JppQtAkVIfjLJ69C57LhW364qjdvyzgqcApncAE+XEMT7qEFbaAwgWd4hTcndV6cd+dj0brmlDMn8EfO5w/k6I9D</latexit><latexit<latexit·<

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexit<sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexit<N···

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

··

···

N
sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit···rsha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexitrsha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexitrlatexitsha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexit<N<

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit>

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

sha1_base64="1geheZ8OEVc4P/xyjJq8gn3vuMI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OJJWrAf0Iay2U7atZtN2N0IJfQXePGgiFd/kjf/jds2B219YeHhnRl25g0SwbVx3W+nsLa+sblV3C7t7O7tH5QPj1o6ThXDJotFrDoB1Si4xKbhRmAnUUijQGA7GN/O6u0nVJrH8sFMEvQjOpQ85IwaazXu++WKW3XnIqvg5VCBXPV++as3iFkaoTRMUK27npsYP6PKcCZwWuqlGhPKxnSIXYuSRqj9bL7olJxZZ0DCWNknDZm7vycyGmk9iQLbGVEz0su1mflfrZua8NrPuExSg5ItPgpTQUxMZleTAVfIjJhYoExxuythI6ooMzabkg3BWz55FVoXVc9y47JSu8njKMIJnMI5eHAFNbiDOjSBAcIzvMKb8+i8OO/Ox6K14OQzx/BHzucPppOM0g==</latexit><latexit

latexitN<

Future Work - Acceleration of Tensor Convolution‣ High-order convolution (computation and storage)‣ Fast convolution via tensor network representation
Published as a conference paper at ICLR 2015

2N + 1

O(N 2d )TND
core-based conv.

(a) Full convolution

2N + 1
2N + 1=

2N + 125
2N + 1···
2N + 1

2N + 1(b) TwoO(dN r4 )O(N d r4 )(c) CP-de

Figure 1: Tensor decompositions for speeding u
spond to 3D tensors (map stacks) within a CNN, w
dimensions. Arrows show linear mappings and dboxes corresponding to single elements of the tation (a) computes each element of the target ten
3D subtensor that spans a spatial d ⇥ d window oCollaborations within AIP‣ A novel schema for hyper-spectral image restoration[He et al., CVPR2019]
Naoto Yokoya‣ Dementia detection via tensorizing neural networks[Ruikowski et al., NeurlPS 2018 workshop]
Mihoko Otake‣ Gene data completion via tensor network[Iwata et al., ISMB/ECCB 2019]
Yasuo Tabei

26

Summary‣ Tensor networks are intriguing alternative to
traditional machine learning models‣ Better scaling, efficient algorithms, opportunities fortheoretical insights‣ Promising as a framework for machine learning withquantum computing

27

Achievements in FY2018
Publications (32)
‣ Conference (19) including AAAI, IJCAI, CVPR, ICASSP, NeurIPS
Workshop, ICLR workshop and etc‣ Journal (13) including IEEE TNNLS, Signal Processing and etc

Awards‣ The 3rd IEEE SPS Japan Best Paper Award‣ 2018 SPS Signal Processing Magazine Best Paper

28