Contrastive Multiview Coding

1 INTRODUCTION . A foundational idea in coding theory is to learn compressed representations that nonetheless can be used to reconstruct the raw data . This idea shows up in contemporary representation learning in the form of autoencoders ( Salakhutdinov & Hinton , 2009 ) and generative models ( Kingma & Welling , 2013 ; Goodfellow et al. , 2014 ) , which try to represent a data point or distribution as losslessly as possible . Yet lossless representation might not be what we really want , and indeed it is trivial to achieve – the raw data itself is a lossless representation . What we might instead prefer is to keep the “ good ” information ( signal ) and throw away the rest ( noise ) . How can we identify what information is signal and what is noise ? To an autoencoder , or a max likelihood generative model , a bit is a bit . No one bit is better than any other . Our conjecture in this paper is that some bits are in fact better than others . Some bits code important properties like semantics , physics , and geometry , while others code attributes that we might consider less important , like incidental lighting conditions or thermal noise in a camera ’ s sensor . We hypothesize that the good bits are the ones that are shared between multiple views of the world , for example between multiple sensory modalities like vision , sound , and touch . Under this perspective “ presence of dog ” is good information , since dogs can be seen , heard , and felt , but “ camera pose ” is bad information , since a camera ’ s pose has little or no effect on the acoustic and tactile properties of the imaged scene . There is significant evidence in the cognitive science and neuroscience literature that such cross-view representations are encoded by the brain ( e.g. , Smith & Gasser ( 2005 ) ; Den Ouden et al . ( 2012 ) ; Hohwy ( 2013 ) ) . Our goal is therefore to learn representations that capture information shared between multiple sensory views but that are otherwise compact ( i.e . throw away the bad information ) . To do so , we employ contrastive learning , where we learn a feature embedding such that views of the same scene map to nearby points while views of different scenes map to far apart points . In particular , we adapt the recently proposed method of Contrastive Predictive Coding ( CPC ) ( Oord et al. , 2018 ) , except we simplify it – removing the recurrent network – and generalize it – showing how to apply it to arbitrary collections of views , rather than just to temporal predictions . In reference to CPC , we term our method Contrastive Multiview Coding ( CMC ) . The contrastive objective in our formulation , as in CPC , can be understood as attempting to maximize the mutual information between the representations of each view . We intentionally leave “ good bits ” only loosely defined and treat its definition as an empirical question . Ultimately , the proof is in the pudding : we consider a representation to be good if it makes subsequent problem solving easy , on tasks of human interest . For example , a useful representation of images might be a feature space in which it is easy to learn to recognize objects . We therefore evaluate our method by testing if the learned representations transfer well to standard semantic recognition tasks . On several benchmark tasks , our method achieves state of the art results , compared to other methods for self-supervised representation learning . We additionally find that the quality of the representation improves as a function of the number of views used for training . Finally , we compare the contrastive formulation of multiview learning to the recently popular approach of cross-view prediction , and find that in head-to-head comparisons , the contrastive approach learns stronger representations . The core ideas that we build on : contrastive learning , mutual information maximization , and deep representation learning , are not new and have been explored in the literature on representation and multiview learning ( Li et al. , 2018 ; Xu et al. , 2013 ; Arora et al. , 2019 ) . Our main contribution is to set up a framework to extend these ideas to any number of views , and to empirically study the factors that lead to success in this framework . A review of the related literature is given at the end of the paper , in Section 4 . Fig . 1 gives a pictorial overview of our framework for the different learning tasks we consider in this paper , to learn representations across datasets with different sets of views . Our main contributions are : • We apply contrastive learning to the multiview setting , attemping to maximize mutual information between representations of different views of the same scene ( e.g. , between different image channels , or different modalities ) . • Our approach yields representations that outperform the state-of-the-art in self-supervised learning in head-to-head comparisons . For example , in the ImageNet linear readoff evaluation , we achieve 68.4 % top-1 accuracy , which is slightly above the state of the art concurrent work Bachman et al . ( 2019 ) . • We show that the contrastive objective is superior to cross-view prediction . • We extend the framework to learn from more than two views , and show that the quality of the learned representation improves as number of views increase . • We conduct controlled experiments to measure the effect of mutual information on represen- tation quality . 2 METHOD . Our goal is to learn representations that capture information shared between multiple sensory views without human supervision . We start by reviewing previous predictive learning ( or reconstruction- based learning ) methods , and then elaborate on contrastive learning within two views . We show connections to mutual information maximization and extend it to scenarios including more than two views . We consider a collection of M views of the data , denoted as V1 , . . . , VM . For each view Vi , we denote vi as a random variable representing samples following vi ∼ P ( Vi ) . 2.1 PREDICTIVE LEARNING . Let V1 and V2 represent two views of a dataset . For instance , V1 might be the luminance of a particular image and V2 the chrominance . We define the predictive learning setup as a deep nonlinear transformation from v1 to v2 through latent variables z , as shown in Fig . 2 . Formally , z = f ( v1 ) and v̂2 = g ( z ) , where f and g represent the encoder and decoder respectively and v̂2 is the prediction of v2 given v1 . The parameters of the encoder and decoder models are then trained using an objective function that tries to bring v̂2 “ close to ” v2 . Simple examples of such an objective include the L1 or L2 loss functions . Note that these objectives assume independence between each pixel or element of v2 given v1 , i.e. , p ( v2|v1 ) = Πip ( v2i|v1 ) , thereby reducing their ability to model correlations or complex structure . The predictive approach has been extensively used in representation learning , for example , colorization ( Zhang et al. , 2016 ; 2017 ) and predicting sound from vision ( Owens et al. , 2016 ) . 2.2 CONTRASTIVE LEARNING WITH TWO VIEWS . The idea behind contrastive learning is to learn an embedding that separates ( contrasts ) samples from two different distributions . Given a dataset of V1 and V2 that consists of a collection of samples { vi1 , vi2 } Ni=1 , we consider contrasting congruent and incongruent pairs , i.e . samples from the joint distribution x ∼ p ( v1 , v2 ) or x = { vi1 , vi2 } , which we call positives , versus samples from the product of marginals , y ∼ p ( v1 ) p ( v2 ) or y = { vi1 , vj2 } , which we call negatives . We learn a “ critic ” hθ ( · ) that is high for positives and low for negatives . Similar to recent setups for contrastive learning ( Oord et al. , 2018 ; Gutmann & Hyvärinen , 2010 ; Mnih & Kavukcuoglu , 2013 ) , we train this function to correctly select a single positive sample x out of a set S = { x , y1 , y2 , ... , yk } that contains k negative samples : Lcontrast = −E S [ log hθ ( x ) hθ ( x ) + ∑k i=1 hθ ( yi ) ] ( 1 ) To construct S , we simply fix one view and enumerate positives and negatives from the other view , allowing us to rewrite the objective as : LV1 , V2contrast = − E { v11 , v12 , ... , v k+1 2 } [ log hθ ( { v11 , v12 } ) ∑k+1 j=1 hθ ( { v11 , v j 2 } ) ] ( 2 ) where k is the number of negative samples vj2 for a given sample v 1 1 . In practice , k can be extremely large , and so directly minimizing Eq . 2 is infeasible . In Section 2.4 , we show an approximation based on Noise Contrastive Estimation ( Gutmann & Hyvärinen , 2010 ) that allows for tractable computation . 1 2 1 1 3 2 2 1 3 1 1 6 3 3 1 1 3 1 ( a ) Core View ( b ) Full Graph V1 < latexit sha1_base64= '' BTRfsG7MuUcz7Tb1mYDXTk3FGhE= '' > AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodX3+27Vq3lzkFXiF6QKBRp996s3SFgWc4VMUmO6vpdikFONgkk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YYXQe5UGmGXLHFoiiTBBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHHy4gjrcQQOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB9fVjX4= < /latexit > < latexit sha1_base64= '' BTRfsG7MuUcz7Tb1mYDXTk3FGhE= '' > AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodX3+27Vq3lzkFXiF6QKBRp996s3SFgWc4VMUmO6vpdikFONgkk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YYXQe5UGmGXLHFoiiTBBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHHy4gjrcQQOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB9fVjX4= < /latexit > < latexit sha1_base64= '' BTRfsG7MuUcz7Tb1mYDXTk3FGhE= '' > AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodX3+27Vq3lzkFXiF6QKBRp996s3SFgWc4VMUmO6vpdikFONgkk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YYXQe5UGmGXLHFoiiTBBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHHy4gjrcQQOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB9fVjX4= < /latexit > < latexit sha1_base64= '' BTRfsG7MuUcz7Tb1mYDXTk3FGhE= '' > AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodX3+27Vq3lzkFXiF6QKBRp996s3SFgWc4VMUmO6vpdikFONgkk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YYXQe5UGmGXLHFoiiTBBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHHy4gjrcQQOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB9fVjX4= < /latexit > V2 < latexit sha1_base64= '' NdtrmwDhkKHNNnS1oER0xSZxDQA= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmnXqp5b9e6vKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDZWY1/ < /latexit > < latexit sha1_base64= '' NdtrmwDhkKHNNnS1oER0xSZxDQA= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmnXqp5b9e6vKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDZWY1/ < /latexit > < latexit sha1_base64= '' NdtrmwDhkKHNNnS1oER0xSZxDQA= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmnXqp5b9e6vKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDZWY1/ < /latexit > < latexit sha1_base64= '' NdtrmwDhkKHNNnS1oER0xSZxDQA= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmnXqp5b9e6vKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDZWY1/ < /latexit > V3 < latexit sha1_base64= '' OZ7/eGjiaAd6iFmhE9KemdqLXmM= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6xEnC/YgOlQgFo2ilh1b/sl+uuFV3DrJKvJxUIEejX/7qDWKWRlwhk9SYrucm6GdUo2CST0u91PCEsjEd8q6likbc+Nn81Ck5s8qAhLG2pZDM1d8TGY2MmUSB7YwojsyyNxP/87ophtd+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmldVD236t3XKvWbPI4inMApnIMHV1CHO2hAExgM4Rle4c2Rzovz7nwsWgtOPnMMf+B8/gDa3Y2A < /latexit > < latexit sha1_base64= '' OZ7/eGjiaAd6iFmhE9KemdqLXmM= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6xEnC/YgOlQgFo2ilh1b/sl+uuFV3DrJKvJxUIEejX/7qDWKWRlwhk9SYrucm6GdUo2CST0u91PCEsjEd8q6likbc+Nn81Ck5s8qAhLG2pZDM1d8TGY2MmUSB7YwojsyyNxP/87ophtd+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmldVD236t3XKvWbPI4inMApnIMHV1CHO2hAExgM4Rle4c2Rzovz7nwsWgtOPnMMf+B8/gDa3Y2A < /latexit > < latexit sha1_base64= '' OZ7/eGjiaAd6iFmhE9KemdqLXmM= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6xEnC/YgOlQgFo2ilh1b/sl+uuFV3DrJKvJxUIEejX/7qDWKWRlwhk9SYrucm6GdUo2CST0u91PCEsjEd8q6likbc+Nn81Ck5s8qAhLG2pZDM1d8TGY2MmUSB7YwojsyyNxP/87ophtd+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmldVD236t3XKvWbPI4inMApnIMHV1CHO2hAExgM4Rle4c2Rzovz7nwsWgtOPnMMf+B8/gDa3Y2A < /latexit > < latexit sha1_base64= '' OZ7/eGjiaAd6iFmhE9KemdqLXmM= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6xEnC/YgOlQgFo2ilh1b/sl+uuFV3DrJKvJxUIEejX/7qDWKWRlwhk9SYrucm6GdUo2CST0u91PCEsjEd8q6likbc+Nn81Ck5s8qAhLG2pZDM1d8TGY2MmUSB7YwojsyyNxP/87ophtd+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmldVD236t3XKvWbPI4inMApnIMHV1CHO2hAExgM4Rle4c2Rzovz7nwsWgtOPnMMf+B8/gDa3Y2A < /latexit > V4 < latexit sha1_base64= '' RGbnccGESBHMfrtDKDZHJvcqDa8= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmlfVT236t3XKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDcYY2B < /latexit > < latexit sha1_base64= '' RGbnccGESBHMfrtDKDZHJvcqDa8= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmlfVT236t3XKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDcYY2B < /latexit > < latexit sha1_base64= '' RGbnccGESBHMfrtDKDZHJvcqDa8= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmlfVT236t3XKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDcYY2B < /latexit > < latexit sha1_base64= '' RGbnccGESBHMfrtDKDZHJvcqDa8= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmlfVT236t3XKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDcYY2B < /latexit > V1 < latexit sha1_base64= '' BTRfsG7MuUcz7Tb1mYDXTk3FGhE= '' > AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodX3+27Vq3lzkFXiF6QKBRp996s3SFgWc4VMUmO6vpdikFONgkk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YYXQe5UGmGXLHFoiiTBBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHHy4gjrcQQOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB9fVjX4= < /latexit > < latexit sha1_base64= '' BTRfsG7MuUcz7Tb1mYDXTk3FGhE= '' > AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodX3+27Vq3lzkFXiF6QKBRp996s3SFgWc4VMUmO6vpdikFONgkk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YYXQe5UGmGXLHFoiiTBBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHHy4gjrcQQOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB9fVjX4= < /latexit > < latexit sha1_base64= '' BTRfsG7MuUcz7Tb1mYDXTk3FGhE= '' > AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodX3+27Vq3lzkFXiF6QKBRp996s3SFgWc4VMUmO6vpdikFONgkk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YYXQe5UGmGXLHFoiiTBBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHHy4gjrcQQOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB9fVjX4= < /latexit > < latexit sha1_base64= '' BTRfsG7MuUcz7Tb1mYDXTk3FGhE= '' > AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodX3+27Vq3lzkFXiF6QKBRp996s3SFgWc4VMUmO6vpdikFONgkk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YYXQe5UGmGXLHFoiiTBBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHHy4gjrcQQOawGAIz/AKb450Xpx352PRWnKKmWP4A+fzB9fVjX4= < /latexit > V2 < latexit sha1_base64= '' NdtrmwDhkKHNNnS1oER0xSZxDQA= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmnXqp5b9e6vKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDZWY1/ < /latexit > < latexit sha1_base64= '' NdtrmwDhkKHNNnS1oER0xSZxDQA= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmnXqp5b9e6vKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDZWY1/ < /latexit > < latexit sha1_base64= '' NdtrmwDhkKHNNnS1oER0xSZxDQA= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmnXqp5b9e6vKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDZWY1/ < /latexit > < latexit sha1_base64= '' NdtrmwDhkKHNNnS1oER0xSZxDQA= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmnXqp5b9e6vKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDZWY1/ < /latexit > V3 < latexit sha1_base64= '' OZ7/eGjiaAd6iFmhE9KemdqLXmM= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6xEnC/YgOlQgFo2ilh1b/sl+uuFV3DrJKvJxUIEejX/7qDWKWRlwhk9SYrucm6GdUo2CST0u91PCEsjEd8q6likbc+Nn81Ck5s8qAhLG2pZDM1d8TGY2MmUSB7YwojsyyNxP/87ophtd+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmldVD236t3XKvWbPI4inMApnIMHV1CHO2hAExgM4Rle4c2Rzovz7nwsWgtOPnMMf+B8/gDa3Y2A < /latexit > < latexit sha1_base64= '' OZ7/eGjiaAd6iFmhE9KemdqLXmM= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6xEnC/YgOlQgFo2ilh1b/sl+uuFV3DrJKvJxUIEejX/7qDWKWRlwhk9SYrucm6GdUo2CST0u91PCEsjEd8q6likbc+Nn81Ck5s8qAhLG2pZDM1d8TGY2MmUSB7YwojsyyNxP/87ophtd+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmldVD236t3XKvWbPI4inMApnIMHV1CHO2hAExgM4Rle4c2Rzovz7nwsWgtOPnMMf+B8/gDa3Y2A < /latexit > < latexit sha1_base64= '' OZ7/eGjiaAd6iFmhE9KemdqLXmM= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6xEnC/YgOlQgFo2ilh1b/sl+uuFV3DrJKvJxUIEejX/7qDWKWRlwhk9SYrucm6GdUo2CST0u91PCEsjEd8q6likbc+Nn81Ck5s8qAhLG2pZDM1d8TGY2MmUSB7YwojsyyNxP/87ophtd+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmldVD236t3XKvWbPI4inMApnIMHV1CHO2hAExgM4Rle4c2Rzovz7nwsWgtOPnMMf+B8/gDa3Y2A < /latexit > < latexit sha1_base64= '' OZ7/eGjiaAd6iFmhE9KemdqLXmM= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0m0oMeiF48V7Qe0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GtzO//cS1EbF6xEnC/YgOlQgFo2ilh1b/sl+uuFV3DrJKvJxUIEejX/7qDWKWRlwhk9SYrucm6GdUo2CST0u91PCEsjEd8q6likbc+Nn81Ck5s8qAhLG2pZDM1d8TGY2MmUSB7YwojsyyNxP/87ophtd+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmldVD236t3XKvWbPI4inMApnIMHV1CHO2hAExgM4Rle4c2Rzovz7nwsWgtOPnMMf+B8/gDa3Y2A < /latexit > V4 < latexit sha1_base64= '' RGbnccGESBHMfrtDKDZHJvcqDa8= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmlfVT236t3XKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDcYY2B < /latexit > < latexit sha1_base64= '' RGbnccGESBHMfrtDKDZHJvcqDa8= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmlfVT236t3XKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDcYY2B < /latexit > < latexit sha1_base64= '' RGbnccGESBHMfrtDKDZHJvcqDa8= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmlfVT236t3XKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDcYY2B < /latexit > < latexit sha1_base64= '' RGbnccGESBHMfrtDKDZHJvcqDa8= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkUI9FLx4r2g9oQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ7dzvPKHSPJaPZpqgH9GR5CFn1FjpoT2oDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi814bWfcZmkBiVbLgpTQUxM5n+TIVfIjJhaQpni9lbCxlRRZmw6JRuCt/ryOmlfVT236t3XKo2bPI4inME5XIIHdWjAHTShBQxG8Ayv8OYI58V5dz6WrQUnnzmFP3A+fwDcYY2B < /latexit > v2 < latexit sha1_base64= '' UQ8RRPt2NN02WZInLZUJLciw6fw= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbRqVc+teg9XlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwAKKI2f < /latexit > < latexit sha1_base64= '' UQ8RRPt2NN02WZInLZUJLciw6fw= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbRqVc+teg9XlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwAKKI2f < /latexit > < latexit sha1_base64= '' UQ8RRPt2NN02WZInLZUJLciw6fw= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbRqVc+teg9XlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwAKKI2f < /latexit > < latexit sha1_base64= '' UQ8RRPt2NN02WZInLZUJLciw6fw= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbRqVc+teg9XlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwAKKI2f < /latexit > v3 < latexit sha1_base64= '' fHYLD/bLdDeenuRAHuRzLEbLywI= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbSbt0swm7m0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hobua3xqg0j+WTmSToR3QgecgZNVZ6HPcue+WKW3XnIKvEy0kFctR75a9uP2ZphNIwQbXueG5i/Iwqw5nAaambakwoG9EBdiyVNELtZ/NTp+TMKn0SxsqWNGSu/p7IaKT1JApsZ0TNUC97M/E/r5Oa8MbPuExSg5ItFoWpICYms79JnytkRkwsoUxxeythQ6ooMzadkg3BW355lTQvqp5b9R6uKrXbPI4inMApnIMH11CDe6hDAxgM4Ble4c0Rzovz7nwsWgtOPnMMf+B8/gALrI2g < /latexit > < latexit sha1_base64= '' fHYLD/bLdDeenuRAHuRzLEbLywI= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbSbt0swm7m0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hobua3xqg0j+WTmSToR3QgecgZNVZ6HPcue+WKW3XnIKvEy0kFctR75a9uP2ZphNIwQbXueG5i/Iwqw5nAaambakwoG9EBdiyVNELtZ/NTp+TMKn0SxsqWNGSu/p7IaKT1JApsZ0TNUC97M/E/r5Oa8MbPuExSg5ItFoWpICYms79JnytkRkwsoUxxeythQ6ooMzadkg3BW355lTQvqp5b9R6uKrXbPI4inMApnIMH11CDe6hDAxgM4Ble4c0Rzovz7nwsWgtOPnMMf+B8/gALrI2g < /latexit > < latexit sha1_base64= '' fHYLD/bLdDeenuRAHuRzLEbLywI= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbSbt0swm7m0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hobua3xqg0j+WTmSToR3QgecgZNVZ6HPcue+WKW3XnIKvEy0kFctR75a9uP2ZphNIwQbXueG5i/Iwqw5nAaambakwoG9EBdiyVNELtZ/NTp+TMKn0SxsqWNGSu/p7IaKT1JApsZ0TNUC97M/E/r5Oa8MbPuExSg5ItFoWpICYms79JnytkRkwsoUxxeythQ6ooMzadkg3BW355lTQvqp5b9R6uKrXbPI4inMApnIMH11CDe6hDAxgM4Ble4c0Rzovz7nwsWgtOPnMMf+B8/gALrI2g < /latexit > < latexit sha1_base64= '' fHYLD/bLdDeenuRAHuRzLEbLywI= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbSbt0swm7m0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hobua3xqg0j+WTmSToR3QgecgZNVZ6HPcue+WKW3XnIKvEy0kFctR75a9uP2ZphNIwQbXueG5i/Iwqw5nAaambakwoG9EBdiyVNELtZ/NTp+TMKn0SxsqWNGSu/p7IaKT1JApsZ0TNUC97M/E/r5Oa8MbPuExSg5ItFoWpICYms79JnytkRkwsoUxxeythQ6ooMzadkg3BW355lTQvqp5b9R6uKrXbPI4inMApnIMH11CDe6hDAxgM4Ble4c0Rzovz7nwsWgtOPnMMf+B8/gALrI2g < /latexit > v4 < latexit sha1_base64= '' +c4psOKK6zsb24Fz44doEjLIVSY= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbSuqp5b9R5qlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwANMI2h < /latexit > < latexit sha1_base64= '' +c4psOKK6zsb24Fz44doEjLIVSY= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbSuqp5b9R5qlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwANMI2h < /latexit > < latexit sha1_base64= '' +c4psOKK6zsb24Fz44doEjLIVSY= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbSuqp5b9R5qlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwANMI2h < /latexit > < latexit sha1_base64= '' +c4psOKK6zsb24Fz44doEjLIVSY= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbSuqp5b9R5qlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwANMI2h < /latexit > v1 < latexit sha1_base64= '' iwb6UOVPBQMaW/UYkl1KcM9pxNQ= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0nf65crbtVdgKwTLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjks1IvNTyhbEyHvGupohE3frY4dUYurDIgYaxtKSQL9fdERiNjplFgOyOKI7PqzcX/vG6K4Y2fCZWkyBVbLgpTSTAm87/JQGjOUE4toUwLeythI6opQ5tOyYbgrb68TlpXVc+teg/XlfptHkcRzuAcLsGDGtThHhrQBAZDeIZXeHOk8+K8Ox/L1oKTz5zCHzifPwikjZ4= < /latexit > < latexit sha1_base64= '' iwb6UOVPBQMaW/UYkl1KcM9pxNQ= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0nf65crbtVdgKwTLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjks1IvNTyhbEyHvGupohE3frY4dUYurDIgYaxtKSQL9fdERiNjplFgOyOKI7PqzcX/vG6K4Y2fCZWkyBVbLgpTSTAm87/JQGjOUE4toUwLeythI6opQ5tOyYbgrb68TlpXVc+teg/XlfptHkcRzuAcLsGDGtThHhrQBAZDeIZXeHOk8+K8Ox/L1oKTz5zCHzifPwikjZ4= < /latexit > < latexit sha1_base64= '' iwb6UOVPBQMaW/UYkl1KcM9pxNQ= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0nf65crbtVdgKwTLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjks1IvNTyhbEyHvGupohE3frY4dUYurDIgYaxtKSQL9fdERiNjplFgOyOKI7PqzcX/vG6K4Y2fCZWkyBVbLgpTSTAm87/JQGjOUE4toUwLeythI6opQ5tOyYbgrb68TlpXVc+teg/XlfptHkcRzuAcLsGDGtThHhrQBAZDeIZXeHOk8+K8Ox/L1oKTz5zCHzifPwikjZ4= < /latexit > < latexit sha1_base64= '' iwb6UOVPBQMaW/UYkl1KcM9pxNQ= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0nf65crbtVdgKwTLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjks1IvNTyhbEyHvGupohE3frY4dUYurDIgYaxtKSQL9fdERiNjplFgOyOKI7PqzcX/vG6K4Y2fCZWkyBVbLgpTSTAm87/JQGjOUE4toUwLeythI6opQ5tOyYbgrb68TlpXVc+teg/XlfptHkcRzuAcLsGDGtThHhrQBAZDeIZXeHOk8+K8Ox/L1oKTz5zCHzifPwikjZ4= < /latexit > v3 < latexit sha1_base64= '' fHYLD/bLdDeenuRAHuRzLEbLywI= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbSbt0swm7m0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hobua3xqg0j+WTmSToR3QgecgZNVZ6HPcue+WKW3XnIKvEy0kFctR75a9uP2ZphNIwQbXueG5i/Iwqw5nAaambakwoG9EBdiyVNELtZ/NTp+TMKn0SxsqWNGSu/p7IaKT1JApsZ0TNUC97M/E/r5Oa8MbPuExSg5ItFoWpICYms79JnytkRkwsoUxxeythQ6ooMzadkg3BW355lTQvqp5b9R6uKrXbPI4inMApnIMH11CDe6hDAxgM4Ble4c0Rzovz7nwsWgtOPnMMf+B8/gALrI2g < /latexit > < latexit sha1_base64= '' fHYLD/bLdDeenuRAHuRzLEbLywI= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbSbt0swm7m0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hobua3xqg0j+WTmSToR3QgecgZNVZ6HPcue+WKW3XnIKvEy0kFctR75a9uP2ZphNIwQbXueG5i/Iwqw5nAaambakwoG9EBdiyVNELtZ/NTp+TMKn0SxsqWNGSu/p7IaKT1JApsZ0TNUC97M/E/r5Oa8MbPuExSg5ItFoWpICYms79JnytkRkwsoUxxeythQ6ooMzadkg3BW355lTQvqp5b9R6uKrXbPI4inMApnIMH11CDe6hDAxgM4Ble4c0Rzovz7nwsWgtOPnMMf+B8/gALrI2g < /latexit > < latexit sha1_base64= '' fHYLD/bLdDeenuRAHuRzLEbLywI= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbSbt0swm7m0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hobua3xqg0j+WTmSToR3QgecgZNVZ6HPcue+WKW3XnIKvEy0kFctR75a9uP2ZphNIwQbXueG5i/Iwqw5nAaambakwoG9EBdiyVNELtZ/NTp+TMKn0SxsqWNGSu/p7IaKT1JApsZ0TNUC97M/E/r5Oa8MbPuExSg5ItFoWpICYms79JnytkRkwsoUxxeythQ6ooMzadkg3BW355lTQvqp5b9R6uKrXbPI4inMApnIMH11CDe6hDAxgM4Ble4c0Rzovz7nwsWgtOPnMMf+B8/gALrI2g < /latexit > < latexit sha1_base64= '' fHYLD/bLdDeenuRAHuRzLEbLywI= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbSbt0swm7m0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hobua3xqg0j+WTmSToR3QgecgZNVZ6HPcue+WKW3XnIKvEy0kFctR75a9uP2ZphNIwQbXueG5i/Iwqw5nAaambakwoG9EBdiyVNELtZ/NTp+TMKn0SxsqWNGSu/p7IaKT1JApsZ0TNUC97M/E/r5Oa8MbPuExSg5ItFoWpICYms79JnytkRkwsoUxxeythQ6ooMzadkg3BW355lTQvqp5b9R6uKrXbPI4inMApnIMH11CDe6hDAxgM4Ble4c0Rzovz7nwsWgtOPnMMf+B8/gALrI2g < /latexit > v1 < latexit sha1_base64= '' iwb6UOVPBQMaW/UYkl1KcM9pxNQ= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0nf65crbtVdgKwTLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjks1IvNTyhbEyHvGupohE3frY4dUYurDIgYaxtKSQL9fdERiNjplFgOyOKI7PqzcX/vG6K4Y2fCZWkyBVbLgpTSTAm87/JQGjOUE4toUwLeythI6opQ5tOyYbgrb68TlpXVc+teg/XlfptHkcRzuAcLsGDGtThHhrQBAZDeIZXeHOk8+K8Ox/L1oKTz5zCHzifPwikjZ4= < /latexit > < latexit sha1_base64= '' iwb6UOVPBQMaW/UYkl1KcM9pxNQ= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0nf65crbtVdgKwTLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjks1IvNTyhbEyHvGupohE3frY4dUYurDIgYaxtKSQL9fdERiNjplFgOyOKI7PqzcX/vG6K4Y2fCZWkyBVbLgpTSTAm87/JQGjOUE4toUwLeythI6opQ5tOyYbgrb68TlpXVc+teg/XlfptHkcRzuAcLsGDGtThHhrQBAZDeIZXeHOk8+K8Ox/L1oKTz5zCHzifPwikjZ4= < /latexit > < latexit sha1_base64= '' iwb6UOVPBQMaW/UYkl1KcM9pxNQ= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0nf65crbtVdgKwTLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjks1IvNTyhbEyHvGupohE3frY4dUYurDIgYaxtKSQL9fdERiNjplFgOyOKI7PqzcX/vG6K4Y2fCZWkyBVbLgpTSTAm87/JQGjOUE4toUwLeythI6opQ5tOyYbgrb68TlpXVc+teg/XlfptHkcRzuAcLsGDGtThHhrQBAZDeIZXeHOk8+K8Ox/L1oKTz5zCHzifPwikjZ4= < /latexit > < latexit sha1_base64= '' iwb6UOVPBQMaW/UYkl1KcM9pxNQ= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0nf65crbtVdgKwTLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjks1IvNTyhbEyHvGupohE3frY4dUYurDIgYaxtKSQL9fdERiNjplFgOyOKI7PqzcX/vG6K4Y2fCZWkyBVbLgpTSTAm87/JQGjOUE4toUwLeythI6opQ5tOyYbgrb68TlpXVc+teg/XlfptHkcRzuAcLsGDGtThHhrQBAZDeIZXeHOk8+K8Ox/L1oKTz5zCHzifPwikjZ4= < /latexit > v4 < latexit sha1_base64= '' +c4psOKK6zsb24Fz44doEjLIVSY= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbSuqp5b9R5qlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwANMI2h < /latexit > < latexit sha1_base64= '' +c4psOKK6zsb24Fz44doEjLIVSY= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbSuqp5b9R5qlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwANMI2h < /latexit > < latexit sha1_base64= '' +c4psOKK6zsb24Fz44doEjLIVSY= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbSuqp5b9R5qlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwANMI2h < /latexit > < latexit sha1_base64= '' +c4psOKK6zsb24Fz44doEjLIVSY= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbSuqp5b9R5qlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwANMI2h < /latexit > v2 < latexit sha1_base64= '' UQ8RRPt2NN02WZInLZUJLciw6fw= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbRqVc+teg9XlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwAKKI2f < /latexit > < latexit sha1_base64= '' UQ8RRPt2NN02WZInLZUJLciw6fw= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbRqVc+teg9XlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwAKKI2f < /latexit > < latexit sha1_base64= '' UQ8RRPt2NN02WZInLZUJLciw6fw= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbRqVc+teg9XlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwAKKI2f < /latexit > < latexit sha1_base64= '' UQ8RRPt2NN02WZInLZUJLciw6fw= '' > AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoMeiF48V7Qe0oWy2m3bpZhN2J4US+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSMnGqGW+yWMa6E1DDpVC8iQIl7ySa0yiQvB2M7+Z+e8K1EbF6wmnC/YgOlQgFo2ilx0m/1i9X3Kq7AFknXk4qkKPRL3/1BjFLI66QSWpM13MT9DOqUTDJZ6VeanhC2ZgOeddSRSNu/Gxx6oxcWGVAwljbUkgW6u+JjEbGTKPAdkYUR2bVm4v/ed0Uwxs/EypJkSu2XBSmkmBM5n+TgdCcoZxaQpkW9lbCRlRThjadkg3BW315nbRqVc+teg9XlfptHkcRzuAcLsGDa6jDPTSgCQyG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwAKKI2f < /latexit > Figure 3 : Graphical models and infor- mation diagrams ( inf ) associated with the core view and full graph paradigms , for the case of 4 views , which gives a total of 6 learning objectives . The numbers within the regions show how much “ weight ” the total loss places on each partition of information ( i.e . how many of the 6 objectives that partition contributes to ) . A region with no number corresponds to 0 weight . For example , in the full graph case , the mutual information between all 4 views is considered in all 6 objectives , and hence is marked with the number 6 . Implementing the critic We implement the critic hθ ( · ) as a neural network . To extract compact latent representations of v1 and v2 , we employ two encoders fθ1 ( · ) and fθ2 ( · ) with parameters θ1 and θ2 respectively . The latent representions are extracted as z1 = fθ1 ( v1 ) , z2 = fθ2 ( v2 ) . On top of these features , the score is computed as the exponential of a bivariate function of z1 and z2 , e.g. , a bilinear function parameterized by W12 : hθ ( { v1 , v2 } ) = efθ1 ( v1 ) TW12fθ2 ( v2 ) ( 3 ) Loss LV1 , V2contrast in Eq . 2 treats view V1 as anchor and enumerates over V2 . Symmetrically , we can get LV2 , V1contrast by anchoring at V2 . We add them up as our two-view loss : L ( V1 , V2 ) = LV1 , V2contrast + LV2 , V1contrast ( 4 ) After the contrastive learning phase , we use the representation z1 , z2 , or the concatenation of both , [ z1 , z2 ] , depending on our paradigm . This process is visualized in Fig . 1 . Connecting to mutual information The optimal critic h∗θ is proportional to the density ratio between the joint distribution p ( z1 , z2 ) and the product of marginals p ( z1 ) p ( z2 ) ( proof provided in Sec . C.1 ) : h∗θ ( { v1 , v2 } ) ∝ p ( z1 , z2 ) p ( z1 ) p ( z2 ) ∝ p ( z1|z2 ) p ( z1 ) ( 5 ) This quantity is the pointwise mutual information , and its expectation , in Eq . 2 , yields an estimator related to mutual information . A formal proof is given by Oord et al . ( 2018 ) ; Poole et al . ( 2019 ) , which we recapitulate in Section C , showing that : I ( zi ; zj ) ≥ log ( k ) − Lcontrast ( 6 ) where , as above , k is the number of negative pairs in sample set S. Hence minimizing the objective L maximizes the lower bound on the mutual information I ( zi ; zj ) , which is bounded above by I ( vi ; vj ) by the data processing inequality . The dependency on k also suggests that using more negative samples can lead to an improved representation ; we show that this is indeed the case in Section A.1.2 . We note that recent work ( McAllester & Statos , 2018 ) shows that the bound in Eq . 6 can be very weak ; and finding better estimators of mutual information is an important open problem .

Summary: The paper presented a multi-view learning method that is based on negative sampling in contrastive learning. The core idea is to set an anchor view and the sample positive and negative data points from the other view and maximise the agreement between positive pairs in learning from two views. When more than two views are presented, the learning objective is a sum over all possible combinations of two views. The performance of the proposed model is good, and the ablation study is interesting.