Title: LEARNING TO PROPAGATE LABELS: TRANSDUCTIVE PROPAGATION NETWORK FOR FEW-SHOT LEARNING

Abstract: The goal of few-shot learning is to learn a classifier that generalizes well even when trained with a limited number of training instances per class. The recently introduced meta-learning approaches tackle this problem by learning a generic classifier across a large number of multiclass classification tasks and generalizing the model to a new task. Yet, even with such meta-learning, the low-data problem in the novel classification task still remains. In this paper, we propose Transductive Propagation Network (TPN), a novel meta-learning framework for transductive inference that classifies the entire test set at once to alleviate the low-data problem. Specifically, we propose to learn to propagate labels from labeled instances to unlabeled test instances, by learning a graph construction module that exploits the manifold structure in the data. TPN jointly learns both the parameters of feature embedding and the graph construction in an end-to-end manner. We validate TPN on multiple benchmark datasets, on which it largely outperforms existing few-shot learning approaches and achieves the state-of-the-art results.

1. INTRODUCTION:

Recent breakthroughs in deep learning (Krizhevsky et al., 2012; Simonyan and Zisserman, 2015; He et al., 2016) highly rely on the availability of large amounts of labeled data. However, this reliance on large data increases the burden of data collection, which hinders its potential applications to the low-data regime where the labeled data is rare and difficult to gather. On the contrary, humans have the ability to recognize new objects after observing only one or few instances (Lake et al., 2011) . For example, children can generalize the concept of "apple" after given a single instance of it. This significant gap between human and deep learning has reawakened the research interest on few-shot learning (Vinyals et al., 2016; Snell et al., 2017; Finn et al., 2017; Ravi and Larochelle, 2017; Lee and Choi, 2018; Xu et al., 2017; Wang et al., 2018) .

Few-shot learning aims to learn a classifier that generalizes well with a few examples of each of these classes. Traditional techniques such as fine-tuning (Jia et al., 2014 ) that work well with deep learning models would severely overfit on this task (Vinyals et al., 2016; Finn et al., 2017) , since a single or only a few labeled instances would not accurately represent the true data distribution and will result in learning classifiers with high variance, which will not generalize well to new data.

In order to solve this overfitting problem, Vinyals et al. (2016) proposed a meta-learning strategy which learns over diverse classification tasks over large number of episodes rather than only on the target classification task. In each episode, the algorithm learns the embedding of the few labeled examples (the support set), which can be used to predict classes for the unlabeled points (the query set) by distance in the embedding space. The purpose of episodic training is to mimic Figure 1 : A conceptual illustration of our transductive meta-learning framework, where lines between nodes represent graph connections and their colors represent the potential direction of label propagation. The neighborhood graph is episodic-wisely trained for transductive inference.

the real test environment containing few-shot support set and unlabeled query set. The consistency between training and test environment alleviates the distribution gap and improves generalization. This episodic meta-learning strategy, due to its generalization performance, has been adapted by many follow-up work on few-shot learning. Finn et al. (2017) learned a good initialization that can adapt quickly to the target tasks. Snell et al. (2017) used episodes to train a good representation and predict classes by computing Euclidean distance with respect to class prototypes.

Although episodic strategy is an effective approach for few-shot learning as it aims at generalizing to unseen classification tasks, the fundamental difficulty with learning with scarce data remains for a novel classification task. One way to achieve larger improvements with limited amount of training data is to consider relationships between instances in the test set and thus predicting them as a whole, which is referred to as transduction, or transductive inference. In previous work (Joachims, 1999; Zhou et al., 2004; Vapnik, 1999) , transductive inference has shown to outperform inductive methods which predict test examples one by one, especially in small training sets. One popular approach for transduction is to construct a network on both the labeled and unlabeled data, and propagate labels between them for joint prediction. However, the main challenge with such label propagation (and transduction) is that the label propagation network is often obtained without consideration of the main task, since it is not possible to learn them at the test time.

Yet, with the meta-learning by episodic training, we can learn the label propagation network as the query examples sampled from the training set can be used to simulate the real test set for transductive inference. Motivated by this finding, we propose Transductive Propagation Network (TPN) to deal with the low-data problem. Instead of applying the inductive inference, we utilize the entire query set for transductive inference (see Figure 1 ). Specifically, we first map the input to an embedding space using a deep neural network. Then a graph construction module is proposed to exploit the manifold structure of the novel class space using the union of support set and query set. According to the graph structure, iterative label propagation is applied to propagate labels from the support set to the query set and finally leads to a closed-form solution. With the propagated scores and ground truth labels of the query set, we compute the cross-entropy loss with respect to the feature embedding and graph construction parameters. Finally, all parameters can be updated end-to-end using backpropagation.

The main contribution of this work is threefold.

• To the best of our knowledge, we are the first to model transductive inference explicitly in few-shot learning. Although Nichol et al. (2018) experimented with a transductive setting, they only share information between test examples by batch normalization rather than directly proposing a transductive model.

• In transductive inference, we propose to learn to propagate labels between data instances for unseen classes via episodic meta-learning. This learned label propagation graph is shown to significantly outperform naive heuristic-based label propagation methods (Zhou et al., 2004) .

• We evaluate our approach on two benchmark datasets for few-shot learning, namely miniImageNet and tieredImageNet. The experimental results show that our Transductive Propagation Network outperforms the state-of-the-art methods on both datasets. Also, with semi-supervised learning, our algorithm achieves even higher performance, outperforming all semi-supervised few-shot learning baselines.

2. RELATED WORK:

Meta-learning In recent works, few-shot learning often follows the idea of metalearning (Schmidhuber, 1987; Thrun and Pratt, 2012) . Meta-learning tries to optimize over batches of tasks rather than batches of data points. Each task corresponds to a learning problem, obtaining good performance on these tasks helps to learn quickly and generalize well to the target few-shot problem without suffering from overfitting. The well-known MAML approach (Finn et al., 2017) aims to find more transferable representations with sensitive parameters. A first-order meta-learning approach named Reptile is proposed by Nichol et al. (2018) . It is closely related to first-order MAML but does not need a training-test split for each task. Compared with the above methods, our algorithm has a closed-form solution for label propagation on the query points, thus avoiding gradient computation in the inner updateand usually performs more efficiently.

Embedding and metric learning approaches Another category of few-shot learning approach aims to optimize the transferable embedding using metric learning approaches. Matching networks (Vinyals et al., 2016) produce a weighted nearest neighbor classifier given the support set and adjust feature embedding according to the performance on the query set. Prototypical networks (Snell et al., 2017) first compute a class's prototype to be the mean of its support set in the embedding space. Then the transferability of feature embedding is evaluated by finding the nearest class prototype for embedded query points. An extension of prototypical networks is proposed in Ren et al. (2018) to deal with semi-supervised few-shot learning. Relation Network (Sung et al., 2018) learns to learn a deep distance metric to compare a small number of images within episodes.

Our proposed method is similar to these approaches in the sense that we all focus on learning deep embeddings with good generalization ability. However, our algorithm assumes a transductive setting, in which we utilize the union of support set and query set to exploit the manifold structure of novel class space by using episodic-wise parameters.

Transduction The setting of transductive inference was first introduced by Vapnik (Vapnik, 1999) .

Transductive Support Vector Machines (TSVMs) (Joachims, 1999 ) is a margin-based classification method that minimizes errors of a particular test set. It shows substantial improvements over inductive methods, especially for small training sets. Another category of transduction methods involves graph-based methods (Zhou et al., 2004; Wang and Zhang, 2006; Rohrbach et al., 2013; Fu et al., 2015) . Label propagation is used in Zhou et al. (2004) to transfer labels from labeled to unlabeled data instances guided by the weighted graph. Label propagation is sensitive to variance parameter σ, so Linear Neighborhood Propagation (LNP) (Wang and Zhang, 2006) constructs approximated Laplacian matrix to avoid this issue. In Zhu and Ghahramani (2002) , minimum spanning tree heuristic and entropy minimization are used to learn the parameter σ. In all these prior work, the graph construction is done on a pre-defined feature space using manually selected hyperparamters since it is not possible to learn them at test time. Our approach, on the other hand, is able to learn the graph construction network since it is a meta-learning framework with episodic training, where at each episode we simulate the test set with a subset of the training set.

In few-shot learning, Nichol et al. (2018) experiments with a transductive setting and shows improvements. However, they only share information between test examples via batch normalization (Ioffe and Szegedy, 2015) rather than explicitly model the transductive setting as in our algorithm.

CNN CNN:



Support:



Query:

f ' < l a t e x i t s h a 1 _ b a s e 6 4 = " m w X g 3 2 Y u u c w n / q u 3 K

N Z b c H 2 5 U 3 Y = " > A A A B 8 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 8 E i e C q J C H o s e v F Y w X 5 A G s p m u 2 m X b n b D 7 q R Q Q n + G F w + K e P X X e P P f u G 1 z 0 N Y X F h 7 e m W F n 3 i g V 3 K D n f T u l j c 2 t 7 Z 3 y b m V v / + D w q H p 8 0 j Y q 0 5 S 1 q B J K d y N i m O C S t Z C j Y N 1 U M 5 J E g n W i 8 f 2 8 3 p k w b b i S T z h N W Z i Q o e Q x p w S t F c T 9 v D c h O h 3 x W b 9 a 8 + r e Q u 4 6 + A X U o F C z X / 3 q D R T N E i a R C m J M 4 H s p h j n R y K l g s 0 o v M y w l d E y G L L A o S c J M m C 9 W n r k X 1 h m 4 s d L 2 S X Q X 7 u + J n C T G T J P I d i Y E R 2 a 1 N j f / q w U Z x r d h z m W a I Z N 0 + V G c C R e V O 7 / f H X D N K I q p B U I 1 t 7 u 6 d E Q 0 o W h T q t g Q / N W T 1 6 F 9 V f c t P 1 7 X G n d F H G U 4 g 3 O 4 B B 9 u o A E P 0 I Q W U F D w D K / w 5 q D z 4 r w 7 H 8 v W k l P M n M I f O Z 8 / v 8 q R i w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " m w X g 3 2 Y u u c w n / q u 3 K N Z b c H 2 5 U 3 Y = " > A A A B 8 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 8 E i e C q J C H o s e v F Y w X 5 A G s p m u 2 m X b n b D 7 q R Q Q n + G F w + K e P X X e P P f u G 1 z 0 N Y X F h 7 e m W F n 3 i g V 3 K D n f T u l j c 2 t 7 Z 3 y b m V v / + D w q H p 8 0 j Y q 0 5 S 1 q B J K d y N i m O C S t Z C j Y N 1 U M 5 J E g n W i 8 f 2 8 3 p k w b b i S T z h N W Z i Q o e Q x p w S t F c T 9 v D c h O h 3 x W b 9 a 8 + r e Q u 4 6 + A X U o F C z X / 3 q D R T N E i a R C m J M 4 H s p h j n R y K l g s 0 o v M y w l d E y G L L A o S c J M m C 9 W n r k X 1 h m 4 s d L 2 S X Q X 7 u + J n C T G T J P I d i Y E R 2 a 1 N j f / q w U Z x r d h z m W a I Z N 0 + V G c C R e V O 7 / f H X D N K I q p B U I 1 t 7 u 6 d E Q 0 o W h T q t g Q / N W T 1 6 F 9 V f c t P 1 7 X G n d F H G U 4 g 3 O 4 B B 9 u o A E P 0 I Q W U F D w D K / w 5 q D z 4 r w 7 H 8 v W k l P M n M I f O Z 8 / v 8 q R i w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " m w X g 3 2 Y u u c w n / q u 3 K N Z b c H 2 5 U 3 Y = " > A A A B 8 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 8 E i e C q J C H o s e v F Y w X 5 A G s p m u 2 m X b n b D 7 q R Q Q n + G F w + K e P X X e P P f u G 1 z 0 N Y X F h 7 e m W F n 3 i g V 3 K D n f T u l j c 2 t 7 Z 3 y b m V v / + D w q H p 8 0 j Y q 0 5 S 1 q B J K d y N i m O C S t Z C j Y N 1 U M 5 J E g n W i 8 f 2 8 3 p k w b b i S T z h N W Z i Q o e Q x p w S t F c T 9 v D c h O h 3 x W b 9 a 8 + r e Q u 4 6 + A X U o F C z X / 3 q D R T N E i a R C m J M 4 H s p h j n R y K l g s 0 o v M y w l d E y G L L A o S c J M m C 9 W n r k X 1 h m 4 s d L 2 S X Q X 7 u + J n C T G T J P I d i Y E R 2 a 1 N j f / q w U Z x r d h z m W a I Z N 0 + V G c C R e V O 7 / f H X D N K I q p B U I 1 t 7 u 6 d E Q 0 o W h T q t g Q / N W T 1 6 F 9 V f c t P 1 7 X G n d F H G U 4 g 3 O 4 B B 9 u o A E P 0 I Q W U F D w D K / w 5 q D z 4 r w 7 H 8 v W k l P M n M I f O Z 8 / v 8 q R i w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " m w X g 3 2 Y u u c w n / q u 3 K N Z b c H 2 5 U 3 Y = " > A A A B 8 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 8 E i e C q J C H o s e v F Y w X 5 A G s p m u 2 m X b n b D 7 q R Q Q n + G F w + K e P X X e P P f u G 1 z 0 N Y X F h 7 e m W F n 3 i g V 3 K D n f T u l j c 2 t 7 Z 3 y b m V v / + D w q H p 8 0 j Y q 0 5 S 1 q B J K d y N i m O C S t Z C j Y N 1 U M 5 J E g n W i 8 f 2 8 3 p k w b b i S T z h N W Z i Q o e Q x p w S t F c T 9 v D c h O h 3 x W b 9 a 8 + r e Q u 4 6 + A X U o F C z X / 3 q D R T N E i a R C m J M 4 H s p h j n R y K l g s 0 o v M y w l d E y G L L A o S c J M m C 9 W n r k X 1 h m 4 s d L 2 S X Q X 7 u + J n C T G T J P I d i Y E R 2 a 1 N j f / q w U Z x r d h z m W a I Z N 0 + V G c C R e V O 7 / f H X D N K I q p B U I 1 t 7 u 6 d E Q 0 o W h T q t g Q / N W T 1 6 F 9 V f c t P 1 7 X G n d F H G U 4 g 3 O 4 B B 9 u o A E P 0 I Q W U F D w D K / w 5 q D z 4 r w 7 H 8 v W k l P M n M I f O Z 8 / v 8 q R i w = = < / l a t e x i t > f ' (X)

< l a t e x i t s h a 1 _ b a s e 6 4 = " T r 0 q V k L p z 6

J j A t L l N d U 2 2 d c m v k I = " > A A A B 9 X i c b Z B N S w M x E I Z n 6 1 e t X 1 W P X o J F q J e y K 4 I e i 1 4 8 V r A f 0 N a S T b N t a D a 7 J L O V s v R / e P G g i F f / i z f / j W m 7 B 2 1 9 I f D w z g w z e f 1 Y C o O u + + 3 k 1 t Y 3 N r f y 2 4 W d 3 b 3 9 g + L h U c N E i W a 8 z i I Z 6 Z Z P D Z d C 8 T o K l L w V a 0 5 D X / K m P 7 q d 1 Z t j r o 2 I 1 A N O Y t 4 N 6 U C J Q D C K 1 n o M e m l n T H U 8 F N N y 6 7 x X L L k V d y 6 y C l 4 G J c h U 6 x W / O v 2 I J S F X y C Q 1 p u 2 5 M X Z T q l E w y a e F T m J 4 T N m I D n j b o q I h N 9 1 0 f v W U n F m n T 4 J I 2 6 e Q z N 3 f E y k N j Z m E v u 0 M K Q 7 N c m 1 m / l d r J x h c d 1 O h 4 g S 5 Y o t F Q S I J R m Q W A e k L z R n K i Q X K t L C 3 E j a k m j K 0 Q R V s C N 7 y l 1 e h c V H x L N 9 f l q o 3 W R x 5 O I F T K I M H V 1 C F O 6 h B H R h o e I Z X e H O e n B f n 3 f l Y t O a c b O Y Y / s j 5 / A E 9 i 5 J S < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " T r 0 q V k L p z 6 J j A t L l N d U 2 2 d c m v k I = " > A A A B 9 X i c b Z B N S w M x E I Z n 6 1 e t X 1 W P X o J F q J e y K 4 I e i 1 4 8 V r A f 0 N a S T b N t a D a 7 J L O V s v R / e P G g i F f / i z f / j W m 7 B 2 1 9 I f D w z g w z e f 1 Y C o O u + + 3 k 1 t Y 3 N r f y 2 4 W d 3 b 3 9 g + L h U c N E i W a 8 z i I Z 6 Z Z P D Z d C 8 T o K l L w V a 0 5 D X / K m P 7 q d 1 Z t j r o 2 I 1 A N O Y t 4 N 6 U C J Q D C K 1 n o M e m l n T H U 8 F N N y 6 7 x X L L k V d y 6 y C l 4 G J c h U 6 x W / O v 2 I J S F X y C Q 1 p u 2 5 M X Z T q l E w y a e F T m J 4 T N m I D n j b o q I h N 9 1 0 f v W U n F m n T 4 J I 2 6 e Q z N 3 f E y k N j Z m E v u 0 M K Q 7 N c m 1 m / l d r J x h c d 1 O h 4 g S 5 Y o t F Q S I J R m Q W A e k L z R n K i Q X K t L C 3 E j a k m j K 0 Q R V s C N 7 y l 1 e h c V H x L N 9 f l q o 3 W R x 5 O I F T K I M H V 1 C F O 6 h B H R h o e I Z X e H O e n B f n 3 f l Y t O a c b O Y Y / s j 5 / A E 9 i 5 J S < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " T r 0 q V k L p z 6 J j A t L l N d U 2 2 d c m v k I = " > A A A B 9 X i c b Z B N S w M x E I Z n 6 1 e t X 1 W P X o J F q J e y K 4 I e i 1 4 8 V r A f 0 N a S T b N t a D a 7 J L O V s v R / e P G g i F f / i z f / j W m 7 B 2 1 9 I f D w z g w z e f 1 Y C o O u + + 3 k 1 t Y 3 N r f y 2 4 W d 3 b 3 9 g + L h U c N E i W a 8 z i I Z 6 Z Z P D Z d C 8 T o K l L w V a 0 5 D X / K m P 7 q d 1 Z t j r o 2 I 1 A N O Y t 4 N 6 U C J Q D C K 1 n o M e m l n T H U 8 F N N y 6 7 x X L L k V d y 6 y C l 4 G J c h U 6 x W / O v 2 I J S F X y C Q 1 p u 2 5 M X Z T q l E w y a e F T m J 4 T N m I D n j b o q I h N 9 1 0 f v W U n F m n T 4 J I 2 6 e Q z N 3 f E y k N j Z m E v u 0 M K Q 7 N c m 1 m / l d r J x h c d 1 O h 4 g S 5 Y o t F Q S I J R m Q W A e k L z R n K i Q X K t L C 3 E j a k m j K 0 Q R V s C N 7 y l 1 e h c V H x L N 9 f l q o 3 W R x 5 O I F T K I M H V 1 C F O 6 h B H R h o e I Z X e H O e n B f n 3 f l Y t O a c b O Y Y / s j 5 / A E 9 i 5 J S < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " T r 0 q V k L p z 6 J j A t L l N d U 2 2 d c m v k I = " > A A A B 9 X i c b Z B N S w M x E I Z n 6 1 e t X 1 W P X o J F q J e y K 4 I e i 1 4 8 V r A f 0 N a S T b N t a D a 7 J L O V s v R / e P G g i F f / i z f / j W m 7 B 2 1 9 I f D w z g w z e f 1 Y C o O u + + 3 k 1 t Y 3 N r f y 2 4 W d 3 b 3 9 g + L h U c N E i W a 8 z i I Z 6 Z Z P D Z d C 8 T o K l L w V a 0 5 D X / K m P 7 q d 1 Z t j r o 2 I 1 A N O Y t 4 N 6 U C J Q D C K 1 n o M e m l n T H U 8 F N N y 6 7 x X L L k V d y 6 y C l 4 G J c h U 6 x W / O v 2 I J S F X y C Q 1 p u 2 5 M X Z T q l E w y a e F T m J 4 T N m I D n j b o q I h N 9 1 0 f v W U n F m n T 4 J I 2 6 e Q z N 3 f E y k N j Z m E v u 0 M K Q 7 N c m 1 m / l d r J x h c d 1 O h 4 g S 5 Y o t F Q S I J R m Q W A e k L z R n K i Q X K t L C 3 E j a k m j K 0 Q R V s C N 7 y l 1 e h c V H x L N 9 f l q o 3 W R x 5 O I F T K I M H V 1 C F O 6 h B H R h o e I Z X e H O e n B f n 3 f l Y t O a c b O Y Y / s j 5 / A E 9 i 5 J S < / l a t e x i t >

< l a t e x i t s h a 1 _ b a s e 6 4 = " d w m 2 w e u i / z a / r S v U

Q 7 3 K / R k 4 / 5 k = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 1 G X b o J F c F V m R N B l 0 Y 3 L C v Y C 7 V A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x g b B t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P / M O j l l G Z J r R J F F e 6 E 2 N D O Z O 0 a Z n l t J N q i k X M a T s e 3 8 7 q 7 S e q D V P y w U 5 S G g k 8 l C x h B F t n t X q G D Q X u + 9 W g F s y F V i E s o A q F G n 3 / q z d Q J B N U W s K x M d 0 w S G 2 U Y 2 0 Z 4 X R a 6 W W G p p i M 8 Z B 2 H U o s q I n y + b Z T d O a c A U q U d k 9 a N H d / T + R Y G D M R s e s U 2 I 7 M c m 1 m / l f r Z j a 5 j n I m 0 8 x S S R Y f J R l H V q H Z 6 W j A N C W W T x x g o p n b F Z E R 1 p h Y F 1 D F h R A u n 7 w K r Y t a 6 P j + s l q / K e I o w w m c w j m E c A V 1 u I M G N I H A I z z D K 7 x 5 y n v x 3 r 2 P R W v J K 2 a O 4 Y + 8 z x + c

Y Y 8 j < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " d w m 2 w e u i / z a / r S v U

Q 7 3 K / R k 4 / 5 k = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 1 G X b o J F c F V m R N B l 0 Y 3 L C v Y C 7 V A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x g b B t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P / M O j l l G Z J r R J F F e 6 E 2 N D O Z O 0 a Z n l t J N q i k X M a T s e 3 8 7 q 7 S e q D V P y w U 5 S G g k 8 l C x h B F t n t X q G D Q X u + 9 W g F s y F V i E s o A q F G n 3 / q z d Q J B N U W s K x M d 0 w S G 2 U Y 2 0 Z 4 X R a 6 W W G p p i M 8 Z B 2 H U o s q I n y + b Z T d O a c A U q U d k 9 a N H d / T + R Y G D M R s e s U 2 I 7 M c m 1 m / l f r Z j a 5 j n I m 0 8 x S S R Y f J R l H V q H Z 6 W j A N C W W T x x g o p n b F Z E R 1 p h Y F 1 D F h R A u n 7 w K r Y t a 6 P j + s l q / K e I o w w m c w j m E c A V 1 u I M G N I H A I z z D K 7 x 5 y n v x 3 r 2 P R W v J K 2 a O 4 Y + 8 z x + c

Y Y 8 j < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " d w m 2 w e u i / z a / r S v U

Q 7 3 K / R k 4 / 5 k = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 1 G X b o J F c F V m R N B l 0 Y 3 L C v Y C 7 V A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x g b B t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P / M O j l l G Z J r R J F F e 6 E 2 N D O Z O 0 a Z n l t J N q i k X M a T s e 3 8 7 q 7 S e q D V P y w U 5 S G g k 8 l C x h B F t n t X q G D Q X u + 9 W g F s y F V i E s o A q F G n 3 / q z d Q J B N U W s K x M d 0 w S G 2 U Y 2 0 Z 4 X R a 6 W W G p p i M 8 Z B 2 H U o s q I n y + b Z T d O a c A U q U d k 9 a N H d / T + R Y G D M R s e s U 2 I 7 M c m 1 m / l f r Z j a 5 j n I m 0 8 x S S R Y f J R l H V q H Z 6 W j A N C W W T x x g o p n b F Z E R 1 p h Y F 1 D F h R A u n 7 w K r Y t a 6 P j + s l q / K e I o w w m c w j m E c A V 1 u I M G N I H A I z z D K 7 x 5 y n v x 3 r 2 P R W v J K 2 a O 4 Y + 8 z x + c

Y Y 8 j < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " d w m 2 w e u i / z a / r S v U

Q 7 3 K / R k 4 / 5 k = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 1 G X b o J F c F V m R N B l 0 Y 3 L C v Y C 7 V A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x g b B t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P / M O j l l G Z J r R J F F e 6 E 2 N D O Z O 0 a Z n l t J N q i k X M a T s e 3 8 7 q 7 S e q D V P y w U 5 S G g k 8 l C x h B F t n t X q G D Q X u + 9 W g F s y F V i E s o A q F G n 3 / q z d Q J B N U W s K x M d 0 w S G 2 U Y 2 0 Z 4 X R a 6 W W G p p i M 8 Z B 2 H U o s q I n y + b Z T d O a c A U q U d k 9 a N H d / T + R Y G D M R s e s U 2 I 7 M c m 1 m / l f r Z j a 5 j n I m 0 8 x S S R Y f J R l H V q H Z 6 W j A N C W W T x x g o p n b F Z E R 1 p h Y F 1 D F h R A u n 7 w K r Y t a 6 P j + s l q / K e I o w w m c w j m E c A V 1 u I M G N I H A I z z D K 7 x 5 y n v x 3 r 2 P R W v J K 2 a O 4 Y + 8 z x + c Y Y 8 j < / l a t e x i t > y < l a t e x i t s h a 1 _ b a s e 6 4 = " P V f i O Q D J D y 8 v d U 8 u 5 C G D 6 x q M r D 4 = " > A A A B 8 X i c b V D L S s N A F L 2 p r 1 p f V Z d u B o v g q i Q i 6 L L o x m U F + 8 A 2 l M l 0 0 g 6 d T M L M j V B C / 8 K N C 0 X c + j f u / B s n b R b a e m D g c M 6 9 z L k n S K Q w 6 L r f T m l t f W N z q 7 x d 2 d n d 2 z + o H h 6 1 T Z x q x l s s l r H u B t R w K R R v o U D J u 4 n m N A o k 7 w S T 2 9 z v P H F t R K w e c J p w P 6 I j J U L B K F r p s R 9 R H A d h N p 0 N q j W 3 7 s 5 B V o l X k B o U a A 6 q X / 1 h z N K I K 2 S S G t P z 3 A T 9 j G o U T P J Z p Z 8 a n l A 2 o S P e s 1 T R i B s / m y e e k T O r D E k Y a / s U k r n 6 e y O j k T H T K L C T e U K z 7 O X i f 1 4 v x f D a z 4 R K U u S K L T 4 K U 0 k w J v n 5 Z C g 0 Z y i n l l C m h c 1 K 2 J h q y t C W V L

E l e M s n r 5 L 2 R d 2 z / P 6 y 1 r g p 6 i j D C Z z C O X h w B Q 2 4 g y a 0 g I G C Z 3 i F N 8 c 4 L 8 6 7 8 7 E Y L T n F z j H 8 g f P 5 A / 7 A k R 0 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " P

V f i O Q D J D y 8 v d U 8 u 5 C G D 6 x q M r D 4 = " > A A A B 8 X i c b V D L S s N A F L 2 p r 1 p f V Z d u B o v g q i Q i 6 L L o x m U F + 8 A 2 l M l 0 0 g 6 d T M L M j V B C / 8 K N C 0 X c + j f u / B s n b R b a e m D g c M 6 9 z L k n S K Q w 6 L r f T m l t f W N z q 7 x d 2 d n d 2 z + o H h 6 1 T Z x q x l s s l r H u B t R w K R R v o U D J u 4 n m N A o k 7 w S T 2 9 z v P H F t R K w e c J p w P 6 I j J U L B K F r p s R 9 R H A d h N p 0 N q j W 3 7 s 5 B V o l X k B o U a A 6 q X / 1 h z N K I K 2 S S G t P z 3 A T 9 j G o U T P J Z p Z 8 a n l A 2 o S P e s 1 T R i B s / m y e e k T O r D E k Y a / s U k r n 6 e y O j k T H T K L C T e U K z 7 O X i f 1 4 v x f D a z 4 R K U u S K L T 4 K U 0 k w J v n 5 Z C g 0 Z y i n l l C m h c 1 K 2 J h q y t C W V L

E l e M s n r 5 L 2 R d 2 z / P 6 y 1 r g p 6 i j D C Z z C O X h w B Q 2 4 g y a 0 g I G C Z 3 i F N 8 c 4 L 8 6 7 8 7 E Y L T n F z j H 8 g f P 5 A / 7 A k R 0 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " P

V f i O Q D J D y 8 v d U 8 u 5 C G D 6 x q M r D 4 = " > A A A B 8 X i c b V D L S s N A F L 2 p r 1 p f V Z d u B o v g q i Q i 6 L L o x m U F + 8 A 2 l M l 0 0 g 6 d T M L M j V B C / 8 K N C 0 X c + j f u / B s n b R b a e m D g c M 6 9 z L k n S K Q w 6 L r f T m l t f W N z q 7 x d 2 d n d 2 z + o H h 6 1 T Z x q x l s s l r H u B t R w K R R v o U D J u 4 n m N A o k 7 w S T 2 9 z v P H F t R K w e c J p w P 6 I j J U L B K F r p s R 9 R H A d h N p 0 N q j W 3 7 s 5 B V o l X k B o U a A 6 q X / 1 h z N K I K 2 S S G t P z 3 A T 9 j G o U T P J Z p Z 8 a n l A 2 o S P e s 1 T R i B s / m y e e k T O r D E k Y a / s U k r n 6 e y O j k T H T K L C T e U K z 7 O X i f 1 4 v x f D a z 4 R K U u S K L T 4 K U 0 k w J v n 5 Z C g 0 Z y i n l l C m h c 1 K 2 J h q y t C W V L

E l e M s n r 5 L 2 R d 2 z / P 6 y 1 r g p 6 i j D C Z z C O X h w B Q 2 4 g y a 0 g I G C Z 3 i F N 8 c 4 L 8 6 7 8 7 E Y L T n F z j H 8 g f P 5 A / 7 A k R 0 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " P

V f i O Q D J D y 8 v d U 8 u 5 C G D 6 x q M r D 4 = " > A A A B 8 X i c b V D L S s N A F L 2 p r 1 p f V Z d u B o v g q i Q i 6 L L o x m U F + 8 A 2 l M l 0 0 g 6 d T M L M j V B C / 8 K N C 0 X c + j f u / B s n b R b a e m D g c M 6 9 z L k n S K Q w 6 L r f T m l t f W N z q 7 x d 2 d n d 2 z + o H h 6 1 T Z x q x l s s l r H u B t R w K R R v o U D J u 4 n m N A o k 7 w S T 2 9 z v P H F t R K w e c J p w P 6 I j J U L B K F r p s R 9 R H A d h N p 0 N q j W 3 7 s 5 B V o l X k B o U a A 6 q X / 1 h z N K I K 2 S S G t P z 3 A T 9 j G o U T P J Z p Z 8 a n l A 2 o S P e s 1 T R i B s / m y e e k T O r D E k Y a / s U k r n 6 e y O j k T H T K L C T e U K z 7 O X i f 1 4 v x f D a z 4 R K U u S K L T 4 K U 0 k w J v n 5 Z C g 0 Z y i n l l C m h c 1 K 2 J h q y t C W V L E l e M s n r 5 L 2 R d 2 z / P 6 y 1 r g p 6 i j D C Z z C O X h w B Q 2 4 g y a 0 g I G C Z 3 i F N 8 c 4 L 8 6 7 8 7 E Y L T n F z j H 8 g f P 5 A / 7 A k R 0 = < / l a t e x i t > g < l a t e x i t s h a 1 _ b a s e 6 4 = " A + 3 b a O 5 b X 4 F T h u l Q S 6 + 7 j H X G o n s = " > A A A B 7 3 i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x g m k L b S i b 7 a R d u t n E 3 Y 1 Q Q v + E F w + K e P X v e P P f u G 1 z 0 N Y X F h 7 e m W F n 3 j A V X B v X / X Z K a + s b m 1 v l 7 c r O 7 t 7 + Q f X w q K W T T D H 0 W S I S 1 Q m p R s E l + o Y b g Z 1 U I Y 1 D g e 1 w f D u r t 5 9 Q a Z 7 I B z N J M Y j p U P K I M 2 q s 1 R n 2 8 1 4 6 4 t N + t e b W 3 b n I K n g F 1 K B Q s 1 / 9 6 g 0 S l s U o D R N U 6 6 7 n p i b I q T K c C Z x W e p n G l L I x H W L X o q Q x 6 i C f 7 z s l Z 9 Y Z k C h R 9 k l D 5 u 7 v i Z z G W k / i 0 H b G 1 I z 0 c m 1 m / l f r Z i a 6 D n I u 0 8 y g Z I u P o k w Q k 5 D Z 8 W T A F T I j J h Y o U 9 z u S t i I K s q M j a h i Q / C W T 1 6 F 1 k X d s 3 x / W W v c F H G U 4 Q R O 4 R w 8 u I I G 3 E E T f G A g 4 B l e 4 c 1 5 d F 6 c d + d j 0 V p y i p l j + C P n 8 w d U c p A l < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " A + 3 b a O 5 b X 4 F T h u l Q S 6 + 7 j H X G o n s = " > A A A B 7 3 i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x g m k L b S i b 7 a R d u t n E 3 Y 1 Q Q v + E F w + K e P X v e P P f u G 1 z 0 N Y X F h 7 e m W F n 3 j A V X B v X / X Z K a + s b m 1 v l 7 c r O 7 t 7 + Q f X w q K W T T D H 0 W S I S 1 Q m p R s E l + o Y b g Z 1 U I Y 1 D g e 1 w f D u r t 5 9 Q a Z 7 I B z N J M Y j p U P K I M 2 q s 1 R n 2 8 1 4 6 4 t N + t e b W 3 b n I K n g F 1 K B Q s 1 / 9 6 g 0 S l s U o D R N U 6 6 7 n p i b I q T K c C Z x W e p n G l L I x H W L X o q Q x 6 i C f 7 z s l Z 9 Y Z k C h R 9 k l D 5 u 7 v i Z z G W k / i 0 H b G 1 I z 0 c m 1 m / l f r Z i a 6 D n I u 0 8 y g Z I u P o k w Q k 5 D Z 8 W T A F T I j J h Y o U 9 z u S t i I K s q M j a h i Q / C W T 1 6 F 1 k X d s 3 x / W W v c F H G U 4 Q R O 4 R w 8 u I I G 3 E E T f G A g 4 B l e 4 c 1 5 d F 6 c d + d j 0 V p y i p l j + C P n 8 w d U c p A l < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " A + 3 b a O 5 b X 4 F T h u l Q S 6 + 7 j H X G o n s = " > A A A B 7 3 i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x g m k L b S i b 7 a R d u t n E 3 Y 1 Q Q v + E F w + K e P X v e P P f u G 1 z 0 N Y X F h 7 e m W F n 3 j A V X B v X / X Z K a + s b m 1 v l 7 c r O 7 t 7 + Q f X w q K W T T D H 0 W S I S 1 Q m p R s E l + o Y b g Z 1 U I Y 1 D g e 1 w f D u r t 5 9 Q a Z 7 I B z N J M Y j p U P K I M 2 q s 1 R n 2 8 1 4 6 4 t N + t e b W 3 b n I K n g F 1 K B Q s 1 / 9 6 g 0 S l s U o D R N U 6 6 7 n p i b I q T K c C Z x W e p n G l L I x H W L X o q Q x 6 i C f 7 z s l Z 9 Y Z k C h R 9 k l D 5 u 7 v i Z z G W k / i 0 H b G 1 I z 0 c m 1 m / l f r Z i a 6 D n I u 0 8 y g Z I u P o k w Q k 5 D Z 8 W T A F T I j J h Y o U 9 z u S t i I K s q M j a h i Q / C W T 1 6 F 1 k X d s 3 x / W W v c F H G U 4 Q R O 4 R w 8 u I I G 3 E E T f G A g 4 B l e 4 c 1 5 d F 6 c d + d j 0 V p y i p l j + C P n 8 w d U c p A l < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " A + 3 b a O 5 b X 4 F T h u l Q S 6 + 7 j H X G o n s = " > A A A B 7 3 i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x g m k L b S i b 7 a R d u t n E 3 Y 1 Q Q v + E F w + K e P X v e P P f u G 1 z 0 N Y X F h 7 e m W F n 3 j A V X B v X / X Z K a + s b m 1 v l 7 c r O 7 t 7 + Q f X w q K W T T D H 0 W S I S 1 Q m p R s E l + o Y b g Z 1 U I Y 1 D g e 1 w f D u r t 5 9 Q a Z 7 I B z N J M Y j p U P K I M 2 q s 1 R n 2 8 1 4 6 4 t N + t e b W 3 b n I K n g F 1 K B Q s 1 / 9 6 g 0 S l s U o D R N U 6 6 7 n p i b I q T K c C Z x W e p n G l L I x H W L X o q Q x 6 i C f 7 z s l Z 9 Y Z k C h R 9 k l D 5 u 7 v i Z z G W k / i 0 H b G 1 I z 0 c m 1 m / l f r Z i a 6 D n I u 0 8 y g Z I u P o k w Q k 5 D Z 8 W T A F T I j J h Y o U 9 z u S t i I K s q M j a h i Q / C W T 1 6 F 1 k X d s 3 x / W W v c F H G U 4 Q R O 4 R w 8 u I I G 3 E E T f G A g 4 B l e 4 c 1 5 d F 6 c d + d j 0 V p y i p l j + C P n 8 w d U c p A l < / l a t e x i t > Wij = exp 1 2 d( f'(xi) i , f'(xj) j ) < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 f G m E Y 8 1 k 5 c 1 a x z 7 o V e d A z B L O z k = " > A A A C d n i c f V F N a x s x E N V u 0 y Z 1 P + K 2 p 1 I I I i b U h i b s h k J 7 C Y T 0 0 m M C d R y w z K K V Z 2 0 5 2 g + k 2 W A j 9 B P y 5 3 r r 7 + g l x 2 p t l 6 Z J y I D g 8 e Y 9 R v M m r Z Q 0 G E W / g v D J x t N n m 1 v P W y 9 e v n q 9 3 X 7 z 9 t y U t R b Q F 6 U q 9 U X K D S h Z Q B 8 l K r i o N P A 8 V T B I L 7 8 1 / c E V a C P L 4 g c u K h j l f F L I T A q O n k r a 1 y z n O E 0 z O 3 C J l T N H j y i D e c V S O e n u s 0 x z Y W N n D x 1 D m K O X j V 1 3 x f 7 1 Z d 7 H r r i u p t J 1 5 4 n s O c u M n O Q 8 k e 7 T o 9 L Z P + n M 9 Z q J v a T d i Q 6 i Z d H 7 I F 6 D D l n X a d L + y c a l q H M o U C h u z D C O K h x Z r l E K B a 7 F a g M V F 5 d 8 A k M P C 5 6 D G d l l b I 7 u e W Z M s 1 L 7 V y B d s r c d l u f G L P L U K 5 s N z N 1 e Q z 7 U G 9 a Y f R 1 Z W V Q 1 Q i F W g 7 J a U S x p c w M 6 l h o E q o U H X G j p / 0 r F l P u o 0 F + q 5 U O I 7 6 5 8 H 5 w f H s Q e n 3 3 u H J + s 4 9 g i H 8 g u 6 Z K Y f C H H 5 D s 5 J X 0 i y O / g f b A b d I K b c C f c C z + u p G G w 9 r w j / 1 U Y / Q E d d M J 4 < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 f G m E Y 8 1 k 5 c 1 a x z 7 o V e d A z B L O z k = " > A A A C d n i c f V F N a x s x E N V u 0 y Z 1 P + K 2 p 1 I I I i b U h i b s h k J 7 C Y T 0 0 m M C d R y w z K K V Z 2 0 5 2 g + k 2 W A j 9 B P y 5 3 r r 7 + g l x 2 p t l 6 Z J y I D g 8 e Y 9 R v M m r Z Q 0 G E W / g v D J x t N n m 1 v P W y 9 e v n q 9 3 X 7 z 9 t y U t R b Q F 6 U q 9 U X K D S h Z Q B 8 l K r i o N P A 8 V T B I L 7 8 1 / c E V a C P L 4 g c u K h j l f F L I T A q O n k r a 1 y z n O E 0 z O 3 C J l T N H j y i D e c V S O e n u s 0 x z Y W N n D x 1 D m K O X j V 1 3 x f 7 1 Z d 7 H r r i u p t J 1 5 4 n s O c u M n O Q 8 k e 7 T o 9 L Z P + n M 9 Z q J v a T d i Q 6 i Z d H 7 I F 6 D D l n X a d L + y c a l q H M o U C h u z D C O K h x Z r l E K B a 7 F a g M V F 5 d 8 A k M P C 5 6 D G d l l b I 7 u e W Z M s 1 L 7 V y B d s r c d l u f G L P L U K 5 s N z N 1 e Q z 7 U G 9 a Y f R 1 Z W V Q 1 Q i F W g 7 J a U S x p c w M 6 l h o E q o U H X G j p / 0 r F l P u o 0 F + q 5 U O I 7 6 5 8 H 5 w f H s Q e n 3 3 u H J + s 4 9 g i H 8 g u 6 Z K Y f C H H 5 D s 5 J X 0 i y O / g f b A b d I K b c C f c C z + u p G G w 9 r w j / 1 U Y / Q E d d M J 4 < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 f G m E Y 8 1 k 5 c 1 a x z 7 o V e d A z B L O z k = " > A A A C d n i c f V F N a x s x E N V u 0 y Z 1 P + K 2 p 1 I I I i b U h i b s h k J 7 C Y T 0 0 m M C d R y w z K K V Z 2 0 5 2 g + k 2 W A j 9 B P y 5 3 r r 7 + g l x 2 p t l 6 Z J y I D g 8 e Y 9 R v M m r Z Q 0 G E W / g v D J x t N n m 1 v P W y 9 e v n q 9 3 X 7 z 9 t y U t R b Q F 6 U q 9 U X K D S h Z Q B 8 l K r i o N P A 8 V T B I L 7 8 1 / c E V a C P L 4 g c u K h j l f F L I T A q O n k r a 1 y z n O E 0 z O 3 C J l T N H j y i D e c V S O e n u s 0 x z Y W N n D x 1 D m K O X j V 1 3 x f 7 1 Z d 7 H r r i u p t J 1 5 4 n s O c u M n O Q 8 k e 7 T o 9 L Z P + n M 9 Z q J v a T d i Q 6 i Z d H 7 I F 6 D D l n X a d L + y c a l q H M o U C h u z D C O K h x Z r l E K B a 7 F a g M V F 5 d 8 A k M P C 5 6 D G d l l b I 7 u e W Z M s 1 L 7 V y B d s r c d l u f G L P L U K 5 s N z N 1 e Q z 7 U G 9 a Y f R 1 Z W V Q 1 Q i F W g 7 J a U S x p c w M 6 l h o E q o U H X G j p / 0 r F l P u o 0 F + q 5 U O I 7 6 5 8 H 5 w f H s Q e n 3 3 u H J + s 4 9 g i H 8 g u 6 Z K Y f C H H 5 D s 5 J X 0 i y O / g f b A b d I K b c C f c C z + u p G G w 9 r w j / 1 U Y / Q E d d M J 4 < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 f G m E Y 8 1 k 5 c 1 a x z 7 o V e d A z B L O z k = " > A A A C d n i c f V F N a x s x E N V u 0 y Z 1 P + K 2 p 1 I I I i b U h i b s h k J 7 C Y T 0 0 m M C d R y w z K K V Z 2 0 5 2 g + k 2 W A j 9 B P y 5 3 r r 7 + g l x 2 p t l 6 Z J y I D g 8 e Y 9 R v M m r Z Q 0 G E W / g v D J x t N n m 1 v P W y 9 e v n q 9 3 X 7 z 9 t y U t R b Q F 6 U q 9 U X K D S h Z Q B 8 l K r i o N P A 8 V T B I L 7 8 1 / c E V a C P L 4 g c u K h j l f F L I T A q O n k r a 1 y z n O E 0 z O 3 C J l T N H j y i D e c V S O e n u s 0 x z Y W N n D x 1 D m K O X j V 1 3 x f 7 1 Z d 7 H r r i u p t J 1 5 4 n s O c u M n O Q 8 k e 7 T o 9 L Z P + n M 9 Z q J v a T d i Q 6 i Z d H 7 I F 6 D D l n X a d L + y c a l q H M o U C h u z D C O K h x Z r l E K B a 7 F a g M V F 5 d 8 A k M P C 5 6 D G d l l b I 7 u e W Z M s 1 L 7 V y B d s r c d l u f G L P L U K 5 s N z N 1 e Q z 7 U G 9 a Y f R 1 Z W V Q 1 Q i F W g 7 J a U S x p c w M 6 l h o E q o U H X G j p / 0 r F l P u o 0 F + q 5 U O I 7 6 5 8 H 5 w f H s Q e n 3 3 u H J + s 4 9 g i H 8 g u 6 Z K Y f C H H 5 D s 5 J X 0 i y O / g f b A b d I K b c C f c C z + u p G G w 9 r w j / 1 U Y / Q E d d M J 4 < / l a t e x i t > Query Label LOSS < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > Graph Construction Feature Embedding Label Propagation Loss < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " n n U z z 7 p X G S + A D u 8 C 4 A N Y w L u W L a E = " > A A A B 7 X i c b Z D L S g M x F I b P 1 F u t t 6 p L N 8 E i u C o z I u i y 6 M Z l B X u B z l A y a a a N z W V I M k I Z + g 5 u X C j i 1 v d x 5 9 u Y t r P Q 1 h 8 C H / 8 5 h 5 z z x y l n x v r + t 1 d a W 9 / Y 3 C p v V 3 Z 2 9 / Y P q o d H b a M y T W i L K K 5 0 N 8 a G c i Z p y z L L a T f V F I u Y 0 0 4 8 v p 3 V O 0 9 U G 6 b k g 5 2 k N B J 4 K F n C C L b O a o c x G 4 Z 5 v 1 r z 6 / 5 c a B W C A m p Q q N m v f o U D R T J B p S U c G 9 M L / N R G O d a W E U 6 n l T A z N M V k j I e 0 5 1 B i Q U 2 U z 7 e d o j P n D F C i t H v S o r n 7 e y L H w p i J i F 2 n w H Z k l m s z 8 7 9 a L 7 P J d Z Q z m W a W S r L 4 K M k 4 s g r N T k c D p i m x f O I A E 8 3 c r o i M s M b E u o A q L o R g + e R V a F / U A 8 f 3 l 7 X G T R F H G U 7 g F M 4 h g C t o w B 0 0 o Q U E H u E Z X u H N U 9 6 L 9 + 5 9 L F p L X j F z D H / k f f 4 A j + y P G w = = < / l a t e x i t >

Figure 2 : The overall framework of our algorithm in which the manifold structure of the entire query set helps to learn better decision boundary. The proposed algorithm is composed of four components: feature embedding, graph construction, label propagation, and loss generation.

3. MAIN APPROACH:

In this section, we introduce the proposed algorithm that utilizes the manifold structure of the given few-shot classification task to improve the performance.

3.1. PROBLEM DEFINITION:

We follow the episodic paradigm (Vinyals et al., 2016) that effectively trains a meta-learner for fewshot classification tasks, which is commonly employed in various literature (Snell et al., 2017; Finn et al., 2017; Nichol et al., 2018; Sung et al., 2018; Mishra et al., 2018) . Meta-learning implemented by the episodic training reasonably performs well to few-shot classification tasks. Yet, due to the lack of labeled instances (K is usually very small) in the support set, we observe that a reliable classifier is still difficult to be obtained. This motivates us to consider a transductive setting that utilizes the whole query set for the prediction rather than predicting each example independently. Taking the entire query set into account, we can alleviate the low-data problem and provide more reliable generalization property.

3.2. TRANSDUCTIVE PROPAGATION NETWORK (TPN):

We introduce Transductive Propagation Network (TPN) illustrated in Figure 2 , which consists of four components: feature embedding with a convolutional neural network; graph construction that produces example-wise parameters to exploit the manifold structure; label propagation that spreads labels from the support set S to the query set Q; a loss generation step that computes a crossentropy loss between propagated labels and the ground-truths on Q to jointly train all parameters in the framework.

3.2.1. FEATURE EMBEDDING:

We employ a convolutional neural network f ϕ to extract features of an input x i , where f ϕ (x i ; ϕ) refers to the feature map and ϕ indicates a parameter of the network. Despite the generality, we adopt the same architecture used in several recent works (Snell et al., 2017; Sung et al., 2018; Vinyals et al., 2016) . By doing so, we can provide more fair comparisons in the experiments, highlighting the effects of transductive approach. The network is made up of four convolutional blocks where each block begins with a 2D convolutional layer with a 3 × 3 kernel and filter size of 64. Each convolutional layer is followed by a batch-normalization layer (Ioffe and Szegedy, 2015) , a ReLU nonlinearity and a 2 × 2 max-pooling layer. We use the same embedding function f ϕ for both the support set S and the query set Q.

3.2.2. GRAPH CONSTRUCTION:

Manifold learning (Chung and Graham, 1997; Zhou et al., 2004; Yang et al., 2016) discovers the embedded low-dimensional subspace in the data, where it is critical to choose an appropriate neighborhood graph. A common choice is Gaussian similarity function:

EQUATION

)

where d(•, •) is a distance measure (e.g., Euclidean distance) and σ is the length scale parameter.

The neighborhood structure behaves differently with respect to various σ, which means that it needs to carefully select the optimal σ for the best performance of label propagation (Wang and Zhang, 2006; Zhu and Ghahramani, 2002) . In addition, we observe that there is no principled way to tune the scale parameter in meta-learning framework, though there exist some heuristics for dimensionalty reduction methods (Zelnik-Manor and Perona, 2004; Sugiyama, 2007) .

Example-wise length-scale parameter To obtain a proper neighborhood graph in meta-learning, we propose a graph construction module built on the union set of support set and query set: S ∪ Q. This module is composed of a convolutional neural network g φ which takes the feature map f ϕ (x i ) for x i ∈ S ∪ Q to produce an example-wise length-scale parameter σ i = g φ (f ϕ (x i )). Note that the scale parameter is determined example-wisely and learned in an episodic training procedure, which adapts well to different tasks and makes it suitable for few-shot learning. With the example-wise σ i , our similarity function is then defined as follows:

W ij = exp - 1 2 d f ϕ (x i ) σ i , f ϕ (x j ) σ j (2)

where W ∈ R (N ×K+T )×(N ×K+T ) for all instances in S ∪ Q. We only keep the k-max values in each row of W to construct a k-nearest neighbour graph. Then we apply the normalized graph Laplacians (Chung and Graham, 1997) on W , that is, S = D -1/2 W D -1/2 , where D is a diagonal matrix with its (i, i)-value to be the sum of the i-th row of W .

f ' (x i )

< l a t e x i t s h a 1 _ b a s e 6 4 = " c q Z G n + 3 8 n v b J z J m y c o 5 e 7 r S x a Q 4

= " > A A A C A n i c b V D L S s N A F L 2 p r 1 p f U V f i Z r A I d V M S E X R Z d O O y g n 1 A E 8 J k O m m H T h 7 M T I o l F D f + i h s X i r j 1 K 9 z 5 N 0 7 a L L T 1 w I X D O f d y 7 z 1 + w p l U l v V t l F Z W 1 9 Y 3 y p u V r e 2 d 3 T 1 z / 6 A t 4 1 Q Q 2 i I x j 0 X X x 5 J y F t G W Y o r T b i I o D n 1 O O / 7 o J v c 7 Y y o k i 6 N 7 N U m o G + J B x A J G s N K S Z x 4 F X u a M s U i G b F p z Q q y G f p A 9 e G x 6 5 p l V q 2 7 N g J a J X Z A q F G h 6 5 p f T j 0 k a 0 k g R j q X s 2 V a i 3 A w L x Q i n 0 4 q T S p p g M s I D 2 t M 0 w i G V b j Z 7 Y Y p O t d J H Q S x 0 R Q r N 1 N 8 T G Q 6 l n I S + 7 s y P l I t e L v 7 n 9 V I V X L k Z i 5 J U 0 Y j M F w U p R y p G e R 6 o z w Q l i k 8 0 w U Q w f S s i Q y w w U T q 1 i g 7 B X n x 5 m b T P 6 7 Z V t + 8 u q o 3 r I o 4 y H M M J 1 M C G S 2 j A L T S h B Q Q e 4

R l e 4 c 1 4 M l 6 M d + N j 3 l o y i p l D + A P j 8 w e 6 T p e f < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " c q Z G n + 3 8 n v b J z J m y c o 5 e 7 r S x a Q 4

= " > A A A C A n i c b V D L S s N A F L 2 p r 1 p f U V f i Z r A I d V M S E X R Z d O O y g n 1 A E 8 J k O m m H T h 7 M T I o l F D f + i h s X i r j 1 K 9 z 5 N 0 7 a L L T 1 w I X D O f d y 7 z 1 + w p l U l v V t l F Z W 1 9 Y 3 y p u V r e 2 d 3 T 1 z / 6 A t 4 1 Q Q 2 i I x j 0 X X x 5 J y F t G W Y o r T b i I o D n 1 O O / 7 o J v c 7 Y y o k i 6 N 7 N U m o G + J B x A J G s N K S Z x 4 F X u a M s U i G b F p z Q q y G f p A 9 e G x 6 5 p l V q 2 7 N g J a J X Z A q F G h 6 5 p f T j 0 k a 0 k g R j q X s 2 V a i 3 A w L x Q i n 0 4 q T S p p g M s I D 2 t M 0 w i G V b j Z 7 Y Y p O t d J H Q S x 0 R Q r N 1 N 8 T G Q 6 l n I S + 7 s y P l I t e L v 7 n 9 V I V X L k Z i 5 J U 0 Y j M F w U p R y p G e R 6 o z w Q l i k 8 0 w U Q w f S s i Q y w w U T q 1 i g 7 B X n x 5 m b T P 6 7 Z V t + 8 u q o 3 r I o 4 y H M M J 1 M C G S 2 j A L T S h B Q Q e 4

R l e 4 c 1 4 M l 6 M d + N j 3 l o y i p l D + A P j 8 w e 6 T p e f < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " c q Z G n + 3 8 n v b J z J m y c o 5 e 7 r S x a Q 4

= " > A A A C A n i c b V D L S s N A F L 2 p r 1 p f U V f i Z r A I d V M S E X R Z d O O y g n 1 A E 8 J k O m m H T h 7 M T I o l F D f + i h s X i r j 1 K 9 z 5 N 0 7 a L L T 1 w I X D O f d y 7 z 1 + w p l U l v V t l F Z W 1 9 Y 3 y p u V r e 2 d 3 T 1 z / 6 A t 4 1 Q Q 2 i I x j 0 X X x 5 J y F t G W Y o r T b i I o D n 1 O O / 7 o J v c 7 Y y o k i 6 N 7 N U m o G + J B x A J G s N K S Z x 4 F X u a M s U i G b F p z Q q y G f p A 9 e G x 6 5 p l V q 2 7 N g J a J X Z A q F G h 6 5 p f T j 0 k a 0 k g R j q X s 2 V a i 3 A w L x Q i n 0 4 q T S p p g M s I D 2 t M 0 w i G V b j Z 7 Y Y p O t d J H Q S x 0 R Q r N 1 N 8 T G Q 6 l n I S + 7 s y P l I t e L v 7 n 9 V I V X L k Z i 5 J U 0 Y j M F w U p R y p G e R 6 o z w Q l i k 8 0 w U Q w f S s i Q y w w U T q 1 i g 7 B X n x 5 m b T P 6 7 Z V t + 8 u q o 3 r I o 4 y H M M J 1 M C G S 2 j A L T S h B Q Q e 4

R l e 4 c 1 4 M l 6 M d + N j 3 l o y i p l D + A P j 8 w e 6 T p e f < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " c q Z G n + 3 8 n v b J z J m y c o 5 e 7 r S x a Q 4

= " > A A A C A n i c b V D L S s N A F L 2 p r 1 p f U V f i Z r A I d V M S E X R Z d O O y g n 1 A E 8 J k O m m H T h 7 M T I o l F D f + i h s X i r j 1 K 9 z 5 N 0 7 a L L T 1 w I X D O f d y 7 z 1 + w p l U l v V t l F Z W 1 9 Y 3 y p u V r e 2 d 3 T 1 z / 6 A t 4 1 Q Q 2 i I x j 0 X X x 5 J y F t G W Y o r T b i I o D n 1 O O / 7 o J v c 7 Y y o k i 6 N 7 N U m o G + J B x A J G s N K S Z x 4 F X u a M s U i G b F p z Q q y G f p A 9 e G x 6 5 p l V q 2 7 N g J a J X Z A q F G h 6 5 p f T j 0 k a 0 k g R j q X s 2 V a i 3 A w L x Q i n 0 4 q T S p p g M s I D 2 t M 0 w i G V b j Z 7 Y Y p O t d J H Q S x 0 R Q r N 1 N 8 T G Q 6 l n I S + 7 s y P l I t e L v 7 n 9 V I V X L k Z i 5 J U 0 Y j M F w U p R y p G e R 6 o z w Q l i k 8 0 w U Q w f S s i Q y w w U T q 1 i g 7 B X n x 5 m b T P 6 7 Z V t + 8 u q o 3 r I o 4 y H M M J 1 M C G S 2 j A L T S h B Q Q e 4

R l e 4 c 1 4 M l 6 M d + N j 3 l o y i p l D + A P j 8 w e 6 T p e f < / l a t e x i t > f ' (x j )

< l a t e x i t s h a 1 _ b a s e 6 4 = " y H N 1 w 3

2 8 A 9 Y 3 q W r Q 9 u Y 3 W 9 E L e Y E = " > A A A C A n i c b V B N S 8 N A E J 3 U r 1 q / o p 7 E S 7 A I 9 V I S E f R Y 9 O K x g v 2 A J o T N d t O u 3 W z C 7 q Z Y Q v H i X / H i Q R G v / g p v / h s 3 b Q 5 a f T D w e G + G m X l B w q h U t v 1 l l J a W V 1 b X y u u V j c 2 t 7 R 1 z d 6 8 t 4 1 R g 0 s I x i 0 U 3 Q J I w y k l L U c V I N x E E R Q E j n W B 0 l f u d M R G S x v x W T R L i R W j A a U g x U l r y z Y P Q z 9 w x E s m Q T m t u h N Q w C L N 7 / 2 5 6 4 p t V u 2 7 P Y P 0 l T k G q U K D p m 5 9 u P 8 Z p R L j C D E n Z c + x E e R k S i m J G p h U 3 l S R B e I Q G p K c p R x G R X j Z 7 Y W o d a 6 V v h b H Q x Z U 1 U 3 9 O Z C i S c h I F u j M / U i 5 6 u f i f 1 0 t V e O F l l C e p I h z P F 4 U p s 1 R s 5 X l Y f S o I V m y i C c K C 6 l s t P E Q C Y a V T q + g Q n M W X / 5 L 2 a d 2 x 6 8 7 N W b V x W c R R h k M 4 g h o 4 c A 4 N u I Y m t A D D A z z B C 7 w a j 8 a z 8 W a 8 z 1 t L R j G z D 7 9 g f H w D u 9 S X o A = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " y H N 1 w 3 2 8 A 9 Y 3 q W r Q 9 u Y 3 W 9 E L e Y E = " > A A A C A n i c b V B N S 8 N A E J 3 U r 1 q / o p 7 E S 7 A I 9 V I S E f R Y 9 O K x g v 2 A J o T N d t O u 3 W z C 7 q Z Y Q v H i X / H i Q R G v / g p v / h s 3 b Q 5 a f T D w e G + G m X l B w q h U t v 1 l l J a W V 1 b X y u u V j c 2 t 7 R 1 z d 6 8 t 4 1 R g 0 s I x i 0 U 3 Q J I w y k l L U c V I N x E E R Q E j n W B 0 l f u d M R G S x v x W T R L i R W j A a U g x U l r y z Y P Q z 9 w x E s m Q T m t u h N Q w C L N 7 / 2 5 6 4 p t V u 2 7 P Y P 0 l T k G q U K D p m 5 9 u P 8 Z p R L j C D E n Z c + x E e R k S i m J G p h U 3 l S R B e I Q G p K c p R x G R X j Z 7 Y W o d a 6 V v h b H Q x Z U 1 U 3 9 O Z C i S c h I F u j M / U i 5 6 u f i f 1 0 t V e O F l l C e p I h z P F 4 U p s 1 R s 5 X l Y f S o I V m y i C c K C 6 l s t P E Q C Y a V T q + g Q n M W X / 5 L 2 a d 2 x 6 8 7 N W b V x W c R R h k M 4 g h o 4 c A 4 N u I Y m t A D D A z z B C 7 w a j 8 a z 8 W a 8 z 1 t L R j G z D 7 9 g f H w D u 9 S X o A = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " y H N 1 w 3 2 8 A 9 Y 3 q W r Q 9 u Y 3 W 9 E L e Y E = " > A A A C A n i c b V B N S 8 N A E J 3 U r 1 q / o p 7 E S 7 A I 9 V I S E f R Y 9 O K x g v 2 A J o T N d t O u 3 W z C 7 q Z Y Q v H i X / H i Q R G v / g p v / h s 3 b Q 5 a f T D w e G + G m X l B w q h U t v 1 l l J a W V 1 b X y u u V j c 2 t 7 R 1 z d 6 8 t 4 1 R g 0 s I x i 0 U 3 Q J I w y k l L U c V I N x E E R Q E j n W B 0 l f u d M R G S x v x W T R L i R W j A a U g x U l r y z Y P Q z 9 w x E s m Q T m t u h N Q w C L N 7 / 2 5 6 4 p t V u 2 7 P Y P 0 l T k G q U K D p m 5 9 u P 8 Z p R L j C D E n Z c + x E e R k S i m J G p h U 3 l S R B e I Q G p K c p R x G R X j Z 7 Y W o d a 6 V v h b H Q x Z U 1 U 3 9 O Z C i S c h I F u j M / U i 5 6 u f i f 1 0 t V e O F l l C e p I h z P F 4 U p s 1 R s 5 X l Y f S o I V m y i C c K C 6 l s t P E Q C Y a V T q + g Q n M W X / 5 L 2 a d 2 x 6 8 7 N W b V x W c R R h k M 4 g h o 4 c A 4 N u I Y m t A D D A z z B C 7 w a j 8 a z 8 W a 8 z 1 t L R j G z D 7 9 g f H w D u 9 S X o A = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " y H N 1 w 3 2 8 A 9 Y 3 q W r Q 9 u Y 3 W 9 E L e Y E = " > A A A C A n i c b V B N S 8 N A E J 3 U r 1 q / o p 7 E S 7 A I 9 V I S E f R Y 9 O K x g v 2 A J o T N d t O u 3 W z C 7 q Z Y Q v H i X / H i Q R G v / g p v / h s 3 b Q 5 a f T D w e G + G m X l B w q h U t v 1 l l J a W V 1 b X y u u V j c 2 t 7 R 1 z d 6 8 t 4 1 R g 0 s I x i 0 U 3 Q J I w y k l L U c V I N x E E R Q E j n W B 0 l f u d M R G S x v x W T R L i R W j A a U g x U l r y z Y P Q z 9 w x E s m Q T m t u h N Q w C L N 7 / 2 5 6 4 p t V u 2 7 P Y P 0 l T k G q U K D p m 5 9 u P 8 Z p R L j C D E n Z c + x E e R k S i m J G p h U 3 l S R B e I Q G p K c p R x G R X j Z 7 Y W o d a 6 V v h b H Q x Z U 1 U 3 9 O Z C i S c h I F u j M / U i 5 6 u f i f 1 0 t V e O F l l C e p I h z P F 4 U p s 1 R s 5 X l Y f S o I V m y i C c K C 6 l s t P E Q C Y a V T q + g Q n M W X / 5 L 2 a d 2 x 6 8 7 N W b V x W c R R h k M 4 g h o 4 c A 4 N u I Y m t A D D A z z B C 7 w a j 8 a z 8 W a 8 z 1 t L R j G z D 7 9 g f H w D u 9 S X o A = = < / l a t e x i t > i < l a t e x i t s h a 1 _ b a s e 6 4 = " G 5 l G s T i 4 B G 6 F h y R G k s c Q q z v s m C 4 = " > A A A B 7 3 i c b V D L S g N B E O y N r x h f U Y 9 e B o P g K e y K o M e g F 4 8 R z A O S J c x O Z p M h M 7 P r T K 8 Q Q n 7 C i w d F v P o 7 3 v w b J 8 k e N L G g o a j q p r s r S q W w 6 P v f X m F t f W N z q 7 h d 2 t n d 2 z 8 o H x 4 1 b Z I Z x h s s k Y l p R 9 R y K T R v o E D J 2 6 n h V E W S t 6 L R 7 c x v P X F j R a I f c J z y U N G B F r F g F J 3 U 7 l o x U L Q n e u W K X / X n I K s k y E k F c t R 7 5 a 9 u P 2 G Z 4 h q Z p N Z 2 A j / F c E I N C i b 5 t N T N L E 8 p G 9 E B 7 z i q q e I 2 n M z v n Z I z p / R J n B h X G s l c / T 0 x o c r a s Y p c p 6 I 4 t M v e T P z P 6 2 Q Y X 4 c T o d M M u W a L R X E m C S Z k 9 j z p C 8 M Z y r E j l B n h b i V s S A 1 l 6 C I q u R C C 5 Z d X S f O i G v j V 4 P 6 y U r v J 4 y j C C Z z C O Q R w B T W 4 g z o 0 g I G E Z 3 i F N + / R e / H e v Y 9

F a 8 H L Z 4 7 h D 7 z P H x q 0 j / 8 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " G 5 l G s

T i 4 B G 6 F h y R G k s c Q q z v s m C 4 = " > A A A B 7 3 i c b V D L S g N B E O y N r x h f U Y 9 e B o P g K e y K o M e g F 4 8 R z A O S J c x O Z p M h M 7 P r T K 8 Q Q n 7 C i w d F v P o 7 3 v w b J 8 k e N L G g o a j q p r s r S q W w 6 P v f X m F t f W N z q 7 h d 2 t n d 2 z 8 o H x 4 1 b Z I Z x h s s k Y l p R 9 R y K T R v o E D J 2 6 n h V E W S t 6 L R 7 c x v P X F j R a I f c J z y U N G B F r F g F J 3 U 7 l o x U L Q n e u W K X / X n I K s k y E k F c t R 7 5 a 9 u P 2 G Z 4 h q Z p N Z 2 A j / F c E I N C i b 5 t N T N L E 8 p G 9 E B 7 z i q q e I 2 n M z v n Z I z p / R J n B h X G s l c / T 0 x o c r a s Y p c p 6 I 4 t M v e T P z P 6 2 Q Y X 4 c T o d M M u W a L R X E m C S Z k 9 j z p C 8 M Z y r E j l B n h b i V s S A 1 l 6 C I q u R C C 5 Z d X S f O i G v j V 4 P 6 y U r v J 4 y j C C Z z C O Q R w B T W 4 g z o 0 g I G E Z 3 i F N + / R e / H e v Y 9

F a 8 H L Z 4 7 h D 7 z P H x q 0 j / 8 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " G 5 l G s

T i 4 B G 6 F h y R G k s c Q q z v s m C 4 = " > A A A B 7 3 i c b V D L S g N B E O y N r x h f U Y 9 e B o P g K e y K o M e g F 4 8 R z A O S J c x O Z p M h M 7 P r T K 8 Q Q n 7 C i w d F v P o 7 3 v w b J 8 k e N L G g o a j q p r s r S q W w 6 P v f X m F t f W N z q 7 h d 2 t n d 2 z 8 o H x 4 1 b Z I Z x h s s k Y l p R 9 R y K T R v o E D J 2 6 n h V E W S t 6 L R 7 c x v P X F j R a I f c J z y U N G B F r F g F J 3 U 7 l o x U L Q n e u W K X / X n I K s k y E k F c t R 7 5 a 9 u P 2 G Z 4 h q Z p N Z 2 A j / F c E I N C i b 5 t N T N L E 8 p G 9 E B 7 z i q q e I 2 n M z v n Z I z p / R J n B h X G s l c / T 0 x o c r a s Y p c p 6 I 4 t M v e T P z P 6 2 Q Y X 4 c T o d M M u W a L R X E m C S Z k 9 j z p C 8 M Z y r E j l B n h b i V s S A 1 l 6 C I q u R C C 5 Z d X S f O i G v j V 4 P 6 y U r v J 4 y j C C Z z C O Q R w B T W 4 g z o 0 g I G E Z 3 i F N + / R e / H e v Y 9

F a 8 H L Z 4 7 h D 7 z P H x q 0 j / 8 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " G 5 l G s

T i 4 B G 6 F h y R G k s c Q q z v s m C 4 = " > A A A B 7 3 i c b V D L S g N B E O y N r x h f U Y 9 e B o P g K e y K o M e g F 4 8 R z A O S J c x O Z p M h M 7 P r T K 8 Q Q n 7 C i w d F v P o 7 3 v w b J 8 k e N L G g o a j q p r s r S q W w 6 P v f X m F t f W N z q 7 h d 2 t n d 2 z 8 o H x 4 1 b Z I Z x h s s k Y l p R 9 R y K T R v o E D J 2 6 n h V E W S t 6 L R 7 c x v P X F j R a I f c J z y U N G B F r F g F J 3 U 7 l o x U L Q n e u W K X / X n I K s k y E k F c t R 7 5 a 9 u P 2 G Z 4 h q Z p N Z 2 A j / F c E I N C i b 5 t N T N L E 8 p G 9 E B 7 z i q q e I 2 n M z v n Z I z p / R J n B h X G s l c / T 0 x o c r a s Y p c p 6 I 4 t M v e T P z P 6 2 Q Y X 4 c T o d M M u W a L R X E m C S Z k 9 j z p C 8 M Z y r E j l B n h b i V s S A 1 l 6 C I q u R C C 5 Z d X S f O i G v j V 4 P 6 y U r v J 4 y j C C Z z C O Q R w B T W 4 g z o 0 g I G E Z 3 i F N + / R e / H e v Y 9

F a 8 H L Z 4 7 h D 7 z P H x q 0 j / 8 = < / l a t e x i t > j < l a t e x i t s h a 1 _ b a s e 6 4 = " W G a / 6 v X u 1 4 3 j M e S e C V X A w t s 0 Graph construction structure The structure of the proposed graph construction module is shown in Figure 3 . It is composed of two convolutional blocks and two fully-connected layers, where each block contains a 3-by-3 convolution, batch normalization, ReLU activation, followed by 2by-2 max pooling. The number of filters in each convolutional block is 64 and 1, respectively. To provide an example-wise scaling parameter, the activation map from the second convolutional block is transformed into a scalar by two fully-connected layers in which the number of neurons is 8 and 1, respectively.

Q N Q = " > A A A B 8 H i c b V D L S g N B E O z 1 G e M r 6 t H L Y B A 8 h V 0 R 9 B j 0 4 j G C e U i y h N n J b D J m H s v M r B C W f I U X D 4 p 4 9 X O 8 + T f O J n v Q x I K G o q q b 7 q 4 o 4 c x Y 3 / / 2 V l b X 1 j c 2 S 1 v l 7 Z 3 d v f 3 K w W H L q F Q T 2 i S K K 9 2 J s K G c S d q 0 z H L a S T T F I u K 0 H Y 1 v c r / 9 R L V h S t 7 b S U J D g Y e S x Y x g 6 6 S H n m F D g f u P 5 X 6 l 6 t f 8 G d A y C Q p S h Q K N f u W r N 1 A k F V R a w r E x 3 c B P b J h h b R n h d F r u p Y Y m m I z x k H Y d l V h Q E 2 a z g 6 f o 1 C k D F C v t S l o 0 U 3 9 P Z F g Y M x G R 6 x T Y j s y i l 4 v / e d 3 U x l d h x m S S W i r J f F G c c m Q V y r 9 H A 6 Y p s X z i C C a a u V s R G W G N i X U Z 5 S E E i y 8 v k 9 Z 5 L f B r w d 1 F t X 5 d x F G C Y z i B M w j g E u p w C w 1 o A g E B z / A K b 5 7 2 X r x 3 7 2 P e u u I V M 0 f w B 9 7 n D 1 O I k B Q = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " W G a / 6 v X u 1 4 3 j M e S e C V X A w t s 0 Q N Q = " > A A A B 8 H i c b V D L S g N B E O z 1 G e M r 6 t H L Y B A 8 h V 0 R 9 B j 0 4 j G C e U i y h N n J b D J m H s v M r B C W f I U X D 4 p 4 9 X O 8 + T f O J n v Q x I K G o q q b 7 q 4 o 4 c x Y 3 / / 2 V l b X 1 j c 2 S 1 v l 7 Z 3 d v f 3 K w W H L q F Q T 2 i S K K 9 2 J s K G c S d q 0 z H L a S T T F I u K 0 H Y 1 v c r / 9 R L V h S t 7 b S U J D g Y e S x Y x g 6 6 S H n m F D g f u P 5 X 6 l 6 t f 8 G d A y C Q p S h Q K N f u W r N 1 A k F V R a w r E x 3 c B P b J h h b R n h d F r u p Y Y m m I z x k H Y d l V h Q E 2 a z g 6 f o 1 C k D F C v t S l o 0 U 3 9 P Z F g Y M x G R 6 x T Y j s y i l 4 v / e d 3 U x l d h x m S S W i r J f F G c c m Q V y r 9 H A 6 Y p s X z i C C a a u V s R G W G N i X U Z 5 S E E i y 8 v k 9 Z 5 L f B r w d 1 F t X 5 d x F G C Y z i B M w j g E u p w C w 1 o A g E B z / A K b 5 7 2 X r x 3 7 2 P e u u I V M 0 f w B 9 7 n D 1 O I k B Q = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " W G a / 6 v X u 1 4 3 j M e S e C V X A w t s 0 Q N Q = " > A A A B 8 H i c b V D L S g N B E O z 1 G e M r 6 t H L Y B A 8 h V 0 R 9 B j 0 4 j G C e U i y h N n J b D J m H s v M r B C W f I U X D 4 p 4 9 X O 8 + T f O J n v Q x I K G o q q b 7 q 4 o 4 c x Y 3 / / 2 V l b X 1 j c 2 S 1 v l 7 Z 3 d v f 3 K w W H L q F Q T 2 i S K K 9 2 J s K G c S d q 0 z H L a S T T F I u K 0 H Y 1 v c r / 9 R L V h S t 7 b S U J D g Y e S x Y x g 6 6 S H n m F D g f u P 5 X 6 l 6 t f 8 G d A y C Q p S h Q K N f u W r N 1 A k F V R a w r E x 3 c B P b J h h b R n h d F r u p Y Y m m I z x k H Y d l V h Q E 2 a z g 6 f o 1 C k D F C v t S l o 0 U 3 9 P Z F g Y M x G R 6 x T Y j s y i l 4 v / e d 3 U x l d h x m S S W i r J f F G c c m Q V y r 9 H A 6 Y p s X z i C C a a u V s R G W G N i X U Z 5 S E E i y 8 v k 9 Z 5 L f B r w d 1 F t X 5 d x F G C Y z i B M w j g E u p w C w 1 o A g E B z / A K b 5 7 2 X r x 3 7 2 P e u u I V M 0 f w B 9 7 n D 1 O I k B Q = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " W G a / 6 v X u 1 4 3 j M e S e C V X A w t s 0 Q N Q = " > A A A B 8 H i c b V D L S g N B E O z 1 G e M r 6 t H L Y B A 8 h V 0 R 9 B j 0 4 j G C e U i y h N n J b D J m H s v M r B C W f I U X D 4 p 4 9 X O 8 + T f O J n v Q x I K G o q q b 7 q 4 o 4 c x Y 3 / / 2 V l b X 1 j c 2 S 1 v l 7 Z 3 d v f 3 K w W H L q F Q T 2 i S K K 9 2 J s K G c S d q 0 z H L a S T T F I u K 0 H Y 1 v c r / 9 R L V h S t 7 b S U J D g Y e S x Y x g 6 6 S H n m F D g f u P 5 X 6 l 6 t f 8 G d A y C Q p S h Q K N f u W r N 1 A k F V R a w r E x 3 c B P b J h h b R n h d F r u p Y Y m m I z x k H Y d l V h Q E 2 a z g 6 f o 1 C k D F C v t S l o 0 U 3 9 P Z F g Y M x G R 6 x T Y j s y i l 4 v / e d 3 U x l d h x m S S W i r J f F G c c m Q V y r 9 H A 6 Y p s X z i C C a a u V s R G W G N i X U Z 5 S E E i y 8 v k 9 Z 5 L f B r w d 1 F t X 5 d x F G C Y z i B M w j g E u p w C w 1 o A g E B z / A K b 5 7 2 X r x 3 7 2 P e u u I V M 0 f w B 9 7 n D 1 O I k B Q = < / l a t e x i t > W ij = exp ✓ 1 2 d( f ' (x i ) i , f ' (x j ) j ) ◆ < l a t e x i t s h a 1 _ b a s e 6 4 = " M Y 4 L T B S Z R v 7 t h U s m Y o O n F V H J 1 Y Q = " > A A A C a H i c f Z F d S x w x F I Y z o 1 a 7 2 j r a S i n e B B d h B V 1 m R N C b g u i N l x Z c V 9 g s Q y Z 7 Z j e a + S A 5 I y 5 h 8 D / 2 r j / A G 3 + F 2 Y 8 L P 0 o P B J 6 8 5 z 0 k e Z O U S h o M w 7 + e v 7 C 4 9 G l 5 5 X N j d e 3 L 1 / V g Y / P a F J U W 0 B G F K v R N w g 0 o m U M H J S q 4 K T X w L F H Q T e 7 O J / 3 u P W g j i / w K x y X 0 M z 7 M Z S o F R y f F w W M 3 t v K 2 p r 8 o g 4 e S K U i x d c B S z Y W N a n t Y D 1 q z T R p b d s 9 1 O Z J 1 i 2 U c R 0 l q H 2 J Z 7 9 W W G T n M u O P 9 / 1 l v X 1 k d U 6 b l c I R 7 c d A M 2 + G 0 6 E e I 5 t A k 8 7 q M g z 9 s U I g q g x y F 4 s b 0 o r D E v u U a p V B Q N 1 h l o O T i j g + h 5 z D n G Z i + n Q Z V 0 1 2 n D G h a a L d y p F P 1 9 Y T l m T H j L H H O y c X N + 9 5 E / F e v V 2 F 6 0 r c y L y u E X M w O S i t F s a C T 1 O l A a h C o x g 6 4 0 N L d l Y o R d 2 m h + 5 u G C y F 6 / + S P c H 3 Y j s J 2 9 P u o e X o 2 j 2 O F b J M d 0 i I R O S a n 5 I J c k g 4 R 5 M l b 9 b 5 7 W 9 6 z H / g / / J 8 z q + / N Z 7 6 R N + X v v A C n Z b v K < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " M Y 4 L T B S Z R v 7 t h U s m Y o O n F V H J 1 Y Q = " > A A A C a H i c f Z F d S x w x F I Y z o 1 a 7 2 j r a S i n e B B d h B V 1 m R N C b g u i N l x Z c V 9 g s Q y Z 7 Z j e a + S A 5 I y 5 h 8 D / 2 r j / A G 3 + F 2 Y 8 L P 0 o P B J 6 8 5 z 0 k e Z O U S h o M w 7 + e v 7 C 4 9 G l 5 5 X N j d e 3 L 1 / V g Y / P a F J U W 0 B G F K v R N w g 0 o m U M H J S q 4 K T X w L F H Q T e 7 O J / 3 u P W g j i / w K x y X 0 M z 7 M Z S o F R y f F w W M 3 t v K 2 p r 8 o g 4 e S K U i x d c B S z Y W N a n t Y D 1 q z T R p b d s 9 1 O Z J 1 i 2 U c R 0 l q H 2 J Z 7 9 W W G T n M u O P 9 / 1 l v X 1 k d U 6 b l c I R 7 c d A M 2 + G 0 6 E e I 5 t A k 8 7 q M g z 9 s U I g q g x y F 4 s b 0 o r D E v u U a p V B Q N 1 h l o O T i j g + h 5 z D n G Z i + n Q Z V 0 1 2 n D G h a a L d y p F P 1 9 Y T l m T H j L H H O y c X N + 9 5 E / F e v V 2 F 6 0 r c y L y u E X M w O S i t F s a C T 1 O l A a h C o x g 6 4 0 N L d l Y o R d 2 m h + 5 u G C y F 6 / + S P c H 3 Y j s J 2 9 P u o e X o 2 j 2 O F b J M d 0 i I R O S a n 5 I J c k g 4 R 5 M l b 9 b 5 7 W 9 6 z H / g / / J 8 z q + / N Z 7 6 R N + X v v A C n Z b v K < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " M Y 4 L T B S Z R v 7 t h U s m Y o O n F V H J 1 Y Q = " > A A A C a H i c f Z F d S x w x F I Y z o 1 a 7 2 j r a S i n e B B d h B V 1 m R N C b g u i N l x Z c V 9 g s Q y Z 7 Z j e a + S A 5 I y 5 h 8 D / 2 r j / A G 3 + F 2 Y 8 L P 0 o P B J 6 8 5 z 0 k e Z O U S h o M w 7 + e v 7 C 4 9 G l 5 5 X N j d e 3 L 1 / V g Y / P a F J U W 0 B G F K v R N w g 0 o m U M H J S q 4 K T X w L F H Q T e 7 O J / 3 u P W g j i / w K x y X 0 M z 7 M Z S o F R y f F w W M 3 t v K 2 p r 8 o g 4 e S K U i x d c B S z Y W N a n t Y D 1 q z T R p b d s 9 1 O Z J 1 i 2 U c R 0 l q H 2 J Z 7 9 W W G T n M u O P 9 / 1 l v X 1 k d U 6 b l c I R 7 c d A M 2 + G 0 6 E e I 5 t A k 8 7 q M g z 9 s U I g q g x y F 4 s b 0 o r D E v u U a p V B Q N 1 h l o O T i j g + h 5 z D n G Z i + n Q Z V 0 1 2 n D G h a a L d y p F P 1 9 Y T l m T H j L H H O y c X N + 9 5 E / F e v V 2 F 6 0 r c y L y u E X M w O S i t F s a C T 1 O l A a h C o x g 6 4 0 N L d l Y o R d 2 m h + 5 u G C y F 6 / + S P c H 3 Y j s J 2 9 P u o e X o 2 j 2 O F b J M d 0 i I R O S a n 5 I J c k g 4 R 5 M l b 9 b 5 7 W 9 6 z H / g / / J 8 z q + / N Z 7 6 R N + X v v A C n Z b v K < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " M Y 4 L T B S Z R v 7 t h U s m Y o O n F V H J 1 Y Q = " > A A A C a H i c f Z F d S x w x F I Y z o 1 a 7 2 j r a S i n e B B d h B V 1 m R N C b g u i N l x Z c V 9 g s Q y Z 7 Z j e a + S A 5 I y 5 h 8 D / 2 r j / A G 3 + F 2 Y 8 L P 0 o P B J 6 8 5 z 0 k e Z O U S h o M w 7 + e v 7 C 4 9 G l 5 5 X N j d e 3 L 1 / V g Y / P a F J U W 0 B G F K v R N w g 0 o m U M H J S q 4 K T X w L F H Q T e 7 O J / 3 u P W g j i / w K x y X 0 M z 7 M Z S o F R y f F w W M 3 t v K 2 p r 8 o g 4 e S K U i x d c B S z Y W N a n t Y D 1 q z T R p b d s 9 1 O Z J 1 i 2 U c R 0 l q H 2 J Z 7 9 W W G T n M u O P 9 / 1 l v X 1 k d U 6 b l c I R 7 c d A M 2 + G 0 6 E e I 5 t A k 8 7 q M g z 9 s U I g q g x y F 4 s b 0 o r D E v u U a p V B Q N 1 h l o O T i j g + h 5 z D n G Z i + n Q Z V 0 1 2 n D G h a a L d y p F P 1 9 Y T l m T H j L H H O y c X N + 9 5 E / F e v V 2 F 6 0 r c y L y u E X M w O S i t F s a C T 1 O l A a h C o x g 6 4 0 N L d l Y o R d 2 m h + 5 u G C y F 6 / + S P c H 3 Y j s J 2 9 P u o e X o 2 j 2 O F b J M d 0 i I R O S a n 5 I J c k g 4 R 5 M l b 9 b 5 7 W 9 6 z H / g / / J 8 z q + / N Z 7 6 R N + X v v A C n Z b v K < / l a t e x i t > 3 ⇥ 3 conv BatchNorm ReLU 2 ⇥ 2 max-pool < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 + B U z e A N F l 9 A T S F g S Z Y M 6 N 2 v R s s = " > A A A C R 3 i c b V A 9 T x t B E N 0 z C R + X B A y U N K v Y S D S x 7 k w B J S I N R Y Q A x Y D k c 6 y 9 9 R i v v B + n 3 T m E d f K / S 0 O b j r 9 A k y I I U b K 2 L + L z S S u 9 e T N P M / v S T A q H U X Q T V O Y + f J x f W F w K P 3 3 + s r x S X V 0 7 d S a 3 H F r c S G P P U + Z A C g 0 t F C j h P L P A V C r h L B 1 + n / T P L s E 6 Y f R P H G X Q U e x C i 7 7 g D L 3 U r f 5 K O G g E G 9 a 3 E x Q K H N 2 u 0 w T h C g t u 9 O U 4 S c J Z t c + Q D w 6 N V U / S C f x o T a p 6 s 7 Q 2 / 1 s V u / q W G S P H 3 W o t a k R T 0 L c k L k m N l D j q V v 8 k P c N z 5 Y / i k j n X j q M M O w W z K L i E c Z j k D j L G h + w C 2 p 5 q 5 v d 2 i m k O Y 7 r p l R 7 t G + u f R j p V n z s K p p w b q d R P K o Y D 9 7 o 3 E d / r t X P s 7 3 Y K o b M c Q f P Z o n 4 u K R o 6 C Z X 2 h A W O c u Q J 4 1 b 4 W y k f M M u 4 T 9 a F P o T 4 9 Z f f k t N m I 4 4 a 8 X G z t r d f x r F I N s h X s k V i s k P 2 y A E 5 I i 3 C y W 9 y S / 6 R u + A 6 + B v c B w + z 0 U p Q e t b J C 1 S C R x g 7 s X Y = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 + B U z e A N F l 9 A T S F g S Z Y M 6 N 2 v R s s = " > A A A C R 3 i c b V A 9 T x t B E N 0 z C R + X B A y U N K v Y S D S x 7 k w B J S I N R Y Q A x Y D k c 6 y 9 9 R i v v B + n 3 T m E d f K / S 0 O b j r 9 A k y I I U b K 2 L + L z S S u 9 e T N P M / v S T A q H U X Q T V O Y + f J x f W F w K P 3 3 + s r x S X V 0 7 d S a 3 H F r c S G P P U + Z A C g 0 t F C j h P L P A V C r h L B 1 + n / T P L s E 6 Y f R P H G X Q U e x C i 7 7 g D L 3 U r f 5 K O G g E G 9 a 3 E x Q K H N 2 u 0 w T h C g t u 9 O U 4 S c J Z t c + Q D w 6 N V U / S C f x o T a p 6 s 7 Q 2 / 1 s V u / q W G S P H 3 W o t a k R T 0 L c k L k m N l D j q V v 8 k P c N z 5 Y / i k j n X j q M M O w W z K L i E c Z j k D j L G h + w C 2 p 5 q 5 v d 2 i m k O Y 7 r p l R 7 t G + u f R j p V n z s K p p w b q d R P K o Y D 9 7 o 3 E d / r t X P s 7 3 Y K o b M c Q f P Z o n 4 u K R o 6 C Z X 2 h A W O c u Q J 4 1 b 4 W y k f M M u 4 T 9 a F P o T 4 9 Z f f k t N m I 4 4 a 8 X G z t r d f x r F I N s h X s k V i s k P 2 y A E 5 I i 3 C y W 9 y S / 6 R u + A 6 + B v c B w + z 0 U p Q e t b J C 1 S C R x g 7 s X Y = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 + B U z e A N F l 9 A T S F g S Z Y M 6 N 2 v R s s = " > A A A C R 3 i c b V A 9 T x t B E N 0 z C R + X B A y U N K v Y S D S x 7 k w B J S I N R Y Q A x Y D k c 6 y 9 9 R i v v B + n 3 T m E d f K / S 0 O b j r 9 A k y I I U b K 2 L + L z S S u 9 e T N P M / v S T A q H U X Q T V O Y + f J x f W F w K P 3 3 + s r x S X V 0 7 d S a 3 H F r c S G P P U + Z A C g 0 t F C j h P L P A V C r h L B 1 + n / T P L s E 6 Y f R P H G X Q U e x C i 7 7 g D L 3 U r f 5 K O G g E G 9 a 3 E x Q K H N 2 u 0 w T h C g t u 9 O U 4 S c J Z t c + Q D w 6 N V U / S C f x o T a p 6 s 7 Q 2 / 1 s V u / q W G S P H 3 W o t a k R T 0 L c k L k m N l D j q V v 8 k P c N z 5 Y / i k j n X j q M M O w W z K L i E c Z j k D j L G h + w C 2 p 5 q 5 v d 2 i m k O Y 7 r p l R 7 t G + u f R j p V n z s K p p w b q d R P K o Y D 9 7 o 3 E d / r t X P s 7 3 Y K o b M c Q f P Z o n 4 u K R o 6 C Z X 2 h A W O c u Q J 4 1 b 4 W y k f M M u 4 T 9 a F P o T 4 9 Z f f k t N m I 4 4 a 8 X G z t r d f x r F I N s h X s k V i s k P 2 y A E 5 I i 3 C y W 9 y S / 6 R u + A 6 + B v c B w + z 0 U p Q e t b J C 1 S C R x g 7 s X Y = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 + B U z e A N F l 9 A T S F g S Z Y M 6 N 2 v R s s = " > A A A C R 3 i c b V A 9 T x t B E N 0 z C R + X B A y U N K v Y S D S x 7 k w B J S I N R Y Q A x Y D k c 6 y 9 9 R i v v B + n 3 T m E d f K / S 0 O b j r 9 A k y I I U b K 2 L + L z S S u 9 e T N P M / v S T A q H U X Q T V O Y + f J x f W F w K P 3 3 + s r x S X V 0 7 d S a 3 H F r c S G P P U + Z A C g 0 t F C j h P L P A V C r h L B 1 + n / T P L s E 6 Y f R P H G X Q U e x C i 7 7 g D L 3 U r f 5 K O G g E G 9 a 3 E x Q K H N 2 u 0 w T h C g t u 9 O U 4 S c J Z t c + Q D w 6 N V U / S C f x o T a p 6 s 7 Q 2 / 1 s V u / q W G S P H 3 W o t a k R T 0 L c k L k m N l D j q V v 8 k P c N z 5 Y / i k j n X j q M M O w W z K L i E c Z j k D j L G h + w C 2 p 5 q 5 v d 2 i m k O Y 7 r p l R 7 t G + u f R j p V n z s K p p w b q d R P K o Y D 9 7 o 3 E d / r t X P s 7 3 Y K o b M c Q f P Z o n 4 u K R o 6 C Z X 2 h A W O c u Q J 4 1 b 4 W y k f M M u 4 T 9 a F P o T 4 9 Z f f k t N m I 4 4 a 8 X G z t r d f x r F I N s h X s k V i s k P 2 y A E 5 I i 3 C y W 9 y S / 6 R u + A 6 + B v c B w + z 0 U p Q e t b J C 1 S C R x g 7 s X Y = < / l a t e x i t > 3 ⇥ 3 conv BatchNorm ReLU 2 ⇥ 2 max-pool < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 + B U z e A N F l 9 A T S F g S Z Y M 6 N 2 v R s s = " > A A A C R 3 i c b V A 9 T x t B E N 0 z C R + X B A y U N K v Y S D S x 7 k w B J S I N R Y Q A x Y D k c 6 y 9 9 R i v v B + n 3 T m E d f K / S 0 O b j r 9 A k y I I U b K 2 L + L z S S u 9 e T N P M / v S T A q H U X Q T V O Y + f J x f W F w K P 3 3 + s r x S X V 0 7 d S a 3 H F r c S G P P U + Z A C g 0 t F C j h P L P A V C r h L B 1 + n / T P L s E 6 Y f R P H G X Q U e x C i 7 7 g D L 3 U r f 5 K O G g E G 9 a 3 E x Q K H N 2 u 0 w T h C g t u 9 O U 4 S c J Z t c + Q D w 6 N V U / S C f x o T a p 6 s 7 Q 2 / 1 s V u / q W G S P H 3 W o t a k R T 0 L c k L k m N l D j q V v 8 k P c N z 5 Y / i k j n X j q M M O w W z K L i E c Z j k D j L G h + w C 2 p 5 q 5 v d 2 i m k O Y 7 r p l R 7 t G + u f R j p V n z s K p p w b q d R P K o Y D 9 7 o 3 E d / r t X P s 7 3 Y K o b M c Q f P Z o n 4 u K R o 6 C Z X 2 h A W O c u Q J 4 1 b 4 W y k f M M u 4 T 9 a F P o T 4 9 Z f f k t N m I 4 4 a 8 X G z t r d f x r F I N s h X s k V i s k P 2 y A E 5 I i 3 C y W 9 y S / 6 R u + A 6 + B v c B w + z 0 U p Q e t b J C 1 S C R x g 7 s X Y = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 + B U z e A N F l 9 A T S F g S Z Y M 6 N 2 v R s s = " > A A A C R 3 i c b V A 9 T x t B E N 0 z C R + X B A y U N K v Y S D S x 7 k w B J S I N R Y Q A x Y D k c 6 y 9 9 R i v v B + n 3 T m E d f K / S 0 O b j r 9 A k y I I U b K 2 L + L z S S u 9 e T N P M / v S T A q H U X Q T V O Y + f J x f W F w K P 3 3 + s r x S X V 0 7 d S a 3 H F r c S G P P U + Z A C g 0 t F C j h P L P A V C r h L B 1 + n / T P L s E 6 Y f R P H G X Q U e x C i 7 7 g D L 3 U r f 5 K O G g E G 9 a 3 E x Q K H N 2 u 0 w T h C g t u 9 O U 4 S c J Z t c + Q D w 6 N V U / S C f x o T a p 6 s 7 Q 2 / 1 s V u / q W G S P H 3 W o t a k R T 0 L c k L k m N l D j q V v 8 k P c N z 5 Y / i k j n X j q M M O w W z K L i E c Z j k D j L G h + w C 2 p 5 q 5 v d 2 i m k O Y 7 r p l R 7 t G + u f R j p V n z s K p p w b q d R P K o Y D 9 7 o 3 E d / r t X P s 7 3 Y K o b M c Q f P Z o n 4 u K R o 6 C Z X 2 h A W O c u Q J 4 1 b 4 W y k f M M u 4 T 9 a F P o T 4 9 Z f f k t N m I 4 4 a 8 X G z t r d f x r F I N s h X s k V i s k P 2 y A E 5 I i 3 C y W 9 y S / 6 R u + A 6 + B v c B w + z 0 U p Q e t b J C 1 S C R x g 7 s X Y = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 + B U z e A N F l 9 A T S F g S Z Y M 6 N 2 v R s s = " > A A A C R 3 i c b V A 9 T x t B E N 0 z C R + X B A y U N K v Y S D S x 7 k w B J S I N R Y Q A x Y D k c 6 y 9 9 R i v v B + n 3 T m E d f K / S 0 O b j r 9 A k y I I U b K 2 L + L z S S u 9 e T N P M / v S T A q H U X Q T V O Y + f J x f W F w K P 3 3 + s r x S X V 0 7 d S a 3 H F r c S G P P U + Z A C g 0 t F C j h P L P A V C r h L B 1 + n / T P L s E 6 Y f R P H G X Q U e x C i 7 7 g D L 3 U r f 5 K O G g E G 9 a 3 E x Q K H N 2 u 0 w T h C g t u 9 O U 4 S c J Z t c + Q D w 6 N V U / S C f x o T a p 6 s 7 Q 2 / 1 s V u / q W G S P H 3 W o t a k R T 0 L c k L k m N l D j q V v 8 k P c N z 5 Y / i k j n X j q M M O w W z K L i E c Z j k D j L G h + w C 2 p 5 q 5 v d 2 i m k O Y 7 r p l R 7 t G + u f R j p V n z s K p p w b q d R P K o Y D 9 7 o 3 E d / r t X P s 7 3 Y K o b M c Q f P Z o n 4 u K R o 6 C Z X 2 h A W O c u Q J 4 1 b 4 W y k f M M u 4 T 9 a F P o T 4 9 Z f f k t N m I 4 4 a 8 X G z t r d f x r F I N s h X s k V i s k P 2 y A E 5 I i 3 C y W 9 y S / 6 R u + A 6 + B v c B w + z 0 U p Q e t b J C 1 S C R x g 7 s X Y = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 + B U z e A N F l 9 A T S F g S Z Y M 6 N 2 v R s s = " > A A A C R 3 i c b V A 9 T x t B E N 0 z C R + X B A y U N K v Y S D S x 7 k w B J S I N R Y Q A x Y D k c 6 y 9 9 R i v v B + n 3 T m E d f K / S 0 O b j r 9 A k y I I U b K 2 L + L z S S u 9 e T N P M / v S T A q H U X Q T V O Y + f J x f W F w K P 3 3 + s r x S X V 0 7 d S a 3 H F r c S G P P U + Z A C g 0 t F C j h P L P A V C r h L B 1 + n / T P L s E 6 Y f R P H G X Q U e x C i 7 7 g D L 3 U r f 5 K O G g E G 9 a 3 E x Q K H N 2 u 0 w T h C g t u 9 O U 4 S c J Z t c + Q D w 6 N V U / S C f x o T a p 6 s 7 Q 2 / 1 s V u / q W G S P H 3 W o t a k R T 0 L c k L k m N l D j q V v 8 k P c N z 5 Y / i k j n X j q M M O w W z K L i E c Z j k D j L G h + w C 2 p 5 q 5 v d 2 i m k O Y 7 r p l R 7 t G + u f R j p V n z s K p p w b q d R P K o Y D 9 7 o 3 E d / r t X P s 7 3 Y K o b M c Q f P Z o n 4 u K R o 6 C Z X 2 h A W O c u Q J 4 1 b 4 W y k f M M u 4 T 9 a F P o T 4 9 Z f f k t N m I 4 4 a 8 X G z t r d f x r F I N s h X s k V i s k P 2 y A E 5 I i 3 C y W 9 y S / 6 R u + A 6 + B v c B w + z 0 U p Q e t b J C 1 S C R x g 7 s X Y = < / l a t e x i t > g < l a t e x i t s h a 1 _ b a s e 6 4 = " f 3 l o b H V c Y C U 7 Q l a K o G R q P G r m S l E = " > A A A B 7 3 i c b V B N S 8 N A E J 3 U r 1 q / q h 6 9 L B b B U 0 l E 0 G P R i 8 c K p i 2 0 o W y 2 m 3 b p Z h N 3 J 0 I J / R N e P C j i 1 b / j z X / j t s 1 B W x 8 M P N 6 b Y W Z e m E p h 0 H W / n d L a + s b m V n m 7 s r O 7 t 3 9 Q P T x q m S T T j P s s k Y n u h N R w K R T 3 U a D k n V R z G o e S t 8 P x 7 c x v P 3 F t R K I e c J L y I K Z D J S L B K F q p M + z n v X Q k p v 1 q z a 2 7 c 5 B V 4 h W k B g W a / e p X b 5 C w L O Y K m a T G d D 0 3 x S C n G g W T f F r p Z Y a n l I 3 p k H c t V T T m J s j n 9 0 7 J m V U G J E q 0 L Y V k r v 6 e y G l s z C Q O b W d M c W S W v Z n 4 n 9 f N M L o O c q H S D L l i i 0 V R J g k m Z P Y 8 G Q j N G c q J J Z R p Y W 8 l b E Q 1 Z W g j q t g Q v O W X V 0 n

V c Y C U 7 Q l a K o G R q P G r m S l E = " > A A A B 7 3 i c b V B N S 8 N A E J 3 U r 1 q / q h 6 9 L B b B U 0 l E 0 G P R i 8 c K p i 2 0 o W y 2 m 3 b p Z h N 3 J 0 I J / R N e P C j i 1 b / j z X / j t s 1 B W x 8 M P N 6 b Y W Z e m E p h 0 H W / n d L a + s b m V n m 7 s r O 7 t 3 9 Q P T x q m S T T j P s s k Y n u h N R w K R T 3 U a D k n V R z G o e S t 8 P x 7 c x v P 3 F t R K I e c J L y I K Z D J S L B K F q p M + z n v X Q k p v 1 q z a 2 7 c 5 B V 4 h W k B g W a / e p X b 5 C w L O Y K m a T G d D 0 3 x S C n G g W T f F r p Z Y a n l I 3 p k H c t V T T m J s j n 9 0 7 J m V U G J E q 0 L Y V k r v 6 e y G l s z C Q O b W d M c W S W v Z n 4 n 9 f N M L o O c q H S D L l i i 0 V R J g k m Z P Y 8 G Q j N G c q J J Z R p Y W 8 l b E Q 1 Z W g j q t g Q v O W X V 0 n

V c Y C U 7 Q l a K o G R q P G r m S l E = " > A A A B 7 3 i c b V B N S 8 N A E J 3 U r 1 q / q h 6 9 L B b B U 0 l E 0 G P R i 8 c K p i 2 0 o W y 2 m 3 b p Z h N 3 J 0 I J / R N e P C j i 1 b / j z X / j t s 1 B W x 8 M P N 6 b Y W Z e m E p h 0 H W / n d L a + s b m V n m 7 s r O 7 t 3 9 Q P T x q m S T T j P s s k Y n u h N R w K R T 3 U a D k n V R z G o e S t 8 P x 7 c x v P 3 F t R K I e c J L y I K Z D J S L B K F q p M + z n v X Q k p v 1 q z a 2 7 c 5 B V 4 h W k B g W a / e p X b 5 C w L O Y K m a T G d D 0 3 x S C n G g W T f F r p Z Y a n l I 3 p k H c t V T T m J s j n 9 0 7 J m V U G J E q 0 L Y V k r v 6 e y G l s z C Q O b W d M c W S W v Z n 4 n 9 f N M L o O c q H S D L l i i 0 V R J g k m Z P Y 8 G Q j N G c q J J Z R p Y W 8 l b E Q 1 Z W g j q t g Q v O W X V 0 n r o u 6 5 d e / + s t a 4 K e I o w w m c w j l 4 c A U N u I M m + M B A w j O 8 w p v z 6 L w 4 7 8 7 H o r X k F D P H 8 A f O 5 w 9 U c J A l < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " f 3 l o b H V c Y C U 7 Q l a K o G R q P G r m S l E = " > A A A B 7 3 i c b V B N S 8 N A E J 3 U r 1 q / q h 6 9 L B b B U 0 l E 0 G P R i 8 c K p i 2 0 o W y 2 m 3 b p Z h N 3 J 0 I J / R N e P C j i 1 b / j z X / j t s 1 B W x 8 M P N 6 b Y W Z e m E p h 0 H W / n d L a + s b m V n m 7 s r O 7 t 3 9 Q P T x q m S T T j P s s k Y n u h N R w K R T 3 U a D k n V R z G o e S t 8 P x 7 c x v P 3 F t R K I e c J L y I K Z D J S L B K F q p M + z n v X Q k p v 1 q z a 2 7 c 5 B V 4 h W k B g W a / e p X b 5 C w L O Y K m a T G d D 0 3 x S C n G g W T f F r p Z Y a n l I 3 p k H c t V T T m J s j n 9 0 7 J m V U G J E q 0 L Y V k r v 6 e y G l s z C Q O b W d M c W S W v Z n 4 n 9 f N M L o O c q H S D L l i i 0 V R J g k m Z P Y 8 G Q j N G c q J J Z R p Y W 8 l b E Q 1 Z W g j q t g Q v O W X V 0 n r o u 6 5 d e / + s t a 4 K e I o w w m c w j l 4 c A U N u I M m + M B A w j O 8 w p v z 6 L w 4 7 8 7 H o r X k F D P H 8 A f O 5 w 9 U c J A l < / l a t e x i t > FC layer 1 FC layer 2 < l a t e x i t s h a 1 _ b a s e 6 4 = " 9 V 9 R Q t S s z 6 S 5 U t e G s A o r V S e n S G Y = " > A A A C D H i c b V C 7 T s M w F H X K q 4 R X g Z H F o k V i q p I u M F Z U Q o x F o g + p i S r H v W m t O k 5 k O 0 h R 1 A 9 g 4 V d Y G E C I l Q 9 g 4 2 9 w H 0 N p O Z K l 4 3 P u v f Y 9 Q c K Z 0 o 7 z Y x U 2 N r e 2 d 4 q 7 9 t 7 + w e F R 6 f i k r e J U U m j R m M e y G x A F n A l o a a Y 5 d B M J J A o 4 d I J x Y + p 3 H k E q F o s H n S X g R 2 Q o W M g o 0 U b q l 8 o e B a F B 2 r c N z E k G E l f c i u c t X W s V U + V U n R n w O n E X p I w W a P Z L 3 9 4 g p m l k J l N O l O q 5 T q L 9 n E j N K I e J 7 a U K E k L H Z A g 9 Q w W J Q P n 5 b J k J v j D K A I e x N E d o P F O X O 3 I S K Z V F g a m M i B 6 p V W 8 q / u f 1 U h 1 e + z k T S a p B 0 P l D Y c q x j v E 0 G T x g E q j m m S G E S m b + i u m I S E J N P M o 2 I b i r K 6 + T d q 3 q O l X 3 v l a u 3 y z i K K I z d I 4 u k Y u u U B 3 d o S Z q I Y q e 0 A t 6 Q + / W s / V q f V i f 8 9 K C t e g 5 R X 9 g f f 0 C G h a Y b w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 9 V 9 R Q t S s z 6 S 5 U t e G s A o r V S e n S G Y = " > A A A C D H i c b V C 7 T s M w F H X K q 4 R X g Z H F o k V i q p I u M F Z U Q o x F o g + p i S r H v W m t O k 5 k O 0 h R 1 A 9 g 4 V d Y G E C I l Q 9 g 4 2 9 w H 0 N p O Z K l 4 3 P u v f Y 9 Q c K Z 0 o 7 z Y x U 2 N r e 2 d 4 q 7 9 t 7 + w e F R 6 f i k r e J U U m j R m M e y G x A F n A l o a a Y 5 d B M J J A o 4 d I J x Y + p 3 H k E q F o s H n S X g R 2 Q o W M g o 0 U b q l 8 o e B a F B 2 r c N z E k G E l f c i u c t X W s V U + V U n R n w O n E X p I w W a P Z L 3 9 4 g p m l k J l N O l O q 5 T q L 9 n E j N K I e J 7 a U K E k L H Z A g 9 Q w W J Q P n 5 b J k J v j D K A I e x N E d o P F O X O 3 I S K Z V F g a m M i B 6 p V W 8 q / u f 1 U h 1 e + z k T S a p B 0 P l D Y c q x j v E 0 G T x g E q j m m S G E S m b + i u m I S E J N P M o 2 I b i r K 6 + T d q 3 q O l X 3 v l a u 3 y z i K K I z d I 4 u k Y u u U B 3 d o S Z q I Y q e 0 A t 6 Q + / W s / V q f V i f 8 9 K C t e g 5 R X 9 g f f 0 C G h a Y b w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 9 V 9 R Q t S s z 6 S 5 U t e G s A o r V S e n S G Y = " > A A A C D H i c b V C 7 T s M w F H X K q 4 R X g Z H F o k V i q p I u M F Z U Q o x F o g + p i S r H v W m t O k 5 k O 0 h R 1 A 9 g 4 V d Y G E C I l Q 9 g 4 2 9 w H 0 N p O Z K l 4 3 P u v f Y 9 Q c K Z 0 o 7 z Y x U 2 N r e 2 d 4 q 7 9 t 7 + w e F R 6 f i k r e J U U m j R m M e y G x A F n A l o a a Y 5 d B M J J A o 4 d I J x Y + p 3 H k E q F o s H n S X g R 2 Q o W M g o 0 U b q l 8 o e B a F B 2 r c N z E k G E l f c i u c t X W s V U + V U n R n w O n E X p I w W a P Z L 3 9 4 g p m l k J l N O l O q 5 T q L 9 n E j N K I e J 7 a U K E k L H Z A g 9 Q w W J Q P n 5 b J k J v j D K A I e x N E d o P F O X O 3 I S K Z V F g a m M i B 6 p V W 8 q / u f 1 U h 1 e + z k T S a p B 0 P l D Y c q x j v E 0 G T x g E q j m m S G E S m b + i u m I S E J N P M o 2 I b i r K 6 + T d q 3 q O l X 3 v l a u 3 y z i K K I z d I 4 u k Y u u U B 3 d o S Z q I Y q e 0 A t 6 Q + / W s / V q f V i f 8 9 K C t e g 5 R X 9 g f f 0 C G h a Y b w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 9 V 9 R Q t S s z 6 S 5 U t e G s A o r V S e n S G Y = " > A A A C D H i c b V C 7 T s M w F H X K q 4 R X g Z H F o k V i q p I u M F Z U Q o x F o g + p i S r H v W m t O k 5 k O 0 h R 1 A 9 g 4 V d Y G E C I l Q 9 g 4 2 9 w H 0 N p O Z K l 4 3 P u v f Y 9 Q c K Z 0 o 7 z Y x U 2 N r e 2 d 4 q 7 9 t 7 + w e F R 6 f i k r e J U U m j R m M e y G x A F n A l o a a Y 5 d B M J J A o 4 d I J x Y + p 3 H k E q F o s H n S X g R 2 Q o W M g o 0 U b q l 8 o e B a F B 2 r c N z E k G E l f c i u c t X W s V U + V U n R n w O n E X p I w W a P Z L 3 9 4 g p m l k J l N O l O q 5 T q L 9 n E j N K I e J 7 a U K E k L H Z A g 9 Q w W J Q P n 5 b J k J v j D K A I e x N E d o P F O X O 3 I S K Z V F g a m M i B 6 p V W 8 q / u f 1 U h 1 e + z k T S a p B 0 P l D Y c q x j v E 0 G T x g E q j m m S G E S m b + i u m I S E J N P M o 2 I b i r K 6 + T d q 3 q O l X 3 v l a u 3 y z i K K I z d I 4 u k Y u u U B 3 d o S Z q I Y q e 0 A t 6 Q + / W s / V q f V i f 8 9 K C t e g 5 R X 9 g f f 0 C G h a Y b w = = < / l a t e x i t >

Graph construction in each episode:

We follow the episodic paradigm for few-shot meta-learner training. This means that the graph is individually constructed for each task in each episode, as shown in Figure 1 . Typically, in 5-way 5-shot training, N = 5, K = 5, T = 75, the dimension of W is only 100 × 100, which is quite efficient.

3.2.3. LABEL PROPAGATION:

We now describe how to get predictions for the query set Q using label propagation, before the last cross-entropy loss step. Let F denote the set of (N × K + T ) × N matrix with nonnegative entries. We define a label matrix Y ∈ F with Y ij = 1 if x i is from the support set and labeled as y i = j, otherwise Y ij = 0. Starting from Y , label propagation iteratively determines the unknown labels of instances in the union set S ∪ Q according to the graph structure using the following formulation:

EQUATION

where F t ∈ F denotes the predicted labels at the timestamp t, S denotes the normalized weight, and α ∈ (0, 1) controls the amount of propagated information. It is well known that the sequence {F t } has a closed-form solution as follows:

EQUATION

)

where I is the identity matrix (Zhou et al., 2004) . We directly utilize this result for the label propagation, making a whole episodic meta-learning procedure more efficient in practice.

Time complexity Matrix inversion originally takes O(n 3 ) time complexity, which is inefficient for large n. However, in our setting, n = N × K + T (80 for 1-shot and 100 for 5-shot) is very small. Moreover, there is plenty of prior work on the scalability and efficiency of label propagation, such as Liang and Li (2018); Fujiwara and Irie (2014) , which can extend our work to large-scale data. More discussions are presented in A.4

3.2.4. CLASSIFICATION LOSS GENERATION:

The objective of this step is to compute the classification loss between the predictions of the union of support and query set via label propagation and the ground-truths. We compute the cross-entropy loss between predicted scores F * and ground-truth labels from S ∪ Q to learn all parameters in an end-to-end fashion, where F * is converted to probabilistic score using softmax: Note that in Equation ( 6), the loss is dependent on two set of parameters ϕ, φ (even though the dependency is implicit through F * ij ). All these parameters are jointly updated by the episodic training in an end-to-end manner.

EQUATION

4. EXPERIMENTS:

We evaluate and compare our TPN with state-of-the-art approaches on two datasets, i.e., miniImageNet (Ravi and Larochelle, 2017) and tieredImageNet (Ren et al., 2018) . The former is the most popular few-shot learning benchmark and the latter is a much larger dataset released recently for few-shot learning.

4.1. DATASETS:

miniImageNet. The miniImageNet dataset is a collection of Imagenet (Krizhevsky et al., 2012) for few-shot image recognition. It is composed of 100 classes randomly selected from Imagenet with each class containing 600 examples. In order to directly compare with state-of-the-art algorithms for few-shot learning, we rely on the class splits used by Ravi and Larochelle (2017) , which includes 64 classes for training, 16 for validation, and 20 for test. All images are resized to 84 × 84 pixels.

tieredImageNet. Similar to miniImageNet , tieredImageNet (Ren et al., 2018) is also a subset of Imagenet (Krizhevsky et al., 2012) , but it has a larger number of classes from ILSVRC-12 (608 classes rather than 100 for miniImageNet). Different from miniImageNet, it has a hierarchical structure of broader categories corresponding to high-level nodes in Imagenet. The top hierarchy has 34 categories, which are divided into 20 training (351 classes), 6 validation (97 classes) and 8 test (160 classes) categories. The average number of examples in each class is 1281. This high-level split strategy ensures that the training classes are distinct from the test classes semantically. This is a more challenging and realistic few-shot setting since there is no assumption that training classes should be similar to test classes. Similarly, all images are resized to 84 × 84 pixels.

4.2. EXPERIMENTAL SETUP:

For fair comparison with other methods, we adopt a widely-used CNN (Finn et al., 2017; Snell et al., 2017) as the feature embedding function f ϕ (Section 3.2.1). The hyper-parameter k of k-nearest neighbour graph (Section 3.2.2) is set to 20 and α of label propagation is set to 0.99, as suggested in Zhou et al. (2004) .

Following Snell et al. (2017) , we adopt the episodic training procedure, i.e, we sample a set of N -way K-shot training tasks to mimic the N -way K-shot test problems. Moreover, Snell et al. (2017) proposed a "Higher Way " training strategy which used more training classes in each episode than test case. However, we find that it is beneficial to train with more examples than test phase (Appendix A.1). This is denoted as "Higher Shot" in our experiments. For 1-shot and 5-shot test problem, we adopt 5-shot and 10-shot training respectively. In all settings, the query number is set to 15 and the performance are averaged over 600 randomly generated episodes from the test set.

All our models were trained with Adam (Kingma and Ba, 2015) and an initial learning rate of 10 -3 . For miniImageNet, we cut the learning rate in half every 10, 000 episodes and for tieredImageNet, we cut the learning rate every 25, 000 episodes. The reason for larger decay step is that tieredImageNet has more classes and more examples in each class which needs larger training iterations. We ran the training process until the validation loss reached a plateau.

4.3. FEW-SHOT LEARNING RESULTS:

We compare our method with several state-of-the-art approaches in various settings. Even though the transductive method has never been used explicitly, batch normalization layer was used transductively to share information between test examples. For example, in Finn et al. (2017) ; Nichol et al. (2018) , they use the query batch statistics rather than global BN parameters for the prediction, which leads to performance gain in the query set. Besides, we propose two simple transductive methods as baselines that explicitly utilize the query set. First, we propose the MAML+Transduction with slight modification of loss function to: J (θ) = T i=1 y i log P( y i |x i ) +

N ×K+T i,j=1 W ij y i -y j 2 2

for transductive inference. The additional term serves as transductive regularization. Second, the naive heuristic-based label propagation methods (Zhou et al., 2004) is proposed to explicitly model the transductive inference.

Experimental results are shown in Table 1 and Table2 . Transductive batch normalization methods tend to perform better than pure inductive methods except for the "Higher Way" PROTO NET. Label propagation without learning to propagate outperforms other baseline methods in most cases, which verifies the necessity of transduction. The proposed TPN achieves the state-of-the-art results and surpasses all the others with a large margin even when the model is trained with regular shots. When "Higher Shot" is applied, the performance of TPN continues to improve especially for 1-shot case. This confirms that our model effectively finds the episodic-wise manifold structure of test examples through learning to construct the graph for label propagation.

Another observation is that the advantages of 5-shot classification is less significant than that of 1shot case. For example, in 5-way miniImageNet , the absolute improvement of TPN over published state-of-the-art is 4.13% for 1-shot and 1.66% for 5-shot. To further investigate this, we experimented 5-way k-shot (k = 1, 2, • • • , 10) experiments. The results are shown in Figure 4 . Our TPN performs consistently better than other methods with varying shots. Moreover, it can be seen that we propose a semi-supervised version of TPN, named TPN-semi, which classifies one test example each time by propagating labels from the labeled set and extra unlabeled set.

We use miniImageNet and tieredImageNet with the labeled/unlabeled data split proposed by Ren et al. (2018) . Specifically, they split the images of each class into disjoint labeled and unlabeled sets. For miniImageNet, the ratio of labeled/unlabeled data is 40% and 60% in each class. Likewise, the ratio is 10% and 90% for tieredImageNet. All semi-supervised methods (including TPN-semi) sample support/query data from the labeled set (e.g, 40% from miniImageNet) and sample unlabeled data from the unlabeled sets (e.g, 60% from miniImageNet). In addition, there is a more challenging situation where many unlabelled examples from other distractor classes (different from labelled classes).

Following Ren et al. (2018) , we report the average accuracy over 10 random labeled/unlabeled splits and the uncertainty computed in standard error. Results are shown in Table 3 and Table 4 . It can be seen that TPN-semi outperforms all other algorithms with a large margin, especially for 1-shot case. Although TPN is originally designed to perform transductive inference, we show that it can be successfully adapted to semi-supervised learning tasks with little modification. In certain cases where we can not get all test data, the TPN-semi can be used as an effective alternative algorithm.

5. CONCLUSION:

In this work, we proposed the transductive setting for few-shot learning. Our proposed approach, namely Transductive Propagation Network (TPN), utilizes the entire test set for transductive inference. Specifically, our approach is composed of four steps: feature embedding, graph construction, label propagation, and loss computation. Graph construction is a key step that produces examplewise parameters to exploit the manifold structure in each episode. In our method, all parameters are learned end-to-end using cross-entropy loss with respect to the ground truth labels and the prediction scores in the query set. We obtained the state-of-the-art results on miniImageNet and tieredImageNet. Also, the semi-supervised adaptation of our algorithm achieved higher results than other semi-supervised methods. In future work, we are going to explore the episodic-wise distance metric rather than only using example-wise parameters for the Euclidean distance.

:



Figures and Tables:

Given a relatively large labeled dataset with a set of classes C train , the objective of this setting is to train classifiers for an unseen set of novel classes C test , for which only a few labeled examples are available. Specifically, in each episode, a small subset of N classes are sampled from C train to construct a support set and a query set. The support set contains K examples from each of the N classes (i.e., N -way K-shot setting) denoted as S = {(x 1 , y 1 ), (x 2 , y 2 ), . . . , (x N ×K , y N ×K )}, while the query set Q = {(x * 1 , y * 1 ), (x * 2 , y * 2 ), . . . , (x * T , y * T )} includes different examples from the same N classes. Here, the support set S in each episode serves as the labeled training set on which the model is trained to minimize the loss of its predictions for the query set Q. This procedure mimics training classifiers for C test and goes episode by episode until convergence.

r o u 6 5 d e / + s t a 4 K e I o w w m c w j l 4 c A U N u I M m + M B A w j O 8 w p v z 6 L w 4 7 8 7 H o r X k F D P H 8 A f O 5 w 9 U c J A l < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " f 3 l o b H

r o u 6 5 d e / + s t a 4 K e I o w w m c w j l 4 c A U N u I M m + M B A w j O 8 w p v z 6 L w 4 7 8 7 H o r X k F D P H 8 A f O 5 w 9 U c J A l < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " f 3 l o b H

Figure 3: Detailed architecture of the graph construction module, in which the length-scale parameter is example-wisely determined.

)Here, ỹi denotes the final predicted label for ith instance in the union of support and query set and F * ij denotes the jth component of predicted label from label propagation. Then the loss function is computed as:y i == j) log(P ( ỹi = j|x i )) ,(6)where y i means the ground-truth label of x i and I(b) is an indicator function, I(b) = 1 if b is true and 0 otherwise.

Figure 4: 5-way performance with various training/test shots.

Semi-supervised comparison on tieredImageNet. "w/D" means with distraction. In this setting, many of the unlabelled data are from the so-called distraction classes , which is different from the classes of labelled data.† Due to space limitation, we report the accuracy with 95% confidence intervals in Appendix.

Table content: <table><tr><td>Model</td><td colspan="4">1-shot 5-shot 1-shot w/D 5-shot w/D</td></tr><tr><td>Soft k-Means (Ren et al., 2018)</td><td>51.52</td><td>70.25</td><td>49.88</td><td>68.32</td></tr><tr><td colspan="2">Soft k-Means+Cluster (Ren et al., 2018) 51.85</td><td>69.42</td><td>51.36</td><td>67.56</td></tr><tr><td>Masked Soft k-Means (Ren et al., 2018)</td><td>52.39</td><td>69.88</td><td>51.38</td><td>69.08</td></tr><tr><td>TPN-semi</td><td>55.74</td><td>71.01</td><td>53.45</td><td>69.93</td></tr></table>

Peer review (iclr2019):
 The paper studies few-host learning in a transductive setting: using meta learning to learn to propagate labels from training samples to test samples. There is nothing strikingly novel in this work, using unlabeled test samples in a transductive way seem to help slightly. However, the paper does cover a setup that I am not aware that was studied before. The paper is written clearly, and the experiments seem solid. Comments: -- What can be said about how computationally demanding the procedure is? running label propagation within meta learning might be too costly. -- It is not clear how the per-example scalar sigma-i is learned. (for Eq 2) -- solving Eq 3 by matrix inversion does not scale. Would be best to also show results using iterative optimization