Title: FEW-SHOT TEXT CLASSIFICATION WITH DISTRIBUTIONAL SIGNATURES

Abstract: In this paper, we explore meta-learning for few-shot text classification. Metalearning has shown strong performance in computer vision, where low-level patterns are transferable across learning tasks. However, directly applying this approach to text is challenging-lexical features highly informative for one task may be insignificant for another. Thus, rather than learning solely from words, our model also leverages their distributional signatures, which encode pertinent word occurrence patterns. Our model is trained within a meta-learning framework to map these signatures into attention scores, which are then used to weight the lexical representations of words. We demonstrate that our model consistently outperforms prototypical networks learned on lexical knowledge (Snell et al., 2017) in both few-shot text classification and relation classification by a significant margin across six benchmark datasets (20.0% on average in 1-shot classification). 1 * Equal contribution. 1 Our code is available at https://github.com/YujiaBao/Distributional-Signatures.
In meta-learning, we emulate the above testing scenario during meta-training so our model learns to quickly learn from a few annotations. To create a single training episode, we first sample N classes from Y train . For each of these N classes, we sample K examples as our training data and L examples as our testing data. We update our model based on loss over these testing data. Figure 4a shows an example of an episode. We repeat this procedure to increase the number of training episodes, each of which is constructed over its own set of N classes. In literature, the training data of one episode is commonly denoted as the support set, while the corresponding testing data is known as the query set. Given the support set, we refer to the task of making predictions over the query set as N -way K-shot classification.
Meta-testing After we have finished meta-training, we apply the same episode-based mechanism to test whether our model can indeed adapt quickly to new classes. To create a testing episode, we

1. INTRODUCTION:

In computer vision, meta-learning has emerged as a promising methodology for learning in a lowresource regime. Specifically, the goal is to enable an algorithm to expand to new classes for which only a few training instances are available. These models learn to generalize in these low-resource conditions by recreating such training episodes from the data available. Even in the most extreme low-resource scenario-a single training example per class-this approach yields 99.6% accuracy on the character recognition task (Sung et al., 2018) .

Given this strong empirical performance, we are interested in employing meta-learning frameworks in NLP. The challenge, however, is the degree of transferability of the underlying representation learned across different classes. In computer vision, low-level patterns (such as edges) and their corresponding representations can be shared across tasks. However, the situation is different for language data where most tasks operate at the lexical level. Words that are highly informative for one task may not be relevant for other tasks. Consider, for example, the corpus of HuffPost headlines, categorized into 41 classes. Figure 1 shows that words highly salient for one class do not play a significant role in classifying others. Not surprisingly, when meta-learning is applied directly on lexical inputs, its performance drops below a simple nearest neighbor classifier. The inability of a traditional meta-learner to zoom-in on important features is further illustrated in Figure 2 : when considering the target class fifty (lifestyle for middle-aged), the standard prototypical network (Snell et al., 2017) attends to uninformative words like "date," while downplaying highly predictive words such as "grandma."

In this paper we demonstrate that despite these variations, we can effectively transfer representations across classes and thereby enable learning in a low-resource regime. Instead of directly considering words, our method utilizes their distributional signatures, characteristics of the underlying word distributions, which exhibit consistent behaviour across classification tasks. Within the meta-learning framework, these signatures enable us to transfer attention across tasks, which can consequently be used to weight the lexical representations of words. One broadly used example of such distributional We compute the local mutual information (LMI) (Evert, 2005) between words and classes. For each class, we include its top 2 LMI-ranked words. Darker colors indicate higher LMI. signatures is tf-idf weighting, which explicitly specifies word importance in terms of its frequency in a document collection, and its skewness within a specific document.

Building on this idea, we would like to learn to utilize distributional signatures in the context of cross-class transfer. In addition to word frequency, we assess word importance with respect to a specific class. This latter relation cannot be reliably estimated of the target class due to the scarcity of labeled data. However, we can obtain a noisy estimate of this indicator by utilizing the few provided training examples for the target class, and then further refine this approximation within the meta-learning framework. We note that while the representational power of distributional signatures is weaker than that of their lexical counterparts, meta knowledge built on distributional signatures are better able to generalize.

Our model consists of two components. The first is an attention generator, which translates distributional signatures into attention scores that reflect word importance for classification. Informed by the attention generator's output, our second component, a ridge regressor, quickly learns to make predictions after seeing only a few training examples. The attention generator is shared across all episodes, while the ridge regressor is trained from scratch for each individual episode. The latter's prediction loss provides supervision for the attention generator. Theoretically, we show that the attention generator is robust to word-substitution perturbations.

We evaluate our model on five standard text classification datasets (Lang, 1995; Lewis et al., 2004; Lewis, 1997; He & McAuley, 2016; Misra, 2018) and one relation classification dataset (Han et al., 2018) . Experimental results demonstrate that our model delivers significant performance gains over all baselines. For instance, our model outperforms prototypical networks by 20.6% on average in one-shot text classification and 17.3% in one-shot relation classification. In addition, both qualitative and quantitative analyses confirm that our model generates high-quality attention for unseen classes.

2. RELATED WORK:

Meta-learning Meta-learning has been shown to be highly effective in computer vision, where lowlevel features are transferable across classes. Existing approaches include learning a metric space over input features (Koch, 2015; Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018) , developing a prior over the optimization procedure (Ravi & Larochelle, 2016; Finn et al., 2017; Nichol & Schulman, 2018; Antoniou et al., 2018) , and exploiting the relations between classes (Garcia & Bruna, 2017) . These methods have been adapted with some success to specific applications in NLP, including machine translation (Gu et al., 2018 ), text classification (Yu et al., 2018; Geng et al., 2019; Guo et al., 2018; Jiang et al., 2019) and relation classification (Han et al., 2018) . Such models primarily build meta-knowledge on lexical representations. However, as our experiments show, there exist innate differences in transferable knowledge between image data and language data, and lexicon-aware meta-learners fail to generalize on standard multiclass classification datasets.

In this work, we observe that even though salient features in text may not be transferable, their distributional behaviors are alike. Thus, we focus on learning the connection between word importance and distributional signatures. As a result, our model can reliably identify important features from novel classes.

Transfer learning Our work is also closely related to transfer learning: we assume access to a large number of labeled examples from source classes, and we would like to identify word importance for the target classification task. Current approaches transfer knowledge from the source to the target by either fine-tuning a pre-trained encoder (Howard & Ruder, 2018; Peters et al., 2018; Radford et al., 2018; Bertinetto et al., 2019) , or multi-task learning with a shared encoder (Collobert & Weston, 2008; Liu et al., 2015; Luong et al., 2015; Strubell et al., 2018) . Recently, Bao et al. (2018) also successfully transferred task-specific attention through human rationales.

In contrast to these methods, where the transfer mechanism is pre-designed, we learn to transfer based on the performance of downstream tasks. Specifically, we utilize distributional statistics to transfer attention across tasks. We note that while Wei et al. (2017) and Sun et al. (2018) also learn transfer mechanisms for image recognition, their methods do not directly apply to NLP.

3. BACKGROUND:

In this section, we first summarize the standard meta-learning framework for few-shot classification and describe the terminology (Vinyals et al., 2016) . Next, we introduce our extensions to the framework. Figure 3 and 4 graphically illustrate our framework.

Problem statement Suppose we are given labeled examples from a set of classes Y train . Our goal is to develop a model that acquires knowledge from these training data, so that we can make predictions over new (but related) classes, for which we only have a few annotations. first sample N new classes from Y test . Then we sample the support set and the query set from the N classes. We evaluate the average performance on the query set across all testing episodes.

Our extension:

We observe that even though all examples from Y train are accessible throughout meta-training, the standard meta-learning framework (Vinyals et al., 2016) only learns from small subsets of these data per training episode. In contrast, our model leverages distributional statistics over all training examples for more robust inference. To accommodate this adjustment, we augment each episode with a source pool (Figure 4b ). During meta-training (Figure 3a ), this source pool includes all examples from training classes not selected for the particular episode. During metatesting (Figure 3b ), this source pool includes all training examples.

4. METHOD:

Overview Our goal is to improve few-shot classification performance by learning high-quality attention from the distributional signatures of the inputs. Given a particular episode, we extract relevant statistics from the source pool and the support set. Since these statistics only roughly approximate word importance for classification, we utilize an attention generator to translate them into high-quality attention that operates over words. This generated attention provides guidance for the downstream predictor, a ridge regressor, to quickly learn from a few labeled examples.2 

We note that the attention generator is optimized over all training episodes, while the ridge regressor is trained from scratch for each episode. Figure 5 illustrates the two components of our model.

• Attention generator: This module generates class-specific attention by combining the distributional statistics of the source pool and the support set (Figure 5a ). The generated attention provides the ridge regressor an inductive bias on the word importance. We train this module based on feedback from the ridge regressor (Section 4.1).

• Ridge regressor: For each episode, this module constructs lexical representations using the attention derived from distributional signatures (Figure 5b ). The goal of this module is to make predictions over the query set, after learning from the support set (Figure 5c and 5d ). Its prediction loss is end-to-end differentiable with respect to the attention generator which leads to efficient training (Section 4.2).

In our theoretical analysis, we show that the attention generator's outputs are invariant to wordsubstitution perturbations (Section 4.3).

4.1. ATTENTION GENERATOR:

The goal of the attention generator is to assess word importance from the distributional signatures of each input example. There are many choices of distributional signatures. Among them, we focus on functions of unigram statistics, which are provably robust to word-substitution perturbations (Section 4.3). We utilize the large source pool to inform the model of general word importance and leverage the small support set to estimate class-specific word importance. The generated attention will be used later to construct the input representation for downstream classification.

W < l a t e x i t s h a 1 _ b a s e 6 4 = " Q W 0 f 9 v L n V t b 9 / N G z h 8 C p q x t v 8 + o = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E e q x 6 M V j C / Y D 2 l A 2 2 0 m 7 d r M J u x u h h P 4 C L x 4 U 8 e p P 8 u a / c d v m o K 0 v L D y 8 M 8 P O v E E i u D a u + + 0 U N j a 3 t n e K u 6 W 9 / Y P D o / L x S V v H q W L Y Y r G I V T e g G g W X 2 D L c C O w m C m k U C O w E k 7 t 5 v f O E S v N Y P p h p g n 5 E R 5 K H n F F j r W Z n U K 6 4 V X c h s g 5 e D h X I 1 R i U v / r D m K U R S s M E 1 b r n u Y n x M 6 o M Z w J n p X 6 q M a F s Q k f Y s y h p h N r P F o v O y I V 1 h i S M l X 3 S k I X 7 e y K j k d b T K L C d E T V j v V q b m / / V e q k J b / y M y y Q 1 K N n y o z A V x M R k f j U Z c o X M i K k F y h S 3 u x I 2 p o o y Y 7 M p 2 R C 8 1 Z P X o X 1 V 9 S w 3 r y v 1 2 z y O I p z B O V y C B z W o w z 0 0 o A U M E J 7 h F d 6 c R + f F e X c + l q 0 F J 5 8 5 h T 9 y P n 8 A t D e M 2 w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Q W 0 f 9 v L n V t b 9 / N G z h 8 C p q x t v 8 + o = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E e q x 6 M V j C / Y D 2 l A 2 2 0 m 7 d r M J u x u h h P 4 C L x 4 U 8 e p P 8 u a / c d v m o K 0 v L D y 8 M 8 P O v E E i u D a u + + 0 U N j a 3 t n e K u 6 W 9 / Y P D o / L x S V v H q W L Y Y r G I V T e g G g W X 2 D L c C O w m C m k U C O w E k 7 t 5 v f O E S v N Y P p h p g n 5 E R 5 K H n F F j r W Z n U K 6 4 V X c h s g 5 e D h X I 1 R i U v / r D m K U R S s M E 1 b r n u Y n x M 6 o M Z w J n p X 6 q M a F s Q k f Y s y h p h N r P F o v O y I V 1 h i S M l X 3 S k I X 7 e y K j k d b T K L C d E T V j v V q b m / / V e q k J b / y M y y Q 1 K N n y o z A V x M R k f j U Z c o X M i K k F y h S 3 u x I 2 p o o y Y 7 M p 2 R C 8 1 Z P X o X 1 V 9 S w 3 r y v 1 2 z y O I p z B O V y C B z W o w z 0 0 o A U M E J 7 h F d 6 c R + f F e X c + l q 0 F J 5 8 5 h T 9 y P n 8 A t D e M 2 w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Q W 0 f 9 v L n V t b 9 / N G z h 8 C p q x t v 8 + o = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E e q x 6 M V j C / Y D 2 l A 2 2 0 m 7 d r M J u x u h h P 4 C L x 4 U 8 e p P 8 u a / c d v m o K 0 v L D y 8 M 8 P O v E E i u D a u + + 0 U N j a 3 t n e K u 6 W 9 / Y P D o / L x S V v H q W L Y Y r G I V T e g G g W X 2 D L c C O w m C m k U C O w E k 7 t 5 v f O E S v N Y P p h p g n 5 E R 5 K H n F F j r W Z n U K 6 4 V X c h s g 5 e D h X I 1 R i U v / r D m K U R S s M E 1 b r n u Y n x M 6 o M Z w J n p X 6 q M a F s Q k f Y s y h p h N r P F o v O y I V 1 h i S M l X 3 S k I X 7 e y K j k d b T K L C d E T V j v V q b m / / V e q k J b / y M y y Q 1 K N n y o z A V x M R k f j U Z c o X M i K k F y h S 3 u x I 2 p o o y Y 7 M p 2 R C 8 1 Z P X o X 1 V 9 S w 3 r y v 1 2 z y O I p z B O V y C B z W o w z 0 0 o A U M E J 7 h F d 6 c R + f F e X c + l q 0 F J 5 8 5 h T 9 y P n 8 A t D e M 2 w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Q W 0 f 9 v L n V t b 9 / N G z h 8 C p q x t v 8 + o = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E e q x 6 M V j C / Y D 2 l A 2 2 0 m 7 d r M J u x u h h P 4 C L x 4 U 8 e p P 8 u a / c d v m o K 0 v L D y 8 M 8 P O v E E i u D a u + + 0 U N j a 3 t n e K u 6 W 9 / Y P D o / L x S V v H q W L Y Y r G I V T e g G g W X 2 D L c C O w m C m k U C O w E k 7 t 5 v f O E S v N Y P p h p g n 5 E R 5 K H n F F j r W Z n U K 6 4 V X c h s g 5 e D h X I 1 R i U v / r D m K U R S s M E 1 b r n u Y n x M 6 o M Z w J n p X 6 q M a F s Q k f Y s y h p h N r P F o v O y I V 1 h i S M l X 3 S k I X 7 e y K j k d b T K L C d E T V j v V q b m / / V e q k J b / y M y y Q 1 K N n y o z A V x M R k f j U Z c o X M i K k F y h S 3 u x I 2 p o o y Y 7 M p 2 R C 8 1 Z P X o X 1 V 9 S w 3 r y v 1 2 z y O I p z B O V y C B z W o w z 0 0 o A U M E J 7 h F d 6 c R + f F e X c + l q 0 F J 5 8 5 h T 9 y P n 8 A t D e M 2 w = = < / l a t e x i t > Y S < l a t e x i t s h a 1 _ b a s e 6 4 = " Z L c 9 x 4 e Z J F j c N j v E l j 2 c Z 5 l K e 0 c = " > A A A B 6 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x U v s h b S i b 7 a R d u t m E 3 Y 1 Q Q n + C F w + K e P U X e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 C p X k s H 8 w k Q T + i Q 8 l D z q i x V u O x 3 + i X K 2 7 V n Y u s g p d D B X L V + + W v 3 i B m a Y T S M E G 1 7 n p u Y v y M K s O Z w G m p l 2 p M K B v T I X Y t S h q h 9 r P 5 q l N y Z p 0 B C W N l n z R k 7 v 6 e y G i k 9 S Q K b G d E z U g v 1 2 b m f 7 V u a s J r P + M y S Q 1 K t v g o T A U x M Z n d T Q Z c I T N i Y o E y x e 2 u h I 2 o o s z Y d E o 2 B G / 5 5 F V o X V Q 9 y / e X l d p N H k c R T u A U z s G D K 6 j B H d S h C Q y G 8 A y v 8 O Y I 5 8 V 5 d z 4 W r Q U n n z m G P 3 I + f w A Q A I 2 j < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Z L c 9 x 4 e Z J F j c N j v E l j 2 c Z 5 l K e 0 c = " > A A A B 6 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x U v s h b S i b 7 a R d u t m E 3 Y 1 Q Q n + C F w + K e P U X e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 C p X k s H 8 w k Q T + i Q 8 l D z q i x V u O x 3 + i X K 2 7 V n Y u s g p d D B X L V + + W v 3 i B m a Y T S M E G 1 7 n p u Y v y M K s O Z w G m p l 2 p M K B v T I X Y t S h q h 9 r P 5 q l N y Z p 0 B C W N l n z R k 7 v 6 e y G i k 9 S Q K b G d E z U g v 1 2 b m f 7 V u a s J r P + M y S Q 1 K t v g o T A U x M Z n d T Q Z c I T N i Y o E y x e 2 u h I 2 o o s z Y d E o 2 B G / 5 5 F V o X V Q 9 y / e X l d p N H k c R T u A U z s G D K 6 j B H d S h C Q y G 8 A y v 8 O Y I 5 8 V 5 d z 4 W r Q U n n z m G P 3 I + f w A Q A I 2 j < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Z L c 9 x 4 e Z J F j c N j v E l j 2 c Z 5 l K e 0 c = " > A A A B 6 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x U v s h b S i b 7 a R d u t m E 3 Y 1 Q Q n + C F w + K e P U X e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 C p X k s H 8 w k Q T + i Q 8 l D z q i x V u O x 3 + i X K 2 7 V n Y u s g p d D B X L V + + W v 3 i B m a Y T S M E G 1 7 n p u Y v y M K s O Z w G m p l 2 p M K B v T I X Y t S h q h 9 r P 5 q l N y Z p 0 B C W N l n z R k 7 v 6 e y G i k 9 S Q K b G d E z U g v 1 2 b m f 7 V u a s J r P + M y S Q 1 K t v g o T A U x M Z n d T Q Z c I T N i Y o E y x e 2 u h I 2 o o s z Y d E o 2 B G / 5 5 F V o X V Q 9 y / e X l d p N H k c R T u A U z s G D K 6 j B H d S h C Q y G 8 A y v 8 O Y I 5 8 V 5 d z 4 W r Q U n n z m G P 3 I + f w A Q A I 2 j < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Z L c 9 x 4 e Z J F j c N j v E l j 2 c Z 5 l K e 0 c = " > A A A B 6 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x U v s h b S i b 7 a R d u t m E 3 Y 1 Q Q n + C F w + K e P U X e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 C p X k s H 8 w k Q T + i Q 8 l D z q i x V u O x 3 + i X K 2 7 V n Y u s g p d D B X L V + + W v 3 i B m a Y T S M E G 1 7 n p u Y v y M K s O Z w G m p l 2 p M K B v T I X Y t S h q h 9 r P 5 q l N y Z p 0 B C W N l n z R k 7 v 6 e y G i k 9 S Q K b G d E z U g v 1 2 b m f 7 V u a s J r P + M y S Q 1 K t v g o T A U x M Z n d T Q Z c I T N i Y o E y x e 2 u h I 2 o o s z Y d E o 2 B G / 5 5 F V o X V Q 9 y / e X l d p N H k c R T u A U z s G D K 6 j B H d S h C Q y G 8 A y v 8 O Y I 5 8 V 5 d z 4 W r Q U n n z m G P 3 I + f w A Q A I 2 j < / l a t e x i t > Y Q < l a t e x i t s h a 1 _ b a s e 6 4 = " J d i x p j t A X z N c E r 9 v 1 U 7 W g t Y J o n 4 = " > A A A B 6 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x R f s h b S i b 7 a R d u t m E 3 Y 1 Q Q n + C F w + K e P U X e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 C p X k s H 8 w k Q T + i Q 8 l D z q i x 1 v 1 j v 9 E v V 9 y q O x d Z B S + H C u S q 9 8 t f v U H M 0 g i l Y Y J q 3 f X c x P g Z V Y Y z g d N S L 9 W Y U D a m Q + x a l D R C 7 W f z V a f k z D o D E s b K P m n I 3 P 0 9 k d F I 6 0 k U 2 M 6 I m p F e r s 3 M / 2 r d 1 I T X f s Z l k h q U b P F R m A p i Y j K 7 m w y 4 Q m b E x A J l i t t d C R t R R Z m x 6 Z R s C N 7 y y a v Q u q h 6 l h u X l d p N H k c R T u A U z s G D K 6 j B H d S h C Q y G 8 A y v 8 O Y I 5 8 V 5 d z 4 W r Q U n n z m G P 3 I + f w A M + I 2 h < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " J d i x p j t A X z N c E r 9 v 1 U 7 W g t Y J o n 4 = " > A A A B 6 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x R f s h b S i b 7 a R d u t m E 3 Y 1 Q Q n + C F w + K e P U X e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 C p X k s H 8 w k Q T + i Q 8 l D z q i x 1 v 1 j v 9 E v V 9 y q O x d Z B S + H C u S q 9 8 t f v U H M 0 g i l Y Y J q 3 f X c x P g Z V Y Y z g d N S L 9 W Y U D a m Q + x a l D R C 7 W f z V a f k z D o D E s b K P m n I 3 P 0 9 k d F I 6 0 k U 2 M 6 I m p F e r s 3 M / 2 r d 1 I T X f s Z l k h q U b P F R m A p i Y j K 7 m w y 4 Q m b E x A J l i t t d C R t R R Z m x 6 Z R s C N 7 y y a v Q u q h 6 l h u X l d p N H k c R T u A U z s G D K 6 j B H d S h C Q y G 8 A y v 8 O Y I 5 8 V 5 d z 4 W r Q U n n z m G P 3 I + f w A M + I 2 h < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " J d i x p j t A X z N c E r 9 v 1 U 7 W g t Y J o n 4 = " > A A A B 6 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x R f s h b S i b 7 a R d u t m E 3 Y 1 Q Q n + C F w + K e P U X e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 C p X k s H 8 w k Q T + i Q 8 l D z q i x 1 v 1 j v 9 E v V 9 y q O x d Z B S + H C u S q 9 8 t f v U H M 0 g i l Y Y J q 3 f X c x P g Z V Y Y z g d N S L 9 W Y U D a m Q + x a l D R C 7 W f z V a f k z D o D E s b K P m n I 3 P 0 9 k d F I 6 0 k U 2 M 6 I m p F e r s 3 M / 2 r d 1 I T X f s Z l k h q U b P F R m A p i Y j K 7 m w y 4 Q m b E x A J l i t t d C R t R R Z m x 6 Z R s C N 7 y y a v Q u q h 6 l h u X l d p N H k c R T u A U z s G D K 6 j B H d S h C Q y G 8 A y v 8 O Y I 5 8 V 5 d z 4 W r Q U n n z m G P 3 I + f w A M + I 2 h < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " J d i x p j t A X z N c E r 9 v 1 U 7 W g t Y J o n 4 = " > A A A B 6 n i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x R f s h b S i b 7 a R d u t m E 3 Y 1 Q Q n + C F w + K e P U X e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 C p X k s H 8 w k Q T + i Q 8 l D z q i x 1 v 1 j v 9 E v V 9 y q O x d Z B S + H C u S q 9 8 t f v U H M 0 g i l Y Y J q 3 f X c x P g Z V Y Y z g d N S L 9 W Y U D a m Q + x a l D R C 7 W f z V a f k z D o D E s b K P m n I 3 P 0 9 k d F I 6 0 k U 2 M 6 I m p F e r s 3 M / 2 r d 1 I T X f s Z l k h q U b P F R m A p i Y j K 7 m w y 4 Q m b E x A J l i t t d C R t R R Z m x 6 Z R s C N 7 y y a v Q u q h 6 l h u X l d p N H k c R T u A U z s G D K 6 j B H d S h C Q y G 8 A y v 8 O Y I 5 8 V 5 d z 4 W r Q U n n z m G P 3 I + f w A M + I 2 h < / l a t e x i t > L CE

< l a t e x i t s h a 1 _ b a s e 6 4 = " c v 7 9 M 9 A S / J l N F n r V < l a t e x i t s h a 1 _ b a s e 6 4 = " r O 7 T u w F q s p p y r e h K 5 R Y C 7 e n e B e 0 = " >

/ i x E X m 2 a 5 D s = " > A A A B / n i c b V D L S s N A F J 3 U V 6 2 v q L h y M 1 g E V y U R Q Z f F I r h w U c E + o I l l M p 2 0 Q y c P Z m 7 E E g L + i h s X i r j 1 O 9 z 5 N 0 7 a L L T 1 w M D h n H u 5 Z 4 4 X C 6 7 A s r 6 N 0 t L y y u p a e b 2 y s b m 1 v W P u 7 r V V l E j K W j Q S k e x 6 R D H B Q 9 Y C D o J 1 Y 8 l I 4 A n W 8 c a N 3 O 8 8 M K l 4 F N 7 B J G Z u Q I Y h 9 z k l o K W + e e A E B E a U i P Q m u 3 e A P U L a u M r 6 Z t W q W V P g R W I X p I o K N P v m l z O I a B K w E K g g S v V s K w Y 3 J R I 4 F S y r O I l i M a F j M m Q 9 T U M S M O W m 0 / g Z P t b K A P u R 1 C 8 E P F V / b 6 Q k U G o S e H o y D 6 v m v V z 8 z + s l 4 F + 4 K Q / j B F h I Z 4 f 8 R G C I c N 4 F H n D J K I i J J o R K r r N i O i K S U N C N V X Q J 9 v y X F 0 n 7 t G Z r f n t W r V 8 W d Z T R I T p C J 8 h G 5 6 i O r l E T t R B F K X p G r + j N e D J e j H f j Y z Z a M o q d f f Q H x u c P u V S V 9 g = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " c v 7 9 M 9 A S / J l N F n r V / i x E X m 2 a 5 D s = " > A A A B / n i c b V D L S s N A F J 3 U V 6 2 v q L h y M 1 g E V y U R Q Z f F I r h w U c E + o I l l M p 2 0 Q y c P Z m 7 E E g L + i h s X i r j 1 O 9 z 5 N 0 7 a L L T 1 w M D h n H u 5 Z 4 4 X C 6 7 A s r 6 N 0 t L y y u p a e b 2 y s b m 1 v W P u 7 r V V l E j K W j Q S k e x 6 R D H B Q 9 Y C D o J 1 Y 8 l I 4 A n W 8 c a N 3 O 8 8 M K l 4 F N 7 B J G Z u Q I Y h 9 z k l o K W + e e A E B E a U i P Q m u 3 e A P U L a u M r 6 Z t W q W V P g R W I X p I o K N P v m l z O I a B K w E K g g S v V s K w Y 3 J R I 4 F S y r O I l i M a F j M m Q 9 T U M S M O W m 0 / g Z P t b K A P u R 1 C 8 E P F V / b 6 Q k U G o S e H o y D 6 v m v V z 8 z + s l 4 F + 4 K Q / j B F h I Z 4 f 8 R G C I c N 4 F H n D J K I i J J o R K r r N i O i K S U N C N V X Q J 9 v y X F 0 n 7 t G Z r f n t W r V 8 W d Z T R I T p C J 8 h G 5 6 i O r l E T t R B F K X p G r + j N e D J e j H f j Y z Z a M o q d f f Q H x u c P u V S V 9 g = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " c v 7 9 M 9 A S / J l N F n r V / i x E X m 2 a 5 D s = " > A A A B / n i c b V D L S s N A F J 3 U V 6 2 v q L h y M 1 g E V y U R Q Z f F I r h w U c E + o I l l M p 2 0 Q y c P Z m 7 E E g L + i h s X i r j 1 O 9 z 5 N 0 7 a L L T 1 w M D h n H u 5 Z 4 4 X C 6 7 A s r 6 N 0 t L y y u p a e b 2 y s b m 1 v W P u 7 r V V l E j K W j Q S k e x 6 R D H B Q 9 Y C D o J 1 Y 8 l I 4 A n W 8 c a N 3 O 8 8 M K l 4 F N 7 B J G Z u Q I Y h 9 z k l o K W + e e A E B E a U i P Q m u 3 e A P U L a u M r 6 Z t W q W V P g R W I X p I o K N P v m l z O I a B K w E K g g S v V s K w Y 3 J R I 4 F S y r O I l i M a F j M m Q 9 T U M S M O W m 0 / g Z P t b K A P u R 1 C 8 E P F V / b 6 Q k U G o S e H o y D 6 v m v V z 8 z + s l 4 F + 4 K Q / j B F h I Z 4 f 8 R G C I c N 4 F H n D J K I i J J o R K r r N i O i K S U N C N V X Q J 9 v y X F 0 n 7 t G Z r f n t W r V 8 W d Z T R I T p C J 8 h G 5 6 i O r l E T t R B F K X p G r + j N e D J e j H f j Y z Z a M o q d f f Q H x u c P u V S V 9 g = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " c v 7 9 M 9 A S / J l N F n r V / i x E X m 2 a 5 D s = " > A A A B / n i c b V D L S s N A F J 3 U V 6 2 v q L h y M 1 g E V y U R Q Z f F I r h w U c E + o I l l M p 2 0 Q y c P Z m 7 E E g L + i h s X i r j 1 O 9 z 5 N 0 7 a L L T 1 w M D h n H u 5 Z 4 4 X C 6 7 A s r 6 N 0 t L y y u p a e b 2 y s b m 1 v W P u 7 r V V l E j K W j Q S k e x 6 R D H B Q 9 Y C D o J 1 Y 8 l I 4 A n W 8 c a N 3 O 8 8 M K l 4 F N 7 B J G Z u Q I Y h 9 z k l o K W + e e A E B E a U i P Q m u 3 e A P U L a u M r 6 Z t W q W V P g R W I X p I o K N P v m l z O I a B K w E K g g S v V s K w Y 3 J R I 4 F S y r O I l i M a F j M m Q 9 T U M S M O W m 0 / g Z P t b K A P u R 1 C 8 E P F V / b 6 Q k U G o S e H o y D 6 v m v V z 8 z + s l 4 F + 4 K Q / j B F h I Z 4 f 8 R G C I c N 4 F H n D J K I i J J o R K r r N i O i K S U N C N V X Q J 9 v y X F 0 n 7 t G Z r f n t W r V 8 W d Z T R I T p C J 8 h G 5 6 i O r l E T t R B F K X p G r + j N e D J e j H f j Y z Z a M o q d f f Q H x u c P u V S V 9 g = = < / l a t e x i

T i Y e K Q E k Q h L B 3 f U I d w 3 9 K C J k U = " > A A A B 7 X i c b Z B N S w M x E I Z n / a z 1 q + r R S 7 A I n s q u C H o s e v F Y 0 X 5 A u 5 R s O t v G Z p M l y Q q l 9 D 9 4 8 a C I V / + P N / + N a b s H b X 0 h 8 P D O D J l 5 o 1 R w Y 3 3 / 2 1 t Z X V v f 2 C x s F b d 3 d v f 2 S w e H D a M y z b D O l F C 6 F V G D g k u s W 2 4 F t l K N N I k E N q P h z b T e f E J t u J I P d p R i m N C + 5 D F n 1 D q r 0 a k N e P e + W y r 7 F X 8 m s g x B D m X I V e u W v j o 9 x b I E p W W C G t M O / N S G Y 6 o t Z w I n x U 5 m M K V s S P v Y d i h p g i Y c z 7 a d k F P n 9 E i s t H v S k p n 7 e 2 J M E 2 N G S e Q 6 E 2 o H Z r E 2 N f + r t T M b X 4 V j L t P M o m T z j + J M E K v I 9 H T S 4 x q Z F S M H l G n u d i V s Q D V l 1 g V U d C E E i y c v Q + O 8 E j i + u y h X r / M 4 C n A M J 3 A G A V x C F W 6 h B n V g 8 A j P 8 A p v n v J e v H f v Y 9 6 6 4 u U z R / B H 3 u c P P d C O 5 Q = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " m K T i Y e K Q E k Q h L B 3 f U I d w 3 9 K C J k U = " > A A A B 7 X i c b Z B N S w M x E I Z n / a z 1 q + r R S 7 A I n s q u C H o s e v F Y 0 X 5 A u 5 R s O t v G Z p M l y Q q l 9 D 9 4 8 a C I V / + P N / + N a b s H b X 0 h 8 P D O D J l 5 o 1 R w Y 3 3 / 2 1 t Z X V v f 2 C x s F b d 3 d v f 2 S w e H D a M y z b D O l F C 6 F V G D g k u s W 2 4 F t l K N N I k E N q P h z b T e f E J t u J I P d p R i m N C + 5 D F n 1 D q r 0 a k N e P e + W y r 7 F X 8 m s g x B D m X I V e u W v j o 9 x b I E p W W C G t M O / N S G Y 6 o t Z w I n x U 5 m M K V s S P v Y d i h p g i Y c z 7 a d k F P n 9 E i s t H v S k p n 7 e 2 J M E 2 N G S e Q 6 E 2 o H Z r E 2 N f + r t T M b X 4 V j L t P M o m T z j + J M E K v I 9 H T S 4 x q Z F S M H l G n u d i V s Q D V l 1 g V U d C E E i y c v Q + O 8 E j i + u y h X r / M 4 C n A M J 3 A G A V x C F W 6 h B n V g 8 A j P 8 A p v n v J e v H f v Y 9 6 6 4 u U z R / B H 3 u c P P d C O 5 Q = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " m K T i Y e K Q E k Q h L B 3 f U I d w 3 9 K C J k U = " > A A A B 7 X i c b Z B N S w M x E I Z n / a z 1 q + r R S 7 A I n s q u C H o s e v F Y 0 X 5 A u 5 R s O t v G Z p M l y Q q l 9 D 9 4 8 a C I V / + P N / + N a b s H b X 0 h 8 P D O D J l 5 o 1 R w Y 3 3 / 2 1 t Z X V v f 2 C x s F b d 3 d v f 2 S w e H D a M y z b D O l F C 6 F V G D g k u s W 2 4 F t l K N N I k E N q P h z b T e f E J t u J I P d p R i m N C + 5 D F n 1 D q r 0 a k N e P e + W y r 7 F X 8 m s g x B D m X I V e u W v j o 9 x b I E p W W C G t M O / N S G Y 6 o t Z w I n x U 5 m M K V s S P v Y d i h p g i Y c z 7 a d k F P n 9 E i s t H v S k p n 7 e 2 J M E 2 N G S e Q 6 E 2 o H Z r E 2 N f + r t T M b X 4 V j L t P M o m T z j + J M E K v I 9 H T S 4 x q Z F S M H l G n u d i V s Q D V l 1 g V U d C E E i y c v Q + O 8 E j i + u y h X r / M 4 C n A M J 3 A G A V x C F W 6 h B n V g 8 A j P 8 A p v n v J e v H f v Y 9 6 6 4 u U z R / B H 3 u c P P d C O 5 Q = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " m K T i Y e K Q E k Q h L B 3 f U I d w 3 9 K C J k U = " > A A A B 7 X i c b Z B N S w M x E I Z n / a z 1 q + r R S 7 A I n s q u C H o s e v F Y 0 X 5 A u 5 R s O t v G Z p M l y Q q l 9 D 9 4 8 a C I V / + P N / + N a b s H b X 0 h 8 P D O D J l 5 o 1 R w Y 3 3 / 2 1 t Z X V v f 2 C x s F b d 3 d v f 2 S w e H D a M y z b D O l F C 6 F V G D g k u s W 2 4 F t l K N N I k E N q P h z b T e f E J t u J I P d p R i m N C + 5 D F n 1 D q r 0 a k N e P e + W y r 7 F X 8 m s g x B D m X I V e u W v j o 9 x b I E p W W C G t M O / N S G Y 6 o t Z w I n x U 5 m M K V s S P v Y d i h p g i Y c z 7 a d k F P n 9 E i s t H v S k p n 7 e 2 J M E 2 N G S e Q 6 E 2 o H Z r E 2 N f + r t T M b X 4 V j L t P M o m T z j + J M E K v I 9 H T S 4 x q Z F S M H l G n u d i V s Q D V l 1 g V U d C E E i y c v Q + O 8 E j i + u y h X r / M 4 C n A M J 3 A G A V x C F W 6 h B n V g 8 A j P 8 A p v n v J e v H f v Y 9 6 6 4 u U z R / B H 3 u c P P d C O 5 Q = = < / l a t e x i t > Q < l a t e x i t s h a 1 _ b a s e 6 4 = " t y c d 4 G o i m 7 G / 9 g T T d v f 9 0 y N 8 R e U = " > A A A B 7 X i c b Z B N S w M x E I Z n / a z 1 q + r R S 7 A I n s q u C H o s e v H Y g v 2 A d i n Z N N v G Z p M l m R V K 6 X / w 4 k E R r / 4 f b / 4 b 0 3 Y P 2 v p C 4 O G d G T L z R q k U F n 3 / 2 1 t b 3 9 j c 2 i 7 s F H f 3 9 g 8 O S 0 f H T a s z w 3 i D a a l N O 6 K W S 6 F 4 A w V K 3 k 4 N p 0 k k e S s a 3 c 3 q r S d u r N D q A c c p D x M 6 U C I W j K K z m t 3 a U P T q v V L Z r / h z k V U I c i h D r l q v 9 N X t a 5 Y l X C G T 1 N p O 4 K c Y T q h B w S S f F r u Z 5 S l l I z r g H Y e K J t y G k / m 2 U 3 L u n D 6 J t X F P I Z m 7 v y c m N L F 2 n E S u M 6 E 4 t M u 1 m f l f r Z N h f B N O h E o z 5 I o t P o o z S V C T 2 e m k L w x n K M c O K D P C 7 U r Y k B r K 0 A V U d C E E y y e v Q v O y E j i u X 5 W r t 3 k c B T i F M 7 i A A K 6 h C v d Q g w Y w e I R n e I U 3 T 3 s v 3 r v 3 s W h d 8 / K Z E / g j 7 / M H O s i O 4 w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t y c d 4 G o i m 7 G / 9 g T T d v f 9 0 y N 8 R e U = " > A A A B 7 X i c b Z B N S w M x E I Z n / a z 1 q + r R S 7 A I n s q u C H o s e v H Y g v 2 A d i n Z N N v G Z p M l m R V K 6 X / w 4 k E R r / 4 f b / 4 b 0 3 Y P 2 v p C 4 O G d G T L z R q k U F n 3 / 2 1 t b 3 9 j c 2 i 7 s F H f 3 9 g 8 O S 0 f H T a s z w 3 i D a a l N O 6 K W S 6 F 4 A w V K 3 k 4 N p 0 k k e S s a 3 c 3 q r S d u r N D q A c c p D x M 6 U C I W j K K z m t 3 a U P T q v V L Z r / h z k V U I c i h D r l q v 9 N X t a 5 Y l X C G T 1 N p O 4 K c Y T q h B w S S f F r u Z 5 S l l I z r g H Y e K J t y G k / m 2 U 3 L u n D 6 J t X F P I Z m 7 v y c m N L F 2 n E S u M 6 E 4 t M u 1 m f l f r Z N h f B N O h E o z 5 I o t P o o z S V C T 2 e m k L w x n K M c O K D P C 7 U r Y k B r K 0 A V U d C E E y y e v Q v O y E j i u X 5 W r t 3 k c B T i F M 7 i A A K 6 h C v d Q g w Y w e I R n e I U 3 T 3 s v 3 r v 3 s W h d 8 / K Z E / g j 7 / M H O s i O 4 w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t y c d 4 G o i m 7 G / 9 g T T d v f 9 0 y N 8 R e U = " > A A A B 7 X i c b Z B N S w M x E I Z n / a z 1 q + r R S 7 A I n s q u C H o s e v H Y g v 2 A d i n Z N N v G Z p M l m R V K 6 X / w 4 k E R r / 4 f b / 4 b 0 3 Y P 2 v p C 4 O G d G T L z R q k U F n 3 / 2 1 t b 3 9 j c 2 i 7 s F H f 3 9 g 8 O S 0 f H T a s z w 3 i D a a l N O 6 K W S 6 F 4 A w V K 3 k 4 N p 0 k k e S s a 3 c 3 q r S d u r N D q A c c p D x M 6 U C I W j K K z m t 3 a U P T q v V L Z r / h z k V U I c i h D r l q v 9 N X t a 5 Y l X C G T 1 N p O 4 K c Y T q h B w S S f F r u Z 5 S l l I z r g H Y e K J t y G k / m 2 U 3 L u n D 6 J t X F P I Z m 7 v y c m N L F 2 n E S u M 6 E 4 t M u 1 m f l f r Z N h f B N O h E o z 5 I o t P o o z S V C T 2 e m k L w x n K M c O K D P C 7 U r Y k B r K 0 A V U d C E E y y e v Q v O y E j i u X 5 W r t 3 k c B T i F M 7 i A A K 6 h C v d Q g w Y w e I R n e I U 3 T 3 s v 3 r v 3 s W h d 8 / K Z E / g j 7 / M H O s i O 4 w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t y c d 4 G o i m 7 G / 9 g T T d v f 9 0 y N 8 R e U = " > A A A B 7 X i c b Z B N S w M x E I Z n / a z 1 q + r R S 7 A I n s q u C H o s e v H Y g v 2 A d i n Z N N v G Z p M l m R V K 6 X / w 4 k E R r / 4 f b / 4 b 0 3 Y P 2 v p C 4 O G d G T L z R q k U F n 3 / 2 1 t b 3 9 j c 2 i 7 s F H f 3 9 g 8 O S 0 f H T a s z w 3 i D a a l N O 6 K W S 6 F 4 A w V K 3 k 4 N p 0 k k e S s a 3 c 3 q r S d u r N D q A c c p D x M 6 U C I W j K K z m t 3 a U P T q v V L Z r / h z k V U I c i h D r l q v 9 N X t a 5 Y l X C G T 1 N p O 4 K c Y T q h B w S S f F r u Z 5 S l l I z r g H Y e K J t y G k / m 2 U 3 L u n D 6 J t X F P I Z m 7 v y c m N L F 2 n E S u M 6 E 4 t M u 1 m f l f r Z N h f B N O h E o z 5 I o t P o o z S V C T 2 e m k L w x n K M c O K D P C 7 U r Y k B r K 0 A V U d C E E y y e v Q v O y E j i u X 5 W r t 3 k c B T i F M 7 i A A K 6 h C v d Q g w Y w e I R n e I U 3 T 3 s v 3 r v 3 s W h d 8 / K Z E / g j 7 / M H O s i O 4 w = = < / l a t e x i t > (•) < l a t e x i t s h a 1 _ b a s e 6 4 = " Z I 4 b l F 2 I S + + S Y c r Q Q j N 2 g H v k R J Q = " > A A A B 8 n i c b Z D L S s N A F I Y n X m u 9 V V 2 6 G S x C 3 Z R E B F 0 W d O G y g r 1 A E s p k M m m H T m b C z I l Q Q h / D j Q t F 3 P o 0 7 n w b p 2 0 W 2 v r D w M d / z m H O + a N M c A O u + + 2 s r W 9 s b m 1 X d q q 7 e / s H h 7 W j 4 6 5 R u a a s Q 5 V Q u h 8 R w w S X r A M c B O t n m p E 0 E q w X j W 9 n 9 d 4 T 0 4 Y r + Q i T j I U p G U q e c E r A W n 6 Q j X g j o L G C i 0 G t 7 j b d u f A q e C X U U a n 2 o P Y V x I r m K Z N A B T H G 9 9 w M w o J o 4 F S w a T X I D c s I H Z M h 8 y 1 K k j I T F v O V p / j c O j F O l L Z P A p 6 7 v y c K k h o z S S P b m R I Y m e X a z P y v 5 u e Q 3 I Q F l 1 k O T N L F R 0 k u M C g 8 u x / H X D M K Y m K B U M 3 t r p i O i C Y U b E p V G 4 K 3 f P I q d C + b n u W H q 3 r r r o y j g k 7 R G W o g D 1 2 j F r p H b d R B F C n 0 j F 7 R m w P O i / P u f C x a 1 5 x y 5 g T 9 k f P 5 A 7 T d k N 4 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Z I 4 b l F 2 I S + + S Y c r Q Q j N 2 g H v k R J Q = " > A A A B 8 n i c b Z D L S s N A F I Y n X m u 9 V V 2 6 G S x C 3 Z R E B F 0 W d O G y g r 1 A E s p k M m m H T m b C z I l Q Q h / D j Q t F 3 P o 0 7 n w b p 2 0 W 2 v r D w M d / z m H O + a N M c A O u + + 2 s r W 9 s b m 1 X d q q 7 e / s H h 7 W j 4 6 5 R u a a s Q 5 V Q u h 8 R w w S X r A M c B O t n m p E 0 E q w X j W 9 n 9 d 4 T 0 4 Y r + Q i T j I U p G U q e c E r A W n 6 Q j X g j o L G C i 0 G t 7 j b d u f A q e C X U U a n 2 o P Y V x I r m K Z N A B T H G 9 9 w M w o J o 4 F S w a T X I D c s I H Z M h 8 y 1 K k j I T F v O V p / j c O j F O l L Z P A p 6 7 v y c K k h o z S S P b m R I Y m e X a z P y v 5 u e Q 3 I Q F l 1 k O T N L F R 0 k u M C g 8 u x / H X D M K Y m K B U M 3 t r p i O i C Y U b E p V G 4 K 3 f P I q d C + b n u W H q 3 r r r o y j g k 7 R G W o g D 1 2 j F r p H b d R B F C n 0 j F 7 R m w P O i / P u f C x a 1 5 x y 5 g T 9 k f P 5 A 7 T d k N 4 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Z I 4 b l F 2 I S + + S Y c r Q Q j N 2 g H v k R J Q = " > A A A B 8 n i c b Z D L S s N A F I Y n X m u 9 V V 2 6 G S x C 3 Z R E B F 0 W d O G y g r 1 A E s p k M m m H T m b C z I l Q Q h / D j Q t F 3 P o 0 7 n w b p 2 0 W 2 v r D w M d / z m H O + a N M c A O u + + 2 s r W 9 s b m 1 X d q q 7 e / s H h 7 W j 4 6 5 R u a a s Q 5 V Q u h 8 R w w S X r A M c B O t n m p E 0 E q w X j W 9 n 9 d 4 T 0 4 Y r + Q i T j I U p G U q e c E r A W n 6 Q j X g j o L G C i 0 G t 7 j b d u f A q e C X U U a n 2 o P Y V x I r m K Z N A B T H G 9 9 w M w o J o 4 F S w a T X I D c s I H Z M h 8 y 1 K k j I T F v O V p / j c O j F O l L Z P A p 6 7 v y c K k h o z S S P b m R I Y m e X a z P y v 5 u e Q 3 I Q F l 1 k O T N L F R 0 k u M C g 8 u x / H X D M K Y m K B U M 3 t r p i O i C Y U b E p V G 4 K 3 f P I q d C + b n u W H q 3 r r r o y j g k 7 R G W o g D 1 2 j F r p H b d R B F C n 0 j F 7 R m w P O i / P u f C x a 1 5 x y 5 g T 9 k f P 5 A 7 T d k N 4 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Z I 4 b l F 2 I S + + S Y c r Q Q j N 2 g H v k R J Q = " > A A A B 8 n i c b Z D L S s N A F I Y n X m u 9 V V 2 6 G S x C 3 Z R E B F 0 W d O G y g r 1 A E s p k M m m H T m b C z I l Q Q h / D j Q t F 3 P o 0 7 n w b p 2 0 W 2 v r D w M d / z m H O + a N M c A O u + + 2 s r W 9 s b m 1 X d q q 7 e / s H h 7 W j 4 6 5 R u a a s Q 5 V Q u h 8 R w w S X r A M c B O t n m p E 0 E q w X j W 9 n 9 d 4 T 0 4 Y r + Q i T j I U p G U q e c E r A W n 6 Q j X g j o L G C i 0 G t 7 j b d u f A q e C X U U a n 2 o P Y V x I r m K Z N A B T H G 9 9 w M w o J o 4 F S w a T X I D c s I H Z M h 8 y 1 K k j I T F v O V p / j c O j F O l L Z P A p 6 7 v y c K k h o z S S P b m R I Y m e X a z P y v 5 u e Q 3 I Q F l 1 k O T N L F R 0 k u M C g 8 u x / H X D M K Y m K B U M 3 t r p i O i C Y U b E p V G 4 K 3 f P I q d C + b n u W H q 3 r r r o y j g k 7 R G W o g D 1 2 j F r p H b d R B F C n 0 j F 7 R m w P O i / P u f C x a 1 5 x y 5 g T 9 k f P 5 A 7 T d k N 4 = < / l a t e x i t > ŶQ < l a t e x i t s h a 1 _ b a s e 6 4 = " 5 n G a E B Y h D c s A j i U y f 3 0 I Z s 6 M 1 + E = " > A A A B 8 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 8 E i e C q J C H o s e v H Y g v 2 i D W W z 3 b R L d z d h d y K U 0 F / h x Y M i X v 0 5 3 v w 3 b t s c t P W F h Y d 3 Z t i Z N 0 w E N + h 5 3 0 5 h Y 3 N r e 6 e 4 W 9 r b P z g 8 K h + f t E y c a s q a N B a x 7 o T E M M E V a y J H w T q J Z k S G g r X D y f 2 8 3 n 5 i 2 v B Y P e I 0 Y Y E k I 8 U j T g l a q 9 s f E 8 y 6 s 0 F j U K 5 4 V W 8 h d x 3 8 H C q Q q z 4 o f / W H M U 0 l U 0 g F M a b n e w k G G d H I q W C z U j 8 1 L C F 0 Q k a s Z 1 E R y U y Q L R a e u R f W G b p R r O 1 T 6 C 7 c 3 x M Z k c Z M Z W g 7 J c G x W a 3 N z f 9 q v R S j 2 y D j K k m R K b r 8 K E q F i 7 E 7 v 9 4 d c s 0 o i q k F Q j W 3 u 7 p 0 T D S h a D M q 2 R D 8 1 Z P X o X V V 9 S 0 3 r i u 1 u z y O I p z B O V y C D z d Q g w e o Q x M o S H i G V 3 h z t P P i v D s f y 9 a C k 8 + c w h 8 5 n z / c B 5 B u < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 5 n G a E B Y h D c s A j i U y f 3 0 I Z s 6 M 1 + E = " > A A A B 8 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 8 E i e C q J C H o s e v H Y g v 2 i D W W z 3 b R L d z d h d y K U 0 F / h x Y M i X v 0 5 3 v w 3 b t s c t P W F h Y d 3 Z t i Z N 0 w E N + h 5 3 0 5 h Y 3 N r e 6 e 4 W 9 r b P z g 8 K h + f t E y c a s q a N B a x 7 o T E M M E V a y J H w T q J Z k S G g r X D y f 2 8 3 n 5 i 2 v B Y P e I 0 Y Y E k I 8 U j T g l a q 9 s f E 8 y 6 s 0 F j U K 5 4 V W 8 h d x 3 8 H C q Q q z 4 o f / W H M U 0 l U 0 g F M a b n e w k G G d H I q W C z U j 8 1 L C F 0 Q k a s Z 1 E R y U y Q L R a e u R f W G b p R r O 1 T 6 C 7 c 3 x M Z k c Z M Z W g 7 J c G x W a 3 N z f 9 q v R S j 2 y D j K k m R K b r 8 K E q F i 7 E 7 v 9 4 d c s 0 o i q k F Q j W 3 u 7 p 0 T D S h a D M q 2 R D 8 1 Z P X o X V V 9 S 0 3 r i u 1 u z y O I p z B O V y C D z d Q g w e o Q x M o S H i G V 3 h z t P P i v D s f y 9 a C k 8 + c w h 8 5 n z / c B 5 B u < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 5 n G a E B Y h D c s A j i U y f 3 0 I Z s 6 M 1 + E = " > A A A B 8 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 8 E i e C q J C H o s e v H Y g v 2 i D W W z 3 b R L d z d h d y K U 0 F / h x Y M i X v 0 5 3 v w 3 b t s c t P W F h Y d 3 Z t i Z N 0 w E N + h 5 3 0 5 h Y 3 N r e 6 e 4 W 9 r b P z g 8 K h + f t E y c a s q a N B a x 7 o T E M M E V a y J H w T q J Z k S G g r X D y f 2 8 3 n 5 i 2 v B Y P e I 0 Y Y E k I 8 U j T g l a q 9 s f E 8 y 6 s 0 F j U K 5 4 V W 8 h d x 3 8 H C q Q q z 4 o f / W H M U 0 l U 0 g F M a b n e w k G G d H I q W C z U j 8 1 L C F 0 Q k a s Z 1 E R y U y Q L R a e u R f W G b p R r O 1 T 6 C 7 c 3 x M Z k c Z M Z W g 7 J c G x W a 3 N z f 9 q v R S j 2 y D j K k m R K b r 8 K E q F i 7 E 7 v 9 4 d c s 0 o i q k F Q j W 3 u 7 p 0 T D S h a D M q 2 R D 8 1 Z P X o X V V 9 S 0 3 r i u 1 u z y O I p z B O V y C D z d Q g w e o Q x M o S H i G V 3 h z t P P i v D s f y 9 a C k 8 + c w h 8 5 n z / c B 5 B u < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 5 n G a E B Y h D c s A j i U y f 3 0 I Z s 6 M 1 + E = " > A A A B 8 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 8 E i e C q J C H o s e v H Y g v 2 i D W W z 3 b R L d z d h d y K U 0 F / h x Y M i X v 0 5 3 v w 3 b t s c t P W F h Y d 3 Z t i Z N 0 w E N + h 5 3 0 5 h Y 3 N r e 6 e 4 W 9 r b P z g 8 K h + f t E y c a s q a N B a x 7 o T E M M E V a y J H w T q J Z k S G g r X D y f 2 8 3 n 5 i 2 v B Y P e I 0 Y Y E k I 8 U j T g l a q 9 s f E 8 y 6 s 0 F j U K 5 4 V W 8 h d x 3 8 H C q Q q z 4 o f / W H M U 0 l U 0 g F M a b n e w k G G d H I q W C z U j 8 1 L C F 0 Q k a s Z 1 E R y U y Q L R a e u R f W G b p R r O 1 T 6 C 7 c 3 x M Z k c Z M Z W g 7 J c G x W a 3 N z f 9 q v R S j 2 y D j K k m R K b r 8 K E q F i 7 E 7 v 9 4 d c s 0 o i q k F Q j W 3 u 7 p 0 T D S h a D M q 2 R D 8 1 Z P X o X V V 9 S 0 3 r i u 1 u z y O I p z B O V y C D z d Q g w e o Q x M o S H i G V 3 h z t P P i v D s f y 9 a C k 8 + c w h 8 5 n z / c B 5 B u < / l a t

A A A B 7 n i c b Z D L S g M x F I Z P 6 q 3 W W 9 W l m 2 A R 6 q b M i K D L g i 5 c V r A X a I e S S T N t a C Y T k o x Y h j 6 E G x e K u P V 5 3 P k 2 p u 0 s t P W H w M d / z i H n / K E S 3 F j P + 0 a F t f W N z a 3 i d m l n d 2 / / o H x 4 1 D J J q i l r 0 k Q k u h M S w w S X r G m 5 F a y j N C N x K F g 7 H N / M 6 u 1 H p g 1 P 5 I O d K B b E Z C h 5 x C m x z m r 3 1 I h X n 8 7 7 5 Y p X 8 + b C q + D n U I F c j X 7 5 q z d I a B o z a a k g x n R 9 T 9 k g I 9 p y K t i 0 1 E s N U 4 S O y Z B 1 H U o S M x N k 8 3 W n + M w 5 A x w l 2 j 1 p 8 d z 9 P Z G R 2 J h J H L r O m N i R W a 7 N z P 9 q 3 d R G 1 0 H G p U o t k 3 T x U Z Q K b B M 8 u x 0 P u G b U i o k D Q j V 3 u 2 I 6 I p p Q 6 x I q u R D 8 5 Z N X o X V R 8 x 3 f X 1 b q t 3 k c R T i B U 6 i C D 1 d Q h z t o Q B M o j O E Z X u E N K f S C 3 t H H o r W A 8 p l j + C P 0 + Q O 5 O o 8 o < /

l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " r O 7 T u w F q s p p y r e h K 5 R Y C 7 e n e B e 0 = " >

A A A B 7 n i c b Z D L S g M x F I Z P 6 q 3 W W 9 W l m 2 A R 6 q b M i K D L g i 5 c V r A X a I e S S T N t a C Y T k o x Y h j 6 E G x e K u P V 5 3 P k 2 p u 0 s t P W H w M d / z i H n / K E S 3 F j P + 0 a F t f W N z a 3 i d m l n d 2 / / o H x 4 1 D J J q i l r 0 k Q k u h M S w w S X r G m 5 F a y j N C N x K F g 7 H N / M 6 u 1 H p g 1 P 5 I O d K B b E Z C h 5 x C m x z m r 3 1 I h X n 8 7 7 5 Y p X 8 + b C q + D n U I F c j X 7 5 q z d I a B o z a a k g x n R 9 T 9 k g I 9 p y K t i 0 1 E s N U 4 S O y Z B 1 H U o S M x N k 8 3 W n + M w 5 A x w l 2 j 1 p 8 d z 9 P Z G R 2 J h J H L r O m N i R W a 7 N z P 9 q 3 d R G 1 0 H G p U o t k 3 T x U Z Q K b B M 8 u x 0 P u G b U i o k D Q j V 3 u 2 I 6 I p p Q 6 x I q u R D 8 5 Z N X o X V R 8 x 3 f X 1 b q t 3 k c R T i B U 6 i C D 1 d Q h z t o Q B M o j O E Z X u E N K f S C 3 t H H o r W A 8 p l j + C P 0 + Q O 5 O o 8 o < /

l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " r O 7 T u w F q s p p y r e h K 5 R Y C 7 e n e B e 0 = " > A A A B 7 n i c b Z D L S g M x F I Z P 6 q 3 W W 9 W l m 2 A R 6 q b M i K D L g i 5 c V r A X a I e S S T N t a C Y T k o x Y h j 6 E G x e K u P V 5 3 P k 2 p u 0 s

t P W H w M d / z i H n / K E S 3 F j P + 0 a F t f W N z a 3 i d m l n d 2 / / o H x 4 1 D J J q i l r 0 k Q k u h M S w w S X r G m 5 F a y j N C N x K F g 7 H N / M 6 u 1 H p g 1 P 5 I O d K B b E Z C h 5 x C m x z m r 3 1 I h X n 8 7 7 5 Y p X 8 + b C q + D n U I F c j X 7 5 q z d I a B o z a a k g x n R 9 T 9 k g I 9 p y K t i 0 1 E s N U 4 S O y Z B 1 H U o S M x N k 8 3 W n + M w 5 A x w l 2 j 1 p 8 d z 9 P Z G R 2 J h J H L r O m N i R W a 7 N z P 9 q 3 d R G 1 0 H G p U o t k 3 T x U Z Q K b B M 8 u x 0 P u G b U i o k D Q j V 3 u 2 I 6 I p p Q 6 x I q u R D 8 5 Z N X o X V R 8 x 3 f X 1 b q t 3 k c R T i B U 6 i C D 1 d Q h z t o Q B M o j O E Z X u E N K f S C 3 t H H o r W A 8 p l j + C P 0 + Q O 5 O o 8 o < /

l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " r O 7 T u w F q s p p y r e h K 5 R Y C 7 e n e B e 0 = " > A A A B 7 n i c b Z D L S g M x F I Z P 6 q 3 W W 9 W l m 2 A R 6 q b M i K D L g i 5 c V r A X a I e S S T N t a C Y T k o x Y h j 6 E G x e K u P V 5 3 P k 2 p u 0 s

t P W H w M d / z i H n / K E S 3 F j P + 0 a F t f W N z a 3 i d m l n d 2 / / o H x 4 1 D J J q i l r 0 k Q k u h M S w w S X r G m 5 F a y j N C N x K F g 7 H N / M 6 u 1 H p g 1 P 5 I O d K B b E Z C h 5 x C m x z m r 3 1 I h X n 8 7 7 5 Y p X 8 + b C q + D n U I F c j X 7 5 q z d I a B o z a a k g x n R 9 T 9 k g I 9 p y K t i 0 1 E s N U 4 S O y Z B 1 H U o S M x N k 8 3 W n + M w 5 A x w l 2 j 1 p 8 d z 9 P Z G R 2 J h J H L r O m N i R W a 7 N z P 9 q 3 d R G 1 0 H G p U o t k 3 T x U Z Q K b B M 8 u x 0 P u G b U i o k D Q j V 3 u 2 I 6 I p p Q 6 x I q u R D 8 5 Z N X o X V R 8 x 3 f X 1 b q t 3 k c R T i B U 6 i C D 1 d Q h z t o Q B M o j O E Z X u E N K f S C 3 t H H o r W A 8 p l j + C P 0 + Q O 5 O o 8 o < / l a t e x i t > f ebd (•) < l a t e x i t s h a 1 _ b a s e 6 4 = " + C K A C Y 6 1 o v w H M X G X X 3 n k v r Y E U u c = " > A A A B / H i c b Z B N S 8 N A E I Y 3 9 a v W r 2 i P X o J F q J e S i K D H o h e P F e w H t K V s N p N 2 6 W Y T d i d i C f W v e P G g i F d / i D f / j d s 2 B 2 1 9 Y e H h n R l m 9 v U T w T W 6 7 r d V W F v f 2 N w q b p d 2 d v f 2 D + z D o 5 a O U 8 W g y W I R q 4 5 P N Q g u o Y k c B X Q S B T T y B b T 9 8 c 2 s 3 n 4 A p X k s 7 3 G S Q D + i Q 8 l D z i g a a 2 C X w 0 E P 4 R E z 8 I N p t c e C G M 8 G d s W t u X M 5 q + D l U C G 5 G g P 7 q x f E L I 1 A I h N U 6 6 7 n J t j P q E L O B E x L v V R D Q t m Y D q F r U N I I d D + b H z 9 1 T o 0 T O G G s z J P o z N 3 f E x m N t J 5 E v u m M K I 7 0 c m 1 m / l f r p h h e 9 T M u k x R B s s W i M B U O x s 4 s C S f g C h i K i Q H K F D e 3 O m x E F W V o 8 i q Z E L z l L 6 9 C 6 7 z m G b 6 7 q N S v 8 z i K 5 J i c k C r x y C W p k 1 v S I E 3 C y I Q 8 k 1 f y Z j 1 Z L 9 a 7 9 b F o L V j 5 T J n 8 k f X 5 A 7 U r l M k = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " + C K A C Y 6 1 o v w H M X G X X 3 n k v r Y E U u c = " > A A A B / H i c b Z B N S 8 N A E I Y 3 9 a v W r 2 i P X o J F q J e S i K D H o h e P F e w H t K V s N p N 2 6 W Y T d i d i C f W v e P G g i F d / i D f / j d s 2 B 2 1 9 Y e H h n R l m 9 v U T w T W 6 7 r d V W F v f 2 N w q b p d 2 d v f 2 D + z D o 5 a O U 8 W g y W I R q 4 5 P N Q g u o Y k c B X Q S B T T y B b T 9 8 c 2 s 3 n 4 A p X k s 7 3 G S Q D + i Q 8 l D z i g a a 2 C X w 0 E P 4 R E z 8 I N p t c e C G M 8 G d s W t u X M 5 q + D l U C G 5 G g P 7 q x f E L I 1 A I h N U 6 6 7 n J t j P q E L O B E x L v V R D Q t m Y D q F r U N I I d D + b H z 9 1 T o 0 T O G G s z J P o z N 3 f E x m N t J 5 E v u m M K I 7 0 c m 1 m / l f r p h h e 9 T M u k x R B s s W i M B U O x s 4 s C S f g C h i K i Q H K F D e 3 O m x E F W V o 8 i q Z E L z l L 6 9 C 6 7 z m G b 6 7 q N S v 8 z i K 5 J i c k C r x y C W p k 1 v S I E 3 C y I Q 8 k 1 f y Z j 1 Z L 9 a 7 9 b F o L V j 5 T J n 8 k f X 5 A 7 U r l M k = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " + C K A C Y 6 1 o v w H M X G X X 3 n k v r Y E U u c = " > A A A B / H i c b Z B N S 8 N A E I Y 3 9 a v W r 2 i P X o J F q J e S i K D H o h e P F e w H t K V s N p N 2 6 W Y T d i d i C f W v e P G g i F d / i D f / j d s 2 B 2 1 9 Y e H h n R l m 9 v U T w T W 6 7 r d V W F v f 2 N w q b p d 2 d v f 2 D + z D o 5 a O U 8 W g y W I R q 4 5 P N Q g u o Y k c B X Q S B T T y B b T 9 8 c 2 s 3 n 4 A p X k s 7 3 G S Q D + i Q 8 l D z i g a a 2 C X w 0 E P 4 R E z 8 I N p t c e C G M 8 G d s W t u X M 5 q + D l U C G 5 G g P 7 q x f E L I 1 A I h N U 6 6 7 n J t j P q E L O B E x L v V R D Q t m Y D q F r U N I I d D + b H z 9 1 T o 0 T O G G s z J P o z N 3 f E x m N t J 5 E v u m M K I 7 0 c m 1 m / l f r p h h e 9 T M u k x R B s s W i M B U O x s 4 s C S f g C h i K i Q H K F D e 3 O m x E F W V o 8 i q Z E L z l L 6 9 C 6 7 z m G b 6 7 q N S v 8 z i K 5 J i c k C r x y C W p k 1 v S I E 3 C y I Q 8 k 1 f y Z j 1 Z L 9 a 7 9 b F o L V j 5 T J n 8 k f X 5 A 7 U r l M k = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " + C K A C Y 6 1 o v w H M X G X X 3 n k v r Y E U u c = " > A A A B / H i c b Z B N S 8 N A E I Y 3 9 a v W r 2 i P X o J F q J e S i K D H o h e P F e w H t K V s N p N 2 6 W Y T d i d i C f W v e P G g i F d / i D f / j d s 2 B 2 1 9 Y e H h n R l m 9 v U T w T W 6 7 r d V W F v f 2 N w q b p d 2 d v f 2 D + z D o 5 a O U 8 W g y W I R q 4 5 P N Q g u o Y k c B X Q S B T T y B b T 9 8 c 2 s 3 n 4 A p X k s 7 3 G S Q D + i Q 8 l D z i g a a 2 C X w 0 E P 4 R E z 8 I N p t c e C G M 8 G d s W t u X M 5 q + D l U C G 5 G g P 7 q x f E L I 1 A I h N U 6 6 7 n J t j P q E L O B E x L v V R D Q t m Y D q F r U N I I d D + b H z 9 1 T o 0 T O G G s z J P o z N 3 f E x m N t J 5 E v u m M K I 7 0 c m 1 m / l f r p h h e 9 T M u k x R B s s W i M B U O x s 4 s C S f g C h i K i Q H K F D e 3 O m x E F W V o 8 i q Z E L z l L 6 9 C 6 7 z m G b 6 7 q N S v 8 z i K 5 J i c k C r x y C W p k 1 v S I E 3 C y I Q 8 k 1 f y Z j 1 Z L 9 a 7 9 b F o L V j 5 T J n 8 k f X 5 A 7 U r l M k = < / l a t e x i t > Attention Generator (•) < l a t e x i t s h a 1 _ b a s e 6 4 = " Z I 4 b l F 2 I S + + S Y c r Q Q j N 2 g H v k R J Q = " > A A A B 8 n i c b Z D L S s N A F I Y n X m u 9 V V 2 6 G S x C 3 Z R E B F 0 W d O G y g r 1 A E s p k M m m H T m b C z I l Q Q h / D j Q t F 3 P o 0 7 n w b p 2 0 W 2 v r D w M d / z m H O + a N M c A O u + + 2 s r W 9 s b m 1 X d q q 7 e / s H h 7 W j 4 6 5 R u a a s Q 5 V Q u h 8 R w w S X r A M c B O t n m p E 0 E q w X j W 9 n 9 d 4 T 0 4 Y r + Q i T j I U p G U q e c E r A W n 6 Q j X g j o L G C i 0 G t 7 j b d u f A q e C X U U a n 2 o P Y V x I r m K Z N A B T H G 9 9 w M w o J o 4 F S w a T X I D c s I H Z M h 8 y 1 K k j I T F v O V p / j c O j F O l L Z P A p 6 7 v y c K k h o z S S P b m R I Y m e X a z P y v 5 u e Q 3 I Q F l 1 k O T N L F R 0 k u M C g 8 u x / H X D M K Y m K B U M 3 t r p i O i C Y U b E p V G 4 K 3 f P I q d C + b n u W H q 3 r r r o y j g k 7 R G W o g D 1 2 j F r p H b d R B F C n 0 j F 7 R

m w P O i / P u f C x a 1 5 x y 5 g T 9 k f P 5 A 7 T d k N 4 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Z I 4 b l F

2 I S + + S Y c r Q Q j N 2 g H v k R J Q = " > A A A B 8 n i c b Z D L S s N A F I Y n X m u 9 V V 2 6 G S x C 3 Z R E B F 0 W d O G y g r 1 A E s p k M m m H T m b C z I l Q Q h / D j Q t F 3 P o 0 7 n w b p 2 0 W 2 v r D w M d / z m H O + a N M c A O u + + 2 s r W 9 s b m 1 X d q q 7 e / s H h 7 W j 4 6 5 R u a a s Q 5 V Q u h 8 R w w S X r A M c B O t n m p E 0 E q w X j W 9 n 9 d 4 T 0 4 Y r + Q i T j I U p G U q e c E r A W n 6 Q j X g j o L G C i 0 G t 7 j b d u f A q e C X U U a n 2 o P Y V x I r m K Z N A B T H G 9 9 w M w o J o 4 F S w a T X I D c s I H Z M h 8 y 1 K k j I T F v O V p / j c O j F O l L Z P A p 6 7 v y c K k h o z S S P b m R I Y m e X a z P y v 5 u e Q 3 I Q F l 1 k O T N L F R 0 k u M C g 8 u x / H X D M K Y m K B U M 3 t r p i O i C Y U b E p V G 4 K 3 f P I q d C + b n u W H q 3 r r r o y j g k 7 R G W o g D 1 2 j F r p H b d R B F C n 0 j F 7 R

m w P O i / P u f C x a 1 5 x y 5 g T 9 k f P 5 A 7 T d k N 4 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Z I 4 b l F

2 I S + + S Y c r Q Q j N 2 g H v k R J Q = " > A A A B 8 n i c b Z D L S s N A F I Y n X m u 9 V V 2 6 G S x C 3 Z R E B F 0 W d O G y g r 1 A E s p k M m m H T m b C z I l Q Q h / D j Q t F 3 P o 0 7 n w b p 2 0 W 2 v r D w M d / z m H O + a N M c A O u + + 2 s r W 9 s b m 1 X d q q 7 e / s H h 7 W j 4 6 5 R u a a s Q 5 V Q u h 8 R w w S X r A M c B O t n m p E 0 E q w X j W 9 n 9 d 4 T 0 4 Y r + Q i T j I U p G U q e c E r A W n 6 Q j X g j o L G C i 0 G t 7 j b d u f A q e C X U U a n 2 o P Y V x I r m K Z N A B T H G 9 9 w M w o J o 4 F S w a T X I D c s I H Z M h 8 y 1 K k j I T F v O V p / j c O j F O l L Z P A p 6 7 v y c K k h o z S S P b m R I Y m e X a z P y v 5 u e Q 3 I Q F l 1 k O T N L F R 0 k u M C g 8 u x / H X D M K Y m K B U M 3 t r p i O i C Y U b E p V G 4 K 3 f P I q d C + b n u W H q 3 r r r o y j g k 7 R G W o g D 1 2 j F r p H b d R B F C n 0 j F 7 R

m w P O i / P u f C x a 1 5 x y 5 g T 9 k f P 5 A 7 T d k N 4 = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " Z I 4 b l F 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 j u M / A = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 j u M / A = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 j u M / A = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = "

2 I S + + S Y c r Q Q j N 2 g H v k R J Q = " > A A A B 8 n i c b Z D L S s N A F I Y n X m u 9 V V 2 6 G S x C 3 Z R E B F 0 W d O G y g r 1 A E s p k M m m H T m b C z I l Q Q h / D j Q t F 3 P o 0 7 n w b p 2 0 W 2 v r D w M d / z m H O + a N M c A O u + + 2 s r W 9 s b m 1 X d q q 7 e / s H h 7 W j 4 6 5 R u a a s Q 5 V Q u h 8 R w w S X r A M c B O t n m p E 0 E q w X j W 9 n 9 d 4 T 0 4 Y r + Q i T j I U p G U q e c E r A W n 6 Q j X g j o L G C i 0 G t 7 j b d u f A q e C X U U a n 2 o P Y V x I r m K Z N A B T H G 9 9 w M w o J o 4 F S w a T X I D c s I H Z M h 8 y 1 K k j I T F v O V p / j c O j F O l L Z P A p 6 7 v y c K k h o z S S P b m R I Y m e X a z P y v 5 u e Q 3 I Q F l 1 k O T N L F R 0 k u M C g 8 u x / H X D M K Y m K B U M 3 t r p i O i C Y U b E p V G 4 K 3 f P I q d C + b n u W H q 3 r r r o y j g k 7 R G W o g D 1 2 j F r p H b d R B F C n 0 j F 7 R

I A r Q p D G 4 G w 7 A x + 5 W r i 9 C Z W K D 4 B o = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 Y Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K

I A r Q p D G 4 G w 7 A x + 5 W r i 9 C Z W K D 4 B o = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 Y Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K

I A r Q p D G 4 G w 7 A x + 5 W r i 9 C Z W K D 4 B o = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 Y Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K

I A r Q p D G 4 G w 7 A x + 5 W r i 9 C Z W K D 4 B o = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 Y Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 j u M / A = = < / l a t e x i t > t(w)

< l a t e x i t s h a 1 _ b a s e 6 4 = " 2 P U R 8 7

1 K S V d 2 q R 9 m k G X V N E v R e d E = " > A A A B 9 H i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I t Q N 2 V G B F 0 W 3 b i s Y B / Q D i W T Z t r Q J D M m m U o Z + h 1 u X C j i 1 o 9 x 5 9 + Y a W e h r Q c C h 3 P u 5 Z 6 c I O Z M G 9 f 9 d g p r 6 x u b W 8 X t 0 s 7 u 3 v 5 B + f C o p a N E E d o k E Y 9 U J 8 C a c i Z p 0 z D D a S d W F I u A 0 3 Y w v s 3 8 9 o Q q z S L 5 Y K Y x 9 Q U e S h Y y g o 2 V / J 7 A Z q R E a m b V p / N + u e L W 3 D n Q K v F y U o E c j X 7 5 q z e I S C K o N I R j r b u e G x s / x c o w w u m s 1 E s 0 j T E Z 4 y H t W i q x o N p P 5 6 F n 6 M w q A x R G y j 5 p 0 F z 9 v Z F i o f V U B H Y y C 6 m X v U z 8 z + s m J r z 2 U y b j x F B J F o f C h C M T o a w B N G C K E s O n l m C i m M 2 K y A g r T I z t q W R L 8 J a / v E p a F z X P 8 v v L S v 0 m r 6 M I J 3 A K V f D g C u p w B w 1 o A o F H e I Z X e H M m z o v z 7 n w s R g t O v n M M f + B 8 / g D G O J I V < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 P U R 8 7 1 K S V d 2 q R 9 m k G X V N E v R e d E = " > A A A B 9 H i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I t Q N 2 V G B F 0 W 3 b i s Y B / Q D i W T Z t r Q J D M m m U o Z + h 1 u X C j i 1 o 9 x 5 9 + Y a W e h r Q c C h 3 P u 5 Z 6 c I O Z M G 9 f 9 d g p r 6 x u b W 8 X t 0 s 7 u 3 v 5 B + f C o p a N E E d o k E Y 9 U J 8 C a c i Z p 0 z D D a S d W F I u A 0 3 Y w v s 3 8 9 o Q q z S L 5 Y K Y x 9 Q U e S h Y y g o 2 V / J 7 A Z q R E a m b V p / N + u e L W 3 D n Q K v F y U o E c j X 7 5 q z e I S C K o N I R j r b u e G x s / x c o w w u m s 1 E s 0 j T E Z 4 y H t W i q x o N p P 5 6 F n 6 M w q A x R G y j 5 p 0 F z 9 v Z F i o f V U B H Y y C 6 m X v U z 8 z + s m J r z 2 U y b j x F B J F o f C h C M T o a w B N G C K E s O n l m C i m M 2 K y A g r T I z t q W R L 8 J a / v E p a F z X P 8 v v L S v 0 m r 6 M I J 3 A K V f D g C u p w B w 1 o A o F H e I Z X e H M m z o v z 7 n w s R g t O v n M M f + B 8 / g D G O J I V < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 P U R 8 7 1 K S V d 2 q R 9 m k G X V N E v R e d E = " > A A A B 9 H i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I t Q N 2 V G B F 0 W 3 b i s Y B / Q D i W T Z t r Q J D M m m U o Z + h 1 u X C j i 1 o 9 x 5 9 + Y a W e h r Q c C h 3 P u 5 Z 6 c I O Z M G 9 f 9 d g p r 6 x u b W 8 X t 0 s 7 u 3 v 5 B + f C o p a N E E d o k E Y 9 U J 8 C a c i Z p 0 z D D a S d W F I u A 0 3 Y w v s 3 8 9 o Q q z S L 5 Y K Y x 9 Q U e S h Y y g o 2 V / J 7 A Z q R E a m b V p / N + u e L W 3 D n Q K v F y U o E c j X 7 5 q z e I S C K o N I R j r b u e G x s / x c o w w u m s 1 E s 0 j T E Z 4 y H t W i q x o N p P 5 6 F n 6 M w q A x R G y j 5 p 0 F z 9 v Z F i o f V U B H Y y C 6 m X v U z 8 z + s m J r z 2 U y b j x F B J F o f C h C M T o a w B N G C K E s O n l m C i m M 2 K y A g r T I z t q W R L 8 J a / v E p a F z X P 8 v v L S v 0 m r 6 M I J 3 A K V f D g C u p w B w 1 o A o F H e I Z X e H M m z o v z 7 n w s R g t O v n M M f + B 8 / g D G O J I V < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 P U R 8 7 1 K S V d 2 q R 9 m k G X V N E v R e d E = " > A A A B 9 H i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I t Q N 2 V G B F 0 W 3 b i s Y B / Q D i W T Z t r Q J D M m m U o Z + h 1 u X C j i 1 o 9 x 5 9 + Y a W e h r Q c C h 3 P u 5 Z 6 c I O Z M G 9 f 9 d g p r 6 x u b W 8 X t 0 s 7 u 3 v 5 B + f C o p a N E E d o k E Y 9 U J 8 C a c i Z p 0 z D D a S d W F I u A 0 3 Y w v s 3 8 9 o Q q z S L 5 Y K Y x 9 Q U e S h Y y g o 2 V / J 7 A Z q R E a m b V p / N + u e L W 3 D n Q K v F y U o E c j X 7 5 q z e I S C K o N I R j r b u e G x s / x c o w w u m s 1 E s 0 j T E Z 4 y H t W i q x o N p P 5 6 F n 6 M w q A x R G y j 5 p 0 F z 9 v Z F i o f V U B H Y y C 6 m X v U z 8 z + s m J r z 2 U y b j x F B J F o f C h C M T o a w B N G C K E s O n l m C i m M 2 K y A g r T I z t q W R L 8 J a / v E p a F z X P 8 v v L S v 0 m r 6 M I J 3 A K V f D g C u p w B w 1 o A o F H e I Z X e H M m z o v z 7 n w s R g t O v n M M f + B 8 / g D G O J I V < / l a t e x i t > s(w) < l a t e x i t s h a 1 _ b a s e 6 4 = " t K U m 7 a Y x p X P Q H P 8 I a G R A U x t m D o c = " > A A A B 9 H i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I t Q N 2 V G B F 0 W 3 b i s Y B / Q D i W T Z t r Q J D M m m U o Z + h 1 u X C j i 1 o 9 x 5 9 + Y a W e h r Q c C h 3 P u 5 Z 6 c I O Z M G 9 f 9 d g p r 6 x u b W 8 X t 0 s 7 u 3 v 5 B + f C o p a N E E d o k E Y 9 U J 8 C a c i Z p 0 z D D a S d W F I u A 0 3 Y w v s 3 8 9 o Q q z S L 5 Y K Y x 9 Q U e S h Y y g o 2 V / J 7 A Z q R E q m f V p / N + u e L W 3 D n Q K v F y U o E c j X 7 5 q z e I S C K o N I R j r b u e G x s / x c o w w u m s 1 E s 0 j T E Z 4 y H t W i q x o N p P 5 6 F n 6 M w q A x R G y j 5 p 0 F z 9 v Z F i o f V U B H Y y C 6 m X v U z 8 z + s m J r z 2 U y b j x F B J F o f C h C M T o a w B N G C K E s O n l m C i m M 2 K y A g r T I z t q W R L 8 J a / v E p a F z X P 8 v v L S v 0 m r 6 M I J 3 A K V f D g C u p w B w 1 o A o F H e I Z X e H M m z o v z 7 n w s R g t O v n M M f + B 8 / g D E s J I U < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t K U m 7 a Y x p X P Q H P 8 I a G R A U x t m D o c = " > A A A B 9 H i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I t Q N 2 V G B F 0 W 3 b i s Y B / Q D i W T Z t r Q J D M m m U o Z + h 1 u X C j i 1 o 9 x 5 9 + Y a W e h r Q c C h 3 P u 5 Z 6 c I O Z M G 9 f 9 d g p r 6 x u b W 8 X t 0 s 7 u 3 v 5 B + f C o p a N E E d o k E Y 9 U J 8 C a c i Z p 0 z D D a S d W F I u A 0 3 Y w v s 3 8 9 o Q q z S L 5 Y K Y x 9 Q U e S h Y y g o 2 V / J 7 A Z q R E q m f V p / N + u e L W 3 D n Q K v F y U o E c j X 7 5 q z e I S C K o N I R j r b u e G x s / x c o w w u m s 1 E s 0 j T E Z 4 y H t W i q x o N p P 5 6 F n 6 M w q A x R G y j 5 p 0 F z 9 v Z F i o f V U B H Y y C 6 m X v U z 8 z + s m J r z 2 U y b j x F B J F o f C h C M T o a w B N G C K E s O n l m C i m M 2 K y A g r T I z t q W R L 8 J a / v E p a F z X P 8 v v L S v 0 m r 6 M I J 3 A K V f D g C u p w B w 1 o A o F H e I Z X e H M m z o v z 7 n w s R g t O v n M M f + B 8 / g D E s J I U < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t K U m 7 a Y x p X P Q H P 8 I a G R A U x t m D o c = " > A A A B 9 H i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I t Q N 2 V G B F 0 W 3 b i s Y B / Q D i W T Z t r Q J D M m m U o Z + h 1 u X C j i 1 o 9 x 5 9 + Y a W e h r Q c C h 3 P u 5 Z 6 c I O Z M G 9 f 9 d g p r 6 x u b W 8 X t 0 s 7 u 3 v 5 B + f C o p a N E E d o k E Y 9 U J 8 C a c i Z p 0 z D D a S d W F I u A 0 3 Y w v s 3 8 9 o Q q z S L 5 Y K Y x 9 Q U e S h Y y g o 2 V / J 7 A Z q R E q m f V p / N + u e L W 3 D n Q K v F y U o E c j X 7 5 q z e I S C K o N I R j r b u e G x s / x c o w w u m s 1 E s 0 j T E Z 4 y H t W i q x o N p P 5 6 F n 6 M w q A x R G y j 5 p 0 F z 9 v Z F i o f V U B H Y y C 6 m X v U z 8 z + s m J r z 2 U y b j x F B J F o f C h C M T o a w B N G C K E s O n l m C i m M 2 K y A g r T I z t q W R L 8 J a / v E p a F z X P 8 v v L S v 0 m r 6 M I J 3 A K V f D g C u p w B w 1 o A o F H e I Z X e H M m z o v z 7 n w s R g t O v n M M f + B 8 / g D E s J I U < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " t K U m 7 a Y x p X P Q H P 8 I a G R A U x t m D o c = " > A A A B 9 H i c b V D L S g M x F L 1 T X 7 W + q i 7 d B I t Q N 2 V G B F 0 W 3 b i s Y B / Q D i W T Z t r Q J D M m m U o Z + h 1 u X C j i 1 o 9 x 5 9 + Y a W e h r Q c C h 3 P u 5 Z 6 c I O Z M G 9 f 9 d g p r 6 x u b W 8 X t 0 s 7 u 3 v 5 B + f C o p a N E E d o k E Y 9 U J 8 C a c i Z p 0 z D D a S d W F I u A 0 3 Y w v s 3 8 9 o Q q z S L 5 Y K Y x 9 Q U e S h Y y g o 2 V / J 7 A Z q R E q m f V p / N + u e L W 3 D n Q K v F y U o E c j X 7 5 q z e I S C K o N I R j r b u e G x s / x c o w w u m s 1 E s 0 j T E Z 4 y H t W i q x o N p P 5 6 F n 6 M w q A x R G y j 5 p 0 F z 9 v Z F i o f V U B H Y y C 6 m X v U z 8 z + s m J r z 2 U y b j x F B J F o f C h C M T o a w B N G C K E s O n l m C i m M 2 K y A g r T I z t q W R L 8 J a / v E p a F z X P 8 v v L S v 0 m r 6 M I J 3 A K V f D g C u p w B w 1 o A o F H e I Z X e H M m z o v z 7 n w s R g t O v n M M f + B 8 / g D E s J I U < / l a t e x i t > w < l a t e x i t s h a 1 _ b a s e 6 4 = " s G a g Z R p s B D Q Q X L O T t G o R B 0 V i 9 S c = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 S Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 L e M + w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " s G a g Z R p s B D Q Q X L O T t G o R B 0 V i 9 S c = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 S Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 L e M + w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " s G a g Z R p s B D Q Q X L O T t G o R B 0 V i 9 S c = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 S Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 L e M + w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " s G a g Z R p s B D Q Q X L O T t G o R B 0 V i 9 S c = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 S Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 L e M + w = = < / l a t e x i t > w < l a t e x i t s h a 1 _ b a s e 6 4 = " s G a g Z R p s B D Q Q X L O T t G o R B 0 V i 9 S c = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 S Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 L e M + w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " s G a g Z R p s B D Q Q X L O T t G o R B 0 V i 9 S c = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 S Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 L e M + w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " s G a g Z R p s B D Q Q X L O T t G o R B 0 V i 9 S c = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 S Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 L e M + w = = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " s G a g Z R p s B D Q Q X L O T t G o R B 0 V i 9 S c = " > A A A B 6 H i c b Z B N S 8 N A E I Y n 9 a v W r 6 p H L 4 t F 8 F Q S E f R Y 9 O K x B f s B b S i b 7 a R d u 9 m E 3 Y 1 S Q n + B F w + K e P U n e f P f u G 1 z 0 N Y X F h 7 e m W F n 3 i A R X B v X / X Y K a + s b m 1 v F 7 d L O 7 t 7 + Q f n w q K X j V D F s s l j E q h N Q j Y J L b B p u B H Y S h T Q K B L a D 8 e 2 s 3 n 5 E p X k s 7 8 0 k Q T + i Q 8 l D z q i x V u O p X 6 6 4 V X c u s g p e D h X I V e + X v 3 q D m K U R S s M E 1 b r r u Y n x M 6 o M Z w K n p V 6 q M a F s T I f Y t S h p h N r P 5 o t O y Z l 1 B i S M l X 3 S k L n 7 e y K j k d a T K L C d E T U j v V y b m f / V u q k J r / 2 M y y Q 1 K N n i o z A V x M R k d j U Z c I X M i I k F y h S 3 u x I 2 o o o y Y 7 M p 2 R C 8 5 Z N X o X V R 9 S w 3 L i u 1 m z y O I p z A K Z y D B 1 d Q g z u o Q x M Y I D z D K 7 w 5 D 8 6 L 8 + 5 8 L F o L T j 5 z D H / k f P 4 A 5 L e M + w = = < / l a t e x i t > ↵ < l a t e x i t s h a 1 _ b a s e 6 4 = " 6 E t S d g p K G 2 q h Z b j G Z 4 j 1 Q L t Q z n s = " > A A A B 7 X i c b Z B N S w M x E I Z n 6 1 e t X 1 W P X o J F 8 F R 2 R d B j 0 Y v H C v Y D 2 q X M p m k b m 0 2 W J C u U p f / B i w d F v P p / v P l v T N s 9 a O s L g Y d 3 Z s j M G y W C G + v 7 3 1 5 h b X 1 j c 6 u 4 X d r Z 3 d s / K B 8 e N Y 1 K N W U N q o T S 7 Q g N E 1 y y h u V W s H a i G c a R Y K 1 o f D u r t 5 6 Y N l z J B z t J W B j j U P I B p 2 i d 1 e y i S E b Y K 1 f 8 q j 8 X W Y U g h w r k q v f K X 9 2 + o m n M p K U C j e k E f m L D D L X l V L B p q Z s a l i A d 4 5 B 1 H E q M m Q m z + b Z T c u a c P h k o 7 Z 6 0 Z O 7 + n s g w N m Y S R 6 4 z R j s y y 7 W Z + V + t k 9 r B d Z h x m a S W S b r 4 a J A K Y h W Z n U 7 6 X D N q x c Q B U s 3 d r o S O U C O 1 L q C S C y F Y P n k V m h f V w P H 9 Z a V 2 k 8 d R h B M 4 h X M I 4 A p q c A d 1 a A C F R 3 i G V 3 j z l P f i v X s f i 9 a C l 8 8 c w x 9 5 n z + L g 4 8 Y < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 6 E t S d g p K G 2 q h Z b j G Z 4 j 1 Q L t Q z n s = " > A A A B 7 X i c b Z B N S w M x E I Z n 6 1 e t X 1 W P X o J F 8 F R 2 R d B j 0 Y v H C v Y D 2 q X M p m k b m 0 2 W J C u U p f / B i w d F v P p / v P l v T N s 9 a O s L g Y d 3 Z s j M G y W C G + v 7 3 1 5 h b X 1 j c 6 u 4 X d r Z 3 d s / K B 8 e N Y 1 K N W U N q o T S 7 Q g N E 1 y y h u V W s H a i G c a R Y K 1 o f D u r t 5 6 Y N l z J B z t J W B j j U P I B p 2 i d 1 e y i S E b Y K 1 f 8 q j 8 X W Y U g h w r k q v f K X 9 2 + o m n M p K U C j e k E f m L D D L X l V L B p q Z s a l i A d 4 5 B 1 H E q M m Q m z + b Z T c u a c P h k o 7 Z 6 0 Z O 7 + n s g w N m Y S R 6 4 z R j s y y 7 W Z + V + t k 9 r B d Z h x m a S W S b r 4 a J A K Y h W Z n U 7 6 X D N q x c Q B U s 3 d r o S O U C O 1 L q C S C y F Y P n k V m h f V w P H 9 Z a V 2 k 8 d R h B M 4 h X M I 4 A p q c A d 1 a A C F R 3 i G V 3 j z l P f i v X s f i 9 a C l 8 8 c w x 9 5 n z + L g 4 8 Y < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 6 E t S d g p K G 2 q h Z b j G Z 4 j 1 Q L t Q z n s = " > A A A B 7 X i c b Z B N S w M x E I Z n 6 1 e t X 1 W P X o J F 8 F R 2 R d B j 0 Y v H C v Y D 2 q X M p m k b m 0 2 W J C u U p f / B i w d F v P p / v P l v T N s 9 a O s L g Y d 3 Z s j M G y W C G + v 7 3 1 5 h b X 1 j c 6 u 4 X d r Z 3 d s / K B 8 e N Y 1 K N W U N q o T S 7 Q g N E 1 y y h u V W s H a i G c a R Y K 1 o f D u r t 5 6 Y N l z J B z t J W B j j U P I B p 2 i d 1 e y i S E b Y K 1 f 8 q j 8 X W Y U g h w r k q v f K X 9 2 + o m n M p K U C j e k E f m L D D L X l V L B p q Z s a l i A d 4 5 B 1 H E q M m Q m z + b Z T c u a c P h k o 7 Z 6 0 Z O 7 + n s g w N m Y S R 6 4 z R j s y y 7 W Z + V + t k 9 r B d Z h x m a S W S b r 4 a J A K Y h W Z n U 7 6 X D N q x c Q B U s 3 d r o S O U C O 1 L q C S C y F Y P n k V m h f V w P H 9 Z a V 2 k 8 d R h B M 4 h X M I 4 A p q c A d 1 a A C F R 3 i G V 3 j z l P f i v X s f i 9 a C l 8 8 c w x 9 5 n z + L g 4 8 Y < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " 6 E t S d g p K G 2 q h Z b j G Z 4 j 1 Q L t Q z n s = " > A A A B 7 X i c b Z B N S w M x E I Z n 6 1 e t X 1 W P X o J F 8 F R 2 R d B j 0 Y v H C v Y D 2 q X M p m k b m 0 2 W J C u U p f / B i w d F v P p / v P l v T N s 9 a O s L g Y d 3 Z s j M G y W C G + v 7 3 1 5 h b X 1 j c 6 u 4 X d r Z 3 d s / K B 8 e N Y 1 K N W U N q o T S 7 Q g N E 1 y y h u V W s H a i G c a R Y K 1 o f D u r t 5 6 Y N l z J B z t J W B j j U P I B p 2 i d 1 e y i S E b Y K 1 f 8 q j 8 X W Y U g h w r k q v f K X 9 2 + o m n M p K U C j e k E f m L D D L X l V L B p q Z s a l i A d 4 5 B 1 H E q M m Q m z + b Z T c u a c P h k o 7 Z 6 0 Z O 7 + n s g w N m Y S R 6 4 z R j s y y 7 W Z + V + t k 9 r B d Z h x m a S W S b r 4 a J A K Y h W Z n U 7 6 X D N q x c Q B U s 3 d r o S O U C O 1 L q C S C y F Y P n k V m h f V w P H 9 Z a V 2 k 8 d R h B M 4 h X M I 4 A p q c A d 1 a A C F R 3 i G V 3 j z l P f i v X s f i 9 a C l 8 8 c w x 9 5 n z + L g 4 8 Y < / l a t e x i t > a) b) c) d) media media impact media media tech

Figure 5 : Illustration of our model for an episode with N = 3, K = 1, L = 2. The attention generator translates the distributional signatures from the source pool and the support set into an attention α for each input example x (5a). The ridge regressor utilizes the generated attention to weight the lexical representations (5b). It then learns from the support set (5c) and makes predictions over the query set (5d).

It is well-documented in literature that words which appear frequently are unlikely to be informative (Sparck Jones, 1972) . Thus, we would like to downweigh frequent words and upweight rare words. To measure general word importance, we select an established approach by Arora et al. (2016) :

EQUATION

where ε = 10 -3 , x i is the i th word of input example x, and P (x i ) is the unigram likelihood of x i over the source pool.

On the other hand, words that are discriminative in the support set are likely to be discriminative in the query set. Thus, we define the following statistic to reflect class-specific word importance:

EQUATION

where the conditional likelihood P (y | x i ) is estimated over the support set using a regularized linear classifier 3 and H(•) is the entropy operator. We note that t(•) measures the uncertainty of the class label y, given the word. Thus, words that exhibit a skewed distribution will be highly weighted.

Directly applying these statistics may not result in good performance for two reasons: 1) the two statistics contain complementary information, and it is unclear how to combine them; and 2) these statistics are noisy approximations to word importance for classification. To bridge this gap, we concatenate these signatures and employ a bi-directional LSTM (Hochreiter & Schmidhuber, 1997) to fuse the information across the input: h = biLSTM([s(x); t(x)]). Finally, we use dot-product attention to predict the attention score α i of word x i :

EQUATION

where h i is the output of the biLSTM at position i and v is a learnable vector.

4.2. RIDGE REGRESSOR:

Informed by the attention generator, the ridge regressor quickly learns to make predictions after seeing a few examples. First, for each example in a given episode, we construct a lexical representation that focuses on important words, as indicated by attention scores. Next, given these lexical representations, we train the ridge regressor on the support set from scratch. Finally, we make predictions over the query set and use the loss to teach the attention generator to produce better attention.

is a bijection, the conditional after perturbation can be expressed as P(x | y) = P σ -1 (x) | y , where σ -1 (x) is the result of applying σ -1 to every word in x. Intuitively, Theorem 1 tells us that if a word w is a discriminative feature for the original classification task, then after the change of the input distribution, the word σ(w) should be a discriminative feature for the new task. This property makes sense since word-importance for classification should depends only on the relative differences between classes. In fact, the theorem holds when the input to the attention generator is any function of unigram counts. We also note that the classification performance on the query set can be different, as the pre-trained embedding function f ebd (•) is generally not invariant under such perturbations.

5. EXPERIMENTAL SETUP:



5.1. DATASETS:

We evaluate our approach on five text classification datasets and one relation classification dataset.5 (See Appendix A.4 for more details.)

20 Newsgroups is comprised of informal discourse from news discussion forums (Lang, 1995) . Documents are organized under 20 topics.

RCV1 is a collection of Reuters newswire articles from 1996 to 1997 (Lewis et al., 2004) . These articles are written in formal speech and labeled with a set of topic codes. We consider 71 secondlevel topics as our total class set and discard articles that belong to more than one class.

Reuters-21578 is a collection of shorter Reuters articles from 1987 (Lewis, 1997) . We use the standard ApteMod version of the dataset. We discard articles with more than one label and consider 31 classes that have at least 20 articles.

Amazon product data contains customer reviews from 24 product categories (He & McAuley, 2016) . Our goal is to classify reviews into their respective product categories. Since the original dataset is notoriously large (142.8 million reviews), we select a more tractable subset by sampling 1000 reviews from each category.

HuffPost headlines consists of news headlines published on HuffPost between 2012 and 2018 (Misra, 2018) . These headlines split among 41 classes. They are shorter and less grammatical than formal sentences.

FewRel is a relation classification dataset developed for few-shot learning (Han et al., 2018) . Each example is a single sentence, annotated with a head entity, a tail entity, and their relation. The goal is to predict the correct relation between the head and tail. The public dataset contains 80 relation types.

5.2. BASELINES:

We compare our model (denoted as OUR) to different combinations of representations and learning algorithms. Details of the baselines may be found in Appendix A.11.

Representations We evaluate three representations. AVG represents each example as the mean of its embeddings. IDF represents each example as the weighted average of its word embeddings, with weights given by inverse document frequency over all training examples. CNN applies 1D convolution over the input words and obtains the representation by max-over-time pooling (Kim, 2014) .

Learning algorithms In addition to the ridge regressor (RR) (Bertinetto et al., 2019) , we evaluate two standard supervised learning algorithms and two meta-learning algorithms. NN is a 1-nearestneighbor classifier under Euclidean distance. FT pre-trains a classifier over all training examples, then finetunes the network using the support set (Chen et al., 2019) . MAML meta-learns a prior over model parameters, so that the model can quickly adapt to new classes (Finn et al., 2017) . Prototypical network (PROTO) meta-learns a metric space for few-shot classification by minimizing Table 2 : 5-way 1-shot and 5-way 5-shot classification on HuffPost and FewRel using BERT.

Ablation study We perform ablation studies on the attention generator. These results are shown at the bottom of Table 1 . We observe that both statistics s(•) and t(•) contribute to the performance, though the former has a larger impact. We also note that instead of computing word importance independently for each word using a multilayer perceptron (OUR w/o biLSTM), fusing information across the input with an biLSTM improves performance slightly.

Finally, we observe that restricting meta-knowledge to distributional signatures is essential to performance: when both lexical word embeddings and distributional signatures are fed to the attention generator (OUR w EBD), the performance drops consistently.

Contextualized representations:

For sentence-level datasets (FewRel, HuffPost), we also experiment with contextualized representations, given by BERT (Devlin et al., 2018) . These results are shown in Table 2 . While BERT significantly improves classification performance on FewRel, we observe no performance boost on HuffPost. We postulate that this discrepancy arises because relation classification is highly contextual, while news classification is mostly keyword-based.

Analysis We visualize our attention-weighted representation φ(x) in Figure 7 . Compared to directly using general word importance s(x) or class-specific word importance t(x), our method produces better separation, which enables effective learning from a few examples. This highlights that the power of our approach lies not in the distributional signatures themselves, but rather in the representations learned on top of them.

Figure 8 visualizes the model's input and output on the same query example in two testing episodes.

The example belongs to the class jobs in the Reuters dataset. First, we observe that our model generates meaningful attention from noisy distributional signatures. Furthermore, the generated Fine-grained classification Coarse-grained classification s(x) finnish unemployment was 6.7 pct in december last year compared with 6.8 pct in november and 6.1 pct in december 1985 , the central statistical office said finnish unemployment was 6.7 pct in december last year compared with 6.8 pct in november and 6.1 pct in december 1985 , the central statistical office said t(x) finnish unemployment was 6.7 pct in december last year compared with 6.8 pct in november and 6.1 pct in december 1985 , the central statistical office said finnish unemployment was 6.7 pct in december last year compared with 6.8 pct in november and 6.1 pct in december 1985 , the central statistical office said OURS finnish unemployment was 6.7 pct in december last year compared with 6.8 pct in november and 6.1 pct in december 1985 , the central statistical office said finnish unemployment was 6.7 pct in december last year compared with 6.8 pct in november and 6.1 pct in december 1985 , the central statistical office said Figure 8 : Attention weights generated by our model are specific to task. We visualize our model's inputs s(x) (top), t(x) (middle), and output (bottom) for one query example from class jobs in Reuters dataset. Word "statistical" is downweighed for jobs when compared to other economics classes (left), but it becomes important when considering dissimilar classes (right). Fine-grained classes: jobs, retail, industrial production index, wholesale production index, consumer production index. Coarse-grained classes: jobs, cocoa, aluminum, copper, reserves.

attention is task-specific: in the depicted example, if the episode contains other economics-related classes, the word "statistical" is downweighed by our model. Conversely, "statistical" is upweighted when we compare jobs to other distant classes.

7. CONCLUSION:

In this paper, we propose a novel meta-learning approach that capitalizes on the connection between word importance and distributional signatures to improve few-shot classification. Specifically, we learn an attention generator that translates distributional statistics into high-quality attention. This generated attention then provides guidance for fast adaptation to new classification tasks. Experimental results on both text and relation classification validate that our model identifies important words for new classes. The effectiveness of our approach demonstrates the promise of meta learning with distributional signatures.

Algorithm 1 Meta-training procedure. N train = Y train is the number of training classes. N < N train is the number of classes of each few-shot task. K, L are the number of support and query examples per target class, respectively. To ease notation, we use SAMPLE(S, N ) to denote a subset of S with N elements, chosen uniformly at random. We use D(Y) ⊆ D to denote the set of all elements (x i , y i ) for which y i ∈ Y. classes are distant to val and test. We applied the easy split to one sentence-level dataset (Huff-Post) and one document-level dataset (Reuters-21578). Hard split is used for the other four datasets (details below). This setting tests the generalization capacity of the algorithm, following Xian et al. (2017) .

∈ Y do T S ← T S ∪ SAMPLE(D({y}), K) T Q ← T Q ∪ SAMPLE(D({y}) \ T S ,

20 Newsgroups Each class in 20 Newsgroups belongs to one of six top-level categories, which roughly correspond to computers, recreation, science, politics, religion, and for-sale. We partition the set of labels so that no top-level category spans two splits. Train contains "sci" and "rec," val contains "comp," and test contains all other labels.

Amazon The Amazon dataset does not come with predefined top-level categories. To generate a hard split, we first apply spectral clustering to classes based on their word distributions. Then we select train/val/test from different clusters.

RCV1:

We apply the same approach as above.

FewRel While FewRel does not provide higher-level categories, we observe that most relations occur between named entities of similar types. Thus, we extract the named entity type of the head and tail for each example using a pretrained spaCy parser. 8 For each class, we determine the most common head and tail entity types. Test contains all classes for which the most common head entity type is WORK OF ART. Train and validation were arbitrarily split to contain the remaining relations.

A.5 OTHER BASELINES:

We also consider two other baselines: induction network (Geng et al., 2019) and P-MAML (Zhang et al., 2019) . Implementation details may be found in Appendix A.11. • Induction network encodes input examples using a biLSTM with self-attentive pooling (Lin et al., 2017) . Based on the encoded representation, it then computes a prototype for each class through dynamic routing over the support set (Sabour et al., 2017) . Finally, it uses a neural tensor layer (Socher et al., 2013) to predict the relation between each query example and the class prototypes. • P-MAML combines pre-training with MAML. It first finetunes pre-trained BERT representation on the meta-training data using masked language modeling (Devlin et al., 2018) .

Based on this finetuned representation, it trains first-order MAML Finn et al. (2017) to enable fast adaptation.

Table 4 shows that learning with distributional signatures (OUR) significantly outperforms the two baselines across all datasets. This is not surprising: both baselines build meta-knowledge on top of lexical representations, which may not generalize when the lexical distributions are vastly different between seen classes and unseen classes. Figure 9 shows the learning curve of INDUCTION NET. Similar to Figure 6 , we see that INDUCTION NET overfits to meta-train classes with poor generalization to meta-test classes. Figure 10 depicts the perplexity of BERT's masked language model objective, during finetuning for P-MAML. Again, we see that the meta-train and meta-test classes exhibit vast lexical mismatch, as the perplexity improves only slightly on meta-test classes.

A.6 ANALYSIS OF LEARNED REPRESENTATION:

To further understand the rationale behind our performance boost, we provide a more detailed analysis on our model's empirical behavior.

Visualizing the embedding space We visualize our attention-weghted representation φ(x) in 20 Newsgroups (Figure 11 ) and HuffPost (Figure 12 ). We observe that our model produces better separation than the unweighted average AVG and directly using the distributional statistics, s(x) or t(x). For instance, in 20 Newsgroups, our model recognizes three clusters: {talk. Visualizing the generated attention Figure 14 visualizes the generated attention for a testing episode in HuffPost. We observe that our model identifies meaningful keywords from the sentence.

A.7 EFFECT OF SOFTMAX CALIBRATION:

In Section 4.2, we use ridge regression with softmax calibration as our downstream predictor due to its efficiency and effectiveness. To study the effect of the softmax calibration, we follow Bertinetto et al. (2019) and compare ridge regression against direct optimization of binary logistic classifiers (LR) with Newton's method (five iterations). Since Newton's method admits a closed-form solution at each iteration, the final solution is also end-to-end differentiable. For each N -way K-shot episode, to adapt the binary logistic classifier for multi-class prediction, we train N binary (one-vsall) classifiers using the support set. The output of the N binary classifiers are combined to make the final predictions on the query set (Bishop, 2006, Chapter 4.1.2) .

The results are shown in Table 5 . Overall, RR and LR produce similar results (78.0 vs. 77.1 on 5way 5-shot, 60.1 vs. 58.5 on 5-way 1-shot, respectively), though RR performs slightly better. When using LR as the downstream predictor, learning with distributional signatures improves 5-way 1-shot accuracy by 6.1% and 5-way 5-shot accuracy by 3.2% over the best baseline. Note that LR requires solving N binary classifications for each episode, which is less efficient than RR. In Table 1 , we fix word embeddings during meta-training for all experiments. Table 6 studies the effect of finetuning word embeddings during meta-training. We observe that when the word embeddings are finetuned, performance drops for nearly all models.

To intuitively understand this behavior, we compare the meta-train vocabulary with the meta-test vocabulary. That is, we avoid overfitting to the meta-train vocabulary and destroying the geometry of the pretrained embeddings.

A.9 DISTRIBUTIONAL SIGNATURES WITH OTHER CLASSIFIERS

In this paper, we demonstrate the advantage of learning meta-knowledge on top of distributional signatures. We focus on a ridge regressor (Bertinetto et al., 2019) as our downstream classifier due to its simplicity and effectiveness. However, the idea of learning with distributional signatures is not limited to ridge regression; rather, we can combine distributional signatures with other learning methods such as prototypical networks (Snell et al., 2017) and induction networks (Geng et al., 2019) .

Prototypical networks To augment a prototypical network with features learned from distributional signatures, we can construct per-class prototypes based on the attention-weighted representation φ(x) (Section 4.2). From Table 7 , we see that learning with distributional signatures improves 5way 1-shot accuracy by 9.9% and 5-way 5-shot accuracy by 16.7%, against the best prototypical network baseline for each dataset.

Induction networks We can also augment induction networks with meta-knowledge learned from distributional signatures. Specifically, we directly feed our attention-weighted representation φ(x) (Section 4.2) to the induction module. To avoid over-fitting on meta-train features, we replace the meta-learned relation module (see Geng et al. (2019) for details) by a parameter-free nearest neighbour predictor, similar to that of prototypical networks. From Table 7 , we again see that learning with distributional signatures improves 5-way 1-shot accuracy by 14.3% and 5-way 5-shot accuracy by 25.1% on average.

These empirical results clearly demonstrate the benefit of our approach. However, both prototypical networks and induction networks perform consistently worse than the ridge regressor across all 

:

Note this generated attention can be applied to any downstream predictor(Lee et al., 2019;Snell et al., 2017). Details from these experiments may be found in the Appendix A.9.

See Appendix A.1 for details.

We view this σ as a mapping from the vocabulary set V to itself.

All processed datasets along with their splits are publicly available.

We also provide the same positional embeddings to the baseline CNN.

less than 1 second on a single GeForce GTX TITAN X

https://spacy.io/

Figures and Tables:

Figure1: Different classes exhibit different word distributions in HuffPost headlines. We compute the local mutual information (LMI)(Evert, 2005) between words and classes. For each class, we include its top 2 LMI-ranked words. Darker colors indicate higher LMI.

Figure 3: Episode generation. a) Meta-training: First, sample N classes from Y train . Then, sample the support set and the query set from the N classes. We use examples from the remaining classes to form the source pool. b) Meta-testing: Select N new classes from Y test and sample the support set and the query set from these N classes. We use all examples from Y train to form the source pool.

Figure 4: Single episode with N = 3, K = 1, L = 2. Rectangles denote input examples. The text inside corresponds to their labels. An episode contains a support set, a query set, and a source pool.

t e x i t s h a 1 _ b a s e 6 4 = " m K

m w P O i / P u f C x a 1 5 x y 5 g T 9 k f P 5 A 7 T d k N 4 = < / l a t e x i t > Ridge regressor: Constructing representations support set this gorgeous grandma proves beauty has no expiration date x < l a t e x i t s h a 1 _ b a s e 6 4 = "

Learning curve of CNN+PROTO (left) v.s. OUR (right) on the Reuters dataset. We plot average 5-way 1-shot accuracy over 50 episodes sampled from seen classes (blue) and unseen classes (red). While OUR has weaker representational power, it generalizes better to unseen classes.

Figure 7: PCA visualization of the input representation for the query set of a testing episode (N = 5, K = 5, L = 500) sampled from 20 Newsgroups. Weighted averages of word embeddings given by (a) s(•), (b) t(•), and (c) the attention generator meta-trained on a disjoint set of training classes.

Training set D = {(x1, y1), (x2, y2), . . .} where each yi ∈ Y train = {1, . . . , Ntrain} Hyperparameters: parameters of φ; coefficients for losses λ, a, b repeat Y ← SAMPLE(Y train , N ) sample target classes D src ← D(Y train \ Y) create support pool T S , T Q ← ∅, ∅ sample support set and query set for y

Figure 9: Learning curves of INDUCTION NET on Huffpost and Reuters. We plot average 5-way 5-shot accuracy over 50 episodes sampled from seen classes (blue) and unseen classes (red).

Figure11: PCA visualization of the input representation for a testing episode in 20 Newsgroups with N = 5, K = 5, L = 500 (the query set has 500 examples per class). AVG: average word embeddings. s(•): weighted average of word embeddings with weights given by s(•). t(•): weighted average of word embeddings with weights given by t(•). OUR: weighted average of word embeddings with weights given by the attention generator meta-trained on a disjoint set of training classes.

Figure13: Average cosine similarity to the oracle word importance over the query set of a testing episode with N = 5, K = 5, L = 500. This oracle is estimated using all labeled examples from the N classes. Since examples in HuffPost Headlines are 30 times shorter, the cosine similarities are higher in this corpora. AVG: uniform distribution over the words. s(•): word importance estimated directly by s(•). t(•): word importance estimated directly by t(•). OUR: word importance estimated by the meta-learned attention generator.

L)Generate attention scores using T S , T Q , D src Eq. 3 Construct attention-weighted representations ΦS, ΦQ using φ(•)Eq. 4 Compute closed-form solution W of ridge regression from λ, ΦS, and support labels YS Eq. 6 Given W, a, b, compute cross entropy loss L CE on the query set ΦQ, YQ Update meta parameters (attention generator and a, B) using L CE until stopping criterion is met

Table content: <table><tr><td>Dataset</td><td colspan="6"># tokens / example vocab size # examples / cls # train cls # val cls # test cls</td></tr><tr><td>20 Newsgroups</td><td>340</td><td>32137</td><td>941</td><td>8</td><td>5</td><td>7</td></tr><tr><td>RCV1</td><td>372</td><td>7304</td><td>20</td><td>37</td><td>10</td><td>24</td></tr><tr><td>Reuters</td><td>168</td><td>2234</td><td>20</td><td>15</td><td>5</td><td>11</td></tr><tr><td>Amazon</td><td>140</td><td>17062</td><td>1000</td><td>10</td><td>5</td><td>9</td></tr><tr><td>HuffPost</td><td>11</td><td>8218</td><td>900</td><td>20</td><td>5</td><td>16</td></tr><tr><td>FewRel</td><td>24</td><td>16045</td><td>700</td><td>65</td><td>5</td><td>10</td></tr></table>

Dataset statistics. See Appendix A.4 for details.

Table content: <table/>

Comparison against Induction Net(Geng et al., 2019) and P-MAML(Zhang et al., 2019). We run P-MAML on text classification datasets with shorter documents, for which it is feasible to finetune BERT (after WordPiece tokenization, documents from "longer" datasets exceed BERT's max length of 512 tokens).

Table content: <table><tr><td>Method</td><td colspan="2">20 News</td><td>Amazon</td><td>HuffPost</td><td colspan="2">RCV1</td><td>Reuters</td><td colspan="2">FewRel</td></tr><tr><td/><td colspan="9">1 shot 5 shot 1 shot 5 shot 1 shot 5 shot 1 shot 5 shot 1 shot 5 shot 1 shot 5 shot</td></tr><tr><td colspan="10">INDUCTION NET 27.6 32.1 30.6 37.1 34.9 44.0 32.3 37.3 58.3 66.9 50.4 56.1</td></tr><tr><td>P-MAML</td><td>-</td><td>-</td><td colspan="2">47.1 58.4 31.0 51.3</td><td>-</td><td>-</td><td>53.0 72.9</td><td>-</td><td>-</td></tr><tr><td>OUR</td><td colspan="9">52.1 68.3 62.6 81.1 43.0 63.5 54.1 75.3 81.8 96.0 67.1 83.5</td></tr></table>

On Amazon, 15095 of 28591 (52.8%) tokens in the meta-train vocabulary are not present in meta-test, and 21049 of 34545 (60.9%) tokens in the meta-test vocabulary are not present in metatrain. On Reuters, 3604 of 6372 (56.6%) meta-train tokens are not in meta-test, while 2481 of 5249 (47.2%) meta-test tokens are not in meta-test. Due to this lexical mismatch, finetuning will destroy the original geometry of the pretrained word embeddings, leading to poor generalization on unseen tasks. 5 shot 1 shot 5 shot 1 shot 5 shot 1 shot 5 shot 1 shot 5 shot 1 shot 5 shot 1 shot 5 shot AVG RR 37.6 57.2 50.2 72.7 36.3 54.8 48.1 72.6 63.4 90.0 53.2 72.2 48.1 69.9 IDF RR44.8 64.3 60.2 79.7 37.6 59.5 48.6 72.8 69.1 93.0 55.6 75.3 52.6  74.1 Results of 5-way 1-shot and 5-way 5-shot classification on six datasets using ridge regression vs. logistic regression with Newton's method.

Table content: <table><tr><td colspan="2">Method</td><td colspan="2">20 News</td><td>Amazon</td><td>HuffPost</td><td>RCV1</td><td>Reuters</td><td>FewRel</td><td>Average</td></tr><tr><td colspan="9">Rep. Alg. 1 shot CNN RR 32.2 44.3 37.3 53.8 37.3 49.9 41.8 59.4 71.4 87.9 56.8 71.8 46.1 61.2</td></tr><tr><td>OUR</td><td colspan="8">RR 52.1 68.3 62.6 81.1 43.0 63.5 54.1 75.3 81.8 96.0 67.1 83.5 60.1 78.0</td></tr><tr><td>AVG</td><td colspan="8">LR 37.7 56.8 50.0 71.6 36.3 56.0 47.9 72.5 62.7 89.3 52.4 72.9 47.8 69.9</td></tr><tr><td>IDF</td><td colspan="8">LR 46.1 64.2 60.5 79.9 38.7 59.3 47.3 72.5 68.1 93.0 55.5 74.9 52.7 74.0</td></tr><tr><td>CNN</td><td colspan="8">LR 32.6 47.5 41.0 58.5 37.6 50.9 44.2 63.6 75.0 90.3 58.7 72.1 48.2 63.8</td></tr><tr><td>OUR</td><td colspan="8">LR 49.7 67.6 61.6 80.7 41.9 61.9 54.4 74.1 79.5 95.6 65.8 82.8 58.8 77.1</td></tr><tr><td/><td colspan="2">Method</td><td/><td colspan="2">5-way 5-shot Reuters</td><td/><td colspan="2">5-way 5-shot Amazon</td></tr><tr><td/><td>Rep.</td><td>Alg.</td><td colspan="6">Fix embedding Finetune embedding Fix embedding Finetune embedding</td></tr><tr><td/><td>AVG</td><td>PROTO</td><td/><td>68.15 ±1.53</td><td colspan="2">58.40 ±2.84</td><td>51.99 ±2.72</td><td>41.34 ±1.61</td></tr><tr><td/><td>IDF</td><td>PROTO</td><td/><td>72.07 ±2.69</td><td colspan="2">65.93 ±3.08</td><td>59.24 ±1.10</td><td>48.37 ±2.41</td></tr><tr><td/><td>CNN</td><td>PROTO</td><td/><td>74.33 ±1.86</td><td colspan="2">74.20 ±2.85</td><td>44.49 ±1.53</td><td>44.08 ±2.21</td></tr><tr><td/><td>AVG</td><td>MAML</td><td/><td>62.46 ±0.58</td><td colspan="2">50.59 ±1.88</td><td>47.22 ±1.99</td><td>30.95 ±1.08</td></tr><tr><td/><td>IDF</td><td>MAML</td><td/><td>71.96 ±1.48</td><td colspan="2">68.76 ±1.39</td><td>62.45 ±1.33</td><td>50.63 ±0.61</td></tr><tr><td/><td>CNN</td><td>MAML</td><td/><td>85.00 ±0.76</td><td colspan="2">84.19 ±0.41</td><td>43.70 ±1.38</td><td>43.96 ±1.19</td></tr><tr><td/><td>AVG</td><td>RR</td><td/><td>90.00 ±0.49</td><td colspan="2">86.87 ±0.60</td><td>72.78 ±0.23</td><td>75.65 ±0.51</td></tr><tr><td/><td>IDF</td><td>RR</td><td/><td>93.02 ±0.45</td><td colspan="2">93.01 ±0.47</td><td>79.78 ±0.28</td><td>78.73 ±0.59</td></tr><tr><td/><td>CNN</td><td>RR</td><td/><td>87.93 ±1.49</td><td colspan="2">85.47 ±0.57</td><td>53.89 ±1.54</td><td>52.63 ±2.02</td></tr><tr><td/><td>OUR</td><td>RR</td><td/><td>96.00 ±0.27</td><td colspan="2">95.98 ±0.39</td><td>81.16 ±0.31</td><td>78.94 ±0.56</td></tr></table>

Fixed vs. finetuned word embeddings during meta-training. Since meta-train and meta-test classes exhibit different word distributions, fixing the word embedding yields better generalization.

Table content: <table/>

comp.sys.ibm.pc.hardware rec.sport.baseball soc.religion.christian Top 10 LMI-ranked words for each class in 20 Newsgroup. Class names are shown in italic. Different class exhibit different salient features.

Table content: <table><tr><td>talk.politics.mideast</td><td>sci.space</td><td>misc.forsale</td><td>talk.politics.misc</td><td>comp.graphics</td></tr><tr><td>israel</td><td>space</td><td>sale</td><td>president</td><td>image</td></tr><tr><td>armenian</td><td>nasa</td><td>00</td><td>cramer</td><td>graphics</td></tr><tr><td>turkish</td><td>launch</td><td>shipping</td><td>mr</td><td>jpeg</td></tr><tr><td>armenians</td><td>orbit</td><td>offer</td><td>stephanopoulos</td><td>images</td></tr><tr><td>israeli</td><td>shuttle</td><td>condition</td><td>people</td><td>gif</td></tr><tr><td>jews</td><td>moon</td><td>1st</td><td>government</td><td>format</td></tr><tr><td>armenia</td><td>henry</td><td>price</td><td>optilink</td><td>file</td></tr><tr><td>arab</td><td>earth</td><td>forsale</td><td>myers</td><td>3d</td></tr><tr><td>people</td><td>mission</td><td>asking</td><td>clayton</td><td>ftp</td></tr><tr><td>jewish</td><td>solar</td><td>comics</td><td>gay</td><td>color</td></tr><tr><td>sci.crypt</td><td colspan="3">comp.windows.x comp.os.ms-windows.misc talk.politics.guns</td><td>talk.religion.misc</td></tr><tr><td>key</td><td>window</td><td>ax</td><td>gun</td><td>god</td></tr><tr><td>clipper</td><td>xx</td><td>max</td><td>guns</td><td>jesus</td></tr><tr><td>encryption</td><td>motif</td><td>windows</td><td>fbi</td><td>sandvik</td></tr><tr><td>chip</td><td>server</td><td>g9v</td><td>firearms</td><td>christian</td></tr><tr><td>keys</td><td>widget</td><td>b8f</td><td>atf</td><td>bible</td></tr><tr><td>security</td><td>file</td><td>a86</td><td>batf</td><td>jehovah</td></tr><tr><td>privacy</td><td>xterm</td><td>145</td><td>weapons</td><td>christ</td></tr><tr><td>government</td><td>x11</td><td>pl</td><td>people</td><td>lord</td></tr><tr><td>escrow</td><td>entry</td><td>1d9</td><td>waco</td><td>kent</td></tr><tr><td>des</td><td>dos</td><td>34u</td><td>cdt</td><td>brian</td></tr><tr><td>rec.autos</td><td>sci.med</td><td>comp.sys.mac.hardware</td><td>sci.electronics</td><td>rec.sport.hockey</td></tr><tr><td>car</td><td>medical</td><td>mac</td><td>circuit</td><td>hockey</td></tr><tr><td>cars</td><td>disease</td><td>apple</td><td>wire</td><td>game</td></tr><tr><td>engine</td><td>msg</td><td>centris</td><td>ground</td><td>team</td></tr><tr><td>ford</td><td>cancer</td><td>quadra</td><td>wiring</td><td>nhl</td></tr><tr><td>oil</td><td>health</td><td>lc</td><td>voltage</td><td>play</td></tr><tr><td>dealer</td><td>patients</td><td>monitor</td><td>battery</td><td>season</td></tr><tr><td>callison</td><td>doctor</td><td>duo</td><td>copy</td><td>games</td></tr><tr><td>mustang</td><td>hiv</td><td>nubus</td><td>amp</td><td>25</td></tr><tr><td>com</td><td>food</td><td>drive</td><td>electronics</td><td>ca</td></tr><tr><td>autos</td><td>diet</td><td>simms</td><td>audio</td><td>pit</td></tr><tr><td colspan="2">alt.atheism rec.motorcycles god bike</td><td>scsi</td><td>baseball</td><td>god</td></tr><tr><td>atheism</td><td>dod</td><td>drive</td><td>game</td><td>church</td></tr><tr><td>atheists</td><td>ride</td><td>ide</td><td>year</td><td>jesus</td></tr><tr><td>keith</td><td>bmw</td><td>controller</td><td>team</td><td>christ</td></tr><tr><td>livesey</td><td>com</td><td>card</td><td>players</td><td>sin</td></tr><tr><td>morality</td><td>riding</td><td>bus</td><td>games</td><td>christians</td></tr><tr><td>religion</td><td>bikes</td><td>drives</td><td>hit</td><td>christian</td></tr><tr><td>moral</td><td>motorcycle</td><td>bios</td><td>braves</td><td>rutgers</td></tr><tr><td>islamic</td><td>dog</td><td>disk</td><td>runs</td><td>bible</td></tr><tr><td>say</td><td>rider</td><td>pc</td><td>pitcher</td><td>faith</td></tr></table>

Peer review (iclr2020):
UPDATE: Based on the extensive improvements by the authors, I have updated my rating. However, I still have doubts about the potential of this approach to reach practically useful levels of accuracy. This paper introduces a simple method to weight pretrained lexical features for use in meta learning of few-shot text classification. The method boils down to weighting word inut features, in the form of pretrained word-embeddings, by attention computed from inverse document frequency and class local mutual information. The idea is that this measure of feature informativeness transfers between tasks, whereas lexical features themselves are highly task-specific. The approach is well motivated and is empirically shown to outperform existing approaches to few-shot text classification with a significant margin. While the improvement over existing approaches is quite substantial, I believe the paper should not be accepted to ICLR for the following reasons. First, the contribution is quite limited and not particularly novel. While two weight functions are proposed, the majority of improvement comes simply from normalizing IDF with attention. Based on existing work on delexicalized features for NLP tasks such as parsing, this is quite a straightforward extension. Given the limited contribution, a short paper seems a better fit. Second, the approach simply trades variance for bias. This brings us to the question of how likely the approach is to be a building block in bringing us towards a pratically useful few-shot classification method. Given the weak representational power of the model, I believe this is unlikely. I see a situation similar to syntactic parsing for low-resource languages, where a collection of simple techniques similar in spirit to the current approach, like delexicalization, brought results far above the naive baselines, but never approached practically useful results. I think this is a crucial point to address in meta-learning research in general to make sure we’re not just solving a toy problem with tailored heuristics. Additional notes: What is the motivation for using a BiLSTM to combine inverse document frequency and inverse class entropy? Is the sequence information at all useful, or would a simple projection and nonlinearity give the same result? The theoretical analysis is completely self-evident from the definition of the feature space. Replaing a feature with an equivalent feature of course gives the same result and I don’t see the need to “mathematize” this. The effect of approximating logistic regression with linear regression + calibration is not analyzed and it is not clear what the effect of this approximation is in the text classification scenario. I would suggest to compare to differentiating through a direct optimization of the logistic formulation, for example with Newton’s method, or plain SGD, as in Bertinetto et al. (2019). Table 1. Why not run the attention-based feature aggregator together with all algorithms. The main contribution is at the input representation level, and this should be applicable across algorithms. In fact, if we remove the BiLSTM which seems to have a very small effect the representation function does not contain learnable parameters in itself. Please provide the average across datasets in Table 1.