SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc., 2020.
[56] Haokun Liu, Derek Tam, Muqeeth Mohammed, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[57] Ethan Perez, Douwe Kiela, and Kyunghyun Cho. True few-shot learning with language models. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[58] Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4596-4604. PMLR, 10-15 Jul 2018.
[59] Rémi Lebret, David Grangier, and Michael Auli. Generating text from structured data with application to the biography domain. CoRR, abs/1603.07771, 2016.
A Multi-armed bandits
The Multi-Armed Bandit (MAB) setting is a problem from machine learning where a learner interacts with an environment over  $N$  rounds by following a policy  $\pi$ . At each round  $t$  the learner chooses one of the environment's  $K$  arms,  $a \in \mathcal{A}$  where  $K = |\mathcal{A}|$ , after which the environment provides a reward  $R_{t}$ . Rewards for unplayed arms are not observed. The goal of the learner is to adopt a policy  $\pi$  that selects actions that lead to the largest cumulative reward over  $N$  rounds,  $R = \sum_{t=1}^{N} R_{t}$ . In this work we assume a finite  $K$  and that the underlying reward distribution of each arm may have a variety of properties (e.g. stochasticity or stationarity) depending on the exact scenario, leading to different optimal policies [34].
Adversarial MAB. The adversarial MAB setting assumes that the reward-generating process is controlled by an adversary. This assumption allows for modelling non-stationary and highly stochastic reward signals. We will later show why our FLAD formulation fits into this setting. Under this setting, it is assumed that an adversary is given access to the learner's policy  $\pi$  and determines the sequence of rewards,  $(R_{a,t})_{t=1}^{N}$ , for each arm prior to play [38]. At each turn  $\pi$  determines a distribution over actions,  $p(\mathcal{A})$ , and an action is sampled from the distribution,  $a \sim p(\mathcal{A})$ . See Lattimore & Szechováři [34] for further details.
The EXP3 algorithm. The EXP3 algorithm ("Exponential-weight algorithm for Exploration and Exploitation") targets the adversarial multi-armed bandit problem by choosing arms according to a Gibbs distribution based on the empirically determined importance-weighted rewards of arms [21]. To allow for exploration, EXP3 mixes the Gibbs distribution with a uniform distribution.
Formally, let the exploration rate be  $\gamma \in (0,1]$ . At round  $t$ ,  $\pi$  defines the probability of selecting a given arm,  $a \in \mathcal{A}$ , as a linear combination of Gibbs and uniform distributions
$$
p _ {t} (a) = (1 - \gamma) \frac {\exp \left(\gamma \hat {R} _ {a , t - 1} / K\right)}{\sum_ {a ^ {\prime}} \exp \left(\gamma \hat {R} _ {a ^ {\prime} , t - 1} / K\right)} + \frac {\gamma}{K} \tag {1}
$$
where the importance weighted reward  $\hat{R}_{a,t}$  is calculated as
$$
\hat {R} _ {a, t} = \hat {R} _ {a, t - 1} + \frac {R _ {a , t}}{p _ {t - 1} (a)} \tag {2}
$$
and  $R_{a,t}$  denotes the observed reward. All unplayed arms,  $a' \neq a$  have unchanged importance weighted rewards;  $\hat{R}_{a',t} = \hat{R}_{a',t-1}$ .
Alge. rically, EXP3 takes the following steps at each round: First, calculate the sampling distribution  $p_t$  and sample an arm from the distribution. Then a reward  $R_{a,t}$  is observed and the algorithm updates the importance weighted reward  $\hat{R}_{a,t}$  for the played arm.
Informally, the use of an importance-weighted estimated reward compensates the rewards of actions that are less likely to be chosen, guaranteeing that the expected estimated reward is equal to the actual reward for each action. EXP3 is designed to be nearly optimal in the worst case, but due to the exploration rate it will select "bad" actions at a rate of  $\gamma / K$ . The exploration of EXP3 combined with importance-weighting allows the policy to handle non-stationary reward-generating processes.
The UCB1 algorithm. While the adversarial setting makes almost no assumptions about the reward-generating process and therefore maintains its performance guarantees under almost any circumstances, it can be outperformed in settings that are constrained. In this section we assume that the reward-generating processes are stationary Gaussian distributions. A common policy used to solve this MAB setting is the Upper Confidence Bound (UCB1) algorithm, which assigns each arm a value called the upper confidence bound based on Hoeffding's inequality [22]. The UCB1 algorithm is based on the principle of optimism in the face of uncertainty, meaning that with high probability the upper confidence bound assigned to each arm is an overestimate of the unknown mean reward.
Formally, let the estimated mean reward of arm  $a$  after being played  $n_a$  times be  $\hat{R}_a$  and the true mean reward be  $R_a$ , then
$$
\mathbb {P} \bigg (R _ {a} \geq \hat {R} _ {a} + \sqrt {\frac {2 \ln (1 / \delta)}{n _ {a}}} \bigg) \leq \delta \quad \forall \delta \in (0, 1)
$$
derived from Hoeffding's inequality (following equation 7.1 of Lattimore & Szepesvári [34]), where  $\delta$  is the confidence level that quantifies the degree of certainty in the arm. In this work we let  $\delta = 1 / t$  where  $t$  is the number of rounds played, shrinking the confidence bound over rounds. Thus, we define the upper confidence bound for arm  $a$  at turn  $t$  as
$$
U C B _ {a, t} = \left\{ \begin{array}{l l} \infty , & \text {i f} n _ {a} = 0 \\ \hat {R} _ {a} + \sqrt {\frac {2 \ln t}{n _ {a}}}, & \text {o t h e r w i s e} \end{array} \right. \tag {3}
$$
Algorithmically, UCB1 takes the following steps at each round. First, the UCB1 policy plays the arm with largest upper confidence bound,  $a^{*} = \arg \max_{a\in \mathcal{A}}UCB_{a,t}$ . Next, a reward  $R_{a^*,t}$  is observed and the algorithm updates  $\hat{R}_{a^*}$  (the estimated mean reward for  $a^{*}$ ) and the upper confidence bounds for all  $a$ . Informally, this algorithm suggests that the learner should play arms more often if they either 1. have large expected reward,  $\hat{R}$ , or 2.  $n_a$  is small because the arm is not well explored.
B Pseudo-code
We include here pseudo-code for our 2 proposed algorithms. Algorithm 1 contains the pseudo-code for EXP3-FLAD, and Algorithm 2 contains the pseudo-code for UCB1-FLAD.
Algorithm 1 EXP3-FLAD
Require:  $\mathcal{D}_A,\mathcal{D}_T$  : Auxiliary and target datasets   
Require:  $f_{\theta}$  Parameterized model   
Require:  $G$  Gradient accumulation steps   
1: Initialize:  $K = |\mathcal{A}|$  .  $\mathcal{E}_0 = \frac{1}{K}$  ..  $\forall a\in \mathcal{A}:\nabla_a = 0,\hat{R}_a = 1$    
2: for  $t = 1,2,\ldots ,N$  do   
3:  $\mathcal{E}_t = \min \Bigl \{\frac{1}{K},\sqrt{\frac{\ln K}{K\cdot t}}\Bigr \}$    
4:  $\forall a\in \mathcal{A}:\pi (a)\gets (1 - KE_t)\frac{\exp(\mathcal{E}_{t - 1}\hat{R}_a)}{\sum_{a'}\exp(\mathcal{E}_{t - 1}R_{a'})} +\mathcal{E}_t$    
5: Sample  $a\sim \pi (\mathcal{A})$  and batch  $\{\mathbf{x},\mathbf{y}\} \sim \mathcal{D}_a$    
6:  $\nabla_{a}\leftarrow \nabla_{a} + \nabla_{\theta}\mathcal{L}(f_{\theta},\mathbf{x},\mathbf{y})$    
7: if  $t$  mod  $G)\equiv 0$  then   
8:  $\nabla_{\tau}\gets \nabla_{\theta}\mathcal{L}(f_{\theta},\mathcal{D}_{\tau})$    
9: Update model parameters w.r.t.  $\nabla_{\tau} + \sum_{a}\nabla_{a}$    
10: for all  $\{a\in \mathcal{A}|\nabla_a\neq 0\}$  do   
11:  $\hat{R}_a\gets \hat{R}_a + \frac{R_a,t}{\pi(a)}$    
12:  $\nabla_{a}\gets 0$    
13: end for   
14: end if   
15: end for
Algorithm 2 UCB1-FLAD
Require:  $\mathcal{D}_A,\mathcal{D}_{\mathcal{T}}$  : Auxiliary and target datasets   
Require:  $f_{\theta}$  Parameterized model   
Require:  $G$  Gradient accumulation steps   
Require:  $\beta$  Smoothing factor   
1: Initialize:  $\forall a\in \mathcal{A}:n_a = 1$ $\hat{R}_a = \cos (\nabla_\theta \mathcal{L}(f_\theta ,\mathcal{D}_\tau),\nabla_\theta \mathcal{L}(f_\theta ,\mathcal{D}_a))$    
2: for  $t = 1,2,\ldots ,N$  do   
3:  $a^{*} = \operatorname *{argmax}_{a\in \mathcal{A}}\hat{R}_{a} + \sqrt{\frac{2\ln t}{n_{a}}}$    
4: Sample batch  $\{\mathbf{x},\mathbf{y}\} \sim \mathcal{D}_a^*$    
5:  $\nabla_{a^*}\gets \nabla_{a^*} + \nabla_{\theta}\mathcal{L}(f_{\theta},\mathbf{x},\mathbf{y})$    
6:  $n_{a^*}\gets n_{a^*} + 1$    
7: if  $t$  (mod  $G)\equiv 0$  then   
8:  $\nabla \tau \leftarrow \nabla_{\theta}\mathcal{L}(f_{\theta},\mathcal{D}_{\mathcal{T}})$    
9: Update model parameters w.r.t.  $\nabla \tau +\sum_{a}\nabla_{a}$    
10: for all  $\{a\in \mathcal{A}|\nabla_a\neq 0\}$  do   
11:  $\hat{R}_a\gets (1 - \beta)\hat{R}_a + \beta R_{a,t}$    
12:  $\nabla_{a}\gets 0$    
13: end for   
14: end if   
15: end for
We train all models (FLAD and non-FLAD) on 40Gb A100s.
For all experiments, we use validation-based early stopping, and train for a maximum of 10,000 gradient update steps. In practice, we find that early-stopping leads to significantly fewer than 10,000 updates, usually between 50-150 for direct fine-tuning, and 1-2,000 for other methods.
For the smoothing factor,  $\beta$ , in UCB1-FLAD we ran preliminary experiments using values of  $\{0.99, 0.9, 0.75, 0.5\}$  and found 0.9 to work well across datasets. All reported scores use  $\beta = 0.9$ .
In preliminary experiments we consider rewards using gradients from multiple model partitions: the full model, encoder-only, decoder-only, and language modelling head (token classifier). We find that using the parameters from the LM head provides best performance, followed by the decoder-only, encoder-only, and full model gradients. The differential from best to worst method was  $\sim 3\%$  relative performance. Recall that with a gradient accumulation factor of  $G$ , our algorithms need to store at most  $G + 1$  gradients at any time. So not only does using the LM head provide performance improvements, but also saves memory. For the models we use, the LM head contains only  $2.3\%$  of the full model parameters.
D Full results
The full results of experiments on target-only fine-tuning, explore-only, exploit-only, EXP3-FLAD, and UCB1-FLAD are found on the next page.
E Probing the reward generating processes.
Figure 5: Reward distributions of  $\mathcal{R}^{GA}$  and  $\mathcal{R}^{GMS}$  prior to training and every 100 gradient updates thereafter. We probe the reward distributions using the T5-XL model with the T0Mix auxiliary dataset and WSC [52] as the target dataset.
F EXP3-FLAD and UCB1-FLAD training dynamics
The following 4 pages include a case study on the training dynamics of EXP3-FLAD and UCB1-FLAD when training T5-XL using T0Mix as the auxiliary data. First, we find datasets where EXP3-FLAD and UCB1-FLAD improve significantly over the baseline FLAD methods, but also where either EXP3-FLAD or UCB1-FLAD clearly outperforms the other. The two datasets that fulfill our interests are RTE and COPA.
We find that UCB1-FLAD outperforms EXP3-FLAD on RTE, and show their respective training dynamics in Figure 6 (UCB1) and Figure 7 (EXP3).
We find that EXP3-FLAD outperforms UCB1-FLAD on COPA, and show their respective training dynamics in Figure 8 (UCB1) and Figure 9 (EXP3).
We include details and takeaways in the caption for each figure. For EXP3-FLAD figures, we include charts of the cumulative estimated reward, empirical gradient alignment, instantaneous sampling distribution determined by the policy, and the empirical sampling distribution determined by the total number of samples seen per dataset as a fraction of the total samples seen. For UCB1-FLAD figures, we include charts of the upper confidence index, estimated gradient alignment, and the empirical sampling distribution.
Figure 6: Training dynamics of UCB1-FLAD, a case study using RTE as target dataset and T0Mix as auxiliary data, where UCB1-FLAD outperforms EXP3-FLAD. Colored lines are a sample of auxiliary datasets with interesting properties, the remaining datasets are shown in grey. We find that even though wiki_qa's estimated gradient alignment falls to below 0 (middle), UCB1 does not abandon sampling from it in the future, finding that between 3200 and 4800 batches, it becomes the dataset with largest upper confidence bound (top). Similarly, we see that UCB1 alternates between wiki_qa, amazon_polarity, and qasc as the datasets with higher gradient alignment and upper confidence bounds. kilt_tasks/hotpotqa has a very high gradient alignment prior to training, but UCB1 samples very infrequently from it, due to it's lower upper confidence bound. This is a failure case for transfer learning-based methods. Interestingly, UCB1 never estimates imdb to have a negative gradient, and gradually samples from it more and more frequently over the course of training.
Figure 7: Training dynamics of EXP3-FLAD, a case study using RTE as target dataset and T0Mix as auxiliary data, where UCB1-FLAD outperforms EXP3-FLAD. Colored lines are a sample of auxiliary datasets with interesting properties, the remaining datasets are shown in grey. We find that the gradient alignment signal is particularly noisy for EXP3-FLAD, possibly leading to it's slightly worse performance on RTE. All five highlighted auxiliary datasets have high instantaneous sampling probability, but over the course of training, the empirical sampling distribution is very condensed across the full set of auxiliary datasets, unlike UCB1 which is able to find better separation.
Figure 8: Training dynamics of UCB1-FLAD, a case study using COPA as target dataset and T0Mix as auxiliary data, where EXP3-FLAD outperforms UCB1-FLAD. Colored lines are a sample of auxiliary datasets with interesting properties, the remaining datasets are shown in grey. We find that although qasc and quartz start with very high gradient alignment, they very quickly fall to negative alignment (middle figure, green and yellow). In the end, we find that the algorithm samples much more from qasc than from quartz (bottom figure). Interestingly, we find that although both cnn_dailymail and multi_news start off with very negative gradient alignment, they quickly become the most aligned with the target task (middle figure, blue and red). We find that the three auxiliary datasets with highest upper confidence index (top figure) and largest sampling percent (bottom figure) are cnn_dailymail, multi_news, and trec even though these all considered dissimilar to the target prior to training.
Figure 9: Training dynamics of EXP3-FLAD, a case study using COPA as target dataset and T0Mix as auxiliary data, where EXP3-FLAD outperforms UCB1-FLAD. Colored lines are a sample of auxiliary datasets with interesting properties, the remaining datasets are shown in grey. This is an impressive example of the importance-weighted estimated reward. We see that cnn_dailymail and multi_news both start with very negative alignment, but EXP3 quickly updates it's estimated reward once their alignment becomes positive. Similar to RTE, we see that EXP3 never makes large separations in the empirical sampling distribution, possibly a reason why UCB1 outperforms EXP3 overall. Compared to RTE, we find that gradient alignments are much less variable, with a maximum alignment close to 0.5 and minimum alignment close to -0.5. Whereas in RTE, alignments regularly reach close to 1.0 and -1.0.
Explore-Only	Exploit-Only	EXP3-FLAD (RGA)	UCB1-FLAD (RGA)
|A|=35 (T0Mix)	570.9	549.1	769.1	700.0
|A|=260 (P3)	863.6	692.7	832.7	794.5
% increase	51.3%	26.2%	8.3%	13.5%
Table 3: Number of training iterations for T0-3B to converge using a training method (column) and a set of auxiliary datasets (row). The number of iterations to convergence is averaged across 11 target datasets and 5 seeds, leading to 55 experiments aggregated per cell.
G Effect of scaling  $|\mathcal{A}|$  on time-to-convergence
As we have described in this work, the computational complexity for a single turn of our methods are independent of the number of auxiliary datasets. However, it is unclear whether the computation complexity of the multi-armed bandits are dependent on the number of auxiliary datasets through their exploration rates. Thus, the computational complexity of an individual training run may be influenced by the number of auxiliary datasets  $(|\mathcal{A}|)$ , but it is not possible to characterize this relation explicitly as it relates to the complex and stochastic process of training and large language model.
To better understand the empirical effects of increasing  $|\mathcal{A}|$  on the time-to-model-convergence, we perform a study on the number of iterations to convergence for different FLAD algorithms. Table 3 shows that all methods require longer training to converge when increasing from  $|\mathcal{A}| = 35$  to 260. We find that, compared with baseline methods, our MAB-based methods require more steps for the smaller set of auxiliary datasets, but the number of additional steps required to train our methods only increases modestly ( $\sim 10\%$ ) when increasing  $|\mathcal{A}|$  by a factor of nearly 10. In contrast, the Explore-and Exploit-Only methods do not scale nearly as well when increasing the number of auxiliary datasets. Notably, the Explore-Only method requires over  $50\%$  more training iterations for P3 than for T0Mix, at which point it takes longer to converge than either of the MAB-based methods.
H Auxiliary Datasets
Here we include the full list of auxiliary datasets from P3 [23] used to train models for the ANLI target tasks. Other target datasets have slightly different auxiliary datasets due to test set decontamination, but are generally the same. Datasets are listed by their name as found in HuggingFace Datasets<sup>2</sup>.
Zaid/quac Expanded, acronym identification, ade(corpus_v2/Ade Corpus_v2_classification, ade Corpus_v2/Ade Corpus_v2_drug_adre Relation, ade Corpus_v2/Ade Corpus_v2_drug_dosage Relation, adversarial_qa/adversarialQA, adversarial_qa/dbert, adversarial_qa/dbidaf, adversarial_qa/droberta, aeslc, ag_news, ai2_ar/ARC-Challenge, ai2_ar/ARC-Easy, amazon_polarity, amazonreviews-multi/en, amazon_usreviews/Wireless_v1_00, ambig_qa/light, appreviews, aqua_rat/raw, art, asset/ratings, asset/simplification, banking77, billsum, bing-coronavirus_query_set, biosses, blbooksgenre/title_genre_classfication, blended_skill_talk, cbt/CN, cbt/NE, cbt/P, cbt/V, cbt/raw, cc_news, circa, climate_fever, cnn_dailymail/3.0.0, codah/codah, codah/fold_0, codah/fold_1, codah/fold_2, codah/fold_3, codah/fold_4, code_x/glue_tc_text_to_code, common_gen, commonsense_qa, conv.ai, conv.ai_2, conv.ai_3, cord19/metadata, cos_e/v1.0, cos_e/v1.11, cosmos_qa,ovid_qa_castorini, craffel/openai_lambada, craigslist bargains, crows_pairs, dbpedia_14, discofuse/discofuse-sport, discofuse/discofuse-wikipedia, discovery/discovery, docred, dream, drop, duorc/ParaphraseRC, duorc/SelfRC, e2e_nlg cleansed, ecthr Cases/alleged-violation-prediction, emo, emotion, enriched_web_nlg/en, esnli, evidence_infer_treatment/1.1, evidence_infer_treatment/2.0, fever/v1.0, fever/v2.0, financialPhrasebank/sentences_allagree, freebase_qa generatedreviews_enth, gigaword, glue/ax, glue/cola, glue/mnli, glue/mnli_MATCHed, glue/mnli_mismatched, glue/mrpc, glue/qnli, glue/qqp, glue/rte, glue/sst2, glue/stsb, glue/wnli, google_wellformed_query, great_code, guardian_authorship/cross_genre_1, guardian_authorship/cross-topic_1, guardian_authorship/cross-topic_4, guardian_authorship/cross-topic_7, gutenberg_time_hans, hate_speech18, head_qa/en, health_fact,hlgd,hotpot_qa/distractor, hotpot_qa/fullwiki
$^{2}$ https://huggingface.co/datasets
humicroedit/subtask-1, humicroedit/subtask-2, hyperpartisan_news_detection/byarticle, hyperpartisan_news_detection/bypublisher, imdb, jfleg, kelm, kilt_tasks/hotpotqa, kilt_tasks/nq, lama/trex, lambada, liar, limit, math_dataset/algebra_LINEAR_1d, math_dataset/algebra_LINEAR_1d_composed, math_dataset/algebra_LINEAR_2d, math_dataset/algebra_LINEAR_2d_composed, math_qa, mc_taco, mdd/task1_qa, mdd/task2_recs, mdd/task3_qarecs, medal, medical Questions_pairs, meta_woz/dialogues, mocha, movie_rationales, multi_news, multi_nli, multi_x_science_sum, mwsc, narrativeqa, ncbi_disease, neural_code_search/evaluation_dataset, newpop, nlu.evaluation_data, nq_open, numer Sense, onestop_english, openai_humaneval, openbookqa/additional, openbookqa/main, paws-x/en, paws/ labeled_final, paws/ labeled_swap, paws/unlabeled_final, piqa, poem_sentiment, pubmed_qa/pqa_labeled, qa_srl, qa_zre, qasc, qed, quac, quail, quarel, quartz, quora, quoref, race/all, race/high, race/middle, riddleSense, ropes, rotten_tomatoes, samsum, scan/addprim_jump, scan/addprim_turn_left, scan/filler_num0, scan/filler_num1, scan/filler_num2, scan/filler_num3, scan/length, scan/simple, scan/template_around_right, scan/template_jump_around_right, scan/template_oposite_right, scan/template_right, scicite, scientific_papers/arxiv, scientific_papers/pubmed, sciq, scitail/snli_format, scitail/tsv_format, scitldr/Abstract, selqa/answer_selection_analysis, sem_eval_2010_task_8, sem_eval_2014_task_1, sent_comp, sick, sms_spam, snips_built_in_intents, snli, social_i_qa, species_800, squad, squad_adversarial/AddSent,squad_v2,squadshifts/amazon,squadshifts/new_wiki,squadshifts/nyt sst/default,stsb-multi_mt/en, subjQA/books, subjQA/electronics, subjQA/grocery, subjQA/movies, subjQA/restaurants, subjQA/tripadvisoror,super_clue/axb,super_clue/axg,super_clue/boolq,super_clue/multirc,super_clue/record,swag/regular,tabFact/tabFact,tmu_gfm_dataset,trec,trivia_qa/unfiltered,turk,tweet_eval/emoji,tweet_eval/emotion,tweet_eval/hate,tweet_eval/irony,tweet_eval/offensive,tweet_eval/sentiment,tweet_eval/stance_abortion,tweet_eval/stance atheism,tweet_eval/stance_climate,tweet_eval/stance_feminist,tweet_eval/stance_hillary,tydiaq/primary_task,tydiaq/secondary_task web这些问题,wiki BIO,wikiHop/masked,wiki_hop/original,wiki_qa,wiki_split,wino.bias/type1 Anti,wino.bias/type1_pro,wino.bias/type2 anti,wino.bias/type2 pro,winograd_wsc/wsc273,winograd_wsc/wsc285,wiqa,xnli/en,xquad/xquad_en,xquad_r/en,xsum,yahooAnswers_qa,yahoo_answers_topics,yelp_polarity,yelp_review_full,zest