SlowGuess/NAACL_2022_OCR huggingface dataset:
Category	Task	Rouge-L	F1-Score
Single Task	V-BB	I-BB	Single Task	V-BB	I-BB
NER	AnatEM	84.88	32.30	83.93	85.55	33.50	84.61
BC2GM	77.66	50.87	74.10	78.56	50.86	75.03
BC4CHEMD	88.85	71.05	86.50	89.06	71.44	86.97
BC5CDR	74.83	69.81	74.76	75.13	70.11	75.24
BioNLP11EPI	84.64	50.10	87.60	84.95	52.85	88.04
BioNLP11ID	71.08	59.12	72.64	71.64	60.15	73.39
BioNLP13CG	64.19	55.18	67.72	61.68	53.88	65.09
BioNLP13GE	83.74	49.30	86.71	84.08	51.78	87.39
BioNLP13PC	70.42	53.06	72.46	66.89	51.61	67.77
BioNLP09	85.16	51.54	88.09	85.54	54.31	88.48
CRAFT	63.72	51.85	64.10	63.92	52.31	64.30
Ex-PTM	82.32	49.61	83.73	82.38	52.07	84.49
JNLPBA	71.65	69.37	71.54	70.79	68.60	70.26
NCBI	89.51	74.46	86.11	89.81	75.55	80.91
linnaeus	94.43	44.99	93.46	93.21	44.59	93.77
Average	79.14	55.51	79.54	78.88	56.24	79.45
DI	DI 2006	12.60	46.38	50.82	10.60	43.28	47.45
POS	Genia	71.45	27.94	71.26	70.48	27.50	71.99
QA	BioASQ8b (factoid)	52.95	51.14	47.28	54.67	53.52	49.51
BioASQ8b (list)	38.96	19.87	36.11	-	17.74	35.59
BioASQ8b (yesno)	61.74	62.61	68.25	63.48	62.61	68.25
PubMedQA	27.12	25.48	24.49	31.44	30.74	29.58
Average	45.19	39.78	44.03	46.39	41.15	45.73
RE	ChemProt	76.08	76.00	81.61	63.89	52.17	63.22
DDI	91.78	82.97	89.35	94.10	82.97	89.35
Average	83.04	79.48	85.48	79.00	67.57	76.28
SA	Medical Drugs	47.51	46.39	47.37	47.51	46.39	47.37
SR	Accelerometer	74.65	72.54	81.25	74.65	72.54	81.25
Acromegaly	80.21	81.77	80.71	80.21	81.77	80.71
COVID	74.81	76.30	77.28	74.81	76.30	77.28
Cooking	71.71	82.93	83.25	71.71	82.93	83.25
HRT	75.68	77.17	82.70	75.68	77.17	82.70
Average	75.41	78.14	81.04	75.41	78.14	81.04
DC	HoC	88.53	49.64	82.53	88.53	49.51	82.53
RFI	RFHD 2014 (yesno)	57.21	64.97	69.17	57.21	64.97	69.17
RFHD 2014 (time-riskfactor)	66.18	0.97	85.24	66.18	0.97	85.28
Average	72.87	57.30	77.21	61.69	32.97	77.22
Average	-	70.51	55.55	73.49	70.15	55.21	73.01
Table 2: Results comparison between single-task baseline, Vanilla-BoXBART and In-BoXBART in terms of Rouge-L and  $F_{1}$ -Score. All the results for  $F_{1}$ -Score are presented in %. V-BB: Vanilla-BoXBART, I-BB: In-BoXBART, RFHD: Risk Factor for Heart Disease.
$F_{1}$ -Score, respectively, exhibiting the same performance behaviour as Rouge-L. Hence, we use Rouge-L for further comparisons. From the result, we can observe that Vanilla-BoXBART reduces the complexity compared to the single-task model (i.e., 110 million parameters vs. 32x110 million pa
rameters), however, on an average the performance drops by  $14.96\%$  in terms of Rouge-L, and compared to single-task models. This indicates that multi-task learning in the biomedical domain is more difficult than general domain NLP since many previous works have shown that the multi-task
Figure 5: Comparison of on an average Rouge-L across all instruction tasks between single-task and In-BoXBART based on the average number of training instances per task.
model outperforms the single-task model (Lourie et al., 2021; McCann et al., 2018). On the other hand, In-BoXBART, which has the same complexity as Vanilla-BoXBART, significantly outperforms Vanilla-BoXBART by on average  $17.94\%$ , and also outperforms the single-task model by a  $2.98\%$  margin, precisely. This indicates the benefit of using instructions to achieve the MTL in the biomedical domain.
Effect of instruction in few-shot learning We have compared the average Rouge-L of In-BoxXBART with a single-task baseline for few-shot setting. Figure 5 shows the relative performance of In-BoxXBART compared to single-task baseline. We have shown results for all few-shot learning experiments in Appendix D. From the results, we see that In-BoxXBART achieves on an average  $60.64\%$  Rouge-L and the single-task model achieves  $37.31\%$  for 32 instances per task. In-BoxXBART significantly outperforms the single-task baseline by  $23.33\%$ . From Figure 5, we can see that In-BoxXBART consistently perform better compared to the baseline. As we know, obtaining a large annotated dataset in the biomedical domain is difficult, time-consuming and costly. From few-shot learning, we can see that instructions are beneficial in achieving high performance compared to task-specific models.
For which tasks, instruction is helpful? From Table 2, we can see that In-BoXBART outperforms baselines for 5 categories, i.e., NER, deidentification, RE, SR and risk factor identification. From this, we can see that instructions are
more helpful in these five categories. However, In-BoXBART achieves performance lower or par with the single-task baseline for the tasks from QA, POS tagging, sentiment analysis and document classification which indicates room for improvement in this direction.
Which are harder tasks to solve using instructions? Although instructions help in achieving better performance for some tasks compared to the single-task model, the overall performance is still lower. For example, instruction improves performance for de-identification, but overall performance on this task is only  $50.82\%$  which can be improved. A similar pattern we can see for BioNLP12CG and CRAFT from NER; BioASQ-8b (factoid, list) and PubmedQA from QA; and Medical Drug from the sentiment analysis category. In general, we can observe that tasks that include either multi-class scenario or answer generation from the context are most likely to be harder to solve using instructions. For example, CRAFT and BioNLP13CG have 6 entity types which are higher than any other tasks from NER, and we can see that the performance for these two tasks is lower compared to other tasks of NER.
For which tasks, instruction is the most beneficial in few shot setting? From the results shown in Appendix D, tasks from the NER, de-identification, QA, sentiment analysis and risk factor identification shows on average larger improvement compared to baselines for the few-shot settings (i.e., 32 and 100 instances per task). This indicates that instructions are beneficial for the tasks from the above categories.
Can we design better instructions? Since instruction teach the model how to solve a given task, domain specific information rich instructions can improve model performance. One potential way is to use the knowledge of domain experts. However, designing a good biomedical instruction can be one research direction.
How to handle long-context input? Training instances of many biomedical datasets consist Electronic Health Records (EHRs) or discharge summaries of patients. Because of this, these instances are long and exceed the maximum input length of LMs such as BERT, BART. In this scenario, encoding extra information in terms of prompts or
instructions becomes difficult. One potential solution is to use Longformer (Beltagy et al., 2020), and another solution is to use T5 kind of models which use relative position embeddings so that the inference length can be longer (Luo et al., 2022a).
How to handle multi-class classification tasks?
Multiple classes cause an issue while creating biomedical instructions because we can not present one example per class. If we do that, the encoding of BI and input will exceed the maximum length of LMs. A naive solution is to select examples of a few labels or remove the examples. However, this will cause a label bias issue or performance degradation. Potential future research direction can be designing a methodology to handle multi-class classification tasks.
How far we are from the SOTA? We have presented preliminary comparison of our results w.r.t. state-of-the-art (SOTA) single-task systems for 21 instruction tasks from the BoX as shown in Appendix E. Form the results, we can see that the performance of the proposed model remains far from the SOTA for some tasks, indicating significant room for further research in this domain.
This research shows the impact of instructions in MTL for the first time in the biomedical domain. To this extent, we introduced the BoX, a first benchmark dataset consisting of 32 instruction tasks across various biomedical NLP domains. Using this meta-dataset, we proposed a unified model, i.e., In-BoXBART which outperforms single-task baseline and Vanilla-BoxBART by  $\sim 3\%$  and  $\sim 18\%$ , respectively. Our proposed approach also shows an effective performance for a few-shot setting which is more beneficial in the biomedical domain where obtaining large annotated datasets is difficult. We hope that the BoX benchmark, In-BoXBART, and experimental results encourage future research into more unified models for biomedical NLP.
Authors acknowledge support from ONR award number N00014-20-1-2332 for this project and thank the anonymous reviewers for feedback.
Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. 2021. Muppet: Massive multi-task representations with pre-finetuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5799-5811.
Sultan Alrowili and Vijay Shanker. 2021. BioM-transformers: Building large biomedical language models with BERT, ALBERT and ELECTRA. In Proceedings of the 20th Workshop on Biomedical Language Processing, pages 221-227, Online. Association for Computational Linguistics.
Simon Baker, Ilona Silins, Yufan Guo, Imran Ali, Johan Hogberg, Ulla Stenius, and Anna Korhonen. 2016. Automatic semantic classification of scientific literature according to the hallmarks of cancer. Bioinformatics, 32(3):432-440.
Pratyay Banerjee, Kuntal Kumar Pal, Murthy Devarakonda, and Chitta Baral. 2021. Biomedical named entity recognition via knowledge guidance and question answering. ACM Transactions on Computing for Healthcare, 2(4):1-24.
Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
Gamal Crichton, Sampo Pyysalo, Billy Chiu, and Anna Korhonen. 2017. A neural network multi-task learning approach to biomedical named entity recognition. BMC bioinformatics, 18(1):1-14.
Avia Efrat and Omer Levy. 2020. The turking test: Can language models understand instructions? arXiv preprint arXiv:2010.11982.
Peter Hase and Mohit Bansal. 2021. When can models learn from explanations? a formal framework for understanding the roles of explanation data. arXiv preprint arXiv:2102.02201.
María Herrero-Zazo, Isabel Segura-Bedmar, Paloma Martínez, and Thierry Declerck. 2013. The ddi corpus: An annotated corpus with pharmacological substances and drug-drug interactions. Journal of biomedical informatics, 46(5):914-920.
Kexin Huang, Abhishek Singh, Sitong Chen, Edward Moseley, Chih-Ying Deng, Naomi George, and Charolotta Lindvall. 2020. Clinical XLNet: Modeling sequential clinical notes and predicting prolonged mechanical ventilation. In Proceedings of the 3rd Clinical Natural Language Processing Workshop, pages 94-100, Online. Association for Computational Linguistics.
Rezarta Islamaj Dogan, Sun Kim, Andrew Chatri-Aryamontri, Chih-Hsuan Wei, Donald C Comeau, Rui Antunes, Sérgio Matos, Qingyu Chen, Aparna Elangovan, Nagesh C Panyam, et al. 2019. Overview
of the biocreative vi precision medicine track: mining protein interactions and mutations for precision medicine. Database, 2019.
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. PubMedQA: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2567-2577, Hong Kong, China. Association for Computational Linguistics.
Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. 2020. UNIFIEDQA: Crossing format boundaries with a single QA system. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1896-1907, Online. Association for Computational Linguistics.
Zeljko Kraljevic, Anthony Shek, Daniel Bean, Rebecca Bendayan, James Teo, and Richard Dobson. 2021. MedGPT: Medical Concept Prediction from Clinical Narratives. arXiv preprint arXiv:2107.03134.
Vishesh Kumar, Amber Stubbs, Stanley Shaw, and Ozlem Uzuner. 2015. Creation of a new longitudinal corpus of clinical narratives. Journal of biomedical informatics, 58:S6-S10.
Kirby Kuznia, Swaroop Mishra, Mihir Parmar, and Chitta Baral. 2022. Less is more: Summary of long instructions is better for program synthesis. arXiv preprint arXiv:2203.08597.
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234-1240.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880, Online. Association for Computational Linguistics.
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, et al. 2021. Few-shot learning with multilingual language models. arXiv preprint arXiv:2112.10668.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pretrain, prompt, and predict: A systematic survey of
prompting methods in natural language processing. arXiv preprint arXiv:2107.13586.
Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13480-13488.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786.
Man Luo, Kazuma Hashimoto, Semih Yavuz, Zhiwei Liu, Chitta Baral, and Yingbo Zhou. 2022a. Choose your qa model wisely: A systematic study of generative and extractive readers for question answering. arXiv preprint arXiv:2203.07522.
Man Luo, Arindam Mitra, Tejas Gokhale, and Chitta Baral. 2022b. Improving biomedical information retrieval with neural retrievers. arXiv preprint arXiv:2201.07745.
Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018. The natural language decathlon: Multitask learning as question answering. arXiv preprint arXiv:1806.08730.
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2021. Metaicl: Learning to learn in context. arXiv preprint arXiv:2110.15943.
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi. 2021a. Reframing instructional prompts to gptk's language. ACL Findings.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021b. Cross-task generalization via natural language crowdsourcing instructions. ACL.
Swaroop Mishra, Arindam Mitra, Neeraj Varshney, Bhavdeep Sachdeva, Peter Clark, Chitta Baral, and Ashwin Kalyan. 2022. NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks. ACL.
Anastasios Nentidis, Anastasia Krithara, Konstantinos Bougiatiotis, and Georgios Paliouras. 2020. Overview of bioasq 8a and 8b: Results of the eighth edition of the bioasq tasks a and b. In CLEF (Working Notes).
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Preprint.
Mihir Parmar. 2021. Automation of Title and Abstract Screening for Clinical Systematic Reviews. Master's thesis, Arizona State University, Tempe, AZ, USA.
Yifan Peng, Shankai Yan, and Zhiyong Lu. 2019. Transfer learning in biomedical natural language processing: An evaluation of BERT and ELMo on ten benchmarking datasets. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 58-65, Florence, Italy. Association for Computational Linguistics.
Long N Phan, James T Anibal, Hieu Tran, Shaurya Chanana, Erol Bahadroglu, Alec Peltekian, and Grégoire Altan-Bonnet. 2021. Scifive: a text-to-text transformer model for biomedical literature. arXiv preprint arXiv:2106.03598.
N Poolsawad, C Kambhampati, and JGF Cleland. 2014. Balancing class for performance of classification with a clinical dataset. In proceedings of the World Congress on Engineering, volume 1, pages 1-6.
Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. 2022. Grips: Gradient-free, edit-based instruction search for prompting large language models. arXiv preprint arXiv:2203.07281.
Ravsehaj Singh Puri, Swaroop Mishra, Mihir Parmar, and Chitta Baral. 2022. How many data samples is an additional instruction worth? arXiv preprint arXiv:2203.09161.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2021. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207.
Timo Schick and Hinrich Schütze. 2021. It's not just size that matters: Small language models are also few-shot learners. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2339-2352, Online. Association for Computational Linguistics.
Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni. 2020. Green ai. Communications of the ACM, 63(12):54-63.
Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3645-3650, Florence, Italy. Association for Computational Linguistics.
Amber Stubbs, Christopher Kotfila, Hua Xu, and Ozlem Uzuner. 2015. Identifying risk factors for heart disease over time: Overview of 2014 i2b2/uthealth shared task track 2. Journal of biomedical informatics, 58:S67-S77.
Yuka Tateisi, Akane Yakushiji, Tomoko Ohta, and Jun'ichi Tsujii. 2005. Syntax annotation for the genia corpus. In Companion Volume to the Proceedings of Conference including Posters/Demos and tutorial abstracts.
Ozlem Uzuner, Yuan Luo, and Peter Szolovits. 2007. Evaluating the state-of-the-art in automatic de-identification. Journal of the American Medical Informatics Association, 14(5):550-563.
Benyou Wang, Qianqian Xie, Jiahuan Pei, Prayag Tiwari, Zhao Li, et al. 2021a. Pre-trained language models in biomedical domain: A systematic survey. arXiv preprint arXiv:2110.05006.
Liwen Wang, Rumei Li, Yang Yan, Yuanmeng Yan, Sirui Wang, Wei Wu, and Weiran Xu. 2022. Instructioner: A multi-task instruction-based generative framework for few-shot ner. arXiv preprint arXiv:2203.03903.
Sinong Wang, Han Fang, Madian Khabsa, Hanzi Mao, and Hao Ma. 2021b. Entailment as few-shot learner. arXiv preprint arXiv:2104.14690.
Albert Webson and Ellie Pavlick. 2021. Do prompt-based models really understand the meaning of their prompts? arXiv preprint arXiv:2109.01247.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652.
Orion Weller, Nicholas Lourie, Matt Gardner, and Matthew E. Peters. 2020. Learning from task descriptions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1361-1375, Online. Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System
Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, and Carrie J Cai. 2022. Promptchainer: Chaining large language model prompts through visual programming. arXiv preprint arXiv:2203.06566.
Tongshuang Wu, Michael Terry, and Carrie J Cai. 2021. Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. arXiv preprint arXiv:2110.01691.
Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I Wang, et al. 2022. Unifiedskg: Unifying and multi-tasking structured knowledge grounding with text-to-text language models. arXiv preprint arXiv:2201.05966.
Qinyuan Ye and Xiang Ren. 2021a. Learning to generate task-specific adapters from task description. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 646-653, Online. Association for Computational Linguistics.
Qinyuan Ye and Xiang Ren. 2021b. Zero-shot learning by generating task-specific adapters. arXiv e-prints, pages arXiv-2101.
Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Kai-Wei Chang. 2021. Ethical-advice taker: Do language models understand natural language interventions? In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4158-4164, Online. Association for Computational Linguistics.
Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein. 2021. Adapting language models for zero-shot learning by meta-tuning on dataset and prompt collections. In *Findings of the Association for Computational Linguistics: EMNLP* 2021, pages 2856-2878, Punta Cana, Dominican Republic. Association for Computational Linguistics.
A Statistics of Instruction Tasks
This section provides all the statistics of training, validation and inference data used for experiments in Table 3. All the number of instances provided in Table 3 are calculated after discarding the instances with more than 1024 token length as described in the section 5.1. We have divided the dataset into standard 70/10/20 splits for train/validation/test if there is no separate validation and testing set provided in the dataset.
B Instruction Tasks and Examples
To build all the models (baselines, proposed model and few-shot learning), we adapt the unified format for all the tasks of BoX. We converted all the tasks into the text-to-text format, including the classification tasks. Table 4 shows an example of input and output from each category. Moreover, we have also re-purposed some biomedical datasets to create more than one task as described in the section 3.1.
C Systematic Review Datasets