YATO : Yet Another deep learning based Text analysis Open toolkit (2023)

Abstract: We introduce YATO, an open-source, easy-to-use toolkit for text analysis with deep learning. Different from existing heavily engineered toolkits and platforms, YATO is lightweight and user-friendly for researchers from cross-disciplinary areas. Designed in a hierarchical structure, YATO supports free combinations of three types of widely used features including 1) traditional neural networks (CNN, RNN, etc.); 2) pre-trained language models (BERT, RoBERTa, ELECTRA, etc.); and 3) user-customized neural features via a simple configurable file. Benefiting from the advantages of flexibility and ease of use, YATO can facilitate fast reproduction and refinement of state-of-the-art NLP models, and promote the cross-disciplinary applications of NLP techniques. The code, examples, and documentation are publicly available at https://github.com/jiesutd/YATO. A demo video is also available at https://www.youtube.com/playlist?list=PLJ0mhzMcRuDUlTkzBfAftOqiJRxYTTjXH.

## introduction large language models (llms) such as gpt-3 @xcite , @xcite , and llama @xcite have gained significant progress in natural language processing (nlp), showing strong abilities to understand text and competitive performance across various nlp tasks. however, these models are either closesource or difficult to fine-tune due to the high computational costs, which makes them inconvenient for academic research or practical implementation. alternatively, traditional neural models, such as recurrent neural networks @xcite , convolutional neural networks @xcite , and pre-trained language models (plms, @xcite @xcite have been widely studied and utilized for text understanding. these models benefit from large-scale training data and can be quickly fine-tuned toward specific usages. recent works also show they can offer useful guidance to llms @xcite . therefore, small open-source deep learning models are important in current nlp systems, especially in computation and data resource-limited scenarios. however, due to the complexity of the deep learning model architecture, it is challenging to implement methods or reproduce results from the literature. the different implementations of these models can lead to unfair comparisons or misleading results. most existing frameworks were designed for professional developers, which brings additional obstacles for less experienced users, especially for researchers with less or no artificial intelligence (ai) background @xcite @xcite . in addition, these frameworks seldom support user-defined features required for various domain applications (e.g., in medical named entity recognition, customized lexicons can be supplemented as external features, such that additional labels are tagged as features when a word occurs in the lexicon). for non-expert, cross-domain users, customizing models via source code with additional features is complex. to promote interdisciplinary applications of cutting-edge nlp techniques, it is necessary to build a flexible, user-friendly, and effective text representation framework that supports a wide range of deep learning architectures and customized domain features. there exist several text analysis toolkits in the nlp community. corenlp @xcite and spacy @xcite offer pipelines for many traditional nlp tasks, while the performance is sometimes under-optimal due to the use of less powerful models. allennlp @xcite and flairnlp @xcite user-defined features. fairseq @xcite is designed for sequence-to-sequence tasks like machine translation and document summarization. transformers @xcite ) offers implementation for various tasks by using state-of-the-art models across different modalities, while it is heavily engineered. paddlenlp @xcite and easynlp @xcite are specifically designed for industrial application and commercial usage, which are not lightweight for research purposes. the above toolkits are mostly developed for professional ai researchers or engineers, where heavy coding effort is necessary during model development and deployment. the learning curve is steep to fully leverage these toolkits for crossdisciplinary researchers (e.g., medical, financial) who need to build models with lightweight code. this paper presents a toolkit, yato (yet another deep learning based text analysis open toolkit), for researchers looking for a convenient way of building state-of-the-art models for two most popular types of nlp tasks: sequence labeling (e.g., part-of-speech tagging, named entity recognition) and sequence classification (e.g., sentiment analysis, document classification). yato is built on ncrf++ @xcite , a popular neural sequence labeling toolkit with over 250+ citations from research papers, 1,900+ stars and 120+ merged pull requests on github as of oct. 2023. ncrf++ has been utilized in many crossdisciplinary research projects, including medical @xcite and finance @xcite . yato retains its strengths, integrates advanced pretrained language models, and adds capabilities for sequence classification and data visualization. ## the overall workflow and usage the overall workflow of yato comprises three primary components to facilitate solving downstream tasks: configuration preparation, model training, and model decoding. users can leverage this modular pipeline to tackle a wide range of applications. ## conclusion yato is an open-source toolkit for text analysis that supports various combinations of state-ofthe-art deep learning models and user-customed features, with high flexibility and minimum coding effort. yato is maintained by core developers from ylab ( https://ylab.top/ ). it aims to help ai researchers build state-of-the-art nlp models and assist non-ai researchers in conducting cross-disciplinary research with advanced nlp techniques. given the success of its predecessor, ncrf++, we believe that yato will greatly promote the applications of nlp in various crossdisciplinary fields and reduce disparities of ai application in these areas. in the future, we plan to integrate advanced llms and customize modules that support modeling time series, multimodal features, and specific features for various domains.