Package: paddlenlp
简体中文🀄|English🌎Features|Installation|Quick Start|API Reference|CommunityPaddleNLPis a NLP library that is botheasy to useandpowerful. It aggregates high-quality pretrained models in the industry and provides aplug-and-playdevelopment experience, covering a model library for various NLP scenarios. With practical examples from industry practices, PaddleNLP can meet the needs of developers who requireflexible customization.News 📢2023.6.12:Release of PaddleNLP v2.6rc🔨 LLM Tools：Introduces comprehensive examples of open-source LLM training and inference, includingBloom,ChatGLM,GLM,LlamaandOPT. Added Tensor Parallel capability toTrainer APIfor distributed LLM trainin. Also releasedParameter-Efficient Finetuning,which enables training LLMs on consumer hardware.2023.1.12:Release of PaddleNLP v2.5🔨 NLP Tools:PPDiffusers, our cross-modal diffusion model toolbox based on PaddlePaddle, has been released! It provides a complete training process for diffusion models, and supports FastDeploy inference acceleration and multi-hardware deployment (supports Ascend chips and Kunlun core deployment).💎 Industrial Applications: Information extraction, text classification, sentiment analysis, and intelligent question answering have all been newly upgraded. New releases include document information extractionUIE-X, unified text classificationUTC, unified sentiment analysisUIE-Senta, andunsupervised QA application. At the same time, theERNIE 3.0 Tiny v2series of pretrained small models have been released, which are more effective with low-resource and foreign data. They provide open-source end-to-end deployment solutions such as model pruning, model quantization, FastDeploy inference acceleration, and edge-side deployment to reduce the difficulty of pretrained model deployment.💪 Framework Upgrade: Pretrained modelparameter configuration unification, saving and loading custom parameter configurations no longer requires additional development;Trainer APIhas added BF16 training, recompute recalculations, sharding, and other distributed capabilities. Large-scale pre-training model training can easily be accomplished through simple configuration.Model Compression APIsupports quantization training, vocabulary compression, and other functions. The compressed model has smaller accuracy loss, and the memory consumption of model deployment is greatly reduced.Data Augmentation APIhas been comprehensively upgraded to support three granularities of data augmentation strategy: character, word, and sentence, making it easy to customize data augmentation strategies.🤝 Community: 🤗Huggingface hub officially supports PaddleNLP pretrained models, supporting PaddleNLP Model and Tokenizer downloads and uploads directly from the 🤗Huggingface hub. Everyone is welcome to try out PaddleNLP pretrained models on the 🤗Huggingface hubhere.September 6, 2022:Release of PaddleNLP v2.4🔨 NLP Tools:NLP Pipeline System Pipelineshas been released, supporting the rapid construction of search engines and question-answering systems, and can be extended to support various NLP systems, making it easy, flexible, and efficient to solve NLP tasks like building blocks!💎 Industrial Applications: A newtext classification full-process application solutionhas been added, covering various scenarios such as multi-classification, multi-label, and hierarchical classification, supporting small-sample learning and TrustAI trustworthy computing model training and tuning.🍭 AIGC: The SOTA modelCodeGenfor code generation in various programming languages has been added.💪 Framework Upgrade:Automatic Model Compression APIhas been released, which automatically cuts and quantizes models, greatly reducing the threshold for using model compression technology.Few-shot Promptcapability has been released, integrating classic algorithms such as PET, P-Tuning, and RGL.Features📦 Out-of-Box NLP Toolset🤗 Awesome Chinese Model Zoo🎛️ Industrial End-to-end System🚀 High Performance Distributed Training and InferenceOut-of-Box NLP ToolsetTaskflow aims to provide off-the-shelf NLP pre-built task covering NLU and NLG technique, in the meanwhile with extremely fast inference satisfying industrial scenario.For more usage please refer toTaskflow Docs.Awesome Chinese Model Zoo🀄 Comprehensive Chinese Transformer ModelsWe provide45+network architectures and over500+pretrained models. Not only includes all the SOTA model like ERNIE, PLATO and SKEP released by Baidu, but also integrates most of the high-quality Chinese pretrained model developed by other organizations. UseAutoModelAPI to⚡SUPER FAST⚡download pretrained models of different architecture. We welcome all developers to contribute your Transformer models to PaddleNLP!frompaddlenlp.transformersimport*ernie=AutoModel.from_pretrained('ernie-3.0-medium-zh')bert=AutoModel.from_pretrained('bert-wwm-chinese')albert=AutoModel.from_pretrained('albert-chinese-tiny')roberta=AutoModel.from_pretrained('roberta-wwm-ext')electra=AutoModel.from_pretrained('chinese-electra-small')gpt=AutoModelForPretraining.from_pretrained('gpt-cpm-large-cn')Due to the computation limitation, you can use the ERNIE-Tiny light models to accelerate the deployment of pretrained models.# 6L768Hernie=AutoModel.from_pretrained('ernie-3.0-medium-zh')# 6L384Hernie=AutoModel.from_pretrained('ernie-3.0-mini-zh')# 4L384Hernie=AutoModel.from_pretrained('ernie-3.0-micro-zh')# 4L312Hernie=AutoModel.from_pretrained('ernie-3.0-nano-zh')Unified API experience for NLP task like semantic representation, text classification, sentence matching, sequence labeling, question answering, etc.importpaddlefrompaddlenlp.transformersimport*tokenizer=AutoTokenizer.from_pretrained('ernie-3.0-medium-zh')text=tokenizer('natural language processing')# Semantic Representationmodel=AutoModel.from_pretrained('ernie-3.0-medium-zh')sequence_output,pooled_output=model(input_ids=paddle.to_tensor([text['input_ids']]))# Text Classificaiton and Matchingmodel=AutoModelForSequenceClassification.from_pretrained('ernie-3.0-medium-zh')# Sequence Labelingmodel=AutoModelForTokenClassification.from_pretrained('ernie-3.0-medium-zh')# Question Answeringmodel=AutoModelForQuestionAnswering.from_pretrained('ernie-3.0-medium-zh')Wide-range NLP Task SupportPaddleNLP provides rich examples covering mainstream NLP task to help developers accelerate problem solving. You can find our powerful transformerModel Zoo, and wide-range NLP applicationexampleswith detailed instructions.Also you can run our interactiveNotebook tutorialon AI Studio, a powerful platform withFREEcomputing resource.PaddleNLP Transformer model summary (click to show details)ModelSequence ClassificationToken ClassificationQuestion AnsweringText GenerationMultiple ChoiceALBERT✅✅✅❌✅BART✅✅✅✅❌BERT✅✅✅❌✅BigBird✅✅✅❌✅BlenderBot❌❌❌✅❌ChineseBERT✅✅✅❌❌ConvBERT✅✅✅❌✅CTRL✅❌❌❌❌DistilBERT✅✅✅❌❌ELECTRA✅✅✅❌✅ERNIE✅✅✅❌✅ERNIE-CTM❌✅❌❌❌ERNIE-Doc✅✅✅❌❌ERNIE-GEN❌❌❌✅❌ERNIE-Gram✅✅✅❌❌ERNIE-M✅✅✅❌❌FNet✅✅✅❌✅Funnel-Transformer✅✅✅❌❌GPT✅✅❌✅❌LayoutLM✅✅❌❌❌LayoutLMv2❌✅❌❌❌LayoutXLM❌✅❌❌❌LUKE❌✅✅❌❌mBART✅❌✅❌✅MegatronBERT✅✅✅❌✅MobileBERT✅❌✅❌❌MPNet✅✅✅❌✅NEZHA✅✅✅❌✅PP-MiniLM✅❌❌❌❌ProphetNet❌❌❌✅❌Reformer✅❌✅❌❌RemBERT✅✅✅❌✅RoBERTa✅✅✅❌✅RoFormer✅✅✅❌❌SKEP✅✅❌❌❌SqueezeBERT✅✅✅❌❌T5❌❌❌✅❌TinyBERT✅❌❌❌❌UnifiedTransformer❌❌❌✅❌XLNet✅✅✅❌✅For more pretrained model usage, please refer toTransformer API Docs.Industrial End-to-end SystemWe provide high value scenarios including information extraction, semantic retrieval, question answering high-value.For more details industrial cases please refer toApplications.🔍 Neural Search SystemFor more details please refer toNeural Search.❓ Question Answering SystemWe provide question answering pipeline which can support FAQ system, Document-level Visual Question answering system based on🚀RocketQA.For more details please refer toQuestion AnsweringandDocument VQA.💌 Opinion Extraction and Sentiment AnalysisWe build an opinion extraction system for product review and fine-grained sentiment analysis based onSKEPModel.For more details please refer toSentiment Analysis.🎙️ Speech Command AnalysisIntegrated ASR Model, Information Extraction, we provide a speech command analysis pipeline that show how to use PaddleNLP andPaddleSpeechto solve Speech + NLP real scenarios.For more details please refer toSpeech Command Analysis.High Performance Distributed Training and Inference⚡ FastTokenizer: High Performance Text Preprocessing LibraryAutoTokenizer.from_pretrained("ernie-3.0-medium-zh",use_fast=True)Setuse_fast=Trueto use C++ Tokenizer kernel to achieve 100x faster on text pre-processing. For more usage please refer toFastTokenizer.⚡ FastGeneration: High Performance Generation Librarymodel=GPTLMHeadModel.from_pretrained('gpt-cpm-large-cn')...outputs,_=model.generate(input_ids=inputs_ids,max_length=10,decode_strategy='greedy_search',use_fast=True)Setuse_fast=Trueto achieve 5x speedup for Transformer, GPT, BART, PLATO, UniLM text generation. For more usage please refer toFastGeneration.🚀 Fleet: 4D Hybrid Distributed TrainingFor more super large-scale model pre-training details please refer toGPT-3.InstallationPrerequisitespython >= 3.7paddlepaddle >= 2.3More information about PaddlePaddle installation please refer toPaddlePaddle's Website.Python pip Installationpip install --upgrade paddlenlpor you can install the latest develop branch code with the following command:pipinstall--pre--upgradepaddlenlp-fhttps://www.paddlepaddle.org.cn/whl/paddlenlp.htmlQuick StartTaskflowaims to provide off-the-shelf NLP pre-built task covering NLU and NLG scenario, in the meanwhile with extremely fast inference satisfying industrial applications.frompaddlenlpimportTaskflow# Chinese Word Segmentationseg=Taskflow("word_segmentation")seg("第十四届全运会在西安举办")>>>['第十四届','全运会','在','西安','举办']# POS Taggingtag=Taskflow("pos_tagging")tag("第十四届全运会在西安举办")>>>[('第十四届','m'),('全运会','nz'),('在','p'),('西安','LOC'),('举办','v')]# Named Entity Recognitionner=Taskflow("ner")ner("《孤女》是2010年九州出版社出版的小说，作者是余兼羽")>>>[('《','w'),('孤女','作品类_实体'),('》','w'),('是','肯定词'),('2010年','时间类'),('九州出版社','组织机构类'),('出版','场景事件'),('的','助词'),('小说','作品类_概念'),('，','w'),('作者','人物类_概念'),('是','肯定词'),('余兼羽','人物类_实体')]# Dependency Parsingddp=Taskflow("dependency_parsing")ddp("9月9日上午纳达尔在亚瑟·阿什球场击败俄罗斯球员梅德韦杰夫")>>>[{'word':['9月9日','上午','纳达尔','在','亚瑟·阿什球场','击败','俄罗斯','球员','梅德韦杰夫'],'head':[2,6,6,5,6,0,8,9,6],'deprel':['ATT','ADV','SBV','MT','ADV','HED','ATT','ATT','VOB']}]# Sentiment Analysissenta=Taskflow("sentiment_analysis")senta("这个产品用起来真的很流畅，我非常喜欢")>>>[{'text':'这个产品用起来真的很流畅，我非常喜欢','label':'positive','score':0.9938690066337585}]API ReferenceSupportLUGEdataset loading and compatible with Hugging FaceDatasets. For more details please refer toDataset API.Using Hugging Face style API to load 500+ selected transformer models and download with fast speed. For more information please refer toTransformers API.One-line of code to load pre-trained word embedding. For more usage please refer toEmbedding API.Please find all PaddleNLP API Reference from ourreadthedocs.CommunitySlackTo connect with other users and contributors, welcome to join ourSlack channel.WeChatScan the QR code below with your Wechat⬇️. You can access to official technical exchange group. Look forward to your participation.CitationIf you find PaddleNLP useful in your research, please consider cite@misc{=paddlenlp,
    title={PaddleNLP: An Easy-to-use and High Performance NLP Library},
    author={PaddleNLP Contributors},
    howpublished = {\url{https://github.com/PaddlePaddle/PaddleNLP}},
    year={2021}
}AcknowledgeWe have borrowed from Hugging Face'sTransformers🤗 excellent design on pretrained models usage, and we would like to express our gratitude to the authors of Hugging Face and its open source community.LicensePaddleNLP is provided under theApache-2.0 License.