Overview | Stanza | New Python Natural Language Processing Library | Stanford
By: Chien Vu
Tags: Machine Learning, Data Science, Stanford, NLP, Stanza

Overview | Stanza | New Python Natural Language Processing Library | Stanford

This topic summarizes the new library Python Natural Language from scratch with applications

Background 📙

Recently, The Stanford NLP Group released Stanza : A Python Natural Language Processing Toolkit for Many Human Languages [1] that introduced an open source Python natural language processing. Compared to existing toolkits for NLP task, researchers stated that:

S tanza fully supported neural pipeline for text analysis, including tokenization, multiword token expansion, lemmatization, part-of-speech, morphological feature tagging, dependency parsing, and named entity recognition

The toolkit is built on top of the PyTorch library with support for using GPU and pre-trained neural models. Furthermore, Stanza assists 66 human languages which was trained on a total of 112 datasets, including the Universal Dependencies treebanks and other multilingual corpora. The result showed that the same neural architecture generalizes well and achieves competitive performance on all languages tested. You can run a demo here. Up to now, the popular NLP toolkits such as CoreNLP [2], FLAIR [3], spaCy [4], and UDPipe [5] which has been using in a wide variety of applications, they also suffer from several limitations: few number of supported major languages; under-optimized model and some tools lacking the ability to process raw text from diverse sources. Compared to these NLP toolkits, Stanza has the following advantages:

From raw text to annotations: Stanza features a fully neural pipeline which takes raw text as input, and produces annotations to any specific purpose.

Stanza features a fully neural pipeline which takes raw text as input, and produces annotations to any specific purpose. Multilingual. Stanza architectural design is language-agnostic and data-driven, which allows us to release models supporting 60+ languages, by training the pipeline on the Universal Dependencies (UD) treebanks and other multilingual corpora. Comparing the system against UDPipe and spaCy on treebanks of five major languages where the corresponding pretrained models are publicly available, the result showed better performance over others. Therefore, you can strongly believe and confident to use the result for your downstream application. You can find more information in their paper.

Neural pipeline performance comparisons

State-of-the-art performance: Stanza neural pipeline adapts well to text of different genres, achieving state-of-the-art or competitive performance at each step of the pipeline.

The summarized difference of Stanza with exiting NLP toolkits shows below:

Feature comparisons of Stanza against other popular natural language processing toolkits.

Getting Started 🛠️

First of all you need to install Stanza . Stanza supports Python 3.6 or later. With pip, you can easily run this code:

pip install stanza

If you already install stanza, just run the code below (it helps resolve all of the dependencies of Stanza)

pip install stanza -U

Or with Anaconda:

conda install -c stanfordnlp stanza

If you don’t want to use or get a trouble with pip and Anaconda you can download from git repository, which will give you more flexibility in developing on top of Stanza.