Facebook PyText is an Open Source Framework for Rapid NLP Experimentation
By: Jesus Rodriguez
Tags: Deep Learning, Thesequence, Machine Learning, Artificial Intelligence, Data Science

Facebook PyText is an Open Source Framework for Rapid NLP Experimentation

The new PyTorch-based framework enables the rapid creation and testing of NLP models.

I recently started a new newsletter focus on AI education. TheSequence is a no-BS( meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:

Natural language processing(NLP) has become the best known discipline in the deep learning space in rencet years. Part of that popularity have brought together an explosion of tools and frameworks such as Google Cloud, Azure LUIS, AWS Lex or Watson Assistant, NLP that have enable the implementation of simple NLP applications without requiring any deep learning knowledge. These platforms are only applicable to relatively basic NLP scenarios. For the rest of the use cases, building NLP applications at scale remains incredibly challenging often surfacing strong frictions between the possibilities of research/experimentation and the realities of model serving/deployment. As one of the biggest conversational environments in the market, Facebook has been facing the challenges of building NLP applications at scale for years. Recently, the Facebook engineering team open sourced the first version of PyText, a PyTorch-based framework for building faster and more efficient NLP solutions.

The ultimate goal of PyText is to provide a simpler experience for the end-to-end implementation of NLP workflows. To achieve that, PyText needs to address some of the existing friction points in NLP workflows. From those friction points, the most troublesome is the existing mismatch between the experimentation and model serving stages of the lifecycle of an NLP application.

Solving the Tradeoff Between NLP Experimentation and Production

The implementation of modern NLP solution typically includes very heavy experimentation phases in which data scientists rapidly test new ideas and models, many of them extracted from research literature, in order to achieve certain levels of performance. During experimentation, data scientists tend to favor frameworks that provide easy to use, eager-execution interfaces that facilitates writing advanced and dynamic models quickly. Frameworks such as PyTorch or TensorFlow Eager are great examples of this category. When deployment time comes, the limitations of the dynamic-graph models become a challenge and deep learning technologists gravitate towards frameworks with static computation graphs and are optimized for operating at scale. TensorFlow, Caffe2 or MxNet are well-known members of this type of stack. The end result is that large data science teams often end up using different stacks for experimentation than for model serving and production deployment.

PyTorch was one of the first deep learning frameworks that addressed the often-conflicting gap between rapid experimentation and model serving at scale. PyText builds on the PyTorch foundation to optimize some of those principles for the NLP space.

Understanding PyText

From the conceptual standpoint, PyText was designed to achieve four fundamental goals:

1. Make experimentation with new modeling ideas as easy and as fast as possible.

2. Make it easy to use pre-built models on new data with minimal extra work.

3. Define a clear workflow for both researchers and engineers to build, evaluate, and ship their models to production with minimal overhead.

4. Ensure high performance (low latency and high throughput) on deployed models at inference.

The capabilities of PyText result on a modeling framework that helps researchers and engineers build end-to-end pipelines for training or inference. The current implementation of PyText covers the fundamental stages of the lifecycle of an NLP workflow providing interfaces for rapid experimentation, raw data processing, reporting of metrics, training and serving of trained models. A high level view of the architecture of PyText clearly reveals how those stages are clearly encapsulated by native components of the framework.

As illustrated in the previous figure, the architecture of PyText includes the following building blocks:

· Task: Combines various components required for a training or inference task into a pipeline.

· Data Handler: Processes raw input data and prepare batches of tensors to feed to the model.

· Model: Defines the neural network architecture.

· Optimizer: Encapsulates model parameter optimization using loss from forward pass of the model.

· Metric Reporter: Implements the relevant metric computation and reporting for the models.

· Trainer: Uses the data handler, model, loss and optimizer to train a model and perform model selection by validating against a holdout set.

· Predictor: Uses the data handler and model for inference given a test dataset.

· Exporter: Exports a trained PyTorch model to a Caffe2 graph using ONNX8.

As you can see, PyText leverages the Open Neural Network Exchange Format(ONNX) to transition models from experimentation-friendly PyTorch to production-robust Caffe2 runtimes.

PyText includes a large portfolio of NLP tasks such as text classification, word tagging, semantic parsing, and language modeling which streamline the implementation of NLP workflows. Similarly, PyText ventures in the area of language understanding by using contextual models such as a SeqNN model for intent labeling tasks and a Contextual Intent Slot model for joint training on multiple tasks.

From an NLP workflow standpoint, PyText streamlines the process of transitioning an idea from experimentation to production. The typical workflow of a PyText application includes the following steps:

1. Implement the model in PyText, and make sure offline metrics on the test set look good.

2. Publish the model to the bundled PyTorch-based inference service, and do a real-time small scale evaluation on a live traffic sample.

3. Export it automatically to a Caffe2 net. In some cases, e.g. when using complex control flow logic and custom data-structures, this might not yet be supported via PyTorch 1.0.

4. If the procedure in 3 isn’t supported, use the Py-Torch C++ API9 to rewrite the model (only the torch.nn.Module10 subclass) and wrap it in a Caffe2 operator.

5. Publish the model to the production-grade Caffe2 prediction service and start serving live traffic

Using PyText

Getting started with PyText is relatively simple. The framework can be installed as a typical python package.

$ pip install pytext-nlp

After that, we can train an NLP model using a task configuration.

(pytext) $ cat demo/configs/docnn.json

{

"task": {

"DocClassificationTask": {

"data_handler": {

"train_path": "tests/data/train_data_tiny.tsv",

"eval_path": "tests/data/test_data_tiny.tsv",

"test_path": "tests/data/test_data_tiny.tsv"

}

}

}

}$ pytext train < demo/configs/docnn.json

A Task is the central artifact for define the model artifacts in a PyText application. Every task has an embedded config that defines the relationships between the different components as shown in the following code.

from word_tagging import ModelInputConfig, TargetConfig class WordTaggingTask(Task):

class Config(Task.Config):

features: ModelInputConfig = ModelInputConfig()

targets: TargetConfig = TargetConfig()

data_handler: WordTaggingDataHandler.Config = WordTaggingDataHandler.Config()

model: WordTaggingModel.Config = WordTaggingModel.Config()

trainer: Trainer.Config = Trainer.Config()

optimizer: OptimizerParams = OptimizerParams()

scheduler: Optional[SchedulerParams] = SchedulerParams()

metric_reporter: WordTaggingMetricReporter.Config = WordTaggingMetricReporter.Config()

exporter: Optional[TextModelExporter.Config] = TextModelExporter.Config()

After a model has been trained, we can evaluate the model and also exported to Caffe2.

(pytext) $ pytext test < "$CONFIG"(pytext) $ pytext export --output-path exported_model.c2 < "$CONFIG"

It is important to notice that PyText provides a very extensible architecture and each one of its key building blocks can be customized and extend it.

PyText represents an important milestone in NLP development as one of the first frameworks that addresses the often-conflicting tradeoff between experimentation and production. With the support of Facebook and the PyTorch community, PyText might has the opportunity to become one of the most important NLP stacks in the deep learning ecosystem.