Turbo-charge your spaCy NLP pipeline | by Prashanth Rao | Towards Data Science

Consider you have a large text dataset on which you want to apply some non-trivial NLP transformations, such as stopword removal followed by lemmatizing the words (i.e. reducing them to root form) in the text. spaCy is an industrial-strength NLP library designed for just such a task.
In this post, the New York Times dataset is used to showcase how to significantly speed up a spaCy NLP pipeline. The goal is to take in an article’s text, and speedily return a list of lemmas as well as unnecessary words, i.e. stopwords, removed.
Pandas DataFrames provide a convenient interface to work with tabular data of this nature — the spaCy NLP methods can be conveniently applied directly to the relevant column of the DataFrame. First, the news data is obtained by running the preprocessing notebook (./data/preprocessing.ipynb), which processes the raw text file downloaded from Kaggle and performs some basic cleaning on it. This step generates a file that contains the tabular data (stored as nytimes.tsv). A curated stopword file is also provided in the same directory.
Since we will not be doing any specialized tasks such as dependency parsing and named entity recognition in this exercise, these components are disabled when loading the spaCy model.
Tip: spaCy has a sentencizer component that can be plugged into a blank pipeline.
The sentencizer pipeline simply performs tokenization and sentence boundary detection, following which lemmas can be extracted as token properties.
A method is defined to read in stopwords from a text file and convert it to a set in Python (for efficient lookup).
The pre-processed version of the NYT news dataset is read in as a Pandas DataFrame. The columns are named date, headline and content - the text present in the content column is what will be preprocessed to remove stopwords and generate token lemmas.
Since the news article data comes from a raw HTML dump, it is very messy and contains a host of unnecessary symbols, social media handles, URLs and other artifacts. An easy way to clean it up is to use a regex that parses only alphanumeric strings and hyphens (so as to include hyphenated words) that are between a given length (3 and 50). This filters each document down to only meaningful text for the lemmatizer.
Now that we have just the clean, alphanumeric tokens left over, these can be further cleaned up by removing stopwords before proceeding to lemmatization.
The straightforward way to process this text is to use an existing method, in this case, the lemmatize method shown below, and apply it to the clean column of the DataFrame using pandas.Series.apply. Lemmatization is done using the spaCy's underlying Doc representation of each token, which contains a lemma_ property. Stopwords are removed simultaneously with the lemmatization process, as each of these steps involves iterating through the same list of tokens.
The resulting lemmas are stored as a list in a separate column preproc as shown below.
%%timedf_preproc['preproc'] = df_preproc['clean'].apply(lemmatize)df_preproc[['date', 'content', 'preproc']].head(3)CPU times: user 48.5 s, sys: 146 ms, total: 48.6 s Wall time: 48.6 s
Applying this method to the clean column of the DataFrame and timing it shows that it takes almost a minute to run on 8,800 news articles.
Can we do better? in the spaCy documentation, it is stated that “processing texts as a stream is usually more efficient than processing them one-by-one”. This is done by calling a language pipe, which internally divides the data into batches to reduce the number of pure-Python function calls. This means that the larger the data, the better the performance gain that can be obtained by nlp.pipe.
To use the language pipe to stream texts, a new lemmatizer method is defined that directly works on a spaCy Doc object. This method is then called in batches to work on a sequence of Doc objects that are streamed through the pipe as shown below.
Just as before, a new column is created by passing data from the clean column of the existing DataFrame. Note that unlike in workflow #1 above, we do not use the apply method here - instead, the column of data (an iterable) is directly passed as an argument to the preprocessor pipe method.
%%timedf_preproc['preproc_pipe'] = preprocess_pipe(df_preproc['clean'])df_preproc[['date', 'content', 'preproc_pipe']].head(3)CPU times: user 51.6 s, sys: 144 ms, total: 51.8 s Wall time: 51.8 s
Timing this workflow doesn’t seem to show improvement over the previous workflow, but as per the spaCy documentation, one would expect that as we work on bigger and bigger datasets, this approach should show some timing improvement (on average).
We can do still better! The previous workflows sequentially worked through each news document to produce the lemma lists, which were then appended to the DataFrame as a new column. Because each row’s output is completely independent of the other, this is an embarrassingly parallel problem, making it ideal for using multiple cores.
The joblib library is recommended by spaCy for processing blocks of an NLP pipeline in parallel. Make sure that you pip install joblib before running the below section.
To parallelize the workflow, a few more helper methods must be defined.
Chunking: The news article content is a list of (long) strings where each document represents a single article’s text. This data must be fed in “chunks” to each worker process started by joblib. Each call of the chunker method returns a generator that only contains that particular chunk's text as a list of strings. During lemmatization, each new chunk is retrieved based on the iterator index (with the previous chunks being "forgotten").
Flattening: Once joblib creates a set of worker processes that work on each chunk, each worker returns a “list of lists” containing lemmas for each document. These lists are then combined by the executor to provide a 3-level nested final “list of lists of lists”. To ensure that the length of the output from the executor is the same as the actual number of articles, a “flatten” method is defined to combine the result into a list of lists containing lemmas. As an example, two parallel executors would return a final nested list of the form: [[[a, b, c], [d, e, f]], [[g, h, i], [j, k, l]]], where [[a, b, c], [d, e, f]] and [[g, h, i], [j, k, l]] refer to the output from each executor (the final output is then concatenated to a single list by joblib). A flattened version of this result would be [[a, b, c], [d, e, f], [g, h, i], [j, k, l]], i.e. with one level of nesting removed.
In addition to the above methods, a similar nlp.pipe method is used as in workflow #2, on each chunk of texts. Each of these methods is wrapped into a preprocess_parallel method that defines the number of worker processes to be used (7 in this case), breaks the input data into chunks and returns a flattened result that can then be appended to the DataFrame. For machines with a higher number of physical cores, the number of worker processes can be increased further.
The parallelized workflow using joblib is shown below.
%%timedf_preproc['preproc_parallel'] = preprocess_parallel(df_preproc['clean'], chunksize=1000)CPU times: user 683 ms, sys: 248 ms, total: 932 ms Wall time: 17.2 s
Timing this parallelized workflow shows significant performance gains (almost 3x reduction in run time)! As the number of documents becomes larger, the additional overhead of starting multiple worker processes with joblib is quickly paid for, and this method can significantly outperform the sequential methods.
Note that in the parallelized workflow, two parameters need to be specified — the optimum number can vary depending on the dataset. The chunksize controls the size of each chunk being worked on by each process. In this example, for 8,800 documents, a chunksize of 1000 is used. Too small a chunksize would mean that a large number of workers would spawn to deal with the large number of chunks overall, which can slow down execution. Generally, a chunksize of several hundred documents to a few thousand is a good starting point (of course, this depends on how big each document in the data is so that the chunks can fit into memory).
The batch size is parameter specific to nlp.pipe, and again, a good value depends on the data being worked on. For reasonably long-sized text such as news articles, it makes sense to keep the batch size reasonably small (so that each batch doesn't contain really long texts), so in this case 20 was chosen for the batch size. For other cases (e.g. Tweets) where each document is much shorter in length, a larger batch size can be used.
It is recommended to experiment with either parameter to see which combination produces the best performance.
Important: Use sets over lists for lookups wherever possible.
Note that in the get_stopwords() method defined earlier on, the list of stopwords read in from the stopword file was converted to a set before using it in the lemmatizer method for stopword removal via lookups. This is a very useful trick in general, but specifically for stopword removal, the use of sets becomes all the more important. Why?
In any realistic stopword list, such as this one for a news dataset, it’s reasonable to expect several hundred stopwords. This is because for downstream tasks such as topic modelling or sentiment analysis, there are a number of domain-specific words that need to be removed (very common verbs, useless abbreviations such as timezones, days of the week, etc.). Each word in each and every document needs to be compared against every word in the stopword list, which is an expensive operation over tens of thousands of documents.
It’s well known that sets have O(1) (i.e. constant) lookup time as opposed to lists, which have O(n) lookup time. In the lemmatize() method, since we're checking each word for membership in the set of stopwords, we would expect sets to be much better than lists. To test this, we can rerun workflow #1 but this time, use a stopword list instead.
stopwords = list(stopwords)%%timedf_preproc['preproc_stopword_list'] = df_preproc['clean'].apply(lemmatize)df_preproc[['date', 'content', 'preproc_stopword_list']].head(3)CPU times: user 1min 17s, sys: 108 ms, total: 1min 18s Wall time: 1min 18s
With a stopword list, producing the same result now takes ~ 50% longer than it did before (with the set), which is a 1.5x increase in run time! This makes sense because in this case, the stopword list is about 500 words long, and each and every word in the corpus needs to be checked for membership in this reasonable-sized list.
In this exercise, a news article dataset (NY Times) was processed using a spaCy pipeline to output a list of lemmas representing the useful tokens present in each article’s content. Because real-world news datasets are almost certainly bigger than this one, and can be unbounded in size, a fast, efficient NLP pipeline is necessary to perform any meaningful analysis on the data. The following steps are very useful in speeding up the spaCy pipeline.
Disable unnecessary components in spaCy model: The standard spaCy model’s pipeline contains the tagger (to assign part-of-speech tags), the parser (to generate a dependency parse) and named entity recognition components. If any or none of these actions are desired, these components must be disabled immediately after loading the model (as shown above).
Use sets over lists for lookups: When performing lookups to compare one set of tokens against another, always perform membership checks using sets — lists are significantly slower for lookups! The larger the list/set of stopwords, the bigger the performance gain seen when using sets.
Use custom language pipes when possible: Setting up a language pipe using nlp.pipe is an extremely flexible and efficient way to process large blocks of text. Even better, spaCy allows you to individually disable components for each specific sub-task, for example, when you need to separately perform part-of-speech tagging and named entity recognition (NER). See the spaCy docs for examples on how to disable pipeline components during model loading, processing or handling custom blocks.
Use multiple cores when possible: When processing individual documents completely independent of one another, consider parallelizing the workflow by dividing the computation across multiple cores. As the number of documents becomes larger and larger, the performance gains can be tremendous. One just needs to ensure that the documents are divided up into chunks, all of which must fit into memory at any given time.
I hope this was useful — have fun testing these out in your next NLP project!
Originally published at https://prrao87.github.io on May 2, 2020.