ACHIEVE
NEXT-GENERATION
NATURAL LANGUAGE
PROCESSING
How to keep up with thelatest innovations in NLPCONTENTSI N TROSE C T I O N 1

SE C T I O N 2

S ECTION 3S ECTION 4S ECTION 5

CONCLU SI ON

Natural Language
Processing is at
the Center ofInnovation

Conventional
Technologies

CommonNLP Models

Process
Demands

Compute Barriers

An Extensible
ML Services
Platform for NLP

Achieve
BreakthroughEfficiency

36812141620Achieve Next-Generation Natural Language Processing

2INT RO

Natural Language Processing isat the Center of Innovation
We interact with natural language processing(NLP) models every day. From sentiment analysis
to fraud detection, NLP technology is often at
the center of our day-to-day interactions with AI.
NLP is one of the most actively pursued areas ofmachine learning among researchers, engineers,and data scientists who count on its ubiquitous
automated applications, such as predictive text,
virtual assistants, chatbots, semantic search,and social media listening. These applications
reliably accelerate innovation, establish market
differentiation, and ultimately save a lot of timeand effort.
As critical as they are to the future of AI,
developing state-of-the-art NLP models is very
complex, expensive, and time-consuming. Howcan organizations accelerate time to value?

Achieving Next-Generation Natural Language Processing

3INT ROA Driving Force in AI
NLP adoption is increasing across industries. In fact,
IDC recognizes AI software spend supporting NLPpowered solutions in five key industries: banking,retail, manufacturing, healthcare, and securities
and investments.1 Given its applicability and rising
popularity, it’s no surprise that NLP use cases are
growing faster than organizations can keep up.

”Worldwide AI systems spend has moved
beyond the early adopters to mainstream
industrywide use-case implementation.2

IDC

1 IDC, NLP by AI Industry and Use Case - Nov 2019
2 IDC, NLP by Artificial Intelligence Industry and Use Case, Nov 2019

Achieving Next-Generation Natural Language Processing

4

INT RONLP Technology is Well-known for Contributing to the Modern Innovations We Use Every Day

Sentiment analysis

Fraud detection

Personal assistants

Automatedservice agents

Search engines

Ad content suggestions

Trading signals

Document analysis

ChatbotsAchieve Next-Generation Natural Language Processing

5

S ECTIO N 1

CONVENTIONAL TECHNOLOGIESSE C T ION 1

Conventional Technologies
Can’t Keep Up
NLP is evolving faster than the technology used fortraining and inference in language models. But as
demand for fast, more accurate NLP models grows,
compute performance progressively lags. The twothings model owners say matter most are accuracy
and time to results, outputs that require adequate
compute resources. Today, it’s only a matter of time
before NLP’s requirements overwhelm the hardware
and software capabilities of most organizations. Infact, we’re already starting to see it happen.
From an evolutionary standpoint, NLP models
are trending away from simpler models, such as
LSTM, to computationally intensive, large-capacitytransformer models, such as BERT and GPT-3. To
perform, these larger NLP models require machinelearning expertise and a powerful computational
infrastructure—both of which are difficult resources
to come by for most organizations.

Achieving Next-Generation Natural Language Processing

7

S ECTIO N 2

COMMON NLP MODELSSE C T ION 2

Common NLP Models
Organizations use NLP to increase businessefficiency by automating labor-intensive
or repetitive tasks traditionally performedby people. To successfully leverage NLP forcompetitive advantage, they need the ability
to more quickly develop highly accurate models
to drive better business outcomes, justify NLP
investments, and deliver innovative applications.
While the NLP model-to-application pipeline isripe for optimization and innovation, obstacles
loom large between model owners and cuttingedge breakthroughs. Training models still takes
too much time in most scenarios. And thepipeline itself, with its multiple stages, presents
challenges as model owners work to incorporate
new efficiencies into workflows.

Achieving Next-Generation Natural Language Processing

9SE C T ION 2Models MatterIn the world of NLP modeling, these models dominate:

BERT
In late 2018, Google researchers released BERT
(Bidirectional Encoder Representations from Transformers).
In addition to capturing semantic meanings of words
statically, BERT can pick up on the contextual meaningof words depending on how they appear in a sentence.
BERT models can also capture the intention behind words,
understanding subtle nuances in language—even when
expressed improperly. A properly trained BERT model
can enable virtually unlimited possibilities in language
applications. Facebook, for instance, developed theirown version of BERT called RoBERTa that tackled thesocial network’s challenges in content moderationby having it learn multiple languages simultaneously.
Unfortunately, outside of these hyperscalers, performing
domain-specific pretraining and fine-tuning requires
expertise and resources that are hard to come by for

Dataflow-as-a-Service

for Language

many organizations.

Achieving Next-Generation Natural Language Processing

10

SE C T ION 2GPT-3
Generative Pretrained Transformer-3 (GPT-3) is a
deep learning language model that’s capableof generating state-of-the-art, human-like text.
Achieving remarkable results on a number ofbenchmarks, the full version of GPT-3 features
175 billion parameters—making it one of thelargest natural language models in existence. The
expansiveness of this powerful model comes at aprice, with estimates suggesting that it costs up toa staggering $12 million in compute resources totrain. Despite all of the promising capabilities that itdemonstrates, this places GPT-3 firmly out of reachfor most organizations. While OpenAI offers GPT-3’s
API for beta trials, access requests are waitlisted,and results are not guaranteed.

Achieving Next-Generation Natural Language Processing11

S ECTIO N 3

PROCESS DEMANDSSE C T ION 3

Process Demands, Constraints, and Challenges
Ideally, remedies to deep learning training compute problems should deliver
great model accuracy and be flexible enough to support memory-heavy

Step-by-step, Token-by-token

The standard model to application pipeline follows theseprimary steps:

workloads of any size when needed. Whether performing sentiment analysis
on large amounts of data, or distilling large models into small models to be
deployed quickly—compute capacity matters—and so does efficiently usingthat capacity.
Enabling state-of-the-art machine intelligence is crucial for text andlanguage understanding in many domains. Users are pushing for larger and1. Extract Information

2. Tokenization

larger models to advance machine intelligence for NLP. But training large NLP

Take in data and clean it up

Define, identify, andmodels requires massive compute resources, and the process is constrained

sequence databy the limitations of conventional technologies (i.e., CPUs and GPUs).
From training to inference, NLP requires technical expertise that is in short
supply and expensive. Where once building and deploying large deeplearning models was limited to hyperscale enterprises that can afford to hirehundreds of skilled technical developers, today, organizations of all sizes arelooking for the means to take advantage of machine learning.

3. Model Processing

4. Post-process Output

Operate, compute, andEvaluate and refineperform tasks

Achieve Next-Generation Natural Language Processing

13

S ECTIO N 4

COMPUTE BARRIERSSE C T ION 4As NLP continues on its

Current Compute Solutions
and Small Models

Compute Barriers forLarger Language Models

Highly-detailed, smaller models require
intense compute capacity and can waste
compute resource.

Larger language models require more computeand capacity than today’s solutions deliver.

trajectory to evolve faster thanits supporting technology,
conventional processors will face
more elaborate and complex
bottlenecks that will further clog
the NLP pipeline at the model
processing stage and limit thepace of innovation due toGPUs- Underutilized capacity- Kernel-by-kernel execution mode
- High overhead- Low utilization- One operation at a time

- Hundreds or thousands ofCPUs- Capacity is too low- Not designed for heavy linear

CPUs- Capacity is too low- Subject to many constraints

GPUs- Design does not support

the lack of performance andavailable capacity.

enough memory
GPUs needed

algebra compute

Achieve Next-Generation Natural Language Processing

15

S ECTIO N 5

AN EXTENSIBLE ML SERVICES PLATFORM FOR NLPSE C T ION 5

An Extensible ML Services
Platform for NLP

Quickly Ramp NLP Model Training From Zero to
State-of-the-Art

Enterprises working toward the cutting edge ofOrganizations can now rapidly train NLP applications and scaleinnovation with NLP need solutions that overcome

on-demand. SambaNova’s Dataflow-as-a-Service for Language

compute limitations and remove obstacles toaccelerates AI workloads and provides comprehensive services,developing more advanced machine learningmodels, and a platform—empowering you to deploy customized

models. There is a better way to meet the model

NLP solutions with confidence.

training needs of NLP applications.
SambaNova Systems has developed an optimal
product that enables users to lower the NLP
model training time, from months to days, andkeep models up to date efficiently—Dataflow-asa-ServiceTM for Language.

Dataflow-as-a-Service for Language is designed to help organizations
quickly get up and running with strategic NLP initiatives. This solution
enables users to overcome obstacles that hinder their organization’s
ability to leverage NLP for next-generation AI innovation—and competitiveadvantage. These challenges include:- NLP training is time-prohibitive, and experts are hard to find andcostly to hire.

- Deployment requires hard trade-offs that sacrifice accuracy for
expediency—adding complexity to the NLP production pipeline.

- Exponentially growing datasets and models compound the difficulty,
hindering the advancement of AI applications.

Achieve Next-Generation Natural Language Processing17

SE C T ION 5

Quickly Ramp NLP Model Training
From Zero to State-of-the-Art
By providing an end-to-end platform for training, finetuning, and inference, SambaNova’s Dataflow-as-aService for LanguageTM offers a flexible, subscription-based
NLP solution that effectively addresses these challenges
and more.

The fastest path to AI- Accelerate AI Innovation. Use your data to deploy customized
NLP solutions at best-in-class accuracy.

The easiest path to AI- Augment your organization’s ML expertise. You don’t need to bea tech giant to deploy world-class machine learning

The guaranteed path to AI- Stay current with the latest advancements in NLP seamlessly. Get
the most up-to-date NLP models and algorithmic techniques.

Achieve Next-Generation Natural Language Processing

18SE C T ION 5

A Flexible New Design Architecture to Support NLP Applications
1.

2.
A CoreEnabling Platform

3.
The Next-Generation
Processor

A Complete
Software Stack

Optimize dataflow from algorithms

SambaNova’s Reconfigurable Dataflow

The complete software stack—to silicon using SambaNova Systems

UnitTM—RDU—enables access to native

SambaFlowTM—is designed to take

DataScale®. Build quickly and deploy

dataflow processing and programmable

input from standard machine-learningnext-generation AI technologies at scaleacceleration. RDUs with high compute

frameworks such as PyTorch andusing DataScale as a core enabling

and memory capacity can process huge

TensorFlow. It automatically extracts,

platform. Built on SambaNova’s

language models while still offering theoptimizes, and maps dataflow graphs onto

Reconfigurable Dataflow ArchitectureTM

flexibility to optimize for smaller detailedRDUs, allowing high performance to be(RDA), DataScale enables organizations

models, high accuracy, cost-efficiency,

obtained without the need for low-levelto achieve unparalleled model training

and greater scalability. Easy to use, thekernel tuning.

efficiency and performance acrossnew RDU solutions provide NLP model

NLP applications

owners cost-effective, democratizedaccess to high accuracy and scalability.

Achieving Next-Generation Natural Language Processing19

CONC L US IO N

ACHIEVE BREAKTHROUGH EFFICIENCYC O NCLU S I ON

Achieve Breakthrough Efficiency
in NLP Model Training
With optimized dataflow from the algorithms tothe silicon, SambaNova is always rethinking how
modern compute solutions can more efficiently andaccurately process machine learning models tobetter support next-generation AI.
Dataflow-as-a-Service for Language provides aflexible platform solution that accelerates the entire
NLP pipeline across model training, fine-tuning,
distillation, and deployment. With state-of-the-art
accuracy, performance, scale, and ease of use,
SambaNova’s NLP solution opens up a new realm ofattainable AI innovation for organizations of all sizes.

Achieve Next-Generation Natural Language Processing

21

DI SC OV E R HOW YOUR OR GANIZ ATION CAN D EPLOY ACCUR ATE
A N D P OW ER F UL NLP S OLUTIONS UP TO 18 MONTHS FAS TERCOMPAR ED TO D O- IT- YOUR S ELF S OLUTIONS .

Visit us on the Web at SambaNova.AIAbout SambaNova Systems
SambaNova Systems is an AI innovation company that empowers organizations to deploy best-in-class solutions for computer vision, natural language processing, recommendation, and AI
for science with confidence. SambaNova’s flagship offering, Dataflow-as-a-Service™, helps organizations rapidly deploy AI in days, unlocking new revenue and boosting operational efficiency.
SambaNova’s DataScale® is an integrated software and hardware system using Reconfigurable Dataflow Architecture™, along with open standards and user interfaces. Headquartered in Palo
Alto, California, SambaNova Systems was founded in 2017 by industry luminaries, and hardware and software design experts from Sun/Oracle and Stanford University. Investors include SoftBank
Vision Fund 2, funds and accounts managed by BlackRock, Intel Capital, GV, Walden International, Temasek, GIC, Redline Capital, Atlantic Bridge Ventures, Celesta, and several others. For more
information please visit us at sambanova.ai or contact us at info@sambanova.ai. Follow SambaNova Systems on LinkedIn.

Copyright © 2021 SambaNova Systems, Inc. All rights reserved. | Palo Alto, CA and Austin, TX | (650) 263-1153