How OpenAI and Weights & Biases
Work Together

The OpenAI + Weights & Biases
collaboration enables developers
to access GPT-3 fine-tune
experiments on Weights &Biases’ best-in-class platform,
bringing experiment tracking,dataset versioning, and model
management into their workflows
and building powerful AI
applications faster.
Sync your OpenAI fine-tune
runs in one line of code
We made visualizing and understanding yourGPT-3 models as simple as possible. Nothingchanges with how you’re currently fine-tuning
those models: you’ll still use OpenAI’s API andhave all the same, robust controls you’re used
to. To see and compare runs, visualize andversion your datasets, analyze your results,
and a whole lot more, all you need is one lineof code. It’s really that simple:

Weights & Biases is an entire
MLOps toolbox in 5 lines ofcode. Here’s what we offer:“With the new
W&B and OpenAI
integration, we’reable to consolidate
experiment trackingof our in-house ML
projects and OpenAI
fine-tuning projects inone place. This gives
us huge time savings
towards delivering
insights from
unstructured data forour customers.”
PREM VISWANATHAN$ # one line command

ML Lead at Artifact.io$ openai wandb sync

A central dashboard and source of truth
Weights & Biases is perhaps best known forour experiment tracking dashboard. Onceyou’ve synced your fine-tunes with that one
line of code above, you’ll be able to dig into
performance and compare different metrics
and predictions across your various modeltraining runs.

www.wandb.ai • contact@wandb.ai

SOLUTION SHOWCASE

Our dashboards are customizable so you cancreate the views and look into the metrics
that matter most for your project. You canshare dashboards across your team toincrease collaboration or pin certain notable
insights into our Reports to share withinterested stakeholders.

EXPERIMENT DASHBOARD• W&B lets teams track,compare, log, and visualize
every step of model
development.
• That means teams haveeverything they need toreproduce their models,debug performance inreal- time, capture code,
see live updates on model
development, check foroverfitting, and more.

ARTIFACTS• Data and model pipelines
are complex. They
have deep interlocking
dependencies where asingle change in just one
place can affect models
downstream.
• Artifacts logs and visualizes
those pipelines andrecords every version ofyour datasets so you caneffortlessly understandyour training flow and track
your models.
• Great for companies
with strict regulatory orcompliance requirements

SWEEPS
Sweeps provides a scalable,
customizable tuning systemthat allows teams to optimize
their hyperparameters.

SOLUTION SHOWCASETABLES• Tables makes it easier
to iterate on andevaluate models.
• Log, version, organize, data
in intuitive visualizations
that help teams findpatterns in their datasets.
• Tables lets you see andinteract with your data andmodels in an interactiveway, leading to fasterinsights and deeper
understanding.

REPORTS

Log & explore predictions

Additional resources

A great way to evaluate generative models isby looking at their predictions on data they’ve
not yet seen (a.k.a. validation data). Weights& Biases’ Tables is a great way to both judge
your models performance and share insights
with your team.

If you’d like to get a little more information
about the collaboration, you can checkout our docs on how it works, as well as acouple blog posts that walk you throughsome workflows and tips about how best toleverage W&B with your GPT-3 fine-tunes.

For example, you could easily create a Table
with a prompt, a target, and a generationfrom your fine-tune, then sort them likeyou would a typical csv. This is an easy andintuitive way to make comparisons and findpatterns in your predictions, helping you
understand what your model is excelling at
and where it might need a little extra work.

DOCS:
https://wandb.me/openai-docs
TUTORIAL:
http://wandb.me/openai-report• ML practitioners cansometimes have trouble
sharing their work with
their team and internal
stakeholders. Reports lets
them do just that.
• Reports are interactive
documents wherepractitioners can takenotes, share findings, plan
future research, and, above
all else, collaborate
across themes.

TEXT GENERATION ON DR. WHO
http://wandb.me/dr-who-gpt-3“We are excited to use W&B capabilities to monitor thefine-tuning of our OpenAI models that help improve
how we understand client feedback at Stitch Fix.”REZA SOHRABI
Staff ML Engineer, Stitch Fix Algorithms

LEARN MOREABOUT WEIGHTS & BIASES
Weights & Biases is the premier, developer-first MLOps platform. We help ML teams unlock their productivity
by optimizing, visualizing, collaborating on, and standardizing their model and data pipelines – regardless offramework, environment, or workflow.
www.wandb.ai • contact@wandb.ai© 2022 OpenAI. All rights reserved.