Four Experiments in Handwriting with a Neural Network
=====================================================

Let’s start with generating new strokes based on your handwriting input
-----------------------------------------------------------------------

Play/Pause
Clear
Length of prediction

20
Variation1

0.1

 dt-byline {
 font-size: 12px;
 line-height: 18px;
 display: block;
 border-top: 1px solid rgba(0, 0, 0, 0.1);
 border-bottom: 1px solid rgba(0, 0, 0, 0.1);
 color: rgba(0, 0, 0, 0.5);
 padding-top: 12px;
 padding-bottom: 12px;
 }
 dt-article.centered dt-byline {
 text-align: center;

 }
 dt-byline a,
 dt-article dt-byline a {
 text-decoration: none;
 border-bottom: none;
 }
 dt-article dt-byline a:hover {
 text-decoration: underline;
 border-bottom: none;
 }
 dt-byline .authors {
 text-align: left;
 }
 dt-byline .name {
 display: inline;
 text-transform: uppercase;
 }
 dt-byline .affiliation {
 display: inline;
 }
 dt-byline .date {
 display: block;
 text-align: left;
 }
 dt-byline .year, dt-byline .month {
 display: inline;
 }
 dt-byline .citation {
 display: block;
 text-align: left;
 }
 dt-byline .citation div {
 display: inline;
 }

 @media(min-width: 768px) {
 dt-byline {
 }
 }

 @media(min-width: 1080px) {
 dt-byline {
 border-bottom: none;
 margin-bottom: 70px;
 }

 dt-byline a:hover {
 color: rgba(0, 0, 0, 0.9);
 }

 dt-byline .authors {
 display: inline-block;
 }

 dt-byline .author {
 display: inline-block;
 margin-right: 12px;
 /\*padding-left: 20px;\*/
 /\*border-left: 1px solid #ddd;\*/
 }

 dt-byline .affiliation {
 display: block;
 }

 dt-byline .author:last-child {
 margin-right: 0;
 }

 dt-byline .name {
 display: block;
 }

 dt-byline .date {
 border-left: 1px solid rgba(0, 0, 0, 0.1);
 padding-left: 15px;
 margin-left: 15px;
 display: inline-block;
 }
 dt-byline .year, dt-byline .month {
 display: block;
 }

 dt-byline .citation {
 border-left: 1px solid rgba(0, 0, 0, 0.15);
 padding-left: 15px;
 margin-left: 15px;
 display: inline-block;
 }
 dt-byline .citation div {
 display: block;
 }
 }

[Shan Carter](http://shancarter.com)
[Google Brain](http://g.co/brain)

[David Ha](https://github.com/hardmaru)
[Google Brain](http://g.co/brain)

[Ian Johnson](https://github.com/enjalot)
[Google Cloud](http://cloud.google.com)

[Chris Olah](http://colah.github.io/)
[Google Brain](http://g.co/brain)

Dec. 6
2016

[Citation:
Carter, et al., 2016](#citation)

 Neural networks are an extremely successful approach to machine learning, but it’s tricky to understand why they behave the way they do. This has sparked a lot of interest and effort around trying to understand and visualize them, which we think is so far just scratching the surface of what is possible.

In this article we will try to push forward in this direction by taking a generative model of handwriting2 and visualizing it in a number of ways. The model is quite simple (so as to run well in the browser) so the generated output mostly produces gibberish letters and words (albeit, gibberish that look like real handwriting), but it is still useful for our purposes of exploring visualization techniques.

In the end we don’t have some ultimate answer or visualization, but we do have some interesting ideas to share. Ultimately we hope they make it easier to divine some meaning from the internals of these model.

Looking at the Output of the Model
----------------------------------

Our first experiment is the most obvious: when we want to see how well someone has learned a task we usually ask them to demonstrate it. So, let’s ask our model to write something for us and see how well it does.

Play/Pause
Variation[1](#footnote1)

0.6

Most of the marks are gibberish, but many of them are surprisingly convincing. Some real (or real-ish) words even start to appear. One surprising thing you’ll notice is that the general style of handwriting is more or less consistent within a sample. This is because the type of architecture used for this model (LSTM) has a mechanism for remembering previous strokes. It is therefore able to remember things like how loopy or jerky the handwriting is, or which letter preceeded the current one. (For more on LSTMs and how they can remember, Chris Olah has a [good primer](http://colah.github.io/posts/2015-08-Understanding-LSTMs/) [3].)

Even with this memory, the technique for generating samples from neural networks is probabilistic, meaning each of these samples is one of a much, much larger possibility space. Viewing them one at a time feels unsatisfying — how can we infer much from one or two samples when there are nearly infinite possibilities? If something looks wrong, how do we know if it was because there is something fundamentally wrong with our model or if it was just dumb luck?

![](assets/color-key.svg)

At each iteration our model could have produced many paths. Instead of picking one and throwing the rest away, let’s draw, say, 50 of them. Green strokes below are places where the model would have veered rightward from the chosen stroke, orange is where it would have veered leftward.

Generated sample 1
Generated sample 2
Generated sample 3
Generated sample 4
Generated sample 5
Straight line

Steps

4
Variation[1](#footnote1)

0.5

With this technique we are casting a light into a wider possibility space. You can see some areas where there was general consensus as to what to do next. Other areas had more of a “anything can happen next” feeling. Others seem to show a possible fork in the road. We can be drawing an “a” or “g”. A few steps later it’s clear the model has converged on a cursive “g”.

![](assets/color-key.svg)

In addition to visualizing samples generated by the model, this technique can also be applied to human-generated samples. Below we can see what the model would have done at each point if it had taken over for the person.

Validation sample 0
Validation sample 1
Validation sample 2
Validation sample 3
Validation sample 4
Validation sample 5
Validation sample 6

Steps

4
Variation[1](#footnote1)

0.5

It is obvious from these experiments that this model has learned quite a lot about human handwriting. Which sort of raises the question, can we extract that knowledge in any meaningful way, rather than just blindly using it to mimic handwriting?

Examining the Internals of the Model
------------------------------------

Our model has 500 cells which act as a sort of memory which it will use as part of its input when deciding what to generate. If we can see what those cells are doing as the model progresses we may be able to gain some intuitive understanding about what the model is doing.

We begin by showing the activation of the cells over time. Each column in the heatmap below represents one line segment of the handwriting. Each row represents one cell of the model and is colored by its activations on that part of the stroke. By inspecting the diagram you may be able to see some patterns in the way certain cells activate for certain types of strokes. You can change which cell is used to color the strokes by clicking on the diagram.

You may have been able to pick out one or two cells with interesting patterns, but we have reason to believe that cells work in tandem, such that an individual cell’s activity may not be as interesting as a group of cells.

Is there a way to order the cells to make this structure clearer? We’ve found that applying one-dimensional [t-SNE](https://lvdmaaten.github.io/tsne/) [4] to the activations of the cells over time can organize them, bringing cells with similar behavior close together. This makes the diagram easier to read. We use a few small tricks to achieve the best results.3

In this version we can find a few clear behaviors. For example, cells 11-70 at the top seem sensitive to slightly different directions and curvatures of the pen path — see in particular cells 25 and 55. On the other hand, at the bottom, cells below 427 seem focused on pen lifts; for example, cell 494 seems to predict whether the pen is about to be lifted. In the middle, we see a variety of cells, including some tracking absolute position. Cell 136 and its neighbors seem concerned with horizontal position, while cells around 236 seem concerned with vertical position. Cell 242 appears to track position within a word.

Another way to explore the activations is to give it a sample and interactively see the activations. Below you can write and see the activations of all the cells in real time.

Clear

---

Conclusion
----------

The black box reputation of machine learning models is well deserved, but we believe part of that reputation has been born from the programming context into which they have been locked. The experience of having an easily inspectable model available in the same programming context as the interactive visualization environment (here, javascript) proved to be very productive for prototyping and exploring new ideas for this post.

As we are able to move them more and more into the same programming context that user interface work is done, we believe we will see richer modes of human-ai interactions flourish. This could have a marked impact on debugging and building models, for sure, but also in how the models are used. Machine learning research typically seeks to mimic and substitute humans, and increasingly it’s able to. What seems less explored is using machine learning to augment humans. This sort of complicated human-machine interaction is best explored when the full capabilities of the model are available in the user interface context.

Several interactive visualizations of a generative model of handwriting. Some are fun, some are serious.


Deconvolution and Checkerboard Artifacts
========================================

 dt-byline {
 font-size: 12px;
 line-height: 18px;
 display: block;
 border-top: 1px solid rgba(0, 0, 0, 0.1);
 border-bottom: 1px solid rgba(0, 0, 0, 0.1);
 color: rgba(0, 0, 0, 0.5);
 padding-top: 12px;
 padding-bottom: 12px;
 }
 dt-article.centered dt-byline {
 text-align: center;

 }
 dt-byline a,
 dt-article dt-byline a {
 text-decoration: none;
 border-bottom: none;
 }
 dt-article dt-byline a:hover {
 text-decoration: underline;
 border-bottom: none;
 }
 dt-byline .authors {
 text-align: left;
 }
 dt-byline .name {
 display: inline;
 text-transform: uppercase;
 }
 dt-byline .affiliation {
 display: inline;
 }
 dt-byline .date {
 display: block;
 text-align: left;
 }
 dt-byline .year, dt-byline .month {
 display: inline;
 }
 dt-byline .citation {
 display: block;
 text-align: left;
 }
 dt-byline .citation div {
 display: inline;
 }

 @media(min-width: 768px) {
 dt-byline {
 }
 }

 @media(min-width: 1080px) {
 dt-byline {
 border-bottom: none;
 margin-bottom: 70px;
 }

 dt-byline a:hover {
 color: rgba(0, 0, 0, 0.9);
 }

 dt-byline .authors {
 display: inline-block;
 }

 dt-byline .author {
 display: inline-block;
 margin-right: 12px;
 /\*padding-left: 20px;\*/
 /\*border-left: 1px solid #ddd;\*/
 }

 dt-byline .affiliation {
 display: block;
 }

 dt-byline .author:last-child {
 margin-right: 0;
 }

 dt-byline .name {
 display: block;
 }

 dt-byline .date {
 border-left: 1px solid rgba(0, 0, 0, 0.1);
 padding-left: 15px;
 margin-left: 15px;
 display: inline-block;
 }
 dt-byline .year, dt-byline .month {
 display: block;
 }

 dt-byline .citation {
 border-left: 1px solid rgba(0, 0, 0, 0.15);
 padding-left: 15px;
 margin-left: 15px;
 display: inline-block;
 }
 dt-byline .citation div {
 display: block;
 }
 }

Augustus Odena
[Google Brain](http://g.co/brain)

[Vincent Dumoulin](http://vdumoulin.github.io/)
[Université de Montréal](https://mila.umontreal.ca/en/)

[Chris Olah](http://colah.github.io)
[Google Brain](http://g.co/brain)

Oct. 17
2016

[Citation:
Odena, et al., 2016](#citation)

When we look very closely at images generated by neural networks, we often see a strange checkerboard pattern of artifacts. It’s more obvious in some cases than others, but a large fraction of recent models exhibit this behavior.

.pattern-examples {
 width: 100%;
}
.pattern-examples > figure {
 display: -ms-flexbox;
 display: -webkit-flex;
 display: flex;
 position: relative;
}

.pattern-examples > figure:after {
 overflow: visible;
 content: "";
 background-image: url("assets/pointer.svg");
 width: 27px;
 height: 27px;
 position: absolute;
 right: -51px;
}

.pattern-examples .example {
 width: 100%;
 margin-right: 2.5%;
}

.pattern-examples .example:last-of-type {
 margin-right: 0;
}

.pattern-examples .original {
 position: relative;
}

.pattern-examples .original:after {
 content: "";
 display: block;
 width: 100%;
 height: 150%;
 position: absolute;
 top: -20%;
 z-index: 1;
}

.pattern-examples .example .reticle {
 position: absolute;
 pointer-events: none;
}

.pattern-examples .example .reticle:after {
 box-sizing: border-box;
 position: relative;
 width: 100%;
 height: 100%;
 left: -50%;
 top: -50%;
 border-radius: 50%;
 border: 1px solid white;
 display: block;
 content: '';
 background-color: rgba(255, 255, 255, 0.2);
 box-shadow: 0 0 4px rgba(0, 0, 0, 0.5);
}

.pattern-examples img {
 display: block;
 width: 100%;
 -ms-interpolation-mode: nearest-neighbor;
 image-rendering: optimizeSpeed;
 image-rendering: pixelated;
}

.pattern-examples .closeup {
 display: block;
 width: 90%;
 border-radius: 50%;
 margin-bottom: 12px;
 position: relative;
 left: 5%;
 top: -5px;
 border: 2px solid white;
 overflow: hidden;
 box-shadow: 0 2px 8px rgba(0, 0, 0, 0.3);
}

.pattern-examples figcaption {
 width: 90%;
 margin-left: 5%;
 text-align: center;
}

.pattern-examples .closeup img {
 position: absolute;
 width: 800%;
}

.pattern-examples .closeup:after {
 padding-top: 100%;
 display: block;
 content: '';
}

![](assets/im_sample_dcgan.png)

![](assets/im_sample_dcgan_contrast.png)

Radford, et al., 2015 [1]

![](assets/im_sample_salimans.png)

![](assets/im_sample_salimans.png)

Salimans et al., 2016 [2]

![](assets/im_sample_donahue.png)

![](assets/im_sample_donahue.png)

Donahue, et al., 2016 [3]

![](assets/im_sample_dumoulin.png)

![](assets/im_sample_dumoulin.png)

Dumoulin, et al., 2016 [4]

(function() {

var html = d3.select(".pattern-examples");
var original = html.selectAll(".example .original");
html
 .on("mouseleave", () => {
 console.log("out")
 original.each(resetReticle);
 });

original
 .datum(function() {
 var focus = this.getAttribute("data-focus").split(",")
 return {
 zoom: focus[0],
 x: focus[1],
 y: focus[2]
 };
 })
 .on("mouseleave", resetReticle)
 .on("mousemove", updateReticle)
 .each(resetReticle);

function updateReticle(d) {

 original.each(resetReticle);
 var x = d3.event.offsetX / this.getBoundingClientRect().width;
 var y = d3.event.offsetY / this.getBoundingClientRect().height;
 setPosition(this, x, y, d.zoom, 100);
}

function resetReticle(d) {
 setPosition(this, d.x, d.y, d.zoom, 300);
}

function setPosition(element, x, y, zoom, duration) {
 var margin = 1 / zoom / 2;
 x = Math.min(Math.max(margin, x), 1 - margin)
 y = Math.min(Math.max(margin, y), 1 - margin)
 var parent = d3.select(element.parentElement);
 var reticle = parent.select(".reticle");
 var closeup = parent.select(".closeup img");

 console.log(x, y)

 reticle
 .style("width", 100 / zoom + "%")
 .style("height", 100 / zoom + "%")
 .transition()
 .ease(d3.ease("cubic-out"))
 .duration(duration)
 .styleTween("left", function (d, i, a) {
 var from = this.style.left,
 to = x \* 100 + "%";
 return d3.interpolateString(from, to);
 })
 .styleTween("top", function (d, i, a) {
 var from = this.style.top,
 to = y \* 100 + "%";
 return d3.interpolateString(from, to);
 })

 closeup
 .style("width", zoom \* 100 + "%")
 .style("height", zoom \* 100 + "%")
 .transition()
 .ease(d3.ease("cubic-out"))
 .duration(duration)
 .styleTween("left", function (d, i, a) {
 var from = this.style.left,
 to = -x \* 100 \* zoom + 50 + "%";
 return d3.interpolateString(from, to);
 })
 .styleTween("top", function (d, i, a) {
 var from = this.style.top,
 to = -y \* 100 \* zoom + 50 + "%";
 return d3.interpolateString(from, to);
 })
}

})();

Mysteriously, the checkerboard pattern tends to be most prominent in images with strong colors.
 What’s going on? Do neural networks hate bright colors? The actual cause of these artifacts is actually remarkably simple, as is a method for avoiding them.

---

Deconvolution & Overlap
-----------------------

When we have neural networks generate images, we often have them build them up
 from low resolution, high-level descriptions.
 This allows the network to describe the rough image and then fill in the details.

In order to do this, we need some way to go from a lower resolution image to a higher one.
 We generally do this with the *deconvolution* operation.
 Roughly, deconvolution layers allow the model to use every point
 in the small image to “paint” a square in the larger one.

(Deconvolution has a number of interpretations and different names, including “transposed convolution.”
 We use the name “deconvolution” in this article for brevity.
 For excellent discussion of deconvolution, see [5, 6].)

Unfortunately, deconvolution can easily have “uneven overlap,”
 putting more of the metaphorical paint in some places than others [7].
 In particular, deconvolution has uneven overlap when the kernel size (the output window size) is not divisible by the stride (the spacing between points on the top).
 While the network could, in principle, carefully learn weights to avoid this
  — as we’ll discuss in more detail later —
 in practice neural networks struggle to avoid it completely.

deconv1d();

The overlap pattern also forms in two dimensions.
 The uneven overlaps on the two axes multiply together,
 creating a characteristic checkerboard-like pattern of varying magnitudes.

deconv2d();

In fact, the uneven overlap tends to be more extreme in two dimensions!
 Because the two patterns are multiplied together, the unevenness gets squared.
 For example, in one dimension, a stride 2, size 3 deconvolution has some outputs with twice the number of inputs as others,
 but in two dimensions this becomes a factor of four.

Now, neural nets typically use multiple layers of deconvolution when creating images,
 iteratively building a larger image out of a series of lower resolution descriptions.
 While it’s possible for these stacked deconvolutions to cancel out artifacts,
 they often compound, creating artifacts on a variety of scales.

deconv1d\_multi();

Stride 1 deconvolutions —
 which we often see as the last layer in successful models (eg. [2])
  — are quite effective at dampening artifacts.
 They can remove artifacts of frequencies
 that divide their size, and reduce others artifacts of frequency less than their
 size. However, artifacts can still leak through, as seen in many recent models.

In addition to the high frequency checkerboard-like artifacts we observed above,
 early deconvolutions can create lower-frequency artifacts,
 which we’ll explore in more detail later.

These artifacts tend to be most prominent when outputting unusual colors.
 Since neural network layers typically have a bias
 (a learned value added to the output) it’s easy to output the average color.
 The further a color — like bright red — is away from the average color,
 the more deconvolution needs to contribute.

---

Overlap & Learning
------------------

Thinking about things in terms of uneven overlap is — while a useful framing —
 kind of simplistic. For better or worse, our models learn weights for their deconvolutions.

In theory, our models could learn to carefully write to unevenly overlapping positions so that the output
 is evenly balanced.

![](assets/upsample_LearnedConvUneven.svg)

This is a tricky balancing act to achieve, especially when one has multiple channels interacting.
 Avoiding artifacts significantly restricts the possible filters, sacrificing model capacity.
 In practice, neural networks struggle to learn to completely avoid these patterns.

In fact, not only do models with uneven overlap not learn to avoid this,
 but models with even overlap often learn kernels that cause similar artifacts!
 While it isn’t their default behavior the way it is for uneven overlap,
 it’s still very easy for even overlap deconvolution to cause artifacts.

![](assets/upsample_LearnedConvEven.svg)

Completely avoiding artifacts is still a significant restriction on filters,
 and in practice the artifacts are still present in these models, although they seem milder.
 (See [4],
 which uses stride 2 size 4 deconvolutions, as an example.)

There are probably a lot of factors at play here.
 For example, in the case of Generative Adversarial Networks (GANs), one issue may be the discriminator and its gradients
 (we’ll discuss this more later).
 But a big part of the problem seems to be deconvolution.
 At best, deconvolution is fragile because it very easily represents artifact creating functions, even when the size is carefully chosen.
 At worst, creating artifacts is the default behavior of deconvolution.

Is there a different way to upsample that is more resistant to artifacts?

---

Better Upsampling
-----------------

To avoid these artifacts, we’d like an alternative to regular deconvolution (“transposed convolution”).
 Unlike deconvolution, this approach to upsampling shouldn’t have artifacts as its default behavior.
 Ideally, it would go further, and be biased against such artifacts.

One approach is to make sure you use a kernel size that is divided by your stride,
 avoiding the overlap issue.
 This is equivalent to “sub-pixel convolution,” a technique which has recently
 had success in image super-resolution [8].
 However, while this approach helps, it is still easy for deconvolution to fall into creating artifacts.

Another approach is to separate out upsampling to a higher resolution from convolution to compute features.
 For example, you might resize the image (using [nearest-neighbor interpolation](https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation) or [bilinear interpolation](https://en.wikipedia.org/wiki/Bilinear_interpolation)) and then do a convolutional layer.
 This seems like a natural approach, and roughly similar methods have worked well in image super-resolution (eg. [9]).

![](assets/upsample_DeconvTypes.svg)

Both deconvolution and the different resize-convolution approaches are linear operations, and can be interpreted as matrices.
 This a helpful way to see the differences between them.
 Where deconvolution has a unique entry for each output window, resize-convolution is implicitly weight-tying in a way that discourages high frequency artifacts.

We’ve had our best results with nearest-neighbor interpolation, and had difficulty making bilinear resize work.
 This may simply mean that, for our models, the nearest-neighbor happened to work well with hyper-parameters optimized for deconvolution.
 It might also point at trickier issues with naively using bilinear interpolation, where it resists high-frequency image features too strongly.
 We don’t necessarily think that either approach is the final solution to upsampling, but they do fix the checkerboard artifacts.

### Code

Resize-convolution layers can be easily implemented in TensorFlow using [`tf.image.resize_images()`](https://www.tensorflow.org/versions/r0.11/api_docs/python/image.html#resize_images). For best results, use [`tf.pad()`](https://www.tensorflow.org/versions/r0.11/api_docs/python/array_ops.html#pad) before doing convolution with [`tf.nn.conv2d()`](https://www.tensorflow.org/versions/r0.11/api_docs/python/nn.html#conv2d) to avoid boundary artifacts.

---

Image Generation Results
------------------------

Our experience has been that nearest-neighbor resize followed by a convolution works very well, in a wide variety of contexts.

One case where we’ve found this approach to help is Generative Adversarial Networks. Simply switching out the standard deconvolutional layers for nearest-neighbor resize followed by convolution causes artifacts of different frequencies to disappear.

.fix-examples > .row {
 display: -ms-flexbox;
 display: -webkit-flex;
 display: flex;
 /\*align-items: center;\*/
 margin-bottom: 10px;
}

.fix-examples > .row > .example {
 flex-grow: 1;
 width: 100%;
 margin-right: 10px;
}

.fix-examples > .row > .explanation {
 flex-grow: 2;
 width: 200%;
 margin-left: 10px;
}

.fix-examples img {
 display: block;
 width: 100%;
 -ms-interpolation-mode: nearest-neighbor;
 image-rendering: optimizeSpeed;
 image-rendering: pixelated;
}

![](assets/fix_two_deconv_boat.png)

![](assets/fix_two_deconv_deer.png)

![](assets/fix_two_deconv_prototype_plane.png)

![](assets/fix_two_deconv_bird.png)

 Deconv in last two layers.
 Other layers use resize-convolution.
*Artifacts of frequency 2 and 4.*

![](assets/fix_one_deconv_boat.png)

![](assets/fix_one_deconv_deer.png)

![](assets/fix_one_deconv_prototype_plane.png)

![](assets/fix_one_deconv_bird.png)

 Deconv only in last layer.
 Other layers use resize-convolution.
*Artifacts of frequency 2.*

![](assets/fix_zero_deconv_boat.png)

![](assets/fix_zero_deconv_deer.png)

![](assets/fix_zero_deconv_prototype_plane.png)

![](assets/fix_zero_deconv_bird.png)

 All layers use resize-convolution.
*No artifacts.*

In fact, the difference in artifacts can be seen before any training occurs.
 If we look at the images the generator produces, initialized with random weights,
 we can already see the artifacts:

.fix-step0-examples > .row {
 display: -ms-flexbox;
 display: -webkit-flex;
 display: flex;
 /\*align-items: center;\*/
 margin-bottom: 10px;
}

.fix-step0-examples > .row > .exampleA {
 flex-grow: 1;
 width: 100%;
 margin-right: 10px;
}
.fix-step0-examples > .row > .exampleB {
 flex-grow: 1;
 width: 100%;
 margin-right: 20px;
}

.fix-step0-examples > .row > .explanation {
 flex-grow: 2;
 width: 200%;
}

.fix-step0-examples img {
 display: block;
 width: 100%;
 -ms-interpolation-mode: nearest-neighbor;
 image-rendering: optimizeSpeed;
 image-rendering: pixelated;
}

![](assets/im_rand_two_0.png)

![](assets/im_rand_two_1.png)

![](assets/im_rand_one_0.png)

![](assets/im_rand_one_1.png)

![](assets/im_rand_zero_1.png)

![](assets/im_rand_zero_2.png)

 Deconvolution in last two layers.
 *Artifacts prior to any training.*

 Deconvolution only in last layer.
*Artifacts prior to any training.*

 All layers use resize-convolution.
*No artifacts before or after training.*

This suggests that the artifacts are due to this method of generating images, rather than adversarial training.
 (It also suggests that we might be able to learn a lot about good generator design without the slow feedback cycle of training models.)

Another reason to believe these artifacts aren’t GAN specific is that we see them in other kinds of models, and have found that they also go away when we switch to resize-convolution upsampling.
 For example, consider real-time artistic style transfer [10] where a neural net is trained to directly generate style-transferred images.
 We’ve found these to be vulnerable to checkerboard artifacts (especially when the cost doesn’t explicitly resist them).
 However, switching deconvolutional layers for resize-convolution layers makes the artifacts disappear.

.style-examples > .row {
 display: -ms-flexbox;
 display: -webkit-flex;
 display: flex;
 /\*align-items: center;\*/
 margin-bottom: 10px;
}

.style-examples > .row > .example {
 flex-grow: 4;
 width: 400%;
 margin-right: 10px;
}

.style-examples > .row > .explanation {
 flex-grow: 2;
 width: 200%;
 margin-left: 10px;
}

.style-examples img {
 display: block;
 width: 100%;
 -ms-interpolation-mode: nearest-neighbor;
 image-rendering: optimizeSpeed;
 image-rendering: pixelated;
}

![](assets/style_artifacts.png)

 Using deconvolution.
*Heavy checkerboard artifacts.*

![](assets/style_clean.png)

 Using resize-convolution.
*No checkerboard artifacts.*

Forthcoming papers from the Google Brain team will demonstrate the benefits of this technique
 in more thorough experiments and state-of-the-art results.
 (We’ve chosen to present this technique separately because we felt it merited more detailed discussion, and because it cut across multiple papers.)

---

Artifacts in Gradients
----------------------

Whenever we compute the gradients of a convolutional layer,
 we do deconvolution (transposed convolution) on the backward pass.
 This can cause checkerboard patterns in the gradient,
 just like when we use deconvolution to generate images.

The presence of high-frequency “noise” in image model gradients is
 already known in the feature visualization community, where it’s a major challenge.
 Somehow, feature visualization methods must compensate for this noise.

For example, DeepDream [11]
 seems to cause destructive interference between artifacts in a number of ways,
 such as optimizing many features simultaneously, and optimizing at many offsets and scales.
 In particular, the “jitter” of optimizing at different offsets cancels out some of the checkerboard artifacts.

.deepdream-examples {
 position: relative;
}

.deepdream-examples::after {
 overflow: visible;
 content: "";
 background-image: url("assets/pointer.svg");
 width: 27px;
 height: 27px;
 position: absolute;
 right: -51px;
 top: 0;
}

.deepdream-examples .example {
 position: relative;
}

.deepdream-examples .original {
 position: relative;
 width: 45%;
}

.deepdream-examples .original:after {
 content: "";
 display: block;
 width: 100%;
 height: 70%;
 position: absolute;
 top: -20%;
 z-index: 1;
}

.deepdream-examples .example .reticle {
 position: absolute;
 pointer-events: none;
}

.deepdream-examples .example .reticle:after {
 box-sizing: border-box;
 position: relative;
 width: 100%;
 height: 100%;
 left: -50%;
 top: -50%;
 border-radius: 50%;
 border: 1px solid white;
 display: block;
 content: '';
 background-color: rgba(255, 255, 255, 0.2);
 box-shadow: 0 0 4px rgba(0, 0, 0, 0.5);
}

.deepdream-examples img {
 display: block;
 width: 100%;
 -ms-interpolation-mode: nearest-neighbor;
 image-rendering: optimizeSpeed;
 image-rendering: pixelated;
}

.deepdream-examples .closeup img {
 -webkit-filter: contrast(150%);
 filter: contrast(150%);
}

.deepdream-examples .closeup {
 position: absolute;
 right: 34%;
 top: 0;
 width: 22.5%;
 border-radius: 50%;
 margin-bottom: 12px;
 border: 2px solid white;
 overflow: hidden;
 box-shadow: 0 2px 8px rgba(0, 0, 0, 0.3);
}

.deepdream-examples figcaption {
 position: absolute;
 right: 0%;
 top: 0;
 width: 30%;
}

.deepdream-examples .closeup img {
 position: absolute;
 width: 800%;
}

.deepdream-examples .closeup:after {
 padding-top: 100%;
 display: block;
 content: '';
}

![](assets/deepdream_full_gitter_0x0.png)

![](assets/deepdream_full_gitter_0x0.png)

 DeepDream only applying the neural network to a fixed position.

*Severe artifacts.*

![](assets/deepdream_full_gitter_8x8.png)

![](assets/deepdream_full_gitter_8x8.png)

 DeepDream applying the network to a different position each step.

*Reduced artifacts.*

(function() {

var html = d3.select(".deepdream-examples");
var original = html.selectAll(".example .original");
html
 .on("mouseleave", () => {
 console.log("out")
 original.each(resetReticle);
 });

original
 .datum(function() {
 var focus = this.getAttribute("data-focus").split(",")
 return {
 zoom: focus[0],
 x: focus[1],
 y: focus[2]
 };
 })
 .on("mouseleave", resetReticle)
 .on("mousemove", updateReticle)
 .each(resetReticle);

function updateReticle(d) {

 original.each(resetReticle);
 var x = d3.event.offsetX / this.getBoundingClientRect().width;
 var y = d3.event.offsetY / this.getBoundingClientRect().height;
 setPosition(this, x, y, d.zoom, 100);
}

function resetReticle(d) {
 setPosition(this, d.x, d.y, d.zoom, 300);
}

function setPosition(element, x, y, zoom, duration) {
 var marginX = 1 / zoom / 4;
 var marginY = 1 / zoom / 2;
 x = Math.min(Math.max(marginX, x), 1 - marginX)
 y = Math.min(Math.max(marginY, y), 1 - marginY)
 var parent = d3.select(element.parentElement);
 var reticle = parent.select(".reticle");
 var closeup = parent.select(".closeup img");

 reticle
 .style("width", 50 / zoom + "%")
 .style("height", 100 / zoom + "%")
 .transition()
 .ease(d3.ease("cubic-out"))
 .duration(duration)
 .styleTween("left", function (d, i, a) {
 var from = this.style.left,
 to = x \* 100 + "%";
 return d3.interpolateString(from, to);
 })
 .styleTween("top", function (d, i, a) {
 var from = this.style.top,
 to = y \* 100 + "%";
 return d3.interpolateString(from, to);
 })

 closeup
 .style("width", zoom \* 200 + "%")
 .style("height", zoom \* 100 + "%")
 .transition()
 .ease(d3.ease("cubic-out"))
 .duration(duration)
 .styleTween("left", function (d, i, a) {
 var from = this.style.left,
 to = -x \* 200 \* zoom + 50 + "%";
 return d3.interpolateString(from, to);
 })
 .styleTween("top", function (d, i, a) {
 var from = this.style.top,
 to = -y \* 100 \* zoom + 50 + "%";
 return d3.interpolateString(from, to);
 })
}

})();

(While some of the artifacts are our standard checkerboard pattern,
 others are a less organized high-frequency pattern.
 We believe these to be caused by max pooling.
 Max pooling was previously linked to high-frequency artifacts in [12].)

More recent work in feature visualization (eg. [13]),
 has explicitly recognized and compensated for these high-frequency gradient components.
 One wonders if better neural network architectures could make these efforts unnecessary.

Do these gradient artifacts affect GANs?
 If gradient artifacts can affect an image being optimized based on a neural networks gradients in feature visualization,
 we might also expect it to affect the family of images parameterized by the generator as they’re optimized by the discriminator in GANs.

We’ve found that this does happen in some cases.
 When the generator is neither biased for or against checkerboard patterns,
 strided convolutions in the discriminator can cause them.

.discrim-examples > .row {
 display: -ms-flexbox;
 display: -webkit-flex;
 display: flex;
 /\*align-items: center;\*/
 margin-bottom: 10px;
}

.discrim-examples > .row > .exampleA {
 flex-grow: 1;
 width: 100%;
 margin-right: 10px;
}
.discrim-examples > .row > .exampleB {
 flex-grow: 1;
 width: 100%;
 margin-right: 20px;
}

.discrim-examples > .row > .explanationA {
 flex-grow: 3;
 width: 300%;
 padding-right: 20px;
}

.discrim-examples > .row > .explanationB {
 flex-grow: 3;
 width: 300%;
 padding-right: 10px;
}

.discrim-examples img {
 display: block;
 width: 100%;
 -ms-interpolation-mode: nearest-neighbor;
 image-rendering: optimizeSpeed;
 image-rendering: pixelated;
}

![](assets/discrim_stride2_0.png)

![](assets/discrim_stride2_1.png)

![](assets/discrim_stride2_2.png)

![](assets/discrim_stride1_1.png)

![](assets/discrim_stride1_2.png)

![](assets/discrim_stride1_0.png)

 Discriminator has stride 2 convolution in first layer.
 *Strong frequency 2 artifacts.*

 Discriminator has regular convolution in first layer.
 *Very mild artifacts.*

It’s unclear what the broader implications of these gradient artifacts are.
 One way to think about them is that some neurons will get many times the gradient of their neighbors, basically arbitrarily.
 Equivalently, the network will care much more about some pixels in the input than others, for no good reason.
 Neither of those sounds ideal.

It seems possible that having some pixels affect the network output much more than others may exaggerate adversarial counter-examples.
 Because the derivative is concentrated on small number of pixels,
 small perturbations of those pixels may have outsized effects.
 We have not investigated this.

---

Conclusion
----------

The standard approach of producing images with deconvolution — despite its successes! — has some conceptually simple issues that lead to artifacts in produced images.
 Using a natural alternative without these issues causes the artifacts to go away
 (Analogous arguments suggest that standard strided convolutional layers may also have issues).

This seems like an exciting opportunity to us!
 It suggests that there is low-hanging fruit to be found in carefully thinking through neural network architectures, even ones where we seem to have clean working solutions.

In the meantime, we’ve provided an easy to use solution that improves the quality of many approaches to generating images with neural networks. We look forward to seeing what people do with it, and whether it helps in domains like audio, where high frequency artifacts would be particularly problematic.


How to Use t-SNE Effectively

How to Use t-SNE Effectively
============================

Although extremely useful for visualizing high-dimensional data, t-SNE plots can sometimes be mysterious or misleading. By exploring how it behaves in simple cases, we can learn to use it more effectively.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

 #playground {
 overflow: hidden;
 font-family: 'Open Sans', sans-serif;
 border-top: 1px solid rgba(0, 0, 0, 0.1);
 /\*border-bottom: 1px solid rgba(0, 0, 0, 0.1);\*/
 margin-top: 36px;
 padding: 36px 0 0 0;
 /\*background: #fcfcfc;\*/
 z-index: 1000;
 }

 #playground \* {
 box-sizing: border-box;
 }

 #playground.modal {
 position: fixed;
 left: 10px;
 right: 10px;
 top: 50px;
 }

 /\* Playground Canvas \*/
 #playground-canvas {
 float: left;
 width: 55%;
 }

 #playground-canvas canvas {
 width: 100%;
 }

 /\* Data Menu \*/
 #data-menu {
 /\*float: left;\*/
 /\*width: 25%\*/
 width: 40%;
 float: left;
 margin-bottom: 24px;
 overflow: hidden;
 margin-left: 5%;
 }
 #data-menu .demo-data {
 cursor: pointer;
 position: relative;
 font-size: 10px;
 line-height: 1.2em;
 box-sizing: border-box;
 float: left;
 margin: 2px;
 padding: 4px;
 width: calc(33% - 4px);
 background: white;
 border: 1px solid rgba(0, 0, 0, 0.1);
 border-radius: 4px;
 box-shadow: 0 0 3px rgba(0, 0, 0, 0.08);
 }
 @media(min-width: 480px) {
 #data-menu .demo-data {
 width: calc(25% - 8px);
 padding: 8px;
 margin: 4px;
 }
 }
 @media(min-width: 768px) {
 #data-menu .demo-data {
 width: calc(16.5% - 8px);
 padding: 8px;
 margin: 4px;
 }
 }
 #data-menu .demo-data:hover {
 border: 1px solid rgba(0, 0, 0, 0.2);

 }
 #data-menu .demo-data.selected::after {
 content: "";
 border: 2px solid rgba(70, 130, 180, 0.8);
 width: 100%;
 height: 100%;
 position: absolute;
 top: 0;
 left: 0;
 box-sizing: border-box;
 border-radius: 4px;
 }
 #data-menu .demo-data span {
 display: none;
 }
 #data-menu .demo-data:hover canvas {
 opacity: 1;
 }
 #data-menu .demo-data canvas {
 width: 100%;
 opacity: 0.3;
 }
 #data-menu .demo-data.selected canvas {
 opacity: 1;
 }

 /\* Data Details \*/
 #data-details {
 position: relative;
 }

 @media(min-width: 768px) {
 #data-details {
 width: 40%;
 float: right;
 }
 }

 #data-details #data-controls {
 width: 40%;
 float: right;
 position: relative;
 overflow: hidden;
 font-size: 13px;
 }

 @media(min-width: 768px) {
 #data-details #data-controls {
 width: 50%;
 margin-right: 10%;
 float: left;
 }
 }

 #data-details #play-controls {
 margin-bottom: 18px;
 overflow: hidden;
 position: relative;
 }
 #data-details #play-controls button {
 cursor: pointer;
 outline: none;
 border-radius: 50%;
 background: steelblue;
 color: white;
 width: 25%;
 margin-right: 5%;
 padding-top: 25%;
 padding-bottom: 0;
 border: none;
 float: left;
 position: relative;
 }
 #play-controls i {
 display: block;
 position: absolute;
 top: 50%;
 left: 0;
 width: 100%;
 height: 36px;
 font-size: 24px;
 line-height: 0;
 }
 @media(min-width: 768px) {
 #play-controls i {
 font-size: 30px;
 }
 }
 #play-controls #play-pause i {
 display: none;
 position: absolute;
 }
 #play-controls #play-pause.paused i:nth-child(1) {
 display: block;
 }
 #play-controls #play-pause.playing i:nth-child(2) {
 display: block;
 }

 #steps-display {
 float: left;
 text-align: center;
 width: 25%;
 line-height: 1.5em;
 font-size: 13px;
 }

 @media(min-width: 1024px) {
 #steps-display {
 font-size: 16px;
 line-height: 1.6em;
 }
 }
 #data-details #data-description {
 width: 50%;
 margin-right: 10%;
 float: right;
 font-size: 14px;
 line-height: 1.6em;
 }

 @media(min-width: 768px) {
 #data-details #data-description {
 width: 40%;
 float: left;
 margin-right: 0;
 }
 }

 /\* Options \*/
 #data-details #options {
 float: left;
 padding-left: 36px;
 font-size: 13px;
 line-height: 1.5em;
 width: 45%;
 }
 #data-details input {
 display: block;
 width: 100%;
 margin: 8px 0 16px 0;
 }

 #options #data-options {
 width: 45%;
 margin-left: 4px;
 }
 #options #tsne-options {
 width: 45%;
 margin-left: 4px;
 }

 #data-details #share {
 margin-top: 8px;
 display: block;
 color: rgba(0, 0, 0, 0.4);
 text-decoration: none;
 font-size: 12px;
 }

 #data-details #share:hover {
 text-decoration: underline;
 }

 #data-details #share i {
 line-height: 0px;
 position: relative;
 top: 7px;
 }

*play\_arrow**pause*
*refresh*

 Step

*link* Share this view

 dt-byline {
 font-size: 12px;
 line-height: 18px;
 display: block;
 border-top: 1px solid rgba(0, 0, 0, 0.1);
 border-bottom: 1px solid rgba(0, 0, 0, 0.1);
 color: rgba(0, 0, 0, 0.5);
 padding-top: 12px;
 padding-bottom: 12px;
 }
 dt-article.centered dt-byline {
 text-align: center;

 }
 dt-byline a,
 dt-article dt-byline a {
 text-decoration: none;
 border-bottom: none;
 }
 dt-article dt-byline a:hover {
 text-decoration: underline;
 border-bottom: none;
 }
 dt-byline .authors {
 text-align: left;
 }
 dt-byline .name {
 display: inline;
 text-transform: uppercase;
 }
 dt-byline .affiliation {
 display: inline;
 }
 dt-byline .date {
 display: block;
 text-align: left;
 }
 dt-byline .year, dt-byline .month {
 display: inline;
 }
 dt-byline .citation {
 display: block;
 text-align: left;
 }
 dt-byline .citation div {
 display: inline;
 }

 @media(min-width: 768px) {
 dt-byline {
 }
 }

 @media(min-width: 1080px) {
 dt-byline {
 border-bottom: none;
 margin-bottom: 70px;
 }

 dt-byline a:hover {
 color: rgba(0, 0, 0, 0.9);
 }

 dt-byline .authors {
 display: inline-block;
 }

 dt-byline .author {
 display: inline-block;
 margin-right: 12px;
 /\*padding-left: 20px;\*/
 /\*border-left: 1px solid #ddd;\*/
 }

 dt-byline .affiliation {
 display: block;
 }

 dt-byline .author:last-child {
 margin-right: 0;
 }

 dt-byline .name {
 display: block;
 }

 dt-byline .date {
 border-left: 1px solid rgba(0, 0, 0, 0.1);
 padding-left: 15px;
 margin-left: 15px;
 display: inline-block;
 }
 dt-byline .year, dt-byline .month {
 display: block;
 }

 dt-byline .citation {
 border-left: 1px solid rgba(0, 0, 0, 0.15);
 padding-left: 15px;
 margin-left: 15px;
 display: inline-block;
 }
 dt-byline .citation div {
 display: block;
 }
 }

[Martin Wattenberg](http://hint.fm/)
[Google Brain](http://g.co/brain)

[Fernanda Viégas](http://hint.fm/)
[Google Brain](http://g.co/brain)

[Ian Johnson](http://enjalot.github.io/)
[Google Cloud](http://cloud.google.com)

Oct. 13
2016

[Citation:
Wattenberg, et al., 2016](#citation)

A popular method for exploring high-dimensional data is something called t-SNE, introduced by van der Maaten and Hinton in 2008 [1]. The technique has become widespread in the field of machine learning, since it has an almost magical ability to create compelling two-dimensonal “maps” from data with hundreds or even thousands of dimensions.
Although impressive, these images can be tempting to misread. The purpose of this note is to prevent some common misreadings.

We’ll walk through a series of simple examples to illustrate what t-SNE diagrams can and cannot show. The t-SNE technique really is useful—but only if you know how to interpret it.

Before diving in: if you haven’t encountered t-SNE before, here’s what you need to know about the math behind it. The goal is to take a set of points in a high-dimensional space and find a faithful representation of those points in a lower-dimensional space, typically the 2D plane. The algorithm is non-linear and adapts to the underlying data, performing different transformations on different regions. Those differences can be a major source of confusion.

A second feature of t-SNE is a tuneable parameter, “perplexity,” which says (loosely) how to balance attention between local and global aspects of your data. The parameter is, in a sense, a guess about the number of close neighbors each point has. The perplexity value has a complex effect on the resulting pictures. The original paper says, *“The performance of SNE is fairly robust to changes in the perplexity, and typical values are between 5 and 50.”* But the story is more nuanced than that. Getting the most from t-SNE may mean analyzing multiple plots with different perplexities.

That’s not the end of the complications. The t-SNE algorithm doesn’t always produce similar output on successive runs, for example, and there are additional hyperparameters related to the optimization process.

---

1. Those hyperparameters really matter
--------------------------------------

Let’s start with the “hello world” of t-SNE: a data set of two widely separated clusters. To make things as simple as possible, we’ll consider clusters in a 2D plane, as shown in the lefthand diagram. (For clarity, the two clusters are color coded.) The diagrams at right show t-SNE plots for five different perplexity values.

With perplexity values in the range (5 - 50) suggested by van der Maaten & Hinton, the diagrams do show these clusters, although with very different shapes. Outside that range, things get a little weird. With perplexity 2, local variations dominate. The image for perplexity 100, with merged clusters, illustrates a pitfall: for the algorithm to operate properly, the perplexity really should be smaller than the number of points. Implementations can give unexpected behavior otherwise.

Each of the plots above was made with 5,000 iterations with a learning rate (often called “epsilon”) of 10, and had reached a point of stability by step 5,000. How much of a difference do those values make? In our experience, the most important thing is to iterate until reaching a stable configuration.

The images above show five different runs at perplexity 30. The first four were stopped before stability. After 10, 20, 60, and 120 steps you can see layouts with seeming 1-dimensional and even pointlike images of the clusters. If you see a t-SNE plot with strange “pinched” shapes, chances are the process was stopped too early. Unfortunately, there’s no fixed number of steps that yields a stable result. Different data sets can require different numbers of iterations to converge.

Another natural question is whether different runs with the same hyperparameters produce the same results. In this simple two-cluster example, and most of the others we discuss, multiple runs give the same global shape. Certain data sets, however, yield markedly different diagrams on different runs; we’ll give an example of one of these later.

From now on, unless otherwise stated, we’ll show results from 5,000 iterations. That’s generally enough for convergence in the (relatively small) examples in this essay. We’ll keep showing a range of perplexities, however, since that seems to make a big difference in every case.

---

2. Cluster sizes in a t-SNE plot mean nothing
---------------------------------------------

So far, so good. But what if the two clusters have different standard deviations, and so different sizes? (By size we mean bounding box measurements, not number of points.) Below are t-SNE plots for a mixture of Gaussians in plane, where one is 10 times as dispersed as the other.

Surprisingly, the two clusters look about same size in the t-SNE plots.
 What’s going on? The t-SNE algorithm adapts its notion of “distance” to regional density variations in the data set. As a result, it naturally expands dense clusters, and contracts sparse ones, evening out cluster sizes. To be clear, this is a different effect than the run-of-the-mill fact that any dimensionality reduction technique will distort distances. (After all, in this example all data was two-dimensional to begin with.) Rather, density equalization happens by design and is a predictable feature of t-SNE.

The bottom line, however, is that you cannot see relative sizes of clusters in a t-SNE plot.

---

3. Distances between clusters might not mean anything
-----------------------------------------------------

What about distances *between* clusters? The next diagrams show three Gaussians of 50 points each, one pair being 5 times as far apart as another pair.

At perplexity 50, the diagram gives a good sense of the global geometry. For lower perplexity values the clusters look equidistant. When the perplexity is 100, we see the global geometry fine, but one of the cluster appears, falsely, much smaller than the others.
 Since perplexity 50 gave us a good picture in this example, can we always set perplexity to 50 if we want to see global geometry?

Sadly, no. If we add more points to each cluster, the perplexity has to increase to compensate. Here are the t-SNE diagrams for three Gaussian clusters with 200 points each, instead of 50. Now none of the trial perplexity values gives a good result.

It’s bad news that seeing global geometry requires fine-tuning perplexity. Real-world data would probably have multiple clusters with different numbers of elements. There may not be one perplexity value that will capture distances across all clusters—and sadly perplexity is a global parameter. Fixing this problem might be an interesting area for future research.

The basic message is that distances between well-separated clusters in a t-SNE plot may mean nothing.

---

4. Random noise doesn’t always look random.
-------------------------------------------

A classic pitfall is thinking you see patterns in what is really just random data. Recognizing noise when you see it is a critical skill, but it takes time to build up the right intuitions. A tricky thing about t-SNE is that it throws a lot of existing intuition out the window.
 The next diagrams show genuinely random data, 500 points drawn from a unit Gaussian distribution in 100 dimensions. The left image is a projection onto the first two coordinates.

The plot with perplexity 2 seems to show dramatic clusters. If you were tuning perplexity to bring out structure in the data, you might think you’d hit the jackpot.

Of course, since we know the cloud of points was generated randomly, it has no statistically interesting clusters: those “clumps” aren’t meaningful. If you look back at previous examples, low perplexity values often lead to this kind of distribution. Recognizing these clumps as random noise is an important part of reading t-SNE plots.

There’s something else interesting, though, which may be a win for t-SNE. At first the perplexity 30 plot doesn’t look like a Gaussian distribution at all: there’s only a slight density difference across different regions of the cloud, and the points seem suspiciously evenly distributed. In fact, these features are saying useful things about high-dimensional normal distributions, which are very close to uniform distributions on a sphere: evenly distributed, with roughly equal spaces between points. Seen in this light, the t-SNE plot is more accurate than any linear projection could be.

---

5. You can see some shapes, sometimes
-------------------------------------

It’s rare for data to be distributed in a perfectly symmetric way. Let’s take a look at an axis-aligned Gaussian distribution in 50 dimensions, where the standard deviation in coordinate i is 1/i. That is, we’re looking at a long-ish ellipsoidal cloud of points.

For high enough perplexity values, the elongated shapes are easy to read. On the other hand, at low perplexity, local effects and meaningless “clumping” take center stage. More extreme shapes also come through, but again only at the right perplexity. For example, here are two clusters of 75 points each in 2D, arranged in parallel lines with a bit of noise.

For a certain range of perplexity the long clusters look close to correct, which is reassuring.

Even in the best cases, though, there’s a subtle distortion: the lines are slightly curved outwards in the t-SNE diagram. The reason is that, as usual, t-SNE tends to expand denser regions of data. Since the middles of the clusters have less empty space around them than the ends, the algorithm magnifies them.

---

6. For topology, you may need more than one plot
------------------------------------------------

Sometimes you can read topological information off a t-SNE plot, but that typically requires views at multiple perplexities.
 One of the simplest topological properties is containment. The plots below show two groups of 75 points in 50 dimensional space. Both are sampled from symmetric Gaussian distributions centered at the origin, but one is 50 times more tightly dispersed than the other. The “small” distribution is in effect contained in the large one.

The perplexity 30 view shows the basic topology correctly, but again t-SNE greatly exaggerates the size of the smaller group of points. At perplexity 50, there’s a new phenomenon: the outer group becomes a circle, as the plot tries to depict the fact that all its points are about the same distance from the inner group. If you looked at this image alone, it would be easy to misread these outer points as a one-dimensional structure.

What about more complicated types of topology? This may be a subject dearer to mathematicians than to practical data analysts, but interesting low-dimensional structures are occasionally found in the wild.

Consider a set of points that trace a link or a knot in three dimensions. Once again, looking at multiple perplexity values gives the most complete picture. Low perplexity values give two completely separate loops; high ones show a kind of global connectivity.

The trefoil knot is an interesting example of how multiple runs affect the outcome of t-SNE. Below are five runs of the perplexity-2 view.

The algorithm settles twice on a circle, which at least preserves the intrinsic topology. But in three of the runs it ends up with three different solutions which introduce artificial breaks. Using the dot color as a guide, you can see that the first and third runs are far from each other.

Five runs at perplexity 50, however, give results that (up to symmetry) are visually identical. Evidently some problems are easier than others to optimize.

---

Conclusion
----------

There’s a reason that t-SNE has become so popular: it’s incredibly flexible, and can often find structure where other dimensionality-reduction algorithms cannot. Unfortunately, that very flexibility makes it tricky to interpret. Out of sight from the user, the algorithm makes all sorts of adjustments that tidy up its visualizations.
 Don’t let the hidden “magic” scare you away from the whole technique, though. The good news is that by studying how t-SNE behaves in simple cases, it’s possible to develop an intuition for what’s going on.

.tsne-group {
 overflow: visible;
 display: -webkit-flex;
 display: flex;
 flex-direction: column;
 margin-top: 36px;
 margin-bottom: 36px;
}

@media(min-width: 640px) {
 .tsne-group {
 -webkit-flex-direction: row;
 flex-direction: row;
 }
}

.tsne-group .original,
.tsne-group .runner {
 box-sizing: border-box;
 -webkit-flex-grow: 1;
 flex-grow: 1;
 position: relative;
 display: -webkit-flex;
 display: flex;
 margin-bottom: 12px;
}

.tsne-group .original {
 position: relative;
 border-bottom: 1px solid rgba(0, 0, 0, 0.1);
 padding-bottom: 12px;
}

@media(min-width: 640px) {
 .tsne-group .original,
 .tsne-group .runner {
 display: block;
 width: 100%;
 margin-bottom: 0;
 }

 .tsne-group .original {
 margin-left: 0;
 padding-right: 12px;
 border-right: 1px solid rgba(0, 0, 0, 0.1);
 border-bottom: none;
 padding-bottom: 0;
 }

 .tsne-group .runner {
 margin-left: 12px;
 }
}

.tsne-group .runner.no-click {
 cursor: default;
}

.runner.clickable {
 cursor: pointer;
}

.runner.clickable:hover .image {
 box-shadow: 0 2px 12px rgba(0, 0, 0, 0.08);
}

.runner:hover i {
 opacity: 1;
}

.runner.selected i {
 opacity: 1;
}

.runner i {
 font-size: 30px;
 position: absolute;
 top: 1px;
 left: 1px;
 background-color: hsla(207, 44%, 79%, 0.8);
 color: steelblue;
 border-radius: 4px;
 opacity: 0;
 transition: opacity 0.3s;
 line-height: 0;
 width: 150px;
 padding: 75px 0;
 text-align: center;
}

@media(min-width: 640px) {
 .runner i {
 width: 100%;
 padding: 50% 0;
 }
}

.runner.selected .image::after {
 content: "";
 border: 2px solid rgba(70, 130, 180, 1);
 width: 100%;
 height: 100%;
 position: absolute;
 top: 0;
 box-sizing: border-box;
 border-radius: 4px;
 z-index: 10;
}

.tsne-group .image {
 border: 1px solid rgba(0, 0, 0, 0.1);
 border-radius: 4px;
 box-shadow: 0 0 6px rgba(0, 0, 0, 0.08);
 width: 150px;
 padding-top: 150px;
 background: white;
 position: relative;
 transition: border-color 0.3s, box-shadow 0.3s;
 margin-right: 12px;
}

@media(min-width: 640px) {
 .tsne-group .image {
 width: 100%;
 padding-top: 100%;
 margin-right: 0;
 }
}

.tsne-group .original .image {
 box-shadow: none;
}

.tsne-group .image img {
 position: absolute;
 top: 5%;
 left: 5%;
 width: 90%;
}

.tsne-group h3.caption {
 font-size: 15px;
 margin-top: 12px;
}

.tsne-group .caption {
 font-size: 12px;
 line-height: 1.6em;
 margin-top: 12px;
 width: 100px;
}

@media(min-width: 640px) {
 .tsne-group .caption,
 .tsne-group h3.caption {
 margin-top: 8px;
 width: 100%;
 }
}

(function() {
 var format = d3.format(",");
 d3.selectAll(".tsne-group").data(figures, function(d) {
 if(!d) return this.dataset.id;
 return d.id
 })
 .each(generateFigure)

 function generateFigure(figure) {

 var FIGURE = figure.id
 var dis = d3.select(this)
 var original = dis.append("div").classed("original", true)
 original.append("div")
 .classed("image", true)
 .append("img")
 .classed("tsne-plot", true)
 .attr("src", "assets/figure\_" + FIGURE + "\_" + "original.png");
 original.append("h3").classed("caption", true).text("Original")

 // examples
 var runners = dis.selectAll(".runner").data(figure.examples)
 .enter().append("div").classed("runner", true)
 if(!figure.noclick) {
 runners
 .classed("clickable", true)
 .on("click", function(d) {
 d3.selectAll(".runner").classed("selected", false)
 updateStateFromFigure(figure, d, this);
 d3.select(this).classed("selected", true)
 });
 } else {
 runners.classed("no-click", true)
 }

 runners.append("div")
 .classed("image", true)
 .append("img")
 .attr("src", function(d,i) {
 return "assets/figure\_" + FIGURE + "\_" + d.id + ".png"
 });

 if(!figure.noclick) {
 runners.append("i")
 .classed("material-icons", true)
 .text("open\_in\_browser")
 }

 var caption = runners.append("div").classed("caption", true);

 caption.append("div").text(function(d) { return "Perplexity: " + format(d.perplexity); })

 caption.append("div").text(function(d) { return "Step: " + format(d.step); })
 }
})()