Repository: pytorch/vision

Issue:
Add examples of transformation to docs
# Problem
`torchvision` has quite a few transforms, but I often find myself using the wrong one because the I misread/just skimmed the doc strings.

# Solution
I'm suggesting to add examples / visualizations for each transform. This way it's easier to know what a given transform does.

After this improvement there should be an example (maybe a few where it makes sense) for each transform in the docs:
https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision-transforms

Also, a picture is worth a thousand words ;)

What do you think?


Discussion:
I think this is a great idea!
Just a few comments:
We could use the`"matplotlib.sphinxext.plot_directive"` extension from [here](https://matplotlib.org/3.1.1/devel/plot_directive.html).
Then for each transform something like this would do the trick and render a nice little plot: 
```rst
.. plot::
   :include-source:

    import matplotlib.pyplot as plt
    import torch
    from torch.transforms import CenterCrop

    transform = CenterCrop(32)
    x = torch.rand(3, 64, 64)
    x = transform(x)
    plt.imshow(x.permute(1, 2, 0))
```



This seems pretty clean and extensible! And also, updating the function behavior would be very simple.

I'd recommend fixing the random seed for each plot, to avoid regenerating a new image every time the docs are generated.
Great, I'm currently experimenting with something similar. I'll finish that and then create a PR .

I'm not sure if setting the seed prevents regeneration of the image. I'll check.
It will probably regenerate the images, but they might be almost identical, so the diff size would be fairly small?
I'll run a bunch of experiments and get back to you.
@sotte , Can I please pick this up? I am interested to contribute to PyTorch and this could be a good starting point.

Thanks
**Proposal:**
In order to reduce boilerplate code in the examples for the transforms add two helper functions:
- Add `example_image()` helper function to `torchvision.utils` that loads the Grace Hopper image which is already used in tests.
- Add `plot_images()` helper function to `torchvision.utils` similar to make_grid().

With the helpers in place it's quite easy to add interesting examples for each transform.

**Open Questions:**
Should we document TF, the transform classes or both? I have the impression that not many people know about TF. If we only add examples to TF we don't have to set the seed.
If we only add examples to TF xor transform classes, then each Transform class should link/ref the corresponding TF and the other way around so that it's easier to see the examples.
So should we document TF, transform classes, or both?

**Demo:**
This is what the code would look like:
```python
from torchvision.transforms import ColorJitter
from torchvision.utils import plot_images, sample_image

transform = ColorJitter(brightness=0.5)
plot_images(*[transform(sample_image()) for _ in range(5)])

transform = ColorJitter(contrast=0.5)
plot_images(*[transform(sample_image()) for _ in range(5)])

transform = ColorJitter(saturation=0.5)
plot_images(*[transform(sample_image()) for _ in range(5)])

transform = ColorJitter(hue=0.1)
plot_images(*[transform(sample_image()) for _ in range(5)])
```

Just to give you an idea how the docs would look like  (still with duplicate code):
![Screenshot from 2019-10-05 12-01-57 pytorch_transform_colorjitter](https://user-images.githubusercontent.com/79138/66256771-89390380-e791-11e9-947e-39dea53afda1.png)
![Screenshot from 2019-10-05 12-02-35 pytorch_transform_fivecrop](https://user-images.githubusercontent.com/79138/66256773-8b02c700-e791-11e9-9504-6e04e88e0fb1.png)

Misc notes:
- Add `matplotlib` to `docs/requirements.txt`.
- Check seed for random transforms.
- I added **Example** in front of the code for visualizing the transform. It's not on the same level as the **Example** form the doc strings. I think this is fine.

@PyExtreme sorry, I just saw your comment. I'm already in the process of implementing this :( I'm sure we can find you another ticket to get you started contributing to pytorch.
@sotte , please do let me know so that I can get started with it.

Thanks
@sotte that's a great proposal!

Here are my thoughts:

### `example_image()` and `plot_images()` in `torchvision.utils`

What about naming them with an underscore in the beginning to indicate that they are private? Also, the Grace Hopper image is available under `test/assets` I believe, which is not installed with torchvision, so this would mean that we would need to move the image somewhere else. My thinking is that those two functions are very specific to the examples, and thus having them live in `torchvision.utils` as being public functions might not be best.

### Images for TF or transforms, or both?
I think starting with `transforms` would be the simplest, because we wouldn't need to manually find parameters for the transformations (and write it there).

In another node, after having another look at the transforms page in torchvision, I think we should split the page in two, and have one page dedicated for the functional transforms, and one dedicated for the module interface.
I created the PR #1426.
fyi transformations overview or list section on the top/side menu could be handy too. Examples/Images are the best only its slightly more PR effort... Currently to view all transformation options i had to scroll through the whole page:(
I was only interested in the interface/method spec dont care so much about details params 

eg 
- RandomJitter
- RandomPerspective
- Compose
...
May I suggest to use [sphinx-gallery](https://sphinx-gallery.github.io/stable/index.html) for this?

It is used by many mature projects already: scikit-learn, matplotlib, scikit-image...

It allows to have a dedicated [example gallery](https://scikit-learn.org/stable/auto_examples/index.html), and the examples can have plots. The examples can also be written in a [notebook-style](https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_0_23_0.html#sphx-glr-auto-examples-release-highlights-plot-release-highlights-0-23-0-py) for didactic purposes.

The plots can then be [embedded in the user guide](https://scikit-learn.org/stable/modules/linear_model.html#regression) or any `rst` file. `sphinx-gallery` can also automatically add [backlinks](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html#examples-using-sklearn-linear-model-ridge) to a docstring, where are listed all the examples that make use of a given class / function.

I'm happy to submit a PR to integrate `sphinx-gallery` if you think it's worth it @fmassa 

Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -20,6 +20,10 @@
 # import os
 # import sys
 # sys.path.insert(0, os.path.abspath('.'))
+
+from pathlib import Path
+import os
+
 import torch
 import torchvision
 import pytorch_sphinx_theme
@@ -44,8 +48,16 @@
     'sphinx.ext.mathjax',
     'sphinx.ext.napoleon',
     'sphinx.ext.viewcode',
+    'sphinx_gallery.gen_gallery',
 ]
 
+sphinx_gallery_conf = {
+    'examples_dirs': '../../gallery/',   # path to your example scripts
+    'gallery_dirs': 'auto_examples',  # path to where to save gallery generated output
+    'backreferences_dir': 'gen_modules/backreferences',
+    'doc_module': ('torchvision',),
+}
+
 napoleon_use_ivar = True
 napoleon_numpy_docstring = False
 napoleon_google_docstring = True
@@ -248,3 +260,42 @@ def handle_item(fieldarg, content):
 
 
 TypedField.make_field = patched_make_field
+
+
+def inject_minigalleries(app, what, name, obj, options, lines):
+    """Inject a minigallery into a docstring.
+
+    This avoids having to manually write the .. minigallery directive for every item we want a minigallery for,
+    as it would be easy to miss some.
+
+    This callback is called after the .. auto directives (like ..autoclass) have been processed,
+    and modifies the lines parameter inplace to add the .. minigallery that will show which examples
+    are using which object.
+
+    It's a bit hacky, but not *that* hacky when you consider that the recommended way is to do pretty much the same,
+    but instead with templates using autosummary (which we don't want to use):
+    (https://sphinx-gallery.github.io/stable/configuration.html#auto-documenting-your-api-with-links-to-examples)
+
+    For docs on autodoc-process-docstring, see the autodoc docs:
+    https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html
+    """
+
+    conf_file_path = Path(__file__).parent.absolute()
+    backrefs_path = conf_file_path / sphinx_gallery_conf['backreferences_dir'] / (name + '.examples')
+    if not (os.path.isfile(backrefs_path) and os.path.getsize(backrefs_path) > 0):
+        # We avoid showing the (empty) minigallery if there's nothing to show, i.e. if the object
+        # isn't used in any example.
+        # FIXME: this check can be removed once https://github.com/sphinx-gallery/sphinx-gallery/pull/813
+        # is merged and the new sphinx-gallery version (> 0.8.x) is released.
+        return
+
+    if what in ("class", "function"):
+        lines.append(f".. minigallery:: {name}")
+        lines.append(f"    :add-heading: Examples using ``{name.split('.')[-1]}``:")
+        # avoid heading entirely to avoid warning. As a bonud it actually renders better
+        lines.append("    :heading-level: 9")
+        lines.append("\n")
+
+
+def setup(app):
+    app.connect('autodoc-process-docstring', inject_minigalleries)
diff --git a/gallery/plot_transforms.py b/gallery/plot_transforms.py
new file mode 100644
--- /dev/null
+++ b/gallery/plot_transforms.py
@@ -0,0 +1,64 @@
+"""
+==========================
+Illustration of transforms
+==========================
+
+This example illustrates the various transforms available in :mod:`torchvision.transforms`.
+"""
+
+from PIL import Image
+from pathlib import Path
+import matplotlib.pyplot as plt
+import numpy as np
+
+import torchvision.transforms as T
+
+
+orig_img = Image.open(Path('assets') / 'astronaut.jpg')
+
+
+def plot(img, title="", with_orig=True, **kwargs):
+    def _plot(img, title, **kwargs):
+        plt.figure().suptitle(title, fontsize=25)
+        plt.imshow(np.asarray(img), **kwargs)
+        plt.axis('off')
+
+    if with_orig:
+        _plot(orig_img, "Original image")
+    _plot(img, title, **kwargs)
+
+
+####################################
+# Pad
+# ---
+# The :class:`~torchvision.transforms.Pad` transform
+# (see also :func:`~torchvision.transforms.functional.pad`)
+# fills image borders with some pixel values.
+padded_img = T.Pad(padding=30)(orig_img)
+plot(padded_img, "Padded image")
+
+####################################
+# Resize
+# ------
+# The :class:`~torchvision.transforms.Resize` transform
+# (see also :func:`~torchvision.transforms.functional.resize`)
+# resizes an image.
+resized_img = T.Resize(size=30)(orig_img)
+plot(resized_img, "Resized image")
+
+####################################
+# ColorJitter
+# -----------
+# The :class:`~torchvision.transforms.ColorJitter` transform
+# randomly changes the brightness, saturation, and other properties of an image.
+jitted_img = T.ColorJitter(brightness=.5, hue=.3)(orig_img)
+plot(jitted_img, "Jitted image")
+
+####################################
+# Grayscale
+# ---------
+# The :class:`~torchvision.transforms.Grayscale` transform
+# (see also :func:`~torchvision.transforms.functional.to_grayscale`)
+# converts an image to grayscale
+gray_img = T.Grayscale()(orig_img)
+plot(gray_img, "Grayscale image", cmap='gray')


Test patch: