Repository: pytorch/pytorch

Issue:
Improve how type annotations show up in html docs
Maybe some simpler type annotations can be introduced.
Currently it's `kernel_size: Union[T, Tuple[T, ...]], stride: Optional[Union[T, Tuple[T, ...]]]`, and it's hard to parse, especially since it's so comon and clutters the signatures. Can we introduce some type alias `int_or_inttuple` or something in this spirit?

![image](https://user-images.githubusercontent.com/1041752/93633775-c8d31c80-f9ef-11ea-81ce-4e8d11b1aff3.png)


cc @jlin27 @ezyang @malfet @rgommers @xuzhao9 @gramster


Discussion:
Such aliases are already used in the module definitions, but apparently are unwrapped when documentation is generated: 
https://github.com/pytorch/pytorch/blob/acc2a1e5fabe3c322927be72e0aa5d2e02bae46c/torch/nn/modules/pooling.py#L17-L19
maybe it's also worth to promote them to "first class" and remove the leading underscore from "_size_any_t" and give it a more comprehensible name such as "int_or_inttuple" (or sth else, but semantics of word "any" here is not very clear) 
The right fix here is to remove the type annotations from all signatures in the generated html docs I believe. The parameter descriptions already have type information, and there will be many cases where the type annotations will make little sense to the user (e.g., returning typevar `T`, long unions).

That of course does mean there can be mismatches between docs and annotations, but that may be on purpose sometimes. For example, for backwards compat often multiple types of inputs are supported, but there's one preferred one which is the only one documented. For example, I just saw that `ModuleDict.update` accepts not only `Mapping[str, Module]`, but also a length-2 list of str-module tuples. In such cases, the type annotations should be made complete if that's not too cumbersome (otherwise ignores can be added), but the docs should recommend just the preferred form.
Well, the type annotations could go to hover-only message with `title` html attribute. I think, types provide a fast parse over function arguments, I think they are still valuable (at least to me).

And a first-class, public alias of `Union[int, Tuple[int]]` may be useful anyway, as many functions accept such types, and users could even use these type definitions in their code
+ sometimes types are not explicitily mentioned in parameter description (e.g. if torch scalar or Python scalar is supported. if Python scalar, only int or float). Or that description is minorly outdated. So having it generated somewhere in the docs is useful - maybe html `title` attribute could be a compromise
> Well, the type annotations could go to hover-only message with `title` html attribute.

That would be nice, unfortunately there's no `autodoc` setting for that. So I'd defer that as a separate enhancement idea; would need a Sphinx or https://github.com/agronholm/sphinx-autodoc-typehints PR. Right now it's choosing between:

<img width="568" alt="image" src="https://user-images.githubusercontent.com/98330/101998635-54013d00-3cd5-11eb-8474-b9ccdca4989e.png">

and

<img width="568" alt="image" src="https://user-images.githubusercontent.com/98330/101998639-5f546880-3cd5-11eb-87f8-ab075c1d2bbe.png">

In recent versions of Sphinx (>= 3.0) there's a third option, `autodoc_typehints = 'description'`, which moves the annotations into the parameter description. I will have a look if that works without having to change each individual docstring with a marker for where to put that type info.

Oh wait, one more thing to try: `autodoc_type_aliases` to leave aliases unevaluated, that would be useful perhaps.
Also, if `stride=None` is supported, the type should be `Optional[something]`...
Still, we can use aliases for common things like `_TensorOrTensors` and feed those into `autodoc_type_aliases` as a dictionary, but for one-off unions that'd not be practical.
> Also, if `stride=None` is supported, the type should be `Optional[something]`...

Yep, lots of bugs left in the `nn.modules` annotations - first PRs to enable mypy on files in `nn/modules/` are for review now, that should catch a lot of these issues.
Unfortunately Sphinx 3.x doesn't seem to work yet; 2.4.4 does. So we can't use any of the newer features for autodoc type annotation handling, just the on/off toggle. @mattip I couldn't easily find where Sphinx versions are stored. I assume you tried with Sphinx 3.x at some point, do you know the status of that?
I don't remember trying with sphinx3. Based on NumPy's experience, it would be a non-trivial migration. The version used is in [docs/requirements.txt](https://github.com/pytorch/pytorch/blob/master/docs/requirements.txt) and [docs/cpp/requirements.txt](https://github.com/pytorch/pytorch/blob/master/docs/cpp/requirements.txt). The `cpp` one uses `sphinx=3.1.2`
Type annotations no longer show up in the html docs after gh-49294. I'll leave this issue open because it contains some good ideas to make use of type annotations in the docs without making a mess; those can be picked up after migrating the doc build to Sphinx 3.x
Just checking if there is an update? Closed #78311 for this.
Sphinx has made a little progress here I believe. The `autodoc_typehints_description_target` and `autodoc_type_aliases` are relevant, and available in Sphinx >= 3.3 (so [docs/cpp/requirements.txt](https://github.com/pytorch/pytorch/blob/master/docs/cpp/requirements.txt)) would need a version bump.

See https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_typehints_description_target

It'd still be a large job to make this usable in the PyTorch HTML docs though. What would need doing is to use [Type aliases](https://mypy.readthedocs.io/en/latest/kinds_of_types.html#type-aliases) consistently, to get readable types rather than things like `Union[T, [Tuple[T, ...]], <etc>`.
For the documentation, between having no type annotation at all and having sometimes-complicated type annotation, I prefer the latter.

The human-entered documentation does not always contain necessary type information so the current design means the reader sometimes is forced to read the source code to get type information, which contains the same complicated type information, anyway. If the documentation contains the original type hints, regardless of whether a reader thinks the type is too complicated, he has an option to read it or ignore it; he still has a choice.

Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -396,8 +396,34 @@
 # Disable docstring inheritance
 autodoc_inherit_docstrings = False
 
-# Disable displaying type annotations, these can be very verbose
-autodoc_typehints = 'none'
+# Show type hints in the description
+autodoc_typehints = 'description'
+
+# Add parameter types if the parameter is documented in the docstring
+autodoc_typehints_description_target = 'documented_params'
+
+# Type aliases for common types
+# Sphinx type aliases only works with Postponed Evaluation of Annotations
+# (PEP 563) enabled (via `from __future__ import annotations`), which keeps the
+# type annotations in string form instead of resolving them to actual types.
+# However, PEP 563 does not work well with JIT, which uses the type information
+# to generate the code. Therefore, the following dict does not have any effect
+# until PEP 563 is supported by JIT and enabled in files.
+autodoc_type_aliases = {
+    "_size_1_t": "int or tuple[int]",
+    "_size_2_t": "int or tuple[int, int]",
+    "_size_3_t": "int or tuple[int, int, int]",
+    "_size_4_t": "int or tuple[int, int, int, int]",
+    "_size_5_t": "int or tuple[int, int, int, int, int]",
+    "_size_6_t": "int or tuple[int, int, int, int, int, int]",
+    "_size_any_opt_t": "int or None or tuple",
+    "_size_2_opt_t": "int or None or 2-tuple",
+    "_size_3_opt_t": "int or None or 3-tuple",
+    "_ratio_2_t": "float or tuple[float, float]",
+    "_ratio_3_t": "float or tuple[float, float, float]",
+    "_ratio_any_t": "float or tuple",
+    "_tensor_list_t": "Tensor or tuple[Tensor]",
+}
 
 # Enable overriding of function signatures in the first line of the docstring.
 autodoc_docstring_signature = True


Test patch: