Repository: numpy/numpydoc

Issue:
Wrong number of Parameter for numpy array.
```
In [1]: from numpydoc.docscrape import NumpyDocString
   ...: from numpy import array
   ...: len(NumpyDocString(array.__doc__)['Parameters'])
Out[1]: 1
```

(should be 6)

Again due to a dedent issue: 

```
$ git diff
diff --git a/numpydoc/docscrape.py b/numpydoc/docscrape.py
index d79992c..3db20fe 100644
--- a/numpydoc/docscrape.py
+++ b/numpydoc/docscrape.py
@@ -219,6 +219,7 @@ class NumpyDocString(Mapping):
                 yield name, self._strip(data[2:])

     def _parse_param_list(self, content, single_element_is_type=False):
+        content = dedent_lines(content)
         r = Reader(content)
         params = []
         while not r.eof():
```

Fixes it. 

Not quite sure why it is not found in tests...




Discussion:
codecov-commenter: # [Codecov](https://codecov.io/gh/numpy/numpydoc/pull/286?src=pr&el=h1) Report
> Merging [#286](https://codecov.io/gh/numpy/numpydoc/pull/286?src=pr&el=desc) into [master](https://codecov.io/gh/numpy/numpydoc/commit/4e86e815a855502eef1a796a5d3f217b08aaa79f&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `100.00%`.


```diff
@@            Coverage Diff             @@
##           master     #286      +/-   ##
==========================================
+ Coverage   93.10%   93.11%   +0.01%     
==========================================
  Files           7        7              
  Lines        1261     1263       +2     
==========================================
+ Hits         1174     1176       +2     
  Misses         87       87              
```



Carreau: > to get the dedenting correct for the entire docstring when the first line doesn't match the indenting scheme of the rest of the docstring?

That was the first fix I tried, but this make `validate` fails as well as other things which are attempting to tell you when the docstring is not valid or have extra spaces in place it does not like.

Personally I end up always calling it as so `Numpydoc(dedent_but_first(docstring))`, to work around some other inconsistencies.

rossbar: > That was the first fix I tried, but this make validate fails as well as other things which are attempting to tell you when the docstring is not valid or have extra spaces in place it does not like.

Yes, I noticed this too when I was messing around with alternatives (see #287, which will fail one validation test). It seems to me much more straightforward to catch issues due to dedenting errors as far upstream as possible, rather than modifying the downstream functions to handle the corner cases of less-robust docstring cleaning.

However, if there are strong backwards compatibility concerns with the validation module, then the approach in #287 is a non-starter.
Carreau: Yeah, I went with not validating, and just re-emitting the docstring from the Numpydoc `_parsed_data` data structure as the `__repr__` does also some weird things and ended up with a docstring reformatter.
rossbar: Hmm, on the one hand the validation should definitely catch indentation errors, but it would be nice if the docstring parsing were more resilient to indentation problems. 
Carreau: Yeah, I think the two logics are intertwined too much, and the "parsing"/"guessing"/"validating" should be different steps. 

There are many docstrings around that have invalid section names, and right now can't be parsed as Numpydoc Errors immediately, though that might be for a longer term project to clean that up.


rossbar: > There are many docstrings around that have invalid section names, and right now can't be parsed as Numpydoc Errors immediately, though that might be for a longer term project to clean that up.

... including in numpy itself! (numpy/numpy#16791)

I agree it's probably a larger project, so let's not worry about it in this PR. I think the changes here look good. The only additions I'd make is to apply a similar `pytest.mark.parametrize` to the `test_returns` and `test_other_parameters` tests, since they also are affected by the changes to `_parse_param_list`. I also slightly prefer the parametrization I used in #287 as it's a little shorter, but YMMV.
jnothman: > many project have paremeters description list with : afer the name, even if no type is present. If there is no space after the : the parameter name includes the : which is most likely wrong.

The problem being that this matched the relatively obscure syntax for ReST definition lists...
Carreau: ok, I made `doc` a parameterized fixture so every test using it are now parametrized on `''` (flush) vs `'\n    '` (newline indented)

Patch:
diff --git a/doc/install.rst b/doc/install.rst
index ba799e3..46b26f8 100644
--- a/doc/install.rst
+++ b/doc/install.rst
@@ -63,15 +63,6 @@ numpydoc_xref_aliases : dict
   aliases/shortcuts used when specifying the types of parameters.
   The keys should not have any spaces. Together with the ``intersphinx``
   extension, you can map to links in any documentation.
-  The default is an empty ``dict``.
-
-  If you have the following ``intersphinx`` namespace configuration::
-
-      intersphinx_mapping = {
-          'python': ('https://docs.python.org/3/', None),
-          'numpy': ('https://docs.scipy.org/doc/numpy', None),
-          ...
-      }
 
   The default ``numpydoc_xref_aliases`` will supply some common ``Python``
   standard library and ``NumPy`` names for you. Then for your module, a useful
diff --git a/numpydoc/docscrape.py b/numpydoc/docscrape.py
index d79992c..2b992d8 100644
--- a/numpydoc/docscrape.py
+++ b/numpydoc/docscrape.py
@@ -219,12 +219,14 @@ class NumpyDocString(Mapping):
                 yield name, self._strip(data[2:])
 
     def _parse_param_list(self, content, single_element_is_type=False):
+        content = dedent_lines(content)
         r = Reader(content)
         params = []
         while not r.eof():
             header = r.read().strip()
-            if ' : ' in header:
-                arg_name, arg_type = header.split(' : ', maxsplit=1)
+            if ' :' in header:
+                arg_name, arg_type = header.split(' :', maxsplit=1)
+                arg_name, arg_type = arg_name.strip(), arg_type.strip()
             else:
                 if single_element_is_type:
                     arg_name, arg_type = '', header
diff --git a/numpydoc/xref.py b/numpydoc/xref.py
index a4ceaf9..7c6612e 100644
--- a/numpydoc/xref.py
+++ b/numpydoc/xref.py
@@ -95,7 +95,9 @@ DEFAULT_LINKS = {
 
 
 def make_xref(param_type, xref_aliases, xref_ignore):
-    """Enclose str in a :obj: role.
+    """Parse and apply appropriate sphinx role(s) to `param_type`.
+
+    The :obj: role is the default.
 
     Parameters
     ----------
@@ -110,7 +112,7 @@ def make_xref(param_type, xref_aliases, xref_ignore):
     Returns
     -------
     out : str
-        Text with parts that may be wrapped in a
+        Text with fully-qualified names and terms that may be wrapped in a
         ``:obj:`` role.
     """
     if param_type in xref_aliases:


Test patch:
diff --git a/numpydoc/tests/test_docscrape.py b/numpydoc/tests/test_docscrape.py
index 2ab0218..a0b6d2e 100644
--- a/numpydoc/tests/test_docscrape.py
+++ b/numpydoc/tests/test_docscrape.py
@@ -133,7 +133,11 @@ doc_txt = '''\
      :refguide: random;distributions, random;gauss
 
   '''
-doc = NumpyDocString(doc_txt)
+
+@pytest.fixture(params=['','\n    '], ids=["flush", "newline_indented"])
+def doc(request):
+    return NumpyDocString(request.param+doc_txt)
+
 
 doc_yields_txt = """
 Test generator
@@ -169,21 +173,21 @@ c : int
 doc_sent = NumpyDocString(doc_sent_txt)
 
 
-def test_signature():
+def test_signature(doc):
     assert doc['Signature'].startswith('numpy.multivariate_normal(')
     assert doc['Signature'].endswith('spam=None)')
 
 
-def test_summary():
+def test_summary(doc):
     assert doc['Summary'][0].startswith('Draw values')
     assert doc['Summary'][-1].endswith('covariance.')
 
 
-def test_extended_summary():
+def test_extended_summary(doc):
     assert doc['Extended Summary'][0].startswith('The multivariate normal')
 
 
-def test_parameters():
+def test_parameters(doc):
     assert len(doc['Parameters']) == 4
     names = [n for n, _, _ in doc['Parameters']]
     assert all(a == b for a, b in zip(names, ['mean', 'cov', 'shape']))
@@ -205,7 +209,7 @@ def test_parameters():
     assert desc[0].startswith('The type and size')
 
 
-def test_other_parameters():
+def test_other_parameters(doc):
     assert len(doc['Other Parameters']) == 1
     assert [n for n, _, _ in doc['Other Parameters']] == ['spam']
     arg, arg_type, desc = doc['Other Parameters'][0]
@@ -213,7 +217,8 @@ def test_other_parameters():
     assert desc[0].startswith('A parrot off its mortal coil')
 
 
-def test_returns():
+
+def test_returns(doc):
     assert len(doc['Returns']) == 3
     arg, arg_type, desc = doc['Returns'][0]
     assert arg == 'out'
@@ -342,23 +347,23 @@ That should break...
                 or 'function dummy_func' in str(e))
 
 
-def test_notes():
+def test_notes(doc):
     assert doc['Notes'][0].startswith('Instead')
     assert doc['Notes'][-1].endswith('definite.')
     assert len(doc['Notes']) == 17
 
 
-def test_references():
+def test_references(doc):
     assert doc['References'][0].startswith('..')
     assert doc['References'][-1].endswith('2001.')
 
 
-def test_examples():
+def test_examples(doc):
     assert doc['Examples'][0].startswith('>>>')
     assert doc['Examples'][-1].endswith('True]')
 
 
-def test_index():
+def test_index(doc):
     assert doc['index']['default'] == 'random'
     assert len(doc['index']) == 2
     assert len(doc['index']['refguide']) == 2
@@ -382,7 +387,7 @@ def line_by_line_compare(a, b, n_lines=None):
         assert aa == bb
 
 
-def test_str():
+def test_str(doc):
     # doc_txt has the order of Notes and See Also sections flipped.
     # This should be handled automatically, and so, one thing this test does
     # is to make sure that See Also precedes Notes in the output.
@@ -921,6 +926,21 @@ doc7 = NumpyDocString("""
 def test_empty_first_line():
     assert doc7['Summary'][0].startswith('Doc starts')
 
+doc8 = NumpyDocString("""
+
+        Parameters with colon and no types:
+
+        Parameters
+        ----------
+
+        data :
+            some stuff, technically invalid
+        """)
+
+
+def test_trailing_colon():
+    assert doc8['Parameters'][0].name == 'data'
+
 
 def test_no_summary():
     str(SphinxDocString("""
diff --git a/numpydoc/tests/test_xref.py b/numpydoc/tests/test_xref.py
index 31dfacd..5d16919 100644
--- a/numpydoc/tests/test_xref.py
+++ b/numpydoc/tests/test_xref.py
@@ -1,25 +1,15 @@
 # -*- encoding:utf-8 -*-
 import pytest
-from numpydoc.xref import make_xref
-
-xref_aliases = {
-    # python
-    'sequence': ':term:`python:sequence`',
-    'iterable': ':term:`python:iterable`',
-    'string': 'str',
-    # numpy
-    'array': 'numpy.ndarray',
-    'dtype': 'numpy.dtype',
-    'ndarray': 'numpy.ndarray',
-    'matrix': 'numpy.matrix',
-    'array-like': ':term:`numpy:array_like`',
-    'array_like': ':term:`numpy:array_like`',
-}
+from numpydoc.xref import make_xref, DEFAULT_LINKS
+
+# Use the default numpydoc link mapping
+xref_aliases = DEFAULT_LINKS
+
 
 # Comes mainly from numpy
 data = r"""
 (...) array_like, float, optional
-(...) :term:`numpy:array_like`, :obj:`float`, optional
+(...) :term:`numpy:array_like`, :class:`python:float`, optional
 
 (2,) ndarray
 (2,) :obj:`ndarray <numpy.ndarray>`
@@ -31,37 +21,37 @@ data = r"""
 (..., :obj:`M`, :obj:`N`) :term:`numpy:array_like`
 
 (float, float), optional
-(:obj:`float`, :obj:`float`), optional
+(:class:`python:float`, :class:`python:float`), optional
 
 1-D array or sequence
 1-D :obj:`array <numpy.ndarray>` or :term:`python:sequence`
 
 array of str or unicode-like
-:obj:`array <numpy.ndarray>` of :obj:`str` or unicode-like
+:obj:`array <numpy.ndarray>` of :class:`python:str` or unicode-like
 
 array_like of float
-:term:`numpy:array_like` of :obj:`float`
+:term:`numpy:array_like` of :class:`python:float`
 
 bool or callable
-:obj:`bool` or :obj:`callable`
+:ref:`bool <python:bltin-boolean-values>` or :func:`python:callable`
 
 int in [0, 255]
-:obj:`int` in [0, 255]
+:class:`python:int` in [0, 255]
 
 int or None, optional
-:obj:`int` or :obj:`None`, optional
+:class:`python:int` or :data:`python:None`, optional
 
 list of str or array_like
-:obj:`list` of :obj:`str` or :term:`numpy:array_like`
+:class:`python:list` of :class:`python:str` or :term:`numpy:array_like`
 
 sequence of array_like
 :term:`python:sequence` of :term:`numpy:array_like`
 
 str or pathlib.Path
-:obj:`str` or :obj:`pathlib.Path`
+:class:`python:str` or :obj:`pathlib.Path`
 
 {'', string}, optional
-{'', :obj:`string <str>`}, optional
+{'', :class:`python:str`}, optional
 
 {'C', 'F', 'A', or 'K'}, optional
 {'C', 'F', 'A', or 'K'}, optional
@@ -70,16 +60,16 @@ str or pathlib.Path
 {'linear', 'lower', 'higher', 'midpoint', 'nearest'}
 
 {False, True, 'greedy', 'optimal'}
-{:obj:`False`, :obj:`True`, 'greedy', 'optimal'}
+{:data:`python:False`, :data:`python:True`, 'greedy', 'optimal'}
 
 {{'begin', 1}, {'end', 0}}, {string, int}
-{{'begin', 1}, {'end', 0}}, {:obj:`string <str>`, :obj:`int`}
+{{'begin', 1}, {'end', 0}}, {:class:`python:str`, :class:`python:int`}
 
 callable f'(x,*args)
-:obj:`callable` f'(x,*args)
+:func:`python:callable` f'(x,*args)
 
 callable ``fhess(x, *args)``, optional
-:obj:`callable` ``fhess(x, *args)``, optional
+:func:`python:callable` ``fhess(x, *args)``, optional
 
 spmatrix (format: ``csr``, ``bsr``, ``dia`` or coo``)
 :obj:`spmatrix` (format: ``csr``, ``bsr``, ``dia`` or coo``)
@@ -88,28 +78,28 @@ spmatrix (format: ``csr``, ``bsr``, ``dia`` or coo``)
 :ref:`strftime <strftime-strptime-behavior>`
 
 callable or :ref:`strftime <strftime-strptime-behavior>`
-:obj:`callable` or :ref:`strftime <strftime-strptime-behavior>`
+:func:`python:callable` or :ref:`strftime <strftime-strptime-behavior>`
 
 callable or :ref:`strftime behavior <strftime-strptime-behavior>`
-:obj:`callable` or :ref:`strftime behavior <strftime-strptime-behavior>`
+:func:`python:callable` or :ref:`strftime behavior <strftime-strptime-behavior>`
 
 list(int)
-:obj:`list`\(:obj:`int`)
+:class:`python:list`\(:class:`python:int`)
 
 list[int]
-:obj:`list`\[:obj:`int`]
+:class:`python:list`\[:class:`python:int`]
 
 dict(str, int)
-:obj:`dict`\(:obj:`str`, :obj:`int`)
+:class:`python:dict`\(:class:`python:str`, :class:`python:int`)
 
 dict[str,  int]
-:obj:`dict`\[:obj:`str`,  :obj:`int`]
+:class:`python:dict`\[:class:`python:str`,  :class:`python:int`]
 
 tuple(float, float)
-:obj:`tuple`\(:obj:`float`, :obj:`float`)
+:class:`python:tuple`\(:class:`python:float`, :class:`python:float`)
 
 dict[tuple(str, str), int]
-:obj:`dict`\[:obj:`tuple`\(:obj:`str`, :obj:`str`), :obj:`int`]
+:class:`python:dict`\[:class:`python:tuple`\(:class:`python:str`, :class:`python:str`), :class:`python:int`]
 """  # noqa: E501
 
 xref_ignore = {'or', 'in', 'of', 'default', 'optional'}