Repository: sphinx-doc/sphinx

Issue:
`sphinx.ext.autodoc` should respect `docutils` `tab_width` configuration
Currently the `sphinx.ext.autodoc` directives (e.g. `.. automodule::`) expand tabs to 8 spaces irrespective of the `tab_width` configuration of `docutils`.

This could be fixed by changing the `expandtabs()` call in `sphinx.util.docstrings.prepare_docstring` to pass the correct `tab_width` value.

(See #642 for more context).



Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201?src=pr&el=h1) Report
> Merging [#6201](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201?src=pr&el=desc) into [master](https://codecov.io/gh/sphinx-doc/sphinx/commit/2adeb68af1763be46359d5e808dae59d708661b1?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201/graphs/tree.svg?width=650&token=VFlFSELbkw&height=150&src=pr)](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6201      +/-   ##
==========================================
+ Coverage   83.56%   83.56%   +<.01%     
==========================================
  Files         273      273              
  Lines       39517    39525       +8     
  Branches     5889     5889              
==========================================
+ Hits        33021    33029       +8     
  Misses       5154     5154              
  Partials     1342     1342
```


| [Impacted Files](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [tests/test\_autodoc.py](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9hdXRvZG9jLnB5) | `96.76% <100%> (+0.02%)` | :arrow_up: |
| [sphinx/ext/autosummary/\_\_init\_\_.py](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201/diff?src=pr&el=tree#diff-c3BoaW54L2V4dC9hdXRvc3VtbWFyeS9fX2luaXRfXy5weQ==) | `84.48% <100%> (ø)` | :arrow_up: |
| [sphinx/ext/autodoc/directive.py](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201/diff?src=pr&el=tree#diff-c3BoaW54L2V4dC9hdXRvZG9jL2RpcmVjdGl2ZS5weQ==) | `92% <100%> (+0.33%)` | :arrow_up: |
| [sphinx/util/docstrings.py](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201/diff?src=pr&el=tree#diff-c3BoaW54L3V0aWwvZG9jc3RyaW5ncy5weQ==) | `96.87% <100%> (ø)` | :arrow_up: |
| [sphinx/ext/autodoc/\_\_init\_\_.py](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201/diff?src=pr&el=tree#diff-c3BoaW54L2V4dC9hdXRvZG9jL19faW5pdF9fLnB5) | `88.42% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201?src=pr&el=footer). Last update [2adeb68...496229e](https://codecov.io/gh/sphinx-doc/sphinx/pull/6201?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/CHANGES b/CHANGES
index aa94dd076..cbdd0e15b 100644
--- a/CHANGES
+++ b/CHANGES
@@ -98,6 +98,7 @@ Bugs fixed
 * #6331: man: invalid output when doctest follows rubric
 * #6351: "Hyperlink target is not referenced" message is shown even if
   referenced
+* #6165: autodoc: ``tab_width`` setting of docutils has been ignored
 
 Testing
 --------
diff --git a/sphinx/ext/autodoc/__init__.py b/sphinx/ext/autodoc/__init__.py
index 2a4df2159..412c336e3 100644
--- a/sphinx/ext/autodoc/__init__.py
+++ b/sphinx/ext/autodoc/__init__.py
@@ -442,7 +442,8 @@ def get_doc(self, encoding=None, ignore=1):
         docstring = getdoc(self.object, self.get_attr,
                            self.env.config.autodoc_inherit_docstrings)
         if docstring:
-            return [prepare_docstring(docstring, ignore)]
+            tab_width = self.directive.state.document.settings.tab_width
+            return [prepare_docstring(docstring, ignore, tab_width)]
         return []
 
     def process_doc(self, docstrings):
@@ -936,7 +937,9 @@ def _find_signature(self, encoding=None):
             if base not in valid_names:
                 continue
             # re-prepare docstring to ignore more leading indentation
-            self._new_docstrings[i] = prepare_docstring('\n'.join(doclines[1:]))
+            tab_width = self.directive.state.document.settings.tab_width  # type: ignore
+            self._new_docstrings[i] = prepare_docstring('\n'.join(doclines[1:]),
+                                                        tabsize=tab_width)
             result = args, retann
             # don't look any further
             break
@@ -1179,7 +1182,9 @@ def get_doc(self, encoding=None, ignore=1):
                     docstrings = [initdocstring]
                 else:
                     docstrings.append(initdocstring)
-        return [prepare_docstring(docstring, ignore) for docstring in docstrings]
+
+        tab_width = self.directive.state.document.settings.tab_width
+        return [prepare_docstring(docstring, ignore, tab_width) for docstring in docstrings]
 
     def add_content(self, more_content, no_docstring=False):
         # type: (Any, bool) -> None
diff --git a/sphinx/ext/autodoc/directive.py b/sphinx/ext/autodoc/directive.py
index 42415433b..6b002b101 100644
--- a/sphinx/ext/autodoc/directive.py
+++ b/sphinx/ext/autodoc/directive.py
@@ -6,10 +6,14 @@
     :license: BSD, see LICENSE for details.
 """
 
+import warnings
+
 from docutils import nodes
+from docutils.parsers.rst.states import Struct
 from docutils.statemachine import StringList
 from docutils.utils import assemble_option_dict
 
+from sphinx.deprecation import RemovedInSphinx40Warning
 from sphinx.ext.autodoc import Options, get_documenters
 from sphinx.util import logging
 from sphinx.util.docutils import SphinxDirective, switch_source_input
@@ -17,7 +21,7 @@
 
 if False:
     # For type annotation
-    from typing import Callable, Dict, List, Set, Type  # NOQA
+    from typing import Any, Callable, Dict, List, Set, Type  # NOQA
     from docutils.parsers.rst.state import RSTState  # NOQA
     from docutils.utils import Reporter  # NOQA
     from sphinx.config import Config  # NOQA
@@ -50,8 +54,8 @@ def __getitem__(self, key):
 class DocumenterBridge:
     """A parameters container for Documenters."""
 
-    def __init__(self, env, reporter, options, lineno):
-        # type: (BuildEnvironment, Reporter, Options, int) -> None
+    def __init__(self, env, reporter, options, lineno, state=None):
+        # type: (BuildEnvironment, Reporter, Options, int, Any) -> None
         self.env = env
         self.reporter = reporter
         self.genopt = options
@@ -59,6 +63,16 @@ def __init__(self, env, reporter, options, lineno):
         self.filename_set = set()  # type: Set[str]
         self.result = StringList()
 
+        if state:
+            self.state = state
+        else:
+            # create fake object for self.state.document.settings.tab_width
+            warnings.warn('DocumenterBridge requires a state object on instantiation.',
+                          RemovedInSphinx40Warning)
+            settings = Struct(tab_width=8)
+            document = Struct(settings=settings)
+            self.state = Struct(document=document)
+
     def warn(self, msg):
         # type: (str) -> None
         logger.warning(msg, location=(self.env.docname, self.lineno))
@@ -131,7 +145,7 @@ def run(self):
             return []
 
         # generate the output
-        params = DocumenterBridge(self.env, reporter, documenter_options, lineno)
+        params = DocumenterBridge(self.env, reporter, documenter_options, lineno, self.state)
         documenter = doccls(params, self.arguments[0])
         documenter.generate(more_content=self.content)
         if not params.result:
diff --git a/sphinx/ext/autosummary/__init__.py b/sphinx/ext/autosummary/__init__.py
index 952bd9e2a..5840f0ccd 100644
--- a/sphinx/ext/autosummary/__init__.py
+++ b/sphinx/ext/autosummary/__init__.py
@@ -175,7 +175,7 @@ def autosummary_table_visit_html(self, node):
 class FakeDirective(DocumenterBridge):
     def __init__(self):
         # type: () -> None
-        super().__init__({}, None, Options(), 0)  # type: ignore
+        super().__init__({}, None, Options(), 0, None)  # type: ignore
 
 
 def get_documenter(app, obj, parent):
@@ -236,7 +236,7 @@ class Autosummary(SphinxDirective):
     def run(self):
         # type: () -> List[nodes.Node]
         self.bridge = DocumenterBridge(self.env, self.state.document.reporter,
-                                       Options(), self.lineno)
+                                       Options(), self.lineno, self.state)
 
         names = [x.strip().split()[0] for x in self.content
                  if x.strip() and re.search(r'^[~a-zA-Z_]', x.strip()[0])]
diff --git a/sphinx/util/docstrings.py b/sphinx/util/docstrings.py
index 97dd60294..31943b2cb 100644
--- a/sphinx/util/docstrings.py
+++ b/sphinx/util/docstrings.py
@@ -15,8 +15,8 @@
     from typing import List  # NOQA
 
 
-def prepare_docstring(s, ignore=1):
-    # type: (str, int) -> List[str]
+def prepare_docstring(s, ignore=1, tabsize=8):
+    # type: (str, int, int) -> List[str]
     """Convert a docstring into lines of parseable reST.  Remove common leading
     indentation, where the indentation of a given number of lines (usually just
     one) is ignored.
@@ -25,7 +25,7 @@ def prepare_docstring(s, ignore=1):
     ViewList (used as argument of nested_parse().)  An empty line is added to
     act as a separator between this docstring and following content.
     """
-    lines = s.expandtabs().splitlines()
+    lines = s.expandtabs(tabsize).splitlines()
     # Find minimum indentation of any non-blank lines after ignored lines.
     margin = sys.maxsize
     for line in lines[ignore:]:


Test patch:
diff --git a/tests/test_autodoc.py b/tests/test_autodoc.py
index 5f616b791..75d59db14 100644
--- a/tests/test_autodoc.py
+++ b/tests/test_autodoc.py
@@ -11,6 +11,7 @@
 
 import platform
 import sys
+from unittest.mock import Mock
 from warnings import catch_warnings
 
 import pytest
@@ -36,7 +37,9 @@ def do_autodoc(app, objtype, name, options=None):
     app.env.temp_data.setdefault('docname', 'index')  # set dummy docname
     doccls = app.registry.documenters[objtype]
     docoptions = process_documenter_options(doccls, app.config, options)
-    bridge = DocumenterBridge(app.env, LoggingReporter(''), docoptions, 1)
+    state = Mock()
+    state.document.settings.tab_width = 8
+    bridge = DocumenterBridge(app.env, LoggingReporter(''), docoptions, 1, state)
     documenter = doccls(bridge, name)
     documenter.generate()
 
@@ -95,7 +98,9 @@ def setup_test():
         genopt = options,
         result = ViewList(),
         filename_set = set(),
+        state = Mock(),
     )
+    directive.state.document.settings.tab_width = 8
 
     processed_docstrings = []
     processed_signatures = []