Repository: spyder-ide/spyder

Issue:
Generate docstring not working if there are comments on the function definition lines
<!--- **PLEASE READ:** When submitting here, please ensure you've completed the following checklist and checked the boxes to confirm. Issue reports without it may be closed. Thanks! --->

### Issue Report Checklist

* [x] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [ ] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [ ] Reproduced the issue after updating with ``conda update spyder`` (or ``pip``, if not using Anaconda)
* [ ] Could not reproduce inside ``jupyter qtconsole`` (if console-related)
* [ ] Tried basic troubleshooting (if a bug/error)
    * [ ] Restarted Spyder
    * [ ] Reset preferences with ``spyder --reset``
    * [ ] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [ ] Tried the other applicable steps from the Troubleshooting Guide
* [x] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below


## Problem Description
On master branch generating docstrings can fail or result in formatting issues of the docstring if function definition lines are commented. This can be fixed with some simple regex, and I will submit a PR for that.


### What steps reproduce the problem?
Two examples
```python
# Not able to generate docstring if the end of function definition line is commented
def test1(v: str = "#"):  # comment, with '#' and "#"

    ...
    
    
    
# Formatting issues with comments, and commas in comments add extra arguments
def test2(v1: str = "#", # comment, with '#' and "#"
          v2: str = '#') -> str:
    """
    

    Parameters
    ----------
    v1 : str, optional
        DESCRIPTION. The default is "#".
    # comment : TYPE
        DESCRIPTION.
    with '#' and "#"          v2 : str, optional
        DESCRIPTION. The default is '#'.

    Returns
    -------
    str
        DESCRIPTION.

    """

    ...
    
```

### What is the expected output? What do you see instead?



### Paste Traceback/Error Below (if applicable)
<!--- Copy from error dialog or View > Panes > Internal Console --->

```python-traceback

```

## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

* Spyder version: 6.0.0.dev0 11b9e503e (conda)
* Python version: 3.10.13 64-bit
* Qt version: 5.15.8
* PyQt5 version: 5.15.9
* Operating System: Windows-10-10.0.19045-SP0


### Dependencies
<!--- Please go to the menu entry Help > Dependencies,
press the Copy to clipboard button and paste below --->

```
# Mandatory:
atomicwrites >=1.2.0              :  1.4.1 (OK)
chardet >=2.0.0                   :  5.2.0 (OK)
cloudpickle >=0.5.0               :  3.0.0 (OK)
cookiecutter >=1.6.0              :  2.4.0 (OK)
diff_match_patch >=20181111       :  20230430 (OK)
intervaltree >=3.0.2              :  3.1.0 (OK)
IPython >=8.13.0,<9.0.0,!=8.17.1  :  8.17.2 (OK)
jedi >=0.17.2,<0.20.0             :  0.19.1 (OK)
jellyfish >=0.7                   :  1.0.1 (OK)
jsonschema >=3.2.0                :  4.19.2 (OK)
keyring >=17.0.0                  :  24.2.0 (OK)
nbconvert >=4.0                   :  7.11.0 (OK)
numpydoc >=0.6.0                  :  1.5.0 (OK)
paramiko >=2.4.0                  :  3.3.1 (OK)
parso >=0.7.0,<0.9.0              :  0.8.3 (OK)
pexpect >=4.4.0                   :  4.8.0 (OK)
pickleshare >=0.4                 :  0.7.5 (OK)
psutil >=5.3                      :  5.9.5 (OK)
pygments >=2.0                    :  2.16.1 (OK)
pylint >=2.5.0,<3.1               :  3.0.2 (OK)
pylint_venv >=3.0.2               :  3.0.3 (OK)
pyls_spyder >=0.4.0               :  0.4.0 (OK)
pylsp >=1.9.0,<1.10.0             :  1.9.0 (OK)
pylsp_black >=1.2.0,<3.0.0        :  1.3.0 (OK)
pyuca >=1.2                       :  1.2 (OK)
qdarkstyle >=3.2.0,<3.3.0         :  3.2 (OK)
qstylizer >=0.2.2                 :  0.2.2 (OK)
qtawesome >=1.2.1                 :  1.2.3 (OK)
qtconsole >=5.5.0,<5.6.0          :  5.6.0.dev0 (OK)
qtpy >=2.1.0                      :  2.4.1 (OK)
rtree >=0.9.7                     :  1.1.0 (OK)
setuptools >=49.6.0               :  68.2.2 (OK)
sphinx >=0.6.6                    :  7.2.6 (OK)
spyder_kernels >=3.0.0b2,<3.0.0b3 :  3.0.0.dev0 (OK)
superqt >=0.6.1,<1.0.0            :  0.6.1 (OK)
textdistance >=4.2.0              :  4.5.0 (OK)
three_merge >=0.1.1               :  0.1.1 (OK)
watchdog >=0.10.3                 :  3.0.0 (OK)
zmq >=22.1.0                      :  25.1.1 (OK)

# Optional:
cython >=0.21                     :  3.0.5 (OK)
matplotlib >=3.0.0                :  3.8.1 (OK)
numpy >=1.7                       :  1.26.0 (OK)
pandas >=1.1.1                    :  2.1.2 (OK)
scipy >=0.17.0                    :  1.11.3 (OK)
sympy >=0.7.3                     :  1.12 (OK)

```



Discussion:

Resolving patch:
diff --git a/spyder/plugins/editor/extensions/docstring.py b/spyder/plugins/editor/extensions/docstring.py
--- a/spyder/plugins/editor/extensions/docstring.py
+++ b/spyder/plugins/editor/extensions/docstring.py
@@ -120,6 +120,14 @@ def is_in_scope_backward(text):
         text.replace(r"\"", "").replace(r"\'", "")[::-1])
 
 
+def remove_comments(text):
+    """
+    Remove code comments from text while ignoring hash symbols (#) inside
+    quotes, which can be part of function arguments.
+    """
+    return re.sub(pattern=r"""(?<!['"])(#.*)""", repl="", string=text)
+
+
 class DocstringWriterExtension(object):
     """Class for insert docstring template automatically."""
 
@@ -194,6 +202,7 @@ def get_function_definition_from_first_line(self):
 
         for __ in range(min(remain_lines, 20)):
             cur_text = to_text_string(cursor.block().text()).rstrip()
+            cur_text = remove_comments(cur_text)
 
             if is_first_line:
                 if not is_start_of_function(cur_text):
@@ -239,6 +248,7 @@ def get_function_definition_from_below_last_line(self):
 
             cursor.movePosition(QTextCursor.PreviousBlock)
             prev_text = to_text_string(cursor.block().text()).rstrip()
+            prev_text = remove_comments(prev_text)
 
             if is_first_line:
                 if not self.is_end_of_function_definition(


Test patch:
diff --git a/spyder/plugins/editor/extensions/tests/test_docstring.py b/spyder/plugins/editor/extensions/tests/test_docstring.py
--- a/spyder/plugins/editor/extensions/tests/test_docstring.py
+++ b/spyder/plugins/editor/extensions/tests/test_docstring.py
@@ -665,3 +665,60 @@ def test_docstring_line_break(editor_auto_docstring, text, expected):
     editor.writer_docstring.write_docstring_for_shortcut()
 
     assert editor.toPlainText() == expected
+
+
+@pytest.mark.parametrize(
+    'text, expected',
+    [
+        ('''  def test(v: str = "#"):  # comment, with '#' and "#"
+      ''',
+         '''  def test(v: str = "#"):  # comment, with '#' and "#"
+      """\n      \n
+      Parameters
+      ----------
+      v : str, optional
+          DESCRIPTION. The default is "#".
+
+      Returns
+      -------
+      None.
+
+      """
+      '''),
+        ('''  def test(v1: str = "#", # comment, with '#' and "#"
+         v2: str = '#') -> str:
+      ''',
+         '''  def test(v1: str = "#", # comment, with '#' and "#"
+         v2: str = '#') -> str:
+      """\n      \n
+      Parameters
+      ----------
+      v1 : str, optional
+          DESCRIPTION. The default is "#".
+      v2 : str, optional
+          DESCRIPTION. The default is '#'.
+
+      Returns
+      -------
+      str
+          DESCRIPTION.
+
+      """
+      '''),
+    ])
+def test_docstring_comments(editor_auto_docstring, text, expected):
+    """
+    Test auto docstring with comments on lines of function definition.
+    """
+    CONF.set('editor', 'docstring_type', 'Numpydoc')
+    editor = editor_auto_docstring
+    editor.set_text(text)
+
+    cursor = editor.textCursor()
+    cursor.movePosition(QTextCursor.NextBlock)
+    cursor.setPosition(QTextCursor.End, QTextCursor.MoveAnchor)
+    editor.setTextCursor(cursor)
+
+    editor.writer_docstring.write_docstring_for_shortcut()
+
+    assert editor.toPlainText() == expected