Repository: spyder-ide/spyder

Issue:
Line break before type hint breaks docstring generation
<!--- **PLEASE READ:** When submitting here, please ensure you've completed the following checklist and checked the boxes to confirm. Issue reports without it may be closed. Thanks! --->

### Issue Report Checklist

* [x] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [x] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [x] Reproduced the issue after updating with ``conda update spyder`` (or ``pip``, if not using Anaconda)
* [ ] Could not reproduce inside ``jupyter qtconsole`` (if console-related)
* [ ] Tried basic troubleshooting (if a bug/error)
    * [x] Restarted Spyder
    * [ ] Reset preferences with ``spyder --reset``
    * [ ] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [ ] Tried the other applicable steps from the Troubleshooting Guide
* [x] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below


## Problem Description

If a function cantains a line break between a parameter and its type hint like so:
```
def test(v:
         int):
```
Typing """ does not activate the docstring generator and using the shortcut produces the following:
```
def test(v:
    """
    

    Parameters
    ----------
    v : TYPE
        DESCRIPTION.

    Returns
    -------
    None.

    """
         int):
```
It looks like the generator thinks the function ends in the first line (besides the lacking ')') because, see example below:
```
def test(d: str, v:
         int, a: None) -> bool:
```
```
def test(d: str, v:
    """
    

    Parameters
    ----------
    d : str
        DESCRIPTION.
    v : TYPE
        DESCRIPTION.

    Returns
    -------
    None.

    """
         int, a: None) -> bool:
```

### What steps reproduce the problem?

1. Enter the provided example
2. Press the docstring shortcut
3. 

### What is the expected output? What do you see instead?

A valid docstring
See above

### Paste Traceback/Error Below (if applicable)
<!--- Copy from error dialog or View > Panes > Internal Console --->

```python-traceback

PASTE TRACEBACK HERE

```

## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

* Spyder version: 4.2.1
* Python version: 1.9.1
* Qt version: 5.12.10
* PyQt version: 5.12.3
* Operating System name/version: Windows 10

### Dependencies
<!--- Please go to the menu entry Help > Dependencies,
press the Copy to clipboard button and paste below --->

```
PASTE DEPENDENCIES HERE
```

Line break before type hint breaks docstring generation
<!--- **PLEASE READ:** When submitting here, please ensure you've completed the following checklist and checked the boxes to confirm. Issue reports without it may be closed. Thanks! --->

### Issue Report Checklist

* [x] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [x] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [x] Reproduced the issue after updating with ``conda update spyder`` (or ``pip``, if not using Anaconda)
* [ ] Could not reproduce inside ``jupyter qtconsole`` (if console-related)
* [ ] Tried basic troubleshooting (if a bug/error)
    * [x] Restarted Spyder
    * [ ] Reset preferences with ``spyder --reset``
    * [ ] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [ ] Tried the other applicable steps from the Troubleshooting Guide
* [x] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below


## Problem Description

If a function cantains a line break between a parameter and its type hint like so:
```
def test(v:
         int):
```
Typing """ does not activate the docstring generator and using the shortcut produces the following:
```
def test(v:
    """
    

    Parameters
    ----------
    v : TYPE
        DESCRIPTION.

    Returns
    -------
    None.

    """
         int):
```
It looks like the generator thinks the function ends in the first line (besides the lacking ')') because, see example below:
```
def test(d: str, v:
         int, a: None) -> bool:
```
```
def test(d: str, v:
    """
    

    Parameters
    ----------
    d : str
        DESCRIPTION.
    v : TYPE
        DESCRIPTION.

    Returns
    -------
    None.

    """
         int, a: None) -> bool:
```

### What steps reproduce the problem?

1. Enter the provided example
2. Press the docstring shortcut
3. 

### What is the expected output? What do you see instead?

A valid docstring
See above

### Paste Traceback/Error Below (if applicable)
<!--- Copy from error dialog or View > Panes > Internal Console --->

```python-traceback

PASTE TRACEBACK HERE

```

## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

* Spyder version: 4.2.1
* Python version: 1.9.1
* Qt version: 5.12.10
* PyQt version: 5.12.3
* Operating System name/version: Windows 10

### Dependencies
<!--- Please go to the menu entry Help > Dependencies,
press the Copy to clipboard button and paste below --->

```
PASTE DEPENDENCIES HERE
```



Discussion:
Hi @Richardk2n,

I could reproduce this issue, we will work in this for one of our future releases.

Thanks for reporting!
By the way, this issue is caused by the code checking for ":" to determine the function definition end. I have a fix running, that changes that to `endswith("):") or endswith("]:") or (endswith(":") and contains("->"))` this works for most cases i can imagine. If you want, I could try to write a proper fix.
> this works for most cases i can imagine. If you want, I could try to write a proper fix.

Yes, please do! We'd really appreciate your help with that.
Hi @Richardk2n,

I could reproduce this issue, we will work in this for one of our future releases.

Thanks for reporting!
By the way, this issue is caused by the code checking for ":" to determine the function definition end. I have a fix running, that changes that to `endswith("):") or endswith("]:") or (endswith(":") and contains("->"))` this works for most cases i can imagine. If you want, I could try to write a proper fix.
> this works for most cases i can imagine. If you want, I could try to write a proper fix.

Yes, please do! We'd really appreciate your help with that.

Resolving patch:
diff --git a/spyder/plugins/editor/extensions/docstring.py b/spyder/plugins/editor/extensions/docstring.py
--- a/spyder/plugins/editor/extensions/docstring.py
+++ b/spyder/plugins/editor/extensions/docstring.py
@@ -48,6 +48,78 @@ def get_indent(text):
     return indent
 
 
+def is_in_scope_forward(text):
+    """Check if the next empty line could be part of the definition."""
+    text = text.replace(r"\"", "").replace(r"\'", "")
+    scopes = ["'''", '"""', "'", '"']
+    indices = [10**6] * 4  # Limits function def length to 10**6
+    for i in range(len(scopes)):
+        if scopes[i] in text:
+            indices[i] = text.index(scopes[i])
+    if min(indices) == 10**6:
+        return (text.count(")") != text.count("(") or
+                text.count("]") != text.count("[") or
+                text.count("}") != text.count("{"))
+    s = scopes[indices.index(min(indices))]
+    p = indices[indices.index(min(indices))]
+    ls = len(s)
+    if s in text[p+ls:]:
+        text = text[:p] + text[p+ls:][text[p+ls:].index(s)+ls:]
+        return is_in_scope_forward(text)
+    elif ls == 3:
+        text = text[:p]
+        return (text.count(")") != text.count("(") or
+                text.count("]") != text.count("[") or
+                text.count("}") != text.count("{"))
+    else:
+        return False
+
+
+def is_touple_brackets(text):
+    """Check if the return type is a tuple."""
+    scopes = ["(", "[", "{"]
+    compliments = [")", "]", "}"]
+    indices = [10**6] * 4  # Limits return type length to 10**6
+    for i in range(len(scopes)):
+        if scopes[i] in text:
+            indices[i] = text.index(scopes[i])
+    if min(indices) == 10**6:
+        return "," in text
+    s = compliments[indices.index(min(indices))]
+    p = indices[indices.index(min(indices))]
+    if s in text[p+1:]:
+        text = text[:p] + text[p+1:][text[p+1:].index(s)+1:]
+        return is_touple_brackets(text)
+    else:
+        return False
+
+
+def is_touple_strings(text):
+    """Check if the return type is a tuple."""
+    text = text.replace(r"\"", "").replace(r"\'", "")
+    scopes = ["'''", '"""', "'", '"']
+    indices = [10**6] * 4  # Limits return type length to 10**6
+    for i in range(len(scopes)):
+        if scopes[i] in text:
+            indices[i] = text.index(scopes[i])
+    if min(indices) == 10**6:
+        return is_touple_brackets(text)
+    s = scopes[indices.index(min(indices))]
+    p = indices[indices.index(min(indices))]
+    ls = len(s)
+    if s in text[p+ls:]:
+        text = text[:p] + text[p+ls:][text[p+ls:].index(s)+ls:]
+        return is_touple_strings(text)
+    else:
+        return False
+
+
+def is_in_scope_backward(text):
+    """Check if the next empty line could be part of the definition."""
+    return is_in_scope_forward(
+        text.replace(r"\"", "").replace(r"\'", "")[::-1])
+
+
 class DocstringWriterExtension(object):
     """Class for insert docstring template automatically."""
 
@@ -67,6 +139,39 @@ def is_beginning_triple_quotes(text):
 
         return False
 
+    def is_end_of_function_definition(self, text, line_number):
+        """Return True if text is the end of the function definition."""
+        text_without_whitespace = "".join(text.split())
+        if (text_without_whitespace.endswith("):") or
+            text_without_whitespace.endswith("]:") or
+            (text_without_whitespace.endswith(":") and
+             "->" in text_without_whitespace)):
+            return True
+        elif text_without_whitespace.endswith(":") and line_number > 1:
+            complete_text = text_without_whitespace
+            document = self.code_editor.document()
+            cursor = QTextCursor(
+                document.findBlockByNumber(line_number - 2))  # previous line
+            for i in range(line_number - 2, -1, -1):
+                txt = "".join(to_text_string(cursor.block().text()).split())
+                if (txt.endswith("\\") or is_in_scope_backward(complete_text)):
+                    if txt.endswith("\\"):
+                        txt = txt[:-1]
+                    complete_text = txt + complete_text
+                else:
+                    break
+                if i != 0:
+                    cursor.movePosition(QTextCursor.PreviousBlock)
+            if is_start_of_function(complete_text):
+                return (complete_text.endswith("):") or
+                        complete_text.endswith("]:") or
+                        (complete_text.endswith(":") and
+                         "->" in complete_text))
+            else:
+                return False
+        else:
+            return False
+
     def get_function_definition_from_first_line(self):
         """Get func def when the cursor is located on the first def line."""
         document = self.code_editor.document()
@@ -94,20 +199,22 @@ def get_function_definition_from_first_line(self):
                 is_first_line = False
             else:
                 cur_indent = get_indent(cur_text)
-                if cur_indent <= func_indent:
+                if cur_indent <= func_indent and cur_text.strip() != '':
                     return None
                 if is_start_of_function(cur_text):
                     return None
-                if cur_text.strip == '':
+                if (cur_text.strip() == '' and
+                        not is_in_scope_forward(func_text)):
                     return None
 
-            if cur_text[-1] == '\\':
+            if len(cur_text) > 0 and cur_text[-1] == '\\':
                 cur_text = cur_text[:-1]
 
             func_text += cur_text
             number_of_lines_of_function += 1
 
-            if cur_text.endswith(':'):
+            if self.is_end_of_function_definition(
+                    cur_text, line_number + number_of_lines_of_function - 1):
                 return func_text, number_of_lines_of_function
 
             cursor.movePosition(QTextCursor.NextBlock)
@@ -130,13 +237,15 @@ def get_function_definition_from_below_last_line(self):
             prev_text = to_text_string(cursor.block().text()).rstrip()
 
             if is_first_line:
-                if not prev_text.endswith(':'):
+                if not self.is_end_of_function_definition(
+                        prev_text, line_number - 1):
                     return None
                 is_first_line = False
-            elif prev_text.endswith(':') or prev_text == '':
+            elif self.is_end_of_function_definition(
+                    prev_text, line_number - number_of_lines_of_function - 1):
                 return None
 
-            if prev_text[-1] == '\\':
+            if len(prev_text) > 0 and prev_text[-1] == '\\':
                 prev_text = prev_text[:-1]
 
             func_text = prev_text + func_text
@@ -773,7 +882,7 @@ def split_args_text_to_list(self, args_text):
         """Split the text including multiple arguments to list.
 
         This function uses a comma to separate arguments and ignores a comma in
-        brackets ans quotes.
+        brackets and quotes.
         """
         args_list = []
         idx_find_start = 0
@@ -822,12 +931,15 @@ def parse_def(self, text):
         self.func_indent = get_indent(text)
 
         text = text.strip()
-        text = text.replace('\r\n', '')
-        text = text.replace('\n', '')
 
-        return_type_re = re.search(r'->[ ]*([a-zA-Z0-9_,()\[\] ]*):$', text)
+        return_type_re = re.search(
+            r'->[ ]*([\"\'a-zA-Z0-9_,()\[\] ]*):$', text)
         if return_type_re:
-            self.return_type_annotated = return_type_re.group(1)
+            self.return_type_annotated = return_type_re.group(1).strip(" ()\\")
+            if is_touple_strings(self.return_type_annotated):
+                self.return_type_annotated = ("("
+                                              + self.return_type_annotated
+                                              + ")")
             text_end = text.rfind(return_type_re.group(0))
         else:
             self.return_type_annotated = None


Test patch: