Repository: microsoft/ptvsd

Issue:
python without debugging won't start if there is a breakpoint
@tmdag commented on [Thu May 02 2019](https://github.com/microsoft/vscode-python/issues/5540)

## Environment data
- VSCode Version: 1.33.1
- OS Version:Linux 5.0.7-200.fc29.x86_64
- Extension version (available under the Extensions sidebar): Python 2019.4.11987
- Anaconda Extension Pack 1.0.1

## Steps to reproduce:
Open Python code
create a breakpoint
run python without debugging (ctrl+F5)
Terminal output:

cd /home/user/dev/Pytool ; env PYTHONIOENCODING=UTF-8 PYTHONUNBUFFERED=1 /usr/bin/python3 /home/user/.vscode/extensions/ms-python.python-2019.4.11987/pythonFiles/ptvsd_launcher.py --default --nodebug --client --host localhost --port 36019 /home/user/dev/Pytool/mypytool.py
Terminated
Does this issue occur when all extensions are disabled?: Yes/No
Not sure - Python extension is required

## Enabled Extensions:
Bookmarks 10.4.3
C/C++ 0.221
Gist 3.0.3
Git History 0.4.6
GitLens - 9.6.3
markdownlint 0.26.0

Syncing 2.1.6
OpenCL 0.5.2
VEX 0.4.0
TAML 0.4.0



---

@jxramos commented on [Thu May 02 2019](https://github.com/microsoft/vscode-python/issues/5540#issuecomment-488807421)

I'm seeing this too on a MacOS, it immediately exits with `Terminated: 15`. This behavior persists even if the breakpoint is disabled/unchecked.




Discussion:
@tmdag @jxramos Can you share logs, for the debugger? you can get logs by setting `PTVSD_LOG_DIR` environment variable to a directory.
```js
{
    "name": "Terminal",
    "type": "python",
    "request": "launch",
    "pythonPath": "${config:python.pythonPath}",
    "program": "${file}",
    "cwd": "${workspaceFolder}",
    "console": "integratedTerminal",
    "env": {
        "PTVSD_LOG_DIR": "<path to directory>"
    }
},
```
@karthiknadig This is what got rendered....

```
I00000.027: Darwin-17.5.0-x86_64-i386-64bit x86_64
            CPython 2.7.10 (64-bit)
            ptvsd 4.2.8

I00000.027: main(['/Users/$USER/.vscode/extensions/ms-python.python-2019.4.12954/pythonFiles/ptvsd_launcher.py', '--nodebug', '--client', '--host', 'localhost', '--port', '57251', '/Users/$USER/source_dir/test.py', '--retry', '0', '--udids', 'xxxxxxxxxxx', '--emails', 'name@domain.com', '--tests', '19245179', '--description', 'test_description'])

I00000.027: sys.argv after parsing: ['/Users/$USER/.vscode/extensions/ms-python.python-2019.4.12954/pythonFiles/ptvsd_launcher.py', '--retry', '0', '--udids', 'xxxxxxxxxxx', '--emails', 'name@domain.com', '--tests', '19245179', '--description', 'test_description']

D00000.027: sys.prefix: ('/System/Library/Frameworks/Python.framework/Versions/2.7',)

D00000.027: site.getusersitepackages(): /Users/$USER/Library/Python/2.7/lib/python/site-packages

D00000.027: site.getsitepackages(): ['/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/site-python', '/Library/Python/2.7/site-packages']

D00000.028: Folder with "site-packages" in sys.path: /Users/$USER/Library/Python/2.7/lib/python/site-packages

D00000.028: Folder with "site-packages" in sys.path: /Library/Python/2.7/site-packages

D00000.028: Folder with "site-packages" in sys.path: /Internal/Library/Python/2.7/site-packages

D00000.028: sys.argv after patching: ['/Users/$USER/source_dir/test.py', '--retry', '0', '--udids', 'xxxxxxxxxxx', '--emails', 'name@domain.com', '--tests', '19245179', '--description', 'test_description']

D00000.028: Starting client daemon on (u'localhost', 57251).

D00000.031: IDE <-- {
                "body": {
                    "category": "telemetry", 
                    "output": "ptvsd", 
                    "data": {
                        "version": "4.2.8"
                    }
                }, 
                "type": "event", 
                "event": "output", 
                "seq": 0
            }

D00000.031: IDE --> {
                "type": "request", 
                "command": "launch", 
                "arguments": {
                    "console": "integratedTerminal", 
                    "name": "Python: DMTEST", 
                    "__sessionId": "1a6494de-fe2b-4581-aa84-e4cb629d3c85", 
                    "envFile": "/Users/$USER/source/.env", 
                    "pythonPath": "/usr/bin/python", 
                    "args": [
                        "--retry", 
                        "0", 
                        "--udids", 
                        "xxxxxxxxxxx", 
                        "--emails", 
                        "name@domain.com", 
                        "--tests", 
                        "19245179", 
                        "--description", 
                        "test_description"
                    ], 
                    "request": "launch", 
                    "debugOptions": [
                        "ShowReturnValue", 
                        "RedirectOutput"
                    ], 
                    "internalConsoleOptions": "neverOpen", 
                    "noDebug": true, 
                    "program": "/Users/$USER/source_dir/test.py", 
                    "env": {
                        "PTVSD_LOG_DIR": "/Users/$USER/vscode_debug_logs"
                    }, 
                    "justMyCode": true, 
                    "workspaceFolder": "/Users/$USER/source", 
                    "type": "python", 
                    "cwd": "/Users/$USER/source_dir/", 
                    "stopOnEntry": false, 
                    "showReturnValue": true
                }, 
                "seq": 2
            }

D00000.032: (while handling {u'command': u'launch', u'type': u'request', u'seq': 2})
            IDE <-- {
                "request_seq": 2, 
                "body": {}, 
                "seq": 1, 
                "success": true, 
                "command": "launch", 
                "message": "", 
                "type": "response"
            }

D00000.032: IDE --> {
                "type": "request", 
                "command": "initialize", 
                "arguments": {
                    "adapterID": "python", 
                    "locale": "en-us", 
                    "supportsRunInTerminalRequest": true, 
                    "pathFormat": "path", 
                    "clientID": "vscode", 
                    "supportsVariableType": true, 
                    "supportsVariablePaging": true, 
                    "linesStartAt1": true, 
                    "columnsStartAt1": true, 
                    "clientName": "Visual Studio Code"
                }, 
                "seq": 1
            }

D00000.033: (while handling {u'command': u'initialize', u'type': u'request', u'seq': 1})
            IDE <-- {
                "request_seq": 1, 
                "body": {
                    "supportsModulesRequest": true, 
                    "supportsConfigurationDoneRequest": true, 
                    "supportsDelayedStackTraceLoading": true, 
                    "supportsDebuggerProperties": true, 
                    "supportsEvaluateForHovers": true, 
                    "supportsSetExpression": true, 
                    "supportsExceptionOptions": true, 
                    "exceptionBreakpointFilters": [
                        {
                            "filter": "raised", 
                            "default": false, 
                            "label": "Raised Exceptions"
                        }, 
                        {
                            "filter": "uncaught", 
                            "default": true, 
                            "label": "Uncaught Exceptions"
                        }
                    ], 
                    "supportsCompletionsRequest": true, 
                    "supportsExceptionInfoRequest": true, 
                    "supportsLogPoints": true, 
                    "supportsValueFormattingOptions": true, 
                    "supportsHitConditionalBreakpoints": true, 
                    "supportsSetVariable": true, 
                    "supportTerminateDebuggee": true, 
                    "supportsGotoTargetsRequest": true, 
                    "supportsConditionalBreakpoints": true
                }, 
                "seq": 2, 
                "success": true, 
                "command": "initialize", 
                "message": "", 
                "type": "response"
            }

D00000.033: (while handling {u'command': u'initialize', u'type': u'request', u'seq': 1})
            IDE <-- {
                "body": {}, 
                "type": "event", 
                "event": "initialized", 
                "seq": 3
            }

I00000.033: Running file /Users/$USER/source_dir/test.py

D00000.033: Adding /Users/$USER/source_dir to sys.path.

D00000.053: IDE --> {
                "type": "request", 
                "command": "setBreakpoints", 
                "arguments": {
                    "sourceModified": false, 
                    "source": {
                        "path": "git:/Users/$USER/source_dir/test.py?%7B%22path%22%3A%22%2FUsers%2F$USER%2Fsource%2Ftest.py%22%2C%22ref%22%3A%22%22%7D", 
                        "name": "test.py"
                    }, 
                    "lines": [
                        1456
                    ], 
                    "breakpoints": [
                        {
                            "line": 1456
                        }
                    ]
                }, 
                "seq": 4
            }

D00000.096: (while handling {u'command': u'setBreakpoints', u'type': u'request', u'seq': 4})
            IDE <-- {
                "request_seq": 4, 
                "body": {}, 
                "seq": 4, 
                "success": true, 
                "command": "setBreakpoints", 
                "message": "", 
                "type": "response"
            }

D00000.512: IDE --> {
                "type": "request", 
                "command": "disconnect", 
                "arguments": {
                    "restart": false
                }, 
                "seq": 5
            }

D00000.554: (while handling {u'command': u'disconnect', u'type': u'request', u'seq': 5})
            Killing remaining subprocesses: PID=[]

D00000.554: (while handling {u'command': u'disconnect', u'type': u'request', u'seq': 5})
            Disconnecting.

D00000.554: (while handling {u'command': u'disconnect', u'type': u'request', u'seq': 5})
            Handling disconnecting session
```
The path being passed in the `setBreakpoints` request seems wrong. @DonJayamanne this is something that the extension passes in right? and is this path correct?

unescaped version:
 `git:/Users/$USER/source_dir/test.py?%7B%22path%22%3A%22%2FUsers%2F$USER%2Fsource%2Ftest.py%22%2C%22ref%22%3A%22%22%7D`

escaped version:
`git:/Users/$USER/source_dir/test.py?{"path":"/Users/$USER/source/test.py","ref":""}`
That’s very likely a side effect of my manual redaction.
Sorry, misunderstood the meaning of `$USER`. Is that the reduction that you have made? 

The real problem here looks like all the `git:/` stuff, as well as everything following `?`. Is there anything in your launch.json corresponding to it?
I manipulated various aspects of the path and script names, substituting my actual user name to $USER was part of my find and replace operation.

Here's a more careful redaction of that git line...

`"git:/Users/USER_X/source/dir1/dir2/dir3/dir4/test.py?%7B%22path%22%3A%22%2FUsers%2FUSER_X%2Fsource%2Fdir1%2Fdir2%2Fdir3%2Fdir4%2Ftest.py%22%2C%22ref%22%3A%22%22%7D", 
`
which looks like it decodes to...

`"git:/Users/USER_X/source/dir1/dir2/dir3/dir4/test.py?{"path":"/Users/USER_X/source/dir1/dir2/dir3/dir4/test.py","ref":""}",
`
and this is my output
```
I00000.014: Linux-5.0.10-200.fc29.x86_64-x86_64-with-fedora-29-Twenty_Nine x86_64
            CPython 3.7.3 (64-bit)
            ptvsd 4.2.8

I00000.014: main(['/home/tmdag/.vscode/extensions/ms-python.python-2019.4.12954/pythonFiles/ptvsd_launcher.py', '--nodebug', '--client', '--host', 'localhost', '--port', '38491', '/home/tmdag/pyproject/mypython.py'])

I00000.014: sys.argv after parsing: ['/home/tmdag/.vscode/extensions/ms-python.python-2019.4.12954/pythonFiles/ptvsd_launcher.py']

D00000.014: sys.prefix: ('/usr',)

D00000.014: sys.base_prefix: /usr

D00000.014: site.getusersitepackages(): /home/tmdag/.local/lib/python3.7/site-packages

D00000.014: site.getsitepackages(): ['/usr/local/lib64/python3.7/site-packages', '/usr/local/lib/python3.7/site-packages', '/usr/lib64/python3.7/site-packages', '/usr/lib/python3.7/site-packages']

D00000.015: Folder with "site-packages" in sys.path: /usr/local/lib64/python3.7/site-packages

D00000.015: Folder with "site-packages" in sys.path: /usr/local/lib/python3.7/site-packages

D00000.015: Folder with "site-packages" in sys.path: /usr/lib64/python3.7/site-packages

D00000.015: Folder with "site-packages" in sys.path: /usr/lib/python3.7/site-packages

D00000.015: sys.argv after patching: ['/home/tmdag/pyproject/mypython.py']

D00000.015: Starting client daemon on ('localhost', 38491).

D00000.016: IDE <-- {
                "type": "event",
                "seq": 0,
                "event": "output",
                "body": {
                    "category": "telemetry",
                    "output": "ptvsd",
                    "data": {
                        "version": "4.2.8"
                    }
                }
            }

D00000.018: IDE --> {
                "command": "launch",
                "arguments": {
                    "name": "Python: Current File",
                    "type": "python",
                    "request": "launch",
                    "program": "/home/tmdag/pyproject/mypython.py",
                    "console": "integratedTerminal",
                    "env": {
                        "PTVSD_LOG_DIR": "/home/tmdag/Desktop/logs"
                    },
                    "noDebug": true,
                    "pythonPath": "/usr/bin/python3",
                    "cwd": "/home/tmdag/Develop/Standalone/python/mypython",
                    "envFile": "/home/tmdag/Develop/Standalone/python/mypython/.env",
                    "stopOnEntry": false,
                    "showReturnValue": true,
                    "internalConsoleOptions": "neverOpen",
                    "debugOptions": [
                        "ShowReturnValue",
                        "RedirectOutput"
                    ],
                    "justMyCode": true,
                    "workspaceFolder": "/home/tmdag/Develop/Standalone/python/mypython",
                    "__sessionId": "103051a5-b6f9-4546-a784-155f924b274c"
                },
                "type": "request",
                "seq": 2
            }

D00000.019: (while handling {'command': 'launch', 'type': 'request', 'seq': 2})
            IDE <-- {
                "type": "response",
                "seq": 1,
                "request_seq": 2,
                "success": true,
                "command": "launch",
                "message": "",
                "body": {}
            }

I00000.019: Running file /home/tmdag/pyproject/mypython.py

D00000.019: IDE --> {
                "command": "initialize",
                "arguments": {
                    "clientID": "vscode",
                    "clientName": "Visual Studio Code",
                    "adapterID": "python",
                    "pathFormat": "path",
                    "linesStartAt1": true,
                    "columnsStartAt1": true,
                    "supportsVariableType": true,
                    "supportsVariablePaging": true,
                    "supportsRunInTerminalRequest": true,
                    "locale": "en-us"
                },
                "type": "request",
                "seq": 1
            }

D00000.019: Adding /home/tmdag/pyproject to sys.path.

D00000.019: (while handling {'command': 'initialize', 'type': 'request', 'seq': 1})
            IDE <-- {
                "type": "response",
                "seq": 2,
                "request_seq": 1,
                "success": true,
                "command": "initialize",
                "message": "",
                "body": {
                    "supportsCompletionsRequest": true,
                    "supportsConditionalBreakpoints": true,
                    "supportsConfigurationDoneRequest": true,
                    "supportsDebuggerProperties": true,
                    "supportsDelayedStackTraceLoading": true,
                    "supportsEvaluateForHovers": true,
                    "supportsExceptionInfoRequest": true,
                    "supportsExceptionOptions": true,
                    "supportsHitConditionalBreakpoints": true,
                    "supportsLogPoints": true,
                    "supportsModulesRequest": true,
                    "supportsSetExpression": true,
                    "supportsSetVariable": true,
                    "supportsValueFormattingOptions": true,
                    "supportTerminateDebuggee": true,
                    "supportsGotoTargetsRequest": true,
                    "exceptionBreakpointFilters": [
                        {
                            "filter": "raised",
                            "label": "Raised Exceptions",
                            "default": false
                        },
                        {
                            "filter": "uncaught",
                            "label": "Uncaught Exceptions",
                            "default": true
                        }
                    ]
                }
            }

D00000.020: (while handling {'command': 'initialize', 'type': 'request', 'seq': 1})
            IDE <-- {
                "type": "event",
                "seq": 3,
                "event": "initialized",
                "body": {}
            }

D00000.087: IDE --> {
                "command": "setBreakpoints",
                "arguments": {
                    "source": {
                        "name": "mypython.py",
                        "path": "/home/tmdag/pyproject/mypython.py"
                    },
                    "lines": [
                        32
                    ],
                    "breakpoints": [
                        {
                            "line": 32
                        }
                    ],
                    "sourceModified": false
                },
                "type": "request",
                "seq": 4
            }

D00000.087: (while handling {'command': 'setBreakpoints', 'type': 'request', 'seq': 4})
            IDE <-- {
                "type": "response",
                "seq": 4,
                "request_seq": 4,
                "success": true,
                "command": "setBreakpoints",
                "message": "",
                "body": {}
            }

D00000.132: IDE --> {
                "command": "disconnect",
                "arguments": {
                    "restart": false
                },
                "type": "request",
                "seq": 5
            }

D00000.132: (while handling {'command': 'disconnect', 'type': 'request', 'seq': 5})
            Killing remaining subprocesses: PID=[]

D00000.133: (while handling {'command': 'disconnect', 'type': 'request', 'seq': 5})
            Disconnecting.

D00000.133: (while handling {'command': 'disconnect', 'type': 'request', 'seq': 5})
            Handling disconnecting session

```


I'll take a look at this.
I'm able to reproduce this (on Linux) with the code below (it seems it can be any script that does not end too fast):

```
import time
while True:
    time.sleep(.1)
```

I've instrumented `ipcjson.py` to see what's going on when it receives the `setBreakpoints` and it seems that it does nothing in both cases (when breakpoints are received or not) -- in both cases it just calls `on_invalid_request` (from the dummy `MESSAGE_PROCESSOR` in ptvsd/runner.py) which just returns `self.send_response(request, success=True)`.

The difference is that afterwards, `ptvsd` receives a `disconnect` message from VSCode if breakpoints are passed and if no breakpoints are passed it proceeds to receive a `configurationDone` and `threads` (as a note, even by changing the `disconnect` in the failing case to do nothing (just send `self.send_response(request, success=True)`),  the `python` process is killed right afterwards anyways).

Enabling `faulthandler` didn't acknowledge any crash in the application.

To me it doesn't seem like the issue lies on `ptvsd` (it seems like VSCode is promptly killing the `ptvsd` process in the case that breakpoints are passed).

@karthiknadig should this be passed to someone in the VSCode client side?
I suspect the reason why it terminates is because the `setBreakpoints` request is required by the spec to have a response with the matching number of breakpoints. At least according to the spec, it can just be a sequence of `{"verified": false}`, since everything else is optional, even ID.
@int19h you're right, it seems that the problem is that the response was not a valid breakpoints response. I'll provide a pull request shortly.

Resolving patch:
diff --git a/src/ptvsd/runner.py b/src/ptvsd/runner.py
--- a/src/ptvsd/runner.py
+++ b/src/ptvsd/runner.py
@@ -39,6 +39,19 @@ class Daemon(DaemonBase):
 
     class SESSION(DebugSession):
         class MESSAGE_PROCESSOR(VSCLifecycleMsgProcessor):
+
+            def on_setBreakpoints(self, request, args):
+                # Note: breakpoints is required (vscode will terminate
+                # the debugger if that's not the case).
+                # See: https://github.com/microsoft/ptvsd/issues/1408
+                self.send_response(
+                    request,
+                    success=True,
+                    breakpoints=(
+                        [{'verified': False}] * len(args.get('breakpoints', ()))
+                    )
+                )
+
             def on_invalid_request(self, request, args):
                 self.send_response(request, success=True)
 


Test patch:
diff --git a/tests/func/test_run.py b/tests/func/test_run.py
--- a/tests/func/test_run.py
+++ b/tests/func/test_run.py
@@ -76,7 +76,8 @@ def code_to_debug():
     with DebugSession() as session:
         session.no_debug = True
         session.initialize(target=(run_as, code_to_debug), start_method='launch', use_backchannel=True)
-        session.set_breakpoints(code_to_debug, [3])
+        breakpoints = session.set_breakpoints(code_to_debug, [3, 4])
+        assert breakpoints == [{'verified': False}, {'verified': False}]
         session.start_debugging()
 
         session.write_json(None)