Repository: microsoft/ptvsd

Issue:
Debugger not working properly.
_From @JasonWorks on October 11, 2018 17:46_

## Environment data

- VS Code version: 1.27.0 and 1.28.0
- Extension version (available under the Extensions sidebar): 2018.9.0
- OS and version: Debian x86-64 Linux Kernal: 4.17.0
- Python version (& distribution if applicable, e.g. Anaconda): 2.73
- Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A 
- Relevant/affected Python packages and their versions: None

## Actual behavior

Breakpoint pauses execution, but provides no debugger functionality

## Expected  behavior

I'm expecting the debugger to stop at the breakpoint and present me with the various variables and call stack.

## Steps to reproduce:
0. Previous VS code uninstalled, ~/.config/Code deleted, ~/.vscode deleted.  Clean VS Code installed, python extension installed.  
1. Construct a very simple Hello.py: 
msg = "Hello"
print(msg)
2. Place breakpoint on print line.
3. Start debugger.
4a. For vscode 1.27.0, the debugger toolbar comes up with Continue, Step Over, Step Into, Step Out, Restart, and Stop all enabled - breakpoint appears to be hit and Call Stack says "Paused on breakpoint", but no variables or call stack information presented.  If I click on Continue, Step Over, Step Into, Step Out then 'Hello' is printed and the program finishes it's run (this is true even if I place the break point on the msg definition line). 
4b. For vscode 1.28.0, the debugger toolbar comes up with Pause, Restart, and Stop enabled - breakpoint appears to be hit but none of the other buttons are enabled.  Call Stack says "Paused on breakpoint", but no variables or call stack information presented.  Unable to do anything meaningful at this point other than Stop - Pause does nothing.  
4c. If the breakpoint was not set then 'Hello' is printed out in the terminal, the process finishes, and the prompt shows.
![screenshot from 2018-10-11 10-22-25](https://user-images.githubusercontent.com/27971992/46823307-ad126680-cd42-11e8-8e6b-12da2b7d67d7.png)

## Logs
Output for `Python` in the `Output` panel (`View`→`Output`, change the drop-down the upper-right of the `Output` panel to `Python`)

##########Linting Output - pylint##########

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)


Output from `Console` under the `Developer Tools` panel (toggle Developer Tools on under `Help`)
INFO no standard startup: not a new window


_Copied from original issue: Microsoft/vscode-python#2865_


Discussion:
Please try the following:

* Go into your `.vscode/launch.json` file and add the following setting in the configuration you are using.
```js
"logToFile": true,
```
* Start debugging, once done go into the command palette and select the command `Extensions: Open Extensions Folder`
* From there locate the python extension directory `ms-python:2018.xxx`
* In that directory look for `debug.log`, upload the contents of that file here.
_From @JasonWorks on October 11, 2018 18:17_

[debug.log](https://github.com/Microsoft/vscode-python/files/2470159/debug.log)

_From @JasonWorks on October 15, 2018 16:55_

Thoughts?
_From @mikinushu on October 15, 2018 19:26_

Hi,
Same issue here.
As you can see: **PAUSED ON BREAKPOINT**
![image](https://user-images.githubusercontent.com/6978721/46972749-ef7fce80-d0c7-11e8-9245-ce163d401712.png)


**debug.log content:**
10:11:01 PM, 10/15/2018
Started @ Mon Oct 15 2018 22:11:01 GMT+0300 (Jerusalem Daylight Time)
From Client:
Content-Length: 313

{"command":"initialize","arguments":{"clientID":"vscode","clientName":"Visual Studio Code","adapterID":"python","pathFormat":"path","linesStartAt1":true,"columnsStartAt1":true,"supportsVariableType":true,"supportsVariablePaging":true,"supportsRunInTerminalRequest":true,"locale":"en-us"},"type":"request","seq":1}
To Client:
Content-Length: 687

{"seq":1,"type":"response","request_seq":1,"command":"initialize","success":true,"body":{"supportsExceptionInfoRequest":true,"supportsConfigurationDoneRequest":true,"supportsConditionalBreakpoints":true,"supportsSetVariable":true,"supportsExceptionOptions":true,"supportsEvaluateForHovers":true,"supportsModulesRequest":true,"supportsValueFormattingOptions":true,"supportsHitConditionalBreakpoints":true,"supportsSetExpression":true,"supportsLogPoints":true,"supportTerminateDebuggee":true,"supportsCompletionsRequest":true,"exceptionBreakpointFilters":[{"filter":"raised","label":"Raised Exceptions","default":false},{"filter":"uncaught","label":"Uncaught Exceptions","default":true}]}}
From Client:
Content-Length: 617

{"command":"launch","arguments":{"name":"Python: Current File (Integrated Terminal)","type":"python","request":"launch","program":"c:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\Dev\\trials.py","console":"integratedTerminal","logToFile":true,"pythonPath":"C:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\python.exe","cwd":"c:\\Users\\Michael\\Anaconda3\\envs\\mod-prj","envFile":"c:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\.env","stopOnEntry":false,"internalConsoleOptions":"neverOpen","debugOptions":["RedirectOutput","FixFilePathCase"],"__sessionId":"8d4192bf-7bd8-4215-96ca-70da1c204144"},"type":"request","seq":2}
To Client:
Content-Length: 482

{"command":"runInTerminal","arguments":{"kind":"integrated","title":"Python Debug Console","cwd":"c:\\Users\\Michael\\Anaconda3\\envs\\mod-prj","args":["C:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\python.exe","c:\\Users\\Michael\\.vscode\\extensions\\ms-python.python-2018.9.0\\pythonFiles\\experimental\\ptvsd_launcher.py","53246","c:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\Dev\\trials.py"],"env":{"PYTHONIOENCODING":"UTF-8","PYTHONUNBUFFERED":"1"}},"type":"request","seq":2}
From Client:
Content-Length: 94

{"type":"response","seq":3,"command":"runInTerminal","request_seq":2,"success":true,"body":{}}
To Client:
Content-Length: 617


To Client:
{"command":"launch","arguments":{"name":"Python: Current File (Integrated Terminal)","type":"python","request":"launch","program":"c:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\Dev\\trials.py","console":"integratedTerminal","logToFile":true,"pythonPath":"C:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\python.exe","cwd":"c:\\Users\\Michael\\Anaconda3\\envs\\mod-prj","envFile":"c:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\.env","stopOnEntry":false,"internalConsoleOptions":"neverOpen","debugOptions":["RedirectOutput","FixFilePathCase"],"__sessionId":"8d4192bf-7bd8-4215-96ca-70da1c204144"},"type":"request","seq":2}
To Client:
Content-Length: 313


To Client:
{"command":"initialize","arguments":{"clientID":"vscode","clientName":"Visual Studio Code","adapterID":"python","pathFormat":"path","linesStartAt1":true,"columnsStartAt1":true,"supportsVariableType":true,"supportsVariablePaging":true,"supportsRunInTerminalRequest":true,"locale":"en-us"},"type":"request","seq":1}
To Client:
Content-Length: 130

{"type": "event", "seq": 0, "event": "output", "body": {"category": "telemetry", "output": "ptvsd", "data": {"version": "4.1.3"}}}Content-Length: 113

{"type": "response", "seq": 1, "request_seq": 2, "success": true, "command": "launch", "message": "", "body": {}}
To Client:
Content-Length: 787

{"type": "response", "seq": 2, "request_seq": 1, "success": true, "command": "initialize", "message": "", "body": {"supportsCompletionsRequest": true, "supportsConditionalBreakpoints": true, "supportsConfigurationDoneRequest": true, "supportsDebuggerProperties": true, "supportsEvaluateForHovers": true, "supportsExceptionInfoRequest": true, "supportsExceptionOptions": true, "supportsHitConditionalBreakpoints": true, "supportsLogPoints": true, "supportsModulesRequest": true, "supportsSetExpression": true, "supportsSetVariable": true, "supportsValueFormattingOptions": true, "supportTerminateDebuggee": true, "exceptionBreakpointFilters": [{"filter": "raised", "label": "Raised Exceptions", "default": false}, {"filter": "uncaught", "label": "Uncaught Exceptions", "default": true}]}}Content-Length: 63

{"type": "event", "seq": 3, "event": "initialized", "body": {}}
From Client:
Content-Length: 229

{"command":"setBreakpoints","arguments":{"source":{"name":"trials.py","path":"C:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\Dev\\trials.py"},"lines":[7],"breakpoints":[{"line":7}],"sourceModified":false},"type":"request","seq":4}
To Client:
Content-Length: 176


To Client:
{"type": "response", "seq": 4, "request_seq": 4, "success": true, "command": "setBreakpoints", "message": "", "body": {"breakpoints": [{"id": 1, "verified": true, "line": 7}]}}
From Client:
Content-Length: 99

{"command":"setExceptionBreakpoints","arguments":{"filters":["uncaught"]},"type":"request","seq":5}
To Client:
Content-Length: 130


To Client:
{"type": "response", "seq": 5, "request_seq": 5, "success": true, "command": "setExceptionBreakpoints", "message": "", "body": {}}
From Client:
Content-Length: 56

{"command":"configurationDone","type":"request","seq":6}
To Client:
Content-Length: 124

{"type": "response", "seq": 6, "request_seq": 6, "success": true, "command": "configurationDone", "message": "", "body": {}}
From Client:
Content-Length: 46

{"command":"threads","type":"request","seq":7}
To Client:
Content-Length: 92

{"type": "event", "seq": 7, "event": "thread", "body": {"reason": "started", "threadId": 1}}
To Client:
Content-Length: 158

{"type": "response", "seq": 8, "request_seq": 7, "success": true, "command": "threads", "message": "", "body": {"threads": [{"id": 1, "name": "MainThread"}]}}
To Client:
Content-Length: 203


To Client:
{"type": "event", "seq": 9, "event": "process", "body": {"name": "c:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\Dev\\trials.py", "systemProcessId": 6320, "isLocalProcess": true, "startMethod": "launch"}}
To Client:
Content-Length: 160

{"type": "event", "seq": 10, "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1, "text": null, "description": null, "preserveFocusHint": false}}
From Client:
Content-Length: 46

{"command":"threads","type":"request","seq":8}
To Client:
Content-Length: 159

{"type": "response", "seq": 11, "request_seq": 8, "success": true, "command": "threads", "message": "", "body": {"threads": [{"id": 1, "name": "MainThread"}]}}
From Client:
Content-Length: 103

{"command":"stackTrace","arguments":{"threadId":1,"startFrame":0,"levels":20},"type":"request","seq":9}
To Client:
Content-Length: 204

{"type": "event", "seq": 12, "event": "module", "body": {"reason": "new", "module": {"id": 1, "package": null, "path": "C:\\Users\\Michael\\Anaconda3\\envs\\mod-prj\\Dev\\trials.py", "name": "__main__"}}}
From Client:
Content-Length: 104

{"command":"stackTrace","arguments":{"threadId":1,"startFrame":0,"levels":20},"type":"request","seq":10}
To Client:
Content-Length: 153

{"type": "response", "seq": 13, "request_seq": 9, "success": true, "command": "stackTrace", "message": "", "body": {"stackFrames": [], "totalFrames": 0}}
To Client:
Content-Length: 154


To Client:
{"type": "response", "seq": 14, "request_seq": 10, "success": true, "command": "stackTrace", "message": "", "body": {"stackFrames": [], "totalFrames": 0}}
From Client:
Content-Length: 47

{"command":"threads","type":"request","seq":11}
To Client:
Content-Length: 160

{"type": "response", "seq": 15, "request_seq": 11, "success": true, "command": "threads", "message": "", "body": {"threads": [{"id": 1, "name": "MainThread"}]}}
From Client:
Content-Length: 104

{"command":"stackTrace","arguments":{"threadId":1,"startFrame":0,"levels":20},"type":"request","seq":12}
To Client:
Content-Length: 154

{"type": "response", "seq": 16, "request_seq": 12, "success": true, "command": "stackTrace", "message": "", "body": {"stackFrames": [], "totalFrames": 0}}
From Client:
Content-Length: 80

{"command":"disconnect","arguments":{"restart":false},"type":"request","seq":13}
Socket Error
check and shutdown
shutdown
Sending Terminated Event
To Client:
Content-Length: 45


onEventTerminated
To Client:
{"seq":0,"type":"event","event":"terminated"}
killing process
Kill process now
Shutting down debug session
disposing

_From @berkobob on October 16, 2018 10:25_

Same bug here:
Windows 10 Pro 10.0.17137 build 17134
VS Code 1.28.1
ms-python.python 2018.9.0

![capture](https://user-images.githubusercontent.com/6568107/47027117-be9ea880-d15e-11e8-855d-c05b82163685.PNG)

_From @heniu66 on October 24, 2018 10:30_

I have the same problem. Consistently when the Anaconda env is used **(and it is not the base environment).** Tested on Windows 10 Pro and two Ubuntu 18.4 (bionic beaver) machines. 
VS Code 1.28.2 (but it didn't work earlier too)
Debugger stops on the breakpoint but:
- no line indicator is displayed in monitor
- in the upper debug panel you can only press pause, reload and stop
- no variables and stack displayed
- "Paused on breakpoint" displayed

![image](https://user-images.githubusercontent.com/8215975/47424704-93ccd980-d788-11e8-9bad-5fc578f56adb.png)


_From @d3r3kk on October 26, 2018 2:25_

Hmmm. Seems like we are getting 0-length stackframes back from these scenarios. I'm probably going to need some help on this issue @DonJayamanne.

From the user's that have reported the issue along with the `debug.log` information, I'm seeing responses from PTVSD that look like this:

```json
{
  "type": "response",
  "seq": 13,
  "request_seq": 9,
  "success": true,
  "command": "stackTrace",
  "message": "",
  "body": {
    "stackFrames": [], // <== This is not what I see locally!
    "totalFrames": 0
  }
}
```

Here's what the request/response interaction looks like for me, vs. what it looks like for the others:

| My Session | User's Session |
|------------|----------------|
|	REQ: "initialize"	|	REQ: "initialize"	|
|	RES: "initialize"	|	RES: "initialize",	|
|	REQ: "launch"	|	REQ: "launch"	|
|	REQ: "runInTerminal"	|	REQ: "runInTerminal"	|
|	RES: "runInTerminal"	|	RES: "runInTerminal"	|
|	REQ: "launch"	|	REQ: "launch"	|
|	REQ: "initialize"	|	REQ: "initialize"	|
|	RES: "launch"	|	RES: "launch"	|
|	RES: "initialize"	|	RES: "initialize"	|
|	REQ: "setBreakpoints"	|	REQ: "setBreakpoints"	|
|	RES: "setBreakpoints"	|	RES: "setBreakpoints"	|
|	REQ: "setExceptionBreakpoints"	|	REQ: "setExceptionBreakpoints"	|
|	RES: "setExceptionBreakpoints"	|	RES: "setExceptionBreakpoints"	|
|	REQ: "configurationDone"	|	REQ: "configurationDone"	|
|		|	**REQ: "threads"**	|
|	RES: "configurationDone"	|	RES: "configurationDone"	|
|		|	**RES: "threads"**	|
|	REQ: "threads"	|	REQ: "threads"	|
|	RES: "threads"	|	RES: "threads"	|
|		|	**REQ: "stackTrace"**	|
|	REQ: "threads"	|	REQ: "threads"	|
|		|	**REQ: "stackTrace"**	|
|		|	**RES: "stackTrace"**	|
|	RES: "threads"	|	RES: "threads"	|
|		|	**RES: "stackTrace"**	|
|	REQ: "stackTrace"	|	REQ: "stackTrace"	|
|	RES: "stackTrace"	|	RES: "stackTrace"	|
|	**REQ: "scopes"**	|		|
|	**RES: "scopes"**	|		|
|	**REQ: "variables"**	|		|
|	**REQ: "threads"**	|		|
|	**RES: "variables"**	|		|
|	**RES: "threads"**	|		|
|	**REQ: "continue"**	|		|
|	**RES: "continue"**	|		|
|	REQ: "disconnect"	|	REQ: "disconnect"	|
|	**RES: "disconnect"**	|		|


Attached: debug.log parsed data from myself and the user reports in a working vscode workspace. Note that I used Win10, latest vscode-insiders, latest revision of the extension, ptvsd 4.1.3, and Python 2.7.14 using `virtualenv`.

[2865_debugger_gets_lost.zip](https://github.com/Microsoft/vscode-python/files/2517364/2865_debugger_gets_lost.zip)

@JasonWorks 
I'm closing this issue as you have created a file in the Environment directory and are debugging that.
Please move that file out of the environment directory and start debugging.
Please ensure the Python Environment directory is not opened in VSCode (that's generally not considered normal practice).

Note, in your `launch.json`, the program you are debugging is `c:\Users\Michael\Anaconda3\envs\mod-prj\Dev\trials.py`
_From @JasonWorks on October 30, 2018 23:41_

I'm not following.  What do you mean by Environment directory?  The code I'm working with is in /usr/local/google/home/jasonworks/src/test/Hello.py.  That directory/file isn't in my environment's path or anywhere related to vscode as far as I know.  Please elaborate.
_From @JasonWorks on October 30, 2018 23:50_

I think you are seeing the image cut off from resizing and the terminal maintains the original column spacing.  Here is the full terminal command...
/usr/local/google/home/jasonworks/src/test $ cd /usr/local/google/home/jasonworks/src/test ; env "PYTHONIOENCODING=UTF-8" "PYTHONUNBUFFERED=1" python /usr/local/google/home/jasonworks/.vscode/extensions/ms-python.python-2018.9.1/pythonFiles/experimental/ptvsd_launcher.py 46101 /usr/local/google/home/jasonworks/src/test/hello.py

_From @JasonWorks on October 30, 2018 23:51_

Please reopen.
_From @JasonWorks on October 31, 2018 0:16_

@DonJayamanne 
Your note about launch.json is referring to a another commentor's note.  I'm not Michael and I don't use Anaconda3.  Please reopen.
@heniu66 @mikinushu
you have created a file in the Environment directory and are debugging that.
Please move that file out of the environment directory and start debugging.
Please ensure the Python Environment directory is not opened in VSCode (that's generally not considered normal practice).

Note, in your `launch.json`, the program you are debugging is `c:\Users\Michael\Anaconda3\envs\mod-prj\Dev\trials.py`
_From @berkobob on October 31, 2018 9:5_


[debug.log](https://github.com/Microsoft/vscode-python/files/2533506/debug.log)

_From @berkobob on October 31, 2018 16:54_

I get the same problem with Linux Mint 19 Cinnamon 64-bit

I've just noticed that I only seem to have this problem when using a virtual environment. When there's no venv it works fine. 
_From @mattpepin on November 9, 2018 17:0_

I have exactly the same issue as @berkobob but in Windows 10, using either VSCode/VSCode Insiders.  I found that If there's a pyvenv.cfg file, the breakpoints don't work correctly. If I remove it, it raises an Exception on an unknown import, but it pauses correctly on the good line of the import.
_From @berkobob on November 9, 2018 18:42_

I believe that the pyvenv.cfg file is created by venv so I created an environment with virtualenv instead (i.e. no pyvenv.cfg created) and I had the same problem. It would be great to get this problem fixed.
_From @JasonWorks on November 9, 2018 23:44_

I'm not sure if it was occurring earlier, but I'm now noticing that when "Paused on Bbreakpoint" (4a in original post) the orange status bar is indicating... Installing packages... and if I move my mouse over the message it says... Installing package 'Razor Language Server (Linux / x64)' - not sure what that is all about.  

I'm still attempting to continue to use vscode without a python debugger, but may have to go elsewhere if progress cannot be made.  @DonJayamanne, please provide an update if you can.
_From @sellweek on November 10, 2018 16:4_

Same problem for me, Arch Linux, 64-bit, VS Code 1.28.2, ms-python.python 2018.10.1.
[debug24330.log](https://github.com/Microsoft/vscode-python/files/2568537/debug24330.log)

_From @Verkurkie on November 12, 2018 16:0_

Similar issue here on both VScode and VS Code Insider. I see this issue when running my app with the "Python : Django" profile but the "hello.py" exercise works! :-\

With breakpoints turned **on**, the debugger starts but just sits there! No stack, nothing ...
With breakpoints turned **off**, the code runs fine

Platform: Windows 10 x64
VS Code versions: 1.27.1 / 1.29.0-insider
Extension: ms-python.python-2018.10.1

my Django profile in launch.json (obfuscated for privacy):
```
    {
      "name": "Python: Django",
      "type": "python",
      "request": "launch",
      "logToFile": true,
      "console": "integratedTerminal",
      "env": {
           "CASSANDRA_ENV":"dev",
           "ES_LOGGING_INDEX":"{app_name}_{user}_development_logging",
           "ES_OUTPUT_INDEX":"{app_name}_{user}_development",
           "PYTHONPATH":"C:\\Users\\{user}\\dev\\{app_folder}/{sub_folder}/{app_name}/",
           "ELASTIC_APM_SERVER_URL":"http://{server}:8200",
           "ELASTIC_APM_ENVIRONMENT":"test"
      },
      "program": "${workspaceFolder}/manage.py",
      "args": [
        "runserver",
        "8080",
        "--noreload",
        "--nothreading"
      ],
      "django": true
    },
```

My debug log (obfuscated for privacy):
[debug_issue.log](https://github.com/Microsoft/vscode-python/files/2572576/debug_issue.log)

Hope this can be resolved soon!
Thanks for all the awesome work you do!
The debugger includes `sys.prefix`, `sys.base_prefix`, and `sys.real_prefix` in the list of std lib prefixes. On some systems this is just `/usr`. This means that the debugger will consider any user code as library code on those systems.

Resolving patch:
diff --git a/ptvsd/wrapper.py b/ptvsd/wrapper.py
--- a/ptvsd/wrapper.py
+++ b/ptvsd/wrapper.py
@@ -176,13 +176,10 @@ def dont_trace_ptvsd_files(file_path):
 
 pydevd_frame.file_tracing_filter = dont_trace_ptvsd_files
 
-
-STDLIB_PATH_PREFIXES = [os.path.normcase(sys.prefix)]
-if hasattr(sys, 'base_prefix'):
-    STDLIB_PATH_PREFIXES.append(os.path.normcase(sys.base_prefix))
-if hasattr(sys, 'real_prefix'):
-    STDLIB_PATH_PREFIXES.append(os.path.normcase(sys.real_prefix))
-
+# NOTE: Previously this included sys.prefix, sys.base_prefix and sys.real_prefix
+# On some systems those resolve to '/usr'. That means any user code will 
+# also be treated as library code.
+STDLIB_PATH_PREFIXES = []
 if hasattr(site, 'getusersitepackages'):
     site_paths = site.getusersitepackages()
     if isinstance(site_paths, list):


Test patch: