Repository: microsoft/ptvsd

Issue:
Module name conflict while debugging with -m
## Environment data

- PTVSD version: ptvsd 4.2.6
- OS and version: Windows-10-10.0.17134 AMD64
- Python version (& distribution if applicable, e.g. Anaconda): CPython 3.7.2 (64-bit)
- Using VS Code or Visual Studio: VS Code

## Actual behavior

If I have a module with name 'thread' in the package that I want to debug with -m option, ptvsd fails to start. This does not happen when I use a wrapper main.py for my package. It seems that ptvsd wants to import standard thread module from python installation, but it tries to import my thread module instead, because of the wrong sys.path order while executing ptvsd startup code. Log message is attached at the bottom. 

## Expected  behavior

Debugger starts successfully.

## Steps to reproduce:

Actually, I looked into the problem and made a fix. I am not sure this is the right way to do.
The following code near ptsvd/\_\_main\_\_.py:215 causes the problem. Adding  the current directory to the head of sys.path to call find_spec() might have to be reverted at the end of elif block. I believe '' will be added by python interpreter anyway later.

> 
    elif opts.target_kind == 'module':
        # Add current directory to path, like Python itself does for -m. This must
        # be in place before trying to use find_spec below to resolve submodules.
        sys.path.insert(0, '')

        # We want to do the same thing that run_module() would do here, without
        # actually invoking it. On Python 3, it's exposed as a public API, but
        # on Python 2, we have to invoke a private function in runpy for this.
        # Either way, if it fails to resolve for any reason, just leave argv as is.
        try:
            if sys.version_info >= (3,):
                from importlib.util import find_spec
                spec = find_spec(opts.target)
                if spec is not None:
                    sys.argv[0] = spec.origin
            else:
                _, _, _, sys.argv[0] = runpy._get_module_details(opts.target)
        except Exception:
            ptvsd.log.exception('Error determining module path for sys.argv')
        #####################
        ## Added code    
        del sys.path[0]
        #####################
>


LOG MESSAGE from ptvsd

>
I00000.000: Windows-10-10.0.17134 AMD64
            CPython 3.7.2 (64-bit)
            ptvsd 4.2.6

I00000.000: main(['c:\\Users\\gegogi.TS.000\\.vscode\\extensions\\ms-python.python-2019.3.6352\\pythonFiles\\ptvsd_launcher.py', '--client', '--host', 'localhost', '--port', '63124', '-m', 'natuum.server'])

I00000.000: sys.argv after parsing: ['c:\\Users\\gegogi.TS.000\\.vscode\\extensions\\ms-python.python-2019.3.6352\\pythonFiles\\ptvsd_launcher.py']

D00000.000: sys.argv after patching: ['F:\\Code\\nt-src\\sources\\python\\natuum\\server\\__init__.py']

D00000.000: Installing pydevd hooks.

D00000.000: Monkey-patching multiprocess functions.

D00000.016: pydevd.settrace()

D00000.016: Starting client daemon on ('localhost', 63124).

E00000.016: Traceback (most recent call last):
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\daemon.py", line 380, in _start_session_safely
                self._start_session(threadname, **kwargs)
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\daemon.py", line 526, in _start_session
                **kwargs
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\daemon.py", line 488, in _start_session
                **kwargs
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\_util.py", line 322, in start
                self._is_running = self._start(*args, **kwargs)
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\session.py", line 117, in _start
                self._msgprocessor = self._new_msg_processor(**kwargs)
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\session.py", line 188, in _new_msg_processor
                **kwargs
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\session.py", line 112, in _new_msg_processor
                **kwargs
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\wrapper.py", line 1221, in __init__
                timeout=timeout,
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\wrapper.py", line 1023, in __init__
                timeout=timeout,
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\wrapper.py", line 831, in __init__
                timeout=timeout,
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\ipcjson.py", line 86, in __init__
                super(SocketIO, self).__init__(*args, **kwargs)
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\ipcjson.py", line 220, in __init__
                import thread
              File "F:\Code\nt-src\sources\python\natuum\thread.py", line 8, in <module>
                class TimedAtom( thing.Thread.Atom ):
            AttributeError: module 'thing' has no attribute 'Thread'
            

D00000.016: Handling closing session
>




Discussion:
I'm not entirely clear how the *inside* of your package ends up on the path. Are you running it from within the `natuum` directory? Or does it change the directory at some point after starting? I would expect the current directly to be the one inside which `natuum` is a subdirectory.

`runpy.run_module` does not add that `sys.path` entry automatically on all Python versions, unfortunately, hence why we have to do it there. And Python itself also adds the current directory as the first (not the last) entry when you do `-m`. So if you have something like this:
```py
# foo.py
import json

# json.py
print(__file__)
```
and try running it with `py -3 -m foo.py` from the directory where both files are located, it loads `json.py`, not the stdlib module (and consequently, any package that tries to import `json` will break). The debugger is meant to behave identically in that regard. 

We can mitigate this somewhat by pre-importing as much as we can before fiddling with `sys.path`. I don't think we can make it foolproof, though, because sometimes deferred imports are necessary because of circular dependencies, and some standard modules also do that, which we can't control. And, of course, we can't mitigate it for standard or site-packages modules that your code might import directly. So, in general, I would recommend not having local modules with the same name as standard modules in the directory from which you're running, whether running under debugger or without - you never know which of your dependencies might try to import them.

This case is a bit special though since `thread` is not a standard module in Python 3 - it was in Python 2, but it got renamed to `_thread`. Problem is, we try to import it without underscore first, then fall back to underscore if that fails. And in your case it succeeds with the wrong module instead. We should either do an explicit version check here, or just use `threading`. 

The easiest fix, though, is to have ipcjson.py just import `thread` on top level; the comment there that explains why it can't do so is obsolete. Then the import will happen early enough that it won't be affected by `sys.path` changes.
Yes, I can confirm that it is executed in `natuum` directory.
And I also can confirm it runs well after moving `import thread` part to the beginning of `ipcjson.py`.

```
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS F:\Code\nt-src\sources\python\natuum> cd 'f:\Code\nt-src\sources\python\natuum'; ${env:PYTHONIOENCODING}='UTF-8'; ${env:PYTHONUNBUFFERED}='1'; & 'F:/Code/nt-src/sources/Checked_x64/Pything.exe' 'c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\ptvsd_launcher.py' '--default' '--client' '--host' 'localhost' '--port' '50924' '-m' 'natuum.server'
E00000.015: cwd: F:\Code\nt-src\sources\python\natuum

            NoneType: None


E00000.062: Traceback (most recent call last):
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\daemon.py", line 380, in _start_session_safely
                self._start_session(threadname, **kwargs)
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\daemon.py", line 526, in _start_session
                **kwargs
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\daemon.py", line 488, in _start_session
                **kwargs
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\_util.py", line 322, in start
                self._is_running = self._start(*args, **kwargs)
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\session.py", line 117, in _start
                self._msgprocessor = self._new_msg_processor(**kwargs)
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\session.py", line 188, in _new_msg_processor
                **kwargs
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\session.py", line 112, in _new_msg_processor
                **kwargs
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\wrapper.py", line 1221, in __init__
                timeout=timeout,
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\wrapper.py", line 1023, in __init__
                timeout=timeout,
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\wrapper.py", line 831, in __init__
                timeout=timeout,
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\ipcjson.py", line 86, in __init__
                super(SocketIO, self).__init__(*args, **kwargs)
              File "c:\Users\gegogi.TS.000\.vscode\extensions\ms-python.python-2019.3.6352\pythonFiles\lib\python\ptvsd\ipcjson.py", line 220, in __init__
                import thread
              File "F:\Code\nt-src\sources\python\natuum\thread.py", line 8, in <module>
                class TimedAtom( thing.Thread.Atom ):
            AttributeError: module 'thing' has no attribute 'Thread'


PS F:\Code\nt-src\sources\python\natuum>
```
I believe the default working directory is going to be the one you have opened as folder in VS Code. So, in case of a Python package, it's usually best to open the parent directory - e.g. for this project, we usually open the root of the repo, or src/, but not src/ptvsd/. It can also be specified via `"cwd"` in launch.json. 

Resolving patch:
diff --git a/src/ptvsd/ipcjson.py b/src/ptvsd/ipcjson.py
--- a/src/ptvsd/ipcjson.py
+++ b/src/ptvsd/ipcjson.py
@@ -4,13 +4,6 @@
 
 from __future__ import print_function, with_statement, absolute_import
 
-# This module MUST NOT import threading in global scope. This is because
-# in a direct (non-ptvsd) attach scenario, it is loaded on the injected
-# debugger attach thread, and if threading module hasn't been loaded
-# already, it will assume that the thread on which it is being loaded is
-# the main thread. This will cause issues when the thread goes away
-# after attach completes.
-
 import errno
 import itertools
 import json
@@ -19,6 +12,7 @@
 from socket import create_connection
 import sys
 import time
+import threading
 import traceback
 
 import ptvsd.log
@@ -216,14 +210,10 @@ def __init__(self, *args, **kwargs):
         super(IpcChannel, self).__init__(*args, **kwargs)
         # This class is meant to be last in the list of base classes
         # Don't call super because object's __init__ doesn't take arguments
-        try:
-            import thread
-        except ImportError:
-            import _thread as thread
-        # TODO: switch to a single underscore for "private" variables.
+
         self.__seq = itertools.count()
         self.__exit = False
-        self.__lock = thread.allocate_lock()
+        self.__lock = threading.Lock()
         self.__message = []
         self._timeout = timeout
         self._fail_after = None


Test patch: