anon-iclr-submission/benchname-bug-localization huggingface dataset:
Repository: microsoft/ptvs
Issue: ptvsd can't be used as a program

The recent refactoring broke `python -m ptvsd`, because it's trying to import some names that have been moved around, such as `DEFAULT_PORT`.

Changed files:
Python/Product/PythonTools/ptvsd/__main__.py

diff --git a/Python/Product/PythonTools/ptvsd/__main__.py b/Python/Product/PythonTools/ptvsd/__main__.py
index bccbe8cab..930a27c4e 100644
--- a/Python/Product/PythonTools/ptvsd/__main__.py
+++ b/Python/Product/PythonTools/ptvsd/__main__.py
@@ -22,7 +22,7 @@ import sys
 from optparse import OptionParser
 from ptvsd.util import exec_file
 from ptvsd.debugger import DONT_DEBUG
-from ptvsd.attach_server import DEFAULT_PORT, enable_attach, wait_for_attach
+from ptvsd import DEFAULT_PORT, enable_attach, wait_for_attach
 
 parser = OptionParser(prog = 'ptvsd', usage = 'Usage: %prog [<option>]... <file> [- <args>]', version = '%prog ' + __version__)
 parser.add_option('-s', '--secret', metavar = '<secret>', help = 'restrict server to only allow clients that specify <secret> when connecting')
Repository: microsoft/ptvs
Issue: Jupyter client raw_input handling

Our current Jupyter client implementation for the REPL has the following pattern:

```python
            try:
                raw_input()
            except NameError():
                input()
```

Unfortunately, `except NameError()` should be `except NameError`, and so this causes multiple errors when a fatal error occurs.

Changed files:
Python/Product/PythonTools/ptvsd/repl/jupyter_client.py

diff --git a/Python/Product/PythonTools/ptvsd/repl/jupyter_client.py b/Python/Product/PythonTools/ptvsd/repl/jupyter_client.py
index c2c911e0d..51fcb9bc6 100644
--- a/Python/Product/PythonTools/ptvsd/repl/jupyter_client.py
+++ b/Python/Product/PythonTools/ptvsd/repl/jupyter_client.py
@@ -228,7 +228,7 @@ class JupyterClientBackend(ReplBackend):
             traceback.print_exc()
             try:
                 raw_input()
-            except NameError():
+            except NameError:
                 input()
             raise
 
@@ -369,7 +369,7 @@ class JupyterClientBackend(ReplBackend):
             traceback.print_exc()
             try:
                 raw_input()
-            except NameError():
+            except NameError:
                 input()
             self.exit_process()
 
@@ -407,7 +407,7 @@ class JupyterClientBackend(ReplBackend):
             traceback.print_exc()
             try:
                 raw_input()
-            except NameError():
+            except NameError:
                 input()
             self.exit_process()
Repository: microsoft/ptvs
Issue: Unhandled errors in Jupyter client

Any of our calls to `get_shell_msg` and `get_iopub_msg` could raise `zmq.error.ZMQError`, particularly when closing the process. We should handle these and do something more useful.

Similarly for `conn.recv` - `ConnectionResetError` could be raised, and we should do better handling.

Changed files:
Python/Product/PythonTools/ptvsd/ipcjson.py, Python/Product/PythonTools/ptvsd/repl/jupyter_client.py

diff --git a/Python/Product/PythonTools/ptvsd/ipcjson.py b/Python/Product/PythonTools/ptvsd/ipcjson.py
index 7d50c0abd..722545940 100644
--- a/Python/Product/PythonTools/ptvsd/ipcjson.py
+++ b/Python/Product/PythonTools/ptvsd/ipcjson.py
@@ -238,7 +238,13 @@ class IpcChannel(object):
         try:
             msg = self.__message.pop(0)
         except IndexError:
-            self._wait_for_message()
+            msg = None
+
+        if msg is None:
+            try:
+                self._wait_for_message()
+            except OSError:
+                return self.__exit
             try:
                 msg = self.__message.pop(0)
             except IndexError:
diff --git a/Python/Product/PythonTools/ptvsd/repl/jupyter_client.py b/Python/Product/PythonTools/ptvsd/repl/jupyter_client.py
index 51fcb9bc6..533ddb8fa 100644
--- a/Python/Product/PythonTools/ptvsd/repl/jupyter_client.py
+++ b/Python/Product/PythonTools/ptvsd/repl/jupyter_client.py
@@ -36,6 +36,7 @@ from ptvsd.util import to_bytes
 try:
     import jupyter_client
     import jupyter_client.manager
+    import zmq.error
 except ImportError:
     raise UnsupportedReplException("Jupyter mode requires the jupyter_client and ipykernel packages. " + traceback.format_exc())
 
@@ -362,6 +363,8 @@ class JupyterClientBackend(ReplBackend):
                     for callable in on_reply:
                         callable(m)
 
+        except zmq.error.ZMQError:
+            self.exit_process()
         except KeyboardInterrupt:
             self.exit_process()
         except:
@@ -400,6 +403,8 @@ class JupyterClientBackend(ReplBackend):
                     print("Received: " + m.msg_type + ":" + str(m) + "\\n")
                     self.write_stdout(str(m) + '\\n')
 
+        except zmq.error.ZMQError:
+            self.exit_process()
         except KeyboardInterrupt:
             self.exit_process()
         except:
Repository: microsoft/ptvs
Issue: ptvsd 2.2.0b1 and ptvs 2.2rc2 not connecting

Apparently, a regression as I've attached to process using ptvs 2.2rc and ptvsd 2.2.0b1 but after upgrading to ptvs 2.2rc2 I can no longer attach.  Subsequently downgraded to ptvs 2.2rc2.

Any hints on debugging the debugger connection (i.e. ptvsd).  Drop down to pdb (Python Debugger)?  There doesn't appear to be any other process using port 5678 ( `netstat -an` )

Here's my `pydevd_startup.py`

```
import json 
import sys 

print "starting... pydevd_startup.py"

if ':' not in config.version_id: 
  # The default server version_id does not contain ':'  
  sys.path.append("lib")  
  import ptvsd  #ptvsd.settrace() equivalent  
  ptvsd.enable_attach(secret = 'joshua')
  print "Hey World, eh"
  print "tcp://joshua@localhost:5678"

  ptvsd.wait_for_attach()

  if ptvsd.is_attached:
      print "Attached debugger, eh"
```

My startup file: `C:\\Program Files\\Google\\Cloud SDK\\google-cloud-sdk\\lib\\googlecloudsdk\\gcloud\\gcloud.py`

Script arguments:
`preview app run app.yaml --python-startup-script "pydevd_startup.py" --max-module-instances="default:1"`


Changed files:
Python/Product/PythonTools/ptvsd/setup.py

diff --git a/Python/Product/PythonTools/ptvsd/setup.py b/Python/Product/PythonTools/ptvsd/setup.py
index 4d525732a..601af4d66 100644
--- a/Python/Product/PythonTools/ptvsd/setup.py
+++ b/Python/Product/PythonTools/ptvsd/setup.py
@@ -18,7 +18,7 @@
 from distutils.core import setup
 
 setup(name='ptvsd',
-      version='2.2.0b1',
+      version='2.2.0rc2',
       description='Python Tools for Visual Studio remote debugging server',
       license='Apache License 2.0',
       author='Microsoft Corporation',
Repository: microsoft/ptvs
Issue: IPython doc strings, question mark does not work

It seems, that IPython interactive window does not support printing doc strings via "?", for example in normal ipython following statement:

```
>>numpy.abs?
```

is equivalent to:

```
>>print numpy.abs.__doc__
```

and double question mark "??" shows the source code if it is available. Are there any reasons for absence of this feature?

PTVS 2.2 RC2 
VS 2013 Express for Desktop
IPython 3.2.0


Changed files:
Python/Product/PythonTools/visualstudio_ipython_repl.py

diff --git a/Python/Product/PythonTools/visualstudio_ipython_repl.py b/Python/Product/PythonTools/visualstudio_ipython_repl.py
index e15aafb0d..ca08966e1 100644
--- a/Python/Product/PythonTools/visualstudio_ipython_repl.py
+++ b/Python/Product/PythonTools/visualstudio_ipython_repl.py
@@ -97,6 +97,18 @@ class VsShellChannel(DefaultHandler, ShellChannel):
         payload = content['payload']
         
         for item in payload:
+            data = item.get('data')
+            if data is not None:
+                try:
+                    # Could be named km.sub_channel for very old IPython, but
+                    # those versions should not put 'data' in this payload
+                    write_data = self._vs_backend.km.iopub_channel.write_data
+                except AttributeError:
+                    pass
+                else:
+                    write_data(data)
+                    continue
+
             output = item.get('text', None)
             if output is not None:
                 self._vs_backend.write_stdout(output)
Repository: microsoft/ptvs
Issue: Debugger warnings kill import performance

Split from https://github.com/Microsoft/PTVS/issues/3408

I also get a ton of messages on the output screen before the program loads that weren't there before in PTVS:

"Standard library module invoked user code during import; breakpoints disabled for invoked code."


If you use the classifier cookiecutter project and start debuging with a breakpoint on the read_table line (which means it imports pandas, numpy and matplotlib):

- It takes over 120-150 secs to reach breakpoint with 1000s of warning in output window
- It takes 5-10 secs if the warning is removed



Changed files:
Python/Product/PythonTools/ptvsd/debugger.py

diff --git a/Python/Product/PythonTools/ptvsd/debugger.py b/Python/Product/PythonTools/ptvsd/debugger.py
index e28a449aa..4244cf058 100644
--- a/Python/Product/PythonTools/ptvsd/debugger.py
+++ b/Python/Product/PythonTools/ptvsd/debugger.py
@@ -776,6 +776,7 @@ class Thread(object):
         self.django_stepping = None
         self.is_sending = False
         self.is_importing_stdlib = False
+        self.is_importing_stdlib_warned = False
 
         # stackless changes
         if stackless is not None:
@@ -938,7 +939,8 @@ class Thread(object):
         # If we're importing stdlib, don't trace nested calls until we return from the import that started it,
         # but notify the user if these nested calls end up in non-stdlib code.
         if self.is_importing_stdlib:
-            if not is_stdlib(path.normcase(co_filename)):
+            if not self.is_importing_stdlib_warned and not co_filename.startswith('<') and should_debug_code(f_code):
+                self.is_importing_stdlib_warned = True
                 debug_output.write('Standard library module invoked user code during import; breakpoints disabled for invoked code.\\n')
             return self.prev_trace_func
Repository: microsoft/ptvs
Issue: ptvsd.enable_attach causes process to never exit under Python 2.5

This breaks the AttachPtvsd test under 2.5 in particular.


Changed files:
Python/Product/PythonTools/ptvsd/attach_server.py

diff --git a/Python/Product/PythonTools/ptvsd/attach_server.py b/Python/Product/PythonTools/ptvsd/attach_server.py
index 892f63c19..fa750212f 100644
--- a/Python/Product/PythonTools/ptvsd/attach_server.py
+++ b/Python/Product/PythonTools/ptvsd/attach_server.py
@@ -271,7 +271,7 @@ def enable_attach(secret, address = ('0.0.0.0', DEFAULT_PORT), certfile = None,
                     client.close()
 
     server_thread = threading.Thread(target = server_thread_func)
-    server_thread.daemon = True
+    server_thread.setDaemon(True)
     server_thread.start()
 
     frames = []
Repository: microsoft/ptvs
Issue: Test failure in django project

Create Django Project (bootstrap starter template)
Run one of the 3 tests
Result: test failure

I'm using Django 1.8

```
Test Name:  test_about
Test Outcome:   Failed
Result StandardError:   
======================================================================
ERROR: tearDownClass (app.tests.ViewTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\\Users\\huvalo\\AppData\\Local\\Continuum\\Anaconda3\\lib\\site-packages\\django\\test\\testcases.py", line 962, in tearDownClass
    cls._rollback_atomics(cls.cls_atomics)
AttributeError: type object 'ViewTest' has no attribute 'cls_atomics'
----------------------------------------------------------------------
Ran 1 test in 0.516s
FAILED (errors=1)

```


Changed files:
Python/Product/Django/Templates/Projects/StarterDjangoProject/app-tests.py

diff --git a/Python/Product/Django/Templates/Projects/StarterDjangoProject/app-tests.py b/Python/Product/Django/Templates/Projects/StarterDjangoProject/app-tests.py
index 5760bba27..cec7006b5 100644
--- a/Python/Product/Django/Templates/Projects/StarterDjangoProject/app-tests.py
+++ b/Python/Product/Django/Templates/Projects/StarterDjangoProject/app-tests.py
@@ -15,6 +15,7 @@ class ViewTest(TestCase):
         # Django 1.7 requires an explicit setup() when running tests in PTVS
         @classmethod
         def setUpClass(cls):
+            super(ViewTest, cls).setUpClass()
             django.setup()
 
     def test_home(self):
Repository: microsoft/ptvs
Issue: Always check `__stdout__` before using

Hard to be sure, but [this issue](https://developercommunity.visualstudio.com/content/problem/175588/adjusting-console-output.html) seems like it could be fixed by verifying `__stdout__` is not None before using it.

Changed files:
Python/Product/PythonTools/ptvsd/debugger.py

diff --git a/Python/Product/PythonTools/ptvsd/debugger.py b/Python/Product/PythonTools/ptvsd/debugger.py
index 4244cf058..c06953435 100644
--- a/Python/Product/PythonTools/ptvsd/debugger.py
+++ b/Python/Product/PythonTools/ptvsd/debugger.py
@@ -2533,16 +2533,17 @@ def new_external_thread():
     sys.settrace(thread.trace_func)
 
 def do_wait():
-    try:
-        import msvcrt
-    except ImportError:
-        sys.__stdout__.write('Press Enter to continue . . . ')
-        sys.__stdout__.flush()
-        sys.__stdin__.read(1)
-    else:
-        sys.__stdout__.write('Press any key to continue . . . ')
-        sys.__stdout__.flush()
-        msvcrt.getch()
+    if sys.__stdout__ is not None:
+        try:
+            import msvcrt
+        except ImportError:
+            sys.__stdout__.write('Press Enter to continue . . . ')
+            sys.__stdout__.flush()
+            sys.__stdin__.read(1)
+        else:
+            sys.__stdout__.write('Press any key to continue . . . ')
+            sys.__stdout__.flush()
+            msvcrt.getch()
 
 def enable_output_redirection():
     sys.stdout = _DebuggerOutput(sys.stdout, 'stdout')
Repository: microsoft/ptvs
Issue: IPython REPL will not start when startup script specified

I am new to Python and Visual Studio and having issues getting IPython to work (I need to be able interactively plot data using matplotlib).

I've gone into Python Tools\\Interactive Windows\\Interactive Mode and set it to IPython.  Once I start typing in the REPL I get the following error:

Traceback (most recent call last):
  File "C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\Common7\\IDE\\Extensions\\Microsoft\\Python Tools for Visual Studio\\2.2\\visualstudio_py_repl.py", line 1370, in <module>
    _run_repl()
  File "C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\Common7\\IDE\\Extensions\\Microsoft\\Python Tools for Visual Studio\\2.2\\visualstudio_py_repl.py", line 1366, in _run_repl
    BACKEND.execution_loop()  
  File "C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\Common7\\IDE\\Extensions\\Microsoft\\Python Tools for Visual Studio\\2.2\\visualstudio_ipython_repl.py", line 308, in execution_loop
    self.execute_file_as_main(self.launch_file, None)
  File "C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\Common7\\IDE\\Extensions\\Microsoft\\Python Tools for Visual Studio\\2.2\\visualstudio_ipython_repl.py", line 291, in execute_file_as_main
    contents = f.read().replace("\\r\\n", "\\n")
TypeError: expected bytes, bytearray or buffer compatible object
The Python REPL process has exited

I'm using Visual Studio Pro 2013 with PTVS 2.2
Python 3.4
IPtython 3.0

Any insight would be appreciated.


Changed files:
Python/Product/PythonTools/visualstudio_ipython_repl.py

diff --git a/Python/Product/PythonTools/visualstudio_ipython_repl.py b/Python/Product/PythonTools/visualstudio_ipython_repl.py
index ca08966e1..c9e950ce4 100644
--- a/Python/Product/PythonTools/visualstudio_ipython_repl.py
+++ b/Python/Product/PythonTools/visualstudio_ipython_repl.py
@@ -17,6 +17,7 @@
 import re
 import sys
 from visualstudio_py_repl import BasicReplBackend, ReplBackend, UnsupportedReplException, _command_line_to_args_list
+from visualstudio_py_util import to_bytes
 try:
     import thread
 except:
@@ -288,7 +289,7 @@ class IPythonBackend(ReplBackend):
     def execute_file_as_main(self, filename, arg_string):
         f = open(filename, 'rb')
         try:
-            contents = f.read().replace("\\r\\n", "\\n")
+            contents = f.read().replace(to_bytes("\\r\\n"), to_bytes("\\n"))
         finally:
             f.close()
         args = [filename] + _command_line_to_args_list(arg_string)
Repository: microsoft/ptvs
Issue: Default Django Web Role template missing views.py

Visual Studio 2015 Update 2
Python Tools 2.2.3 for VS 2015

To repro, in Visual Studio 2015: New Project -> Python -> Azure Cloud Service -> Django Web Role. No views.py is created after this, even though urls.py is importing it. Running locally gives an ImportError.


Changed files:
Python/Templates/Django/ProjectTemplates/Python/Web/WebRoleDjango/urls.py

diff --git a/Python/Templates/Django/ProjectTemplates/Python/Web/WebRoleDjango/urls.py b/Python/Templates/Django/ProjectTemplates/Python/Web/WebRoleDjango/urls.py
index c9ba27051..db7220196 100644
--- a/Python/Templates/Django/ProjectTemplates/Python/Web/WebRoleDjango/urls.py
+++ b/Python/Templates/Django/ProjectTemplates/Python/Web/WebRoleDjango/urls.py
@@ -4,8 +4,6 @@ Definition of urls for $safeprojectname$.
 
 from django.conf.urls import include, url
 
-import $safeprojectname$.views
-
 # Uncomment the next two lines to enable the admin:
 # from django.contrib import admin
 # admin.autodiscover()
Repository: microsoft/ptvs
Issue: Blank Django Web Project doesn't run out of the box

```
  File "c:\\projects\\temp\\DjangoWebProject5\\DjangoWebProject5\\urls.py", line 7, in <module>
    import DjangoWebProject5.views
ImportError: No module named 'DjangoWebProject5.views'
Press any key to continue . . .
```

If I comment out that import, I get the expected "It Worked" page.


Changed files:
Python/Product/Django/Templates/Projects/DjangoProject/urls.py

diff --git a/Python/Product/Django/Templates/Projects/DjangoProject/urls.py b/Python/Product/Django/Templates/Projects/DjangoProject/urls.py
index 64649a4ad..40efcebcb 100644
--- a/Python/Product/Django/Templates/Projects/DjangoProject/urls.py
+++ b/Python/Product/Django/Templates/Projects/DjangoProject/urls.py
@@ -4,8 +4,6 @@ Definition of urls for $safeprojectname$.
 
 from django.conf.urls import include, url
 
-import $safeprojectname$.views
-
 # Uncomment the next two lines to enable the admin:
 # from django.contrib import admin
 # admin.autodiscover()
Repository: microsoft/ptvs
Issue: Debugger breaks inside ptvsd

Test code:
```py
import ptvsd
ptvsd.enable_attach(None)

x = 1
while True:
    x += 1
    print(x)
```
Breakpoint set on `print`. After attaching, the stack is:
```
>	enable_attach in __init__ line 87	Python	Symbols loaded.
 	PythonApplication1 module line 7	Python	Symbols loaded.
```
where `__init__` is `ptvsd\\__init__.py`. The line in question is inside `enable_attach`. However, the line on which it's actually broken is the one with `print`. So the breakpoint is hit, which is correct, but the stack is weird.

The real problem here is that `__init__.py` is not in `DONT_DEBUG`.


Changed files:
Python/Product/PythonTools/ptvsd/debugger.py, Python/Product/PythonTools/ptvsd/attach_server.py

diff --git a/Python/Product/PythonTools/ptvsd/attach_server.py b/Python/Product/PythonTools/ptvsd/attach_server.py
index 316a7386d..2394d0877 100644
--- a/Python/Product/PythonTools/ptvsd/attach_server.py
+++ b/Python/Product/PythonTools/ptvsd/attach_server.py
@@ -137,11 +137,9 @@ def enable_attach(secret, address=('0.0.0.0', ptvsd.DEFAULT_PORT), certfile=None
 
     frames = []
     f = sys._getframe()
-    while True:
-        f = f.f_back
-        if f is None:
-            break
+    while f is not None:
         frames.append(f)
+        f = f.f_back
     frames.reverse()
     cur_thread = vspd.new_thread()
     for f in frames:
diff --git a/Python/Product/PythonTools/ptvsd/debugger.py b/Python/Product/PythonTools/ptvsd/debugger.py
index 8a4f60cc6..e7407ea03 100644
--- a/Python/Product/PythonTools/ptvsd/debugger.py
+++ b/Python/Product/PythonTools/ptvsd/debugger.py
@@ -43,6 +43,7 @@ import runpy
 import datetime
 from codecs import BOM_UTF8
 
+import ptvsd
 import ptvsd.util as _vspu
 import ptvsd.ipcjson as _vsipc
 import ptvsd.repl as _vspr
@@ -155,7 +156,13 @@ class SynthesizedValue(object):
 
 # Specifies list of files not to debug. Can be extended by other modules
 # (the REPL does this for $attach support and not stepping into the REPL).
-DONT_DEBUG = [path.normcase(__file__), path.normcase(_vspu.__file__)]
+DONT_DEBUG = [
+    path.normcase(__file__),
+    path.normcase(ptvsd.__file__),
+    path.normcase(_vspu.__file__),
+    path.normcase(_vspr.__file__),
+    path.normcase(_vsipc.__file__),
+]
 if sys.version_info >= (3, 3):
     DONT_DEBUG.append(path.normcase('<frozen importlib._bootstrap>'))
 if sys.version_info >= (3, 5):
Repository: microsoft/ptvs
Issue: Cannot debug script that uses colorama

When I try to debug this simple script:

``` Python
import colorama
colorama.init()
```

```
I get the following error:
Traceback (most recent call last):
  File "C:\\Work\\src\\test.py", line 2, in
<module>
    colorama.init()
  File "C:\\Anaconda\\lib\\site-packages\\colorama\\initialise.py", line 31, in init
    wrap_stream(orig_stdout, convert, strip, autoreset, wrap)
  File "C:\\Anaconda\\lib\\site-packages\\colorama\\initialise.py", line 61, in wrap_
stream
    convert=convert, strip=strip, autoreset=autoreset)
  File "C:\\Anaconda\\lib\\site-packages\\colorama\\ansitowin32.py", line 68, in __in
it__
    convert = on_windows and not wrapped.closed and not on_emulated_windows and
is_a_tty(wrapped)
AttributeError: '_ReplOutput' object has no attribute 'closed'
Press any key to continue . . .
```

Adding

```
self.closed = False
```

to class `_ReplOutput` fixes the problem. I am not sure what is the proper way of fixing this. With some hints I could send a PR.

Thanks


Changed files:
Python/Product/PythonTools/visualstudio_py_repl.py

diff --git a/Python/Product/PythonTools/visualstudio_py_repl.py b/Python/Product/PythonTools/visualstudio_py_repl.py
index 373dd9586..ee64cbf0c 100644
--- a/Python/Product/PythonTools/visualstudio_py_repl.py
+++ b/Python/Product/PythonTools/visualstudio_py_repl.py
@@ -1169,6 +1169,7 @@ class DebugReplBackend(BasicReplBackend):
 class _ReplOutput(object):
     """file like object which redirects output to the repl window."""
     errors = None
+    closed = False
 
     def __init__(self, backend, is_stdout, old_out = None):
         self.name = "<stdout>" if is_stdout else "<stderr>"