Repository: microsoft/ptvsd

Issue:
Validate breakpoint targets
It is currently possible to set breakpoints on lines where valid breakpoints aren't possible (e.g. blank lines, or lines in the middle of a multi-line statement). When that happens, there's no error or warning, and the breakpoint doesn't get hit.

We should either automatically move the breakpoint to the nearest valid line, or display an error message in that case. 


Discussion:
https://github.com/Microsoft/PTVS/issues/207
First, the general problem. In CPython, breakpoints don't work on lines of code which don't map to any bytecode range at runtime, since the bytecode interpreter uses those ranges to keep track of the current source line for debugger purposes. This can happen for various reasons: blank lines, obviously, but also `pass` statements (although not in all contexts). 

In addition, on Python 2 especially, all bytecode for multiline statements usually gets mapped to the last line of that statement. E.g. for something like this:
```py
print(1,
      2, 3,
      4, 5, 6)
```
all the bytecode is recorded as corresponding to line 3, so that's the only line where you can set the breakpoint.

Since this all is already dependent on Python version, and likely to do so in the future, we can't rely on parsing the source code ourselves - we need to look at the bytecode at runtime, and extract valid line numbers from there. But when setting the breakpoint, the target file might not even be loaded by Python at all, so there's no corresponding code object; and even if there is, Python doesn't provide any way to enumerate existing code objects, or to find one by filename.

However, the file can be separately loaded, parsed and compiled (without executing it) to produce a code object. This code object only covers the top level of the module, but it contains references to other code objects generated for nested classes, functions etc. Thus, it is possible to walk that entire tree and extract line information from all code objects that get produced from the file. Here's some straightforward recursive code that does that for any file given to it:
```py
import dis
import sys
import types

def get_valid_lines(code):
    for _, lineno in dis.findlinestarts(code):
        yield lineno
    for const in code.co_consts:
        if isinstance(const, types.CodeType) and const.co_filename == code.co_filename:
            for lineno in get_valid_lines(const):
                yield lineno

filename = sys.argv[1] 
with open(filename) as f:
    source = f.read()
code = compile(source, filename, 'exec')
lines = set(get_valid_lines(code))
print(lines)
```
Given  something like:
```py
x = 1
try:
    x /= 0
except Exception:
    pass

print(1,
      2, 3,
      4, 5, 6)


def foo():
    print('begin foo')
    pass
    def bar():
        print('bar!')
        pass
        return
    bar()
    print('end foo')
```
you get this for Python 3:

{1, 2, 3, 4, 5, 7, 8, 9, 12, 13, 15, 16, 18, 19, 20}

and this for Python 2:

{1, 2, 3, 4, 5, 9, 12, 13, 15, 16, 18, 19, 20}

The debugger can do something similar every time a breakpoint is set. I don't even think we'll need caching, because setting a breakpoint is a one-off operation, so the overhead of parsing the file won't be noticeable (but parsing it every time would accommodate e.g. exotic scenarios where the file changes).

If the line on which a breakpoint is being set is not in the list of valid lines, we can report the breakpoint as invalid (like C#), or we can adjust it to the next valid line (like C++). It's hard to say whether one or the other is preferable in general - for a multiline statement in Python 2, the user would probably expect it to adjust, but for all other scenarios like `pass`, moving BP might actually place it in an undesirable location - supposing something like this:
```py
while foo():
    pass
print('done')
```
The breakpoint on `pass` would be invalid, but moving it to `print` makes no sense either. If we only look at Python 3, where individual lines of multiline statements are properly mapped to bytecode, all remaining scenarios are like this, so rejecting the breakpoint is probably a better idea.
/cc @qubitron 
Let us know your opinion on how to handle invalid breakpoints.
Can we also add the same for django/flask templates?
Or should that be tracked as a separate issue?
It needs to be investigated first. I don't think this approach will work there.
For invalid breakpoints I'd prefer to move to the next valid line, though I'm not sure if the VS Code UI/DAP supports that @DonJayamanne.

If that is too difficult, then we can start by at least showing that it's not valid.
Even if the next valid line is semantically unrelated? (as in the loop example... I'd expect that if the breakpoint is adjusted there, it's one line up, not one line down).

Implementation-wise it's all very similar - the only real difference is setting the invalid flag vs setting the line number in response message.
I would like to optimize for minimizing wasted time. If the breakpoint is invalid, I won't find that out until run-time, and it's annoying when the breakpoint gets skipped/missed and I have to restart my program to hit the breakpoint again. If we push it down one line past the point of execution, then again I might need to restart my program to hit the point that I wanted to hit.

So always going up one line means I never miss the point of execution I'm trying to stop on, so that sounds like a good approach to me, even though it is inconsistent with how C++ works in VS. 

Breaking in the wrong spot doesn't really ever "do harm", but not breaking when I wanted to can waste time.

Thinking about it, adjusting to the previous seems to work well in general - in most scenarios it's reasonable. One case where it can be confusing with multiline, e.g.:

```py
print(1,
      2, 3,
      4, 5, 6)
```
In Python 2, this will only break on the last line of three. Suppose the user sets a breakpoint at print, where it's more natural. If we adjust upwards, they will end up on a different statement. If we adjust downwards, it'll do exactly what they wanted, breaking right before print.

However, this is specific to Python 2. 
The case I hit this was:
```
a = [
  1,
  2,
  3]
```

I set the breakpoint on the first line but would expect it to hit on the second instead.

Interesting - that one happens on Python 3 as well.

Either way, we can implement one thing and then try it out. It's an easy change later to adjust in a different direction or invalidate the breakpoints instead. 
Let's go with the adjust to previous line rule, so that we never run past where the user wanted us to execute.

Resolving patch:
diff --git a/src/ptvsd/_util.py b/src/ptvsd/_util.py
--- a/src/ptvsd/_util.py
+++ b/src/ptvsd/_util.py
@@ -8,6 +8,7 @@
 import os
 import threading
 import time
+import types
 import sys
 
 
@@ -413,3 +414,24 @@ def _allow_debug_break(enabled=True):
 
 def _is_debug_break_allowed():
     return _enable_debug_break
+
+
+try:
+    import dis
+except ImportError:
+    def get_code_lines(code):
+        return None
+else:
+    def get_code_lines(code):
+        # First, get all line starts for this code object. This does not include
+        # bodies of nested class and function definitions, as they have their
+        # own objects.
+        for _, lineno in dis.findlinestarts(code):
+            yield lineno
+
+        # For nested class and function definitions, their respective code objects
+        # are constants referenced by this object.
+        for const in code.co_consts:
+            if isinstance(const, types.CodeType) and const.co_filename == code.co_filename:
+                for lineno in get_code_lines(const):
+                    yield lineno
\ No newline at end of file
diff --git a/src/ptvsd/wrapper.py b/src/ptvsd/wrapper.py
--- a/src/ptvsd/wrapper.py
+++ b/src/ptvsd/wrapper.py
@@ -4,6 +4,7 @@
 
 from __future__ import print_function, absolute_import, unicode_literals
 
+import bisect
 import contextlib
 import copy
 import errno
@@ -2282,6 +2283,40 @@ def on_setBreakpoints(self, request, args):
 
         pydevd_request = copy.deepcopy(request)
         del pydevd_request['seq']  # A new seq should be created for pydevd.
+
+        # Validate breakpoints and adjust their positions.
+        args = pydevd_request['arguments']
+        if 'breakpoints' in args and 'path' in args['source']:
+            path = args['source']['path']
+            if sys.version_info < (3,) and not isinstance(path, bytes):
+                path = path.encode(sys.getfilesystemencoding())
+            path = path_to_unicode(pydevd_file_utils.norm_file_to_server(path))
+
+            try:
+                with open(path) as f:
+                    src = f.read()
+            except Exception:
+                pass
+            else:
+                try:
+                    code = compile(src, path, 'exec', 0, dont_inherit=True)
+                except Exception:
+                    pass
+                else:
+                    try:
+                        lines = sorted(_util.get_code_lines(code))
+                    except Exception:
+                        pass
+                    else:
+                        if lines:
+                            for bp in args['breakpoints']:
+                                line = bp['line']
+                                if line not in lines:
+                                    # Adjust to the first preceding valid line.
+                                    idx = bisect.bisect_left(lines, line)
+                                    if idx > 0:
+                                        bp['line'] = lines[idx - 1]
+
         _, _, resp_args = yield self.pydevd_request(
             pydevd_comm.CMD_SET_BREAK,
             pydevd_request,


Test patch:
diff --git a/test_requirements.txt b/test_requirements.txt
--- a/test_requirements.txt
+++ b/test_requirements.txt
@@ -5,7 +5,7 @@ requests
 flask
 psutil
 pygments
-pytest
+pytest<5
 pytest-cov
 pytest-timeout
 pytest-xdist
diff --git a/tests/func/test_breakpoints.py b/tests/func/test_breakpoints.py
--- a/tests/func/test_breakpoints.py
+++ b/tests/func/test_breakpoints.py
@@ -11,6 +11,7 @@
 import re
 import sys
 
+from tests.helpers import get_marked_line_numbers
 from tests.helpers.pathutils import get_test_root
 from tests.helpers.session import DebugSession
 from tests.helpers.timeline import Event
@@ -391,10 +392,91 @@ def code_to_debug():
         session.set_breakpoints(code_to_debug, [])
         session.send_request('continue').wait_for_response(freeze=False)
 
-        session.write_json('done')
         session.wait_for_next(Event('output', ANY.dict_with({'category': 'stdout', 'output': '9'})))
+        session.write_json('done')
         session.wait_for_exit()
 
         output = session.all_occurrences_of(Event('output', ANY.dict_with({'category': 'stdout'})))
         output = sorted(int(o.body['output'].strip()) for o in output if len(o.body['output'].strip()) > 0)
         assert list(range(0, 10)) == output
+
+
+def test_invalid_breakpoints(pyfile, run_as, start_method):
+    @pyfile
+    def code_to_debug():
+        from dbgimporter import import_and_enable_debugger
+        import_and_enable_debugger()
+
+        b = True
+        while b:        #@bp1-expected
+            pass        #@bp1-requested
+            break
+
+        print()         #@bp2-expected
+        [               #@bp2-requested
+            1, 2, 3,    #@bp3-expected
+        ]               #@bp3-requested
+
+        # Python 2.7 only.
+        print()         #@bp4-expected
+        print(1,        #@bp4-requested-1
+              2, 3,     #@bp4-requested-2
+              4, 5, 6)
+
+    line_numbers = get_marked_line_numbers(code_to_debug)
+    from tests.helpers import print
+    print(line_numbers)
+
+    with DebugSession() as session:
+        session.initialize(
+            target=(run_as, code_to_debug),
+            start_method=start_method,
+            ignore_unobserved=[Event('continued')],
+        )
+
+        requested_bps = [
+            line_numbers['bp1-requested'],
+            line_numbers['bp2-requested'],
+            line_numbers['bp3-requested'],
+        ]
+        if sys.version_info < (3,):
+            requested_bps += [
+                line_numbers['bp4-requested-1'],
+                line_numbers['bp4-requested-2'],
+            ]
+
+        actual_bps = session.set_breakpoints(code_to_debug, requested_bps)
+        actual_bps = [bp['line'] for bp in actual_bps]
+
+        expected_bps = [
+            line_numbers['bp1-expected'],
+            line_numbers['bp2-expected'],
+            line_numbers['bp3-expected'],
+        ]
+        if sys.version_info < (3,):
+            expected_bps += [
+                line_numbers['bp4-expected'],
+                line_numbers['bp4-expected'],
+            ]
+
+        assert expected_bps == actual_bps
+
+        # Now let's make sure that we hit all of the expected breakpoints,
+        # and stop where we expect them to be.
+
+        session.start_debugging()
+
+        # If there's multiple breakpoints on the same line, we only stop once,
+        # so remove duplicates first.
+        expected_bps = sorted(set(expected_bps))
+
+        while expected_bps:
+            hit = session.wait_for_thread_stopped()
+            frames = hit.stacktrace.body['stackFrames']
+            line = frames[0]['line']
+            assert line == expected_bps[0]
+            del expected_bps[0]
+            session.send_request('continue').wait_for_response()
+        assert not expected_bps
+
+        session.wait_for_exit()
diff --git a/tests/helpers/__init__.py b/tests/helpers/__init__.py
--- a/tests/helpers/__init__.py
+++ b/tests/helpers/__init__.py
@@ -83,7 +83,7 @@ def get_marked_line_numbers(path):
     with open(path) as f:
         lines = {}
         for i, line in enumerate(f):
-            match = re.search(r'#@(.*?)\s*$', line)
+            match = re.search(r'#@\s*(.*?)\s*$', line)
             if match:
                 marker = match.group(1)
                 lines[marker] = i + 1
diff --git a/tests/helpers/session.py b/tests/helpers/session.py
--- a/tests/helpers/session.py
+++ b/tests/helpers/session.py
@@ -581,10 +581,10 @@ def _wait_for_remaining_output(self):
             thread.join()
 
     def set_breakpoints(self, path, lines=()):
-        self.send_request('setBreakpoints', arguments={
+        return self.send_request('setBreakpoints', arguments={
                 'source': {'path': path},
                 'breakpoints': [{'line': bp_line} for bp_line in lines],
-            }).wait_for_response()
+            }).wait_for_response().body.get('breakpoints', None)
 
     def wait_for_thread_stopped(self, reason=ANY):
         thread_stopped = self.wait_for_next(Event('stopped', ANY.dict_with({'reason': reason})))