Repository: microsoft/ptvsd

Issue:
Logpoint messages aren't printed for "launch"
For breakpoints with "logMessage", when they're hit, pydevd sends an "output" event with `"category": "stdout"`, but doesn't actually print that message to stdout. Since we're now capturing the output directly from the process for launch scenarios, this means that the event from the server is ignored by the adapter on the assumption that launcher will supply more accurate output. In this case, the assumption doesn't hold, but the IDE ends up never receiving that message.

I can see three options to handle this.

1. Send output with `"category": "console"` instead. This shouldn't make any practical difference in the UI - it will show up in the Debug Console, but not in the terminal, same as it is today - but such events will always be propagated by the adapter, since it only relies on the launcher for stdout/stderr.

2. Add a custom property to "output" events that pydevd can use to mark synthetic ones that do not correspond to any actual output. The adapter would then check that property, and always propagate such events, even if the launcher is capturing actual stdout/err.

3. Don't synthesize the events, but rather just print the message directly to stdout. This will make it show in the terminal. 

It seems that the second option isn't really adding anything useful, and the choice between 1 and 3 depends on whether we want logpoint messages to show up in the terminal, or only in Debug Console.

The DAP spec doesn't say anything about this, but so far we've been doing the equivalent of 1. This feels more reasonable, since it doesn't affect the observable behavior of the app; e.g. imagine if it is debugged with its stdout redirected to be parsed by some other app - logpoints being actually printed would mess that up. 


Discussion:
`test_log_point` currently fails in the refactor branch due to this bug.
I think option 1 is OK.
@fabioz /cc
I think doing this change is ok (so that we have a way to know what's generated from the debugger and what comes from the user output), but, regardless of that, the way I see it, on launch scenarios we should not be redirecting the output at all (i.e.: in which case the adapter would not receive any `output` events for user output anyways).
Oh, so basically, just don't propagate "RedirectOutput" debug option to pydevd for "launch"?
> Oh, so basically, just don't propagate "RedirectOutput" debug option to pydevd for "launch"?

Yes
Indeed! Doing this, and propagating all remaining (i.e. synthetic) "output" events from the server, fixes the original issue.

We do have a different problem though... since regular output and logpoint output is now captured differently, this means that logpoints get out of sync wrt output. E.g. if you do:
```py
print("foo")
something()  # logpoint here that prints "bar"
print("baz")
```
the end result is sometimes `"foo\nbaz\nbar"`. 

Fixing this properly would be complicated and slow - the logpoint code in pydevd would need to synchronize with the output capture in adapter, basically. One way to do it would be to pause all threads, flush stdout/stderr, report the logpoint as a reverse request, and then wait for response. The adapter would then have to capture whatever is pending in stdout/stderr streams, then finally report logpoint as "output" to the IDE, and respond to pydevd. Only then it can unpause and allow the code to continue running and producing more output. 

I don't think it's worth it, especially if we're going to switch logpoints to a different output category anyway. For now, I'll just change the test so that it doesn't expect logpoint output to have strict ordering relative to regular output. 

Resolving patch:
diff --git a/src/ptvsd/adapter/ide.py b/src/ptvsd/adapter/ide.py
--- a/src/ptvsd/adapter/ide.py
+++ b/src/ptvsd/adapter/ide.py
@@ -223,6 +223,10 @@ def launch_request(self, request):
 
         self.session.spawn_debuggee(request, sudo, args, console, console_title)
 
+        if "RedirectOutput" in self.session.debug_options:
+            # The launcher is doing output redirection, so we don't need the server.
+            request.arguments["debugOptions"].remove("RedirectOutput")
+
     @_start_message_handler
     def attach_request(self, request):
         if self.session.no_debug:
diff --git a/src/ptvsd/adapter/server.py b/src/ptvsd/adapter/server.py
--- a/src/ptvsd/adapter/server.py
+++ b/src/ptvsd/adapter/server.py
@@ -128,13 +128,6 @@ def continued_event(self, event):
         if self.ide.client_id not in ("visualstudio", "vsformac"):
             self.ide.propagate_after_start(event)
 
-    @message_handler
-    def output_event(self, event):
-        category = event("category", "console")
-        # If there is a launcher, it's handling stdout and stderr.
-        if not self.launcher or category not in ("stdout", "stderr"):
-            self.ide.propagate_after_start(event)
-
     @message_handler
     def exited_event(self, event):
         # If there is a launcher, it's handling the exit code.


Test patch:
diff --git a/tests/ptvsd/server/test_breakpoints.py b/tests/ptvsd/server/test_breakpoints.py
--- a/tests/ptvsd/server/test_breakpoints.py
+++ b/tests/ptvsd/server/test_breakpoints.py
@@ -28,10 +28,7 @@ def test_path_with_ampersand(start_method, run_as):
         session.start_debugging()
 
         session.wait_for_stop(
-            "breakpoint",
-            expected_frames=[
-                some.dap.frame(test_py, line="two"),
-            ],
+            "breakpoint", expected_frames=[some.dap.frame(test_py, line="two")]
         )
 
         session.request_continue()
@@ -200,9 +197,11 @@ def test_log_point(pyfile, start_method, run_as, condition):
     @pyfile
     def code_to_debug():
         import debug_me  # noqa
+        import sys
 
         for i in range(0, 10):
-            print(i * 10)  # @bp
+            sys.stderr.write(str(i * 10) + "\n")  # @bp
+            sys.stderr.flush()
         ()  # @wait_for_output
 
     lines = code_to_debug.lines
@@ -215,7 +214,7 @@ def code_to_debug():
 
         session.request(
             "setBreakpoints",
-            arguments={
+            {
                 "source": {"path": code_to_debug},
                 "breakpoints": [bp, {"line": lines["wait_for_output"]}],
             },
@@ -224,36 +223,34 @@ def code_to_debug():
 
         if condition:
             session.wait_for_stop(
-                "breakpoint",
-                expected_frames=[
-                    some.dap.frame(code_to_debug, line="bp")
-                ],
+                "breakpoint", expected_frames=[some.dap.frame(code_to_debug, line="bp")]
             )
 
             var_i = session.get_variable("i")
             assert var_i == some.dict.containing(
-                {"name": "i",  "evaluateName": "i", "type": "int", "value": "5"},
+                {"name": "i", "evaluateName": "i", "type": "int", "value": "5"}
             )
 
             session.request_continue()
 
         session.wait_for_stop(
             "breakpoint",
-            expected_frames=[
-                some.dap.frame(code_to_debug, line="wait_for_output")
-            ],
+            expected_frames=[some.dap.frame(code_to_debug, line="wait_for_output")],
         )
         session.request_continue()
 
-    assert not session.captured_stderr()
+    # print() should produce both actual output, and "output" events on stderr,
+    # but logpoints should only produce "output" events on stdout.
+    assert not session.captured_stdout()
 
     expected_stdout = "".join(
-        (
-            fmt(r"{0}\r?\n{1}\r?\n", re.escape(str(i)), re.escape(str(i * 10)))
-            for i in range(0, 10)
-        )
+        (fmt(r"{0}\r?\n", re.escape(str(i))) for i in range(0, 10))
+    )
+    expected_stderr = "".join(
+        (fmt(r"{0}\r?\n", re.escape(str(i * 10))) for i in range(0, 10))
     )
     assert session.output("stdout") == some.str.matching(expected_stdout)
+    assert session.output("stderr") == some.str.matching(expected_stderr)
 
 
 def test_package_launch():
@@ -267,10 +264,7 @@ def test_package_launch():
         session.start_debugging()
 
         session.wait_for_stop(
-            "breakpoint",
-            expected_frames=[
-                some.dap.frame(test_py, line="two"),
-            ],
+            "breakpoint", expected_frames=[some.dap.frame(test_py, line="two")]
         )
 
         session.request_continue()
@@ -291,10 +285,7 @@ def code_to_debug():
         session.start_debugging()
 
         session.wait_for_stop(
-            "breakpoint",
-            expected_frames=[
-                some.dap.frame(code_to_debug, line="bp"),
-            ],
+            "breakpoint", expected_frames=[some.dap.frame(code_to_debug, line="bp")]
         )
 
         # Remove breakpoint inside the loop.
@@ -303,9 +294,7 @@ def code_to_debug():
 
         session.wait_for_stop(
             "breakpoint",
-            expected_frames=[
-                some.dap.frame(code_to_debug, line="wait_for_output")
-            ],
+            expected_frames=[some.dap.frame(code_to_debug, line="wait_for_output")],
         )
         session.request_continue()
 
@@ -314,23 +303,22 @@ def code_to_debug():
 
 
 def test_breakpoints_in_filtered_files(pyfile, run_as, start_method):
-
     @pyfile
     def code_to_debug():
-        import debug_me # noqa
+        import debug_me  # noqa
 
     with debug.Session(start_method) as session:
         session.configure(run_as, code_to_debug)
 
-        breakpoints = session.set_breakpoints('invalid_file.py', [1])
-        assert breakpoints == [{
-            'verified': False,
-            'message': 'Breakpoint in file that does not exist.',
-            'source': some.dict.containing({
-                'path': some.path('invalid_file.py')
-            }),
-            'line': 1
-        }]
+        breakpoints = session.set_breakpoints("invalid_file.py", [1])
+        assert breakpoints == [
+            {
+                "verified": False,
+                "message": "Breakpoint in file that does not exist.",
+                "source": some.dict.containing({"path": some.path("invalid_file.py")}),
+                "line": 1,
+            }
+        ]
         session.start_debugging()