Repository: sublimelsp/LSP

Issue:
Reset CompletionState when textDocument/completion gets an ResponseError
We need to reset `CompletionState` to `IDLE`, otherwise, autocompletion will be frozen.


Discussion:
The problem is caused by a very general one. If ANY request is responded with a server error -32000 the response handler is never called. Therefore the CompletionHandler's response_handler in this case is never called. The error response seems not to include the request id? 

Therefore the CompletionHandler gets caught in CANCALLING state and can't recover.
There is some requirement that responses arrive in the same order that the requests are made: https://github.com/Microsoft/language-server-protocol/issues/12

How common should it be for completion requests to return an error? Errors should be shown in the status bar already, the user can always ~restart the server~ re-open the document to recover.

@randy3k, what language server is returning the error, and what is the error returned?
I encountered the error when developing the [R language server](https://github.com/REditorSupport/languageserver). They are runtime errors due to bugs in completion handler where `InternalError` is returned.

Closing and reopening the view should set up a new `CompletionHandler` with fresh state. 

Let me know if that is not a functional or acceptable workaround for what should be rare or development-time language server errors.
No software is perfect. We always should assume something to possibly go wrong. Therefore I think LSP should handle error responses somehow and in this case really cancel the completion. How should a normal user find out why completions suddenly stopped working? I was indeed wondering for a while, too, until I found the error responses to be the reason. See the '`column` parameter is not in a valid range.' issue from #140 as an example. A server always is allowed to respond with an error - doesn't need to be bug triggered. It should not cause LSP to stop working. I am absolutely with you, "id" would need to be sent back as a reference to the request the response belongs to - even in error situation.

But with the assumption of ordered request/response chains, this issue might be solved by adding some kind of fifo and remember all sent requests until the corresponding response was received to catch errors as well and forward them to the response handlers.
I am with @deathaxe. I think in general we need to pass the responses to the corresponding handler without catching them too early. It is not a solution at all to re-open the document if it can be avoided.
Yep, we can keep this issue for later.
Right now I think there are more important issues, like #140.
I thought #140 and this are caused by the same bug.
You are correct, but this bug can have many cases in the future and needs to be fixed separately in the future.
Yes this is a more general issue. This topic is just an example. I guess it needs some time to work out a strategy and may cause bigger changes to solve this issue cleanly. 

Resolving patch:
diff --git a/plugin/completion.py b/plugin/completion.py
--- a/plugin/completion.py
+++ b/plugin/completion.py
@@ -119,7 +119,7 @@ def __init__(self, view):
         self.resolve_details = []  # type: List[Tuple[str, str]]
         self.state = CompletionState.IDLE
         self.completions = []  # type: List[Any]
-        self.next_request = None
+        self.next_request = None  # type: Optional[Tuple[str, List[int]]]
         self.last_prefix = ""
         self.last_location = 0
 
@@ -189,7 +189,7 @@ def on_query_completions(self, prefix, locations):
                 else sublime.INHIBIT_WORD_COMPLETIONS | sublime.INHIBIT_EXPLICIT_COMPLETIONS
             )
 
-    def do_request(self, prefix, locations):
+    def do_request(self, prefix: str, locations: 'List[int]'):
         self.next_request = None
         view = self.view
 
@@ -204,10 +204,11 @@ def do_request(self, prefix, locations):
             if document_position:
                 client.send_request(
                     Request.complete(document_position),
-                    self.handle_response)
+                    self.handle_response,
+                    self.handle_error)
                 self.state = CompletionState.REQUESTING
 
-    def format_completion(self, item) -> 'Tuple[str, str]':
+    def format_completion(self, item: dict) -> 'Tuple[str, str]':
         # Sublime handles snippets automatically, so we don't have to care about insertTextFormat.
         label = item["label"]
         # choose hint based on availability and user preference
@@ -231,7 +232,7 @@ def format_completion(self, item) -> 'Tuple[str, str]':
         # only return label with a hint if available
         return "\t  ".join((label, hint)) if hint else label, insert_text
 
-    def handle_response(self, response):
+    def handle_response(self, response: dict):
         global resolvable_completion_items
 
         if self.state == CompletionState.REQUESTING:
@@ -251,10 +252,16 @@ def handle_response(self, response):
             self.view.run_command("hide_auto_complete")
             self.run_auto_complete()
         elif self.state == CompletionState.CANCELLING:
-            self.do_request(*self.next_request)
+            if self.next_request:
+                prefix, locations = self.next_request
+                self.do_request(prefix, locations)
         else:
             debug('Got unexpected response while in state {}'.format(self.state))
 
+    def handle_error(self, error: dict):
+        sublime.status_message('Completion error: ' + str(error.get('message')))
+        self.state = CompletionState.IDLE
+
     def run_auto_complete(self):
         self.view.run_command(
             "auto_complete", {
diff --git a/plugin/core/rpc.py b/plugin/core/rpc.py
--- a/plugin/core/rpc.py
+++ b/plugin/core/rpc.py
@@ -31,6 +31,7 @@ def __init__(self, process, project_path):
         self.project_path = project_path
         self.request_id = 0
         self._response_handlers = {}  # type: Dict[int, Callable]
+        self._error_handlers = {}  # type: Dict[int, Callable]
         self._request_handlers = {}  # type: Dict[str, Callable]
         self._notification_handlers = {}  # type: Dict[str, Callable]
         self.capabilities = {}  # type: Dict[str, Any]
@@ -47,11 +48,13 @@ def has_capability(self, capability):
     def get_capability(self, capability):
         return self.capabilities.get(capability)
 
-    def send_request(self, request: Request, handler: 'Callable'):
+    def send_request(self, request: Request, handler: 'Callable', error_handler: 'Optional[Callable]' = None):
         self.request_id += 1
         debug(' --> ' + request.method)
         if handler is not None:
             self._response_handlers[self.request_id] = handler
+        if error_handler is not None:
+            self._error_handlers[self.request_id] = error_handler
         self.send_payload(request.to_payload(self.request_id))
 
     def send_notification(self, notification: Notification):
@@ -107,11 +110,7 @@ def read_stdout(self):
                         continue
 
                     try:
-                        if "error" in payload:
-                            error = payload['error']
-                            debug("Got error from server: ", error)
-                            sublime.status_message(error.get('message'))
-                        elif "method" in payload:
+                        if "method" in payload:
                             if "id" in payload:
                                 self.request_handler(payload)
                             else:
@@ -145,7 +144,11 @@ def read_stderr(self):
                 if not content:
                     break
                 if settings.log_stderr:
-                    server_log("(stderr): ", content.strip())
+                    try:
+                        decoded = content.decode("UTF-8")
+                    except UnicodeDecodeError:
+                        decoded = content
+                    server_log(decoded.strip())
             except IOError as err:
                 exception_log("Failure reading stderr", err)
                 return
@@ -154,13 +157,23 @@ def read_stderr(self):
 
     def response_handler(self, response):
         handler_id = int(response.get("id"))  # dotty sends strings back :(
+        error = response.get('error', None)
         result = response.get('result', None)
-        if settings.log_payloads and result:
-            debug('     ' + str(result))
-        if (self._response_handlers[handler_id]):
-            self._response_handlers[handler_id](result)
-        else:
-            debug("No handler found for id" + response.get("id"))
+        if result:
+            if settings.log_payloads:
+                debug('     ' + str(result))
+
+            if handler_id in self._response_handlers:
+                self._response_handlers[handler_id](result)
+            else:
+                debug("No handler found for id" + response.get("id"))
+        elif error:
+            if settings.log_payloads:
+                debug('     ' + str(error))
+            if handler_id in self._error_handlers:
+                self._error_handlers[handler_id](error)
+            else:
+                sublime.status_message(error.get('message'))
 
     def on_request(self, request_method: str, handler: 'Callable'):
         self._request_handlers[request_method] = handler


Test patch: