Repository: ipython/ipython

Issue:
ipython console ignores cursor_start and cursor_end on complete_reply messages
Here is a minimal example using commit dbd30bf using Anaconda 3.4 64bit on Linux 17:

``` python
from IPython.kernel.zmq.kernelbase import Kernel

__version__ = '1'


class TestKernel(Kernel):
    implementation = 'test_kernel'
    implementation_version = __version__
    language = 'test'
    language_version = __version__
    banner = ''

    def do_complete(self, code, cursor_pos):

        matches = [code[1:] + '_complete']

        return {'matches': matches, 'cursor_start': 1,
             'cursor_end': cursor_pos, 'metadata': dict(),
                'status': 'ok'}

    def do_execute(self, code, silent, store_history=True, user_expressions=None, allow_stdin=False):
        return {'status': 'ok', 'execution_count': self.execution_count,
                    'payload': [], 'user_expressions': {}}

if __name__ == '__main__':
    from IPython.kernel.zmq.kernelapp import IPKernelApp
    IPKernelApp.launch_instance(kernel_class=TestKernel)
```

In one terminal, launch this file as `python test_kernel.py`.  In another tab, run `ipython console --existing ...`.  At the prompt type `test<TAB>`.  We expect to get `test_complete`, but instead we get `est_complete`.



Discussion:
Actually, `cursor_end` is not recognized either:

``` python
from IPython.kernel.zmq.kernelbase import Kernel

__version__ = '1'


class TestKernel(Kernel):
    implementation = 'test_kernel'
    implementation_version = __version__
    language = 'test'
    language_version = __version__
    banner = ''

    def do_complete(self, code, cursor_pos):

        matches = [code[1:] + '_complete']

        return {'matches': matches, 'cursor_start': 1,
             'cursor_end': cursor_pos - 2, 'metadata': dict(),
                'status': 'ok'}

    def do_execute(self, code, silent, store_history=True, user_expressions=None, allow_stdin=False):
        return {'status': 'ok', 'execution_count': self.execution_count,
                    'payload': [], 'user_expressions': {}}

if __name__ == '__main__':
    from IPython.kernel.zmq.kernelapp import IPKernelApp
    IPKernelApp.launch_instance(kernel_class=TestKernel)
```

If we try `hello<TAB>`, we expect to see `hello_completelo`, not `ello_complete`, or is the Kernel a Brit?

ping @takluyver.

It appears that the terminal console hasn't been updated for the new completion message format yet. It works correctly in the Qt console. Retitling accordingly.

The first one works in the Qt console, but the second one does not. 

Both work correctly in the notebook.

![screenshot from 2014-10-13 19 24 53](https://cloud.githubusercontent.com/assets/2096628/4622230/91d97508-5338-11e4-8c12-48ed1885a76d.png)

Here's a potential fix for the QtConsole:

``` diff
diff --git a/IPython/qt/console/ipython_widget.py b/IPython/qt/console/ipython_widget.py
index d145330..d7f1c75 100644
--- a/IPython/qt/console/ipython_widget.py
+++ b/IPython/qt/console/ipython_widget.py
@@ -150,9 +150,13 @@ def _handle_complete_reply(self, rep):
             matches = content['matches']
             start = content['cursor_start']
             end = content['cursor_end']
-            
-            offset = end - start
+
             # Move the cursor to the start of the match and complete.
+            line = self._get_input_buffer_cursor_line()
+            rest = line[end:]
+            if rest:
+                matches = [m + rest for m in matches]
+            offset = len(line) - start
             cursor.movePosition(QtGui.QTextCursor.Left, n=offset)
             self._complete_with_items(cursor, matches)

```

![screenshot from 2014-10-14 05 22 53](https://cloud.githubusercontent.com/assets/2096628/4627586/9409a718-538c-11e4-85a3-892d9cd4324d.png)
![screenshot from 2014-10-14 05 23 19](https://cloud.githubusercontent.com/assets/2096628/4627588/95c0778a-538c-11e4-9961-2c7319440cae.png)

``` python
from IPython.kernel.zmq.kernelbase import Kernel

__version__ = '1'


class TestKernel(Kernel):
    implementation = 'test_kernel'
    implementation_version = __version__
    language = 'test'
    language_version = __version__
    banner = ''

    def do_complete(self, code, cursor_pos):

        matches = [code[1:] + '_complete', code[1:] + '_complete2']

        return {'matches': matches, 'cursor_start': 1,
             'cursor_end': cursor_pos - 2, 'metadata': dict(),
                'status': 'ok'}

    def do_execute(self, code, silent, store_history=True, user_expressions=None, allow_stdin=False):
        return {'status': 'ok', 'execution_count': self.execution_count,
                    'payload': [], 'user_expressions': {}}

if __name__ == '__main__':
    from IPython.kernel.zmq.kernelapp import IPKernelApp
    IPKernelApp.launch_instance(kernel_class=TestKernel)
```

Do you want to make pull request ? 

The QtConsole fix above is still not very elegant.  I think the completion text should be displayed as `hello_complete`, and the outcome should remain the same. 

This takes care of the first problem:

``` diff
diff --git a/IPython/qt/console/ipython_widget.py b/IPython/qt/console/ipython_widget.py
index d145330..ff25b7e 100644
--- a/IPython/qt/console/ipython_widget.py
+++ b/IPython/qt/console/ipython_widget.py
@@ -150,10 +150,19 @@ def _handle_complete_reply(self, rep):
             matches = content['matches']
             start = content['cursor_start']
             end = content['cursor_end']
-            
-            offset = end - start
+
             # Move the cursor to the start of the match and complete.
-            cursor.movePosition(QtGui.QTextCursor.Left, n=offset)
+            line = self._get_input_buffer_cursor_line()
+            rest = line[end:]
+            if rest:
+                matches = [m + rest for m in matches]
+            if start:
+                matches = [line[:start] + m for m in matches]
+            if ' ' in matches[0]:
+                offset = matches[0].rindex(' ')
+                matches = [m[offset:] for m in matches]
+                line = line[offset:]
+            cursor.movePosition(QtGui.QTextCursor.Left, n=len(line))
             self._complete_with_items(cursor, matches)

     def _handle_execute_reply(self, msg):
```

@takluyver, can you help me find a spot to get started implementing this for the console?  Grepping for `complete_reply` led me to `IPython.kernel.adapter`, but I could not get any changes there to have any effect.

It shouldn't be in adapter, that's for translating between different versions of the message format. `IPython.terminal.console.completer` is the relevant code. It may be thorny because it has to work with readline.


Resolving patch:
diff --git a/IPython/terminal/console/completer.py b/IPython/terminal/console/completer.py
--- a/IPython/terminal/console/completer.py
+++ b/IPython/terminal/console/completer.py
@@ -45,11 +45,16 @@ def complete_request(self, text):
             code=line,
             cursor_pos=cursor_pos,
         )
-        
+    
         msg = self.client.shell_channel.get_msg(timeout=self.timeout)
         if msg['parent_header']['msg_id'] == msg_id:
-            cursor_start = msg['content']['cursor_start']
-            return [ line[:cursor_start] + m for m in msg['content']['matches'] ]
+            content = msg['content']
+            cursor_start = content['cursor_start']
+            matches = [ line[:cursor_start] + m for m in content['matches'] ]
+            if content["cursor_end"] < cursor_pos:
+                extra = line[content["cursor_end"]: cursor_pos]
+                matches = [m + extra for m in matches]
+            return matches
         return []
     
     def rlcomplete(self, text, state):


Test patch: