Repository: ipython/ipython

Issue:
Crash if ipython history db contains malformed unicode
```
ipython
Traceback (most recent call last):
  File "/home/omer/.local/bin/ipython", line 11, in <module>
    sys.exit(start_ipython())
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/__init__.py", line 120, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/config/application.py", line 563, in launch_instance
    app.initialize(argv)
  File "<string>", line 2, in initialize
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/config/application.py", line 92, in catch_config_error
    return method(app, *args, **kwargs)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/terminal/ipapp.py", line 332, in initialize
    self.init_shell()
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/terminal/ipapp.py", line 348, in init_shell
    ipython_dir=self.ipython_dir, user_ns=self.user_ns)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/config/configurable.py", line 354, in instance
    inst = cls(*args, **kwargs)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 328, in __init__
    **kwargs
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 483, in __init__
    self.init_readline()
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 1884, in init_readline
    self.refill_readline_hist()
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 1896, in refill_readline_hist
    include_latest=True):
  File "<string>", line 2, in get_tail
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/history.py", line 85, in catch_corrupt_db
    return f(self, *a, **kw)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/history.py", line 319, in get_tail
    (n,), raw=raw, output=output)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/history.py", line 253, in _run_sql
    (toget, sqlfrom) + sql, params)
OperationalError: Could not decode to UTF-8 column 'source_raw' with text '��������������������������������'
```

I don't think it should crash though. Maybe just warn that history is not available?

Crash if ipython history db contains malformed unicode
```
ipython
Traceback (most recent call last):
  File "/home/omer/.local/bin/ipython", line 11, in <module>
    sys.exit(start_ipython())
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/__init__.py", line 120, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/config/application.py", line 563, in launch_instance
    app.initialize(argv)
  File "<string>", line 2, in initialize
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/config/application.py", line 92, in catch_config_error
    return method(app, *args, **kwargs)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/terminal/ipapp.py", line 332, in initialize
    self.init_shell()
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/terminal/ipapp.py", line 348, in init_shell
    ipython_dir=self.ipython_dir, user_ns=self.user_ns)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/config/configurable.py", line 354, in instance
    inst = cls(*args, **kwargs)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 328, in __init__
    **kwargs
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 483, in __init__
    self.init_readline()
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 1884, in init_readline
    self.refill_readline_hist()
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 1896, in refill_readline_hist
    include_latest=True):
  File "<string>", line 2, in get_tail
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/history.py", line 85, in catch_corrupt_db
    return f(self, *a, **kw)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/history.py", line 319, in get_tail
    (n,), raw=raw, output=output)
  File "/home/omer/.local/venvs/ipython/lib/python2.7/site-packages/IPython/core/history.py", line 253, in _run_sql
    (toget, sqlfrom) + sql, params)
OperationalError: Could not decode to UTF-8 column 'source_raw' with text '��������������������������������'
```

I don't think it should crash though. Maybe just warn that history is not available?



Discussion:
I'm not sure where the history db is located.
I can try to create a test case out of this if you tell me how to copy it.

I'm not sure where the history db is located.
I can try to create a test case out of this if you tell me how to copy it.


Resolving patch:
diff --git a/IPython/core/history.py b/IPython/core/history.py
--- a/IPython/core/history.py
+++ b/IPython/core/history.py
@@ -70,10 +70,15 @@ def needs_sqlite(f, self, *a, **kw):
 
 if sqlite3 is not None:
     DatabaseError = sqlite3.DatabaseError
+    OperationalError = sqlite3.OperationalError
 else:
     @undoc
     class DatabaseError(Exception):
         "Dummy exception when sqlite could not be imported. Should never occur."
+    
+    @undoc
+    class OperationalError(Exception):
+        "Dummy exception when sqlite could not be imported. Should never occur."
 
 @decorator
 def catch_corrupt_db(f, self, *a, **kw):
@@ -83,7 +88,7 @@ def catch_corrupt_db(f, self, *a, **kw):
     """
     try:
         return f(self, *a, **kw)
-    except DatabaseError:
+    except (DatabaseError, OperationalError):
         if os.path.isfile(self.hist_file):
             # Try to move the file out of the way
             base,ext = os.path.splitext(self.hist_file)


Test patch: