Repository: ipython/ipython

Issue:
Failed magic import appears not to throw error
I'm trying to test that an ipython magic works inside of jupyter by doing this: `jupyter nbconvert --execute --to html tests/bigquery.ipynb`, where the `biquery.ipynb` notebook imports a magic called `%bigquery`. Currently, the `%bigquery` magic, which allows you to query Google's BigQuery service, is not importing due to a config problem.

Unexpectedly, even though the output html says `ERROR:root:Line magic function %bigquery not found.`, the jupyter nbconvert command itself does not crash, even though it's supposed to if any exceptions arise. 

If it doesn't crash when it's supposed to, I can't reliably test whether the magic is in fact working. This appears to be the default ipython behavior. As @minrk put it in gitter: "The error handling in magics is probably an IPython thing. For a long time, the only thing that mattered for IPython was what it looked like. But then Jupyter came around, and it now makes a difference whether an error was raised, or just 'reported' to output. This is the kind of thing that affects nbconvert's "this failed". But there are still a number of 'errors' that are caught and displayed as nicer errors than tracebacks, but now Jupyter frontends don't know that the cell failed."

To reproduce the issue in ipython:
```python
In [2]: import IPython
In [3]: shell = IPython.core.interactiveshell.InteractiveShell.instance()
In [4]: shell.run_cell('%bigquery')
ERROR:root:Line magic function `%bigquery` not found.
Out[4]: <ExecutionResult object at 7f27b7605e50, execution_count=None error_before_exec=None error_in_exec=None result=None>
```

Just wanted to flag this bug!


Discussion:

Resolving patch:
diff --git a/IPython/core/interactiveshell.py b/IPython/core/interactiveshell.py
--- a/IPython/core/interactiveshell.py
+++ b/IPython/core/interactiveshell.py
@@ -2076,7 +2076,7 @@ def run_line_magic(self, magic_name, line, _stack_depth=1):
             etpl = "Line magic function `%%%s` not found%s."
             extra = '' if cm is None else (' (But cell magic `%%%%%s` exists, '
                                     'did you mean that instead?)' % magic_name )
-            error(etpl % (magic_name, extra))
+            raise UsageError(etpl % (magic_name, extra))
         else:
             # Note: this is the distance in the stack to the user's frame.
             # This will need to be updated if the internal calling logic gets
@@ -2115,7 +2115,7 @@ def run_cell_magic(self, magic_name, line, cell):
             etpl = "Cell magic `%%{0}` not found{1}."
             extra = '' if lm is None else (' (But line magic `%{0}` exists, '
                             'did you mean that instead?)'.format(magic_name))
-            error(etpl.format(magic_name, extra))
+            raise UsageError(etpl.format(magic_name, extra))
         elif cell == '':
             message = '%%{0} is a cell magic, but the cell body is empty.'.format(magic_name)
             if self.find_line_magic(magic_name) is not None:


Test patch:
diff --git a/IPython/core/tests/test_magic.py b/IPython/core/tests/test_magic.py
--- a/IPython/core/tests/test_magic.py
+++ b/IPython/core/tests/test_magic.py
@@ -74,6 +74,35 @@ def test_extract_symbols_raises_exception_with_non_python_code():
     with nt.assert_raises(SyntaxError):
         code.extract_symbols(source, "hello")
 
+
+def test_magic_not_found():
+    # magic not found raises UsageError
+    with nt.assert_raises(UsageError):
+        _ip.magic('doesntexist')
+
+    # ensure result isn't success when a magic isn't found
+    result = _ip.run_cell('%doesntexist')
+    assert isinstance(result.error_in_exec, UsageError)
+
+
+def test_cell_magic_not_found():
+    # magic not found raises UsageError
+    with nt.assert_raises(UsageError):
+        _ip.run_cell_magic('doesntexist', 'line', 'cell')
+
+    # ensure result isn't success when a magic isn't found
+    result = _ip.run_cell('%%doesntexist')
+    assert isinstance(result.error_in_exec, UsageError)
+
+
+def test_magic_error_status():
+    def fail(shell):
+        1/0
+    _ip.register_magic_function(fail)
+    result = _ip.run_cell('%fail')
+    assert isinstance(result.error_in_exec, ZeroDivisionError)
+
+
 def test_config():
     """ test that config magic does not raise
     can happen if Configurable init is moved too early into