Repository: spyder-ide/spyder

Issue:
Error when trying to get a very large namespace view from the kernel
I keep getting the same error after running the following code:

`from brian import *`
`brian_sample_run()`

However the output seems to be fine on IPython console.


**Error**

```python-traceback
  File "/home/biomike/anaconda2/lib/python2.7/site-packages/qtconsole/base_frontend_mixin.py", line 163, in _dispatch
Traceback (most recent call last):
  File "/home/biomike/anaconda2/lib/python2.7/site-packages/qtconsole/base_frontend_mixin.py", line 163, in _dispatch
    handler(msg)
  File "/home/biomike/anaconda2/lib/python2.7/site-packages/spyder/widgets/ipythonconsole/namespacebrowser.py", line 214, in _handle_execute_reply
    self.handle_exec_method(msg)
  File "/home/biomike/anaconda2/lib/python2.7/site-packages/spyder/widgets/ipythonconsole/shell.py", line 317, in handle_exec_method
    properties = ast.literal_eval(data['text/plain'])
  File "/home/biomike/anaconda2/lib/python2.7/ast.py", line 49, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/home/biomike/anaconda2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 9001
    ...}
    ^
SyntaxError: invalid syntax
    handler(msg)
  File "/home/biomike/anaconda2/lib/python2.7/site-packages/spyder/widgets/ipythonconsole/namespacebrowser.py", line 214, in _handle_execute_reply
    self.handle_exec_method(msg)
  File "/home/biomike/anaconda2/lib/python2.7/site-packages/spyder/widgets/ipythonconsole/shell.py", line 311, in handle_exec_method
    view = ast.literal_eval(data['text/plain'])
  File "/home/biomike/anaconda2/lib/python2.7/ast.py", line 49, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/home/biomike/anaconda2/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 4001
    ...}
    ^
SyntaxError: invalid syntax
```

## Version and main components

* Spyder Version: 3.2.0 
* Python Version: 2.7.13
* Qt Versions: 5.6.2, PyQt5 5.6 on Linux

## Dependencies
```
pyflakes >=0.5.0  :  1.5.0 (OK)
pycodestyle >=2.3 :  2.3.1 (OK)
pygments >=2.0    :  2.2.0 (OK)
pandas >=0.13.1   :  0.20.3 (OK)
numpy >=1.7       :  1.13.1 (OK)
sphinx >=0.6.6    :  1.6.2 (OK)
rope >=0.9.4      :  0.9.4 (OK)
jedi >=0.9.0      :  0.10.2 (OK)
psutil >=0.3      :  5.2.2 (OK)
nbconvert >=4.0   :  5.2.1 (OK)
sympy >=0.7.3     :  1.1.1 (OK)
cython >=0.21     :  0.26 (OK)
qtconsole >=4.2.0 :  4.3.0 (OK)
IPython >=4.0;<6.0:  5.3.0 (OK)
pylint >=0.25     :  1.6.4 (OK)
```



Discussion:
Update:
i get the same error everytime i am trying to import everything from a module
for example just running:
`from numpy import *`

will result to the same error
I know where this error is coming from. We'll fix it in Spyder 3.2.1, to be released this weekend.

In the meantime, please avoid star imports (i.e. `from foo import *`) because they can't be handled by Spyder.
What is the error @ccordoba12 ?
It's not our fault (I think). IPython is not sending the namespace representation that we build in the kernel correctly, but as a truncated dictionary (because it contains too much elements).

That's why we get `...` at the end of representation, which makes `ast` to fail.
Hmm, ok, how do we even fix that :-| ?
I was thinking to call `repr` in the kernel side, to avoid this truncation. Then we would need to call `ast` twice in our side.
@dalthviz, please help me with this one. I think the idea is to use `repr` in these places:

https://github.com/spyder-ide/spyder/blob/3.x/spyder/utils/ipython/spyder_kernel.py#L113

and

https://github.com/spyder-ide/spyder/blob/3.x/spyder/utils/ipython/spyder_kernel.py#L141

Then we need to use `ast.literal_eval` twice in these places:

https://github.com/spyder-ide/spyder/blob/3.x/spyder/widgets/ipythonconsole/shell.py#L330

and

https://github.com/spyder-ide/spyder/blob/3.x/spyder/widgets/ipythonconsole/shell.py#L336

Resolving patch:
diff --git a/spyder/utils/ipython/spyder_kernel.py b/spyder/utils/ipython/spyder_kernel.py
--- a/spyder/utils/ipython/spyder_kernel.py
+++ b/spyder/utils/ipython/spyder_kernel.py
@@ -110,8 +110,10 @@ def get_namespace_view(self):
         settings = self.namespace_view_settings
         if settings:
             ns = self._get_current_namespace()
-            view = make_remote_view(ns, settings, EXCLUDED_NAMES)
+            view = repr(make_remote_view(ns, settings, EXCLUDED_NAMES))
             return view
+        else:
+            return repr(None)
 
     def get_var_properties(self):
         """
@@ -138,9 +140,9 @@ def get_var_properties(self):
                     'array_ndim': self._get_array_ndim(value)
                 }
 
-            return properties
+            return repr(properties)
         else:
-            return {}
+            return repr(None)
 
     def get_value(self, name):
         """Get the value of a variable"""
diff --git a/spyder/widgets/ipythonconsole/debugging.py b/spyder/widgets/ipythonconsole/debugging.py
--- a/spyder/widgets/ipythonconsole/debugging.py
+++ b/spyder/widgets/ipythonconsole/debugging.py
@@ -9,6 +9,8 @@
 mode and Spyder
 """
 
+import ast
+
 from qtpy.QtCore import Qt
 
 from qtconsole.rich_jupyter_widget import RichJupyterWidget
@@ -52,10 +54,12 @@ def refresh_from_pdb(self, pdb_state):
             self.sig_pdb_step.emit(fname, lineno)
 
         if 'namespace_view' in pdb_state:
-            self.sig_namespace_view.emit(pdb_state['namespace_view'])
+            self.sig_namespace_view.emit(ast.literal_eval(
+                    pdb_state['namespace_view']))
 
         if 'var_properties' in pdb_state:
-            self.sig_var_properties.emit(pdb_state['var_properties'])
+            self.sig_var_properties.emit(ast.literal_eval(
+                    pdb_state['var_properties']))
 
     # ---- Private API (overrode by us) ----------------------------
     def _handle_input_request(self, msg):
diff --git a/spyder/widgets/ipythonconsole/shell.py b/spyder/widgets/ipythonconsole/shell.py
--- a/spyder/widgets/ipythonconsole/shell.py
+++ b/spyder/widgets/ipythonconsole/shell.py
@@ -327,13 +327,15 @@ def handle_exec_method(self, msg):
                 data = reply.get('data')
                 if 'get_namespace_view' in method:
                     if data is not None and 'text/plain' in data:
-                        view = ast.literal_eval(data['text/plain'])
+                        literal = ast.literal_eval(data['text/plain'])
+                        view = ast.literal_eval(literal)
                     else:
                         view = None
                     self.sig_namespace_view.emit(view)
                 elif 'get_var_properties' in method:
                     if data is not None and 'text/plain' in data:
-                        properties = ast.literal_eval(data['text/plain'])
+                        literal = ast.literal_eval(data['text/plain'])
+                        properties = ast.literal_eval(literal)
                     else:
                         properties = None
                     self.sig_var_properties.emit(properties)


Test patch:
diff --git a/spyder/plugins/tests/test_ipythonconsole.py b/spyder/plugins/tests/test_ipythonconsole.py
--- a/spyder/plugins/tests/test_ipythonconsole.py
+++ b/spyder/plugins/tests/test_ipythonconsole.py
@@ -77,6 +77,21 @@ def close_console():
 #==============================================================================
 # Tests
 #==============================================================================
+@flaky(max_runs=3)
+def test_console_import_namespace(ipyconsole, qtbot):
+    """Test an import of the form 'from foo import *'."""
+    # Wait until the window is fully up
+    shell = ipyconsole.get_current_shellwidget()
+    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
+
+    # Import numpy
+    with qtbot.waitSignal(shell.executed):
+        shell.execute('from numpy import *')
+
+    # Assert we get the e value correctly
+    assert shell.get_value('e') == 2.718281828459045
+
+
 @flaky(max_runs=3)
 def test_console_disambiguation(ipyconsole, qtbot):
     """Test the disambiguation of dedicated consoles."""