Repository: ipython/ipython

Issue:
nbconvert: Default encoding problem on OS X
Greetings.

I am using IPython 1.1.0 via MacPorts on OSX 10.7.5.  The following problem is reproducible on the master git branch (IPython 2.0.0-dev).

On any call to nbconvert, I get the following failure:

```
[NbConvertApp] Using existing profile dir: u'/Users/USERNAME_REDACTED/.ipython/profile_default'
[NbConvertApp] Converting notebook ticks.ipynb to html
[NbConvertApp] Support files will be in ticks_files/
Traceback (most recent call last):
  File "/opt/local/bin/ipython", line 6, in <module>
    start_ipython()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/__init__.py", line 118, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/config/application.py", line 545, in launch_instance
    app.start()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/terminal/ipapp.py", line 358, in start
    return self.subapp.start()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/nbconvert/nbconvertapp.py", line 267, in start
    self.convert_notebooks()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/nbconvert/nbconvertapp.py", line 300, in convert_notebooks
    output, resources = exporter.from_filename(notebook_filename, resources=resources)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/nbconvert/exporters/exporter.py", line 288, in from_filename
    with io.open(filename) as f:
LookupError: unknown encoding: 

If you suspect this is an IPython bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@scipy.org

You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    c.Application.verbose_crash=True
```

This is an easy fix: I change the troublesome line such that it reads,

```
with io.open(filename, encoding='ascii') as f:
```

However, this ad hoc and likely a suboptimal solution.  I wanted to bring this to the developers' attention and inquire about a proper solution.  Thanks!

System info:

```
python -c "import IPython; print(IPython.sys_info())"
{'codename': 'An Afternoon Hack',
 'commit_hash': '7c2ea3a',
 'commit_source': 'installation',
 'default_encoding': 'US-ASCII',
 'ipython_path': '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython',
 'ipython_version': '1.1.0',
 'os_name': 'posix',
 'platform': 'Darwin-11.4.2-x86_64-i386-64bit',
 'sys_executable': '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python',
 'sys_platform': 'darwin',
 'sys_version': '2.7.6 (default, Nov 19 2013, 16:37:14) \n[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)]'}
```



Discussion:
I hate the fact that `io.open` has an environment-dependent default, it's so easy for things like this to get overlooked. PR coming up.

That said, you're using a modern Mac - why on earth is the default encoding anything other than UTF-8? :confused: 


Resolving patch:
diff --git a/IPython/nbconvert/exporters/exporter.py b/IPython/nbconvert/exporters/exporter.py
--- a/IPython/nbconvert/exporters/exporter.py
+++ b/IPython/nbconvert/exporters/exporter.py
@@ -139,7 +139,7 @@ def from_filename(self, filename, resources=None, **kw):
         modified_date = datetime.datetime.fromtimestamp(os.path.getmtime(filename))
         resources['metadata']['modified_date'] = modified_date.strftime(text.date_format)
 
-        with io.open(filename) as f:
+        with io.open(filename, encoding='utf-8') as f:
             return self.from_notebook_node(nbformat.read(f, 'json'), resources=resources, **kw)
 
 


Test patch:
diff --git a/IPython/core/tests/test_magic.py b/IPython/core/tests/test_magic.py
--- a/IPython/core/tests/test_magic.py
+++ b/IPython/core/tests/test_magic.py
@@ -911,7 +911,7 @@ def _run_edit_test(arg_s, exp_filename=None,
     if exp_filename is not None:
         nt.assert_equal(exp_filename, filename)
     if exp_contents is not None:
-        with io.open(filename, 'r') as f:
+        with io.open(filename, 'r', encoding='utf-8') as f:
             contents = f.read()
         nt.assert_equal(exp_contents, contents)
     if exp_lineno != -1:
diff --git a/IPython/html/services/notebooks/tests/test_notebooks_api.py b/IPython/html/services/notebooks/tests/test_notebooks_api.py
--- a/IPython/html/services/notebooks/tests/test_notebooks_api.py
+++ b/IPython/html/services/notebooks/tests/test_notebooks_api.py
@@ -108,7 +108,8 @@ def setUp(self):
 
         for d, name in self.dirs_nbs:
             d = d.replace('/', os.sep)
-            with io.open(pjoin(nbdir, d, '%s.ipynb' % name), 'w') as f:
+            with io.open(pjoin(nbdir, d, '%s.ipynb' % name), 'w',
+                         encoding='utf-8') as f:
                 nb = new_notebook(name=name)
                 write(nb, f, format='ipynb')
 
diff --git a/IPython/html/services/sessions/tests/test_sessions_api.py b/IPython/html/services/sessions/tests/test_sessions_api.py
--- a/IPython/html/services/sessions/tests/test_sessions_api.py
+++ b/IPython/html/services/sessions/tests/test_sessions_api.py
@@ -59,7 +59,8 @@ def setUp(self):
             if e.errno != errno.EEXIST:
                 raise
 
-        with io.open(pjoin(nbdir, 'foo', 'nb1.ipynb'), 'w') as f:
+        with io.open(pjoin(nbdir, 'foo', 'nb1.ipynb'), 'w',
+                     encoding='utf-8') as f:
             nb = new_notebook(name='nb1')
             write(nb, f, format='ipynb')