Repository: ipython/ipython

Issue:
Notebooks with filenames with accented characters ("ã", "í", etc.) don't open in 2.0
I'm using Python 2.7, Conda 3.4.0 and IPython 2.0 (on windows, with chrome).

To reproduce the problem, create a notebook, change the name to include an accented character, close the notebook and try to reopen it. This worked before 2.0.



Discussion:
This might be Windows specific, it does not appear to be the case on OS X or Linux.  What is `IPython.utils.encoding.DEFAULT_ENCODING` on your machine?

'cp1252'

Hi, I am experiencing the same issue. Windows 7, Python 2.7 (64 bits), IPython 2.0, firefox. Furthermore, using nbconvert on a file with a non-ascii character will report the following error:
`[NbConvertApp] WARNING | pattern u'Validation des mod\xdeles uniaxial.ipynb' matched no files`

Where the filename is "Validation des modèles uniaxial.ipynb"

I have the same problem on Japanese Windows 7 with Japanese filename notebook.
I debuged this and found that:

Japanese filename notebook will cause the following code in main.js failed:

```
var opts = {
    base_url : IPython.utils.get_body_data("baseUrl"),
    notebook_path : IPython.utils.get_body_data("notebookPath"),
    notebook_name : IPython.utils.get_body_data('notebookName')
};
```

`get_body_data()` will call `decodeURIComponent()` which raises URIError: URI malformed.

You can reproduce the error by execute following code in a javascript console:

```
decodeURIComponent("02%20%93d%97%ACPI%90%A7%8C%E4.ipynb")
```

So the problem is that IPython quote the filename in SHIFT-JIS encoding.

After adding `encoding="utf8"` to the following function in IPython.html.utils, I can 
open and save Japanese filename notebook correctly:

```
def url_escape(path):
    """Escape special characters in a URL path

    Turns '/foo bar/' into '/foo%20bar/'
    """
    parts = py3compat.unicode_to_str(path, encoding="utf8").split('/')
    return u'/'.join([quote(p) for p in parts])
```

Yes, we need to be careful about encodings when we are talking to the system (use system encoding), and encodings when we are talking to the web frontend (always utf8).

FWIW I've fixed the issue with nbconvert, it's related to active code page in windows and sys.stdin.encoding not reporting the correct code page. I can submit a pull request if you're interested, or just describe the fix it's rather simple. I haven't gotten notebook to open accented notebooks yet but zhangruoyu seems to be onto something.

@auxym sorry for this dropping off my radar. Yes, it would be helpful to get a fix for this, if you have one.

On windows, one should _not_ use the system encoding; use the unicode APIs instead! After all, on Windows, filenames are text, not bytes.

For instance, if you have a file called ポケモン.txt ('pokémon'), and are running a Western-European Windows version:

``` python
>>> glob.glob("*.txt") # str/bytes API
['????.txt']
>>> glob.glob(u"*.txt") # unicode API
[u'\u30dd\u30b1\u30e2\u30f3.txt']
```

The error auxym reports is somewhat strange:

``` python
[NbConvertApp] WARNING | pattern u'Validation des mod\xdeles uniaxial.ipynb' matched no files
```
- i.e. u'Validation des modÞles uniaxial.ipynb', which is obviously different from modèles. I can't come up with an obvious reason for that mojibake. Auxym, what's the exact command you use for that, and what's the output of chcp in the same cmd window?


Resolving patch:
diff --git a/IPython/html/utils.py b/IPython/html/utils.py
--- a/IPython/html/utils.py
+++ b/IPython/html/utils.py
@@ -55,7 +55,7 @@ def url_escape(path):
     
     Turns '/foo bar/' into '/foo%20bar/'
     """
-    parts = py3compat.unicode_to_str(path).split('/')
+    parts = py3compat.unicode_to_str(path, encoding='utf8').split('/')
     return u'/'.join([quote(p) for p in parts])
 
 def url_unescape(path):
@@ -64,8 +64,8 @@ def url_unescape(path):
     Turns '/foo%20bar/' into '/foo bar/'
     """
     return u'/'.join([
-        py3compat.str_to_unicode(unquote(p))
-        for p in py3compat.unicode_to_str(path).split('/')
+        py3compat.str_to_unicode(unquote(p), encoding='utf8')
+        for p in py3compat.unicode_to_str(path, encoding='utf8').split('/')
     ])
 
 def is_hidden(abs_path, abs_root=''):


Test patch: