Repository: conda/conda-build

Issue:
TypeError on directory_size (in utils) on non-English Windows
<!--
Thanks for opening an issue! A few things to keep in mind:

- This issue tracker is for issues with or enhancements for conda-build.

For conda issues, you should open an issue at https://github.com/conda/conda/issues
For Anaconda issues, you should open an issue at https://github.com/ContinuumIO/anaconda-issues/issues
For an issue with a particular conda-forge package, you should open an issue on the 
corresponding feedstock: https://github.com/conda-forge
-->

### Actual Behavior
Following error occurs when trying to build a recipe with `conda build`
```
Traceback (most recent call last):
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\utils.py", line 217, in _execute
    self.disk = max(directory_size(disk_usage_dir), self.disk)
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\utils.py", line 128, in directory_size
    out = re.search("([\d,]+)\sbytes", out).group(1).replace(',', '')
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\re.py", line 182, in search
    return _compile(pattern, flags).search(string)
TypeError: cannot use a string pattern on a bytes-like object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\Scripts\conda-build-script.py", line 10, in <module>
    sys.exit(main())
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\cli\main_build.py", line 420, in main
    execute(sys.argv[1:])
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\cli\main_build.py", line 411, in execute
    verify=args.verify)
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\api.py", line 199, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\build.py", line 2097, in build_tree
    notest=notest,
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\build.py", line 1338, in build
    windows.build(m, build_file, stats=build_stats)
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\windows.py", line 274, in build
    check_call_env(cmd, cwd=src_dir, stats=stats)
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\utils.py", line 294, in check_call_env
    return _func_defaulting_env_to_os_environ('call', *popenargs, **kwargs)
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\utils.py", line 268, in _func_defaulting_env_to_os_environ
    proc = PopenWrapper(_args, **kwargs)
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\utils.py", line 154, in __init__
    self.out, self.err = self._execute(*args, **kwargs)
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\utils.py", line 229, in _execute
    self.disk = max(directory_size(disk_usage_dir), self.disk)
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\site-packages\conda_build\utils.py", line 128, in directory_size
    out = re.search("([\d,]+)\sbytes", out).group(1).replace(',', '')
  File "C:\Users\*****\AppData\Local\Continuum\Anaconda3\lib\re.py", line 182, in search
    return _compile(pattern, flags).search(string)
TypeError: cannot use a string pattern on a bytes-like object
```

### Expected Behavior
No error should occurs

It seems, there is two problems in `directory_size` function:

```python
def directory_size(path):
    '''
    '''
    try:
        if on_win:
            command = "dir /s {}"
            out = subprocess.check_output(command.format(path), shell=True)
        else:
            command = "du -s {}"
            out = subprocess.check_output(command.format(path).split(), stderr=subprocess.PIPE)

        if hasattr(out, 'decode'):
            try:
                out = out.decode()
            # give up.  this isn't important anyway.
            except UnicodeDecodeError:
                pass
        if on_win:
            out = re.search("([\d,]+)\sbytes", out).group(1).replace(',', '')
        else:
            out = out.split()[0]
    except subprocess.CalledProcessError:
        out = directory_size_slow(path)

    return int(out)  # size in bytes
```

1. `out = out.decode()` assumes there is no wrong-character in the output. Using `out = out.decode(errors='replace')` should fix this.
2. the regexp used assumes that the out put of the command will be in English but the output of `dir` is translated. In my case, the correct regexep would be "([\d,]+)\soctets".

Example output of `dir /s`:

- Before `decode()`:
```
b" Le volume dans le lecteur C n'a pas de nom.\r\n Le num\x82ro de s\x82rie du volume est ****-****\r\n\r\n R\x82pertoire de C:\\Users\\*****\\Downloads\\conda-recipes-master\\recipes\\rdkit\r\n\r\n12/03/2018  13:07    <REP>          .\r\n12/03/2018  13:07    <REP>          ..\r\n13/03/2018  09:48             1\xff213 bld.bat\r\n12/03/2018  13:07               876 build.sh\r\n13/03/2018  09:53             1\xff360 meta.yaml\r\n12/03/2018  13:07               715 rdconfig.patch\r\n12/03/2018  13:07               714 rdpaths.patch\r\n12/03/2018  13:07            21\xff412 vs2015.patch\r\n               6 fichier(s)           26\xff290 octets\r\n\r\n     Total des fichiers list\x82s\xff:\r\n               6 fichier(s)           26\xff290 octets\r\n               2 R\x82p(s)  12\xff446\xff318\xff592 octets libres\r\n"
```

- After `decode(errors='replace')`:
```
 Le volume dans le lecteur C n'a pas de nom.
 Le numro de srie du volume est ****-****

 Rpertoire de C:\Users\*****\Downloads\conda-recipes-master\recipes\rdkit

12/03/2018  13:07    <REP>          .
12/03/2018  13:07    <REP>          ..
13/03/2018  09:48             1213 bld.bat
12/03/2018  13:07               876 build.sh
13/03/2018  09:53             1360 meta.yaml
12/03/2018  13:07               715 rdconfig.patch
12/03/2018  13:07               714 rdpaths.patch
12/03/2018  13:07            21412 vs2015.patch
               6 fichier(s)           26290 octets

     Total des fichiers lists:
               6 fichier(s)           26290 octets
               2 Rp(s)  12446318592 octets libres
```
### Steps to Reproduce
Start `conda build any-package` on a non-English Windows version (French for me).

##### `Output of conda info`
```
     active environment : None
       user config file : C:\Users\*****\.condarc
 populated config files : C:\Users\*****\.condarc
          conda version : 4.4.11
    conda-build version : 3.7.1
         python version : 3.6.1.final.0
       base environment : C:\Users\*****\AppData\Local\Continuum\Anaconda3  (writ
able)
           channel URLs : https://conda.anaconda.org/anaconda-fusion/win-64
                          https://conda.anaconda.org/anaconda-fusion/noarch
                          https://repo.continuum.io/pkgs/main/win-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/win-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/win-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/win-64
                          https://repo.continuum.io/pkgs/pro/noarch
                          https://repo.continuum.io/pkgs/msys2/win-64
                          https://repo.continuum.io/pkgs/msys2/noarch
          package cache : C:\Users\*****\AppData\Local\Continuum\Anaconda3\pkgs
                          C:\Users\*****\AppData\Local\conda\conda\pkgs
       envs directories : C:\Users\*****\AppData\Local\Continuum\Anaconda3\envs
                          C:\Users\*****\AppData\Local\conda\conda\envs
                          C:\Users\*****\.conda\envs
               platform : win-64
             user-agent : conda/4.4.11 requests/2.18.4 CPython/3.6.1 Windows/7 W
indows/6.1.7601
          administrator : False
             netrc file : None
           offline mode : False
```




Discussion:
Thanks.  I'll see what we can do here.  Generalizing the translated word for bytes probably isn't feasible, so I'll see if there's another way.

Resolving patch:
diff --git a/conda_build/utils.py b/conda_build/utils.py
--- a/conda_build/utils.py
+++ b/conda_build/utils.py
@@ -120,15 +120,15 @@ def directory_size(path):
 
         if hasattr(out, 'decode'):
             try:
-                out = out.decode(errors='replace')
+                out = out.decode(errors='ignore')
             # This isn't important anyway so give up. Don't try search on bytes.
             except (UnicodeDecodeError, IndexError):
                 if on_win:
                     return 0
                 else:
-                    pass                   
+                    pass
         if on_win:
-            # Windows can give long output, we need only 2nd to last line 
+            # Windows can give long output, we need only 2nd to last line
             out = out.strip().rsplit('\r\n', 2)[-2]
             pattern = "\s([\d\W]+).+"  # Language and punctuation neutral
             out = re.search(pattern, out.strip()).group(1).strip()
@@ -234,11 +234,8 @@ def _execute(self, *args, **kwargs):
             _popen.kill()
             raise
 
-        finally:
-            # Get disk usage
-            self.disk = max(directory_size(disk_usage_dir), self.disk)
-            self.elapsed = time.time() - start_time
-
+        self.disk = max(directory_size(disk_usage_dir), self.disk)
+        self.elapsed = time.time() - start_time
         return _popen.stdout, _popen.stderr
 
     def __repr__(self):


Test patch: