Repository: conda/conda

Issue:
Installing from environment.yml produces unexpected error, maybe due to character encoding
<!--
Hi!  Read this; it's important.

This is an issue tracker for conda -- the package manager.  File feature requests
for conda here, as well as bug reports about something conda has messed up.

Anaconda Community Code of Conduct: https://www.anaconda.com/community-code-of-conduct/

If your issue is a bug report for:
  * a specific conda package from Anaconda ('defaults' channel):
      ==> file at https://github.com/ContinuumIO/anaconda-issues
  * a specific conda package from conda-forge:
      ==> file at the corresponding feedstock under https://github.com/conda-forge
  * repo.anaconda.com access and service:
      ==> file at https://github.com/ContinuumIO/anaconda-issues
  * anaconda.org access and service:
      ==> file at https://anaconda.org/contact/report
  * commands under 'conda build':
      ==> file at https://github.com/conda/conda-build
  * commands under 'conda env':
      ==> please file it here!
  * all other conda commands that start with 'conda':
      ==> please file it here!

If you continue on,

   **please include all requested information below.**

If a maintainer determines the information is required to understand 
your issue, and if it is not provided, your issue may be
closed automatically.

-->

## Current Behavior
<!-- What actually happens?
     If you want to include console output, please use "Steps to Reproduce" below. -->
I'm using `conda env export` to generate an `environment.yml` file. It's my understanding that these files should make conda environments portable, thus easily replicated on a different machine. Please correct me if I'm wrong with that.

I'm able to successfully create an environment, activate and export it, deactivate and remove it, but when I try to recreate it from the `environment.yml` file, I'm getting a message stating `An unexpected error has occurred.`

### Steps to Reproduce
<!-- If the current behavior is a bug, please provide specific, minimal steps to independently reproduce.
     Include the exact conda commands that reproduce the issue and their output between the ticks below. -->
```
# create the environment
conda create --name testenv python=3.8

# activate and export the environment to yml
conda activate testenv
conda env export > environment.yml

# deactivate and remove the environment
conda deactivate
conda remove --name testenv --all

# try to restore environment from environment.yml
conda env create -v -v -v -f .\environment.yml
```
All of the commands succeed, except for `conda env create`
<details open><summary><code>`conda env create -f environment.yml`</code> <strong>ERROR REPORT</strong></summary><p>
<!-- between the ticks below, paste the output of 'conda info' -->

```
DEBUG conda.gateways.logging:set_verbosity(231): verbosity set to 3

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "D:\Program_Files\miniconda3\lib\site-packages\conda\exceptions.py", line 1079, in __call__
        return func(*args, **kwargs)
      File "D:\Program_Files\miniconda3\lib\site-packages\conda_env\cli\main.py", line 80, in do_call
        exit_code = getattr(module, func_name)(args, parser)
      File "D:\Program_Files\miniconda3\lib\site-packages\conda_env\cli\main_create.py", line 80, in execute
        directory=os.getcwd())
      File "D:\Program_Files\miniconda3\lib\site-packages\conda_env\specs\__init__.py", line 40, in detect
        if spec.can_handle():
      File "D:\Program_Files\miniconda3\lib\site-packages\conda_env\specs\yaml_file.py", line 18, in can_handle
        self._environment = env.from_file(self.filename)
      File "D:\Program_Files\miniconda3\lib\site-packages\conda_env\env.py", line 151, in from_file
        return from_yaml(yamlstr, filename=filename)
      File "D:\Program_Files\miniconda3\lib\site-packages\conda_env\env.py", line 136, in from_yaml
        data = yaml_load_standard(yamlstr)
      File "D:\Program_Files\miniconda3\lib\site-packages\conda\common\serialize.py", line 76, in yaml_load_standard
        return yaml.load(string, Loader=yaml.Loader, version="1.2")
      File "D:\Program_Files\miniconda3\lib\site-packages\ruamel_yaml\main.py", line 933, in load
        loader = Loader(stream, version, preserve_quotes=preserve_quotes)
      File "D:\Program_Files\miniconda3\lib\site-packages\ruamel_yaml\loader.py", line 50, in __init__
        Reader.__init__(self, stream, loader=self)
      File "D:\Program_Files\miniconda3\lib\site-packages\ruamel_yaml\reader.py", line 85, in __init__
        self.stream = stream  # type: Any  # as .read is called
      File "D:\Program_Files\miniconda3\lib\site-packages\ruamel_yaml\reader.py", line 117, in stream
        self.check_printable(val)
      File "D:\Program_Files\miniconda3\lib\site-packages\ruamel_yaml\reader.py", line 255, in check_printable
        'special characters are not allowed',
    ruamel_yaml.reader.ReaderError: unacceptable character #x0000: special characters are not allowed
      in "<unicode string>", position 3

`$ D:\Program_Files\miniconda3\Scripts\conda-env-script.py create -v -v -v -f .\environment.yml`

  environment variables:
                 CIO_TEST=<not set>
  CONDA_AUTO_UPDATE_CONDA=false
        CONDA_DEFAULT_ENV=base
                CONDA_EXE=D:\Program_Files\miniconda3\Scripts\conda.exe
             CONDA_PREFIX=D:\Program_Files\miniconda3
    CONDA_PROMPT_MODIFIER=(base)
         CONDA_PYTHON_EXE=D:\Program_Files\miniconda3\python.exe
               CONDA_ROOT=D:\Program_Files\miniconda3
              CONDA_SHLVL=1
                 HOMEPATH=\
                     PATH=D:\Program_Files\miniconda3;D:\Program_Files\miniconda3\Library\mingw-
                          w64\bin;D:\Program_Files\miniconda3\Library\usr\bin;D:\Program_Files\m
                          iniconda3\Library\bin;D:\Program_Files\miniconda3\Scripts;D:\Program_F
                          iles\miniconda3\bin;D:\Program_Files\miniconda3;D:\Program_Files\minic
                          onda3\Library\mingw-w64\bin;D:\Program_Files\miniconda3\Library\usr\bi
                          n;D:\Program_Files\miniconda3\Library\bin;D:\Program_Files\miniconda3\
                          Scripts;D:\Program_Files\miniconda3\bin;D:\Program_Files\miniconda3\co
                          ndabin;C:\Oracle11\Ora11G-64\11.2.0\client_1\bin;C:\Oracle\ORA11\11.2.
                          0\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDO
                          WS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Prog
                          ram Files (x86)\WebEx\Productivity Tools;C:\Program
                          Files\Git\cmd;C:\Program Files (x86)\Microsoft SQL
                          Server\150\DTS\Binn;C:\Program Files (x86)\ArcGIS\Bin;C:\Program Files
                          (x86)\ArcGIS\EsriProductionMapping\Desktop10.5\Bin;C:\Program
                          Files\nodejs;C:\ProgramData\chocolatey\bin;C:\Program Files (x86)\Webe
                          x\Webex\Applications;C:\Users\avitale\AppData\Local\Microsoft\WindowsA
                          pps;C:\Users\avitale\AppData\Local\Programs\Microsoft VS Code\bin;C:\U
                          sers\avitale\AppData\Roaming\npm;C:\Python3X;C:\OSGeo4W\bin;C:\Python3
                          X;C:\OSGeo4W\bin;C:\Python3X;C:\OSGeo4W\bin;C:\OSGeo4W\bin;C:\Program
                          Files\dotnet;C:\Users\avitale\AppData\Local\Microsoft\WindowsApps;.;C:
                          \Users\avitale\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\a
                          vitale\AppData\Roaming\npm;C:\Users\avitale\.dotnet\tools
             PSMODULEPATH=C:\Users\avitale\Documents\WindowsPowerShell\Modules;C:\Program Files\
                          WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\M
                          odules
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : base
    active env location : D:\Program_Files\miniconda3
            shell level : 1
       user config file : C:\Users\avitale\.condarc
 populated config files : C:\Users\avitale\.condarc
          conda version : 4.8.2
    conda-build version : not installed
         python version : 3.7.4.final.0
       virtual packages : __cuda=10.2
       base environment : D:\Program_Files\miniconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : D:\Program_Files\miniconda3\pkgs
                          C:\Users\avitale\.conda\pkgs
                          C:\Users\avitale\AppData\Local\conda\conda\pkgs
       envs directories : D:\Program_Files\miniconda3\envs
                          C:\Users\avitale\.conda\envs
                          C:\Users\avitale\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.4 Windows/10 Windows/10.0.17134
          administrator : False
             netrc file : None
           offline mode : False


An unexpected error has occurred. Conda has prepared the above report.

If submitted, this report will be used by core maintainers to improve
future releases of conda.
Would you like conda to send this report to the core maintainers?

[y/N]:

No report sent. To permanently opt-out, use

    $ conda config --set report_errors false

```
</p></details>

## Expected Behavior
<!-- What do you think should happen? -->
I believe that the `conda env create -f .\environment.yml` command should install the packages listed in the `environment.yml` file without error.

### Workaround
I found a strange workaround, that seems to work on two separate Windows 10 machines. If I open the `environment.yml` file in VS Code, change the character encoding to UTF-8 and "reopen with encoding", hit `ctrl+z` to undo the change, then save the file, the install works as expected. The default character encoding for the `environment.yml` file that's output from conda is UTF-16 LE.

The error from the YAML parser led me down this path:
```python
      File "D:\Program_Files\miniconda3\lib\site-packages\conda_env\env.py", line 136, in from_yaml
        data = yaml_load_standard(yamlstr)
      File "D:\Program_Files\miniconda3\lib\site-packages\conda\common\serialize.py", line 76, in yaml_load_standard
        return yaml.load(string, Loader=yaml.Loader, version="1.2")
      File "D:\Program_Files\miniconda3\lib\site-packages\ruamel_yaml\main.py", line 933, in load
        loader = Loader(stream, version, preserve_quotes=preserve_quotes)
      File "D:\Program_Files\miniconda3\lib\site-packages\ruamel_yaml\loader.py", line 50, in __init__
        Reader.__init__(self, stream, loader=self)
      File "D:\Program_Files\miniconda3\lib\site-packages\ruamel_yaml\reader.py", line 85, in __init__
        self.stream = stream  # type: Any  # as .read is called
      File "D:\Program_Files\miniconda3\lib\site-packages\ruamel_yaml\reader.py", line 117, in stream
        self.check_printable(val)
      File "D:\Program_Files\miniconda3\lib\site-packages\ruamel_yaml\reader.py", line 255, in check_printable
        'special characters are not allowed',
    ruamel_yaml.reader.ReaderError: unacceptable character #x0000: special characters are not allowed
      in "<unicode string>", position 3
```

## Environment Information
<details open><summary><code>`conda info`</code></summary><p>
<!-- between the ticks below, paste the output of 'conda info' -->

```
     active environment : base
    active env location : D:\Program_Files\miniconda3
            shell level : 1
       user config file : C:\Users\avitale\.condarc
 populated config files : C:\Users\avitale\.condarc
          conda version : 4.8.2
    conda-build version : not installed
         python version : 3.7.4.final.0
       virtual packages : __cuda=10.2
       base environment : D:\Program_Files\miniconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : D:\Program_Files\miniconda3\pkgs
                          C:\Users\avitale\.conda\pkgs
                          C:\Users\avitale\AppData\Local\conda\conda\pkgs
       envs directories : D:\Program_Files\miniconda3\envs
                          C:\Users\avitale\.conda\envs
                          C:\Users\avitale\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.4 Windows/10 Windows/10.0.17134
          administrator : False
             netrc file : None
           offline mode : False
```
</p></details>


<details open><summary><code>`conda config --show-sources`</code></summary><p>
<!-- between the ticks below, paste the output of 'conda config --show-sources' -->

```
==> C:\Users\avitale\.condarc <==
ssl_verify: False
```
</p></details>


<details><summary><code>`conda list --show-channel-urls`</code></summary><p>
<!-- between the ticks below, paste the output of 'conda list --show-channel-urls' -->

```
# packages in environment at D:\Program_Files\miniconda3:
#
# Name                    Version                   Build  Channel
asn1crypto                1.3.0                    py37_0    defaults
ca-certificates           2020.1.1                      0    defaults
certifi                   2019.11.28               py37_0    defaults
cffi                      1.14.0           py37h7a1dbc1_0    defaults
chardet                   3.0.4                 py37_1003    defaults
conda                     4.8.2                    py37_0    defaults
conda-package-handling    1.6.0            py37h62dcd97_0    defaults
console_shortcut          0.1.1                         3    defaults
cryptography              2.8              py37h7a1dbc1_0    defaults
idna                      2.8                      py37_0    defaults
menuinst                  1.4.16           py37he774522_0    defaults
openssl                   1.1.1d               he774522_4    defaults
pip                       20.0.2                   py37_1    defaults
powershell_shortcut       0.0.1                         2    defaults
pycosat                   0.6.3            py37he774522_0    defaults
pycparser                 2.19                     py37_0    defaults
pyopenssl                 19.1.0                   py37_0    defaults
pysocks                   1.7.1                    py37_0    defaults
python                    3.7.4                h5263a28_0    defaults
pywin32                   227              py37he774522_1    defaults
requests                  2.22.0                   py37_1    defaults
ruamel_yaml               0.15.87          py37he774522_0    defaults
setuptools                45.2.0                   py37_0    defaults
six                       1.14.0                   py37_0    defaults
sqlite                    3.31.1               he774522_0    defaults
tqdm                      4.42.1                     py_0    defaults
urllib3                   1.25.8                   py37_0    defaults
vc                        14.1                 h0510ff6_4    defaults
vs2015_runtime            14.16.27012          hf0eaf9b_1    defaults
wheel                     0.34.2                   py37_0    defaults
win_inet_pton             1.1.0                    py37_0    defaults
wincertstore              0.2                      py37_0    defaults
yaml                      0.1.7                hc54c509_2    defaults
```
</p></details>



Discussion:
I had the some error. The YML file was created by conda on a Windows 10 machine, and I tried to recreate the environment on another Windows 10 machine.

I converted the YML file in Notepad++ to UTF-8, after that it worked.
The workaround using VS code that vitale232 described worked for me, too.
It looks like the bad encoding happens when you write out using `>`

I was getting the same error, had to change the export to:
 `conda env export -f environment.yml`
or
`conda env export | Out-File environment.yml -Encoding utf8NoBom`

Resolving patch:
diff --git a/conda_env/env.py b/conda_env/env.py
--- a/conda_env/env.py
+++ b/conda_env/env.py
@@ -155,8 +155,12 @@ def from_file(filename):
     elif not os.path.exists(filename):
         raise exceptions.EnvironmentFileNotFound(filename)
     else:
-        with open(filename, 'r') as fp:
-            yamlstr = fp.read()
+        with open(filename, 'rb') as fp:
+            yamlb = fp.read()
+            try:
+                yamlstr = yamlb.decode('utf-8')
+            except UnicodeDecodeError:
+                yamlstr = yamlb.decode('utf-16')
     return from_yaml(yamlstr, filename=filename)
 
 


Test patch: