Repository: ansible/ansible

Issue:
Don't set LANG to C on modules which don't need it
##### Issue Type:

Improvement
##### Ansible Version:

1.6
##### Environment:

N/A
##### Summary:

Modules should not set LANG to C, unless it is really needed. This is especially important for modules like `command` and `shell`.

This issue is a follow up to the following forum discussion https://groups.google.com/forum/#!topic/ansible-project/XwJmFPfMfo0
##### Steps To Reproduce:

N/A
##### Expected Results:

N/A
##### Actual Results:

N/A



Discussion:
I believe this workaround is available, but I completely agree with the above.  (Needs testing):

```
- shell: /usr/bin/foo
  env:
     LANG: your_language_here
```

Equivalently

```
- shell: /usr/bin/foo
  env: "{{ env }}"
```

I would suggest we probably code it for shell and command right now but think more about the larger question for the future.

I looked at what is the easiest way to accomplish this and I think it would be best to add one more parameter to `AnsibleModule` class constructor e.g. `set_default_lang=True`, so if a module doesn't want the `LANG` to be set, they would just pass `set_default_lang=False`.

In the future, we could easily set this to `False` by default and only set it to `True` in modules that require `LANG` to be `C`.

Also I'm not sure if `LANG` could be unset, so it might be a good idea to also check if it is empty and set it to default in that case.

I could throw in a pull request in the next few days if you like this idea?

Please ignore my commit above, I haven't tested anything. I removed that commit, so it should remove it from this issue until I test everything.

Commit 993aa96424e9eafd59fc879f9551b0c10f976e20 should be fine. I tried running `echo $LANG` with the `shell` module (since `command` module runs the `shell` module) and it worked as expected. By default it printed the default system `LANG`. Then I tried setting `LANG` with what you suggested as a workaround, but it is not `env`, but `environment`:

```
- shell: /usr/bin/foo
  environment:
     LANG: your_language_here
```

and it set the `LANG` to what was configured by the `environment`, which is also expected.

I also tried running `setup` module and it printed that `LANG=c`, which means all other modules will still use `c` by default.

cc @jimi-c just made some changes here - https://github.com/ansible/ansible/commit/c115c34a1f5d570422b2560cc3ff0ce0ebe53dff - though this clearly isn't the solution fix (assumes English)

Just linking for future record.

I'm not sure if that commit is a good idea. Setting default `LANG` to `en_US.UTF-8` could bring a lot of regression problems in other modules. If I understand correctly the reason why Ansible sets `LANG=c` is because some modules throw errors if the `LANG` is different than `c`.

Maybe a better option would be to merge this pull request and modify the `subversion` module to pass `set_default_lang=False`. It should solve all the problems with `subversion` module and it won't cause any possible problems in other modules.

Also should I added setting `LC_CTYPE` as well?

@kustodian my feeling on this was that LANG=C is essentially an ASCII environment, and where en_us.UTF-8 is a superset of ASCII it should not cause problems. For those who had internationalization issues before when LANG=C they were most likely having to set the default_lang variable in ansible.cfg already, so I thought this would at least prevent a large class of unicode errors.

I just remember that @mpdehaan said once in the forums, what is the reasoning behind LANG=C, that some modules caused errors, or output different results in different language settings, so they set a default LANG=C to resolve this problem. Maybe you are right, maybe everything will work with en_us.UTF-8, but we don't know and this could cause a lot of regression problems, which we don't need, so why not just leave LANG=C on all modules and just modify modules which have problems to use the default system. Don't get me wrong, I'm all that en_US.UTF-8 is the default LANG, since I had a few problems because of this which took me some time to figure out why they are happening, but I would rather play it safe.

This is probably @mpdehaan decision, to leave `C` as default, or change it to `en_US.UTF-8`, but some modules like `command` (and probably `subversion`) would benefit more if they didn't set LANG at all, but use the system default.

I just realized that only some modules have problems with some languages, so if we allow changing default LANG in ansible.cfg, that means a user could still break those modules. We should probably reconsider and make it possible for some modules to override the default `LANG` and set a LANG which works for them. Of course this would mean that we would need to go over history to find which modules are that, but would also mean that some new modules could have problems.

In my opinion Ansible should not set LANG and use the default system LANG for all modules, except those modules which we know have problems with some languages, those modules should set LANG which works for them.

We have two solutions to this problem:
1. If we know which modules have problems and we don't think there will be regression problems:
   - drop ANSIBLE_MODULE_LANG and make the system LANG the default
   - make it possible for modules to set their LANG (we will identify which modules will need this)
2. If we are not sure what modules will have problems and we would like to minimize regression bugs:
   - leave the default LANG=C
   - make it possible for modules to set their LANG (we will identify which modules will need this)
   - make it possible for modules to use default system LANG (e.g. `command` module)

What do you think about this?

So after discussing further, I think we're going to handle it as follows:

We will modify the `_compute_environment_string()` function in `runner/__init__.py` to add the proper localization environment. This will allow users to override the environment per-task if they so choose, but we will continue to default to something sane (probably en_US.UTF-8).


Resolving patch:
diff --git a/lib/ansible/module_common.py b/lib/ansible/module_common.py
--- a/lib/ansible/module_common.py
+++ b/lib/ansible/module_common.py
@@ -28,7 +28,6 @@
 
 REPLACER = "#<<INCLUDE_ANSIBLE_MODULE_COMMON>>"
 REPLACER_ARGS = "\"<<INCLUDE_ANSIBLE_MODULE_ARGS>>\""
-REPLACER_LANG = "\"<<INCLUDE_ANSIBLE_MODULE_LANG>>\""
 REPLACER_COMPLEX = "\"<<INCLUDE_ANSIBLE_MODULE_COMPLEX_ARGS>>\""
 
 class ModuleReplacer(object):
@@ -140,12 +139,10 @@ def modify_module(self, module_path, complex_args, module_args, inject):
                 encoded_args = repr(module_args.encode('utf-8'))
             except UnicodeDecodeError:
                 encoded_args = repr(module_args)
-            encoded_lang = repr(C.DEFAULT_MODULE_LANG)
             encoded_complex = repr(complex_args_json)
 
             # these strings should be part of the 'basic' snippet which is required to be included
             module_data = module_data.replace(REPLACER_ARGS, encoded_args)
-            module_data = module_data.replace(REPLACER_LANG, encoded_lang)
             module_data = module_data.replace(REPLACER_COMPLEX, encoded_complex)
 
             if module_style == 'new':
@@ -154,7 +151,6 @@ def modify_module(self, module_path, complex_args, module_args, inject):
                     facility = inject['ansible_syslog_facility']
                 module_data = module_data.replace('syslog.LOG_USER', "syslog.%s" % facility)
 
-
             lines = module_data.split("\n")
             shebang = None
             if lines[0].startswith("#!"):
diff --git a/lib/ansible/module_utils/basic.py b/lib/ansible/module_utils/basic.py
--- a/lib/ansible/module_utils/basic.py
+++ b/lib/ansible/module_utils/basic.py
@@ -30,7 +30,6 @@
 # == BEGIN DYNAMICALLY INSERTED CODE ==
 
 MODULE_ARGS = "<<INCLUDE_ANSIBLE_MODULE_ARGS>>"
-MODULE_LANG = "<<INCLUDE_ANSIBLE_MODULE_LANG>>"
 MODULE_COMPLEX_ARGS = "<<INCLUDE_ANSIBLE_MODULE_COMPLEX_ARGS>>"
 
 BOOLEANS_TRUE = ['yes', 'on', '1', 'true', 1]
@@ -191,8 +190,6 @@ def __init__(self, argument_spec, bypass_checks=False, no_log=False,
                 if k not in self.argument_spec:
                     self.argument_spec[k] = v
 
-        os.environ['LANG'] = MODULE_LANG
-        os.environ['LC_CTYPE'] = MODULE_LANG
         (self.params, self.args) = self._load_params()
 
         self._legal_inputs = ['CHECKMODE', 'NO_LOG']
diff --git a/lib/ansible/runner/__init__.py b/lib/ansible/runner/__init__.py
--- a/lib/ansible/runner/__init__.py
+++ b/lib/ansible/runner/__init__.py
@@ -287,15 +287,21 @@ def _transfer_str(self, conn, tmp, name, data):
     def _compute_environment_string(self, inject=None):
         ''' what environment variables to use when running the command? '''
 
-        if not self.environment:
-            return ""
-        enviro = template.template(self.basedir, self.environment, inject, convert_bare=True)
-        enviro = utils.safe_eval(enviro)
-        if type(enviro) != dict:
-            raise errors.AnsibleError("environment must be a dictionary, received %s" % enviro)
+        default_environment = collections.OrderedDict([
+            ('LANG', C.DEFAULT_MODULE_LANG),
+            ('LC_CTYPE', C.DEFAULT_MODULE_LANG),
+        ])
+
+        if self.environment:
+            enviro = template.template(self.basedir, self.environment, inject, convert_bare=True)
+            enviro = utils.safe_eval(enviro)
+            if type(enviro) != dict:
+                raise errors.AnsibleError("environment must be a dictionary, received %s" % enviro)
+            default_environment.update(enviro)
+
         result = ""
-        for (k,v) in enviro.iteritems():
-            result = "%s=%s %s" % (k, pipes.quote(unicode(v)), result)
+        for (k,v) in default_environment.iteritems():
+            result = "%s %s=%s" % (result, k, pipes.quote(unicode(v)))
         return result
 
     # *****************************************************


Test patch: