Repository: openshift/openshift-ansible

Issue:
Failures with ansible 2.2.1.0
We're seeing yaml rendering issues with our code and 2.2.1.0. Need to sort out if we need to fix something or if ansible will be fixing things.

Upstream references https://github.com/ansible/ansible/issues/20290 https://github.com/ansible/ansible/issues/20253

```
    TASK [openshift_master : set_fact] *********************************************
    task path: /home/rdu/sdodson/git/Openshift/openshift-ansible/roles/openshift_master/tasks/main.yml:146
    An exception occurred during task execution. The full traceback is:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
        res = self._execute()
      File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 443, in _execute
        self._task.post_validate(templar=templar)
      File "/usr/lib/python2.7/site-packages/ansible/playbook/task.py", line 248, in post_validate
        super(Task, self).post_validate(templar)
      File "/usr/lib/python2.7/site-packages/ansible/playbook/base.py", line 373, in post_validate
        value = templar.template(getattr(self, name))
      File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 427, in template
        disable_lookups=disable_lookups,
      File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 383, in template
        disable_lookups=disable_lookups,
      File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 583, in do_template
        res = j2_concat(rf)
      File "<template>", line 9, in root
      File "/home/rdu/sdodson/git/Openshift/openshift-ansible/filter_plugins/openshift_master.py", line 488, in translate_idps
        return yaml.safe_dump([idp.to_dict() for idp in idp_list], default_flow_style=False)
      File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 218, in safe_dump
        return dump_all([data], stream, Dumper=SafeDumper, **kwds)
      File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 190, in dump_all
        dumper.represent(data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 28, in represent
        node = self.represent_data(data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 57, in represent_data
        node = self.yaml_representers[data_types[0]](self, data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 215, in represent_list
        return self.represent_sequence(u'tag:yaml.org,2002:seq', data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 101, in represent_sequence
        node_item = self.represent_data(item)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 57, in represent_data
        node = self.yaml_representers[data_types[0]](self, data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 223, in represent_dict
        return self.represent_mapping(u'tag:yaml.org,2002:map', data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 123, in represent_mapping
        node_value = self.represent_data(item_value)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 67, in represent_data
        node = self.yaml_representers[None](self, data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 247, in represent_undefined
        raise RepresenterError("cannot represent an object: %s" % data)
    RepresenterError: cannot represent an object: htpasswd_auth
     
    fatal: [ose3-master.example.com]: FAILED! => {
        "failed": true,
        "msg": "Unexpected failure during module execution.",
        "stdout": ""
    }
```


Discussion:
@abutcher had some proposed changes that may address this
CC: @tdawson @detiber 
Casting `ansible.vars.unsafe_proxy.AnsibleUnsafeText` to string in the `IdentityProviderBase` class allowed this to continue. WIP commit here https://github.com/abutcher/openshift-ansible/commit/8d3714f7efe9833659a01ab37c27436ecac4b707
Hi, trying to install v1.4.0 cluster and Ansible is failing because of

```
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: RepresenterError: cannot represent an object: htpasswd_auth
```

I have tried to checkout `release-1.4` branch as well as `openshift-ansible-3.4.55-1` tag.

Is there some workaround that would allow me to finish the installation process except of downgrading Ansible itself?

thanks
@marekjelen I believe you would need to either need to use the commit that @abutcher linked to above. Ansible is working on a new build that resolves the problem, so I do not expect that we will ship a workaround.
Playing whack-a-mole with openshift-ansible workarounds. The next place we hit the issue is within the `to_padded_yaml` filter.

```
TASK [openshift_master : Create master config] *********************************
task path: /home/abutcher/rhat/openshift-ansible/roles/openshift_master/tasks/main.yml:150
fatal: [master1.abutcher.com]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "invocation": {
        "module_args": {
            "backup": true, 
            "dest": "/etc/origin/master/master-config.yaml", 
            "group": "root", 
            "mode": 384, 
            "owner": "root", 
            "src": "master.yaml.v1.j2"
        }, 
        "module_name": "template"
    }, 
    "msg": "AnsibleFilterError: Failed to convert: cannot represent an object: BuildDefaults"
}
```
I can confirm that downgrading **ansible-2.2.1.0-1.el7.noarch.rpm** from **epel-release-7-9** to **ansible-2.2.0.0-4.el7.noarch.rpm** from **epel-release-7-8** is a workaround for a successful openshift-ansible v1.4 installation. Regards, Armin.
@ArminMW Dumb question, but where did you find the old version? I'm not seeing it in EPEL.
@amdonov Just run into exactly the same issue. Installed 2.2.0.0 as a pip.
I found the downlevel ansible in an epel mirror that seems to be lagging behind: 
http://ftp.uni-hannover.de/pub/mirror/linux/epel/7Server/x86_64/a/ansible-2.2.0.0-4.el7.noarch.rpm
@wicksy Thanks for the tip. I took your advice and did something similar to the following. (Hope I'm not missing anything)

sudo yum -y erase ansible
sudo yum install -y "@Development Tools" python2-pip openssl-devel python-devel
sudo pip install -Iv ansible==2.2.0.0

Before going that route I tried applying the patch that closed the Ansible issue. I think that additional work will need to happen to the openshift playbooks to work going forward.
We've also got ansible-2.2.0.0 in the centos paas sig common repo http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin/common/ansible-2.2.0.0-1.el7.noarch.rpm If you choose to downgrade to 2.2.0.0 it should be noted that version is vulnerable to CVE-2016-9587, see https://bugzilla.redhat.com/show_bug.cgi?id=1404378
We'll put a workaround in unless we're sure that we'll have a fixed version of ansible available soon.
Just another confirmation that 2.2.0.0 downgrade results in a pass for me as well. Thanks for linking to the Ansible issues to track.

Resolving patch:
diff --git a/filter_plugins/openshift_master.py b/filter_plugins/openshift_master.py
--- a/filter_plugins/openshift_master.py
+++ b/filter_plugins/openshift_master.py
@@ -10,6 +10,7 @@
 from distutils.version import LooseVersion  # pylint: disable=no-name-in-module,import-error
 
 from ansible import errors
+from ansible.parsing.yaml.dumper import AnsibleDumper
 from ansible.plugins.filter.core import to_bool as ansible_bool
 from six import string_types
 
@@ -485,7 +486,10 @@ def translate_idps(idps, api_version, openshift_version, deployment_type):
             idp_list.append(idp_inst)
 
         IdentityProviderBase.validate_idp_list(idp_list, openshift_version, deployment_type)
-        return yaml.safe_dump([idp.to_dict() for idp in idp_list], default_flow_style=False)
+        return yaml.dump([idp.to_dict() for idp in idp_list],
+                         allow_unicode=True,
+                         default_flow_style=False,
+                         Dumper=AnsibleDumper)
 
     @staticmethod
     def validate_pcs_cluster(data, masters=None):


Test patch: