Repository: openshift/openshift-ansible

Issue:
Long DN string with spaces can cause incorrect YAML to be generated in master-config
#### Description
I have a configuration file where my settings for an LDAP identity provider has a long string containing spaces, when this is converted to YAML, line feeds are inserted on some of the longer DN/LDAP filter strings, this results in the master API service failing to start due to invalid config.

Modifying the following `yaml.dump()` to include `width=1000` for example results in a working master-config.yaml.

https://github.com/openshift/openshift-ansible/blob/7496b1235f72bd4241e4917f50df722174bf90fa/roles/openshift_master_facts/filter_plugins/openshift_master.py#L496-L499


##### Version
```
ansible 2.2.1.0
atomic-openshift-utils-3.4.67-1.git.0.14a0b4d.el7.noarch
openshift-ansible-3.4.67-1.git.0.14a0b4d.el7.noarch
```

##### Steps To Reproduce
Create a long LDAP identity provider in openshift_master_identity_providers that has spaces in the DN. eg. `'url':'ldap://url.to.some.ldap.server/DC=blah,DC=foo,DC=bar?uid??(memberof=CN=UserGroupForPaaS,OU=Groups,OU=Unit With Spaces For Some Reason,OU=Also With - In Case,DC=blah,DC=foo,DC=bar)'`

##### Expected Results
master-config.yaml with a DN that is readable by the OpenShift master service.

##### Observed Results
Master service fails to start.
Long DN string with spaces can cause incorrect YAML to be generated in master-config
#### Description
I have a configuration file where my settings for an LDAP identity provider has a long string containing spaces, when this is converted to YAML, line feeds are inserted on some of the longer DN/LDAP filter strings, this results in the master API service failing to start due to invalid config.

Modifying the following `yaml.dump()` to include `width=1000` for example results in a working master-config.yaml.

https://github.com/openshift/openshift-ansible/blob/7496b1235f72bd4241e4917f50df722174bf90fa/roles/openshift_master_facts/filter_plugins/openshift_master.py#L496-L499


##### Version
```
ansible 2.2.1.0
atomic-openshift-utils-3.4.67-1.git.0.14a0b4d.el7.noarch
openshift-ansible-3.4.67-1.git.0.14a0b4d.el7.noarch
```

##### Steps To Reproduce
Create a long LDAP identity provider in openshift_master_identity_providers that has spaces in the DN. eg. `'url':'ldap://url.to.some.ldap.server/DC=blah,DC=foo,DC=bar?uid??(memberof=CN=UserGroupForPaaS,OU=Groups,OU=Unit With Spaces For Some Reason,OU=Also With - In Case,DC=blah,DC=foo,DC=bar)'`

##### Expected Results
master-config.yaml with a DN that is readable by the OpenShift master service.

##### Observed Results
Master service fails to start.


Discussion:
If the lines should never wrap the surefire fix would probably be a `yaml.dump(idp, width=float("inf"))`
I can reproduce this and verify your suggested solution would work for the general case of not-wrapping long lines (that is desirable).

```python
In [41]: from ansible.parsing.yaml.dumper import AnsibleDumper

In [42]: import yaml

In [43]: s = """---
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
bar: foofoofoofoofoofoofoofoofoofoo
"""

In [44]: s_loaded = yaml.load(s)

In [45]: s, s_loaded
Out[45]: 
('---\nfoo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar \nbar: foofoofoofoofoofoofoofoofoofoo\n',
 {'bar': 'foofoofoofoofoofoofoofoofoofoo',
  'foo': 'bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar'})

In [46]: no_width = 'no_width.yaml'

In [47]: with_width = 'with_width.yaml'

In [48]: with open(with_width, 'w') as fp:
    fp.write(yaml.dump(s_loaded, default_flow_style=False, width=500, Dumper=AnsibleDumper))
   ....: 

In [49]: with open(no_width, 'w') as fp:
    fp.write(yaml.dump(s_loaded, default_flow_style=False, Dumper=AnsibleDumper))
   ....: 
```

Which produces

```
$ echo "---sep---" | cat no_width.yaml - with_width.yaml
bar: foofoofoofoofoofoofoofoofoofoo
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
  bar bar bar bar bar bar bar bar bar bar bar bar bar
---sep---
bar: foofoofoofoofoofoofoofoofoofoo
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
```

Works with the `width=float("inf")` kwarg added too

```python
In [56]: inf_width = "inf_width.yaml"

In [57]: with open(inf_width, 'w') as fp:
    fp.write(yaml.dump(s_loaded, default_flow_style=False, width=float("inf"), Dumper=AnsibleDumper))
   ....: 
```

Which produces the same

```
$ cat no_width.yaml with_width.yaml inf_width.yaml 
bar: foofoofoofoofoofoofoofoofoofoo
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
  bar bar bar bar bar bar bar bar bar bar bar bar bar
bar: foofoofoofoofoofoofoofoofoofoo
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
bar: foofoofoofoofoofoofoofoofoofoo
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
```
Should I submit a PR or you got this? :)

(thanks for the quick response)
oooooooooo boy. There are a lot of instances of `yaml.dump` in our repo

```
$ grep -r 'yaml\.dump(' playbooks roles *_plugins | wc -l
61
```

But about 55 of those are in the generated `lib_openshift` files which include two code fragments from `roles/lib_utils/src/class/yedit.py` and `roles/lib_openshift/src/lib/base.py`

```
$ grep -r 'yaml\.dump(' playbooks roles *_plugins | grep -v -E '^roles/lib_openshift/library/' | wc -l                                                                                            
6
```

Updating the `yedit.py` and `base.py` files will require regenerating all of the `lib_openshift/library/` modules and that can get a little dicey if you're not comfortable with that process. One of us can take that on and track the work.

Would you mind submitting a PR for the fix in the file you pointed out in the OP? That is the quickest way to get you on the road again properly.
If the lines should never wrap the surefire fix would probably be a `yaml.dump(idp, width=float("inf"))`
I can reproduce this and verify your suggested solution would work for the general case of not-wrapping long lines (that is desirable).

```python
In [41]: from ansible.parsing.yaml.dumper import AnsibleDumper

In [42]: import yaml

In [43]: s = """---
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
bar: foofoofoofoofoofoofoofoofoofoo
"""

In [44]: s_loaded = yaml.load(s)

In [45]: s, s_loaded
Out[45]: 
('---\nfoo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar \nbar: foofoofoofoofoofoofoofoofoofoo\n',
 {'bar': 'foofoofoofoofoofoofoofoofoofoo',
  'foo': 'bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar'})

In [46]: no_width = 'no_width.yaml'

In [47]: with_width = 'with_width.yaml'

In [48]: with open(with_width, 'w') as fp:
    fp.write(yaml.dump(s_loaded, default_flow_style=False, width=500, Dumper=AnsibleDumper))
   ....: 

In [49]: with open(no_width, 'w') as fp:
    fp.write(yaml.dump(s_loaded, default_flow_style=False, Dumper=AnsibleDumper))
   ....: 
```

Which produces

```
$ echo "---sep---" | cat no_width.yaml - with_width.yaml
bar: foofoofoofoofoofoofoofoofoofoo
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
  bar bar bar bar bar bar bar bar bar bar bar bar bar
---sep---
bar: foofoofoofoofoofoofoofoofoofoo
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
```

Works with the `width=float("inf")` kwarg added too

```python
In [56]: inf_width = "inf_width.yaml"

In [57]: with open(inf_width, 'w') as fp:
    fp.write(yaml.dump(s_loaded, default_flow_style=False, width=float("inf"), Dumper=AnsibleDumper))
   ....: 
```

Which produces the same

```
$ cat no_width.yaml with_width.yaml inf_width.yaml 
bar: foofoofoofoofoofoofoofoofoofoo
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
  bar bar bar bar bar bar bar bar bar bar bar bar bar
bar: foofoofoofoofoofoofoofoofoofoo
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
bar: foofoofoofoofoofoofoofoofoofoo
foo: bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar
```
Should I submit a PR or you got this? :)

(thanks for the quick response)
oooooooooo boy. There are a lot of instances of `yaml.dump` in our repo

```
$ grep -r 'yaml\.dump(' playbooks roles *_plugins | wc -l
61
```

But about 55 of those are in the generated `lib_openshift` files which include two code fragments from `roles/lib_utils/src/class/yedit.py` and `roles/lib_openshift/src/lib/base.py`

```
$ grep -r 'yaml\.dump(' playbooks roles *_plugins | grep -v -E '^roles/lib_openshift/library/' | wc -l                                                                                            
6
```

Updating the `yedit.py` and `base.py` files will require regenerating all of the `lib_openshift/library/` modules and that can get a little dicey if you're not comfortable with that process. One of us can take that on and track the work.

Would you mind submitting a PR for the fix in the file you pointed out in the OP? That is the quickest way to get you on the road again properly.

Resolving patch:
diff --git a/roles/openshift_master_facts/filter_plugins/openshift_master.py b/roles/openshift_master_facts/filter_plugins/openshift_master.py
--- a/roles/openshift_master_facts/filter_plugins/openshift_master.py
+++ b/roles/openshift_master_facts/filter_plugins/openshift_master.py
@@ -496,6 +496,7 @@ def translate_idps(idps, api_version, openshift_version, deployment_type):
         return u(yaml.dump([idp.to_dict() for idp in idp_list],
                            allow_unicode=True,
                            default_flow_style=False,
+                           width=float("inf"),
                            Dumper=AnsibleDumper))
 
     @staticmethod


Test patch: