internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: openshift/openshift-ansible

Issue:
Parameterize some defaults for Logging to prevent cluster collaps 
Lets see if we can prevent https://github.com/openshift/origin/issues/8868 at the installer level, at least temporarily if not permanantly.

TL;DR :We don't want `docker logs` from noisy containers to fill up the disks and kill your cluster!



Discussion:
cc @timothysc @rrati 

What logging options are you suggesting to resolve this? Can they not already be set via openshift_docker_log_driver and openshift_docker_log_options variables?

We need to configure defaults for docker to log using journald + and set sane roll-over defaults.  

Right now the defaults OOTB will explode a reasonable sized cluster. 

I think setting docker to log to the journal and configure the journals defaults.  

We have yet to determine what are sane defaults on the journald for densely packed nodes.  

Can you drive coordinating with cluster infra and containers teams on what the best defaults should be? We're happy to implement whatever the decision is but I don't think that's something that our team should decide.

CC: @ncdc @jwhonce @mfojtik @cgwalters 

Isn't the default already the journald log driver?

@ncdc no. 

@timothysc ??? isn't the RHEL/Docker default journald?

Nevermind, I see that's just in Fedora.


Resolving patch:
diff --git a/roles/openshift_facts/library/openshift_facts.py b/roles/openshift_facts/library/openshift_facts.py
--- a/roles/openshift_facts/library/openshift_facts.py
+++ b/roles/openshift_facts/library/openshift_facts.py
@@ -1714,7 +1714,9 @@ def get_defaults(self, roles, deployment_type):
                                     set_node_ip=False)
 
         if 'docker' in roles:
-            docker = dict(disable_push_dockerhub=False, hosted_registry_insecure=True)
+            docker = dict(disable_push_dockerhub=False,
+                          hosted_registry_insecure=True,
+                          options='--log-driver=json-file --log-opt max-size=50m')
             version_info = get_docker_version_info()
             if version_info is not None:
                 docker['api_version'] = version_info['api_version']


Test patch:
Repository: openshift/openshift-ansible

Issue:
RequestHeaderIdentityProvider clientCommonNames option
https://github.com/openshift/origin/pull/8443
https://github.com/openshift/openshift-docs/pull/1888/files

RequestHeaderIdentityProvider now has an option to specify which client certificates are accepted. Previously, any client certificate validated by the specified `clientCA` would be accepted. Now, a list of allowed common names may also be provided in `clientCommonNames`.



Discussion:

Resolving patch:
diff --git a/filter_plugins/openshift_master.py b/filter_plugins/openshift_master.py
--- a/filter_plugins/openshift_master.py
+++ b/filter_plugins/openshift_master.py
@@ -237,7 +237,11 @@ def __init__(self, api_version, idp):
         self._required += [['headers']]
         self._optional += [['challengeURL', 'challenge_url'],
                            ['loginURL', 'login_url'],
-                           ['clientCA', 'client_ca']]
+                           ['clientCA', 'client_ca'],
+                           ['clientCommonNames', 'client_common_names'],
+                           ['emailHeaders', 'email_headers'],
+                           ['nameHeaders', 'name_headers'],
+                           ['preferredUsernameHeaders', 'preferred_username_headers']]
 
     def validate(self):
         ''' validate this idp instance '''


Test patch:
Repository: openshift/openshift-ansible

Issue:
enable service-serving-cert-signer by default
Commands like `oadm router` want to take advantage of the service-serving-cert-signing controller.  This requires that 

``` yaml
controllerConfig:
  serviceServingCert:
    signer:
      certFile: service-signer.crt
      keyFile: service-signer.key
```

in the master-config.yaml be configured with a signer.  This signs for internal DNS names and can be created using a command like:

``` bash
oadm ca create-signer-cert --cert=service-signer.crt --key=service-signer.key --serial=service-signer-serial.txt
```

https://github.com/openshift/openshift-docs/pull/2324 has docs about using it.



Discussion:
@pecameron you need this fix.

@knobunc, @sdodson  assigned @abutcher to do this.

changing to p0. unable to deploy 3.3 routers without.


Resolving patch:
diff --git a/filter_plugins/openshift_master.py b/filter_plugins/openshift_master.py
--- a/filter_plugins/openshift_master.py
+++ b/filter_plugins/openshift_master.py
@@ -550,6 +550,9 @@ def certificates_to_synchronize(hostvars, include_keys=True):
             certs += ['openshift-master.crt',
                       'openshift-master.key',
                       'openshift-master.kubeconfig']
+        if bool(hostvars['openshift']['common']['version_gte_3_3_or_1_3']):
+            certs += ['service-signer.crt',
+                      'service-signer.key']
         return certs
 
     @staticmethod


Test patch:
Repository: openshift/openshift-ansible

Issue:
Named certificates with SANs cause duplicate name entries when cacert is specified
When deploying custom named certificates, if you don't specify the names, [they will be detected from the certificate's CN and SANs](https://github.com/openshift/openshift-ansible/blob/openshift-ansible-3.4.1-1/filter_plugins/oo_filters.py#L509-L537). Now, the first entry in a certificate's SAN list is always the CN. As such, the resulting list needs to be deduplicated. And, as it turns out, [this is done](https://github.com/openshift/openshift-ansible/blob/openshift-ansible-3.4.1-1/filter_plugins/oo_filters.py#L533).

Unfortunately, for some reason, this is only done if the `cacert` field on the Ansible certificate object [is unspecified](https://github.com/openshift/openshift-ansible/blob/openshift-ansible-3.4.1-1/filter_plugins/oo_filters.py#L531-L533). I'm not quite sure the purpose of the preceding line but I think the aim is to not use self-signed certificates for internal communications.

In any event, the deduplication of the name list should be performed regardless of whether or not a value for `cacert` is provided. As it stands, if you specify `cacert` but not `names` for a certificate with SANs, you'll get an entry in `master-config.yaml`'s `named_certificates` field that looks like this:

``` yaml
- certFile: /etc/origin/master/named_certificates/console.crt
  keyFile: /etc/origin/master/named_certificates/console.key
  names:
  - "subdomain.example.com"
  - "subdomain.example.com"
  - "example.com"
```

Which will then cause the `origin-master` service to fail with the following error:

> servingInfo.namedCertificates[0].names[1]: Invalid value: "subdomain.example.com": this name is already used in another named certificate



Discussion:

Resolving patch:
diff --git a/filter_plugins/oo_filters.py b/filter_plugins/oo_filters.py
--- a/filter_plugins/oo_filters.py
+++ b/filter_plugins/oo_filters.py
@@ -529,9 +529,9 @@ def oo_parse_named_certificates(certificates, named_certs_dir, internal_hostname
                 raise errors.AnsibleFilterError(("|failed to parse certificate '%s', " % certificate['certfile'] +
                                                  "please specify certificate names in host inventory"))
 
+            certificate['names'] = list(set(certificate['names']))
             if 'cafile' not in certificate:
                 certificate['names'] = [name for name in certificate['names'] if name not in internal_hostnames]
-                certificate['names'] = list(set(certificate['names']))
                 if not certificate['names']:
                     raise errors.AnsibleFilterError(("|failed to parse certificate '%s' or " % certificate['certfile'] +
                                                      "detected a collision with internal hostname, please specify " +


Test patch:
Repository: openshift/openshift-ansible

Issue:
hosted registry created a NFS pvc claim when using S3
Inventory had s3 set as the provider, and was deployed successfully. 

```
openshift_hosted_registry_storage_kind=object
openshift_hosted_registry_storage_provider=s3
```

However a NFS PVC claim was created with the name `registry-claim`



Discussion:

Resolving patch:
diff --git a/filter_plugins/oo_filters.py b/filter_plugins/oo_filters.py
--- a/filter_plugins/oo_filters.py
+++ b/filter_plugins/oo_filters.py
@@ -735,13 +735,14 @@ def oo_persistent_volume_claims(hostvars, persistent_volume_claims=None):
         if 'hosted' in hostvars['openshift']:
             for component in hostvars['openshift']['hosted']:
                 if 'storage' in hostvars['openshift']['hosted'][component]:
-                    kind = hostvars['openshift']['hosted'][component]['storage']['kind']
-                    create_pv = hostvars['openshift']['hosted'][component]['storage']['create_pv']
-                    create_pvc = hostvars['openshift']['hosted'][component]['storage']['create_pvc']
-                    if kind != None and create_pv and create_pvc:
-                        volume = hostvars['openshift']['hosted'][component]['storage']['volume']['name']
-                        size = hostvars['openshift']['hosted'][component]['storage']['volume']['size']
-                        access_modes = hostvars['openshift']['hosted'][component]['storage']['access_modes']
+                    params = hostvars['openshift']['hosted'][component]['storage']
+                    kind = params['kind']
+                    create_pv = params['create_pv']
+                    create_pvc = params['create_pvc']
+                    if kind not in [None, 'object'] and create_pv and create_pvc:
+                        volume = params['volume']['name']
+                        size = params['volume']['size']
+                        access_modes = params['access_modes']
                         persistent_volume_claim = dict(
                             name="{0}-claim".format(volume),
                             capacity=size,


Test patch:
Repository: openshift/openshift-ansible

Issue:
Ansible 2.2 bug in AWS cloud provider ini file
We are testing a new version of Ansible (ansible-2.2.0.0-0.70.rc3.el7.noarch).

In the new version of Ansible, the ini module will die if the ini file is NOT there.  According to the ini module docs, it looks like there is a new option called "create:"  However, I'm not sure if create is backwards compatible.

I see 2 options. 1) "touch" the file before it's created.  2) use "create: true"

Here is where it's bombing in the ansible code:

https://github.com/openshift/openshift-ansible/blob/master/roles/openshift_cloud_provider/tasks/aws.yml

Here is the ini module doc.  Please see the "create" statement.

http://docs.ansible.com/ansible/ini_file_module.html



Discussion:
Thanks for testing the new build, this is really helpful.


Resolving patch:
diff --git a/utils/src/ooinstall/cli_installer.py b/utils/src/ooinstall/cli_installer.py
--- a/utils/src/ooinstall/cli_installer.py
+++ b/utils/src/ooinstall/cli_installer.py
@@ -121,11 +121,6 @@ def collect_hosts(oo_cfg, existing_env=False, masters_set=False, print_summary=T
 are prompted to identify a *separate* system to act as the load balancer for
 your cluster once you define all masters and nodes.
 
-If only one master is specified, an etcd instance is embedded within the
-OpenShift master service to use as the datastore. This can be later replaced
-with a separate etcd instance, if required. If multiple masters are specified,
-then a separate etcd cluster is configured with each master serving as a member.
-
 Any masters configured as part of this installation process are also
 configured as nodes. This enables the master to proxy to pods
 from the API. By default, this node is unschedulable, but this can be changed
@@ -263,11 +258,8 @@ def print_host_summary(all_hosts, host):
             click.echo("  - Load Balancer (Preconfigured)")
         else:
             click.echo("  - Load Balancer (HAProxy)")
-    if host.is_master():
-        if host.is_etcd_member(all_hosts):
-            click.echo("  - Etcd Member")
-        else:
-            click.echo("  - Etcd (Embedded)")
+    if host.is_etcd():
+        click.echo("  - Etcd")
     if host.is_storage():
         click.echo("  - Storage")
 
diff --git a/utils/src/ooinstall/oo_config.py b/utils/src/ooinstall/oo_config.py
--- a/utils/src/ooinstall/oo_config.py
+++ b/utils/src/ooinstall/oo_config.py
@@ -120,6 +120,10 @@ def is_master_lb(self):
     def is_storage(self):
         return 'storage' in self.roles
 
+    def is_etcd(self):
+        """ Does this host have the etcd role """
+        return 'etcd' in self.roles
+
     def is_etcd_member(self, all_hosts):
         """ Will this host be a member of a standalone etcd cluster. """
         if not self.is_master():


Test patch:
Repository: openshift/openshift-ansible

Issue:
Flannel vars renamed in flannel 0.5.5
It seems that the upcoming flannel-0.5.5 uses different variable names in /etc/sysconfig/flannel.
##### Version

rhel 7.3
flannel-0.5.5-1.el7.x86_64
openshift-ansible commit 553ce409a40305a984894a980fc3969c11f41aba
##### Steps To Reproduce

deploy with flannel on rhel 7.3
##### Current Result

flanneld service fails to start
##### Expected Result

flanneld service starts
##### Additional Information

This fix works for the new version:
roles/flannel/tasks/main.yml
   lineinfile:
￼      dest: /etc/sysconfig/flanneld
￼      backrefs: yes
￼ -    regexp: "^(FLANNEL_ETCD=)"
￼ +    regexp: "^(FLANNEL_ETCD_ENDPOINTS=)"
￼      line: '\1{{ etcd_hosts|join(",") }}'
￼  
￼  - name: Set flannel etcd key
￼    become: yes
￼    lineinfile:
￼      dest: /etc/sysconfig/flanneld
￼      backrefs: yes
￼ -    regexp: "^(FLANNEL_ETCD_KEY=)"
￼ +    regexp: "^(FLANNEL_ETCD_PREFIX=)"
￼      line: '\1{{ flannel_etcd_key }}'



Discussion:
@ingvagabund Do you know if we could set both old and new values without breaking flannel?

The problem is the `flannel.service` file that expects new names of envs. We could grep the service file and check if it expects the new names or the old names and regenerate the `flanneld` file accordingly. 

I'm suggesting we just add both and we avoid trying to do conditional logic in ansible based on whether it/s 0.5.5 or earlier. Does it ignore envars it doesn't know about?

That is okay as long as the config file is touched by automatic scripts only. It can be confusing for the user and cause problems when only part of the vars is modified.

Yes, the service ignores all unused envs.

https://bugzilla.redhat.com/show_bug.cgi?id=1391515


Resolving patch:
diff --git a/utils/src/ooinstall/cli_installer.py b/utils/src/ooinstall/cli_installer.py
--- a/utils/src/ooinstall/cli_installer.py
+++ b/utils/src/ooinstall/cli_installer.py
@@ -306,6 +306,7 @@ def validate_prompt_lb(hostname):
     else:
         return None
 
+
 def set_cluster_hostname(oo_cfg):
     message = """
 You have chosen to install a single master cluster (non-HA).
@@ -321,6 +322,7 @@ def set_cluster_hostname(oo_cfg):
                                     default='')
     oo_cfg.deployment.variables['openshift_master_cluster_hostname'] = cluster_hostname
 
+
 def collect_storage_host(hosts):
     """
     Get a valid host for storage from the user and append it to the list of


Test patch:
Repository: openshift/openshift-ansible

Issue:
error because pyOpenSSL is not installed
I get an error because pyOpenSSL is not installed when I run this from an f25 system (the host where ansible is installed). 

```
TASK [Verify Ansible version is greater than or equal to 2.1.0.0] **************
fatal: [localhost]: FAILED! => {"failed": true, "msg": "The conditional check 'not ansible_version.full | version_compare('2.1.0.0', 'ge')' failed. The error was: No module
 named OpenSSL.crypto\n\nThe error appears to have been in '/sharedfolder/code/github.com/openshift/openshift-ansible/playbooks/common/openshift-cluster/verify_ansible_vers
ion.yml': line 8, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n  - name: Verify Ansi
ble version is greater than or equal to 2.1.0.0\n    ^ here\n"}
```

After I install pyOpenSSL everything works fine. Maybe that is a dep that should get installed by the rpm. Maybe not. If not, we should give a better error message to the users of openshift-ansible. 

##### Version
Working from 90fb07f and using `ansible-2.2.0.0-3.fc25.noarch.rpm` on fedora 25. No pyOpenSSL is installed. 


Discussion:
@mscherer @detiber - thoughts? 
I agree we need a better error message. Let me take a look.
So the problem is on https://github.com/openshift/openshift-ansible/blob/master/filter_plugins/oo_filters.py#L13 

We do import the module without making sure it is here. 

We can either install it prior to using the filter, or importing openssl conditionally and if we can't, disable oo_parse_named_certificates . Or just fail that method if openssl is not present.

I am gonna try the last method.

Resolving patch:
diff --git a/filter_plugins/oo_filters.py b/filter_plugins/oo_filters.py
--- a/filter_plugins/oo_filters.py
+++ b/filter_plugins/oo_filters.py
@@ -10,7 +10,14 @@
 from distutils.util import strtobool
 from distutils.version import LooseVersion
 from operator import itemgetter
-import OpenSSL.crypto
+
+HAS_OPENSSL=False
+try:
+    import OpenSSL.crypto
+    HAS_OPENSSL=True
+except ImportError:
+    pass
+
 import os
 import pdb
 import pkg_resources
@@ -516,6 +523,9 @@ def oo_parse_named_certificates(certificates, named_certs_dir, internal_hostname
         if not isinstance(internal_hostnames, list):
             raise errors.AnsibleFilterError("|failed expects internal_hostnames is list")
 
+        if not HAS_OPENSSL:
+            raise errors.AnsibleFilterError("|missing OpenSSL python bindings")
+
         for certificate in certificates:
             if 'names' in certificate.keys():
                 continue


Test patch:
Repository: openshift/openshift-ansible

Issue:
openshift_facts received a lower value for openshift.master.master_count
I'm testing a cluster environment in which I've scaled it's masters down (from 3 to 2 in native ha), I've updated the masterCount in master-config.yaml but not able to track down why the facts `openshift_facts received a lower value for openshift.master.master_count` error me out


Discussion:

Resolving patch:
diff --git a/roles/openshift_facts/library/openshift_facts.py b/roles/openshift_facts/library/openshift_facts.py
--- a/roles/openshift_facts/library/openshift_facts.py
+++ b/roles/openshift_facts/library/openshift_facts.py
@@ -1460,7 +1460,7 @@ def merge_facts(orig, new, additive_facts_to_overwrite, protected_facts_to_overw
             dict: the merged facts
     """
     additive_facts = ['named_certificates']
-    protected_facts = ['ha', 'master_count']
+    protected_facts = ['ha']
 
     # Facts we do not ever want to merge. These originate in inventory variables
     # and contain JSON dicts. We don't ever want to trigger a merge
@@ -1513,14 +1513,6 @@ def merge_facts(orig, new, additive_facts_to_overwrite, protected_facts_to_overw
             # it so we will determine if it is okay to change this
             # fact.
             elif key in protected_facts and key not in [x.split('.')[-1] for x in protected_facts_to_overwrite]:
-                # The master count (int) can only increase unless it
-                # has been passed as a protected fact to overwrite.
-                if key == 'master_count' and new[key] is not None and new[key] is not '':
-                    if int(value) <= int(new[key]):
-                        facts[key] = copy.deepcopy(new[key])
-                    else:
-                        # pylint: disable=line-too-long
-                        module.fail_json(msg='openshift_facts received a lower value for openshift.master.master_count')  # noqa: F405
                 # ha (bool) can not change unless it has been passed
                 # as a protected fact to overwrite.
                 if key == 'ha':


Test patch:
Repository: openshift/openshift-ansible

Issue:
Invalid boolean conversion for extraAuthorizeParameters include_granted_scopes
When trying to define an OpenIDConnect ID provider in Ansible hosts by setting include_granted_scopes. If the value is “true” with the double quotes then the definition is invalid. 

For instance extraAuthorizeParameters': {'include_granted_scopes': ‘\”true\"'} results in include_granted_scopes: false in the configuration.

The issue seems to be that the value of include_granted_scopes is evaluated as ansible_bool.

https://github.com/openshift/openshift-ansible/blob/b8e72ea2143834317043f19cbc3dddf8ad9047a5/filter_plugins/openshift_master.py#L388



Discussion:
@bagnaram what is the goal of using the double quotes? I would expect for `{'include_granted_scopes': 'true'}` or `{'include_granted_scopes': true}` to produce the correct output in the master config.

I think it is because the master-config expects the value as a string. include_granted_scopes: "true"
The boolean parser is likely evaluating the string constant as always true

@smunilla ptal. I suspect we either need to avoid treating this value as a boolean in the filter or we need to explicitly convert it to a quoted string after evaluating ansible_bool().

I've moved this to P2 based on it not being a regression.

Resolving patch:
diff --git a/roles/openshift_master_facts/filter_plugins/openshift_master.py b/roles/openshift_master_facts/filter_plugins/openshift_master.py
--- a/roles/openshift_master_facts/filter_plugins/openshift_master.py
+++ b/roles/openshift_master_facts/filter_plugins/openshift_master.py
@@ -383,7 +383,7 @@ def __init__(self, api_version, idp):
         if 'extraAuthorizeParameters' in self._idp:
             if 'include_granted_scopes' in self._idp['extraAuthorizeParameters']:
                 val = ansible_bool(self._idp['extraAuthorizeParameters'].pop('include_granted_scopes'))
-                self._idp['extraAuthorizeParameters']['include_granted_scopes'] = val
+                self._idp['extraAuthorizeParameters']['include_granted_scopes'] = '"true"' if val else '"false"'
 
     def validate(self):
         ''' validate this idp instance '''


Test patch: