Repository: openshift/openshift-ansible

Issue #9988: 3.10 - openid_auth_openid_ca.crt is added to master-config.yaml when it doesn't exist
Created: 2018-09-11T07:48:46Z
Description
When installing a new cluster, and using OpenID without variable openshift_master_request_header_ca or openshift_master_request_header_ca_file defined, the master-config.yaml is still updated with ca: /etc/origin/master/openid_auth_openid_ca.crt, causing the master API to fail to start, due to an invalid master-config.yaml, because openid_auth_openid_ca.crt does not exist.
Version
Please put the following version information in the code block
indicated below.


ansible 2.6.3.post0cd op


openshift-ansible-3.10.46-1


Steps To Reproduce


Deploy OpenShift cluster with a defined openshift_master_identity_providers, without using a CA, e.g.
openshift_master_identity_providers=[{"name": "openid_auth", "login": "true", "challenge": "false", "kind": "OpenIDIdentityProvider", "client_id": "openshift-uat.lab", "client_secret": "000000-000000-000000-000000", "claims": {"id": ["sub"], "preferredUsername": ["preferred_username"], "name": ["name"], "email": ["email"]}, "urls": {"authorize": "https://sso-uat.lab/auth/realms/lab/protocol/openid-connect/auth", "token": "https://sso-uat.lab/auth/realms/lab/protocol/openid-connect/token"}}]


Run prerequisites


Run deploy_cluster


Expected Results
Cluster deployed successfully. This has previously deployed successfully, however after a recent rebuild and redeployment using a newer openshift-ansible version, the same inventory no longer functions due to this issue.
ansible-playbook openshift-ansible/playbooks/redeploy-certificates.yml --inventory-file=lab-openshift/lab-openshift-ansible-prod/hosts.ini

Observed Results
Describe what is actually happening.
Sep 11 02:35:09 lab-ose-master-uat-1.ose-uat.lab dnsmasq[6953]: setting upstream servers from DBus
Sep 11 02:35:09 lab-ose-master-uat-1.ose-uat.lab dnsmasq[6953]: using nameserver 217.19.60.80#53
Sep 11 02:35:09 lab-ose-master-uat-1.ose-uat.lab dnsmasq[6953]: using nameserver 217.19.48.80#53
Sep 11 02:35:09 lab-ose-master-uat-1.ose-uat.lab dnsmasq[6953]: using nameserver 127.0.0.1#53 for domain in-addr.arpa
Sep 11 02:35:09 lab-ose-master-uat-1.ose-uat.lab dnsmasq[6953]: using nameserver 127.0.0.1#53 for domain cluster.local
Sep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: I0911 02:35:10.654302    8219 kuberuntime_manager.go:513] Container {Name:api Image:docker.io/openshift/origin-control-plane:v3.10.0 Command:[/bin/bash -c] ArgSep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: set -euo pipefail
Sep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: if [[ -f /etc/origin/master/master.env ]]; then
Sep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: set -o allexport
Sep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: source /etc/origin/master/master.env
Sep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: fi
Sep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: exec openshift start master api --config=/etc/origin/master/master-config.yaml --loglevel=${DEBUG_LOGLEVEL:-2}
Sep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: ] WorkingDir: Ports:[] EnvFrom:[] Env:[] Resources:{Limits:map[] Requests:map[]} VolumeMounts:[{Name:master-config ReadOnly:false MountPath:/etc/origin/master/Sep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: I0911 02:35:10.654497    8219 kuberuntime_manager.go:757] checking backoff for container "api" in pod "master-api-lab-ose-master-uat-1.ose-uat.lab_kube-systSep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: I0911 02:35:10.654665    8219 kuberuntime_manager.go:767] Back-off 5m0s restarting failed container=api pod=master-api-lab-ose-master-uat-1.ose-uat.lab_kubeSep 11 02:35:10 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: E0911 02:35:10.654698    8219 pod_workers.go:186] Error syncing pod 9ca23c5815da8ed1d3dca61d87e1f6ab ("master-api-lab-ose-master-uat-1.ose-uat.lab_kube-systSep 11 02:35:13 lab-ose-master-uat-1.ose-uat.lab origin-node[8219]: I0911 02:35:13.655619    8219 kuberuntime_manager.go:513] Container {Name:controllers Image:docker.io/openshift/origin-control-plane:v3.10.0 Command:[/bin/bashS

Additional Information

CentOS 7.3

lab-ose-master-uat-1.ose-uat.lab ansible_port=22 ansibe_host=192.19.50.25
lab-ose-master-uat-2.ose-uat.lab ansible_port=22 ansibe_host=192.19.50.26
lab-ose-master-uat-3.ose-uat.lab ansible_port=22 ansibe_host=192.19.50.27
lab-ose-node-uat-1.ose-uat.lab ansible_port=22 ansibe_host=192.19.50.28
lab-ose-node-uat-2.ose-uat.lab ansible_port=22 ansibe_host=192.19.50.29
lab-ose-node-uat-3.ose-uat.lab ansible_port=22 ansibe_host=192.19.50.30

[nodes]
lab-ose-node-uat-1.ose-uat.lab openshift_node_labels="{'region': 'primary', 'zone': 'default'}" openshift_node_group_name="node-config-compute" openshift_hostname="lab-ose-node-uat-1.ose-uat.lab" openshift_public_hostname="lab-ose-node-uat-1.ose-uat.lab" openshift_public_ip=192.19.50.28 openshift_ip=192.19.50.28
lab-ose-node-uat-2.ose-uat.lab openshift_node_labels="{'region': 'primary', 'zone': 'default'}" openshift_node_group_name="node-config-compute" openshift_hostname="lab-ose-node-uat-2.ose-uat.lab" openshift_public_hostname="lab-ose-node-uat-2.ose-uat.lab" openshift_public_ip=192.19.50.29 openshift_ip=192.19.50.29
lab-ose-node-uat-3.ose-uat.lab openshift_node_labels="{'region': 'primary', 'zone': 'default'}" openshift_node_group_name="node-config-compute" openshift_hostname="lab-ose-node-uat-3.ose-uat.lab" openshift_public_hostname="lab-ose-node-uat-3.ose-uat.lab" openshift_public_ip=192.19.50.30 openshift_ip=192.19.50.30
lab-ose-master-uat-1.ose-uat.lab openshift_node_labels="{'region': 'infra', 'zone': 'default'}" openshift_node_group_name="node-config-master-infra" openshift_hostname="lab-ose-master-uat-1.ose-uat.lab" openshift_public_hostname="lab-ose-master-uat-1.ose-uat.lab" openshift_public_ip=192.19.50.25 openshift_ip=192.19.50.25 openshift_schedulable=true 
lab-ose-master-uat-2.ose-uat.lab openshift_node_labels="{'region': 'infra', 'zone': 'default'}" openshift_node_group_name="node-config-master-infra" openshift_hostname="lab-ose-master-uat-2.ose-uat.lab" openshift_public_hostname="lab-ose-master-uat-2.ose-uat.lab" openshift_public_ip=192.19.50.26 openshift_ip=192.19.50.26 openshift_schedulable=true 
lab-ose-master-uat-3.ose-uat.lab openshift_node_labels="{'region': 'infra', 'zone': 'default'}" openshift_node_group_name="node-config-master-infra" openshift_hostname="lab-ose-master-uat-3.ose-uat.lab" openshift_public_hostname="lab-ose-master-uat-3.ose-uat.lab" openshift_public_ip=192.19.50.27 openshift_ip=192.19.50.27 openshift_schedulable=true 

[masters]
lab-ose-master-uat-[1:3].ose-uat.lab

[etcd]
lab-ose-master-uat-[1:3].ose-uat.lab 

[OSEv3:children]
masters
nodes
etcd

[OSEv3:vars]
#Deployment and version
openshift_deployment_type="origin"
openshift_release="3.10"

# This enables all the system containers except for docker:
openshift_use_system_containers=true
# In either case, system_images_registry must be specified to be able to find the system images
system_images_registry="docker.io"

#Enable Firewalld automatic configuration
os_firewall_use_firewalld=true
os_firewall_enabled=true

#Disable logging and delete any logging reminants
openshift_logging_install_logging=false
#Ensure there are no log reminants
openshift_logging_purge_logging=true
#turn off a second ES/kibana for infra logging. defaults to false
openshift_logging_use_ops=false

#Enable Ansible and Template service brokers
ansible_service_broker_install=true
template_service_broker_install=true

# Configure UAT Keycloak OAuth authentication
openshift_master_identity_providers=[{"name": "openid_auth", "login": "true", "challenge": "false", "kind": "OpenIDIdentityProvider", "client_id": "openshift-uat.lab", "client_secret": "000000-000000-000000-0000", "claims": {"id": ["sub"], "preferredUsername": ["preferred_username"], "name": ["name"], "email": ["email"]}, "urls": {"authorize": "https://sso-uat.lab/auth/realms/lab/protocol/openid-connect/auth", "token": "https://sso-uat.lab/auth/realms/lab/protocol/openid-connect/token"}}]

#Enable HTTPS via wildcard certificate for portal Public URL and Internal URL
openshift_master_overwrite_named_certificates=false
openshift_master_named_certificates=[{"certfile": "/tmp/lab.crt", "keyfile": "/tmp/lab.key", "names": ["openshift-uat.lab"], "cafile": "/tmp/ca-lab.crt"}]
openshift_master_named_certificates=[{"certfile": "/tmp/lab.crt", "keyfile": "/tmp/lab.key", "names": ["lab-ose-master-uat-cluster.lab"], "cafile": "/tmp/ca-lab.crt"}]

#Configure SSH user
ansible_user=batch
ansible_become=true

# default subdomain to use for exposed routes, you should have wildcard dns
openshift_master_default_subdomain=apps.ose-uat.lab
#Set openshift_master_cluster_hostname to point at your load balancer
openshift_master_cluster_hostname=lab-ose-master-uat-cluster.lab
#Set PUBLIC openshift_master_cluster_public_hostname to point at your load balancer AND DIFFERENT TO ABOVE
openshift_master_cluster_public_hostname=openshift-uat.lab

# Configure master API and console ports.
openshift_master_api_port=8443
openshift_master_console_port=8443

Duplicate issue #10216: ca option injected into oidc configuration in master-config.yaml even when no ca is specified
Created: 2018-09-25T02:30:26Z
Description
When specifying an oidc idp in your inventory with no certificate file attribute the playbooks are injecting the ca attribute with the value of idpname_openid_ca.crt where idpname is the name of the identity provider.  So as an example the following inventory variable: openshift_master_identity_providers=[{"name": "unison","challenge": false,"login": true,"mappingMethod": "claim","kind": "OpenIDIdentityProvider","clientID": "openshift","clientSecret": "","claims": {"id": ["sub"],"preferredUsername": ["preferred_username"],"name": ["name"],"email": ["email"]},"urls": {"authorize": "https://idm.customer.com/auth/idp/OpenShiftIdP/auth","token":"https://idm.customer.com/auth/idp/OpenShiftIdP/token"}}] will generate a ca attribute with the value unison_openid_ca.crt.  If your idp is signed by a commercial certificate or the certificate is already trusted by the servers because the CA has been imported into the trust store this causes an extra configuration that is not needed.   If the ca directive points to a file that doesn't exist the api server fails to start.
It appears this pull request implemented this issue - 845e90a#diff-37bf817a5aae06cfc3e26ff12c2c1d6c
Version
Please put the following version information in the code block
indicated below.

Your ansible version per ansible --version

2.4
If you're operating from a git clone:

The output of git describe

release-3.10

Steps To Reproduce

add the above openshift_master_identity_providers to your inventory
deploy OKD 3.10

Expected Results
Describe what you expected to happen.
The ca should not be set if none is specified.  Based on the PR that introduced this issue, there should be a flag of some kind to disable this behavior.

Closing comment (2018-09-26T08:44:57Z): Duplicate of #9988

If your idp is signed by a commercial certificate or the certificate is already trusted by the servers because the CA has been imported into the trust store this causes an extra configuration that is not needed

This CA is used by API server, which is running in a static pod (a separate container). So importing CA into the host's trust store won't be sufficient. For clarity the trust store should be specified explicitly.
The installer assumes this file is already present on the host, if its not specified.