Change:
This example shows how to run OpenShift Origin as a pod on an existing Kubernetes cluster. <del> This example demonstrates usage of a pod with a secret volume mount. </del> <ins> OpenShift Origin runs with a rich set of role based policy rules out of the box that requires authentication from users via certificates.  When run as a pod on an existing Kubernetes cluster, it proxies access to the underlying Kubernetes services to provide security. As a result, this example is a complex end-to-end configuration that shows how to configure certificates for a service that runs on Kubernetes, and requires a number of configuration files to be injected dynamically via a secret volume to the pod. </ins> ### Step 0: Prerequisites <del> This example assumes that you have a basic understanding of Kubernetes and that you have forked the repository and [turned up a Kubernetes cluster](https://github.com/GoogleCloudPlatform/kubernetes#contents): </del> <ins> This example assumes that you have an understanding of Kubernetes and that you have forked the repository. OpenShift Origin creates privileged containers when running Docker builds during the source-to-image process. </ins> <del> This example has been tested against the **gce** and **vagrant** based KUBERNETES_PROVIDER. </del> <ins> If you are using a Salt based KUBERNETES_PROVIDER (**gce**, **vagrant**, **aws**), you should enable the ability to create privileged containers via the API. </ins> ```shell $ cd kubernetes <ins> $ vi cluster/saltbase/pillar/privilege.sls # If true, allow privileged containers to be created by API allow_privileged: true ``` Now spin up a cluster using your preferred KUBERNETES_PROVIDER ```shell </ins> $ export KUBERNETES_PROVIDER=gce <del> $ hack/dev-build-and-up.sh </del> <ins> $ cluster/kube-up.sh </ins> ``` <del> ### Step 1: Generate resources </del> <ins> Next, let's setup some variables, and create a local folder that will hold generated configuration files. </ins> <del> The demonstration will require the following resources: </del> <ins> ```shell $ export OPENSHIFT_EXAMPLE=$(pwd)/examples/openshift-origin $ export OPENSHIFT_CONFIG=${OPENSHIFT_EXAMPLE}/config $ mkdir ${OPENSHIFT_CONFIG} ``` </ins> <del> 1. A Kubernetes Secret that contains information needed to securely communicate to your Kubernetes master as an administrator 2. A Kubernetes Pod that contains information for how to run OpenShift Origin that consumes this Secret securely 3. A Kubernetes Service that exposes OpenShift Origin API via an external load balancer 4. A Kubernetes Service that exposes OpenShift Origin UI via an external load balancer </del> <ins> ### Step 1: Export your Kubernetes configuration file for use by OpenShift pod </ins> <del> To generate these resources, we will run a script that introspects your configured KUBERNETES_PROVIDER: </del> <ins> OpenShift Origin uses a configuration file to know how to access your Kubernetes cluster with administrative authority. </ins> <del> ```shell $ examples/openshift-origin/resource-generator.sh </del> ``` <del> A Kubernetes Secret was generated that contains the following data: </del> <ins> $ cluster/kubectl.sh config view --output=yaml --flatten=true --minify=true > ${OPENSHIFT_CONFIG}/kubeconfig ``` The output from this command will contain a single file that has all the required information needed to connect to your Kubernetes cluster that you previously provisioned.   This file should be considered sensitive, so do not share this file with untrusted parties. We will later use this file to tell OpenShift how to bootstap its own configuration. </ins> <del> 1. kubeconfig: a valid kubeconfig file that is used by OpenShift Origin to communicate to the master 2. kube-ca: a certificate authority for the Kubernetes master 3. kube-auth-path: a Kubernetes authorization file 4. kube-cert: a Kubernetes certificate 5. kube-key: a Kubernetes key file </del> <ins> ### Step 2: Create an External Load Balancer to Route Traffic to OpenShift </ins> <del> As required by a Kubernetes secret, each piece of data is base64 encoded - with no line wraps. </del> <ins> An external load balancer is needed to route traffic to our OpenShift master service that will run as a pod on your Kubernetes cluster. </ins> <del> You can view the file by doing: </del> ```shell <del> $ cat examples/openshift-origin/secret.json </del> <ins> $ cluster/kubectl.sh create -f $OPENSHIFT_EXAMPLE/openshift-service.yaml </ins> ``` <del> Caution:  This file contains all of the required information to operate as a Kubernetes admin on your cluster, so only share this file with trusted parties. </del> <ins> ### Step 3: Generate configuration file for your OpenShift master pod </ins> <del> A Kubernetes Pod file was generated that can run OpenShift Origin on your cluster. </del> <ins> The OpenShift master requires a configuration file as input to know how to bootstrap the system. </ins> <del> The OpenShift Origin pod file has a volume mount that references the Kubernetes secret we created to know how to work with the underlying Kubernetes provider. </del> <ins> In order to build this configuration file, we need to know the public IP address of our external load balancer in order to build default certificates. </ins> <del> You can view the file by doing: </del> <ins> Grab the public IP address of the service we previously created. </ins> ```shell <del> $ cat examples/openshift-origin/pod.json </del> <ins> $ export PUBLIC_IP=$(cluster/kubectl.sh get services openshift --template="{{ index .spec.publicIPs 0 }}") $ echo $PUBLIC_IP </ins> ``` <del> Finally, a Kubernetes service was generated for the UI and the API and available via an external load balancer: </del> <ins> Ensure you have a valid PUBLIC_IP address before continuing in the example. </ins> <del> ``shell $ cat examples/openshift-origin </del> <ins> We now need to run a command on your host to generate a proper OpenShift configuration.  To do this, we will volume mount the configuration directory that holds your Kubernetes kubeconfig file from the prior step. </ins> <del> ### Step 2: Create the secret in Kubernetes </del> <ins> ```shell docker run --privileged -v ${OPENSHIFT_CONFIG}:/config openshift/origin start master --write-config=/config --kubeconfig=/config/kubeconfig --master=https://localhost:8443 --public-master=https://${PUBLIC_IP}:8443 ``` </ins> <del> To provision the secret on Kubernetes: </del> <ins> You should now see a number of certificates minted in your configuration directory, as well as a master-config.yaml file that tells the OpenShift master how to execute.  In the next step, we will bundle this into a Kubernetes Secret that our OpenShift master pod will consume. </ins> <del> ```shell $ cluster/kubectl.sh create -f examples/openshift-origin/secret.json </del> <ins> ### Step 4: Bundle the configuration into a Secret We now need to bundle the contents of our configuration into a secret for use by our OpenShift master pod. OpenShift includes an experimental command to make this easier. First, update the ownership for the files previously generated: ``` $ sudo -E chown ${USER} -R ${OPENSHIFT_CONFIG} </ins> ``` <del> You should see your secret resource was created by listing: </del> <ins> Then run the following command to collapse them into a Kubernetes secret. </ins> ```shell <del> $ cluster/kubectl.sh get secrets </del> <ins> docker run -i -t --privileged -e="OPENSHIFTCONFIG=/config/admin.kubeconfig" -v ${OPENSHIFT_CONFIG}:/config openshift/origin ex bundle-secret openshift-config -f /config &> ${OPENSHIFT_EXAMPLE}/secret.json </ins> ``` <del> ### Step 3: Provisioning OpenShift Origin To create the OpenShift Origin pod: </del> <ins> Now, lets create the secret in your Kubernetes cluster. </ins> ```shell <del> $ cluster/kubectl.sh create -f examples/openshift-origin/pod.json </del> <ins> $ cluster/kubectl.sh create -f ${OPENSHIFT_EXAMPLE}/secret.json </ins> ``` <del> ### Step 4: Provisioning OpenShift Origin Services </del> <ins> **NOTE: This secret is secret and should not be shared with untrusted parties.** ### Step 5: Deploy OpenShift Master We are now ready to deploy OpenShift. </ins> <del> To create the OpenShift Origin Services that expose the API and UI: </del> <ins> We will deploy a pod that runs the OpenShift master.  The OpenShift master will delegate to the underlying Kubernetes system to manage Kubernetes specific resources.  For the sake of simplicity, the OpenShift master will run with an embedded etcd to hold OpenShift specific content.  This demonstration will evolve in the future to show how to run etcd in a pod so that content is not destroyed if the OpenShift master fails. </ins> ```shell <del> $ cluster/kubectl.sh create -f examples/openshift-origin/ui-service.json $ cluster/kubectl.sh create -f examples/openshift-origin/api-service.json </del> <ins> $  cluster/kubectl.sh create -f ${OPENSHIFT_EXAMPLE}/openshift-controller.yaml </ins> ``` <del> ### Step 5: Open Firewall Ports If you are running on GCE, you need to open the following ports: </del> <ins> You should now get a pod provisioned whose name begins with openshift. </ins> ```shell <del> $ gcloud compute instances list </del> <ins> $ cluster/kubectl.sh get pods | grep openshift $ cluster/kubectl.sh log openshift-t7147 origin Running: cluster/../cluster/gce/../../cluster/../_output/dockerized/bin/linux/amd64/kubectl log openshift-t7t47 origin 2015-04-30T15:26:00.454146869Z I0430 15:26:00.454005       1 start_master.go:296] Starting an OpenShift master, reachable at 0.0.0.0:8443 (etcd: [https://10.0.27.2:4001]) 2015-04-30T15:26:00.454231211Z I0430 15:26:00.454223       1 start_master.go:297] OpenShift master public address is https://104.197.73.241:8443 ``` </ins> <del> FIND THE MINION NAME PREFIX </del> <ins> Depending upon your cloud provider, you may need to open up an external firewall rule for tcp:8443.  For GCE, you can run the following: </ins> <del> $ gcloud compute firewall-rules create openshift-origin-node-8444 --allow tcp:8444 --target-tags kubernetes-minion-prq8 $ gcloud compute firewall-rules create openshift-origin-node-8443 --allow tcp:8443 --target-tags kubernetes-minion-prq8 </del> <ins> ```shell gcloud compute --project "your-project" firewall-rules create "origin" --allow tcp:8443 --network "your-network" --source-ranges "0.0.0.0/0" </ins> ``` <del> ### Step 4: Try out OpenShift Origin TODO add more detail here: </del>  No newline at end of file <ins> Consult your cloud provider's documentation for more information. Open a browser and visit the OpenShift master public address reported in your log. You can use the CLI commands by running the following: ```shell $ docker run --privileged --entrypoint="/usr/bin/bash" -it -e="OPENSHIFTCONFIG=/config/admin.kubeconfig" -v ${OPENSHIFT_CONFIG}:/config openshift/origin $ osc config use-context public-default $ osc --help ``` </ins>

Discussion:
This PR was prematurely merged: Need to update the PR based on the actual steps that were needed to get what was demonstrated at the public Kubernetes meet-up last week.
Any chance you can get rid of the hostPort from the example?
- depending on the Kubernetes provider, yes. For Vagrant based provider, I think I still need the hostPort because I do not have routes. For the GCE provider, I should be able to get rid of it because I have routes. In both providers, there were a number of setup steps that did not make the merged PR. For those that are tracking this issue (which I know folks have pinged me on separately) needs to be configured to allow privelaged containers needs to be configured to allow pushing to an insecure registry (this is point in time) Ideally, I could use a ReplicationController in the example to load balance OpenShift, but until we can provide a pattern to run etcd as a pod, I am unable to do this and need to use the embedded etcd we end up running.
Note: I will resolve this issue in 2 phases. Vagrant (which was demonstrated) to document using GCE (which has some additional steps on firewall rules)
Turns out OpenShift Origin is too secure now ;-) This is blocked until OpenShift Origin allows you to mint certs on the client, and then supply the certs on startup as a secret to the pod. This feature is coming soon. /cc
I now have all the required pieces to get this to work. I hope to get a PR submitted to fix this issue by end of this week.