Make kube-down more robust for GCE provider

This PR makes `kube-down.sh` more robust for the GCE provider. Before, the deletion of resources did not always succeed and running kube-down a second time would not always clean up remaining resources. This PR changes kube-down to find out what the running resources (instances, firewall rules, routes) are and then attempts to delete them. Occasionally, something may go wrong:
```
Deleting firewall rules kubernetes-minion-1-all kubernetes-minion-2-all kubernetes-minion-3-all kubernetes-minion-4-all
ERROR: (gcloud.compute.firewall-rules.delete) Some requests did not succeed:
 - Backend Error
```
in which case `kube-down.sh` can be run again.

For taking down very large clusters this is very useful to avoid a lot of manual deletion (or writing custom scripts).

Another change I made was to give the gcloud commands a list of instances etc. are arguments rather than use a loop and run a separate gcloud command per resource. These commands are run in the foreground so the final wait has been removed. We may need to revisit this decision for large clusters.

@jbeda @cjcullen 
One worry I have about the composite expression is that if the gcloud
command fails for any reason (e.g. wrong project, zone etc.) then this
tear-down script will go into a futile infinite loops (of course, being a
lazy functional programmer, I realize that not all infinite loops are
futile).


On Wed, Dec 10, 2014 at 3:34 PM, Satnam Singh <satnam@google.com> wrote:

> Yes, I know, and I did temporarily have a version which did that but
> reverted for some reason. Will revisit.
>
> On Wed, Dec 10, 2014 at 3:28 PM, Joe Beda <notifications@github.com>
> wrote:
>
>> In cluster/gce/util.sh
>> <https://github.com/GoogleCloudPlatform/kubernetes/pull/2803#discussion-diff-21647571>
>> :
>>
>> >      --project "${PROJECT}" \
>> >      --quiet \
>> > -    "${MASTER_NAME}-https" &
>> > -
>> > -  local minion
>> > -  for minion in "${MINION_NAMES[@]}"; do
>> > -    gcloud compute firewall-rules delete \
>> > +    --delete-disks all \
>> > +    --zone "${ZONE}" \
>> > +    "${MASTER_NAME}" || true
>> > +  # Find out what minions are running.
>> > +  local m=`gcloud compute instances list --project "${PROJECT}" --zone "${ZONE}" --regexp "${INSTANCE_PREFIX}"-minion-[0-9]+ | tail -n +2 | awk '{ print $1 }'`
>> > +  local -a minions=(`echo "${m}"`)
>>
>> 2 things here:
>>
>>    - I don't think you need to do this in 2 commands. You can do local
>>    -a "minions=($(gcloud compute instances list...)
>>    - Don't use backticks -- use $(...) instead.
>>
>> —
>> Reply to this email directly or view it on GitHub
>> <https://github.com/GoogleCloudPlatform/kubernetes/pull/2803/files#r21647571>
>> .
>>
>
>