Issue 1:
Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.):

No, this is a bug report

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.):

Searched for: "error validating data: couldn't find type: v1beta1.StorageClass"

Is this a BUG REPORT or FEATURE REQUEST? (choose one):

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.0+2831379", GitCommit:"283137936a498aed572ee22af6774b6fb6e9fd94", GitTreeState:"not a git tree", BuildDate:"2016-07-05T15:40:13Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.5+coreos.0", GitCommit:"d7a04b1c6044647f5919fadf3cecb9ee70c10fc5", GitTreeState:"clean", BuildDate:"2016-08-15T21:01:42Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}

Environment:

Cloud provider or hardware configuration: AWS

OS (e.g. from /etc/os-release): Mac OSX 10.11.5

Kernel (e.g. uname -a): Darwin Adams-MacBook-Pro.local 15.5.0 Darwin Kernel Version 15.5.0: Tue Apr 19 18:36:36 PDT 2016; root:xnu-3248.50.21~8/RELEASE_X86_64 x86_64

Install tools:

Others:

What happened: I want to create a StorageClass. I ran the following code:

kubectl create -f - <<EOF
kind: StorageClass
apiVersion: extensions/v1beta1
metadata:
name: slow
provisioner: kubernetes.io/aws-ebs
parameters:
type: io1
zone: us-east-1d
iopsPerGB: "10"
EOF

And got the following error:

error validating "STDIN": error validating data: couldn't find type: v1beta1.StorageClass; if you choose to ignore these errors, turn validation off with --validate=false

What you expected to happen:

A StorageClass is created succesfully

How to reproduce it (as minimally and precisely as possible):

Run this:

kubectl create -f - <<EOF
kind: StorageClass
apiVersion: extensions/v1beta1
metadata:
name: slow
provisioner: kubernetes.io/aws-ebs
parameters:
type: io1
zone: us-east-1d
iopsPerGB: "10"
EOF

Anything else do we need to know:

Issue 2:
Note: I'm happy to write a more formal proposal out, I just wanted to quickly float the idea and see if it makes any sense to have.

Introduction

A hostPath volume source is probably the simplest one to define, needing only a single path. However, that simplicity comes with many assumptions and caveats.

Right now, the largest caveat with a hostPath volume under Docker is that the given target will be created as an empty directory owned by root if it does not already exist. This is rarely what the user actually wants because hostPath volumes are typically used to express a dependency on an existing external file or directory.

To copy from a different comment: hostPath volumes have the following uses:

Hostpath use-cases

Communicating with an external system that produced data, is producing data, or is listening on a unix socket, such as a user who produced a configuration directory for the application, or X and the X11 socket.

Communicating with external systems that will consume data created by the application; e.g. a container that produces logs at /var/log/fooApp and wishes to bindmount it to the host at /var/log/fooApp as well because the operator knows an external logging process will scan through that directory and collect them.

Communicating between instances of yourself: e.g. a container that bindmounts in /var/cache/appName/thumbnails for each run and populates it with any thumbnails not present, but prefers to re-use existing ones between invocations simply to avoid unneeded processing.

Proposed API change

I propose that the v1.HostPathVolumeSource be changed to include the following field:

type - optional string of auto|exists|file|device|socket|directory - If not set, defaults to auto, whereby any existing path will be accepted, and non-existent paths be created as new directories If set to a given value, something of the given type must be present at that path. If there is nothing present, or it is not of the specified type, the pod will fail to run.

Note: For the default case, the presence or absence of the given path is resolved at container-runtime for any container that references the given path. For the other cases, it is resolved when the pod is placed prior to any containers being run.

Reason for change

In the first use-case above, creating the nonexistent thing is not a desirable action and can end up breaking things (aside, docker once had a bug where it created the docker.sock after starting old containers, so any restart=always containers bindmounting the docker.sock would create a directory there and crashloop the daemon).

In the second and third use-cases, you do want the directory created. I think the first use-case is, however, the most common one.

Because Kubernetes runs across many machines and should value correctness, I propose that the hostPath volume should allow a user to specify additional expectations about what already exists, or does not, at that path such that it can fail quickly if those assumptions are not met, rather than mutate the host filesystem.

This ensures that a host that is in an odd state (e.g. with some socket not existing that should) does not end up in a worse state, and also allows a pod to more clearly express the dependency it has such that an operator who accidentally runs it on a system it is not suited for can see quickly and clearly what went wrong.

Other considerations

the subPath feature is also relevant because it is a way in which one can reference directories which might-or-might not exist within a volume as well. One possible way to resolve both hostPath and subPath at the same time would be making my above proposed api-changes attributes of the reference within the container, not the volume source outside of it. I think that having the attribute of the volume on the host attached to the source is neater and makes more logical sense however, so I'd prefer keeping it there. Other solutions are to simply assume subPaths should always be created, or to duplicate these options for subPath as well. Opinions welcome!

The examples folder contains many examples of pods where this feature would be useful. For example:

sysdig-cloud example -- If /boot does not exist or /var/run/docker.sock is at an alternate location, this should fail to run and the operator should edit the pod, not clobber the host

storage/vitess example -- If /etc/ssl/certs is not present (e.g. on /etc/pki/ distros) it should not be created, it should be an error

newrelic example -- Same as sysdig, a docker.sock reference

... many others in the wild I'm sure

Opinions on whether this increase in expressiveness is worth the increased API surface area. I think it is, but second opinions would be much appreciated.

cc @kubernetes/sig-storage @thockin @jonboulle