Add initial resources.md for discussion

The first draft of a Kubernetes resource model.
Thanks for all the comments.  Summarizing:

> In the name of precision, 1123 defines "host name", but 1035 defines
> "label" and "subdomain" with a pseudo-BNF. We use the 1035 nomenclature
> everywhere else.
> Second, you don't want a host name (aka label) here (which is a single
> token: myhost). You want a subdomain (myhost.example.com).

Done.

Can we not mix formats for a given quantity type? E.g. If CPU is int
> millicores in one place it is never float cores elsewhere.

1. I don't see how to do that for the usage data, which is inherently
statistical in nature.
2. Also: CPU cores are always in units of cores in the request
specification files - it's only the internal representations that will use
millicores.

Can we clarify where these representations apply? I.e. are these just
> shorthand for use in things like UIs and CLIs or do we expect all of the
> various APIs to accept both forms?

For example, in JSON we can say 'memory: 1234000' or 'memory: "1234K"', but
> only if we explicitly go out of the way to support it as such. Something
> like protobuf is less flexible. What I don't want is for numeric types to
> be required to be specified as strings. 'cpu: "300"' would be terrible,
> since we use native ints everywhere else.


Done.  (This is what "external" meant, but I expanded its definition to
make it clearer.)  Internal APIs should use internal units: it's only
human-user-visible formats that should behave this way.  (Developers don't
count!)

Next, I am worried that the ability to specify both SI base-10 and base-2
> will be more confusing than helpful. Many systems allow you to say "1024M"
> for memory but I have NEVER EVER seen that mean "1024 * 10^6" - memory is
> just never expressed as base-10. But disk space is sometimes. And network
> is always.

We do *not* want to embed semantics of resource types into quantities.

Last pedantry: Does case matter, or do we allow "k" and "K" ?

It matters: M and m mean different things.  Clarified.

shouldn't all the CPU numbers here be ints?

Huh?  Why?  Fixed point numbers are explicitly permitted.  or do you want
the smallest granularity of CPU allocation to be whole cores?  That seems
wrong.

and all the memory numbers be Mi (see - it even confused you, and you wrote
> it!)

Actually, I really didn't care - I just needed examples.  But I've changed
them to Mi, to make the point.

Again, we should be clear what this representation is. It's not JSON
> syntax, so it's not clearly the primary REST API. But it does describe a
> structure, albeit loosely. Somewhere we should be more precise.

It was meant to be valid YAML.  Please help me fix it if that's wrong.

For the sake of clarity, given that I was arguing for the other
> representation, I presume that the case of named resources (assuming we
> wanted to do that at all) would look something like the following?


I proposed a slightly different model at the very end of the doc.  PTAL.

[on request and limit defaults]

> There's at least two places where defaults can apply. There's the API into
> the apiserver and the API into the kubelet. Do we expect the same defaults
> at both edges, or can, for example, the kubelet be more restrictive? This
> is how we specced internally - the master can decide the defaulting
> behavior and the 'let expected an absolute answer.
> E.g. if the master said "default is unbounded" it would pass 2^64 to the
> 'let.


I think that's perfectly fine.  I was more focused on what the external
view would be.  I've add a note with yuor example.

Do we expect to see a single structure with all of these dimensions
> (request and capacity at the same time) or are we actually just defining
> what the dimensions could be and their meanings?

The latter.  (I've added a note.)

"a scheduler will be able to treat a pod that has a request as having an
> (implicit) capacity for placing containers inside it"
> Can we just strike this? It suggests that users should not be able to
> over-commit inside a pod, and I absolutely think they should. I'd rather
> see this say something like:
> Initially, resource specs for containers and nodes are supported. We will
> extend this to include pods, too.

Schedulers need pod resource specs to do placement, so I now think that the
"initially" is actually pointless, so I took this bit out.


> Initially the resources at a given scope bound the resources of the sum of
> inner scopes. This may eventually be loosened.

Nice. Added.

do we want resources.kubermetes.io/cpu or kubernetes.io/resources/cpu - I
> am wary of consuming subdomains here, and the latter seems easier to
> actually implement as a URL handler that returns docs or something.

The former, I think: because of the way that typename namespaces are
defined.

what about swap and zswap? This *sort of* makes memory compressible.. How
> do you want to express that?

By adding "at least initially" :-).

A note on unbounded. Things like memory are very tricky here. If you do not
> specify memory bounds, then the kernel does not always try to reclaim from
> you, which means you end up in this weird bi-modal behavior.
> When you apply a limit the kernel actively tries to keep you alive by
> reclaiming. When you take off the limit, the kernel will happily use as
> much memory as it can for caches. If there is a sudden spike on the
> machine, the kernel tries to reclaim, but only to a limit. Those jobs
> without limits are the ones worst hit.
>
I'll happily put any words in here that you like  ...

We need to continue beating on the kernel to behave better.

:-)


> [resource metadata]  Do you expect this to be a REST GET'able object?
> PUT'able?

Definitely GETable.  Not sure about PUTable, but maybe so for user-defined
resource types.


On Sat, Aug 23, 2014 at 2:24 PM, Tim Hockin <notifications@github.com>
wrote:

> In docs/resources.md:
>
> > +To do good pod placement, Kubernetes needs to know how big pods are, as well as the sizes of the nodes onto which they are being placed.  The definition of "how big" is given by the Kubernetes resource model - the subject of this document.
> > +
> > +The resource model aims to be:
> > +* simple, for common cases;
> > +* extensible, to accommodate future growth;
> > +* regular, with few special cases; and
> > +* precise, to avoid misunderstandings and promote pod portability.
> > +
> > +## The resource model
> > +A Kubernetes _resource_ is something that can be requested by, allocated to, or consumed by a pod or container.  Examples include memory (RAM), CPU, disk-time, and network bandwidth.
> > +
> > +Once resources on a node have been allocated to one pod, they should not be allocated to another until that pod is removed or exits. This means that Kubernetes schedulers should ensure that the sum of the resources allocated (requested and granted) to its pods never exceeds the usable capacity of the node. Testing whether a pod will fit on a node is called _feasibility checking_. Note that the resource model currently prohibits over-committing resources; we will want to relax that restriction later.
> > +### Resource types
> > +All resources have a _type_ that is identified by their _typename_ (a string, e.g., "memory").  Several resource types are predefined by Kubernetes (a full list is below), although only two will be supported at first: CPU and memory.  Users and system administrators can define their own resource types if they wish (e.g., Hadoop slots).
> > +
> > +A fully-qualified resource typename has two parts: a DNS-style _namespace_ (that conforms to [RFC 1123 'host name' syntax](http://tools.ietf.org/html/rfc1123#section-2.1)) and a _path_ (e.g., `resources.kubernetes.io/memory` <http://resources.kubernetes.io/memory>, `myveryown.org/shinyNewResource/v2` <http://myveryown.org/shinyNewResource/v2>, in which the namespace is `myveryown.org` and the path is `shinyNewResource/v2`).
>
> In the name of precision, 1123 defines "host name", but 1035 defines
> "label" and "subdomain" with a pseudo-BNF. We use the 1035 nomenclature
> everywhere else.
>
> Second, you don't want a host name (aka label) here (which is a single
> token: myhost). You want a subdomain (myhost.example.com).
>
> --
> Reply to this email directly or view it on GitHub
> <https://github.com/GoogleCloudPlatform/kubernetes/pull/617/files#r16632782>
> .
>