Initial ipv6 / iptables work

More for discussion than for actual merging (for now).

I think IPv6 could solve the address allocation problem that seems to be holding k8s back on EC2.  EC2 "tolerates" IPv6 for internal networking by using protocol 41 encapsulation.

I'm currently working on getting cluster/kube-up.sh to work, but in the meantime feedback on this idea would be helpful!
I was specifically asking if it random is what mattered or if "don't reuse"
is what matters.  I agree with the latter, the former causes me a very
small but of angst wrt static addresses for cluster services like DNS.

On Mon, Nov 17, 2014 at 9:42 AM, Justin Santa Barbara <
notifications@github.com> wrote:

> In pkg/registry/service/ip_allocator.go:
>
> > -			nextBit, err := ffs(freeMask)
> > -			if err != nil {
> > -				// If this happens, something really weird is going on.
> > -				glog.Errorf("ffs(%#x) had an unexpected error: %s", freeMask, err)
> > -				return nil, err
> > -			}
> > -			ipa.used[i] |= 1 << nextBit
> > -			offset := (i * 8) + int(nextBit)
> > -			ip := ipAdd(ipa.subnet.IP, offset)
> > +	if int64(ipa.used.Size()) == ipa.ipSpaceSize {
> > +		return nil, fmt.Errorf("can't find a free IP in %s", ipa.subnet)
> > +	}
> > +
> > +	// Try randomly first
> > +	for i := 0; i < ipa.randomAttempts; i++ {
> > +		ip := ipa.createRandomIp()
>
> Keeping a full bitmap is out of the question for e.g. a /64, which is what
> motivated the change.
>
> This should be more efficient than a linear scan, if we expect the address
> space to be sparsely populated. But there are also correctness aspects:
>
> I've had problems with IP address reuse in the past:
>
>    - where ARP or the ipv6 equivalents got confused (surmountable with
>    unsolicited ARP and equivalents)
>    - where the kernel cgroups or bridge got confused (particularly with
>    IPv6; the symptom was that attempting to assign the IPv6 address to the LXC
>    instance would just fail, but after a few instance restarts / time-delay it
>    would eventually work. I don't know if this still happens, or whether I was
>    just doing something wrong.)
>
> Also, it seems a little risky to assign an IP address immediately to the
> next requester, in case that is a different tenant. Having an LRU queue
> would probably be better.
>
> Of course, these are real problems, and randomizing just buries them in
> the long-tails. We can change randomAttempts to 0 or just remove the
> randomizing code, to see if any of these problems still occur.
>
> --
> Reply to this email directly or view it on GitHub
> <https://github.com/GoogleCloudPlatform/kubernetes/pull/2147/files#r20450818>
> .
>