Issue 1:
Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.): No

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): Everything 'ELB' for past few months

Is this a BUG REPORT or FEATURE REQUEST? (choose one): Bug report

Kubernetes version (use kubectl version): 1.4.6

Environment:

Cloud provider or hardware configuration: AWS

OS (e.g. from /etc/os-release): Jessie

Kernel (e.g. uname -a): 4.4.26-k8s

Install tools: kops 1.4.1

Others:

What happened:

My goal was to expose a k8s Service using an AWS ELB, using both HTTP and HTTPS. HTTPS should use an AWS cert, passing through both HTTP and HTTPS to an nginx container.

Scenario A:

With these three options set:

service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:xxxxxx

service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https

service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https

The listeners created were:

TCP/80 -> TCP/(nodeport for container's 80)

HTTPS/443 -> HTTPS/(nodeport for container's 443), using AWS cert

While this does move traffic on both 80 and 443, the source IP is lost on port 80. 443 works properly.

Scenario B:

Backend changed to http, otherwise the same:

service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http

The listeners created this time were:

HTTP/80 -> HTTP/(nodeport for container's 80)

HTTPS/443 -> HTTP/(nodeport for container's 443), using AWS cert

In this instance, 80 works properly, but 443 is completely broken as it's attempting to deliver HTTP traffic to the container over port 443.

What you expected to happen:

In scenario A (backend-protocol: https), I expected:

HTTP/80 -> HTTP/(nodeport for container's 80) [or alternatively, -> HTTPS/(nodeport for 443)]

HTTPS/443 -> HTTPS/(nodeport for container's 443)

In short, I expected HTTP on 80, not TCP, which would enable proper L7 on both ports.

In scenario B (backend-protocol: http), I expected:

HTTP/80 -> HTTP/(nodeport for container's 80)

HTTPS/443 -> HTTP/(nodeport for container's 80)

Here I expected the backend HTTP to be directed to the NodePort that forwards to container's port 80 in both cases.

In my mind, in the presence of aws-load-balancer-ssl-cert, k8s already assumes traffic should now be L7. I expect L7 (and the benefits of the X-Forwarded-* set of headers) to happen on both 80 and 443, not just 443.

Further, I expect the Instance Protocol and the NodePort assigned to Instance Port to match, and not get HTTP traffic directed to a NodePort for HTTPS, or vice versa.

How to reproduce it (as minimally and precisely as possible):

I installed Kubernetes via Kops 1.4.1, targeting K8s 1.4.6. Then I built a simple nginx container (that exposes 80 + 443) and used the following Service definition:

apiVersion: v1
kind: Service
metadata:
name: web
labels:
app: web
annotations:
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:xxxxxx
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https
# or http
spec:
type: LoadBalancer
ports:
- name: http
port: 80
protocol: TCP
- name: https
port: 443
protocol: TCP
selector:
app: web

Anything else do we need to know:

Ideally, I'd like backend-protocol of both http and https to be working, as we do run some sensitive services that require https. If only https worked, we'd accept the internal slowdown on less critical services.

Issue 2:
Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.): Yes

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): elasticsearch

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Kubernetes version (use kubectl version): 1.4.6_coreos.0

Environment:

Cloud provider or hardware configuration: on premise, vCloud

OS (e.g. from /etc/os-release): CoreOS alpha (1221.0.0)

Kernel (e.g. uname -a): Linux f1 4.8.6-coreos #1 SMP Thu Nov 3 04:43:35 UTC 2016 x86_64 Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz GenuineIntel GNU/Linux

Install tools: None. Default installation

Others:

What happened:

When I tried to install es-controller.yaml, The containers are created successfully and but terminated with the following error:

Exception in thread "main" java.lang.IllegalStateException: Unable to access 'path.data' (/data/kubernetes-logging)
Likely root cause: java.nio.file.AccessDeniedException: /data/kubernetes-logging
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384)
at java.nio.file.Files.createDirectory(Files.java:674)
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781)
at java.nio.file.Files.createDirectories(Files.java:767)
at org.elasticsearch.bootstrap.Security.ensureDirectoryExists(Security.java:337)
at org.elasticsearch.bootstrap.Security.addPath(Security.java:314)
at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:259)
at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:212)
at org.elasticsearch.bootstrap.Security.configure(Security.java:118)
at org.elasticsearch.bootstrap.Bootstrap.setupSecurity(Bootstrap.java:212)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:183)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Refer to the log for complete error details.

What you expected to happen:

The container runs successfully.

How to reproduce it (as minimally and precisely as possible):

in a running kube cluster, run

kubectl create -f cluster/addons/fluentd-elasticsearch/es-controller.yaml

The pod will restart itself because of the error. The error message is in the container log.

Anything else do we need to know:

It seems that when container mount the volume /data, it mounted as root:root (755). However, elasticsearch is running under user elasticsearch. So it couldn't write the files under /data.

After I run chown elasticsearch:elasticsearch /data, the container runs fine.

/cc @crassirostris