add http health checks.

Yes, I meant that we should return healthy == true on any HTTP response. However, I now take that back. Nevermind. I'm not sure it would work as intended given failure modes of popular external webservers (e.g., apache, nginx, php, nodejs), and it definitely wouldn't work if a proxy or load balancer or somesuch were in between Kubelet and the application.

Maybe we can make the failure condition configurable in the future.

It does mean that your example of using index.html (or even /healthz) as the path to probe is unlikely to work in many cases, such as webservers that return 503 when offline during data reloading, draining, and/or maintenance, since returning 503 is intended behavior in such cases.

Once we have a means to report it to the user, we should record the HTTP status code that triggered the failure. HTTP errors (and non-200 success status codes, such as 204) are likely to be the result of misconfiguration more than anything else.