Repository: kubernetes/kubernetes (language: go)

Issue:
no defined semantics for EndpointSlices of type "FQDN"
The API docs (and the end-user docs) state that "FQDN" is a valid value for EndpointSlice.AddressType, but nothing explains what the expected semantics of an FQDN slice are. And in fact, the expected semantics seem to be "[it gets ignored](https://github.com/kubernetes/kubernetes/blob/v1.26.0-rc.0/pkg/proxy/endpoints_test.go#L1590)".

We should document what FQDN slices are for, and if the answer is (and always will be) "nothing" then we should mark that value deprecated.

/assign @robscott 
/kind bug
/kind documentation
/sig network
/priority backlog


Discussion / hints:
> . And in fact, the expected semantics seem to be "[it gets ignored](https://github.com/kubernetes/kubernetes/blob/v1.26.0-rc.0/pkg/proxy/endpoints_test.go#L1590)".

that is kube-proxy but slices can be consumed by others.

However, I've tried to find the motivation or some use cases but I couldn't find them https://github.com/kubernetes/kubernetes/pull/84091
So this came out of an OpenShift bug, where a customer was expecting FQDN EndpointSlices to work basically like ExternalName services, except being able to specify multiple remote endpoints. But CoreDNS doesn't implement that, and I don't see any way that it _could_ implement that, because you can't have multiple CNAMEs attached to the same service DNS name. It seems like the only thing that would be able to handle FQDN slices in a useful way would be an L7 proxy?
xref https://github.com/kubernetes/kubernetes/issues/105172
> I don't see any way that it could implement that, because you can't have multiple CNAMEs attached to the same service DNS name. 

I think any implementation would have to involve "flattening" the endpoint FQDNS. e.g.

For a type A query of `example.default.svc.cluster.local.`

The following answer would be illegal in DNS, since there are two CNAMEs with the same name.
```
example.default.svc.cluster.local. CNAME fqdn1.example.com.
example.default.svc.cluster.local. CNAME fqdn2.example.com.
fqdn1.example.com. A  10.0.0.1
fqdn2.example.com. A 10.0.0.2
```
so instead the answer could be flattened to ...
```
example.default.svc.cluster.local. A  10.0.0.1
example.default.svc.cluster.local. A 10.0.0.2
```

This would be like treating the FQDN endpoints as ALIASes.  ALIAS afaik not being an official DNS spec concept, but implemented by various DNS servers more or less as an "invisible" CNAME.


> so instead the answer could be flattened to ...

Ah, true. You can't really resolve the FQDNs to IPs _ahead of time_ and have `example.default.svc.cluster.local` resolve to that list of IPs (this runs into all the same problems as attempting to do FQDN NetworkPolicies), but I suppose CoreDNS could implement it by looking up the FQDNs _at the point when someone tries to resolve `example.default.svc.cluster.local`_ and returning the aggregated list of results.
Could the cluster DNS service publish `SVCB` records rather than `CNAME`s?
ISTR we left this in as a nod towards being more general-purpose, but I can't honestlrecall why it was a good idea. 
Agree with @thockin here. This came from the idea that EndpointSlices could be used by different applications for entirely independent purposes, published and consumed by custom implementations. In retrospect I don't think we had sufficiently clear use cases for this, and I don't know of any implementations that make use of this feature today.
What if two applications each decided to use `addressType: FQDN` but interpreted it in incompatible ways?

I don't think EndpointSlice is currently designed correctly to be extensible. (eg, `addressType` must be a registered type, so you're pretty limited in what sorts of "independent purposes" you could use EndpointSlices for)

> We should document what FQDN slices are for, and if the answer is (and always will be) "nothing" then we should mark that value deprecated.

We could also mark it as “reserved” (and even warn if it's used, until whatever release where we start using it in earnest).
Agree - we should mark it deprecated and return an API warning when it is used.  Seems like an easy one for a newish contrib to jump on!

/help-wanted
/triage accepted
Some guidance for the people that may not be familiar with the codebase, this means to add some function herer

https://github.com/kubernetes/kubernetes/blob/8c8a4cf3e4a18e97359ce750530a4fa27bbd3b88/pkg/registry/discovery/endpointslice/strategy.go#L91-L93

and here

https://github.com/kubernetes/kubernetes/blob/8c8a4cf3e4a18e97359ce750530a4fa27bbd3b88/pkg/registry/discovery/endpointslice/strategy.go#L111-L114

that parses the object and return the corresponding `[]string` with the warnings

One example here https://github.com/kubernetes/kubernetes/commit/1b804fc87c5586989392116a81bc0d07047dbb87
/triage accepted
/assign kl52752
@aojea If AddressType field is immutable the I don't think there is a need to generate the warning on the Update? WDYT?
> @aojea If AddressType field is immutable the I don't think there is a need to generate the warning on the Update? WDYT?

agree
/reopen
needs an update to `k8s.io/api` too
@danwinship: Reopened this issue.

<details>

In response to [this](https://github.com/kubernetes/kubernetes/issues/114210#issuecomment-1384122891):

>/reopen
>needs an update to `k8s.io/api` too


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
I think we should rollback this deprecation - or maybe scope the use of FQDN type to only having a single entry.

Knative was planning on making use of the FQDN type to work around restrictions with Service type=ExternalName

The K8s CVE-2021-25740 mentions https://github.com/kubernetes/kubernetes/issues/103675

> A similar attack is possible using Ingress implementations that support forwarding to ExternalName Services. This can be used to forward to Services in other namespaces or, in some cases, sensitive endpoints within the Ingress implementation. If you are using the Ingress API, we recommend confirming that the implementation you’re using either does not support forwarding to ExternalName Services or supports disabling the functionality.

This was possible with Contour and hence they issued a CVE - https://github.com/projectcontour/contour/security/advisories/GHSA-5ph6-qq5x-7jwc restricting the feature.

With ingresses dropping support for ExternalName we were planning on [migrating](https://github.com/knative/serving/issues/11821) to using EndpointSlices with FQDN. The reasoning being is users don't interact with EndpointSlices and operators can use RBAC rules to limit their creation/modification - but allow the Knative control plane to do so.

Also an observation I noticed is that ingress implementations sorta ignored FQDN because when it became stable in K8s 1.21 it didn't actually work/did anything. It was only until CoreDNS 1.8.4 added support (which kubeadm used for K8s 1.22)  would FQDN act like a CNAME record.






I was also hoping ingress implementations (via the Gateway API) support these types of Services variants - by expanding their conformance tests - https://github.com/kubernetes-sigs/gateway-api/issues/1718
> I was also hoping ingress implementations (via the Gateway API) support these types of Services variants - by expanding their conformance tests - [kubernetes-sigs/gateway-api#1718](https://github.com/kubernetes-sigs/gateway-api/issues/1718)

/cc @robscott 
>With ingresses dropping support for ExternalName we were planning on https://github.com/knative/serving/issues/11821 to using EndpointSlices with FQDN. The reasoning being is users don't interact with EndpointSlices and operators can use RBAC rules to limit their creation/modification - but allow the Knative control plane to do so.

This does feel marginally safer, but I'm still not sure how safe it would be. Can you explain some of the specific use case(s) for FQDN addresses?
Knative creates K8s services of type external name which point to a cluster local proxy (typically the proxies service dns name - istio-ingressgateway.istio-system.svc.cluster.local). This allows us to perform L7 type routing within a cluster without a mesh. 
> I think we should rollback this deprecation - or maybe scope the use of FQDN type to only having a single entry.

If there are use cases for new EndpointSlice AddressTypes, we could add them. (This would presumably require a KEP.) The problem is that there was no definition of what the existing FQDN type was supposed to mean.

But OTOH, do you actually need to be using `discoveryv1.EndpointSlice`? The slices you want aren't going to be automatically created by the EndpointSlice controller anyway... so why not just make a CRD type for the feature you want?

> If there are use cases for new EndpointSlice AddressTypes, we could add them. (This would presumably require a KEP.) The problem is that there was no definition of what the existing FQDN type was supposed to mean.

Why not backfill the use cases based on empirical usage - generally it seems like FQDN is being used to introduce CNAME records into Kubernetes's DNS services.

ie. using a kind cluster 1.22+
```yaml
apiVersion: discovery.k8s.io/v1
kind: EndpointSlice
metadata:
  name: helloworld
  labels:
    kubernetes.io/service-name: helloworld
addressType: FQDN
ports:
- name: http
  port: 80
  protocol: TCP
endpoints:
  - addresses:
      - "google.com"
    conditions:
      ready: true
---
apiVersion: v1
kind: Service
metadata:
  name: helloworld
spec:
  clusterIP: None
  ports:
    - name: http
      protocol: TCP
      port: 80
```
```sh
$ kubectl run -ti --restart=Never --rm --image registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 dnsutils --  nslookup helloworld
Server:		10.96.0.10
Address:	10.96.0.10#53

helloworld.default.svc.cluster.local	canonical name = google.com.
Name:	google.com
Address: 142.251.33.174

pod "dnsutils" deleted
```

Here's an example of a downstream usage in practice - https://github.com/vmware-tanzu/cross-cluster-connectivity/issues/96

> But OTOH, do you actually need to be using discoveryv1.EndpointSlice? The slices you want aren't going to be automatically created by the EndpointSlice controller anyway... so why not just make a CRD type for the feature you want?

Unless I'm missing something - I don't see an alternative to EndpointSlices that let me provide DNS records to the cluster. I don't think creating a CRD is going to help here.  
> generally it seems like FQDN is being used to introduce CNAME records into Kubernetes's DNS services.

see @chrisohaver comment in https://github.com/kubernetes/kubernetes/issues/114210#issuecomment-1337646747

> The following answer would be illegal in DNS, since there are two CNAMEs with the same name.

However, it seems that it works in CoreDNS with one record, @chrisohaver any idea how this can work ? I can't see you process those types on CoreDNS
https://github.com/coredns/coredns/blob/915b74f2232573746bcc76e1e987775b076a9ef4/plugin/kubernetes/object/endpoint.go#L106


I'm wary of seeing CNAME as a big limitation, because we have records like SVCB likely to come available as standards. And we already have SRV.
So. This is a bit embarrassing. I’ve been following this and scratching my head because I don’t remember this being a feature of CoreDNS. I haven’t had time to confirm, but I think the CoreDNS behavior is  an unintentional result of how the endpointslice is passed to the “backend” to be rendered to a dns record. The backend renders any non-ip record value as a CNAME.
> generally it seems like FQDN is being used to introduce CNAME records into Kubernetes's DNS services

This seems more like a bug than an intentional feature.  We have a supported way of introducing a CNAME record - `Service.spec.type: ExternalName`.  We should not add another equivalent-but-different mechanism.

>> With ingresses dropping support for ExternalName we were planning on knative/serving#11821 to using EndpointSlices with FQDN. The reasoning being is users don't interact with EndpointSlices and operators can use RBAC rules to limit their creation/modification - but allow the Knative control plane to do so.
>
> This does feel marginally safer, but I'm still not sure how safe it would be. 

My thoughts exactly.  Epsilon safer than a literal CVE is not "safe".

> Knative creates K8s services of type external name which point to a cluster local proxy (typically the proxies service dns name - istio-ingressgateway.istio-system.svc.cluster.local). This allows us to perform L7 type routing within a cluster without a mesh.

And this works, right?  As long as you don't try to create an Ingress.

Let's recall what that CVE was about - it was about not letting user B expose user A's workloads to user C (canonical example of C being "the internet"). 

Thinking out loud: 

@robscott this kind of feels like a job for ReferenceGrant, doesn't it?

If one were to drop a refgrant into namespace `istio-system`, expressing "allow access to Service istio-ingressgateway from namespace *", it seems clearer that the controller should allow it.  But actually, wait.  That doesn't distinguish between internal and external use, and once you have access to the proxy, I guess it uses hostname routing?

So could I do this:

User A has a service X intended only for internal traffic, ExternalName = the cluster proxy.
User B creates a service Y, ExternalName = the cluster proxy
User B gives service Y an internet-facing Ingress, which gets IP 1.2.3.4
User C accesses 1.2.3.4 with the hostname header set to "x.a.svc.cluster.local" 

User B has done exactly what that CVE tried to prevent.

Changing ExternalName to FQDN EPSlice limits this only to users who can write EPSlices, which is a small consolation.  Fundamentally you have a shared proxy, whose routing instructions are in the attacker's hands (hostname).  That just can't work, unless the routing is taken out of the attacker's control (e.g. destination IP or port).

Please forgive me if I am making some incorrect leap in logic.
> This seems more like a bug than an intentional feature. 

Your bug is my feature ;)

> And this works, right? As long as you don't try to create an Ingress.

Yup it works but _now_ it depends on the ingress implementation and their posture towards Service type=ExternalName.

> My thoughts exactly. Epsilon safer than a literal CVE is not "safe". ... Changing ExternalName to FQDN EPSlice limits this only to users who can write EPSlices, which is a small consolation.

I think you're on point where users don't create Endpoint[Slice]s and operators can easily create RBAC rules to prevent them. Since the creation of EndpointSlice type=FQDN can be limited to control planes (ie. Knative) I hope ingress implementations feel confident routing to these types of Services by default.

> Please forgive me if I am making some incorrect leap in logic.

Your example doesn't really apply because Knative has a common use case where people create internal Knative Services and then expose a few them to the internet using a DomainMapping feature.  But we have complimentary resources for the operator to limit _who_ can create _what_ mappings.

For this to work we need affordances that Ingress implementations trust.




> You're example doesn't really apply because Knative has a common use case where people create internal Knative Services and then expose a few them to the internet using a DomainMapping feature. But we have complimentary resources for the operator to limit who can create what mappings.

Once traffic arrives at the proxy, how does the proxy decide which backend service to route to? 
> Once traffic arrives at the proxy, how does the proxy decide which backend service to route to?

There are logically two proxies - an external one and an internal one. The external one would look at the host of the request - `foo.com` and then send that traffic to the desired backend - `some-revision.default.cluster.local` while rewriting the host. The internal proxy then routes based on traffic splitting and other rules to a set of K8s services representing Knative Revisions.
Is that "literally" two proxies or just "logically"?

What I want to understand is what stops user C from accessing service A via ingress B (as described in https://github.com/kubernetes/kubernetes/issues/114210#issuecomment-1447418634)?

Can you walk me thru the protections?
> Is that "literally" two proxies or just "logically"?

Depends on the implementation. For Contour we have multiple deployments, for Istio we use the same deployment but have distinct Gateways on different listeners.

When we apply internal routing rules we only program the internal gateway/deployment.

> Can you walk me thru the protections?

For Istio only the external listener is exposed via a Service type=LoadBalancer. Same for Contour.

So the proxy that user C is hitting won't have any `cluster.local` hosts. 

In the meantime should we revert - https://github.com/kubernetes/kubernetes/pull/114677 and hold the linked PRs?
I don't think we should roll back until we're sure. 

> For Istio only the external listener is exposed via a Service type=LoadBalancer. Same for Contour.

listener here meaning a distinct TCP port with distinct routing rules attached which are only evaluated if the client's original destination was provably an external LB?

How do we PROVE that?

Assume internal-only service A, type "ExternalName", which aliases to your proxy.  Your proxy listens on :443.

In-cluster client accesses the "cluster-local" name (which might not actually be ".cluster.local", but I assume you handle that).  That name is CNAME to your proxy on :443.  Proxy accepts the connection and looks at the hostname header.  It finds "A...local" and routes back to service A.

Repeat for any number of services, B, C, D, ...

Someone wants to put their service X on the internet.  They create an LB which routes hostname "external.example.com" to an IP, which is delivered to your proxy on :8443 (different than above).   Proxy accepts the connection and looks at the hostname header.  It finds "external.example.com" and routes back to service X.

Presumably the route-map for :8443 is different than for :443.  If a connection arrives at :8443 with hostname header "A...local", it has to be an error, right?

But what ensures that the LB config used :8443 and not :443?  Who is configuring that, and via what API?
> Presumably the route-map for :8443 is different than for :443. If a connection arrives at :8443 with hostname header "A...local", it has to be an error, right?

Yeah they don't share the same routing rules & hosts

> But what ensures that the LB config used :8443 and not :443? Who is configuring that, and via what API?

Knative Control Plane - using whatever underying network resource the users have installed (ie. Istio = VirtualService/Gateway, Contour=HTTPProxy)



Something still feels off but I can't put my finger on it.  

How do I request that my knative app be exposed on an external LB?  I know how to do it with a regular Service/Ingress/Gateway (and all of their respective *Class parameters).  But those mechanisms will not work with either ExternalName or FQDN endpoints.  Is there a knative native way to request an external LB?
Operators deploy the underlying networking stack (Istio, Contour) and then configure Knative to use certain infrastructure for the public and private/cluster local knative services. For Istio that means pointing to a two separate Gateways. For Contour you point to two separate instances.

Users of the Knative API can make their KnService cluster local using an annotation - https://knative.dev/docs/serving/services/private-services/
I appreciate the patience here, @dprotaso .  I'm trying to see the whole picture :)

So, as a user, I build my knative app and deploy it.  Why do I need a Service?  Is it *just* to get an internal name?

I want my internal name to send traffic to some other managed proxy (the mechanism is what we are discussing) in another namespace.

The managed portion of the stack has/will/can establish LB(s) which bring traffic into the cluster and to the managed proxy.  The proxy is clever enough to not route to knative services which didn't ask for it.

Right so far?

ExternalName services *do* work (though clunky for internal use, IMO).  An in-cluster user who access such a service will find a CNAME to the real proxy (which presumably has the original service name in its TLS certs?).

But why does the external LB->proxy path need the ExternalName, if it already knows the Service?  If traffic already arrives at the proxy, isn't that enough? 
> I appreciate the patience here, @dprotaso . I'm trying to see the whole picture :)

Yeah no worries, a white-board would help but I have time to write replies. I tried to make a [mermaid diagram](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-diagrams) but it was too hard to just look at the final diagram. You have to be there for the journey.

> Why do I need a Service? Is it just to get an internal name?

I guess to clarify Knative Service (ksvc) is it's own type distinct from a Kubernetes Service (svc). Creating a `ksvc` will trigger the Knative control plane to create a `svc` so that it's routable in the cluster using `(ksvc-name}.{namespace}.svc.cluster.local` address. This points to the internal proxy using `ExternalName` to perform L7 magic (traffic splitting).

> Right so far?

Yup

> ExternalName services do work (though clunky for internal use, IMO). An in-cluster user who access such a service will find a CNAME to the real proxy (which presumably has the original service name in its TLS certs?).

Yup - but in-cluster is currently unencrypted (but we're working on securing that)

> But why does the external LB->proxy path need the ExternalName, if it already knows the Service? If traffic already arrives at the proxy, isn't that enough?

I was a bit too terse [earlier](https://github.com/kubernetes/kubernetes/issues/114210#issuecomment-1447488637) so let me expand on DomainMapping a bit.

When you create an public `ksvc` the external URL is created from a template defined by the operator. Typically the template is setup to prevent collisions - ie. `{ksvc-name}.{namespace}.example.com`. DomainMapping let's users bring their own vanity domain - ie. `k8s.gcr.io`. But first the operator needs to authorize a namespace the ability to use a certain domain using a `DomainClaim`. Only once that's created can a user create a `DomainMapping` pointing to their `ksvc`. The way this is accomplish in our network programming is forwarding traffic for `k8s.gcr.io`(external proxy) to the `ksvc` internal URL `(ksvc-name}.{namespace}.svc.cluster.local` and performing a host-rewrite.  `(ksvc-name}.{namespace}.svc.cluster.local` points to the internal proxy via ExternalName/CNAME.

DomainMapping treats the target's as opaque. It's not documented but you can point them to any Kubernetes Service or some thing that conforms to a Knative 'Addressable' duck type (ie. resources that have a `status.address.url` property). Because we treat targets as opaque we don't do any special handling for our own types. Even if we did have special handling there's a few edge cases that would still require forwarding to an internal `svc` - ie. users are creating private `ksvc`s and using DomainMapping to expose a subset of them with vanity domains.



> The way this is accomplish in our network programming is forwarding traffic for k8s.gcr.io(external proxy) to the ksvc internal URL (ksvc-name}.{namespace}.svc.cluster.local and performing a host-rewrite

So the exposure here is that the service, being an ExternalName and in the user's NS, could be subverted. I could update it to point to any domain and make the proxy do my bidding. This is exactly the CVE that started the whole snowball. 

The "you can point to any Kubernetes Service or duck type"  is the same.  I can make your proxy hit any URL I want.  That seems like a bad bad bad feature.  Tell me I misunderstand? 

If you exclude ExternalName, and the DomainMapping is in the same NS as the target, you're back to what we have today, which doesn't do what you want.

So an FQDN endpoint is slightly better, in that most users can't write those. But you would still need to codify the DNS behavior, which doesn't work with CNAME unless you have just 1 endpoint.

It feels to me like what you want is an "alias" service which was subject to something like reference grant. The proxy allows refs from services in other namespaces, and  that produces a CNAME, but only if permitted.

I really don't want to extend service though.

Am I piecing it together? 


> So the exposure here is that the service, being an ExternalName and in the user's NS, could be subverted. I could update it to point to any domain and make the proxy do my bidding. This is exactly the CVE that started the whole snowball.

FWIW Operators can limit Service creation in clusters that have Knative installed mitigating the CVE.

> The "you can point to any Kubernetes Service or duck type" is the same. I can make your proxy hit any URL I want. That seems like a bad bad bad feature. Tell me I misunderstand?

There are restrictions ie. same namespace etc. 

> So an FQDN endpoint is slightly better, in that most users can't write those. But you would still need to codify the DNS behavior, which doesn't work with CNAME unless you have just 1 endpoint.
>
> It feels to me like what you want is an "alias" service which was subject to something like reference grant. The proxy allows refs from services in other namespaces, and that produces a CNAME, but only if permitted.

Using EndpointSlices works well since it can support more than just FQDN but also IP addresses. 

An 'alias' service is an interesting idea - curious how you would imagine the mechanics of that working. But a thing to consider is the [internal knative ingress spec](https://github.com/knative/networking/blob/74471b95eeda7e401d1b3e7e3cea76ef32ed8521/pkg/apis/networking/v1alpha1/ingress_types.go#L299-L321) allows proxies to be off cluster and referenced using a domain name. We don't have any networking layers implementing this currently but I would imagine this is something we'll see when migrating to Gateway API (ie. Gateway's with AddressType=Hostname)
If we deprecate setting FDQN as the `addressType` for an EndpointSlice, what does that mean for APIService?

I'm asking because one way people might have hoped to use APIService is to specify that there is an aggregated APIServer somewhere that you can reach via a hostname, implementing some API group. The control plane API servers could resolve that DNS record when connecting (a well established pattern!)

If we take the lack of defined semantics as grounds for deprecation, we're making it harder to run aggregated APIs outside the cluster. You still could, but you'd either need to make sure you use fixed IP addressing :grimacing: or you'd need some sort of DNS-to-endpoint glue that wouldn't be fun to write or use.
another use case for targeting things off cluster

From: https://www.nginx.com/blog/announcing-nginx-ingress-controller-for-kubernetes-release-1-5-0/#ExternalName

> Support for ExternalName services makes it easier to migrate your services into a Kubernetes cluster. You can run your HTTP load balancers (NGINX Plus) within Kubernetes, then route traffic both to the services in the cluster and to services that have not yet been moved to the cluster.
So, the problem here arises from two things:
- there are lots of places where you want to use a relatively-opaque hostname to direct somewhere _outside_ the cluster
- there are some, _limited_ use cases where it's good to be able to use a hostname to direct traffic inside a cluster, _using other ways to manage the security risks inherent in doing so_.

I think that it all comes down to treating the hostname field as opaque. I _think_ that we could make large swaths of at least my concern about all these hostname fields that control traffic flow go away, _purely using API constructs_ using some rules that looked like this:
- ReferenceGrant is available in the cluster
- Any field that uses a hostname for forwarding traffic _must_ be parsed and checked for things that look like `<service>.<namespace>.svc.<cluster-domain>`, and if that pattern is matched, they must be validated by checking that `<service>` in `<namespace>` has a ReferenceGrant allowing the traffic.
- For things that aren't services inside the cluster, knock yourself out.

Of course, this really moves the problem one layer out, in that it means that an attacker with access to a running pod inside the cluster can:
- Find the name and namespace of some service they shouldn't have access to
- Look up the IP associated with `service.namespace.svc.cluster.local`, call it `1.2.3.4`
- Create an external hostname that resolves to `1.2.3.4`
- Use that to get a confused deputy (like an Ingress or Gateway API controller proxy) to forward traffic to that service for you.

(This is why the Gateway API recommendations for ExternalName services use such strong language, because it's reasonably likely that a motivated attacker can fool whatever allowlist mechanisms an implementation puts in place).

The only way to definitely mitigate this problem is to have something that implements network policy, _and, more importantly_, a full set of network policies that only allow access to services from what they're supposed to.
> Any field that uses a hostname for forwarding traffic must be parsed and checked for things that look like `<service>.<namespace>.svc.<cluster-domain>`, and if that pattern is matched, they must be validated by checking that `<service>` in <namespace> has a ReferenceGrant allowing the traffic.

Is this enough? What happens if I find a SaaS app running on Kubernetes and tell it that, as a SaaS tenant, the webhook to report to is `https://foo.kube-system.svc.cluster.example/vulnerablething`?

In other words, I think that confused deputy risk is there regardless of EndpointSlice, and we need an answer to the general concern as well as one for EndpointSlice specifically.

To bring this discussion back to the issue - `no defined semantics for EndpointSlices of type FQDN`. I chimed in on this issue to highlight that there are _actual_ semantics for FQDN that have been functional for five releases (1.22-1.27). And a few more use cases have surfaced. It would be great to come to a conclusion here and decide on the deprecation of this address type. 

If we decide not to deprecate then it seems like some action items out of this discussion are
1. Update the documention/website about how FQDN can be used as an CNAME alternative for Service type=ExternalName
    1. Should there also be a conformance test?
1.  Mention the risk of a confused deputy attack if operators don't following the [security advisory about users creating/editing Endpoints/Slices](https://discuss.kubernetes.io/t/security-advisory-cve-2021-25740-endpoint-endpointslice-permissions-allow-cross-namespace-forwarding/16700). 
1. Highlight `NetworkPolicy` as a means of preventing similar attacks






> In other words, I think that confused deputy risk is there regardless of EndpointSlice, and we need an answer to the general concern as well as one for EndpointSlice specifically.

Yeah, I think this is fair. I was trying to get some sort of risk mitigation rule solely using API things, but I think that the risk can't entirely be mitigated like that.

Circling back - I brought this up in a sig-networking meeting in April (recording is [here](https://www.youtube.com/watch?v=wNPZHoYX2s4&list=PL69nYSiGNLP2E8vmnqo5MwPOY25sDWIxb&t=2200s))

@robscott proposed a path that would piggy back onto @aojea's Cluster IP allocation KEP - where we would provide a way to allocate DNS names.

@thockin had the following points
1. There's a discussion ongoing about Service IP allocation and how it relates to Pod CIDR allocation - this gates the discussion on what we do with DNS
2. Alternatively we can look at the problem more fundamentally (it's an auth issue) and try to creatively come up with a different solution

There was some discussion about considering ReferenceGrant (the one being worked on in sig-auth)


> > . And in fact, the expected semantics seem to be "[it gets ignored](https://github.com/kubernetes/kubernetes/blob/v1.26.0-rc.0/pkg/proxy/endpoints_test.go#L1590)".
> 
> that is kube-proxy but slices can be consumed by others.
> 
> However, I've tried to find the motivation or some use cases but I couldn't find them #84091

I can add one of potential use cases utilizing EndpointSlices of the type "FQDN".

Use case details: Once the proxy on the edge cluster receives specific user request pattern, it will forward the request to ANOTHER cluster through DNS resolution.  This can be done through the proxy utilizing Service and EndpointSlice with FQDN entities inside.


This issue has not been updated in over 1 year, and should be re-triaged.

You can:
- Confirm that this issue is still relevant with `/triage accepted` (org members only)
- Close this issue with `/close`

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted
Revisiting in sig-network: We should try to revive https://github.com/kubernetes/kubernetes/pull/115160

No voluteers yet, revisit next meeting?

Fix patch:
diff --git a/api/openapi-spec/swagger.json b/api/openapi-spec/swagger.json
index 44a26bbc5cbdd..4463aed9c17ec 100644
--- a/api/openapi-spec/swagger.json
+++ b/api/openapi-spec/swagger.json
@@ -12132,7 +12132,7 @@
       "description": "Endpoint represents a single logical \"backend\" implementing a service.",
       "properties": {
         "addresses": {
-          "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267",
+          "description": "addresses of this endpoint. For EndpointSlices of addressType \"IPv4\" or \"IPv6\", the values are IP addresses in canonical form. The syntax and semantics of other addressType values are not defined. This must contain at least one address but no more than 100. EndpointSlices generated by the EndpointSlice controller will always have exactly 1 address. No semantics are defined for additional addresses beyond the first, and kube-proxy does not look at them.",
           "items": {
             "type": "string"
           },
@@ -12180,15 +12180,15 @@
       "description": "EndpointConditions represents the current condition of an endpoint.",
       "properties": {
         "ready": {
-          "description": "ready indicates that this endpoint is prepared to receive traffic, according to whatever system is managing the endpoint. A nil value indicates an unknown state. In most cases consumers should interpret this unknown state as ready. For compatibility reasons, ready should never be \"true\" for terminating endpoints, except when the normal readiness behavior is being explicitly overridden, for example when the associated Service has set the publishNotReadyAddresses flag.",
+          "description": "ready indicates that this endpoint is ready to receive traffic, according to whatever system is managing the endpoint. A nil value should be interpreted as \"true\". In general, an endpoint should be marked ready if it is serving and not terminating, though this can be overridden in some cases, such as when the associated Service has set the publishNotReadyAddresses flag.",
           "type": "boolean"
         },
         "serving": {
-          "description": "serving is identical to ready except that it is set regardless of the terminating state of endpoints. This condition should be set to true for a ready endpoint that is terminating. If nil, consumers should defer to the ready condition.",
+          "description": "serving indicates that this endpoint is able to receive traffic, according to whatever system is managing the endpoint. For endpoints backed by pods, the EndpointSlice controller will mark the endpoint as serving if the pod's Ready condition is True. A nil value should be interpreted as \"true\".",
           "type": "boolean"
         },
         "terminating": {
-          "description": "terminating indicates that this endpoint is terminating. A nil value indicates an unknown state. Consumers should interpret this unknown state to mean that the endpoint is not terminating.",
+          "description": "terminating indicates that this endpoint is terminating. A nil value should be interpreted as \"false\".",
           "type": "boolean"
         }
       },
@@ -12220,7 +12220,7 @@
           "type": "string"
         },
         "port": {
-          "description": "port represents the port number of the endpoint. If this is not specified, ports are not restricted and must be interpreted in the context of the specific consumer.",
+          "description": "port represents the port number of the endpoint. If the EndpointSlice is derived from a Kubernetes service, this must be set to the service's target port. EndpointSlices used for other purposes may have a nil port.",
           "format": "int32",
           "type": "integer"
         },
@@ -12233,10 +12233,10 @@
       "x-kubernetes-map-type": "atomic"
     },
     "io.k8s.api.discovery.v1.EndpointSlice": {
-      "description": "EndpointSlice represents a subset of the endpoints that implement a service. For a given service there may be multiple EndpointSlice objects, selected by labels, which must be joined to produce the full set of endpoints.",
+      "description": "EndpointSlice represents a set of service endpoints. Most EndpointSlices are created by the EndpointSlice controller to represent the Pods selected by Service objects. For a given service there may be multiple EndpointSlice objects which must be joined to produce the full set of endpoints; you can find all of the slices for a given service by listing EndpointSlices in the service's namespace whose `kubernetes.io/service-name` label contains the service's name.",
       "properties": {
         "addressType": {
-          "description": "addressType specifies the type of address carried by this EndpointSlice. All addresses in this slice must be the same type. This field is immutable after creation. The following address types are currently supported: * IPv4: Represents an IPv4 Address. * IPv6: Represents an IPv6 Address. * FQDN: Represents a Fully Qualified Domain Name.",
+          "description": "addressType specifies the type of address carried by this EndpointSlice. All addresses in this slice must be the same type. This field is immutable after creation. The following address types are currently supported: * IPv4: Represents an IPv4 Address. * IPv6: Represents an IPv6 Address. * FQDN: Represents a Fully Qualified Domain Name. (Deprecated) The EndpointSlice controller only generates, and kube-proxy only processes, slices of addressType \"IPv4\" and \"IPv6\". No semantics are defined for the \"FQDN\" type.",
           "type": "string"
         },
         "apiVersion": {
@@ -12260,7 +12260,7 @@
           "description": "Standard object's metadata."
         },
         "ports": {
-          "description": "ports specifies the list of network ports exposed by each endpoint in this slice. Each port must have a unique name. When ports is empty, it indicates that there are no defined ports. When a port is defined with a nil port value, it indicates \"all ports\". Each slice may include a maximum of 100 ports.",
+          "description": "ports specifies the list of network ports exposed by each endpoint in this slice. Each port must have a unique name. Each slice may include a maximum of 100 ports. Services always have at least 1 port, so EndpointSlices generated by the EndpointSlice controller will likewise always have at least 1 port. EndpointSlices used for other purposes may have an empty ports list.",
           "items": {
             "$ref": "#/definitions/io.k8s.api.discovery.v1.EndpointPort"
           },
diff --git a/api/openapi-spec/v3/apis__discovery.k8s.io__v1_openapi.json b/api/openapi-spec/v3/apis__discovery.k8s.io__v1_openapi.json
index 85360ad92e8e0..ccf2bf0dee309 100644
--- a/api/openapi-spec/v3/apis__discovery.k8s.io__v1_openapi.json
+++ b/api/openapi-spec/v3/apis__discovery.k8s.io__v1_openapi.json
@@ -40,7 +40,7 @@
         "description": "Endpoint represents a single logical \"backend\" implementing a service.",
         "properties": {
           "addresses": {
-            "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267",
+            "description": "addresses of this endpoint. For EndpointSlices of addressType \"IPv4\" or \"IPv6\", the values are IP addresses in canonical form. The syntax and semantics of other addressType values are not defined. This must contain at least one address but no more than 100. EndpointSlices generated by the EndpointSlice controller will always have exactly 1 address. No semantics are defined for additional addresses beyond the first, and kube-proxy does not look at them.",
             "items": {
               "default": "",
               "type": "string"
@@ -103,15 +103,15 @@
         "description": "EndpointConditions represents the current condition of an endpoint.",
         "properties": {
           "ready": {
-            "description": "ready indicates that this endpoint is prepared to receive traffic, according to whatever system is managing the endpoint. A nil value indicates an unknown state. In most cases consumers should interpret this unknown state as ready. For compatibility reasons, ready should never be \"true\" for terminating endpoints, except when the normal readiness behavior is being explicitly overridden, for example when the associated Service has set the publishNotReadyAddresses flag.",
+            "description": "ready indicates that this endpoint is ready to receive traffic, according to whatever system is managing the endpoint. A nil value should be interpreted as \"true\". In general, an endpoint should be marked ready if it is serving and not terminating, though this can be overridden in some cases, such as when the associated Service has set the publishNotReadyAddresses flag.",
             "type": "boolean"
           },
           "serving": {
-            "description": "serving is identical to ready except that it is set regardless of the terminating state of endpoints. This condition should be set to true for a ready endpoint that is terminating. If nil, consumers should defer to the ready condition.",
+            "description": "serving indicates that this endpoint is able to receive traffic, according to whatever system is managing the endpoint. For endpoints backed by pods, the EndpointSlice controller will mark the endpoint as serving if the pod's Ready condition is True. A nil value should be interpreted as \"true\".",
             "type": "boolean"
           },
           "terminating": {
-            "description": "terminating indicates that this endpoint is terminating. A nil value indicates an unknown state. Consumers should interpret this unknown state to mean that the endpoint is not terminating.",
+            "description": "terminating indicates that this endpoint is terminating. A nil value should be interpreted as \"false\".",
             "type": "boolean"
           }
         },
@@ -148,7 +148,7 @@
             "type": "string"
           },
           "port": {
-            "description": "port represents the port number of the endpoint. If this is not specified, ports are not restricted and must be interpreted in the context of the specific consumer.",
+            "description": "port represents the port number of the endpoint. If the EndpointSlice is derived from a Kubernetes service, this must be set to the service's target port. EndpointSlices used for other purposes may have a nil port.",
             "format": "int32",
             "type": "integer"
           },
@@ -161,11 +161,11 @@
         "x-kubernetes-map-type": "atomic"
       },
       "io.k8s.api.discovery.v1.EndpointSlice": {
-        "description": "EndpointSlice represents a subset of the endpoints that implement a service. For a given service there may be multiple EndpointSlice objects, selected by labels, which must be joined to produce the full set of endpoints.",
+        "description": "EndpointSlice represents a set of service endpoints. Most EndpointSlices are created by the EndpointSlice controller to represent the Pods selected by Service objects. For a given service there may be multiple EndpointSlice objects which must be joined to produce the full set of endpoints; you can find all of the slices for a given service by listing EndpointSlices in the service's namespace whose `kubernetes.io/service-name` label contains the service's name.",
         "properties": {
           "addressType": {
             "default": "",
-            "description": "addressType specifies the type of address carried by this EndpointSlice. All addresses in this slice must be the same type. This field is immutable after creation. The following address types are currently supported: * IPv4: Represents an IPv4 Address. * IPv6: Represents an IPv6 Address. * FQDN: Represents a Fully Qualified Domain Name.",
+            "description": "addressType specifies the type of address carried by this EndpointSlice. All addresses in this slice must be the same type. This field is immutable after creation. The following address types are currently supported: * IPv4: Represents an IPv4 Address. * IPv6: Represents an IPv6 Address. * FQDN: Represents a Fully Qualified Domain Name. (Deprecated) The EndpointSlice controller only generates, and kube-proxy only processes, slices of addressType \"IPv4\" and \"IPv6\". No semantics are defined for the \"FQDN\" type.",
             "type": "string"
           },
           "apiVersion": {
@@ -199,7 +199,7 @@
             "description": "Standard object's metadata."
           },
           "ports": {
-            "description": "ports specifies the list of network ports exposed by each endpoint in this slice. Each port must have a unique name. When ports is empty, it indicates that there are no defined ports. When a port is defined with a nil port value, it indicates \"all ports\". Each slice may include a maximum of 100 ports.",
+            "description": "ports specifies the list of network ports exposed by each endpoint in this slice. Each port must have a unique name. Each slice may include a maximum of 100 ports. Services always have at least 1 port, so EndpointSlices generated by the EndpointSlice controller will likewise always have at least 1 port. EndpointSlices used for other purposes may have an empty ports list.",
             "items": {
               "allOf": [
                 {
diff --git a/pkg/generated/openapi/zz_generated.openapi.go b/pkg/generated/openapi/zz_generated.openapi.go
index ba89f317420f5..c1856e758f7ac 100644
--- a/pkg/generated/openapi/zz_generated.openapi.go
+++ b/pkg/generated/openapi/zz_generated.openapi.go
@@ -33137,7 +33137,7 @@ func schema_k8sio_api_discovery_v1_Endpoint(ref common.ReferenceCallback) common
 							},
 						},
 						SchemaProps: spec.SchemaProps{
-							Description: "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267",
+							Description: "addresses of this endpoint. For EndpointSlices of addressType \"IPv4\" or \"IPv6\", the values are IP addresses in canonical form. The syntax and semantics of other addressType values are not defined. This must contain at least one address but no more than 100. EndpointSlices generated by the EndpointSlice controller will always have exactly 1 address. No semantics are defined for additional addresses beyond the first, and kube-proxy does not look at them.",
 							Type:        []string{"array"},
 							Items: &spec.SchemaOrArray{
 								Schema: &spec.Schema{
@@ -33224,21 +33224,21 @@ func schema_k8sio_api_discovery_v1_EndpointConditions(ref common.ReferenceCallba
 				Properties: map[string]spec.Schema{
 					"ready": {
 						SchemaProps: spec.SchemaProps{
-							Description: "ready indicates that this endpoint is prepared to receive traffic, according to whatever system is managing the endpoint. A nil value indicates an unknown state. In most cases consumers should interpret this unknown state as ready. For compatibility reasons, ready should never be \"true\" for terminating endpoints, except when the normal readiness behavior is being explicitly overridden, for example when the associated Service has set the publishNotReadyAddresses flag.",
+							Description: "ready indicates that this endpoint is ready to receive traffic, according to whatever system is managing the endpoint. A nil value should be interpreted as \"true\". In general, an endpoint should be marked ready if it is serving and not terminating, though this can be overridden in some cases, such as when the associated Service has set the publishNotReadyAddresses flag.",
 							Type:        []string{"boolean"},
 							Format:      "",
 						},
 					},
 					"serving": {
 						SchemaProps: spec.SchemaProps{
-							Description: "serving is identical to ready except that it is set regardless of the terminating state of endpoints. This condition should be set to true for a ready endpoint that is terminating. If nil, consumers should defer to the ready condition.",
+							Description: "serving indicates that this endpoint is able to receive traffic, according to whatever system is managing the endpoint. For endpoints backed by pods, the EndpointSlice controller will mark the endpoint as serving if the pod's Ready condition is True. A nil value should be interpreted as \"true\".",
 							Type:        []string{"boolean"},
 							Format:      "",
 						},
 					},
 					"terminating": {
 						SchemaProps: spec.SchemaProps{
-							Description: "terminating indicates that this endpoint is terminating. A nil value indicates an unknown state. Consumers should interpret this unknown state to mean that the endpoint is not terminating.",
+							Description: "terminating indicates that this endpoint is terminating. A nil value should be interpreted as \"false\".",
 							Type:        []string{"boolean"},
 							Format:      "",
 						},
@@ -33307,7 +33307,7 @@ func schema_k8sio_api_discovery_v1_EndpointPort(ref common.ReferenceCallback) co
 					},
 					"port": {
 						SchemaProps: spec.SchemaProps{
-							Description: "port represents the port number of the endpoint. If this is not specified, ports are not restricted and must be interpreted in the context of the specific consumer.",
+							Description: "port represents the port number of the endpoint. If the EndpointSlice is derived from a Kubernetes service, this must be set to the service's target port. EndpointSlices used for other purposes may have a nil port.",
 							Type:        []string{"integer"},
 							Format:      "int32",
 						},
@@ -33334,7 +33334,7 @@ func schema_k8sio_api_discovery_v1_EndpointSlice(ref common.ReferenceCallback) c
 	return common.OpenAPIDefinition{
 		Schema: spec.Schema{
 			SchemaProps: spec.SchemaProps{
-				Description: "EndpointSlice represents a subset of the endpoints that implement a service. For a given service there may be multiple EndpointSlice objects, selected by labels, which must be joined to produce the full set of endpoints.",
+				Description: "EndpointSlice represents a set of service endpoints. Most EndpointSlices are created by the EndpointSlice controller to represent the Pods selected by Service objects. For a given service there may be multiple EndpointSlice objects which must be joined to produce the full set of endpoints; you can find all of the slices for a given service by listing EndpointSlices in the service's namespace whose `kubernetes.io/service-name` label contains the service's name.",
 				Type:        []string{"object"},
 				Properties: map[string]spec.Schema{
 					"kind": {
@@ -33360,7 +33360,7 @@ func schema_k8sio_api_discovery_v1_EndpointSlice(ref common.ReferenceCallback) c
 					},
 					"addressType": {
 						SchemaProps: spec.SchemaProps{
-							Description: "addressType specifies the type of address carried by this EndpointSlice. All addresses in this slice must be the same type. This field is immutable after creation. The following address types are currently supported: * IPv4: Represents an IPv4 Address. * IPv6: Represents an IPv6 Address. * FQDN: Represents a Fully Qualified Domain Name.\n\nPossible enum values:\n - `\"FQDN\"` represents a FQDN.\n - `\"IPv4\"` represents an IPv4 Address.\n - `\"IPv6\"` represents an IPv6 Address.",
+							Description: "addressType specifies the type of address carried by this EndpointSlice. All addresses in this slice must be the same type. This field is immutable after creation. The following address types are currently supported: * IPv4: Represents an IPv4 Address. * IPv6: Represents an IPv6 Address. * FQDN: Represents a Fully Qualified Domain Name. (Deprecated) The EndpointSlice controller only generates, and kube-proxy only processes, slices of addressType \"IPv4\" and \"IPv6\". No semantics are defined for the \"FQDN\" type.\n\nPossible enum values:\n - `\"FQDN\"` represents a FQDN.\n - `\"IPv4\"` represents an IPv4 Address.\n - `\"IPv6\"` represents an IPv6 Address.",
 							Default:     "",
 							Type:        []string{"string"},
 							Format:      "",
@@ -33393,7 +33393,7 @@ func schema_k8sio_api_discovery_v1_EndpointSlice(ref common.ReferenceCallback) c
 							},
 						},
 						SchemaProps: spec.SchemaProps{
-							Description: "ports specifies the list of network ports exposed by each endpoint in this slice. Each port must have a unique name. When ports is empty, it indicates that there are no defined ports. When a port is defined with a nil port value, it indicates \"all ports\". Each slice may include a maximum of 100 ports.",
+							Description: "ports specifies the list of network ports exposed by each endpoint in this slice. Each port must have a unique name. Each slice may include a maximum of 100 ports. Services always have at least 1 port, so EndpointSlices generated by the EndpointSlice controller will likewise always have at least 1 port. EndpointSlices used for other purposes may have an empty ports list.",
 							Type:        []string{"array"},
 							Items: &spec.SchemaOrArray{
 								Schema: &spec.Schema{
diff --git a/staging/src/k8s.io/api/discovery/v1/generated.proto b/staging/src/k8s.io/api/discovery/v1/generated.proto
index 8ddf0dc5d3ef9..5214fc8e6e020 100644
--- a/staging/src/k8s.io/api/discovery/v1/generated.proto
+++ b/staging/src/k8s.io/api/discovery/v1/generated.proto
@@ -31,12 +31,12 @@ option go_package = "k8s.io/api/discovery/v1";
 
 // Endpoint represents a single logical "backend" implementing a service.
 message Endpoint {
-  // addresses of this endpoint. The contents of this field are interpreted
-  // according to the corresponding EndpointSlice addressType field. Consumers
-  // must handle different types of addresses in the context of their own
-  // capabilities. This must contain at least one address but no more than
-  // 100. These are all assumed to be fungible and clients may choose to only
-  // use the first element. Refer to: https://issue.k8s.io/106267
+  // addresses of this endpoint. For EndpointSlices of addressType "IPv4" or "IPv6",
+  // the values are IP addresses in canonical form. The syntax and semantics of
+  // other addressType values are not defined. This must contain at least one
+  // address but no more than 100. EndpointSlices generated by the EndpointSlice
+  // controller will always have exactly 1 address. No semantics are defined for
+  // additional addresses beyond the first, and kube-proxy does not look at them.
   // +listType=set
   repeated string addresses = 1;
 
@@ -82,26 +82,25 @@ message Endpoint {
 
 // EndpointConditions represents the current condition of an endpoint.
 message EndpointConditions {
-  // ready indicates that this endpoint is prepared to receive traffic,
+  // ready indicates that this endpoint is ready to receive traffic,
   // according to whatever system is managing the endpoint. A nil value
-  // indicates an unknown state. In most cases consumers should interpret this
-  // unknown state as ready. For compatibility reasons, ready should never be
-  // "true" for terminating endpoints, except when the normal readiness
-  // behavior is being explicitly overridden, for example when the associated
-  // Service has set the publishNotReadyAddresses flag.
+  // should be interpreted as "true". In general, an endpoint should be
+  // marked ready if it is serving and not terminating, though this can
+  // be overridden in some cases, such as when the associated Service has
+  // set the publishNotReadyAddresses flag.
   // +optional
   optional bool ready = 1;
 
-  // serving is identical to ready except that it is set regardless of the
-  // terminating state of endpoints. This condition should be set to true for
-  // a ready endpoint that is terminating. If nil, consumers should defer to
-  // the ready condition.
+  // serving indicates that this endpoint is able to receive traffic,
+  // according to whatever system is managing the endpoint. For endpoints
+  // backed by pods, the EndpointSlice controller will mark the endpoint
+  // as serving if the pod's Ready condition is True. A nil value should be
+  // interpreted as "true".
   // +optional
   optional bool serving = 2;
 
   // terminating indicates that this endpoint is terminating. A nil value
-  // indicates an unknown state. Consumers should interpret this unknown state
-  // to mean that the endpoint is not terminating.
+  // should be interpreted as "false".
   // +optional
   optional bool terminating = 3;
 }
@@ -132,8 +131,9 @@ message EndpointPort {
   optional string protocol = 2;
 
   // port represents the port number of the endpoint.
-  // If this is not specified, ports are not restricted and must be
-  // interpreted in the context of the specific consumer.
+  // If the EndpointSlice is derived from a Kubernetes service, this must be set
+  // to the service's target port. EndpointSlices used for other purposes may have
+  // a nil port.
   optional int32 port = 3;
 
   // The application protocol for this port.
@@ -155,9 +155,12 @@ message EndpointPort {
   optional string appProtocol = 4;
 }
 
-// EndpointSlice represents a subset of the endpoints that implement a service.
-// For a given service there may be multiple EndpointSlice objects, selected by
-// labels, which must be joined to produce the full set of endpoints.
+// EndpointSlice represents a set of service endpoints. Most EndpointSlices are created by
+// the EndpointSlice controller to represent the Pods selected by Service objects. For a
+// given service there may be multiple EndpointSlice objects which must be joined to
+// produce the full set of endpoints; you can find all of the slices for a given service
+// by listing EndpointSlices in the service's namespace whose `kubernetes.io/service-name`
+// label contains the service's name.
 message EndpointSlice {
   // Standard object's metadata.
   // +optional
@@ -169,7 +172,10 @@ message EndpointSlice {
   // supported:
   // * IPv4: Represents an IPv4 Address.
   // * IPv6: Represents an IPv6 Address.
-  // * FQDN: Represents a Fully Qualified Domain Name.
+  // * FQDN: Represents a Fully Qualified Domain Name. (Deprecated)
+  // The EndpointSlice controller only generates, and kube-proxy only processes,
+  // slices of addressType "IPv4" and "IPv6". No semantics are defined for
+  // the "FQDN" type.
   optional string addressType = 4;
 
   // endpoints is a list of unique endpoints in this slice. Each slice may
@@ -178,10 +184,11 @@ message EndpointSlice {
   repeated Endpoint endpoints = 2;
 
   // ports specifies the list of network ports exposed by each endpoint in
-  // this slice. Each port must have a unique name. When ports is empty, it
-  // indicates that there are no defined ports. When a port is defined with a
-  // nil port value, it indicates "all ports". Each slice may include a
+  // this slice. Each port must have a unique name. Each slice may include a
   // maximum of 100 ports.
+  // Services always have at least 1 port, so EndpointSlices generated by the
+  // EndpointSlice controller will likewise always have at least 1 port.
+  // EndpointSlices used for other purposes may have an empty ports list.
   // +optional
   // +listType=atomic
   repeated EndpointPort ports = 3;
diff --git a/staging/src/k8s.io/api/discovery/v1/types.go b/staging/src/k8s.io/api/discovery/v1/types.go
index d6a9d0fcedb74..9ddcda269c413 100644
--- a/staging/src/k8s.io/api/discovery/v1/types.go
+++ b/staging/src/k8s.io/api/discovery/v1/types.go
@@ -25,9 +25,12 @@ import (
 // +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
 // +k8s:prerelease-lifecycle-gen:introduced=1.21
 
-// EndpointSlice represents a subset of the endpoints that implement a service.
-// For a given service there may be multiple EndpointSlice objects, selected by
-// labels, which must be joined to produce the full set of endpoints.
+// EndpointSlice represents a set of service endpoints. Most EndpointSlices are created by
+// the EndpointSlice controller to represent the Pods selected by Service objects. For a
+// given service there may be multiple EndpointSlice objects which must be joined to
+// produce the full set of endpoints; you can find all of the slices for a given service
+// by listing EndpointSlices in the service's namespace whose `kubernetes.io/service-name`
+// label contains the service's name.
 type EndpointSlice struct {
 	metav1.TypeMeta `json:",inline"`
 
@@ -41,7 +44,10 @@ type EndpointSlice struct {
 	// supported:
 	// * IPv4: Represents an IPv4 Address.
 	// * IPv6: Represents an IPv6 Address.
-	// * FQDN: Represents a Fully Qualified Domain Name.
+	// * FQDN: Represents a Fully Qualified Domain Name. (Deprecated)
+	// The EndpointSlice controller only generates, and kube-proxy only processes,
+	// slices of addressType "IPv4" and "IPv6". No semantics are defined for
+	// the "FQDN" type.
 	AddressType AddressType `json:"addressType" protobuf:"bytes,4,rep,name=addressType"`
 
 	// endpoints is a list of unique endpoints in this slice. Each slice may
@@ -50,10 +56,11 @@ type EndpointSlice struct {
 	Endpoints []Endpoint `json:"endpoints" protobuf:"bytes,2,rep,name=endpoints"`
 
 	// ports specifies the list of network ports exposed by each endpoint in
-	// this slice. Each port must have a unique name. When ports is empty, it
-	// indicates that there are no defined ports. When a port is defined with a
-	// nil port value, it indicates "all ports". Each slice may include a
+	// this slice. Each port must have a unique name. Each slice may include a
 	// maximum of 100 ports.
+	// Services always have at least 1 port, so EndpointSlices generated by the
+	// EndpointSlice controller will likewise always have at least 1 port.
+	// EndpointSlices used for other purposes may have an empty ports list.
 	// +optional
 	// +listType=atomic
 	Ports []EndpointPort `json:"ports" protobuf:"bytes,3,rep,name=ports"`
@@ -76,12 +83,12 @@ const (
 
 // Endpoint represents a single logical "backend" implementing a service.
 type Endpoint struct {
-	// addresses of this endpoint. The contents of this field are interpreted
-	// according to the corresponding EndpointSlice addressType field. Consumers
-	// must handle different types of addresses in the context of their own
-	// capabilities. This must contain at least one address but no more than
-	// 100. These are all assumed to be fungible and clients may choose to only
-	// use the first element. Refer to: https://issue.k8s.io/106267
+	// addresses of this endpoint. For EndpointSlices of addressType "IPv4" or "IPv6",
+	// the values are IP addresses in canonical form. The syntax and semantics of
+	// other addressType values are not defined. This must contain at least one
+	// address but no more than 100. EndpointSlices generated by the EndpointSlice
+	// controller will always have exactly 1 address. No semantics are defined for
+	// additional addresses beyond the first, and kube-proxy does not look at them.
 	// +listType=set
 	Addresses []string `json:"addresses" protobuf:"bytes,1,rep,name=addresses"`
 
@@ -127,26 +134,25 @@ type Endpoint struct {
 
 // EndpointConditions represents the current condition of an endpoint.
 type EndpointConditions struct {
-	// ready indicates that this endpoint is prepared to receive traffic,
+	// ready indicates that this endpoint is ready to receive traffic,
 	// according to whatever system is managing the endpoint. A nil value
-	// indicates an unknown state. In most cases consumers should interpret this
-	// unknown state as ready. For compatibility reasons, ready should never be
-	// "true" for terminating endpoints, except when the normal readiness
-	// behavior is being explicitly overridden, for example when the associated
-	// Service has set the publishNotReadyAddresses flag.
+	// should be interpreted as "true". In general, an endpoint should be
+	// marked ready if it is serving and not terminating, though this can
+	// be overridden in some cases, such as when the associated Service has
+	// set the publishNotReadyAddresses flag.
 	// +optional
 	Ready *bool `json:"ready,omitempty" protobuf:"bytes,1,name=ready"`
 
-	// serving is identical to ready except that it is set regardless of the
-	// terminating state of endpoints. This condition should be set to true for
-	// a ready endpoint that is terminating. If nil, consumers should defer to
-	// the ready condition.
+	// serving indicates that this endpoint is able to receive traffic,
+	// according to whatever system is managing the endpoint. For endpoints
+	// backed by pods, the EndpointSlice controller will mark the endpoint
+	// as serving if the pod's Ready condition is True. A nil value should be
+	// interpreted as "true".
 	// +optional
 	Serving *bool `json:"serving,omitempty" protobuf:"bytes,2,name=serving"`
 
 	// terminating indicates that this endpoint is terminating. A nil value
-	// indicates an unknown state. Consumers should interpret this unknown state
-	// to mean that the endpoint is not terminating.
+	// should be interpreted as "false".
 	// +optional
 	Terminating *bool `json:"terminating,omitempty" protobuf:"bytes,3,name=terminating"`
 }
@@ -183,8 +189,9 @@ type EndpointPort struct {
 	Protocol *v1.Protocol `json:"protocol,omitempty" protobuf:"bytes,2,name=protocol"`
 
 	// port represents the port number of the endpoint.
-	// If this is not specified, ports are not restricted and must be
-	// interpreted in the context of the specific consumer.
+	// If the EndpointSlice is derived from a Kubernetes service, this must be set
+	// to the service's target port. EndpointSlices used for other purposes may have
+	// a nil port.
 	Port *int32 `json:"port,omitempty" protobuf:"bytes,3,opt,name=port"`
 
 	// The application protocol for this port.
diff --git a/staging/src/k8s.io/api/discovery/v1/types_swagger_doc_generated.go b/staging/src/k8s.io/api/discovery/v1/types_swagger_doc_generated.go
index 41c3060568f25..22ad6f945e379 100644
--- a/staging/src/k8s.io/api/discovery/v1/types_swagger_doc_generated.go
+++ b/staging/src/k8s.io/api/discovery/v1/types_swagger_doc_generated.go
@@ -29,7 +29,7 @@ package v1
 // AUTO-GENERATED FUNCTIONS START HERE. DO NOT EDIT.
 var map_Endpoint = map[string]string{
 	"":                   "Endpoint represents a single logical \"backend\" implementing a service.",
-	"addresses":          "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267",
+	"addresses":          "addresses of this endpoint. For EndpointSlices of addressType \"IPv4\" or \"IPv6\", the values are IP addresses in canonical form. The syntax and semantics of other addressType values are not defined. This must contain at least one address but no more than 100. EndpointSlices generated by the EndpointSlice controller will always have exactly 1 address. No semantics are defined for additional addresses beyond the first, and kube-proxy does not look at them.",
 	"conditions":         "conditions contains information about the current status of the endpoint.",
 	"hostname":           "hostname of this endpoint. This field may be used by consumers of endpoints to distinguish endpoints from each other (e.g. in DNS names). Multiple endpoints which use the same hostname should be considered fungible (e.g. multiple A values in DNS). Must be lowercase and pass DNS Label (RFC 1123) validation.",
 	"targetRef":          "targetRef is a reference to a Kubernetes object that represents this endpoint.",
@@ -45,9 +45,9 @@ func (Endpoint) SwaggerDoc() map[string]string {
 
 var map_EndpointConditions = map[string]string{
 	"":            "EndpointConditions represents the current condition of an endpoint.",
-	"ready":       "ready indicates that this endpoint is prepared to receive traffic, according to whatever system is managing the endpoint. A nil value indicates an unknown state. In most cases consumers should interpret this unknown state as ready. For compatibility reasons, ready should never be \"true\" for terminating endpoints, except when the normal readiness behavior is being explicitly overridden, for example when the associated Service has set the publishNotReadyAddresses flag.",
-	"serving":     "serving is identical to ready except that it is set regardless of the terminating state of endpoints. This condition should be set to true for a ready endpoint that is terminating. If nil, consumers should defer to the ready condition.",
-	"terminating": "terminating indicates that this endpoint is terminating. A nil value indicates an unknown state. Consumers should interpret this unknown state to mean that the endpoint is not terminating.",
+	"ready":       "ready indicates that this endpoint is ready to receive traffic, according to whatever system is managing the endpoint. A nil value should be interpreted as \"true\". In general, an endpoint should be marked ready if it is serving and not terminating, though this can be overridden in some cases, such as when the associated Service has set the publishNotReadyAddresses flag.",
+	"serving":     "serving indicates that this endpoint is able to receive traffic, according to whatever system is managing the endpoint. For endpoints backed by pods, the EndpointSlice controller will mark the endpoint as serving if the pod's Ready condition is True. A nil value should be interpreted as \"true\".",
+	"terminating": "terminating indicates that this endpoint is terminating. A nil value should be interpreted as \"false\".",
 }
 
 func (EndpointConditions) SwaggerDoc() map[string]string {
@@ -67,7 +67,7 @@ var map_EndpointPort = map[string]string{
 	"":            "EndpointPort represents a Port used by an EndpointSlice",
 	"name":        "name represents the name of this port. All ports in an EndpointSlice must have a unique name. If the EndpointSlice is derived from a Kubernetes service, this corresponds to the Service.ports[].name. Name must either be an empty string or pass DNS_LABEL validation: * must be no more than 63 characters long. * must consist of lower case alphanumeric characters or '-'. * must start and end with an alphanumeric character. Default is empty string.",
 	"protocol":    "protocol represents the IP protocol for this port. Must be UDP, TCP, or SCTP. Default is TCP.",
-	"port":        "port represents the port number of the endpoint. If this is not specified, ports are not restricted and must be interpreted in the context of the specific consumer.",
+	"port":        "port represents the port number of the endpoint. If the EndpointSlice is derived from a Kubernetes service, this must be set to the service's target port. EndpointSlices used for other purposes may have a nil port.",
 	"appProtocol": "The application protocol for this port. This is used as a hint for implementations to offer richer behavior for protocols that they understand. This field follows standard Kubernetes label syntax. Valid values are either:\n\n* Un-prefixed protocol names - reserved for IANA standard service names (as per RFC-6335 and https://www.iana.org/assignments/service-names).\n\n* Kubernetes-defined prefixed names:\n  * 'kubernetes.io/h2c' - HTTP/2 prior knowledge over cleartext as described in https://www.rfc-editor.org/rfc/rfc9113.html#name-starting-http-2-with-prior-\n  * 'kubernetes.io/ws'  - WebSocket over cleartext as described in https://www.rfc-editor.org/rfc/rfc6455\n  * 'kubernetes.io/wss' - WebSocket over TLS as described in https://www.rfc-editor.org/rfc/rfc6455\n\n* Other protocols should use implementation-defined prefixed names such as mycompany.com/my-custom-protocol.",
 }
 
@@ -76,11 +76,11 @@ func (EndpointPort) SwaggerDoc() map[string]string {
 }
 
 var map_EndpointSlice = map[string]string{
-	"":            "EndpointSlice represents a subset of the endpoints that implement a service. For a given service there may be multiple EndpointSlice objects, selected by labels, which must be joined to produce the full set of endpoints.",
+	"":            "EndpointSlice represents a set of service endpoints. Most EndpointSlices are created by the EndpointSlice controller to represent the Pods selected by Service objects. For a given service there may be multiple EndpointSlice objects which must be joined to produce the full set of endpoints; you can find all of the slices for a given service by listing EndpointSlices in the service's namespace whose `kubernetes.io/service-name` label contains the service's name.",
 	"metadata":    "Standard object's metadata.",
-	"addressType": "addressType specifies the type of address carried by this EndpointSlice. All addresses in this slice must be the same type. This field is immutable after creation. The following address types are currently supported: * IPv4: Represents an IPv4 Address. * IPv6: Represents an IPv6 Address. * FQDN: Represents a Fully Qualified Domain Name.",
+	"addressType": "addressType specifies the type of address carried by this EndpointSlice. All addresses in this slice must be the same type. This field is immutable after creation. The following address types are currently supported: * IPv4: Represents an IPv4 Address. * IPv6: Represents an IPv6 Address. * FQDN: Represents a Fully Qualified Domain Name. (Deprecated) The EndpointSlice controller only generates, and kube-proxy only processes, slices of addressType \"IPv4\" and \"IPv6\". No semantics are defined for the \"FQDN\" type.",
 	"endpoints":   "endpoints is a list of unique endpoints in this slice. Each slice may include a maximum of 1000 endpoints.",
-	"ports":       "ports specifies the list of network ports exposed by each endpoint in this slice. Each port must have a unique name. When ports is empty, it indicates that there are no defined ports. When a port is defined with a nil port value, it indicates \"all ports\". Each slice may include a maximum of 100 ports.",
+	"ports":       "ports specifies the list of network ports exposed by each endpoint in this slice. Each port must have a unique name. Each slice may include a maximum of 100 ports. Services always have at least 1 port, so EndpointSlices generated by the EndpointSlice controller will likewise always have at least 1 port. EndpointSlices used for other purposes may have an empty ports list.",
 }
 
 func (EndpointSlice) SwaggerDoc() map[string]string {


Test patch: