Change:
An can contain up to 1000 , which can each contain up to 100 . Or at least, that's what the docs say, but actually, kube-proxy (or more specifically, ) only looks at the first Address in each Endpoint: (This is compatible with how works, since it would only generate multi-Address Endpoints if a pod had multiple of the same address family, which is not allowed. But it's not compatible with manually-generated EndpointSlices.) Either all are used, or else the docs state that only is used and the field is an array solely for historical reasons.
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network /priority important-longterm /cc
All addresses in the slice must be of the same type, do you mean creating an Slice with different address types in the Addresses field?
I'm not sure if/how kube-proxy should handle an endpoint with more than 1 address. I think the goal of kube-proxy is to give each endpoint an equivalent probability of being selected. To extend that logic here, I think we'd need to take the probability assigned to an endpoint and divide it by the number of addresses attached to that endpoint. So if an endpoint has a 50% probability of being selected, and it provides 50 addresses, each should end up with a 1% probability of being selected. Of course with the way we need to do descending probabilities, the logic may get a bit complicated. This isn't as clear as it should be, but I don't think kube-proxy or any consumer of the EndpointSlice API needs to commit to supporting every value specified by that API. For example, many may filter by address type, Service, or some other means. I think it's also valid for a consumer to say they only support a single address per family per endpoint. So all of that to say, I'd support either documenting that kube-proxy only selects the first address in an endpoint, or updating kube-proxy to support more than one address per endpoint. Given the number of changes currently in flight for kube-proxy, documentation seems like a safer initial approach to me.
/assign
can I have this editing only the first address(Address[0]) in Endpoint.Address to be used. The use of an array in this field is based on historical reasons. Isn't that right
thanks for volunteering to help with this one! I know has some big kube-proxy changes/fixes in flight right now, so it would be good to be sure you don't conflict with him. Otherwise, as long as isn't already working on this, I think you're free to have it.
I don't think we agreed that the array is for historical reasons only. We aren't currently using it, but we aren't ready to say that we won't use it in the future at this point. So, the docs should warn that kube-proxy currently ignores additional addresses, but it shouldn't imply that it's actually wrong to have additional addresses I'm not
thanks i will work on this
It was for a reason, but I think that reason was mixed-family slices (before we decided to break it up into single-family slices). What would it mean to have multiple addresses here? Are they assumed to be equivalent? Or do they have different context (like a split-horizon)? If the latter, there isn't any metadata here (i.e. it's not a struct like to allow new fields). I am inclined to believe it was meant as equivalent, and as such it's probably OK to document that assumption clearly. That said, I don't see how anyone could use multiple addresses in a meaningful way - they are either assumed equivalent or they lack enough metadata to distinguish their differences. ?? If we do document the equivalence, it should be generically worded. E.g. "These addresses are assumed to be equivalent and some consumers may choose to ignore all but the first value."
if it was meant as equivalent, why put multiple same addresses into Addresses? I don't understand what the purpose of this is.
The first version of the API had both IP families in a single slice, so it made sense to have multiple addresses. We changed that, though we didn't immediately remove the old support. So my best guess is that we should have converted this to a singular value when we did that, but we missed it. On Wed, Nov 24, 2021, 5:45 PM Cyclinder
so we should change the API doc? Or It's just that documenting it will only use the first address in the , like you said above
I'd like some of the others to comment, but it's this a US holiday, so it might be a few days..
So maybe we should just declare that the array is essentially a bug, and only the first address is used? Maybe something should warn if you ever set additional addresses? (FTR, the context that I noticed this in was that I was adding unit tests, and was creating EndpointSlices with a single Endpoint and multiple Addresses, which doesn't work; you need multiple Endpoints with a single Address each.)
I think the simplest and most compat approach is to say "these are all assumed to be fungible and clients may choose to only use the first element". Thoughts?
I think the simplest and most compat approach is to say "these are all assumed to be fungible and clients may choose to only use the first element". Big +1 to that approach. I don't think it's necessarily invalid to set multiple addresses here, there may actually be some use cases for it, but adding this kind of statement should help clear things up significantly.
hmm as we push for multi homed pod this will make a lot of sense. There might also be a chance for community to provide a drop-in replacement for proxy that does much more than we do now. I think we should update the docs and don't modify anything for now.
Note that this API doesn't include per-IP metadata, so a multi-homed pod would be challenging with this mechanic. I think it's maybe time to reconsider multi-net more holistically. On Mon, Dec 13, 2021, 10:24 AM Khaled Henidak (Kal) <
:thinking:
If they're fungible, then "clients may choose to use only one of the elements" (or "only a subset"?). No reason to call out the first one specifically.
That's fair On Tue, Dec 14, 2021, 6:09 AM Dan Winship
I would like to add to this issue that it would makes sense to have kube-proxy reduce the subset of endpoints in the endpointslice based on the label of the node and the & attributes of the endpoint. Taking only into consideration the first endpoint which is in the same zone of the node or has no zone dependency and is ready for serving. & Would it make sense to reopen this issue?
I don't follow entirely what you are suggesting, can you elaborate?
kube-proxy should understand which zone a node is in and prefer endpoints in the same zone.
but that is unrelated to this issue, topology is considered See KEPs related, topology-aware-routing was replaced by topology-aware-hints
thank you for clarifying. It seems ambiguous to me to have a zone attribute and a hint attribute. I fail to implement the hinting on a service in v1.23.8, but this is urelated to this issue.
heh, don't worry, this topic is complex and we went through different solutions, you have more historical context on the KEPs I linked above and this thread in the mailing list

Discussion:
"description": "Endpoint represents a single logical "backend" implementing a service.", "properties": { "addresses": { <del> "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100.", </del> <ins> "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267", </ins> "items": { "type": "string" },

"description": "Endpoint represents a single logical "backend" implementing a service.", "properties": { "addresses": { <del> "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100.", </del> <ins> "description": "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267", </ins> "items": { "default": "", "type": "string"

// AUTO-GENERATED FUNCTIONS START HERE. DO NOT EDIT. var map_Endpoint = map[string]string{ "":                   "Endpoint represents a single logical "backend" implementing a service.", <del> "addresses":          "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100.", </del> <ins> "addresses":          "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267", </ins> "conditions":         "conditions contains information about the current status of the endpoint.", "hostname":           "hostname of this endpoint. This field may be used by consumers of endpoints to distinguish endpoints from each other (e.g. in DNS names). Multiple endpoints which use the same hostname should be considered fungible (e.g. multiple A values in DNS). Must be lowercase and pass DNS Label (RFC 1123) validation.", "targetRef":          "targetRef is a reference to a Kubernetes object that represents this endpoint.",

// according to the corresponding EndpointSlice addressType field. Consumers // must handle different types of addresses in the context of their own // capabilities. This must contain at least one address but no more than <del> // 100. </del> <ins> // 100. These are all assumed to be fungible and clients may choose to only // use the first element. Refer to: https://issue.k8s.io/106267 </ins> // +listType=set repeated string addresses = 1;

// according to the corresponding EndpointSlice addressType field. Consumers // must handle different types of addresses in the context of their own // capabilities. This must contain at least one address but no more than <del> // 100. </del> <ins> // 100. These are all assumed to be fungible and clients may choose to only // use the first element. Refer to: https://issue.k8s.io/106267 </ins> // +listType=set Addresses []string `json:"addresses" protobuf:"bytes,1,rep,name=addresses"` // conditions contains information about the current status of the endpoint.

// AUTO-GENERATED FUNCTIONS START HERE. DO NOT EDIT. var map_Endpoint = map[string]string{ "":           "Endpoint represents a single logical "backend" implementing a service.", <del> "addresses":  "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100.", </del> <ins> "addresses":  "addresses of this endpoint. The contents of this field are interpreted according to the corresponding EndpointSlice addressType field. Consumers must handle different types of addresses in the context of their own capabilities. This must contain at least one address but no more than 100. These are all assumed to be fungible and clients may choose to only use the first element. Refer to: https://issue.k8s.io/106267", </ins> "conditions": "conditions contains information about the current status of the endpoint.", "hostname":   "hostname of this endpoint. This field may be used by consumers of endpoints to distinguish endpoints from each other (e.g. in DNS names). Multiple endpoints which use the same hostname should be considered fungible (e.g. multiple A values in DNS). Must be lowercase and pass DNS Label (RFC 1123) validation.", "targetRef":  "targetRef is a reference to a Kubernetes object that represents this endpoint.",