Code change:
We allow users to specify in the Service object. If the user later removes this from the Service Spec, the controller will fail to sync. For example: creates an LB for Service.Spec.LoadBalancerIP = "1.1.1.1" with a user-owned static IP address. updates the Service to remove the requested LoadBalancerIP tries to used by the existing forwarding rule (i.e., "1.1.1.1"). It encounters a conflict. Controller assumes it owns the IP and by name but fails. We can fix the controller to continue syncing and using the user-owned IP for the LB, or create a new address for the LB. The former seems less disruptive to me, but the downside is that if user will not be able to let other components use this IP (i.e., 1.1.1.1) unless they explicitly delete the Service. I think it's ok but would like more opinions. thoughts?
If a service is updated to remove a static address, then it seems like that would be a deliberate move on the part of the user. It seems like it should probably release the address and provision a dynamic one, even if this is disruptive. How do we handle the reverse? If a user has an already existing LoadBalancer service, and they add the after the fact? Would this release the dynamic address and recreate the forwarding rule with the reserved static address?
It seems like the creating a new address makes sense here... General comment: given the combinations (and the bugs we have been running into), I think this is a problem that all of the potential state transitions for the load balancer have not been fully spec'd out.
Creating a new address seems nice but the implementation is a nightmare. The service controller doesn't give you a previous Service spec, so you don't know what's changed (controller might not know either if spec was modified while offline). Given a desired service spec without a how do you decide to ditch your current IP and grab a new one? If we were to call to evaluate the situation, it's highly likely users will accidentally cause LBs to change IPs during typical usage. Imagine a user has a dynamically allocated IP for a service, but they want to promote it for safe keeping. The controller would think it needs to surrender the IP to the user's reservation. The only correct process would be to set before promoting the address; however, that will surely be a missed point in docs and result in unhappy users. I lean towards the LB continuing use of the IP. Besides the easy option of deleting & re-creating the service, they could change the to another static address then demote the address. I'm not too familiar with the ideological boundaries of controllers and spec, so let me know if this crosses the line. What if the service controller set to the created IP on first sync if the IP wasn't pre-specified? The controller could rely on loadBalancerIP to always indicate which IP to use. If the field is empty (blanked by user), then it's a signal to get a new IP.
By spec, I mean we should write down the careful reasoning that you just wrote in a permanent place :-)
would it then make sense for to be immutable on an already created service?
+1 +1 Was thinking the same. Unlike the many annotations we support, updates to this actual field could be validated and blocked at the apiserver... Changing IP is disruptive anyway. Could we simply ask the user to re-create the Service?
Of course, this could be an API-breaking change, and other cloud providers may actually be supporting this properly.
I just ran across this today, but in a more likely (IMO) case. 1) create the Service, type=LoadBalancer 2) let the IP be auto-allocated 3) manually promote that IP to static and give it a meaningful name 4) observe errors from service controller trying to allocate that IP Since my Service does not spec "loadBalancerIP", the question is, do we: a) use the IP, but leave it as static b) use the IP, but demote it back to ephemeral c) allocate and use a new IP C is completely hostile. B is subtly hostile. A seems correct. Can it be implemented? If we find that the IP exists but the name of it is not what we expect, we should treat it as user-provided and leave it alone. This is still nasty because it logs a failure in the GCP activities log for every loop, but I don't know how to avoid that without getting a list of IPs and scanning for the IP we want.
Filed for the looping.
+1 as long as we can rely on the "naming" of the address to decide whether the controller owns the address and can promote/demote it.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale Still relevant
/assign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale Still relevant
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
any update on this?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/lifecycle frozen
If this issue has been triaged, please comment . If you aren't able to handle this issue, consider unassigning yourself and/or adding the label. I am a bot run by vllry. ‍

Related discussion:
text: if err != nil || !existed || ip != ipPrime { t.Fatalf(`ensureStaticIP(%v, %v, %v, %v, %v) = %v, %v, %v; want %v, true, nil`, gce, ipName, serviceName, gce.region, ip, ipPrime, existed, err, ip) } <ins> // Ensure call with different name ipName = "another-name-for-static-ip" ipPrime, existed, err = ensureStaticIP(gce, ipName, serviceName, gce.region, ip, cloud.NetworkTierDefault) if err != nil || !existed || ip != ipPrime { t.Fatalf(`ensureStaticIP(%v, %v, %v, %v, %v) = %v, %v, %v; want %v, true, nil`, gce, ipName, serviceName, gce.region, ip, ipPrime, existed, err, ip) } </ins> } func TestEnsureStaticIPWithTier(t *testing.T) {