Change:
// or has already been reserved.  ErrFull will be returned if there // are no addresses left. func (r *Range) Allocate(ip net.IP) error { <ins> label := r.CIDR() </ins> ok, offset := r.contains(ip) if !ok { <ins> // update metrics clusterIPAllocationErrors.WithLabelValues(label.String()).Inc() </ins> return &ErrNotInRange{r.net.String()} } allocated, err := r.alloc.Allocate(offset) if err != nil { <ins> // update metrics clusterIPAllocationErrors.WithLabelValues(label.String()).Inc() </ins> return err } if !allocated { <ins> // update metrics clusterIPAllocationErrors.WithLabelValues(label.String()).Inc() </ins> return ErrAllocated } <ins> // update metrics clusterIPAllocations.WithLabelValues(label.String()).Inc() clusterIPAllocated.WithLabelValues(label.String()).Set(float64(r.Used())) clusterIPAvailable.WithLabelValues(label.String()).Set(float64(r.Free())) </ins> return nil } // AllocateNext reserves one of the IPs from the pool. ErrFull may // be returned if there are no addresses left. func (r *Range) AllocateNext() (net.IP, error) { <ins> label := r.CIDR() </ins> offset, ok, err := r.alloc.AllocateNext() if err != nil { <ins> // update metrics clusterIPAllocationErrors.WithLabelValues(label.String()).Inc() </ins> return nil, err } if !ok { <ins> // update metrics clusterIPAllocationErrors.WithLabelValues(label.String()).Inc() </ins> return nil, ErrFull } <ins> // update metrics clusterIPAllocations.WithLabelValues(label.String()).Inc() clusterIPAllocated.WithLabelValues(label.String()).Set(float64(r.Used())) clusterIPAvailable.WithLabelValues(label.String()).Set(float64(r.Free())) </ins> return utilnet.AddIPOffset(r.base, offset), nil }

Discussion:
<!-- Feature requests are unlikely to make progress as an issue. Instead, please suggest enhancements by engaging with SIGs on slack and mailing lists. A proposal that works through the design along with the implications of the change can be opened as a KEP: --Add metrics to apiserver tracking the service IP allocator - around how many IP addresses are used(allocated) vs available, and/or if there were any errors allocating service IPs to a service. We want to track if we are approaching the limit of the ServiceClusterIPRange, which is a fixed CIDR and difficult to change. Currently while we can approximate the usage looking at services in the cluster, it is not coming from the source allocator which is within the apiserver. Similar metrics are currently available for the node's ipam allocatior: /area apiserver /sig instrumentation /sig network
/triage accepted /assign I see something like this, gauges for and can you expand on this? a counter incremented each time an IP allocation fails?
that is one option. for contrast the nodeipam metric has an 'allocationtriesper_request' metric - i was imagining something similar to an 'attempts', but with a success:true/false tag/facet so we could compare how often allocations are attempted/how often they fail. I will defer to the experts here as I'm not sure what the status quo for reporting attempts/failures are. I'm not sure if there's any code paths that would cause allocation failures other than usage total (or available <= 0) - but having them separated out like that would be handy in that scenario. (maybe etcd internal allocation is corrupt? or context deadline exceeded)