Repository: kubernetes
Issue: [Umbrella] Make client-go lighter and easier to consume
Labels: sig/api-machinery,sig/architecture,area/code-organization,triage/accepted

### Description

The current structure of client-go, with its dependencies on `k8s.io/api` and `k8s.io/apimachinery` present some significant challenges:

* **Dependency bloat:** projects end up pulling in a large number of transitive dependencies, increasing binary size and potentially leading to conflicts.
* **Versioning headaches:** Updating client-go often necessitates updating the dependent projects as well, which can be complex and time-consuming, especially with the need for coordinated releases.
* **Tight coupling:** This structure creates tight coupling between client-go and the underlying API definitions, making necessary to update the clients to be able to use the new API types.

### How to repro

Using the workqueue example https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/client-go/examples/workqueue and creating a new project, after doing `go mod tidy` the `go.mod` file looks like:

<details>
<summary>go.mod</summary>

```
module client-go-example

go 1.23

require (
        k8s.io/api v0.31.1
        k8s.io/apimachinery v0.31.1
        k8s.io/client-go v0.31.1
        k8s.io/klog/v2 v2.130.1
)

require (
        github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
        github.com/emicklei/go-restful/v3 v3.11.0 // indirect
        github.com/fxamacker/cbor/v2 v2.7.0 // indirect
        github.com/go-logr/logr v1.4.2 // indirect
        github.com/go-openapi/jsonpointer v0.19.6 // indirect
        github.com/go-openapi/jsonreference v0.20.2 // indirect
        github.com/go-openapi/swag v0.22.4 // indirect
        github.com/gogo/protobuf v1.3.2 // indirect
        github.com/golang/protobuf v1.5.4 // indirect
        github.com/google/gnostic-models v0.6.8 // indirect
        github.com/google/go-cmp v0.6.0 // indirect
        github.com/google/gofuzz v1.2.0 // indirect
        github.com/google/uuid v1.6.0 // indirect
        github.com/imdario/mergo v0.3.6 // indirect
        github.com/josharian/intern v1.0.0 // indirect
        github.com/json-iterator/go v1.1.12 // indirect
        github.com/mailru/easyjson v0.7.7 // indirect
        github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
        github.com/modern-go/reflect2 v1.0.2 // indirect
        github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
        github.com/spf13/pflag v1.0.5 // indirect
        github.com/x448/float16 v0.8.4 // indirect
        golang.org/x/net v0.26.0 // indirect
        golang.org/x/oauth2 v0.21.0 // indirect
        golang.org/x/sys v0.21.0 // indirect
        golang.org/x/term v0.21.0 // indirect
        golang.org/x/text v0.16.0 // indirect
        golang.org/x/time v0.3.0 // indirect
        google.golang.org/protobuf v1.34.2 // indirect
        gopkg.in/inf.v0 v0.9.1 // indirect
        gopkg.in/yaml.v2 v2.4.0 // indirect
        gopkg.in/yaml.v3 v3.0.1 // indirect
        k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 // indirect
        k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 // indirect
        sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
        sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
        sigs.k8s.io/yaml v1.4.0 // indirect
)
```
</details>

and its size is 

```
ls -lah main
-rwxr-x--- 1 aojea primarygroup 51M Oct  6 08:41 main
```

### Proposal

* **Modularization:** Breaking down client-go into smaller, more focused modules would allow users to import only the functionalities they need, reducing dependency bloat and improving maintainability.
* **Generic clients and Versioned APIs:** Providing generic clients that can use versioned API clients would enable users to target specific Kubernetes versions without being forced to update the entire client-go library.
* **Reducing dependencies:** eliminate unnecessary dependencies within client-go and its related projects.

### Design Details

#### k8s.io/api

Ideally this should be a self contained module without dependencies,  but it depends on apimachinery for the generated code and registering the types

https://github.com/kubernetes/kubernetes/blob/7b28a115ba04651bc31aa1d7089abbd67ec5c067/staging/src/k8s.io/api/go.mod#L3-L13

It requires `k8s.io/apimachinery` because of:

* schema
```
register.go:     "k8s.io/apimachinery/pkg/runtime"
register.go:     "k8s.io/apimachinery/pkg/runtime/schema"
zz_generated.deepcopy.go:        runtime "k8s.io/apimachinery/pkg/runtime"
```
* metav1
```
register.go:     metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
zz_generated.deepcopy.go:        v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
```

* utils for types

```
"k8s.io/apimachinery/pkg/util/validation"
zz_generated.deepcopy.go:  intstr "k8s.io/apimachinery/pkg/util/intstr"
resource/v1alpha3/types.go:     "k8s.io/apimachinery/pkg/util/validation"
```

**NOTE** This validation can be easily avoided and it seems is only to match a constant  (cc: @pohly )

https://github.com/kubernetes/kubernetes/blob/7b28a115ba04651bc31aa1d7089abbd67ec5c067/staging/src/k8s.io/api/resource/v1alpha3/types.go#L186

#### k8s.io/apimachinery

This is the more tricky, and it is bringing a lot of dependencies

https://github.com/kubernetes/kubernetes/blob/7b28a115ba04651bc31aa1d7089abbd67ec5c067/staging/src/k8s.io/apimachinery/go.mod#L3-L33

#### k8s.io/client-go

These module contains all the generated code, the typed clients and the dynamic client. Thanks to @skitt we already have  [Generics to share code in client-go](#121439 )

In addition to the dynamic client we could have a Generic client, that will be able to work with different versions of k8s.io/api, allowing users to work with skew between k8s.io/client-go and  k8s.io/api, though this requires a very high bar on these modules to avoid breaking clients and provide stability across the ecosystem.

https://github.com/kubernetes/kubernetes/blob/7b28a115ba04651bc31aa1d7089abbd67ec5c067/staging/src/k8s.io/client-go/go.mod#L3-L37

Ideally we should be able to split the API types from the tooling and allow consumers to use generics, something like

```go
import (
  corev1 "k8s.io/api/core/v1"
	k8s.io/client-go/generic
)


func main(
	client := generic.NewClient[corev1.Pod]
	list, err := client.Namespace("foo").List()

)
```

and use generic controllers, we already have some intree

https://github.com/kubernetes/kubernetes/blob/a7fcc89ac0e7124158eee58820bbf517e0a15377/staging/src/k8s.io/apiserver/pkg/admission/plugin/policy/internal/generic/interface.go#L27-L59

#### Tasks

- [ ] remove unnecessary dependencies, per example, testing frameworks only offer cosmetic benefits but bring dependencies, despite they are not used to build the binary the make more noisy the dependency tree https://github.com/kubernetes/kubernetes/pull/127876
- [ ] consolidate, per example,  we have our own fork of go-yaml https://github.com/kubernetes-sigs/yaml/pull/76 so we can switch all the dependencies to it to have just one path and also avoid possible code drifting
- [ ] stricter rules for client-go and apimachinery exported code, @pohly  already has setup a job as optional https://github.com/kubernetes/test-infra/pull/33579
- [ ] https://github.com/kubernetes/kubernetes/issues/127889


### References

*  #124380 
*  #106846
* [Analyzing Go Binary Sizes](https://blog.howardjohn.info/posts/go-binary-size/)

#### Notes

I don't create a KEP because there are a considerable number of dimensions in this problem, so as first step I just want to gather feedback and take a more iterative approach, there are things like dependency pruning that can be worked out without a KEP, once we reach a point we need to implement some user facing change I will open the corresponding KEP

/area code-organization
/sig architecture
/sig api-machinery