jiebi/CodeConvo huggingface dataset:
each kube-apiserver. You can inspect Leases owned by each kube-apiserver by checking for lease objects in the `kube-system` namespace <del> with the name `kube-apiserver-<sha256-hash>`. Alternatively you can use the label selector `k8s.io/component=kube-apiserver`: </del> <ins> with the name `kube-apiserver-<sha256-hash>`. Alternatively you can use the label selector `apiserver.kubernetes.io/identity=kube-apiserver`: </ins> ```shell <del> kubectl -n kube-system get lease -l k8s.io/component=kube-apiserver </del> <ins> kubectl -n kube-system get lease -l apiserver.kubernetes.io/identity=kube-apiserver </ins> ``` ``` NAME                                        HOLDER                                                                           AGE <del> kube-apiserver-c4vwjftbvpc5os2vvzle4qg27a   kube-apiserver-c4vwjftbvpc5os2vvzle4qg27a_9cbf54e5-1136-44bd-8f9a-1dcd15c346b4   5m33s kube-apiserver-dz2dqprdpsgnm756t5rnov7yka   kube-apiserver-dz2dqprdpsgnm756t5rnov7yka_84f2a85d-37c1-4b14-b6b9-603e62e4896f   4m23s kube-apiserver-fyloo45sdenffw2ugwaz3likua   kube-apiserver-fyloo45sdenffw2ugwaz3likua_c5ffa286-8a9a-45d4-91e7-61118ed58d2e   4m43s </del> <ins> apiserver-07a5ea9b9b072c4a5f3d1c3702        apiserver-07a5ea9b9b072c4a5f3d1c3702_0c8914f7-0f35-440e-8676-7844977d3a05        5m33s apiserver-7be9e061c59d368b3ddaf1376e        apiserver-7be9e061c59d368b3ddaf1376e_84f2a85d-37c1-4b14-b6b9-603e62e4896f        4m23s apiserver-1dfef752bcb36637d2763d1868        apiserver-1dfef752bcb36637d2763d1868_c5ffa286-8a9a-45d4-91e7-61118ed58d2e        4m43s </ins> ``` The SHA256 hash used in the lease name is based on the OS hostname as seen by that API server. Each kube-apiserver should be
hostname used by kube-apisever by checking the value of the `kubernetes.io/hostname` label: ```shell <del> kubectl -n kube-system get lease kube-apiserver-c4vwjftbvpc5os2vvzle4qg27a -o yaml </del> <ins> kubectl -n kube-system get lease apiserver-07a5ea9b9b072c4a5f3d1c3702 -o yaml </ins> ``` ```yaml apiVersion: coordination.k8s.io/v1 kind: Lease metadata: <del> creationTimestamp: "2022-11-30T15:37:15Z" </del> <ins> creationTimestamp: "2023-07-02T13:16:48Z" </ins> labels: <del> k8s.io/component: kube-apiserver kubernetes.io/hostname: kind-control-plane name: kube-apiserver-c4vwjftbvpc5os2vvzle4qg27a </del> <ins> apiserver.kubernetes.io/identity: kube-apiserver kubernetes.io/hostname: master-1 name: apiserver-07a5ea9b9b072c4a5f3d1c3702 </ins> namespace: kube-system <del> resourceVersion: "18171" uid: d6c68901-4ec5-4385-b1ef-2d783738da6c </del> <ins> resourceVersion: "334899" uid: 90870ab5-1ba9-4523-b215-e4d4e662acb1 </ins> spec: <del> holderIdentity: kube-apiserver-c4vwjftbvpc5os2vvzle4qg27a_9cbf54e5-1136-44bd-8f9a-1dcd15c346b4 </del> <ins> holderIdentity: apiserver-07a5ea9b9b072c4a5f3d1c3702_0c8914f7-0f35-440e-8676-7844977d3a05 </ins> leaseDurationSeconds: 3600 <del> renewTime: "2022-11-30T18:04:27.912073Z" </del> <ins> renewTime: "2023-07-04T21:58:48.065888Z" </ins> ``` Expired leases from kube-apiservers that no longer exist are garbage collected by new kube-apiservers after 1 hour.
You should now be able to curl the nginx Service on `<CLUSTER-IP>:<PORT>` from any node in your cluster. Note that the Service IP is completely virtual, it never hits the wire. If you're curious about how this works you can read more <del> about the [service proxy](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies). </del> <ins> about the [service proxy](/docs/reference/networking/virtual-ips/). </ins> ## Accessing the Service
| Session affinity | Ensures that connections from a particular client are passed to the same Pod each time. | Windows Server 2022 | Set `service.spec.sessionAffinity` to "ClientIP" | | Direct Server Return (DSR) | Load balancing mode where the IP address fixups and the LBNAT occurs at the container vSwitch port directly; service traffic arrives with the source IP set as the originating pod IP. | Windows Server 2019 | Set the following flags in kube-proxy: `--feature-gates="WinDSR=true" --enable-dsr=true` | | Preserve-Destination | Skips DNAT of service traffic, thereby preserving the virtual IP of the target service in packets reaching the backend Pod. Also disables node-node forwarding. | Windows Server, version 1903 | Set `"preserve-destination": "true"` in service annotations and enable DSR in kube-proxy. | <del> | IPv4/IPv6 dual-stack networking | Native IPv4-to-IPv4 in parallel with IPv6-to-IPv6 communications to, from, and within a cluster | Windows Server 2019 | See [IPv4/IPv6 dual-stack](#ipv4ipv6-dual-stack) | </del> <ins> | IPv4/IPv6 dual-stack networking | Native IPv4-to-IPv4 in parallel with IPv6-to-IPv6 communications to, from, and within a cluster | Windows Server 2019 | See [IPv4/IPv6 dual-stack](/docs/concepts/services-networking/dual-stack/#windows-support) | </ins> | Client IP preservation | Ensures that source IP of incoming ingress traffic gets preserved. Also disables node-node forwarding. |  Windows Server 2019  | Set `service.spec.externalTrafficPolicy` to "Local" and enable DSR in kube-proxy | {{< /table >}}
I componenti del Control Plane sono responsabili di tutte le decisioni globali sul cluster (ad esempio, lo scheduling) oltre che a rilevare e rispondere agli eventi del cluster (ad esempio, l'avvio di un nuovo {{< glossary_tooltip text="pod" term_id="pod">}} quando il valore `replicas` di un deployment non è soddisfatto). I componenti della Control Plane possono essere eseguiti su qualsiasi nodo del cluster stesso. Solitamente, per semplicità, gli script di installazione tendono a eseguire tutti i componenti della Control Plane sulla stessa macchina, separando la Control Plane dai workload dell'utente. <del> Vedi [creare un cluster in High-Availability](/docs/admin/high-availability/) per un esempio di un'installazione multi-master. </del> <ins> Vedi [creare un cluster in High-Availability](/docs/setup/production-environment/tools/kubeadm/high-availability/) per un esempio di un'installazione multi-master. </ins> ### kube-apiserver
body.cid-community .community-section#meetups p:last-of-type { margin-bottom: 6em; /* extra space for background */ } <ins> } @media only screen and (max-width: 767px) { body.cid-community .community-section h2:before, body.cid-community .community-section h2:after { display: none; } </ins> }  No newline at end of file
## Labels, annotations and taints used on API objects <ins> ### apf.kubernetes.io/autoupdate-spec Type: Annotation Example: `apf.kubernetes.io/autoupdate-spec: "true"` Used on: [`FlowSchema` and `PriorityLevelConfiguration` Objects](/concepts/cluster-administration/flow-control/#defaults) If this annotation is set to true on a FlowSchema or PriorityLevelConfiguration, the `spec` for that object is managed by the kube-apiserver. If the API server does not recognize an APF object, and you annotate it for automatic update, the API server deletes the entire object. Otherwise, the API server does not manage the object spec. For more details, read  [Maintenance of the Mandatory and Suggested Configuration Objects](/docs/concepts/cluster-administration/flow-control/#maintenance-of-the-mandatory-and-suggested-configuration-objects). </ins> ### app.kubernetes.io/component Type: Label
Limiting port ranges of communication | This recommendation may be a bit self-explanatory, but wherever possible you should only expose the ports on your service that are absolutely essential for communication or metric gathering. | 3rd Party Dependency Security | It is a good practice to regularly scan your application's third party libraries for known security vulnerabilities. Each programming language has a tool for performing this check automatically. | Static Code Analysis | Most languages provide a way for a snippet of code to be analyzed for any potentially unsafe coding practices. Whenever possible you should perform checks using automated tooling that can scan codebases for common security errors. Some of the tools can be found at: https://owasp.org/www-community/Source_Code_Analysis_Tools | <del> Dynamic probing attacks | There are a few automated tools that you can run against your service to try some of the well known service attacks. These include SQL injection, CSRF, and XSS. One of the most popular dynamic analysis tools is the [OWASP Zed Attack proxy](https://owasp.org/www-project-zap/) tool. | </del> <ins> Dynamic probing attacks | There are a few automated tools that you can run against your service to try some of the well known service attacks. These include SQL injection, CSRF, and XSS. One of the most popular dynamic analysis tools is the [OWASP Zed Attack proxy](https://www.zaproxy.org/) tool. | </ins> {{< /table >}}
<p>If you haven't worked through the earlier sections, start from <a href="/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro/">Using minikube to create a cluster</a>.</p> <p><em>Scaling</em> is accomplished by changing the number of replicas in a Deployment</p> <ins> <p> <b> NOTE </b> If you are trying this after <a href="/docs/tutorials/kubernetes-basics/expose/expose-intro/">the previous section </a>, you may need to start from <a href="/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro/">creating a cluster</a> as the services may have been deleted </p> </ins> </div> <div class="col-md-4"> <div class="content__box content__box_lined">
To reflect the change on kubeadm nodes you must do the following: - Log in to a kubeadm node - Run `kubeadm upgrade node phase kubelet-config` to download the latest `kubelet-config` <del> ConfigMap contents into the local file `/var/lib/kubelet/config.conf` </del> <ins> ConfigMap contents into the local file `/var/lib/kubelet/config.yaml` </ins> - Edit the file `/var/lib/kubelet/kubeadm-flags.env` to apply additional configuration with flags - Restart the kubelet service with `systemctl restart kubelet`
{{< note >}} During `kubeadm upgrade`, kubeadm downloads the `KubeletConfiguration` from the <del> `kubelet-config` ConfigMap and overwrite the contents of `/var/lib/kubelet/config.conf`. </del> <ins> `kubelet-config` ConfigMap and overwrite the contents of `/var/lib/kubelet/config.yaml`. </ins> This means that node local configuration must be applied either by flags in `/var/lib/kubelet/kubeadm-flags.env` or by manually updating the contents of <del> `/var/lib/kubelet/config.conf` after `kubeadm upgrade`, and then restarting the kubelet. </del> <ins> `/var/lib/kubelet/config.yaml` after `kubeadm upgrade`, and then restarting the kubelet. </ins> {{< /note >}} ### Applying kube-proxy configuration changes
#### Persisting kubelet reconfiguration <del> Any changes to the `KubeletConfiguration` stored in `/var/lib/kubelet/config.conf` will be overwritten on </del> <ins> Any changes to the `KubeletConfiguration` stored in `/var/lib/kubelet/config.yaml` will be overwritten on </ins> `kubeadm upgrade` by downloading the contents of the cluster wide `kubelet-config` ConfigMap. <del> To persist kubelet node specific configuration either the file `/var/lib/kubelet/config.conf` </del> <ins> To persist kubelet node specific configuration either the file `/var/lib/kubelet/config.yaml` </ins> has to be updated manually post-upgrade or the file `/var/lib/kubelet/kubeadm-flags.env` can include flags. The kubelet flags override the associated `KubeletConfiguration` options, but note that some of the flags are deprecated. <del> A kubelet restart will be required after changing `/var/lib/kubelet/config.conf` or </del> <ins> A kubelet restart will be required after changing `/var/lib/kubelet/config.yaml` or </ins> `/var/lib/kubelet/kubeadm-flags.env`. ## {{% heading "whatsnext" %}}
and bare metal workloads. * [Flannel](https://github.com/flannel-io/flannel#deploying-flannel-manually) is an overlay network provider that can be used with Kubernetes. <ins> * [Gateway API](/docs/concepts/services-networking/gateway/) is an open source project managed by the [SIG Network](https://github.com/kubernetes/community/tree/master/sig-network) community and provides an expressive, extensible, and role-oriented API for modeling service networking. </ins> * [Knitter](https://github.com/ZTE/Knitter/) is a plugin to support multiple network interfaces in a Kubernetes pod. * [Multus](https://github.com/k8snetworkplumbingwg/multus-cni) is a Multi plugin for
to be reachable from outside your cluster. - [Ingress](/docs/concepts/services-networking/ingress/) provides extra functionality specifically for exposing HTTP applications, websites and APIs. <ins> - [Gateway API](/docs/concepts/services-networking/gateway/) is an {{<glossary_tooltip text="add-on" term_id="addons">}} that provides an expressive, extensible, and role-oriented family of API kinds for modeling service networking. </ins> - You can also use Services to [publish services only for consumption inside your cluster](/docs/concepts/services-networking/service-traffic-policy/).
* Read about [Ingress](/docs/concepts/services-networking/ingress/), which exposes HTTP and HTTPS routes from outside the cluster to Services within your cluster. <del> * Read about [Gateway](https://gateway-api.sigs.k8s.io/), an extension to </del> <ins> * Read about [Gateway](/docs/concepts/services-networking/gateway/), an extension to </ins> Kubernetes that provides more flexibility than Ingress. For more context, read the following:
<ins> --- title: Gateway API id: gateway-api date: 2023-10-19 full_link: /docs/concepts/services-networking/gateway/ short_description: > An API for modeling service networking in Kubernetes. aka: tags: - networking - architecture - extension --- A family of API kinds for modeling service networking in Kubernetes. <!--more--> Gateway API provides a family of extensible, role-oriented, protocol-aware API kinds for modeling service networking in Kubernetes. </ins>
You can use {{< glossary_tooltip term_id="containerd" text="ContainerD" >}} 1.4.0+ as the container runtime for Kubernetes nodes that run Windows. <del> Learn how to [install ContainerD on a Windows node](/docs/setup/production-environment/container-runtimes/#install-containerd). </del> <ins> Learn how to [install ContainerD on a Windows node](/docs/setup/production-environment/container-runtimes/#containerd). </ins> {{< note >}} There is a [known limitation](/docs/tasks/configure-pod-container/configure-gmsa/#gmsa-limitations) when using GMSA with containerd to access Windows network shares, which requires a
- `kubectl edit cm kubelet-config-x.yy -n kube-system`を呼び出します(`x.yy`はKubernetesのバージョンに置き換えてください)。 - 既存の`cgroupDriver`の値を修正するか、以下のような新しいフィールドを追加します。 <del> ``yaml </del> <ins> ```yaml </ins> cgroupDriver: systemd ```
communicates with). 1. (Recommended) If you have plans to upgrade this single control-plane `kubeadm` cluster <del> to high availability you should specify the `--control-plane-endpoint` to set the shared endpoint for all control-plane nodes. Such an endpoint can be either a DNS name or an IP address of a load-balancer. </del> <ins> to [high availability](/docs/setup/production-environment/tools/kubeadm/high-availability/) you should specify the `--control-plane-endpoint` to set the shared endpoint for all control-plane nodes. Such an endpoint can be either a DNS name or an IP address of a load-balancer. </ins> 1. Choose a Pod network add-on, and verify whether it requires any arguments to be passed to `kubeadm init`. Depending on which third-party provider you choose, you might need to set the `--pod-network-cidr` to
kubectl apply -f <add-on.yaml> ``` <ins> {{< note >}} Only a few CNI plugins support Windows. More details and setup instructions can be found in [Adding Windows worker nodes](/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes/#network-config). {{< /note >}} </ins> You can install only one Pod network per cluster. Once a Pod network has been installed, you can confirm that it is working by
* A compatible Linux host. The Kubernetes project provides generic instructions for Linux distributions based on Debian and Red Hat, and those distributions without a package manager. * 2 GB or more of RAM per machine (any less will leave little room for your apps). <del> * 2 CPUs or more. </del> <ins> * 2 CPUs or more for control plane machines. </ins> * Full network connectivity between all machines in the cluster (public or private network is fine). * Unique hostname, MAC address, and product_uuid for every node. See [here](#verify-mac-address) for more details. * Certain ports are open on your machines. See [here](#check-required-ports) for more details.
## Enabling sidecar containers <del> Starting with Kubernetes 1.28, a </del> <ins> Enabled by default with Kubernetes 1.29, a </ins> [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) named `SidecarContainers` allows you to specify a `restartPolicy` for containers listed in a Pod's `initContainers` field. These restartable _sidecar_ containers are independent with
Here's an example of a Job with two containers, one of which is a sidecar: {{% code_sample language="yaml" file="application/job/job-sidecar.yaml" %}} <del> By default, this feature is not available in Kubernetes. To avail this feature, you need to enable the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) named `SidecarContainers`. </del> ## Differences from regular containers
--- Allow setting the `restartPolicy` of an init container to `Always` so that the container becomes a sidecar container (restartable init containers). <del> See [Sidecar containers and restartPolicy](/docs/concepts/workloads/pods/init-containers/#sidecar-containers-and-restartpolicy) </del> <ins> See [Sidecar containers and restartPolicy](/docs/concepts/workloads/pods/sidecar-containers/) for more details. </ins>
Selector:     tier=frontend Labels:       app=guestbook tier=frontend <del> Annotations:  kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"apps/v1","kind":"ReplicaSet","metadata":{"annotations":{},"labels":{"app":"guestbook","tier":"frontend"},"name":"frontend",... </del> <ins> Annotations:  <none> </ins> Replicas:     3 current / 3 desired Pods Status:  3 Running / 0 Waiting / 0 Succeeded / 0 Failed Pod Template: Labels:  tier=frontend Containers: php-redis: <del> Image:        gcr.io/google_samples/gb-frontend:v3 </del> <ins> Image:        us-docker.pkg.dev/google-samples/containers/gke/gb-frontend:v5 </ins> Port:         <none> Host Port:    <none> Environment:  <none>
Events: Type    Reason            Age   From                   Message ----    ------            ----  ----                   ------- <del> Normal  SuccessfulCreate  117s  replicaset-controller  Created pod: frontend-wtsmm Normal  SuccessfulCreate  116s  replicaset-controller  Created pod: frontend-b2zdv Normal  SuccessfulCreate  116s  replicaset-controller  Created pod: frontend-vcmts </del> <ins> Normal  SuccessfulCreate  13s   replicaset-controller  Created pod: frontend-gbgfx Normal  SuccessfulCreate  13s   replicaset-controller  Created pod: frontend-rwz57 Normal  SuccessfulCreate  13s   replicaset-controller  Created pod: frontend-wkl7w </ins> ``` And lastly you can check for the Pods brought up:
``` NAME             READY   STATUS    RESTARTS   AGE <del> frontend-b2zdv   1/1     Running   0          6m36s frontend-vcmts   1/1     Running   0          6m36s frontend-wtsmm   1/1     Running   0          6m36s </del> <ins> frontend-gbgfx   1/1     Running   0          10m frontend-rwz57   1/1     Running   0          10m frontend-wkl7w   1/1     Running   0          10m </ins> ``` You can also verify that the owner reference of these pods is set to the frontend ReplicaSet. To do this, get the yaml of one of the Pods running: ```shell <del> kubectl get pods frontend-b2zdv -o yaml </del> <ins> kubectl get pods frontend-gbgfx -o yaml </ins> ``` The output will look similar to this, with the frontend ReplicaSet's info set in the metadata's ownerReferences field:
apiVersion: v1 kind: Pod metadata: <del> creationTimestamp: "2020-02-12T07:06:16Z" </del> <ins> creationTimestamp: "2024-02-28T22:30:44Z" </ins> generateName: frontend- labels: tier: frontend <del> name: frontend-b2zdv </del> <ins> name: frontend-gbgfx </ins> namespace: default ownerReferences: - apiVersion: apps/v1
## {{% heading "prerequisites" %}} <del> {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} </del> <ins> {{< include "task-tutorial-prereqs-node-upgrade.md" >}} {{< version-check >}} </ins> * Familiarize yourself with [the process for upgrading the rest of your kubeadm cluster](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade). You will want to upgrade the control plane nodes before upgrading your Linux Worker nodes.
## {{% heading "prerequisites" %}} <del> {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} </del> <ins> {{< include "task-tutorial-prereqs-node-upgrade.md" >}} {{< version-check >}} </ins> * Familiarize yourself with [the process for upgrading the rest of your kubeadm cluster](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade). You will want to upgrade the control plane nodes before upgrading your Windows nodes.
<code>string</code> </td> <td> <del> <p><code>ontrolPlaneEndpoint</code> sets a stable IP address or DNS name for the control plane; </del> <ins> <p><code>controlPlaneEndpoint</code> sets a stable IP address or DNS name for the control plane; </ins> It can be a valid IP address or a RFC-1123 DNS subdomain, both with optional TCP port. In case the <code>controlPlaneEndpoint</code> is not specified, the <code>advertiseAddress</code> + <code>bindPort</code> are used; in case the <code>controlPlaneEndpoint</code> is specified but without a TCP port,
<td> <p><code>localAPIEndpoint</code> represents the endpoint of the API server instance that's deployed on this control plane node. In HA setups, this differs from <code>ClusterConfiguration.controlPlaneEndpoint</code> <del> in the sense that <code>ontrolPlaneEndpoint</code> is the global endpoint for the cluster, which then </del> <ins> in the sense that <code>controlPlaneEndpoint</code> is the global endpoint for the cluster, which then </ins> loadbalances the requests to each individual API server. This configuration object lets you customize what IP/DNS name and port the local API server advertises it's accessible on. By default, kubeadm tries to auto-detect the IP of the default
Some Kubernetes features exclusively use cgroup v2 for enhanced resource management and isolation. For example, the <del> [MemoryQoS](/blog/2021/11/26/qos-memory-resources/) feature improves memory QoS </del> <ins> [MemoryQoS](/docs/concepts/workloads/pods/pod-qos/#memory-qos-with-cgroup-v2) feature improves memory QoS </ins> and relies on cgroup v2 primitives.
returned. In this case, the client will need to start from the beginning or omit the `limit` parameter. <del> For example, if there are 1,253 pods on the cluster and you wants to receive chunks </del> <ins> For example, if there are 1,253 pods on the cluster and you want to receive chunks </ins> of 500 pods at a time, request those chunks as follows: 1. List all of the pods on a cluster, retrieving up to 500 pods each time.
In Kubernetes v{{< skew currentVersion >}}, the value of `.spec.os.name` does not affect how the {{< glossary_tooltip text="kube-scheduler" term_id="kube-scheduler" >}} <del> picks a Pod to run a node. In any cluster where there is more than one operating system for </del> <ins> picks a node for the Pod to run on. In any cluster where there is more than one operating system for </ins> running nodes, you should set the [kubernetes.io/os](/docs/reference/labels-annotations-taints/#kubernetes-io-os) label correctly on each node, and define pods with a `nodeSelector` based on the operating system
* [Borg](https://research.google.com/pubs/pub43438.html) * [Marathon](https://mesosphere.github.io/marathon/docs/rest-api.html) * [Omega](https://research.google/pubs/pub41684/) <del> * [Tupperware](https://engineering.fb.com/data-center-engineering/tupperware/). </del>  No newline at end of file <ins> * [Tupperware](https://engineering.fb.com/data-center-engineering/tupperware/). </ins>