Project: Kubernetes (Orchestration & Management / Scheduling & Orchestration)
File: CHANGELOG-1.23.md

<!-- BEGIN MUNGE: GENERATED_TOC --> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - <!-- END MUNGE: GENERATED_TOC --> filename | sha512 hash -- | -- | 886292f279417ed9e63978c685df3c20ef677cd45344cce2c7dd3942a5ca1682a1c6743c4713df794b82152b059ceee437aded2eae0ab81a29320ad6eb0942d0 | f39f80416da9835ed912f04e54579ebd98b0b6e3c2cc1097abc1c98ec4d9b020f0f60ac733f323b38e6230e39ebf9d40a272885af2fbdf842a6932b051053ec5 filename | sha512 hash -- | -- | 5ce19fcaa82dc4794e804650835495aa49ee6edd9509536384804e9aebf29e34a152c077d25605abcac391dfcab6fa98c3b911bb8e7863193f8e62875836315c | f9fea145d182921eee9bc19c5fe9051d581dbec550a7ea645eeee942ddcb41f1427951754bedf5dcfc6ea4a2f5962542306312ca82f22548bb53991bc6cfc43f | 5947ef238998131c7441c25426fc42b5a7f19dfeac0a5d85f8eddc20b1a22a85b5b34b7100e2a9139f83f84108d23bdda2220e0976efe2e7e4ec6104efd4dad4 | 3b1ebf273e48809984cccd8f3a4c999e1f9f1b9a8f91637f4cdef29b0b74408cec3f68898c5c12ef5675f8e4d335100aba765730f0f4cf0d13ee75d28c571bf3 | 19b14ea8c14d894e5ed0fe7cb891f261ff6d81ac5bf6de96f4a516cb122ff27a6760372459c3c1f2513271c42093bd90634c1e47b8dfc2d13e66372c4638887a | 9585290aea3606edc2a9ba7ab3cd9dfcca20e48ea7e431ee59922015b5c05cc4c4b7aab321824a0944b111a98cbcaa9eca9998f224ddfce3158832f385490fa8 | 02a344446415d789e5a5091c4e47d6f6c0f93bbb490320cce984792b2870e9dd1ef2e006d380d28f4a2e8dfa8b1693b879d25ce1c7724bee43543131cb3d1722 | b976198ac6299eb683978921f82da27487b77c08ed6af6ab4d2c4c1e8487479911ed71c739fa64644e135b8dc2d15db0b0486a5cefe01f4daf5e4c5915088018 | 20ad752499a50c6ff008d9bb379bc0584f8b776a0cb09f20da44f692df9442b41471109bc0d1aab0ceb1b58e3713744fa100cb3e88a4a9f9c76e929d89496c80 | bf75fe667a8ddf19a8acbe9fd9e8ade9ff6ef8894eabe188fae43699f467fd3a7138f4024ab14f1f26cf7d03055cc214d93e8164a81a58183c1da310535fac00 | e30ed886c242c927723f11a48864d894b3890b22a8cbab1e1a952e259ad05fb1dd7b10cefe370f81316d18ad8e0cf418af6c66cc8ee6b7cd2a44da5afc94f48d filename | sha512 hash -- | -- | a9a4b132f5b5cce0b68bd55a63d3c71564cc28e9a122787eea743cb043063d74982d798be8407ee961df8063db57d7e61ca6137b7a93b18ff650e8f1d044d712 | 65e75a70bae70899abb4c1bf3edaccc74b873d8808f8f85a4b29754260934b3309116a2e1b7c3e571eb278fc5fb4de13a5cce5222bcd72f092471a26fa2d472f | ffb5b4de396ad1e51fc7bf30891f2d87d9735849c39aaf492a9e0bf3c6a91c953b897be80a0c312001937d32b8bb648b3001655a752ec88939413dacf4d32074 | eaecef5c67d88e0f9da832ad96153068addb4d9280f22d2a3c2080b0e27561eb4ec3a0bbb5febc2f86f0cb7132caf122e29fd7511d963fa6915443d4efe31573 | af876a0a893e340634bd4589da00aca445781046ed7b471f834341f8266e29daa9cddcc1cacb29f9a028dc972c3d1766fdfee2ac121b7359e349a4b188343242 filename | sha512 hash -- | -- | 3c123cc4e171e8755faa4a03a23234cb23024c9e29ce11914d40613191e9911c18264c944accb740bbbc976988dfae652a0829cb02460955fd33244b363a2fe1 | cfd08aafb09b0864b317726cb89230cdcb56806aa272ba8365c2dcf8d85b670d6beff0a11fdae7bca27fe2570f65845c6d14eda236f2d4d089e8152b062961bd | fb7664b11ea4b520b1d52664e17af8673c73b84431ffe4d6ec84e28b1a5cfcbc2348adb046eac9951b2abf4a2ead94f92e7bb81ea13d4d9fc646bc563154d22e | ba0774345de9427683f15efe6d7de535d47232483566346b6e4e9858fd2988eb47181e7b3383198668bc0836adeb6be73bc831abfe82874a46a0731c5eb7a69f | a8dafd03e094b8acd05ea91f5cb18bd735a97c04b7286fb69ecdc3e38f0c443a431920b89795f51850d3e8df13deae45754648e9e8df16ca3644223a069e7244 | fecc305f6047044dc3d50213922e0d317b45f210bb066e442cb6ce04a9096fe6d8377d98f2b9b4adabebfa29d58a6c7a835ab9ea6d496f98973416098225243d All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Kubelet TCP and HTTP probes are more effective using networking resources: conntrack entries, sockets, ... This is achieved by reducing the TIME-WAIT state of the connection to 1 second, instead of the defaults 60 seconds. This allows kubelet to free the socket, and free conntrack entry and ephemeral port associated. (, ) [SIG Network and Node] Kubernetes is now built with Go 1.19.6 (, ) [SIG Cloud Provider, Instrumentation, Release and Testing] Golang.org/x/net updates to v0.7.0 to fix CVE-2022-41723 (, ) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Security and Storage] Nothing has changed. golang.org/x/net: 1e63c2f v0.7.0 golang.org/x/sys: v0.3.0 v0.5.0 golang.org/x/term: v0.3.0 v0.5.0 golang.org/x/text: v0.5.0 v0.7.0 Nothing has changed. filename | sha512 hash -- | -- | f4e872ef6b315607428920907a9a55a9d7546d84ca6102b79ba107d642c9878f188cb3d4331428c4f15d59cbec65ab24f8038b65202e86acdd31fb65cbe7d2ab | edebfdbd83ba33ca46b242da35b7cd41ca57e4d218e7dbed3ad74df58cab8f2e9837f50031f9c8ab82806e55a1bed0edaffedf337c60f238d807987cc4e172b8 filename | sha512 hash -- | -- | 40cca087af79f67a260b93c692da6c6bbf5ab791d38ee1df33903ed8f817c0fd3bc5eb62adfe6b2f36cd209bc652b0746858201ebd7893ab50fad5bb375a3276 | 8bc8cc244ee5d0d578280cbb7c9e203294f5cf3a85bb42c2f98803841f2346009bb010347451cbfe1ca847fd4a89a60d7e2cc2ae43fb0401d50fa7503425be05 | d5b40c11113c4758e4f7d9b645dbde1668f2c1f048a88a283640b9416219438755f7943b1156be9a9c9b1f8a3025f95065495711e36e43d6d3809e4c13405904 | b91e9b9654cdc962390648d81d2099ee22ab8444efff18775fee7d88ddcd6b5b442ce978960211e1b2a179115c426ce1ce85fdebd4faf901695d301376f1361c | 8d33aba4ab24ccceb5b17fb29c6a4fe53b6986124fa95d9fed7a9e770ecf9987ed87e6d316e9e898f2cb588ffd9d534bcd654014a5188c49713773997d9f62f1 | 33c5d05b18d71095cc9e7f7c3cf0b81b3cafc1819327483d9d757da73539ee3aa20b53f9f33b4bdba299eab06ecc1fe6bcc00f231e005b1f80c5240b8c7d922f | 86c3b36db1b7ee491efa6201c64c307f0d01bc22668f9ff39bbc96ee2dde90d80142a4d7b7cc2b44891e2ab2574309d781a5c42c09a268de6815a12340a44a74 | 294ce4d00553d720d7a862b09a153209e5de39578d52e44b2af17c104fdbe78e78671d6ee078efc22a19df935342fd22b92109ed2ff83cd930cbb966547b3860 | 6c1c758fa377271a4a30c300dfa6cf24984b66e09a085f4b2e136bc368c1e1f10841cba3d5f3a4794c408a5e227227af632b8c162dccf895b835b210756a47ca | a407333fd7abff500f423ba8ee6b7bff4f9434943d630061d4aba9b5781c5a64ba6ecd72e0d527099247e53d6348a9fda220880d6d8890429c17ebf98f6d300e | a096749df3864ee2ea85179850b05b6afd53842390ac91161b3f14f3fceb0003a8efa5518acd52a751d206f97d2d0006a50dedb8a06caed3544d29a2811240f2 filename | sha512 hash -- | -- | b9ac9a70ac41969b9532f6b22225924214c7a7a015e459642818207974e95440bc3533d353f43bce80556e25729d2227b2ff0485ca0ee7b6fc040eac948db243 | ce898b7a658e75034738a4fbde47306df85f2fc10ba6c6d238316b786ed0553d30d37034956167ea6456da497fe203661dd4e297f8e4013b0eb0a5ef439e7fbf | 789abee922dd618ddb9a0189341f30196614e942c41572f593b4ba0c8afa962f63defae303c5ebf067c5e26c38f61d714b0778c901ea4d07dc36141be4d91e62 | d23d0946be48770e4bf644463fa4910f800f365231af25dd0929c9c1a8fee8d2613d7012451f697f36a905d4d63907408e03c24cd72df2648dbe2d37f8bdc4f2 | a6b29082c52a9ebd562132e6ad47075a4b49bdf7c98899a47997b62463764350247d506ad9de9fc8ef031b9f5d3af17bf93472cfc507ec4010b8c697af61035c filename | sha512 hash -- | -- | 3fed26409429c87bbd60da45ba72c67576f036da16242ef92567554180f18d1960a1a2dfb525ecba915ef926f00915bf29bcd3033369f96741ba917b95d34086 | 514201f6925573ec0fbfe0a2519025b4ff0fc4bdda01e36cec2345a92ef432b3b3fcd7fdcdf03e7809c3faa655f143d1694bbb8c0b86f7646024f24d9a8ba929 | 0e8ae7a52fbef727840cd8fa9d7ee324bf6fe7a129135aa0b2bc2dd36bae536070a61063490a2cb711319a4b2b9c06c2aefd6f0681a8752be64d30b1385a1e91 | 80941bcdd3ab9afd65a8200dfc0c8b0133b94ee40f88196385c6bbaebece3c32ea61b961ea99ed6389483077ca74072b86aefbe2d16e1ce9fa090b1f11e3718b | 5a514a3458aa1ae8676ae07c09cea882973a82f8bc15ad43d6485618ad6509afe78d109e73e82197bb20862bb6bde6914c9e48c9d8d9b2a87d246d3eba21ca69 | 711c264f937205cbe5b90903e6a18a4c7bbd0c5f7c179f635691822c78f494045f245ea215f1edb6369fe5509bb53d5be5a5f7c4644af4b85bd5ab348cdb2839 All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image

name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Kubernetes 1.23 is now built with go1.19.4. To match behavior of previous Kubernetes 1.23 patch releases: `kube-apiserver` defaults the GOGC setting to 63, to approximate go1.17 garbage collection memory performance in heavily loaded API servers `kube-apiserver` defaults the GODEBUG x509sha1 setting to `x509sha1=1` to match go1.17 support for sha1 certificates (, ) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scheduling, Storage and Testing] Kubernetes is now built with Go 1.19.5 (, ) [SIG Cloud Provider, Release and Testing] Client-go: fixes potential data races retrying requests using a custom io.Reader body; with this fix, only requests with no body or with string / , ) [SIG API Machinery] Do not include preemptor pod metadata in the event message (, ) [SIG Scheduling] Failed pods associated with a job with `parallelism = 1` are recreated by the job controller honoring exponential backoff delay again. However, for jobs with `parallelism > 1`, pods might be created without exponential backoff delay. (, ) [SIG Apps] Fixed StatefulSet to show the valid status even if the new replica creation fails. (, ) [SIG Apps and Testing] Kube-apiserver: bugfix DeleteCollection API fails if request body is non-empty (, ) [SIG API Machinery] Kubelet: make the image pull time more accurate in event (, ) [SIG Node] , ) [SIG Cloud Provider] Nothing has changed. github.com/yuin/goldmark: golang.org/x/crypto: 32db794 8634188 golang.org/x/mod: v0.4.2 86c51ed golang.org/x/net: 491a49a 1e63c2f golang.org/x/oauth2: 2bc19b1 d3ed0bb golang.org/x/sync: 036812b 886fb93 golang.org/x/sys: f4d4317 v0.3.0 golang.org/x/term: 6886f2d v0.3.0 golang.org/x/text: v0.3.7 v0.5.0 golang.org/x/time: 1f47c86 90d013b golang.org/x/tools: d4cc65f v0.1.12 sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.33 v0.0.35 Nothing has changed. filename | sha512 hash -- | -- | 75abc129db8c36f0e6c3f617bf07c404da1718f3e49ee02b61da1aca263063ed4513e344ea92507b85a418094fd1f8ac4b9d1e650bbea32024d0f134a8bbec9c | c2987ea018c02c9d74efcc3b1ea11737427a76dd4689ec8e1975951c50bb046a507d997dd02c85fa75ac8401d4212a90d848549955288f002117f169569593aa filename | sha512 hash -- | -- | 95ce601c970d1bbc14c59b5a90771c74461a32cbb62c688ee2fef29c17f6347f43439b2e16802857844dce4a6f380d07ecde5f0cfa99924cb4fceb9b67032bfd | 10ae6d54f21b5520b8e870b819ee382bb87c8bff4a1d8309a33d5e18f5407d6f1d18f2b0fea360007da786d12731d91c78bbefb33abdc697d16557304a795327 | 31d3ef04469842609c33f9abd55edfa5b6edb60f7f5eff2131b138f7504bbb56f45fd3c6ca7ff2362771f980a1089364b0bebb8d952f638762cf68ad58e71e05 | 7396f8f3597bf4ea97dc2788b415934385b829fa2c91f3f7458fedd543cf9123137be118a2de46fc0a5cd80378f77ebe97bb3c3dc44bd4fbd0cd921983694287 | c3189ca8c3ca36bc345a1339be4a142c801e62f0570c5847b4bcd2da612be31392c9fbebf3ca6d6340e8d8e5e83b9b600ee87f17b3611c84af159a6107da93c7 | 4650b7c2172b55e37f13d2eb596b37b50b33608e3017ca06cdc1e8ffb15ddf9cfc5c44a4740d285007b320d0bd37049da2b58cbb07495276d1cd01320615a617 | 26d46dd1db4b4bc0cdc00055c2e434015879de97ef8a2be9a01579e202ac1f2b966392544ca1c42d22e8ae2758f98cea1b9707835915a94e990bc65413f56453 | 0f385dd49aecb4493e059543b132ca4a9b5ac3b71002f3495bc37d390ba7866bb3a650fb26236a719f82d11edbbc6397737d58dead3dc29be282027d8b97a5a8 | 3286a73cfa3e39f75524f08990148b970bdba6120548f47e4e341ca3689dc0bf5c1ab8826611475d4df3f52c708ba81db5b0c594096045d2b7024aa139a4f61a | 3120763fd2708a2a8b0cb875bee6e4e466a1b97cad6f66faa7ce81235925e9240d1cce30286cfcc79ffc083845d41fabd1c1d242b5dd02ec3693e77ce9500ee1 | f6231faf75e31726c4e14c063469e50a249458dda29d159d1e93071be81cd14169bc757268291b5d4f5193e4de818afaa358b6c00d9e0f23ebf0b5a8c41c1018 filename | sha512 hash -- | -- | a8513eaaad75ea8a05d014187e5d6a2c58b52105296d3fb69620719973e6ff49f32bbbbeb594441f0e9a21c5ff18a81982aad9712a64dc3694f0dd06f1f5cf41 | 60e3ac978e6b0ea9d6fc5e4f580b996fdff2c134af0c6afb7400c9962bc83340e7338d190b49da6bbd41d3c0ab15e59e6dcdca6856f8f56e8ee8287714c4b280 | 287b7643b07cbe706e92cfb9078af48c58e1cb79188d1c08efe4e46b68b2ccbc8724f5eb0dbc0e1c56cea2ca9dfd5a9d2edb41e07ad46b3e2ff41fa1197834ac | f3a82d6ed388c104fd569314232de92f6b679d7f9a2e5b17a6acf49a1e4b3bc5e466a74cb1fe8e3ad0e5ea93ffe8624908364a380ff921fecb80aabcb80df66e | 10b5bc8b58f5b7ff15abdfc34aa59b09302134a32d4f30d8bcf5eb32df1569964d6e3b5c6b273f51580c31e406d50665e74104e32850cfd6f153629a8f789476 filename | sha512 hash -- | -- | e4bc3f661d5e944143bf7b02524ced414015207e48a0605f889d4ef982a87046b16dbb681fdc10a0cc1daa8ec966a85d7e844a2e10a0d0050fbada17e8ec0cfc | e4980a49f4c501f966a9c709c2de1e8fc58851de81ea7a8088a9784ef49d1efb93fb2496d3078af9cbeaf775e61ab0c5a2cfd77468aac88047e3bf52946bfe5a | 4989abc084f5441fc6261e9d9315dc49ec631b3274a3b65271df976c9562b2d59963531b95633a8cd3256cf8b8c4e8bdccb782508346961e7d463e1f4ecb6bee | 53619e15364e03a6c5367699a67f0acc92e1b5f0b971f35e3b752660d246369d5ae68d3c11bd400dcaedfc5906a04c684f7fdde8fbe461861c0054bde9dd7fb8 | 486a14bd1fd78672cbbbe440092b5a0726385de4ad10764a18d70516404563c54ba2cc769c0df5fb3243ba3d3ca1709d65623589d6b641218ed3cf861c59cf21 | 169e5df5a8170ea0f2d91840b44bcb8778afd69598295070437259a085dfa7d9d1470753826fbf82def2bd78daa5236cbf0e6f5479be3e31e9a20eb1a07b85d0 All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Kubeadm: use the image registry registry.k8s.io instead of k8s.gcr.io for new clusters. During upgrade, migrate users to registry.k8s.io if they were using the default of k8s.gcr.io. (, ) [SIG Cluster Lifecycle] Fix endpoint reconciler not being able to delete the apiserver lease on shutdown (, ) [SIG API Machinery] Fix for volume reconstruction of CSI ephemeral volumes (, ) [SIG Node, Storage and Testing] Fix performance issue when creating large objects using SSA with fully unspecified schemas (preserveUnknownFields). (, ) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Storage and Testing] Kubelet now defaults to pulling the pause image from registry.k8s.io (, ) [SIG Node] Nothing has changed. sigs.k8s.io/structured-merge-diff/v4: v4.2.1 v4.2.3 Nothing has

filename | sha512 hash -- | -- | b4ef60e57a8590e428b3b9a7b871ad3918c56bde47c215f51534c1698bf75fcd479b7c0fb8e43f5e4f647e37fa6a125c152b16ecd0378d85201118ff206c1679 | 923b56a0e45c9b58d8f42cc499191fd526805eb9ab1ebf6c5a0d37b4724f1a27a5be8297f5913b3cfeaa4bfed138f6073569d58a05b60d1396bd1888bc2448ea filename | sha512 hash -- | -- | 10af1dc225e95c1aedeeb0f9b955b6774bccf5dff4a3f242c26294d633b8e4ad593f1fd058cadf533d25b481ce7eb4442ad84f2c28373753650d834b9ffb0afb | 2974b5237b7f99728355dc3cffd12915f2e291a08800f5faca8a802824e4dbfc2b48387f26ed909404cf518977eb9f440ea5e64b253fb34deb7b36fd2606a135 | 026e0cd6076f8f2249a9829b82153762b8b918a61d72a292b7da45d24b8c1aed3007d084097b212df1e6d7e5d83581ebf8533eabae6eb376f16135899c28e677 | 02c650f38d79065543d2d0a003a5b8c9c668bc81b4abd086b0596e964ed16a2fdbf2d16dd84d239a17a8bf7f2ca7bcc66c5daf7b96a22e9f74c0f4e22eb2c46d | 21ea1e7fc0bc9a2ff31c06b95410047a74b66bcd88b27bf8aecbe90a17a001acb6770f6ef4fdb0599993c1eb5497bf3517b61f81b4ef645bbc168b6da0e38107 | 0c9011fef724067e4fa81d085ff1389c6681ddc9745d2a264c5a1c7173a6a24a6dcbcd76b10fbf9e8272c0d7ef0a53eb1b51648755e499234a32deacead26704 | 95201598d72abdc8582d125468b343966b588cd610d00adc93424424659791f83944581120495f87025f52fa6707ab111e4dc1ac14f0f6554aa2f863da83051c | 588d034bd4ba6d056cf320b1a85b3319a7824c15a37f38e0be1a1825263c043546619f39842fc1297b55881814a5207ecd02e50b0ff87745cb4af0a3b6883fc3 | 0fe4d9c2177b849fec0e315dcf4d5a578de60f3838f340645b1cd80a57c2545f4d17b825de8ae6d8a16e42b769e66321c490f51f82bfade5a1250980f382dc51 | fba076820775283a611be753b6842e6c764512fe49e648cc40666dc3d376d402479a4d00eb8c7e53fdd73609004ae2f525c2d315671b958209f3de9a0ddce447 | fca4a5c44d7dd9716720181fe09828ad83187c269573a67cc09e6b524db5a0e1b7d14f2d1257cced36aff82902262abb1294884e713ac2e044975c495f6db4d3 filename | sha512 hash -- | -- | f642668f7af84aea2817813ce40e12f416158e40284532ff8bc6dd721ee53ca714d1a88fbf14ddad4c4c3ca575cd196d5f6de044e41268ee7ab25aea749689dc | dc1fb1486b159e9856cac0c68c9e6ad70112de94c4c9cd8b37321a03e0a2185782704933087212a39358658f9a182674a8a349d420dfdbd548dda75b73ccf424 | 61616198b40bb64ab770501ddb06099cc65a9fb6cf762ec5cc7c1b31f5df69250d20228fd718398468672c6f33bf92f4aad84a0622d46b526c23c07a9027a1f6 | b12f73b3118478a792f0bc807138e4db5425b740d5bb92f51ea11140e120c92d304006e9b21bb945c3e38ef7000fac1da3c4c4884f0bf3034e38333d11b7e4f6 | c95a6a97a15181993bd463fb20e08be778ead6a55701aed37d5d69378d6de33ac2d79b98bac3ff861321b82bff495363d032dc999abe2b5a6d171fd5994c1b42 filename | sha512 hash -- | -- | c65e71c583a140a4789cdec4444ab6335ed6e49d808475d6704c1ba3dc67593e087092f1a0e99c08f692667d72ddab38d7bc39c152e384aae858685530ac6763 | e098e856536470b4d5b9cb7d3ea7340a23db76932defd0d9d1a1002ff856199850586d0ec98d6b0cf5bb3afe62ab188f75d5c6e8db96f73eff1175c22b68099c | abf8e6232853ff11599c1735af6d0dfb55645e14903c757ebd9dc7ee77d801b7932ea39e817a9ba33964107ef5829f603f6b3ad7c739887c5d3ae85b67a88c66 | 833733e3387859bc7abf4f217abc829095051ba116980673c57e3f24ebc4353a323d75b156b21cdb1c7c4b93ed7ef4c60cadfac5a327fae814b0f060e4330eea | 67f517e04dac3b672ed75802c0f4b547a11b23ef5306825f7e4120da9db8ad43d61621f6c098871de3a143bb516e5a7b3f6b12bdf09502b431a6163786b8c48b | d0ba80907465ca9e6c39510c13985c4be5d10dcc9e7fa984235959425faa8b5d19499a2265c99964b41fe2f8977d95eae26f7963925a39ad66c09a203cb63483 All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , This release contains changes that address the following vulnerabilities: A security issue was discovered in Kubernetes where users authorized to list or watch one type of namespaced custom resource cluster-wide can read custom resources of a different type in the same API group they are not authorized to read. Affected Versions: kube-apiserver v1.25.0 - v1.25.3 kube-apiserver v1.24.0 - v1.24.7 kube-apiserver v1.23.0 - v1.23.13 kube-apiserver v1.22.0 - v1.22.15 kube-apiserver <= v1.21.? Fixed Versions: kube-apiserver v1.25.4 kube-apiserver v1.24.8 kube-apiserver v1.23.14 kube-apiserver v1.22.16 This vulnerability was reported by Richard Turnbull of NCC Group as part of the Kubernetes Audit CVSS Rating: Medium (6.5) A security issue was discovered in Kubernetes where users may have access to secure endpoints in the control plane network. Kubernetes clusters are only affected if an untrusted user can to modify Node objects and send requests proxying through them. Kubernetes supports node proxying, which allows clients of kube-apiserver to access endpoints of a Kubelet to establish connections to Pods, retrieve container logs, and more. While Kubernetes already validates the proxying address for Nodes, a bug in kube-apiserver made it possible to bypass this validation. Bypassing this validation could allow authenticated requests destined for Nodes to be redirected to the API Server through its private network. The merged fix enforces validation against the proxying address for a Node. In some cases, the fix can break clients that depend on the `nodes/proxy` subresource, specifically if a kubelet advertises a localhost or link-local address to the Kubernetes control plane. Configuring an egress proxy for egress to the cluster network can also mitigate this vulnerability. Affected Versions: kube-apiserver v1.25.0 - v1.25.3 kube-apiserver v1.24.0 - v1.24.7 kube-apiserver v1.23.0 - v1.23.13 kube-apiserver v1.22.0 - v1.22.15 kube-apiserver <= v1.21.? Fixed Versions: kube-apiserver v1.25.4 kube-apiserver v1.24.8 kube-apiserver v1.23.14 kube-apiserver v1.22.16 This vulnerability was reported by Yuval Avrahami of Palo Alto Networks CVSS Rating: Medium (6.6) Make STS available replicas optional again, (, ) [SIG API Machinery and Apps] Make STS available replicas optional again. (, ) [SIG Apps] Protobuf serialization of metav1.MicroTime timestamps (used in `Lease` and `Event` API objects) has been corrected to truncate to microsecond precision, to match the documented behavior and JSON/YAML serialization. Any existing persisted data is truncated to microsecond when read from etcd. (, ) [SIG API Machinery] Consider only plugin directory and not entire kubelet root when cleaning up mounts (, ) [SIG Storage] Etcd: Update to v3.5.5 (, ) [SIG API Machinery, Cloud Provider, Cluster Lifecycle and Testing] Fixed a bug where a change in the `appProtocol` for a Service did not trigger a load balancer update. (, ) [SIG Cloud Provider and Network] Kube-proxy, will restart in case it detects that the Node assigned pod.Spec.PodCIDRs have changed (, ) [SIG Network] Kubelet no longer reports terminated container metrics from cAdvisor (, ) [SIG Node] Kubelet: fix GetAllocatableCPUs method in cpumanager (, ) [SIG Node] Pod logs using --timestamps are not broken up with timestamps anymore. (, ) [SIG Node] Nothing has changed. github.com/stretchr/objx: github.com/stretchr/testify: go.uber.org/goleak: v1.1.10 v1.2.0 gopkg.in/yaml.v3: 496545a v3.0.1 sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.30 v0.0.33 Nothing has

filename | sha512 hash -- | -- | 0995ad8ee7e2a8733db5287f9dedcfa565cc7782f83a633459718a155ae60b18dd8a6978e134abdecb05bdfd28c85744e402c53e54fd1ec7332f627b2e76c2fa | 0e3129af29a28c96528f4842caa9234dcd198d9b1712cdf262581a535fa851013ce31c6543014145bb984c8994673bc73f88bbc83ea7142bc92737b81df8bae3 filename | sha512 hash -- | -- | 1f94745e78d8ee50725d5145d4eac928a7ae481e51758ed259232013f245961a9a54f106b556330d710a7d8c45e7c13a304fec6f56f8c5ce8e959e204efb6b27 | 255843cdf86ba0a18b60d8dd5b515fa9a3225119875e9c6a3f7a128d1a0545d92c28abf8d1e2c02194f3f047417ade6752c791bfbc1469e566d305122522f60e | 4076aa1723fff8d1d9a455159994ee002855bec419c4e56d894ee95dbf527a222236bbd65e711aa967df185bdcd71d9ac3bd6dd0143cdc18c80a28ec09e675e2 | 5dee83bd9ef3c8ce3923c7f8fa3a6cbbaab81158ae065f0721117debdbd6412cb2f091eda9e9cbcc7637533c0f328228a9714f56de303bbe61c93b94398a91f7 | 2c5026cb1204e4a3a78280c837193bed7cd1287c5053ab83f3660762e18168c5b3896709d1a3f18cf66f92b359b258b1fcc4bc9acea651531bd7fca5fa991e9d | f0f9505bfdf6e9b89c995797cec522dedfec34b1880a52be042c81a797ddcf080e27f6af366944247ce5edc0757aaa9f67425840bedad276e9e18180c4363334 | ebd37318e83e42d8a1833598b6f31ee28d358987e5727e0bb0e9ce4e4de7c26c299a26beb4f857d852e04db569c92750f44061239493bcf040b403b41b3e865b | 037d6b51e4b0e7011048087940637b3653034ad22219c281b49ee76a2e7194b79502a9f4a299349772b68b3b256570a6ce7e44896746e761c606ef04e104453b | 1e938c7b44a4093d84b5370c51590105d64b5283c3543bcb579252733b062db613c5203fd336de621442963a1dd4f51bf4799289e98fe5e09f953065e503255f | 0f2ed66cc2c2471c3574320654b19e8894a7ee42e7e1cb9b579119abb3f9dc82087acf41b060627e3ddf0aaae6a0ed04cbbe740cbbef26eaa754cce715ae32de | 02032b45f01c2b2a1ebbe48aace0f44297c677f64e122c2933f0b66ecb0ca48c13795097cce9007efa6f70f2dbdac42b6a08219ecebe29404a728202ebdabddc filename | sha512 hash -- | -- | 59b8bb9a28799c2f07c961bceac6b600c443df30e0a5051950f784f031c69dd7deb46555a734696a262a95d7d1d0b5e1623440ca4017806896d06758a3a08d8b | 2061a03b768b9e24055dd7280e1695717c5eba32cecbdd8c96be0755f95907505106bbd0e24d8cb9098c5d9cd1aa6f70529550a411317cf749fd89457923b33f | b4ca99506f8efa6232e7a4a7d6d37973b317fb053f21b73a5bebe2f2e685cafca75a29345a67ffe41b631e5af9a7c5e2625e6113b0eebdd2781547728c976a70 | fb3d036fde212363bbcb5c8e04cca76eeea6ec90e3513a9f2f4cbef4655548b4e061143be9c6e0e17b2dc496ae1990847a11c948b7d72d9e9fd0555d7a4df5a7 | 62f09c403cb97ddb536a67ec4816846e9b7fa136faa8958831ec980d4755b2b88483292f705ca12c0c7ced0e3e3f0daa811835ff578c8d31b24c25389c09033e filename | sha512 hash -- | -- | 97167211801a737b52f766a2fd7b0afddacac22dbf4b03cbbeae734b2a7e1243f8e5d715c84f4f8dfa862c62892f01a42d2561c96386bf2283dd21e64bb91a72 | f1e75a62c8360296cd4bae498c46721e44f2219322f7361ca9d32a0026a5c9f797c99c3b34df2f985466aa11eaacda425ec6aab9f6368c426860efcd0496af5c | 86fb0f9cea876a7e8331ac82de1feebadca56accc3495c30c558b5851e4242a9c9fc8420c7e23c2db734b6ef8134cc0455dfb7a67798838e921c3120936386c9 | 997190a1d39a0a2684d65922ef583e72eff9fe5c77011f3d1edd837ee7ff4ce4aa1f898b9e2f1acba4d239b2f3bf33f874ace28105b2b51825b57f699309622d | 3210ae564dc1066c70ea2bfbb6b961290fea748db8169d3ac9a00cd3082b24edbebe0b5fe0fbb276f794360ff82c502209c7c155cb2d6e58eb82a95d5fd829cb | 2cdac85dc652cc0dfde4556343e96747aefae678b14324b4eef7adfc9a9bfc2d6e7b3c9801da3ccdf9d18a52282edab054c2f0e1f6a7a06e3845c9915100195f All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Azure: Skip "instance not found" error for LB backend address pools (, ) [SIG Cloud Provider] Fix list cost estimation in Priority and Fairness for list requests with metadata.name specified. (, ) [SIG API Machinery] Fixes an issue affecting log rotation with CRI runtimes where kubelet tries to compress Windows logs before closing file handles (, ) [SIG Node and Windows] For raw block CSI volumes on Kubernetes, kubelet was incorrectly calling CSI NodeStageVolume for every single "map" (i.e. raw block "mount") operation for a volume already attached to the node. This PR ensures it is only called once per volume per node. (, ) [SIG Storage] Make sure auto-mounted subpath mount source is already mounted (, ) [SIG Storage] Nothing has changed. Nothing has changed. Nothing has changed. filename | sha512 hash -- | -- | c6ef86c6510a6aaff54c36190b22161d910b6ae6d9a1f0304094112ead3ee25bdbf6349cff9d5f926b287b4f75aba7befb8898d31c87f1ba19a687183e4ada09 | 5736949e8ea5dbea4c7a3ff332d4d69dca6db558c4337f189fcb0e0d712e5bddc18d580d1b61f979acd4262369a22d9e6c6c5ca36e62e1d0f5a3f51225832e9c filename | sha512 hash -- | -- | eacd137e88441b56b827460eb949a01f72dbc975a62b941b3d8093d168c191f0c59a6eddc7d27165071ba827b87d1d6ae80c372f1b1bf3a3462fe55fe6946136 | 6c4ef41a6c324d28aef169b320552acd40d3cd64165c99b213b5834b64977cc6b2d153c50be576fc9d196303431b2a67cf53f0bb812ad7f50c06f0f6ec873abd | 1d045d962c97bc064eae749981f60ecab48c38966dd4fbb69dc1fb861d5112360b5deae6e5ddab586cf11c574f1f1ba16c00d98557dc835a8d0521ac79265354 | 55478ab38a23ffdeca0b4e9a45f75431d8bf18f6ad08f7f2a1151be2b92479abab590e526ccbc7196f737d52b85382a01c7b8fae0af186c5a6ce4baf2e4e03f4 | c80b390cbd3039b90ac956268eac321448264b3ab8e753e6651933b7a26bfb4c0f50717f2ca3d8298b175165bb56133c5d7305c34bfa7838e4c029cb05868168 | 1abda0aa71ecddd6a4128bf1985070490cb83d16d2c5ebed257c312a764bccd71f54ea67a21c728a4cf8675ff5dfb8525709ab4e553aaca9ddcd448c26265a31 | 5747070f19780cfe1b2fc9b38f004ce542136cf637cd4ab70de57ae87c6576f5d5efe58d4d8fe6870a224f26775f0865760b03cf06773c9c11d6503bbfa5ee64 | fff1e69c75b64be5269afc1f801d8bf7e33343f386c9d932c4d96ec3271040793df0ce185260429c45f6cf9fbc025a2604cbe12ce01c9fe917a1684ad32398ce | 02851aaf61e8ffc2834d9baf825b7411ff25663a94778029e629142d296da97d1ab42e4deb6e12bd23f0bc09313ae477b192491d0d369d9136ad03c8f690d4ff | 95d47f3888f803a88f759c01f25d9b410a8534f745c79f925337b1d636734de71822859a427f1b5bc0d361ea391d397132031951602c984f5d4d56c61da483cf | ae333672d0caee1b58d71514e4734ee7a2ef8851d2fa674c6af6947c7caadace3ab1d12d4852f377ac2e23a0ec95c28fb95b1e46601aa88f5e1ead2d9e19a1dc filename | sha512 hash -- | -- | 6e80857d208a72387f66b548a067268042b465712df6c2babc4fc5e2ac9349a3754a32cc44af5c244d23e34b0ceb9c973fe9a59fe51f558bb786c993a03a8a47 | 6dc1d09f58e4c615eea96e754449932ddffd2deac540dc0c1dc657fa1650dddce5c7364ca67b68288afa1176a76ead53baecf95abd4dbcaff93e8e4b09c1a0c2 | 506303e92dca80821259a1842131b536cacafd0fe63d48bab0348c4fb6f5ca86456605bfb30a0f2993b7ab4a5d48b25eb1dabf8cc5eb4be91743343808db833c | 0aad929cdd11e2494baa46bd69256086281e98f5f76beb50b21f3494b3a24da607a6f1bd87fa960c36e6ff2ffd39a23a7cf342bffa337b12ca492c1c6db5b61c | 1a02255daddfd3c510f27bb480062493d8b0336136ec52e74b130362d7abe006193219cba115ec8296fc79f80a2c544e0fcc8a655cd7f38c3736af1f28b1c1e5 filename | sha512 hash -- | -- | e35f9f226a214ee977deb467e60f6724cb626eed6a806c2e1c707f23f8bd6aac59d315dd3dab6d706889c4fb5f48af5919bfc606e01e6281ffa8c63b29c8bd62 | b808f29dbfeb825adc0eaf799e933286c47f0978d642b240737bfd9cbb36cb40f0499a31d3b7c2201f8e1cbafd29f4d5edda02c79df1952b0079354839532ff1 | 3f7e085d8b0ed71128752beb2422438256443329a9a60812657149751b6a7e68382701472f2994fa23d28d7d00be3de7991080b9928576a72aefe0dcc29644aa | 4e4c00160811404bba2c3581cc61351ab79e90d697c886ca5b0215eec54353c474915c2b4274b48326f563b14c01ef5c50249a921335b5171c4d7582eff1658c | 701c8d8bf175cd89b670082d88db186d85c9cb68774c297e8dc3ed576475562f02e8e2bce7cb440d35d274139762d1814045b35f021e805bf0d1ad3ecbee50d9 | 3f8f84063803add6281b5afe883c4b7ff594111dea89156a4fd9ad66f8e9e6c089cf167f3f57e4903d58d7c5b18f7c4ad5684a590c66581fbabfba6ef44232aa All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Kube-apiserver: gzip compression switched from level 4 to level 1 to improve large list call latencies in exchange for higher network bandwidth usage (10-50% higher). This increases the headroom before very large unpaged list calls exceed request timeout limits. (, ) [SIG API Machinery] Kube-apiserver: resolved a regression that treated `304 Not Modified` responses from aggregated API servers as internal errors (, ) [SIG API Machinery] Kubeadm: allow RSA and ECDSA format keys in preflight check (, ) [SIG Cluster Lifecycle] Nothing has changed. Nothing has changed. Nothing has changed. filename | sha512 hash -- | -- | 90fb263651c00d4b801feeed82446e0bfd69a42e804fdc98a08cf83c8f6fab24712d88b812bc8f70f6ffd0d33fc03099e889ec1cb141ef423c1455a59905a51a | e25ba52f86f8fc1afc88e4ed814a75761b2ba85f5df27c96a4fca5c26358dee8e53cde914197d2884e69155f5b45bb09e7d1a1c340f35ba44e15bc958bf5cea3 filename | sha512 hash -- | -- | 9991740791a6ff7f1ca7de7869cdc3d9c3ba24a2ed57ed38213a17d714954db3d746b6ca004edd8ec869ead0423716b52fd97c24177a2284ff293fbdc2c50913 | d986a0ade1a9ef3200319d1b7b5445a7980e54e9febe7d3ad35d3b1fb3c5c1bd04e6d69727de7d1303cda9ac8b47f154270d63edf323d9a9065b0d4f0deda8bb | 291bd7e7c63618d48a58f17efc07f815d554efc61167a908f44e494d58928d408bc1772c1c231d4ecb41a73e1a2fa34e836691ced24ff16b5614c621dddc3bca | 7f4e290d5057b5d8376690d6ffa79e45217228df94fb23061b1be0adc71a9a573009058654dba6b58c5deaa457080e3a31121ea860e47f480f1a4c042e55f979 | c625bc6d1c1f7a454116fc7397ec7a65674aff022166eb06319e45c4bfd1297d1a218e7581bea180201e6bc6232cf6110c5a211201517108cd3164151869eac8 | eb3c01b02b5ddc5d8f40ec401cdb81916711bed8ec49578ad1ff32f994638c9f5786032d24107b41879a4242415685174e3c94d0ee7496087b64c12f654e66e7 | e3f2993fca11589eaf27d1ee2ba513dd349e577677fc3d8b357361ecb3fa1a99e911865ff88caccaa75c2340e3535bbcaee20592aedbe53da540e79c2e2ab845 | 63a3dcd8f24754df3c5a2f5c970ce22d75753d027ac32da1c5a9d6bf60033e9093bf233df4d7594536e3419a6673cbf8ee361896adbea8078e895bd37254f74f | 1ff486a460ce6d5526c7f73f8fb6e87525c139aaf935787b6decaa00095dd1e49a5ec1126007926f737394c57b19f10de3480c9c513d728ee677f688c8998066 | 9406aece47869024872b912d3ace880e016491ec709e28dc0e5019999782a93ec736b8af8cc453f5b5bb92cc801492056f60494be4a4d75314a73eb9dccd261c | 9e8493a3fca2d1003092a9d8c078543dfa4e6492c98daf6b84c7ec7cb7bf8e5bdc88083f815a42903c1c2cdc8a0ef4c2220aa217c330597ca1e9c8171314846f filename | sha512 hash -- | -- | 767eec5ae28f8c0151379236e3f46870573964af4bf68ff38da28fc900c057d521bd6e2ca79458a95acdc6062069928f5a246ad69aff2e751f772ae9baec548a | 4cbba22196d0002494d785f03138a6b1f747c4d00c686c9a52b61fe73e57fb28ac9314edd44d6136b3c82a457b5059f267c7dfc0270e55c7ff3ed8305cee850b | d69a5a95a05532b19cef65a020e6b8594f63b5d0fa2d29768c3c633a4170ac69f2b41e98aba1a4039b5f500d7719ad84f76010e229b296fd89ded9ac21298735 | 9c9ba9a27567b5ca253bee66a65e978783ce02a56d13312e933ce829dba1c7d6c00648e552ea647dccbd1f858fb0d90842a7acabecea47be8248f3e46af5d043 | b0b32c62216b18a8456788c6763eeac2c443e7c3781b29fcb356025d1c836c248c3f157ff65a94412b3b02306196259411e3f2e49a888b5f233d176b9e69541b filename | sha512 hash -- | -- | aace59e2477138f50799f5d536ae2370b1cb76425f0411d5022fc9ee438715be6ce27b8e6c400ede704ed2d6b599d13fb12b15eed272a4c73cec60ff08857747 | 044216a04fb7c995038d1f36237fb268396621aabbee7a9c0ac492ab6e70f697b80ff833b1a4aad7f32cdbd35dd90e6931028747c73e04adec4eb786a658140c | b31cdddeaffec6618cb1826b28f73e657bbd6b272801e4d0399ef6a5db1ea0afcd9fea050ca0a6aaa2714eeda573d0cf9441c462f0492777f07ac94d46aef8b9 | bdb3ddde36fc29374d5f16bf7ee078aab94f20b0b4b356eb6ceba5088d5de694f26acf79c82a7758ed0976ae754a4a7b6122e31cb0cf66a0cf3a3d1e971bc88c | 170f42796052cb451793c42757e3e8fe69d200e20372edcfaecd42403b8a0efd7571c2456d6a21bc2c3a35c7b0c6ebcab6f83d21a772b6522a36f3d4b1c8d0ab | 3bb5daf6403bcc88a32470feac9a248d2a0a39f21375f5b0cc5fe081062061872d4269b4d2d977eb74bf6259cc1908e1bd138bb7eeb95d373e337f85979da1b0 All container images are available as manifest lists and support the described

It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , This release contains changes that address the following vulnerabilities: A security issue was discovered in kube-apiserver that could allow an attacker controlled aggregated API server to redirect client traffic to any URL. This could lead to the client performing unexpected actions as well as leaking the client's credentials to third parties. There is no mitigation from this issue. Cluster admins should take care to secure aggregated API servers and should not grant access to mutate `APIService`s to untrusted parties. Affected Versions: kube-apiserver v1.25.0 kube-apiserver v1.24.0 - v1.24.4 kube-apiserver v1.23.0 - v1.23.10 kube-apiserver v1.22.0 - v1.22.14 kube-apiserver <= v1.21.? Fixed Versions: kube-apiserver v1.25.1 kube-apiserver v1.24.5 kube-apiserver v1.23.11 kube-apiserver v1.22.14 This vulnerability was reported by Nicolas Joly & Weinong Wang from Microsoft CVSS Rating: Medium (5.1) A security issue was discovered in Kubernetes that could allow Windows workloads to run as `ContainerAdministrator` even when those workloads set the `runAsNonRoot` option to `true `. This issue has been rated low and assigned CVE-2021-25749 Am I vulnerable? All Kubernetes clusters with following versions, running Windows workloads with `runAsNonRoot` are impacted Affected Versions: kubelet v1.20 - v1.21 kubelet v1.22.0 - v1.22.13 kubelet v1.23.0 - v1.23.10 kubelet v1.24.0 - v1.24.4 How do I mitigate this vulnerability? There are no known mitigations to this vulnerability. Fixed Versions: kubelet v1.22.14 kubelet v1.23.11 kubelet v1.24.5 kubelet v1.25.0 To upgrade, refer to this documentation For core Kubernetes: https://kubernetes.io/docs/tasks/administer-cluster/cluster-management/#upgrading-a-cluster Detection: Kubernetes Audit logs may indicate if the user name was misspelled to bypass the restriction placed on which user is a pod allowed to run as. If you find evidence that this vulnerability has been exploited, please contact security@kubernetes.io Additional Details: See the GitHub issue for more details: https://github.com/kubernetes/kubernetes/issues/112192 Acknowledgements: This vulnerability was reported and fixed by Mark Rosetti (@marosset) CVSS Rating: Low (3.4) Fix an ephemeral port exhaustion bug caused by improper connection management that occurred when a large number of objects were handled by kubectl while exec auth was in use. (, ) [SIG API Machinery and Auth] Fix problem in updating VolumeAttached in node status (, ) [SIG Apps] Kube-apiserver: redirect responses are no longer returned from backends by default. Set `--aggregator-reject-forwarding-redirect=false` to continue forwarding redirect responses. (, ) [SIG API Machinery] UserName check for 'ContainerAdministrator' is now case-insensitive if runAsNonRoot is set to true on Windows. (, ) [SIG Node, Testing and Windows] Nothing has changed. Nothing has changed. Nothing has changed. filename | sha512 hash -- | -- | 0e986db4a54b5d07773b17fae599680bf221c87d3c7ec185feba7aa23fe990f999af6ea28d8f5631b30f797104cbca8ac9a15bc029452d48b2eb9c6536c8abb3 | 4e44687dc3594ede040199eea620ca330c41d323c70a69e1cf0d4b9ead28afca5b477d55e89eaf9ee05462bdda1aecb6c2382e2fdc0d156c1177d5f703e7c2ad filename | sha512 hash -- | -- | 90d6164c0ae119eb187c7d878ea41fc825f3d9cdf466bba3e0758613fce67736131f28fc16ca9c26754b4d117c5ffa436e144b59df84da753b6066c67f92c453 | 72fb1c734a6f0392ac98db78b365c0816fdf3c995fbfce35bd46034959c8cb5939b09affdcbc8e57866f754866456646c84dccc7a098909bfd57ec559329e64c | 59dcae5a3cdf9e240017cba82ec07fa97ad5fbd8a17d09388aabc38529027fb4c9c67c55741157cc888e514929bcd5277fda5fe7eedfe7d63c3374b62dcb0e0d | fae98e5e61dbf869428d14e05a2daa60d221a41688faf016157c5a09b400ed2a4a49c90681fea0bbacc75e0e158f75da62d026a6fedd37f9e52d56e3112f25c9 | 681728f0ab837b1fb679e881fb0c9e728dc93bdfff9dbe3e333234166192d79b907a991d21e66a25ac56b16bc446332cb4bdc9891bb5334826f07c159617ab49 | 58339517a0a89204f357354173df493bea41149a100dd91f73f290f19ad8903b2e7733bf1c2acd21be9266b50a55f1dd0fc039af1f42ef98dc10ce0f86185ab3 | a3c703613749e61f4bb618fce2091fc5c10eb85fc4507838ca0c79765139948cfbc46ee60a2218d95863e0b2ff3369650a4ce0a59fdfd548a921fe6a3cea54f4 | 8f4d343205b3d2b0ba61f1639e76edf7934da3c7316095af510c9adc9a3a623065b4a62da4bbfe6ea95809020143751cf21ba774ba6b5ece4dd2de11efeefb70 | 292f13a099c04dd74af026f67a13d5f7ff85f215e2b5fbb784f07e2408ddf2f0041cac4205043415a5e8fdf4183e9589c8c68bdef90e426df0747f743fb3123b | f89a6ecef52164f5f77274ea4240cb5000a9a3b200af85187d7194850ff2e5b8fc89fd41ad4311452ed80c8785b2a5f8cafe3c3de727eb68aa1a0b688f198647 | 16893e512a991180ca8e0d7c052e74b5e35cffb3a37ad92623ff376e2fc9d4b77edcdb6d7fb147c4de00a7b419ec04043aeb4f33a1e7114179885494c36c2948 filename | sha512 hash -- | -- | d587c2106ef717e6c32d4480ff9cab768a4c58c40b65c8da61446b18aa071ac41a09d5d854e1577f3550b2df84fdf5685a3b9f5766acb9eaf51e2ae587af3b08 | 244d57c42709c3f45aba508eb800ff228d091d474b8985689d455f916bbf15a93c081fdc6c1e1dfc7fac1905ea411cef085b88ebde41e0296d15146d2b49490b | 4a512a1440d0cbdc2dd6a4e43e4aa6dd23835be576b6aa58b4cc9c5c502f607d3a7aebefbbedaf55c5778f3acf102d5c19a8822203e9638b3bd4752d23a46d38 | 1198ab579e87585f335c310fae173d344897fdc0a503c560612d7961b0cb737cced2a1a9408b144b15eeddbb37298dc997a75c984242e5b97b7f58009d0fc71a | 3f655435f01106d64de55a2c688e086456ec2265c6719ab0c80064c49666c9b27b7664e69d76df32540873f78480d6f8751566c018d06c4018aa08f51c9e15c3 filename | sha512 hash -- | -- | fc0a7e46e0b3611f7c3e3d50af377ab24b154590c50d75bf17572df71c5fb5fa2de50db246dba114f0714e8f0b5e7da70f01065bda6a9be2ada958af61d9c0a9 | 6b00d621d862899aa858ee838b3767bf094b36ff44cd9f414b5d429a40f26f410942f4325652908fd92dd1117208a1163d246a309a6b3344750f6dece847ca9e | 4fa47526aab15142e88d3954d68299583427511154076bcc849826671185621b9f2ddc9fdd31f958f5a257234d9def2f0795497cf4fdfaf5b6901204f1044862 | 56522b8eecdc8a85e337dab05957277fe1f086e7029d649ce234423d014ad48557d4aba160fc48f0df72c108a43b0862f7fea24ebd7e80a643a2e4a041181de1 | 3bded271505dc71328ba9e16917929e0b83c9a89615fc4683bf5dde565a9e4c5f54e85618bfb70457ecf3218317754efe2d599cc46efb9f2ffa6ea355c6336ec | 5408c4da6e6866cc8a962b43e1f91981d7b95e530c3783d4f9849b9f848237569f061e8739f7cc134e6e5807627cc46e52c9a38ce9908eeacf319d09df7b738b All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Kubernetes is now built with Golang 1.17.12 (, ) [SIG Cloud Provider, Instrumentation, Release and Testing] Kubernetes is now built with Golang

(, ) [SIG Cloud Provider, Instrumentation, Release and Testing] Fix JobTrackingWithFinalizers when a pod succeeds after the job is considered failed, which led to API conflicts that blocked finishing the job. (, ) [SIG Apps and Testing] Fix memory leak in the job controller related to JobTrackingWithFinalizers (, ) [SIG Apps] Fix memory leak on kube-scheduler preemption (, ) [SIG Scheduling] Fixing issue on Windows nodes where HostProcess containers may not be created as expected. (, ) [SIG Node and Windows] If the parent directory of the file specified in the `--audit-log-path` argument does not exist, Kubernetes now creates it. (, ) [SIG Auth] Reduce API server memory when many CRDs are loaded by sharing a single etcd3 client logger across all clients (, ) [SIG API Machinery] Updating kubelet permissions check for Windows nodes to see if process is elevated instead of checking if process owner is in Administrators group (, ) [SIG Node and Windows] Nothing has changed. Nothing has changed. Nothing has changed. filename | sha512 hash -- | -- | 78d2929a7582e3b875bf00f921c580de42538aa05dd27c9cc16b64109c504aa11961408e07363d77b8ea0177ff17ebc4b37dc729dff32dddb713e864425d7494 | 91e861a195735695a41035a694863b0483b7f4f7956b719c07ee0b31dd1a2f16b3e5e91643c0e35ad3f5ddedadabc307a728ede03ca94c901698353c65622431 filename | sha512 hash -- | -- | 1ef3d70708efc694932ba94b860044dcb0ad3e1513f25f88140f8391faa142a2160b35e479feee5612d6ff3f156dae8d9fd4b23e3f35471fe4932fb64ca8a188 | ad087e869b0b9b79fb8a0f31ecb4145ee7fac338ad73dd3f64c904518ef04bc0e8a7f3b7ef403c1f3633980b368e2c9739dc7797ce2aa99df47d58766f169ee2 | c8c1111988d918f385f97ba1563369d33fef8b52204f621919e61980be0e6ee79242face11b46841f17dac8a0d5164bcf54de3668655debd90b831be0b7033df | 3128800d337cc9be19494bcc724b9ea112444be6194136ef0e49b969394522533262a3f92554cf3543ae9b1e0bf063de7694f73ba230154bf2e59b74599f40ee | b5e125472055b35344b2ccfe34ecef545d8e3501a150479a1adb9c9218a96011de3ef36c9bfbb352ded41fdbcbd460bf4c600648fb586fec3bf9685eaac405b8 | 0bdde0bca2208de991ff3196ca73886f7129cd8037608ca8079973fe73d2742adb162fb693db6f30e0f4e4d15f7591e19e95ac1ce70d02cae77ded8e746f7b35 | 325a42eefc77305015ae23d61185ab728055aa622576c79ef5264120d0c00e118a2fe9c1bcbcc3250ee0088f380b9ea7f81755331ad552fe16e841fee55072e9 | 7586d553bee0a49226e973f0851b8f03394154849904e4b88f632d80924906c02321d78ba3ddfbd2be1c6328b7675a568fbc8ca8973bacfb6d11b6252d996479 | 68eb204d5bb4411d63c87838088db05d97349c10e6fae7f2e9f55ee934a347246f305045cd756195f20702937ab9c5082adaa84264f72841d81dbd8cbbcef6c2 | e21c10e5fe4f1b39f3c69e5c48030661ffc8e5fdfb20c8274ea61ad4adb4bccca557fa403a6250aa5ce17daeec11c330f9f29153748f5114b7275c2e87e10693 | 142a19090b3a169fc43982bd8b9a5d1d67855e27fe9ce2a9e863d83ba948d2e880c3b2ee3561a5d821f27dbdbc2b15af3eb2152d3244f1dff2829cf83b48e793 filename | sha512 hash -- | -- | 90d5ac43e6e2794ade95b74328f68d1b87a6034b3ccfbb73ca8fadf14b2a98fd1700fe2790b4c963375516972505774e30c13a7cd4f10099f80dc4dafd357c0c | 5dad2818d5457572afd0a8b6c74004d9c36c94614601c237c72585eca214330fb3aefdf3cb778c17cf0fa3a062f6d9416b11ae32229479ed0ae603c148a3f28b | 2cb1ae6078250a26dd405ad98d9440cc390f87b56543497299c0d51338cb563c38c57308fbea3e96f8c2ef56102d48b8bdd4ae6c22093ae884edebe14b0f766c | f1905fa13f308aabe453a35bdc2d194d6a55e5db221e61ddc3436a628d850f708da19240801210ae4804c8786d82d3005d0f252dc969bedeccfc5141a409fb68 | e05f415e64424fb5c2ce17dc6008c9340e8a69964cc397a542286ed2a5970c19d415cfa13b2122958411b24893cbb3f817a627a1ac7f79fefa39ee0de204fed6 filename | sha512 hash -- | -- | 44140e5ebe0441e4cf6a155250b08c6e321cd32208ac672ff46af97c01b40de42142ca019a892453f024db7894acb77e0421c3fc6f6058243d6c0ae55d212cd3 | 797905b2ae3b07934ef6a3c546e1343d82ad3f45555d9963a1c4997c9bfaca79226b6825db8b36078289ec350864d4759dbe1e84548ebdc1a91439c9f0e4fdc1 | 4e9487951ed430e307686500ff44d371aac184d02b453495f6b643f24943d347739a9441322e94173afdb801ca47954c99d8e2744c7bbc6707b79a9733fb3903 | a1d74ef1d83aa1b095537a729acc61995ca1b18467febcc5ccf25399579d69aea05b0bdcab3c97e04a172edc61f54fca2ccebfbe9461a72079fbb800095375e6 | 4b4c5441f5d3d9ac7347a4da2db5e453986a2e0cb53cd17b77dfca57edb543ec6a96b7a9ca2e3732063dffe84cb1873c883dcc0562db336b367f8c902ee7f9b5 | 4710f1b88761e735f59ca4828c4e60fb9a125a86dca826355d10c6c14e9d195a300693e1a26c86b75fd9790ccf4fa803bb77969f20a64e096037b7a3dbb02294 All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Fix a bug that caused the wrong result length when using --chunk-size and --selector together (, ) [SIG API Machinery and Testing] Fix bug that prevented the job controller from enforcing activeDeadlineSeconds when set (, ) [SIG Apps] Fix image pulling failure when IMDS is unavailable in kubelet startup (, ) [SIG Cloud Provider] Fix printing resources with int64 fields (, ) [SIG API Machinery] Fixed a regression introduced in 1.23.0 where Azure load balancers were not kept up to date with the state of cluster nodes. In particular, nodes that are not in the ready state and are not newly created (i.e. not having the `node.cloudprovider.kubernetes.io/uninitialized` taint) now get removed from Azure load balancers. (, ) [SIG Cloud Provider] Fixed potential scheduler crash when scheduling with unsatisfied nodes in PodTopologySpread. (, ) [SIG Scheduling] Kubeadm: fix the bug that configurable KubernetesVersion not respected during kubeadm join (, ) [SIG Cluster Lifecycle] Reduced time taken to sync proxy rules on Windows kube-proxy with kernelspace mode (, ) [SIG Network and Windows] Updated cAdvisor to v0.43.1 to pick up a kubelet fix where network metrics can be missing in some cases when used with containerd (, ) [SIG Node] Nothing has changed. github.com/google/cadvisor: Nothing has

filename | sha512 hash -- | -- | de69d27776c066cdc450b65d075fff9efe39175823aa3cd65cf8e431ba397c38add39ef53fc4d21b74c900d9bb5ed055a0f05d7a25a429c2f1f3967c2c5ff01c | 25386d6c7c0fc9829bc178c8a24b3d0fb65895495c0c3f25e6a3072728ee39e4b55855e1587a6dbf2fe75bd8a192d908dde7843c118f00fd71c9f76b14a9180a filename | sha512 hash -- | -- | e4a95fd15f417b51baf73f096a940adbe4426609f0fde2077c93b50e7b0cd5da8086e753c47112bc71575b00f8efaa030f92abc5f100a7113b9375ba4b9f58e9 | 4296920ddfea2fee14b680a231a72688771030387112f51f502c52662b3c72f51d254d579ca0f24ed6d6327056f1b047e52d43f2ff2bb6bf4267b3b30f1cbdbc | d600fb12e7fa00374875efb6d2ede65790cb64b149bca6db2eecdba1fc8845a0a46126041cef3bb3489180ce2825ed116257098f6009912635f1bb6560977c21 | 071de5987abc6295c3f9843a31b3fc0bea106f1ca3498bf089f867c3914a20a8fc563fc0763640c589691fbd9ab7dc4280d8f8de2b2904bde13f8c4f059580c9 | 4677affdf764a5983ab64af3876c120919ce319e0eb67f922439a93f13a3f62de8b81c652f9d1947572c1f051f49a54157717d6a1416b49cde7f76c49b8e342e | 3ac6eb2dd77a71ea608777429a8efa0c26aa3bf835f125a114c3d2312337a00c5b96eaf32edaef319c055be023d9862ff6f7fc06c0056d4b2491eaafe34315b0 | 233efa4ace4c9ac36b9eae75c32d477adc01c0690ecc069b4daecbbe2aabb50514255ef659bc9a2ffd2ceeeadd33965aa743299bf02814d200b11dfd2061d170 | 841d6a53cae230bcdf58e65ab6d780737c41c99f0144443905caad62b508b94f738640e2fb7a43ea39bb82387ba537389fb86d555572de1eb6adcf8d80fe31b4 | eefcb9290ac80c6d0e50fa77d4fabd8b71949412c37272ad33468bc31251d031b877cb26e17ee56d51616347331f5f9d17fa170a44d1f0f9530be9c3605981dc | 30b82098d3dc3fbc045174dddc273ba38ade9a64e6fb5a4a70646343752ed882f7342bc78c70b6af02fe9b42a20ef84763ea89418aa98618a19c39945aa34eb3 | 76c7f0e5ae9158d52a3bf76e638fc750bda8a4f207a9cfe3d99f2f1035b85d4feb8443de01d2daae2bf2414814286929b523341d896c144b6383b7a98b482cf1 filename | sha512 hash -- | -- | ca0c4023c1bbeb1aafafb073348e42a016f6793b64a84b33f9619971950ee0698cc6f008418c707cf5126ca82bff936508e63e02ecf89f38997d2aeb2f5facf0 | e33be64370b1418aa3ef935f1a01e4aff3285c0f3dbd8b0af1be60af5bb245f291671b557cda6c41e4af82da49b42bbe1562e568011099195e24f8add036227d | 86350da9262bdad99d997bd4a6f1f20067a77e2a8d0b4359d7681dfe0a272b42fa30bc4de47920aa8aadcfb5ab0d9b626c8fe3c268db0ff9c35d0f32f94e8cb0 | d0954bd675eae099195f8986a9c2ada9eadf804794f8bd91371c53423173398457987ef9ca73664f097f8301486f6416f31a7540bbbcbc844fe3ca7fb58529ee | 017d4026ff30bb10460d22571a592c50f6c9496b49acecb8a333174b33a8a98cd3f234d49546d45947d9af1c3ab5a1e337b42015968e9572b482bf7a4a58d181 filename | sha512 hash -- | -- | 3003ba3faa6d175c2e0b898b3a07b3bb22e6509a9fc7948582d92009a648b83bbe329a8c2f843a466f94089fd6a26529c89abef8c769f406a55f39a09bc55017 | 3938d4d33cbf021108ea1dc03534286c58ad68ee059027a239548cd2bce3708eb288f6893368014f885b20ebe26c4d556d27cf8fdf29bf478e5ed23bdbfe5325 | badf58767eae4a20f81200f609fd5966479b455a3bd9d3833d5cbb0e78c31e6d624c7301cc366b5b07da6299d2d5e0228db2aaf78c43e947a3a80a4db210965b | 301c234325e9a588315ff128469c49b64f471c6a40ce0567b745f8315bbe8847b8ba9e6e098c317bf20f0613e65e3d604afba12edcb84154e636427139344cc5 | f862838e1f7294de4b58359f0ff4e4644ff6229f83bd615f35aa5b625a1754a4c6e538fb2f911c13ed434fd81d2e2feefdbd4fd0148149ea24d3f6df8c01a7ec | cecb68bad4aea5790afc72c895d20d93bac53985437dbddb0917c7b9b8f876831758a66595c70fd8c142140a2eecc8d2fc2a52885ca4d062ea34586a2871f133 All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Kubernetes is now built with Golang 1.17.11 (, ) [SIG Cloud Provider, Instrumentation, Release and Testing] EndpointSlices marked for deletion are now ignored during reconciliation. (, ) [SIG Apps and Network] Fixed a kubelet issue that could result in invalid pod status updates to be sent to the api-server where pods would be reported in a terminal phase but also report a ready condition of true in some cases. (, ) [SIG Node and Testing] Pods will now post their readiness during termination. (, ) [SIG Network, Node and Testing] The pod phase lifecycle guarantees that terminal Pods, those whose states are Unready or Succeeded, can not regress and will have all container stopped. Hence, terminal Pods will never be reachable and should not publish their IP addresses on the Endpoints or EndpointSlices, independently of the Service TolerateUnready option. (, ) [SIG Apps, Network, Node and Testing] Nothing has changed. Nothing has changed. Nothing has changed. filename | sha512 hash -- | -- | 3dcd42ec4fb7fbc3b00dc18414bcab7438c22b8040e9d66998e844b289fba31b3b951800cc9e6d88722d1868c27e7dd3873c6d761d43c247570f83e8e6e5ac97 | 80482eef8ea1c07c0c5e272d8b0fb56c22998d20f40c275d19d3d33197f4a7a1c4a71783f3cbb71ceb21c5ac990ee253d28888eae68e8dbfbee3c39d0f047af6 filename | sha512 hash -- | -- | 15ed16d69f01e585e5272e0284f7c9112eaee22d73afd781de40f8f6d40e71ecc46cd6951beb3a171b539c8b343ad695ed67ba67779b18d8c668a20ef07e1c35 | 0f8eecfef713c2eea9793f36c90150c2d7dc444c69504f303f025c74b1f74a4d92fd61af379765c01eebdfb696812f98f256225d5cdbbfc34c26d0d9c4874ced | 16a86a9f6bf6c28c541d8b73e6704693d31fed4a09afa16d8b5414fd091133e2746c5f477f6571eec9a7540be88a9ab18ce644e99e6c8ad6e79506e73b0a8211 | 971d2bb7fb2ac098829a91a5451687bf016f8cccda80fa605b19a55e99b5a411ff838d8ec25b5abd2e3984b6807c7ecb9dd1ba744279be65e45495443ecb1726 | 6c003e494b26f7ff03653667db2abce5533817d733c95c64b5782374a4ca40eaa54682a5f7d0e6bb5043c3e43676fed868f6bcbb233b410bc999b276bf65a8e7 | 9a07bc8671a398c01aa31c5c3facc1469b708a7f388081c4e4110ee852af8a6d74cc700c925b3c867852e259477a232a992eda4c1211a9ac91233dc3de5f7ee1 | c60592be14d218799dd15360d004adbabd26baf73137f3ada526fc2d6e6b7e76dc8b2dcba939fc2c92d23f1a837fb5ae788ddc1ff06b5274f5aee503e27cefd4 | 6958d77d1f126a41ddeb4f5e7b5dea953c8174924165a5ded0c5b518349ae526cfcc869f8d9aa6f4cf078480877b89a9dd6a5b4c2f40b556e16f1618cf74e32b | 5ec708da681544ee7c8729f5c3afc44c869c3be4b8fdaaed4668f13c2b9a40f8a138dd01b93595f84ec4b3982027344a3146b7aea1617f60ff752f8182b2f792 | 60e52a530a4dba6d79abe21c55a3214b070840dab97f6da193cbe4f41b0a5a0707378ae7d118a018d5aec16b8a149784064d37f1f29cad635b9c876e0df744d1 | d5d35a538ba553c92601bd487b309269b0cd8c26c108d6ea4d107326d31ca58c44292bf0fc12612a5a2357f64053cb042a09cddb6fea6aac6a4d4e62019fe53d filename | sha512 hash -- | -- | de3dccf8a7e47542094528f6738b1d34073977c6031318bacb1656656860054e6cfe21280b0f314a1f1c73d393929341ad4866604ee73eff37af5087825a766b | 5ccdc82f5289bb7177b81e909c742a43f6dc8c5cc80abeb97808a7097ab08019eea0ae5c959f7f938457cc684eb1ed8c90e5f436a1b8e85d91a7dbe9a34e331b | a463d637fe8becb4e1b5b99b1cec44538ff31a386c7d331ba7791904773f931630840657f723212af65e519a7830b47bf50063616ee8c906212a6751bf426333 | 09ce40a6a684e0060ba41a9dd5eeb6da9aad9098f284b4b082ac386fe12fdb55514c4987455cb6aaf80acb74c637b77828ce7909da82bcf3f2afc094f3864bee | 3d3f904f935915aa12b36ac80c43275250c7a4b9a169aa641e225592bbdbc9da28c923056458211e5d3aec6f7344477d5fa9a9239d59f47599836082926f5de6 filename | sha512 hash -- | -- | 9dd2ec04902bb6dc0eb0a75735c8f2958ab7dc106487ac962564e6fc514c90581ac32e15e8fc658c598f0b6c3d199a938c4824bda23157f379510a8cdecc4a7f | d73d8cee8bac5b977ec0b11917a4d4eb8d5d33c8a2548210a806a3882e449d242504934ec199bcac80c31c53542f883f69d48c599eab3fe2cd2e3d95760ec966 | 4498a1b60813f0e1461c00d5cb2351a522f3e7c9d9d81ffcc99bf925378885aa5dbb64fae00dda3afdbe2f9517eb30ea28cc18c11158a9a46f69c0bc3244f2c2 | 50353816f8772d830d8ae05e1a535553fbcc2d552d00ba822b777c17b8492ba4ef5acd03b1d9f56583064893f173e3cf7b67faa4bdd77c11261a21c32bac8e4a | c2c4b35a42ae84d498c5e27175bcbc641e74aa42056df2f36c6bdb35f6330b5aca26c2d5b8656750f13b75d02c092d0f0ac5d4ed287d13033f696f243af9ca39 | dde3429d945c4b8c22a551e8663678e76d7a1136b9f9012ead3af534582f738092b0d5e629a983ad446d0396db143188a440edf471ee8c102a331006fcd00ab8 All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Sets JobTrackingWithFinalizers, beta feature, as disabled by default, due to unresolved bug https://github.com/kubernetes/kubernetes/issues/109485 (, ) [SIG Apps, Auth, CLI, Network, Node, Scheduling, Storage and Testing] Kubernetes is now built with Golang 1.17.10 (, ) [SIG Cloud Provider, Instrumentation, Release and Testing] Allow KUBETESTREPO_LIST to be a remote url (, ) [SIG Cloud Provider and Testing] Correct event registration for multiple scheduler plugins; this fixes a potential significant delay in re-queueing unschedulable pods. (, ) [SIG Scheduling and Testing] Existing InTree AzureFile PVs which don't have a secret namespace defined will now work properly after enabling CSI migration - the namespace will be obtained from ClaimRef. (, ) [SIG Cloud Provider and Storage] Fix JobTrackingWithFinalizers that: was declaring a job finished before counting all the created pods in the status was leaving pods with finalizers, blocking pod and job deletions JobTrackingWithFinalizers is still disabled by default. (, ) [SIG Apps and Testing] Fix a bug that out-of-tree plugin is misplaced when using scheduler v1beta3 config (, ) [SIG Scheduling] Fix kubectl completion zsh to use any command name rather than hardcoded kubectl (, ) [SIG CLI] Kubeadm: add the flag "--experimental-initial-corrupt-check" to etcd static Pod manifests to ensure etcd member data consistency (, ) [SIG Cluster Lifecycle] Nothing has changed. Nothing has changed. Nothing has

filename | sha512 hash -- | -- | 6cf7b3b00ab86a74cbaff599b121189ae73f7506b5041ee5351def4c2b9ee689f0fe23ff5a6ee682d994d3fdc47e54ba32640fd915a384cd33dfe7213f0e001b | 3a1acc0889add88f72bdfdababb54da8ef61e2d82176e2b102a5b73c3f3110123a9732bc5a8c657c75ce1bf379d1b13b3cc1941042c9f67644f39cf55891d24b filename | sha512 hash -- | -- | 96faea4fb5df5c34b4f9affcf1612a70370af718d87587ed1f1e28d9df8c57e469d5902c09444413fed5f5dd578aff88a9efec9c2df201b1cbc461d701469c7b | c30428e83089d38f47fd39a583324b122fc19a9f39ee4454f218f61352a4f7638e94f9bc1d223ce76732115999434ce039e4762df954629126460566238c0027 | a0da9c6f41c0164c81b7e805533020221969fb68f0816c58e1e982a2a744350eb552aaec2c5e4a9410568dbb6b72225c901140cc308f132843cc53a058e6f47f | 626620f7943575f4568b625b27e4df5e43073cf395f5666b7e44424a5964f913a6453e8fdd13990a52ca3187eb15c0e73a5f8d6ca2f297ca842afa0d111a7254 | d8ab4acff263761d89ca95cfc207d17d51fbc8c6b79f1cfa9782ea2afa15cad8341265b74a834b0bad808f24bc40bfeed1e12e4dd4a31cf82aaf103a5ce6c4a1 | 28006d52582842479f91094506696957a99a74fefaf3dd1e44ded910468f616ef918b40935027cd431026c9a8ac70dfb5adb57fd6f2ec96e717258d1326dbca6 | db8e85c87fb12beb12798e66a49eccd72f590eec1167be226960786276c8d31f9aed694d4a0e96efccdd29886b587f5e4ff6ccddaf354867dabba910ef7396f2 | 9e3e11398fb5f980517ef4eb9e732db6fb0301e820a50d141e6bf90775728c131c0f0eeae183c0b9314533089e7f44643abb0b55b1c2a1e38a7cea4f739acbae | b0afd0e85826fc061978e753682926b61a7b4a8618a8e0033185f69774afc7e7bd0de459d7cfb2bdc42f34398959953a8e57752766b9c283f517d6c6ce1c6ce2 | 76c7564d4910a03836eb43d5c18692b69d54862e30dea11994f5c4656c8911c08f083a2f0b0ee1e4756a4dd659b1a549bdeece8aa571e4e2ca7beaffb8523c44 | 1f9974ebeba3b8f751c561a64cc51c7ea109fb595c5177dfae87e9786804b19302fa96d7bac90fc8d99c04ef48d7400a0b11636a0264034749c7931bd01cf687 filename | sha512 hash -- | -- | a33fd6a01b9a10e00fd6cea33c0e621fc96c83ed434564ec3c36d1c2589d5466413b3c2993dacf5d4d40560ea3f99afff03e126b4707f1aa0816324d3595af24 | d6d7a7d16538a3484aff2c25358db81f1bbeb01c8a41930712e086a407fe9099e0100b7e4981b86e85824478aa1091d71deab211dd87582fb2869190cb8bfbeb | c5f57460c375f7eaae7fcb14781c8131a1ae20392319648191daddd2397246831f1c5d2b8d3e4d859df5e7c17e23056279553a4b3e72835a9fe08a55da500140 | 219bb24fc0a483e325bde0b982ba10a561588dc39b48eb5fd0e1cc31d1b6c2cbb6217a424a2eb5c64f4210fc3d9f692308171341b075a6b07c10c7932037c0ae | 90d0cca8028df8bfa260f4c9abba70747a5aeedcd59a92459bb5e94c2f0caefa39c68f11f38f916994d13fb5553134faa92ed735ed329f588b2c10874f95202b filename | sha512 hash -- | -- | 8083afd4dd05659e364d3592f71b0694f9c3d4c20d2260efbb2814ff8b914be3ced6d4690f9cf546522168b1ad6aab2ea70d9092cff0c7207db61514294b452d | 1c46b5cf9a2baf9d4c117a57efa64447701d1e55e249ab8a367008630e05036607fa0afc6c0e3eedec3efecd1a4d5c13316ac1034ae97e9d4971fba0f0c2e2ff | 5e36c75eeede07c31ac29961460ee5cc32e6ffa7f53cf6f674e2087d11668f6bf27013daf72a4137e041f17c114d263f5c7b6c891ba084ecc76029f63bba1011 | c4eba116db85a94e5e647d8ca7f5d334a9aa8d42c27e10077eebce28e7948adb06d6cefcf10db8abd762960dcd7d4b6ab2c1caf7b079fc84bfef356cb822eb07 | 50905829928ac041d95243bac78879aa607d7a5bdadb92495e022f856e1f332204ec4e0cc2b9c848b1afddf1cb8b0fe249944537005a97967237581bc42e48b9 | ff71310b0453efb8ba364572eb5ade35a9826f3a6b1d5d215764b5e165054c3cd77e24b7f75a443c9ab577f39899212fea0fe849e3d0a8bbf786ac568f2c63be All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Omits alpha-level enums from the static openapi file captured in api/openapi-spec (, ) [SIG Apps and Auth] Kubernetes is now built with Golang 1.17.9 (, ) [SIG Cloud Provider, Instrumentation, Release and Testing] Bug: client-go clientset was not defaulting the user agent, using the default golang agent for all the requests. (, ) [SIG API Machinery and Instrumentation] E2e tests wait for kube-root-ca.crt to be populated in namespaces for use with projected service account tokens, reducing delays starting those test pods and errors in the logs. (, ) [SIG Testing] Failure to start a container cannot accidentally result in the pod being considered "Succeeded" in the presence of deletion. (, ) [SIG Node] Fix indexer bug that resulted in incorrect index updates if number of index values for a given object was changing during update (, ) [SIG API Machinery] Fix the overestimated cost of delegated API requests in kube-apiserver API priority&fairness (, ) [SIG API Machinery] Fixed a regression that could incorrectly reject pods with OutOfCpu errors if they were rapidly scheduled after other pods were reported as complete in the API. The Kubelet now waits to report the phase of a pod as terminal in the API until all running containers are guaranteed to have stopped and no new containers can be started. Short-lived pods may take slightly longer (~1s) to report Succeeded or Failed after this change. (, ) [SIG Apps, Node and Testing] Nothing has changed. Nothing has changed. Nothing has changed. filename | sha512 hash -- | -- | ced23c737deb52c9e84af8d2937b19d13822c05d588c29bde120a125c97276ea1f008c649e9a2117b6f026f5fc057afe0089d8ee1154c75e67e054e8ec73b847 | 7fe3cf10bb534a26ee74fa1eedad0c98d874d8de0b8e2c91ed10ebf880e74968bd39d4b6e7170d694ea68bbdc775b3ca70ab78ec5f4f2d15bcb20d6e869f4b2c filename | sha512 hash -- | -- | 0c5dfa5cd898aba6599273da7c16aa81f11d0e3a031d92820c98daabb2d2372c8a1dcaf2cb203d88ff172066239c4c783810356840a0b19603366f7760f41ae4 | f41e58a9118378685ff42499d082f34bed72a23ab55c1e94cd6ec209aad8e5de943c9385d9b449bb70325b4a01f58f63f0784c43230a681bd13603ce034d85e0 | 795f9f68447b36ed912cad4b3f874780bbc28ef7dd34881927467ee42961563a8e6cb1d6d150f79a579ebfeaca2631bd99703625770b851e30dacfec5fbdde81 | 51f5679a0cb11a65f25c3479bbfdfd21c4d0acd8814d3cbaf5aaeea7682178a3820c3555b17ea6ee24470ac67ebfd0f78cc98513e5b526436494350be64bda69 | 056d62df16a9725c7ae8b072cadb4b713bbd2230fff95d777721a951b6a1443c480920dd000039f1ef4da1df4ea8fabef096d7166a6dace5e6e6fe3b3e67da53 | a6b5b19a71e971cdfb0b4819add8f7ef3c24a99b6201b67f52ef6c65787a4d9008ba69b6f950bd162d05e7620c34971346a6b4b7cbeefe2ffe4a84a32661f208 | 969d000f87e991755f91f9b16c114c8606d342f669625111609c1991537e7085eef6c20a815c0c0890e2a405e015f9c5daacaab526b444fa5e0c075f0ac4d017 | 64a1bb89a47a37e7de1bc8835963d8ef3253c86306e5c4ef4d6b2609699fa5dce1d8611406c1d8bf22b3a7ec6194f61e526c8276381f79bad0fd173d6cdaf50f | 381d1602538fd7926758fad59a64dba6fa560ecc48593cb55f7b3bcb494ac221dab84be1f9e945036a8cb6336c8a0ecdfc33c0e94f8613112dcda2e020e2646f | a794fc29d9d2de0d5550a05dd7712b91dd39dc6c75bd9f291c25fb4acd3a5b6fcfaea07c768e291bd62b10c9b27d3d0f57af450ab489d872a3af7b89852c5878 | f4225d21f12a270019c7a96330f14159fe8a34b869370deeff3920b8dee3cd78e9173605033b9e138ccdef7304bbc39ac41ed582c0360b580a1b0cd67091e6b9 filename | sha512 hash -- | -- | 063ad74fb1463ee7a7bf4fb746eef1e02980c170cfa89c93444bee0841a84133bdbe91035f6608ece15096fec3e0c9aa50ebdc2b15ce589d86e2f07d10a1d747 | baa1a310236fc5baad609285fc4717913791cbad920dca17d8736e0052af9cad07ee454323e8d8c03cad70456bd69d79cececb7019b3f3a48978552bed3d2b68 | b3388b6da8fcbbaa30ac881f0f0dbf6ca501bd5fc52aca33174025fe6234af2872ab1a25a5a74049cfc627359a748c7cbc0f129ad3b800c4707d44f98ac69d52 | 1eab49c6ad3bc7f368b2756239f60b59cb946f6bf56974a5c62688f5bfc5175e2d4ae33453bf8b1e1baedfcd9cba23cad913ca282a57be87fed18ddb6c28e754 | 3abcaa6fda41b19a0b5e2627e93b0004759d291ea22c8698008f0924a7c8a5c2aece81c508e69e37a60980f94adb6873a68fe79a7c659641a453d706c90b26c4 filename | sha512 hash -- | -- | 57eeae81081e06e35484b353be04f18bf5c2556175a0355d63cbe3eea80d51decfae28eb42cb5fc8907492a70e4e9bae54bd86956caea7c3a51b1fc909feaea6 | 26510f1f342efa97d3a340db699f8d18d3b8430e082ff32c4596fc3efb629cdc0d427af6f749c4ed845ea2c0f2594ece7e23bc875970e5433e585812e8eef84c | 70a73dc630df6fe60682e5379624116607a6fef571f17423de309d1ce20895eb358472809e736743740d4a2df1f708799975013d4cdc47499c5f1df3d4b2d630 | ebf98d17dc9ffba9ab895bf4877099563cc01ae930b9ec0342936ec53dd6b5335cb2ff01baaf620dbb9d7270e2f6831e12201c3da6c0ccbdccba30288bfb1317 | 708a9101cf73f5c78cd7d7199833b5f2f74a7cca8a4f1e0db629e3a47250a3e5338269de4b3fba703d9c084ccb4f3f30e57a88f5dedc09fe47964a0751275460 | 8aa1ad4a60edc6b677f21509eb6120dae4bd396317f28ef2d73a49986e3aafb899a5e07c7944cf5337a9b9bd514e14c3d49757c9041d5aaf80bd93f493288101 All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - | , , , , | , , , , | , , , , | , , , , | , , , , Fixes a regression in v1beta1 PodDisruptionBudget handling of "strategic merge patch"-type API requests for the `selector` field. Prior to 1.21, these requests would merge `matchLabels` content and replace `matchExpressions` content. In 1.21, patch requests touching the `selector` field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. (, ) [SIG Auth and Testing] Kubernetes is now built with Golang 1.17.8 (, ) [SIG Cloud Provider, Instrumentation, Release and Testing] Bump sigs.k8s.io/apiserver-network-proxy/konnectivity-client to v0.0.30, fixing goroutine leaks in

(, ) [SIG API Machinery, Auth and Cloud Provider] Fix kubectl config flags incorrectly setting burst and discovery limits (, ) [SIG CLI] Fix static pod restarts in cases where the container is not present. (, ) [SIG Node] Fixes a bug where a partial EndpointSlice update could cause node name information to be dropped from endpoints that were not updated. (, ) [SIG Network] Fixes a regression in the kubelet restarting static pods. (, ) [SIG Node and Testing] Fixes error handling in a kubectl method used in downstream packages. (, ) [SIG CLI] Increase Azure ACR credential provider timeout (, ) [SIG Cloud Provider] Kube-apiserver: removed apf_fd from server logs (added in 1.23.0) which could contain data identifying the requesting user (, ) [SIG API Machinery and Scalability] Nothing has changed. sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.27 v0.0.30 Nothing has changed. filename | sha512 hash -- | -- | c88f633b0b418469aa381cd39da1581236ce3e7df6f983434d4ce95fbd810a63005a78745f20f16661fdc6280d83be2cb52c8777c0ab5a8c526dea30669b463c | ae34f80b5a13f717179954a99bb5d0481b3b9bb1ea27e805341f6911d0cd1b3f2d58e5cd375b8b84efb0325fc13d7536f9642790aad129e8361d60f36169e2e5 filename | sha512 hash -- | -- | 6cdbaffea1ed917cddf6c4d7630eb33dddc36c7194421bb291ea7c7a8acdc235ea061cb3a8d72d84e5d314f61497594b9a5b40dbf4a69baaac88cfb48eadccdf | 200025cd65155ef8d6a854a96f8fc28f819dc3d4f7417af4c0da141d31036d48eebeb3fbfb4efd5d1adf974de54eb549f6c76f26c585b689d46484ca12902edb | 111157e8a37ddf5d746018a4c8c8b16e86f6c2e18b228d9935b5eb2da630959e22362e3c328435ce0b5976eaba420f331c8ac0eb217d81577f0a5a956b529397 | 280c0b62d7b19c23b30e52e2b6d3aad676de003cbe711ee20bf03227362133782ab6c40f9bf4ac2ae264d0522a863084050e0283eb7b46d70393091994aca30c | 0d2c1d7091fcfa37b439e3c8ae5643385860e1cf6578fe58df85913b607f356c5e6e2b348e8f156fd293d53c8b4b520527640c5b585990397f42699c3ba0e146 | fd995845ebd87195b8de662097f423f6c4c71addeefd95387303be16814374eaf044894e28e84cf1906cf33569c5b486919fff97e7aa32e7528fe591c100f3c1 | f88356c7f1bf84eba54ad83de924f35f893b3f8dd6ed78518907f8f69cc85048c3b1eb176e945e8b3ecf5b2cbdd0f95985e4f5ffe91eccd3cb24050d6a3c89ed | 899ae660288a6dee79e3e9b64ca8c3c37c8bcdf290748074172f546da785d805f9ed281b3f9a443a0dd1e86b8e112c8b98c0bc85592526e10ab8420dee90ca59 | 0bd0219cf5653204ac89cf246521f4ce56f89218ce4ace979504a70947a7aad9c7e783fbcf2e090996bedc90ffc3084683527ba5df76858f31727a58e41b2740 | aad2ae58858017484683347849d04baf3bfc7eee6984548383b3bb0150fc8cc25b1aa73bf4e1fcb9878e05cc2f71c44811f5b50b3bdf9662b79627f3ef3d3d9f | c5d6a22f8264b38b67c316a44619095c1826e46996ffd93e4a78f6f7cf8d85f6e9835270659dbe01b0c495873e40d9373fdfafa26e5fc6d8e078407c763b22b6 filename | sha512 hash -- | -- | cd5e90d25bd48dbbd1755eb5d328676c6424d191c2936d7477c9dde72cd374389f888bfee6ffd02836bbe136e8bf9a2cb8adcad2c650714dc6112075949c06c7 | cef6eaea9e1f65cbcf8cbfd254d61e6faacd1550bb691ee5cf7f157efc61a4cce6a6abdb26c42c628706f55b062f7892ab5d40665703e22d6c1dc16dfa8e8ad0 | a7762d0b380fb06675bad6d4b987e3ddfe0c2b54ce1592c9d2c853d3a8a4d85bbfa77013e92985f3d47269010ff03fcef1166f91aa691169e920b367b3babdb0 | e8390ca2a4cf7d2e4b1ab5a42da4a47a0761fea200ee83626c8e81b2790cc6a20b25b090ae2b67a0a81a946a35f3468b3a09dbfabf195fe359a1a159260c424f | 1501c640e22acbe03ec06b76ccad8bece1038039f2a7d71d9504843c8173551b04b2d144ec437d3ea1998e8ee9f25df3739e85788c738f6efca274dac919a947 filename | sha512 hash -- | -- | 9f75ec2e71469bf5d53f0ac305128d7b685f7d2daf4dd3218a8c89b36bd3f3e73f69696ece76bb4ed1254e1fcd89abc1fe49896db73222a7ce8c0795afd37e93 | f9085bf9b750dbebcf713aa4b5166e65aa8a313be468e1e8014e79a516018b930a9dd4e093fbb632a1538e59c6b42017c0c15050fe67407c92e7141ae073af49 | 5e6f30f07f1f49092c2201303d5f3843343c1453cbcb1603617df9aa43bf4549581afd9332d634127320fa543747f730c57c00f437cde3f912b6476b571e5bb8 | a63da0682d139a9ccc261fef8e9b944b31d8cf020416f5fe3216ea5da6e2b76c65778064df51009e8a4630f41c811e9c082682a5731766647b372de2b04f1035 | 3c7e7295f1b133a0469f9a70a56891cef9d990e7959c75f4c5dfa1dbe8f2e6bc689743b32cf5d5ba1bf854c5dac812328f4fcff6606a97c77ae06c24848de2e7 | 434c0a397a10c06bbed9228944741605a97eee3c0fad6f4de341a5ac882439879433f84a5e269019ad599d440218a1277bac0dbc38e01ff294ec7044a1283076 All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - k8s.gcr.io/conformance:v1.23.4 | amd64, arm, arm64, ppc64le, s390x k8s.gcr.io/kube-apiserver:v1.23.4 | amd64, arm, arm64, ppc64le, s390x k8s.gcr.io/kube-controller-manager:v1.23.4 | amd64, arm, arm64, ppc64le, s390x k8s.gcr.io/kube-proxy:v1.23.4 | amd64, arm, arm64, ppc64le, s390x k8s.gcr.io/kube-scheduler:v1.23.4 | amd64, arm, arm64, ppc64le, s390x Fix OpenAPI serialization of the x-kubernetes-validations field (, ) [SIG API Machinery] Kubernetes is now built with Golang 1.17.7 (, ) [SIG Cloud Provider, Instrumentation, Release and Testing] Fix Azurefile volumeid collision issue in csi migration (, ) [SIG Cloud Provider and Storage] Fix e2e test "Services should respect internalTrafficPolicy=Local Pod and Node, to Pod (hostNetwork: true)" (, ) [SIG Network and Testing] Fixes a regression in 1.23 where update requests to previously persisted `Service` objects that have not been modified since 1.19 can be rejected with an incorrect `spec.clusterIPs: Required value` error (, ) [SIG Network and Testing] Fixes static pod add and removes restarts in certain cases. (, ) [SIG Node] Nothing has changed. Nothing has changed. Nothing has changed. filename | sha512 hash -- | -- | 339d208b86206272494d4f31a384fd8430911a1f8205d4a73605f412b4653fd816e79653bd0a0dacf52c9b9f6a3194279cc1059683c2ffd560c1ad3fa185e20f | 9530d46878aff36b26b6e8f8bb04c53eb402bd822851f5aa65a2ac6e46064f67820a63a2153ffd317a0b6fa3383cdbc58fe55484a0dd3197862f3133fede5a5b filename | sha512 hash -- | -- | 96810fc0f294bfe412f06125b257ae4e7d4ddc7d11247435a3f500830af334560b61399ba8ac9753c63857b42a7fd13c8f7d55a927cb7981aecd886a84ae8a90 | c9d1bad58e46908d190760fb1615188180ca1c734ab4137437e62baa76d92e3b79acf8c39e693c5b947bf650bf894f5fb96651d4b617d35992cd82c1d1f64709 | cbc63934798cb57f0be188346e852fb9ca8da071c38c5b75c8199b5637bc8df7df9019855607c72ac634d7944e86db0b9485b29b36365d93f8b85f2703b0f3c9 | 7ee6292a77d7042ed3589f998231985e82abd90143496a65e29b8141dd39dced5f9cd87a7eeba1efa4dbf61e5ddec9e7929c14b7afcdf01d83af322ddf839efb | 36147a76eb16869bc07608f947b78ff15c3d60b73deef208e5c135d93e1a48d17e5f8a447894a450a6cf1a1d5f058b07463dcbf1d0120d17133d2d98cbce2444 | fb66a9735f40e2df40388df1f8e17aedd1ac87f7190d76e3eb2a5dd1a11494be56ae312be5aea0f7613b826f3b9b3ee923eef9736b689415605de351eb8861df | 6b3103ade6e0d7d918461ae33978636343b3d122dda5a68287c21eaba7c6abc2de49277828640e4d25904134aa4311665615ac78783b5fec48314bb3ce09a3ee | 13edeefe00b9d9c151ba27a4190a0dfe5fbb7fdf409a83787eb90a0f38be1cfffa7be50b180d5fe96acbdb6ff335ef9ba8c90fb828f4e9953e4798aa3b20963c | 4c761bf7ddf59a980cc800602c5ae1379d9b39b3a15fc35a0a9b2b34fe18150dfea5e6c84b2548477df4a563ffd3290fe84cac80e5f9bc6f07863a2efdfe049e | 92cc39b07c62ce5c436f167bb8929f678e362484d7c40b7ef76562ea61db93a38e811325e0992b42349edfc5136b110aff169a6423a29408e65266cbd204c8f8 | 71a136733fc032af8c825c402355b73e1049f3706c8b88fc3c7c78da0d3e0e6e7eb1fea455a833092ec7d9e12e27230c1097fad1129f487fcdcc4e54b81cceb2 filename | sha512 hash -- | -- | 667bc04778070685e5fb5b6281fe78263c5081af0613adfe9a68df0695210cb2273e89a1d37a27e4cbf947b9e565ef7697d8b90ddfba23aeeb4c9f8474a373c5 | dd3dbb478185819a4783ac5ed923282ce5be6d9595228226dd8c8c447ac78e5ab7b4558c452c39767df571dd0669552ce096a6bc4d0ae993511e5c3274759961 | 1eab0b0102cf6635a3e92249d95c08ac36baaccc14a612f512009e0aff28d563b4fd818deb4aaa471aa8399a3a5bb67b10b484dcf0d3e54615d906a1bd861cdd | e648010752a1db8d23119ffd44b672f67040e4f841b014df699bfa328c32cf97bb928a1c17d1719098a229d83844a593a9cfb1d0833f562a7d64e71ff20120d9 | 0da7c96e2360f8272dd6cab9da7a3a6b516760b39a9fcd38e86365c2b6f7bff0e839892b58910921aa25d8dce0c94b46cee58bd686b9369995641b6771fa0191 filename | sha512 hash -- | -- | 9fd17ed04dc8e13ba5b4d67ec657b8afba721c344bd9785669af3def481dcbd8a2eecb02e54e5eebd0559645c6e819f757c49de731e53073f06a12d871e569eb | 8c934bc5b3a545a8a5c2fa9f6df7358127125509ebd9ddf1b074121f78c6520ff0e4af9217b9930dd21b28c4960946673914f35060283b83606073d308ef28e7 | 6319071775767b4eab400f3068ec4c0901756ccb79db63fa9ed6754047bacbaec55cbef366a92e057ca610eb6f20d6e4341b9a1f7a36ced44435d07bdb0af0f3 | 8bf110dac7e4e61ca9a2a513a6e296bf36bfd8dee85e7c2c46f831e4eacdeaf6b238b361b044b5a7de0f14f1d735f41e8e169cdb676ae4f4c109da457593918f | 35afc0d3f31b6795a280cb7005cc7c5253e897758aba36f4d4558a0ffc2b34ac4f0e7a1e7a42ffc3c67ca1a52365bd3052ea554e43b8cca4bcf2ab38c6ea7929 | 8d687018bf4b70065d4871406702d57f0ef14abb6c8e8bd7635d2d94f8a56aead9a641ede4477e34534bc705e76bb94cec10dbb9414c5885ad0a5d07d1105401 All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures | - k8s.gcr.io/conformance:v1.23.3 | amd64, arm, arm64, ppc64le, s390x k8s.gcr.io/kube-apiserver:v1.23.3 | amd64, arm, arm64, ppc64le, s390x k8s.gcr.io/kube-controller-manager:v1.23.3 | amd64, arm, arm64, ppc64le, s390x k8s.gcr.io/kube-proxy:v1.23.3 | amd64, arm, arm64, ppc64le, s390x k8s.gcr.io/kube-scheduler:v1.23.3 | amd64, arm, arm64, ppc64le, s390x Kubernetes is now built with Golang 1.17.6 (, ) [SIG Cloud Provider, Instrumentation, Release and Testing] Fix: delete non existing Azure disk issue (, ) [SIG Cloud Provider] Fixes a regression in

that incorrectly pruned data from array items of a custom resource that set `x-kubernetes-preserve-unknown-fields: true` (, ) [SIG API Machinery] Nothing has changed. k8s.io/utils: cb0fa31 6203023 Nothing has changed. filename | sha512 hash -- | -- | f30d444bd0fc62bd8f7d352dacbdc2fe8904707f3c4f6d719e62f6c9509d5d544a1b26964228c3ff29b9c451534d9f85fe25a60b09b332fa5291e542720cfa05 | 6a54e73972672415c9d1472764f6f266700da807a6ee9cd530e28a5158d33280702f3d94948e914347e32df24d9bebdc0d4627adef5221fc7bd3b12e2f8d2a93 filename | sha512 hash -- | -- | 7371cbf87768e49cbc7ef2776fe037eae2809eefefc5242733da119328d49facf714cb04113c15cfbda99b6b5f62fb82b66951c04011fcb642bd056adb11acb3 | a5c8aa760e1cd94e469beb1e73c5abc786bc294278b2e92082dc1afc7ce6e3f8c7f157752331c45579738642d3c159c23d0ac303e0838157cf6fa06d649ef800 | 84af3cd829626a8737e38650dd231e158edce11706612f357e19c2e8ef316b31239d6ab82c5ba565c1bbf2975556eeefb3ca6c757d17d27303ff6cf1ad4b9f0a | c8653aa2bce09a29041b7347ce2d45710abd8bc3cfe79265e0aa04a24c2028344f0b280f52b2858e869bf997cd0e71b6ac8f22ec8a2b4b39328e73339746f565 | 35a00f6296ab70d0af2838915b848b9f0df5b778935ab8089b4b180c7406b958a1c909adc1bbdf12857d0c75b18be3c236c617f74d392d3eb2f8cb85eb862eca | b00539ec1d993e272b77d4ed3a46be743af645cecc6320c9a017c0c5f4f48dd0272377f67e993c2f06aae0fdeb6a174da901be9ff1e9f6c8cd311e16c5cf60ce | 135e2aa8ae000ac6fe88ec8afa0f671147b9d8936def510a53d2a456191805daceaef8152d22d91a57bcf4bc7e8b47e587440ed260d3f8b2d5013c90d125fcbe | 860ad0d3eb064e1ca3b2ce74a296fde1fffe3e620ddfd579f7d022032419bb8f0c7300565bd94d6681cc50ba2268bb2d64a94c1cc3b9b999f0c031de1b92999b | 907c3043a1f06912238ee5d91f7d76d9bbc5417363deb3d9f2cef86bd79e72ad4f7d9cbeaaa7003afa5570bf2a689a8d619ef60e5cb89e61d53eb5ef373102e7 | 1abd5aeeaaff5884238ee39022ef18b91518026c93e4305f25de9d2cc2136fbacb9799deaf138c691ad82928ac22f7156046793b04b7240625b2e5043b65e9ee | ab5617a9a6b154af6a2329523c2fd356583b8266cf9dd512dccdab5a21a9f82ab5eab4802d65e0bc191cc492215e2d5fb850f6f8c02c403635cd0efbe8349450 filename | sha512 hash -- | -- | 7f4cc97250be176a9af0136c25a549fc491caa0ef6300ccca798c599c5ed5182c08ebba1d8f669ddf3011633cd7b12523d7e37abc65faa369defdde9e351eb22 | 572fbad7f0edcb1e1294d81e222f4add78d302d80ea28d7456b8020b53f01fbe54bee424d4807ed05d6eadaaab46a323faf36b3a3da7427bdc075749c3365d9e | 971ce29019cf248c167c27fd081458fde613f7f92c5fe4ad3816eb12ab157c30eb78105b999144d0b31e7093c0f826b1df9cc275b120d0ac269062726669d0c8 | e1cc8146a2c2a0b5774a3516548323aed948dd03e93545ec11c23be8bc5b23e3395b593a656dafb36711e7b368b3cf1c8dd2b55bf5a486fc3ac4875bd011dd22 | c3f766166e64f878c9077c1d070326da0ce71881b40204d60596921d62d568282f6527137e0f362ff3895ad156ab92dc026ed19f305120872aa385386033e6cd filename | sha512 hash -- | -- | 2fab5c395fbf5f88e56430c51979670db0be5119d79a8a74846900601a313917b91615efabe49a92b3b8ad2f11d2dc5892f701549b0debeb16012c3705612d01 | cfe8e40981aec67fbe9cfcba4af18b1ea4f5b4f34e5b9b11cdb6dee076ad8be2b351d0871b1e9d715cec4ca629c87a5cb598a70b6f8e1dcd15486c1f387a5a42 | b75d8d1e0b9ba1b06b3f08d67ffbff01784eee8973797d0f1565efda4d61bdb89fbea45c385ca3224fc75237ffe6a41920fd1f51b73d5021007ec9e4ef88af73 | fc47a39b6cbe9d4237740d060234c065c7bcf33fbb10b3cffc670b6f7eacfb9f44c2695c620a42fd37a2e30e375f29d206ef4d4fc8a1ded7f57bb9894c85b178 | 1e9b0d5197b436a14a1e772027c951ba580c5a047acf201102c585190eefc8a8871fee8e20a40efcc01c0475776540ec8cb8f09aef43c149e335a722670cd855 | 37d6edc06bb5a555c0594875f917a80f42486e59252c0a8b813b3a935352306a19178c4b5c0af4251b40be58fbe3b99377387d1c861e32f4599ef0e275bd4299 Kube-apiserver: when merging lists, Server Side Apply now prefers the order of the submitted request instead of the existing persisted object (, ) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Storage and Testing] An inefficient lock in EndpointSlice controller metrics cache has been reworked. Network programming latency may be significantly reduced in certain scenarios, especially in clusters with a large number of Services. (, ) [SIG Apps and Network] Client-go: fix that paged list calls with ResourceVersionMatch set would fail once paging kicked in. (, ) [SIG API Machinery] Fix a panic when using invalid output format in kubectl create secret command (, ) [SIG CLI] Fix: azuredisk parameter lowercase translation issue (, ) [SIG Cloud Provider and Storage] Fixed a bug that a pod's .status.nominatedNodeName is not cleared properly, and thus over-occupied system resources. (, ) [SIG Scheduling and Testing] Fixes a rare race condition handling requests that timeout (, ) [SIG API Machinery] Mount-utils: Detect potential stale file handle (, ) [SIG Storage] The feature gate was mentioned as `csiMigrationRBD` where it should have been `CSIMigrationRBD` to be in parity with other migration plugins. This release correct the same and keep it as `CSIMigrationRBD`. users who have configured this feature gate as `csiMigrationRBD` has to reconfigure the same to `CSIMigrationRBD` from this release. (, ) [SIG Storage] Updates konnectivity-network-proxy to v0.0.27. This includes a memory leak fix for the network proxy (, ) [SIG API Machinery, Auth and Cloud Provider] Nothing has changed. sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.25 v0.0.27 sigs.k8s.io/structured-merge-diff/v4: v4.1.2 v4.2.1 Nothing has changed. filename | sha512 hash -- | -- | d7b53be1a9695143b780fb9ff1271c65dd1584e09ef77fe5aa3db4f965a9a7a8b59af8981b3dfeba1f89dd48a81e30f1cd4d443b7b9bee9f1695b3346b41c8fc | 00e07c8f2b42bda04f780f74fb5625f52d7a16b99424a0f7a4c67101923eabe495f440f66317ef151b9ae48253c0dd2fa4730b8a4e28b86f13c7741e96cdaaa8 filename | sha512 hash -- | -- | b7e858cdb049e710d3961a791d0ae4b0c2309b024f98a80dd470e4e6a2ab30bd2aae72eed57561af27dafdb317617d60c8b7f454c2a0873bd2801751d8cc0260 | df441b4757c7ccc3e13145ad264c1de8101e442cf267b3537d2506e9c62f5da4738fca14967367c58933554fb5deaa47cfed146bfb03e0b81b0fac656696b4cd | 3dcc223be7562a78d01e491fe101196d9c644490e4d9a11f5a2314cd25c0f1870ed1dc0c2b872c24d1802348a3d209d0e1304ed16a5cac76d121090e914f4191 | aaf6c4f2f65b27902ce02069aa5a7c5b195099deb522fbe7638c5458fc1ba6c2fbe2eb00fa18edc952989dfc27c7a252b37792987c55dc044b88d4e350569891 | 5091b2731725a53a9d0db7e45dcd3f534978f6f0361dff13f36e8c22259506df622e25d0c3b81867328a3314d98d9545b351bbf49ff45f9fc5444f2f67359546 | dbd3ce1ec9cc3e89a0510ce3809966b38fbd95e538b9b9426b9c303e1dbb71eb44c1446bf0b4c70a58d9f1c29a75bff71543fe0e8c724a5d5a03082eabdd9f02 | 677e733f714b148478b9576bd8fe56e78e7517b2fcc5b1d276e92c958933002b9565bc8bdc8f22c2c41857c9b1e92a008baf48477d31d485cfca6a68a82f32ee | 8019fae5c10b146594e300fb5591eba45df7a637db4bcbea8943bbef37d06f2d349f338c4b48500ad92e477a869c29e980dfc80e42d2a759ad62236757c53718 | b9d3831e78220abc15b0384ab3739f4ba105e324270012a2519ffae4b22c963f2eb602963b68f6353807ab42ae1a1699ea82114edb5e2650af59bd7b6c700288 | d661b48b7d5dedadcd644a9dd4eb99b756784e43026a39cd837194f8eae5d4356fe0c4de7f4b85ef7ef6fb27a5bac8b9cc99ab92e8bb5bc092f38d3d781e9921 | 833ba3c0afafc1fa5fd40d20e0450eeb591fe3eb2b73cbcea74fcc029cdf17caabc3a2ae27d6f3227b30c90c31c0d9a2c57656ed3f8fd1173d38938b98e74374 filename | sha512 hash -- | -- | be1b895d3b6e7d36d6e8b9415cb9b52bb47d34a3cb289029a8bce8a26933c6cc5c0fe2e9760c3e02b20942aa8b0633b543715ccf629119946c3f2970245f6446 | e5518962fc1543cfba096693c9ca3fb026a11395dd6eabdf50fc577c598e6c546860a96def681681dbc84025e8f06e3bbb13b28a1ba829bee359b779e884f626 | c546831cca738c3178ff464891d15f84c10d754c1c9b70742b1fa638d108afabf320aabae9dcfd1ec2e6e77439e5151c561ca6a2cb8989cb533035f48509bda8 | 6e72592a8ab51d6e7875c327159918a737deca88e168574b0dae77c08e0325acc575b62bf3c166e3c6da438fdfc2f996911e134de0c57a1c5001ce29d3b7d97e | 1092c009c518f28b089c962e33c73f97af2226e5d100856a3ac996f47da4519a4c450dd98ac5d78519fe12e47b955df98675bf4ecbf0d8a722d994d5777b4107 filename | sha512 hash -- | -- | 77bdd46d93d6dcdb2e5a66ecf0e5f9b6fffe49c53920a36c2e0c4ceb3625631a4a09662dbdf4c170bc5ee35968da257b80048d21d4d1f7302123a549fe99429d | fb619f27a72cb014e5e97c287d768fbd23bcd21dd8f5d2cc1a4e4d7a21727781ba914a1af9fc33436795e7e7c835bca3e9afa4cbc5d691691f39fcb5108617c7 | 6f3a98d8929f1e7088ad8751dea6a9fe588ff6a7c40d29b5723ebc47f1badc2130ff4a72c6b2fd3bc0c9dce2b93329ee3c7cef3be93fdfcaf82162bbe0314adc | 3c8b14d8680612af73f56dbf8b1de4c7c45addf98cda1aa7cdf0b8a0e8f12a95dffe336fd3aa4cd93e072cda5dcde1a88da3034ce8cf5bd11718800596342986 | 6f8a30f6ea0114156e636208ab09de9763ed08844d1278f7d034575ce0b4b486aa3fd3cec8992651d6de8263b66aa8285a1180ff2f6011a2904eabcd872eaebb | a0cf768d92b51d370842dd7b819c20d6fac2bee955763ead4700f095ac0bead030a9a4b2347987e5ee862402f3d5a7a306343229777011be21544286a2ad9448 Kubernetes is now built with Golang 1.17.5 golang.org/x/net to v0.0.0-20211209124913-491a49abca63 (, ) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Release, Storage and Testing] Kubeadm: allow the "certs check-expiration" command to not require the existence of the cluster CA key (ca.key file) when checking the expiration of managed certificates in kubeconfig files. (, ) [SIG Cluster Lifecycle] Kubeadm: during execution of the "check expiration" command, treat the etcd CA as external if there is a missing etcd CA key file (etcd/ca.key) and perform the proper validation on certificates signed by the etcd CA. Additionally, make sure that the CA for all entries in the output table is included - for both certificates on disk and in kubeconfig files. (, ) [SIG Cluster Lifecycle] Kubectl: restores `--dry-run`, `--dry-run=true`, and `--dry-run=false` for compatibility with pre-1.23 invocations. (, ) [SIG CLI and Testing] Reverts graceful node shutdown to match

behavior of setting pods that have not yet successfully completed to "Failed" phase if the GracefulNodeShutdown feature is enabled in kubelet. The GracefulNodeShutdown feature is beta and must be explicitly configured via kubelet config to be enabled in 1.21+. This changes 1.22 and 1.23 behavior on node shutdown to match 1.21. If you do not want pods to be marked terminated on node shutdown in 1.22 and 1.23, disable the GracefulNodeShutdown feature. (, ) [SIG Node and Testing] Nothing has changed. golang.org/x/net: e898025 491a49a Nothing has changed. filename | sha512 hash -- | -- | `850f92f4a4f397773ceabdacdb0513fa3cd2eb8867f7e3697f42bc595c3c710f81a8b9b34679d783ca2e1900dd272e0af209126cf55719e321af8da04a4b1c2b` | `ee53eb3b32bc4745a3f58dd0af1a8f4157e74b71b896eb39ae0658f6f3d0497b8a1334205cc19a3fe1cc7056b7606b0c745e89860f73b14ffaf4ed7bf9b6ef11` filename | sha512 hash -- | -- | `ec7acc668cf32ec2ecf9ff515096cfe0d421c096522f4d4f6dd5504046051d6b4a15a130aab67e0d545078b26c1a0d27f2f567b1f4ac68b76324e15a216799f5` | `5c8dcc6c847d44bef1d739015627369b8b7f0b27d96bb0264bac1ea029d3e247565639cdfb9041e28d74f27c7650661ec275e452ede448181c454d77916fc432` | `a94aa935b348dfeee09b47f3a34e8cb7b2d7213dc28e8df189a4b8438a317cfd575d97ae09651fb6a46a528cd00f0808c6e6c95d9e176837b5be463620593acd` | `82574d81696693510d8becd2a2319d517dda2b424b63c5525299ed24c9f4abe1b1803fc799b2a75c650175a5a70ef03eab6068e5bf2a286a43fcaabee9681747` | `8dd15ed487883f76ed869458df3b5e859518491ac5b3aedb7ec95fd6c8ba1bde081859e0a7c171d9674a773b655db67975dbab9b842d1c0864a300341e58035a` | `125f51a712fa4cad241e84a57baa4bc7950b4977bb4f7275ec21e82758ea90137d00d39841061cd9c4665144a2cf8ab87b99e185a64152f9682ef524266c45a3` | `ebf7130485c33a59fc81c2a6b8d19b847470f57f4be49ead06e0405a9b34891de456199a87ef105b0428f3d9767ac39f83b8199171ba60ab7b7c538b0558d1de` | `dc0a122755d096f18de5d33a7596bae3c8cce058d22999c6754377cd074b8d69f597c62b5468cc80eda4144edea026b9b946b522b144e8d9ad66874de2d0bd9b` | `d64f72835dca883e666fc44f80db6b75467b5f952bf40f241b8b1034a8b7565aa101b3d8d9be4b47523cf22a1bb51db3924376405fdf40cd3cab688c5e9db21c` | `915b7e23517dba67db9aa8b20f18f3451897fe7ab2bae1cd64bc22810e38efa3f2f62483ba377c6d1fed738de24874c2c14ef772f02f201f41a35d80eb2f872b` | `2702663c0bc4316e83573c6c262040e72c13c6ea50b9dd042dd8d375e719e8729db759db3bffa8bd6a838661278e02a3047ce402b0348ae082393b37643047cb` filename | sha512 hash -- | -- | `633cba8102648735b93d91a9840a39597b3242d2489081e71b0131a9bf246e2f52060bc8f8abcc2154f39c1e2410dec5e8b189d220743b55f861062c86edd2f8` | `39eb3f85a46f6c71ba70ffa391e706b6c57c8f9fb7eea95960f944a3fee7883da56e3dca9eec7f3121800911e45681b2bcc78c4585ce7081720a8658e1837f47` | `91236a70b0ff67b54c939215ac71a7e03e4202e71d8b11f687fc6406eb54da6271e8f095a4e812b3aaad23275e0a04b4cfa7bfa8f455a98ccf8c5a22b8b5e59f` | `a41f590fcc271861d73cae14032c51d7674efba48f160550da8be7095240be39a59bdd8012886eb94afa8062576e43643a76c2796ce847d6df33d7147a807e9f` | `2bb2d3087e911e5c296ae194697190470606c1ac761fb3e69533492109d012e756428daac93e89665f6bbac7c4b4fd1ab9e554718244234f43c6e7219c396eff` filename | sha512 hash -- | -- | `4b0d9188914f3b8dbb7cab384b9f2a63b5d0f53b78c7c9d4920613e780d17afffd95809eb828753556048481634b5a5bd0b89b50360e4d3f2a37d21bb88d5a36` | `039c77a9a5f7e12d826ded7f8590b9f63c79863fcc463e150848ab6304de171aebbdc948f9398478b96a763f982d02f955f8327f651a04e8caced4c6fd2b8e12` | `d60c740f2f5b2ffe95bc9d1ea0e26ca6af1aef718d5c498339eabe6a9cc1cdff8cb69f9cc0f2389adb1eaa3dacdbcaacae3e19fd27ce035f561813d8f29b5f90` | `163b33fcab0226e950c2d2415ece74ac840c592ede2e7276babab466ef3ef1cdbee95102ca4cbefeb836ce1b2897d19b0534967dbd0dcb6706753b453adb27eb` | `c01db605cc3b744a6a13962e318f94ae66eb221d4ed3758a868452bc610cdcf79297332e158cf9e88002e3ac72c9489b7c27b001ba40df1852c6ba098afb9586` | `d5431cd60990bd56649ce11e3c5a72b92a733e6abafaee6517cec333a3dcabd6469ebd4e3d9b052d250ecb35a0c1dcac16cd841612d73886438731f0f817e2d1` FlexVolume is deprecated. Out-of-tree CSI driver is the recommended way to write volume drivers in Kubernetes. See for more information. Maintainers of FlexVolume drivers should implement a CSI driver and move users of FlexVolume to CSI. Users of FlexVolume should move their workloads to CSI driver. To simplify the code base, several in Kubernetes 1.23. The code which implements them will be removed in a future release, so users of those need to start replacing the deprecated flags with some alternative solutions. Kubernetes releases are now generating provenance attestation files describing the staging and release phases of the release process and artifacts are verified as they are handed over from one phase to the next. This final piece completes the work needed to comply with Level 1 of the (Supply-chain Levels for Software Artifacts). graduates to GA. Since 1.21, Kubernetes clusters are enabled to support dual-stack networking by default. In 1.23, the `IPv6DualStack` feature gate is removed. The use of dual-stack networking is not mandatory. Although clusters are enabled to support dual-stack networking, Pods and Services continue to default to single-stack. To use dual-stack networking: Kubernetes nodes have routable IPv4/IPv6 network interfaces, a dual-stack capable CNI network plugin is used, Pods are configured to be dual-stack and Services have their `.spec.ipFamilyPolicy` field set to either `PreferDualStack` or `RequireDualStack`. Version 2 of the HorizontalPodAutoscaler API graduates to stable in the 1.23 release. The HorizontalPodAutoscaler `autoscaling/v2beta2` API is deprecated in favor of the new `autoscaling/v2` API, which the Kubernetes project recommends for all use cases. This release does not deprecate the v1 HorizontalPodAutoscaler API. The generic ephemeral volume feature moved to GA in 1.23. This feature allows any existing storage driver that supports dynamic provisioning to be used as an ephemeral volume with the volumes lifecycle bound to the Pod. All StorageClass parameters for volume provisioning and all features supported with PersistentVolumeClaims are supported. The feature to configure volume permission and ownership change policy for Pods moved to GA in 1.23. This allows users to skip recursive permission changes on mount and speeds up the pod start up time. The feature to allow CSI Drivers to declare support for fsGroup based permissions graduates to GA in 1.23. moves to Beta. `PodSecurity` replaces the deprecated `PodSecurityPolicy` admission controller. `PodSecurity` is an admission controller that enforces Pod Security Standards on Pods in a Namespace based on specific namespace labels that set the enforcement level. In 1.23, the `PodSecurity` feature gate is enabled by default. The Kubelet now supports the CRI `v1` API, which is now the project-wide

If a container runtime does not support the `v1` API, Kubernetes will fall back to the `v1alpha2` implementation. There is no intermediate action required by end-users, because `v1` and `v1alpha2` do not differ in their implementation. It is likely that `v1alpha2` will be removed in one of the future Kubernetes releases to be able to develop `v1`. Structured logging reached its Beta milestone. Most log messages from kubelet and kube-scheduler have been converted. Users are encouraged to try out JSON output or parsing of the structured text format and provide feedback on possible solutions for the open issues, such as handling of multi-line strings in log values. The kube-scheduler is adding a new, simplified config field for Plugins to allow multiple extension points to be enabled in one spot. The new `multiPoint` plugin field is intended to simplify most scheduler setups for administrators. Plugins that are enabled via `multiPoint` will automatically be registered for each individual extension point that they implement. For example, a plugin that implements Score and Filter extensions can be simultaneously enabled for both. This means entire plugins can be enabled and disabled without having to manually edit individual extension point settings. These extension points can now be abstracted away due to their irrelevance for most users. CSI Migration enables the replacement of existing in-tree storage plugins such as `kubernetes.io/gce-pd` or `kubernetes.io/aws-ebs` with a corresponding CSI driver. If CSI Migration is working properly, Kubernetes end users shouldnt notice a difference. After migration, Kubernetes users may continue to rely on all the functionality of in-tree storage plugins using the existing interface. CSI Migration feature is turned on by default but stays in Beta for GCE PD, AWS EBS, and Azure Disk in 1.23. CSI Migration is introduced as an Alpha feature for Ceph RBD and Portworx in 1.23. Kubeadm: remove the deprecated flag `--experimental-patches` for the `init|join|upgrade` commands. The flag `--patches` is no longer allowed in a mixture with the flag `--config`. Please use the kubeadm configuration for setting patches for a node using `{Init|Join}Configuration.patches`. (, ) Log messages in JSON format are written to stderr by default now (same as text format) instead of stdout. Users who expected JSON output on stdout must now capture stderr instead or in addition to stdout. (, ) [SIG API Machinery, Architecture, Cluster Lifecycle and Instrumentation] Support for the seccomp annotations `seccomp.security.alpha.kubernetes.io/pod` and `container.seccomp.security.alpha.kubernetes.io/, ) is included in release tar balls. It can be used to replace the deprecated `--log-file` parameter. (, ) [SIG API Machinery, Architecture, Cloud Provider, Cluster Lifecycle and Instrumentation] Kubernetes is built using golang 1.17. This version of go removes the ability to use a `GODEBUG=x509ignoreCN=0` environment setting to re-enable deprecated legacy behavior of treating the CommonName of X.509 serving certificates as a host name. This behavior has been disabled by default since Kubernetes 1.19 / go 1.15. Serving certificates used by admission webhooks, custom resource conversion webhooks, and aggregated API servers must now include valid Subject Alternative Names. If you are running Kubernetes 1.22 with `GODEBUG=x509ignoreCN=0` set, check the `apiserverkubeaggregatorx509missingsantotal` and `apiserverwebhooksx509missingsan_total` metrics for non-zero values to see if the API server is connecting to webhooks or aggregated API servers using certificates that will be considered invalid in Kubernetes 1.23+. Data corruption issue was found in etcd v3.5.0 release that was shipped with 1.22 Kubernetes release. Please read up-to-date . A deprecation notice has been added when using the kube-proxy userspace proxier, which will be removed in v1.25. (#103860) (, ) Added `apiserverlongrunningrequests` metric to replace the soon to be deprecated `apiserverlongrunninggauge` metric. (, ) Controller-manager: the following flags have no effect and would be removed in

`--port` `--address` The insecure port flags `--port` may only be set to 0 now. Kube-scheduler: the `--port` and `--address` flags have no effect and would be removed in v1.24. The insecure port flags `--port` may only be set to 0 now. Also `metricsBindAddress` and `healthzBindAddress` fields from `kubescheduler.config.k8s.io/v1beta1` are no-op and expected to be empty. Removed in `kubescheduler.config.k8s.io/v1beta2` completely. (, ) In addition, please be careful that: kube-scheduler MUST start with `--authorization-kubeconfig` and `--authentication-kubeconfig` correctly set to get authentication/authorization working. liveness/readiness probes to kube-scheduler MUST use HTTPS now, and the default port has been changed to 10259. Applications that fetch metrics from kube-scheduler should use a dedicated service account which is allowed to access nonResourceURLs `/metrics`. (, ) [SIG Cloud Provider, Scheduling and Testing] Feature-gate `VolumeSubpath` has been deprecated and cannot be disabled. It will be completely removed in 1.25 (, ) Kubeadm: add a new `output/v1alpha2` API that is identical to the `output/v1alpha1`, but attempts to resolve some internal dependencies with the `kubeadm/v1beta2` API. The `output/v1alpha1` API is now deprecated and will be removed in a future release. (, ) Kubeadm: add the kubeadm specific, Alpha (disabled by default) feature gate UnversionedKubeletConfigMap. When this feature is enabled kubeadm will start using a new naming format for the ConfigMap where it stores the KubeletConfiguration structure. The old format included the Kubernetes version - "kube-system/kubelet-config-1.22", while the new format does not - "kube-system/kubelet-config". A similar formatting change is done for the related RBAC rules. The old format is now DEPRECATED and will be removed after the feature graduates to GA. When writing the ConfigMap kubeadm (init, upgrade apply) will respect the value of UnversionedKubeletConfigMap, while when reading it (join, reset, upgrade), it would attempt to use new format first and fallback to the legacy format if needed. (, ) [SIG Cluster Lifecycle and Testing] Kubeadm: remove the deprecated / NO-OP phase `update-cluster-status` in `kubeadm reset` (, ) Remove 'master' as a valid EgressSelection type in the EgressSelectorConfiguration API. (, ) Removed `kubectl --dry-run` empty default value and boolean values. `kubectl --dry-run` usage must be specified with `--dry-run=(server|client|none)`. (, ) Removed deprecated metric `schedulervolumeschedulingdurationseconds`. (, ) The deprecated `--experimental-bootstrap-kubeconfig` flag has been removed. This can be set via `--bootstrap-kubeconfig`. (, ) A new field `omitManagedFields` has been added to both `audit.Policy` and `audit.PolicyRule` so cluster operators can opt in to omit managed fields of the request and response bodies from being written to the API audit log. (, ) [SIG API Machinery, Auth, Cloud Provider and Testing] A small regression in Service updates was fixed. The circumstances are so unlikely that probably nobody would ever hit it. (, ) Added a feature gate `StatefulSetAutoDeletePVC`, which allows PVCs automatically created for StatefulSet pods to be automatically deleted. (, ) Client-go impersonation config can specify a UID to pass impersonated uid information through in requests. (, ) Create HPA v2 from v2beta2 with some fields changed. (, ) [SIG API Machinery, Apps, Auth, Autoscaling and Testing] Ephemeral containers graduated to beta and are now available by default. (, ) Fix kube-proxy regression on UDP services because the logic to detect stale connections was not considering if the endpoint was ready. (, ) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Contributor Experience, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows] If a conflict occurs when creating an object with `generateName`, the server now returns an "AlreadyExists" error with a retry

(, ) Implement support for recovering from volume expansion failures (, ) [SIG API Machinery, Apps and Storage] In kubelet, log verbosity and flush frequency can also be configured via the configuration file and not just via command line flags. In other commands (kube-apiserver, kube-controller-manager), the flags are listed in the "Logs flags" group and not under "Global" or "Misc". The type for `-vmodule` was made a bit more descriptive (`pattern=N,...` instead of `moduleSpec`). (, ) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling] Introduce `OS` field in the PodSpec (, ) Introduce `v1beta3` API for scheduler. This version increases the weight of user specifiable priorities. The weights of following priority plugins are increased `TaintTolerations` to 3 - as leveraging node tainting to group nodes in the cluster is becoming a widely-adopted practice `NodeAffinity` to 2 `InterPodAffinity` to 2 Won't have `HealthzBindAddress`, `MetricsBindAddress` fields (, ) Introduce v1beta2 for Priority and Fairness with no changes in API spec. (, ) JSON log output is configurable and now supports writing info messages to stdout and error messages to stderr. Info messages can be buffered in memory. The default is to write both to stdout without buffering, as before. (, ) JobTrackingWithFinalizers graduates to beta. Feature is enabled by default. (, ) Kube-apiserver: Fixes handling of CRD schemas containing literal null values in enums. (, ) Kube-apiserver: The `rbac.authorization.k8s.io/v1alpha1` API version is removed; use the `rbac.authorization.k8s.io/v1` API, available since v1.8. The `scheduling.k8s.io/v1alpha1` API version is removed; use the `scheduling.k8s.io/v1` API, available since v1.14. (, ) Kube-scheduler: support for configuration file version `v1beta1` is removed. Update configuration files to v1beta2(xref: https://github.com/kubernetes/enhancements/issues/2901) or v1beta3 before upgrading to 1.23. (, ) KubeSchedulerConfiguration provides a new field `MultiPoint` which will register a plugin for all valid extension points (, ) [SIG Scheduling and Testing] Kubelet should reject pods whose OS doesn't match the node's OS label. (, ) [SIG Apps and Node] Kubelet: turn the KubeletConfiguration v1beta1 `ResolverConfig` field from a `string` to `*string`. (, ) Kubernetes is now built using go 1.17. (, ) Performs strict server side schema validation requests via the `fieldValidation=, ) Promote `IPv6DualStack` feature to stable. Controller Manager flags for the node IPAM controller have slightly changed: When configuring a dual-stack cluster, the user must specify both `--node-cidr-mask-size-ipv4` and `--node-cidr-mask-size-ipv6` to set the per-node IP mask sizes, instead of the previous `--node-cidr-mask-size` flag. The `--node-cidr-mask-size` flag is mutually exclusive with `--node-cidr-mask-size-ipv4` and `--node-cidr-mask-size-ipv6`. Single-stack clusters do not need to change, but may choose to use the more specific flags. Users can use either the older `--node-cidr-mask-size` flag or one of the newer `--node-cidr-mask-size-ipv4` or `--node-cidr-mask-size-ipv6` flags to configure the per-node IP mask size, provided that the flag's IP family matches the cluster's IP family (--cluster-cidr). (, ) Remove `NodeLease` feature gate that was graduated and locked to stable in 1.17 release. (, ) Removed deprecated `--seccomp-profile-root`/`seccompProfileRoot` config. (, ) Since golang 1.17 both net.ParseIP and net.ParseCIDR rejects leading zeros in the dot-decimal notation of IPv4 addresses, Kubernetes will keep allowing leading zeros on IPv4 address to not break the compatibility. IMPORTANT: Kubernetes interprets leading zeros on IPv4 addresses as decimal, users must not rely on parser alignment to not being impacted by the associated security advisory: CVE-2021-29923 golang standard library "net" - Improper Input Validation of octal literals in golang 1.16.2 and below standard library "net" results in indeterminate SSRF & RFI vulnerabilities. Reference: https://nvd.nist.gov/vuln/detail/CVE-2021-29923 (, ) StatefulSet `minReadySeconds` is promoted to beta. (, ) Support pod priority based node graceful shutdown. (, ) The "Generic Ephemeral Volume" feature graduates to GA. It is now enabled unconditionally. (, ) The Kubelet's `--register-with-taints` option is now available via the Kubelet config file field registerWithTaints (, ) [SIG Node and Scalability] The

can now be modified. (, ) The `CSIVolumeFSGroupPolicy` feature has moved from beta to GA. (, ) The `IngressClass.Spec.Parameters.Namespace` field is now GA. (, ) The `Service.spec.ipFamilyPolicy` field is now required in order to create or update a Service as dual-stack. This is a breaking change from the beta behavior. Previously the server would try to infer the value of that field from either `ipFamilies` or `clusterIPs`, but that caused ambiguity on updates. Users who want a dual-stack Service MUST specify `ipFamilyPolicy` as either "PreferDualStack" or "RequireDualStack". (, ) The `TTLAfterFinished` feature gate is now GA and enabled by default. (, ) The `kube-controller-manager` supports `--concurrent-ephemeralvolume-syncs` flag to set the number of ephemeral volume controller workers. (, ) The legacy scheduler policy config is removed in v1.23, the associated flags `policy-config-file`, `policy-configmap`, `policy-configmap-namespace` and `use-legacy-policy-config` are also removed. Migrate to Component Config instead, see https://kubernetes.io/docs/reference/scheduling/config/ for details. (, ) Track the number of Pods with a Ready condition in Job status. The feature is alpha and needs the feature gate JobReadyPods to be enabled. (, ) Users of `LogFormatRegistry` in component-base must update their code to use the logr v1.0.0 API. The JSON log output now uses the format from go-logr/zapr (no `v` field for error messages, additional information for invalid calls) and has some fixes (correct source code location for warnings about invalid log calls). (, ) Validation rules for Custom Resource Definitions can be written in the using the `x-kubernetes-validations` extension in OpenAPIv3 schemas (alpha). This is gated by the alpha "CustomResourceValidationExpressions" feature gate. (, ) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing] (beta feature) If the CSI driver supports the NodeServiceCapability `VOLUMEMOUNTGROUP` and the `DelegateFSGroupToCSIDriver` feature gate is enabled, kubelet will delegate applying FSGroup to the driver by passing it to NodeStageVolume and NodePublishVolume, regardless of what other FSGroup policies are set. (, ) [SIG Storage] Add a new `distribute-cpus-across-numa` option to the static `CPUManager` policy. When enabled, this will trigger the `CPUManager` to evenly distribute CPUs across NUMA nodes in cases where more than one NUMA node is required to satisfy the allocation. (, ) Add fish shell completion to kubectl. (, ) Add mechanism to load simple sniffer class into fluentd-elasticsearch image (, ) Add support for Portworx plugin to csi-translation-lib. Alpha release Portworx CSI driver is required to enable migration. This PR adds support of the `CSIMigrationPortworx` feature gate, which can be enabled by: Adding the feature flag to the kube-controller-manager `--feature-gates=CSIMigrationPortworx=true` Adding the feature flag to the kubelet config: featureGates: CSIMigrationPortworx: true (, ) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows] Add support to generate client-side binaries for windows/arm64 platform (, ) Added PowerShell completion generation by running `kubectl completion powershell`. (, ) Added a `Processing` condition for the `workqueue` API. Changed `Shutdown` for the `workqueue` API to wait until the work queue finishes processing all in-flight items. (, ) Added a new feature gate `CustomResourceValidationExpressions` to enable expression validation for Custom Resource. (, ) Added a new flag `--append-server-path` to `kubectl proxy` that will automatically append the kube context server path to each request. (, ) Added ability for `kubectl wait` to wait on arbitary JSON path (, ) Added support for `PodAndContainerStatsFromCRI` feature gate, which allows a user to specify their pod stats must also come from the CRI, not `cAdvisor`. (, ) Added support for setting controller-manager log level

(, ) Added the ability to specify whether to use an RFC7396 JSON Merge Patch, an RFC6902 JSON Patch, or a Strategic Merge Patch to perform an override of the resources created by `kubectl run` and `kubectl expose`. (, ) Adding option for `kubectl cp` to resume on network errors until completion, requires tar in addition to tail inside the container image (, ) Adding support for multiple `--from-env-file flags`. (, ) Adding support for multiple `--from-env-file` flags. (, ) Adds `--as-uid` flag to `kubectl` to allow uid impersonation in the same way as user and group impersonation. (, ) Adds new , ) Allow node expansion of local volumes. (, ) Allow to build kubernetes with a custom `kube-cross` image. (, ) Allows users to prevent garbage collection on pinned images (, ) [SIG Node] CRI v1 is now the project default. If a container runtime does not support the v1 API, Kubernetes will fall back to the v1alpha2 implementation. (, ) Changed feature `CSIMigrationAWS` to on by default. This feature requires the AWS EBS CSI driver to be installed. (, ) Client-go: pass `DeleteOptions` down to the fake client `Reactor` (, ) Cloud providers can set service account names for cloud controllers. (, ) Display Labels when kubectl describe ingress. (, ) Enhance scheduler `VolumeBinding` plugin to handle Lost PVC as `UnschedulableAndUnresolvable` (, ) Ensures that volume is deleted from the storage backend when the user tries to delete the PV object manually and the PV `ReclaimPolicy` is set to `Delete`. (, ) Expose a `NewUnstructuredExtractor` from apply configurations `meta/v1` package that enables extracting objects into unstructured apply configurations. (, ) Feature gate `StorageObjectInUseProtection` has been deprecated and cannot be disabled. It will be completely removed in 1.25 (, ) Graduating `controlleradmissiondurationseconds`, `stepadmissiondurationseconds`, `webhookadmissiondurationseconds`, `apiservercurrentinflightrequests` and `apiserverresponsesizes` metrics to stable. (, ) [SIG API Machinery, Instrumentation and Testing] Graduating `pendingpods`, `preemptionattemptstotal`, `preemptionvictims` and `scheduleattemptstotal` metrics to stable. Also `e2eschedulingdurationseconds` is renamed to `schedulingattemptdurationseconds` and the latter is graduated to stable. (, ) [SIG Instrumentation, Scheduling and Testing] Health check of kube-controller-manager now includes each controller. (, ) Integration testing now takes periodic Prometheus scrapes from the etcd server. There is a new script ,`hack/run-prometheus-on-etcd-scrapes.sh`, that runs a containerized Prometheus server against an archive of such scrapes. (, ) [SIG API Machinery and Testing] Introduce a feature gate `DisableKubeletCloudCredentialProviders` which allows disabling the in-tree kubelet credential providers. The feature gate `DisableKubeletCloudCredentialProviders` is currently in Alpha, which means is currently disabled by default. Once this feature gate moves to beta, in-tree credential providers will be disabled by default, and users will need to migrate to use external credential providers. (, ) Introduces a new metric: `admissionwebhookrequest_total` with the following labels: name (string) - the webhook name, type (string) - the admission type, operation (string) - the requested verb, code (int) - the HTTP status code, rejected (bool) - whether the request was rejected, namespace (string) - the namespace of the requested resource. (, ) Kubeadm: add support for dry running `kubeadm join`. The new flag `kubeadm join --dry-run` is similar to the existing flag for `kubeadm init/upgrade` and allows you to see what changes would be applied. (, ) Kubeadm: do not check if the `/etc/kubernetes/manifests` folder is empty on joining worker nodes during preflight (, ) Kubectl will now provide shell completion choices for the `--output/-o` flag (, ) Kubelet should reconcile `kubernetes.io/os` and `kubernetes.io/arch` labels on the node object. The side-effect of this is kubelet would deny admission to pod which has nodeSelector with label `kubernetes.io/os` or `kubernetes.io/arch` which doesn't match the underlying OS or arch on the host

The label reconciliation happens as part of periodic status update which can be configured via flag `--node-status-update-frequency` (, ) [SIG Node, Testing and Windows] Kubernetes is now built with Golang 1.16.7. (, ) Kubernetes is now built with Golang 1.17.1. (, ) Kubernetes is now built with Golang 1.17.2 (, ) Kubernetes is now built with Golang 1.17.3 (, ) [SIG API Machinery, Cloud Provider, Instrumentation, Release and Testing] Move `ConfigurableFSGroupPolicy` to GA and rename metric `volumefsgrouprecursiveapply` to `volumeapplyaccesscontrol` (, ) Move the `GetAllocatableResources` Endpoint in PodResource API to the beta that will make it enabled by default. (, ) Moving `WindowsHostProcessContainers` feature to beta (, ) Node affinity, Node selectors, and tolerations are now mutable for Jobs that are suspended and have never been started (, ) Pod template annotations and labels are now mutable for Jobs that are suspended and have never been started (, ) PodSecurity: in 1.23+ restricted policy levels, Pods and containers which set `runAsUser=0` are forbidden at admission-time; previously, they would be rejected at runtime (, ) Shell completion now knows to continue suggesting resource names when the command supports it. For example `kubectl get pod pod1 <TAB>` will suggest more Pod names. (, ) Support to enable Hyper-V in GCE Windows Nodes created with `kube-up` (, ) The CPUManager policy options are now enabled, and we introduce a graduation path for the new CPU Manager policy options. (, ) The Pods and Pod controllers that are exempted from the PodSecurity admission process are now marked with the `pod-security.kubernetes.io/exempt: user/namespace/runtimeClass` annotation, based on what caused the exemption. The enforcement level that allowed or denied a Pod during PodSecurity admission is now marked by the `pod-security.kubernetes.io/enforce-policy` annotation. The annotation that informs about audit policy violations changed from `pod-security.kubernetes.io/audit` to `pod-security.kubernetes.io/audit-violation`. (, ) The `/openapi/v3` endpoint will be populated with OpenAPI v3 if the feature flag is enabled (, ) The `CSIMigrationGCE` feature flag is turned `ON` by default (, ) The `DownwardAPIHugePages` feature is now enabled by default. (, ) The `PodSecurity `admission plugin has graduated to `beta` and is enabled by default. The admission configuration version has been promoted to `pod-security.admission.config.k8s.io/v1beta1`. See https://kubernetes.io/docs/concepts/security/pod-security-admission/ for usage guidelines. (, ) The `ServiceAccountIssuerDiscovery` feature gate is removed. It reached GA in Kubernetes 1.21. (, ) The `constants/variables` from k8s.io for STABLE metrics is now supported. (, ) The `kubectl describe namespace` now shows Conditions (, ) The etcd container image now supports Windows. (, ) The kube-apiserver's Prometheus metrics have been extended with some that describe the costs of handling LIST requests. They are as follows. apiserver_cache_list_total: Counter of LIST requests served from watch cache, broken down by resourceprefix and indexname apiserver_cache_list_fetched_objects_total: Counter of objects read from watch cache in the course of serving a LIST request, broken down by resourceprefix and indexname apiserver_cache_list_evaluated_objects_total: Counter of objects tested in the course of serving a LIST request from watch cache, broken down by resource_prefix apiserver_cache_list_returned_objects_total: Counter of objects returned for a LIST request from watch cache, broken down by resource_prefix apiserver_storage_list_total: Counter of LIST requests served from etcd, broken down by resource apiserver_storage_list_fetched_objects_total: Counter of objects read from etcd in the course of serving a LIST request, broken down by resource apiserver_storage_list_evaluated_objects_total: Counter of objects tested in the course of serving a LIST request from etcd, broken down by resource apiserver_storage_list_returned_objects_total: Counter of objects returned for a LIST request from etcd, broken down by resource (, ) The pause image list now contains Windows Server 2022. (, ) The script `kube-up.sh` installs `csi-proxy

(, ) This PR adds the following metrics for API Priority and Fairness. apiserver_flowcontrol_priority_level_seat_count_samples: histograms of seats occupied by executing requests (both regular and final-delay phases included), broken down by priority_level; the observations are taken once per millisecond. apiserver_flowcontrol_priority_level_seat_count_watermarks: histograms of high and low watermarks of number of seats occupied by executing requests (both regular and final-delay phases included), broken down by priority_level. apiserver_flowcontrol_watch_count_samples: histograms of number of watches relevant to a given mutating request, broken down by that request's prioritylevel and flowschema. (, ) [SIG API Machinery, Instrumentation and Testing] Topology Aware Hints have graduated to beta. (, ) [SIG Network] Turn on CSIMigrationAzureDisk by default on 1.23 (, ) Update the system-validators library to v1.6.0 (, ) [SIG Cluster Lifecycle and Node] Updated Cluster Autosaler to version `1.22.0`. Release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.22.0. (, ) Updates `debian-iptables` to v1.6.7 to pick up CVE fixes. (, ) Updates the following images to pick up CVE fixes: `debian` to v1.9.0 `debian-iptables` to v1.6.6 `setcap` to v2.0.4 (, ) Upgrade etcd to 3.5.1 (, ) [SIG Cloud Provider, Cluster Lifecycle and Testing] When feature gate `JobTrackingWithFinalizers` is enabled: Limit the number of Pods tracked in a single Job sync to avoid starvation of small Jobs. The metric `jobpodfinished_total` counts the number of finished Pods tracked by the Job controller. (, ) When using `RequestedToCapacityRatio` ScoringStrategy, empty shape will cause error. (, ) [SIG Scheduling] `client-go` event library allows customizing spam filtering function. It is now possible to override `SpamKeyFunc`, which is used by event filtering to detect spam in the events. (, ) `client-go`, using log level 9, traces the following events of a HTTP request: DNS lookup TCP dialing TLS handshake Time to get a connection from the pool Time to process a request (, ) Graduating `podschedulingdurationseconds`, `podschedulingattempts`, `frameworkextensionpointdurationseconds`, `pluginexecutiondurationseconds` and `queueincomingpods_total` metrics to stable. (, ) [SIG Instrumentation, Scheduling and Testing] The test ", ) Update description of `--audit-log-maxbackup` to describe behavior when `value = 0`. (, ) Users should not rely on unsupported CRON_TZ variable when specifying schedule, both the API server and cronjob controller will emit warnings pointing to https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/ containing explanation (, ) [SIG Apps] Fixes hostPath storage E2E tests within SELinux enabled env (, ) (PodSecurity admission) errors validating workload resources (deployment, replicaset, etc.) no longer block admission. (, ) [SIG Auth] A pod that the Kubelet rejects was still considered as being accepted for a brief period of time after rejection, which might cause some pods to be rejected briefly that could fit on the node. A pod that is still terminating (but has status indicating it has failed) may also still be consuming resources and so should also be considered. (, ) Add Kubernetes Events to the `Kubelet Graceful Shutdown` feature. (, ) Add Pod Security admission metrics: `podsecurityevaluationstotal`, `podsecurityexemptionstotal`, `podsecurityerrors_total` (, ) Add support for Windows Network stats in Containerd (, ) [SIG Node, Testing and Windows] Added show-capacity option to `kubectl top node` to show `Capacity` resource usage (, ) [SIG CLI] Apimachinery: Pretty printed JSON and YAML output is now indented consistently. (, ) Be able to create a Pod with Generic Ephemeral Volumes as raw block devices. (, ) CA, certificate and key bundles for the `generic-apiserver` based servers will be reloaded immediately after the files are changed. (, ) Change `kubectl diff --invalid-arg` status code from 1 to 2 to match docs (, ) Changed kubectl describe to compute age of an event using the `EventSeries.count` and

(, ) Changes behaviour of kube-proxy start; does not attempt to set specific `sysctl` values (which does not work in recent Kernel versions anymore in non-init namespaces), when the current sysctl values are already set higher. (, ) Client-go uses the same HTTP client for all the generated groups and versions, allowing to share customized transports for multiple groups versions. (, ) Disable aufs module for gce clusters. (, ) Do not unmount and mount subpath bind mounts during container creation unless bind mount changes (, ) [SIG Storage] Don't prematurely close reflectors in case of slow initialization in watch based manager to fix issues with inability to properly mount secrets/configmaps. (, ) Don't use a custom dialer for the kubelet if is not rotating certificates, so we can reuse TCP connections and have only one between the apiserver and the kubelet. If users experiment problems with stale connections using HTTP1.1, they can force the previous behavior of the kubelet by setting the environment variable DISABLE_HTTP2. (, ) [SIG API Machinery, Auth and Node] EndpointSlice Mirroring controller now cleans up managed EndpointSlices when a Service selector is added (, ) [SIG Apps, Network and Testing] Enhanced event messages for pod failed for exec probe timeout (, ) [SIG Node] Ensure Pods are removed from the scheduler cache when the scheduler misses deletion events due to transient errors (, ) [SIG Scheduling] Ensure `InstanceShutdownByProviderID` return false for creating Azure VMs. (, ) Evicted and other terminated Pods will no longer revert to the `Running` phase. (, ) Fix `kube-apiserver` metric reporting for the deprecated watch path of `/api/<version>/watch/...`. (, ) Fix a regression where the Kubelet failed to exclude already completed pods from calculations about how many resources it was currently using when deciding whether to allow more pods. (, ) Fix detach disk issue on deleting vmss node. (, ) Fix job controller syncs: In case of conflicts, ensure that the sync happens with the most up to date information. Improves reliability of JobTrackingWithFinalizers. (, ) Fix job tracking with finalizers for more than 500 pods, ensuring all finalizers are removed before counting the Pod. (, ) Fix pod name of NonIndexed Jobs to not include rogue `-1` substring (, ) Fix scoring for `NodeResourcesBalancedAllocation` plugins when nodes have containers with no requests. (, ) Fix system default topology spreading when nodes don't have zone labels. Pods correctly spread by default now. (, ) Fix: do not try to delete a LoadBalancer that does not exist (, ) Fix: ignore non-VMSS error for VMAS nodes in `reconcileBackendPools`. (, ) Fix: leave the probe path empty for TCP probes (, ) Fix: remove VMSS and VMSS instances from SLB backend pool only when necessary (, ) Fix: skip `instance not found` when decoupling VMSSs from LB (, ) Fix: skip case sensitivity when checking Azure NSG rules. (, ) Fixed a bug that prevents a PersistentVolume that has a PersistentVolumeClaim UID which doesn't exist in local cache but exists in etcd from being updated to the Released phase. (, ) Fixed a bug where using `kubectl patch` with `$deleteFromPrimitiveList` on a nonexistent or empty list would add the item to the list (, ) Fixed a bug which could cause webhooks to have an incorrect copy of the old object after an Apply or Update (, ) [SIG API Machinery] Fixed a bug which kubectl would emit duplicate warning messages for flag names that contain an underscore and recommend using a nonexistent flag in some cases. (, ) Fixed a panic in `kubectl` when creating secrets with an improper output type (, ) Fixed a regression setting `--audit-log-path=-` to log to stdout in 1.22

(, ) Fixed an issue which didn't append OS's environment variables with the one provided in Credential Provider Config file, which may fail execution of external credential provider binary. See https://github.com/kubernetes/kubernetes/issues/102750. (, ) Fixed applying of SELinux labels to CSI volumes on very busy systems (with "error checking for SELinux support: could not get consistent content of /proc/self/mountinfo after 3 attempts") (, ) [SIG Storage] Fixed architecture within manifest for non `amd64` etcd images. (, ) Fixed architecture within manifest for non `amd64` etcd images. (, ) Fixed azure disk translation issue due to lower case `managed` kind. (, ) Fixed client IP preservation for NodePort service with protocol SCTP in ipvs. (, ) Fixed concurrent map access causing panics when logging timed-out API calls. (, ) Fixed consolidate logs for `instance not found` error Fixed skip `not found` nodes when reconciling LB backend address pools (, ) Fixed occasional pod cgroup freeze when using cgroup v1 and systemd driver. (, ) Fixed the issue where logging output of kube-scheduler configuration files included line breaks and escape characters. The output also attempted to output the configuration file in one section without showing the user a more readable format. (, ) [SIG Scheduling] Fixes a bug that could result in the EndpointSlice controller unnecessarily updating EndpointSlices associated with a Service that had Topology Aware Hints enabled. (, ) Fixes a regression that could cause panics in LRU caches in controller-manager, kubelet, kube-apiserver, or client-go. (, ) Fixes an issue where an admission webhook can observe a v1 Pod object that does not have the `defaultMode` field set in the injected service account token volume in kube-api-server. (, ) Fixes the `should support building a client with a CSR` E2E test to work with clusters configured with short certificate lifetimes (, ) Graceful node shutdown, allow the actual inhibit delay to be greater than the expected inhibit delay. (, ) Handle Generic Ephemeral Volumes properly in the node limits scheduler filter and the kubelet `hostPath` check. (, ) Headless Services with no selector which were created without dual-stack enabled will be defaulted to RequireDualStack instead of PreferDualStack. This is consistent with such Services which are created with dual-stack enabled. (, ) Ignore `not a vmss instance` error for VMAS nodes in `EnsureBackendPoolDeleted`. (, ) Ignore the case when comparing azure tags in service annotation. (, ) Ignore the case when updating Azure tags. (, ) Introduce a new server run option 'shutdown-send-retry-after'. If true the HTTP Server will continue listening until all non longrunning request(s) in flight have been drained, during this window all incoming requests will be rejected with a status code `429` and a 'Retry-After' response header. (, ) Kube-apiserver: Avoid unnecessary repeated calls to `admission webhooks` that reject an update or delete request. (, ) Kube-apiserver: Server Side Apply merge order is reverted to match v1.22 behavior until `http://issue.k8s.io/104641` is resolved. (, ) Kube-apiserver: events created via the `events.k8s.io` API group for cluster-scoped objects are now permitted in the default namespace as well for compatibility with events clients and the `v1` API (, ) Kube-apiserver: fix a memory leak when deleting multiple objects with a `deletecollection`. (, ) Kube-proxy health check ports used to listen to `:<port>` for each of the services. This is not needed and opens ports in addresses the cluster user may not have intended. The PR limits listening to all node address which are controlled by `--nodeport-addresses`

if no addresses are provided then we default to existing behavior by listening to `:<port>` for each service (, ) Kube-proxy: delete stale conntrack UDP entries for loadbalancer ingress IP. (, ) Kube-scheduler now doesn't print any usage message when unknown flag is specified. (, ) Kube-up now includes CoreDNS version v1.8.6 (, ) [SIG Cloud Provider] Kubeadm: When adding an etcd peer to an existing cluster, if an error is returned indicating the peer has already been added, this is accepted and a `ListMembers` call is used instead to return the existing cluster. This helps to diminish the exponential backoff when the first AddMember call times out, while still retaining a similar performance when the peer has already been added from a previous call. (, ) Kubeadm: do not allow empty `--config` paths to be passed to `kubeadm kubeconfig user` (, ) Kubeadm: fix a bug on Windows worker nodes, where the downloaded KubeletConfiguration from the cluster can contain Linux paths that do not work on Windows and can trip the kubelet binary. (, ) [SIG Cluster Lifecycle and Windows] Kubeadm: switch the preflight check (called 'Swap') that verifies if swap is enabled on Linux hosts to report a warning instead of an error. This is related to the graduation of the NodeSwap feature gate in the kubelet to Beta and being enabled by default in 1.23 - allows swap support on Linux hosts. In the next release of kubeadm (1.24) the preflight check will be removed, thus we recommend that you stop using it - e.g. via `--ignore-preflight-errors` or the kubeadm config. (, ) Kubelet did not report `kubeletvolumestats_*` metrics for Generic Ephemeral Volumes. (, ) Kubelet's Node Grace Shutdown will terminate probes when shutting down (, ) Kubelet: fixes a file descriptor leak in log rotation (, ) [SIG Node] Kubelet: the printing of flags at the start of kubelet now uses the final logging configuration. (, ) Make the etcd client (used by the API server) retry certain types of errors. The full list of retriable (codes.Unavailable) errors can be found at https://github.com/etcd-io/etcd/blob/main/api/v3rpc/rpctypes/error.go#L72 (, ) Metrics changes: Fix exposed buckets of `schedulervolumeschedulingdurationseconds_bucket` metric. (, ) Migrated kubernetes object references (= name + namespace) to structured logging when using JSON as log output format (, ) Pass additional flags to subpath mount to avoid flakes in certain conditions. (, ) Pod SecurityContext sysctls name parameter for update requests where the existing object's sysctl contains slashes and kubelet sysctl whitelist support contains slashes. (, ) [SIG Apps, Auth, Node, Storage and Testing] Pod will not start when Init container was OOM killed. (, ) [SIG Node] PodResources interface was changed, now it returns only isolated CPUs (, ) Provide IPv6 support for internal load balancer. (, ) Reduce the number of calls to docker for stats via dockershim. For Windows this reduces the latency when calling docker, for Linux this saves cpu cycles. (, ) [SIG Node and Windows] Removed the error message label from the `kubeletstartedpodserrorstotal` metric (, ) Resolves a potential issue with GC and NS controllers which may delete objects after getting a 404 response from the server during its startup. This PR ensures that requests to aggregated APIs will get 503, not 404 while the APIServiceRegistrationController hasn't finished its job. (, ) Respect grace period when updating static pods. (, ) [SIG Node and Testing] Revert building binaries with PIE mode. (, ) Reverts adding namespace label to admission metrics (and histogram exansion) due to cardinality issues. (, ) Reverts the CRI API version surfaced by dockershim to `v1alpha2`. (, ) Scheduler resource metrics over fractional binary quantities (2.5Gi, 1.1Ki) were incorrectly reported as very small

(, ) Support more than 100 disk mounts on Windows (, ) Support using negative array index in JSON patch replace operations. (, ) The `--leader-elect*` CLI args are now honored in scheduler. (, ) The `--leader-elect*` CLI args are now honored in the scheduler. (, ) The `client-go` dynamic client sets the header `Content-Type: application/json` by default (, ) The `kube-Proxy` now correctly filters out unready endpoints for Services with Topology. (, ) The `pods/binding` subresource now honors `metadata.uid` and `metadata.resourceVersion` (, ) The kube-proxy syncproxyrulesiptablestotal metric now gives the correct number of rules, rather than being off by one. Fixed multiple iptables proxy regressions introduced in 1.22: When using Services with SessionAffinity, client affinity for an endpoint now gets broken when that endpoint becomes non-ready (rather than continuing until the endpoint is fully deleted). Traffic to a service IP now starts getting rejected (as opposed to merely dropped) as soon as there are no longer any usable endpoints, rather than waiting until all of the terminating endpoints have terminated even when those terminating endpoints were not being used. Chains for endpoints that won't be used are no longer output to iptables, saving a bit of memory/time/cpu. (, ) [SIG Network] Topology Aware Hints now ignores unready endpoints when assigning hints. (, ) Topology Hints now excludes control plane notes from capacity calculations. (, ) Update Go used to build migrate script in etcd image to v1.16.7. (, ) Updated json representation for a conflicted taint to `Key=Effect` when a conflicted taint occurs in kubectl taint. (, ) Upgrades functionality of `kubectl kustomize` as described at https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv4.4.1 (, ) [SIG CLI] Watch requests that are delegated to aggregated API servers no longer reserve concurrency units (seats) in the API Priority and Fairness dispatcher for their entire duration. (, ) When a static pod file is deleted and recreated while using a fixed UID, the pod was not properly restarted. (, ) XFS-filesystems are now force-formatted (option `-f`) in order to avoid problems being formatted due to detection of magic super-blocks. This aligns with the behaviour of formatting of ext3/4 filesystems. (, ) `--log-flush-frequency` had no effect in several commands or was missing. Help and warning texts were not always using the right format for a command (`adddirheader` instead of `add-dir-header`). Fixing this included cleaning up flag handling in component-base/logs: that package no longer adds flags to the global flag sets. Commands which want the klog and `--log-flush-frequency` flags must explicitly call `logs.AddFlags`; the new `cli.Run` does that for commands. That helper function also covers flag normalization and printing of usage and errors in a consistent way (print usage text first if parsing failed, then the error). (, ) All `klog` flags except for `-v` and `-vmodule` are deprecated. Support for `-vmodule` is only guaranteed for the text log format. (, ) Better pod events ("waiting for ephemeral volume controller to create the persistentvolumeclaim"" instead of "persistentvolumeclaim not found") when using generic ephemeral volumes. (, ) Changed buckets in apiserverrequestduration_seconds metric from , ) [SIG API Machinery, Instrumentation and Testing] Deprecate `apiserverlongrunninggauge` and `apiserverregisterwatchers` in 1.23.0. (, ) Enhanced error message for nodes not selected by scheduler due to pod's PersistentVolumeClaim(s) bound to PersistentVolume(s) that do not exist. (, ) Fix an issue in cleaning up `CertificateSigningRequest` objects with an unparseable `status.certificate` field. (, ) Kube-apiserver: requests to node, Service, and Pod `/proxy` subresources with no additional URL path now only automatically redirect GET and HEAD

(, ) Kube-apiserver: sets an upper-bound on the lifetime of idle keep-alive connections and the time to read the headers of incoming requests. (, ) Kubeadm: external etcd endpoints passed in the `ClusterConfiguration` that have Unicode characters are no longer IDNA encoded (converted to Punycode). They are now just URL encoded as per Go's implementation of RFC-3986, have duplicate "/" removed from the URL paths, and passed like that directly to the `kube-apiserver` `--etcd-servers` flag. If you have etcd endpoints that have Unicode characters, it is advisable to encode them in advance with tooling that is fully IDNA compliant. If you don't do that, the Go standard library (used in k8s and etcd) would do it for you when making requests to the endpoints. (, ) Kubeadm: remove the `--port` flag from the manifest for the `kube-controller-manager` since the flag has been a NO-OP since 1.22 and insecure serving was removed for the component. (, ) Kubeadm: remove the `--port` flag from the manifest for the kube-scheduler since the flag has been a NO-OP since 1.23 and insecure serving was removed for the component. (, ) Kubeadm: update references to legacy artifacts locations, the `ci-cross` prefix has been removed from the version match as it does not exist in the new `gs://k8s-release-dev` bucket. (, ) Kubectl: deprecated command line flags (like several of the klog flags) now have a `DEPRECATED: <explanation>` comment. (, ) [SIG CLI] Kubemark is now built as a portable, static binary. (, ) [SIG Scalability and Testing] Migrate `cmd/proxy/{config, healthcheck, winkernel}` to structured logging (, ) Migrate `pkg/proxy` to structured logging (, ) Migrate `pkg/scheduler/framework/plugins/interpodaffinity/filtering.go`,`pkg/scheduler/framework/plugins/podtopologyspread/filtering.go`, `pkg/scheduler/framework/plugins/volumezone/volume_zone.go` to structured logging (, ) Migrate `pkg/scheduler` to structured logging. (, ) Migrate cmd/proxy/app and pkg/proxy/meta_proxier to structured logging (, ) Migrated `cmd/kube-scheduler/app/server.go`, `pkg/scheduler/framework/plugins/nodelabel/nodelabel.go`, `pkg/scheduler/framework/plugins/nodevolumelimits/csi.go`, `pkg/scheduler/framework/plugins/nodevolumelimits/noncsi.go` to structured logging (, ) Migrated `pkg/proxy/ipvs` to structured logging (, ) Migrated `pkg/proxy/userspace` to structured logging. (, ) Migrated `pkg/proxy` to structured logging (, ) Migrated `pkg/scheduler/framework/plugins/volumebinding/assume_cache.go` to structured logging. (, ) [SIG Instrumentation, Scheduling and Storage] Migrated `pkg/scheduler/framework/preemption/preemption.go`, `pkg/scheduler/framework/plugins/examples/stateful/stateful.go`, and `pkg/scheduler/framework/plugins/noderesources/resource_allocation.go` to structured logging (, ) [SIG Instrumentation, Node and Scheduling] Migrated pkg/proxy/winuserspace to structured logging (, ) Migrated scheduler file `cache.go` to structured logging (, ) [SIG Instrumentation and Scheduling] Migrated scheduler files `comparer.go`, `dumper.go`, `node_tree.go` to structured logging (, ) [SIG Instrumentation and Scheduling] More detailed logging has been added to the EndpointSlice controller for Topology. (, ) Remove deprecated and not supported old cronjob controller. (, ) [SIG Apps] Remove ignore error flag for drain, and set this feature as default (, ) [SIG CLI] Remove the deprecated flags `--csr-only` and `--csr-dir` from `kubeadm certs renew`. Please use `kubeadm certs generate-csr` instead. (, ) Support allocating whole NUMA nodes in the CPUManager when there is not a 1:1 mapping between socket and NUMA node (, ) Support for Windows Server 2022 was added to the `k8s.gcr.io/pause:3.6` image. (, ) Surface warning when users don't set `propagationPolicy` for jobs while deleting. (, ) The `AllowInsecureBackendProxy` feature gate is removed. It reached GA in Kubernetes 1.21. (, ) The `BoundServiceAccountTokenVolume` feature gate that is GA since v1.22 is unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. (, ) The `StartupProbe` feature gate that is GA since v1.20 is unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. (, ) The `SupportPodPidsLimit` and `SupportNodePidsLimit` feature gates that are GA since v1.20 are unconditionally enabled, and can no longer be specified via the `--feature-gates`

(, ) The `apiserver` exposes 4 new metrics that allow to track the status of the Service CIDRs allocations: current number of available IPs per Service CIDR current number of used IPs per Service CIDR total number of allocation per Service CIDR total number of allocation errors per ServiceCIDR (, ) The flag `--deployment-controller-sync-period` has been deprecated and will be removed in v1.24. (, ) The image `gcr.io/kubernetes-e2e-test-images` will no longer be used in E2E / CI testing, `k8s.gcr.io/e2e-test-images` will be used instead. (, ) The kube-proxy image contains `/go-runner` as a replacement for deprecated klog flags. (, ) The maximum length of the `CSINode` id field has increased to 256 bytes to match the CSI spec. (, ) Troubleshooting: informers log handlers that take more than 100 milliseconds to process an object if the `DeltaFIFO` queue starts to grow beyond 10 elements. (, ) Update `cri-tools` dependency to v1.22.0. (, ) Update `migratecmd/kube-proxy/app` logs to structured logging. (, ) Update build images to Debian 11 (Bullseye) debian-base:bullseye-v1.0.0 debian-iptables:bullseye-v1.0.0 go-runner:v2.3.1-go1.17.1-bullseye.0 kube-cross:v1.23.0-go1.17.1-bullseye.1 setcap:bullseye-v1.0.0 cluster/images/etcd: Build 3.5.0-2 image test/conformance/image: Update runner image to base-debian11 (, ) Update conformance image to use `debian-base:buster-v1.9.0`. (, ) `volume.kubernetes.io/storage-provisioner` annotation will be added to dynamic provisioning required PVC. `volume.beta.kubernetes.io/storage-provisioner` annotation is deprecated. (, ) bazil.org/fuse: 371fbbd github.com/OneOfOne/xxhash: github.com/antlr/antlr4/runtime/Go/antlr: github.com/cespare/xxhash: github.com/cncf/xds/go: github.com/getkin/kin-openapi: github.com/go-logr/zapr: github.com/google/cel-go: github.com/google/cel-spec: github.com/google/martian/v3: github.com/kr/fs: github.com/pkg/sftp: github.com/spaolacci/murmur3: sigs.k8s.io/json: c049b76 cloud.google.com/go/bigquery: v1.4.0 v1.8.0 cloud.google.com/go/storage: v1.6.0 v1.10.0 cloud.google.com/go: v0.54.0 v0.81.0 github.com/GoogleCloudPlatform/k8s-cloud-provider: github.com/Microsoft/go-winio: github.com/Microsoft/hcsshim: github.com/benbjohnson/clock: github.com/bketelsen/crypt: github.com/containerd/cgroups: github.com/containerd/containerd: github.com/containerd/continuity: github.com/containerd/fifo: github.com/containerd/go-runc: github.com/containerd/typeurl: github.com/coredns/corefile-migration: github.com/docker/docker: github.com/envoyproxy/go-control-plane: github.com/evanphx/json-patch: github.com/go-logr/logr: github.com/golang/glog: github.com/golang/mock: github.com/google/cadvisor: github.com/google/pprof: github.com/hashicorp/golang-lru: github.com/ianlancetaylor/demangle: github.com/json-iterator/go: github.com/magiconair/properties: github.com/mitchellh/go-homedir: github.com/mitchellh/mapstructure: github.com/modern-go/reflect2: github.com/opencontainers/runc: github.com/pelletier/go-toml: github.com/prometheus/common: github.com/spf13/afero: github.com/spf13/cast: github.com/spf13/cobra: github.com/spf13/jwalterweatherman: github.com/spf13/viper: github.com/yuin/goldmark: go.opencensus.io: v0.22.3 v0.23.0 go.uber.org/zap: v1.17.0 v1.19.0 golang.org/x/crypto: 5ea612d 32db794 golang.org/x/net: 37e1c6a e898025 golang.org/x/oauth2: bf48bf1 2bc19b1 golang.org/x/sys: 59db8d7 f4d4317 golang.org/x/term: 6a3ed07 6886f2d golang.org/x/text: v0.3.6 v0.3.7 golang.org/x/tools: v0.1.2 d4cc65f google.golang.org/api: v0.20.0 v0.46.0 google.golang.org/appengine: v1.6.5 v1.6.7 google.golang.org/genproto: f16073e fe13028 google.golang.org/grpc: v1.38.0 v1.40.0 google.golang.org/protobuf: v1.26.0 v1.27.1 gopkg.in/ini.v1: v1.51.0 v1.62.0 honnef.co/go/tools: v0.0.1-2020.1.3 v0.0.1-2020.1.4 k8s.io/gengo: b6c5ce2 485abfe k8s.io/klog/v2: v2.9.0 v2.30.0 k8s.io/kube-openapi: 9528897 e816edb k8s.io/system-validators: v1.5.0 v1.6.0 k8s.io/utils: 4b05e18 cb0fa31 sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.22 v0.0.25 sigs.k8s.io/kustomize/api: v0.8.11 v0.10.1 sigs.k8s.io/kustomize/cmd/config: v0.9.13 v0.10.2 sigs.k8s.io/kustomize/kustomize/v4: v4.2.0 v4.4.1 sigs.k8s.io/kustomize/kyaml: v0.11.0 v0.13.0 cloud.google.com/go/datastore: v1.1.0 cloud.google.com/go/pubsub: v1.2.0 github.com/alecthomas/units: github.com/coreos/bbolt: github.com/coreos/etcd: github.com/coreos/go-systemd: github.com/coreos/pkg: github.com/dgrijalva/jwt-go: github.com/google/martian: github.com/jpillora/backoff: gotest.tools: v2.2.0+incompatible filename | sha512 hash -- | -- | 4cf838ebcd3bb756cc604b194dd3b58716b41a34c35f636a7c23af4a501829c7ce23def2fae43547f86bff326bb9268a16b34058e83cada4626930b60a928d97 | 88af717a64f237de86f287c3715540fcaf6fd9e526a715832b395965ae27187319cc8955434f2511bcace46be455f56d22d569b083bf65f63bd0539fc6ce76a0 filename | sha512 hash -- | -- | 2aa8dbe6fe7926dd78083a243323dd0090f19acb22cc6ab6e7fb74ae3efd494f9927469b466a78eda1a210657935237ae235fbcf99dcc765908e590d8b5979ee | 8aaaa11ecdefc349a64b12775656c9f1ed9bfdb72e3dac0f651c629f4f113dc701eb01d07e529c9633bbd5d0eca455a2bf0f654d52830b77917c28ec7bb84dd5 | 03f482611ec7d12b30c03e382e53b1b00d7263882204d146a0fb74185d19eb90487ee87d898ec8b6c2ed388ac7b31455d79bafc3cf490efacea06825c1ff49c4 | bac04a5d242ff32fbed1a7e756107afd68652ed489c405aee026132f8da57107ea9292ad776ec9f68c5b8a4bc5391a56d6a0ea538461d78778a4f09633fe3bba | cc3ac01dd58f01e8d85b4c0ef70914331addc5b4abde05175d5194d3754c11b7f113a2e2ff79d15a17562565cd7572e643054b67457261563e9f316c3e20a473 | 8fd0109f6ff07656dc0a265248c3fba81cf2faeb3a872dfb4423f14a92bff793e9ca8c330d83a1e957cdf9358287e325dd4f2ea7e52a0bf66ff1f2631b900a2c | 0610276ae835a9b151ad5ac9b192ef7d15b385279c9676ac4f2dfcb0600cb66dd4dcd39e07d65d637f8ae07f0f808f9eddf3312c45c6c435c2db0e0716cf3c29 | bbba68df6895eb722ea7f8678e25e7be6e2d34a50e30c806445515e02b00e3d6982af8e2d54ee757c4fb661fea2f9fa61bd46ffe576897f0583598e3fb0be080 | 1c41e825e83b0363967af94e759d6f8adb6f77f7f176c804922b089262df933d21528734dd8ebffd3d49fa7196a558a61d178d64f45444708688fe571dd2b4be | 2b39c66f4d69cdaeff9fa8f46331b2eece1714998df7a9300b6be85e9fec7ccc8cccdc02b3900ab7aa989816242b410234a1b9562b2dd51cff89ae19fd9be491 | 77d00160870636703c9604613b6829c3727241616723de638653f5044373ed233608d8680357d16a37ee8fe88f0c158d93da644ddcddd27047f1d76da1a98a68 filename | sha512 hash -- | -- | d54cbd2ed123ab30ad8662523a9f28af530bba0fe15b22738dbc2a5c3f6917c7ef4e0fcd03eaea36cee38420e968b91607e8d5022598e55030a21777448c57c0 | 835d94a3833e7b3b257f2174840e974f957d17e026dc2e1781f2d4590738afc947a27982cd06579a711c31dad5933beaa01c3629950dcb3e36f21b4f8e57aad8 | 1fed95a3649d87390cb266918f5ed1b8fb00b4ef027cf004f6bffb3d05d1f6f84497c4ac11f34f3d8334d83cab929f46b62122e107ce6c0c6d628c01bb957fc7 | cb563f6d5625eaa5db4f92b239508c0f30e2c7d9e6200651f23cf59f1962301c2113830e2718c515bcc05cce1a5c1b34d77281c3dc2253e4429e8425a852c238 | a382396ddf43b6a4f5e95a95f58c2618cb63e5d5bbe3d8f8c74bdc1f23855eec14e4aa32e6ad6b2d2bfc0ecd2f9da602b2f410f17b0e680dc8f2e69d7f0f0127 filename | sha512 hash -- | -- | fca9757fe2c67a01841810890fd62fb546c6a7ccb3f475d68a1960a054c0d4db44f71b2669e96d4c7bdb9ac727156448c9eeea64dbdb7442a04e91e5f88fdd76 | 30a29ba0a71d5cf29a359da78c0c42abc1c478d8cb4736b1e4dfb905770f537e170eba6b1079173867e6973e34b6d51f9d9f86c5573e1e6e7539de0bdd617c7b | 066a437d780b0c871e5635246f9434ba9e7652393546f8b3edba83a56e5a431b4e9aa443a82670d8c5f906bb630426f8a3d8c8b0d146426a20816f588182ea38 | 8816dc1907a743cf8345acc41f5831d195b8531e313024060a5f322e5ba53061c2878f483e1d7253ed06bdb7edbbcf8784d2f5ef1d6c134e486ee8db9d1852b2 | 671e9513d0aeace5352bf22a1522aa64e4f0b041110f6d11663d4fa3c97dc1c60a6197341dd000c8fd35e0cd51b279e748bea04b220dbecd9f589058632e59cc | cae457dde9c3bd813b97b2f57912f72cb08668b31700ceb356234ad78183fac45312e9a92e83e094ba7d991152bfe4375d098dfa6db6e3f2675630dd61441039 Kube-apiserver: Server Side Apply merge order is reverted to match v1.22 behavior until http://issue.k8s.io/104641 is resolved. (, ) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Storage and Testing] Reverts the CRI API version surfaced by dockershim to v1alpha2 (, ) [SIG Network and Node] Nothing has changed. sigs.k8s.io/structured-merge-diff/v4: v4.2.0 v4.1.2 Nothing has changed. filename | sha512 hash -- | -- | ede62f7d1bde6a11e60b8ff119366c42902090c5b005ca73590856645c16ff12c904cdf45528cb5f48d4ece31db62f8a2c6a2fc4f10d29052c660036f5a47b5b | 6103bde6ceeb7b6c40e6e7391731acc4228cf799ee8b7cf612baa8327212a183f16fd560f25b1d608e7f629c230310c585e2e1551436f9569a9d7d5a8c3dbb38 filename | sha512 hash -- | -- | 0266edfb98cf69c62466c87caa1028510cdb0600dfee9f25ba13b6936935011f6b90e8fe6b008a4c8d060012066adff45b28f01cf1c7f9e24293800576c5dc74 | 8bebf2537a53670a8487ccb43faec62b00439124c99d67ae88c5f1360bb863f03648c430be836580b56dc4526609fd53ef35cabf7a4622c346c87731d5e94575 | 7cb542707711b5c4cf1402f07d2102b8633b4b75c43be4424f3e15da047badf484b458f6c3e27ecc82343f58f99437c4ab92a0a17f4976f806636d7b070c3396 | eec960213ccb94b1cf1dc47aaf508eb12c1d04c13474b66960db4e0379d23a59aa484bba06cc04e6e7aba22a6e0eb8afd11a35799829dd540767eaedd66b37d1 | 7db536134da64c586058603283d752bc0bd7c2ea63b312513fff95d65bc24f978b24c0145446efea6f0b6b8f87f3c74c9a4dd581fa5a186ac630e6d58d30ee9d | 93a33630fe6bd89fb06f739f7a4184c151c4ac5a8230798b5c3a9137f553f59495e8cc2231f155d6ba51517f70ee095752872c067a12e7bba69ba2b4d9724ee9 | afc9ffb6632b4c85837f87d6764e54a8111d4df3a23320294ac0b942dd089789d018faf91a4d5a22ddc4496d9204470267fba09b2a1d6d6f8ad74353df060675 | d5c28f9e65d6a910cf6478342c3e1bd968b16820b2aae6d7ab51d1a646a3b4e46dd7bc2f255f5a02e516d36c90320f4c7b0f836192aae37f45da283fbd76dc88 | cccb34fd97fb3f05aaf900569bd07772e4ba95f723f7ea71f191926fe05f01b4e608ab49d876b65405051c504043c74b725d09c92321e17a35587703269a37a8 | 61298763df834a36a1f10c0a45cc7c0b520ad38c2f013fd39e0ab11cb07b7f416b9aa695c4289cf21baf9281c3a9ec4e99e93b4b82b368658b91fa19f6486c69 | 99d52ce6c6f620464239e4610711daa8d79e0551bab587d9d2342fe315a07fb94b19ffbfdc8f4a769ae51c848f2896517ac3abd256504492bc24389878749775 filename | sha512 hash -- | -- | fff4bdfce528a16abce7d075570bad7e5fd3b64baa5bd154595b44d25945379e4ea6fb56869681cd2222ad55965214d7e96ea1d32ddd519629a1b943dfbceb2b | dc777f74f6d6eef8d56d379cad36e535566993df3abf0be5e00cf22790f01f4336cd757b395aa724b5743db2ad74ce966746755d4970e04a599861f59ea8f12c | a4d935e6816e6bd14e037819949644626813885ef308c7e5ab0a680f71b155cd164c46b963dcdbdcd91cadbf1d0870c66934bb62c9f95698699d1ae3dcc25cb5 | 9a69761c04556e246e046e18bd4b875e813aef1a1e01931ad0aef61ad11d741d4a1f416949dc9f5a8b7103823f6061557d075a35b4766a8e1192872a3fedb637 | 6f64559358d05d659faf79476524101420b6b4e83e27bba11c407624723926adfd14045792a8a874100527d21cf334b56293a94ac1c3e8916b7ec3e9926b52e4 filename | sha512 hash -- | -- | ed3972e5bb9550d0999b4f4b9da315607bfdb349fae9395e23bb36e72c6cc30aa6df42084b0faa0f5f890b983ed7132357d49d755cde081fe4c53bf78a935f83 | c87b42358cc75f362eb5a1b52780ee62fa7c18a8760cf7ee744e1910558ffbe5869fea75e49bf8822795c5255ddd935a9364aa067bffeca500b8fb61d70402c5 | 34cf8aea703f279559765fdf5c0a079e4679e407134c666082a4ee56d1352c1b66291c847a983ee547a64209143a83a15d89f73d4f2fbb3473dfefb8f9aa630e | 86e8bb17b715aea6df3c4ddfd68e4423414ac9a9b86cd38e6272a2226849a456d86496d6154b468b97c22164dc22f80f1dd456a2011ca22f1b782791cf4d3c84 | 61723328d454ceade01b1ad5e71bf96caae9badd58b02e877833eafae8465cbc9d09cfd1f414749c4040419bbc594a9256f02a44ce7e762a9753ae50511e57de | 3c22248012c2e301209832ded5ecd20d87eb4350a09592763c7de6b46c821b158abbad816c76e012e692ff21ba5da9fb0e861b914166f339f9a0d676c3d4b60a Add gRPC probe to Pod.Spec.Container.{Liveness,Readiness,Startup}Probe (#106463, @SergeyKanzhelev) [SIG API Machinery, Apps, CLI, Node and Testing] Adds a feature gate StatefulSetAutoDeletePVC, which allows PVCs automatically created for StatefulSet pods to be automatically deleted. (#99728, @mattcary) [SIG API Machinery, Apps, Auth and Testing] Performs strict server side schema validation requests via the `fieldValidation=[Strict,Warn,Ignore]` query

(#105916, @kevindelgado) [SIG API Machinery, Apps, Auth, Cloud Provider and Testing] Support pod priority based node graceful shutdown (#102915, @wzshiming) [SIG Node and Testing] CRI v1 is now the project default. If a container runtime does not support the v1 API, Kubernetes will fall back to the v1alpha2 implementation. (#106501, @ehashman) [SIG Network, Node and Testing] Kube-Proxy now correctly filters out unready endpoints for Services with Topology Aware Hints enabled. (#106507, @robscott) [SIG Network] Kubelet: the printing of flags at the start of kubelet now uses the final logging configuration (#106520, @pohly) [SIG Node] Topology Aware Hints now ignores unready endpoints when assigning hints. (#106510, @robscott) [SIG Apps and Network] Nothing has changed. Nothing has changed. Nothing has changed. filename | sha512 hash -- | -- | 048cc297840fd70dc571863bbed9da8176a479ca6b8ff17c9a2cc1b1dbf286377d85eb7fccc5d85e1d652658c393ea1eab7ab518631510e1e7462ea638a56b2b | 1d3f6f5bb54b61312934169845417dffc428bed0f51342dc2b0eebf7f16899843b0f66f9fb2dcdb2a6e9f25bbdc930ea9adac552b0b011e656151c8cae2f4f71 filename | sha512 hash -- | -- | e22ce7199acf369eacf8422c8ee417041289e927bfc03c238f45faec75c2dabd7f8201c77ed39f20ac311d1ba289766825b7b2f738cfc59b5652a20b98117180 | 22fa13ca86eb5837db3844b6b7fd134c3ffa3ba5a008635bfa83613a100fa48b3e2331cdf5d368cb267c3cd27e3947fe08ac2540342f1b221192e972695a2cd6 | 8e239ce934d121b21b534a6d521ca02bf1c6709831e181d103c8d86cdab01b296546be25902162b1060876744f3b579de018b7c2d198e5d5efdd9c849b3ba7ef | e9355264e3ca91da833fe3c8c1dcc55c287a9b813aad91f26b09e6a75f48be57d12cb235c5f9c6fe2a0aceee09e2b5da84568d81d8002066c8e77d848a03f112 | 80e93b6c8cce8221f9a5aba8018fcd95b7ec57728a202fdd158b8df86a733e32d6bb60d8b7ea78da9556058074e9bb88c072b4207a43a4fd2f256cce2593a8df | 769a1aa41988bbf11a11ef40f42c76740fcbe7fe1fd5d6da948729e1a62bf9c4f28101f47fa9ccd12de50a378b3654e1e4c2d50afad59182c03b8d1e972341e7 | 4a9346caef2714f03e65dc3e5e46ade1b311b91ef184b8a47466583e834f44dcdb21c3800793e87c20064b25c3eac2c34637ff6817f1752d52425cdfd5a912fb | f2129ea05e581a38bdc2771cfdd92ad990620fabf9655f7343c56541a544aa4c6c1e1a2e91a338d06dd0064f35fb5e3027259c317a0909badcbadc9e418c6ced | 2dc9459b02f4ed564a7d0e2062e3590c5240debc6a64449d1c714382ded197d5fcf99feecb80ba6483d265ab34126958737cd692783e675b39159be94729c018 | e58cb2f87f619d34afbb2c2c0f2bab484970406216698b79129637cb27c5508b2ca4bd2a3a91847868631bd72947887317692a73fec0f8d67c26aa59868c9d8f | 515bd2e3c95afe613db998ed42ea5456771c488e0963c9fe0328816a6baba09ea4e915d22538e05d478556d17f1678d6a96b75cae25ba742be73da23d04f72ff filename | sha512 hash -- | -- | adc6c0e5c07c3e1d24ac4399ea725da5d72a043feaea0063f26188e469b4b8cf537df245015631f1efce9d5e457724858327da3c7c9763f6ca4538aaf77a5e67 | e6e673cb9baecc56ae03d716569769391cd6f8d38d85810f0199e71b20a4d4c3c92efe7b31a67af463fb01029d94cbcb0c6fe7a0918123055f3fa8f373e76c49 | f91dc6e948b702784909ca0c4b8758ad9dbfbcd202ec4e329666b07d42488df00ad64de6a68405668ed881e62e0515271c8168e8316519cd95802239abde4951 | fbbf3daff8caa89f8249122ba19d67a0d9298fb47d327c0bebd7a54adad4fe6e809164d8bf8e563c79b1f9c8b646f29d18789ec938cbc5746e30649b392c7121 | a4ccda542f1b86667e6bf29afd091a2ce6f3a30165ff8b918585fc7794be26d00bd846acaa5b805b270a60df69fbe9827bab6ee472129996e28052bbbe1b0593 filename | sha512 hash -- | -- | 4d7dd2e50fe65fd1140c51deeb90d8d9f89bbba59502becf626757e2e9eb59fb781bbf3ecb899f1b8e391746329c5c017177287004195387151799e73887f05b | d38cd4a06b983a7253d99a6d927c40cbacc636bd73d33172ee03cda502f806638d3cc6f096bc13a55a2faf11ab3e85d77dfd20559e2c880cf54f45ba0875c75c | fa1fa35f30ca589e031485affd2a1016ba5ca0efdf64b35d49c7738342acb55c40733e53fb3b477734bab68d97b00f9adcfb5954ab365169d8f00ac804cc60fb | 412b3a133a7711e32455e49d1aac4ce9ee0e44df89afca40dfa8ac52a8aa98649bd4dd7eff85addd8a525bb16b65966dbde1df0c62a994213b4cfa1a7a3b8128 | 7e0e217893665a56406b6f1404d616da8578396890b04474fed12ea6b48f5fbf52432efd43c13f66a643284fd54c0fd3441940c777eb1cd0796443fd72d69b6f | 768dfe871a028ff7d972d9b59935c1ebdcc8ea0ccf990ee84060ef3bb995ddecb48a49d9fb2ff12dc44ed404d6d9362ee78af3492a4206bb23eb8a0ac8d63ca2 Log messages in JSON format are written to stderr by default now (same as text format) instead of stdout. Users who expected JSON output on stdout must now capture stderr instead or in addition to stdout. (#106146, @pohly) [SIG API Machinery, Architecture, Cluster Lifecycle and Instrumentation] is included in release tar balls. It can be used to replace the deprecated `--log-file` parameter. (#106123, @pohly) [SIG API Machinery, Architecture, Cloud Provider, Cluster Lifecycle and Instrumentation] Kubeadm: add a new output/v1alpha2 API that is identical to the output/v1alpha1, but attempts to resolve some internal dependencies with the kubeadm/v1beta2 API. The output/v1alpha1 API is now deprecated and will be removed in a future release. (#105295, @neolit123) [SIG Cluster Lifecycle] Kubeadm: add the kubeadm specific, Alpha (disabled by default) feature gate UnversionedKubeletConfigMap. When this feature is enabled kubeadm will start using a new naming format for the ConfigMap where it stores the KubeletConfiguration structure. The old format included the Kubernetes version - "kube-system/kubelet-config-1.22", while the new format does not - "kube-system/kubelet-config". A similar formatting change is done for the related RBAC rules. The old format is now DEPRECATED and will be removed after the feature graduates to GA. When writing the ConfigMap kubeadm (init, upgrade apply) will respect the value of UnversionedKubeletConfigMap, while when reading it (join, reset, upgrade), it would attempt to use new format first and fallback to the legacy format if needed. (#105741, @neolit123) [SIG Cluster Lifecycle and Testing] A new field `omitManagedFields` has been added to both `audit.Policy` and `audit.PolicyRule` so cluster operators can opt in to omit managed fields of the request and response bodies from being written to the API audit log. (#94986, @tkashem) [SIG API Machinery, Auth, Cloud Provider and Testing] Create HPA v2 from v2beta2 with some fields changed. (#102534, @wangyysde) [SIG API Machinery, Apps, Auth, Autoscaling and Testing] Fix kube-proxy regression on UDP services because the logic to detect stale connections was not considering if the endpoint was ready. (#106163, @aojea) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Contributor Experience, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows] Implement support for recovering from volume expansion failures (#106154, @gnufied) [SIG API Machinery, Apps and Storage] In kubelet, log verbosity and flush frequency can also be configured via the configuration file and not just via command line

In other commands (kube-apiserver, kube-controller-manager), the flags are listed in the "Logs flags" group and not under "Global" or "Misc". The type for `-vmodule` was made a bit more descriptive (`pattern=N,...` instead of `moduleSpec`). (#106090, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling] IngressClass.Spec.Parameters.Namespace field is now GA. (#104636, @hbagdi) [SIG Network and Testing] KubeSchedulerConfiguration provides a new field `MultiPoint` which will register a plugin for all valid extension points (#105611, @damemi) [SIG Scheduling and Testing] Kubelet should reject pods whose OS doesn't match the node's OS label. (#105292, @ravisantoshgudimetla) [SIG Apps and Node] The CSIVolumeFSGroupPolicy feature has moved from beta to GA. (#105940, @dobsonj) [SIG Storage] The Kubelet's `--register-with-taints` option is now available via the Kubelet config file field registerWithTaints (#105437, @cmssczy) [SIG Node and Scalability] Validation rules for Custom Resource Definitions can be written in the using the `x-kubernetes-validations` extension in OpenAPIv3 schemas (alpha). This is gated by the alpha "CustomResourceValidationExpressions" feature gate. (#106051, @jpbetz) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing] (beta feature) If the CSI driver supports the NodeServiceCapability `VOLUMEMOUNTGROUP` and the `DelegateFSGroupToCSIDriver` feature gate is enabled, kubelet will delegate applying FSGroup to the driver by passing it to NodeStageVolume and NodePublishVolume, regardless of what other FSGroup policies are set. (#106330, @verult) [SIG Storage] /openapi/v3 endpoint will be populated with OpenAPI v3 if the feature flag is enabled (#105945, @Jefftree) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing] Add support for PodAndContainerStatsFromCRI featuregate, which allows a user to specify their pod stats must also come from the CRI, not cAdvisor. (#103095, @haircommander) [SIG Node] Add support for Portworx plugin to csi-translation-lib. Alpha release Portworx CSI driver is required to enable migration. This PR adds support of the `CSIMigrationPortworx` feature gate, which can be enabled by: Adding the feature flag to the kube-controller-manager `--feature-gates=CSIMigrationPortworx=true` Adding the feature flag to the kubelet config: featureGates: CSIMigrationPortworx: true (#103447, @trierra) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows] Added ability for kubectl wait to wait on arbitary JSON path (#105776, @lauchokyip) [SIG CLI] Added the ability to specify whether to use an RFC7396 JSON Merge Patch, an RFC6902 JSON Patch, or a Strategic Merge Patch to perform an override of the resources created by kubectl run and kubectl expose. (#105140, @brianpursley) [SIG CLI] Adding option for kubectl cp to resume on network errors until completion, requires tar in addition to tail inside the container image (#104792, @matthyx) [SIG CLI] Adds --as-uid flag to kubectl to allow uid impersonation in the same way as user and group impersonation. (#105794, @margocrawf) [SIG API Machinery, Auth, CLI and Testing] Allows users to prevent garbage collection on pinned images (#103299, @wgahnagl) [SIG Node] CSIMigrationGCE feature flag is turned ON by default (#104722, @leiyiz) [SIG Apps, Cloud Provider, Node, Storage and Testing] Changed feature CSIMigrationAWS to on by default. This feature requires the AWS EBS CSI driver to be installed. (#106098, @wongma7) [SIG Storage] Ensures that volume is deleted from the storage backend when the user tries to delete the PV object manually and the PV ReclaimPolicy is Delete. (#105773, @deepakkinni) [SIG Apps and Storage] Graduating `controlleradmissiondurationseconds`, `stepadmissiondurationseconds`, `webhookadmissiondurationseconds`, `apiservercurrentinflightrequests` and `apiserverresponsesizes` metrics to stable. (#106122, @rezakrimi) [SIG API Machinery, Instrumentation and Testing] Graduating `pendingpods`, `preemptionattemptstotal`, `preemptionvictims` and `scheduleattemptstotal` metrics to stable. Also `e2eschedulingdurationseconds` is renamed to `schedulingattemptdurationseconds` and the latter is graduated to stable. (#105941, @rezakrimi) [SIG Instrumentation, Scheduling and Testing] Integration testing now takes periodic Prometheus scrapes from the etcd server. There is a new script

that runs a containerized Prometheus server against an archive of such scrapes. (#106190, @MikeSpreitzer) [SIG API Machinery and Testing] Kube-apiserver: when merging lists, Server Side Apply now prefers the order of the submitted request instead of the existing persisted object (#105983, @jiahuif) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Storage and Testing] Kubectl describe namespace now shows Conditions (#106219, @dlipovetsky) [SIG CLI] Kubelet should reconcile `kubernetes.io/os` and `kubernetes.io/arch` labels on the node object. The side-effect of this is kubelet would deny admission to pod which has nodeSelector with label `kubernetes.io/os` or `kubernetes.io/arch` which doesn't match the underlying OS or arch on the host OS. The label reconciliation happens as part of periodic status update which can be configured via flag `--node-status-update-frequency` (#104613, @ravisantoshgudimetla) [SIG Node, Testing and Windows] Kubernetes is now built with Golang 1.17.3 (#106209, @cpanato) [SIG API Machinery, Cloud Provider, Instrumentation, Release and Testing] Move ConfigurableFSGroupPolicy to GA Rename metric volumefsgrouprecursiveapply to volumeapplyaccesscontrol (#105885, @gnufied) [SIG Instrumentation and Storage] Moving WindowsHostProcessContainers feature to beta (#106058, @marosset) [SIG Windows] The DownwardAPIHugePages feature is now enabled by default. (#106271, @mysunshine92) [SIG Node] The PodSecurity admission plugin has graduated to beta and is enabled by default. The admission configuration version has been promoted to pod-security.admission.config.k8s.io/v1beta1. See https://kubernetes.io/docs/concepts/security/pod-security-admission/ for usage guidelines. (#106089, @liggitt) [SIG Auth and Testing] This PR adds the following metrics for API Priority and Fairness. apiserver_flowcontrol_priority_level_seat_count_samples: histograms of seats occupied by executing requests (both regular and final-delay phases included), broken down by priority_level; the observations are taken once per millisecond. apiserver_flowcontrol_priority_level_seat_count_watermarks: histograms of high and low watermarks of number of seats occupied by executing requests (both regular and final-delay phases included), broken down by priority_level. apiserver_flowcontrol_watch_count_samples: histograms of number of watches relevant to a given mutating request, broken down by that request's prioritylevel and flowschema. (#105873, @MikeSpreitzer) [SIG API Machinery, Instrumentation and Testing] Topology Aware Hints have graduated to beta. (#106433, @robscott) [SIG Network] Update the system-validators library to v1.6.0 (#106323, @neolit123) [SIG Cluster Lifecycle and Node] Upgrade etcd to 3.5.1 (#105706, @uthark) [SIG Cloud Provider, Cluster Lifecycle and Testing] When using `RequestedToCapacityRatio` ScoringStrategy, empty shape will cause error. (#106169, @kerthcet) [SIG Scheduling] This release enables in-tree RBD migration to CSI driver with a couple of feature gates. These featuregates are alpha in this release. `CSIMigrationRBD`: when enabled, it will redirect traffic from in tree rbd plugin ( kubernetes.io/rbd ) to CSI driver ( rbd.csi.ceph.com) , default to `false` now. `IntreePluginRBDUnregister`: Disables the RBD in-tree driver The feature gates can be enabled by: Adding the feature flag to the kube-controller-manager `--feature-gates=CSIMigrationRBD=true` Adding the feature flag to the kubelet config: featureGates: `CSIMigrationRBD`: true As a Kubernetes cluster operator that administers storage, here are the prerequisites that you must complete before you attempt migration to the RBD CSI driver: You must install the Ceph CSI driver (rbd.csi.ceph.com), v3.5.0 or above, into your Kubernetes cluster. Considering the clusterID field is a required parameter for CSI driver for its operations, but in-tree StorageClass has monitors field as a required parameter, a Kubernetes storage admin has to create a clusterID based on the monitors hash ( ex:#echo -n '<monitors_string>' | md5sum) in the CSI config map and keep the monitors under this clusterID configuration. Also, if the value of adminId in the in-tree Storageclass is different from admin, the adminSecretName mentioned in the in-tree Storageclass has to be patched with the base64 value of the adminId parameter value, otherwise this step can be skipped.(#95361, @humblec) [SIG API Machinery, Node, Scheduling, Storage] Graduating `podschedulingdurationseconds`, `podschedulingattempts`, `frameworkextensionpointdurationseconds`, `pluginexecutiondurationseconds` and `queueincomingpods_total` metrics to

(#106266, @ahg-g) [SIG Instrumentation, Scheduling and Testing] Users should not rely on unsupported CRON_TZ variable when specifying schedule, both the API server and cronjob controller will emit warnings pointing to https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/ containing explanation (#106455, @soltysh) [SIG Apps] (PodSecurity admission) errors validating workload resources (deployment, replicaset, etc.) no longer block admission. (#106017, @tallclair) [SIG Auth] Add support for Windows Network stats in Containerd (#105744, @jsturtevant) [SIG Node, Testing and Windows] Added show-capacity option to `kubectl top node` to show `Capacity` resource usage (#102917, @bysnupy) [SIG CLI] Do not unmount and mount subpath bind mounts during container creation unless bind mount changes (#105512, @gnufied) [SIG Storage] Don't use a custom dialer for the kubelet if is not rotating certificates, so we can reuse TCP connections and have only one between the apiserver and the kubelet. If users experiment problems with stale connections using HTTP1.1, they can force the previous behavior of the kubelet by setting the environment variable DISABLE_HTTP2. (#104844, @aojea) [SIG API Machinery, Auth and Node] EndpointSlice Mirroring controller now cleans up managed EndpointSlices when a Service selector is added (#105997, @robscott) [SIG Apps, Network and Testing] Enhanced event messages for pod failed for exec probe timeout (#106201, @yxxhero) [SIG Node] Ensure Pods are removed from the scheduler cache when the scheduler misses deletion events due to transient errors (#106102, @alculquicondor) [SIG Scheduling] Fix a panic in kubectl when creating secrets with an improper output type (#106317, @lauchokyip) [SIG CLI] Fixed a bug which could cause webhooks to have an incorrect copy of the old object after an Apply or Update (#106195, @alexzielenski) [SIG API Machinery] Fixed applying of SELinux labels to CSI volumes on very busy systems (with "error checking for SELinux support: could not get consistent content of /proc/self/mountinfo after 3 attempts") (#105934, @jsafrane) [SIG Storage] Fixed bug where using kubectl patch with $deleteFromPrimitiveList on a nonexistent or empty list would add the item to the list (#105421, @brianpursley) [SIG API Machinery] Fixed the issue where logging output of kube-scheduler configuration files included line breaks and escape characters. The output also attempted to output the configuration file in one section without showing the user a more readable format. (#106228, @sanchayanghosh) [SIG Scheduling] Kube-up now includes CoreDNS version v1.8.6 (#106091, @rajansandeep) [SIG Cloud Provider] Kubeadm: fix a bug on Windows worker nodes, where the downloaded KubeletConfiguration from the cluster can contain Linux paths that do not work on Windows and can trip the kubelet binary. (#105992, @hwdef) [SIG Cluster Lifecycle and Windows] Kubectl port-forward service will now properly exit when the attached pod dies (#103526, @brianpursley) [SIG API Machinery] Kubelet: fixes a file descriptor leak in log rotation (#106382, @rphillips) [SIG Node] Pod SecurityContext sysctls name parameter for update requests where the existing object's sysctl contains slashes and kubelet sysctl whitelist support contains slashes. (#102393, @mengjiao-liu) [SIG Apps, Auth, Node, Storage and Testing] Pod will not start when Init container was OOM killed. (#104650, @yxxhero) [SIG Node] Reduce the number of calls to docker for stats via dockershim. For Windows this reduces the latency when calling docker, for Linux this saves cpu cycles. (#104287, @jsturtevant) [SIG Node and Windows] Respect grace period when updating static pods. (#104743, @gjkim42) [SIG Node and Testing] The kube-proxy syncproxyrulesiptablestotal metric now gives the correct number of rules, rather than being off by one. Fixed multiple iptables proxy regressions introduced in 1.22: When using Services with SessionAffinity, client affinity for an endpoint now gets broken when that endpoint becomes non-ready (rather than continuing until the endpoint is fully

Traffic to a service IP now starts getting rejected (as opposed to merely dropped) as soon as there are no longer any usable endpoints, rather than waiting until all of the terminating endpoints have terminated even when those terminating endpoints were not being used. Chains for endpoints that won't be used are no longer output to iptables, saving a bit of memory/time/cpu. (#106030, @danwinship) [SIG Network] Upgrades functionality of `kubectl kustomize` as described at https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv4.4.1 (#106389, @natasha41575) [SIG CLI] Changed buckets in apiserverrequestduration_seconds metric from [0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0,1.25, 1.5, 1.75, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5, 6, 7, 8, 9, 10, 15, 20, 25, 30, 40, 50, 60] to [0.05, 0.1, 0.2, 0.4, 0.6, 0.8, 1.0, 1.25, 1.5, 2, 3, 4, 5, 6, 8, 10, 15, 20, 30, 45, 60] (#106306, @pawbana) [SIG API Machinery, Instrumentation and Testing] Kubectl: deprecated command line flags (like several of the klog flags) now have a `DEPRECATED: <explanation>` comment. (#106172, @pohly) [SIG CLI] Kubemark is now built as a portable, static binary. (#106150, @pohly) [SIG Scalability and Testing] Migrated `pkg/scheduler/framework/plugins/volumebinding/assume_cache.go` to structured logging. (#105904, @mengjiao-liu) [SIG Instrumentation, Scheduling and Storage] Migrated `pkg/scheduler/framework/preemption/preemption.go`, `pkg/scheduler/framework/plugins/examples/stateful/stateful.go`, and `pkg/scheduler/framework/plugins/noderesources/resource_allocation.go` to structured logging (#105967, @shivanshu1333) [SIG Instrumentation, Node and Scheduling] Migrated scheduler file `cache.go` to structured logging (#105969, @shivanshu1333) [SIG Instrumentation and Scheduling] Migrated scheduler files `comparer.go`, `dumper.go`, `node_tree.go` to structured logging (#105968, @shivanshu1333) [SIG Instrumentation and Scheduling] Remove deprecated and not supported old cronjob controller. (#106126, @soltysh) [SIG Apps] Remove ignore error flag for drain, and set this feature as default (#105571, @yuzhiquan) [SIG CLI] The kube-proxy image contains /go-runner as a replacement for deprecated klog flags. (#106301, @pohly) [SIG Testing] github.com/OneOfOne/xxhash: github.com/antlr/antlr4/runtime/Go/antlr: github.com/cespare/xxhash: github.com/cncf/xds/go: github.com/getkin/kin-openapi: github.com/google/cel-go: github.com/google/cel-spec: github.com/spaolacci/murmur3: github.com/containerd/containerd: github.com/coredns/corefile-migration: github.com/docker/docker: github.com/envoyproxy/go-control-plane: github.com/golang/glog: github.com/google/cadvisor: golang.org/x/net: 60bc85c e898025 golang.org/x/sys: 41cdb87 f4d4317 golang.org/x/text: v0.3.6 v0.3.7 google.golang.org/genproto: f16073e fe13028 google.golang.org/grpc: v1.38.0 v1.40.0 google.golang.org/protobuf: v1.26.0 v1.27.1 k8s.io/kube-openapi: 7fbd8d5 e816edb k8s.io/system-validators: v1.5.0 v1.6.0 sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.23 v0.0.25 sigs.k8s.io/kustomize/api: v0.8.11 v0.10.1 sigs.k8s.io/kustomize/cmd/config: v0.9.13 v0.10.2 sigs.k8s.io/kustomize/kustomize/v4: v4.2.0 v4.4.1 sigs.k8s.io/kustomize/kyaml: v0.11.0 v0.13.0 sigs.k8s.io/structured-merge-diff/v4: v4.1.2 v4.2.0 Nothing has changed. filename | sha512 hash -- | -- | aeb10a3fbb89694c52d47203cc958d3543b21426938a9664348163aacd41e20ea7670617a28d8ce6d8d51492980facd5fab062e8ad664dafd7b8dbff1c2bb54f | b7a8999335ce15b68360478b22af4daaed10e9db50d597e077d731de194208355d1b2134f5635331d9049dc638d05f1f792d52c5890e521f0af3dc2f3e64fbb8 filename | sha512 hash -- | -- | 5654879ac03f4c7193a8df49cfd4b7253add031c197f50bada40942738bf5720d1c06e31a1d1a7bd1b1a540aa46897e4b34ad8a7e087bd206a7b69b9ffaf5edb | 5dce9fee32436c971ef17595f88f3c74f5644ab3af0e3f854a79fb42f3c8d6d8f507fbb0d7b5bcba52ddf1a49ada2559c477278037cf2dadccba72f0398a1093 | e9eb7dab22801c043da2833fde89d2cd721b9dd622df0ff42b25a6742cfab5cff8bfe3ebbda6cc584cf92db3940b95e25aff935863ed999374ee8923ca0b1215 | 570eeaed029bb05235c58138a777cfd6a4b17d4d91aba346b1fc9a0e573781947599d31a8997e889165db561a18a7ab4d613c2b40a8b2dc0d0225f2411b0fd73 | 298923762745cc064a4489aa01d55f57076b84538aef3a6a3554b60257d9959b4eebbb8aeeecdaf14246fa4f1c17750e1b69c63d4940ae71f87010692e41675c | 498527f1cf2d16af576a6b6d27b5ddbb876e24bd85e34e2c91cf39ef467d366b2059e580fdcccb91e0b61a5f52795273b77ed94a1073b5c0bd574b8661afbe0e | 2632b0fb69565819ef1b6797a834e65f96629df4fd8bec01fce7370672a39afa181854d6ab44afc1c4a6b8143158cf170f5a8e61b75a48071ade2d5ab89d1b2c | b793a5a8fce9109343ada86f29cf356c6973cd80d81ca47af5c7e4fa11ffccc273f77aba52b1db42ee12abb94ee23677c21910f57c9385646e35742a1c60e17e | b92e34ee58e1247c1c444134dd9fa78033d0fda1f51509b43016543596cb211128f8aff730d9a3a9118dfeba139186db2a5dd45455427c7521776e63ee77218c | 0b5ea6a2de0ff6f71647f428fdbee67c7eb2b918d725cf236ce60daa02e94bd998d15ea0ebb20c4106453e220d11d31506161d5dee3cde6c616dfb5efd11c25e | a4e570be453d1df779bb85c62efb41e98209bb93b57b7655a94a737d552c90f9d3061df9088204c7787344dc6a3eb3f843c58651394c0436d2c90b55e499bfab filename | sha512 hash -- | -- | cf215ee7372edd7d5dbf07faee8ccf83de477c8cd431c0fac58357bb8e027349d8edf87364e7db5cec0936f991388f7b183e81e5f92cb6cdf6303efd8cd65d83 | d6281a6727fcdab956170dca7563fc5099ef79b06c96b2f6bc87fcd0b74f1dea0e14aca344cb41b5b6811919cf4a6d6f60cb08b7fb7034690fd0c4ead82e55ca | 4ec30cdfd8128ca405201c0c40750e10bac016e1e53a7662265328564b09e4feb831a259125bcdd64169d221145cbb166a463216e884dd76f4bb9a72a00e64e0 | 42b31174a95d0999c78750a1d2c866918c91d11d6406df4e984913f64806708add35c27c0daf255b5d28e98eb815355d1913911f921d34e618dea4d2ebf91949 | c4e2b38681c0858d560adc8a330f27e95a035cb0e426c6ff332dcd435cefe88441ea866badab5514c4055191324c48aac108d5d6934a9fd4697da179168b6632 filename | sha512 hash -- | -- | 708b40c9c0d2cfcb6f9874aa3f1b5a27796cbe2bfe7a2345f381e0d9062df8a6769b2bf29b8b641929ef1f5952897c5739e876e8315eb51cced460b13994c247 | f89448106af23d6658b9c2e7b43240fb82051d2f89a302ee61fc1cc78e593535993ba12f412c3df907f415e55c38f1783ce9141075198bb9f197b6fa26328d49 | bcd8d9fbb244048a3ef3f79f1d4e8f2645bbd69caf353e67ee5c5a4ffd4443da420e5984422933cd4c622c58017a942e20af076f26bfa22f5f38f73a831370ca | c0724d053c601d4e80ea19957bd32005aeba0cf8f5e03e8e36412aed0777e860ae680302eef632c8e7d4ef1a8e789e48dc58489ad1a7bf7fd20cb0f755e797af | 0245f592b92d79ccd102961e5b23a9f5b275829e627254fe8ce5f0a7df53ec2c4a9436942686b9d31b696635ab88131cf92e1002869369fa1cf6f080f8073b5f | 2a5c6c79ea65f47a42d25b236709a00eafb793e5d87b5f56516da16b85b06e03020679f7cabfb7dc4bc252ff57da0afa52e357915cb1d3801dc3d5c32f096edf A deprecation notice has been added when using the kube-proxy Userspace proxier, which will be removed in v1.25. (#103860) (#104631, @perithompson) [SIG Network] Feature-gate VolumeSubpath has been deprecated and cannot be disabled. It will be completely removed in 1.25 (#105474, @mauriciopoppe) [SIG Storage] Kubeadm: remove the deprecated / NO-OP phase "update-cluster-status" in "kubeadm reset" (#105888, @neolit123) [SIG Cluster Lifecycle] Removed kubectl --dry-run empty default value and boolean values. kubectl --dry-run usage must be specified with --dry-run=(server|client|none). (#105327, @julianvmodesto) [SIG CLI and Testing] #### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: <!-- This section can be blank if this pull request does not require a release note. When adding links which point to resources within git repositories, like KEPs or supporting documentation, please reference a specific commit and avoid linking directly to the master branch. This ensures that links reference a specific point in time, rather than a document that may change over time. See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files Please use the following format for linking documentation: [KEP]: <link> [Usage]: <link> [Other doc]: <link>

(#104782, @kerthcet) [SIG Scheduling and Testing] Ephemeral containers have reached beta maturity and are now available by default. (#105405, @verb) [SIG API Machinery, Apps, Node and Testing] Introduce OS field in the Pod Spec (#104693, @ravisantoshgudimetla) [SIG API Machinery and Apps] Introduce v1beta3 api for scheduler. This version increases the weight of user specifiable priorities. The weights of following priority plugins are increased TaintTolerations to 3 - as leveraging node tainting to group nodes in the cluster is becoming a widely-adopted practice NodeAffinity to 2 InterPodAffinity to 2 Won't have HealthzBindAddress, MetricsBindAddress fields (#104251, @ravisantoshgudimetla) [SIG Scheduling and Testing] JSON log output is configurable and now supports writing info messages to stdout and error messages to stderr. Info messages can be buffered in memory. The default is to write both to stdout without buffering, as before. (#104873, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling] JobTrackingWithFinalizers graduates to beta. Feature is enabled by default. (#105687, @alculquicondor) [SIG Apps and Testing] Remove NodeLease feature gate that was graduated and locked to stable in 1.17 release. (#105222, @cyclinder) [SIG Apps, Node and Testing] TTLAfterFinished is now GA and enabled by default (#105219, @sahilvv) [SIG API Machinery, Apps, Auth and Testing] The "Generic Ephemeral Volume" feature graduates to GA. It is now enabled unconditionally. (#105609, @pohly) [SIG API Machinery, Apps, Auth, Node, Scheduling, Storage and Testing] The legacy scheduler policy config is removed in v1.23, the associated flags policy-config-file, policy-configmap, policy-configmap-namespace and use-legacy-policy-config are also removed. Migrate to Component Config instead, see https://kubernetes.io/docs/reference/scheduling/config/ for details. (#105424, @kerthcet) [SIG Scheduling and Testing] Track the number of Pods with a Ready condition in Job status. The feature is alpha and needs the feature gate JobReadyPods to be enabled. (#104915, @alculquicondor) [SIG API Machinery, Apps, CLI and Testing] Add a new `distribute-cpus-across-numa` option to the static `CPUManager` policy. When enabled, this will trigger the `CPUManager` to evenly distribute CPUs across NUMA nodes in cases where more than one NUMA node is required to satisfy the allocation. (#105631, @klueska) [SIG Node] Add support to generate client-side binaries for windows/arm64 platform (#104894, @pacoxu) [SIG CLI, Testing and Windows] Added a new feature gate `CustomResourceValidationExpressions` to enable expression validation for Custom Resource. (#105107, @cici37) [SIG API Machinery] Adds new [alpha] command 'kubectl events' (#99557, @bboreham) [SIG CLI] Client-go, using log level 9, trace the following events of an http request: dns lookup tcp dialing tls handshake time to get a connection from the pool time to process a request (#105156, @aojea) [SIG API Machinery] Client-go: pass `DeleteOptions` down to the fake client `Reactor` (#102945, @chenchun) [SIG API Machinery, Apps and Auth] Enhance scheduler volumebinding plugin to handle Lost PVC as UnschedulableAndUnresolvable during PreFilter stage (#105245, @yibozhuang) [SIG Scheduling and Storage] Feature-gate `StorageObjectInUseProtection` has been deprecated and cannot be disabled. It will be completely removed in 1.25 (#105495, @ikeeip) [SIG Apps] Kubectl will now provide shell completion choices for the --output/-o flag (#105851, @marckhouzam) [SIG CLI] Kubernetes is now built with Golang 1.17.2 (#105563, @mengjiao-liu) [SIG API Machinery, Cloud Provider, Instrumentation, Release and Testing] Move the getAllocatableResources endpoint in podresource-api to the beta that will make it enabled by default. (#105003, @swatisehgal) [SIG Node and Testing] Node affinity, node selector and tolerations are now mutable for jobs that are suspended and have never been started (#105479, @ahg-g) [SIG Apps, Scheduling and Testing] Pod template annotations and labels are now mutable for jobs that are suspended and have never been started (#105980, @ahg-g) [SIG Apps] PodSecurity: add a container image and manifests for the PodSecurity validating admission

(#105923, @liggitt) [SIG Auth] PodSecurity: in 1.23+ restricted policy levels, pods and containers which set runAsUser=0 are forbidden at admission-time; previously, they would be rejected at runtime (#105857, @liggitt) [SIG Auth] Shell completion now knows to continue suggesting resource names when the command supports it. For example "kubectl get pod pod1 <TAB>" will suggest more pod names. (#105711, @marckhouzam) [SIG CLI] Support to enable Hyper-V in GCE Windows Nodes created with kube-up (#105999, @mauriciopoppe) [SIG Cloud Provider and Windows] The CPUManager policy options are now enabled, and we introduce a graduation path for the new CPU Manager policy options. (#105012, @fromanirh) [SIG Node and Testing] The etcd container image now supports Windows. (#92433, @claudiubelu) [SIG API Machinery and Windows] The pods and pod controllers that are exempted from the PodSecurity admission process are now marked with the "pod-security.kubernetes.io/exempt: user/namespace/runtimeClass" annotation, based on what caused the exemption. The enforcement level that allowed or denied pod during PodSecurity admission is now marked by the "pod-security.kubernetes.io/enforce-policy" annotation. The annotation that informs about audit policy violations changed from ""pod-security.kubernetes.io/audit" to ""pod-security.kubernetes.io/audit-violation". (#105908, @stlaz) [SIG Auth] When feature gate JobTrackingWithFinalizers is enabled: Limit the number of pods tracked in a single job sync to avoid starvation of small jobs. The metric jobpodfinished_total counts the number of finished pods tracked by the job controller (#105197, @alculquicondor) [SIG Apps, Instrumentation and Testing] Fixes hostpath storage e2e tests within SELinux enabled env (#104551, @Elbehery) [SIG Testing] (PodSecurity admission) errors validating workload resources (deployment, replicaset, etc.) no longer block admission. (#106017, @tallclair) [SIG Auth] Add Pod Security admission metrics: podsecurityevaluationstotal, podsecurityexemptionstotal, podsecurityerrors_total (#105898, @tallclair) [SIG Auth, Instrumentation and Testing] Apimachinery: pretty-printed json and yaml output is now indented consistently (#105466, @liggitt) [SIG API Machinery] Change `kubectl diff --invalid-arg` status code from 1 to 2 to match docs (#105445, @ardaguclu) [SIG CLI] Client-go uses the same http client for all the generated groups and versions, allowing to share customized transports for multiple groups versions. (#105490, @aojea) [SIG API Machinery, Auth, Instrumentation and Testing] Evicted and other terminated pods will no longer revert to Running phase (#105462, @ehashman) [SIG Node and Testing] Fix pod name of NonIndexed jobs to not include rogue -1 substring (#105676, @alculquicondor) [SIG Apps] Fix scoring for NodeResourcesBalancedAllocation plugins when nodes have containers with no requests. (#105845, @ahmad-diaa) [SIG Scheduling] Fix: consolidate logs for instance not found error fix: skip not found nodes when reconciling LB backend address pools (#105188, @nilo19) [SIG Cloud Provider] Fix: do not delete the lb that does not exist (#105777, @nilo19) [SIG Cloud Provider] Fix: ignore not a VMSS error for VMAS nodes in EnsureBackendPoolDeleted. (#105185, @ialidzhikov) [SIG Cloud Provider] Fix: leave the probe path empty for TCP probes (#105253, @nilo19) [SIG Cloud Provider] Fix: remove VMSS and VMSS instances from SLB backend pool only when necessary (#105839, @nilo19) [SIG Cloud Provider] Fix: skip instance not found when decoupling vmss from lb (#105666, @nilo19) [SIG Cloud Provider] Fixed a bug that prevents PersistentVolume that has a Claim UID which doesn't exist in local cache but exists in ETCD from being updated to Released phase. (#105211, @xiaopingrubyist) [SIG Apps] Fixed architecture within manifest for non `amd64` etcd images. (#105484, @saschagrunert) [SIG API Machinery] Fixes a bug that could result in the EndpointSlice controller unnecessarily updating EndpointSlices associated with a Service that had Topology Aware Hints enabled. (#105267, @llhuii) [SIG Apps and Network] Fixes the `should support building a client with a CSR` e2e test to work with clusters configured with short certificate

(#105396, @liggitt) [SIG Auth and Testing] Generic ephemeral volumes can be used also as raw block devices, but the Pod validation was refusing to create pods with that combination. (#105682, @pohly) [SIG Apps, Storage and Testing] Generic ephemeral volumes were not considered properly by the node limits scheduler filter and the kubelet hostpath check. (#100482, @pohly) [SIG Node, Scheduling, Storage and Testing] Kube-apiserver: fix a memory leak when deleting multiple objects with a deletecollection. (#105606, @sxllwx) [SIG API Machinery] Kubeadm: do not allow empty "--config" paths to be passed to "kubeadm kubeconfig user" (#105649, @navist2020) [SIG Cluster Lifecycle] Kubelet did not report `kubeletvolumestats_*` metrics for generic ephemeral voiumes. (#105569, @pohly) [SIG Node] Kubelet's Node Grace Shutdown will terminate probes when shutting down. (#105215, @rphillips) [SIG Node] Kubernetes object references (= name + namespace) were not logged as struct when using JSON as log output format. (#104877, @pohly) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation and Storage] Podresources interface was changed, now it returns only isolated cpus (#97415, @AlexeyPerevalov) [SIG Node and Testing] Release-note Removed error message label from kubeletstartedpodserrorstotal metric (#105213, @yxxhero) [SIG Instrumentation and Node] Resolves a potential issue with GC and NS controllers which may delete objects after getting a 404 response from the server during its startup. This PR ensures that requests to aggregated APIs will get 503, not 404 while the APIServiceRegistrationController hasn't finished its job. (#104748, @p0lyn0mial) [SIG API Machinery] Revert building binaries with PIE mode. (#105352, @ehashman) [SIG Node, Release and Security] Support more than 100 disk mounts on Windows (#105673, @andyzhangx) [SIG Storage and Windows] Support using negative array index in json patch replace operations. (#105896, @zqzten) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation and Storage] The --leader-elect* CLI args are now honored in scheduler. (#105712, @Huang-Wei) [SIG Scheduling] The client-go dynamic client sets the header 'Content-Type: application/json' by default (#104327, @sxllwx) [SIG API Machinery] The pods/binding subresource now honors `metadata.uid` and `metadata.resourceVersion` preconditions (#105913, @aholic) [SIG Scheduling] Topology Hints now excludes control plane notes from capacity calculations. (#104744, @robscott) [SIG Apps and Network] Watch requests that are delegated to aggregated apiservers no longer reserve concurrency units (seats) in the API Priority and Fairness dispatcher for their entire duration. (#105511, @benluddy) [SIG API Machinery] `--log-flush-frequency` had no effect in several commands or was missing. Help and warning texts were not always using the right format for a command (`adddirheader` instead of `add-dir-header`). Fixing this included cleaning up flag handling in component-base/logs: that package no longer adds flags to the global flag sets. Commands which want the klog and --log-flush-frequency flags must explicitly call logs.AddFlags; the new cli.Run does that for commands. That helper function also covers flag normalization and printing of usage and errors in a consistent way (print usage text first if parsing failed, then the error). (#105076, @pohly) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scheduling and Testing] All klog flags except for `-v` and `-vmodule` are deprecated. Support for `-vmodule` is only guaranteed for the text log format. (#105042, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle and Instrumentation] Kube-apiserver: requests to node, service, and pod `/proxy` subresources with no additional URL path now only automatically redirect GET and HEAD requests. (#95128, @Riaankl) [SIG API Machinery, Architecture and Testing] Migrate `pkg/scheduler/framework/plugins/interpodaffinity/filtering.go`,`pkg/scheduler/framework/plugins/podtopologyspread/filtering.go`, `pkg/scheduler/framework/plugins/volumezone/volume_zone.go` to structured logging (#105931, @mengjiao-liu) [SIG Instrumentation and Scheduling] Migrated `cmd/kube-scheduler/app/server.go`, `pkg/scheduler/framework/plugins/nodelabel/nodelabel.go`, `pkg/scheduler/framework/plugins/nodevolumelimits/csi.go`, `pkg/scheduler/framework/plugins/nodevolumelimits/noncsi.go` to structured logging (#105855, @shivanshu1333) [SIG Instrumentation and Scheduling] Migrated pkg/proxy to structured logging (#104891, @shivanshu1333) [SIG Network] Migrated pkg/proxy/ipvs to structured

(#104932, @shivanshu1333) [SIG Network] Support allocating whole NUMA nodes in the CPUManager when there is not a 1:1 mapping between socket and NUMA node (#102015, @klueska) [SIG Node] sigs.k8s.io/json: c049b76 github.com/evanphx/json-patch: github.com/go-logr/logr: github.com/go-logr/zapr: k8s.io/klog/v2: v2.20.0 v2.30.0 k8s.io/utils: bdf08cb cb0fa31 Nothing has changed. filename | sha512 hash -- | -- | 083e6ca03c9d701768b1b5666f354223a3f7dca9fc6410ce45bbf5947152620e300b46df9b6019134e7d736ba44916537eb3bea8fa57e5f7bc3cc34898b4a5dd | c3fc74d52e1b7e808c03b9caa30e3e73be30eb8330ce676000b93d5324bbdba93bd005d125b999ba937b79d4751af99b37986911365416f7175d223345f95914 filename | sha512 hash -- | -- | 31d8adc657afbd305df18bfec397a825536357e23b241a19aa538b6ddefefc59743f737db98756e04deea89cc6f260d40a80f02b4d1dc34af1d19e8d796dcd8a | b69c4d6cde1c476bafa2ca9916ce3e5bf7286be0ff6a08193bdd1a954ba89b64b1b14193d1acec17ccc141024ee3097971448017b5c9f1327e0961b1e92b2224 | 059f25ee48aa4b0d1621d6ba87af8fb7e765634d723d98a4e9739f50d3703e7dd3973f4d1ed886c0f3ad6eba165ed81d4e63ecde3b39e66fcbec7d3aa2dfed2e | 291dba14160803065895799adcde39bdad7a5b0372403f283d6d5e9a094fe1fc79c70e7546f93ee692b9fd297e2667cb558e4209161ecb4bf89965df5746ed4d | 988e12cd7466033578acc487447df376c409e4f79726a4721af1aedbe931e927b22a93d6224891b61b55c7a0ec12e42d8cfcd40e15a9a0cbbc1dbf0e59ab0341 | b3f21dac41b38e671fa7a95892468e2c27fab51abf9c77b336550e5ec213af204e16cac11dd76262fedb0087cf5ad1950af7e36599a38d50cc270cf831cd4f0b | beebf01e2e4ff09bb711284bb9a5c7cc519e4ac8a826dc829394fa28bd9a3149ba73088eaf6712d39a8cab96b0a1c2859e9d5955fee892b759eaddcdeaa8b93c | 87e5d3d8ba01f9fefb2300e9f06146a254d39d72eaa10cad8c444428b738b3763483ee9eb82f0a13d2ff5aba35fdcb4320598fd5a6a2a07ea3fd00b4ac682d3c | 71bfc5a1df9c47735476af10225830212f68c83357ff7d443e18f9b7881524db910781a95d11ff6697cb587352059b5841f7b24fda40b5302ad252bfb6da7e51 | 078b0c698f9535f3eee41ecf162d57e2ace67243da36067b78b30cfbb7b27cfcf97af4c5db48cdd592953e26b42b31794002eb96317476849e89e2126c6df99d filename | sha512 hash -- | -- | 951b790158dadf46c32e1a1e9c12f2cc8f41e1645602ebff6b4130a08a377bc6d92549186b420332d620d67191123d98a5d717ac0f5ee9643bebe88947ead8fa | 0e7a5b9f39b4f45c45bdb5a19dd3695d28f53e1039d76bc572421c707917944d28b1dbfc36e59214b5bc2b93a787900d8e6eb0b587aa801ea8a8faacdb814a4e | 921e060120b8651a0f80977360faca9f207189cee10bc61f669ceba4e540ef48c0ceff1a877ee4c7d31b01b88096bce93c577f68f93b2341c8542dfd89972b60 | 292cde446b754a87f4ef5384fadbd30017e53ed2744d45a724be467c86ccd9837bfb490db6396642a869937f2f0d080d9655e89ca3345f8365d109a9bcdd18d9 | e0ea667f828ce3b36ca4b2a05fb286da5eb321852c50caf0957694553caf2908b27bcc37a5a82277a2606cf6ff4d9e33617ad61628845d9c21f5cf68c960ca92 filename | sha512 hash -- | -- | e13cd3f75628d354bd1544a5495600fb905741431eb4af4da3d980cc0b7565e3f9c1585d9686cc4e967e54fb854f05bbedfe0c60bb7b855fa027ac8ac45b26e0 | 6c91b42350528692ff558b667bffd41c5b967c7aa6101471274e4b16b0ac6f84afe01722881328fd4f6f8fe71c7852620fa000186c6f7e56e498fcc2c67ad793 | 81728e1388e9cdb436d6847c868f28ab2771331e5e40cd5a7af13cb8dc80a7e4e66a215c12f8183b4884807a3962f913ef5343b889e3c4ecd0e410e8d53aaea9 | 299649f1b25cc38f3a7543ef4d3ee6d42c85e24ac41b4eb61927bc5c5f0c533a39f9ddd4d5ad1df54c625d77aeb41f6c31b1ca7fd8983262f84fefdf1cb2cfd0 | fd6cbc93f98abff9803b43215af6e75a4f7b91ca06969220a779468f34b5ec5ec69f20b529e0cd7b10ba8769bbe2507d46f84ce1d8cd0760380ab9264dd94672 | a5bfaf2e3ad8d3d2127c3e3e0f131c615a03563253da6bf0e1fd793f6ef71287f341ce1bd0d35eb9a81e0721a5baf03e7c72863b5ed8eb45e8fe70573904ed54 Remove 'master' as a valid EgressSelection type in the EgressSelectorConfiguration API. (, ) [SIG API Machinery and Cloud Provider] Remove VolumeSubpath feature gate (, ) [SIG Apps, Node and Storage] The deprecated --experimental-bootstrap-kubeconfig flag has been removed. This can be set via --bootstrap-kubeconfig. (, ) [SIG Node] Client-go impersonation config can specify a UID to pass impersonated uid information through in requests. (, ) [SIG API Machinery, Auth and Testing] IPv6DualStack feature moved to stable. Controller Manager flags for the node IPAM controller have slightly changed: When configuring a dual-stack cluster, the user must specify both --node-cidr-mask-size-ipv4 and --node-cidr-mask-size-ipv6 to set the per-node IP mask sizes, instead of the previous --node-cidr-mask-size flag. The --node-cidr-mask-size flag is mutually exclusive with --node-cidr-mask-size-ipv4 and --node-cidr-mask-size-ipv6. Single-stack clusters do not need to change, but may choose to use the more specific flags. Users can use either the older --node-cidr-mask-size flag or one of the newer --node-cidr-mask-size-ipv4 or --node-cidr-mask-size-ipv6 flags to configure the per-node IP mask size, provided that the flag's IP family matches the cluster's IP family (--cluster-cidr). (, ) [SIG API Machinery, Apps, Auth, Cloud Provider, Cluster Lifecycle, Network, Node and Testing] Kubelet: turn the KubeletConfiguration v1beta1 `ResolverConfig` field from a `string` to `*string`. (, ) [SIG Cluster Lifecycle and Node] Add mechanism to load simple sniffer class into fluentd-elasticsearch image (, ) [SIG Cloud Provider and Instrumentation] Kubeadm: do not check if the '/etc/kubernetes/manifests' folder is empty on joining worker nodes during preflight (, ) [SIG Cluster Lifecycle] The kube-apiserver's Prometheus metrics have been extended with some that describe the costs of handling LIST requests. They are as follows. apiserver_cache_list_total: Counter of LIST requests served from watch cache, broken down by resourceprefix and indexname apiserver_cache_list_fetched_objects_total: Counter of objects read from watch cache in the course of serving a LIST request, broken down by resourceprefix and indexname apiserver_cache_list_evaluated_objects_total: Counter of objects tested in the course of serving a LIST request from watch cache, broken down by resource_prefix apiserver_cache_list_returned_objects_total: Counter of objects returned for a LIST request from watch cache, broken down by resource_prefix apiserver_storage_list_total: Counter of LIST requests served from etcd, broken down by resource apiserver_storage_list_fetched_objects_total: Counter of objects read from etcd in the course of serving a LIST request, broken down by resource apiserver_storage_list_evaluated_objects_total: Counter of objects tested in the course of serving a LIST request from etcd, broken down by resource apiserver_storage_list_returned_objects_total: Counter of objects returned for a LIST request from etcd, broken down by resource (, ) [SIG API Machinery and Instrumentation] Turn on CSIMigrationAzureDisk by default on 1.23 (, ) [SIG Cloud Provider] Changes behaviour of kube-proxy start; does not attempt to set specific sysctl values (which does not work in recent Kernel versions anymore in non-init namespaces), when the current sysctl values are already set higher. (, ) [SIG Network] Fix job controller syncs: In case of conflicts, ensure that the sync happens with the most up to date information. Improves reliability of

(, ) [SIG Apps] Fix system default topology spreading when nodes don't have zone labels. Pods correctly spread by default now. (, ) [SIG Scheduling] Headless Services with no selector which were created without dual-stack enabled will be defaulted to RequireDualStack instead of PreferDualStack. This is consistent with such Services which are created with dual-stack enabled. (, ) [SIG Network] Kube-apiserver: events created via the `events.k8s.io` API group for cluster-scoped objects are now permitted in the default namespace as well for compatibility with events clients and the `v1` API (, ) [SIG API Machinery, Apps and Testing] Kube-controller incorrectly enabled support for generic ephemeral inline volumes if the storage object in use protection feature was enabled. (, ) [SIG API Machinery] Kubeadm: switch the preflight check (called 'Swap') that verifies if swap is enabled on Linux hosts to report a warning instead of an error. This is related to the graduation of the NodeSwap feature gate in the kubelet to Beta and being enabled by default in 1.23 - allows swap support on Linux hosts. In the next release of kubeadm (1.24) the preflight check will be removed, thus we recommend that you stop using it - e.g. via --ignore-preflight-errors or the kubeadm config. (, ) [SIG Cluster Lifecycle] Makes the etcd client (used by the API server) retry certain types of errors. The full list of retriable (codes.Unavailable) errors can be found at https://github.com/etcd-io/etcd/blob/main/api/v3rpc/rpctypes/error.go#L72 (, ) [SIG API Machinery] When a static pod file is deleted and recreated while using a fixed UID, the pod was not properly restarted. (, ) [SIG Node and Testing] XFS-filesystems are now force-formatted (option `-f`) in order to avoid problems being formatted due to detection of magic super-blocks. This aligns with the behaviour of formatting of ext3/4 filesystems. (, ) [SIG Storage] Enhanced error message for nodes not selected by scheduler due to pod's PersistentVolumeClaim(s) bound to PersistentVolume(s) that do not exist. (, ) [SIG Scheduling and Storage] Kubeadm: remove the --port flag from the manifest for the kube-scheduler since the flag has been a NO-OP since 1.23 and insecure serving was removed for the component. (, ) [SIG Cluster Lifecycle] Migrate `cmd/proxy/{config, healthcheck, winkernel}` to structured logging (, ) [SIG Network] Migrate cmd/proxy/app and pkg/proxy/meta_proxier to structured logging (, ) [SIG Apps, Cluster Lifecycle, Network, Node and Testing] Migrate pkg/proxy to structured logs (, ) [SIG Network] Migrated pkg/proxy/winuserspace to structured logging (, ) [SIG Network] The `BoundServiceAccountTokenVolume` feature gate that is GA since v1.22 is unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. (, ) [SIG Auth] The `SupportPodPidsLimit` and `SupportNodePidsLimit` feature gates that are GA since v1.20 are unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. (, ) [SIG Node] Update build images to Debian 11 (Bullseye) debian-base:bullseye-v1.0.0 debian-iptables:bullseye-v1.0.0 go-runner:v2.3.1-go1.17.1-bullseye.0 kube-cross:v1.23.0-go1.17.1-bullseye.1 setcap:bullseye-v1.0.0 cluster/images/etcd: Build 3.5.0-2 image test/conformance/image: Update runner image to base-debian11 (, ) [SIG API Machinery, Architecture, Release and Testing] Nothing has changed. github.com/json-iterator/go: github.com/modern-go/reflect2: Nothing has changed. filename | sha512 hash -- | -- | 121d51f42a52b28e27a4b2f914a4f80fa3fba6328e6a4a5c96dec39c5b28c05461fcc290ef35a49058e237091532b24db3cd8c61801bcb6736aee1dd7dbcffc3 | 641d47241acfadb3b13bccec57795749d2c9e3e07ffa7aa4b30df3a488643631eb8e5cd581bcfb764dff4ac5ed755f72d94e80746142123b09e1675e81421a91 filename | sha512 hash -- | -- | f734cb514ee56adcb2d991a6f0550df907c72f8a61cc2a13117e61b8d5826ff942a582a2e9383deb1a61d5df2243362f1327942a3b4883490eb3296647ce3737 | 24d1f851cd5782f8f39054e37beda1554dadd8a28cb3272b00d50fc095d1fc3018768c1ea72a44eda61ff0f58f71b33dd28cbdc54467d620e87c3694ecf14cc2 | 082ad4abea58de3b629fc2ed4560a836cdbeb1adefb0c4cf47044bf33c750d8fcd8a06e2c4ce365853e83a58d52e0129d510a698dd894bd1261f8184dd1cab42 | b3b0b23479c05b57ca574cf17cdcde7e716033bc4f6a80532d1175d8e533e3202bece0dcf503731d5a60319c526ce1ce4a0bc900bf87536321208a59cf890e35 | f5dac2976ce04310f74bba6102080554309b851fbd966ff1220d3eb23089db8eb8da519a6bd8865c94f2f24346a4d27eb40fd0a3ff06ca9c6874e1fc6f356b67 | 057b372150749b13a38e04802c7cf566765e0fbb27f1b5f7bf6d3cc3f71eb3020916ea7f8579ecc7fcc10e2db1b5c8caa31a1e8a3aac80da86e4e777f515d42f | 9a090d22aeba011c6d039bff59dbdc23ac4a112828db3cbba588d8b0ee1cd14d16e0eacefbb000e5a3ff26bcce4730824819f86a99b7a9826f35fa9964f9f27a | 435e20055badb619289dc7c572af300bd2f86068d0b8f326e8d9abfda5347f2449e316158c412e9b946a2541208c3e8cc6e5c823946e74ac4fc2d594d410179a | 55f192a4d095d494bb53af1b7133124b762a677eb46247b9dba71d10ea6830b37c30d603908e7a9c63f371baff508b19406e89b231ed5ece0497627f09753f68 | 944059d1f1918a793490b95be8130d06189508ba8e79e79ca8cfd2ab98bf396ac551786514b093cc6afe4b3fd15736d728cfcdce18bb32fbee41bc0a97f5c4be filename | sha512 hash -- | -- | a76a4b86ee151ba027f7cf4a2072451ae4c829182bb14e00ce1967421744bfc1e58f141b6eaf2ab27ece67054ae307f8e0768477ab9c3c4749eaad397d495182 | 95aeb4eb473ab4920d81904bc89c6126732b9c6888f9e57493ee99d692042ca44f6844ac1dade1409565f4d9fbec59445402e1f7deac6cbf5b6df16ac814b58c | 3c56e906aafc2a1ac72300352a334662bec5d59e3e523c19b9d65bc52ad9075dc2631f259513efd0f654e220fe0e7d54dfa5028d7eaad81d5d87ca251653f75d | b74bacafe9bb6a7cf407747b03e78ae3873e50deec4eaa08758d5e1d5287ac23af59b3ef26f888fe4cd44ccb1455beafcd1384e700230eb445720e3acae5f2e3 | d3f8f8d9c233b114129f615252d42782cd366978a49506393a40af3f8b5b1250ce99e9806881675e112a69270a0411fb2f00ea19b99ad7415b9e0074beb2726d filename | sha512 hash -- | -- | 146e2f762c179178a57a8c7af7c26470c5d580b8ff8400615162ad1056625f87ce2b32598538d82652f88639e54afb782810529b074c36eb52cc6374414a6181 | 9357d1b387e1b049fb6cec06a7081afc2ce7e906484c9b061fb0449d147a6c4f9c9dc7a9219cdca5ed71df6c73784f360018d9e48d4fa2aa7eeabef60649d7a4 | 8394f8f9d6ee823cb9a470ea67e15d4d0c6aca7065fe826788f50955905373fc3cdddd6db43901c07736588d8d6a3d3e2916bc8d45fd6bd06307583686137a0a | 7211cb426834484bff39f1ab3c9541203429039f8f5e522ca9e28c43da749e197128a3cae28db0467fc339305d2f23f85e8b4ed9ec116506c3d8076744a88d5e | a7c1a38250398171d3df5865749e9928867c4f44106ae66d44cf9f948ce4f4eed9d1f273a5d369996425b1e12482fceccde4c7652770a8c9fb3f161811323b69 | 2007b3b16597cc06b486f87f35b6c637404f07c11d88b8c8e1c2c9bbea97f762bd7d4f9a31f42f78a917c595af5cb89e6885dd88f3766836dc6e4ec79cf084f2 Controller-manager: the following flags have no effect and would be removed in v1.24: `--port` `--address` The insecure port flags `--port` may only be set to 0

Also `metricsBindAddress` and `healthzBindAddress` fields from `kubescheduler.config.k8s.io/v1beta1` are no-op and expected to be empty. Removed in `kubescheduler.config.k8s.io/v1beta2` completely. In addition, please be careful that: kube-scheduler MUST start with `--authorization-kubeconfig` and `--authentication-kubeconfig` correctly set to get authentication/authorization working. liveness/readiness probes to kube-scheduler MUST use HTTPS now, and the default port has been changed to 10259. Applications that fetch metrics from kube-scheduler should use a dedicated service account which is allowed to access nonResourceURLs `/metrics`. (, ) [SIG Cloud Provider, Scheduling and Testing] Removed deprecated metric `schedulervolumeschedulingdurationseconds` (, ) [SIG Instrumentation, Scheduling and Storage] A small regression in Service updates was fixed. The circumstances are so unlikely that probably nobody would ever hit it. (, ) [SIG Network] Introduce v1beta2 for Priority and Fairness with no changes in API spec (, ) [SIG API Machinery and Testing] Kube-apiserver: Fixes handling of CRD schemas containing literal null values in enums. (, ) [SIG API Machinery, Apps and Network] Kubelet: turn the KubeletConfiguration v1beta1 `ResolverConfig` field from a `string` to `*string`. (, ) [SIG Cluster Lifecycle and Node] Kubernetes is now built using go1.17 (, ) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scheduling, Storage and Testing] Removed deprecated `--seccomp-profile-root`/`seccompProfileRoot` config (, ) [SIG Node] Since golang 1.17 both net.ParseIP and net.ParseCIDR rejects leading zeros in the dot-decimal notation of IPv4 addresses. Kubernetes will keep allowing leading zeros on IPv4 address to not break the compatibility. IMPORTANT: Kubernetes interprets leading zeros on IPv4 addresses as decimal, users must not rely on parser alignment to not being impacted by the associated security advisory: CVE-2021-29923 golang standard library "net" - Improper Input Validation of octal literals in golang 1.16.2 and below standard library "net" results in indeterminate SSRF & RFI vulnerabilities. Reference: https://nvd.nist.gov/vuln/detail/CVE-2021-29923 (, ) [SIG API Machinery, Apps, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage and Testing] StatefulSet minReadySeconds is promoted to beta (, ) [SIG Apps and Testing] The `Service.spec.ipFamilyPolicy` field is now required in order to create or update a Service as dual-stack. This is a breaking change from the beta behavior. Previously the server would try to infer the value of that field from either `ipFamilies` or `clusterIPs`, but that caused ambiguity on updates. Users who want a dual-stack Service MUST specify `ipFamilyPolicy` as either "PreferDualStack" or "RequireDualStack". (, ) [SIG API Machinery, Apps, Network and Testing] Users of LogFormatRegistry in component-base must update their code to use the logr v1.0.0 API. The JSON log output now uses the format from go-logr/zapr (no `v` field for error messages, additional information for invalid calls) and has some fixes (correct source code location for warnings about invalid log calls). (, ) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation and Storage] When creating an object with generateName, if a conflict occurs the server now returns an AlreadyExists error with a retry option. (, ) [SIG API Machinery] Add fish shell completion to kubectl (, ) [SIG CLI] Added PowerShell completion generation by running `kubectl completion powershell` (, ) [SIG CLI] Added a `Processing` condition for the workqueue API Changed `Shutdown` for the workqueue API to wait until the work queue finishes processing all in-flight items. (, ) [SIG API Machinery and Apps] Added a new flag `--append-server-path` to `kubectl proxy` that will automatically append the kube context server path to each

(, ) [SIG API Machinery, CLI and Testing] Added support for setting controller-manager log level online (, ) [SIG API Machinery, Apps and Cloud Provider] Adding support for multiple --from-env-file flags (, ) [SIG CLI] Cloud providers can set service account names for cloud controllers. (, ) [SIG API Machinery and Cloud Provider] Health check of kube-controller-manager now includes each controller. (, ) [SIG API Machinery and Cloud Provider] Kubernetes is now built with Golang 1.17.1 (, ) [SIG API Machinery, Cloud Provider, Instrumentation, Release and Testing] The pause image list now contains Windows Server 2022 (, ) [SIG Windows] Updates debian-iptables to v1.6.7 to pick up CVE fixes (, ) [SIG API Machinery, Network, Release, Security and Testing] Conformance: the test ", ) [SIG Architecture, Network and Testing] A pod that the Kubelet rejects was still considered as being accepted for a brief period of time after rejection, which might cause some pods to be rejected briefly that could fit on the node. A pod that is still terminating (but has status indicating it has failed) may also still be consuming resources and so should also be considered. (, ) [SIG Node] Changed kubectl describe to compute Age of an event using the count and lastObservedTime fields available in the event series (, ) [SIG CLI] Don't prematurely close reflectors in case of slow initialization in watch based manager to fix issues with inability to properly mount secrets/configmaps. (, ) [SIG Node] Fix Job tracking with finalizers for more than 500 pods, ensuring all finalizers are removed before counting the Pod. (, ) [SIG Apps and Instrumentation] Fix a regression where the Kubelet failed to exclude already completed pods from calculations about how many resources it was currently using when deciding whether to allow more pods. (, ) [SIG Node] Fix detach disk issue on deleting vmss node (, ) [SIG Cloud Provider] Fix: ensure InstanceShutdownByProviderID return false for creating Azure VMs (, ) [SIG Cloud Provider] Fix: ignore the case when comparing azure tags in service annotation (, ) [SIG Cloud Provider] Fix: ignore the case when updating Azure tags (, ) [SIG Cloud Provider] Fixed bug where kubectl would emit duplicate warning messages for flag names that contain an underscore and recommend using a nonexistent flag in some cases (, ) [SIG CLI and Cluster Lifecycle] Fixed client IP preservation for NodePort service with protocol SCTP in ipvs mode (, ) [SIG Network] Fixed occasional pod cgroup freeze when using cgroup v1 and systemd driver. (, ) [SIG Node] Fixes a regression that could cause panics in LRU caches in controller-manager, kubelet, kube-apiserver, or client-go (, ) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation and Storage] Kube-apiserver: fixes an issue where an admission webhook can observe a v1 Pod object that does not have the `defaultMode` field set in the injected service account token volume (, ) [SIG Auth] Kube-proxy health check ports used to listen to :<port> for each of the services. This is not needed and opens ports in addresses the cluster user may not have intended. The PR limits listening to all node address which are controlled by `--nodeport-addresses` flag. if no addresses are provided then we default to existing behavior by listening to :<port> for each service (, ) [SIG Network] Kube-scheduler now doesn't print any usage message when unknown flag is specified (, ) [SIG Scheduling] Metrics changes: Fix exposed buckets of `schedulervolumeschedulingdurationseconds_bucket` metric (, ) [SIG Apps, Instrumentation, Scheduling and Storage] Scheduler resource metrics over fractional binary quantities (2.5Gi, 1.1Ki) were incorrectly reported as very small

(, ) [SIG API Machinery and Scheduling] Generic ephemeral volumes: better pod events ("waiting for ephemeral volume controller to create the persistentvolumeclaim"" instead of "persistentvolumeclaim not found") (, ) [SIG Scheduling and Storage] Kubeadm: remove the deprecated flags "--csr-only" and "--csr-dir" from "kubeadm certs renew". Please use "kubeadm certs generate-csr" instead. (, ) [SIG Cluster Lifecycle] Migrate `pkg/scheduler` to structured logging (, ) [SIG Scheduling] Migrated pkg/proxy/userspace to structured logging (, ) [SIG Network] More detailed logging has been added to the EndpointSlice controller for Topology Aware Hints. (, ) [SIG Apps and Network] Support for Windows Server 2022 was added to the k8s.gcr.io/pause:3.6 image. (, ) [SIG CLI, Cloud Provider, Cluster Lifecycle, Node, Release and Testing] The maximum length of the CSINode id field has increased to 256 bytes to match the CSI spec (, ) [SIG Storage] Update conformance image to use debian-base:buster-v1.9.0 (, ) [SIG Architecture, Release, Security and Testing] `volume.kubernetes.io/storage-provisioner` annotation will be added to dynamic provisioning required PVC. `volume.beta.kubernetes.io/storage-provisioner` annotation is deprecated. (, ) [SIG Apps and Storage] bazil.org/fuse: 371fbbd github.com/go-logr/zapr: github.com/kr/fs: github.com/pkg/sftp: github.com/Microsoft/go-winio: github.com/Microsoft/hcsshim: github.com/benbjohnson/clock: github.com/bketelsen/crypt: github.com/containerd/cgroups: github.com/containerd/containerd: github.com/containerd/continuity: github.com/containerd/fifo: github.com/containerd/go-runc: github.com/containerd/typeurl: github.com/go-logr/logr: github.com/magiconair/properties: github.com/mitchellh/go-homedir: github.com/mitchellh/mapstructure: github.com/opencontainers/runc: github.com/pelletier/go-toml: github.com/spf13/afero: github.com/spf13/cast: github.com/spf13/cobra: github.com/spf13/jwalterweatherman: github.com/spf13/viper: github.com/yuin/goldmark: go.uber.org/zap: v1.17.0 v1.19.0 golang.org/x/crypto: 5ea612d 32db794 golang.org/x/net: abc4532 60bc85c golang.org/x/oauth2: f6687ab 2bc19b1 golang.org/x/sys: 59db8d7 41cdb87 golang.org/x/term: 6a3ed07 6886f2d golang.org/x/tools: v0.1.2 d4cc65f gopkg.in/ini.v1: v1.51.0 v1.62.0 k8s.io/klog/v2: v2.9.0 v2.20.0 k8s.io/utils: efc7438 bdf08cb sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.22 v0.0.23 github.com/coreos/bbolt: github.com/coreos/etcd: github.com/coreos/go-systemd: github.com/coreos/pkg: github.com/dgrijalva/jwt-go: gotest.tools: v2.2.0+incompatible filename | sha512 hash -- | -- | f7c76f1e077b5d98019347b2c9b79eaa0c79d428542b9c15dab23886c276ca16314f200ca37af914c52264c0e1e5d0bde639d6adf37368d5e7b29d230df00d95 | f267f26eca20cd7018e68abeeed38aed5c10dbbae7c531c4e08e507196a4dd3f511eb8d41ee8b09495544337d8e1940a8ca04e94084f8dd172698a96564fb070 filename | sha512 hash -- | -- | deb110839c2c3cf94ca9b29df2f0b07b3fad6937d7bb6e9d2516d01345c8e324f6ab86fe1d34f1443f04c3d1fc328b53b3d756c295f4ed22f1994071fbc8c9cb | 1473cb9fc4847b0daff6c9e3189ce55fadc22fb6190161e744e5438066a714cb467fdebfb35f6445a27f5010df94ee602fff492a2382e0f308fda111d53af1f4 | ed5f5b0777ca51790d185764afc2c812f82ae27c35d897570fc86cabee90dc0a445d9d8c37c981bd3684ba9cd47dc0d75d0094578e79ef7b591d3c1b6564280f | 39f2a888e7a43c9e4a4018301894786f6babe23d79ab7a143e06444f69bc14aec2e158d355c5b48da4356e7bd72ec9b1268f8b12815c8b709395f36ad9a68a2f | b6b8333d8adb4bc6a943bcd2c6cd1a0aeaf0b926d06aa03b759e3c723c81ccc91804debc64fedcd7d678eefdee9bdacc52b2891bd084a15fd5f7918a70e51a15 | 3cb8217b9a5363cebad4989253e02c8a37259b61eafc2f08681508c11c5f68448cad43282257c3d90ad510cc9a62645b7f1adeb99fedf5e13c181495e3754ee4 | e411700fb13b25deca6347983cdafe47199f0df00086ccd7b3e7d52a7b3bee7e96a85c2568dd52c956fd4ea8b4a6991859c57c9b73a13e06440b456c65b11687 | 6c1395792a175de77436352d0893476363497b0f6a616f4415f91aed5e780d1f25b515021939a7563046237c7b651caba0d1fbf7c4c461677d1b9308b227e94c | f3aec7136c21d24a99145ce294a859078fcbf11bae132b8b4081555a6656c0d95ccbaca02a86dc257d557ecebc0673d0771b9cdd10593712a643e8cc0f61d681 | b29697ba0a25f3d871ffbe5800dcb23ec9fd27c0122a284e17c21f1258f7dd9d341813aeb7826159c7999581a16db19fbb6eeeab48f5c89975df7595d19102c3 filename | sha512 hash -- | -- | a5b3edca559b84cd9d22b43b23d0607951d434e185dcb313b831604d83dd306cfc017599994d3944ce77360116024eb59a302851325bb2c29c185a80db2e6eac | 2334dbcff3ba22a50f252998eb63991b6c816659dbaa5f749370fc1b1f78f0af7739e50ab64c14a23c4e7dfa8917568e2a3b85bdffdb2cc691ee23ae8f5c8326 | 58674443ce6e359a995dd7c4289bf730e616bcaf336837b77333a206d4e98693d9356a0a670ffbe0b274e2997a8b76a164153cf084f0ff5f91f40f00b5512684 | f60ebdd04e2348b1ba51540cad93fa24cb133fd25db97150000bffaff8ccb41e1b6506bcde6b7d913aee7701478f975a97775430a82980105383fdb1cc13d260 | ff008aa0ba1bf755f32c7251c6aceb12b6f9de00d2e2729302b51960e70e486bd82da62d21d70ad81c14e01910ab2afe0fd2509ebfdec050d36f88ee1f0330b2 filename | sha512 hash -- | -- | 352502f10fbc4579bd9556e3f73ca7513184371ea563d12a39d655d39bb14ccf0f485f4f2b54a77d984c91ff0de2acea7225f98532a1247da5b9ecc65081bc1a | af9de95e2b9e4c1f39cb9757d4dca020f7d276b6702302a2d92e7a93e9986528615ce54531e62b96f6e8a0b9863cddbb264f42b1f59374948ac3499af60d9532 | 45a286cb1d469b16d046af02047cf63a8407222e4a39fe696f5652e0587e0c9ffbdbab6505ce85e2726ba10db3189a7fbe70e316bc610caedc8cbb49fed28076 | 7a540a3ff0295998a1679b0ccd50cb1825faf1d0afd6ed08138ab3767c83a2743aa43b122c8da89ee00161f57c0af8d76012e890f9fe6d77b4ee8aff4e32e50f | 3cd7656221ac2fa161abcf237878cff26c1d97cf77d9b784736c97a56841397ff859e43947d81a83f8fe4164701da41a1dad69b551c4e1fee49b3f8196878236 | 21e63913024e88a48244a598cd400fbae6ce8f8910202f1b635812fbc9281b7c6097eb10a321dd18846484a198845bba58970d83b5119a367862cf8418d4d08c #### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: <!-- This section can be blank if this pull request does not require a release note. When adding links which point to resources within git repositories, like KEPs or supporting documentation, please reference a specific commit and avoid linking directly to the master branch. This ensures that links reference a specific point in time, rather than a document that may change over time. See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files Please use the following format for linking documentation: [KEP]: <link> [Usage]: <link> [Other doc]: <link> --> (, ) [SIG Node] Kubeadm: remove the deprecated flag --experimental-patches for the init|join|upgrade commands. The flag --patches is no longer allowed in a mixture with the flag --config. Please use the kubeadm configuration for setting patches for a node using {Init|Join}Configuration.patches. (, ) [SIG Cluster Lifecycle] Add apiserverlongrunningrequests metric to replace the soon to be deprecated apiserverlongrunninggauge metric. (, ) [SIG API Machinery, Cluster Lifecycle and Instrumentation] Kubeadm: remove the --port flag from the manifest for the kube-controller-manager since the flag has been a NO-OP since 1.22 and insecure serving was removed for the component. (, ) [SIG Cluster Lifecycle] CSIDriver.Spec.StorageCapacity can now be modified. (, ) [SIG Storage] Kube-apiserver: The `rbac.authorization.k8s.io/v1alpha1` API version is removed; use the `rbac.authorization.k8s.io/v1` API, available since v1.8. The `scheduling.k8s.io/v1alpha1` API version is removed; use the `scheduling.k8s.io/v1` API, available since v1.14. (, ) [SIG API Machinery, Auth, Network and Testing] Kube-controller-manager supports '--concurrent-ephemeralvolume-syncs' flag to set the number of ephemeral volume controller

(, ) [SIG API Machinery and Apps] Adding support for multiple --from-env-file flags (, ) [SIG CLI] All folks to build kubernetes with a custom kube-cross image (, ) [SIG Release and Testing] Allow node expansion of local volumes (, ) [SIG Storage and Testing] Client-go event library allows customizing spam filtering function. It is now possible to override `SpamKeyFunc`, which is used by event filtering to detect spam in the events. (, ) [SIG API Machinery and Instrumentation] Constants/variables from k8s.io for STABLE metrics is now supported (, ) [SIG Auth, Instrumentation, Node and Testing] Display Labels when kubectl describe ingress (, ) [SIG CLI] Expose a `NewUnstructuredExtractor` from apply configurations `meta/v1` package that enables extracting objects into unstructured apply configurations (, ) [SIG API Machinery, Cluster Lifecycle, Release and Testing] Introduce a feature gate DisableKubeletCloudCredentialProviders which allows disabling the in-tree kubelet credential providers. The DisableKubeletCloudCredentialProviders FeatureGate is currently in Alpha, which means is currently disabled by default. Once the FeatureGate moves to beta, in-tree credential providers will be disabled by default, and users will need to migrate to using external credential providers. (, ) [SIG Cloud Provider] Introduces a new metric: admissionwebhookrequest_total with the following labels: name (string) - the webhook name, type (string) - the admission type, operation (string) - the requested verb, code (int) - the HTTP status code, rejected (bool) - whether the request was rejected, namespace (string) - the namespace of the requested resource. (, ) [SIG API Machinery and Instrumentation] Kube-up.sh installs csi-proxy v1.0.1-gke.0 (, ) [SIG Cloud Provider, Storage and Windows] Kubeadm: add support for dry running "kubeadm join". The new flag "kubeadm join --dry-run" is similar to the existing flag for "kubeadm init/upgrade" and allows you to see what changes would be applied. (, ) [SIG Cluster Lifecycle] Kubernetes is now built with Golang 1.16.7 (, ) [SIG Cloud Provider, Instrumentation, Release and Testing] The ServiceAccountIssuerDiscovery feature gate is removed. It reached GA in Kubernetes 1.21. (, ) [SIG API Machinery and Auth] Updated Cluster Autosaler to version 1.22.0. Release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.22.0 (, ) [SIG Autoscaling and Cloud Provider] Updates the following images to pick up CVE fixes: debian to v1.9.0 debian-iptables to v1.6.6 setcap to v2.0.4 (, ) [SIG API Machinery, Release and Testing] Update description of --audit-log-maxbackup to describe behavior when value = 0 (, ) [SIG API Machinery] Changes json representation for a conflicted taint to Key=Effect when a conflicted taint occurs in kubectl taint. (, ) [SIG CLI] A new server run option 'shutdown-send-retry-after' has been introduced. If true the HTTP Server will continue listening until all non longrunning request(s) in flight have been drained, during this window all incoming requests will be rejected with a status code 429 and a 'Retry-After' response header. (, ) [SIG API Machinery] Adds Kubernetes Events to the Kubelet Graceful Shutdown feature (, ) [SIG Node] CA, certificate and key bundles for the generic-apiserver based servers will be reloaded immediately after the files are changed. (, ) [SIG API Machinery and Testing] Fix kube-apiserver metric reporting for the deprecated watch path of /api/<version>/watch/... (, ) [SIG API Machinery and Instrumentation] Fix: skip case sensitivity when checking Azure NSG rules (, ) [SIG Cloud Provider] Fixed an issue which didn't append OS's environment variables with the one provided in Credential Provider Config file, which may lead to failed execution of external credential provider binary. See https://github.com/kubernetes/kubernetes/issues/102750 (, ) [SIG Auth and Node] Fixed architecture within manifest for non `amd64` etcd

(, ) [SIG API Machinery] Fixed bug where kubectl would emit duplicate warning messages for flag names that contain an underscore and recommend using a nonexistent flag in some cases (, ) [SIG CLI and Cluster Lifecycle] Graceful node shutdown, allow the actual inhibit delay to be greater than the expected inhibit delay (, ) [SIG Node] Kube-apiserver: Avoids unnecessary repeated calls to admission webhooks that reject an update or delete request. (, ) [SIG API Machinery] Kube-proxy: delete stale conntrack UDP entries for loadbalancer ingress IP. (, ) [SIG Network] Kubeadm: When adding an etcd peer to an existing cluster, if an error is returned indicating the peer has already been added, this is accepted and a ListMembers call is used instead to return the existing cluster. This helps diminish the exponential backoff when the first AddMember call times out, while still retaining a similar performance when the peer had already been added from a previous call. (, ) [SIG Cluster Lifecycle] Pass additional flags to subpath mount to avoid flakes in certain conditions (, ) [SIG Storage] Update Go used to build migrate script in etcd image to v1.16.7 (, ) [SIG API Machinery and Release] Deprecate apiserverlongrunninggauge and apiserverregisterwatchers in 1.23.0 (, ) [SIG API Machinery, Cluster Lifecycle and Instrumentation] Kube-apiserver: sets an upper-bound on the lifetime of idle keep-alive connections and time to read the headers of incoming requests (, ) [SIG API Machinery and Node] Kubeadm: external etcd endpoints passed in the ClusterConfiguration that have Unicode characters are no longer IDNA encoded (converted to Punycode). They are now just URL encoded as per Go's implementation of RFC-3986, have duplicate "/" removed from the URL paths, and passed like that directly to the kube-apiserver --etcd-servers flag. If you have etcd endpoints that have Unicode characters, it is advisable to encode them in advance with tooling that is fully IDNA compliant. If you don't do that, the Go standard library (used in k8s and etcd) would do it for you when making requests to the endpoints. (, ) [SIG Cluster Lifecycle] Kubeadm: update references to legacy artifacts locations, the 'ci-cross' prefix has been removed from the version match as it does not exist in the new 'gs://k8s-release-dev' bucket (, ) [SIG Cluster Lifecycle] Migratecmd/kube-proxy/app logs to structured logging (, ) [SIG Network] Surface warning when users don't set propagationPolicy for jobs while deleting (, ) [SIG Apps] The AllowInsecureBackendProxy feature gate is removed. It reached GA in Kubernetes 1.21. (, ) [SIG API Machinery] The `StartupProbe` feature gate that is GA since v1.20 is unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. (, ) [SIG Node] The apiserver exposes 4 new metrics that allow to track the status of the Service CIDRs allocations: current number of available IPs per Service CIDR current number of used IPs per Service CIDR total number of allocation per Service CIDR total number of allocation errors per ServiceCIDR (, ) [SIG Apps, Instrumentation and Network] The flag `--deployment-controller-sync-period` has no effect now, deprecate it and will be removed in v1.24. (, ) [SIG Apps] Troubleshooting: informers log handlers that take more than 100 milliseconds to process an object if the DeltaFIFO queue starts to grow beyond 10 elements. (, ) [SIG API Machinery] Update cri-tools dependency to v1.22.0 (, ) [SIG Cloud Provider and Node] ``gcr.io/kubernetes-e2e-test-images`` will no longer be used in E2E / CI testing, ``k8s.gcr.io/e2e-test-images`` will be used instead. (, ) [SIG API Machinery and Testing] github.com/google/martian/v3: github.com/kr/fs: github.com/pkg/sftp: cloud.google.com/go/bigquery: v1.4.0 v1.8.0 cloud.google.com/go/storage: v1.6.0 v1.10.0 cloud.google.com/go: v0.54.0 v0.81.0 github.com/GoogleCloudPlatform/k8s-cloud-provider: github.com/bketelsen/crypt: github.com/golang/mock: github.com/google/pprof: github.com/hashicorp/golang-lru: github.com/ianlancetaylor/demangle: github.com/magiconair/properties: github.com/mitchellh/go-homedir: