Repository: tomplus/kubernetes_asyncio
Pull request #359 (2025-05-20 20:34:49)

Allow for async refresh_api_key_hook methods.
Rather than creating an issue, I created this draft pull request to demonstrate what the issue is.

My specific use case is I have implemented the ability to obtain AWS EKS credentials which expire every 15 minutes using an AWS async client. I suspect that more often than not, refreshing credentials will involve i/o and would benefit from async support.

tomplus: Thanks for your PR. Could you provide more details about your EKS cluster? I think EKS uses "exec" in kubeconfig and it should work out of the box. Does `kubectl` work for you?
iciclespider: Yes, kubectl works for me, however the clusters I want to access are not configured for it.

What I have implemented is essentially replicating what the kubectl exec is doing, but in pure python. The EKS clusters I am accessing are dynamically discovered and do not exist in my kubectl configuration. The program shows how this works using hard coded values instead of being dynamically discovered.
```python
import aiobotocore.session
import asyncio
import atexit
import base64
import kubernetes_asyncio
import os
import tempfile
import time


async def main():
    aws_session = aiobotocore.session.AioSession()
    async with aws_session.create_client('eks') as eks:
        response = await eks.describe_cluster(name='enablement')
        configuration = EksConfiguration(aws_session, response['cluster'])
        async with kubernetes_asyncio.client.ApiClient(configuration) as api:
            v1 = kubernetes_asyncio.client.CoreV1Api(api)
            response = await v1.list_node()
            for node in response.items:
                print(node.metadata.name)


class EksConfiguration(kubernetes_asyncio.client.Configuration):
    def __init__(self, aws_session, cluster):
        super(EksConfiguration, self).__init__(
            cluster['endpoint'],
            {'BearerToken': True},
            {'BearerToken': 'Bearer'},
            ssl_ca_cert=temp_file(
                base64.b64decode(cluster['certificateAuthority']['data']).decode()
            ),
        )
        self.aws_session = aws_session
        self.cluster_name = cluster['name']
        self.token_expires = 0
        self.refresh_api_key_hook = self.refresh_eks_token

    @staticmethod
    async def refresh_eks_token(self):
        if time.time() < self.token_expires:
            return
        async with self.aws_session.create_client('sts') as sts:
            sts.meta.events.register(
                'before-sign.sts.GetCallerIdentity', self.inject_cluster_name_header
            )
            url = await sts.generate_presigned_url(
                ClientMethod='get_caller_identity',
                ExpiresIn=15 * 60,  # max 15 minutes
                HttpMethod='GET',
            )
        self.api_key[
            'BearerToken'
        ] = f"k8s-aws-v1.{base64.urlsafe_b64encode(url.encode()).decode().rstrip('=')}"
        self.token_expires = time.time() + (10 * 60)  # refresh every 10 minutes

    def inject_cluster_name_header(self, request, **_kwargs):
        request.headers['x-k8s-aws-id'] = self.cluster_name


temp_files = []

def temp_file(content):
    if not temp_files:
        atexit.register(cleanup_temp_files)
    name = tempfile.mkstemp()[1]
    temp_files.append(name)
    with open(name, "w") as fd:
        fd.write(content)
    return name

def cleanup_temp_files():
    for temp_file in temp_files:
        try:
            os.remove(temp_file)
        except OSError:
            pass
    temp_files.clear()


if __name__ == '__main__':
    asyncio.run(main())
```
This is just one example of having to perform I/O to refresh an api token. In my opinion, such token refreshes are more likely than not to involved I/O.
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/tomplus/kubernetes_asyncio/pull/359?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tomasz+Prus) Report
Attention: Patch coverage is `90.90909%` with `1 line` in your changes missing coverage. Please review.
> Project coverage is 27.06%. Comparing base [(`b377772`)](https://app.codecov.io/gh/tomplus/kubernetes_asyncio/commit/b3777725c70c3e7499f8634633d648f936b1bea6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tomasz+Prus) to head [(`e70b107`)](https://app.codecov.io/gh/tomplus/kubernetes_asyncio/commit/e70b1072fd3f6baac01be0943e2793fde9bbdf30?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tomasz+Prus).

| [Files with missing lines](https://app.codecov.io/gh/tomplus/kubernetes_asyncio/pull/359?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tomasz+Prus) | Patch % | Lines |
|---|---|---|
| [kubernetes\_asyncio/client/configuration.py](https://app.codecov.io/gh/tomplus/kubernetes_asyncio/pull/359?src=pr&el=tree&filepath=kubernetes_asyncio%2Fclient%2Fconfiguration.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tomasz+Prus#diff-a3ViZXJuZXRlc19hc3luY2lvL2NsaWVudC9jb25maWd1cmF0aW9uLnB5) | 83.33% | [1 Missing :warning: ](https://app.codecov.io/gh/tomplus/kubernetes_asyncio/pull/359?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tomasz+Prus) |

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##           master     #359   +/-   ##
=======================================
  Coverage   27.06%   27.06%           
=======================================
  Files         805      805           
  Lines       98717    98720    +3     
=======================================
+ Hits        26715    26717    +2     
- Misses      72002    72003    +1     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/tomplus/kubernetes_asyncio/pull/359?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tomasz+Prus).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Tomasz+Prus).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>

Patch:
diff --git a/kubernetes_asyncio/client/api_client.py b/kubernetes_asyncio/client/api_client.py
index 81c12e00..02c9d0f2 100644
--- a/kubernetes_asyncio/client/api_client.py
+++ b/kubernetes_asyncio/client/api_client.py
@@ -165,7 +165,7 @@ class ApiClient(object):
             post_params.extend(self.files_parameters(files))
 
         # auth setting
-        self.update_params_for_auth(
+        await self.update_params_for_auth(
             header_params, query_params, auth_settings,
             request_auth=_request_auth)
 
@@ -548,7 +548,7 @@ class ApiClient(object):
         else:
             return content_types[0]
 
-    def update_params_for_auth(self, headers, queries, auth_settings,
+    async def update_params_for_auth(self, headers, queries, auth_settings,
                                request_auth=None):
         """Updates header and query params based on authentication setting.
 
@@ -566,7 +566,7 @@ class ApiClient(object):
             return
 
         for auth in auth_settings:
-            auth_setting = self.configuration.auth_settings().get(auth)
+            auth_setting = (await self.configuration.auth_settings()).get(auth)
             if auth_setting:
                 self._apply_auth_params(headers, queries, auth_setting)
 
diff --git a/kubernetes_asyncio/client/configuration.py b/kubernetes_asyncio/client/configuration.py
index 720bb81f..b2522c23 100644
--- a/kubernetes_asyncio/client/configuration.py
+++ b/kubernetes_asyncio/client/configuration.py
@@ -12,6 +12,7 @@
 
 from __future__ import absolute_import
 
+import asyncio
 import copy
 import logging
 import sys
@@ -370,7 +371,7 @@ conf = client.Configuration(
         self.__logger_format = value
         self.logger_formatter = logging.Formatter(self.__logger_format)
 
-    def get_api_key_with_prefix(self, identifier, alias=None):
+    async def get_api_key_with_prefix(self, identifier, alias=None):
         """Gets API key (with prefix if set).
 
         :param identifier: The identifier of apiKey.
@@ -378,7 +379,9 @@ conf = client.Configuration(
         :return: The token for api key authentication.
         """
         if self.refresh_api_key_hook is not None:
-            self.refresh_api_key_hook(self)
+            result = self.refresh_api_key_hook(self)
+            if asyncio.iscoroutine(result):
+                await result
         key = self.api_key.get(identifier, self.api_key.get(alias) if alias is not None else None)
         if key:
             prefix = self.api_key_prefix.get(identifier)
@@ -402,7 +405,7 @@ conf = client.Configuration(
             basic_auth=username + ':' + password
         ).get('authorization')
 
-    def auth_settings(self):
+    async def auth_settings(self):
         """Gets Auth Settings dict for api client.
 
         :return: The Auth Settings information dict.
@@ -413,7 +416,7 @@ conf = client.Configuration(
                 'type': 'api_key',
                 'in': 'header',
                 'key': 'authorization',
-                'value': self.get_api_key_with_prefix(
+                'value': await self.get_api_key_with_prefix(
                     'BearerToken',
                 ),
             }
diff --git a/scripts/api_client_async_refresh_api_key_hook.diff b/scripts/api_client_async_refresh_api_key_hook.diff
new file mode 100644
index 00000000..9fd5df4c
--- /dev/null
+++ b/scripts/api_client_async_refresh_api_key_hook.diff
@@ -0,0 +1,31 @@
+diff --git a/kubernetes_asyncio/client/api_client.py b/kubernetes_asyncio/client/api_client.py
+index 81c12e00..02c9d0f2 100644
+--- a/kubernetes_asyncio/client/api_client.py
++++ b/kubernetes_asyncio/client/api_client.py
+@@ -165,7 +165,7 @@ class ApiClient(object):
+             post_params.extend(self.files_parameters(files))
+
+         # auth setting
+-        self.update_params_for_auth(
++        await self.update_params_for_auth(
+             header_params, query_params, auth_settings,
+             request_auth=_request_auth)
+
+@@ -548,7 +548,7 @@ class ApiClient(object):
+         else:
+             return content_types[0]
+
+-    def update_params_for_auth(self, headers, queries, auth_settings,
++    async def update_params_for_auth(self, headers, queries, auth_settings,
+                                request_auth=None):
+         """Updates header and query params based on authentication setting.
+
+@@ -566,6 +566,6 @@ class ApiClient(object):
+             return
+
+         for auth in auth_settings:
+-            auth_setting = self.configuration.auth_settings().get(auth)
++            auth_setting = (await self.configuration.auth_settings()).get(auth)
+             if auth_setting:
+                 self._apply_auth_params(headers, queries, auth_setting)
+
diff --git a/scripts/client_configuration_async_refresh_api_key_hook.diff b/scripts/client_configuration_async_refresh_api_key_hook.diff
new file mode 100644
index 00000000..3929ba1e
--- /dev/null
+++ b/scripts/client_configuration_async_refresh_api_key_hook.diff
@@ -0,0 +1,50 @@
+diff --git a/kubernetes_asyncio/client/configuration.py b/kubernetes_asyncio/client/configuration.py
+index 720bb81f..b2522c23 100644
+--- a/kubernetes_asyncio/client/configuration.py
++++ b/kubernetes_asyncio/client/configuration.py
+@@ -12,6 +12,7 @@
+
+ from __future__ import absolute_import
+
++import asyncio
+ import copy
+ import logging
+ import sys
+@@ -370,7 +371,7 @@ conf = client.Configuration(
+         self.__logger_format = value
+         self.logger_formatter = logging.Formatter(self.__logger_format)
+
+-    def get_api_key_with_prefix(self, identifier, alias=None):
++    async def get_api_key_with_prefix(self, identifier, alias=None):
+         """Gets API key (with prefix if set).
+
+         :param identifier: The identifier of apiKey.
+@@ -378,7 +379,9 @@ conf = client.Configuration(
+         :return: The token for api key authentication.
+         """
+         if self.refresh_api_key_hook is not None:
+-            self.refresh_api_key_hook(self)
++            result = self.refresh_api_key_hook(self)
++            if asyncio.iscoroutine(result):
++                await result
+         key = self.api_key.get(identifier, self.api_key.get(alias) if alias is not None else None)
+         if key:
+             prefix = self.api_key_prefix.get(identifier)
+@@ -402,7 +405,7 @@ conf = client.Configuration(
+             basic_auth=username + ':' + password
+         ).get('authorization')
+
+-    def auth_settings(self):
++    async def auth_settings(self):
+         """Gets Auth Settings dict for api client.
+
+         :return: The Auth Settings information dict.
+@@ -413,7 +416,7 @@ conf = client.Configuration(
+                 'type': 'api_key',
+                 'in': 'header',
+                 'key': 'authorization',
+-                'value': self.get_api_key_with_prefix(
++                'value': await self.get_api_key_with_prefix(
+                     'BearerToken',
+                 ),
+             }
diff --git a/scripts/update-client.sh b/scripts/update-client.sh
index 1db50ed7..fa345c73 100755
--- a/scripts/update-client.sh
+++ b/scripts/update-client.sh
@@ -83,6 +83,9 @@ echo ">>> don't deep-copy configuration for local_vars_configuration in models"
 patch "${CLIENT_ROOT}/client/configuration.py" "${SCRIPT_ROOT}/client_configuration_get_default_patch.diff"
 find "${CLIENT_ROOT}/client/models/" -type f -print0 | xargs -0 sed -i 's/local_vars_configuration = Configuration.get_default_copy()/local_vars_configuration = Configuration.get_default()/g'
 
+echo ">>> fix generated api client and configuration for async token refreshing..."
+patch "${CLIENT_ROOT}/client/api_client.py" "${SCRIPT_ROOT}/api_client_async_refresh_api_key_hook.diff"
+patch "${CLIENT_ROOT}/client/configuration.py" "${SCRIPT_ROOT}/client_configuration_async_refresh_api_key_hook.diff"
 
 echo ">>> Remove invalid tests (workaround https://github.com/OpenAPITools/openapi-generator/issues/5377)"
 grep -r make_instance "${CLIENT_ROOT}/test/" | awk '{ gsub(":", ""); print $1}' | sort | uniq | xargs rm


Test patch:
diff --git a/kubernetes_asyncio/config/incluster_config_test.py b/kubernetes_asyncio/config/incluster_config_test.py
index f49a0b3d..614831b5 100644
--- a/kubernetes_asyncio/config/incluster_config_test.py
+++ b/kubernetes_asyncio/config/incluster_config_test.py
@@ -99,13 +99,13 @@ class InClusterConfigTest(unittest.IsolatedAsyncioTestCase):
         self.assertEqual(cert_filename, loader.ssl_ca_cert)
         self.assertEqual('Bearer ' + _TEST_TOKEN, loader.token)
 
-    def test_refresh_token(self):
+    async def test_refresh_token(self):
         loader = self.get_test_loader()
         config = Configuration()
         loader.load_and_set(config)
 
         self.assertEqual('Bearer ' + _TEST_TOKEN,
-                         config.get_api_key_with_prefix('BearerToken'))
+                         await config.get_api_key_with_prefix('BearerToken'))
         self.assertEqual('Bearer ' + _TEST_TOKEN, loader.token)
         self.assertIsNotNone(loader.token_expires_at)
 
@@ -113,15 +113,15 @@ class InClusterConfigTest(unittest.IsolatedAsyncioTestCase):
         loader._token_filename = self._create_file_with_temp_content(
             _TEST_NEW_TOKEN)
         self.assertEqual('Bearer ' + _TEST_TOKEN,
-                         config.get_api_key_with_prefix('BearerToken'))
+                         await config.get_api_key_with_prefix('BearerToken'))
 
         loader.token_expires_at = datetime.datetime.now()
         self.assertEqual('Bearer ' + _TEST_NEW_TOKEN,
-                         config.get_api_key_with_prefix('BearerToken'))
+                         await config.get_api_key_with_prefix('BearerToken'))
         self.assertEqual('Bearer ' + _TEST_NEW_TOKEN, loader.token)
         self.assertGreater(loader.token_expires_at, old_token_expires_at)
 
-    def test_refresh_token_default_config_with_copies(self):
+    async def test_refresh_token_default_config_with_copies(self):
         loader = self.get_test_loader()
         loader.load_and_set()
 
@@ -132,7 +132,7 @@ class InClusterConfigTest(unittest.IsolatedAsyncioTestCase):
 
         for config in configs:
             self.assertEqual('Bearer ' + _TEST_TOKEN,
-                             config.get_api_key_with_prefix('BearerToken'))
+                             await config.get_api_key_with_prefix('BearerToken'))
         self.assertEqual('Bearer ' + _TEST_TOKEN, loader.token)
         self.assertIsNotNone(loader.token_expires_at)
 
@@ -142,13 +142,13 @@ class InClusterConfigTest(unittest.IsolatedAsyncioTestCase):
 
         for config in configs:
             self.assertEqual('Bearer ' + _TEST_TOKEN,
-                             config.get_api_key_with_prefix('BearerToken'))
+                             await config.get_api_key_with_prefix('BearerToken'))
 
         loader.token_expires_at = datetime.datetime.now()
 
         for config in configs:
             self.assertEqual('Bearer ' + _TEST_NEW_TOKEN,
-                             config.get_api_key_with_prefix('BearerToken'))
+                             await config.get_api_key_with_prefix('BearerToken'))
 
         self.assertEqual('Bearer ' + _TEST_NEW_TOKEN, loader.token)
         self.assertGreater(loader.token_expires_at, old_token_expires_at)