Repository: bimmerconnected/bimmer_connected (python)

Issue:
Connection Issue: Authentication failed
### Describe the issue

Connection is no longer possible.
MyBMW-App is still working, password has not been changed. Last successful update was about 24h ago, error also occured before this last update

```DEBUG:bimmer_connected.account:Getting vehicle list
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): cocoapi.bmwgroup.com:443
DEBUG:urllib3.connectionpool:https://cocoapi.bmwgroup.com:443 "GET /eadrax-ucs/v1/presentation/oauth/config HTTP/1.1" 200 580
DEBUG:bimmer_connected.account:Authenticating against GCDM with MyBMW flow.
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): customer.bmwgroup.com:443
DEBUG:urllib3.connectionpool:https://customer.bmwgroup.com:443 "POST /gcdm/oauth/authenticate HTTP/1.1" 401 190
ERROR:bimmer_connected.account:Authentication failed (invalid_client): Client authentication failed (e.g., login failure, unknown client, no client authentication included or unsupported authentication method)
ERROR:bimmer_connected.account:401 Client Error: Unauthorized for url: https://customer.bmwgroup.com/gcdm/oauth/authenticate
Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/account.py", line 94, in _get_oauth_token
    token_data = self._login_row_na()
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/account.py", line 220, in _login_row_na
    raise ex
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/account.py", line 170, in _login_row_na
    response.raise_for_status()
  File "/usr/local/lib/python3.7/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://customer.bmwgroup.com/gcdm/oauth/authenticate
Traceback (most recent call last):
  File "/home/pi/.local/bin/bimmerconnected", line 10, in <module>
    sys.exit(main())
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/cli.py", line 282, in main
    args.func(args)
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/cli.py", line 127, in fingerprint
    log_responses=time_dir)
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/account.py", line 83, in __init__
    self._get_vehicles()
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/account.py", line 376, in _get_vehicles
    self._get_oauth_token()
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/account.py", line 108, in _get_oauth_token
    raise ex
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/account.py", line 94, in _get_oauth_token
    token_data = self._login_row_na()
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/account.py", line 220, in _login_row_na
    raise ex
  File "/home/pi/.local/lib/python3.7/site-packages/bimmer_connected/account.py", line 170, in _login_row_na
    response.raise_for_status()
  File "/usr/local/lib/python3.7/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://customer.bmwgroup.com/gcdm/oauth/authenticate
```

### Expected behavior

Authentication is working, data is fetched correctly

### Which Home Assistant version are you using?

None

### What was the last working version of Home Assistant Core?

_No response_

### What is your region?

Rest of world

### ConnectedDrive website

- [ ] I can still successfully login to the BMW ConnectedDrive website and the car status is available there.
- [ ] I have ConnectedDrive enabled for my vehicle.

### Number of cars

- [ ] I have 2 or more cars linked to the ConnectedDrive account.
- [ ] I have a Mini vehicle linked to my account.

### Output of bimmer_connected fingerprint

_No response_

### Anything in the logs that might be useful for us?

_No response_

### Additional information

_No response_

Pull request:
Fix login unauthorized
## Proposed change
<!--
  Describe the big picture of your changes here to communicate to the
  maintainers why we should accept this pull request. If it fixes a bug
  or resolves a feature request, be sure to link to that issue in the
  additional information section.
-->
This PR includes various changes to improve the flaky login/unauthorized situation:
* Add `-10s` offset when storing token expiry to mitigate timing issues on `/vehicles` endpoint
* Add retry strategy to `north_america`/`rest_of_world` login
  * Sometimes the `/authenticate` endpoint does not seem to work correctly, happend to me in the Android app as well

## Type of change
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (which adds functionality to this library)
- [ ] Breaking change (fix/feature causing existing functionality to break)
- [ ] Code quality improvements to existing code or addition of tests

## Additional information
<!--
  Details are important, and help maintainers processing your PR.
  Please be sure to fill out additional details, if applicable.
-->

- This PR fixes or closes issue: fixes #393
- This PR is related to issue: https://github.com/home-assistant/core/issues/63214

## Checklist
<!--
  Put an `x` in the boxes that apply. You can also fill these out after
  creating the PR. If you're unsure about any of them, don't hesitate to ask.
  We're here to help! This is simply a reminder of what we are going to look
  for before merging your code.
-->

- [ ] The code change is tested and works locally.
- [ ] Tests have been added to verify that the new code works.


Patch:
diff --git a/bimmer_connected/account.py b/bimmer_connected/account.py
index 2f30326..e24b468 100644
--- a/bimmer_connected/account.py
+++ b/bimmer_connected/account.py
@@ -10,10 +10,8 @@ This library is not affiliated with or endorsed by BMW Group.
 
 import base64
 import datetime
-import hashlib
 import json
 import logging
-import os
 import pathlib
 import urllib
 from threading import Lock
@@ -40,8 +38,15 @@ from bimmer_connected.country_selector import (
     get_ocp_apim_key,
     get_server_url
 )
+from bimmer_connected.utils import (
+    RetrySession,
+    create_s256_code_challenge,
+    generate_token
+)
 from bimmer_connected.vehicle import CarBrand, ConnectedDriveVehicle
 
+VALID_UNTIL_OFFSET = datetime.timedelta(seconds=10)
+
 _LOGGER = logging.getLogger(__name__)
 
 
@@ -111,7 +116,7 @@ class ConnectedDriveAccount:  # pylint: disable=too-many-instance-attributes
         """Login to Rest of World and North America."""
         try:
             # We need a session for cross-request cookies
-            oauth_session = requests.Session()
+            oauth_session = RetrySession(status_forcelist=[401], allowed_methods=frozenset(["GET", "POST"]))
             r_oauth_settings = oauth_session.get(
                 OAUTH_CONFIG_URL.format(server=self.server_url),
                 headers={
@@ -125,19 +130,19 @@ class ConnectedDriveAccount:  # pylint: disable=too-many-instance-attributes
             # My BMW login flow
             _LOGGER.debug("Authenticating against GCDM with MyBMW flow.")
 
-            # Setting up PKCS data
-            verifier_bytes = os.urandom(64)
-            code_verifier = base64.urlsafe_b64encode(verifier_bytes).rstrip(b'=')
-
-            challenge_bytes = hashlib.sha256(code_verifier).digest()
-            code_challenge = base64.urlsafe_b64encode(challenge_bytes).rstrip(b'=')
+            code_verifier = generate_token(86)
+            code_challenge = create_s256_code_challenge(code_verifier)
 
-            state_bytes = os.urandom(16)
-            state = base64.urlsafe_b64encode(state_bytes).rstrip(b'=')
+            state = generate_token(22)
 
             authenticate_url = AUTH_URL.format(gcdm_base_url=oauth_settings["gcdmBaseUrl"])
             authenticate_headers = {
                 "Content-Type": "application/x-www-form-urlencoded",
+                "Referer": "https://customer.bmwgroup.com/oneid/",
+                "User-Agent": (
+                    "Mozilla/5.0 (Linux; Android 7.1.2; One) AppleWebKit/537.36 (KHTML, like Gecko) "
+                    "Chrome/95.0.4638.74 Mobile Safari/537.36"
+                ),
             }
 
             # we really need all of these parameters
@@ -152,15 +157,13 @@ class ConnectedDriveAccount:  # pylint: disable=too-many-instance-attributes
                 "code_challenge_method": "S256",
             }
 
-            authenticate_data = urllib.parse.urlencode(
-                dict(
-                    oauth_base_values,
-                    **{
-                        "grant_type": "authorization_code",
-                        "username": self._username,
-                        "password": self._password,
-                    }
-                )
+            authenticate_data = dict(
+                oauth_base_values,
+                **{
+                    "grant_type": "authorization_code",
+                    "username": self._username,
+                    "password": self._password,
+                }
             )
             response = oauth_session.post(
                 authenticate_url,
@@ -205,7 +208,7 @@ class ConnectedDriveAccount:  # pylint: disable=too-many-instance-attributes
             response_json = response.json()
 
             expiration_time = int(response_json["expires_in"])
-            expires_at = current_utc_time + datetime.timedelta(seconds=expiration_time)
+            expires_at = current_utc_time + datetime.timedelta(seconds=expiration_time) - VALID_UNTIL_OFFSET
 
             return {
                 "access_token": response_json["access_token"],
@@ -253,7 +256,7 @@ class ConnectedDriveAccount:  # pylint: disable=too-many-instance-attributes
 
             return {
                 "access_token": response_json["access_token"],
-                "expires_at": datetime.datetime.utcfromtimestamp(decoded_token["exp"])
+                "expires_at": datetime.datetime.utcfromtimestamp(decoded_token["exp"]) - VALID_UNTIL_OFFSET
             }
         except HTTPError as ex:
             try:
@@ -265,7 +268,6 @@ class ConnectedDriveAccount:  # pylint: disable=too-many-instance-attributes
 
     def request_header(self, brand: CarBrand = None) -> Dict[str, str]:
         """Generate a header for HTTP requests to the server."""
-        self._get_oauth_token()
         brand = brand or CarBrand.BMW
         headers = {
             "accept": "application/json",
diff --git a/bimmer_connected/remote_services.py b/bimmer_connected/remote_services.py
index 99cbd51..954314b 100644
--- a/bimmer_connected/remote_services.py
+++ b/bimmer_connected/remote_services.py
@@ -48,6 +48,7 @@ class _Services(str, Enum):
     REMOTE_DOOR_UNLOCK = 'DOOR_UNLOCK'
     REMOTE_HORN = 'HORN_BLOW'
     REMOTE_AIR_CONDITIONING = 'CLIMATE_NOW'
+    REMOTE_CHARGE_NOW = 'CHARGE_NOW'
 
 
 class RemoteServiceStatus:  # pylint: disable=too-few-public-methods
@@ -134,6 +135,17 @@ class RemoteServices:
         event_id = self._trigger_remote_service(_Services.REMOTE_HORN)
         return self._block_until_done(_Services.REMOTE_HORN, event_id)
 
+    def trigger_charge_now(self) -> RemoteServiceStatus:
+        """Trigger the vehicle to start charging.
+
+        A state update is NOT triggered after this, as the vehicle state is unchanged.
+        """
+        _LOGGER.debug('Triggering charge now')
+        event_id = self._trigger_remote_service(_Services.REMOTE_CHARGE_NOW)
+        result = self._block_until_done(_Services.REMOTE_CHARGE_NOW, event_id)
+        self._trigger_state_update()
+        return result
+
     def trigger_remote_air_conditioning(self) -> RemoteServiceStatus:
         """Trigger the air conditioning to start.
 
diff --git a/bimmer_connected/utils.py b/bimmer_connected/utils.py
index d6ac681..f998e92 100644
--- a/bimmer_connected/utils.py
+++ b/bimmer_connected/utils.py
@@ -1,14 +1,62 @@
 """General utils and base classes used in the library."""
 
-from abc import ABC
+import base64
 import datetime
+import hashlib
 import inspect
 import json
 import logging
+import random
+import string
 import sys
+from abc import ABC
+from collections import OrderedDict
+
+import requests
+from requests.adapters import HTTPAdapter
+from urllib3.util.retry import Retry
 
 _LOGGER = logging.getLogger(__name__)
 
+UNICODE_CHARACTER_SET = string.ascii_letters + string.digits + '-._~'
+
+
+def generate_token(length=30, chars=UNICODE_CHARACTER_SET):
+    """Generate a random token with given length and characters."""
+    rand = random.SystemRandom()
+    return ''.join(rand.choice(chars) for _ in range(length))
+
+
+def create_s256_code_challenge(code_verifier):
+    """Create S256 code_challenge with the given code_verifier."""
+    data = hashlib.sha256(code_verifier.encode('ascii')).digest()
+    return base64.urlsafe_b64encode(data).rstrip(b'=').decode('UTF-8')
+
+
+class RetrySession(requests.Session):
+    """A `requests.Session` with `Retry`."""
+
+    def __init__(
+        self,
+        retries=3,
+        backoff_factor=0.3,
+        status_forcelist=None,
+        allowed_methods=None,
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+        retry = Retry(
+            total=retries,
+            read=retries,
+            connect=retries,
+            backoff_factor=backoff_factor,
+            status_forcelist=status_forcelist,
+            allowed_methods=allowed_methods,
+        )
+        self.adapters = OrderedDict()
+        self.mount('http://', HTTPAdapter(max_retries=retry))
+        self.mount('https://', HTTPAdapter(max_retries=retry))
+
 
 def serialize_for_json(obj: object, excluded: list = None, exclude_hidden: bool = True) -> dict:
     """


Test patch:
diff --git a/test/test_remote_services.py b/test/test_remote_services.py
index ffdba9b..e773f28 100644
--- a/test/test_remote_services.py
+++ b/test/test_remote_services.py
@@ -91,6 +91,7 @@ class TestRemoteServices(TestCase):
             ("CLIMATE_STOP", "trigger_remote_air_conditioning_stop", True),
             ("VEHICLE_FINDER", "trigger_remote_vehicle_finder", False),
             ("HORN_BLOW", "trigger_remote_horn", False),
+            ("CHARGE_NOW", "trigger_charge_now", True),
             ("SEND_POI", "trigger_send_poi", False),
         ]