Repository: ultrabug/py3status

Issue:
[weather_owm] API 2.5 is dying in June, 2024
**How can we help you today?**

Openweathermap.org sent out information stating that API 2.5 will be killed off in June, and to migrate to API 3.0. I'm not a coder (and never played one on TV), so I'm clueless as to going about changing the module to adapt to the 3.0 API.

**Your py3status version**

py3status version 3.57 (python 3.12.3) on sway

**Share your configuration**

Config: https://0x0.st/XX0o.txt

**Additional context**

I've gotten a 3.0 API key. This is the full output (in imperial units) of a 3.0 call for my location (no exclusions):

Full output: https://0x0.st/XX0X.output

I ran the output thru a JSON formatter:

JSON: https://0x0.st/XX08.json

Thanks in advance for any help!!!

ETA: I'm willing to provide an API 3.0 key (via email) to help with modifying the module.



Discussion:
>Please fill you billing information for subscription

This is unfortunate.

It looks like the users would need to start filling out personal / billing plan in order to subscribe / get new API key even although they might not want to.

The users need to know they can get charged if they made excess calls after the limit.

I ask @ultrabug what he want to do here. Possible removal just like https://github.com/ultrabug/py3status/commit/a29ac2ca16540561a79a8f88ae02bbabbcb42310.
What if it was relegated to a 'user' module (put in $XDG_CONFIG_HOME/py3status/modules)? That way, whoever wants to subscribe, can. If they don't, then no loss to them (other than it not working anymore).

I really hope I don't have to lose this...
/cc author @alexoneill
It's bank holiday week here, I'll have a look next week and will do my best to keep this module up
So it seems my current (pro) subscription does not include 3.0 API :(

I'll have a deeper look ... damn
Tested new endpoint, seems like the response payload is the same

```diff
diff --git a/py3status/modules/weather_owm.py b/py3status/modules/weather_owm.py
index 8aa47dce..2721c869 100644
--- a/py3status/modules/weather_owm.py
+++ b/py3status/modules/weather_owm.py
@@ -265,7 +265,7 @@ import datetime
 # API information
 OWM_CURR_ENDPOINT = "https://api.openweathermap.org/data/2.5/weather?"
 OWM_FUTURE_ENDPOINT = "https://api.openweathermap.org/data/2.5/forecast?"
-OWM_ONECALL_ENDPOINT = "https://api.openweathermap.org/data/2.5/onecall?"
+OWM_ONECALL_ENDPOINT = "https://api.openweathermap.org/data/3.0/onecall?exclude=alerts,minutely"
 IP_ENDPOINT = "http://geo.ultrabug.fr"
 
 # Paths of information to extract from JSON
```


@mlmatlock could you try using 
[weather_owm.py.gz](https://github.com/ultrabug/py3status/files/15337925/weather_owm.py.gz)

and confirm it works for you as intended?
If it works, the questioning will relate to the fact that people could be charged by using this module so I really wonder if we want to add some kind of counter to make sure that the module stays within the free limit of api calls.

Since users have to enter their credit card to use that endpoint, they should be aware enough anyway so on the principle it's not the module's business.

As of now, the cache timeout is 30min which makes the 1000 calls / day limit hard to reach, but... you never know.
> @mlmatlock could you try using [weather_owm.py.gz](https://github.com/ultrabug/py3status/files/15337925/weather_owm.py.gz)
> 
> and confirm it works for you as intended?

@ultrabug, it seems to be working!

Noticed one thing, though...it's working with my old (2+ years) API key. If I generate a new API key and put that one in the config, I get a 401 error. A little digging, and if I call the 2.5 API with the new key from a web browser, I get the same 401 error:

`{"cod":401, "message": "Invalid API key. Please see https://openweathermap.org/faq#error401 for more info."}`

Calling the 2.5 API with the old key works, old key also calls the 3.0 API with no problems.

Seems new API keys are forcing the depreciation, and old keys might stop working when they actually turn the 2.5 API off.

As always, I appreciate the help!
Some thoughts.

>As of now, the cache timeout is 30min which makes the 1000 calls / day limit hard to reach, but... you never know.

1) If users doesn't like this, they'll change it to every 5 minutes or such... Suggestion... Add `self.cache_timeout = max(600, self.cache_timeout)` too as a preventive measure. The API said it updates every 10 minutes anyway.

>to add some kind of counter to make sure that the module stays within the free limit of api calls.

2) Users refreshing/restarting i3 frequently..... is a possible occurrence. Storing cache between i3-msg restarts if it is under `max(600, self.cache_timeout)` would be nice to have too.

3) Option to shutdown on 999 calls?

This is given. Clean up code. Get rid of 2.5 and get everything from 3.0 will force users to refresh API key and they'll learn about this new billing subscription. No issue there. Do document the paid plan for existing/new users anyway.

I think `1)` and `2)` matters. `3)` is overkill.

4) Onecall 3.0 likely expose new placeholders. See if we should make `format_*` for some of them. Looks like we may be able to customize `exclude` request based on placeholders to make calls smaller too. Idk.
Thanks @lasers indeed I considered some of those options. I think even if it requires more work, spending time on `3` would be more future proof.

I implemented 1/ and 2/ anyway

```diff
diff --git a/py3status/modules/weather_owm.py b/py3status/modules/weather_owm.py
index 8aa47dce..54c80669 100644
--- a/py3status/modules/weather_owm.py
+++ b/py3status/modules/weather_owm.py
@@ -261,11 +261,12 @@ diff
 """
 
 import datetime
+import json
 
 # API information
 OWM_CURR_ENDPOINT = "https://api.openweathermap.org/data/2.5/weather?"
 OWM_FUTURE_ENDPOINT = "https://api.openweathermap.org/data/2.5/forecast?"
-OWM_ONECALL_ENDPOINT = "https://api.openweathermap.org/data/2.5/onecall?"
+OWM_ONECALL_ENDPOINT = "https://api.openweathermap.org/data/3.0/onecall?exclude=alerts,minutely"
 IP_ENDPOINT = "http://geo.ultrabug.fr"
 
 # Paths of information to extract from JSON
@@ -448,6 +449,18 @@ class Py3status:
         # Generate our icon array
         self.icons = self._get_icons()
 
+        # Implement safe-to-reload rate limit
+        cached_hour = datetime.datetime.now(datetime.UTC).strftime("%H")
+        self.cached_hits = json.loads(
+            self.py3.storage_get("cached_hits") or json.dumps({cached_hour: 0})
+        )
+        self.cached_onecall_response = self.py3.storage_get("cached_onecall_response")
+
+        # We want to make sure users to not exceed the request limit
+        # to 3.0 API and get billed while taking into account that
+        # OWM does refresh its API data every 10min anyway.
+        self.cache_timeout = max(600, self.cache_timeout)
+
         # Verify the units configuration
         if self.unit_rain.lower() not in RAIN_UNITS:
             raise Exception("unit_rain is not recognized")
@@ -817,6 +830,9 @@ class Py3status:
         return self.py3.safe_format(self.format, today)
 
     def weather_owm(self):
+        # Prepare rate limit cache hour
+        cached_hour = datetime.datetime.now(datetime.UTC).strftime("%H")
+        cached_hits = self.cached_hits.get(cached_hour, 0)
         # Get weather information
         loc_tz_info = self._get_loc_tz_info()
         text = ""
@@ -840,9 +856,19 @@ class Py3status:
             except Exception:
                 raise Exception("no latitude/longitude found for your config")
 
-            # onecall = forecasts
-            onecall_api_params = {"lat": lat, "lon": lon}
-            onecall = self._get_onecall(onecall_api_params)
+            # onecall = forecasts rate limited
+            if cached_hits < 999:
+                onecall_api_params = {"lat": lat, "lon": lon}
+                onecall = self._get_onecall(onecall_api_params)
+                # update and store caches
+                self.cached_onecall_response = onecall
+                self.cached_hits[cached_hour] = cached_hits + 1
+                self.py3.storage_set("cached_onecall_response", onecall)
+                self.py3.storage_set(
+                    "cached_hits", json.dumps({cached_hour: self.cached_hits[cached_hour]})
+                )
+            else:
+                onecall = self.cached_onecall_response
             onecall_daily = onecall["daily"]
 
             fcsts_days = self.forecast_days + 1
```

Resolving patch:
diff --git a/py3status/modules/weather_owm.py b/py3status/modules/weather_owm.py
--- a/py3status/modules/weather_owm.py
+++ b/py3status/modules/weather_owm.py
@@ -8,11 +8,11 @@
 
 This module uses OpenWeatherMap API (https://openweathermap.org).
 
-setting `location` or `city` allows you to specify the location for the weather
-you want displaying.
+Requires a 3.0 API key for OpenWeatherMap (OWM) with a subscription which this
+module will try as hard as it can to stay under the free tier limit.
 
-Requires an API key for OpenWeatherMap (OWM), but the free tier allows you
-enough requests/sec to get accurate weather even up to the minute.
+Setting `location` or `city` allows you to specify the location for the weather
+you want displaying.
 
 I would highly suggest you install an additional font, such as the incredible
 (and free!) Weather Icons font (https://erikflowers.github.io/weather-icons),
@@ -261,11 +261,12 @@
 """
 
 import datetime
+import json
 
 # API information
 OWM_CURR_ENDPOINT = "https://api.openweathermap.org/data/2.5/weather?"
 OWM_FUTURE_ENDPOINT = "https://api.openweathermap.org/data/2.5/forecast?"
-OWM_ONECALL_ENDPOINT = "https://api.openweathermap.org/data/2.5/onecall?"
+OWM_ONECALL_ENDPOINT = "https://api.openweathermap.org/data/3.0/onecall?exclude=alerts,minutely"
 IP_ENDPOINT = "http://geo.ultrabug.fr"
 
 # Paths of information to extract from JSON
@@ -448,6 +449,18 @@ def post_config_hook(self):
         # Generate our icon array
         self.icons = self._get_icons()
 
+        # Implement safe-to-reload rate limit
+        cached_day = datetime.datetime.now(datetime.UTC).strftime("%Y%m%d")
+        self.cached_hits = json.loads(
+            self.py3.storage_get("cached_hits") or json.dumps({cached_day: 0})
+        )
+        self.cached_onecall_response = self.py3.storage_get("cached_onecall_response")
+
+        # We want to make sure users to not exceed the request limit
+        # to 3.0 API and get billed while taking into account that
+        # OWM does refresh its API data every 10min anyway.
+        self.cache_timeout = max(600, self.cache_timeout)
+
         # Verify the units configuration
         if self.unit_rain.lower() not in RAIN_UNITS:
             raise Exception("unit_rain is not recognized")
@@ -817,6 +830,9 @@ def _format(self, current_wthr, fcsts, city, country):
         return self.py3.safe_format(self.format, today)
 
     def weather_owm(self):
+        # Prepare rate limit cache
+        cached_day = datetime.datetime.now(datetime.UTC).strftime("%Y%m%d")
+        cached_hits = self.cached_hits.get(cached_day, 0)
         # Get weather information
         loc_tz_info = self._get_loc_tz_info()
         text = ""
@@ -840,9 +856,19 @@ def weather_owm(self):
             except Exception:
                 raise Exception("no latitude/longitude found for your config")
 
-            # onecall = forecasts
-            onecall_api_params = {"lat": lat, "lon": lon}
-            onecall = self._get_onecall(onecall_api_params)
+            # onecall = forecasts rate limited
+            if cached_hits < 999:
+                onecall_api_params = {"lat": lat, "lon": lon}
+                onecall = self._get_onecall(onecall_api_params)
+                # update and store caches
+                self.cached_onecall_response = onecall
+                self.cached_hits[cached_day] = cached_hits + 1
+                self.py3.storage_set("cached_onecall_response", onecall)
+                self.py3.storage_set(
+                    "cached_hits", json.dumps({cached_day: self.cached_hits[cached_day]})
+                )
+            else:
+                onecall = self.cached_onecall_response
             onecall_daily = onecall["daily"]
 
             fcsts_days = self.forecast_days + 1


Test patch: