Repository: mampfes/hacs_waste_collection_schedule

Issue:
[Bug]: westberks_gov_uk source not showing 2024 collection
### I Have A Problem With:

A specific source

### What's Your Problem

**Issue**: The westberks_gov_uk integration / date collections are not updating beyond the end of this year (2023). 

- I have tried testing with the westberks_gov_uk.md example attributes, and have tried UPRN as well as postcode/housenumberorname (as separate configs to rule out an attribute issue).
- Using UPRN attribute results in the error shown in the logs.
- Using postcode/housenumberorname returned recycling for the week of 24-30 Dec 2023 but not rubbish in the first week of Jan 2024.
- I have tried separating out sensors, to have a rubbish sensor and recycling sensor. The issue persists.

### Source (if relevant)

westberks_gov_uk

### Logs

```Shell
2023-12-29 20:22:43.890 ERROR (SyncWorker_2) [waste_collection_schedule.source_shell] fetch failed for source West Berkshire Council:
Traceback (most recent call last):
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 134, in fetch
    entries = self._source.fetch()
              ^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/westberks_gov_uk.py", line 101, in fetch
    if not rubbish_data["result"]["nextRubbishDateSubText"]:
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'nextRubbishDateSubText'
```


### Relevant Configuration

```YAML
#Waste collection
waste_collection_schedule:
  sources:
    - name: westberks_gov_uk
      args:
        postcode: "my_postcode"
        housenumberorname: "my_house_number"

#Waste collection sensor
sensor:
  - platform: waste_collection_schedule
    name: Waste collection
```


### Checklist Source Error

- [X] Use the example parameters for your source (often available in the documentation) (don't forget to restart Home Assistant after changing the configuration)
- [X] Checked that the website of your service provider is still working
- [X] Tested my attributes on the service provider website (if possible)
- [X] I have tested with the latest version of the integration (master) (for HACS in the 3 dot menu of the integration click on "Redownload" and choose master as version)

### Checklist Sensor Error

- [X] Checked in the Home Assistant Calendar tab if the event names match the types names (if types argument is used)

### Required

- [X] I have searched past (closed AND opened) issues to see if this bug has already been reported, and it hasn't been.
- [X] I understand that people give their precious time for free, and thus I've done my very best to make this problem as easy as possible to investigate.


Discussion:
@EPicLURcher - FYI as you're having the same issue with the same source, mentioned in #1590 

Resolving patch:
diff --git a/custom_components/waste_collection_schedule/waste_collection_schedule/source/tmbc_gov_uk.py b/custom_components/waste_collection_schedule/waste_collection_schedule/source/tmbc_gov_uk.py
--- a/custom_components/waste_collection_schedule/waste_collection_schedule/source/tmbc_gov_uk.py
+++ b/custom_components/waste_collection_schedule/waste_collection_schedule/source/tmbc_gov_uk.py
@@ -1,7 +1,9 @@
+from datetime import datetime
+
 import requests
 from bs4 import BeautifulSoup
 from dateutil import parser
-from waste_collection_schedule import Collection
+from waste_collection_schedule import Collection  # type: ignore[attr-defined]
 
 # mostly copied from braintree_gov_uk
 
@@ -9,8 +11,14 @@
 DESCRIPTION = "Tonbridge and Malling Borough Council, UK - Waste Collection"
 URL = "https://www.tmbc.gov.uk"
 TEST_CASES = {
-    "High Street, West Malling": {"address": "138 High Street", "post_code": "ME19 6NE"},
-    "Nutfields, Ightham, Sevenoaks": {"address": "5 Nutfields, Ightham, Sevenoaks", "post_code": "TN15 9EA"},
+    "High Street, West Malling": {
+        "address": "138 High Street",
+        "post_code": "ME19 6NE",
+    },
+    "Nutfields, Ightham, Sevenoaks": {
+        "address": "5 Nutfields, Ightham, Sevenoaks",
+        "post_code": "TN15 9EA",
+    },
 }
 
 ICON_MAP = {
@@ -40,34 +48,38 @@ def fetch(self):
         for address in BeautifulSoup(address_lookup.text, "html.parser").find_all(
             "option"
         ):
-            if not "..." in address["value"]:
+            if "..." not in address["value"]:
                 addresses[address["value"]] = address.text.strip()
         id = [
             address
             for address in addresses
-            if addresses[address].startswith(self.address)]
+            if addresses[address].startswith(self.address)
+        ]
         if len(id) == 0:
             raise Exception("Address not found")
         if len(id) > 1:
             raise Exception("Address is not unique")
         id = id[0]
-        
-        
-        self.form_data["q752eec300b2ffef2757e4536b77b07061842041a_1_0"] = (
-            None, id)
+
+        self.form_data["q752eec300b2ffef2757e4536b77b07061842041a_1_0"] = (None, id)
         self.form_data["next"] = (None, "Next")
         collection_lookup = requests.post(
             "https://www.tmbc.gov.uk/xfp/form/167", files=self.form_data
         )
         collection_lookup.raise_for_status()
         entries = []
-        for rows in BeautifulSoup(collection_lookup.text, "html.parser").find(
-            "table", class_="waste-collections-table"
-        ).find("tbody").find_all("tr"):
+        for rows in (
+            BeautifulSoup(collection_lookup.text, "html.parser")
+            .find("table", class_="waste-collections-table")
+            .find("tbody")
+            .find_all("tr")
+        ):
             date_td = rows.find_all("td")[0]
             bins_td = rows.find_all("td")[1]
 
             date = parser.parse(date_td.text.strip(), dayfirst=True).date()
+            if datetime.now().month == 12 and date.month in (1, 2):
+                date = date.replace(year=date.year + 1)
             for bin in bins_td.find("div", class_="collections").findAll("p"):
                 bin = bin.text.strip()
                 entries.append(
diff --git a/custom_components/waste_collection_schedule/waste_collection_schedule/source/westberks_gov_uk.py b/custom_components/waste_collection_schedule/waste_collection_schedule/source/westberks_gov_uk.py
--- a/custom_components/waste_collection_schedule/waste_collection_schedule/source/westberks_gov_uk.py
+++ b/custom_components/waste_collection_schedule/waste_collection_schedule/source/westberks_gov_uk.py
@@ -3,7 +3,7 @@
 from datetime import date, datetime
 
 import requests
-from waste_collection_schedule import Collection
+from waste_collection_schedule import Collection  # type: ignore[attr-defined]
 
 TITLE = "West Berkshire Council"
 DESCRIPTION = "Source for westberks.gov.uk services for West Berkshire Council"
@@ -32,6 +32,12 @@
 COLLECTIONS = {"Rubbish", "Recycling"}
 
 
+def fix_date(d: date):
+    if datetime.now().month == 12 and d.month in (1, 2):
+        d = d.replace(year=d.year + 1)
+    return d
+
+
 class Source:
     def __init__(
         self, uprn=None, postcode=None, housenumberorname=None
@@ -74,7 +80,6 @@ def fetch(self):
 
         # Get the collection days based on the UPRN (either supplied through arguments or searched for above)
         if self._uprn is not None:
-
             # POST request - one for each type as it is a separate method in the api
             rubbish_args = {
                 "jsonrpc": "2.0",
@@ -98,51 +103,53 @@ def fetch(self):
 
             # Extract dates from json
             waste_type = "Rubbish"
-            if not rubbish_data["result"]["nextRubbishDateSubText"]:
-                dt_str = (
-                    rubbish_data["result"]["nextRubbishDateText"]
-                    + " "
-                    + str(date.today().year)
-                )
-            else:
-                if len(rubbish_data["result"]["nextRubbishDateText"]) < 12:
+            if "nextRubbishDateSubText" in rubbish_data["result"]:
+                if not rubbish_data["result"]["nextRubbishDateSubText"]:
                     dt_str = (
-                        rubbish_data["result"]["nextRubbishDateSubText"]
+                        rubbish_data["result"]["nextRubbishDateText"]
                         + " "
                         + str(date.today().year)
                     )
-            dt_zulu = datetime.strptime(dt_str, "%A %d %B %Y")
-            dt_local = dt_zulu.astimezone(None)
-            entries.append(
-                Collection(
-                    date=dt_local.date(),
-                    t=waste_type,
-                    icon=ICON_MAP.get(waste_type.upper()),
+                else:
+                    if len(rubbish_data["result"]["nextRubbishDateText"]) < 12:
+                        dt_str = (
+                            rubbish_data["result"]["nextRubbishDateSubText"]
+                            + " "
+                            + str(date.today().year)
+                        )
+                dt_zulu = datetime.strptime(dt_str, "%A %d %B %Y")
+                dt_local = dt_zulu.astimezone(None)
+                entries.append(
+                    Collection(
+                        date=fix_date(dt_local.date()),
+                        t=waste_type,
+                        icon=ICON_MAP.get(waste_type.upper()),
+                    )
                 )
-            )
 
             waste_type = "Recycling"
-            if not recycling_data["result"]["nextRecyclingDateSubText"]:
-                dt_str = (
-                    recycling_data["result"]["nextRecyclingDateText"]
-                    + " "
-                    + str(date.today().year)
-                )
-            else:
-                if len(recycling_data["result"]["nextRecyclingDateText"]) < 12:
+            if "nextRecyclingDateSubText" in recycling_data["result"]:
+                if not recycling_data["result"]["nextRecyclingDateSubText"]:
                     dt_str = (
-                        recycling_data["result"]["nextRecyclingDateSubText"]
+                        recycling_data["result"]["nextRecyclingDateText"]
                         + " "
                         + str(date.today().year)
                     )
-            dt_zulu = datetime.strptime(dt_str, "%A %d %B %Y")
-            dt_local = dt_zulu.astimezone(None)
-            entries.append(
-                Collection(
-                    date=dt_local.date(),
-                    t=waste_type,
-                    icon=ICON_MAP.get(waste_type.upper()),
+                else:
+                    if len(recycling_data["result"]["nextRecyclingDateText"]) < 12:
+                        dt_str = (
+                            recycling_data["result"]["nextRecyclingDateSubText"]
+                            + " "
+                            + str(date.today().year)
+                        )
+                dt_zulu = datetime.strptime(dt_str, "%A %d %B %Y")
+                dt_local = dt_zulu.astimezone(None)
+                entries.append(
+                    Collection(
+                        date=fix_date(dt_local.date()),
+                        t=waste_type,
+                        icon=ICON_MAP.get(waste_type.upper()),
+                    )
                 )
-            )
 
         return entries


Test patch: