Repository: mampfes/hacs_waste_collection_schedule

Issue:
[Bug]: rctcbc_gov_uk returning extra incorrect dates 
### I Have A Problem With:

A specific source

### What's Your Problem

I've been using this integration for a while without issue.
I've been away for a few weeks, and come home to find 3 entries for each entry for each week, when there should only be one.

See attached screenshot.
The Wednesday entries are the only valid ones. Not sure why they're being duplicated though. :-/

<img width="1184" alt="Screenshot 2024-03-28 at 03 44 55" src="https://github.com/mampfes/hacs_waste_collection_schedule/assets/63560223/7cecf6a6-9ee5-42cb-875f-50d91877ceeb">


### Source (if relevant)

rctcbc_gov_uk

### Logs

```Shell
no relevant logs
```


### Relevant Configuration

```YAML
waste_collection_schedule:
  sources:
    - name: rctcbc_gov_uk
      args:
        uprn: "200003766278"
      calendar_title: "Bin Collection"
```


### Checklist Source Error

- [X] Use the example parameters for your source (often available in the documentation) (don't forget to restart Home Assistant after changing the configuration)
- [X] Checked that the website of your service provider is still working
- [X] Tested my attributes on the service provider website (if possible)
- [X] I have tested with the latest version of the integration (master) (for HACS in the 3 dot menu of the integration click on "Redownload" and choose master as version)

### Checklist Sensor Error

- [X] Checked in the Home Assistant Calendar tab if the event names match the types names (if types argument is used)

### Required

- [X] I have searched past (closed AND opened) issues to see if this bug has already been reported, and it hasn't been.
- [X] I understand that people give their precious time for free, and thus I've done my very best to make this problem as easy as possible to investigate.


Discussion:
I am having the same issue. Going to the council website and the calendar looks ok:
![image](https://github.com/mampfes/hacs_waste_collection_schedule/assets/8913897/d58527c4-9e1b-4db4-a06e-e9bcee31591b)

https://www.rctcbc.gov.uk/EN/Resident/RecyclingandWaste/RecyclingandWasteCollectionDays.aspx?uprn=<xxxx>&month=0

Resolving patch:
diff --git a/custom_components/waste_collection_schedule/waste_collection_schedule/source/rctcbc_gov_uk.py b/custom_components/waste_collection_schedule/waste_collection_schedule/source/rctcbc_gov_uk.py
--- a/custom_components/waste_collection_schedule/waste_collection_schedule/source/rctcbc_gov_uk.py
+++ b/custom_components/waste_collection_schedule/waste_collection_schedule/source/rctcbc_gov_uk.py
@@ -1,8 +1,8 @@
 from datetime import datetime
 
 import requests
-from bs4 import BeautifulSoup
-from waste_collection_schedule import Collection
+from bs4 import BeautifulSoup, Tag
+from waste_collection_schedule import Collection  # type: ignore[attr-defined]
 
 TITLE = "Rhondda Cynon Taf County Borough Council"
 DESCRIPTION = "Source for rctcbc.gov.uk services for Rhondda Cynon Taf County Borough Council, Wales, UK"
@@ -24,34 +24,73 @@ class Source:
     def __init__(self, uprn):
         self._uprn = str(uprn)
 
-    def fetch(self):
+    def extract_collections(self, calendar: Tag | BeautifulSoup) -> list[Collection]:
+        calendar_month = calendar.find("div", {"class": "calendar-month"})
+        if not calendar_month or not isinstance(calendar_month, Tag):
+            return []
+        month = calendar_month.text.strip()
+        calendar_days = calendar.find_all(
+            "div", {"class": "card-body card-body-padding"}
+        )
+
+        entries = []
+        for day in calendar_days:
+            pickups = day.find_all("a")
+            if len(pickups) != 0:
+                d = day.find("div", {"class": "card-title"})
+                if not d or not isinstance(d, Tag):
+                    continue
+                dt = d.text.strip() + " " + month
+                for pickup in pickups:
+                    entries.append(
+                        Collection(
+                            date=datetime.strptime(
+                                dt,
+                                "%d %B %Y",
+                            ).date(),
+                            t=pickup.text,
+                            icon=ICON_MAP.get(pickup.text.upper()),
+                        )
+                    )
+        return entries
+
+    def extract_from_printable_calendar(
+        self, soup: BeautifulSoup
+    ) -> list[Collection] | None:
+        entries = []
+        printable_calendar = soup.find("div", {"class": "printableCalendar"})
+        if not printable_calendar or not isinstance(printable_calendar, Tag):
+            return None
+
+        calendars = printable_calendar.find_all(
+            "div", {"class": "calendar-wrap onlyPrint"}
+        )
+        if not calendars:
+            return None
+
+        for calendar in calendars:
+            if not calendar or not isinstance(calendar, Tag):
+                continue
+            entries += self.extract_collections(calendar)
+        return entries or None
+
+    def fetch(self) -> list[Collection]:
         s = requests.Session()
         # website appears to display ~4 months worth of collections, so iterate through those pages
-        entries = []
+        entries: list[Collection] = []
         for month in range(0, 4):
             r = s.get(
                 f"https://www.rctcbc.gov.uk/EN/Resident/RecyclingandWaste/RecyclingandWasteCollectionDays.aspx?uprn={self._uprn}&month={month}"
             )
             soup = BeautifulSoup(r.text, "html.parser")
-            calendar_month = soup.find("div", {"class": "calendar-month"})
-            calendar_day = soup.find_all(
-                "div", {"class": "card-body card-body-padding"}
-            )
-            for day in calendar_day:
-                pickups = day.find_all("a")
-                if len(pickups) != 0:
-                    d = day.find("div", {"class": "card-title"})
-                    dt = d.text.strip() + " " + calendar_month.text.strip()
-                    for pickup in pickups:
-                        entries.append(
-                            Collection(
-                                date=datetime.strptime(
-                                    dt,
-                                    "%d %B %Y",
-                                ).date(),
-                                t=pickup.text,
-                                icon=ICON_MAP.get(pickup.text.upper()),
-                            )
-                        )
+            printable_calendar_entries = self.extract_from_printable_calendar(soup)
+            if printable_calendar_entries:
+                return printable_calendar_entries
+
+            # OLD METHOD IF THEY EVER REMOVE THE PRINTABLE CALENDAR AGAIN:
+            calendar = soup.find("div", {"class": "monthlyCalendar"}) or soup
+            if not isinstance(calendar, Tag):
+                continue
+            entries += self.extract_collections(calendar)
 
         return entries


Test patch: