Repository: mampfes/hacs_waste_collection_schedule

Issue:
AW Harburg throws
With 1.28.0 fetching the calendar for aw_harburg_de stopped working.
Last working version probably was 1.26.x +- ... (skipped some updates)
config:```
```

waste_collection_schedule:
  sources:
    - name: aw_harburg_de
      args:
        level_1: "XXX"
        level_2: "YYY"
      customize:
        - type: Hausmüll 14-täglich
          alias: "waste"
          icon: mdi:trash-can
        - type: Gelber Sack
          alias: "recycle"
          icon: mdi:recycle
        - type: Altpapier
          alias: "paper"
          icon: mdi:trash-can-outline
  fetch_time: "04:00"
  day_switch_time: "10:00"
```
level_1 + 2 are set to real levels according to website and were not changed
Pls find log attached
```

Logger: waste_collection_schedule.scraper
Source: custom_components/waste_collection_schedule/waste_collection_schedule/scraper.py:143
Integration: waste_collection_schedule ([documentation](https://github.com/mampfes/hacs_waste_collection_schedule#readme))
First occurred: 17:00:41 (1 occurrences)
Last logged: 17:00:41
fetch failed for source AW Harburg: Traceback (most recent call last): File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/scraper.py", line 141, in fetch entries = self._source.fetch() File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/aw_harburg_de.py", line 108, in fetch dates = self._ics.convert(r.text) File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/service/ICS.py", line 38, in convert events: List[Any] = icalevents.events( File "/usr/local/lib/python3.10/site-packages/icalevents/icalevents.py", line 50, in events found_events += parse_events(content, start=start, end=end) File "/usr/local/lib/python3.10/site-packages/icalevents/icalparser.py", line 250, in parse_events raise ValueError('Content is invalid!') ValueError: Content is invalid!
```


Discussion:
edit: Similar to issue #417 and pr #418. Might add the same quick fix, or maybe find a general solution

This code finds all links on the website:
```python
soup = BeautifulSoup(r.text, features="html.parser")
links = soup.find_all("a")
ical_url = ""
for any_link in links:
    if " als iCal" in any_link.text:
           ical_url = any_link.get("href")
```

There are 2 Links on the Website one for 2022 and one for 2023, but the 2023 ical file is empty. 

The source will work again as soon the dates for 2023 are updated.

Until then both links should probably be fetched and the output ics files concatenated. I don't know the code enough to implement this though.

Resolving patch:
diff --git a/custom_components/waste_collection_schedule/waste_collection_schedule/source/aw_harburg_de.py b/custom_components/waste_collection_schedule/waste_collection_schedule/source/aw_harburg_de.py
--- a/custom_components/waste_collection_schedule/waste_collection_schedule/source/aw_harburg_de.py
+++ b/custom_components/waste_collection_schedule/waste_collection_schedule/source/aw_harburg_de.py
@@ -91,25 +91,26 @@ def fetch(self):
 
         soup = BeautifulSoup(r.text, features="html.parser")
         links = soup.find_all("a")
-        ical_url = ""
+        ical_urls = []
         for any_link in links:
             if " als iCal" in any_link.text:
-                ical_url = any_link.get("href")
-
-        if "ical.html" not in ical_url:
-            raise Exception("No ical Link in the result: " + str(links))
-
-        # Get the final data
-        r = requests.get(ical_url, headers=HEADERS)
-        if not r.ok:
-            raise Exception(f"Error: failed to fetch url: {ical_url}")
-
-        # Parse ics file
-        dates = self._ics.convert(r.text)
+                # multiple links occur during year transition
+                ical_urls.append(any_link.get("href"))
 
+        # Get the final data for all links
         entries = []
-        for d in dates:
-            entries.append(Collection(d[0], d[1]))
+        for ical_url in ical_urls:
+            r = requests.get(ical_url, headers=HEADERS)
+            r.raise_for_status()
+
+            # Parse ics file
+            try:
+                dates = self._ics.convert(r.text)
+
+                for d in dates:
+                    entries.append(Collection(d[0], d[1]))
+            except ValueError:
+                pass    # during year transition the ical for the next year may be empty
         return entries
 
     def parse_level(self, response, level):


Test patch: