Repository: mampfes/hacs_waste_collection_schedule

Issue:
[Bug]: Chichester District Council is not working
### I Have A Problem With:

A specific source

### What's Your Problem

The source has stopped working since Tuesday 13th February 2024. All the collection days no longer show on the calendar at all. The Chichester District Council website still shows me the days.

### Source (if relevant)

chichester_gov_uk

### Logs

```Shell
This error originated from a custom integration.

Logger: waste_collection_schedule.source_shell
Source: custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py:136
Integration: waste_collection_schedule (documentation)
First occurred: 11:36:47 (1 occurrences)
Last logged: 11:36:47

fetch failed for source Chichester District Council: Traceback (most recent call last): File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py", line 134, in fetch entries = self._source.fetch() ^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/waste_collection_schedule/waste_collection_schedule/source/chichester_gov_uk.py", line 37, in fetch form_url = form["action"] ~~~~^^^^^^^^^^ TypeError: 'NoneType' object is not subscriptable
```


### Relevant Configuration

```YAML
waste_collection_schedule:
  sources:
    - name: chichester_gov_uk
      args:
        uprn: 10002466648
```


### Checklist Source Error

- [X] Use the example parameters for your source (often available in the documentation) (don't forget to restart Home Assistant after changing the configuration)
- [X] Checked that the website of your service provider is still working
- [X] Tested my attributes on the service provider website (if possible)
- [X] I have tested with the latest version of the integration (master) (for HACS in the 3 dot menu of the integration click on "Redownload" and choose master as version)

### Checklist Sensor Error

- [X] Checked in the Home Assistant Calendar tab if the event names match the types names (if types argument is used)

### Required

- [X] I have searched past (closed AND opened) issues to see if this bug has already been reported, and it hasn't been.
- [X] I understand that people give their precious time for free, and thus I've done my very best to make this problem as easy as possible to investigate.


Discussion:

Resolving patch:
diff --git a/custom_components/waste_collection_schedule/waste_collection_schedule/source/chichester_gov_uk.py b/custom_components/waste_collection_schedule/waste_collection_schedule/source/chichester_gov_uk.py
--- a/custom_components/waste_collection_schedule/waste_collection_schedule/source/chichester_gov_uk.py
+++ b/custom_components/waste_collection_schedule/waste_collection_schedule/source/chichester_gov_uk.py
@@ -33,13 +33,13 @@ def fetch(self):
         soup = BeautifulSoup(r.text, features="html.parser")
 
         # Extract form submission url
-        form = soup.find("form", attrs={"id": "WASTECOLLECTIONCALENDARV2_FORM"})
+        form = soup.find("form", attrs={"id": "WASTECOLLECTIONCALENDARV5_FORM"})
         form_url = form["action"]
 
         # Submit form
         form_data = {
-            "WASTECOLLECTIONCALENDARV2_FORMACTION_NEXT": "Submit",
-            "WASTECOLLECTIONCALENDARV2_CALENDAR_UPRN": self._uprn,
+            "WASTECOLLECTIONCALENDARV5_FORMACTION_NEXT": "Submit",
+            "WASTECOLLECTIONCALENDARV5_CALENDAR_UPRN": self._uprn,
         }
         r = session.post(form_url, data=form_data)
         r.raise_for_status()
@@ -47,16 +47,18 @@ def fetch(self):
         # Extract collection dates
         soup = BeautifulSoup(r.text, features="html.parser")
         entries = []
-        data = soup.find_all("div", attrs={"class": "bin-days"})
-        for bin in data:
-            if "print-only" in bin["class"]:
-                continue
-
-            type = bin.find("span").contents[0].replace("bin", "").strip().title()
-            list_items = bin.find_all("li")
-            if list_items:
-                for item in list_items:
-                    date = datetime.strptime(item.text, "%d %B %Y").date()
+        tables = soup.find_all("table", attrs={"class": "bin-collection-dates"})
+        # Data is presented in two tables side-by-side
+        for table in tables:
+            # Each collection is a table row
+            data = table.find_all("tr")
+            for bin in data:
+                cells = bin.find_all("td")
+                # Ignore the header row
+                if len(cells) == 2:
+                    date = datetime.strptime(cells[0].text, "%d %B %Y").date()
+                    # Maintain backwards compatibility - it used to be General Waste and now it is General waste
+                    type = cells[1].text.title()
                     entries.append(
                         Collection(
                             date=date,


Test patch: