Repository: alltheplaces/alltheplaces

Issue:
@spider=zabka_pl opening hours are missing Sunday data, proposed `opening_hours` are broken
https://www.alltheplaces.xyz/map/#16.47/50.073227/20.037421

![screen](https://github.com/alltheplaces/alltheplaces/assets/899988/a19656c7-65ad-4db2-8907-b5e0f2186f26)

https://www.openstreetmap.org/node/4271289403

Opening hours in OSM (`Mo-Sa 06:00-22:00; Su 11:00-20:00`) are correct.

At https://www.zabka.pl/znajdz-sklep it refuses to show Sunday opening hours.

Maybe it is caused by their internal structure? This shops are franchises and maybe they are obligated to be open `Mo-Sa 06:00-22:00` and Sundays can be decided by operator?

Overall it seems that Monday to Saturday data is likely correct.


Discussion:
> Maybe it is caused by their internal structure? This shops are franchises and maybe they are obligated to be open `Mo-Sa 06:00-22:00` and Sundays can be decided by operator?

https://github.com/alltheplaces/alltheplaces/blob/a0bc9ae0f99026e6235a4e71a1b70d83231c3f5c/locations/spiders/zabka_pl.py#L30-L35

https://github.com/alltheplaces/alltheplaces/pull/5586

Sounds like you and @alufers agree, and it sounds like at least some of them have Sunday hours available. It would be nice to collect them, especially when all the Sundays in `nonTradingDays` agree.

Not entirely sure what should be done?

Ideally parsing `nonTradingDays` would be nice, but not sure what should be done with shops missing it. Are they closed on Sundays? Missing data and their opening hours should be skipped?

For now I would propose to not emit any opening hours rather than known incomplete ones.
> For now I would propose to not emit any opening hours rather than known incomplete ones.

Interesting, we don't actually output `Su off`. But I acknowledge, in reality, OSM doesn't differentiate. We also don't output any PH data, anyone conflating with OSM should take that into account.

For this specific spider, currently 262 are missing (length 0) `nonTradingDays`, 9,365 have it. Ideally someone with knowledge of the chain could work out if the 262 are closed on Sunday, or missing data.

More than 9,000 currently have the same `nonTradingDays` opening hours, we can collect them. But what happens near Christmas? I imagine that number will be a lot lower, I wouldn't want you to update OSM then.

Resolving patch:
diff --git a/locations/spiders/zabka_pl.py b/locations/spiders/zabka_pl.py
--- a/locations/spiders/zabka_pl.py
+++ b/locations/spiders/zabka_pl.py
@@ -1,3 +1,5 @@
+from datetime import datetime
+
 import scrapy
 from scrapy.http import JsonRequest
 
@@ -20,6 +22,7 @@ def start_requests(self):
         yield JsonRequest(url=self.start_urls[0], headers=headers)
 
     def parse(self, response):
+        today = datetime.now()
         for location in response.json():
             item = DictParser.parse(location)
             item["street_address"] = item.pop("addr_full", "")
@@ -33,4 +36,25 @@ def parse(self, response):
             item["opening_hours"].add_days_range(
                 ["Mo", "Tu", "We", "Th", "Fr", "Sa"], location["openTime"], location["closeTime"]
             )
+
+            if location["nonTradingDays"]:
+                sunday_open = None
+                sunday_close = None
+                for rule in location["nonTradingDays"]:
+                    d = datetime.strptime(rule["date"], "%Y-%m-%d")
+                    if d.weekday() != 6 or d < today:
+                        continue  # In the past, ignore
+                    if sunday_open is None:
+                        sunday_open = rule["openTime"]
+                        sunday_close = rule["closeTime"]
+                    else:
+                        if sunday_open != rule["openTime"] or sunday_close != rule["closeTime"]:
+                            self.crawler.stats.inc_value("atp/zabka_pl/nonTradingDays/mismatching")
+                            break  # Mismatching future Sundays, skip
+                else:
+                    self.crawler.stats.inc_value("atp/zabka_pl/nonTradingDays/fine")
+                    item["opening_hours"].add_range("Su", sunday_open, sunday_close)
+            else:
+                self.crawler.stats.inc_value("atp/zabka_pl/nonTradingDays/missing")  # Sunday closed? Missing data?
+
             yield item


Test patch: