Repository: alltheplaces/alltheplaces

Issue:
Spider heb is broken
During the global build at 2021-08-18-14-42-26, spider **heb** failed with **320 features** and **1 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-08-18-14-42-26/logs/heb.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-08-18-14-42-26/output/heb.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-08-18-14-42-26/output/heb.geojson))


Discussion:
During the global build at 2021-09-01-14-42-16, spider **heb** failed with **320 features** and **1 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-09-01-14-42-16/logs/heb.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-09-01-14-42-16/output/heb.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-09-01-14-42-16/output/heb.geojson))

Resolving patch:
diff --git a/locations/spiders/heb.py b/locations/spiders/heb.py
--- a/locations/spiders/heb.py
+++ b/locations/spiders/heb.py
@@ -1,8 +1,10 @@
 # -*- coding: utf-8 -*-
+import json
+
 import scrapy
-import re
 
 from locations.items import GeojsonPointItem
+from locations.hours import OpeningHours
 
 
 class HEBSpider(scrapy.Spider):
@@ -23,18 +25,35 @@ def parse(self, response):
             yield scrapy.Request(url=url, callback=self.parse_store, meta={"url": url})
 
     def parse_store(self, response):
-        ref = "_".join(re.search(r".+/(.+?)/(.+?)/(.+?)/?(?:\.html|$)", response.url).groups())
-
-        properties = {
-            'name': response.xpath('//h1[@class="store-details__store-name"]/text()').extract_first(),
-            'ref': ref,
-            'addr_full': response.xpath('//p[@itemprop="streetAddress"]/text()').extract_first(),
-            'city': response.xpath('//div[@class="store-details__location"]/p[2]/span[1]/text()').extract_first(),
-            'state': response.xpath('//div[@class="store-details__location"]/p[2]/span[2]/text()').extract_first(),
-            'postcode': response.xpath('//div[@class="store-details__location"]/p[2]/span[3]/text()').extract_first(),
-            'phone': response.xpath('//a[@class="store-details__link store-details__link--phone"]/@content/text()').extract_first(),
-            'lat': (response.xpath('//div[@id="map-wrap"]/@data-map-lat').extract_first()),
-            'lon': (response.xpath('//div[@id="map-wrap"]/@data-map-lon').extract_first()),
-            'website': response.url
-        }
-        yield GeojsonPointItem(**properties)
+        if response.request.meta.get('redirect_urls'):
+            return
+
+        store_json = json.loads(
+            response.xpath('//script[@type="application/ld+json"]/text()').extract_first()
+        )
+        yield GeojsonPointItem(
+            ref=response.url.split('/')[-1],
+            name=store_json['name'],
+            lat=float(store_json['geo']['latitude']),
+            lon=float(store_json['geo']['longitude']),
+            addr_full=store_json['address']['streetAddress'],
+            city=store_json['address']['addressLocality'],
+            state=store_json['address']['addressRegion'],
+            postcode=store_json['address']['postalCode'],
+            country=store_json['address']['addressCountry'],
+            phone=store_json['telephone'],
+            website=response.url,
+            opening_hours=self.parse_hours(store_json['openingHoursSpecification'])
+        )
+
+    def parse_hours(self, hours):
+        opening_hours = OpeningHours()
+
+        for hour in hours:
+            opening_hours.add_range(
+                day=hour["dayOfWeek"][0:2].capitalize(),
+                open_time=hour["opens"],
+                close_time=hour["closes"]
+            )
+
+        return opening_hours.as_opening_hours()


Test patch: