Repository: alltheplaces/alltheplaces

Issue:
Spider afcurgentcare is broken
During the global build at 2021-05-21-20-28-08, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-05-21-20-28-08/logs/afcurgentcare.log) and [the output](https://data.alltheplaces.xyz/runs/2021-05-21-20-28-08/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-05-21-20-28-08/output/afcurgentcare.geojson))


Discussion:
During the global build at 2021-05-21-20-39-25, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-05-21-20-39-25/logs/afcurgentcare.log) and [the output](https://data.alltheplaces.xyz/runs/2021-05-21-20-39-25/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-05-21-20-39-25/output/afcurgentcare.geojson))
During the global build at 2021-05-24-14-04-37, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-05-24-14-04-37/logs/afcurgentcare.log) and [the output](https://data.alltheplaces.xyz/runs/2021-05-24-14-04-37/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-05-24-14-04-37/output/afcurgentcare.geojson))
During the global build at 2021-05-26-14-42-23, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-05-26-14-42-23/logs/afcurgentcare.log) and [the output](https://data.alltheplaces.xyz/runs/2021-05-26-14-42-23/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-05-26-14-42-23/output/afcurgentcare.geojson))
During the global build at 2021-06-02-14-42-40, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-06-02-14-42-40/logs/afcurgentcare.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-06-02-14-42-40/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-06-02-14-42-40/output/afcurgentcare.geojson))
During the global build at 2021-06-09-14-42-21, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-06-09-14-42-21/logs/afcurgentcare.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-06-09-14-42-21/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-06-09-14-42-21/output/afcurgentcare.geojson))
During the global build at 2021-06-16-14-42-20, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-06-16-14-42-20/logs/afcurgentcare.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-06-16-14-42-20/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-06-16-14-42-20/output/afcurgentcare.geojson))
During the global build at 2021-06-23-14-42-18, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-06-23-14-42-18/logs/afcurgentcare.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-06-23-14-42-18/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-06-23-14-42-18/output/afcurgentcare.geojson))
During the global build at 2021-06-30-14-42-26, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-06-30-14-42-26/logs/afcurgentcare.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-06-30-14-42-26/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-06-30-14-42-26/output/afcurgentcare.geojson))
During the global build at 2021-07-07-14-42-19, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-07-07-14-42-19/logs/afcurgentcare.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-07-07-14-42-19/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-07-07-14-42-19/output/afcurgentcare.geojson))
During the global build at 2021-07-14-14-42-22, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-07-14-14-42-22/logs/afcurgentcare.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-07-14-14-42-22/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-07-14-14-42-22/output/afcurgentcare.geojson))
During the global build at 2021-07-21-14-42-39, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-07-21-14-42-39/logs/afcurgentcare.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-07-21-14-42-39/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-07-21-14-42-39/output/afcurgentcare.geojson))
During the global build at 2021-07-28-14-42-16, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-07-28-14-42-16/logs/afcurgentcare.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-07-28-14-42-16/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-07-28-14-42-16/output/afcurgentcare.geojson))
During the global build at 2021-08-04-14-42-45, spider **afcurgentcare** failed with **0 features** and **0 errors**.

Here's [the log](https://data.alltheplaces.xyz/runs/2021-08-04-14-42-45/logs/afcurgentcare.txt) and [the output](https://data.alltheplaces.xyz/runs/2021-08-04-14-42-45/output/afcurgentcare.geojson) ([on a map](https://data.alltheplaces.xyz/map.html?show=https://data.alltheplaces.xyz/runs/2021-08-04-14-42-45/output/afcurgentcare.geojson))

Resolving patch:
diff --git a/locations/spiders/afcurgentcare.py b/locations/spiders/afcurgentcare.py
--- a/locations/spiders/afcurgentcare.py
+++ b/locations/spiders/afcurgentcare.py
@@ -1,45 +1,48 @@
 import json
-import re
+import urllib.parse
+
 import scrapy
+
 from locations.items import GeojsonPointItem
-from locations.hours import OpeningHours
+from locations.hours import OpeningHours, DAYS
+
 
 class AfcUrgentCareSpider(scrapy.Spider):
     name = "afcurgentcare"
-    item_attributes = { 'brand': "AFC Urgent Care" }
+    item_attributes = {"brand": "AFC Urgent Care"}
     allowed_domains = ["afcurgentcare.com"]
-    download_delay = 0.2
     start_urls = (
-        'https://www.afcurgentcare.com/locations/',
+        "https://www.afcurgentcare.com/modules/multilocation/?near_lat=39&near_lon=-98",
     )
 
     def parse(self, response):
-        for url in response.xpath('//li[@class="location"]/@data-href').extract():
-            yield scrapy.Request(
-                response.urljoin(url),
-                callback=self.parse_store,
-            )
+        j = json.loads(response.body)
+        if j["meta"]["next"] is not None:
+            qs = "?" + urllib.parse.urlparse(j["meta"]["next"]).query
+            yield scrapy.Request(urllib.parse.urljoin(response.url, qs))
+        for obj in j["objects"]:
+            yield from self.parse_store(obj)
 
-    def parse_store(self, response):
+    def parse_store(self, obj):
         properties = {
-            'ref': response.url,
-            'lat': response.xpath('//div[@class="map-container"]/div/@data-latitude').extract_first(),
-            'lon': response.xpath('//div[@class="map-container"]/div/@data-longitude').extract_first(),
-            'phone': response.xpath('//a[@class="phone-link"]/span/text()').extract_first(),
-            'addr_full': response.xpath('//span[@itemprop="streetAddress"]/text()').extract_first().strip(),
-            'name': response.xpath('//meta[@itemprop="name legalName"]/@content').extract_first(),
-            'city': response.xpath('//span[@itemprop="addressLocality"]/text()').extract_first()[:-1],
-            'state': response.xpath('//span[@itemprop="addressRegion"]/text()').extract_first().strip(),
-            'postcode': response.xpath('//span[@itemprop="postalCode"]/text()').extract_first().strip(),
+            "ref": obj["id"],
+            "lat": obj["lat"],
+            "lon": obj["lon"],
+            "phone": obj["phonemap_e164"].get("phone"),
+            "addr_full": obj["street"],
+            "name": obj["location_name"],
+            "city": obj["city"],
+            "state": obj["state"],
+            "postcode": obj["postal_code"],
+            "website": obj["location_url"],
         }
 
         o = OpeningHours()
-        for h in response.css('#LocalMapAreaOpenHourBanner li.h-day'):
-            day = h.xpath('em/span/text()').extract_first().strip()[:2]
-            day_range = h.xpath('em/text()').extract_first().strip(':').strip()
-            open_time, close_time = day_range.split(' - ')
-
-            o.add_range(day, open_time, close_time, '%I:%M %p')
-        properties['opening_hours'] = o.as_opening_hours()
+        for ([h, _], day) in zip(obj["hours_of_operation"], DAYS):
+            if not h:
+                continue
+            open_time, close_time = h
+            o.add_range(day, open_time, close_time, "%H:%M:%S")
+        properties["opening_hours"] = o.as_opening_hours()
 
         yield GeojsonPointItem(**properties)


Test patch: