Repository: alltheplaces/alltheplaces

Issue:
Dia (ES) ~1.7k shop=supermarket
### Brand name

Dia

### Wikidata ID

Q925132

### Store finder url(s)

https://www.dia.es/tiendas/buscador-tiendas-folletos


Discussion:
GZIP'd JSON file containing location IDs and coordinates: `https://www.dia.es/clubdia/ES/tiendas.v2031.json.gz` (note: the v2031 is possibly a constantly changing version number, hence this URL may need to be extracted from the store locator page?)

Then for each location ID, an API request can be made to (for example) `https://www.dia.es/tiendas/buscadorTiendas.html?action=buscarInformacionTienda&id=7817` to return location details.

There doesn't appear to be a webpage specific to each location, hence no mention in the sitemap of location pages.
GZIP'd JSON file is indeed versioned and now: `https://www.dia.es/clubdia/ES/tiendas.v2034.json.gz`

Resolving patch:
diff --git a/locations/spiders/dia_es.py b/locations/spiders/dia_es.py
new file mode 100644
--- /dev/null
+++ b/locations/spiders/dia_es.py
@@ -0,0 +1,51 @@
+import json
+from gzip import decompress
+
+from scrapy import Request, Spider
+from scrapy.http import JsonRequest
+
+from locations.hours import DAYS, OpeningHours
+from locations.items import Feature
+
+
+class DiaESSpider(Spider):
+    name = "dia_es"
+    item_attributes = {"brand": "Dia", "brand_wikidata": "Q925132"}
+    allowed_domains = ["www.dia.es"]
+    start_urls = ["https://www.dia.es/tiendas/buscador-tiendas-folletos"]
+
+    def parse(self, response):
+        store_list_url = response.xpath('.//input[@id="gz"]/@value').get()
+        yield Request(url=store_list_url, callback=self.parse_store_list)
+
+    def parse_store_list(self, response):
+        locations = json.loads(decompress(response.body).decode("utf-8"))
+        for location in locations:
+            store_id = location["idTienda"]
+            yield JsonRequest(
+                url=f"https://www.dia.es/tiendas/buscadorTiendas.html?action=buscarInformacionTienda&id={store_id}",
+                meta={"lat": location["posicionX"], "lon": location["posicionY"]},
+                callback=self.parse_store,
+            )
+
+    def parse_store(self, response):
+        properties = {
+            "ref": response.json()["tiendaId"],
+            "lat": response.meta["lat"],
+            "lon": response.meta["lon"],
+            "street_address": response.json()["direccionPostal"].strip(),
+            "housenumber": response.json()["numeroVia"].strip(),
+            "street": response.json()["nombreVia"].strip() + " " + response.json()["tipoVia"].strip(),
+            "city": response.json()["localidad"].strip(),
+            "postcode": response.json()["codigoPostal"].strip(),
+        }
+        properties["opening_hours"] = OpeningHours()
+        for day_number, day_hours in response.json()["horariosTienda"].items():
+            if "|" in day_hours:
+                for day_hours_range in day_hours.split(" | "):
+                    properties["opening_hours"].add_range(
+                        DAYS[int(day_number) - 1], *day_hours_range.split(" - ", 1), "%H:%M"
+                    )
+            else:
+                properties["opening_hours"].add_range(DAYS[int(day_number) - 1], *day_hours.split(" - ", 1), "%H:%M")
+        yield Feature(**properties)


Test patch: