Repository: alltheplaces/alltheplaces

Issue:
Dunelm spider output is missing 41 branches (dunelm_gb)
The Dunelm spider dunelm_gb is consistently returning 138 branches for the last few weeks. However, Dunelm's own online store-finder at https://www.dunelm.com/stores/a-z lists 179 branches. All of the 138 are included in the 179, meaning the spider is missing 41.

For example, the following branches appear on Dunelm's website, but aren't returned by the spider:
- https://www.dunelm.com/stores/altrincham
- https://www.dunelm.com/stores/basildon
- https://www.dunelm.com/stores/beckton
- https://www.dunelm.com/stores/beverley

I'm afraid I can't figure out how to manually replicate the spider's request, to check whether the missing branches are missing from the API return, or are just not being picked up by the spider for some reason.

I don't know if there's any connection between the missing stores. The Basildon one only opened recently in April 2022 ([source](https://www.echo-news.co.uk/news/20100489.dunelm-opens-mayflower-retail-park-basildon/)) but the Altrincham store has been around since 2017 ([source](https://www.messengernewspapers.co.uk/news/whereyoulive/15122706.customers-attend-opening-of-dunelms-new-altrincham-store/)). I've checked a few of the missing branches and found facebook supprt groupswith recent posts, suggesting that the stores are indeed still open.

If the API isn't returning all the stores, then perhaps the online list at https://www.dunelm.com/stores/a-z could be used by the spider instead, or maybe https://www.dunelm.com/sitemap/static-sitemap.xml (which also seems to include all 179).


Discussion:
We can replicate it with curl: `curl "https://fy8plebn34-dsn.algolia.net/1/indexes/*/queries?x-algolia-application-id=FY8PLEBN34&x-algolia-api-key=ae9bc9ca475f6c3d7579016da0305a33" --data-raw '{"requests":[{"indexName":"stores_prod","params": "hitsPerPage=300"}]}'`

And we can use jq to process the json output and check on it. So: `curl "https://fy8plebn34-dsn.algolia.net/1/indexes/*/queries?x-algolia-application-id=FY8PLEBN34&x-algolia-api-key=ae9bc9ca475f6c3d7579016da0305a33" --data-raw '{"requests":[{"indexName":"stores_prod","params": "hitsPerPage=300"}]}' | jq ".results[0].hits | length"` gives us 179 as there should be.

Checking the [log](https://data.alltheplaces.xyz/runs/2023-01-07-13-32-28/logs/dunelm_gb.txt) gives us the obvious answer. The spider crashes because one of the stores doesn't have an email. We can fix that by doing `store.get("email")`, or we can just remove that line because `DictParser.parse` handles the email in this case anyway.

Do you want to do a PR for this? https://github.com/alltheplaces/alltheplaces/blob/master/docs/GETTING_STARTED.md
Given that I don't really have any experience coding in Python or using git repositories, I'm not sure me trying to write a pull request would be a very good idea! But many thanks for the curl tip and the pointer to the ATP logs - I now know to look there if anything looks wrong with the other spiders. :)
I didn't know any Python before contributing either. Since we're only working on one file, you can completely ignore Git and just use the GitHub website, and you're not really doing much code. I think this is actually the easiest chance you'll ever get.

You go the the [Dunelm spider](https://github.com/alltheplaces/alltheplaces/blob/master/locations/spiders/dunelm_gb.py) and click the [edit/pencil](https://github.com/alltheplaces/alltheplaces/edit/master/locations/spiders/dunelm_gb.py) button in the top right. Delete line 40. Scroll to "Propose changes" at the bottom, something like "Fix Dunelm crash (closes #4632)" and then click the "Propose changes" button.

If you really don't want to do it, it's fine, I'll do it before the next weekly is ran, but you are clearly technical enough and I think at some stage you're going to go from just consuming to consuming and contributing. It may as well be now.

Resolving patch:
diff --git a/locations/spiders/dunelm_gb.py b/locations/spiders/dunelm_gb.py
--- a/locations/spiders/dunelm_gb.py
+++ b/locations/spiders/dunelm_gb.py
@@ -37,7 +37,6 @@ def parse(self, response, **kwargs):
 
             item["opening_hours"] = oh.as_opening_hours()
 
-            item["email"] = store["email"]
             item["extras"] = {"storeType": store.get("storeType")}
 
             yield item


Test patch: