Repository: alltheplaces/alltheplaces

Issue:
McDonald's
JSON endpoint: http://rl.mcdonalds.com/googleapps/GoogleSearchUSAction.do?method=searchLocation&searchTxtLatlng=(43.1272254%2C-87.9432837)&actionType=searchRestaurant&language=en&country=us

Search by lat/lon only? Looks like they geocode using Google Maps API and then call this endpoint with a lat/lon.



Discussion:
this issue was solved?
The existing spider covers the UK, Canada, and the US only. There are several other countries with McDonald's that can be included.
I see.
let me see this when I have time.
@iandees 
could you possibly let me know which countries is available on this task as many as you know?
thanks.
There's a Wikipedia article on the subject (with links to the restaurant locators, too!): https://en.wikipedia.org/wiki/List_of_countries_with_McDonald%27s_restaurants
https://en.wikipedia.org/wiki/List_of_countries_with_McDonald%27s_restaurants
yes. :)
@iandees please review this and let me know.
thanks.
@iandees it seems we should write many spiders for mcdonalds website for each country.
here are some websites I researched to scrap.
https://www.mcdonalds.com/se/sv-se.html
https://www.restaurants.mcdonalds.fr/
http://www.mcdonalds.de/restaurant-suche
https://mcdonalds.com.au/find-us/restaurants
http://www.mcdonalds.com.pa/locales
http://www.mcdonalds.co.cr/locales
http://www.mcdonalds.com.pr/locales
...

as you know see, these website has different types.
so I think we can't do crawling in one Spider.

what do you think about this?
please let me know about your thought.
thanks.
Maybe it should be even split into multiple issues.
@iandees , @nvkelso 
could you possibly let me know about your thought for McDonalds websites on several countries?
hopefully look forward for hearing from you soon.
thanks.
It makes sense to have multiple spiders for McDonald's. For consistency's sake maybe make the file names `mcdonalds_nl.py`, `mcdonalds_de.py`, etc. where the last part is the TLD of the domain?
okay, got you.
well..., @iandees 
I'll not assign new issues to me, I'd like to focus on mcdonalds websites.
okay for you?
Yep, that's ok with me.
thanks.
I prefer someone tackles this on head on, for all global McDonalds locations. We have a PR that's been merged that just does USA and couple other countries. The others remain TODO.
@nvkelso I already did it.
but we didn't get expected result because websites on other countries has different website url and response to parse.
I already said about it to @iandees 
@iandees 
I've added McLocalizer Spider to #301 
please review it.
thanks.
@iandees 
is it possible to create new issues for each countries of McDonalds by you or me?
hopefully look forward for hearing from you soon.
thanks.
If you'd like to make multiple issues you should have the permissions to do that. Make sure they link back to this one, though. Otherwise just make pull requests for each of the countries and link them to this one.
okay, got you.
hi, @iandees 
https://map.mcdonalds.co.jp/
could you possibly look at this website?
I can't get the rest api result, so can't crawling this website.
any guide for me?
thanks.
Yea, not sure what's going on there. I wouldn't worry about it for now and move on to another country.
okay

Resolving patch:
diff --git a/locations/spiders/mcdonalds_localizer.py b/locations/spiders/mcdonalds_localizer.py
--- a/locations/spiders/mcdonalds_localizer.py
+++ b/locations/spiders/mcdonalds_localizer.py
@@ -7,11 +7,12 @@
 class McLocalizer(scrapy.Spider):
 
     name = "mclocalizer"
-    allowed_domains = ["www.mcdonalds.com", "www.mcdonalds.com.pr", "www.mcdonalds.co.cr", "www.mcdonalds.com.ar"]
+    allowed_domains = ["www.mcdonalds.com", "www.mcdonalds.com.pr", "www.mcdonalds.co.cr", "www.mcdonalds.com.ar", "www.mcdonalds.com.pa"]
     start_urls = (
         'http://www.mcdonalds.com.pr/api/restaurantsByCountry?country=PR',
         'http://www.mcdonalds.co.cr/api/restaurantsByCountry?country=CR',
-        'http://www.mcdonalds.com.ar/api/restaurantsByCountry?country=AR'
+        'http://www.mcdonalds.com.ar/api/restaurantsByCountry?country=AR',
+        'http://www.mcdonalds.com.pa/api/restaurantsByCountry?country=PA'
     )
 
     def parse(self, response):
@@ -33,6 +34,5 @@ def parse(self, response):
 
             properties["name"] = name
             properties["addr_full"] = data['name'][data['name'].find("<small>"):-8][8:]
-            #  = address[8:]
 
             yield GeojsonPointItem(**properties)
\ No newline at end of file


Test patch: