Repository: alltheplaces/alltheplaces

Issue:
Pizza Hut
```
curl 'https://www.pizzahut.com/api.php/site/api_ajax/find_nearby' \
 -H 'Accept: application/json, text/plain, */*' \
 -H 'Referer: https://www.pizzahut.com/' \
 --data-binary '{"occasion":"C","zip":"24450","near":"24450","address":"","dine_in":false,"store_limit":10,"mob_find":true}'
```


Discussion:
A bit stumped by this one, the only way to get the data seems to be to send a request to their api (like the one mentioned in iandees's comment) that uses a cookie which likely expires. Unassigning myself for now.
Hello
Same situation as Subway and Wallgreens
Just need lat/lon pair and we can request:
https://locator-svc.subway.com/v2//GetLocations.ashx?q={"InputText":"","GeoCode":{"Latitude":47.5768927,"Longitude":-120.20598330000001,"Accuracy":null,"CountryCode":"US","RegionCode":null,"PostalCode":null,"City":null,"LocalityType":null},"DetectedLocation":{"Latitude":0,"Longitude":0,"Accuracy":0},"Paging":{"StartIndex":1,"PageSize":10},"ConsumerParameters":{"metric":false,"culture":"en-US","country":"US","size":"D","template":"","rtl":false,"clientId":"17","key":"SUBWAY_PROD"},"Filters":[],"LocationType":2}

Work stopped, safe for me to reassign to myself? I can tackle this right now.
Yes

Sent from my handsful device.



> On Dec 18, 2017, at 18:40, whoissteven <notifications@github.com> wrote:
> 
> Work stopped, safe for me to reassign to myself? I can tackle this right now.
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub, or mute the thread.
> 

Are we allowed to use splash to render javascript on page?
As developer, who already analyzed Pizza Hut I can say it's not necessary to render any javascript on this site to grab data.
Only one problem I see - searching by position or ZIP.
Last time I used to scan all USA ZIPs, I got message I shouldn't use this method because of huge amount of requests and overlapped zones.
Correct me if I wrong, please. Do you find another way to scan it?

By the way, question about splash interested me too, because I have javascript-building site to scrap.

This issue hasn't been worked on in a while. I'm removing the assignment so others can work on it. If you're still working on it, please reassign yourself and reference a pull request or branch where you're doing your work.
Looks like zipcode search no longer needed:  https://locations.pizzahut.com/

Resolving patch:
diff --git a/locations/spiders/pizza_hut.py b/locations/spiders/pizza_hut.py
new file mode 100644
--- /dev/null
+++ b/locations/spiders/pizza_hut.py
@@ -0,0 +1,53 @@
+# -*- coding: utf-8 -*-
+import re
+import scrapy
+from locations.items import GeojsonPointItem
+
+class PizzaHutSpider(scrapy.Spider):
+    name = "pizza_hut"
+    allowed_domains = ["pizzahut.com"]
+    download_delay = 0.1
+    start_urls = (
+        'https://locations.pizzahut.com/',
+    )
+
+    def parse(self, response):
+        urls = response.xpath('//li[@class="Directory-listItem"]/a/@href').extract()
+
+        for url in urls:
+            if len(url.split('/')) == 2:
+                yield scrapy.Request(response.urljoin(url), callback=self.parse_stores)
+            else:
+                yield scrapy.Request(response.urljoin(url), callback=self.parse_cities)
+
+    def parse_cities(self,response):
+        urls = response.xpath('//a[@class="Directory-listLink"]/@href').extract()
+
+        for url in urls:
+            if len( url.split('/')) == 2:
+                yield scrapy.Request(response.urljoin(url), callback=self.parse_stores)
+            else:
+                yield scrapy.Request(response.urljoin(url), callback=self.parse_store)
+
+    def parse_stores(self, response):
+        urls = response.xpath('//div[@class="Directory-content"]//li//a[@class="Teaser-titleLink"]/@href').extract()
+        for url in urls:
+            yield scrapy.Request(response.urljoin(url), callback=self.parse_store)
+
+    def parse_store(self, response):
+        ref = re.findall(r".com/(.+?)/(.+?)/(.+)", response.url)[0]
+        ref = "_".join(ref)
+        properties = {
+            'name': response.xpath('//span[@class="LocationName-geo"]/text()').extract_first(),
+            'addr_full': response.xpath('//span[@class="c-address-street-1"]/text()').extract_first(),
+            'phone': response.xpath('//span[@itemprop="telephone"]/text()').extract_first(),
+            'city': response.xpath('//span[@class="c-address-city"]/text()').extract_first(),
+            'state': response.xpath('//abbr[@class="c-address-state"]/text()').extract_first(),
+            'postcode': response.xpath('//span[@class="c-address-postal-code"]/text()').extract_first(),
+            'ref': ref,
+            'website': response.url,
+            'lat': float(response.xpath('//meta[@itemprop="latitude"]/@content').extract_first()),
+            'lon': float(response.xpath('//meta[@itemprop="longitude"]/@content').extract_first()),
+        }
+
+        yield GeojsonPointItem(**properties)
\ No newline at end of file


Test patch: