Repository: alltheplaces/alltheplaces

Issue:
T-Market supermarkets
### Brand name

T-Market

### Wikidata ID

Q64033983

### Store finder url(s)

https://tmarket.bg/sitemap/page/1.xml

Each location has a separate URL that looks like this: `https://tmarket.bg/page/magazin-{shop_name}`. They may have forgotten to put a few dozen closing tags which makes finding the opening hours quite difficult. Having only coordinates would be fine at this stage.


Discussion:
----
### Brand name

T-Market

Bulgarian supermarket chain

### Wikidata ID

Q64033983
https://www.wikidata.org/wiki/Q64033983
https://www.wikidata.org/wiki/Special:EntityData/Q64033983.json

### Store finder url(s)

Official Url(s): http://www.tmarket.bg/
- [x] I have run the below and if anything in generated, added it.
- [x] I have manually checked for a storefinder
- [ ] This is not a manufacturer listing outlets operating under their own brands / this will not create duplicates
`pipenv run scrapy sf --brand-wikidata=Q64033983 http://www.tmarket.bg/`
```
```

https://tmarket.bg/page/magazin-gorna-orqhovitsa
No structured data

```
<div class="_text-description">
            <div class="qqvbed-p83tee">
<h3 class="qqvbed-p83tee-lTBxed">Магазин - Горна Оряховица</h3>
<p class="qqvbed-p83tee-lTBxed">Адрес:</p>
<p class="qqvbed-p83tee-lTBxed">Горна Оряховица, ул. Васил Левски 10</p>
</div>
<div class="qqvbed-p83tee">
<div class="qqvbed-p83tee-lTBxed">Телефон на потребителя:</div>
<div class="qqvbed-p83tee-lTBxed">
<div class="phone"><strong>0700 701 71</strong></div>
<div class="phone">&nbsp;</div>
<div class="phone">
<div>Работно време:</div>
<div></div>
<div class="phone">
<div class="_4bl9">
<div id="u_fetchstream_1_1" class="_6a uiPopover openToggler selected">
<div id="u_fetchstream_1_2" class="clearfix _ikh _5jau _p" role="button" aria-haspopup="true" aria-expanded="false" aria-controls="u_fetchstream_1_0">
<div class="_4bl7"><span><strong>Понеделник - Неделя</strong></span></div>
<div class="_4bl7">08:00 ч. - 22:00 ч.</div>
</div>
</div>
</div>
</div>
<div class="phone"></div>
<div class="phone">
<div>Email:</div>
<div class="phone"><span><strong>maxima@maximabulgaria.bg</strong></span></div>
</div>
</div>
</div>
</div>
        </div>
```
If you use innerHTML, you get a bit of structure.

```
Магазин - Горна Оряховица

Адрес:

Горна Оряховица, ул. Васил Левски 10

Телефон на потребителя:
0700 701 71
 
Работно време:
Понеделник - Неделя
08:00 ч. - 22:00 ч.
Email:
maxima@maximabulgaria.bg
```
I mean more specifically no linked data, rdfa, jsonld, etc - https://validator.schema.org/ surfaces nothing



Resolving patch:
diff --git a/locations/spiders/t_market_bg.py b/locations/spiders/t_market_bg.py
new file mode 100644
--- /dev/null
+++ b/locations/spiders/t_market_bg.py
@@ -0,0 +1,56 @@
+import re
+
+from scrapy.spiders import SitemapSpider
+
+from locations.google_url import extract_google_position
+from locations.hours import DAYS_BG, OpeningHours, day_range, sanitise_day
+from locations.items import Feature
+
+
+class T_Market_BGSpider(SitemapSpider):
+    name = "t_market_bg"
+    item_attributes = {"brand": "T-Market", "brand_wikidata": "Q64033983"}
+    allowed_domains = ["tmarket.bg"]
+    sitemap_urls = ["https://tmarket.bg/sitemap/page/1.xml"]
+    sitemap_rules = [(r"https://tmarket\.bg/page/magazin[\w-]+", "parse")]
+    no_refs = True
+
+    def parse(self, response):
+        item = Feature()
+        extract_google_position(item, response)
+        item["addr_full"] = (
+            response.xpath('//p[contains(text(), "Адрес:")]/following-sibling::p[1]/text()').get(default="").strip()
+            if response.xpath('//p[contains(text(), "Адрес:")]')
+            else None
+        )
+
+        item["opening_hours"] = OpeningHours()
+        # Some pages have split the opening days in multiple elements like <strong>Понеделник - <b>Неделя</b></strong>
+        oh_days = response.xpath(
+            '//*[contains(text(), "Понеделник") or contains(text(), "Вторник") or contains(text(), "Сряда") or contains(text(), "Четвъртък") or contains(text(), "Петък") or contains(text(), "Събота") or contains(text(), "Неделя")]//text()'
+        ).getall()
+        self.log(oh_days)
+        # Others lack days entirely, so it's Mo-Su
+        if len(oh_days) == 1 and oh_days[0] is None:
+            oh_days[0] = "Понеделник - Неделя"
+        oh_days = [re.sub(r"[^а-я-]", "", days.lower()) for days in oh_days]
+
+        # Merge the split day ranges
+        for index, day in enumerate(oh_days):
+            if day.endswith("-"):
+                oh_days[index] = "".join([oh_days[index], oh_days.pop(index + 1)])
+        oh_days = [[sanitise_day(day, DAYS_BG) for day in days.split("-")] for days in oh_days]
+
+        oh_hours = response.xpath('(//*[contains(text(), ":00")])/text()').getall()
+        oh_hours = [re.sub(r"[^0-9:-]", "", hours).split("-") for hours in oh_hours]
+
+        item["opening_hours"] = OpeningHours()
+        for index, days in enumerate(oh_days):
+            if len(days) == 2:
+                item["opening_hours"].add_days_range(
+                    day_range(days[0], days[1]), oh_hours[index][0], oh_hours[index][1]
+                )
+            else:
+                item["opening_hours"].add_range(days[0], oh_hours[index][0], oh_hours[index][1])
+
+        yield item


Test patch: