Repository: mdn/kuma

Issue:
React breaks on certain IDs within the content
Originally filed on https://github.com/mdn/sprints/issues/2148

> Just wanted to point out some rendering issues that are caused by translations (like French language for example) which makes some selectors not valid.
> Take a look : https://developer.mozilla.org/fr/docs/Web/CSS/line-height




Discussion:
Quoting @schalkneethling 


> Thank you for reporting this @DamChtlv
> 
> The problem is because of:
> 
> ```
> #Utiliser_les_transitions_pour_accentuer_les_éléments_d'un_menu ~ * pre[class*=html]' is not a valid selector
> ```
> 
> My first guess is that it is the characters such as `é` and `d'u` causing the problem

The reasons this happened is one thing. The more pressing thing is that we didn't have any test automation, that uses a real browser, that checks if pages render correctly. 
Able to reproduce locally:
<img width="1559" alt="Screen Shot 2019-09-12 at 10 10 52 AM" src="https://user-images.githubusercontent.com/26739/64791387-a5a4a000-d545-11e9-90a7-6792c74b0343.png">

It looks like it might actually be characters such as `'` as an id such as `Relzione_tra_length_e_proprietà_numeriche` does not break it. Specifically referring to the `à` character in that string.

https://developer.mozilla.org/it/docs/Web/JavaScript/Reference/Global_Objects/Array

So far I have only found the problem with the `fr` locale
I wonder, should the ID have become `Relzione_tra_length_e_proprieta_numeriche` (instead of `Relzione_tra_length_e_proprietà_numeriche`) by the Wiki post-processing?
Yep, @schalkneethling is right. My bad. We just need to make sure the `iframes` don't have a `'` in their ID. 
https://codepen.io/peterbe/pen/vYBrOzj?editors=1111
Sorry, not iframes. These:
```html
<h3 id="Utiliser_les_transitions_pour_accentuer_les_éléments_d'un_menu">Utiliser les transitions pour accentuer les éléments d'un menu</h3>
```
Well sorry for the French :)
Rewording the ID could be a workaround
Adding the `'` in `non_url_safe` https://github.com/mozilla/kuma/blob/master/kuma/wiki/content.py#L556 could help as well

I came to the same conclusion. 
Now I'm just curious what we should do to pages that have "already been allowed" to keep the `'`.
> Well sorry for the French :)

Don't apologize for your beautiful language! It's Kuma who should say sorry. 
> what we should do to pages that have "already been allowed" to keep the `'`

I might have an [old crawler of mine](https://gist.github.com/SphinxKnight/f3b232450fea15cf0118ed2a207b6060#file-scanl10n-js)  that stores the content of `?raw` view on files that I could run to detect the pages having such IDs.
Once Kuma would do something differently, a manual save on those pages should be sufficient (they should not be too many)

A drawback is that I have to "choose" which section I want to crawl and I may miss some pages if I don't know the list of sections beforehand.
Another drawback is that it's quite slow to gather all the pages for all the translations for all the sections.

If possible a fallback display in any case would be nice :s


Resolving patch:
diff --git a/kuma/wiki/content.py b/kuma/wiki/content.py
--- a/kuma/wiki/content.py
+++ b/kuma/wiki/content.py
@@ -577,7 +577,7 @@ def gen_id(self):
     non_url_safe = ['"', '#', '$', '%', '&', '+',
                     ',', '/', ':', ';', '=', '?',
                     '@', '[', '\\', ']', '^', '`',
-                    '{', '|', '}', '~']
+                    '{', '|', '}', '~', "'"]
 
     def slugify(self, text):
         """


Test patch:
diff --git a/kuma/wiki/tests/test_content.py b/kuma/wiki/tests/test_content.py
--- a/kuma/wiki/tests/test_content.py
+++ b/kuma/wiki/tests/test_content.py
@@ -491,6 +491,8 @@ def test_non_ascii_section_headers(self):
              u'HTML5_엘리먼트'),
             (u'Non safe title "#$%&+,/:;=?@[\\]^`{|}~',
              u'Non_safe_title'),
+            (u"Five o'clock",
+             u'Five_oclock'),
         ]
 
         section_filter = SectionIDFilter('')