Repository: mui/material-ui (TypeScript)
Date: 2020-05-15 05:22:48+00:00
Task type: Feature

Issue:
[docs] Keep the same header between locales
<!-- Provide a general summary of the feature in the Title above -->

<!--
  Thank you very much for contributing to Material-UI by creating an issue! ❤️
  To avoid duplicate issues we ask you to check off the following list.
-->

<!-- Checked checkbox should look like this: [x] -->

- [X] I have searched the [issues](https://github.com/mui-org/material-ui/issues) of this repository and believe that this is not a duplicate.

## Intention

Regardless of how the translation affects the content, it should not affect the anchor (#) that keeps pointing to the content.

## Summary 💡

The request is that the translations keep the same header, facilitating the way the links are made available between locales.
Currently when a content is translated, the anchor for this content also changes, resulting in the scenario below.

<!-- Describe how it should work. -->

## Examples 🌈

- https://material-ui.com/components/rating/#simple-ratings

- https://material-ui.com/zh/components/rating/#%E7%AE%80%E5%8D%95%E8%AF%84%E5%88%86

## Expected

Regardless of locale, same content anchor #

- https://material-ui.com/components/rating/#simple-ratings

- https://material-ui.com/zh/components/rating/#simple-ratings

- https://material-ui.com/pt/components/rating/#simple-ratings

## Motivation 🔦

[Discussion](https://github.com/mui-org/material-ui/pull/20779#discussion_r415352929) started on this pull request #20779.

<!--
  What are you trying to accomplish? How has the lack of this feature affected you?
  Providing context helps us come up with a solution that is most useful in the real world.
-->



Issue discussion:
@jaironalves Thanks, regarding the implementation, it would require to:

1. Change the logic of `parseMarkdown.js` to process the English version first.
2. In the `heading` handler, use the English hash for the non-English locales

Done
Note that using the id from another translation can break if the translations aren't synced. It's not ideal for the master deploy but manageable. We just need to make sure that we never deploy outdated translations to `material-ui.com`.
@eps1lon Great point. Depending on the level of quality we want to provide on the translations, we could either fail or fallback to the translated hash. Maybe the latter? Crowdin guarantees that it will be eventually kept in sync.
> Crowdin guarantees that it will be eventually kept in sync.

That is hardly a useful guarantee. When is "eventually"?
@eps1lon The "when" depends on the frequency we merge the changes into the repository. I think that it can be automated. Right now, it's every time we manually do it. For instance, once per release.
That's what I was saying: It's not as important if they're out-of-sync on `master`. But they need to be synced before we deploy `material-ui.com` which happens on release.

Solution patch:
diff --git a/docs/src/modules/utils/parseMarkdown.js b/docs/src/modules/utils/parseMarkdown.js
--- a/docs/src/modules/utils/parseMarkdown.js
+++ b/docs/src/modules/utils/parseMarkdown.js
@@ -10,6 +10,7 @@ const titleRegExp = /# (.*)[\r\n]/;
 const descriptionRegExp = /<p class="description">(.*)<\/p>[\r\n]/;
 const headerKeyValueRegExp = /(.*): (.*)/g;
 const emptyRegExp = /^\s*$/;
+const notEnglishMarkdownRegExp = /-([a-z]{2})\.md$/;
 
 /**
  * Extract information from the top of the markdown.
@@ -130,19 +131,32 @@ export function prepareMarkdown(config) {
 
   const demos = {};
   const docs = {};
+  const headingHashes = {};
+
+  // Process the English markdown before the other locales.
+  let filenames = [];
   requireRaw.keys().forEach((filename) => {
+    if (filename.match(notEnglishMarkdownRegExp)) {
+      filenames.push(filename);
+    } else {
+      filenames = [filename].concat(filenames);
+    }
+  });
+
+  filenames.forEach((filename) => {
     if (filename.indexOf('.md') !== -1) {
-      const match = filename.match(/-([a-z]{2})\.md$/);
+      const matchNotEnglishMarkdown = filename.match(notEnglishMarkdownRegExp);
 
       const userLanguage =
-        match && LANGUAGES_IN_PROGRESS.indexOf(match[1]) !== -1 ? match[1] : 'en';
+        matchNotEnglishMarkdown && LANGUAGES_IN_PROGRESS.indexOf(matchNotEnglishMarkdown[1]) !== -1
+          ? matchNotEnglishMarkdown[1]
+          : 'en';
 
       const markdown = requireRaw(filename);
-      const contents = getContents(markdown);
       const headers = getHeaders(markdown);
-
       const title = headers.title || getTitle(markdown);
       const description = headers.description || getDescription(markdown);
+      const contents = getContents(markdown);
 
       if (headers.components.length > 0) {
         contents.push(`
@@ -157,8 +171,10 @@ ${headers.components
   `);
       }
 
-      const headingHashes = {};
       const toc = [];
+      const headingHashesFallbackTranslated = {};
+      let headingIndex = -1;
+
       const rendered = contents.map((content) => {
         if (demos && demoRegexp.test(content)) {
           try {
@@ -186,17 +202,28 @@ ${headers.components
               ) // remove emojis
               .replace(/<\/?[^>]+(>|$)/g, '') // remove HTML
               .trim();
-            const hash = textToHash(headingText, headingHashes);
+
+            // Standardizes the hash from the default location (en) to different locations
+            // Need english.md file parsed first
+            let hash;
+            if (userLanguage === 'en') {
+              hash = textToHash(headingText, headingHashes);
+            } else {
+              headingIndex += 1;
+              hash = Object.keys(headingHashes)[headingIndex];
+              if (!hash) {
+                hash = textToHash(headingText, headingHashesFallbackTranslated);
+              }
+            }
 
             // enable splitting of long words from function name + first arg name
             // Closing parens are less interesting since this would only allow breaking one character earlier.
             // Applying the same mechanism would also allow breaking of non-function signatures like "Community help (free)".
             // To detect that we enabled breaking of open/closing parens we'd need a context-sensitive parser.
             const displayText = headingText.replace(/([^\s]\()/g, '$1&#8203;');
-            /**
-             * create a nested structure with 2 levels starting with level 2 e.g.
-             * [{...level2, children: [level3, level3, level3]}, level2]
-             */
+
+            // create a nested structure with 2 levels starting with level 2 e.g.
+            // [{...level2, children: [level3, level3, level3]}, level2]
             if (level === 2) {
               toc.push({
                 text: displayText,
@@ -247,6 +274,7 @@ ${headers.components
           },
         });
       });
+
       // fragment link symbol
       rendered.unshift(`<svg style="display: none;" xmlns="http://www.w3.org/2000/svg">
   <symbol id="anchor-link-icon" viewBox="0 0 16 16">
@@ -254,11 +282,13 @@ ${headers.components
   </symbol>
 </svg>`);
 
-      const location = headers.filename || `/docs/src/pages/${pageFilename}/${filename}`;
-
-      const localized = { description, location, rendered, toc, title };
-
-      docs[userLanguage] = localized;
+      docs[userLanguage] = {
+        description,
+        location: headers.filename || `/docs/src/pages/${pageFilename}/${filename}`,
+        rendered,
+        toc,
+        title,
+      };
     } else if (filename.indexOf('.tsx') !== -1) {
       const demoName = `pages/${pageFilename}/${filename
         .replace(/\.\//g, '')


Test patch:
diff --git a/docs/src/modules/utils/parseMarkdown.test.js b/docs/src/modules/utils/parseMarkdown.test.js
--- a/docs/src/modules/utils/parseMarkdown.test.js
+++ b/docs/src/modules/utils/parseMarkdown.test.js
@@ -104,5 +104,189 @@ describe('parseMarkdown', () => {
         },
       ]);
     });
+
+    it('use english hash for different locales', () => {
+      const markdownEn = `
+# Localization
+## Locales
+### Example
+### Use same hash
+`;
+
+      const markdownPt = `
+# Localização
+## Idiomas
+### Exemplo
+### Usar o mesmo hash
+`;
+
+      const markdownZh = `
+# 所在位置
+## 语言环境
+### 例
+### 使用相同的哈希
+`;
+      // mock require.context
+      function requireRaw(filename) {
+        switch (filename) {
+          case 'localization-pt.md':
+            return markdownPt;
+          case 'localization-zh.md':
+            return markdownZh;
+          default:
+            return markdownEn;
+        }
+      }
+      requireRaw.keys = () => ['localization-pt.md', 'localization.md', 'localization-zh.md'];
+
+      const {
+        docs: {
+          en: { toc: tocEn },
+          pt: { toc: tocPt },
+          zh: { toc: tocZh },
+        },
+      } = prepareMarkdown({
+        pageFilename: 'same-hash-test',
+        requireRaw,
+      });
+
+      expect(tocZh).to.have.deep.ordered.members([
+        {
+          children: [
+            {
+              hash: 'example',
+              level: 3,
+              text: '例',
+            },
+            {
+              hash: 'use-same-hash',
+              level: 3,
+              text: '使用相同的哈希',
+            },
+          ],
+          hash: 'locales',
+          level: 2,
+          text: '语言环境',
+        },
+      ]);
+
+      expect(tocPt).to.have.deep.ordered.members([
+        {
+          children: [
+            {
+              hash: 'example',
+              level: 3,
+              text: 'Exemplo',
+            },
+            {
+              hash: 'use-same-hash',
+              level: 3,
+              text: 'Usar o mesmo hash',
+            },
+          ],
+          hash: 'locales',
+          level: 2,
+          text: 'Idiomas',
+        },
+      ]);
+
+      expect(tocEn).to.have.deep.ordered.members([
+        {
+          children: [
+            {
+              hash: 'example',
+              level: 3,
+              text: 'Example',
+            },
+            {
+              hash: 'use-same-hash',
+              level: 3,
+              text: 'Use same hash',
+            },
+          ],
+          hash: 'locales',
+          level: 2,
+          text: 'Locales',
+        },
+      ]);
+    });
+
+    it('use translated hash for translations are not synced', () => {
+      const markdownEn = `
+# Localization
+## Locales
+### Example
+### Use same hash
+`;
+
+      const markdownPt = `
+# Localização
+## Idiomas
+### Exemplo
+### Usar o mesmo hash
+### Usar traduzido
+`;
+
+      // mock require.context
+      function requireRaw(filename) {
+        return filename === 'localization-pt.md' ? markdownPt : markdownEn;
+      }
+      requireRaw.keys = () => ['localization-pt.md', 'localization.md'];
+
+      const {
+        docs: {
+          en: { toc: tocEn },
+          pt: { toc: tocPt },
+        },
+      } = prepareMarkdown({
+        pageFilename: 'same-hash-test',
+        requireRaw,
+      });
+
+      expect(tocPt).to.have.deep.ordered.members([
+        {
+          children: [
+            {
+              hash: 'example',
+              level: 3,
+              text: 'Exemplo',
+            },
+            {
+              hash: 'use-same-hash',
+              level: 3,
+              text: 'Usar o mesmo hash',
+            },
+            {
+              hash: 'usar-traduzido',
+              level: 3,
+              text: 'Usar traduzido',
+            },
+          ],
+          hash: 'locales',
+          level: 2,
+          text: 'Idiomas',
+        },
+      ]);
+
+      expect(tocEn).to.have.deep.ordered.members([
+        {
+          children: [
+            {
+              hash: 'example',
+              level: 3,
+              text: 'Example',
+            },
+            {
+              hash: 'use-same-hash',
+              level: 3,
+              text: 'Use same hash',
+            },
+          ],
+          hash: 'locales',
+          level: 2,
+          text: 'Locales',
+        },
+      ]);
+    });
   });
 });