Repository: mondeja/mkdocs-include-markdown-plugin (python)

Issue:
Relative paths to images inside markdown are broken when doing an include
Hello @mondeja ! I haven't touched this module in over a year and as soon as you take it over, I have a feature request! :)

I'm including another section from our README into the docs. This section contains images, in the `![](path_to_image)` format. 

**README.md**
```md
....
<!--examples-start-->
![](docs/assets/example.png)
<!--examples-end-->
...
```

**docs/examples.md**
```md
{%
   include-markdown "../README.md"
   start="<!--examples-start-->"
   end="<!--examples-end-->"
%}
```

But the trouble is, when the snippet from README.md is included, the relative path breaks because it's relative to a different file. I guess the same problem would occur for relative hyperlinks, too.

I'm wondering about a flag that can be used on include-markdown, which would rewrite URLs to be relative to the inclusion location. So something like 
```md
{%
   include-markdown "../README.md"
   start="<!--examples-start-->"
   end="<!--examples-end-->"
   rewrite_relative_urls=true
%}
```

What do you think?


Pull request:
rewrite relative urls
Fix #1

- Add processing code to translate relative paths
- Integrate with the directive and add integration test


Patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 6fd6e63..8c58852 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -12,9 +12,12 @@ on:
 jobs:
   test:
     name: Test
-    runs-on: ubuntu-latest
+    runs-on: ${{ matrix.os }}
     strategy:
       matrix:
+        os:
+          - ubuntu-latest
+          - windows-latest
         python-version:
           - 3.6
           - 3.7
diff --git a/README.md b/README.md
index a097585..38caf30 100644
--- a/README.md
+++ b/README.md
@@ -39,6 +39,10 @@ content to include.
 
 - **start**: Delimiter that marks the beginning of the content to include.
 - **end**: Delimiter that marks the end of the content to include.
+- **rewrite_relative_urls**: When this option is enabled, Markdown links and 
+  images in the content that are specified by a relative URL are 
+  rewritten to work correctly in their new location. Default: `true`. Possible
+  values: `true` , `false`.
 
 ##### Example
 
@@ -50,6 +54,17 @@ content to include.
 %}
 ```
 
+##### Example
+
+```
+{%
+   include-markdown "docs/includes/header.md"
+   start="<!--table-start-->"
+   end="<!--table-end-->"
+   rewrite_relative_urls=false
+%}
+```
+
 #### **`include`**
 
 Includes the content of a file.
diff --git a/mkdocs_include_markdown_plugin/event.py b/mkdocs_include_markdown_plugin/event.py
index db21766..6e55f29 100644
--- a/mkdocs_include_markdown_plugin/event.py
+++ b/mkdocs_include_markdown_plugin/event.py
@@ -2,6 +2,8 @@ import html
 import re
 from pathlib import Path
 
+from mkdocs_include_markdown_plugin import process
+
 
 INCLUDE_TAG_REGEX = re.compile(
     r'''
@@ -25,6 +27,7 @@ INCLUDE_MARKDOWN_TAG_REGEX = re.compile(
         "(?P<filename>[^"]+)" # "filename"
         (?:\s+start="(?P<start>[^"]+)")? # optional start expression
         (?:\s+end="(?P<end>[^"]+)")? # optional end expression
+        (?:\s+rewrite_relative_urls=(?P<rewrite_relative_urls>\w*))? # option
         \s*
         %} # closing tag
     ''',
@@ -56,6 +59,18 @@ def _on_page_markdown(markdown, page, **kwargs):
         start = match.group('start')
         end = match.group('end')
 
+        option_value = match.group('rewrite_relative_urls')
+        if option_value in [None, 'true']:
+            # if unspecified, default to true
+            should_rewrite_relative = True
+        elif option_value == 'false':
+            should_rewrite_relative = False
+        else:
+            raise ValueError(
+                'Unknown value for rewrite_relative_urls. Possible values '
+                'are: true, false'
+            )
+
         file_path_abs = Path(page_src_path).parent / filename
 
         if not file_path_abs.exists():
@@ -69,6 +84,13 @@ def _on_page_markdown(markdown, page, **kwargs):
         if end:
             text_to_include, _, _ = text_to_include.partition(end)
 
+        if should_rewrite_relative:
+            text_to_include = process.rewrite_relative_urls(
+                text_to_include,
+                source_path=file_path_abs,
+                destination_path=Path(page_src_path),
+            )
+
         return (
             '<!-- BEGIN INCLUDE %s %s %s -->\n' % (
                 filename, html.escape(start or ''), html.escape(end or '')
diff --git a/mkdocs_include_markdown_plugin/process.py b/mkdocs_include_markdown_plugin/process.py
new file mode 100644
index 0000000..7cd868e
--- /dev/null
+++ b/mkdocs_include_markdown_plugin/process.py
@@ -0,0 +1,147 @@
+import os
+import re
+from functools import partial
+from pathlib import Path
+from urllib.parse import urlparse, urlunparse
+
+
+# Markdown regular expressions. Taken from the original Markdown.pl by John
+# Gruber, and modified to work in Python
+
+# Matches markdown links.
+# e.g. [scikit-learn](https://github.com/scikit-learn/scikit-learn)
+MARKDOWN_LINK_REGEX = re.compile(
+    r'''
+        (                 # wrap whole match in $1
+          (?<!!)          # don't match images - negative lookbehind
+          \[
+            (             # link text = $2
+                [^\[\]]*  # not bracket
+                (?:
+                    \[[^\[\]]+\]  # another level of nested bracket with
+                                  # something inside
+                    [^\[\]]*      # not bracket
+                )*
+            )
+          \]
+          \(             # literal paren
+            [ \t]*
+            <?(.*?)>?    # href = $3
+            [ \t]*
+            (            # $4
+              (['"])     # quote char = $5
+              (.*?)      # Title = $6
+              \5         # matching quote
+            )?           # title is optional
+          \)
+        )
+    ''',
+    flags=re.VERBOSE,
+)
+
+# Matches markdown inline images.
+# e.g. ![alt-text](path/to/image.png)
+MARKDOWN_IMAGE_REGEX = re.compile(
+    r'''
+        (                # wrap whole match in $1
+          !\[
+            (.*?)        # alt text = $2
+          \]
+          \(             # literal paren
+            [ \t]*
+            <?(\S+?)>?   # src url = $3
+            [ \t]*
+            (            # $4
+              (['"])     # quote char = $5
+              (.*?)      # title = $6
+              \5         # matching quote
+              [ \t]*
+            )?           # title is optional
+          \)
+        )
+    ''',
+    flags=re.VERBOSE,
+)
+
+# Matches markdown link definitions.
+# e.g. [scikit-learn]: https://github.com/scikit-learn/scikit-learn
+MARKDOWN_LINK_DEFINITION_REGEX = re.compile(
+    r'''
+        ^[ ]{0,4}\[(.+)\]:   # id = $1
+        [ \t]*
+        \n?                # maybe *one* newline
+        [ \t]*
+        <?(\S+?)>?           # url = $2
+        [ \t]*
+        \n?                # maybe one newline
+        [ \t]*
+        (?:
+            (?<=\s)          # lookbehind for whitespace
+            ["(]
+            (.+?)            # title = $3
+            [")]
+            [ \t]*
+        )?                   # title is optional
+        (?:\n+|\Z)
+    ''',
+    flags=re.VERBOSE | re.MULTILINE,
+)
+
+
+def rewrite_relative_urls(
+    markdown: str, source_path: Path, destination_path: Path
+) -> str:
+    '''
+      Rewrites markdown so that relative links that were written at
+      source_path will still work when inserted into a file at destination_path
+    '''
+    def rewrite_url(url: str) -> str:
+        scheme, netloc, path, params, query, fragment = urlparse(url)
+
+        is_absolute = path.startswith('/')
+        if is_absolute:
+            return url
+
+        trailing_slash = path.endswith('/')
+
+        path = os.path.relpath(source_path.parent / path,
+                               destination_path.parent)
+
+        # ensure forward slashes are used, on Windows
+        path = Path(path).as_posix()
+
+        if trailing_slash:
+            # the above operation removes a trailing slash. Add it back if it
+            # was present in the input
+            path = path + '/'
+
+        return urlunparse((scheme, netloc, path, params, query, fragment))
+
+    def found_href(m: 're.Match', url_group_index=-1) -> str:
+        match_start, match_end = m.span(0)
+        href = m.group(url_group_index)
+        href_start, href_end = m.span(url_group_index)
+        rewritten_url = rewrite_url(href)
+        return (
+            m.string[match_start:href_start]
+            + rewritten_url
+            + m.string[href_end:match_end]
+        )
+
+    markdown = re.sub(
+        MARKDOWN_LINK_REGEX,
+        partial(found_href, url_group_index=3),
+        markdown
+    )
+    markdown = re.sub(
+        MARKDOWN_IMAGE_REGEX,
+        partial(found_href, url_group_index=3),
+        markdown
+    )
+    markdown = re.sub(
+        MARKDOWN_LINK_DEFINITION_REGEX,
+        partial(found_href, url_group_index=2),
+        markdown
+    )
+
+    return markdown


Test patch:
diff --git a/tests/test_include_markdown.py b/tests/test_include_markdown.py
index d69878d..f7ef8c1 100644
--- a/tests/test_include_markdown.py
+++ b/tests/test_include_markdown.py
@@ -1,5 +1,6 @@
 import os
 import tempfile
+import textwrap
 
 import pytest
 
@@ -133,3 +134,91 @@ end="<!--end-here-->"
 
         with pytest.raises(ValueError):
             _on_page_markdown(page_content, page(f_includer.name))
+
+
+@pytest.mark.parametrize('should_rewrite', [True, False, None])
+def test_include_markdown_relative_rewrite(page, tmp_path, should_rewrite):
+    option_value = {
+        True: 'rewrite_relative_urls=true',
+        False: 'rewrite_relative_urls=false',
+        None: '',
+    }[should_rewrite]
+
+    includer_path = tmp_path / 'includer.md'
+    includer_path.write_text(textwrap.dedent(f'''
+        # Heading
+
+        {{%
+            include-markdown "docs/page.md"
+            start="<!--start-here-->"
+            end="<!--end-here-->"
+            {option_value}
+        %}}
+    '''))
+
+    (tmp_path / 'docs').mkdir()
+    included_file_path = tmp_path / 'docs' / 'page.md'
+    included_file_path.write_text(textwrap.dedent('''
+        # Subpage Heading
+        <!--start-here-->
+        Here's [a link](page2.md) and here's an image: ![](image.png)
+
+        Here's a [reference link][ref-link].
+
+        [ref-link]: page3.md
+        <!--end-here-->
+    '''))
+
+    output = _on_page_markdown(
+        includer_path.read_text(),
+        page(str(includer_path))
+    )
+
+    if should_rewrite in [True, None]:
+        assert output == textwrap.dedent('''
+            # Heading
+
+            <!-- BEGIN INCLUDE docs/page.md &lt;!--start-here--&gt; &lt;!--end-here--&gt; -->
+
+            Here's [a link](docs/page2.md) and here's an image: ![](docs/image.png)
+
+            Here's a [reference link][ref-link].
+
+            [ref-link]: docs/page3.md
+
+            <!-- END INCLUDE -->
+        ''')  # noqa: E501
+    else:
+        # include without rewriting
+        assert output == textwrap.dedent('''
+            # Heading
+
+            <!-- BEGIN INCLUDE docs/page.md &lt;!--start-here--&gt; &lt;!--end-here--&gt; -->
+
+            Here's [a link](page2.md) and here's an image: ![](image.png)
+
+            Here's a [reference link][ref-link].
+
+            [ref-link]: page3.md
+
+            <!-- END INCLUDE -->
+        ''')  # noqa: E501
+
+
+def test_include_markdown_relative_rewrite_invalid_option(page):
+    page_content = textwrap.dedent('''
+        # Header
+
+        {%
+            include-markdown "subfile.md"
+            rewrite_relative_urls=invalidoption
+        %}
+    ''')
+
+    with pytest.raises(ValueError) as excinfo:
+        _on_page_markdown(
+            page_content,
+            page('page.md'),
+        )
+
+    assert "Unknown value for rewrite_relative_urls" in str(excinfo.value)
diff --git a/tests/test_process.py b/tests/test_process.py
new file mode 100644
index 0000000..9eff58d
--- /dev/null
+++ b/tests/test_process.py
@@ -0,0 +1,93 @@
+import textwrap
+from pathlib import Path
+
+from mkdocs_include_markdown_plugin.process import rewrite_relative_urls
+
+
+def test_relative_link():
+    input = '''
+        Here's a [link](CHANGELOG.md) to the changelog.
+    '''
+    output = rewrite_relative_urls(input, Path('README'), Path('docs/nav.md'))
+    assert output == '''
+        Here's a [link](../CHANGELOG.md) to the changelog.
+    '''
+
+
+def test_multiline_link():
+    input = '''
+        Here's a [link whose text is really long and so is broken across
+        multiple lines](CHANGELOG.md) to the changelog.
+    '''
+    output = rewrite_relative_urls(input, Path('README'), Path('docs/nav.md'))
+    assert output == '''
+        Here's a [link whose text is really long and so is broken across
+        multiple lines](../CHANGELOG.md) to the changelog.
+    '''
+
+
+def test_relative_link_down():
+    input = '''
+        Check [this link](foobar.md) for more information
+    '''
+    output = rewrite_relative_urls(
+        input,
+        Path('docs/includes/feature_a/index.md'),
+        Path('docs/setup.md')
+    )
+    assert output == '''
+        Check [this link](includes/feature_a/foobar.md) for more information
+    '''
+
+
+def test_link_with_hash():
+    input = '''
+        Here's a [link](CHANGELOG.md#v1.2.3) to the changelog.
+    '''
+    output = rewrite_relative_urls(input, Path('README'), Path('docs/nav.md'))
+    assert output == '''
+        Here's a [link](../CHANGELOG.md#v1.2.3) to the changelog.
+    '''
+
+
+def test_link_reference():
+    input = textwrap.dedent('''
+        Here's a [link][changelog] to the changelog.
+
+        [changelog]: CHANGELOG.md
+    ''')
+    output = rewrite_relative_urls(input, Path('README'), Path('docs/nav.md'))
+    assert output == textwrap.dedent('''
+        Here's a [link][changelog] to the changelog.
+
+        [changelog]: ../CHANGELOG.md
+    ''')
+
+
+def test_image():
+    input = '''
+        Here's a diagram: ![diagram](assets/diagram.png)
+    '''
+    output = rewrite_relative_urls(input, Path('README'), Path('docs/home.md'))
+    assert output == '''
+        Here's a diagram: ![diagram](../assets/diagram.png)
+    '''
+
+
+def test_image_inside_link():
+    input = '''
+        Build status: [![Build Status](badge.png)](build/)
+    '''
+    output = rewrite_relative_urls(input, Path('README'), Path('docs/home.md'))
+    assert output == '''
+        Build status: [![Build Status](../badge.png)](../build/)
+    '''
+
+
+def test_dont_touch_absolute_urls():
+    input = '''
+        [Homepage](/) [Github](https://github.com/user/repo)
+        [Privacy policy](/privacy)
+    '''
+    output = rewrite_relative_urls(input, Path('README'), Path('docs/nav.md'))
+    assert output == input