Repository: facelessuser/pymdown-extensions

Issue:
Snippets : warn against file or part changes through checksum
### Description

Issue #1462, closed not long ago, added a feature to allow including only specified line ranges from external file with snippets. 
One concern regarding this feature was inconsistency it could create if file changed (e.g if lines were added/removed, creating an offset).  
I think a checksum mechanism could be used to warn against changes, and it isn't limited to range inclusion, but could also benefit general snippets.

## Context

I use pymdownx through Material for Mkdocs to build a computer science course. The course is for absolute beginners, and quite young. For this reason, there are heavy constraints :
 
1. I can't give them full code if the file is large, otherwise they get confused, as for example if tests are included.
2. I often need to explain the code in text, and thus make references to lines.
3. I refer to some snippets at different places, for example exercises are often referred to in the course and in an "exercises" part of the doc.

Range inclusion solves problem 1, but inconsistency is still a danger here, as even in file fully included, an inconsistency in lines numbers can confuse the students. 

I think a lot of tutorials and docs have this problems to a certain extent, and it would be beneficial to have a change warning mechanism.

### Benefits

Protects against possible inconsistency when a file included through snippets in the docs changes, makes easier to structure large docs with multiple references to the same file, or with references to out-of-docs files, as test cases.

### Solution Idea


Here is a draw at a possible solution : 

Through the inclusion syntax, snippets could allow user to provide, at the end of file name chain, a checksum between bracket. So for example : 

``somefile::4[EA8A25E440]``

Then snippet extension would check this checksum against either : 
- The entire file (likely easier to implement, but a slight bit less practical, yet sufficient i think)
- The snippet included (more practical, but maybe more difficult to implement) 

In case a checksum isn't matching, a warning message would be issued, mentioning both checksum : 

```
Inclusion somefile::4[EA8A25E440] has mismatching checksum for somefile::4 : expected EA8A25E440, yet found EA5E4408A2.
```

This way, it is simple for user to update checksum after checking file content. Warning would likely be failing the build, or maybe an option in config could allow to choose whether warn or fail.

An empty checksum `[]` would be considered as a checksum as well, always failing, simply to allow user to use snippets to compute checksum.

Discussion:
facelessuser: @gir-bot lgtm

Patch:
diff --git a/docs/src/markdown/about/changelog.md b/docs/src/markdown/about/changelog.md
index 1454dd66..a30a20c7 100644
--- a/docs/src/markdown/about/changelog.md
+++ b/docs/src/markdown/about/changelog.md
@@ -4,6 +4,7 @@
 
 - **NEW**: Tabbed: Add new syntax to allow forcing a specific tab to be select by default.
 - **NEW**: Snippets: Add new option to pass arbitrary HTTP headers.
+- **NEW**: Snippets: Allow specifying sections in a snippet and including just the specified section.
 
 ## 9.6
 
diff --git a/docs/src/markdown/extensions/snippets.md b/docs/src/markdown/extensions/snippets.md
index d21e38e4..965362f2 100644
--- a/docs/src/markdown/extensions/snippets.md
+++ b/docs/src/markdown/extensions/snippets.md
@@ -120,6 +120,44 @@ include.md::3
 ;--8<--
 ```
 
+### Snippet Sections
+
+!!! new "New 9.7"
+
+Specifying snippet lines may not always be as the source could change moving, adding, and/or removing lines. A way
+around this is to partition a snippet into named sections and then targeting a specific section to be included instead
+of specific line numbers.
+
+Snippet sections can be specified by surrounding a block of text with `--8<-- [start:name]` and `--8<-- [end:name]`.
+Then a file can simply specify the snippet using the name instead of line numbers.
+
+```
+;--8<-- "include.md:name"
+
+;--8<--
+include.md:name
+;--8<--
+```
+
+Unlike other snippet syntax, the section start and end syntax do not have to be on a line by themselves. This allows
+you to embed them in comments depending on the file type. When a section is included, the line with the start and end
+are always omitted.
+
+If we wanted to include a function from a Python source, we could specify the snippet as follows:
+
+```python
+# --8<-- [start:func]
+def my_function(var):
+    pass
+# --8<-- [end:func]
+```
+
+And then just include it in our document:
+
+```
+;--8<-- "example.py:func"
+```
+
 ### Escaping Snippets Notation
 
 !!! new "New 9.6"
diff --git a/pymdownx/snippets.py b/pymdownx/snippets.py
index 67f4a3ad..24dd6f0d 100644
--- a/pymdownx/snippets.py
+++ b/pymdownx/snippets.py
@@ -63,7 +63,16 @@ class SnippetPreprocessor(Preprocessor):
         '''
     )
 
-    RE_SNIPPET_FILE = re.compile(r'(.*?)(:[0-9]*)?(:[0-9]*)?$')
+    RE_SNIPPET_SECTION = re.compile(
+        r'''(?xi)
+        ^.*?
+        (?P<inline_marker>-{2,}8<-{2,}[ \t]+)
+        \[[ \t]*(?P<type>start|end)[ \t]*:[ \t]*(?P<name>[a-z][0-9a-z]*)[ \t]*\]
+        .*?$
+        '''
+    )
+
+    RE_SNIPPET_FILE = re.compile(r'(?i)(.*?)(?:(:[0-9]*)?(:[0-9]*)?|(:[a-z][0-9a-z]*)?)$')
 
     def __init__(self, config, md):
         """Initialize."""
@@ -82,6 +91,37 @@ class SnippetPreprocessor(Preprocessor):
         self.tab_length = md.tab_length
         super(SnippetPreprocessor, self).__init__()
 
+    def extract_section(self, section, lines):
+        """Extract the specified section from the lines."""
+
+        new_lines = []
+        start = False
+        for l in lines:
+
+            # Found a snippet section marker with our specified name
+            m = self.RE_SNIPPET_SECTION.match(l)
+            if m is not None and m.group('name') == section:
+
+                # We found the start
+                if not start and m.group('type') == 'start':
+                    start = True
+                    continue
+
+                # We found the end
+                elif start and m.group('type') == 'end':
+                    start = False
+                    break
+
+                # We found an end, but no start
+                else:
+                    return []
+
+            # We are currently in a section, so append the line
+            if start:
+                new_lines.append(l)
+
+        return new_lines
+
     def get_snippet_path(self, path):
         """Get snippet path."""
 
@@ -135,7 +175,7 @@ class SnippetPreprocessor(Preprocessor):
                 return ['']
 
             # Process lines
-            return [l.decode(self.encoding) for l in response.readlines()]
+            return [l.decode(self.encoding).rstrip('\r\n') for l in response.readlines()]
 
     def parse_snippets(self, lines, file_name=None, is_url=False):
         """Parse snippets snippet."""
@@ -201,6 +241,7 @@ class SnippetPreprocessor(Preprocessor):
                 # Get line numbers (if specified)
                 end = None
                 start = None
+                section = None
                 m = self.RE_SNIPPET_FILE.match(path)
                 path = m.group(1).strip()
                 # Looks like we have an empty file and only lines specified
@@ -215,6 +256,9 @@ class SnippetPreprocessor(Preprocessor):
                 starting = m.group(2)
                 if starting and len(starting) > 1:
                     start = max(1, int(starting[1:]) - 1)
+                section_name = m.group(4)
+                if section_name:
+                    section = section_name[1:]
 
                 # Ignore path links if we are in external, downloaded content
                 is_link = path.lower().startswith(('https://', 'http://'))
@@ -235,10 +279,12 @@ class SnippetPreprocessor(Preprocessor):
                     if not url:
                         # Read file content
                         with codecs.open(snippet, 'r', encoding=self.encoding) as f:
-                            s_lines = [l for l in f]
+                            s_lines = [l.rstrip('\r\n') for l in f]
                             if start is not None or end is not None:
                                 s = slice(start, end)
                                 s_lines = s_lines[s]
+                            elif section:
+                                s_lines = self.extract_section(section, s_lines)
                     else:
                         # Read URL content
                         try:
@@ -246,6 +292,8 @@ class SnippetPreprocessor(Preprocessor):
                             if start is not None or end is not None:
                                 s = slice(start, end)
                                 s_lines = s_lines[s]
+                            elif section:
+                                s_lines = self.extract_section(section, s_lines)
                         except SnippetMissingError:
                             if self.check_paths:
                                 raise
@@ -255,7 +303,7 @@ class SnippetPreprocessor(Preprocessor):
                     new_lines.extend(
                         [
                             space + l2 for l2 in self.parse_snippets(
-                                [l.rstrip('\r\n') for l in s_lines],
+                                s_lines,
                                 snippet,
                                 is_url=url
                             )


Test patch:
diff --git a/tests/test_extensions/_snippets/section.txt b/tests/test_extensions/_snippets/section.txt
new file mode 100644
index 00000000..8a05e940
--- /dev/null
+++ b/tests/test_extensions/_snippets/section.txt
@@ -0,0 +1,19 @@
+/* --8<-- [start: cssSection] */
+div {
+    color: red;
+}
+/* --8<-- [end: cssSection] */
+
+<!-- --8<-- [start: htmlSection] -->
+<div><p>content</p></div>
+<!-- --8<-- [end: htmlSection] -->
+
+/* --8<-- [end: cssSection2] */
+/* --8<-- [start: cssSection2] */
+div {
+    color: red;
+}
+/* --8<-- [end: cssSection2] */
+
+<!-- --8<-- [start: htmlSection2] -->
+<div><p>content</p></div>
diff --git a/tests/test_extensions/test_snippets.py b/tests/test_extensions/test_snippets.py
index 8687e540..5261b2d3 100644
--- a/tests/test_extensions/test_snippets.py
+++ b/tests/test_extensions/test_snippets.py
@@ -11,7 +11,7 @@ class TestSnippets(util.MdCase):
     """Test snippet cases."""
 
     extension = [
-        'pymdownx.snippets',
+        'pymdownx.snippets', 'pymdownx.superfences'
     ]
 
     extension_configs = {
@@ -223,6 +223,68 @@ class TestSnippets(util.MdCase):
             True
         )
 
+    def test_section_inline(self):
+        """Test section partial in inline snippet."""
+
+        self.check_markdown(
+            R'''
+            ```
+            --8<-- "section.txt:cssSection"
+            ```
+            ''',
+            '''
+            <div class="highlight"><pre><span></span><code>div {
+                color: red;
+            }
+            </code></pre></div>
+            ''',
+            True
+        )
+
+    def test_section_block(self):
+        """Test section partial in inline snippet."""
+
+        self.check_markdown(
+            R'''
+            --8<--
+            section.txt:htmlSection
+            --8<--
+            ''',
+            '''
+            <div><p>content</p></div>
+            ''',
+            True
+        )
+
+    def test_section_end_first(self):
+        """Test section when the end is specified first."""
+
+        self.check_markdown(
+            R'''
+            --8<--
+            section.txt:cssSection2
+            --8<--
+            ''',
+            '''
+            ''',
+            True
+        )
+
+    def test_section_no_end(self):
+        """Test section when the end is not specified."""
+
+        self.check_markdown(
+            R'''
+            --8<--
+            section.txt:htmlSection2
+            --8<--
+            ''',
+            '''
+            <div><p>content</p></div>
+            ''',
+            True
+        )
+
 
 class TestSnippetsFile(util.MdCase):
     """Test snippet file case."""
@@ -502,7 +564,7 @@ class TestURLSnippets(util.MdCase):
 
     @patch('urllib.request.urlopen')
     def test_url_lines(self, mock_urlopen):
-        """Test nested file in URL."""
+        """Test specifying specific lines in a URL."""
 
         content = []
         length = 0
@@ -515,7 +577,7 @@ class TestURLSnippets(util.MdCase):
         cm.status = 200
         cm.code = 200
         cm.readlines.return_value = content
-        cm.headers = {'content-length': '183'}
+        cm.headers = {'content-length': length}
         cm.__enter__.return_value = cm
         mock_urlopen.return_value = cm
 
@@ -615,6 +677,35 @@ class TestURLSnippets(util.MdCase):
             True
         )
 
+    @patch('urllib.request.urlopen')
+    def test_url_sections(self, mock_urlopen):
+        """Test specifying a section in a URL."""
+
+        content = []
+        length = 0
+        with open('tests/test_extensions/_snippets/section.txt', 'rb') as f:
+            for l in f:
+                length += len(l)
+                content.append(l)
+
+        cm = MagicMock()
+        cm.status = 200
+        cm.code = 200
+        cm.readlines.return_value = content
+        cm.headers = {'content-length': length}
+        cm.__enter__.return_value = cm
+        mock_urlopen.return_value = cm
+
+        self.check_markdown(
+            R'''
+            --8<-- "https://test.com/myfile.md:htmlSection"
+            ''',
+            '''
+            <div><p>content</p></div>
+            ''',
+            True
+        )
+
 
 class TestURLSnippetsNoMax(util.MdCase):
     """Test snippet URL cases no max size."""