Repository: causify-ai/helpers
Pull request #481: Helpers task293 link does not exist when it points to a section
Author: login: sandeepthalapanane
Created: 2025-04-02T20:22:57Z | Merged: 2025-04-04T19:33:00Z

Updated `amp_fix_md_links.py` to correctly handle markdown links that include section references #293. 
```
'/docs/coding/all.coding_style.how_to_guide.md#logging' does not exist [fix_md_links]
```
### Changes
- Updated the linter to:
  - Split the link on the `#` symbol.
  - Check if the base file exists.
  - If the file exists, verify if the section exists and raise a **warning** if it's missing.

- Added corresponding unit tests in `./linters/test/test_amp_fix_md_links.py`:
  - Created a reference markdown file and a test markdown file.
  - Validated reference file and section existence using the test markdown file.


Diff:
diff --git a/linters/amp_fix_md_links.py b/linters/amp_fix_md_links.py
index d481f6071..bbc573651 100644
--- a/linters/amp_fix_md_links.py
+++ b/linters/amp_fix_md_links.py
@@ -14,6 +14,7 @@
 import helpers.hdbg as hdbg
 import helpers.hgit as hgit
 import helpers.hio as hio
+import helpers.hmarkdown as hmarkdo
 import helpers.hparser as hparser
 import helpers.hstring as hstring
 import linters.action as liaction
@@ -63,6 +64,29 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_header_exists(
+    markdown_link_path: str, header: str, level: int = 6
+) -> bool:
+    """
+    Check if a header exists in the markdown file.
+
+    :param markdown_link_path: the path to the Markdown file in which the header will be looked up
+    :param header: the heading text to look for in the file, e.g., `test`, `test-two`
+    :param level: the maximum depth of headers to extract (Markdown supports levels 1 to 6)
+        - E.g., level 2 matches only `##` and `#` headers, not `###` or deeper
+    :return: True if the header is found, False if not found
+    """
+    with open(markdown_link_path, "r", encoding="utf-8") as file:
+        content = file.read()
+    # Get the headers of the markdown file.
+    headers_md = hmarkdo.extract_headers_from_markdown(content, level)
+    # Replace '-' with a white space.
+    header = header.replace("-", " ").lower()
+    # Check if the header matches any extracted header of the markdown file.
+    found = any(header == h.description.lower() for h in headers_md)
+    return found
+
+
 def _check_md_link_format(
     link_text: str, link: str, line: str, file_name: str, line_num: int
 ) -> Tuple[str, List[str]]:
@@ -121,11 +145,19 @@ def _check_md_link_format(
     # Replace the link in the line with its updated version.
     new_link_txt = f"[{link_text}]({link})"
     updated_line = line.replace(old_link_txt, new_link_txt)
-    # Check that the file referenced by the link exists.
-    link_in_cur_module = _make_path_module_agnostic(link)
+    # Split the link into file path and header using the '#' delimiter.
+    link_path, _, header = link.partition("#")
+    link_in_cur_module = _make_path_module_agnostic(link_path)
     if not os.path.exists(link_in_cur_module):
-        msg = f"{file_name}:{line_num}: '{link}' does not exist"
+        # Warn that the file referenced by the link does not exist.
+        msg = f"{file_name}:{line_num}: '{link_path}' does not exist"
         warnings.append(msg)
+    elif header:
+        # Check if the header referenced by the link exists.
+        header_exists = _check_md_header_exists(link_in_cur_module, header)
+        if not header_exists:
+            msg = f"{file_name}:{line_num}: Header '{header}' does not exist in '{link_path}'"
+            warnings.append(msg)
     return updated_line, warnings
 
 
diff --git a/linters/test/outcomes/Test_fix_links.test5/output/test.txt b/linters/test/outcomes/Test_fix_links.test5/output/test.txt
new file mode 100644
index 000000000..0b2a15116
--- /dev/null
+++ b/linters/test/outcomes/Test_fix_links.test5/output/test.txt
@@ -0,0 +1,15 @@
+# linter warnings
+
+/app/linters/test/outcomes/Test_fix_links.test5/tmp.scratch/valid_header_test.md:3: '/docs/markdown_exam.md' does not exist
+/app/linters/test/outcomes/Test_fix_links.test5/tmp.scratch/valid_header_test.md:5: Header 'introduce' does not exist in '/linters/test/outcomes/Test_fix_links.test5/tmp.scratch/reference.md'
+
+# linted file
+
+
+    Markdown link: [Valid Markdown and header Link](/linters/test/outcomes/Test_fix_links.test5/tmp.scratch/reference.md#introduction)
+
+    Markdown link: [InValid Markdown Link](/docs/markdown_exam.md#introduction)
+
+    Markdown link: [Invalid header in the Markdown Link](/linters/test/outcomes/Test_fix_links.test5/tmp.scratch/reference.md#introduce)
+
+    Markdown link: [Valid Markdown and header Link](/linters/test/outcomes/Test_fix_links.test5/tmp.scratch/reference.md#hyphen-test)
\ No newline at end of file
diff --git a/linters/test/test_amp_fix_md_links.py b/linters/test/test_amp_fix_md_links.py
index af4e0b7e3..9f26e6613 100644
--- a/linters/test/test_amp_fix_md_links.py
+++ b/linters/test/test_amp_fix_md_links.py
@@ -101,7 +101,7 @@ def test3(self) -> None:
         )
         self.check_string(output)
 
-    def test4(self):
+    def test4(self) -> None:
         """
         Test links with a filepath with a tag ("/image.png") to check for its
         preservation.
@@ -127,6 +127,52 @@ def test4(self):
         """
         self.assert_equal(actual, expected, fuzzy_match=True)
 
+    def test5(self) -> None:
+        """
+        Test Markdown file references to another Markdown file and its headers.
+        """
+        reference_file_md_content = """
+# Reference test file
+
+- [Introduction](#introduction)
+- [Hyphen test](#hyphen-test)
+
+## Introduction
+
+A test header with one word in the reference file.
+
+## Hyphen test
+
+A test to check two words header in the reference file.
+        """
+        reference_file_name = "reference.md"
+        reference_file_link = self._write_input_file(
+            reference_file_md_content, reference_file_name
+        )
+        # Remove '/app' prefix from the file path.
+        reference_file_link = reference_file_link.removeprefix("/app")
+        test_md_content = f"""
+    Markdown link: [Valid Markdown and header Link]({reference_file_link}#introduction)
+
+    Markdown link: [InValid Markdown Link](docs/markdown_exam.md#introduction)
+
+    Markdown link: [Invalid header in the Markdown Link]({reference_file_link}#introduce)
+
+    Markdown link: [Valid Markdown and header Link]({reference_file_link}#hyphen-test)
+        """
+        test_file_name = "valid_header_test.md"
+        test_file_link = self._write_input_file(test_md_content, test_file_name)
+        # Run.
+        _, updated_lines, out_warnings = lafimdli.fix_links(test_file_link)
+        # Check.
+        output = "\n".join(
+            ["# linter warnings", ""]
+            + out_warnings
+            + ["", "# linted file", ""]
+            + updated_lines
+        )
+        self.check_string(output)
+
     def _get_txt_with_incorrect_links(self) -> str:
         txt_incorrect = r"""
 - Markdown-style link with a text label


Reviews:
user: login: gpsaggese
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-03T16:48:27Z
user: login: sonniki
body: Very good work 
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-03T17:18:17Z
user: login: sandeepthalapanane
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-03T18:05:35Z
user: login: sonniki
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-03T18:08:43Z
user: login: sandeepthalapanane
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-03T20:17:04Z
user: login: sonniki
body: The content looks good, just some style things
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-04T08:15:18Z
user: login: sandeepthalapanane
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-04T16:03:37Z
user: login: sonniki
body: All good, just need to resolve the conflict (another PR on the same file was merged in the meantime) and then we'll merge
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-04T19:17:56Z
user: login: sonniki
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-04T19:32:54Z

Review comments:
diff_hunk: @@ -121,11 +153,19 @@ def _check_md_link_format(
     # Replace the link in the line with its updated version.
     new_link_txt = f"[{link_text}]({link})"
     updated_line = line.replace(old_link_txt, new_link_txt)
-    # Check that the file referenced by the link exists.
-    link_in_cur_module = _make_path_module_agnostic(link)
+    link_path, _, section = link.partition("#")
+    link_in_cur_module = _make_path_module_agnostic(link_path)
     if not os.path.exists(link_in_cur_module):
-        msg = f"{file_name}:{line_num}: '{link}' does not exist"
+        # Check if the file referenced by the link does not exist.
path: linters/amp_fix_md_links.py
user: login: sonniki
body: "Check if" -> "Warn that"
created_at: 2025-04-03T16:35:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -121,11 +153,19 @@ def _check_md_link_format(
     # Replace the link in the line with its updated version.
     new_link_txt = f"[{link_text}]({link})"
     updated_line = line.replace(old_link_txt, new_link_txt)
-    # Check that the file referenced by the link exists.
-    link_in_cur_module = _make_path_module_agnostic(link)
+    link_path, _, section = link.partition("#")
+    link_in_cur_module = _make_path_module_agnostic(link_path)
     if not os.path.exists(link_in_cur_module):
-        msg = f"{file_name}:{line_num}: '{link}' does not exist"
+        # Check if the file referenced by the link does not exist.
+        msg = f"{file_name}:{line_num}: '{link_path}' does not exist"
         warnings.append(msg)
+    elif section:
+        # Check that the section referenced by the link exists.
path: linters/amp_fix_md_links.py
user: login: sonniki
body: "Check that" -> "Check if"
created_at: 2025-04-03T16:38:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
+    """
+    Check if a section or heading exists in the markdown file.
+
+    Searches the specified file for a Markdown-style section that
+    matches the provided section name. Returns a warning message if the
+    section is not found.
path: linters/amp_fix_md_links.py
user: login: sonniki
body: Nit: I would continue using imperatives, ("Search", "Return"), similarly to the first line of the docstring -- just for continuity.
created_at: 2025-04-03T16:39:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
path: linters/amp_fix_md_links.py
user: login: gpsaggese
body: We use Optional[str], so let's do that for consistency but we should switch the code base to the new approach at some point
created_at: 2025-04-03T16:42:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
+    """
+    Check if a section or heading exists in the markdown file.
+
+    Searches the specified file for a Markdown-style section that
+    matches the provided section name. Returns a warning message if the
+    section is not found.
+
+    :param link_in_cur_module: the path to the Markdown file to check
path: linters/amp_fix_md_links.py
user: login: gpsaggese
body: -> ... in_curr_module (a bit more readable)
created_at: 2025-04-03T16:43:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
+    """
+    Check if a section or heading exists in the markdown file.
+
+    Searches the specified file for a Markdown-style section that
+    matches the provided section name. Returns a warning message if the
+    section is not found.
+
+    :param link_in_cur_module: the path to the Markdown file to check
+    :param section: the heading text to look for in the file
path: linters/amp_fix_md_links.py
user: login: gpsaggese
body: let's make an example. E.g., does it include also the level?
created_at: 2025-04-03T16:43:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
+    """
+    Check if a section or heading exists in the markdown file.
+
+    Searches the specified file for a Markdown-style section that
+    matches the provided section name. Returns a warning message if the
+    section is not found.
+
+    :param link_in_cur_module: the path to the Markdown file to check
+    :param section: the heading text to look for in the file
+    :param file_name: the original file with the link
+    :param line_num: the number of the line in the file
+    :return: a warning message if the referenced markdown section is
+        missing, or None if found
+    """
+    with open(link_in_cur_module, "r", encoding="utf-8") as file:
+        content = file.read()
+    section_pattern = re.compile(r"\[.*?\]\((.*?)\)", re.MULTILINE)
+    matches = section_pattern.findall(content)
+    if f"#{section}" in matches:
+        # Return None if the specified section is found.
+        return None
+    warning = f"{file_name}:{line_num}: The section '{section}' does not exist in the '{link_path}'"
path: linters/amp_fix_md_links.py
user: login: gpsaggese
body: You are passing extra stuff to build the warning. But this is not competence of this function.
This function should only return yes/no, then the caller decides what to do
created_at: 2025-04-03T16:44:44Z
author_association: CONTRIBUTOR
diff_hunk: @@ -121,11 +153,19 @@ def _check_md_link_format(
     # Replace the link in the line with its updated version.
     new_link_txt = f"[{link_text}]({link})"
     updated_line = line.replace(old_link_txt, new_link_txt)
-    # Check that the file referenced by the link exists.
-    link_in_cur_module = _make_path_module_agnostic(link)
+    link_path, _, section = link.partition("#")
path: linters/amp_fix_md_links.py
user: login: gpsaggese
body: need a comment to explain the intent
created_at: 2025-04-03T16:46:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -101,6 +101,49 @@ def test3(self) -> None:
         )
         self.check_string(output)
 
+    def test4(self) -> None:
+        """
+        Test Markdown file references to another Markdown file and it's
path: linters/test/test_amp_fix_md_links.py
user: login: gpsaggese
body: -> its
created_at: 2025-04-03T16:47:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
+    """
+    Check if a section or heading exists in the markdown file.
+
+    Searches the specified file for a Markdown-style section that
+    matches the provided section name. Returns a warning message if the
+    section is not found.
+
+    :param link_in_cur_module: the path to the Markdown file to check
+    :param section: the heading text to look for in the file
+    :param file_name: the original file with the link
+    :param line_num: the number of the line in the file
+    :return: a warning message if the referenced markdown section is
+        missing, or None if found
+    """
+    with open(link_in_cur_module, "r", encoding="utf-8") as file:
+        content = file.read()
+    section_pattern = re.compile(r"\[.*?\]\((.*?)\)", re.MULTILINE)
+    matches = section_pattern.findall(content)
+    if f"#{section}" in matches:
path: linters/amp_fix_md_links.py
user: login: sonniki
body: Do I understand correctly that you are looking for links with the form `[...](#section_name)`? So presumably, from the table of contents? This contains a risk that if the section _is_ in the markdown but the table of contents _isn't_, then we are going to say that the markdown doesn't have the section, correct? Or the same if there _is_ a table of contents but it's not updated and therefore doesn't include the section we are looking for.

Of course, in an ideal world, all Markdowns are linted, and Linter does add/update the table of contents automatically. But for the real world, we might want to make this search more robust.
created_at: 2025-04-03T17:09:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
+    """
+    Check if a section or heading exists in the markdown file.
+
+    Searches the specified file for a Markdown-style section that
+    matches the provided section name. Returns a warning message if the
+    section is not found.
+
+    :param link_in_cur_module: the path to the Markdown file to check
+    :param section: the heading text to look for in the file
+    :param file_name: the original file with the link
path: linters/amp_fix_md_links.py
user: login: sonniki
body: `link_path` is missing from the param descriptions. But I agree with GP's comment, it might be cleaner if this function just returns True or False about whether or not the section is in the file, and the message is constructed downstream (and then we don't need these params)
created_at: 2025-04-03T17:12:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -101,6 +101,49 @@ def test3(self) -> None:
         )
         self.check_string(output)
 
+    def test4(self) -> None:
+        """
+        Test Markdown file references to another Markdown file and it's
+        section.
+        """
+        reference_file_md_content = """
+# Reference test file
+
+- [Introduction](#introduction)
+- [Usage](#usage)
+
+## Introduction
+
+A test introduction in the reference file.
+
+## Usage
+        """
+        reference_file_name = "reference.md"
+        reference_file_link = self._write_input_file(
+            reference_file_md_content, reference_file_name
+        )
+        # Remove '/app' prefix from the file path
path: linters/test/test_amp_fix_md_links.py
user: login: sonniki
body: Missing `.` at the end of the comment
created_at: 2025-04-03T17:14:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
+    """
+    Check if a section or heading exists in the markdown file.
+
+    Searches the specified file for a Markdown-style section that
+    matches the provided section name. Returns a warning message if the
+    section is not found.
+
+    :param link_in_cur_module: the path to the Markdown file to check
+    :param section: the heading text to look for in the file
+    :param file_name: the original file with the link
+    :param line_num: the number of the line in the file
+    :return: a warning message if the referenced markdown section is
+        missing, or None if found
+    """
+    with open(link_in_cur_module, "r", encoding="utf-8") as file:
+        content = file.read()
+    section_pattern = re.compile(r"\[.*?\]\((.*?)\)", re.MULTILINE)
+    matches = section_pattern.findall(content)
+    if f"#{section}" in matches:
+        # Return None if the specified section is found.
+        return None
+    warning = f"{file_name}:{line_num}: The section '{section}' does not exist in the '{link_path}'"
path: linters/amp_fix_md_links.py
user: login: sonniki
body: Nit: I would remove both instances of "the", just say "Section '{section}' does not exist in '{link_path}'"
created_at: 2025-04-03T17:16:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
+    """
+    Check if a section or heading exists in the markdown file.
+
+    Searches the specified file for a Markdown-style section that
+    matches the provided section name. Returns a warning message if the
+    section is not found.
+
+    :param link_in_cur_module: the path to the Markdown file to check
+    :param section: the heading text to look for in the file
+    :param file_name: the original file with the link
+    :param line_num: the number of the line in the file
+    :return: a warning message if the referenced markdown section is
+        missing, or None if found
+    """
+    with open(link_in_cur_module, "r", encoding="utf-8") as file:
+        content = file.read()
+    section_pattern = re.compile(r"\[.*?\]\((.*?)\)", re.MULTILINE)
+    matches = section_pattern.findall(content)
+    if f"#{section}" in matches:
path: linters/amp_fix_md_links.py
user: login: sandeepthalapanane
body: Indeed, the code assumes that markdown files contain a table of contents. I could also verify the presence of headings even without a table of contents. I am considering a combined approach to check for either the table of contents or the headings.
created_at: 2025-04-03T18:05:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
+    """
+    Check if a section or heading exists in the markdown file.
+
+    Searches the specified file for a Markdown-style section that
+    matches the provided section name. Returns a warning message if the
+    section is not found.
+
+    :param link_in_cur_module: the path to the Markdown file to check
+    :param section: the heading text to look for in the file
+    :param file_name: the original file with the link
+    :param line_num: the number of the line in the file
+    :return: a warning message if the referenced markdown section is
+        missing, or None if found
+    """
+    with open(link_in_cur_module, "r", encoding="utf-8") as file:
+        content = file.read()
+    section_pattern = re.compile(r"\[.*?\]\((.*?)\)", re.MULTILINE)
+    matches = section_pattern.findall(content)
+    if f"#{section}" in matches:
path: linters/amp_fix_md_links.py
user: login: sonniki
body: SG. 
BTW `helpers/hmarkdown.py` contains a bunch of useful functions so feel free to browse for something you might use. E.g. there is `extract_headers_from_markdown()`.
created_at: 2025-04-03T18:08:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +63,38 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_section_exists(
+    link_in_cur_module: str,
+    section: str,
+    link_path: str,
+    file_name: str,
+    line_num: int,
+) -> str | None:
+    """
+    Check if a section or heading exists in the markdown file.
+
+    Searches the specified file for a Markdown-style section that
+    matches the provided section name. Returns a warning message if the
+    section is not found.
+
+    :param link_in_cur_module: the path to the Markdown file to check
+    :param section: the heading text to look for in the file
path: linters/amp_fix_md_links.py
user: login: sandeepthalapanane
body: I included the level by utilizing the functions from `helpers/hmarkdown.py`, and I also provided an example.
created_at: 2025-04-03T20:17:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +64,36 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_header_exists(
+    markdown_link_path: str, header: str, level: int = 6
+) -> bool:
+    """
+    Check if a header or heading exists in the markdown file.
+
+    Search the specified file for a Markdown-style header that
+    matches the provided header name. Return True if the header is
+    found, otherwise return False.
path: linters/amp_fix_md_links.py
user: login: sonniki
body: IMO this is not needed (it's already not difficult to understand just from the first docstring line)
created_at: 2025-04-04T07:46:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +64,36 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_header_exists(
+    markdown_link_path: str, header: str, level: int = 6
+) -> bool:
+    """
+    Check if a header or heading exists in the markdown file.
path: linters/amp_fix_md_links.py
user: login: sonniki
body: Header or heading?:) I'd say these are more or less synonyms, so you should just pick one and call it that consistently throughout
created_at: 2025-04-04T07:47:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +64,36 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_header_exists(
+    markdown_link_path: str, header: str, level: int = 6
+) -> bool:
+    """
+    Check if a header or heading exists in the markdown file.
+
+    Search the specified file for a Markdown-style header that
+    matches the provided header name. Return True if the header is
+    found, otherwise return False.
+
+    E.g., return True if a header like '#test' or '##test-two' exists in the markdown file,
+    otherwise return False.
path: linters/amp_fix_md_links.py
user: login: sonniki
body: Same as above -- not needed IMO. You can put `e.g., "#test", "#test-two"` in the param description of `header` if you want
created_at: 2025-04-04T07:49:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +64,36 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_header_exists(
+    markdown_link_path: str, header: str, level: int = 6
+) -> bool:
+    """
+    Check if a header or heading exists in the markdown file.
+
+    Search the specified file for a Markdown-style header that
+    matches the provided header name. Return True if the header is
+    found, otherwise return False.
+
+    E.g., return True if a header like '#test' or '##test-two' exists in the markdown file,
+    otherwise return False.
+
+    :param markdown_link_path: the path to the Markdown file in which the header will be looked up.
+    :param header: the heading text to look for in the file
+    :param level: the maximum depth of headers to extract (Markdown supports levels 1 to 6)
+        - "E.g., level 2 matches only '##' and '#' headers, not '###' or deeper."
+    :return: True if the header is found, False if not found.
path: linters/amp_fix_md_links.py
user: login: sonniki
body: There should be no `.` at the end of param/return descriptions
created_at: 2025-04-04T07:49:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +64,36 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_header_exists(
+    markdown_link_path: str, header: str, level: int = 6
+) -> bool:
+    """
+    Check if a header or heading exists in the markdown file.
+
+    Search the specified file for a Markdown-style header that
+    matches the provided header name. Return True if the header is
+    found, otherwise return False.
+
+    E.g., return True if a header like '#test' or '##test-two' exists in the markdown file,
+    otherwise return False.
+
+    :param markdown_link_path: the path to the Markdown file in which the header will be looked up.
+    :param header: the heading text to look for in the file
+    :param level: the maximum depth of headers to extract (Markdown supports levels 1 to 6)
+        - "E.g., level 2 matches only '##' and '#' headers, not '###' or deeper."
path: linters/amp_fix_md_links.py
user: login: sonniki
body: Why quotation marks here?
created_at: 2025-04-04T07:50:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -121,11 +152,19 @@ def _check_md_link_format(
     # Replace the link in the line with its updated version.
     new_link_txt = f"[{link_text}]({link})"
     updated_line = line.replace(old_link_txt, new_link_txt)
-    # Check that the file referenced by the link exists.
-    link_in_cur_module = _make_path_module_agnostic(link)
+    # Split the link into file path and header using the '#' delimiter if exist.
path: linters/amp_fix_md_links.py
user: login: sonniki
body: "if exist" can be removed
created_at: 2025-04-04T08:04:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -101,6 +101,52 @@ def test3(self) -> None:
         )
         self.check_string(output)
 
+    def test4(self) -> None:
+        """
+        Test Markdown file references to another Markdown file and its header.
path: linters/test/test_amp_fix_md_links.py
user: login: sonniki
body: -> headers
created_at: 2025-04-04T08:05:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +64,36 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_header_exists(
+    markdown_link_path: str, header: str, level: int = 6
+) -> bool:
+    """
+    Check if a header or heading exists in the markdown file.
+
+    Search the specified file for a Markdown-style header that
+    matches the provided header name. Return True if the header is
+    found, otherwise return False.
+
+    E.g., return True if a header like '#test' or '##test-two' exists in the markdown file,
+    otherwise return False.
+
+    :param markdown_link_path: the path to the Markdown file in which the header will be looked up.
+    :param header: the heading text to look for in the file
+    :param level: the maximum depth of headers to extract (Markdown supports levels 1 to 6)
+        - "E.g., level 2 matches only '##' and '#' headers, not '###' or deeper."
+    :return: True if the header is found, False if not found.
+    """
+    with open(markdown_link_path, "r", encoding="utf-8") as file:
+        content = file.read()
+    # Get the headers of the markdown file.
+    headers_md = hmarkdo.extract_headers_from_markdown(content, level)
+    # Replace '-' with a white space, if '-' exists.
path: linters/amp_fix_md_links.py
user: login: sonniki
body: Same here -- no need to specify "if X exists", this is usually implied
created_at: 2025-04-04T08:08:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +64,36 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_header_exists(
path: linters/amp_fix_md_links.py
user: login: sonniki
body: Can you also pls confirm that if you run Linter in this branch on a file that has links with headers, you now don't get the "file doesn't exist" warning? E.g. [this one](https://github.com/causify-ai/helpers/blob/master/docs/coding/all.submit_code_for_review.how_to_guide.md), which has this link: `[here](/docs/work_organization/all.use_github.how_to_guide.md#pr-labels)`
created_at: 2025-04-04T08:15:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -63,6 +64,36 @@ def _make_path_module_agnostic(path: str) -> str:
     return upd_path
 
 
+def _check_md_header_exists(
path: linters/amp_fix_md_links.py
user: login: sandeepthalapanane
body: I ran the command `i lint --files="docs/coding/all.submit_code_for_review.how_to_guide.md"`

Output -
```
Linting file: 'docs/coding/all.submit_code_for_review.how_to_guide.md'
////////////////////////////////////////////////////////////////////////////////
./linter_warnings.txt
////////////////////////////////////////////////////////////////////////////////
cmd line='./linters/base.py --files docs/coding/all.submit_code_for_review.how_to_guide.md --num_threads -1'
file_paths=1 ['docs/coding/all.submit_code_for_review.how_to_guide.md']
actions=24 ['add_python_init_files', 'add_toc_to_notebook', 'fix_md_links', 'lint_md', 'check_md_toc_headers', 'autoflake', 'fix_whitespaces', 'doc_formatter', 'isort', 'class_method_order', 'normalize_imports', 'format_separating_line', 'add_class_frames', 'black', 'process_jupytext', 'check_file_size', 'check_filename', 'check_merge_conflict', 'check_import', 'warn_incorrectly_formatted_todo', 'check_md_reference', 'flake8', 'pylint', 'mypy']
////////////////////////////////////////////////////////////////////////////////
11:39:35 - INFO  hdbg.py init_logger:1018                               > cmd='./dev_scripts_helpers/documentation/lint_notes.py -i docs/coding/all.submit_code_for_review.how_to_guide.md --in_place' [lint_md]
docs/coding/all.submit_code_for_review.how_to_guide.md: 'docs/coding/all.submit_code_for_review.how_to_guide.md' is not referenced in README.md [check_md_reference]
docs/coding/all.submit_code_for_review.how_to_guide.md:159: Missing syntax tag in ``` [lint_md]
docs/coding/all.submit_code_for_review.how_to_guide.md:168: Missing syntax tag in ``` [lint_md]
docs/coding/all.submit_code_for_review.how_to_guide.md:187: Missing syntax tag in ``` [lint_md]
docs/coding/all.submit_code_for_review.how_to_guide.md:192: Missing syntax tag in ``` [lint_md]
docs/coding/all.submit_code_for_review.how_to_guide.md:46: Missing syntax tag in ``` [lint_md]
docs/coding/all.submit_code_for_review.how_to_guide.md:50: Missing syntax tag in ``` [lint_md]
docs/coding/all.submit_code_for_review.how_to_guide.md:58: Missing syntax tag in ``` [lint_md]

////////////////////////////////////////////////////////////////////////////////
```

I could confirm that no warnings were raised for `/docs/work_organization/all.use_github.how_to_guide.md#pr-labels`.  
created_at: 2025-04-04T16:03:37Z
author_association: CONTRIBUTOR

PR comments:
user: login: sandeepthalapanane
created_at: 2025-04-02T20:53:23Z
body: @samarth9008 @sonniki 

I am confused about why the linter keeps failing, saying the git client is not clean despite my git being clean. Can you assist me with this?

```
git status
On branch HelpersTask293_Link_does_not_exist_when_it_points_to_a_section
Your branch is up to date with 'origin/HelpersTask293_Link_does_not_exist_when_it_points_to_a_section'.

nothing to commit, working tree clean
```
author_association: CONTRIBUTOR
user: login: sonniki
created_at: 2025-04-02T21:39:46Z
body: > @samarth9008 @sonniki
> 
> I am confused about why the linter keeps failing, saying the git client is not clean despite my git being clean. Can you assist me with this?
> 
> ```
> git status
> On branch HelpersTask293_Link_does_not_exist_when_it_points_to_a_section
> Your branch is up to date with 'origin/HelpersTask293_Link_does_not_exist_when_it_points_to_a_section'.
> 
> nothing to commit, working tree clean
> ```

How about after running `i lint` on the files listed in the failing Linter check?
author_association: CONTRIBUTOR
user: login: sandeepthalapanane
created_at: 2025-04-02T22:23:24Z
body: I am facing an error while linting the `linters/amp_fix_md_links.py`
```
Linting file: './linters/amp_fix_md_links.py'
Traceback (most recent call last):
  File "/app/./linters/base.py", line 519, in <module>
    _main(args_)
  File "/app/./linters/base.py", line 497, in _main
    lints = _run_linter(file_paths, action_names, action_classes, args)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/./linters/base.py", line 393, in _run_linter
    cur_file_lints = _lint(
                     ^^^^^^
  File "/app/./linters/base.py", line 354, in _lint
    cur_action_lints = action_class.execute(file_path, pedantic)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/linters/action.py", line 62, in execute
    output = self._execute(file_name, pedantic)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/linters/amp_normalize_import.py", line 1034, in _execute
    fixed_content = import_docstring_generator.process_file(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/linters/amp_normalize_import.py", line 799, in process_file
    hdbg.dassert_in(long_import, long_to_short_import.keys())
  File "/app/helpers/hdbg.py", line 295, in dassert_in
    _dfatal(txt, msg, *args, only_warning=only_warning)
  File "/app/helpers/hdbg.py", line 142, in _dfatal
    dfatal(dfatal_txt)
  File "/app/helpers/hdbg.py", line 71, in dfatal
    raise assertion_type(ret)
AssertionError: 
################################################################################
* Failed assertion *
'..linters.amp_fix_md_links' in 'dict_keys(['dev_scripts_helpers.old.linter.pre_pr_checklist', 'dev_scripts_helpers.old.linter.linter_master_report', 'dev_scripts_helpers.old.linter.linter', 'dev_scripts_helpers.old.linter.process_jupytext', 'dev_scripts_helpers.old.create_conda.install.print_conda_packages', 'dev_scripts_helpers.old.create_conda.install.create_conda', 'dev_scripts_helpers.old.create_conda.install.check_develop_packages', 'dev_scripts_helpers.old.create_conda._bootstrap', 'dev_scripts_helpers.old.create_conda._setenv_amp', 'dev_scripts_helpers.old.create_conda._setenv_lib', 'dev_scripts_helpers.chatgpt.run_simple_chatgpt', 'dev_scripts_helpers.chatgpt.manage_chatgpt_assistant', 'dev_scripts_helpers.chatgpt.run_chatgpt', 'dev_scripts_helpers.poetry.run_poetry_debug', 'dev_scripts_helpers.encrypt_models.encrypt_model', 'dev_scripts_helpers.system_tools.capture_notebook_cells', 'dev_scripts_helpers.system_tools.replace_text', 'dev_scripts_helpers.system_tools.save_screenshot', 'dev_scripts_helpers.system_tools.jackdoc', 'dev_scripts_helpers.system_tools.tg', 'dev_scripts_helpers.system_tools.remove_escape_chars', 'dev_scripts_helpers.system_tools.email_notify', 'dev_scripts_helpers.system_tools.ffind', 'dev_scripts_helpers.system_tools.zip_files', 'dev_scripts_helpers.documentation.dockerized_pandoc', 'dev_scripts_helpers.documentation.replace_latex', 'dev_scripts_helpers.documentation.run_pandoc', 'dev_scripts_helpers.documentation.dockerized_prettier', 'dev_scripts_helpers.documentation.convert_docx_to_markdown', 'dev_scripts_helpers.documentation.dockerized_tikz_to_bitmap', 'dev_scripts_helpers.documentation.extract_headers_from_markdown', 'dev_scripts_helpers.documentation.transform_notes', 'dev_scripts_helpers.documentation.render_images', 'dev_scripts_helpers.documentation.lint_notes', 'dev_scripts_helpers.documentation.dockerized_latex', 'dev_scripts_helpers.documentation.preprocess_notes', 'dev_scripts_helpers.documentation.notes_to_pdf', 'dev_scripts_helpers.documentation.generate_script_catalog', 'dev_scripts_helpers.documentation.generate_latex_sty', 'dev_scripts_helpers.documentation.dockerized_mermaid', 'dev_scripts_helpers.documentation.dockerized_graphviz', 'dev_scripts_helpers.documentation.publish_notes', 'dev_scripts_helpers.llms.dockerized_llm_transform', 'dev_scripts_helpers.llms.llm_prompts', 'dev_scripts_helpers.llms.llm_transform', 'dev_scripts_helpers.llms.snippets', 'dev_scripts_helpers.misc.string_to_file', 'dev_scripts_helpers.infra.old.ssh_tunnels', 'dev_scripts_helpers.infra.gdrive', 'dev_scripts_helpers.github.set_secrets_and_variables', 'dev_scripts_helpers.release_sorrentum.filter_repo.lint_history', 'dev_scripts_helpers.lib_tasks_data_qa', 'dev_scripts_helpers.coding_tools.traceback_to_cfile', 'dev_scripts_helpers.coding_tools.manage_cache', 'dev_scripts_helpers.coding_tools.parallel_script_skeleton', 'dev_scripts_helpers.coding_tools.measure_import_times', 'dev_scripts_helpers.coding_tools.invite_github_collaborator', 'dev_scripts_helpers.coding_tools.url', 'dev_scripts_helpers.coding_tools.grsync', 'dev_scripts_helpers.coding_tools.toml_merge', 'dev_scripts_helpers.coding_tools.process_prof', 'dev_scripts_helpers.coding_tools.script_skeleton', 'dev_scripts_helpers.coding_tools.find_unused_golden_files', 'dev_scripts_helpers.coding_tools.compile_all', 'dev_scripts_helpers.coding_tools.transform_skeleton', 'dev_scripts_helpers.coding_tools.diff_to_vimdiff', 'dev_scripts_helpers.git.git_hooks.translate', 'dev_scripts_helpers.git.git_hooks.pre-commit', 'dev_scripts_helpers.git.git_hooks.install_hooks', 'dev_scripts_helpers.git.git_hooks.commit-msg', 'dev_scripts_helpers.git.git_hooks.pre-commit-dry-run', 'dev_scripts_helpers.git.git_hooks.utils', 'dev_scripts_helpers.git.git_submodules', 'dev_scripts_helpers.git.gd_notebook', 'dev_scripts_helpers.git.gup', 'dev_scripts_helpers.git.gsp', 'dev_scripts_helpers.dockerize.dockerized_template', 'dev_scripts_helpers.aws.am_aws', 'dev_scripts_helpers.to_clean.gen_utils', 'dev_scripts_helpers.to_clean.gen_utils.ORIG', 'dev_scripts_helpers.thin_client.thin_client_utils', 'dev_scripts_helpers.thin_client.build', 'dev_scripts_helpers.thin_client.tmux', 'dev_scripts_helpers.notebooks.run_notebook_test_case', 'dev_scripts_helpers.notebooks.publish_notebook', 'dev_scripts_helpers.notebooks.ipynb_format', 'dev_scripts_helpers.notebooks.add_toc_to_notebook', 'dev_scripts_helpers.notebooks.process_jupytext', 'dev_scripts_helpers.notebooks.extract_notebook_images', 'dev_scripts_helpers.notebooks.run_notebook', 'dev_scripts_helpers.notebooks.run_jupyter_server', 'dev_scripts_helpers.cleanup_scripts.CmTask1292_Rename_old_aws_env_vars', 'dev_scripts_helpers.cleanup_scripts.CmTask972_Merge_dataframe_to_str_and_df_to_short_str_into_hpandas', 'dev_scripts_helpers.lib_tasks_data_reconcile', 'import_check.example.input.subdir4.file1', 'import_check.example.input.subdir4.file2', 'import_check.example.input.subdir4.file3', 'import_check.example.input.subdir2.file1', 'import_check.example.input.subdir2.subdir3.file1', 'import_check.example.input.subdir2.subdir3.file2', 'import_check.example.input.subdir2.subdir3.file3', 'import_check.show_imports', 'import_check.detect_import_cycles', 'linters.amp_isort', 'linters.amp_check_filename', 'linters.amp_check_file_size', 'linters.amp_class_method_order', 'linters.amp_check_import', 'linters.amp_format_separating_line', 'linters.amp_add_toc_to_notebook', 'linters.amp_processjupytext', 'linters.amp_lint_md', 'linters.amp_check_md_reference', 'linters.amp_normalize_import', 'linters.amp_flake8', 'linters.base', 'linters.amp_fix_comments', 'linters.amp_fix_whitespaces', 'linters.amp_pylint', 'linters.amp_doc_formatter', 'linters.amp_warn_incorrectly_formatted_todo', 'linters.amp_black', 'linters.amp_add_class_frames', 'linters.amp_autoflake', 'linters.amp_fix_md_links', 'linters.action', 'linters.amp_check_shebang', 'linters.amp_mypy', 'linters.amp_check_merge_conflict', 'linters.add_python_init_files', 'linters.amp_check_md_toc_headers', 'linters.utils', 'helpers.lib_tasks_docker_release', 'helpers.stage_linked_file', 'helpers.hgoogle_drive_api', 'helpers.repo_config_utils', 'helpers.hmoto', 'helpers.haws', 'helpers.old.user_credentials', 'helpers.old.conda', 'helpers.old.env2', 'helpers.old.tunnels', 'helpers.hio', 'helpers.unit_test_skeleton', 'helpers.htable', 'helpers.hdatetime', 'helpers.hnumba', 'helpers.hasyncio', 'helpers.hnetwork', 'helpers.hlatex', 'helpers.htest_logger', 'helpers.hprint', 'helpers.lib_tasks_lint', 'helpers.hopen', 'helpers.htqdm', 'helpers.htypes', 'helpers.hsystem', 'helpers.hobject', 'helpers.hplayback', 'helpers.hpytest', 'helpers.logging_testing.logging_main', 'helpers.logging_testing.logging_module', 'helpers.htraceback', 'helpers.hstring', 'helpers.hwarnings', 'helpers.hemail', 'helpers.hserver', 'helpers.hwall_clock_time', 'helpers.hsql_implementation', 'helpers.hchatgpt', 'helpers.lib_tasks_gh', 'helpers.hpandas', 'helpers.hdbg', 'helpers.telegram_notify.telegram_notify', 'helpers.telegram_notify.get_chat_id', 'helpers.telegram_notify.config', 'helpers.lib_tasks_aws', 'helpers.hgit', 'helpers.htranslate', 'helpers.hlist', 'helpers.hdocker', 'helpers.hs3', 'helpers.lib_tasks_perms', 'helpers.hchatgpt_instructions', 'helpers.hdataframe', 'helpers.hunit_test_utils', 'helpers.hsql', 'helpers.lib_tasks', 'helpers.lib_tasks_git', 'helpers.henv', 'helpers.hdict', 'helpers.hcsv', 'helpers.create_links', 'helpers.htimer', 'helpers.hparser', 'helpers.hsql_test', 'helpers.hcache_simple', 'helpers.lib_tasks_docker', 'helpers.hpickle', 'helpers.lib_tasks_pytest', 'helpers.hparquet', 'helpers.hopenai', 'helpers.lib_tasks_integrate', 'helpers.hlogging', 'helpers.hsftp', 'helpers.hnumpy', 'helpers.hjoblib', 'helpers.hretry', 'helpers.hintrospection', 'helpers.hjupyter', 'helpers.hversion', 'helpers.hcache', 'helpers.hthreading', 'helpers.lib_tasks_utils', 'helpers.hunit_test', 'helpers.lib_tasks_print', 'helpers.lib_tasks_find', 'helpers.hsecrets', 'helpers.hmarkdown', 'config_root.config.config_list_builder', 'config_root.config.config_list', 'config_root.config.config_utils', 'config_root.config.config_builder', 'config_root.config.config_', 'main_pytest'])'
################################################################################
```
author_association: CONTRIBUTOR
user: login: sonniki
created_at: 2025-04-03T07:24:57Z
body: > I am facing an error while linting the linters/amp_fix_md_links.py

- You've practiced linting files before during your onboarding, correct? And that worked? Why do you think this is not working now?
- When you report an error, you must always include the command you are running, not only the output
- The problem is likely because you are using an incorrect file path. Pls refer to the [docs](https://github.com/causify-ai/helpers/blob/master/docs/coding/all.submit_code_for_review.how_to_guide.md#run-linter) and make sure the path is correct
- You can generally spend some time first debugging such errors yourself. If you are convinced there is a genuine bug in the system, you are welcome to open an issue and describe the problem. But many things that are related to your setup you can likely figure out without our assistance after some trial and error.    
author_association: CONTRIBUTOR
user: login: sandeepthalapanane
created_at: 2025-04-04T19:28:44Z
body: > All good, just need to resolve the conflict (another PR on the same file was merged in the meantime) and then we'll merge

I resolved the conflict and just changed a small thing from the another issue in `linters/test/test_amp_fix_md_links.py`. 

```
def test4(self): 
to
def test4(self) -> None:
```

It raised a warning while linting the file. 
author_association: CONTRIBUTOR