Repository: jackdewinter/pymarkdown

Issue:
Support for file extensions other than .md
Hi,

In current version, I noticed that usage of the pymarkdown tool is limited to files with extension `.md`.

That is due to https://github.com/jackdewinter/pymarkdown/blob/0476c414eb04f60cc0dde952578de43de028517f/pymarkdown/main.py#L212

I understand that this may be a default, but markdown can technically be used in any text file. I personally have a use case where the files are forced to have a different extension (due to a toolchain that renders the files), but for the most part they are still markdown and could very much use some linting :)

Expanding the existing function to checking against a list of extensions should be easy enough - if that would be a viable solution, I could perhaps propose a PR.

Thoughts?

Discussion:
sourcery-ai[bot]: # Sourcery Code Quality Report

✅ &nbsp;Merging this PR will **increase** code quality in the affected files by **0.18%**.

| Quality metrics | Before | After | Change |
| --- | :---: | :---: | :---: |
| [Complexity](https://docs.sourcery.ai/GitHub/Quality-Report/#complexity) | 2.36 ⭐ | 2.28 ⭐ | -0.08 👍 |
| [Method Length](https://docs.sourcery.ai/GitHub/Quality-Report/#method-length) | 56.97 ⭐ | 56.22 ⭐ | -0.75 👍 |
| [Working memory](https://docs.sourcery.ai/GitHub/Quality-Report/#working-memory) | 7.65 🙂 | 7.73 🙂 | 0.08 👎 |
| **[Quality](https://docs.sourcery.ai/GitHub/Quality-Report/#quality-score)** | **76.13%** ⭐| **76.31%** ⭐| **0.18%** 👍 |

| Other metrics | Before | After | Change |
| --- | :---: | :---: | :---: |
| Lines | 2374 | 2651 | 277 |

| Changed files | Quality Before | Quality After | Quality Change |
| --- | :---: | :---: | :---: |
| pymarkdown/main.py | 69.34% 🙂 | 68.03% 🙂 | -1.31% 👎|
| test/test_listfiles.py | 82.80% ⭐ | 82.80% ⭐ | 0.00% |
| test/test_main.py | 78.42% ⭐ | 79.24% ⭐ | 0.82% 👍|

Here are some functions in these files that still need a tune-up:

| File | Function | Complexity | Length | Working Memory | Quality | Recommendation |
| --- | --- | :---: | :---: | :---: | :---: | --- |
| pymarkdown/main.py | PyMarkdownLint.\_\_parse\_arguments | 2 ⭐ | 297 ⛔ | 11 😞 | 49.42% 😞 | Try splitting into smaller methods. Extract out complex expressions |
| pymarkdown/main.py | PyMarkdownLint.\_\_determine\_files\_to\_scan | 11 🙂 | 107 🙂 | 13 😞 | 52.24% 🙂 | Extract out complex expressions |
| pymarkdown/main.py | PyMarkdownLint.\_\_scan\_file | 6 ⭐ | 171 😞 | 9 🙂 | 56.96% 🙂 | Try splitting into smaller methods |
| pymarkdown/main.py | PyMarkdownLint.main | 14 🙂 | 134 😞 | 7 🙂 | 58.08% 🙂 | Try splitting into smaller methods |
| pymarkdown/main.py | PyMarkdownLint.\_\_initialize\_logging | 9 🙂 | 121 😞 | 9 🙂 | 59.53% 🙂 | Try splitting into smaller methods |

## Legend and Explanation

The emojis denote the absolute quality of the code:

 - ⭐ excellent
 - 🙂 good
 - 😞 poor
 - ⛔ very poor


The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.

***

Please see our documentation [here](https://docs.sourcery.ai/GitHub/Quality-Report/) for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Help us [improve](https://research.typeform.com/to/j06Spdfr?type=quality_report&github_login=jackdewinter&base_repo=https%3A%2F%2Fgithub.com%2Fjackdewinter%2Fpymarkdown.git&base_remote_ref=refs%2Fpull%2F428%2Fhead&base_ref=issue-413&base_sha=28370d5674fa1e34fe45f40126c98d9178bce43b&head_repo=https%3A%2F%2Fgithub.com%2Fjackdewinter%2Fpymarkdown.git&head_ref=sourcery%2Fissue-413&base_pr_number=428&base_pr_state=open) this quality report!
codecov[bot]: # [Codecov](https://codecov.io/gh/jackdewinter/pymarkdown/pull/428?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
> Merging [#428](https://codecov.io/gh/jackdewinter/pymarkdown/pull/428?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (28370d5) into [main](https://codecov.io/gh/jackdewinter/pymarkdown/commit/1a83e3f3633e23d567b0d3b6868b8b9016205034?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) (1a83e3f) will **not change** coverage.
> The diff coverage is `n/a`.

```diff
@@            Coverage Diff            @@
##              main      #428   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          106       106           
  Lines        12221     12233   +12     
  Branches      1746      1751    +5     
=========================================
+ Hits         12221     12233   +12     
```


| [Impacted Files](https://codecov.io/gh/jackdewinter/pymarkdown/pull/428?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | |
|---|---|---|
| [main.py](https://codecov.io/gh/jackdewinter/pymarkdown/pull/428/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-bWFpbi5weQ==) | `100.00% <0.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/jackdewinter/pymarkdown/pull/428?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/jackdewinter/pymarkdown/pull/428?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Last update [1a83e3f...28370d5](https://codecov.io/gh/jackdewinter/pymarkdown/pull/428?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).


Patch:
diff --git a/publish/coverage.json b/publish/coverage.json
index a283d8ef..acc8d6d4 100644
--- a/publish/coverage.json
+++ b/publish/coverage.json
@@ -2,12 +2,12 @@
     "projectName": "pymarkdown",
     "reportSource": "pytest",
     "branchLevel": {
-        "totalMeasured": 3493,
-        "totalCovered": 3493
+        "totalMeasured": 3503,
+        "totalCovered": 3503
     },
     "lineLevel": {
-        "totalMeasured": 12221,
-        "totalCovered": 12221
+        "totalMeasured": 12233,
+        "totalCovered": 12233
     }
 }
 
diff --git a/pymarkdown/main.py b/pymarkdown/main.py
index d887fdbd..e2c26130 100644
--- a/pymarkdown/main.py
+++ b/pymarkdown/main.py
@@ -74,6 +74,29 @@ class PyMarkdownLint:
             return argument
         raise ValueError(f"Value '{argument}' is not a valid log level.")
 
+    @staticmethod
+    def alternate_extension_type(argument: str) -> str:
+        """
+        Function to help argparse limit the valid log levels.
+        """
+        split_argument = argument.split(",")
+        for next_split in split_argument:
+            if not next_split.startswith("."):
+                raise argparse.ArgumentTypeError(
+                    f"Extension '{next_split}' must start with a period."
+                )
+            clean_split = next_split[1:]
+            if not clean_split:
+                raise argparse.ArgumentTypeError(
+                    f"Extension '{next_split}' must have at least one character after the period."
+                )
+            for clean_split_char in clean_split:
+                if not clean_split_char.isalnum():
+                    raise argparse.ArgumentTypeError(
+                        f"Extension '{next_split}' must only contain alphanumeric characters after the period."
+                    )
+        return argument.lower()
+
     def __parse_arguments(self) -> argparse.Namespace:
         parser = argparse.ArgumentParser(description="Lint any found Markdown files.")
 
@@ -184,6 +207,15 @@ class PyMarkdownLint:
             default=False,
             help="recursively scan directories",
         )
+        new_sub_parser.add_argument(
+            "-ae",
+            "--alternate-extensions",
+            dest="alternate_extensions",
+            action="store",
+            default=".md",
+            type=PyMarkdownLint.alternate_extension_type,
+            help="provider an alternate set of file extensions to scan",
+        )
         new_sub_parser.add_argument(
             "paths",
             metavar="path",
@@ -205,11 +237,16 @@ class PyMarkdownLint:
         return parse_arguments
 
     @classmethod
-    def __is_file_eligible_to_scan(cls, path_to_test: str) -> bool:
+    def __is_file_eligible_to_scan(
+        cls, path_to_test: str, eligible_extensions: List[str]
+    ) -> bool:
         """
         Determine if the presented path is one that we want to scan.
         """
-        return path_to_test.endswith(".md")
+        return any(
+            path_to_test.endswith(next_extension)
+            for next_extension in eligible_extensions
+        )
 
     def __scan_file(self, args: argparse.Namespace, next_file: str) -> None:
         """
@@ -255,7 +292,11 @@ class PyMarkdownLint:
             raise
 
     def __process_next_path(
-        self, next_path: str, files_to_parse: Set[str], recurse_directories: bool
+        self,
+        next_path: str,
+        files_to_parse: Set[str],
+        recurse_directories: bool,
+        eligible_extensions: List[str],
     ) -> bool:
 
         did_find_any = False
@@ -268,10 +309,10 @@ class PyMarkdownLint:
             POGGER.debug("Provided path '$' does not exist.", next_path)
         elif os.path.isdir(next_path):
             self.__process_next_path_directory(
-                next_path, files_to_parse, recurse_directories
+                next_path, files_to_parse, recurse_directories, eligible_extensions
             )
             did_find_any = True
-        elif self.__is_file_eligible_to_scan(next_path):
+        elif self.__is_file_eligible_to_scan(next_path, eligible_extensions):
             POGGER.debug(
                 "Provided path '$' is a valid file. Adding.",
                 next_path,
@@ -290,7 +331,11 @@ class PyMarkdownLint:
         return did_find_any
 
     def __process_next_path_directory(
-        self, next_path: str, files_to_parse: Set[str], recurse_directories: bool
+        self,
+        next_path: str,
+        files_to_parse: Set[str],
+        recurse_directories: bool,
+        eligible_extensions: List[str],
     ) -> None:
         POGGER.debug("Provided path '$' is a directory. Walking directory.", next_path)
         normalized_next_path = next_path.replace("\\", "/")
@@ -305,13 +350,20 @@ class PyMarkdownLint:
             )
             for file in files:
                 rooted_file_path = f"{normalized_root}/{file}"
-                if self.__is_file_eligible_to_scan(rooted_file_path):
+                if self.__is_file_eligible_to_scan(
+                    rooted_file_path, eligible_extensions
+                ):
                     files_to_parse.add(rooted_file_path)
 
     def __determine_files_to_scan(
-        self, eligible_paths: List[str], recurse_directories: bool
+        self,
+        eligible_paths: List[str],
+        recurse_directories: bool,
+        eligible_extensions: str,
     ) -> Tuple[List[str], bool]:
 
+        split_eligible_extensions: List[str] = eligible_extensions.split(",")
+
         did_error_scanning_files = False
         files_to_parse: Set[str] = set()
         for next_path in eligible_paths:
@@ -327,10 +379,16 @@ class PyMarkdownLint:
                 for next_globbed_path in globbed_paths:
                     next_globbed_path = next_globbed_path.replace("\\", "/")
                     self.__process_next_path(
-                        next_globbed_path, files_to_parse, recurse_directories
+                        next_globbed_path,
+                        files_to_parse,
+                        recurse_directories,
+                        split_eligible_extensions,
                     )
             elif not self.__process_next_path(
-                next_path, files_to_parse, recurse_directories
+                next_path,
+                files_to_parse,
+                recurse_directories,
+                split_eligible_extensions,
             ):
                 did_error_scanning_files = True
                 break
@@ -429,13 +487,9 @@ class PyMarkdownLint:
             self.__properties.set_manual_property(args.set_configuration)
 
     def __initialize_strict_mode(self, args: argparse.Namespace) -> None:
-        effective_strict_configuration = args.strict_configuration
-        if not effective_strict_configuration:
-            effective_strict_configuration = self.__properties.get_boolean_property(
-                "mode.strict-config", strict_mode=True
-            )
-
-        if effective_strict_configuration:
+        if args.strict_configuration or self.__properties.get_boolean_property(
+            "mode.strict-config", strict_mode=True
+        ):
             self.__properties.enable_strict_mode()
 
     def __initialize_logging(
@@ -546,7 +600,7 @@ class PyMarkdownLint:
 
             POGGER.info("Determining files to scan.")
             files_to_scan, did_error_scanning_files = self.__determine_files_to_scan(
-                args.paths, args.recurse_directories
+                args.paths, args.recurse_directories, args.alternate_extensions
             )
             if did_error_scanning_files:
                 total_error_count = 1


Test patch:
diff --git a/publish/test-results.json b/publish/test-results.json
index 7dca0a64..2c51121a 100644
--- a/publish/test-results.json
+++ b/publish/test-results.json
@@ -1324,7 +1324,7 @@
         },
         {
             "name": "test.test_main",
-            "totalTests": 38,
+            "totalTests": 46,
             "failedTests": 0,
             "errorTests": 0,
             "skippedTests": 0,
diff --git a/test/test_listfiles.py b/test/test_listfiles.py
index 82a30d0f..d2be990a 100644
--- a/test/test_listfiles.py
+++ b/test/test_listfiles.py
@@ -14,15 +14,17 @@ def test_markdown_with_dash_h():
     supplied_arguments = ["scan", "-h"]
 
     expected_return_code = 0
-    expected_output = """usage: main.py scan [-h] [-l] [-r] path [path ...]
+    expected_output = """usage: main.py scan [-h] [-l] [-r] [-ae ALTERNATE_EXTENSIONS] path [path ...]
 
 positional arguments:
-  path              one or more paths to scan for eligible Markdown files
+  path                  one or more paths to scan for eligible Markdown files
 
 optional arguments:
-  -h, --help        show this help message and exit
-  -l, --list-files  list the markdown files found and exit
-  -r, --recurse     recursively scan directories
+  -h, --help            show this help message and exit
+  -l, --list-files      list the markdown files found and exit
+  -r, --recurse         recursively scan directories
+  -ae ALTERNATE_EXTENSIONS, --alternate-extensions ALTERNATE_EXTENSIONS
+                        provider an alternate set of file extensions to scan
 """
     expected_error = ""
 
@@ -46,7 +48,7 @@ def test_markdown_with_dash_l_only():
 
     expected_return_code = 2
     expected_output = ""
-    expected_error = """usage: main.py scan [-h] [-l] [-r] path [path ...]
+    expected_error = """usage: main.py scan [-h] [-l] [-r] [-ae ALTERNATE_EXTENSIONS] path [path ...]
 main.py scan: error: the following arguments are required: path
 """
 
diff --git a/test/test_main.py b/test/test_main.py
index f0fe21c0..3b4d5567 100644
--- a/test/test_main.py
+++ b/test/test_main.py
@@ -1536,3 +1536,241 @@ def test_markdown_with_multiple_errors_reported():
     execute_results.assert_results(
         expected_output, expected_error, expected_return_code
     )
+
+
+def test_markdown_with_dash_ae_with_invalid_file_extension():
+    """
+    Test to make sure
+    """
+
+    # Arrange
+    scanner = MarkdownScanner()
+    file_to_scan = "test/resources/double-line-with-blank-and-trailing.txt"
+    supplied_arguments = [
+        "scan",
+        "-ae",
+        ".abc",
+        file_to_scan,
+    ]
+
+    expected_return_code = 1
+    expected_output = ""
+    expected_error = (
+        f"Provided file path '{file_to_scan}' is not a valid file. Skipping."
+    )
+
+    # Act
+    execute_results = scanner.invoke_main(arguments=supplied_arguments)
+
+    # Assert
+    execute_results.assert_results(
+        expected_output, expected_error, expected_return_code
+    )
+
+
+def test_markdown_with_dash_ae_with_valid_file_extension():
+    """
+    Test to make sure
+    """
+
+    # Arrange
+    scanner = MarkdownScanner()
+    file_to_scan = "test/resources/double-line-with-blank-and-trailing.txt"
+    supplied_arguments = [
+        "--stack-trace",
+        "scan",
+        "-ae",
+        ".txt",
+        file_to_scan,
+    ]
+
+    expected_return_code = 0
+    expected_output = ""
+    expected_error = ""
+
+    # Act
+    execute_results = scanner.invoke_main(arguments=supplied_arguments)
+
+    # Assert
+    execute_results.assert_results(
+        expected_output, expected_error, expected_return_code
+    )
+
+
+def test_markdown_with_dash_ae_with_valid_file_extension_multiple():
+    """
+    Test to make sure
+    """
+
+    # Arrange
+    scanner = MarkdownScanner()
+    file_to_scan = "test/resources/double-line-with-blank-and-trailing.txt"
+    supplied_arguments = [
+        "--stack-trace",
+        "scan",
+        "-ae",
+        ".txt,.md",
+        file_to_scan,
+    ]
+
+    expected_return_code = 0
+    expected_output = ""
+    expected_error = ""
+
+    # Act
+    execute_results = scanner.invoke_main(arguments=supplied_arguments)
+
+    # Assert
+    execute_results.assert_results(
+        expected_output, expected_error, expected_return_code
+    )
+
+
+def test_markdown_with_dash_ae_with_invalid_file_extension_no_period():
+    """
+    Test to make sure
+    """
+
+    # Arrange
+    scanner = MarkdownScanner()
+    file_to_scan = "test/resources/double-line-with-blank-and-trailing.txt"
+    supplied_arguments = [
+        "--stack-trace",
+        "scan",
+        "-ae",
+        "md",
+        file_to_scan,
+    ]
+
+    expected_return_code = 2
+    expected_output = ""
+    expected_error = """usage: main.py scan [-h] [-l] [-r] [-ae ALTERNATE_EXTENSIONS] path [path ...]
+main.py scan: error: argument -ae/--alternate-extensions: Extension 'md' must start with a period."""
+
+    # Act
+    execute_results = scanner.invoke_main(arguments=supplied_arguments)
+
+    # Assert
+    execute_results.assert_results(
+        expected_output, expected_error, expected_return_code
+    )
+
+
+def test_markdown_with_dash_ae_with_invalid_file_extension_no_alphanum():
+    """
+    Test to make sure
+    """
+
+    # Arrange
+    scanner = MarkdownScanner()
+    file_to_scan = "test/resources/double-line-with-blank-and-trailing.txt"
+    supplied_arguments = [
+        "--stack-trace",
+        "scan",
+        "-ae",
+        ".*",
+        file_to_scan,
+    ]
+
+    expected_return_code = 2
+    expected_output = ""
+    expected_error = """usage: main.py scan [-h] [-l] [-r] [-ae ALTERNATE_EXTENSIONS] path [path ...]
+main.py scan: error: argument -ae/--alternate-extensions: Extension '.*' must only contain alphanumeric characters after the period."""
+
+    # Act
+    execute_results = scanner.invoke_main(arguments=supplied_arguments)
+
+    # Assert
+    execute_results.assert_results(
+        expected_output, expected_error, expected_return_code
+    )
+
+
+def test_markdown_with_dash_ae_with_invalid_file_extension_only_period():
+    """
+    Test to make sure
+    """
+
+    # Arrange
+    scanner = MarkdownScanner()
+    file_to_scan = "test/resources/double-line-with-blank-and-trailing.txt"
+    supplied_arguments = [
+        "--stack-trace",
+        "scan",
+        "-ae",
+        ".",
+        file_to_scan,
+    ]
+
+    expected_return_code = 2
+    expected_output = ""
+    expected_error = """usage: main.py scan [-h] [-l] [-r] [-ae ALTERNATE_EXTENSIONS] path [path ...]
+main.py scan: error: argument -ae/--alternate-extensions: Extension '.' must have at least one character after the period."""
+
+    # Act
+    execute_results = scanner.invoke_main(arguments=supplied_arguments)
+
+    # Assert
+    execute_results.assert_results(
+        expected_output, expected_error, expected_return_code
+    )
+
+
+def test_markdown_with_dash_ae_with_invalid_file_extension_semicolon_as_sep():
+    """
+    Test to make sure
+    """
+
+    # Arrange
+    scanner = MarkdownScanner()
+    file_to_scan = "test/resources/double-line-with-blank-and-trailing.txt"
+    supplied_arguments = [
+        "--stack-trace",
+        "scan",
+        "-ae",
+        ".md;.txt",
+        file_to_scan,
+    ]
+
+    expected_return_code = 2
+    expected_output = ""
+    expected_error = """usage: main.py scan [-h] [-l] [-r] [-ae ALTERNATE_EXTENSIONS] path [path ...]
+main.py scan: error: argument -ae/--alternate-extensions: Extension '.md;.txt' must only contain alphanumeric characters after the period."""
+
+    # Act
+    execute_results = scanner.invoke_main(arguments=supplied_arguments)
+
+    # Assert
+    execute_results.assert_results(
+        expected_output, expected_error, expected_return_code
+    )
+
+
+def test_markdown_with_dash_ae_with_invalid_file_extension_empty():
+    """
+    Test to make sure
+    """
+
+    # Arrange
+    scanner = MarkdownScanner()
+    file_to_scan = "test/resources/double-line-with-blank-and-trailing.txt"
+    supplied_arguments = [
+        "--stack-trace",
+        "scan",
+        "-ae",
+        "",
+        file_to_scan,
+    ]
+
+    expected_return_code = 2
+    expected_output = ""
+    expected_error = """usage: main.py scan [-h] [-l] [-r] [-ae ALTERNATE_EXTENSIONS] path [path ...]
+main.py scan: error: argument -ae/--alternate-extensions: Extension '' must start with a period."""
+
+    # Act
+    execute_results = scanner.invoke_main(arguments=supplied_arguments)
+
+    # Assert
+    execute_results.assert_results(
+        expected_output, expected_error, expected_return_code
+    )