Repository: fortran-lang/fortls

Issue:
`incl_suffixes` ignored if they do not start with a dot
**Describe the bug**
If I pass include suffixes via `--incl_suffixes` to fortls that do not start with a dot, they are simply ignored, with no warning whatsoever. This is especially surprising, since the description of the setting in the Modern Fortran VS Code extension lists the defaults without dots (see [package.json#L443](https://github.com/fortran-lang/vscode-fortran-support/blob/b35cbf5732102e54dc9a8b01a3853a10f11727d2/package.json#L443C100-L443C129))

**To Reproduce**

Having a source repository with

```
<root>
 |- main.f90
 |- subdir
   |- include.inc
```

running `python -m fortls --debug_rootpath=C:\source\test-fortls --incl_suffixes inc`

gives

```
Testing "initialize" request:
  Root = "C:\source\test-fortls"
[INFO - 12:03:10] fortls - Fortran Language Server 2.13.1.dev172+gc04d2ab.d20230629 Initialized
  Successful!

  Source directories:
    C:\source\test-fortls
```

but running `python -m fortls --debug_rootpath=C:\source\test-fortls --incl_suffixes .inc`

gives the correct:

```
Testing "initialize" request:
  Root = "C:\source\test-fortls"
[INFO - 12:03:13] fortls - Fortran Language Server 2.13.1.dev172+gc04d2ab.d20230629 Initialized
  Successful!

  Source directories:
    C:\source\test-fortls\subdir
    C:\source\test-fortls
```

**Expected behavior**

In the order of my personal preference:
 1. It should not matter whether I pass the include suffixes with or without a dot in the beginning
 2. **OR** I should get a warning when a passed argument is incorrect and ignored
 3. **OR** The documentation should at least very explicitly state that a dot in the beginning is a mandatory requirement.


Comments:

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 935d35c..68b380a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -18,6 +18,9 @@
 
 ### Changed
 
+- Changed `--incl_suffixes` option to faithfully match the suffixes that are
+  provided in the option, without performing any type of modification.
+  ([#300](https://github.com/fortran-lang/fortls/issues/300))
 - Changed the completion signature to include the full Markdown documentation
   for the completion item.
   ([#219](https://github.com/fortran-lang/fortls/issues/219))
diff --git a/docs/options.rst b/docs/options.rst
index 91ceb82..c532741 100644
--- a/docs/options.rst
+++ b/docs/options.rst
@@ -102,11 +102,15 @@ incl_suffixes
 .. code-block:: json
 
    {
-      "incl_suffixes": [".h", ".FYP"]
+      "incl_suffixes": [".h", ".FYP", "inc"]
    }
 
 ``fortls`` will parse only files with ``incl_suffixes`` extensions found in
-``source_dirs``. By default ``incl_suffixes`` are defined as
+``source_dirs``. Using the above example, ``fortls`` will match files by the
+``file.h`` and ``file.FYP``, but not ``file.fyp`` or ``filefyp``.
+It will also match ``file.inc`` and ``fileinc`` but not ``file.inc2``.
+
+By default, ``incl_suffixes`` are defined as
 .F .f .F03 .f03 .F05 .f05 .F08 .f08 .F18 .f18 .F77 .f77 .F90 .f90 .F95 .f95 .FOR .for .FPP .fpp.
 Additional source file extensions can be defined in ``incl_suffixes``.
 
diff --git a/fortls/fortls.schema.json b/fortls/fortls.schema.json
index 695c963..f6a8c09 100644
--- a/fortls/fortls.schema.json
+++ b/fortls/fortls.schema.json
@@ -61,7 +61,7 @@
     },
     "incl_suffixes": {
       "title": "Incl Suffixes",
-      "description": "Consider additional file extensions to the default (default: F,F77,F90,F95,F03,F08,FOR,FPP (lower & upper casing))",
+      "description": "Consider additional file extensions to the default (default: .F, .F77, .F90, .F95, .F03, .F08, .FOR, .FPP (lower & upper casing))",
       "default": [],
       "type": "array",
       "items": {},
diff --git a/fortls/interface.py b/fortls/interface.py
index ac136aa..05cc8c1 100644
--- a/fortls/interface.py
+++ b/fortls/interface.py
@@ -126,7 +126,7 @@ def cli(name: str = "fortls") -> argparse.ArgumentParser:
         metavar="SUFFIXES",
         help=(
             "Consider additional file extensions to the default (default: "
-            "F,F77,F90,F95,F03,F08,FOR,FPP (lower & upper casing))"
+            ".F, .F77, .F90, .F95, .F03, .F08, .FOR, .FPP (lower & upper casing))"
         ),
     )
     group.add_argument(
diff --git a/fortls/langserver.py b/fortls/langserver.py
index f98ebb6..33dc6c3 100644
--- a/fortls/langserver.py
+++ b/fortls/langserver.py
@@ -61,7 +61,7 @@ from fortls.objects import (
     get_use_tree,
 )
 from fortls.parse_fortran import FortranFile, get_line_context
-from fortls.regex_patterns import src_file_exts
+from fortls.regex_patterns import create_src_file_exts_str
 from fortls.version import __version__
 
 # Global regexes
@@ -89,7 +89,9 @@ class LangServer:
 
         self.sync_type: int = 2 if self.incremental_sync else 1
         self.post_messages = []
-        self.FORTRAN_SRC_EXT_REGEX: Pattern[str] = src_file_exts(self.incl_suffixes)
+        self.FORTRAN_SRC_EXT_REGEX: Pattern[str] = create_src_file_exts_str(
+            self.incl_suffixes
+        )
         # Intrinsic (re-loaded during initialize)
         (
             self.statements,
@@ -1569,7 +1571,7 @@ class LangServer:
         self.source_dirs = set(config_dict.get("source_dirs", self.source_dirs))
         self.incl_suffixes = set(config_dict.get("incl_suffixes", self.incl_suffixes))
         # Update the source file REGEX
-        self.FORTRAN_SRC_EXT_REGEX = src_file_exts(self.incl_suffixes)
+        self.FORTRAN_SRC_EXT_REGEX = create_src_file_exts_str(self.incl_suffixes)
         self.excl_suffixes = set(config_dict.get("excl_suffixes", self.excl_suffixes))
 
     def _load_config_file_general(self, config_dict: dict) -> None:
diff --git a/fortls/regex_patterns.py b/fortls/regex_patterns.py
index 80a793e..6f59040 100644
--- a/fortls/regex_patterns.py
+++ b/fortls/regex_patterns.py
@@ -149,30 +149,63 @@ class FortranRegularExpressions:
     OBJBREAK: Pattern = compile(r"[\/\-(.,+*<>=$: ]", I)
 
 
-def src_file_exts(input_exts: list[str] = []) -> Pattern[str]:
-    """Create a REGEX for which file extensions the Language Server should parse
-    Default extensions are
-    F F03 F05 F08 F18 F77 F90 F95 FOR FPP f f03 f05 f08 f18 f77 f90 f95 for fpp
+# TODO: use this in the main code
+def create_src_file_exts_regex(input_exts: list[str] = []) -> Pattern[str]:
+    r"""Create a REGEX for which sources the Language Server should parse.
+
+    Default extensions are (case insensitive):
+    F F03 F05 F08 F18 F77 F90 F95 FOR FPP
 
     Parameters
     ----------
     input_exts : list[str], optional
-        Additional Fortran, by default []
+        Additional list of file extensions to parse, in Python REGEX format
+        that means special characters must be escaped
+        , by default []
+
+    Examples
+    --------
+    >>> regex = create_src_file_exts_regex([r"\.fypp", r"\.inc"])
+    >>> regex.search("test.fypp")
+    <re.Match object; span=(4, 9), match='.fypp'>
+    >>> regex.search("test.inc")
+    <re.Match object; span=(4, 8), match='.inc'>
+
+    >>> regex = create_src_file_exts_regex([r"\.inc.*"])
+    >>> regex.search("test.inc.1")
+    <re.Match object; span=(4, 10), match='.inc.1'>
+
+    Invalid regex expressions will cause the function to revert to the default
+    extensions
+
+    >>> regex = create_src_file_exts_regex(["*.inc"])
+    >>> regex.search("test.inc") is None
+    True
 
     Returns
     -------
     Pattern[str]
-        A compiled regular expression, by default
-        '.(F|F03|F05|F08|F18|F77|F90|F95|FOR|FPP|f|f03|f05|f08|f18|f77|f90|f95|for|fpp)?'
+        A compiled regular expression for matching file extensions
     """
-    EXTS = ["", "77", "90", "95", "03", "05", "08", "18", "OR", "PP"]
-    FORTRAN_FILE_EXTS = []
-    for e in EXTS:
-        FORTRAN_FILE_EXTS.extend([f"F{e}".upper(), f"f{e}".lower()])
-    # Add the custom extensions for the server to parse
-    for e in input_exts:
-        if e.startswith("."):
-            FORTRAN_FILE_EXTS.append(e.replace(".", ""))
-    # Cast into a set to ensure uniqueness of extensions & sort for consistency
-    # Create a regular expression from this
-    return compile(rf"\.({'|'.join(sorted(set(FORTRAN_FILE_EXTS)))})?$")
+    import re
+
+    DEFAULT = r"\.[fF](77|90|95|03|05|08|18|[oO][rR]|[pP]{2})?"
+    EXPRESSIONS = [DEFAULT]
+    try:
+        EXPRESSIONS.extend(input_exts)
+        # Add its expression as an OR and force they match the end of the string
+        return re.compile(rf"(({'$)|('.join(EXPRESSIONS)}$))")
+    except re.error:
+        # TODO: Add a warning to the logger
+        return re.compile(rf"({DEFAULT}$)")
+
+
+def create_src_file_exts_str(input_exts: list[str] = []) -> Pattern[str]:
+    """This is a version of create_src_file_exts_regex that takes a list
+    sanitises the list of input_exts before compiling the regex.
+    For more info see create_src_file_exts_regex
+    """
+    import re
+
+    input_exts = [re.escape(ext) for ext in input_exts]
+    return create_src_file_exts_regex(input_exts)
diff --git a/setup.cfg b/setup.cfg
index d9651b2..e6c8048 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -62,7 +62,7 @@ dev =
     black
     isort
     pre-commit
-    pydantic
+    pydantic==1.9.1
 docs =
     sphinx >= 4.0.0
     sphinx-argparse


Test patch:
diff --git a/test/test_regex_patterns.py b/test/test_regex_patterns.py
new file mode 100644
index 0000000..0b08b6d
--- /dev/null
+++ b/test/test_regex_patterns.py
@@ -0,0 +1,45 @@
+from __future__ import annotations
+
+import pytest
+
+from fortls.regex_patterns import create_src_file_exts_regex
+
+
+@pytest.mark.parametrize(
+    "input_exts, input_files, matches",
+    [
+        (
+            [],
+            [
+                "test.f",
+                "test.F",
+                "test.f90",
+                "test.F90",
+                "test.f03",
+                "test.F03",
+                "test.f18",
+                "test.F18",
+                "test.f77",
+                "test.F77",
+                "test.f95",
+                "test.F95",
+                "test.for",
+                "test.FOR",
+                "test.fpp",
+                "test.FPP",
+            ],
+            [True] * 16,
+        ),
+        ([], ["test.ff", "test.f901", "test.f90.ff"], [False, False, False]),
+        ([r"\.inc"], ["test.inc", "testinc", "test.inc2"], [True, False, False]),
+        (["inc.*"], ["test.inc", "testinc", "test.inc2"], [True, True, True]),
+    ],
+)
+def test_src_file_exts(
+    input_exts: list[str],
+    input_files: list[str],
+    matches: list[bool],
+):
+    regex = create_src_file_exts_regex(input_exts)
+    results = [bool(regex.search(file)) for file in input_files]
+    assert results == matches