Repository: fox-it/dissect.cstruct

Issue:
Stripping comments does not preserve line numbers
If there are any parse errors after multi-line comments, the reported line numbers for the errors are incorrect. To fix this, we can preserve any newlines within comments.

Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/fox-it/dissect.cstruct/pull/43?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fox-it) Report
> Merging [#43](https://app.codecov.io/gh/fox-it/dissect.cstruct/pull/43?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fox-it) (e7c1c7f) into [main](https://app.codecov.io/gh/fox-it/dissect.cstruct/commit/38d332f92776d3d31d1d12951a7a3e4cd32ad310?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fox-it) (38d332f) will **not change** coverage.
> The diff coverage is `100.00%`.

```diff
@@           Coverage Diff           @@
##             main      #43   +/-   ##
=======================================
  Coverage   88.23%   88.23%           
=======================================
  Files          20       20           
  Lines        1709     1709           
=======================================
  Hits         1508     1508           
  Misses        201      201           
```

| Flag | Coverage Δ | |
|---|---|---|
| unittests | `88.23% <100.00%> (ø)` | |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fox-it#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Files Changed](https://app.codecov.io/gh/fox-it/dissect.cstruct/pull/43?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fox-it) | Coverage Δ | |
|---|---|---|
| [dissect/cstruct/parser.py](https://app.codecov.io/gh/fox-it/dissect.cstruct/pull/43?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fox-it#diff-ZGlzc2VjdC9jc3RydWN0L3BhcnNlci5weQ==) | `71.42% <100.00%> (ø)` | |

:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=fox-it)


Patch:
diff --git a/dissect/cstruct/parser.py b/dissect/cstruct/parser.py
index 64b2447..0394a00 100644
--- a/dissect/cstruct/parser.py
+++ b/dissect/cstruct/parser.py
@@ -299,11 +299,11 @@ class TokenParser(Parser):
         # second group captures comments (//single-line or /* multi-line */)
         regex = re.compile(pattern, re.MULTILINE | re.DOTALL)
 
-        def _replacer(match):
+        def _replacer(match: re.Match) -> str:
             # if the 2nd group (capturing comments) is not None,
             # it means we have captured a non-quoted (real) comment string.
-            if match.group(2) is not None:
-                return ""  # so we will return empty to remove the comment
+            if comment := match.group(2):
+                return "\n" * comment.count("\n")  # so we will return empty to remove the comment
             else:  # otherwise, we will return the 1st group
                 return match.group(1)  # captured quoted-string
 
@@ -314,7 +314,7 @@ class TokenParser(Parser):
         """Quick and dirty line number calculator"""
 
         match = tok.match
-        return match.string.count("\n", 0, match.start())
+        return match.string.count("\n", 0, match.start()) + 1
 
     def _config_flag(self, tokens: TokenConsumer) -> None:
         flag_token = tokens.consume()


Test patch:
diff --git a/tests/test_parser.py b/tests/test_parser.py
new file mode 100644
index 0000000..bdc408b
--- /dev/null
+++ b/tests/test_parser.py
@@ -0,0 +1,26 @@
+from unittest.mock import Mock
+
+from dissect.cstruct.parser import TokenParser
+
+
+def test_preserve_comment_newlines():
+    cdef = """
+    // normal comment
+    #define normal_anchor
+    /*
+     * Multi
+     * line
+     * comment
+     */
+    #define multi_anchor
+    """
+    data = TokenParser._remove_comments(cdef)
+    print(repr(data))
+
+    mock_token = Mock()
+    mock_token.match.string = data
+    mock_token.match.start.return_value = data.index("#define normal_anchor")
+    assert TokenParser._lineno(mock_token) == 3
+
+    mock_token.match.start.return_value = data.index("#define multi_anchor")
+    assert TokenParser._lineno(mock_token) == 9