Repository: checkstyle/checkstyle
Pull request #16462: Issue #14446: Annotations inside javadoc inline tags should not be pa…
Author: login: mahfouz72
Created: 2025-03-03T19:14:06Z | Merged: 2025-03-07T03:56:31Z

closes #14446 

We should support `@snippet` ofc but this PR introduces a workaround to fix the bug.

Diff Regression config: https://gist.githubusercontent.com/mohitsatr/f5a524a06294d907c84b9fc5b68661dc/raw/604872961df92db35e66c26218dc74c61bb10097/config.xml

Linked issue #14446: Parse error when Javadoc contains @snippet with code example that uses Java annotation
I have read check documentation: there are few Checks
I have downloaded the latest checkstyle from: https://checkstyle.org/cmdline.html#Download_and_Run
I have executed the cli and showed it below, as cli describes the problem better than 1,000 words

https://blogs.oracle.com/javamagazine/post/java-javadoc-snippet
https://openjdk.org/jeps/413

```cmd
C:\DEV\checkstyle>javac JsonSerializableStringConverter.java

C:\DEV\checkstyle>type checkstyle.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE module PUBLIC "-//Checkstyle//DTD Checkstyle Configuration 1.3//EN" "https://checkstyle.org/dtds/configuration_1_3.dtd">
<module name="Checker">
    <module name="SuppressWarningsFilter"/>
    <property name="charset" value="UTF-8"/>
    <property name="severity" value="warning"/>
    <property name="fileExtensions" value="java, properties, xml"/>

    <module name="TreeWalker">
        <module name="RequireEmptyLineBeforeBlockTagGroup"/>
        <module name="NonEmptyAtclauseDescription"/>
        <module name="AtclauseOrder">
            <property name="tagOrder" value="@param, @return, @throws, @deprecated"/>
            <property name="target" value="CLASS_DEF, INTERFACE_DEF, ENUM_DEF, METHOD_DEF, CTOR_DEF, VARIABLE_DEF"/>
        </module>
        <module name="SingleLineJavadoc"/>
    </module>
</module>

C:\DEV\checkstyle>type JsonSerializableStringConverter.java
package com.company.example;

/**
 * Some comment describing stuff:
 *
 * {@snippet :
 *     import jakarta.persistence.Converter;
 *     @Converter
 *     public static class SomeSpecificClassConverter extends SomeBaseConverter {
 *         // Some stuff here
 *     }
 *}
 *
 * Note: it supports {@code null} values.
 */
public abstract class JsonSerializableStringConverter {
    // Contents omitted
}

C:\DEV\checkstyle>set RUN_LOCALE="-Duser.language=en -Duser.country=US"

C:\DEV\checkstyle>java %RUN_LOCALE% -jar checkstyle-10.13.0-all.jar -c checkstyle.xml JsonSerializableStringConverter.java
Starting audit...
[WARN] C:\DEV\checkstyle\JsonSerializableStringConverter.java:8: Javadoc comment at column 0 has parse error. Details: mismatched input ' *' expecting JAVADOC_INLINE_TAG_END while parsing JAVADOC_INLINE_TAG [AtclauseOrder]
[WARN] C:\DEV\checkstyle\JsonSerializableStringConverter.java:8: Javadoc comment at column 0 has parse error. Details: mismatched input ' *' expecting JAVADOC_INLINE_TAG_END while parsing JAVADOC_INLINE_TAG [NonEmptyAtclauseDescription]
[WARN] C:\DEV\checkstyle\JsonSerializableStringConverter.java:8: Javadoc comment at column 0 has parse error. Details: mismatched input ' *' expecting JAVADOC_INLINE_TAG_END while parsing JAVADOC_INLINE_TAG [RequireEmptyLineBeforeBlockTagGroup]
[WARN] C:\DEV\checkstyle\JsonSerializableStringConverter.java:8: Javadoc comment at column 0 has parse error. Details: mismatched input ' *' expecting JAVADOC_INLINE_TAG_END while parsing JAVADOC_INLINE_TAG [SingleLineJavadoc]
Audit done.

C:\DEV\checkstyle>
```

If I comment or remove the `@Converter` annotation inside Javadoc snippet like this:
```
/**
 * Some comment describing stuff:
 *
 * {@snippet :
 *     import jakarta.persistence.Converter;
 *     //@Converter
 *     public static class SomeSpecificClassConverter extends SomeBaseConverter {
 *         // Some stuff here
 *     }
 *}
 *
 * Note: it supports {@code null} values.
 */
public abstract class JsonSerializableStringConverter {
    // Contents omitted
}
```
then Checkstyle produces no warnings:
```
Starting audit...
Audit done.
```

Which makes it seem that annotations inside javadoc `@snippet`s are not supported.
Possibly part of https://github.com/checkstyle/checkstyle/issues/11455 ?

---

Expectation: Checkstyle checks not failing when annotations are used inside Javadoc `@snippet`.
Linked issue comments: user: login: romani
created_at: 2024-02-09T14:06:34Z
body: code of grammar is at https://github.com/checkstyle/checkstyle/tree/master/src/main/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc
author_association: MEMBER
user: login: marinier
created_at: 2024-09-06T13:35:01Z
body: I am also having this issue.
author_association: NONE
user: login: lrozenblyum
created_at: 2024-09-10T10:28:15Z
body: The workaround I've found is to use `&#64;` instead of `@` in the annotation declaration inside the snippet.
author_association: NONE
user: login: mahfouz72
created_at: 2025-03-08T00:18:44Z
body: CLI proof that issue is fixed

```
D:\CS\test
cat config.xml
<?xml version="1.0"?>
<!DOCTYPE module PUBLIC
        "-//Checkstyle//DTD Checkstyle Configuration 1.3//EN"
        "https://checkstyle.org/dtds/configuration_1_3.dtd">
<module name="Checker">
    <module name="TreeWalker">
        <module name="RequireEmptyLineBeforeBlockTagGroup"/>
        <module name="NonEmptyAtclauseDescription"/>
        <module name="AtclauseOrder">
            <property name="tagOrder" value="@param, @return, @throws, @deprecated"/>
            <property name="target" value="CLASS_DEF, INTERFACE_DEF, ENUM_DEF, METHOD_DEF, CTOR_DEF, VARIABLE_DEF"/>
        </module>
        <module name="SingleLineJavadoc"/>
    </module>
</module>
D:\CS\test
cat src/Test.java
/**
 * Some comment describing stuff:
 *
 * {@snippet :
 *     import jakarta.persistence.Converter;
 *     @Converter
 *     public static class SomeSpecificClassConverter extends SomeBaseConverter {
 *         // Some stuff here
 *     }
 *}
 *
 * Note: it supports {@code null} values.
 */
abstract class JsonSerializableStringConverter {
    // Contents omitted
}
D:\CS\test
java  -jar checkstyle-10.22.0-SNAPSHOT-all.jar -c config.xml src/Test.java
Starting audit...
Audit done.
```
author_association: MEMBER
user: login: romani
created_at: 2025-03-08T01:36:43Z
body: @mahfouz72 , thanks a lot.
author_association: MEMBER

Diff:
diff --git a/config/checkstyle-non-main-files-suppressions.xml b/config/checkstyle-non-main-files-suppressions.xml
index 8cfc3dbd894b..184b82c3ce50 100644
--- a/config/checkstyle-non-main-files-suppressions.xml
+++ b/config/checkstyle-non-main-files-suppressions.xml
@@ -93,6 +93,10 @@
   <suppress id="lineLength"
          files="src[\\/]test[\\/]resources[\\/].*[\\/]checks[\\/]translation[\\/]Expected.*\.xml"/>
 
+  <!-- the files in this folder cannot be line wrapped, we can not change expected AST -->
+  <suppress id="lineLength"
+         files="src[\\/]test[\\/]resources[\\/].*[\\/]grammar[\\/]javadoc[\\/]javadocTags[\\/]expected.*\.txt"/>
+
   <!-- property names a too long, nothing we can do -->
   <suppress id="lineLength"
          files="config[\\/]org.eclipse.jdt.core.prefs"/>
diff --git a/src/main/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocLexer.g4 b/src/main/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocLexer.g4
index 9a6df389d856..9f99027ed471 100644
--- a/src/main/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocLexer.g4
+++ b/src/main/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocLexer.g4
@@ -80,6 +80,8 @@ import com.puppycrawl.tools.checkstyle.grammar.CrAwareLexerSimulator;
             } else if (previousTokenType != WS && previousTokenType != LEADING_ASTERISK) {
                   isJavadocTagAvailable = false;
             }
+
+            isJavadocTagAvailable = isJavadocTagAvailable && insideJavadocInlineTag == 0;
       }
 
       public void skipCurrentTokenConsuming() {
diff --git a/src/test/java/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocParseTreeTest.java b/src/test/java/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocParseTreeTest.java
index 3f2d2f92bd2a..07a6aff44a9d 100644
--- a/src/test/java/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocParseTreeTest.java
+++ b/src/test/java/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocParseTreeTest.java
@@ -412,4 +412,10 @@ public void testDosLineEndingAsNewlineMultiline() throws Exception {
         verifyJavadocTree(getPath("expectedDosLineEndingAsNewlineMultiline.txt"),
                 getPath("InputDosLineEndingAsNewlineMultiline.javadoc"));
     }
+
+    @Test
+    public void testAnnotationsInsideInlineTag() throws Exception {
+        verifyJavadocTree(getDocPath("expectedAnnotationsInsideInlineTagAst.txt"),
+                getDocPath("InputAnnotationsInsideInlineTag.javadoc"));
+    }
 }
diff --git a/src/test/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/javadocTags/InputAnnotationsInsideInlineTag.javadoc b/src/test/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/javadocTags/InputAnnotationsInsideInlineTag.javadoc
new file mode 100644
index 000000000000..f9f146009f1b
--- /dev/null
+++ b/src/test/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/javadocTags/InputAnnotationsInsideInlineTag.javadoc
@@ -0,0 +1,12 @@
+ * Some comment describing stuff:
+ *
+ * {@snippet :
+ *     import jakarta.persistence.Converter;
+ *     @Converter
+ *     public static class SomeSpecificClassConverter extends SomeBaseConverter {
+ *         // Some stuff here
+ * }
+ * {@code @code int x;}
+ * {@snippet :
+ *    @author int x;
+ *  }
diff --git a/src/test/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/javadocTags/expectedAnnotationsInsideInlineTagAst.txt b/src/test/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/javadocTags/expectedAnnotationsInsideInlineTagAst.txt
new file mode 100644
index 000000000000..b6f5ff579456
--- /dev/null
+++ b/src/test/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/javadocTags/expectedAnnotationsInsideInlineTagAst.txt
@@ -0,0 +1,57 @@
+JAVADOC -> JAVADOC [0:0]
+|--LEADING_ASTERISK ->  * [0:0]
+|--TEXT ->  Some comment describing stuff: [0:2]
+|--NEWLINE -> \n [0:33]
+|--LEADING_ASTERISK ->  * [1:0]
+|--NEWLINE -> \n [1:2]
+|--LEADING_ASTERISK ->  * [2:0]
+|--TEXT ->   [2:2]
+|--JAVADOC_INLINE_TAG -> JAVADOC_INLINE_TAG [2:3]
+|   |--JAVADOC_INLINE_TAG_START -> { [2:3]
+|   |--CUSTOM_NAME -> @snippet [2:4]
+|   |--WS ->   [2:12]
+|   |--DESCRIPTION -> DESCRIPTION [2:13]
+|   |   |--TEXT -> : [2:13]
+|   |   |--NEWLINE -> \n [2:14]
+|   |   |--LEADING_ASTERISK ->  * [3:0]
+|   |   |--TEXT ->      import jakarta.persistence.Converter; [3:2]
+|   |   |--NEWLINE -> \n [3:44]
+|   |   |--LEADING_ASTERISK ->  * [4:0]
+|   |   |--TEXT ->      @Converter [4:2]
+|   |   |--NEWLINE -> \n [4:17]
+|   |   |--LEADING_ASTERISK ->  * [5:0]
+|   |   |--TEXT ->      public static class SomeSpecificClassConverter extends SomeBaseConverter { [5:2]
+|   |   |--NEWLINE -> \n [5:81]
+|   |   |--LEADING_ASTERISK ->  * [6:0]
+|   |   |--TEXT ->          // Some stuff here [6:2]
+|   |   |--NEWLINE -> \n [6:29]
+|   |   |--LEADING_ASTERISK ->  * [7:0]
+|   |   `--TEXT ->   [7:2]
+|   `--JAVADOC_INLINE_TAG_END -> } [7:3]
+|--NEWLINE -> \n [7:4]
+|--LEADING_ASTERISK ->  * [8:0]
+|--TEXT ->   [8:2]
+|--JAVADOC_INLINE_TAG -> JAVADOC_INLINE_TAG [8:3]
+|   |--JAVADOC_INLINE_TAG_START -> { [8:3]
+|   |--CODE_LITERAL -> @code [8:4]
+|   |--WS ->   [8:9]
+|   |--TEXT -> @code int x; [8:10]
+|   `--JAVADOC_INLINE_TAG_END -> } [8:23]
+|--NEWLINE -> \n [8:24]
+|--LEADING_ASTERISK ->  * [9:0]
+|--TEXT ->   [9:2]
+|--JAVADOC_INLINE_TAG -> JAVADOC_INLINE_TAG [9:3]
+|   |--JAVADOC_INLINE_TAG_START -> { [9:3]
+|   |--CUSTOM_NAME -> @snippet [9:4]
+|   |--WS ->   [9:12]
+|   |--DESCRIPTION -> DESCRIPTION [9:13]
+|   |   |--TEXT -> : [9:13]
+|   |   |--NEWLINE -> \n [9:14]
+|   |   |--LEADING_ASTERISK ->  * [10:0]
+|   |   |--TEXT ->     @author int x; [10:2]
+|   |   |--NEWLINE -> \n [10:20]
+|   |   |--LEADING_ASTERISK ->  * [11:0]
+|   |   `--TEXT ->    [11:2]
+|   `--JAVADOC_INLINE_TAG_END -> } [11:4]
+|--NEWLINE -> \n [11:5]
+`--EOF -> <EOF> [12:0]


Reviews:
user: login: mahfouz72
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T19:21:54Z
user: login: nrmancuso
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T03:16:28Z
user: login: mahfouz72
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-05T12:57:16Z
user: login: nrmancuso
body: Nice hack 🎉 
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-06T02:54:13Z
user: login: rnveach
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T03:56:22Z
user: login: romani
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T16:30:09Z
user: login: nrmancuso
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T18:49:11Z
user: login: romani
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T23:31:38Z
user: login: mahfouz72
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T00:19:06Z

Review comments:
diff_hunk: @@ -0,0 +1,12 @@
+ * Some comment describing stuff:
+ *
+ * {@snippet :
+ *     import jakarta.persistence.Converter;
+ *     @Converter
+ *     public static class SomeSpecificClassConverter extends SomeBaseConverter {
+ *         // Some stuff here
+ * }
path: src/test/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/javadocTags/InputAnnotationsInsideInlineTag.javadoc
user: login: mahfouz72
body: The lexer incorrectly classifies `@converter` as a [custom name Javadoc tag](https://github.com/checkstyle/checkstyle/blob/2219dd8daefec057cd538ec44515eba5f8122582/src/main/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocLexer.g4#L146), while the parser expects to hit `JAVADOC_INLINE_TAG_END` first, causing a mismatch.

The additional condition `insideJavadocInlineTag == 0` ensures that the lexer does not consider `@something` as a Javadoc tag when inside a Javadoc inline tag.

My workaround is to ensure annotations are not treated as Javadoc tags when inside a Javadoc inline tag.
created_at: 2025-03-03T19:21:53Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,12 @@
+ * Some comment describing stuff:
+ *
+ * {@snippet :
+ *     import jakarta.persistence.Converter;
+ *     @Converter
+ *     public static class SomeSpecificClassConverter extends SomeBaseConverter {
+ *         // Some stuff here
+ * }
path: src/test/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/javadocTags/InputAnnotationsInsideInlineTag.javadoc
user: login: nrmancuso
body: Is there any changes in performance?
created_at: 2025-03-05T03:16:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,12 @@
+ * Some comment describing stuff:
+ *
+ * {@snippet :
+ *     import jakarta.persistence.Converter;
+ *     @Converter
+ *     public static class SomeSpecificClassConverter extends SomeBaseConverter {
+ *         // Some stuff here
+ * }
path: src/test/resources/com/puppycrawl/tools/checkstyle/grammar/javadoc/javadocTags/InputAnnotationsInsideInlineTag.javadoc
user: login: mahfouz72
body: No
created_at: 2025-03-05T12:57:15Z
author_association: MEMBER
diff_hunk: @@ -412,4 +412,10 @@ public void testDosLineEndingAsNewlineMultiline() throws Exception {
         verifyJavadocTree(getPath("expectedDosLineEndingAsNewlineMultiline.txt"),
                 getPath("InputDosLineEndingAsNewlineMultiline.javadoc"));
     }
+
+    @Test
+    public void testAnnotationsInsideInlineTag() throws Exception {
+        verifyJavadocTree(getDocPath("expectedAnnotationsInsideInlineTagAst.txt"),
path: src/test/java/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocParseTreeTest.java
user: login: romani
body: Issue was opened on AtclauseOrder to work.

I did not notice that it was proven to be working.
created_at: 2025-03-07T16:30:09Z
author_association: MEMBER
diff_hunk: @@ -412,4 +412,10 @@ public void testDosLineEndingAsNewlineMultiline() throws Exception {
         verifyJavadocTree(getPath("expectedDosLineEndingAsNewlineMultiline.txt"),
                 getPath("InputDosLineEndingAsNewlineMultiline.javadoc"));
     }
+
+    @Test
+    public void testAnnotationsInsideInlineTag() throws Exception {
+        verifyJavadocTree(getDocPath("expectedAnnotationsInsideInlineTagAst.txt"),
path: src/test/java/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocParseTreeTest.java
user: login: nrmancuso
body: Issue was opened for four different checks in the configuration
created_at: 2025-03-07T18:49:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -412,4 +412,10 @@ public void testDosLineEndingAsNewlineMultiline() throws Exception {
         verifyJavadocTree(getPath("expectedDosLineEndingAsNewlineMultiline.txt"),
                 getPath("InputDosLineEndingAsNewlineMultiline.javadoc"));
     }
+
+    @Test
+    public void testAnnotationsInsideInlineTag() throws Exception {
+        verifyJavadocTree(getDocPath("expectedAnnotationsInsideInlineTagAst.txt"),
path: src/test/java/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocParseTreeTest.java
user: login: romani
body: At least we should share CLI on user config to see if output  is reasonable 
created_at: 2025-03-07T23:31:38Z
author_association: MEMBER
diff_hunk: @@ -412,4 +412,10 @@ public void testDosLineEndingAsNewlineMultiline() throws Exception {
         verifyJavadocTree(getPath("expectedDosLineEndingAsNewlineMultiline.txt"),
                 getPath("InputDosLineEndingAsNewlineMultiline.javadoc"));
     }
+
+    @Test
+    public void testAnnotationsInsideInlineTag() throws Exception {
+        verifyJavadocTree(getDocPath("expectedAnnotationsInsideInlineTagAst.txt"),
path: src/test/java/com/puppycrawl/tools/checkstyle/grammar/javadoc/JavadocParseTreeTest.java
user: login: mahfouz72
body: https://github.com/checkstyle/checkstyle/issues/14446#issuecomment-2707769020
created_at: 2025-03-08T00:19:06Z
author_association: MEMBER

PR comments:
user: login: nrmancuso
created_at: 2025-03-05T03:14:17Z
body: I am good to close #14446 via the changes in this PR, but we should move some details from there to https://github.com/checkstyle/checkstyle/issues/11455
author_association: CONTRIBUTOR
user: login: romani
created_at: 2025-03-05T17:05:11Z
body: @nrmancuso , please move details that you need, I am not sure what are they 
author_association: MEMBER
user: login: nrmancuso
created_at: 2025-03-06T02:56:10Z
body: @mahfouz72 please generate a regression report with all javadoc checks (preferably on OpenJDK21+)
author_association: CONTRIBUTOR
user: login: mahfouz72
created_at: 2025-03-06T17:23:08Z
body: Github, generate report
author_association: MEMBER
user: login: github-actions[bot]
created_at: 2025-03-06T18:40:14Z
body: https://checkstyle-diff-reports.s3.us-east-2.amazonaws.com/16f5a89_2025183943/reports/diff/index.html
author_association: CONTRIBUTOR
user: login: mahfouz72
created_at: 2025-03-06T19:29:06Z
body: All new violations are on javadoc that wasn't parsed before.

Annotaions inside `@snippet` or any other inline tag.

diff are on jdk21 only
author_association: MEMBER
user: login: rnveach
created_at: 2025-03-07T03:03:52Z
body: > All new violations are on javadoc that wasn't parsed before.

Removal of `SuppressionXpathSingleFilter` would prove your point.
author_association: CONTRIBUTOR
user: login: rnveach
created_at: 2025-03-07T03:04:54Z
body: I am waiting on a follow up to https://github.com/checkstyle/checkstyle/pull/16462#issuecomment-2699675472 . I don't see anything else needed here.
author_association: CONTRIBUTOR
user: login: nrmancuso
created_at: 2025-03-07T03:31:49Z
body: > I am waiting on a follow up to [#16462 (comment)](https://github.com/checkstyle/checkstyle/pull/16462#issuecomment-2699675472) . I don't see anything else needed here.

Upon closer inspection, we have everything we need in https://github.com/checkstyle/checkstyle/issues/11455
author_association: CONTRIBUTOR