Repository: MyIntervals/PHP-CSS-Parser
Pull request #1169: [BUGFIX] Include comments for all rules in declaration block
Author: login: JakeQZ
Labels: name: bug
name: to-backport
Created: 2025-03-15T00:41:33Z | Merged: 2025-03-16T16:25:25Z

- `Rule::parse()` will no longer consume anything after the semicolon terminating the rule - it does not belong to that rule;
- The whitespace and comments before a rule will be processed by `RuleSet::parseRuleSet()` and passed as a parameter to `Rule::parse()` -
  - This is only required while 'strict mode' parsing is an option, to avoid having an exception thrown during normal operation (i.e. when `Rule::parse()` encounters normal `}` as opposed to some other junk, which is not distinguished).

Fixes #173.
See also #663, #672, #741.

Linked issue #173: bug with parsing multiple comments
When parsing a rule or csslist, trailing comments (those comments after the rule/list) are consumed as part of a final consumeWhitespace call, meaning we don't see these reported for the next rule in the set (or the next set).

Consider the following cases:

`div {/*Find Me!*/left:10px; /*Find Me Too!*/text-align:left;}`
Both of these comments should be returned (one for each rule), but the latter is not.

`/*Find Me!*/div {left:10px; text-align:left;} /*Find Me Too!*/a {left:10px;}`
Both of these comments should be returned (one for each csslist), but the latter is not.

I'd like this fixed so I can use this to parse RTL directives for css rules/lists reliably again. Moodle recently upgraded to 8.3.0 and this broke our RTL.

Please see the linked fix proposal.
Linked issue comments: user: login: andrewnicols
created_at: 2025-03-13T04:00:54Z
body: This should be reopened as #672 was reverted.
author_association: NONE
user: login: oliverklee
created_at: 2025-03-13T08:18:48Z
body: Indeed, thanks! 🙏 Reopening.
author_association: COLLABORATOR

Diff:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f6bcb8058..73fed056a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -66,6 +66,7 @@ Please also have a look at our
 
 ### Fixed
 
+- Include comments for all rules in declaration block (#1169)
 - Render rules in line and column number order (#1059)
 - Don't render `rgb` colors with percentage values using hex notation (#803)
 - Parse `@font-face` `src` property as comma-delimited list (#790)
diff --git a/src/Rule/Rule.php b/src/Rule/Rule.php
index 42fef09d4..dac05cd10 100644
--- a/src/Rule/Rule.php
+++ b/src/Rule/Rule.php
@@ -68,14 +68,16 @@ public function __construct($rule, int $lineNumber = 0, $columnNumber = 0)
     }
 
     /**
+     * @param list<Comment> $commentsBeforeRule
+     *
      * @throws UnexpectedEOFException
      * @throws UnexpectedTokenException
      *
      * @internal since V8.8.0
      */
-    public static function parse(ParserState $parserState): Rule
+    public static function parse(ParserState $parserState, array $commentsBeforeRule = []): Rule
     {
-        $comments = $parserState->consumeWhiteSpace();
+        $comments = \array_merge($commentsBeforeRule, $parserState->consumeWhiteSpace());
         $rule = new Rule(
             $parserState->parseIdentifier(!$parserState->comes('--')),
             $parserState->currentLine(),
@@ -98,8 +100,6 @@ public static function parse(ParserState $parserState): Rule
             $parserState->consume(';');
         }
 
-        $parserState->consumeWhiteSpace();
-
         return $rule;
     }
 
diff --git a/src/RuleSet/RuleSet.php b/src/RuleSet/RuleSet.php
index 311f06bed..9d5ab4118 100644
--- a/src/RuleSet/RuleSet.php
+++ b/src/RuleSet/RuleSet.php
@@ -65,11 +65,15 @@ public static function parseRuleSet(ParserState $parserState, RuleSet $ruleSet):
         while ($parserState->comes(';')) {
             $parserState->consume(';');
         }
-        while (!$parserState->comes('}')) {
+        while (true) {
+            $commentsBeforeRule = $parserState->consumeWhiteSpace();
+            if ($parserState->comes('}')) {
+                break;
+            }
             $rule = null;
             if ($parserState->getSettings()->usesLenientParsing()) {
                 try {
-                    $rule = Rule::parse($parserState);
+                    $rule = Rule::parse($parserState, $commentsBeforeRule);
                 } catch (UnexpectedTokenException $e) {
                     try {
                         $consumedText = $parserState->consumeUntil(["\n", ';', '}'], true);
@@ -87,7 +91,7 @@ public static function parseRuleSet(ParserState $parserState, RuleSet $ruleSet):
                     }
                 }
             } else {
-                $rule = Rule::parse($parserState);
+                $rule = Rule::parse($parserState, $commentsBeforeRule);
             }
             if ($rule instanceof Rule) {
                 $ruleSet->addRule($rule);
diff --git a/tests/ParserTest.php b/tests/ParserTest.php
index 47ec1ffaf..930f12f7c 100644
--- a/tests/ParserTest.php
+++ b/tests/ParserTest.php
@@ -1091,9 +1091,11 @@ public function flatCommentExtractingOneComment(): void
     {
         $parser = new Parser('div {/*Find Me!*/left:10px; text-align:left;}');
         $document = $parser->parse();
+
         $contents = $document->getContents();
         $divRules = $contents[0]->getRules();
         $comments = $divRules[0]->getComments();
+
         self::assertCount(1, $comments);
         self::assertSame('Find Me!', $comments[0]->getComment());
     }
@@ -1101,16 +1103,50 @@ public function flatCommentExtractingOneComment(): void
     /**
      * @test
      */
-    public function flatCommentExtractingTwoComments(): void
+    public function flatCommentExtractingTwoConjoinedCommentsForOneRule(): void
     {
-        self::markTestSkipped('This is currently broken.');
+        $parser = new Parser('div {/*Find Me!*//*Find Me Too!*/left:10px; text-align:left;}');
+        $document = $parser->parse();
+
+        $contents = $document->getContents();
+        $divRules = $contents[0]->getRules();
+        $comments = $divRules[0]->getComments();
 
+        self::assertCount(2, $comments);
+        self::assertSame('Find Me!', $comments[0]->getComment());
+        self::assertSame('Find Me Too!', $comments[1]->getComment());
+    }
+
+    /**
+     * @test
+     */
+    public function flatCommentExtractingTwoSpaceSeparatedCommentsForOneRule(): void
+    {
+        $parser = new Parser('div { /*Find Me!*/ /*Find Me Too!*/ left:10px; text-align:left;}');
+        $document = $parser->parse();
+
+        $contents = $document->getContents();
+        $divRules = $contents[0]->getRules();
+        $comments = $divRules[0]->getComments();
+
+        self::assertCount(2, $comments);
+        self::assertSame('Find Me!', $comments[0]->getComment());
+        self::assertSame('Find Me Too!', $comments[1]->getComment());
+    }
+
+    /**
+     * @test
+     */
+    public function flatCommentExtractingCommentsForTwoRules(): void
+    {
         $parser = new Parser('div {/*Find Me!*/left:10px; /*Find Me Too!*/text-align:left;}');
         $document = $parser->parse();
+
         $contents = $document->getContents();
         $divRules = $contents[0]->getRules();
         $rule1Comments = $divRules[0]->getComments();
         $rule2Comments = $divRules[1]->getComments();
+
         self::assertCount(1, $rule1Comments);
         self::assertCount(1, $rule2Comments);
         self::assertSame('Find Me!', $rule1Comments[0]->getComment());


Reviews:
user: login: oliverklee
body: Looks good! I think we should have some more testcases, though.
state: CHANGES_REQUESTED
author_association: COLLABORATOR
submitted_at: 2025-03-15T09:47:30Z
user: login: JakeQZ
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-16T00:52:33Z
user: login: oliverklee
body: LGTM.
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-16T09:34:26Z
user: login: oliverklee
body: Added two more nitpicks.
state: CHANGES_REQUESTED
author_association: COLLABORATOR
submitted_at: 2025-03-16T09:38:24Z
user: login: oliverklee
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-16T09:38:58Z
user: login: oliverklee
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-16T12:26:43Z
user: login: oliverklee
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-16T16:25:03Z

Review comments:
diff_hunk: @@ -1103,8 +1103,6 @@ public function flatCommentExtractingOneComment(): void
      */
     public function flatCommentExtractingTwoComments(): void
     {
-        self::markTestSkipped('This is currently broken.');
-
         $parser = new Parser('div {/*Find Me!*/left:10px; /*Find Me Too!*/text-align:left;}');
path: tests/ParserTest.php
user: login: oliverklee
body: I think we also should have some more tests:
- two comments directly after another without whitespace
- two comments directly after another, but separated by whitespace
- three comments

created_at: 2025-03-15T09:47:13Z
author_association: COLLABORATOR
diff_hunk: @@ -1103,8 +1103,6 @@ public function flatCommentExtractingOneComment(): void
      */
     public function flatCommentExtractingTwoComments(): void
     {
-        self::markTestSkipped('This is currently broken.');
-
         $parser = new Parser('div {/*Find Me!*/left:10px; /*Find Me Too!*/text-align:left;}');
path: tests/ParserTest.php
user: login: JakeQZ
body: > I think we also should have some more tests:

I was also thinking the same.  Have added the first two suggestions.

For consistency, I've continued with the original method-naming pattern and location for now.  Though I don't know what `flat` is supposed to mean, and I think these tests belong in functional tests for either `Rule` or `RuleSet` (not sure which), but that is beyond the scope of this PR.

>     * three comments

Not sure what you had in mind here.  If one works and two works, is that not sufficient?  How far do we go (and [how long is](https://dictionary.cambridge.org/dictionary/english/how-long-is-a-piece-of-string) [a piece of string](https://www.youtube.com/watch?v=T1TYiixJ9P8))?
created_at: 2025-03-16T00:52:33Z
author_association: COLLABORATOR
diff_hunk: @@ -65,11 +65,15 @@ public static function parseRuleSet(ParserState $parserState, RuleSet $ruleSet):
         while ($parserState->comes(';')) {
             $parserState->consume(';');
         }
-        while (!$parserState->comes('}')) {
+        for (;;) {
path: src/RuleSet/RuleSet.php
user: login: oliverklee
body: Let's communicate the intention behind this line more clearly:

```suggestion
        while (true) {
```
created_at: 2025-03-16T09:35:36Z
author_association: COLLABORATOR
diff_hunk: @@ -68,14 +68,16 @@ public function __construct($rule, int $lineNumber = 0, $columnNumber = 0)
     }
 
     /**
+     * @param list<Comment> $commentsBeforeRule
path: src/Rule/Rule.php
user: login: oliverklee
body: For this to be consistent on higher PHPStan levels, we also need to change the return type annotation of `ParserState::consumeWhiteSpace()` to be a `list`.
created_at: 2025-03-16T09:38:15Z
author_association: COLLABORATOR
diff_hunk: @@ -1103,8 +1103,6 @@ public function flatCommentExtractingOneComment(): void
      */
     public function flatCommentExtractingTwoComments(): void
     {
-        self::markTestSkipped('This is currently broken.');
-
         $parser = new Parser('div {/*Find Me!*/left:10px; /*Find Me Too!*/text-align:left;}');
path: tests/ParserTest.php
user: login: oliverklee
body: After thinking about this some more, I think testing for 2 comments should suffice.
created_at: 2025-03-16T09:38:58Z
author_association: COLLABORATOR
diff_hunk: @@ -68,14 +68,16 @@ public function __construct($rule, int $lineNumber = 0, $columnNumber = 0)
     }
 
     /**
+     * @param list<Comment> $commentsBeforeRule
path: src/Rule/Rule.php
user: login: oliverklee
body: Done in #1173.
created_at: 2025-03-16T12:26:43Z
author_association: COLLABORATOR

PR comments:
user: login: coveralls
created_at: 2025-03-15T00:43:08Z
body: 
[![Coverage Status](https://coveralls.io/builds/72747848/badge)](https://coveralls.io/builds/72747848)

coverage: 55.727% (-0.1%) from 55.84%
when pulling **d503f75aaa2014337868c4bf43787301449141c6 on bugfix/ruleset-comments**
into **b561b72a07d117c5a31137556d0873754524a5b3 on main**.

author_association: NONE
user: login: oliverklee
created_at: 2025-03-16T23:03:30Z
body: @JakeQZ Would you be willing to take care of creating the backport?
author_association: COLLABORATOR
user: login: JakeQZ
created_at: 2025-03-17T00:03:24Z
body: > @JakeQZ Would you be willing to take care of creating the backport?

Done - #1182.
author_association: COLLABORATOR