Repository: Shopify/ruby-lsp (Ruby)

Issue:
incorrect doc reference when methods are too close
While developing PR #2733, I accidentally found this very tiny bug! But there may be a different way to look at it. Anyway, just bring it up!

## Description

If we have two methods like the example, the `bar` will share the doc from `foo`. 

```ruby
module Test
  # This method shows foo
  def foo; end
  def bar; end
end

class Something
  include Test

  def foobar
    bar
  end
end
```

![Image](https://github.com/user-attachments/assets/3d04a059-becc-4aa8-9ca0-da988dd8532f)


But if we add one line between the two methods, the `bar` will act normally! 

```ruby
module Test
  # This method shows foo
  def foo
  end
  def bar; end
end

class Something
  include Test

  def foobar
    bar
  end
end
```

![Image](https://github.com/user-attachments/assets/06ce87d9-083b-4e3a-aa00-24c6514089f4)

## Additional Info

Here is where I found it over the [conversation](https://github.com/Shopify/ruby-lsp/pull/2733#discussion_r1813457346) on PR #2733





Discussion:
Thank you for creating the issue!

The mistake comes from [this line](https://github.com/Shopify/ruby-lsp/blob/3881550ceb87ac1a7567f35ca594e55b7ad09a11/lib/ruby_indexer/lib/ruby_indexer/declaration_listener.rb#L664). Essentially, we wanted to ensure that we didn't miss documentation for projects that add comments to their declaration with a blank line in between. Like this:

```ruby
# Documentation for Foo

class Foo
  # Documentation for bar  

  def bar
  end
end
```

However, our check is a bit naive. If there's no comment immediately above the declaration, we check if there's a comment in the line above that one, without checking if the contents of the line in between is another declaration, which leads to the bug.

We should only consider comments as part of the documentation if the line in between is blank.
So, you mean whether a method will have a document, which depends on whether there is a comment above it. The comment can be above the method within one line break.

So, I think the solution could be to check whether the code above the method has `#` or not. 
> So, I think the solution could be to check whether the code above the method has # or not.

Essentially, [this  line](https://github.com/Shopify/ruby-lsp/blob/3881550ceb87ac1a7567f35ca594e55b7ad09a11/lib/ruby_indexer/lib/ruby_indexer/declaration_listener.rb#L664) has to change to be

```ruby
start_line -= 1 unless @comments_by_line.key?(start_line) || !source[start_line - 1].empty?
```

We don't have direct access to the source, but the parse result has a handle to it somewhere.
@vinistock Thanks, that's clear enough! I am gonna create a PR for this issue.
@vinistock, I tried out what you said. I didn't find a way to use parse_result as you do in the pseudo-code. 
But I just came up with an idea: why not remove comments or mark them as visited from `@comments_by_line` while traversing nodes? This approach doesn't need to access the source. Alternatively, when collecting comments for each node, we just need to look up twice (first start_line is `current line - 1` and second start_line is `current line - 2`) and break if `comment` returns nil    

```
# Documentation for Foo <- will be removed or marked when visiting the Foo node

class Foo
  # ####################  <- will be removed or marked when visiting the bar node
  # Documentation for bar <- will be removed or marked when visiting the bar node
  # ####################  <- will be removed or marked when visiting the bar node
  #                       <- will be removed or marked when visiting the bar node
  def bar
  end

  # Documentation for baz <- will be removed or marked when visiting the baz node
  def baz; end
  def ban; end <- do nothing when visiting the ban node because there is no key or all marked as visited
end
```
Good idea, I think deleting the comments from the hash is a good strategy 👍.



Patch:
diff --git a/lib/ruby_indexer/lib/ruby_indexer/declaration_listener.rb b/lib/ruby_indexer/lib/ruby_indexer/declaration_listener.rb
index b29bdd7dbd..2815ec1a8e 100644
--- a/lib/ruby_indexer/lib/ruby_indexer/declaration_listener.rb
+++ b/lib/ruby_indexer/lib/ruby_indexer/declaration_listener.rb
@@ -37,6 +37,7 @@ def initialize(index, dispatcher, parse_result, file_path, collect_comments: fal
         parse_result.code_units_cache(@index.configuration.encoding),
         T.any(T.proc.params(arg0: Integer).returns(Integer), Prism::CodeUnitsCache),
       )
+      @source_lines = T.let(parse_result.source.lines, T::Array[String])
 
       # The nesting stack we're currently inside. Used to determine the fully qualified name of constants, but only
       # stored by unresolved aliases which need the original nesting to be lazily resolved
@@ -661,8 +662,7 @@ def collect_comments(node)
       comments = +""
 
       start_line = node.location.start_line - 1
-      start_line -= 1 unless @comments_by_line.key?(start_line)
-
+      start_line -= 1 unless comment_exists_at?(start_line)
       start_line.downto(1) do |line|
         comment = @comments_by_line[line]
         break unless comment
@@ -683,6 +683,11 @@ def collect_comments(node)
       comments
     end
 
+    sig { params(line: Integer).returns(T::Boolean) }
+    def comment_exists_at?(line)
+      @comments_by_line.key?(line) || !@source_lines[line - 1].to_s.strip.empty?
+    end
+
     sig { params(name: String).returns(String) }
     def fully_qualify_name(name)
       if @stack.empty? || name.start_with?("::")


Test patch:
diff --git a/lib/ruby_indexer/test/method_test.rb b/lib/ruby_indexer/test/method_test.rb
index 809980e18a..6cf261f708 100644
--- a/lib/ruby_indexer/test/method_test.rb
+++ b/lib/ruby_indexer/test/method_test.rb
@@ -149,6 +149,39 @@ def bar; end
       end
     end
 
+    def test_comments_documentation
+      index(<<~RUBY)
+        # Documentation for Foo
+
+        class Foo
+          # ####################
+          # Documentation for bar
+          # ####################
+          #
+          def bar
+          end
+
+          # test
+
+          # Documentation for baz
+          def baz; end
+          def ban; end
+        end
+      RUBY
+
+      foo_comment = @index["Foo"].first.comments
+      assert_equal("Documentation for Foo", foo_comment)
+
+      bar_comment = @index["bar"].first.comments
+      assert_equal("####################\nDocumentation for bar\n####################\n", bar_comment)
+
+      baz_comment = @index["baz"].first.comments
+      assert_equal("Documentation for baz", baz_comment)
+
+      ban_comment = @index["ban"].first.comments
+      assert_empty(ban_comment)
+    end
+
     def test_method_with_parameters
       index(<<~RUBY)
         class Foo
diff --git a/test/requests/completion_resolve_test.rb b/test/requests/completion_resolve_test.rb
index 3435967081..15cadf66d1 100644
--- a/test/requests/completion_resolve_test.rb
+++ b/test/requests/completion_resolve_test.rb
@@ -50,8 +50,8 @@ def initialize
       end
 
       class Foo
-        # Foo!
         def initialize
+          # Foo!
           @a = 1
         end
       end