Repository: objectionary/lints
Pull request #479: bug(#478): L sized program in ProgramsBench
Author: login: h1alexbel
Created: 2025-04-08T13:21:17Z | Merged: 2025-04-10T19:51:31Z

In this PR I've fixed the issue with `ProgramsBench.scansLargeProgram()` benchmark run, now benchmark using L sized program from `ProgramSize.java`. Also, I added benchmark reporting in PRs via [jmh-benchmark-action](https://github.com/volodya-lombrozo/jmh-benchmark-action).

closes #478

Linked issue #478: `ProgramsBench.scansLargeProgram()` fails
I run this:

```bash
mvn jmh:benchmark
```

And getting this:

```text
java.lang.IllegalStateException: java.io.IOException: The resource "program-0.xmir" was not found in jdk.internal.loader.ClassLoaders$AppClassLoader@2c854dc5 (jdk.internal.loader.ClassLoaders.AppClassLoader)
	at org.cactoos.Fallback$None.apply(Fallback.java:103)
	at org.cactoos.Fallback$None.apply(Fallback.java:95)
	at org.cactoos.scalar.ScalarWithFallback.fallback(ScalarWithFallback.java:108)
	at org.cactoos.scalar.ScalarWithFallback.value(ScalarWithFallback.java:74)
	at org.cactoos.scalar.Checked.value(Checked.java:56)
	at org.cactoos.scalar.IoChecked.value(IoChecked.java:44)
	at org.cactoos.scalar.Unchecked.value(Unchecked.java:37)
	at org.cactoos.bytes.UncheckedBytes.asBytes(UncheckedBytes.java:53)
	at fixtures.BytecodeClass.lambda$value$0(BytecodeClass.java:79)
	at com.yegor256.farea.Farea.together(Farea.java:179)
	at fixtures.BytecodeClass.value(BytecodeClass.java:67)
	at fixtures.BytecodeClass.value(BytecodeClass.java:25)
	at org.cactoos.scalar.Checked.value(Checked.java:56)
	at org.cactoos.scalar.IoChecked.value(IoChecked.java:44)
	at benchmarks.ProgramsBench.<init>(ProgramsBench.java:55)
	at benchmarks.jmh_generated.ProgramsBench_jmhType_B1.<init>(ProgramsBench_jmhType_B1.java:3)
	at benchmarks.jmh_generated.ProgramsBench_jmhType_B2.<init>(ProgramsBench_jmhType_B2.java:3)
	at benchmarks.jmh_generated.ProgramsBench_jmhType_B3.<init>(ProgramsBench_jmhType_B3.java:2)
	at benchmarks.jmh_generated.ProgramsBench_jmhType.<init>(ProgramsBench_jmhType.java:2)
	at benchmarks.jmh_generated.ProgramsBench_scansLargeProgram_jmhTest._jmh_tryInit_f_programsbench0_G(ProgramsBench_scansLargeProgram_jmhTest.java:355)
	at benchmarks.jmh_generated.ProgramsBench_scansLargeProgram_jmhTest.scansLargeProgram_AverageTime(ProgramsBench_scansLargeProgram_jmhTest.java:142)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:527)
	at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:504)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.io.IOException: The resource "program-0.xmir" was not found in jdk.internal.loader.ClassLoaders$AppClassLoader@2c854dc5 (jdk.internal.loader.ClassLoaders.AppClassLoader)
	at org.cactoos.io.ResourceOf.lambda$new$3(ResourceOf.java:148)
	at org.cactoos.func.IoCheckedFunc.lambda$apply$0(IoCheckedFunc.java:38)
	at org.cactoos.scalar.Checked.value(Checked.java:56)
	at org.cactoos.scalar.IoChecked.value(IoChecked.java:44)
	at org.cactoos.func.IoCheckedFunc.apply(IoCheckedFunc.java:38)
	at org.cactoos.io.ResourceOf.stream(ResourceOf.java:219)
	at org.cactoos.io.TeeInput.stream(TeeInput.java:848)
	at org.cactoos.bytes.InputAsBytes.asBytes(InputAsBytes.java:59)
	at org.cactoos.bytes.BytesOf.asBytes(BytesOf.java:732)
	at org.cactoos.scalar.ScalarWithFallback.value(ScalarWithFallback.java:68)
	... 27 more
```

Let's fix it

Linked issue comments: user: login: h1alexbel
created_at: 2025-04-08T12:58:33Z
body: @yegor256 looks like bug
author_association: MEMBER
user: login: 0crat
created_at: 2025-04-09T11:12:13Z
body: @h1alexbel Great job reporting that new bug! As per our work policy, you've earned **8** points for your contribution. Your keen eye helps boost our project's quality. Remember, if you spot anything else in the repository that [doesn't look](https://www.yegor256.com/2018/02/06/where-to-find-more-bugs.html) up to par, [don't hesitate](https://www.yegor256.com/2014/04/13/bugs-are-welcome.html) to [report](https://www.yegor256.com/2018/04/24/right-way-to-report-bugs.html) it. Every bug you find is a chance to improve and earn more points. Your current balance stands at [+116](https://www.eolang.org/zerocracy/objectionary-vitals.html). Keep up the excellent work!

<!-- Thanks for reporting a new bug! You've earned +8 points. By reporting bugs, you help our project improve its quality. If you find anything else in the repository that [doesn't look](https://www.yegor256.com/2018/02/06/where-to-find-more-bugs.html) as good as you might expect, [do not hesitate](https://www.yegor256.com/2014/04/13/bugs-are-welcome.html) to [report](https://www.yegor256.com/2018/04/24/right-way-to-report-bugs.html) it. Your running balance is [+116](https://www.eolang.org/zerocracy/objectionary-vitals.html).  -->
author_association: NONE

Diff:
diff --git a/.github/workflows/comment.yml b/.github/workflows/comment.yml
new file mode 100644
index 000000000..54b5d0894
--- /dev/null
+++ b/.github/workflows/comment.yml
@@ -0,0 +1,35 @@
+# SPDX-FileCopyrightText: Copyright (c) 2016-2025 Objectionary.com
+# SPDX-License-Identifier: MIT
+---
+name: comment
+on:
+  workflow_run:
+    workflows: [ "jmh" ]
+    types:
+      - completed
+jobs:
+  comment:
+    timeout-minutes: 15
+    runs-on: ubuntu-24.04
+    steps:
+      - name: Dump GitHub context
+        env:
+          GITHUB_CONTEXT: ${{ toJson(github) }}
+        run: echo "$GITHUB_CONTEXT"
+      - name: Download PR Number and Benchmark Comment
+        uses: actions/download-artifact@v4
+        with:
+          run-id: ${{ github.event.workflow_run.id }}
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+      - name: Read and display PR Number
+        run: |
+          PR_NUMBER=$(cat pr-number/pr-number)
+          echo "PR_NUMBER=$PR_NUMBER" >> "$GITHUB_ENV"
+          echo "The PR number is $PR_NUMBER"
+      - name: Post Comment on PR
+        uses: mshick/add-pr-comment@v2
+        with:
+          issue: ${{ env.PR_NUMBER }}
+          message-path: |
+            benchmark-comment/benchmark-comment.md
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/jmh.yml b/.github/workflows/jmh.yml
new file mode 100644
index 000000000..9abc0de67
--- /dev/null
+++ b/.github/workflows/jmh.yml
@@ -0,0 +1,21 @@
+# SPDX-FileCopyrightText: Copyright (c) 2016-2025 Objectionary.com
+# SPDX-License-Identifier: MIT
+---
+name: jmh
+on:
+  pull_request:
+    branches:
+      - master
+jobs:
+  jmh:
+    timeout-minutes: 25
+    runs-on: ubuntu-24.04
+    steps:
+      - name: Run JMH Benchmark Action
+        uses: volodya-lombrozo/jmh-benchmark-action@main
+        with:
+          java-version: "11"
+          base-ref: "master"
+          benchmark-command: |
+            mvn jmh:benchmark -Djmh.rf=json -Djmh.rff=benchmark.json
+          benchmark-file: "benchmark.json"
diff --git a/src/test/java/benchmarks/ProgramBench.java b/src/test/java/benchmarks/ProgramBench.java
index eecfe4a28..f1c188bab 100644
--- a/src/test/java/benchmarks/ProgramBench.java
+++ b/src/test/java/benchmarks/ProgramBench.java
@@ -27,10 +27,6 @@
  * Benchmark for {@link Program}.
  *
  * @since 0.0.34
- * @todo #143:35min Add JMH benchmarks result reporting.
- *  Currently, there is no reporting of benchmark results. Let's add
- *  <a href="https://github.com/volodya-lombrozo/jmh-benchmark-action">this</a> GitHub Actions
- *  plugin and configure it to run on each PR.
  * @checkstyle DesignForExtensionCheck (10 lines)
  * @checkstyle NonStaticMethodCheck (100 lines)
  */
diff --git a/src/test/java/benchmarks/ProgramsBench.java b/src/test/java/benchmarks/ProgramsBench.java
index a63aa41e8..1b22152b1 100644
--- a/src/test/java/benchmarks/ProgramsBench.java
+++ b/src/test/java/benchmarks/ProgramsBench.java
@@ -5,6 +5,7 @@
 package benchmarks;
 
 import fixtures.BytecodeClass;
+import fixtures.ProgramSize;
 import java.io.IOException;
 import java.nio.charset.StandardCharsets;
 import java.nio.file.Files;
@@ -51,7 +52,7 @@ public ProgramsBench() {
                 final String name = String.format("program-%d.xmir", idx);
                 Files.write(
                     this.home.resolve(String.format("%s.xmir", name)),
-                    new IoChecked<>(new BytecodeClass(name))
+                    new IoChecked<>(new BytecodeClass(name, ProgramSize.L.java()))
                         .value().toString().getBytes(StandardCharsets.UTF_8)
                 );
             }


Reviews:
user: login: volodya-lombrozo
body: @h1alexbel Thank you for the contribution! Are you solving the https://github.com/objectionary/lints/issues/470 issue? or the the https://github.com/objectionary/lints/issues/478?
I would highly recommend you creating a single PR for a single issue. At least, next time. Otherwise you make a scope of a PR too broad, which makes it hard to review. 

state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-04-09T08:00:09Z
user: login: h1alexbel
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-09T08:56:41Z
user: login: volodya-lombrozo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-09T11:44:48Z
user: login: volodya-lombrozo
body: @h1alexbel Looks good to me. Thank you!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-09T11:45:09Z
user: login: yegor256
body: @h1alexbel many thanks for the contribution! A few suggestions above.
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-04-10T02:28:16Z
user: login: h1alexbel
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T15:04:40Z

Review comments:
diff_hunk: @@ -51,7 +52,7 @@ public ProgramsBench() {
                 final String name = String.format("program-%d.xmir", idx);
                 Files.write(
                     this.home.resolve(String.format("%s.xmir", name)),
-                    new IoChecked<>(new BytecodeClass(name))
+                    new IoChecked<>(new BytecodeClass(name, ProgramSize.L.java()))
path: src/test/java/benchmarks/ProgramsBench.java
user: login: volodya-lombrozo
body: @h1alexbel What is this change fixing? We had green pipelines before, and we have green pipelines now.
created_at: 2025-04-09T07:59:34Z
author_association: MEMBER
diff_hunk: @@ -51,7 +52,7 @@ public ProgramsBench() {
                 final String name = String.format("program-%d.xmir", idx);
                 Files.write(
                     this.home.resolve(String.format("%s.xmir", name)),
-                    new IoChecked<>(new BytecodeClass(name))
+                    new IoChecked<>(new BytecodeClass(name, ProgramSize.L.java()))
path: src/test/java/benchmarks/ProgramsBench.java
user: login: h1alexbel
body: @volodya-lombrozo they were green, since we didn't run `mvn jmh:benchmark` there. To show that now its working as expected, and prevent this kind of issues in future, I added missing `jmh` action. WDYT? Maybe I missing something.
created_at: 2025-04-09T08:56:41Z
author_association: MEMBER
diff_hunk: @@ -51,7 +52,7 @@ public ProgramsBench() {
                 final String name = String.format("program-%d.xmir", idx);
                 Files.write(
                     this.home.resolve(String.format("%s.xmir", name)),
-                    new IoChecked<>(new BytecodeClass(name))
+                    new IoChecked<>(new BytecodeClass(name, ProgramSize.L.java()))
path: src/test/java/benchmarks/ProgramsBench.java
user: login: volodya-lombrozo
body: @h1alexbel I didn't understand that. Thank you for the explanation. Then, it's fine.
created_at: 2025-04-09T11:44:48Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,21 @@
+# SPDX-FileCopyrightText: Copyright (c) 2016-2025 Objectionary.com
+# SPDX-License-Identifier: MIT
+---
+name: Performance Regression Check
path: .github/workflows/jmh-benchmark.yml
user: login: yegor256
body: @h1alexbel just `benchmark` is enough. Moreover, I suggest keeping names consistent: file name, workflow name, and job name -- all three should be the same.
created_at: 2025-04-10T02:26:43Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,35 @@
+# SPDX-FileCopyrightText: Copyright (c) 2016-2025 Objectionary.com
+# SPDX-License-Identifier: MIT
+---
+name: Post Benchmark Comment
path: .github/workflows/jmh-comment.yml
user: login: yegor256
body: @h1alexbel the same here, let's keep all three identical: file name, workflow name, and job name
created_at: 2025-04-10T02:27:32Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,21 @@
+# SPDX-FileCopyrightText: Copyright (c) 2016-2025 Objectionary.com
+# SPDX-License-Identifier: MIT
+---
+name: Performance Regression Check
path: .github/workflows/jmh-benchmark.yml
user: login: h1alexbel
body: @yegor256 since `benchmark` is already reserved, let's use `jmh`.
created_at: 2025-04-10T15:04:40Z
author_association: MEMBER

PR comments:
user: login: h1alexbel
created_at: 2025-04-08T14:00:55Z
body: @volodya-lombrozo could you take a look, please?
author_association: MEMBER
user: login: h1alexbel
created_at: 2025-04-09T11:49:43Z
body: @yegor256 take a look, please
author_association: MEMBER
user: login: h1alexbel
created_at: 2025-04-10T15:28:21Z
body: @yegor256 updated. Take a look, please
author_association: MEMBER
user: login: yegor256
created_at: 2025-04-10T19:51:39Z
body: @h1alexbel now it's perfect, thanks!
author_association: MEMBER
user: login: 0crat
created_at: 2025-04-14T16:16:57Z
body: @h1alexbel Thanks for your contribution! Here's the breakdown of your +4 points: +4 base, -16 for no code review, +16 to meet the minimum. While we appreciate your effort, remember that [code reviews](https://www.yegor256.com/2018/03/06/speed-vs-quality.html) are crucial for quality. Your balance is now [+124](https://www.eolang.org/zerocracy/objectionary-vitals.html). Keep improving and contributing!

<!-- Thanks for the contribution! You've earned +4 points for this: +4 as a basis; -16 for the lack of code review; +16 to give you at least something. Please, [keep](https://www.yegor256.com/2018/03/06/speed-vs-quality.html) them coming. Your running balance is [+124](https://www.eolang.org/zerocracy/objectionary-vitals.html).  -->
author_association: NONE