Repository: junit-team/junit-framework (2025-07-03T05:49:57Z)

Issue:
Improve "Ineffective @Order annotation" discovery issue by mentioning meta-annotations
The commit https://github.com/junit-team/junit-framework/commit/786de2932b4ae749af2753777cc93962f182ec12#diff-4b290a2104fbe0c8f688dfe991a5b5c668f62c6a561abb608421953fd8d5edfe introduced a warning about use of `@Order` without an orderer.

```
(1) [INFO] Ineffective @Order annotation on class 'com.example.OrderTest'. It will not be applied because ClassOrderer.OrderAnnotation is not in use.
    Source: ClassSource [className = 'com.example.OrderTest', filePosition = null]
            at com.example.OrderTest.<no-method>(SourceFile:0)
```

I do understand the problem, but the information does not help: in my case, a meta-annotation was annotated with `@Order` and **I had to use debug mode** to finally find this was due to a meta annotation.

The annotation was annotated like this:

```java
@Retention(RetentionPolicy.RUNTIME)
@Order(1)
public @interface ComposedAnnotation {
}
```

And the test:

```java
@ComposedAnnotation
public class OrderTest {

    @Test
    void test() {
        assertTrue(true);
    }
}
```

This result in the following warnings:

```
juil. 02, 2025 10:49:05 AM org.junit.platform.launcher.core.DiscoveryIssueNotifier logIssues
INFOS: TestEngine with ID 'junit-jupiter' encountered a non-critical issue during test discovery:

(1) [INFO] Ineffective @Order annotation on class 'com.example.OrderTest'. It will not be applied because ClassOrderer.OrderAnnotation is not in use.
    Source: ClassSource [className = 'com.example.OrderTest', filePosition = null]
            at com.example.OrderTest.<no-method>(SourceFile:0)
juil. 02, 2025 10:49:05 AM org.junit.platform.launcher.core.DiscoveryIssueNotifier logIssues
INFOS: TestEngine with ID 'junit-jupiter' encountered a non-critical issue during test discovery:

(1) [INFO] Ineffective @Order annotation on class 'com.example.OrderTest'. It will not be applied because ClassOrderer.OrderAnnotation is not in use.
    Source: ClassSource [className = 'com.example.OrderTest', filePosition = null]
            at com.example.OrderTest.<no-method>(SourceFile:0)
```

Could it be possible to add more details in those warnings?

## Context

 - Used versions (Jupiter/Vintage/Platform): Jupiter 5.13.2
 - Build Tool/IDE: Eclipse / Maven



Discussion:
> Could it be possible to add more details in those warnings ?

To avoid confusion, we could point out that the `@Order` annotation may be present as a meta-annotation, and I've updated the title of this issue to reflect that.
Here is a demo: [demo.tar.gz](https://github.com/user-attachments/files/21017757/demo.tar.gz)

Simply run `./mvnw verify` with a JDK 17 in the path and you'll see the warning:

```
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
2025-07-02T12:18:08.593694600Z main WARN Ignoring call to `j.u.l.Logger.setLevel(WARNING)`, since the Log4j API does not provide methods to modify the underlying implementation.
To modify the configuration using JUL, use an `AbstractLoggerAdapter` appropriate for your logging implementation.
See https://logging.apache.org/log4j/3.x/log4j-jul.html#log4j.jul.loggerAdapter for more information.
14:18:08.739 [main] INFO  org.junit.platform.launcher.core.DiscoveryIssueNotifier - TestEngine with ID 'junit-jupiter' encountered a non-critical issue during test discovery:

(1) [INFO] Ineffective @Order annotation on class 'com.example.demo.DemoApplicationTest'. It will not be applied because ClassOrderer.OrderAnnotation is not in use.
    Source: ClassSource [className = 'com.example.demo.DemoApplicationTest', filePosition = null]
            at com.example.demo.DemoApplicationTest.<no-method>(SourceFile:0)
[INFO] Running com.example.demo.DemoApplicationTest
14:18:08.816 [main] WARN  com.example.demo.DemoApplicationTest - TEST
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.036 s -- in com.example.demo.DemoApplicationTest
14:18:08.835 [main] INFO  org.junit.platform.launcher.core.DiscoveryIssueNotifier - TestEngine with ID 'junit-jupiter' encountered a non-critical issue during test discovery:

(1) [INFO] Ineffective @Order annotation on class 'com.example.demo.DemoApplicationTest'. It will not be applied because ClassOrderer.OrderAnnotation is not in use.
    Source: ClassSource [className = 'com.example.demo.DemoApplicationTest', filePosition = null]
            at com.example.demo.DemoApplicationTest.<no-method>(SourceFile:0)
[INFO]
```

And I think most people don't see the issue in other IDE or in Maven because JUnit seems to use `java.util.logging`: the default configuration is probably to log nothing, but somehow on my Eclipse, it is configured to log something and then I see the warning in red (stderr) in the console.



> > Could it be possible to add more details in those warnings ?
> 
> To avoid confusion, we could point out that the `@Order` annotation may be present as a meta-annotation, and I've updated the title of this issue to reflect that.

That would a be a start. 



Cherry-picked to `releases/5.13.x` in 8a0be1322e14d08f0a69146b307b52ddefb94190



Patch:
diff --git a/documentation/src/docs/asciidoc/release-notes/release-notes-5.13.3.adoc b/documentation/src/docs/asciidoc/release-notes/release-notes-5.13.3.adoc
index a386778d50e0..10dc0f391461 100644
--- a/documentation/src/docs/asciidoc/release-notes/release-notes-5.13.3.adoc
+++ b/documentation/src/docs/asciidoc/release-notes/release-notes-5.13.3.adoc
@@ -59,6 +59,7 @@ repository on GitHub.
   that both `null` and _blank_ values are supported for reason strings and that such
   values will result in an _empty_ `Optional` returned from
   `ConditionEvaluationResult.getReason()`.
+* Improve message of discovery issues reported for ineffective `@Order` annotations.
 
 
 [[release-notes-5.13.3-junit-vintage]]
diff --git a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/discovery/ClassOrderingVisitor.java b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/discovery/ClassOrderingVisitor.java
index 2d6502f6c898..b45250e06fbe 100644
--- a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/discovery/ClassOrderingVisitor.java
+++ b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/discovery/ClassOrderingVisitor.java
@@ -28,7 +28,6 @@
 import org.junit.platform.engine.DiscoveryIssue;
 import org.junit.platform.engine.DiscoveryIssue.Severity;
 import org.junit.platform.engine.TestDescriptor;
-import org.junit.platform.engine.support.descriptor.ClassSource;
 import org.junit.platform.engine.support.discovery.DiscoveryIssueReporter;
 import org.junit.platform.engine.support.discovery.DiscoveryIssueReporter.Condition;
 
@@ -49,10 +48,13 @@ class ClassOrderingVisitor extends AbstractOrderingVisitor {
 		this.globalOrderer = createGlobalOrderer(configuration);
 		this.noOrderAnnotation = issueReporter.createReportingCondition(
 			testDescriptor -> !isAnnotated(testDescriptor.getTestClass(), Order.class), testDescriptor -> {
-				String message = "Ineffective @Order annotation on class '%s'. It will not be applied because ClassOrderer.OrderAnnotation is not in use.".formatted(
-					testDescriptor.getTestClass().getName());
+				String message = """
+						Ineffective @Order annotation on class '%s'. \
+						It will not be applied because ClassOrderer.OrderAnnotation is not in use. \
+						Note that the annotation may be either directly present or meta-present on the class."""//
+						.formatted(testDescriptor.getTestClass().getName());
 				return DiscoveryIssue.builder(Severity.INFO, message) //
-						.source(ClassSource.from(testDescriptor.getTestClass())) //
+						.source(testDescriptor.getSource()) //
 						.build();
 			});
 	}
diff --git a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/discovery/MethodOrderingVisitor.java b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/discovery/MethodOrderingVisitor.java
index de4646820d37..6d5e6e339ab9 100644
--- a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/discovery/MethodOrderingVisitor.java
+++ b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/discovery/MethodOrderingVisitor.java
@@ -30,7 +30,6 @@
 import org.junit.platform.engine.DiscoveryIssue;
 import org.junit.platform.engine.DiscoveryIssue.Severity;
 import org.junit.platform.engine.TestDescriptor;
-import org.junit.platform.engine.support.descriptor.MethodSource;
 import org.junit.platform.engine.support.discovery.DiscoveryIssueReporter;
 import org.junit.platform.engine.support.discovery.DiscoveryIssueReporter.Condition;
 
@@ -50,10 +49,13 @@ class MethodOrderingVisitor extends AbstractOrderingVisitor {
 		this.configuration = configuration;
 		this.noOrderAnnotation = issueReporter.createReportingCondition(
 			testDescriptor -> !isAnnotated(testDescriptor.getTestMethod(), Order.class), testDescriptor -> {
-				String message = "Ineffective @Order annotation on method '%s'. It will not be applied because MethodOrderer.OrderAnnotation is not in use.".formatted(
-					testDescriptor.getTestMethod().toGenericString());
+				String message = """
+						Ineffective @Order annotation on method '%s'. \
+						It will not be applied because MethodOrderer.OrderAnnotation is not in use. \
+						Note that the annotation may be either directly present or meta-present on the method."""//
+						.formatted(testDescriptor.getTestMethod().toGenericString());
 				return DiscoveryIssue.builder(Severity.INFO, message) //
-						.source(MethodSource.from(testDescriptor.getTestMethod())) //
+						.source(testDescriptor.getSource()) //
 						.build();
 			});
 		this.methodsBeforeNestedClassesOrderer = createMethodsBeforeNestedClassesOrderer();


Test patch:
diff --git a/jupiter-tests/src/test/java/org/junit/jupiter/engine/extension/OrderedClassTests.java b/jupiter-tests/src/test/java/org/junit/jupiter/engine/extension/OrderedClassTests.java
index 1954b93d35e0..dd76fa677ce9 100644
--- a/jupiter-tests/src/test/java/org/junit/jupiter/engine/extension/OrderedClassTests.java
+++ b/jupiter-tests/src/test/java/org/junit/jupiter/engine/extension/OrderedClassTests.java
@@ -205,7 +205,9 @@ private static void assertIneffectiveOrderAnnotationIssues(List<DiscoveryIssue>
 		assertThat(discoveryIssues).extracting(DiscoveryIssue::severity).containsOnly(Severity.INFO);
 		assertThat(discoveryIssues).extracting(DiscoveryIssue::message) //
 				.allMatch(it -> it.startsWith("Ineffective @Order annotation on class")
-						&& it.endsWith("It will not be applied because ClassOrderer.OrderAnnotation is not in use."));
+						&& it.contains("It will not be applied because ClassOrderer.OrderAnnotation is not in use.")
+						&& it.endsWith(
+							"Note that the annotation may be either directly present or meta-present on the class."));
 		assertThat(discoveryIssues).extracting(DiscoveryIssue::source).extracting(Optional::orElseThrow) //
 				.containsExactlyInAnyOrder(ClassSource.from(A_TestCase.class), ClassSource.from(C_TestCase.class));
 	}
diff --git a/jupiter-tests/src/test/java/org/junit/jupiter/engine/extension/OrderedMethodTests.java b/jupiter-tests/src/test/java/org/junit/jupiter/engine/extension/OrderedMethodTests.java
index 0d191dfadd99..b19ef8e08bbf 100644
--- a/jupiter-tests/src/test/java/org/junit/jupiter/engine/extension/OrderedMethodTests.java
+++ b/jupiter-tests/src/test/java/org/junit/jupiter/engine/extension/OrderedMethodTests.java
@@ -374,7 +374,9 @@ private static void assertIneffectiveOrderAnnotationIssues(List<DiscoveryIssue>
 		assertThat(discoveryIssues).extracting(DiscoveryIssue::severity).containsOnly(Severity.INFO);
 		assertThat(discoveryIssues).extracting(DiscoveryIssue::message) //
 				.allMatch(it -> it.startsWith("Ineffective @Order annotation on method")
-						&& it.endsWith("It will not be applied because MethodOrderer.OrderAnnotation is not in use."));
+						&& it.contains("It will not be applied because MethodOrderer.OrderAnnotation is not in use.")
+						&& it.endsWith(
+							"Note that the annotation may be either directly present or meta-present on the method."));
 		var testClass = WithoutTestMethodOrderTestCase.class;
 		assertThat(discoveryIssues).extracting(DiscoveryIssue::source).extracting(Optional::orElseThrow) //
 				.containsExactlyInAnyOrder(MethodSource.from(testClass.getDeclaredMethod("test1")),