Repository: junit-team/junit-framework (Java)

Issue:
Provide a concise test feed details mode for the ConsoleLauncher
The ability to have a good report (tree) does not prevent to have some real time hints about executed tests.
Currently, these requirements are not fulfilled OOTB by the console launcher.
Would be great to have a listener logging each test being executed (a bit like surefire does "Running X") + the global report at the end.


Discussion: You may activate "some real time hints" by passing `--details=flat` to the Console Launcher program.

There's also the `--details=verbose` mode, which combines `flat` and `tree` modes.

If those (and other) OOTB listeners/printers don't fulfill your needs, consider to provide your own: [Plugging in your own Test Execution Listener](https://junit.org/junit5/docs/current/user-guide/#launcher-api-listeners-custom)
@sormuras verbose logs all internal details of junit (uniqueId and so on) which is useless most of the time, however all builds need to know what is currently running or in which order test were ran (using the log to track flacky tests) and have a summary for a quick review of the execution (in particular in case of failure). So I really think junit5 built-in listeners are not enough until you can use `--details=tree,flat` or something like that.
Last note being a custom listener is not an option in "application projects" in general so this would end up in a reusable module released outside junit5 whereas it benefits all junit5 users IMHO so I think it should be available OOTB.

edit: flat mode is quite too verbose "per test" in real time mode too (registered event log is useless most of the time, and wonder if [engine:junit-jupiter] logging makes sense when there is a single engine. Makes it hard to read for no real gain IMHO)
Tentatively slated for 5.8 Backlog for _team discussion_
Some more precise feedback:

I turned on flat instead of tree on my setup. Here are the conclusions:
- tree/summary enables to know which test failed easily
- flat enables to know which test hangs or is getting executed (don't assume a test is <1s ;))

=> both features are required at the same time

With flat, when there is a failure it is hard to know which test until you go in the xml report which is not that readable or even browsable - at least in CLI without xml tools.
With tree I have to jstack the console launcher process to know what happens - if possible, in some cases it does not help.

Therefore I really think that combining listeners is required to make the experience at least not worse than alternate existing runners.
**Team decision:** Add a new `testfeed` details mode and make `--details` take a comma-separated list of modes.

The output would look similar to this (fewer/more blank lines up for debate):
```
JUnit Jupiter > Some container display name > Some nested container display name > Some test display name STARTED
JUnit Jupiter > Some container display name > Some nested container display name > Some test display name SUCCEEDED

JUnit Jupiter > Some container display name > Some nested container display name > Some other test display name SKIPPED
    Reason: ...

JUnit Jupiter > Some container display name > Some nested container display name > Some failing test display name STARTED

JUnit Jupiter > Some container display name > Some nested container display name > Some failing test display name FAILED/ABORTED
   java.lang.AssertionError: some message
       at SomeClass.java:42

JUnit Jupiter > Some container display name > Some nested container display name > Yet another test display name STARTED
JUnit Jupiter > Some container display name > Some nested container display name > Yet another test display name SUCCEEDED
```
@rmannibucau Would that suit your needs?
@marcphilipp perfectly, thanks a lot!
My only question would be, whether the test names are escaped? E.g. how would a test like this be logged `@DisplayName("test a > b")`, same with `\n` and oder special chars.

@rmannibucau what is the intended audience, a human or a maschine?
@leonard84 clearly a human, machine audience only uses file reports (legacy xml until there is a new standard).
> My only question would be, whether the test names are escaped?

Since this intended for human consumption, I don't think we need to escape `>`.



Fix (patch):
diff --git a/documentation/src/docs/asciidoc/release-notes/release-notes-5.10.0-M1.adoc b/documentation/src/docs/asciidoc/release-notes/release-notes-5.10.0-M1.adoc
index b1a7b61e62d7..dc23033cb043 100644
--- a/documentation/src/docs/asciidoc/release-notes/release-notes-5.10.0-M1.adoc
+++ b/documentation/src/docs/asciidoc/release-notes/release-notes-5.10.0-M1.adoc
@@ -38,6 +38,8 @@ repository on GitHub.
   methods of the former. Please refer to the
   <<../user-guide/index.adoc#launcher-api-launcher-interceptors-custom, User Guide>> for
   details.
+* The new `testfeed` details mode for `ConsoleLauncher` prints test execution events as
+  they occur in a concise format.
 
 
 [[release-notes-5.10.0-M1-junit-jupiter]]
diff --git a/junit-platform-console/src/main/java/org/junit/platform/console/options/Details.java b/junit-platform-console/src/main/java/org/junit/platform/console/options/Details.java
index 7e6e4bdd850d..3482c6497fa9 100644
--- a/junit-platform-console/src/main/java/org/junit/platform/console/options/Details.java
+++ b/junit-platform-console/src/main/java/org/junit/platform/console/options/Details.java
@@ -43,10 +43,17 @@ public enum Details {
 	/**
 	 * Combines {@link #TREE} and {@link #FLAT} modes.
 	 */
-	VERBOSE;
+	VERBOSE,
 
 	/**
-	 * Return lower case {@link #name} for easier usage in help text for
+	 * Test plan execution events are rendered as they occur in a concise format.
+	 *
+	 * @since 1.10
+	 */
+	TESTFEED;
+
+	/**
+	 * Return lower case {@link #name()} for easier usage in help text for
 	 * available options.
 	 */
 	@Override
diff --git a/junit-platform-console/src/main/java/org/junit/platform/console/tasks/ConsoleTestExecutor.java b/junit-platform-console/src/main/java/org/junit/platform/console/tasks/ConsoleTestExecutor.java
index aebafa32134a..1c5c6c18cac3 100644
--- a/junit-platform-console/src/main/java/org/junit/platform/console/tasks/ConsoleTestExecutor.java
+++ b/junit-platform-console/src/main/java/org/junit/platform/console/tasks/ConsoleTestExecutor.java
@@ -148,6 +148,8 @@ private Optional<DetailsPrintingListener> createDetailsPrintingListener(PrintWri
 				return Optional.of(new TreePrintingListener(out, colorPalette, theme));
 			case VERBOSE:
 				return Optional.of(new VerboseTreePrintingListener(out, colorPalette, 16, theme));
+			case TESTFEED:
+				return Optional.of(new TestFeedPrintingListener(out, colorPalette));
 			default:
 				return Optional.empty();
 		}
diff --git a/junit-platform-console/src/main/java/org/junit/platform/console/tasks/DetailsPrintingListener.java b/junit-platform-console/src/main/java/org/junit/platform/console/tasks/DetailsPrintingListener.java
index 3eceb2721b6a..ece60eb52fbf 100644
--- a/junit-platform-console/src/main/java/org/junit/platform/console/tasks/DetailsPrintingListener.java
+++ b/junit-platform-console/src/main/java/org/junit/platform/console/tasks/DetailsPrintingListener.java
@@ -10,18 +10,18 @@
 
 package org.junit.platform.console.tasks;
 
-import static org.apiguardian.api.API.Status.EXPERIMENTAL;
+import java.util.regex.Pattern;
 
-import org.apiguardian.api.API;
 import org.junit.platform.launcher.TestExecutionListener;
 import org.junit.platform.launcher.TestPlan;
 
-/**
- * @since 1.9
- */
-@API(status = EXPERIMENTAL, since = "1.9")
-public interface DetailsPrintingListener extends TestExecutionListener {
+interface DetailsPrintingListener extends TestExecutionListener {
+
+	Pattern LINE_START_PATTERN = Pattern.compile("(?m)^");
 
 	void listTests(TestPlan testPlan);
 
+	static String indented(String message, String indentation) {
+		return LINE_START_PATTERN.matcher(message).replaceAll(indentation).trim();
+	}
 }
diff --git a/junit-platform-console/src/main/java/org/junit/platform/console/tasks/FlatPrintingListener.java b/junit-platform-console/src/main/java/org/junit/platform/console/tasks/FlatPrintingListener.java
index f572d71d1907..304cedc27a2f 100644
--- a/junit-platform-console/src/main/java/org/junit/platform/console/tasks/FlatPrintingListener.java
+++ b/junit-platform-console/src/main/java/org/junit/platform/console/tasks/FlatPrintingListener.java
@@ -11,7 +11,6 @@
 package org.junit.platform.console.tasks;
 
 import java.io.PrintWriter;
-import java.util.regex.Pattern;
 
 import org.junit.platform.commons.util.ExceptionUtils;
 import org.junit.platform.engine.TestExecutionResult;
@@ -24,8 +23,6 @@
  */
 class FlatPrintingListener implements DetailsPrintingListener {
 
-	private static final Pattern LINE_START_PATTERN = Pattern.compile("(?m)^");
-
 	static final String INDENTATION = "             ";
 
 	private final PrintWriter out;
@@ -102,7 +99,7 @@ private void println(Style style, String format, Object... args) {
 	 * @return indented message
 	 */
 	private static String indented(String message) {
-		return LINE_START_PATTERN.matcher(message).replaceAll(INDENTATION).trim();
+		return DetailsPrintingListener.indented(message, INDENTATION);
 	}
 
 	@Override
diff --git a/junit-platform-console/src/main/java/org/junit/platform/console/tasks/TestFeedPrintingListener.java b/junit-platform-console/src/main/java/org/junit/platform/console/tasks/TestFeedPrintingListener.java
new file mode 100644
index 000000000000..c7c273c1a92e
--- /dev/null
+++ b/junit-platform-console/src/main/java/org/junit/platform/console/tasks/TestFeedPrintingListener.java
@@ -0,0 +1,130 @@
+/*
+ * Copyright 2015-2023 the original author or authors.
+ *
+ * All rights reserved. This program and the accompanying materials are
+ * made available under the terms of the Eclipse Public License v2.0 which
+ * accompanies this distribution and is available at
+ *
+ * https://www.eclipse.org/legal/epl-v20.html
+ */
+
+package org.junit.platform.console.tasks;
+
+import static org.junit.platform.engine.TestExecutionResult.Status.SUCCESSFUL;
+
+import java.io.PrintWriter;
+import java.util.ArrayList;
+import java.util.List;
+
+import org.junit.platform.commons.util.ExceptionUtils;
+import org.junit.platform.engine.TestExecutionResult;
+import org.junit.platform.engine.UniqueId;
+import org.junit.platform.launcher.TestIdentifier;
+import org.junit.platform.launcher.TestPlan;
+
+class TestFeedPrintingListener implements DetailsPrintingListener {
+
+	private static final String INDENTATION = "\t";
+	private static final String STATUS_SEPARATOR = " :: ";
+
+	private final PrintWriter out;
+	private final ColorPalette colorPalette;
+	private TestPlan testPlan;
+
+	TestFeedPrintingListener(PrintWriter out, ColorPalette colorPalette) {
+		this.out = out;
+		this.colorPalette = colorPalette;
+	}
+
+	@Override
+	public void testPlanExecutionStarted(TestPlan testPlan) {
+		this.testPlan = testPlan;
+	}
+
+	@Override
+	public void testPlanExecutionFinished(TestPlan testPlan) {
+		this.testPlan = null;
+	}
+
+	@Override
+	public void executionSkipped(TestIdentifier testIdentifier, String reason) {
+		if (shouldPrint(testIdentifier)) {
+			String msg = formatTestIdentifier(testIdentifier);
+			String indentedReason = indented(String.format("Reason: %s", reason));
+			println(Style.SKIPPED,
+				String.format("%s" + STATUS_SEPARATOR + "SKIPPED%n" + INDENTATION + "%s", msg, indentedReason));
+		}
+	}
+
+	@Override
+	public void executionStarted(TestIdentifier testIdentifier) {
+		if (shouldPrint(testIdentifier)) {
+			String msg = formatTestIdentifier(testIdentifier);
+			println(Style.NONE, String.format("%s" + STATUS_SEPARATOR + "STARTED", msg));
+		}
+	}
+
+	@Override
+	public void executionFinished(TestIdentifier testIdentifier, TestExecutionResult testExecutionResult) {
+		TestExecutionResult.Status status = testExecutionResult.getStatus();
+		if (testExecutionResult.getThrowable().isPresent()) {
+			Style style = Style.valueOf(testExecutionResult);
+			String msg = formatTestIdentifier(testIdentifier);
+			Throwable throwable = testExecutionResult.getThrowable().get();
+			String stacktrace = indented(ExceptionUtils.readStackTrace(throwable));
+			println(style,
+				String.format("%s" + STATUS_SEPARATOR + "%s%n" + INDENTATION + "%s", msg, status, stacktrace));
+		}
+		else if (shouldPrint(testIdentifier) || testExecutionResult.getStatus() != SUCCESSFUL) {
+			Style style = Style.valueOf(testExecutionResult);
+			String msg = formatTestIdentifier(testIdentifier);
+			println(style, String.format("%s" + STATUS_SEPARATOR + "%s", msg, status));
+		}
+	}
+
+	private String formatTestIdentifier(TestIdentifier testIdentifier) {
+		return String.join(" > ", collectDisplayNames(testIdentifier.getUniqueIdObject()));
+	}
+
+	private void println(Style style, String message) {
+		this.out.println(colorPalette.paint(style, message));
+	}
+
+	private List<String> collectDisplayNames(UniqueId uniqueId) {
+		int size = uniqueId.getSegments().size();
+		List<String> displayNames = new ArrayList<>(size);
+		for (int i = 0; i < size; i++) {
+			displayNames.add(0, testPlan.getTestIdentifier(uniqueId).getDisplayName());
+			if (i < size - 1) {
+				uniqueId = uniqueId.removeLastSegment();
+			}
+		}
+		return displayNames;
+	}
+
+	private static String indented(String message) {
+		return DetailsPrintingListener.indented(message, INDENTATION);
+	}
+
+	@Override
+	public void listTests(TestPlan testPlan) {
+		this.testPlan = testPlan;
+		try {
+			testPlan.accept(new TestPlan.Visitor() {
+				@Override
+				public void visit(TestIdentifier testIdentifier) {
+					if (shouldPrint(testIdentifier)) {
+						println(Style.NONE, formatTestIdentifier(testIdentifier));
+					}
+				}
+			});
+		}
+		finally {
+			this.testPlan = null;
+		}
+	}
+
+	private static boolean shouldPrint(TestIdentifier testIdentifier) {
+		return testIdentifier.isTest();
+	}
+}