Repository: dotnet/aspire
Pull request #9223: [CI] Generate simple test summary
Author: login: radical
Labels: name: area-engineering-systems
description: infrastructure helix infra engineering repo stuff
Created: 2025-05-10T03:36:27Z | Merged: 2025-05-12T05:50:10Z

`Usage: dotnet tools run GenerateTestSummary --dirPathOrTrxFilePath <path> [--output <output>] [--combined]`
- Generates a summary report from trx files.
   - And writes to $GITHUB_STEP_SUMMARY if running in GitHub Actions

- This can generate individual summary for a test assembly showing details of the test failures.
- And a combined report for multiple test assemblies with the total counters shown.


Diff:
diff --git a/.github/workflows/run-tests.yml b/.github/workflows/run-tests.yml
index 8eeba91a4c0..9ff92b739bc 100644
--- a/.github/workflows/run-tests.yml
+++ b/.github/workflows/run-tests.yml
@@ -283,6 +283,7 @@ jobs:
           docker network ls
 
       - name: Upload logs, and test results
+        id: upload-logs
         if: always()
         uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
         with:
@@ -292,3 +293,15 @@ jobs:
             testresults/**
             artifacts/bin/Aspire.Templates.Tests/Debug/net8.0/logs/**
             artifacts/log/test-logs/**
+
+      - name: Generate test results summary
+        if: always()
+        env:
+          CI: false
+        run: >
+          ${{ github.workspace }}/dotnet.sh
+          run
+          --project ${{ github.workspace }}/tools/GenerateTestSummary/GenerateTestSummary.csproj
+          --
+          ${{ github.workspace }}/testresults
+          -u ${{ steps.upload-logs.outputs.artifact-url }}
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index 9f14c896e2f..e14bdfba4ce 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -148,6 +148,9 @@ jobs:
     name: Final Results
     needs: [ integrations_test_lin, integrations_test_win, templates_test_lin, templates_test_win, endtoend_tests ]
     steps:
+      - name: Checkout code
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+
       # get all the test-job-result* artifacts into a single directory
       - uses: actions/download-artifact@cc203385981b70ca67e1cc392babf9cc229d5806 # v4.1.9
         with:
@@ -159,20 +162,32 @@ jobs:
         with:
           pattern: logs-*-ubuntu-latest
           merge-multiple: true
-          path: testresults/ubuntu-latest
+          path: ${{ github.workspace }}/testresults/ubuntu-latest
 
       - uses: actions/download-artifact@cc203385981b70ca67e1cc392babf9cc229d5806 # v4.1.9
         with:
           pattern: logs-*-windows-latest
           merge-multiple: true
-          path: testresults/windows-latest
+          path: ${{ github.workspace }}/testresults/windows-latest
 
       - name: Upload test results
         if: always()
         uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
         with:
           name: All-TestResults
-          path: testresults/**/*.trx
+          path: ${{ github.workspace }}/testresults/**/*.trx
+
+      - name: Generate test results summary
+        if: always()
+        env:
+          CI: false
+        run: >
+          ${{ github.workspace }}/dotnet.sh
+          run
+          --project ${{ github.workspace }}/tools/GenerateTestSummary/GenerateTestSummary.csproj
+          --
+          ${{ github.workspace }}/testresults
+          --combined
 
       # return success if zero result-failed-* files are found
       - name: Compute result
diff --git a/Aspire.sln b/Aspire.sln
index 1a8387b15d3..efe28447dac 100644
--- a/Aspire.sln
+++ b/Aspire.sln
@@ -678,6 +678,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "WebApplication1", "playgrou
 EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "WebApplication2", "playground\SqlServerScript\WebApplication2\WebApplication2.csproj", "{554D72B3-F0B0-FB9A-67ED-BBDF55A6DE81}"
 EndProject
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GenerateTestSummary", "tools\GenerateTestSummary\GenerateTestSummary.csproj", "{29950A00-A83A-48D3-8739-EE3D667B5229}"
+EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Any CPU = Debug|Any CPU
@@ -3976,6 +3978,18 @@ Global
 		{554D72B3-F0B0-FB9A-67ED-BBDF55A6DE81}.Release|x64.Build.0 = Release|Any CPU
 		{554D72B3-F0B0-FB9A-67ED-BBDF55A6DE81}.Release|x86.ActiveCfg = Release|Any CPU
 		{554D72B3-F0B0-FB9A-67ED-BBDF55A6DE81}.Release|x86.Build.0 = Release|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Debug|x64.ActiveCfg = Debug|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Debug|x64.Build.0 = Debug|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Debug|x86.ActiveCfg = Debug|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Debug|x86.Build.0 = Debug|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Release|Any CPU.Build.0 = Release|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Release|x64.ActiveCfg = Release|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Release|x64.Build.0 = Release|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Release|x86.ActiveCfg = Release|Any CPU
+		{29950A00-A83A-48D3-8739-EE3D667B5229}.Release|x86.Build.0 = Release|Any CPU
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE
@@ -4302,6 +4316,7 @@ Global
 		{3928CF69-B803-43A2-8AE5-5E29CB3E8D24} = {02EA681E-C7D8-13C7-8484-4AC65E1B71E8}
 		{E79A95EA-08D9-9947-377D-6F2213B36E1B} = {02EA681E-C7D8-13C7-8484-4AC65E1B71E8}
 		{554D72B3-F0B0-FB9A-67ED-BBDF55A6DE81} = {02EA681E-C7D8-13C7-8484-4AC65E1B71E8}
+		{29950A00-A83A-48D3-8739-EE3D667B5229} = {2136E31D-2CBB-41BB-8618-716FF8E46E9E}
 	EndGlobalSection
 	GlobalSection(ExtensibilityGlobals) = postSolution
 		SolutionGuid = {47DCFECF-5631-4BDE-A1EC-BE41E90F60C4}
diff --git a/tools/GenerateTestSummary/GenerateTestSummary.csproj b/tools/GenerateTestSummary/GenerateTestSummary.csproj
new file mode 100644
index 00000000000..ef372971aa6
--- /dev/null
+++ b/tools/GenerateTestSummary/GenerateTestSummary.csproj
@@ -0,0 +1,14 @@
+﻿<Project Sdk="Microsoft.NET.Sdk">
+
+  <PropertyGroup>
+    <OutputType>Exe</OutputType>
+    <TargetFramework>net9.0</TargetFramework>
+    <ImplicitUsings>enable</ImplicitUsings>
+    <Nullable>enable</Nullable>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <PackageReference Include="System.CommandLine" />
+  </ItemGroup>
+
+</Project>
diff --git a/tools/GenerateTestSummary/Program.cs b/tools/GenerateTestSummary/Program.cs
new file mode 100644
index 00000000000..7f1adc98e63
--- /dev/null
+++ b/tools/GenerateTestSummary/Program.cs
@@ -0,0 +1,76 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System.Text;
+using System.CommandLine;
+using Aspire.TestTools;
+
+// Usage: dotnet tools run GenerateTestSummary --dirPathOrTrxFilePath <path> [--output <output>] [--combined]
+// Generate a summary report from trx files.
+// And write to $GITHUB_STEP_SUMMARY if running in GitHub Actions.
+
+var dirPathOrTrxFilePathArgument = new Argument<string>("dirPathOrTrxFilePath");
+var outputOption = new Option<string>("--output", "-o") { Description = "Output file path" };
+var combinedSummaryOption = new Option<bool>("--combined", "-c") { Description = "Generate combined summary report" };
+var urlOption = new Option<string>("--url", "-u") { Description = "URL for test links" };
+
+var rootCommand = new RootCommand
+{
+    dirPathOrTrxFilePathArgument,
+    outputOption,
+    combinedSummaryOption,
+    urlOption
+};
+
+rootCommand.SetAction(result =>
+{
+    var dirPathOrTrxFilePath = result.GetValue<string>(dirPathOrTrxFilePathArgument);
+    if (string.IsNullOrEmpty(dirPathOrTrxFilePath))
+    {
+        Console.WriteLine("Please provide a directory path with trx files or a trx file path.");
+        return;
+    }
+
+    var combinedSummary = result.GetValue<bool>(combinedSummaryOption);
+
+    string report;
+    if (combinedSummary)
+    {
+        report = TestSummaryGenerator.CreateCombinedTestSummaryReport(dirPathOrTrxFilePath);
+    }
+    else
+    {
+        var reportBuilder = new StringBuilder();
+        if (Directory.Exists(dirPathOrTrxFilePath))
+        {
+            var trxFiles = Directory.EnumerateFiles(dirPathOrTrxFilePath, "*.trx", SearchOption.AllDirectories);
+            foreach (var trxFile in trxFiles)
+            {
+                TestSummaryGenerator.CreateSingleTestSummaryReport(trxFile, reportBuilder);
+            }
+        }
+        else
+        {
+            TestSummaryGenerator.CreateSingleTestSummaryReport(dirPathOrTrxFilePath, reportBuilder, result.GetValue<string>(urlOption));
+        }
+
+        report = reportBuilder.ToString();
+    }
+
+    var outputFilePath = result.GetValue<string>(outputOption);
+    if (outputFilePath is not null)
+    {
+        File.WriteAllText(outputFilePath, report);
+        Console.WriteLine($"Report written to {outputFilePath}");
+    }
+
+    if (report.Length > 0
+        && Environment.GetEnvironmentVariable("GITHUB_ACTIONS") == "true"
+        && Environment.GetEnvironmentVariable("GITHUB_STEP_SUMMARY") is string summaryPath
+        && !string.IsNullOrEmpty(summaryPath))
+    {
+        File.WriteAllText(summaryPath, report);
+    }
+});
+
+return rootCommand.Parse(args).Invoke();
diff --git a/tools/GenerateTestSummary/TestSummaryGenerator.cs b/tools/GenerateTestSummary/TestSummaryGenerator.cs
new file mode 100644
index 00000000000..22ae9c377e1
--- /dev/null
+++ b/tools/GenerateTestSummary/TestSummaryGenerator.cs
@@ -0,0 +1,171 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System.Text;
+using System.Globalization;
+using System.Text.RegularExpressions;
+
+namespace Aspire.TestTools;
+
+sealed partial class TestSummaryGenerator
+{
+    public static string CreateCombinedTestSummaryReport(string basePath)
+    {
+        if (!Directory.Exists(basePath))
+        {
+            throw new DirectoryNotFoundException($"The directory '{basePath}' does not exist.");
+        }
+
+        var trxFiles = System.IO.Directory.EnumerateFiles(basePath, "*.trx", System.IO.SearchOption.AllDirectories);
+
+        int overallTotalTestCount = 0;
+        int overallPassedTestCount = 0;
+        int overallFailedTestCount = 0;
+        int overallSkippedTestCount = 0;
+
+        // Update to use markdown tables instead of HTML
+        var tableBuilder = new StringBuilder();
+        tableBuilder.AppendLine("| Name | Passed | Failed | Skipped | Total |");
+        tableBuilder.AppendLine("|------|--------|--------|---------|-------|");
+
+        foreach (var file in trxFiles.OrderBy(f => Path.GetFileName(f)))
+        {
+            TestRun? testRun;
+            try
+            {
+                testRun = TrxReader.DeserializeTrxFile(file);
+                if (testRun == null || testRun.ResultSummary?.Counters == null)
+                {
+                    Console.WriteLine($"Failed to deserialize or find results in file: {file}, tr: {testRun}");
+                    continue;
+                }
+            }
+            catch (Exception ex)
+            {
+                Console.WriteLine($"Failed to deserialize file: {file}, exception: {ex}");
+                continue;
+            }
+
+            // emit row for each trx file
+            var counters = testRun.ResultSummary.Counters;
+            int total = counters.Total;
+            int passed = counters.Passed;
+            int failed = counters.Failed;
+            int skipped = counters.NotExecuted;
+
+            overallTotalTestCount += total;
+            overallPassedTestCount += passed;
+            overallFailedTestCount += failed;
+            overallSkippedTestCount += skipped;
+
+            tableBuilder.AppendLine(CultureInfo.InvariantCulture, $"| {(failed > 0 ? "❌" : "✅")} {GetTestTitle(file)} | {passed} | {failed} | {skipped} | {total} |");
+        }
+
+        var overallTableBuilder = new StringBuilder();
+        overallTableBuilder.AppendLine("## Overall Summary");
+
+        overallTableBuilder.AppendLine("| Passed | Failed | Skipped | Total |");
+        overallTableBuilder.AppendLine("|--------|--------|---------|-------|");
+        overallTableBuilder.AppendLine(CultureInfo.InvariantCulture, $"| {overallPassedTestCount} | {overallFailedTestCount} | {overallSkippedTestCount} | {overallTotalTestCount} |");
+
+        overallTableBuilder.AppendLine();
+        overallTableBuilder.Append(tableBuilder);
+
+        return overallTableBuilder.ToString();
+    }
+
+    public static void CreateSingleTestSummaryReport(string trxFilePath, StringBuilder reportBuilder, string? url = null)
+    {
+        if (!File.Exists(trxFilePath))
+        {
+            throw new FileNotFoundException($"The file '{trxFilePath}' does not exist.");
+        }
+
+        TestRun? testRun;
+        try
+        {
+            testRun = TrxReader.DeserializeTrxFile(trxFilePath);
+            if (testRun == null || testRun.ResultSummary?.Counters == null)
+            {
+                throw new InvalidOperationException($"Failed to deserialize or find results in file: {trxFilePath}");
+            }
+        }
+        catch (Exception ex)
+        {
+            throw new InvalidOperationException($"Failed to process file: {trxFilePath}", ex);
+        }
+
+        var counters = testRun.ResultSummary.Counters;
+        var failed = counters.Failed;
+        if (failed == 0)
+        {
+            return;
+        }
+
+        var total = counters.Total;
+        var passed = counters.Passed;
+        var skipped = counters.NotExecuted;
+
+        reportBuilder.AppendLine(CultureInfo.InvariantCulture, $"### {GetTestTitle(trxFilePath)}");
+        reportBuilder.AppendLine("| Passed | Failed | Skipped | Total |");
+        reportBuilder.AppendLine("|--------|--------|---------|-------|");
+        reportBuilder.AppendLine(CultureInfo.InvariantCulture, $"| {passed} | {failed} | {skipped} | {total} |");
+
+        reportBuilder.AppendLine();
+        if (testRun.Results?.UnitTestResults is null)
+        {
+            return;
+        }
+
+        var failedTests = testRun.Results.UnitTestResults.Where(r => r.Outcome == "Failed");
+        if (failedTests.Any())
+        {
+            foreach (var test in failedTests)
+            {
+                var title = string.IsNullOrEmpty(url)
+                                ? $"🔴 <b>{test.TestName}</b>"
+                                : $"🔴 <a href=\"{url}\">{test.TestName}</a>";
+
+                reportBuilder.AppendLine("<div>");
+                reportBuilder.AppendLine(CultureInfo.InvariantCulture, $"""
+                    <details><summary>{title}</summary>
+                """);
+
+                var errorMsgBuilder = new StringBuilder();
+                errorMsgBuilder.AppendLine(test.Output?.ErrorInfo?.InnerText ?? string.Empty);
+                errorMsgBuilder.AppendLine(test.Output?.StdOut ?? string.Empty);
+
+                // Truncate long error messages for readability
+                var errorMsgTruncated = TruncateTheStart(errorMsgBuilder.ToString(), 50_000);
+
+                reportBuilder.AppendLine();
+                reportBuilder.AppendLine("```yml");
+                reportBuilder.AppendLine(errorMsgTruncated);
+                reportBuilder.AppendLine("```");
+                reportBuilder.AppendLine();
+                reportBuilder.AppendLine("</div>");
+            }
+        }
+        reportBuilder.AppendLine();
+    }
+
+    public static string GetTestTitle(string trxFileName)
+    {
+        var filename = Path.GetFileNameWithoutExtension(trxFileName);
+        var match = TestNameFromTrxFileNameRegex().Match(filename);
+        if (match.Success)
+        {
+            return $"{match.Groups["testName"].Value} ({match.Groups["tfm"].Value})";
+        }
+
+        return filename;
+    }
+
+    [GeneratedRegex(@"(?<testName>.*)_(?<tfm>net\d+\.0)_.*")]
+    private static partial Regex TestNameFromTrxFileNameRegex();
+
+    private static string? TruncateTheStart(string? s, int maxLength)
+        => s is null || s.Length <= maxLength
+            ? s
+            : "... (truncated) " + s[^maxLength..];
+}
diff --git a/tools/GenerateTestSummary/TrxReader.cs b/tools/GenerateTestSummary/TrxReader.cs
new file mode 100644
index 00000000000..4ccc2e389ff
--- /dev/null
+++ b/tools/GenerateTestSummary/TrxReader.cs
@@ -0,0 +1,149 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System.Globalization;
+using System.Xml;
+using System.Xml.Serialization;
+
+namespace Aspire.TestTools;
+
+public class TrxReader
+{
+    public static IList<TestResult> GetTestResultsFromTrx(string filepath, Func<string, string, bool>? testFilter = null)
+    {
+        XmlSerializer serializer = new(typeof(TestRun));
+        using FileStream fileStream = new(filepath, FileMode.Open);
+
+        if (serializer.Deserialize(fileStream) is not TestRun testRun || testRun.Results?.UnitTestResults is null)
+        {
+            return Array.Empty<TestResult>();
+        }
+
+        var testResults = new List<TestResult>();
+
+        foreach (var unitTestResult in testRun.Results.UnitTestResults)
+        {
+            if (string.IsNullOrEmpty(unitTestResult.TestName) || string.IsNullOrEmpty(unitTestResult.Outcome))
+            {
+                continue;
+            }
+
+            if (testFilter is not null && !testFilter(unitTestResult.TestName, unitTestResult.Outcome))
+            {
+                continue;
+            }
+
+            var startTime = unitTestResult.StartTime;
+            var endTime = unitTestResult.EndTime;
+
+            testResults.Add(new TestResult(
+                Name: unitTestResult.TestName,
+                Outcome: unitTestResult.Outcome,
+                StartTime: startTime is null ? TimeSpan.MinValue : TimeSpan.Parse(startTime, CultureInfo.InvariantCulture),
+                EndTime: endTime is null ? TimeSpan.MinValue : TimeSpan.Parse(endTime, CultureInfo.InvariantCulture),
+                ErrorMessage: unitTestResult.Output?.ErrorInfoString,
+                Stdout: unitTestResult.Output?.StdOut
+            ));
+        }
+
+        return testResults;
+    }
+
+    public static TestRun? DeserializeTrxFile(string filePath)
+    {
+        if (string.IsNullOrWhiteSpace(filePath))
+        {
+            throw new ArgumentException($"{nameof(filePath)} cannot be null or empty.", nameof(filePath));
+        }
+
+        XmlSerializer serializer = new(typeof(TestRun));
+
+        using FileStream fileStream = new(filePath, FileMode.Open, FileAccess.Read);
+        return serializer.Deserialize(fileStream) as TestRun;
+    }
+}
+
+[XmlRoot("TestRun", Namespace = "http://microsoft.com/schemas/VisualStudio/TeamTest/2010")]
+public class TestRun
+{
+    public Results? Results { get; set; }
+
+    public ResultSummary? ResultSummary { get; set; }
+}
+
+public class Results
+{
+    [XmlElement("UnitTestResult")]
+    public List<UnitTestResult>? UnitTestResults { get; set; }
+}
+
+public class UnitTestResult
+{
+    [XmlAttribute("testName")]
+    public string? TestName { get; set; }
+
+    [XmlAttribute("outcome")]
+    public string? Outcome { get; set; }
+
+    [XmlAttribute("startTime")]
+    public string? StartTime { get; set; }
+
+    [XmlAttribute("endTime")]
+    public string? EndTime { get; set; }
+
+    public Output? Output { get; set; }
+}
+
+public class Output
+{
+    [XmlAnyElement]
+    public XmlElement? ErrorInfo { get; set; }
+    public string? StdOut { get; set; }
+
+    [XmlIgnore]
+    public string ErrorInfoString => ErrorInfo?.InnerText ?? string.Empty;
+}
+
+public class ResultSummary
+{
+    public string? Outcome { get; set; }
+    public Counters? Counters { get; set; }
+}
+
+public class Counters
+{
+    [XmlAttribute("total")]
+    public int Total { get; set; }
+
+    [XmlAttribute("executed")]
+    public int Executed { get; set; }
+
+    [XmlAttribute("passed")]
+    public int Passed { get; set; }
+
+    [XmlAttribute("failed")]
+    public int Failed { get; set; }
+
+    [XmlAttribute("error")]
+    public int Error { get; set; }
+
+    [XmlAttribute("timeout")]
+    public int Timeout { get; set; }
+
+    [XmlAttribute("aborted")]
+    public int Aborted { get; set; }
+
+    [XmlAttribute("inconclusive")]
+    public int Inconclusive { get; set; }
+
+    [XmlAttribute("passedButRunAborted")]
+    public int PassedButRunAborted { get; set; }
+
+    [XmlAttribute("notRunnable")]
+    public int NotRunnable { get; set; }
+
+    [XmlAttribute("notExecuted")]
+    public int NotExecuted { get; set; }
+}
+
+public record TestResult(string Name, string Outcome, TimeSpan StartTime, TimeSpan EndTime, string? ErrorMessage = null, string? Stdout = null);


Reviews:
user: login: copilot-pull-request-reviewer[bot]
body: ## Pull Request Overview

This PR adds a new tool for generating test summary reports from trx files, including both combined and single reports, and integrates it into the CI workflows.  
- Introduces a new project, GenerateTestSummary, with logic for reading trx files and creating markdown reports.  
- Updates GitHub workflows to use the new tool for generating test result summaries.  
- Integrates the new project into the solution.

### Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

<details>
<summary>Show a summary per file</summary>

| File | Description |
| ---- | ----------- |
| tools/GenerateTestSummary/TrxReader.cs | Adds functionality to deserialize and process trx files into test results. |
| tools/GenerateTestSummary/TestSummaryGenerator.cs | Implements report generation logic for both combined and single test summaries. |
| tools/GenerateTestSummary/Program.cs | Provides command-line entry point to run the test summary generation. |
| tools/GenerateTestSummary/GenerateTestSummary.csproj | New project file setup for the test summary tool. |
| Aspire.sln | Integrates the new project into the solution. |
| .github/workflows/tests.yml | Updates CI steps to use the new test summary tool with absolute paths. |
| .github/workflows/run-tests.yml | Adds a step to generate test summaries and pass URLs for logs. |
</details>





state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T05:21:47Z
user: login: davidfowl
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T05:36:54Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T09:22:27Z

Review comments:
diff_hunk: @@ -0,0 +1,178 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System.Text;
+using System.Globalization;
+using System.Text.RegularExpressions;
+
+namespace Aspire.TestTools;
+
+sealed partial class TestSummaryGenerator
+{
+    public static string CreateCombinedTestSummaryReport(string basePath)
+    {
+        if (!Directory.Exists(basePath))
+        {
+            throw new DirectoryNotFoundException($"The directory '{basePath}' does not exist.");
+        }
+
+        var trxFiles = System.IO.Directory.EnumerateFiles(basePath, "*.trx", System.IO.SearchOption.AllDirectories);
+
+        int overallTotalTestCount = 0;
+        int overallPassedTestCount = 0;
+        int overallFailedTestCount = 0;
+        int overallSkippedTestCount = 0;
+
+        // Update to use markdown tables instead of HTML
+        var tableBuilder = new StringBuilder();
+        tableBuilder.AppendLine("| Name | Passed | Failed | Skipped | Total |");
+        tableBuilder.AppendLine("|------|--------|--------|---------|-------|");
+
+        foreach (var file in trxFiles.OrderBy(f => Path.GetFileName(f)))
+        {
+            TestRun? testRun;
+            try
+            {
+                testRun = TrxReader.DeserializeTrxFile(file);
+                if (testRun == null || testRun.ResultSummary?.Counters == null)
+                {
+                    Console.WriteLine($"Failed to deserialize or find results in file: {file}, tr: {testRun}");
+                    continue;
+                }
+            }
+            catch (Exception ex)
+            {
+                Console.WriteLine($"Failed to deserialize file: {file}, exception: {ex}");
+                continue;
+            }
+
+            // emit row for each trx file
+            var counters = testRun.ResultSummary.Counters;
+            int total = counters.Total;
+            int passed = counters.Passed;
+            int failed = counters.Failed;
+            int skipped = counters.NotExecuted;
+
+            overallTotalTestCount += total;
+            overallPassedTestCount += passed;
+            overallFailedTestCount += failed;
+            overallSkippedTestCount += skipped;
+
+            tableBuilder.AppendLine(CultureInfo.InvariantCulture, $"| {(failed > 0 ? "❌" : "✅")} {GetTestTitle(file)} | {passed} | {failed} | {skipped} | {total} |");
+        }
+
+        var overallTableBuilder = new StringBuilder();
+        overallTableBuilder.AppendLine("## Overall Summary");
+
+        overallTableBuilder.AppendLine("| Passed | Failed | Skipped | Total |");
+        overallTableBuilder.AppendLine("|--------|--------|---------|-------|");
+        overallTableBuilder.AppendLine(CultureInfo.InvariantCulture, $"| {overallPassedTestCount} | {overallFailedTestCount} | {overallSkippedTestCount} | {overallTotalTestCount} |");
+
+        overallTableBuilder.AppendLine();
+        overallTableBuilder.Append(tableBuilder);
+
+        return overallTableBuilder.ToString();
+    }
+
+    public static void CreateSingleTestSummaryReport(string trxFilePath, StringBuilder reportBuilder, string? url = null)
+    {
+        if (!File.Exists(trxFilePath))
+        {
+            throw new FileNotFoundException($"The file '{trxFilePath}' does not exist.");
+        }
+
+        TestRun? testRun;
+        try
+        {
+            testRun = TrxReader.DeserializeTrxFile(trxFilePath);
+            if (testRun == null || testRun.ResultSummary?.Counters == null)
+            {
+                throw new InvalidOperationException($"Failed to deserialize or find results in file: {trxFilePath}");
+            }
+        }
+        catch (Exception ex)
+        {
+            throw new InvalidOperationException($"Failed to process file: {trxFilePath}", ex);
+        }
+
+        var counters = testRun.ResultSummary.Counters;
+        var failed = counters.Failed;
+        if (failed == 0)
+        {
+            return;
+        }
+
+        var total = counters.Total;
+        var passed = counters.Passed;
+        var skipped = counters.NotExecuted;
+
+        reportBuilder.AppendLine(CultureInfo.InvariantCulture, $"### {GetTestTitle(trxFilePath)}");
+        reportBuilder.AppendLine("| Passed | Failed | Skipped | Total |");
+        reportBuilder.AppendLine("|--------|--------|---------|-------|");
+        reportBuilder.AppendLine(CultureInfo.InvariantCulture, $"| {passed} | {failed} | {skipped} | {total} |");
+
+        reportBuilder.AppendLine();
+        if (testRun.Results?.UnitTestResults is null)
+        {
+            return;
+        }
+
+        var failedTests = testRun.Results.UnitTestResults.Where(r => r.Outcome == "Failed");
+        if (failedTests.Any())
+        {
+            foreach (var test in failedTests)
+            {
+                string title;
+                if (string.IsNullOrEmpty(url))
+                {
+                    title = $"🔴 <b>{test.TestName}</b>";
+                }
+                else
+                {
+
+                    title = $"🔴 <a href=\"{url}\">{test.TestName}</a>";
+                }
+                reportBuilder.AppendLine("<div>");
+                reportBuilder.AppendLine(CultureInfo.InvariantCulture, $"""
+                    <details><summary>{title}</summary>
+                """);
+
+                var errorMsgBuilder = new StringBuilder();
+                errorMsgBuilder.AppendLine(test.Output?.ErrorInfo?.InnerText ?? string.Empty);
+                errorMsgBuilder.AppendLine(test.Output?.StdOut ?? string.Empty);
+
+                var errorMsgTruncated = TruncateTheStart(errorMsgBuilder.ToString(), 50_000); // Truncate long error messages for readability
+
+                // Truncate long error messages for readability
path: tools/GenerateTestSummary/TestSummaryGenerator.cs
user: login: Copilot
body: Duplicate comment regarding truncating error messages is present. Consider removing the redundant comment to improve readability.
```suggestion

```
created_at: 2025-05-12T05:21:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,178 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System.Text;
+using System.Globalization;
+using System.Text.RegularExpressions;
+
+namespace Aspire.TestTools;
+
+sealed partial class TestSummaryGenerator
+{
+    public static string CreateCombinedTestSummaryReport(string basePath)
+    {
+        if (!Directory.Exists(basePath))
+        {
+            throw new DirectoryNotFoundException($"The directory '{basePath}' does not exist.");
+        }
+
+        var trxFiles = System.IO.Directory.EnumerateFiles(basePath, "*.trx", System.IO.SearchOption.AllDirectories);
+
+        int overallTotalTestCount = 0;
+        int overallPassedTestCount = 0;
+        int overallFailedTestCount = 0;
+        int overallSkippedTestCount = 0;
+
+        // Update to use markdown tables instead of HTML
+        var tableBuilder = new StringBuilder();
+        tableBuilder.AppendLine("| Name | Passed | Failed | Skipped | Total |");
+        tableBuilder.AppendLine("|------|--------|--------|---------|-------|");
+
+        foreach (var file in trxFiles.OrderBy(f => Path.GetFileName(f)))
+        {
+            TestRun? testRun;
+            try
+            {
+                testRun = TrxReader.DeserializeTrxFile(file);
+                if (testRun == null || testRun.ResultSummary?.Counters == null)
+                {
+                    Console.WriteLine($"Failed to deserialize or find results in file: {file}, tr: {testRun}");
+                    continue;
+                }
+            }
+            catch (Exception ex)
+            {
+                Console.WriteLine($"Failed to deserialize file: {file}, exception: {ex}");
+                continue;
+            }
+
+            // emit row for each trx file
+            var counters = testRun.ResultSummary.Counters;
+            int total = counters.Total;
+            int passed = counters.Passed;
+            int failed = counters.Failed;
+            int skipped = counters.NotExecuted;
+
+            overallTotalTestCount += total;
+            overallPassedTestCount += passed;
+            overallFailedTestCount += failed;
+            overallSkippedTestCount += skipped;
+
+            tableBuilder.AppendLine(CultureInfo.InvariantCulture, $"| {(failed > 0 ? "❌" : "✅")} {GetTestTitle(file)} | {passed} | {failed} | {skipped} | {total} |");
+        }
+
+        var overallTableBuilder = new StringBuilder();
+        overallTableBuilder.AppendLine("## Overall Summary");
+
+        overallTableBuilder.AppendLine("| Passed | Failed | Skipped | Total |");
+        overallTableBuilder.AppendLine("|--------|--------|---------|-------|");
+        overallTableBuilder.AppendLine(CultureInfo.InvariantCulture, $"| {overallPassedTestCount} | {overallFailedTestCount} | {overallSkippedTestCount} | {overallTotalTestCount} |");
+
+        overallTableBuilder.AppendLine();
+        overallTableBuilder.Append(tableBuilder);
+
+        return overallTableBuilder.ToString();
+    }
+
+    public static void CreateSingleTestSummaryReport(string trxFilePath, StringBuilder reportBuilder, string? url = null)
+    {
+        if (!File.Exists(trxFilePath))
+        {
+            throw new FileNotFoundException($"The file '{trxFilePath}' does not exist.");
+        }
+
+        TestRun? testRun;
+        try
+        {
+            testRun = TrxReader.DeserializeTrxFile(trxFilePath);
+            if (testRun == null || testRun.ResultSummary?.Counters == null)
+            {
+                throw new InvalidOperationException($"Failed to deserialize or find results in file: {trxFilePath}");
+            }
+        }
+        catch (Exception ex)
+        {
+            throw new InvalidOperationException($"Failed to process file: {trxFilePath}", ex);
+        }
+
+        var counters = testRun.ResultSummary.Counters;
+        var failed = counters.Failed;
+        if (failed == 0)
+        {
+            return;
+        }
+
+        var total = counters.Total;
+        var passed = counters.Passed;
+        var skipped = counters.NotExecuted;
+
+        reportBuilder.AppendLine(CultureInfo.InvariantCulture, $"### {GetTestTitle(trxFilePath)}");
+        reportBuilder.AppendLine("| Passed | Failed | Skipped | Total |");
+        reportBuilder.AppendLine("|--------|--------|---------|-------|");
+        reportBuilder.AppendLine(CultureInfo.InvariantCulture, $"| {passed} | {failed} | {skipped} | {total} |");
+
+        reportBuilder.AppendLine();
+        if (testRun.Results?.UnitTestResults is null)
+        {
+            return;
+        }
+
+        var failedTests = testRun.Results.UnitTestResults.Where(r => r.Outcome == "Failed");
+        if (failedTests.Any())
+        {
+            foreach (var test in failedTests)
+            {
+                string title;
+                if (string.IsNullOrEmpty(url))
+                {
+                    title = $"🔴 <b>{test.TestName}</b>";
+                }
+                else
+                {
+
+                    title = $"🔴 <a href=\"{url}\">{test.TestName}</a>";
+                }
+                reportBuilder.AppendLine("<div>");
+                reportBuilder.AppendLine(CultureInfo.InvariantCulture, $"""
+                    <details><summary>{title}</summary>
+                """);
+
+                var errorMsgBuilder = new StringBuilder();
+                errorMsgBuilder.AppendLine(test.Output?.ErrorInfo?.InnerText ?? string.Empty);
+                errorMsgBuilder.AppendLine(test.Output?.StdOut ?? string.Empty);
+
+                var errorMsgTruncated = TruncateTheStart(errorMsgBuilder.ToString(), 50_000); // Truncate long error messages for readability
path: tools/GenerateTestSummary/TestSummaryGenerator.cs
user: login: Copilot
body: [nitpick] Consider refactoring the magic number 50_000 into a named constant to clarify its purpose and improve maintainability.
created_at: 2025-05-12T05:21:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,76 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System.Text;
+using System.CommandLine;
+using Aspire.TestTools;
+
+// Usage: dotnet tools run GenerateTestSummary --dirPathOrTrxFilePath <path> [--output <output>] [--combined]
+// Generate a summary report from trx files.
+// And write to $GITHUB_STEP_SUMMARY if running in GitHub Actions.
+
+var dirPathOrTrxFilePathArgument = new Argument<string>("dirPathOrTrxFilePath");
path: tools/GenerateTestSummary/Program.cs
user: login: RussKie
body: I'm struggling to understand all use-cases for this argument; more comments would be really helpful.

I understand when this points to a folder where *.trx files are, but what does it mean when it is a file? Why would I pass a file?

What does the following combination means?
```
 -- D:\a\aspire\aspire/testresults -u https://github.com/dotnet/aspire/actions/runs/14966104757/artifacts/3103892024
```
created_at: 2025-05-12T09:22:27Z
author_association: CONTRIBUTOR

PR comments:
user: login: davidfowl
created_at: 2025-05-10T04:18:00Z
body: @radical can you translate this to a C# project under a tools folder? We don't need this to be in powershell 😄 (and it'll let us all vibe code a bit better on it) 
author_association: CONTRIBUTOR
user: login: radical
created_at: 2025-05-10T04:25:34Z
body: > @radical can you translate this to a C# project under a tools folder? We don't need this to be in powershell 😄 (and it'll let us all vibe code a bit better on it)

This is essentially a stopgap till @russkie and @joperezr have the other test reporter back again. But we could do this as our own thing in C# too. Copilot seems to be good with powershell too btw. That's what I am using right now lol.
author_association: MEMBER
user: login: danmoseley
created_at: 2025-05-10T17:19:20Z
body: >  Copilot seems to be good with powershell too btw. 

pick your AI, you'll find one that can easily translate that to C#
author_association: MEMBER
user: login: davidfowl
created_at: 2025-05-10T17:20:33Z
body: You can stop gap in c#
author_association: CONTRIBUTOR
user: login: danmoseley
created_at: 2025-05-10T22:13:49Z
body: Actually thinking about it -- if psh is on all the test machines, isn't it the best tool for the job? No compilation and easy to read/modify (even debug).. but idc
author_association: MEMBER
user: login: danmoseley
created_at: 2025-05-10T22:19:07Z
body: ` /home/runner/work/aspire/aspire/dotnet.sh ` ... looks like a workaround may be to use $GITHUB_WORKSPACE instead?
https://github.com/actions/checkout/issues/785#issuecomment-2616058824
author_association: MEMBER
user: login: davidfowl
created_at: 2025-05-11T18:04:02Z
body: Can you inline the logs for the failed test under a nested collapsible section? Like <details> in markdown 
author_association: CONTRIBUTOR
user: login: RussKie
created_at: 2025-05-12T01:56:38Z
body: Have you explored the MTP extensibility model or chatted with the Test team? The MTP supports custom plugins that can be invoked at specific points in the test execution workflow. 

On a heel of https://github.com/dotnet/aspire/pull/8811#discussion_r2062965334, I had a chat with @Youssef1313, here's some details from that convo:

> Registration of extensions happens in the auto-generated entry point. We use an MSBuild item for that (named `TestingPlatformBuilderHook`)
>  
>  
> ```XML
>   <ItemGroup>
>     <TestingPlatformBuilderHook Include="UNIQUE_GUID_FOR_YOUR_EXTENSION" >
>       <DisplayName>Display Name for your extension</DisplayName>
>       <TypeFullName>FQN of TestingPlatformBuilderHook</TypeFullName>
>     </TestingPlatformBuilderHook>
>   </ItemGroup>
> ```
> 
> The TestingPlatformBuilderHook class should have AddExtensions method with two parameters (`ITestApplicationBuilder`, and `string[]`). The latter in most cases ends up unused (it represents the command-line arguments, i.e, same as `string[] args` of Main)
> On the` ITestApplicationBuilder`, you do stuff like `builder.TestHost.AddDataConsumer` as explained in https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-architecture-extensions#the-idataconsumer-extensions.
>  
> See https://github.com/microsoft/testfx/blob/5bcd432d2e9289609395ab2ae1f8489ec9ed9610/samples/public/mstest-runner/CustomReportExtension/CustomReportExtension/Program.cs#L12-L19 for example. The code in this sample is what you can have in `AddExtensions`.

@nohwnd also had a presentation for an AzDO reporting plugin during the latest .NET Showcase. Jakub also told me that a GHA reported was something on the team's backlog.

Last, but not least @jeffhandley has built an awesome reporting for https://github.com/dotnet/issue-labeler (https://github.com/dotnet/issue-labeler/pull/104) using [GitHub.Actions.Core](https://github.com/IEvangelist/dotnet-github-actions-sdk) (which is built by our own @IEvangelist).
You can see the examples in our actions. E.g.:
* https://github.com/dotnet/aspire/actions/runs/14827645654
* https://github.com/dotnet/aspire/actions/runs/14960322655

![image](https://github.com/user-attachments/assets/9e268499-2dea-466c-937b-2e7659da4a48)


With the above, I'm not entirely convinced this change may be going the right direction; we'll definitely need the data analysis and the reporting capabilities, but the overall strategy is worth reconsidering.
author_association: CONTRIBUTOR
user: login: radical
created_at: 2025-05-12T02:05:17Z
body: > With the above, I'm not entirely convinced this change may be going the right direction; we'll definitely need the data analysis and the reporting capabilities, but the overall strategy is worth reconsidering.

Thank you, that's useful information! I'm aware that you have great ideas around this and are going to possibly work on it. But that might take some time, so my effort here is a just a quick solution to act as a stop-gap till your proper solution is merged.
author_association: MEMBER
user: login: davidfowl
created_at: 2025-05-12T04:33:35Z
body: Yea, lets do that in a follow up PR. We just need something better than what we have now at the moment.
author_association: CONTRIBUTOR
user: login: radical
created_at: 2025-05-12T04:35:09Z
body: <img width="1182" alt="Screenshot 2025-05-12 at 00 30 07" src="https://github.com/user-attachments/assets/41b5cad4-e905-4741-922b-1c4a586caae1" />
<img width="889" alt="Screenshot 2025-05-12 at 00 30 25" src="https://github.com/user-attachments/assets/bb136f07-c1df-419a-ae14-9f0b82dec884" />


author_association: MEMBER
user: login: radical
created_at: 2025-05-12T04:36:02Z
body: test run - https://github.com/dotnet/aspire/actions/runs/14963668924?pr=9223

Just cleaning up the PR now.
author_association: MEMBER
user: login: davidfowl
created_at: 2025-05-12T04:37:34Z
body: Super nice. Small nit, delete the Program class and static void Main. Top level statements are made for these micro level programs 😄 
author_association: CONTRIBUTOR
user: login: davidfowl
created_at: 2025-05-12T04:45:41Z
body: @radical can we see inline console logs for the test that failed? Where is that?
author_association: CONTRIBUTOR
user: login: radical
created_at: 2025-05-12T04:47:26Z
body: > @radical can we see inline console logs for the test that failed? Where is that?

That should be captured in the `.trx` file. I am trying to surface that here too though, but the github actions summary has a limit of 64k. You can download the trx and log files from the artifacts named like `logs-Hosting-ubuntu-latest`.
author_association: MEMBER
user: login: davidfowl
created_at: 2025-05-12T04:54:14Z
body: OK some suggestions, add a link to the TRX file and logs per test failure. That will make it trivial to find and download the logs instead of showing them inline.
author_association: CONTRIBUTOR
user: login: RussKie
created_at: 2025-05-12T05:10:59Z
body: > <img alt="Screenshot 2025-05-12 at 00 30 07" width="1182" src="https://private-user-images.githubusercontent.com/1472/442585081-41b5cad4-e905-4741-922b-1c4a586caae1.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDcwMjY5MzEsIm5iZiI6MTc0NzAyNjYzMSwicGF0aCI6Ii8xNDcyLzQ0MjU4NTA4MS00MWI1Y2FkNC1lOTA1LTQ3NDEtOTIyYi0xYzRhNTg2Y2FhZTEucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDUxMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTA1MTJUMDUxMDMxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OTMwZGRlYzZlNzU2NTE3YWRhNWE0YThlOTQxOGU1YTUzYTc4ZmMzOWM1ZDQ3NTQ1ZTZjZmEzMmRiZDJlNWI1MyZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.FGF8CkBvT9UK2QLiiWeB15S7nBTmLgkfifV3OwRMDpg"> <img alt="Screenshot 2025-05-12 at 00 30 25" width="889" src="https://private-user-images.githubusercontent.com/1472/442585095-bb136f07-c1df-419a-ae14-9f0b82dec884.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDcwMjY5MzEsIm5iZiI6MTc0NzAyNjYzMSwicGF0aCI6Ii8xNDcyLzQ0MjU4NTA5NS1iYjEzNmYwNy1jMWRmLTQxOWEtYWUxNC05ZjBiODJkZWM4ODQucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDUxMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTA1MTJUMDUxMDMxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MzJhMGEyNDQ4OWI1NzViZGQ2ZDNkNmE2ZTdkMmQ4MTVmMDg1ZjhkNDllYWU1ZTczZThhMDg3NzlmODAyODRjYSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.fCiiCLKxTNvJGyH1YrPl7Zp_zzV7SdsnmAPJ3t0Hfdg">

These look awesome!
author_association: CONTRIBUTOR
user: login: radical
created_at: 2025-05-12T05:18:58Z
body: > OK some suggestions, add a link to the TRX file and logs per test failure. That will make it trivial to find and download the logs

Added link to the artifacts for the test assembly, which includes the trx, log file etc.

> instead of showing them inline.

I retained this one though, good for a quick look at the error.
author_association: MEMBER
user: login: davidfowl
created_at: 2025-05-12T05:22:34Z
body: @radical this PR is good, extra credit for a follow up PR that posts a GitHub comment with the direct link to the summary 😄 (and updates that same comment as re-runs happen) 
author_association: CONTRIBUTOR
user: login: radical
created_at: 2025-05-12T05:26:50Z
body: > @radical this PR is good, extra credit for a follow up PR that posts a GitHub comment with the direct link to the summary 😄 (and updates that same comment as re-runs happen)

I was trying to add the reports on the `Checks` page actually, which would surface that in the PR nicely too. But a quick attempt at it failed because of permissions to post from the workflow.

PS: I have a different tool that I am working on which would add a github app and we could post from that.
author_association: MEMBER
user: login: davidfowl
created_at: 2025-05-12T05:37:59Z
body: github app seems over engineered what what we need. Actions can use the token to post comments like our backport bot.

Anyhow, approved. 
author_association: CONTRIBUTOR
user: login: radical
created_at: 2025-05-12T05:48:41Z
body: > Actions can use the token to post comments like our backport bot.

I'll check that out 👍 
author_association: MEMBER
user: login: davidfowl
created_at: 2025-05-12T05:56:35Z
body: https://github.com/dotnet/arcade/blob/80c4e4d26cb85c86f7e1be77d2d9eceeef0f3493/.github/workflows/backport-base.yml#L62-L73
author_association: CONTRIBUTOR
user: login: davidfowl
created_at: 2025-05-12T06:02:59Z
body: @radical include the OS in the test name. It looks like there are duplicate runs in the summary
author_association: CONTRIBUTOR