Repository: dotnet/aspire
Pull request #8498: Migrate from VSTest to Microsoft.Testing.Platform
Author: login: Youssef1313
Labels: name: area-engineering-systems
description: infrastructure helix infra engineering repo stuff
Created: 2025-04-02T09:03:50Z | Merged: 2025-04-09T17:12:10Z

## Description

Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #8293

## Checklist

- Is this feature complete?
  - [ ] Yes. Ready to ship.
  - [ ] No. Follow-up changes expected.
- Are you including unit tests for the changes and scenario tests if relevant?
  - [ ] Yes
  - [ ] No
- Did you add public API?
  - [ ] Yes
    - If yes, did you have an API Review for it?
      - [ ] Yes
      - [ ] No
    - Did you add `<remarks />` and `<code />` elements on your triple slash comments?
      - [ ] Yes
      - [ ] No
  - [ ] No
- Does the change make any security assumptions or guarantees?
  - [ ] Yes
    - If yes, have you done a threat model and had a security review?
      - [ ] Yes
      - [ ] No
  - [ ] No
- Does the change require an update in our Aspire docs?
  - [ ] Yes
    - Is this introducing a breaking change?
      - [ ] Yes
        - Link to aspire-docs issue (please use this [`breaking-change` template](https://github.com/dotnet/docs-aspire/issues/new?template=04-breaking-change.yml)):
      - [ ] No
        - Link to aspire-docs issue (please use this [`doc-idea` template](https://github.com/dotnet/docs-aspire/issues/new?template=02-docs-request.yml)):
  - [ ] No


Linked issue #8293: Migrating to xunit.v3 and to Microsoft.Testing.Platform
@danmoseley @joperezr I'm opening this issue per our internal discussion.

Basically I want to discuss moving from xunit v2 to xunit.v3 as a first step, then afterwards, move from VSTest to the newer platform, Microsoft.Testing.Platform.

Moving from v2 to v3 will probably need some changes on Arcade side, so that attributes like ActiveIssue etc are still supported without any extra work to be done on Aspire side.

I'm anticipating that the move to v3 will need to happen in future anyways.

A future move to MTP will also mean that custom test loggers that are written against VSTest will not work. Currently, the only custom logger used is GitHubActionsTestLogger. Trx is for sure supported with MTP via Microsoft.Testing.Extensions.TrxReport.

cc @davidfowl 
Linked issue comments: user: login: danmoseley
created_at: 2025-03-25T17:36:25Z
body: There'll certainly be some .NET team that wants to move to v3 so if you want you can do the Arcade changes now without waiting for a decision here.

I'm guessing we don't have any particular reason to move to v3 any time soon in Aspire, but I'm also guessing your offer to do the migration has an expiry time which might be a reason to do it.
author_association: MEMBER
user: login: joperezr
created_at: 2025-03-25T17:41:47Z
body: Agree with everything shared by Dan, the only thing I'm interested in also learning would be in the move to MTP. Can you share a bit more details on what would this entail and perhaps also some of the pros and cons of such move?
author_association: MEMBER

Diff:
diff --git a/.github/workflows/run-tests.yml b/.github/workflows/run-tests.yml
index c88a972da65..3d2300a9545 100644
--- a/.github/workflows/run-tests.yml
+++ b/.github/workflows/run-tests.yml
@@ -13,10 +13,10 @@ on:
         # relative the repo root
         required: false
         type: string
-      testSessionTimeoutMs:
+      testSessionTimeout:
         required: false
         type: string
-        default: "900000"
+        default: "15m"
       testHangTimeout:
         required: false
         type: string
@@ -220,18 +220,14 @@ jobs:
           TEST_LOG_PATH: ${{ github.workspace }}/artifacts/log/test-logs
           TestsRunningOutsideOfRepo: true
         run: >
-          dotnet test -s .runsettings -v:n ${{ env.TEST_ASSEMBLY_NAME }}.dll
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          --blame
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
+          dotnet ${{ env.TEST_ASSEMBLY_NAME }}.dll
+          --report-trx --report-trx-filename "${{ inputs.testShortName }}.trx"
+          --hangdump --hangdump-timeout ${{ inputs.testHangTimeout }}
+          --crashdump
           --results-directory ${{ github.workspace }}/testresults
+          --filter-not-trait "category=failing"
+          --timeout ${{ inputs.testSessionTimeout }}
           ${{ inputs.extraTestArgs }}
-          --
-          RunConfiguration.CollectSourceInformation=true
-          RunConfiguration.TestSessionTimeout=${{ inputs.testSessionTimeoutMs }}
 
       - name: Run tests
         if: ${{ ! inputs.requiresNugets }}
@@ -240,24 +236,24 @@ jobs:
           CI: false
           DCP_DIAGNOSTICS_LOG_LEVEL: debug
           DCP_DIAGNOSTICS_LOG_FOLDER: ${{ github.workspace }}/testresults/dcp
+          # During restore and build, we use -ci, which causes NUGET_PACKAGES to point to a local cache (Arcade behavior).
+          # In this step, we are not using Arcade, but want to make sure that MSBuild is able to evaluate correctly.
+          # So, we manually set NUGET_PACKAGES
+          NUGET_PACKAGES: ${{ github.workspace }}/.packages
         run: >
           ${{ env.DOTNET_SCRIPT }} test ${{ env.TEST_PROJECT_PATH }}
           /p:ContinuousIntegrationBuild=true
-          -s eng/testing/.runsettings
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          "--blame"
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
-          --results-directory testresults
+          /p:TrxFileNamePrefix="${{ inputs.testShortName }}"
+          -bl:${{ github.workspace }}/testresults/test.binlog
           --no-restore
           --no-build
-          -bl:${{ github.workspace }}/testresults/test.binlog
-          ${{ inputs.extraTestArgs }}
           --
-          RunConfiguration.CollectSourceInformation=true
-          RunConfiguration.TestSessionTimeout=${{ inputs.testSessionTimeoutMs }}
+          --report-trx
+          --hangdump --hangdump-timeout ${{ inputs.testHangTimeout }}
+          --crashdump
+          --results-directory ${{ github.workspace }}/testresults
+          --timeout ${{ inputs.testSessionTimeout }}
+          ${{ inputs.extraTestArgs }}
 
       # Save the result of the previous steps - success or failure
       # in the form of a file result-success/result-failure -{name}.rst
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index c85e92e8c6b..4f40b71ec06 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -77,7 +77,7 @@ jobs:
     with:
       testShortName: ${{ matrix.shortname }}
       os: "ubuntu-latest"
-      extraTestArgs: "--filter \"(quarantined!=true)\""
+      extraTestArgs: "--filter-not-trait \"quarantined=true\""
 
   integrations_test_win:
     uses: ./.github/workflows/run-tests.yml
@@ -90,7 +90,7 @@ jobs:
     with:
       testShortName: ${{ matrix.shortname }}
       os: "windows-latest"
-      extraTestArgs: "--filter \"(quarantined!=true)\""
+      extraTestArgs: "--filter-not-trait \"quarantined=true\""
 
   templates_test_lin:
     name: Templates Linux
@@ -103,10 +103,9 @@ jobs:
       testShortName: ${{ matrix.shortname }}
       os: "ubuntu-latest"
       testProjectPath: tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj
-      testSessionTimeoutMs: 1200000
+      testSessionTimeout: 20m
       testHangTimeout: 12m
-      # append '.' to the name so only the test class with exactly that name is run
-      extraTestArgs: "--filter \"(quarantined!=true)&(FullyQualifiedName~Aspire.Templates.Tests.${{ matrix.shortname }}.)\""
+      extraTestArgs: "--filter-not-trait quarantined=true --filter-class Aspire.Templates.Tests.${{ matrix.shortname }}"
       requiresNugets: true
       requiresTestSdk: true
 
@@ -121,10 +120,9 @@ jobs:
       testShortName: ${{ matrix.shortname }}
       os: "windows-latest"
       testProjectPath: tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj
-      testSessionTimeoutMs: 1200000
+      testSessionTimeout: 20m
       testHangTimeout: 12m
-      # append '.' to the name so only the test class with exactly that name is run
-      extraTestArgs: "--filter \"(quarantined!=true)&(FullyQualifiedName~Aspire.Templates.Tests.${{ matrix.shortname }}.)\""
+      extraTestArgs: "--filter-not-trait quarantined=true --filter-class Aspire.Templates.Tests.${{ matrix.shortname }}"
       requiresNugets: true
       requiresTestSdk: true
 
diff --git a/eng/.runsettings b/eng/.runsettings
deleted file mode 100644
index 955954d2c90..00000000000
--- a/eng/.runsettings
+++ /dev/null
@@ -1,33 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <RunConfiguration>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx" />
-      <Logger friendlyName="html" />
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>Minimal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-    <Logger friendlyName="blame" enabled="True" />
-  </LoggerRunSettings>
-  <DataCollectionRunSettings>
-    <DataCollectors>
-      <!-- Enables blame -->
-      <DataCollector friendlyName="blame" enabled="True">
-        <Configuration>
-          <!-- Enables crash dump, with dump type "Full" or "Mini". -->
-          <CollectDump DumpType="Full" />
-          <!-- Enables hang dump or testhost and its child processes
-          when a test hangs for more than 10 minutes. -->
-          <CollectDumpOnTestSessionHang TestTimeout="10min" HangDumpType="Full" />
-        </Configuration>
-      </DataCollector>
-    </DataCollectors>
-  </DataCollectionRunSettings>
-</RunSettings>
diff --git a/eng/Testing.props b/eng/Testing.props
index f43be60cb0a..50bb41f48db 100644
--- a/eng/Testing.props
+++ b/eng/Testing.props
@@ -9,23 +9,18 @@
     <RunOnAzdoCILinux>true</RunOnAzdoCILinux>
     <RunOnAzdoHelixWindows>true</RunOnAzdoHelixWindows>
     <RunOnAzdoHelixLinux>true</RunOnAzdoHelixLinux>
-  </PropertyGroup>
-
-  <PropertyGroup Condition="'$(UseVSTestRunner)' != 'true'">
-    <_QuarantinedTestRunAdditionalArgs>-trait "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
-    <_NonQuarantinedTestRunAdditionalArgs>-notrait "quarantined=true"</_NonQuarantinedTestRunAdditionalArgs>
-  </PropertyGroup>
 
-  <PropertyGroup Condition="'$(UseVSTestRunner)' == 'true'">
-    <_QuarantinedTestRunAdditionalArgs>--filter "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
-    <_NonQuarantinedTestRunAdditionalArgs>--filter "quarantined!=true"</_NonQuarantinedTestRunAdditionalArgs>
+    <_QuarantinedTestRunAdditionalArgs>--filter-trait "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
+    <_NonQuarantinedTestRunAdditionalArgs>--filter-not-trait "quarantined=true"</_NonQuarantinedTestRunAdditionalArgs>
   </PropertyGroup>
 
   <PropertyGroup>
     <BlameHangTimeout Condition="'$(BlameHangTimeout)' == ''">10m</BlameHangTimeout>
-    <_BlameArgs>--blame --blame-hang-timeout $(BlameHangTimeout) --blame-crash</_BlameArgs>
+    <_BlameArgs>--hangdump --hangdump-timeout $(BlameHangTimeout) --crashdump</_BlameArgs>
+
+    <TestRunnerAdditionalArguments>$(TestRunnerAdditionalArguments) --filter-not-trait &quot;category=failing&quot;</TestRunnerAdditionalArguments>
 
-    <TestRunnerAdditionalArguments Condition="'$(RunQuarantinedTests)' == ''">$(TestRunnerAdditionalArguments) $(_NonQuarantinedTestRunAdditionalArgs) $(TestRunnerAdditionalArguments) $(_BlameArgs)</TestRunnerAdditionalArguments>
-    <TestRunnerAdditionalArguments Condition="'$(RunQuarantinedTests)' == 'true'">$(TestRunnerAdditionalArguments) $(_QuarantinedTestRunAdditionalArgs) $(TestRunnerAdditionalArguments) $(_BlameArgs)</TestRunnerAdditionalArguments>
+    <TestRunnerAdditionalArguments Condition="'$(RunQuarantinedTests)' != 'true'">$(TestRunnerAdditionalArguments) $(_NonQuarantinedTestRunAdditionalArgs) $(_BlameArgs)</TestRunnerAdditionalArguments>
+    <TestRunnerAdditionalArguments Condition="'$(RunQuarantinedTests)' == 'true'">$(TestRunnerAdditionalArguments) $(_QuarantinedTestRunAdditionalArgs) $(_BlameArgs)</TestRunnerAdditionalArguments>
   </PropertyGroup>
 </Project>
diff --git a/eng/Versions.props b/eng/Versions.props
index 0c9c4c8b8e5..3c4f1e25ca2 100644
--- a/eng/Versions.props
+++ b/eng/Versions.props
@@ -12,10 +12,10 @@
     <DotNetRuntimePreviousVersionForTesting>8.0.13</DotNetRuntimePreviousVersionForTesting>
     <!-- dotnet 8.0 versions for running tests - used for templates tests -->
     <DotNetSdkPreviousVersionForTesting>8.0.406</DotNetSdkPreviousVersionForTesting>
-    <UseVSTestRunner>true</UseVSTestRunner>
     <XunitV3Version>2.0.0</XunitV3Version>
     <XUnitAnalyzersVersion>1.20.0</XUnitAnalyzersVersion>
     <XunitRunnerVisualStudioVersion>3.0.2</XunitRunnerVisualStudioVersion>
+    <MicrosoftTestingPlatformVersion>1.6.3</MicrosoftTestingPlatformVersion>
     <!-- Enable to remove prerelease label. -->
     <StabilizePackageVersion Condition="'$(StabilizePackageVersion)' == ''">false</StabilizePackageVersion>
     <DotNetFinalVersionKind Condition="'$(StabilizePackageVersion)' == 'true'">release</DotNetFinalVersionKind>
diff --git a/eng/Xunit3/Microsoft.Testing.Platform.targets b/eng/Xunit3/Microsoft.Testing.Platform.targets
new file mode 100644
index 00000000000..e58467d236d
--- /dev/null
+++ b/eng/Xunit3/Microsoft.Testing.Platform.targets
@@ -0,0 +1,92 @@
+<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. -->
+<Project>
+
+  <Target Name="RunTests"
+          Outputs="%(TestToRun.ResultsStdOutPath)"
+          Condition="'@(TestToRun)' != ''">
+    <Telemetry EventName="NETCORE_ENGINEERING_TELEMETRY" EventData="Category=Test" />
+
+    <PropertyGroup>
+      <_TestResultDirectory>$([System.IO.Path]::GetDirectoryName('%(TestToRun.ResultsTrxPath)'))</_TestResultDirectory>
+      <_TestResultTrxFileName>$([System.IO.Path]::GetFileName('%(TestToRun.ResultsTrxPath)'))</_TestResultTrxFileName>
+      <_TestResultXmlFileName>$([System.IO.Path]::GetFileName('%(TestToRun.ResultsXmlPath)'))</_TestResultXmlFileName>
+      <_TestResultHtmlFileName>$([System.IO.Path]::GetFileName('%(TestToRun.ResultsHtmlPath)'))</_TestResultHtmlFileName>
+    </PropertyGroup>
+
+    <PropertyGroup>
+      <_TestEnvironment>%(TestToRun.EnvironmentDisplay)</_TestEnvironment>
+      <_TestAssembly>%(TestToRun.Identity)</_TestAssembly>
+      <_TestAssembly Condition="'$(OS)'=='Windows_NT'">$([System.IO.Path]::ChangeExtension($(_TestAssembly), '.exe'))</_TestAssembly>
+      <_TestAssembly Condition="'$(OS)'!='Windows_NT'">$(_TestAssembly.TrimEnd('.dll'))</_TestAssembly>
+      <_TestRuntime>%(TestToRun.TestRuntime)</_TestRuntime>
+      <_TestTimeout>%(TestToRun.TestTimeout)</_TestTimeout>
+      <_TestRunnerAdditionalArguments>%(TestToRun.TestRunnerAdditionalArguments)</_TestRunnerAdditionalArguments>
+
+      <_TestRunner>$(_TestAssembly)</_TestRunner>
+      <_TestRunnerArgs>$(_TestRunnerAdditionalArguments) --results-directory "$(_TestResultDirectory)" --report-xunit --report-xunit-filename "$(_TestResultXmlFileName)" --report-xunit-html --report-xunit-html-filename "$(_TestResultHtmlFileName)" --report-trx --report-trx-filename "$(_TestResultTrxFileName)"</_TestRunnerArgs>
+    </PropertyGroup>
+
+    <PropertyGroup Condition="'$(_TestRuntime)' == 'Core'">
+      <_TestRunnerArgs>$(_TestRunnerArgs) --auto-reporters off</_TestRunnerArgs>
+    </PropertyGroup>
+
+    <PropertyGroup>
+      <_TestRunnerCommand>"$(_TestRunner)" $(_TestRunnerArgs)</_TestRunnerCommand>
+
+      <!--
+        Redirect std output of the runner.
+        Note that xUnit outputs failure info to both STDOUT (stack trace, message) and STDERR (failed test name)
+      -->
+      <_TestRunnerCommand Condition="'$(TestCaptureOutput)' != 'false'">$(_TestRunnerCommand) > "%(TestToRun.ResultsStdOutPath)" 2>&amp;1</_TestRunnerCommand>
+    </PropertyGroup>
+
+    <ItemGroup>
+      <_OutputFiles Include="%(TestToRun.ResultsXmlPath)" />
+      <_OutputFiles Include="%(TestToRun.ResultsHtmlPath)" />
+      <_OutputFiles Include="%(TestToRun.ResultsStdOutPath)" />
+    </ItemGroup>
+
+    <MakeDir Directories="@(_OutputFiles->'%(RootDir)%(Directory)')"/>
+    <Delete Files="@(_OutputFiles)" />
+
+    <Message Text="Running tests: $(_TestAssembly) [$(_TestEnvironment)]" Importance="high"/>
+    <Exec Command='$(_TestRunnerCommand)'
+          LogStandardErrorAsError="false"
+          WorkingDirectory="$(_TargetDir)"
+          IgnoreExitCode="true"
+          Timeout="$(_TestTimeout)"
+          EnvironmentVariables="DOTNET_ROOT=$(DotNetRoot);DOTNET_ROOT_X86=$(DotNetRoot)x86"
+          ContinueOnError="WarnAndContinue">
+      <Output TaskParameter="ExitCode" PropertyName="_TestErrorCode" />
+    </Exec>
+
+    <!--
+      Add command line to the log.
+    -->
+    <WriteLinesToFile File="%(TestToRun.ResultsStdOutPath)"
+                      Overwrite="false"
+                      Lines=";=== COMMAND LINE ===;$(_TestRunnerCommand)"
+                      Condition="'$(TestCaptureOutput)' != 'false'" />
+
+    <!--
+      Report test status.
+    -->
+    <Message Text="Tests succeeded: $(_TestAssembly) [$(_TestEnvironment)]" Condition="'$(_TestErrorCode)' == '0'" Importance="high" />
+
+    <PropertyGroup>
+      <_ResultsFileToDisplay>%(TestToRun.ResultsHtmlPath)</_ResultsFileToDisplay>
+      <_ResultsFileToDisplay Condition="!Exists('$(_ResultsFileToDisplay)')">%(TestToRun.ResultsStdOutPath)</_ResultsFileToDisplay>
+    </PropertyGroup>
+
+    <!--
+      Ideally we would set ContinueOnError="ErrorAndContinue" so that when a test fails in multi-targeted test project
+      we'll still run tests for all target frameworks. ErrorAndContinue doesn't work well on Linux though: https://github.com/Microsoft/msbuild/issues/3961.
+    -->
+    <Error Text="Tests failed: $(_ResultsFileToDisplay) [$(_TestEnvironment)]" Condition="'$(_TestErrorCode)' != '0' and '$(_ErrorOnTestFailure)' != 'false'" File="XUnit" />
+
+    <ItemGroup>
+      <FileWrites Include="@(_OutputFiles)"/>
+    </ItemGroup>
+  </Target>
+
+</Project>
diff --git a/eng/Xunit3/Xunit3.targets b/eng/Xunit3/Xunit3.targets
index 2cda46b44b6..71b6dccb5c6 100644
--- a/eng/Xunit3/Xunit3.targets
+++ b/eng/Xunit3/Xunit3.targets
@@ -4,15 +4,23 @@
     <PackageVersion Include="xunit.v3.core" Version="$(XunitV3Version)" />
     <PackageVersion Include="xunit.analyzers" Version="$(XunitAnalyzersVersion)" />
     <PackageVersion Include="xunit.v3.assert" Version="$(XunitV3Version)" />
-    <PackageVersion Include="xunit.runner.visualstudio" Version="$(XunitRunnerVisualStudioVersion)" />
     <PackageVersion Include="xunit.v3.runner.console" Version="$(XunitV3Version)" />
+    <PackageVersion Include="Microsoft.Testing.Platform" Version="$(MicrosoftTestingPlatformVersion)" />
+    <PackageVersion Include="Microsoft.Testing.Platform.MSBuild" Version="$(MicrosoftTestingPlatformVersion)" />
+    <PackageVersion Include="Microsoft.Testing.Extensions.HangDump" Version="$(MicrosoftTestingPlatformVersion)" />
+    <PackageVersion Include="Microsoft.Testing.Extensions.CrashDump" Version="$(MicrosoftTestingPlatformVersion)" />
+    <PackageVersion Include="Microsoft.Testing.Extensions.TrxReport" Version="$(MicrosoftTestingPlatformVersion)" />
 
     <PackageReference Include="xunit.v3" />
     <PackageReference Include="xunit.v3.core" />
     <PackageReference Include="xunit.analyzers" />
     <PackageReference Include="xunit.v3.assert" />
-    <PackageReference Include="xunit.runner.visualstudio" />
+    <PackageReference Include="Microsoft.Testing.Platform" />
+    <PackageReference Include="Microsoft.Testing.Platform.MSBuild" />
+    <PackageReference Include="Microsoft.Testing.Extensions.HangDump" />
+    <PackageReference Include="Microsoft.Testing.Extensions.CrashDump" />
+    <PackageReference Include="Microsoft.Testing.Extensions.TrxReport" />
   </ItemGroup>
 
-  <Import Project="..\tools\VSTest.targets" Sdk="Microsoft.DotNet.Arcade.Sdk" />
-</Project>
\ No newline at end of file
+  <Import Project="Microsoft.Testing.Platform.targets" />
+</Project>
diff --git a/eng/testing/.runsettings b/eng/testing/.runsettings
deleted file mode 100644
index 88a92c090ab..00000000000
--- a/eng/testing/.runsettings
+++ /dev/null
@@ -1,35 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <RunConfiguration>
-    <!-- Timeout in ms, 20 minutes -->
-    <TestSessionTimeout>1200000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="blame" enabled="True" />
-    </Loggers>
-  </LoggerRunSettings>
-  <DataCollectionRunSettings>
-    <DataCollectors>
-      <!-- Enables blame -->
-      <DataCollector friendlyName="blame" enabled="True">
-        <Configuration>
-          <CollectDump DumpType="Full" />
-          <CollectDumpOnTestSessionHang TestTimeout="7min" HangDumpType="Full" />
-        </Configuration>
-      </DataCollector>
-    </DataCollectors>
-  </DataCollectionRunSettings>
-</RunSettings>
diff --git a/tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj b/tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
index efa2d072e97..3e291ae6127 100644
--- a/tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
+++ b/tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
@@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
   </PropertyGroup>
 
   <ItemGroup>
diff --git a/tests/Aspire.EndToEnd.Tests/.runsettings b/tests/Aspire.EndToEnd.Tests/.runsettings
deleted file mode 100644
index 348b55730ea..00000000000
--- a/tests/Aspire.EndToEnd.Tests/.runsettings
+++ /dev/null
@@ -1,26 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
diff --git a/tests/Aspire.EndToEnd.Tests/Aspire.EndToEnd.Tests.csproj b/tests/Aspire.EndToEnd.Tests/Aspire.EndToEnd.Tests.csproj
index 5694b015626..a074d720d09 100644
--- a/tests/Aspire.EndToEnd.Tests/Aspire.EndToEnd.Tests.csproj
+++ b/tests/Aspire.EndToEnd.Tests/Aspire.EndToEnd.Tests.csproj
@@ -20,7 +20,6 @@
     <DefineConstants Condition="'$(_BuildForTestsRunningOutsideOfRepo)' == 'true'">BUILD_FOR_TESTS_RUNNING_OUTSIDE_OF_REPO;$(DefineConstants)</DefineConstants>
 
     <XunitRunnerJson>xunit.runner.json</XunitRunnerJson>
-    <RunSettingsFilePath>$(MSBuildThisFileDirectory).runsettings</RunSettingsFilePath>
     <TestArchiveTestsDir>$(TestArchiveTestsDirForEndToEndTests)</TestArchiveTestsDir>
   </PropertyGroup>
 
diff --git a/tests/Aspire.Hosting.Containers.Tests/.runsettings b/tests/Aspire.Hosting.Containers.Tests/.runsettings
deleted file mode 100644
index 591a45e7ab7..00000000000
--- a/tests/Aspire.Hosting.Containers.Tests/.runsettings
+++ /dev/null
@@ -1,23 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
diff --git a/tests/Aspire.Hosting.Keycloak.Tests/.runsettings b/tests/Aspire.Hosting.Keycloak.Tests/.runsettings
deleted file mode 100644
index 591a45e7ab7..00000000000
--- a/tests/Aspire.Hosting.Keycloak.Tests/.runsettings
+++ /dev/null
@@ -1,23 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
diff --git a/tests/Aspire.Playground.Tests/.runsettings b/tests/Aspire.Playground.Tests/.runsettings
deleted file mode 100644
index 1eb28e08584..00000000000
--- a/tests/Aspire.Playground.Tests/.runsettings
+++ /dev/null
@@ -1,26 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 20 minutes -->
-    <TestSessionTimeout>1200000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
diff --git a/tests/Aspire.Playground.Tests/Aspire.Playground.Tests.csproj b/tests/Aspire.Playground.Tests/Aspire.Playground.Tests.csproj
index 2b048c12806..d07a3033a95 100644
--- a/tests/Aspire.Playground.Tests/Aspire.Playground.Tests.csproj
+++ b/tests/Aspire.Playground.Tests/Aspire.Playground.Tests.csproj
@@ -17,7 +17,6 @@
 
     <PlaygroundSourceDir>$(MSBuildThisFileDirectory)..\..\playground\</PlaygroundSourceDir>
     <TestsSharedDir>$(MSBuildThisFileDirectory)..\Shared\</TestsSharedDir>
-    <RunSettingsFilePath>$(MSBuildThisFileDirectory).runsettings</RunSettingsFilePath>
     <!-- Skip emulators that don't start consistently when running in CI. -->
     <SkipUnstableEmulators Condition="'$(SkipUnstableEmulators)' == '' and ('$(RepoRoot)' == '' or '$(ContinuousIntegrationBuild)' == 'true' or '$(CODESPACES)' == 'true')">true</SkipUnstableEmulators>
 
diff --git a/tests/Aspire.Templates.Tests/.runsettings b/tests/Aspire.Templates.Tests/.runsettings
deleted file mode 100644
index 1eb28e08584..00000000000
--- a/tests/Aspire.Templates.Tests/.runsettings
+++ /dev/null
@@ -1,26 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 20 minutes -->
-    <TestSessionTimeout>1200000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
diff --git a/tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj b/tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj
index 7e9cb00475b..78faf4d9376 100644
--- a/tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj
+++ b/tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj
@@ -7,7 +7,6 @@
     <InstallWorkloadForTesting>true</InstallWorkloadForTesting>
 
     <XunitRunnerJson>xunit.runner.json</XunitRunnerJson>
-    <RunSettingsFilePath>$(MSBuildThisFileDirectory).runsettings</RunSettingsFilePath>
     <TestArchiveTestsDir>$(TestArchiveTestsDirForTemplateTests)</TestArchiveTestsDir>
 
     <InstallBrowsersForPlaywright Condition="'$(InstallBrowsersForPlaywright)' == '' and '$(CODESPACES)' == 'true'">true</InstallBrowsersForPlaywright>
diff --git a/tests/Directory.Build.props b/tests/Directory.Build.props
index 8c0aabbcfa9..beb44b89dcd 100644
--- a/tests/Directory.Build.props
+++ b/tests/Directory.Build.props
@@ -5,8 +5,6 @@
   <Import Project="$(TestsSharedRepoTestingDir)Aspire.RepoTesting.props" />
 
   <PropertyGroup>
-    <!-- Set RunSettingsFilePath property which is read by VSTest. -->
-    <RunSettingsFilePath>$(RepositoryEngineeringDir)testing\.runsettings</RunSettingsFilePath>
     <TestArchiveTestsDir Condition="'$(TestArchiveTestsDir)' == ''">$([MSBuild]::NormalizeDirectory($(ArtifactsDir), 'helix', 'tests'))</TestArchiveTestsDir>
     <TestArchiveTestsDirForTemplateTests Condition="'$(TestArchiveTestsDirForTemplateTests)' == ''">$([MSBuild]::NormalizeDirectory($(ArtifactsDir), 'helix', 'templates-tests'))</TestArchiveTestsDirForTemplateTests>
     <TestArchiveTestsDirForEndToEndTests Condition="'$(TestArchiveTestsDirForEndToEndTests)' == ''">$([MSBuild]::NormalizeDirectory($(ArtifactsDir), 'helix', 'e2e-tests'))</TestArchiveTestsDirForEndToEndTests>
@@ -16,14 +14,8 @@
 
     <!-- This is useful for local runs -->
     <VSTestResultsDirectory>$(ArtifactsTestResultsDir)</VSTestResultsDirectory>
-
-    <OutputType Condition="$(MSBuildProjectName.EndsWith('.Tests'))">Exe</OutputType>
   </PropertyGroup>
 
-  <ItemGroup>
-    <PackageReference Include="GitHubActionsTestLogger" PrivateAssets="all" IncludeAssets="runtime; build; native; contentfiles; analyzers" />
-  </ItemGroup>
-
   <PropertyGroup>
     <UsePublicApiAnalyzers>false</UsePublicApiAnalyzers>
   </PropertyGroup>
diff --git a/tests/Directory.Build.targets b/tests/Directory.Build.targets
index a18ad0736dc..bf1b4461737 100644
--- a/tests/Directory.Build.targets
+++ b/tests/Directory.Build.targets
@@ -4,17 +4,12 @@
   <Import Project="$(TestsSharedRepoTestingDir)Aspire.RepoTesting.targets" />
 
   <PropertyGroup>
-    <!-- forward the settings file path to RunTests target in arcade -->
-    <VSTestRunSettingsFile Condition="'$(VSTestRunSettingsFile)' == ''">$(RunSettingsFilePath)</VSTestRunSettingsFile>
-
-    <DeployRunSettingsFile Condition="'$(DeployRunSettingsFile)' == ''">true</DeployRunSettingsFile>
     <!-- Use a separate xunit.runner.json for helix that disables parallel test runs -->
     <XunitRunnerJson Condition="'$(XunitRunnerJson)' == '' and '$(PrepareForHelix)' == 'true'">$(RepoRoot)tests\helix\xunit.runner.json</XunitRunnerJson>
     <XunitRunnerJson Condition="'$(XunitRunnerJson)' == ''">$(RepositoryEngineeringDir)testing\xunit.runner.json</XunitRunnerJson>
   </PropertyGroup>
 
   <ItemGroup>
-    <None Include="$(RunSettingsFilePath)" CopyToOutputDirectory="PreserveNewest" Condition="'$(DeployRunSettingsFile)' == 'true'" />
     <None Include="$(XunitRunnerJson)" CopyToOutputDirectory="PreserveNewest" />
   </ItemGroup>
 
@@ -41,7 +36,7 @@
     <Error Condition="'$(ExtractTestClassNamesPrefix)' == ''"
            Text="%24(ExtractTestClassNamesPrefix) should be set, for example - Aspire.Templates.Tests" />
 
-    <Exec Command="&quot;$(DotNetTool)&quot; test --no-build -c $(Configuration) -s $(RunSettingsFilePath) --list-tests --nologo -v:q -p:VsTestUseMSBuildOutput=false" ConsoleToMSBuild="true" EnvironmentVariables="DOTNET_ROOT=$(DotNetRoot);DOTNET_ROOT_X86=$(DotNetRoot)x86">
+    <Exec Command="&quot;$(RunCommand)&quot; --filter-not-trait category=failing --list-tests" ConsoleToMSBuild="true" EnvironmentVariables="DOTNET_ROOT=$(DotNetRoot);DOTNET_ROOT_X86=$(DotNetRoot)x86">
       <Output TaskParameter="ConsoleOutput" ItemName="_ListOfTestsLines" />
     </Exec>
 
@@ -66,7 +61,8 @@
 
   <Target Name="GetRunTestsOnGithubActions" Returns="@(TestProject)">
     <ItemGroup>
-      <TestProject Include="$(MSBuildProjectFullPath)" RunTestsOnGithubActions="$(RunOnGithubActions)" />
+      <TestProject Condition="'$(OS)' == 'Windows_NT'" Include="$(MSBuildProjectFullPath)" RunTestsOnGithubActions="$(RunOnGithubActionsWindows)" />
+      <TestProject Condition="'$(OS)' != 'Windows_NT'" Include="$(MSBuildProjectFullPath)" RunTestsOnGithubActions="$(RunOnGithubActionsLinux)" />
     </ItemGroup>
   </Target>
 
diff --git a/tests/Shared/RepoTesting/Aspire.RepoTesting.props b/tests/Shared/RepoTesting/Aspire.RepoTesting.props
index f4c225f2e54..4bf72ed654f 100644
--- a/tests/Shared/RepoTesting/Aspire.RepoTesting.props
+++ b/tests/Shared/RepoTesting/Aspire.RepoTesting.props
@@ -14,6 +14,12 @@
     <!-- https://github.com/dotnet/arcade/issues/15654 -->
     <!-- We currently have our own Xunit3.targets in eng/Xunit3, which Arcade will import -->
     <TestRunnerName>Xunit3</TestRunnerName>
+
+    <TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
+    <UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner>
+    <TestingPlatformCaptureOutput>false</TestingPlatformCaptureOutput>
+    <OutputType Condition="$(MSBuildProjectName.EndsWith('.Tests'))">Exe</OutputType>
+
   </PropertyGroup>
 
   <!-- IsAspireHost property might not be available here, so allow infering that from the project name -->
diff --git a/tests/Shared/RepoTesting/Aspire.RepoTesting.targets b/tests/Shared/RepoTesting/Aspire.RepoTesting.targets
index ac54f9b7fd3..4f3c9834786 100644
--- a/tests/Shared/RepoTesting/Aspire.RepoTesting.targets
+++ b/tests/Shared/RepoTesting/Aspire.RepoTesting.targets
@@ -41,6 +41,9 @@
 
     <IncludeTestPackages Condition="'$(IncludeTestPackages)' == '' and ('$(IsTestProject)' == 'true' or '$(IsTestUtilityProject)' == 'true')">true</IncludeTestPackages>
     <GeneratedPackagesVersionsPropsPath Condition="'$(GeneratedPackagesVersionsPropsPath)' == ''">$(IntermediateOutputPath)Directory.Packages.Versions.props</GeneratedPackagesVersionsPropsPath>
+
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --filter-not-trait "category=failing"</TestingPlatformCommandLineArguments>
+    <TestingPlatformCommandLineArguments Condition="'$(TrxFileNamePrefix)' != ''">$(TestingPlatformCommandLineArguments) --report-trx-filename  "$(TrxFileNamePrefix)_$(TargetFramework)_$([System.DateTime]::UtcNow.ToString('yyyyMMddhhmmss')).trx"</TestingPlatformCommandLineArguments>
   </PropertyGroup>
 
   <ItemGroup Condition="'$(TestsRunningOutsideOfRepo)' != 'true'">
@@ -67,8 +70,11 @@
     <PackageReference Include="xunit.v3.core" />
     <PackageReference Include="xunit.analyzers" />
     <PackageReference Include="xunit.v3.assert" />
-    <PackageReference Include="xunit.runner.visualstudio" />
-    <PackageReference Include="Microsoft.NET.Test.Sdk" />
+    <PackageReference Include="Microsoft.Testing.Platform" />
+    <PackageReference Include="Microsoft.Testing.Platform.MSBuild" />
+    <PackageReference Include="Microsoft.Testing.Extensions.HangDump" />
+    <PackageReference Include="Microsoft.Testing.Extensions.CrashDump" />
+    <PackageReference Include="Microsoft.Testing.Extensions.TrxReport" />
   </ItemGroup>
 
   <ItemGroup Condition="'$(DeployOutsideOfRepoSupportFiles)' == 'true'">
diff --git a/tests/Shared/RepoTesting/Directory.Packages.Helix.props b/tests/Shared/RepoTesting/Directory.Packages.Helix.props
index a36826bcfe9..3a74f01ca7e 100644
--- a/tests/Shared/RepoTesting/Directory.Packages.Helix.props
+++ b/tests/Shared/RepoTesting/Directory.Packages.Helix.props
@@ -86,7 +86,5 @@
 
   <ItemGroup Label="For tests">
     <PackageVersion Include="mstest.testadapter" Version="$(MSTestTestAdapterVersion)" />
-    <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="$(MicrosoftNETTestSdkVersion)" />
-    <PackageVersion Include="Microsoft.DotNet.XUnitAssert" Version="$(MicrosoftDotNetXUnitAssertVersion)" />
   </ItemGroup>
 </Project>
diff --git a/tests/Shared/TemplatesTesting/BuildEnvironment.cs b/tests/Shared/TemplatesTesting/BuildEnvironment.cs
index ad3454b5e05..afc83b1ae3f 100644
--- a/tests/Shared/TemplatesTesting/BuildEnvironment.cs
+++ b/tests/Shared/TemplatesTesting/BuildEnvironment.cs
@@ -158,8 +158,6 @@ public BuildEnvironment(bool useSystemDotNet = false, string sdkDirName = "dotne
         // Avoid using the msbuild terminal logger, so the output can be read
         // in the tests
         EnvVars["_MSBUILDTLENABLED"] = "0";
-        // .. and disable new output style for vstest
-        EnvVars["VsTestUseMSBuildOutput"] = "false";
         EnvVars["SkipAspireWorkloadManifest"] = "true";
 
         DotNet = Path.Combine(sdkForTemplatePath!, "dotnet");
diff --git a/tests/helix/send-to-helix-basictests.targets b/tests/helix/send-to-helix-basictests.targets
index fcfb8bdc47d..29b05b8a3e1 100644
--- a/tests/helix/send-to-helix-basictests.targets
+++ b/tests/helix/send-to-helix-basictests.targets
@@ -21,8 +21,7 @@
       <!-- needed for Aspire.Hosting.Container.Tests -->
       <HelixPreCommand Include="$(_EnvVarSetKeyword) DOCKER_BUILDKIT=1" />
 
-      <_TestRunCommandArguments Condition="'$(OS)' != 'Windows_NT'" Include="--filter &quot;quarantined!=true&quot; -- RunConfiguration.TestSessionTimeout=$TEST_TIMEOUT" />
-      <_TestRunCommandArguments Condition="'$(OS)' == 'Windows_NT'" Include="--filter &quot;quarantined!=true&quot; -- RunConfiguration.TestSessionTimeout=%TEST_TIMEOUT%" />
+      <_TestRunCommandArguments Include="--filter-not-trait &quot;quarantined=true&quot;" />
     </ItemGroup>
 
     <PropertyGroup>
@@ -37,16 +36,9 @@
       <_DefaultWorkItems TestName="$([System.Text.RegularExpressions.Regex]::Replace(%(FileName), $(TargetFrameworkSuffixRegex), '$1'))" />
       <_DefaultWorkItems TestNameSuffix="$([System.Text.RegularExpressions.Regex]::Replace(%(FileName), $(TargetFrameworkSuffixRegex), '$2'))" />
 
-      <!-- runsettings timeout in ms, default to 15 mins -->
-      <_DefaultWorkItems TimeoutMs="900000" />
-
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Hosting.Elasticsearch.Tests'))" TimeoutMs="3600000" />
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Hosting.Oracle.Tests'))" TimeoutMs="1200000" />
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Pomelo.EntityFrameworkCore.MySql.Tests'))" TimeoutMs="1200000" />
-
       <HelixWorkItem Include="@(_DefaultWorkItems -> '%(FileName)')">
         <PayloadArchive>%(Identity)</PayloadArchive>
-        <PreCommands>$(_EnvVarSetKeyword) &quot;TEST_NAME=%(TestName)&quot; $(_ShellCommandSeparator) $(_EnvVarSetKeyword) TEST_TIMEOUT=%(TimeoutMs) $(_ShellCommandSeparator) $(_EnvVarSetKeyword) &quot;TEST_NAME_SUFFIX=%(TestNameSuffix)&quot; $(_ShellCommandSeparator) $(_EnvVarSetKeyword) &quot;CODE_COV_FILE_SUFFIX=%(TestNameSuffix)&quot;</PreCommands>
+        <PreCommands>$(_EnvVarSetKeyword) &quot;TEST_NAME=%(TestName)&quot; $(_ShellCommandSeparator) $(_EnvVarSetKeyword) $(_ShellCommandSeparator) $(_EnvVarSetKeyword) &quot;TEST_NAME_SUFFIX=%(TestNameSuffix)&quot; $(_ShellCommandSeparator) $(_EnvVarSetKeyword) &quot;CODE_COV_FILE_SUFFIX=%(TestNameSuffix)&quot;</PreCommands>
 
         <Command>$(_TestRunCommand)</Command>
         <Timeout>$(_workItemTimeout)</Timeout>
diff --git a/tests/helix/send-to-helix-buildonhelixtests.targets b/tests/helix/send-to-helix-buildonhelixtests.targets
index 096bf419beb..19e0c189d19 100644
--- a/tests/helix/send-to-helix-buildonhelixtests.targets
+++ b/tests/helix/send-to-helix-buildonhelixtests.targets
@@ -31,14 +31,14 @@
       <_TestRunCommandArguments Remove="@(_TestRunCommandArguments)" />
 
       <_TestBlameArguments Remove="@(_TestBlameArguments)" />
-      <_TestBlameArguments Include="--blame-crash" />
-      <_TestBlameArguments Include="--blame-crash-dump-type full" />
+      <_TestBlameArguments Include="--crashdump" />
+      <_TestBlameArguments Include="--crashdump-type Full" />
 
       <!-- Using `dotnet test` for the project directly here -->
-      <_TestRunCommandArguments Include="dotnet test -s .runsettings --results-directory $(_HelixLogsPath) -v:n" />
+      <_TestRunCommandArguments Include="dotnet test -- --results-directory $(_HelixLogsPath) --report-trx --report-trx-filename TestResults.trx" />
       <_TestRunCommandArguments Include="@(_TestBlameArguments, ' ')" />
 
-      <_TestRunCommandArguments Include="--filter &quot;quarantined!=true&quot;" />
+      <_TestRunCommandArguments Include="--filter-not-trait &quot;quarantined=true&quot;" />
     </ItemGroup>
 
     <PropertyGroup>
diff --git a/tests/helix/send-to-helix-endtoendtests.targets b/tests/helix/send-to-helix-endtoendtests.targets
index a64f35da2a4..1e9b2d70f33 100644
--- a/tests/helix/send-to-helix-endtoendtests.targets
+++ b/tests/helix/send-to-helix-endtoendtests.targets
@@ -21,7 +21,7 @@
            Text="Could not find EndToEnd tests at %24(_E2ETestsArchivePath)=$(_E2ETestsArchivePath)" />
 
     <ItemGroup>
-      <_TestRunCommandArguments Include="--filter &quot;(quarantined!=true)&amp;(scenario=$(_TestScenarioEnvVar))&quot;" />
+      <_TestRunCommandArguments Include="--filter-not-trait &quot;quarantined=true&quot; --filter-trait &quot;scenario=$(_TestScenarioEnvVar)&quot;" />
     </ItemGroup>
 
     <PropertyGroup>
diff --git a/tests/helix/send-to-helix-inner.proj b/tests/helix/send-to-helix-inner.proj
index 61095f4a741..3ad04b6656e 100644
--- a/tests/helix/send-to-helix-inner.proj
+++ b/tests/helix/send-to-helix-inner.proj
@@ -58,13 +58,19 @@
   </PropertyGroup>
 
   <ItemGroup>
-    <_TestBlameArguments Include="--blame-hang" />
-    <_TestBlameArguments Include="--blame-hang-dump-type full" />
-    <_TestBlameArguments Include="--blame-hang-timeout 10m" />
-    <_TestBlameArguments Include="--blame-crash" />
-    <_TestBlameArguments Include="--blame-crash-dump-type full" />
-
-    <_TestRunCommandArguments Include="$(_SetExecutableBitCommand)dotnet test -s .runsettings $(_TestNameEnvVar).dll --ResultsDirectory:$(_HelixLogsPath) -v:n" />
+    <_TestBlameArguments Include="--hangdump" />
+    <_TestBlameArguments Include="--hangdump-type Full" />
+    <_TestBlameArguments Include="--hangdump-timeout 10m" />
+    <_TestBlameArguments Include="--crashdump" />
+    <_TestBlameArguments Include="--crashdump-type Full" />
+
+    <!--
+      At time of writing, Aspire.Components.Common.Tests doesn't have runnable tests in CI. In that case, we don't want to fail.
+      Also EndToEnd.Tests doesn't have tests matching scenario=cosmos
+      So, ignore exit code 8 (zero tests ran)
+      https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes
+    -->
+    <_TestRunCommandArguments Include="$(_SetExecutableBitCommand)dotnet exec $(_TestNameEnvVar).dll --results-directory:$(_HelixLogsPath) --report-trx --report-trx-filename TestResults.trx --filter-not-trait &quot;category=failing&quot; --ignore-exit-code 8" />
     <_TestRunCommandArguments Include="@(_TestBlameArguments, ' ')" />
   </ItemGroup>
 
diff --git a/tests/helix/send-to-helix-templatestests.targets b/tests/helix/send-to-helix-templatestests.targets
index cdd6cee0918..f731f3b41c4 100644
--- a/tests/helix/send-to-helix-templatestests.targets
+++ b/tests/helix/send-to-helix-templatestests.targets
@@ -59,8 +59,8 @@
 
           To fix that, append '.' to the name so only the test class with exactly that name is run.
         -->
-        <PreCommands Condition="'$(OS)' == 'Windows_NT'">%(PreCommands) $(_ShellCommandSeparator) set &quot;TEST_ARGS=--filter quarantined^^!=true^&amp;category^^!=failing^&amp;FullyQualifiedName~%(Identity).&quot;</PreCommands>
-        <PreCommands Condition="'$(OS)' != 'Windows_NT'">%(PreCommands) $(_ShellCommandSeparator) export &quot;TEST_ARGS=--filter quarantined!=true&amp;category!=failing&amp;FullyQualifiedName~%(Identity).&quot;</PreCommands>
+        <PreCommands Condition="'$(OS)' == 'Windows_NT'">%(PreCommands) $(_ShellCommandSeparator) set &quot;TEST_ARGS=--filter-not-trait quarantined=true --filter-not-trait category=failing --filter-class %(Identity)&quot;</PreCommands>
+        <PreCommands Condition="'$(OS)' != 'Windows_NT'">%(PreCommands) $(_ShellCommandSeparator) export &quot;TEST_ARGS=--filter-not-trait quarantined=true --filter-not-trait category=failing --filter-class %(Identity)&quot;</PreCommands>
 
         <Command>$(_TestRunCommand)</Command>
         <Timeout>$(_workItemTimeout)</Timeout>


Reviews:
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:08:37Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:12:19Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:14:14Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:15:02Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:17:42Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:19:03Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:19:19Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:19:59Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:20:23Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:22:13Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:22:58Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:55:05Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:56:10Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T16:58:38Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:00:34Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:02:21Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:02:56Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:03:49Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:04:06Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:04:28Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:05:58Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:08:15Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:10:08Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:15:30Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T17:18:39Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T18:14:50Z
user: login: danmoseley
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T18:20:29Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-03T06:32:16Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-03T06:34:25Z
user: login: RussKie
body: 🚀 

I had a chat with @nohwnd, and he mentioned once we're on MTP, we may be able to use [`--retry-failed-tests`](https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-extensions-policy) functionality.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-03T06:36:46Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-03T07:05:45Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-03T07:11:20Z
user: login: radical
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-03T07:21:40Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-03T07:40:06Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-03T10:45:06Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-03T11:23:15Z
user: login: radical
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-04T18:27:14Z
user: login: radical
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-04T19:43:53Z
user: login: eerhardt
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T15:42:14Z
user: login: eerhardt
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T15:43:01Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T15:57:25Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T15:59:13Z
user: login: eerhardt
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T18:47:15Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-07T22:00:11Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-07T22:03:10Z
user: login: radical
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T22:06:48Z
user: login: radical
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T22:38:41Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T02:48:37Z
user: login: radical
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T21:02:09Z
user: login: radical
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-09T01:53:32Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T08:55:48Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-09T09:46:27Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-09T09:50:55Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-09T09:51:31Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-09T13:32:45Z
user: login: radical
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-09T14:47:02Z
user: login: radical
body: This looks great! Test counts, and artifacts look great. Anything missed can be handled in a follow up.
Thank you for all the work! Ready to merge 👍 
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-09T16:59:50Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-10T03:37:41Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T04:52:56Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-10T04:54:34Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T05:07:06Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T05:07:51Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T05:08:37Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-10T05:40:59Z
user: login: RussKie
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-10T06:32:48Z
user: login: Youssef1313
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T06:35:53Z

Review comments:
diff_hunk: @@ -1,26 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
+--show-live-output on
path: tests/Aspire.EndToEnd.Tests/.runsettings
user: login: danmoseley
body: if we wanted to experiment with parallel tests runs for a particular library, we'd put it in a file like this presumably? (right now none may be stable for parallel runs, nor diagnosable, but for traditional unit tests it may be possible)
created_at: 2025-04-02T16:08:37Z
author_association: MEMBER
diff_hunk: @@ -240,24 +235,23 @@ jobs:
           CI: false
           DCP_DIAGNOSTICS_LOG_LEVEL: debug
           DCP_DIAGNOSTICS_LOG_FOLDER: ${{ github.workspace }}/testresults/dcp
+          # During restore and build, we use -ci, which causes NUGET_PACKAGES to point to a local cache (Arcade behavior).
+          # In this step, we are not using Arcade, but want to make sure that MSBuild is able to evaluate correctly.
+          # So, we manually set NUGET_PACKAGES
+          NUGET_PACKAGES: ${{ github.workspace }}/.packages
+        
         run: >
           ${{ env.DOTNET_SCRIPT }} test ${{ env.TEST_PROJECT_PATH }}
           /p:ContinuousIntegrationBuild=true
-          -s eng/testing/.runsettings
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          "--blame"
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
-          --results-directory testresults
+          -bl:${{ github.workspace }}/testresults/test.binlog
           --no-restore
           --no-build
-          -bl:${{ github.workspace }}/testresults/test.binlog
-          ${{ inputs.extraTestArgs }}
           --
-          RunConfiguration.CollectSourceInformation=true
-          RunConfiguration.TestSessionTimeout=${{ inputs.testSessionTimeoutMs }}
+          --report-trx --report-trx-filename "${{ inputs.testShortName }}.trx"
path: .github/workflows/run-tests.yml
user: login: danmoseley
body: did you notice whether hang/crash dumps go into the folder that is uploaded as part of test results? I can't find a doc on where they are written if you don't set filename.
created_at: 2025-04-02T16:12:19Z
author_association: MEMBER
diff_hunk: @@ -102,8 +102,7 @@ jobs:
       testProjectPath: tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj
       testSessionTimeoutMs: 1200000
       testHangTimeout: 12m
-      # append '.' to the name so only the test class with exactly that name is run
-      extraTestArgs: "--filter \"(quarantined!=true)&(FullyQualifiedName~Aspire.Templates.Tests.${{ matrix.shortname }}.)\""
+      extraTestArgs: "--filter-not-trait quarantined=true --filter-class Aspire.Templates.Tests.${{ matrix.shortname }}"
path: .github/workflows/tests.yml
user: login: danmoseley
body: these are implicitly anded? I guess we would notice if not
created_at: 2025-04-02T16:14:13Z
author_association: MEMBER
diff_hunk: @@ -1,17 +1,12 @@
 <Project>
-  <PropertyGroup Condition="'$(UseVSTestRunner)' != 'true'">
-    <_QuarantinedTestRunAdditionalArgs>-trait "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
-    <_NonQuarantinedTestRunAdditionalArgs>-notrait "quarantined=true"</_NonQuarantinedTestRunAdditionalArgs>
-  </PropertyGroup>
-
-  <PropertyGroup Condition="'$(UseVSTestRunner)' == 'true'">
-    <_QuarantinedTestRunAdditionalArgs>--filter "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
-    <_NonQuarantinedTestRunAdditionalArgs>--filter "quarantined!=true"</_NonQuarantinedTestRunAdditionalArgs>
+  <PropertyGroup>
+    <_QuarantinedTestRunAdditionalArgs>--filter-trait "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
+    <_NonQuarantinedTestRunAdditionalArgs>--filter-not-trait "quarantined=true"</_NonQuarantinedTestRunAdditionalArgs>
   </PropertyGroup>
 
   <PropertyGroup>
     <BlameHangTimeout Condition="'$(BlameHangTimeout)' == ''">10m</BlameHangTimeout>
path: eng/Testing.props
user: login: danmoseley
body: seems this is 12min elsewhere. maybe we should pick one.
created_at: 2025-04-02T16:15:01Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,91 @@
+<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. -->
path: eng/Xunit3/Microsoft.Testing.Platform.targets
user: login: danmoseley
body: this is src\Microsoft.DotNet.Arcade.Sdk\tools\Microsoft.Testing.Platform.targets from Arcade? here temporarily?
created_at: 2025-04-02T16:17:41Z
author_association: MEMBER
diff_hunk: @@ -1,35 +1,2 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <RunConfiguration>
-    <!-- Timeout in ms, 20 minutes -->
-    <TestSessionTimeout>1200000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="blame" enabled="True" />
-    </Loggers>
-  </LoggerRunSettings>
-  <DataCollectionRunSettings>
-    <DataCollectors>
-      <!-- Enables blame -->
-      <DataCollector friendlyName="blame" enabled="True">
-        <Configuration>
-          <CollectDump DumpType="Full" />
-          <CollectDumpOnTestSessionHang TestTimeout="7min" HangDumpType="Full" />
-        </Configuration>
-      </DataCollector>
-    </DataCollectors>
-  </DataCollectionRunSettings>
-</RunSettings>
+# Filter out failing (wrong framework, platform, runtime or activeissue) tests
+--filter-not-trait "category=failing"
path: eng/testing/.runsettings
user: login: danmoseley
body: how is eng/testing/.runsettings discovered if there's no tests below it?
created_at: 2025-04-02T16:19:03Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,7 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: danmoseley
body: ```suggestion
    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code </TestingPlatformCommandLineArguments>
```
?
created_at: 2025-04-02T16:19:19Z
author_association: MEMBER
diff_hunk: @@ -1,26 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
+--show-live-output on
+
+# Filter out failing (wrong framework, platform, runtime or activeissue) tests
+--filter-not-trait "category=failing"
path: tests/Aspire.EndToEnd.Tests/.runsettings
user: login: danmoseley
body: I guess these could go into tests/.runsettings so we don't need lots of files? but perhaps they don't aggregate?
created_at: 2025-04-02T16:19:59Z
author_association: MEMBER
diff_hunk: @@ -20,10 +20,6 @@
     <OutputType Condition="$(MSBuildProjectName.EndsWith('.Tests'))">Exe</OutputType>
   </PropertyGroup>
 
-  <ItemGroup>
-    <PackageReference Include="GitHubActionsTestLogger" PrivateAssets="all" IncludeAssets="runtime; build; native; contentfiles; analyzers" />
path: tests/Directory.Build.props
user: login: danmoseley
body: are we losing anything from dropping this?
created_at: 2025-04-02T16:20:23Z
author_association: MEMBER
diff_hunk: @@ -30,15 +30,16 @@
     <ItemGroup>
       <_TestRunCommandArguments Remove="@(_TestRunCommandArguments)" />
 
+      <!-- TODO: Fix this for MTP -->
path: tests/helix/send-to-helix-buildonhelixtests.targets
user: login: danmoseley
body: noting todo here.
created_at: 2025-04-02T16:22:13Z
author_association: MEMBER
diff_hunk: @@ -30,15 +30,16 @@
     <ItemGroup>
       <_TestRunCommandArguments Remove="@(_TestRunCommandArguments)" />
 
+      <!-- TODO: Fix this for MTP -->
       <_TestBlameArguments Remove="@(_TestBlameArguments)" />
       <_TestBlameArguments Include="--blame-crash" />
       <_TestBlameArguments Include="--blame-crash-dump-type full" />
 
       <!-- Using `dotnet test` for the project directly here -->
-      <_TestRunCommandArguments Include="dotnet test -s .runsettings --results-directory $(_HelixLogsPath) -v:n" />
+      <_TestRunCommandArguments Include="dotnet exec @.runsettings --results-directory $(_HelixLogsPath)" />
path: tests/helix/send-to-helix-buildonhelixtests.targets
user: login: danmoseley
body: this is finding .runsettings next to the test csproj?
created_at: 2025-04-02T16:22:58Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,7 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: Youssef1313
body: The exit code 8 is "zero tests ran". Currently, all tests in this project seem to be ignored when running in CI. I'll add a link to https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-exit-codes above this line so that it doesn't seem like a magic number that can't be understood.
created_at: 2025-04-02T16:55:05Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,91 @@
+<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. -->
path: eng/Xunit3/Microsoft.Testing.Platform.targets
user: login: Youssef1313
body: It will be supported in Arcade, hopefully. My PR there is still WIP.

The whole `eng/Xunit3` directory should be removable once everything is in Arcade and you move to .NET 10
created_at: 2025-04-02T16:56:10Z
author_association: MEMBER
diff_hunk: @@ -102,8 +102,7 @@ jobs:
       testProjectPath: tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj
       testSessionTimeoutMs: 1200000
       testHangTimeout: 12m
-      # append '.' to the name so only the test class with exactly that name is run
-      extraTestArgs: "--filter \"(quarantined!=true)&(FullyQualifiedName~Aspire.Templates.Tests.${{ matrix.shortname }}.)\""
+      extraTestArgs: "--filter-not-trait quarantined=true --filter-class Aspire.Templates.Tests.${{ matrix.shortname }}"
path: .github/workflows/tests.yml
user: login: Youssef1313
body: Hmm, I actually need to make test to make sure if they are ANDed or ORed.
created_at: 2025-04-02T16:58:38Z
author_association: MEMBER
diff_hunk: @@ -240,24 +235,23 @@ jobs:
           CI: false
           DCP_DIAGNOSTICS_LOG_LEVEL: debug
           DCP_DIAGNOSTICS_LOG_FOLDER: ${{ github.workspace }}/testresults/dcp
+          # During restore and build, we use -ci, which causes NUGET_PACKAGES to point to a local cache (Arcade behavior).
+          # In this step, we are not using Arcade, but want to make sure that MSBuild is able to evaluate correctly.
+          # So, we manually set NUGET_PACKAGES
+          NUGET_PACKAGES: ${{ github.workspace }}/.packages
+        
         run: >
           ${{ env.DOTNET_SCRIPT }} test ${{ env.TEST_PROJECT_PATH }}
           /p:ContinuousIntegrationBuild=true
-          -s eng/testing/.runsettings
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          "--blame"
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
-          --results-directory testresults
+          -bl:${{ github.workspace }}/testresults/test.binlog
           --no-restore
           --no-build
-          -bl:${{ github.workspace }}/testresults/test.binlog
-          ${{ inputs.extraTestArgs }}
           --
-          RunConfiguration.CollectSourceInformation=true
-          RunConfiguration.TestSessionTimeout=${{ inputs.testSessionTimeoutMs }}
+          --report-trx --report-trx-filename "${{ inputs.testShortName }}.trx"
path: .github/workflows/run-tests.yml
user: login: Youssef1313
body: They should go into results directory (whatever specified as `--results-directory`). If `--results-directory` isn't specified, it should be defaulting to `TestResults` (relative to the test executable AFAIK, but I think you always specify `--results-directory` anyways)
created_at: 2025-04-02T17:00:34Z
author_association: MEMBER
diff_hunk: @@ -1,26 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
+--show-live-output on
path: tests/Aspire.EndToEnd.Tests/.runsettings
user: login: Youssef1313
body: Either that, or add it to `TestingPlatformCommandLineArguments`.

Note: the current file is no longer really runsettings, it's a response file. RunSettings is actually not supported by core MTP (there is limited support for it through VSTestBridge, but xUnit doesn't use the bridge).
created_at: 2025-04-02T17:02:20Z
author_association: MEMBER
diff_hunk: @@ -1,26 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
+--show-live-output on
path: tests/Aspire.EndToEnd.Tests/.runsettings
user: login: Youssef1313
body: You can also do `dotnet run --project path/to/test-project.csproj -- --help` and it will show you all the command-line options available.
created_at: 2025-04-02T17:02:56Z
author_association: MEMBER
diff_hunk: @@ -102,8 +102,7 @@ jobs:
       testProjectPath: tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj
       testSessionTimeoutMs: 1200000
       testHangTimeout: 12m
-      # append '.' to the name so only the test class with exactly that name is run
-      extraTestArgs: "--filter \"(quarantined!=true)&(FullyQualifiedName~Aspire.Templates.Tests.${{ matrix.shortname }}.)\""
+      extraTestArgs: "--filter-not-trait quarantined=true --filter-class Aspire.Templates.Tests.${{ matrix.shortname }}"
path: .github/workflows/tests.yml
user: login: danmoseley
body: it would be great if you could check test counts before and after this change, before we merge the PR..!
created_at: 2025-04-02T17:03:49Z
author_association: MEMBER
diff_hunk: @@ -1,17 +1,12 @@
 <Project>
-  <PropertyGroup Condition="'$(UseVSTestRunner)' != 'true'">
-    <_QuarantinedTestRunAdditionalArgs>-trait "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
-    <_NonQuarantinedTestRunAdditionalArgs>-notrait "quarantined=true"</_NonQuarantinedTestRunAdditionalArgs>
-  </PropertyGroup>
-
-  <PropertyGroup Condition="'$(UseVSTestRunner)' == 'true'">
-    <_QuarantinedTestRunAdditionalArgs>--filter "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
-    <_NonQuarantinedTestRunAdditionalArgs>--filter "quarantined!=true"</_NonQuarantinedTestRunAdditionalArgs>
+  <PropertyGroup>
+    <_QuarantinedTestRunAdditionalArgs>--filter-trait "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
+    <_NonQuarantinedTestRunAdditionalArgs>--filter-not-trait "quarantined=true"</_NonQuarantinedTestRunAdditionalArgs>
   </PropertyGroup>
 
   <PropertyGroup>
     <BlameHangTimeout Condition="'$(BlameHangTimeout)' == ''">10m</BlameHangTimeout>
path: eng/Testing.props
user: login: Youssef1313
body: There is a bit of inconsistency in how tests are setup currently, with too many runsettings, and `dotnet test` calls that specify things that are already in runsettings (and the command-line options then take precedence over runsettings).

created_at: 2025-04-02T17:04:06Z
author_association: MEMBER
diff_hunk: @@ -1,26 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
+--show-live-output on
path: tests/Aspire.EndToEnd.Tests/.runsettings
user: login: danmoseley
body: why do we still have .runsettings if it's not really supported?
created_at: 2025-04-02T17:04:28Z
author_association: MEMBER
diff_hunk: @@ -1,35 +1,2 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <RunConfiguration>
-    <!-- Timeout in ms, 20 minutes -->
-    <TestSessionTimeout>1200000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="blame" enabled="True" />
-    </Loggers>
-  </LoggerRunSettings>
-  <DataCollectionRunSettings>
-    <DataCollectors>
-      <!-- Enables blame -->
-      <DataCollector friendlyName="blame" enabled="True">
-        <Configuration>
-          <CollectDump DumpType="Full" />
-          <CollectDumpOnTestSessionHang TestTimeout="7min" HangDumpType="Full" />
-        </Configuration>
-      </DataCollector>
-    </DataCollectors>
-  </DataCollectionRunSettings>
-</RunSettings>
+# Filter out failing (wrong framework, platform, runtime or activeissue) tests
+--filter-not-trait "category=failing"
path: eng/testing/.runsettings
user: login: Youssef1313
body: tests/Directory.Build.props has:

```xml
    <RunSettingsFilePath>$(RepositoryEngineeringDir)testing\.runsettings</RunSettingsFilePath>
```

That's then used in:

```xml
    <TestingPlatformCommandLineArguments Condition="'$(RunSettingsFilePath)'!=''">$(TestingPlatformCommandLineArguments) @"$(RunSettingsFilePath)"</TestingPlatformCommandLineArguments>
```
created_at: 2025-04-02T17:05:58Z
author_association: MEMBER
diff_hunk: @@ -30,15 +30,16 @@
     <ItemGroup>
       <_TestRunCommandArguments Remove="@(_TestRunCommandArguments)" />
 
+      <!-- TODO: Fix this for MTP -->
       <_TestBlameArguments Remove="@(_TestBlameArguments)" />
       <_TestBlameArguments Include="--blame-crash" />
       <_TestBlameArguments Include="--blame-crash-dump-type full" />
 
       <!-- Using `dotnet test` for the project directly here -->
-      <_TestRunCommandArguments Include="dotnet test -s .runsettings --results-directory $(_HelixLogsPath) -v:n" />
+      <_TestRunCommandArguments Include="dotnet exec @.runsettings --results-directory $(_HelixLogsPath)" />
path: tests/helix/send-to-helix-buildonhelixtests.targets
user: login: Youssef1313
body: I think next to test executable. And that will be there because you already have this in tests/Directory.Build.targets:

```
    <None Include="$(RunSettingsFilePath)" CopyToOutputDirectory="PreserveNewest" Condition="'$(DeployRunSettingsFile)' == 'true'" />

```
created_at: 2025-04-02T17:08:15Z
author_association: MEMBER
diff_hunk: @@ -20,10 +20,6 @@
     <OutputType Condition="$(MSBuildProjectName.EndsWith('.Tests'))">Exe</OutputType>
   </PropertyGroup>
 
-  <ItemGroup>
-    <PackageReference Include="GitHubActionsTestLogger" PrivateAssets="all" IncludeAssets="runtime; build; native; contentfiles; analyzers" />
path: tests/Directory.Build.props
user: login: Youssef1313
body: Yes.

From GitHubActionsTestLogger README, it does things similar to the following:

![image](https://github.com/user-attachments/assets/816a04f3-a414-4a90-9b69-9f596b2fa3b7)

![image](https://github.com/user-attachments/assets/ceb2ed6d-efd3-4517-8909-d8eb71999a5f)

We can discuss internally if it's important for you though.
created_at: 2025-04-02T17:10:07Z
author_association: MEMBER
diff_hunk: @@ -1,26 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
+--show-live-output on
path: tests/Aspire.EndToEnd.Tests/.runsettings
user: login: Youssef1313
body: The file name is just `.runsettings` but it acts as a command-line response file. I can rename for clarity.

The original intent is very unclear to me, there are too many .runsettings scattered and it's not claer which is used for what and when.

If you can clarify the original intent, maybe there can be a good room to clean it up more.
created_at: 2025-04-02T17:15:29Z
author_association: MEMBER
diff_hunk: @@ -102,8 +102,7 @@ jobs:
       testProjectPath: tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj
       testSessionTimeoutMs: 1200000
       testHangTimeout: 12m
-      # append '.' to the name so only the test class with exactly that name is run
-      extraTestArgs: "--filter \"(quarantined!=true)&(FullyQualifiedName~Aspire.Templates.Tests.${{ matrix.shortname }}.)\""
+      extraTestArgs: "--filter-not-trait quarantined=true --filter-class Aspire.Templates.Tests.${{ matrix.shortname }}"
path: .github/workflows/tests.yml
user: login: Youssef1313
body: Yup sure.
created_at: 2025-04-02T17:18:39Z
author_association: MEMBER
diff_hunk: @@ -1,26 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
+--show-live-output on
path: tests/Aspire.EndToEnd.Tests/.runsettings
user: login: danmoseley
body: I cannot clarify. Possibly @radical can or perhaps it's just happenstance, this isn't a mature repo. Any cleanup welcome!
created_at: 2025-04-02T18:14:50Z
author_association: MEMBER
diff_hunk: @@ -20,10 +20,6 @@
     <OutputType Condition="$(MSBuildProjectName.EndsWith('.Tests'))">Exe</OutputType>
   </PropertyGroup>
 
-  <ItemGroup>
-    <PackageReference Include="GitHubActionsTestLogger" PrivateAssets="all" IncludeAssets="runtime; build; native; contentfiles; analyzers" />
path: tests/Directory.Build.props
user: login: danmoseley
body: It's OK and we can worry about this later/discuss internally later.
created_at: 2025-04-02T18:20:28Z
author_association: MEMBER
diff_hunk: @@ -144,7 +147,6 @@
     <PackageVersion Include="Testcontainers.Oracle" Version="$(TestcontainersPackageVersion)" />
     <PackageVersion Include="Testcontainers.Elasticsearch" Version="$(TestcontainersPackageVersion)" />
     <PackageVersion Include="Testcontainers" Version="$(TestcontainersPackageVersion)" />
-    <PackageVersion Include="GitHubActionsTestLogger" Version="2.4.1" />
path: Directory.Packages.props
user: login: RussKie
body: 🎉 
created_at: 2025-04-03T06:32:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -20,10 +20,6 @@
     <OutputType Condition="$(MSBuildProjectName.EndsWith('.Tests'))">Exe</OutputType>
   </PropertyGroup>
 
-  <ItemGroup>
-    <PackageReference Include="GitHubActionsTestLogger" PrivateAssets="all" IncludeAssets="runtime; build; native; contentfiles; analyzers" />
path: tests/Directory.Build.props
user: login: RussKie
body: I'm glad we're removing this package.
created_at: 2025-04-03T06:34:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,26 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
+--show-live-output on
path: tests/Aspire.EndToEnd.Tests/.runsettings
user: login: Youssef1313
body: Okay, I went ahead and cleaned up all the `.runsettings`.

Any customizations that are needed for a specific project can be done via `TestingPlatformCommandLineArguments` MSBuild property. This allows more flexibility as you can customize in `Directory.Build.[props|targets]` as well, based on any conditions. Previously, only one `.runsettings` file can be passed, and it was too hard to track which runsettings is used where.
created_at: 2025-04-03T07:05:45Z
author_association: MEMBER
diff_hunk: @@ -102,8 +102,7 @@ jobs:
       testProjectPath: tests/Aspire.Templates.Tests/Aspire.Templates.Tests.csproj
       testSessionTimeoutMs: 1200000
       testHangTimeout: 12m
-      # append '.' to the name so only the test class with exactly that name is run
-      extraTestArgs: "--filter \"(quarantined!=true)&(FullyQualifiedName~Aspire.Templates.Tests.${{ matrix.shortname }}.)\""
+      extraTestArgs: "--filter-not-trait quarantined=true --filter-class Aspire.Templates.Tests.${{ matrix.shortname }}"
path: .github/workflows/tests.yml
user: login: Youssef1313
body: I made a quick test and yes they are ANDed.
created_at: 2025-04-03T07:11:20Z
author_association: MEMBER
diff_hunk: @@ -1,26 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
-<RunSettings>
-  <xUnit>
-    <ShowLiveOutput>true</ShowLiveOutput>
-  </xUnit>
-  <RunConfiguration>
-    <!-- Timeout in ms, 15 minutes -->
-    <TestSessionTimeout>900000</TestSessionTimeout>
-    <!-- Filter out failing (wrong framework, platform, runtime or activeissue) tests -->
-    <TestCaseFilter>category!=failing</TestCaseFilter>
-  </RunConfiguration>
-  <LoggerRunSettings>
-    <Loggers>
-      <Logger friendlyName="trx">
-        <Configuration>
-          <LogFileName>TestResults.trx</LogFileName>
-        </Configuration>
-      </Logger>
-      <Logger friendlyName="console">
-        <Configuration>
-          <Verbosity>normal</Verbosity>
-        </Configuration>
-      </Logger>
-    </Loggers>
-  </LoggerRunSettings>
-</RunSettings>
+--show-live-output on
path: tests/Aspire.EndToEnd.Tests/.runsettings
user: login: radical
body: We have various runsettings to have different logging and timeout setup, and the files themselves weren't compostable. This was useful for local vs CI runs. I did start moving to overriding on the command line. But cleaning all this up to use command line parameters via msbuild properties will be great!!
created_at: 2025-04-03T07:21:40Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: RussKie
body: ```suggestion
    <!--
        https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes
        Exit code 8 is "zero tests ran", currently, none of the tests in this project run in CI. All are ignored
      -->
```

https://learn.microsoft.com/help/platform/links-how-to?branch=main#locale-codes-in-external-links
created_at: 2025-04-03T07:40:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -21,8 +21,8 @@
     <DefineConstants Condition="'$(_BuildForTestsRunningOutsideOfRepo)' == 'true'">BUILD_FOR_TESTS_RUNNING_OUTSIDE_OF_REPO;$(DefineConstants)</DefineConstants>
 
     <XunitRunnerJson>xunit.runner.json</XunitRunnerJson>
-    <RunSettingsFilePath>$(MSBuildThisFileDirectory).runsettings</RunSettingsFilePath>
     <TestArchiveTestsDir>$(TestArchiveTestsDirForEndToEndTests)</TestArchiveTestsDir>
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --show-live-output on</TestingPlatformCommandLineArguments>
path: tests/Aspire.EndToEnd.Tests/Aspire.EndToEnd.Tests.csproj
user: login: Youssef1313
body: NOTE: `TestingPlatformCommandLineArguments` works only when doing `dotnet test`. But if the build happens on one machine, and test is run on another machine on the already-built assembly, this won't work.

@radical @danmoseley Is that okay for you?
I don't know if this was done for easier local debugging, or is it meant for CI scenarios, and is it relevant for both helix and non-helix?

One way this could be addressed is if xUnit supported testconfig.json: https://github.com/xunit/xunit/issues/3247
created_at: 2025-04-03T10:45:06Z
author_association: MEMBER
diff_hunk: @@ -21,8 +21,8 @@
     <DefineConstants Condition="'$(_BuildForTestsRunningOutsideOfRepo)' == 'true'">BUILD_FOR_TESTS_RUNNING_OUTSIDE_OF_REPO;$(DefineConstants)</DefineConstants>
 
     <XunitRunnerJson>xunit.runner.json</XunitRunnerJson>
-    <RunSettingsFilePath>$(MSBuildThisFileDirectory).runsettings</RunSettingsFilePath>
     <TestArchiveTestsDir>$(TestArchiveTestsDirForEndToEndTests)</TestArchiveTestsDir>
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --show-live-output on</TestingPlatformCommandLineArguments>
path: tests/Aspire.EndToEnd.Tests/Aspire.EndToEnd.Tests.csproj
user: login: Youssef1313
body: Or, we could always `--show-live-output` in all places where we invoke tests, and have consistency for all projects rather than being specified for only 3 projects currently. I'm trying to go with this approach now
created_at: 2025-04-03T11:23:14Z
author_association: MEMBER
diff_hunk: @@ -21,8 +21,8 @@
     <DefineConstants Condition="'$(_BuildForTestsRunningOutsideOfRepo)' == 'true'">BUILD_FOR_TESTS_RUNNING_OUTSIDE_OF_REPO;$(DefineConstants)</DefineConstants>
 
     <XunitRunnerJson>xunit.runner.json</XunitRunnerJson>
-    <RunSettingsFilePath>$(MSBuildThisFileDirectory).runsettings</RunSettingsFilePath>
     <TestArchiveTestsDir>$(TestArchiveTestsDirForEndToEndTests)</TestArchiveTestsDir>
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --show-live-output on</TestingPlatformCommandLineArguments>
path: tests/Aspire.EndToEnd.Tests/Aspire.EndToEnd.Tests.csproj
user: login: radical
body: We did it for specific projects only because they would have crashes, or take too long and it would help to watch the running output. I think it might be too verbose for other tests. We can start with no `--show-live-output` at all for now, and we can turn it on if we hit a need for it again.
created_at: 2025-04-04T18:27:14Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: radical
body: This is not a real test project. It needs `<IsTestUtilityProject>true</IsTestUtilityProject>` which got  dropped in #8279 . Adding that back should mean the project would get skipped completely.
created_at: 2025-04-04T19:43:53Z
author_association: MEMBER
diff_hunk: @@ -220,18 +220,14 @@ jobs:
           TEST_LOG_PATH: ${{ github.workspace }}/artifacts/log/test-logs
           TestsRunningOutsideOfRepo: true
         run: >
-          dotnet test -s .runsettings -v:n ${{ env.TEST_ASSEMBLY_NAME }}.dll
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          --blame
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
+          dotnet exec ${{ env.TEST_ASSEMBLY_NAME }}.dll
path: .github/workflows/run-tests.yml
user: login: eerhardt
body: Is this really how normal developers are supposed to execute tests using the Microsoft.Testing.Platform? In general, we don't suggest to people to use `dotnet exec`. You can `dotnet XXX.dll` or just run the platform-specific executable `${{ env.TEST_ASSEMBLY_NAME }}[.exe]`
created_at: 2025-04-07T15:42:13Z
author_association: MEMBER
diff_hunk: @@ -240,24 +236,25 @@ jobs:
           CI: false
           DCP_DIAGNOSTICS_LOG_LEVEL: debug
           DCP_DIAGNOSTICS_LOG_FOLDER: ${{ github.workspace }}/testresults/dcp
+          # During restore and build, we use -ci, which causes NUGET_PACKAGES to point to a local cache (Arcade behavior).
+          # In this step, we are not using Arcade, but want to make sure that MSBuild is able to evaluate correctly.
+          # So, we manually set NUGET_PACKAGES
+          NUGET_PACKAGES: ${{ github.workspace }}/.packages
+        
         run: >
           ${{ env.DOTNET_SCRIPT }} test ${{ env.TEST_PROJECT_PATH }}
path: .github/workflows/run-tests.yml
user: login: eerhardt
body: Why is this one still `dotnet test`?
created_at: 2025-04-07T15:43:01Z
author_association: MEMBER
diff_hunk: @@ -220,18 +220,14 @@ jobs:
           TEST_LOG_PATH: ${{ github.workspace }}/artifacts/log/test-logs
           TestsRunningOutsideOfRepo: true
         run: >
-          dotnet test -s .runsettings -v:n ${{ env.TEST_ASSEMBLY_NAME }}.dll
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          --blame
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
+          dotnet exec ${{ env.TEST_ASSEMBLY_NAME }}.dll
path: .github/workflows/run-tests.yml
user: login: Youssef1313
body: All the three approaches will work (dotnet exec dll, dotnet dll, or the executable directly). If you have preference let me know and I can refactor.
created_at: 2025-04-07T15:57:25Z
author_association: MEMBER
diff_hunk: @@ -240,24 +236,25 @@ jobs:
           CI: false
           DCP_DIAGNOSTICS_LOG_LEVEL: debug
           DCP_DIAGNOSTICS_LOG_FOLDER: ${{ github.workspace }}/testresults/dcp
+          # During restore and build, we use -ci, which causes NUGET_PACKAGES to point to a local cache (Arcade behavior).
+          # In this step, we are not using Arcade, but want to make sure that MSBuild is able to evaluate correctly.
+          # So, we manually set NUGET_PACKAGES
+          NUGET_PACKAGES: ${{ github.workspace }}/.packages
+        
         run: >
           ${{ env.DOTNET_SCRIPT }} test ${{ env.TEST_PROJECT_PATH }}
path: .github/workflows/run-tests.yml
user: login: Youssef1313
body: I'm not intending to do any broader refactorings with this PR. This was `dotnet test`, so I'm keeping as is. We can have a separate build step and then run the executable, but I'm trying to not change much as part of this PR.
created_at: 2025-04-07T15:59:13Z
author_association: MEMBER
diff_hunk: @@ -220,18 +220,14 @@ jobs:
           TEST_LOG_PATH: ${{ github.workspace }}/artifacts/log/test-logs
           TestsRunningOutsideOfRepo: true
         run: >
-          dotnet test -s .runsettings -v:n ${{ env.TEST_ASSEMBLY_NAME }}.dll
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          --blame
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
+          dotnet exec ${{ env.TEST_ASSEMBLY_NAME }}.dll
path: .github/workflows/run-tests.yml
user: login: eerhardt
body: What's the preference from the test team?

I guess I would prefer invoking the executable directly.
created_at: 2025-04-07T18:47:14Z
author_association: MEMBER
diff_hunk: @@ -220,18 +220,14 @@ jobs:
           TEST_LOG_PATH: ${{ github.workspace }}/artifacts/log/test-logs
           TestsRunningOutsideOfRepo: true
         run: >
-          dotnet test -s .runsettings -v:n ${{ env.TEST_ASSEMBLY_NAME }}.dll
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          --blame
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
+          dotnet exec ${{ env.TEST_ASSEMBLY_NAME }}.dll
path: .github/workflows/run-tests.yml
user: login: RussKie
body: I'm reworking the infra with expectation that we'd be running `build.cmd/sh` for all build, test, package, etc. needs.
created_at: 2025-04-07T22:00:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: RussKie
body: @radical, no, this _IS_ a test project. It contains tests. For example:
https://github.com/dotnet/aspire/blob/8489a323a5799e475f931c67f77362cb1646f47f/tests/Aspire.Components.Common.Tests/ConformanceTests.cs#L106-L109
https://github.com/dotnet/aspire/blob/8489a323a5799e475f931c67f77362cb1646f47f/tests/Aspire.Components.Common.Tests/ConformanceTests.cs#L120-L121

All non-test infra was moved to Aspire.TestUtils.
created_at: 2025-04-07T22:03:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: radical
body: But these tests are never run by themselves. They are run through other test classes that extend this `ConformanceTests` class.
created_at: 2025-04-07T22:06:48Z
author_association: MEMBER
diff_hunk: @@ -220,18 +220,14 @@ jobs:
           TEST_LOG_PATH: ${{ github.workspace }}/artifacts/log/test-logs
           TestsRunningOutsideOfRepo: true
         run: >
-          dotnet test -s .runsettings -v:n ${{ env.TEST_ASSEMBLY_NAME }}.dll
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          --blame
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
+          dotnet exec ${{ env.TEST_ASSEMBLY_NAME }}.dll
path: .github/workflows/run-tests.yml
user: login: radical
body: That's not a correct expectation, as for nuget testing, and tests run on helix we run just the binaries, and thus no arcade or repository is available.
created_at: 2025-04-07T22:38:41Z
author_association: MEMBER
diff_hunk: @@ -220,18 +220,14 @@ jobs:
           TEST_LOG_PATH: ${{ github.workspace }}/artifacts/log/test-logs
           TestsRunningOutsideOfRepo: true
         run: >
-          dotnet test -s .runsettings -v:n ${{ env.TEST_ASSEMBLY_NAME }}.dll
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          --blame
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
+          dotnet exec ${{ env.TEST_ASSEMBLY_NAME }}.dll
path: .github/workflows/run-tests.yml
user: login: RussKie
body: We won't be running on Helix at all.
created_at: 2025-04-08T02:48:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -220,18 +220,14 @@ jobs:
           TEST_LOG_PATH: ${{ github.workspace }}/artifacts/log/test-logs
           TestsRunningOutsideOfRepo: true
         run: >
-          dotnet test -s .runsettings -v:n ${{ env.TEST_ASSEMBLY_NAME }}.dll
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          --blame
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
+          dotnet exec ${{ env.TEST_ASSEMBLY_NAME }}.dll
path: .github/workflows/run-tests.yml
user: login: radical
body: We are using `dotnet exec` because the tests are being run without the project file available. But since the test assembly is executable now, we can directly run that.
created_at: 2025-04-08T21:02:09Z
author_association: MEMBER
diff_hunk: @@ -240,24 +236,25 @@ jobs:
           CI: false
           DCP_DIAGNOSTICS_LOG_LEVEL: debug
           DCP_DIAGNOSTICS_LOG_FOLDER: ${{ github.workspace }}/testresults/dcp
+          # During restore and build, we use -ci, which causes NUGET_PACKAGES to point to a local cache (Arcade behavior).
+          # In this step, we are not using Arcade, but want to make sure that MSBuild is able to evaluate correctly.
+          # So, we manually set NUGET_PACKAGES
+          NUGET_PACKAGES: ${{ github.workspace }}/.packages
+        
path: .github/workflows/run-tests.yml
user: login: radical
body: ```suggestion

```
created_at: 2025-04-08T23:40:36Z
author_association: MEMBER
diff_hunk: @@ -9,23 +9,18 @@
     <RunOnAzdoCILinux>true</RunOnAzdoCILinux>
     <RunOnAzdoHelixWindows>true</RunOnAzdoHelixWindows>
     <RunOnAzdoHelixLinux>true</RunOnAzdoHelixLinux>
-  </PropertyGroup>
-
-  <PropertyGroup Condition="'$(UseVSTestRunner)' != 'true'">
-    <_QuarantinedTestRunAdditionalArgs>-trait "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
-    <_NonQuarantinedTestRunAdditionalArgs>-notrait "quarantined=true"</_NonQuarantinedTestRunAdditionalArgs>
-  </PropertyGroup>
 
-  <PropertyGroup Condition="'$(UseVSTestRunner)' == 'true'">
-    <_QuarantinedTestRunAdditionalArgs>--filter "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
-    <_NonQuarantinedTestRunAdditionalArgs>--filter "quarantined!=true"</_NonQuarantinedTestRunAdditionalArgs>
+    <_QuarantinedTestRunAdditionalArgs>--filter-trait "quarantined=true"</_QuarantinedTestRunAdditionalArgs>
+    <_NonQuarantinedTestRunAdditionalArgs>--filter-not-trait "quarantined=true"</_NonQuarantinedTestRunAdditionalArgs>
   </PropertyGroup>
 
   <PropertyGroup>
     <BlameHangTimeout Condition="'$(BlameHangTimeout)' == ''">10m</BlameHangTimeout>
-    <_BlameArgs>--blame --blame-hang-timeout $(BlameHangTimeout) --blame-crash</_BlameArgs>
+    <_BlameArgs>--hangdump --hangdump-timeout $(BlameHangTimeout) --crashdump</_BlameArgs>
+
+    <TestRunnerAdditionalArguments>$(TestRunnerAdditionalArguments) --filter-not-trait &quot;category=failing&quot;</TestRunnerAdditionalArguments>
 
-    <TestRunnerAdditionalArguments Condition="'$(RunQuarantinedTests)' == ''">$(TestRunnerAdditionalArguments) $(_NonQuarantinedTestRunAdditionalArgs) $(TestRunnerAdditionalArguments) $(_BlameArgs)</TestRunnerAdditionalArguments>
-    <TestRunnerAdditionalArguments Condition="'$(RunQuarantinedTests)' == 'true'">$(TestRunnerAdditionalArguments) $(_QuarantinedTestRunAdditionalArgs) $(TestRunnerAdditionalArguments) $(_BlameArgs)</TestRunnerAdditionalArguments>
+    <TestRunnerAdditionalArguments Condition="'$(RunQuarantinedTests)' == ''">$(TestRunnerAdditionalArguments) $(_NonQuarantinedTestRunAdditionalArgs) $(_BlameArgs)</TestRunnerAdditionalArguments>
path: eng/Testing.props
user: login: radical
body: ```suggestion
    <TestRunnerAdditionalArguments Condition="'$(RunQuarantinedTests)' != 'true'">$(TestRunnerAdditionalArguments) $(_NonQuarantinedTestRunAdditionalArgs) $(_BlameArgs)</TestRunnerAdditionalArguments>
```
created_at: 2025-04-08T23:42:20Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: radical
body: The changes here can be dropped, replaced by `<IsTestUtilityProject>true</IsTestUtilityProject>`.
created_at: 2025-04-08T23:48:38Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,92 @@
+<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. -->
+<Project>
+
+  <Target Name="RunTests"
+          Outputs="%(TestToRun.ResultsStdOutPath)"
+          Condition="'@(TestToRun)' != ''">
+    <Telemetry EventName="NETCORE_ENGINEERING_TELEMETRY" EventData="Category=Test" />
+
+    <PropertyGroup>
+      <_TestResultDirectory>$([System.IO.Path]::GetDirectoryName('%(TestToRun.ResultsTrxPath)'))</_TestResultDirectory>
+      <_TestResultTrxFileName>$([System.IO.Path]::GetFileName('%(TestToRun.ResultsTrxPath)'))</_TestResultTrxFileName>
+      <_TestResultXmlFileName>$([System.IO.Path]::GetFileName('%(TestToRun.ResultsXmlPath)'))</_TestResultXmlFileName>
+      <_TestResultHtmlFileName>$([System.IO.Path]::GetFileName('%(TestToRun.ResultsHtmlPath)'))</_TestResultHtmlFileName>
+    </PropertyGroup>
+
+    <PropertyGroup>
+      <_TestEnvironment>%(TestToRun.EnvironmentDisplay)</_TestEnvironment>
+      <_TestAssembly>%(TestToRun.Identity)</_TestAssembly>
+      <_TestAssembly Condition="'$(OS)'=='Windows_NT'">$([System.IO.Path]::ChangeExtension($(_TestAssembly), '.exe'))</_TestAssembly>
+      <_TestAssembly Condition="'$(OS)'!='Windows_NT'">$(_TestAssembly.TrimEnd('.dll'))</_TestAssembly>
+      <_TestRuntime>%(TestToRun.TestRuntime)</_TestRuntime>
+      <_TestTimeout>%(TestToRun.TestTimeout)</_TestTimeout>
path: eng/Xunit3/Microsoft.Testing.Platform.targets
user: login: radical
body: Can this timeout be set by test project itself? We would need that because some tests need higher timeouts than others.
created_at: 2025-04-08T23:51:39Z
author_association: MEMBER
diff_hunk: @@ -37,16 +36,9 @@
       <_DefaultWorkItems TestName="$([System.Text.RegularExpressions.Regex]::Replace(%(FileName), $(TargetFrameworkSuffixRegex), '$1'))" />
       <_DefaultWorkItems TestNameSuffix="$([System.Text.RegularExpressions.Regex]::Replace(%(FileName), $(TargetFrameworkSuffixRegex), '$2'))" />
 
-      <!-- runsettings timeout in ms, default to 15 mins -->
-      <_DefaultWorkItems TimeoutMs="900000" />
-
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Hosting.Elasticsearch.Tests'))" TimeoutMs="3600000" />
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Hosting.Oracle.Tests'))" TimeoutMs="1200000" />
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Pomelo.EntityFrameworkCore.MySql.Tests'))" TimeoutMs="1200000" />
-
       <HelixWorkItem Include="@(_DefaultWorkItems -> '%(FileName)')">
         <PayloadArchive>%(Identity)</PayloadArchive>
-        <PreCommands>$(_EnvVarSetKeyword) &quot;TEST_NAME=%(TestName)&quot; $(_ShellCommandSeparator) $(_EnvVarSetKeyword) TEST_TIMEOUT=%(TimeoutMs) $(_ShellCommandSeparator) $(_EnvVarSetKeyword) &quot;TEST_NAME_SUFFIX=%(TestNameSuffix)&quot; $(_ShellCommandSeparator) $(_EnvVarSetKeyword) &quot;CODE_COV_FILE_SUFFIX=%(TestNameSuffix)&quot;</PreCommands>
path: tests/helix/send-to-helix-basictests.targets
user: login: radical
body: Can we still set these custom timeouts somewhere?
created_at: 2025-04-08T23:54:46Z
author_association: MEMBER
diff_hunk: @@ -58,13 +58,19 @@
   </PropertyGroup>
 
   <ItemGroup>
-    <_TestBlameArguments Include="--blame-hang" />
-    <_TestBlameArguments Include="--blame-hang-dump-type full" />
-    <_TestBlameArguments Include="--blame-hang-timeout 10m" />
-    <_TestBlameArguments Include="--blame-crash" />
-    <_TestBlameArguments Include="--blame-crash-dump-type full" />
-
-    <_TestRunCommandArguments Include="$(_SetExecutableBitCommand)dotnet test -s .runsettings $(_TestNameEnvVar).dll --ResultsDirectory:$(_HelixLogsPath) -v:n" />
+    <_TestBlameArguments Include="--hangdump" />
+    <_TestBlameArguments Include="--hangdump-type Full" />
+    <_TestBlameArguments Include="--hangdump-timeout 10m" />
+    <_TestBlameArguments Include="--crashdump" />
+    <_TestBlameArguments Include="--crashdump-type Full" />
+
+    <!--
+      At time of writing, Aspire.Components.Common.Tests doesn't have runnable tests in CI. In that case, we don't want to fail.
+      Also EndToEnd.Tests doesn't have tests matching scenario=cosmos
path: tests/helix/send-to-helix-inner.proj
user: login: radical
body: Opened https://github.com/dotnet/aspire/issues/8659 to track this work.
created_at: 2025-04-09T00:02:28Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: RussKie
body: > But these tests are never run by themselves. They are run through other test classes that extend this `ConformanceTests` class.

In this case the project should be renamed to remove ".Tests" suffix.
created_at: 2025-04-09T08:55:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,92 @@
+<!-- Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. -->
+<Project>
+
+  <Target Name="RunTests"
+          Outputs="%(TestToRun.ResultsStdOutPath)"
+          Condition="'@(TestToRun)' != ''">
+    <Telemetry EventName="NETCORE_ENGINEERING_TELEMETRY" EventData="Category=Test" />
+
+    <PropertyGroup>
+      <_TestResultDirectory>$([System.IO.Path]::GetDirectoryName('%(TestToRun.ResultsTrxPath)'))</_TestResultDirectory>
+      <_TestResultTrxFileName>$([System.IO.Path]::GetFileName('%(TestToRun.ResultsTrxPath)'))</_TestResultTrxFileName>
+      <_TestResultXmlFileName>$([System.IO.Path]::GetFileName('%(TestToRun.ResultsXmlPath)'))</_TestResultXmlFileName>
+      <_TestResultHtmlFileName>$([System.IO.Path]::GetFileName('%(TestToRun.ResultsHtmlPath)'))</_TestResultHtmlFileName>
+    </PropertyGroup>
+
+    <PropertyGroup>
+      <_TestEnvironment>%(TestToRun.EnvironmentDisplay)</_TestEnvironment>
+      <_TestAssembly>%(TestToRun.Identity)</_TestAssembly>
+      <_TestAssembly Condition="'$(OS)'=='Windows_NT'">$([System.IO.Path]::ChangeExtension($(_TestAssembly), '.exe'))</_TestAssembly>
+      <_TestAssembly Condition="'$(OS)'!='Windows_NT'">$(_TestAssembly.TrimEnd('.dll'))</_TestAssembly>
+      <_TestRuntime>%(TestToRun.TestRuntime)</_TestRuntime>
+      <_TestTimeout>%(TestToRun.TestTimeout)</_TestTimeout>
path: eng/Xunit3/Microsoft.Testing.Platform.targets
user: login: Youssef1313
body: This is Arcade stuff. The TestTimeout metadata is calculated via `TestTimeout` MSBuild property. https://github.com/dotnet/arcade/blob/a7bc103497a422a60c85accfc01e8f91f2c28a7e/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.targets#L64

So yes, you can set the property.
created_at: 2025-04-09T09:46:27Z
author_association: MEMBER
diff_hunk: @@ -37,16 +36,9 @@
       <_DefaultWorkItems TestName="$([System.Text.RegularExpressions.Regex]::Replace(%(FileName), $(TargetFrameworkSuffixRegex), '$1'))" />
       <_DefaultWorkItems TestNameSuffix="$([System.Text.RegularExpressions.Regex]::Replace(%(FileName), $(TargetFrameworkSuffixRegex), '$2'))" />
 
-      <!-- runsettings timeout in ms, default to 15 mins -->
-      <_DefaultWorkItems TimeoutMs="900000" />
-
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Hosting.Elasticsearch.Tests'))" TimeoutMs="3600000" />
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Hosting.Oracle.Tests'))" TimeoutMs="1200000" />
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Pomelo.EntityFrameworkCore.MySql.Tests'))" TimeoutMs="1200000" />
-
       <HelixWorkItem Include="@(_DefaultWorkItems -> '%(FileName)')">
         <PayloadArchive>%(Identity)</PayloadArchive>
-        <PreCommands>$(_EnvVarSetKeyword) &quot;TEST_NAME=%(TestName)&quot; $(_ShellCommandSeparator) $(_EnvVarSetKeyword) TEST_TIMEOUT=%(TimeoutMs) $(_ShellCommandSeparator) $(_EnvVarSetKeyword) &quot;TEST_NAME_SUFFIX=%(TestNameSuffix)&quot; $(_ShellCommandSeparator) $(_EnvVarSetKeyword) &quot;CODE_COV_FILE_SUFFIX=%(TestNameSuffix)&quot;</PreCommands>
path: tests/helix/send-to-helix-basictests.targets
user: login: Youssef1313
body: Yes, in `_TestRunCommandArguments` you can add `--timeout`. Documented [here](https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-intro?tabs=dotnetcli#options). But I personally don't see the need to customize it, everything just works with simplified test infra.
created_at: 2025-04-09T09:50:55Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: Youssef1313
body: Could we do in a follow-up PR?
created_at: 2025-04-09T09:51:30Z
author_association: MEMBER
diff_hunk: @@ -220,18 +220,14 @@ jobs:
           TEST_LOG_PATH: ${{ github.workspace }}/artifacts/log/test-logs
           TestsRunningOutsideOfRepo: true
         run: >
-          dotnet test -s .runsettings -v:n ${{ env.TEST_ASSEMBLY_NAME }}.dll
-          -l "console;verbosity=normal"
-          -l "trx;LogFilePrefix=${{ inputs.testShortName }}"
-          -l "GitHubActions;summary.includePassedTests=true;summary.includeSkippedTests=true"
-          --blame
-          --blame-hang-timeout ${{ inputs.testHangTimeout }}
-          --blame-crash
+          dotnet exec ${{ env.TEST_ASSEMBLY_NAME }}.dll
path: .github/workflows/run-tests.yml
user: login: Youssef1313
body: I tried to move over to running executable directly, but it's not in the zip file when using `ZipTestArchive`. I have no idea why. Maybe we are zipping too early or something. I'm doing dotnet test.dll for now. We can follow-up with investigation to include the exe in the zip.
created_at: 2025-04-09T13:32:45Z
author_association: MEMBER
diff_hunk: @@ -37,16 +36,9 @@
       <_DefaultWorkItems TestName="$([System.Text.RegularExpressions.Regex]::Replace(%(FileName), $(TargetFrameworkSuffixRegex), '$1'))" />
       <_DefaultWorkItems TestNameSuffix="$([System.Text.RegularExpressions.Regex]::Replace(%(FileName), $(TargetFrameworkSuffixRegex), '$2'))" />
 
-      <!-- runsettings timeout in ms, default to 15 mins -->
-      <_DefaultWorkItems TimeoutMs="900000" />
-
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Hosting.Elasticsearch.Tests'))" TimeoutMs="3600000" />
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Hosting.Oracle.Tests'))" TimeoutMs="1200000" />
-      <_DefaultWorkItems Condition="$([System.String]::new('%(FileName)').StartsWith('Aspire.Pomelo.EntityFrameworkCore.MySql.Tests'))" TimeoutMs="1200000" />
-
       <HelixWorkItem Include="@(_DefaultWorkItems -> '%(FileName)')">
         <PayloadArchive>%(Identity)</PayloadArchive>
-        <PreCommands>$(_EnvVarSetKeyword) &quot;TEST_NAME=%(TestName)&quot; $(_ShellCommandSeparator) $(_EnvVarSetKeyword) TEST_TIMEOUT=%(TimeoutMs) $(_ShellCommandSeparator) $(_EnvVarSetKeyword) &quot;TEST_NAME_SUFFIX=%(TestNameSuffix)&quot; $(_ShellCommandSeparator) $(_EnvVarSetKeyword) &quot;CODE_COV_FILE_SUFFIX=%(TestNameSuffix)&quot;</PreCommands>
path: tests/helix/send-to-helix-basictests.targets
user: login: radical
body: We can set it in the individual project files. 👍 
created_at: 2025-04-09T14:47:02Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: RussKie
body: @Youssef1313 this doesn't appear to be used in Microsoft.Testing.Platform.targets, which results in the tests to fail
![image](https://github.com/user-attachments/assets/40dfba01-b7da-4eeb-8d2e-879316b11370)

Should this be tucked on to `_TestRunnerArgs`?
https://github.com/dotnet/aspire/blob/fd413eb66b219013ad4b8bc64a459c84a82fa446/eng/Xunit3/Microsoft.Testing.Platform.targets#L23-L31
created_at: 2025-04-10T03:37:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: Youssef1313
body: @RussKie `TestingPlatformCommandLineArguments` is currently used when invoking `dotnet test` but not when invoked via Arcade.

created_at: 2025-04-10T04:52:56Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: RussKie
body: What would be the right way to fix this?
created_at: 2025-04-10T04:54:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: Youssef1313
body: I think you can simply append `TestingPlatformCommandLineArguments` to `_TestRunnerArgs`.
created_at: 2025-04-10T05:07:06Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: Youssef1313
body: Hmm, what I'm suggesting won't probably be possible without my PR on Arcade side. For now I think you could duplicate the `--ignore-exit-code 8` on both sides.
created_at: 2025-04-10T05:07:51Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: Youssef1313
body: Or simply just rename Aspire.Components.Common.Tests to end with TestUtilities if it's really not intended to be a test project, then you don't need the --ignore-exit-code 8 at all.
created_at: 2025-04-10T05:08:36Z
author_association: MEMBER
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: RussKie
body: > Or simply just rename Aspire.Components.Common.Tests to end with TestUtilities if it's really not intended to be a test project, then you don't need the --ignore-exit-code 8 at all.

Yep, working on it
created_at: 2025-04-10T05:40:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: RussKie
body: I don't think we can do it easily (if at all).
The project uses XUnit API (like fact and theory), and we need to import the Xunit assemblies. Those are defined in eng/Xunit3/Xunit3.targets and can only be imported by test projects. If the targets imported explicitly, then the build gets failed by `_XunitValidateBuild` declared in xunit.v3.core.targets
```
D:\Development\dotnet-aspire\.packages\xunit.v3.core\2.0.0\buildTransitive\xunit.v3.core.targets(15,5): xUnit.net v3 test projects must be executable (set project property '<OutputType>Exe</OutputType>'). If this is not a test project, reference xunit.v3.extensibilty.core instead. [D:\Development\dotnet-aspire\tests\Aspire.Components.Common.Tests\Aspire.Components.Common.TestUtilities.csproj]
```
created_at: 2025-04-10T06:32:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2,6 +2,10 @@
 
   <PropertyGroup>
     <TargetFramework>$(DefaultTargetFramework)</TargetFramework>
+    <!-- https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes -->
+    <!-- Exit code 8 is "zero tests ran" -->
+    <!-- Currently, none of the tests in this project run in CI. All are ignored -->
+    <TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --ignore-exit-code 8</TestingPlatformCommandLineArguments>
path: tests/Aspire.Components.Common.Tests/Aspire.Components.Common.Tests.csproj
user: login: Youssef1313
body: @RussKie You need to reference only `xunit.v3.extensibility.core` in that project, and not `xunit.v3.core`, nor `xunit.v3`
created_at: 2025-04-10T06:35:53Z
author_association: MEMBER

PR comments:
user: login: Youssef1313
created_at: 2025-04-04T14:29:37Z
body: @radical I have a single failure here which is specific to Linux helix run and is specific to `Aspire.Playground.Tests`

```
/datadisks/disk1/work/AEFB09D7/w/9BFE08AE/e/.nuget/xunit.v3.core/2.0.0/buildTransitive/xunit.v3.core.targets(15,5): error : xUnit.net v3 test projects must be executable (set project property '<OutputType>Exe</OutputType>'). If this is not a test project, reference xunit.v3.extensibilty.core instead. [/datadisks/disk1/work/AEFB09D7/w/9BFE08AE/e/tests/Aspire.Playground.Tests/Aspire.Playground.Tests.csproj]
```

Do you have a guess why Aspire.Playground.Tests could be special here? Output type is already set here:

https://github.com/dotnet/aspire/blob/e5648453d89e09ab7a226115b48aaab5ecf48920/tests/Directory.Build.props#L20

It also seems we are not grabbing binlogs from helix, which makes it harder to investigate this.
author_association: MEMBER
user: login: danmoseley
created_at: 2025-04-04T15:54:07Z
body: seems like we _ought_ to grab binlogs from helix..
author_association: MEMBER
user: login: Youssef1313
created_at: 2025-04-04T16:29:12Z
body: @danmoseley I took a quick look and it seems like `DownloadFilesFromResults` metadata on `HelixWorkItem` items is what determines that a file should be downloaded from the remote machine? It looks like currently [only TRX files are specified](https://github.com/dotnet/aspire/blob/c09db91f625e1b51fda027b8864a892e406d8017/tests/helix/send-to-helix-buildonhelixtests.targets#L66) (or it's very possible too that I'm misunderstanding something). I appreciate if you can get that fixed, and will help understanding what's currently going wrong with the changes in this PR.
author_association: MEMBER
user: login: danmoseley
created_at: 2025-04-04T16:54:26Z
body: @radical will have context on that line.
https://github.com/dotnet/aspire/blob/9b1c4a1537ad953330a8fd069d9b389bf3728cee/tests/helix/send-to-helix-buildonhelixtests.targets#L66

It shouldn't stop us merging this, right?

We keep getting unactionable crashes in the test runner process, and it sounds like MTP may help there so excited to get this merged...
author_association: MEMBER
user: login: Youssef1313
created_at: 2025-04-04T17:14:58Z
body: > It shouldn't stop us merging this, right?

The PR is currently failing and I don't have any "guess" so need a binlog.

> We keep getting unactionable crashes in the test runner process, and it sounds like MTP may help there so excited to get this merged...

Can you link me to a run that has a crash?
author_association: MEMBER
user: login: radical
created_at: 2025-04-04T18:36:36Z
body: re:trx-binlogs, the binlogs and other logs are collected from helix and available for download from there. The trx is downloaded like this so it can be included in the Azdo artifacts.
author_association: MEMBER
user: login: radical
created_at: 2025-04-04T18:41:43Z
body: Add `<OutputType>Exe</OutputType>` explicitly to `Aspire.Playground.Tests.csproj` because the project is built on helix without `tests/Directory.Build.props`.
author_association: MEMBER
user: login: danmoseley
created_at: 2025-04-04T19:04:33Z
body: > Can you link me to a run that has a crash?

https://dev.azure.com/dnceng-public/public/_build/results?buildId=1005544&view=results
https://dev.azure.com/dnceng-public/public/_build/results?buildId=1004803&view=results


author_association: MEMBER
user: login: Youssef1313
created_at: 2025-04-04T19:05:14Z
body: > Add `<OutputType>Exe</OutputType>` explicitly to `Aspire.Playground.Tests.csproj` because the project is built on helix without `tests/Directory.Build.props`.

@radical But current main is already xunit.v3 which requires OutputType Exe even for VSTest. I also fail to see why this specific project is special and why the build error is Linux only.

---

> re:trx-binlogs, the binlogs and other logs are collected from helix and available for download from there. The trx is downloaded like this so it can be included in the Azdo artifacts.

Do you mean the binlogs are available to download somewhere else other than AzDO artifacts? I'm not sure where to access them so far
author_association: MEMBER
user: login: radical
created_at: 2025-04-04T19:37:04Z
body: > > Add `<OutputType>Exe</OutputType>` explicitly to `Aspire.Playground.Tests.csproj` because the project is built on helix without `tests/Directory.Build.props`.
> 
> @radical But current main is already xunit.v3 which requires OutputType Exe even for VSTest. I also fail to see why this specific project is special and why the build error is Linux only.

Oh, you are correct about already being on xunit.v3 . This project is run only for Linux because we don't have docker on windows on CI.

> > re:trx-binlogs, the binlogs and other logs are collected from helix and available for download from there. The trx is downloaded like this so it can be included in the Azdo artifacts.
> 
> Do you mean the binlogs are available to download somewhere else other than AzDO artifacts? I'm not sure where to access them so far

Yeah. I usually follow the `Details` link from output in https://dev.azure.com/dnceng-public/public/_build/results?buildId=1005447&view=logs&j=acc6f692-388e-5b31-5997-e154da29f5b3&t=59fafa2a-4be0-5f74-c845-0779cc54f4a3 . And from that json follow `ListUrl` -> `DetailsUrl` and that shows all the collected helix artifacts. I can share the binlog with you directly.
author_association: MEMBER
user: login: radical
created_at: 2025-04-04T19:54:19Z
body: `.trx` files are missing for the runs in https://github.com/dotnet/aspire/blob/f15a1c158bdc98c8737e8404ac2abe3575ac7090/.github/workflows/run-tests.yml#L246-L258
author_association: MEMBER
user: login: Youssef1313
created_at: 2025-04-05T06:29:03Z
body: @radical The missing TRXes should be fixed now hopefully.
author_association: MEMBER
user: login: RussKie
created_at: 2025-04-07T00:08:31Z
body: @Youssef1313, am I correct in understanding that we can't take this change until VS 17.14 is publicly available?

https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-vs-vstest#visual-studio

> _Visual Studio Test Explorer supports the new test platform starting with version 17.14. If you are using an earlier version, you might need to update your Visual Studio to the latest version._
author_association: CONTRIBUTOR
user: login: Youssef1313
created_at: 2025-04-07T06:16:44Z
body: The documentation seems misleading. Will let @drognanar give more history to fix the documentation.
author_association: MEMBER
user: login: Youssef1313
created_at: 2025-04-07T09:34:24Z
body: @radical Could you please help with validating test counts etc here?
author_association: MEMBER
user: login: radical
created_at: 2025-04-09T14:49:36Z
body: The Playground tests shouldn't be running on Windows at all. This seems to have regressed in #8518 . cc @RussKie 
author_association: MEMBER
user: login: danmoseley
created_at: 2025-04-09T15:03:45Z
body: How close is this now? 

[Here's](https://[helixr1107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-aspire-refs-heads-release-92-be88d12d588d4064a5/Aspire.Hosting.Nats.Tests/1/console.feb7ea04.log?skoid=8eda00af-b5ec-4be9-b69b-0919a2338892&sktid=72f988bf-86f1-41af-91ab-2d7cd011db47&skt=2025-04-09T14%3A54%3A21Z&ske=2025-04-09T15%3A54%3A21Z&sks=b&skv=2024-11-04&sv=2024-11-04&st=2025-04-09T14%3A54%3A21Z&se=2025-04-09T15%3A54%3A21Z&sr=c&sp=r&sig=%2F5Kvbv7ahfePR6F7m85ZS6XG%2F2hAwE%2BBslATmJUPxAA%3D](https://helixr1107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-aspire-refs-heads-release-92-be88d12d588d4064a5/Aspire.Hosting.Nats.Tests/1/console.feb7ea04.log?skoid=8eda00af-b5ec-4be9-b69b-0919a2338892&sktid=72f988bf-86f1-41af-91ab-2d7cd011db47&skt=2025-04-09T14%3A54%3A21Z&ske=2025-04-09T15%3A54%3A21Z&sks=b&skv=2024-11-04&sv=2024-11-04&st=2025-04-09T14%3A54%3A21Z&se=2025-04-09T15%3A54%3A21Z&sr=c&sp=r&sig=%2F5Kvbv7ahfePR6F7m85ZS6XG%2F2hAwE%2BBslATmJUPxAA%3D)) yet another log where the only stack is the pipe disconnecting, and I'm hoping this will help diagnose such cases
author_association: MEMBER
user: login: radical
created_at: 2025-04-09T15:27:20Z
body: > How close is this now?
> 
> [Here's](https://%5Bhelixr1107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-aspire-refs-heads-release-92-be88d12d588d4064a5/Aspire.Hosting.Nats.Tests/1/console.feb7ea04.log?skoid=8eda00af-b5ec-4be9-b69b-0919a2338892&sktid=72f988bf-86f1-41af-91ab-2d7cd011db47&skt=2025-04-09T14%3A54%3A21Z&ske=2025-04-09T15%3A54%3A21Z&sks=b&skv=2024-11-04&sv=2024-11-04&st=2025-04-09T14%3A54%3A21Z&se=2025-04-09T15%3A54%3A21Z&sr=c&sp=r&sig=%2F5Kvbv7ahfePR6F7m85ZS6XG%2F2hAwE%2BBslATmJUPxAA%3D%5D(https://helixr1107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-aspire-refs-heads-release-92-be88d12d588d4064a5/Aspire.Hosting.Nats.Tests/1/console.feb7ea04.log?skoid=8eda00af-b5ec-4be9-b69b-0919a2338892&sktid=72f988bf-86f1-41af-91ab-2d7cd011db47&skt=2025-04-09T14%3A54%3A21Z&ske=2025-04-09T15%3A54%3A21Z&sks=b&skv=2024-11-04&sv=2024-11-04&st=2025-04-09T14%3A54%3A21Z&se=2025-04-09T15%3A54%3A21Z&sr=c&sp=r&sig=%2F5Kvbv7ahfePR6F7m85ZS6XG%2F2hAwE%2BBslATmJUPxAA%3D)) yet another log where the only stack is the pipe disconnecting, and I'm hoping this will help diagnose such cases

I think we could merge this today.
author_association: MEMBER
user: login: Youssef1313
created_at: 2025-04-09T15:27:53Z
body: @danmoseley I think it's close. Hopefully if the current build goes green, it would be good if you can do validation regarding number of tests etc to make sure everything is good.
author_association: MEMBER
user: login: RussKie
created_at: 2025-04-09T23:44:58Z
body: @Youssef1313 with this change the command line is logged as the last entry in the log files where previously it was at the start of the file

* old
![image](https://github.com/user-attachments/assets/7a4ce404-74d6-41bc-884b-e5f38faa30eb)

* new
![image](https://github.com/user-attachments/assets/7528403a-bf64-45fa-ac43-fdd610b51a56)

author_association: CONTRIBUTOR
user: login: Youssef1313
created_at: 2025-04-10T02:44:56Z
body: @RussKie Why is the placement of this mattering? Are you parsing the log file programmatically somewhere?

The discrepancy mostly comes from Arcade. With VSTest, the command line comes before the test execution:

https://github.com/dotnet/arcade/blob/03b4b258fc72e5663f2ad701df5d70129aa2df8a/src/Microsoft.DotNet.Arcade.Sdk/tools/VSTest.targets#L44-L58

For xunit.v2 runner, it comes after:

https://github.com/dotnet/arcade/blob/03b4b258fc72e5663f2ad701df5d70129aa2df8a/src/Microsoft.DotNet.Arcade.Sdk/tools/XUnit/XUnit.Runner.targets#L69-L84

For xunit.v3:

- The Arcade logic is duplicated in this repo currently, which you adjusted in #8685
- When you move to .NET 10 and Arcade 10, hopefully https://github.com/dotnet/arcade/pull/15671 would have been merged, and we should aim to drop the custom logic in the repo and use it from Arcade directly. But at this point, I'm not sure for my PR https://github.com/dotnet/arcade/pull/15671 where I should add the command line, at first or at end. FYI @ViktorHofer 
author_association: MEMBER
user: login: RussKie
created_at: 2025-04-10T02:49:04Z
body: Some logs can be very log (hundreds and thousands of lines long), and having the command line at the end hinders the process.
author_association: CONTRIBUTOR
user: login: RussKie
created_at: 2025-04-10T02:50:04Z
body: I didn't realise the VSTest and XUnit had different implementations 😲
author_association: CONTRIBUTOR
user: login: captainsafia
created_at: 2025-04-11T03:55:31Z
body: Have we validated that these changes work with the Test Explorer in VS Code?

I've ran into a few issues debugging in the repo (see https://github.com/microsoft/vscode-dotnettools/issues/1923 and https://github.com/microsoft/vscode-dotnettools/issues/1922) since this change went in. Test discovery and execution appear to behave much more reliably in VS Code if I revert the commit associated with this PR.

Does anyone have any insights on the state of Microsoft.Testing.Platform support on VS Code?
author_association: CONTRIBUTOR
user: login: Youssef1313
created_at: 2025-04-11T03:56:45Z
body: @captainsafia It's expected to work in VS Code if you are using DevKit
author_association: MEMBER
user: login: Youssef1313
created_at: 2025-04-11T03:59:39Z
body: @captainsafia I don't know if the option is enabled by default or not, but you need to make sure this is turned on:

![image](https://github.com/user-attachments/assets/2ccb7283-899d-4db3-ae62-8224d96fe426)

(we did a change to no longer mark it "experimental" but not sure if the change has already shipped or not)
author_association: MEMBER
user: login: captainsafia
created_at: 2025-04-11T04:14:19Z
body: @Youssef1313 Yep, I'm on a release of the extension without the experimental flag and have the feature enabled. I'm running into issues with debugging tests and lack of reliability with the test discovery.

<img width="1096" alt="image" src="https://github.com/user-attachments/assets/a73b10bd-f05e-4c95-bcbc-f0d983d84677" />

Is there an SDK requirement associated with this change?

author_association: CONTRIBUTOR
user: login: Youssef1313
created_at: 2025-04-11T04:16:53Z
body: There shouldn't be SDK requirement associated. From your logs:

> No test is available in /Users/captainsafia/git/aspire/acr-resource/artifacts/bin/Aspire.Oracle.EntityFrameworkCore.Tests/Debug/net8.0/Aspire.Oracle.EntityFrameworkCore.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

It seems like VS Code is treating the project as VSTest and not MTP. @drognanar would be the right contact.
author_association: MEMBER
user: login: mitchdenny
created_at: 2025-04-15T01:18:27Z
body: Looks like this could have broken me as well (I'm _trying_ to use Codespaces as my primary environment).
author_association: MEMBER
user: login: Youssef1313
created_at: 2025-04-15T04:01:52Z
body: @captainsafia @mitchdenny As a potential temporary workaround, could you try adding a PackageReference to `xunit.runner.visualstudio` (version 3.0.2)?

This would go in:

- https://github.com/dotnet/aspire/blob/3fc603bdd765bea1c0d9ebb1f7ba5351f4031903/tests/Shared/RepoTesting/Aspire.RepoTesting.targets#L65
- https://github.com/dotnet/aspire/blob/3fc603bdd765bea1c0d9ebb1f7ba5351f4031903/eng/Xunit3/Xunit3.targets#L2

and version would be defined in:

https://github.com/dotnet/aspire/blob/3fc603bdd765bea1c0d9ebb1f7ba5351f4031903/eng/Versions.props#L2
author_association: MEMBER
user: login: mitchdenny
created_at: 2025-04-15T04:08:29Z
body: Happy to give it a shot. Can you explain why this might fix it?
author_association: MEMBER
user: login: Youssef1313
created_at: 2025-04-15T04:17:06Z
body: `xunit.runner.visualstudio` is what supports VSTest. So you'll be using VSTest in your case. But command-line runs will remain using MTP, as well as any client that correctly supports and detects the projects as MTP.
author_association: MEMBER
user: login: captainsafia
created_at: 2025-04-15T04:46:04Z
body: @Youssef1313 I tried this and unfortunately I still run into issues debugging tests in the repo. :/
author_association: CONTRIBUTOR
user: login: danmoseley
created_at: 2025-04-15T21:16:17Z
body: @Youssef1313 can you repro this with VS Code?

If we can't fix this we'll presumably have to revert your change until we figure it out.
author_association: MEMBER
user: login: captainsafia
created_at: 2025-04-15T22:32:06Z
body: @danmoseley I think we should revert this. https://github.com/microsoft/vscode-dotnettools/issues/1922 has been validated as a bug on the VS Code side and is under investigation.
author_association: CONTRIBUTOR
user: login: Youssef1313
created_at: 2025-04-16T07:34:58Z
body: @captainsafia I'm not able to reproduce. Added a comment in https://github.com/dotnet/aspire/pull/8802#issuecomment-2808668894
author_association: MEMBER
user: login: Youssef1313
created_at: 2025-04-16T12:28:14Z
body: @captainsafia Meanwhile, while we are trying to figure it out, can you try again the xunit.runner.visualstudio workaround, but also add `Microsoft.NET.Test.Sdk`? Both packages are needed for VSTest. Sorry I forgot to mention `Microsoft.NET.Test.Sdk` in my last comment.
author_association: MEMBER