Repository: microsoft/terminal (language: cpp)

Issue:
Re-running just the tests will fail in CI, even if the tests pass
Probably regressed in #15808 

Case in point:

PR: https://github.com/microsoft/terminal/pull/15867/checks?check_run_id=16176312961.
AZP: https://dev.azure.com/ms/terminal/_build/results?buildId=491204&view=logs&jobId=715ae6b5-7bc1-5485-dd41-2ac3c0010e51&j=715ae6b5-7bc1-5485-dd41-2ac3c0010e51&t=efe7f887-b87a-5492-dbfa-782df7fade32.

Log:

```
Starting: PublishPipelineArtifact
==============================================================================
Task         : Publish Pipeline Artifacts
Description  : Publish (upload) a file or directory as a named artifact for the current run
Version      : 1.199.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/publish-pipeline-artifact
==============================================================================
Artifact name input: test-logs-x64-Release
Uploading pipeline artifact from D:\a\_work\1\a\Release\x64\test-logs for build #491204
Using default max parallelism.
Max dedup parallelism: 192
ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session 20065d8c-75d0-4839-8a6f-1fb162ba1281
Hashtype: Dedup64K
DedupManifestArtifactClient will correlate http requests with X-TFS-Session 20065d8c-75d0-4839-8a6f-1fb162ba1281
1 files processed.
Processed 1 files from D:\a\_work\1\a\Release\x64\test-logs successfully.
Uploaded 1,703,262 out of 1,703,262 bytes
Content upload is done!

Content upload statistics:
Total Content: 3.4 MB
Physical Content Uploaded: 0.1 MB
Logical Content Uploaded: 1.7 MB
Compression Saved: 1.6 MB
Deduplication Saved: 1.7 MB
Number of Chunks Uploaded: 17
Total Number of Chunks: 49

ApplicationInsightsTelemetrySender correlated 2 events with X-TFS-Session 20065d8c-75d0-4839-8a6f-1fb162ba1281
##[error]Artifact test-logs-x64-Release already exists for build 491204.
Finishing: PublishPipelineArtifact
```

The first run, tests flaked out. On rerun, they passed. But the CI reported it as a failure, because `Artifact test-logs-x64-Release already exists for build 491204`




Discussion / hints:
Oh, interesting. This is an unexpected limitation in Azure DevOps! I wonder if there's a way to disambiguate the pipeline artifact...

Fix patch:
diff --git a/build/pipelines/ci.yml b/build/pipelines/ci.yml
index 3a142688443..4ad72566a15 100644
--- a/build/pipelines/ci.yml
+++ b/build/pipelines/ci.yml
@@ -97,6 +97,8 @@ stages:
           - template: ./templates-v2/job-test-project.yml
             parameters:
               platform: ${{ platform }}
+              # The tests might be run more than once; log one artifact per attempt.
+              outputArtifactStem: -$(System.JobAttempt)
 
   - ${{ if ne(variables['Build.Reason'], 'PullRequest') }}:
     - stage: CodeIndexer


Test patch:
diff --git a/build/pipelines/templates-v2/job-test-project.yml b/build/pipelines/templates-v2/job-test-project.yml
index 78f5fa4db6a..1cd8e2bef67 100644
--- a/build/pipelines/templates-v2/job-test-project.yml
+++ b/build/pipelines/templates-v2/job-test-project.yml
@@ -2,7 +2,8 @@ parameters:
   configuration: 'Release'
   platform: ''
   testLogPath: '$(Build.BinariesDirectory)\$(BuildPlatform)\$(BuildConfiguration)\testsOnBuildMachine.wtl'
-  artifactStem: ''
+  inputArtifactStem: ''
+  outputArtifactStem: ''
 
 jobs:
 - job: Test${{ parameters.platform }}${{ parameters.configuration }}
@@ -37,7 +38,7 @@ jobs:
   - task: DownloadPipelineArtifact@2
     displayName: Download artifacts
     inputs:
-      artifactName: build-${{ parameters.platform }}-$(BuildConfiguration)${{ parameters.artifactStem }}
+      artifactName: build-${{ parameters.platform }}-$(BuildConfiguration)${{ parameters.inputArtifactStem }}
       downloadPath: $(Terminal.BinDir)
 
   - task: PowerShell@2
@@ -94,5 +95,5 @@ jobs:
       flattenFolders: true
 
   - publish: '$(Build.ArtifactStagingDirectory)/$(BuildConfiguration)/$(BuildPlatform)/test-logs'
-    artifact: test-logs-$(BuildPlatform)-$(BuildConfiguration)${{ parameters.artifactStem }}
+    artifact: test-logs-$(BuildPlatform)-$(BuildConfiguration)${{ parameters.outputArtifactStem }}
     condition: always()