Repository: dotnet/performance
Pull request #4770: Setup new dotnet new android measurements using Mono or CoreCLR runtimes
Author: login: matouskozak
Labels: name: enhancement
description: New feature or request
name: Scenario
name: python
description: Pull requests that update Python code
Created: 2025-03-13T18:18:45Z | Merged: 2025-04-13T03:17:34Z

Switching MAUI-Android (and .NET Android) scenarios to use latest .NET SDK + dotnet10 feed, the steps are:
1. Install the latest .NET SDK: `.dotnet-install.sh --channel 10.0 --quality daily`
2. Using the latest feed ([dotnet10](https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet10/nuget/v3/index.json)), find the latest packs using `dotnet package search`.
3. Put the latest packs to a rollback file and install maui workload.

Adding new jobs:
- `dotnet new android` running CoreCLR.
- `dotnet new maui` (Android) running CoreCLR.
- `dotnet new maui-blazor` (Android) running CoreCLR.

Internal pipeline run: https://dev.azure.com/dnceng/internal/_build/results?buildId=2684265

TODO:
- [x] Remove added android feeds from `Nuget.config` (should be added by maestro)

Out of scope
- Fix flaky memory consumption measurements for .NET Android and .NET MAUI Android https://github.com/dotnet/performance/issues/4826

---

https://github.com/dotnet/performance/issues/4754





Diff:
diff --git a/eng/performance/maui_scenarios_android.proj b/eng/performance/maui_scenarios_android.proj
index 27a9ad0951d..d01ef11a031 100644
--- a/eng/performance/maui_scenarios_android.proj
+++ b/eng/performance/maui_scenarios_android.proj
@@ -11,6 +11,10 @@
     <PreparePayloadOutDirectoryName>scenarios_out</PreparePayloadOutDirectoryName>
     <PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' == 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)\</PreparePayloadWorkItemBaseDirectory>
     <PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' != 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)/</PreparePayloadWorkItemBaseDirectory>
+
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'mono'">/p:UseMonoRuntime=true</_MSBuildArgs>
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">/p:UseMonoRuntime=false</_MSBuildArgs>
+    <RunConfigsString>$(RuntimeFlavor)_$(CodegenType)</RunConfigsString>
   </PropertyGroup>
 
   <Target Name="RemoveDotnetFromCorrelationStaging" BeforeTargets="BeforeTest">
@@ -25,19 +29,19 @@
   </ItemDefinitionGroup>
 
   <ItemGroup>
-    <MAUIAndroidScenario Include="Maui Android Default">
-      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
-      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
-      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
-      <PackageName>com.companyname.mauiandroiddefault</PackageName>
-    </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include=".NET Android Default">
+    <MAUIAndroidScenario Include=".NET Android Default Template">
       <ScenarioDirectoryName>netandroid</ScenarioDirectoryName>
       <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
       <ApkName>com.companyname.netandroiddefault-Signed</ApkName>
       <PackageName>com.companyname.NetAndroidDefault</PackageName>
     </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include="Maui Blazor Android Default">
+    <MAUIAndroidScenario Include="MAUI Android Default Template">
+      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
+      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
+      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
+      <PackageName>com.companyname.mauiandroiddefault</PackageName>
+    </MAUIAndroidScenario>
+    <MAUIAndroidScenario Include="MAUI Blazor Android Default Template (TTFD)">
       <ScenarioDirectoryName>mauiblazorandroid</ScenarioDirectoryName>
       <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
       <ApkName>com.companyname.mauiblazorandroiddefault-Signed</ApkName>
@@ -49,7 +53,7 @@
 
   <ItemGroup>
     <PreparePayloadWorkItem Include="@(MAUIAndroidScenario)">
-      <Command>$(Python) pre.py publish -f $(PERFLAB_Framework)-android -r android-arm64 --self-contained -o $(PreparePayloadWorkItemBaseDirectory)%(PreparePayloadWorkItem.ScenarioDirectoryName)</Command>
+      <Command>$(Python) pre.py publish -f $(PERFLAB_Framework)-android -r android-arm64 --self-contained --msbuild=&quot;$(_MSBuildArgs)&quot; --binlog %(PreparePayloadWorkItem.ScenarioDirectoryName).$(RunConfigsString).binlog  -o $(PreparePayloadWorkItemBaseDirectory)%(PreparePayloadWorkItem.ScenarioDirectoryName)</Command>
       <WorkingDirectory>%(PreparePayloadWorkItem.PayloadDirectory)</WorkingDirectory>
     </PreparePayloadWorkItem>
   </ItemGroup>
diff --git a/eng/pipelines/sdk-perf-jobs.yml b/eng/pipelines/sdk-perf-jobs.yml
index 6fde591456e..9c9b7b9b42d 100644
--- a/eng/pipelines/sdk-perf-jobs.yml
+++ b/eng/pipelines/sdk-perf-jobs.yml
@@ -388,7 +388,7 @@ jobs:
         ${{ each parameter in parameters.jobParameters }}:
           ${{ parameter.key }}: ${{ parameter.value }}
 
-  # Maui Android scenario benchmarks
+  # Maui Android scenario benchmarks (Mono)
   - template: /eng/pipelines/templates/build-machine-matrix.yml
     parameters:
       jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml
@@ -399,10 +399,30 @@ jobs:
       jobParameters:
         runKind: maui_scenarios_android
         projectFileName: maui_scenarios_android.proj
-        dotnetVersionsLinks:
-          9.0: ./eng/Version.Details.xml
         channels:
-          - 8.0
+          - main
+        runtimeFlavor: mono
+        codeGenType: ProfiledAOT
+        additionalJobIdentifier: Mono
+        ${{ each parameter in parameters.jobParameters }}:
+          ${{ parameter.key }}: ${{ parameter.value }}
+
+  # Maui Android scenario benchmarks (CoreCLR)
+  - template: /eng/pipelines/templates/build-machine-matrix.yml
+    parameters:
+      jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml
+      buildMachines:
+        - win-x64-android-arm64-pixel
+        - win-x64-android-arm64-galaxy
+      isPublic: false
+      jobParameters:
+        runKind: maui_scenarios_android
+        projectFileName: maui_scenarios_android.proj
+        channels:
+          - main
+        runtimeFlavor: coreclr
+        codeGenType: JIT
+        additionalJobIdentifier: CoreCLR
         ${{ each parameter in parameters.jobParameters }}:
           ${{ parameter.key }}: ${{ parameter.value }}
 
diff --git a/eng/pipelines/templates/run-performance-job.yml b/eng/pipelines/templates/run-performance-job.yml
index b9266adcf21..7d716fa1442 100644
--- a/eng/pipelines/templates/run-performance-job.yml
+++ b/eng/pipelines/templates/run-performance-job.yml
@@ -186,7 +186,7 @@ jobs:
               - '--is-scenario'
             - ${{ if ne(length(parameters.runEnvVars), 0) }}:
               - "--run-env-vars ${{ join(' ', parameters.runEnvVars)}}"
-            - ${{ if and(eq(parameters.runKind, 'maui_scenarios_ios'), ne(parameters.runtimeFlavor, '')) }}:
+            - ${{ if and(in(parameters.runKind, 'maui_scenarios_ios', 'maui_scenarios_android'), ne(parameters.runtimeFlavor, '')) }}:
               - '--runtime-flavor ${{ parameters.runtimeFlavor }}'
             - ${{ if eq(parameters.hybridGlobalization, true) }}:
               - '--hybrid-globalization'
diff --git a/scripts/run_performance_job.py b/scripts/run_performance_job.py
index 77024df9b65..a380e5c1d0c 100644
--- a/scripts/run_performance_job.py
+++ b/scripts/run_performance_job.py
@@ -529,6 +529,7 @@ def run_performance_job(args: RunPerformanceJobArgs):
 
     runtime_type = ""
 
+    # dotnet/runtime Android sample app scenarios
     if args.run_kind == "android_scenarios":
         # Mapping runtime_type to runtime_flavor before sending to helix
         if args.runtime_type == "AndroidMono":
@@ -540,6 +541,13 @@ def run_performance_job(args: RunPerformanceJobArgs):
         configurations["CodegenType"] = str(args.codegen_type)
         configurations["RuntimeType"] = str(args.runtime_flavor)
 
+    # .NET Android and .NET MAUI Android sample app scenarios
+    if args.run_kind == "maui_scenarios_android":
+        if not args.runtime_flavor in ("mono", "coreclr"):
+            raise Exception("Runtime flavor must be specified for maui_scenarios_android")
+        configurations["CodegenType"] = str(args.codegen_type)
+        configurations["RuntimeType"] = str(args.runtime_flavor)
+
     if ios_mono:
         runtime_type = "Mono"
         configurations["iOSLlvmBuild"] = str(args.ios_llvm_build)
@@ -901,6 +909,7 @@ def run_performance_job(args: RunPerformanceJobArgs):
             os.environ["HelixTargetQueues"] = args.queue
             os.environ["Python"] = agent_python
             os.environ["RuntimeFlavor"] = args.runtime_flavor or ''
+            os.environ["CodegenType"] = args.codegen_type or ''
             os.environ["HybridGlobalization"] = str(args.hybrid_globalization)
 
             # TODO: See if these commands are needed for linux as they were being called before but were failing.
@@ -1089,7 +1098,7 @@ def run_performance_job(args: RunPerformanceJobArgs):
         helix_build=args.build_number,
         partition_count=args.partition_count,
         runtime_flavor=args.runtime_flavor or "",
-        codegen_type=args.codegen_type,
+        codegen_type=args.codegen_type or "",
         hybrid_globalization=args.hybrid_globalization,
         target_csproj=args.target_csproj,
         work_item_command=work_item_command or None,
diff --git a/src/scenarios/mauiandroid/pre.py b/src/scenarios/mauiandroid/pre.py
index f251328b1c8..8b438a7f99b 100644
--- a/src/scenarios/mauiandroid/pre.py
+++ b/src/scenarios/mauiandroid/pre.py
@@ -5,15 +5,19 @@
 import sys
 from performance.logger import setup_loggers, getLogger
 from shared import const
-from shared.mauisharedpython import remove_aab_files, install_versioned_maui
+from shared.mauisharedpython import remove_aab_files, install_latest_maui
 from shared.precommands import PreCommands
-from shared.versionmanager import versions_write_json, get_version_from_dll_powershell
+from shared.versionmanager import versions_write_json, get_sdk_versions
 from test import EXENAME
 
 setup_loggers(True)
+logger = getLogger(__name__)
+logger.info("Starting pre-command for MAUI Android template app (dotnet new maui)")
 
 precommands = PreCommands()
-install_versioned_maui(precommands)
+
+install_latest_maui(precommands)
+precommands.print_dotnet_info()
 
 # Setup the Maui folder
 precommands.new(template='maui',
@@ -24,7 +28,6 @@
                 no_restore=False)
 
 # Build the APK
-shutil.copy('./MauiNuGet.config', './app/Nuget.config')
 precommands.execute([])
 
 # Remove the aab files as we don't need them, this saves space
@@ -33,8 +36,7 @@
     output_dir = precommands.output
 remove_aab_files(output_dir)
 
-# Copy the MauiVersion to a file so we have it on the machine
-maui_version = get_version_from_dll_powershell(rf".\{const.APPDIR}\obj\Release\{precommands.framework}\android-arm64\linked\Microsoft.Maui.dll")
-version_dict = { "mauiVersion": maui_version }
+# Extract the versions of used SDKs from the linked folder DLLs
+version_dict = get_sdk_versions(rf".\{const.APPDIR}\obj\Release\{precommands.framework}\android-arm64\linked")
 versions_write_json(version_dict, rf"{output_dir}\versions.json")
-print(f"Versions: {version_dict}")
+print(f"Versions: {version_dict} from location " + rf".\{const.APPDIR}\obj\Release\{precommands.framework}\android-arm64\linked")
diff --git a/src/scenarios/mauiblazorandroid/pre.py b/src/scenarios/mauiblazorandroid/pre.py
index 9a383b2837f..cce988310b3 100644
--- a/src/scenarios/mauiblazorandroid/pre.py
+++ b/src/scenarios/mauiblazorandroid/pre.py
@@ -5,14 +5,19 @@
 import sys
 from performance.logger import setup_loggers, getLogger
 from shared import const
-from shared.mauisharedpython import remove_aab_files, install_versioned_maui
+from shared.mauisharedpython import remove_aab_files, install_latest_maui
 from shared.precommands import PreCommands
-from shared.versionmanager import versions_write_json, get_version_from_dll_powershell
+from shared.versionmanager import versions_write_json, get_sdk_versions
 from test import EXENAME
 
 setup_loggers(True)
+logger = getLogger(__name__)
+logger.info("Starting pre-command for MAUI Blazor Android template app (dotnet new maui-blazor)")
+
 precommands = PreCommands()
-install_versioned_maui(precommands)
+
+install_latest_maui(precommands)
+precommands.print_dotnet_info()
 
 # Setup the Maui folder
 precommands.new(template='maui-blazor',
@@ -54,7 +59,6 @@
             mainActivityFile.write(line)
 
 # Build the APK
-shutil.copy('./MauiNuGet.config', './app/Nuget.config')
 precommands.execute([])
 
 output_dir = const.PUBDIR
@@ -62,9 +66,7 @@
     output_dir = precommands.output
 remove_aab_files(output_dir)
 
-# Copy the MauiVersion to a file so we have it on the machine
-maui_version = get_version_from_dll_powershell(rf".\{const.APPDIR}\obj\Release\{precommands.framework}\android-arm64\linked\Microsoft.Maui.dll")
-version_dict = { "mauiVersion": maui_version }
+# Extract the versions of used SDKs from the linked folder DLLs
+version_dict = get_sdk_versions(rf".\{const.APPDIR}\obj\Release\{precommands.framework}\android-arm64\linked")
 versions_write_json(version_dict, rf"{output_dir}\versions.json")
-print(f"Versions: {version_dict}")
-
+print(f"Versions: {version_dict} from location " + rf".\{const.APPDIR}\obj\Release\{precommands.framework}\android-arm64\linked")
diff --git a/src/scenarios/netandroid/pre.py b/src/scenarios/netandroid/pre.py
index bb565a98146..c2f4c78f0d1 100644
--- a/src/scenarios/netandroid/pre.py
+++ b/src/scenarios/netandroid/pre.py
@@ -5,15 +5,19 @@
 import sys
 from performance.logger import setup_loggers, getLogger
 from shared import const
-from shared.mauisharedpython import remove_aab_files, install_versioned_maui
+from shared.mauisharedpython import remove_aab_files,install_latest_maui
 from shared.precommands import PreCommands
-from shared.versionmanager import versions_write_json, get_version_from_dll_powershell
+from shared.versionmanager import versions_write_json, get_sdk_versions
 from test import EXENAME
 
 setup_loggers(True)
+logger = getLogger(__name__)
+logger.info("Starting pre-command for .NET Android template app (dotnet new android)")
 
 precommands = PreCommands()
-install_versioned_maui(precommands)
+
+install_latest_maui(precommands)
+precommands.print_dotnet_info()
 
 # Setup the app folder
 precommands.new(template='android',
@@ -24,7 +28,6 @@
                 no_restore=False)
 
 # Build the APK
-shutil.copy('./MauiNuGet.config', './app/Nuget.config')
 precommands.execute([])
 
 # Remove the aab files as we don't need them, this saves space
@@ -33,8 +36,7 @@
     output_dir = precommands.output
 remove_aab_files(output_dir)
 
-# Copy the netAndroidVersion to a file so we have it on the machine
-net_android_version = get_version_from_dll_powershell(rf".\{const.APPDIR}\obj\Release\{precommands.framework}\android-arm64\linked\Mono.Android.dll")
-version_dict = { "netAndroidVersion": net_android_version }
+# Extract the versions of used SDKs from the linked folder DLLs
+version_dict = get_sdk_versions(rf".\{const.APPDIR}\obj\Release\{precommands.framework}\android-arm64\linked")
 versions_write_json(version_dict, rf"{output_dir}\versions.json")
-print(f"Versions: {version_dict} from location " + rf".\{const.APPDIR}\obj\Release\{precommands.framework}\android-arm64\linked\Mono.Android.dll")
+print(f"Versions: {version_dict} from location " + rf".\{const.APPDIR}\obj\Release\{precommands.framework}\android-arm64\linked")
diff --git a/src/scenarios/shared/mauisharedpython.py b/src/scenarios/shared/mauisharedpython.py
index e73253c24d9..e0c153edee2 100644
--- a/src/scenarios/shared/mauisharedpython.py
+++ b/src/scenarios/shared/mauisharedpython.py
@@ -5,6 +5,7 @@
 import urllib.request
 from performance.common import get_repo_root_path
 from shared.precommands import PreCommands
+from logging import getLogger
 
 # Remove the aab files as we don't need them, this saves space in the correlation payload
 def remove_aab_files(output_dir="."):
@@ -93,4 +94,118 @@ def install_versioned_maui(precommands: PreCommands):
         dump_dict_to_json_file(rollback_dict, f"rollback_{target_framework_wo_platform}.json")
         workload_install_args += ['--from-rollback-file', f'rollback_{target_framework_wo_platform}.json']
 
-    precommands.install_workload('maui', workload_install_args) 
+    precommands.install_workload('maui', workload_install_args)
+
+def extract_latest_dotnet_feed_from_nuget_config(path: str) -> str:
+    '''
+        Extract the latest dotnet feed from the NuGet.config file.
+        The latest feed is the one that has the highest version number.
+        Supports only single number versioning (dotnet9, dotnet10, etc.)
+    '''
+    tree = ET.parse(path)
+    root = tree.getroot()
+
+    # Find the <packageSources> element
+    package_sources = root.find(".//packageSources")
+    if package_sources is None:
+        raise ValueError("No <packageSources> element found in NuGet.config")
+
+    # Extract all <add> elements with keys starting with "dotnet" followed by a number
+    dotnet_feeds = dict[int, str]()
+    for add_element in package_sources.findall("add"):
+        key = add_element.get("key")
+        value = add_element.get("value")
+        if key and value:
+            match = re.match(r"dotnet(\d+)$", key)
+            if match:
+                version = int(match.group(1))
+                dotnet_feeds[version] = value
+
+    # Find the highest version
+    if not dotnet_feeds:
+        raise ValueError("No dotnet feeds found in NuGet.config")
+
+    latest_version = max(dotnet_feeds.keys())
+    latest_feed = dotnet_feeds[latest_version]
+
+    return latest_feed
+
+def install_latest_maui(
+        precommands: PreCommands, 
+        feed=extract_latest_dotnet_feed_from_nuget_config(path=os.path.join(get_repo_root_path(), "NuGet.config"))
+        ):
+    '''
+        Install the latest maui workload using the provided feed. 
+        This function will create a rollback file and install the maui workload using that file.
+    '''
+
+    if precommands.has_workload:
+        getLogger().info("Skipping maui installation due to --has-workload=true")
+        return
+
+    maui_rollback_dict: dict[str, str] = {
+        "microsoft.net.sdk.android" : "",
+        "microsoft.net.sdk.ios" : "",
+        "microsoft.net.sdk.maccatalyst" : "",
+        "microsoft.net.sdk.macos" : "",
+        "microsoft.net.sdk.maui" : "",
+        "microsoft.net.sdk.tvos" : ""
+    }
+
+    getLogger().info(f"Installing the latest maui workload from feed {feed}")
+
+    # Get the latest published version of the maui workloads
+    for workload in maui_rollback_dict.keys():
+        packages = precommands.get_packages_for_sdk_from_feed(workload, feed)
+
+        try:
+            packages = json.loads(packages)["searchResult"][0]['packages']
+        except json.JSONDecodeError as e:
+            raise Exception(f"Error parsing JSON output: {e}")
+
+        # Filter out packages that have ID that matches the pattern 'Microsoft.NET.Sdk.<workload>.Manifest-<version>'
+        packages = [pkg for pkg in packages if re.match(r'Microsoft\.NET\.Sdk\..*\.Manifest\-\d+\.\d+\.\d+(\-(preview|rc|alpha)\.\d+)?$', pkg['id'])]
+
+        # Extract the .NET version from the package ID (Manifest-<version>($|-<preview|rc|alpha>.*))
+        for package in packages:
+            match = re.search(r'Manifest-(.+)$', package["id"])
+            if match:
+                sdk_version = match.group(1)
+                package['sdk_version'] = sdk_version
+
+                # Extract the .NET version from sdk_version (first integer)
+                match = re.search(r'^\d+\.\d+', sdk_version)
+                if match:
+                    dotnet_version = match.group(0)
+                    package['dotnet_version'] = dotnet_version
+                else:
+                    raise Exception("Unable to find .NET version in SDK version")
+            else:
+                raise Exception("Unable to find .NET SDK version in package ID")
+            
+        # Filter out packages that have lower 'dotnet_version' than the rest of the packages
+        # Sometimes feed can contain packages from previous release versions, so we need to filter them out
+        highest_dotnet_version = max(float(pkg['dotnet_version']) for pkg in packages)
+        packages = [pkg for pkg in packages if float(pkg['dotnet_version']) == highest_dotnet_version]
+
+        # Check if we have non-preview packages available, if so, check if the version is greater than the preview version
+        non_preview_packages = [pkg for pkg in packages if not re.search(r'\-(preview|rc|alpha)\.\d+$', pkg['id'])]
+        if non_preview_packages:
+            packages = non_preview_packages
+
+        # Sort the packages first by by 'sdk_version'
+        packages.sort(key=lambda x: x['sdk_version'], reverse=True)
+
+        latest_package = packages[0]
+
+        getLogger().info(f"Latest package for {workload} found: {latest_package['id']} {latest_package['latestVersion']}")
+        
+        maui_rollback_dict[workload] = f"{latest_package['latestVersion']}/{latest_package['sdk_version']}"
+
+    # Create the rollback file
+    with open("rollback_maui.json", "w", encoding="utf-8") as f:
+        f.write(json.dumps(maui_rollback_dict, indent=4))
+
+    # Install the workload using the rollback file
+    getLogger().info("Installing maui workload with rollback file")
+    precommands.install_workload('maui', ['--from-rollback-file', 'rollback_maui.json'])
diff --git a/src/scenarios/shared/precommands.py b/src/scenarios/shared/precommands.py
index 4a34e34a2f6..bae0af4cff8 100644
--- a/src/scenarios/shared/precommands.py
+++ b/src/scenarios/shared/precommands.py
@@ -288,6 +288,35 @@ def uninstall_workload(self, workloadid: str):
         if self.has_workload and not self.readonly_dotnet:
             subprocess.run(["dotnet", "workload", "uninstall", workloadid])
 
+    def setup_workload_update_mode(self, update_mode: str):
+        'Sets the workload update mode to the given value <manifests|workload-set>'
+        if update_mode != 'manifests' and update_mode != 'workload-set':
+            raise Exception('workload config --update-mode must be either manifests or workload-set')
+        
+        if self.readonly_dotnet:
+            raise Exception('workload config --update-mode not supported with readonly-dotnet=true')
+        
+        subprocess.run(["dotnet", "workload", "config", "--update-mode", update_mode], check=True)
+
+    def print_dotnet_info(self):
+        'Prints the dotnet info'
+        subprocess.run(["dotnet", "--info"], check=True)
+
+
+    def get_packages_for_sdk_from_feed(self, sdk_name: str, feed: str):
+        'Gets the packages for the given sdk from the given feed'
+
+        getLogger().debug(f"dotnet package search {sdk_name} --prerelease --take 999 --format json --source {feed}")
+
+        # Query the given feed for published SDKs using the `dotnet package search` command
+        result = subprocess.run(
+            ["dotnet", "package", "search", sdk_name, "--prerelease", "--take", "999", "--format", "json", "--source", feed], check=True, capture_output=True, text=True)
+        
+        if result.returncode != 0:
+            raise Exception(f"Error querying package feed: {result.stderr}")
+        
+        return result.stdout
+
     def _addstaticmsbuildproperty(self, projectfile: str):
         'Insert static msbuild property in the specified project file'
         if self.msbuildstatic:
diff --git a/src/scenarios/shared/versionmanager.py b/src/scenarios/shared/versionmanager.py
index 5befb00e5d4..119ef6b1dc4 100644
--- a/src/scenarios/shared/versionmanager.py
+++ b/src/scenarios/shared/versionmanager.py
@@ -4,7 +4,7 @@
 import json
 import os
 import subprocess
-
+from performance.logger import getLogger
 from typing import Dict
 
 def versions_write_json(versiondict: Dict[str, str], outputfile: str = 'versions.json'):
@@ -34,4 +34,63 @@ def get_version_from_dll_powershell(dll_path: str):
 
 def get_version_from_dll_powershell_ios(dll_path: str):
     result = subprocess.run(['pwsh', '-Command', rf'Get-ChildItem {dll_path} | Select-Object -ExpandProperty VersionInfo | Select-Object -ExpandProperty ProductVersion'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=False)
-    return result.stdout.decode('utf-8').strip()
\ No newline at end of file
+    return result.stdout.decode('utf-8').strip()
+
+def get_sdk_versions(dll_folder_path: str, windows_powershell: bool = True) -> dict[str, str]:
+    '''
+    Get the SDK versions from the used dlls
+    :param dll_folder_path: The folder path where the dlls are located
+    :return: A dictionary with the SDK version identifiers and commit hashes
+    '''
+
+    def parse_version_output(output: str) -> tuple[str, str]:
+        """
+        Parse the output of the PowerShell command to extract version and commit hash.
+        :param output: The output string from the PowerShell command.
+        :return: A tuple containing the version and commit hash.
+        """
+        version = None
+        commit = None
+
+        if '+' in output: # Handle "versi<version>+<commit>" format
+            parts = output.split('+')
+            version = parts[0].strip()
+            commit = parts[1].strip()
+        else: # Handle "<version>; git-rev-head:<commit>; git-branch:<branch>" format
+            parts = output.split(';')
+            version = parts[0].strip()
+
+            for part in parts:
+                if 'git-rev-head:' in part:
+                    commit = part.split(':', 1)[1].strip()
+                    break
+        
+        assert version is not None, "Version parsing failed"
+        assert commit is not None, "Commit parsing failed"
+
+        return version, commit
+
+    powershell_cmd = get_version_from_dll_powershell if windows_powershell else get_version_from_dll_powershell_ios
+
+    mobile_sdks = {
+        "net_android": "Mono.Android.dll",
+        "net_ios": "Mono.iOS.dll",
+        "net_maui": "Microsoft.Maui.dll",
+        "runtime": "System.Runtime.dll"
+    }
+    results = dict[str, str]()
+
+    for sdk, dll_name in mobile_sdks.items():
+        dll_path = os.path.join(dll_folder_path, dll_name)
+        print(f"Getting version from {dll_path}")
+        result = powershell_cmd(dll_path)
+        
+        if "Cannot find path" in result:
+            getLogger().warning(f"Cannot find {dll_name} in {dll_folder_path}. Skipping version extraction.")
+            continue
+
+        version, commit = parse_version_output(result)
+        results[f"{sdk}_version"] = version
+        results[f"PERFLAB_DATA_{sdk}_commit_hash"] = commit
+            
+    return results


Reviews:
user: login: LoopedBard3
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T22:33:16Z
user: login: LoopedBard3
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T23:39:35Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-19T13:35:24Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-19T13:36:07Z
user: login: caaavik-msft
body: Looks good to me. One question I have, are any of these changes (e.g. the _MSBuildArgs) being made because they are coupled to the runtime repository itself or would you want them set even if we decided to run against an older version of the runtime? 

As part of the recent changes we made to move the yaml from the runtime to the performance repository, I'd like to know if any of the things we moved over were better living in the runtime repository. If so, we can look at restructuring where these live to better couple related changes.
state: DISMISSED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T23:52:47Z
state: DISMISSED
author_association: NONE
submitted_at: 2025-03-24T08:29:36Z
user: login: ivanpovazan
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T10:52:24Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T14:18:43Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T14:21:51Z
user: login: ivanpovazan
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T08:29:23Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T09:01:43Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-25T11:07:37Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-01T08:40:18Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-01T16:14:53Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T07:20:51Z
user: login: LoopedBard3
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-02T21:06:49Z
user: login: ivanpovazan
body: LGTM, only left few minor notes.
Thank you!
state: DISMISSED
author_association: MEMBER
submitted_at: 2025-04-07T13:23:03Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T08:59:37Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T09:00:15Z
user: login: ivanpovazan
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T10:56:28Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T16:44:51Z
user: login: matouskozak
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T17:00:30Z
user: login: LoopedBard3
body: This looks really great! I am excited to have what seems to be a more robust solution for building the apps. I left one comment but am also wondering if this supports 9.0 and 8.0 mobile app runs. It seems that it currently sets up the runs to only run main.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T17:46:21Z
user: login: LoopedBard3
state: DISMISSED
author_association: MEMBER
submitted_at: 2025-04-10T18:20:07Z
user: login: copilot-pull-request-reviewer[bot]
body: 

Copilot reviewed 9 out of 10 changed files in this pull request and generated 2 comments.

<details>
<summary>Files not reviewed (1)</summary>

* **eng/performance/maui_scenarios_android.proj**: Language not supported
</details>

<details>
<summary>Comments suppressed due to low confidence (1)</summary>

**eng/pipelines/sdk-perf-jobs.yml:405**
* [nitpick] The pipeline uses 'codeGenType' (PascalCase) while the corresponding code in run_performance_job.py references 'codegen_type' (lowercase). Consider using consistent naming conventions.
```
codeGenType: ProfiledAOT
```
</details>


state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-10T21:03:14Z
user: login: ivanpovazan
body: This looks neat, thank you!

The only thoughts left (not for this PR) are:
1. It seems like the rollback file could be provided by some other service rather than us manually creating it as part of performance runs 
2. Think about a way of doing performance runs on specific releases, versions i.e., answering a question: what is the performance of our preview.X release - and with that freeze/tag such measurements for reference

Other than that, really nice work!
state: DISMISSED
author_association: MEMBER
submitted_at: 2025-04-11T13:48:59Z
user: login: caaavik-msft
body: LGTM, just the one minor comment on the yaml condition
state: DISMISSED
author_association: CONTRIBUTOR
submitted_at: 2025-04-12T13:02:43Z
user: login: LoopedBard3
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-13T03:15:40Z

Review comments:
diff_hunk: @@ -288,6 +288,37 @@ def uninstall_workload(self, workloadid: str):
         if self.has_workload and not self.readonly_dotnet:
             subprocess.run(["dotnet", "workload", "uninstall", workloadid])
 
+    def create_nuget_config(self, feeds: dict[str, str]={"nuget.org": "https://api.nuget.org/v3/index.json"}):
path: src/scenarios/shared/precommands.py
user: login: LoopedBard3
body: We will likely get workitems to pop up if we use nuget.org, would it instead be possible to instead use the one at the top level of the performance repo? If there are some packages that only occasionally update but they are not being found in our feeds, we can have them added.

Also, this does not appear to currently be called anywhere.
created_at: 2025-03-18T22:00:39Z
author_association: MEMBER
diff_hunk: @@ -541,6 +541,17 @@ def run_performance_job(args: RunPerformanceJobArgs):
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    if args.run_kind == "maui_scenarios_android":
+        if args.runtime_flavor == "mono":
+            runtime_type = "Mono"
+            configurations["CompilationMode"] = "ProfiledAOT"
path: scripts/run_performance_job.py
user: login: LoopedBard3
body: We should make sure other similar android app testing matches this for Mono.
created_at: 2025-03-18T23:39:35Z
author_association: MEMBER
diff_hunk: @@ -288,6 +288,37 @@ def uninstall_workload(self, workloadid: str):
         if self.has_workload and not self.readonly_dotnet:
             subprocess.run(["dotnet", "workload", "uninstall", workloadid])
 
+    def create_nuget_config(self, feeds: dict[str, str]={"nuget.org": "https://api.nuget.org/v3/index.json"}):
path: src/scenarios/shared/precommands.py
user: login: matouskozak
body: removed
created_at: 2025-03-19T13:35:24Z
author_association: MEMBER
diff_hunk: @@ -541,6 +541,17 @@ def run_performance_job(args: RunPerformanceJobArgs):
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    if args.run_kind == "maui_scenarios_android":
+        if args.runtime_flavor == "mono":
+            runtime_type = "Mono"
+            configurations["CompilationMode"] = "ProfiledAOT"
path: scripts/run_performance_job.py
user: login: matouskozak
body: I added a comment explaining which scenario is which. We run different configuration for dotnet/runtime android sample than the default configuration used in .NET Android / .NET MAUI Android samples.
created_at: 2025-03-19T13:36:07Z
author_association: MEMBER
diff_hunk: @@ -11,6 +11,10 @@
     <PreparePayloadOutDirectoryName>scenarios_out</PreparePayloadOutDirectoryName>
     <PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' == 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)\</PreparePayloadWorkItemBaseDirectory>
     <PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' != 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)/</PreparePayloadWorkItemBaseDirectory>
+
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'mono'">/p:UseMonoRuntime=true</_MSBuildArgs>
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">/p:UseMonoRuntime=false;/p:AndroidEnableMarshalMethods=false</_MSBuildArgs>
path: eng/performance/maui_scenarios_android.proj
user: login: ivanpovazan
body: ```suggestion
    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr' and '$(PublishAot)' != 'true'">/p:UseMonoRuntime=false</_MSBuildArgs>
```
`AndroidEnableMarshalMethods ` is not needed anymore as of: https://github.com/dotnet/android/pull/9912
created_at: 2025-03-24T10:20:36Z
author_association: MEMBER
diff_hunk: @@ -25,19 +29,19 @@
   </ItemDefinitionGroup>
 
   <ItemGroup>
-    <MAUIAndroidScenario Include="Maui Android Default">
-      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
-      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
-      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
-      <PackageName>com.companyname.mauiandroiddefault</PackageName>
-    </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include=".NET Android Default">
+    <MAUIAndroidScenario Include=".NET Android Default $(RunConfigsString)">
path: eng/performance/maui_scenarios_android.proj
user: login: ivanpovazan
body: I think we should revisit the naming here and/or introduce new cases.
The `Default` is different between Mono, CoreCLR and NativeAOT, Mono's defaults to profiled AOT, while CoreCLR defaults to JIT only *at the moment* etc, which can lead to confusion when analysing the measurements.
I suggest we add new more descriptive scenarios (JIT, AOT, ..) conditioned on the runtime type.
Lastly, marking which one of the scenarios is the default one is still valueable.
created_at: 2025-03-24T10:41:26Z
author_association: MEMBER
diff_hunk: @@ -25,19 +29,19 @@
   </ItemDefinitionGroup>
 
   <ItemGroup>
-    <MAUIAndroidScenario Include="Maui Android Default">
-      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
-      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
-      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
-      <PackageName>com.companyname.mauiandroiddefault</PackageName>
-    </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include=".NET Android Default">
+    <MAUIAndroidScenario Include=".NET Android Default $(RunConfigsString)">
       <ScenarioDirectoryName>netandroid</ScenarioDirectoryName>
       <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
       <ApkName>com.companyname.netandroiddefault-Signed</ApkName>
       <PackageName>com.companyname.NetAndroidDefault</PackageName>
     </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include="Maui Blazor Android Default">
+    <MAUIAndroidScenario Include="Maui Android Default $(RunConfigsString)" Condition="'$(RuntimeFlavor)' == 'mono'">
+      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
+      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
+      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
+      <PackageName>com.companyname.mauiandroiddefault</PackageName>
+    </MAUIAndroidScenario>
+    <MAUIAndroidScenario Include="Maui Blazor Android Default (TTFD) $(RunConfigsString)" Condition="'$(RuntimeFlavor)' == 'mono'">
path: eng/performance/maui_scenarios_android.proj
user: login: ivanpovazan
body: Why do we only care about TTFD measurement here? Or why don't we measure both TTID and TTFD?
created_at: 2025-03-24T10:42:02Z
author_association: MEMBER
diff_hunk: @@ -531,16 +531,30 @@ def run_performance_job(args: RunPerformanceJobArgs):
 
     runtime_type = ""
 
+    # dotnet/runtime AndroidSampleApp using Mono runtime
     if android_mono:
         runtime_type = "Mono"
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    # dotnet/runtime AndroidSampleApp using CoreCLR runtime
     if android_coreclr:
         runtime_type = "CoreCLR"
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    # .NET Android and .NET MAUI Android sample app scenarios
+    if args.run_kind == "maui_scenarios_android":
+        if args.runtime_flavor == "mono":
+            runtime_type = "Mono"
+            configurations["CompilationMode"] = "ProfiledAOT"
path: scripts/run_performance_job.py
user: login: ivanpovazan
body: Can we use `CompilationMode` in the `MAUIAndroidScenario` naming above?
created_at: 2025-03-24T10:45:51Z
author_association: MEMBER
diff_hunk: @@ -11,9 +11,14 @@
 from test import EXENAME
 
 setup_loggers(True)
+logger = getLogger(__name__)
+logger.info("Starting pre-command for Maui Android Default App")
 
 precommands = PreCommands()
-install_versioned_maui(precommands)
+
+precommands.setup_workload_update_mode("manifests")
path: src/scenarios/mauiandroid/pre.py
user: login: ivanpovazan
body: Should this be the default behavior? 
My understanding is that we have to set the update mode only if we want to use the nightly builds
created_at: 2025-03-24T10:51:55Z
author_association: MEMBER
diff_hunk: @@ -25,19 +29,19 @@
   </ItemDefinitionGroup>
 
   <ItemGroup>
-    <MAUIAndroidScenario Include="Maui Android Default">
-      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
-      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
-      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
-      <PackageName>com.companyname.mauiandroiddefault</PackageName>
-    </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include=".NET Android Default">
+    <MAUIAndroidScenario Include=".NET Android Default $(RunConfigsString)">
       <ScenarioDirectoryName>netandroid</ScenarioDirectoryName>
       <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
       <ApkName>com.companyname.netandroiddefault-Signed</ApkName>
       <PackageName>com.companyname.NetAndroidDefault</PackageName>
     </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include="Maui Blazor Android Default">
+    <MAUIAndroidScenario Include="Maui Android Default $(RunConfigsString)" Condition="'$(RuntimeFlavor)' == 'mono'">
+      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
+      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
+      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
+      <PackageName>com.companyname.mauiandroiddefault</PackageName>
+    </MAUIAndroidScenario>
+    <MAUIAndroidScenario Include="Maui Blazor Android Default (TTFD) $(RunConfigsString)" Condition="'$(RuntimeFlavor)' == 'mono'">
path: eng/performance/maui_scenarios_android.proj
user: login: matouskozak
body: I don't know. I just saw that it is passing the `--use-fully-drawn-time` so I added it to the scenario name to make it clear. I can probably add TTID as well if this is something we want.
created_at: 2025-03-24T14:18:43Z
author_association: MEMBER
diff_hunk: @@ -11,9 +11,14 @@
 from test import EXENAME
 
 setup_loggers(True)
+logger = getLogger(__name__)
+logger.info("Starting pre-command for Maui Android Default App")
 
 precommands = PreCommands()
-install_versioned_maui(precommands)
+
+precommands.setup_workload_update_mode("manifests")
path: src/scenarios/mauiandroid/pre.py
user: login: matouskozak
body: > My understanding is that we have to set the update mode only if we want to use the nightly builds

This is true. 

> Should this be the default behavior?

The change is intended to be the new default, and I wasn't planning for us to have 2 separate pipelines, one of released and another of nightly builds. These measurements should serve as a tool for internal monitoring and not public benchmarks. Do you see benefits in having both scenarios (released/nightly) running in the lab?

created_at: 2025-03-24T14:21:51Z
author_association: MEMBER
diff_hunk: @@ -11,9 +11,14 @@
 from test import EXENAME
 
 setup_loggers(True)
+logger = getLogger(__name__)
+logger.info("Starting pre-command for Maui Android Default App")
 
 precommands = PreCommands()
-install_versioned_maui(precommands)
+
+precommands.setup_workload_update_mode("manifests")
path: src/scenarios/mauiandroid/pre.py
user: login: ivanpovazan
body: If we checkout the repo and want to run the scenario locally with a specific release how would we do that?
created_at: 2025-03-25T08:29:23Z
author_association: MEMBER
diff_hunk: @@ -11,9 +11,14 @@
 from test import EXENAME
 
 setup_loggers(True)
+logger = getLogger(__name__)
+logger.info("Starting pre-command for Maui Android Default App")
 
 precommands = PreCommands()
-install_versioned_maui(precommands)
+
+precommands.setup_workload_update_mode("manifests")
path: src/scenarios/mauiandroid/pre.py
user: login: matouskozak
body: I see, I have not considered this as a user scenario when designing this change. 

We can specify local dotnet install by `. ./init.sh -dotnetdir <custom dotnet directory>` so I think it could be enough to pass an argument to the `pre.py` command to skip installing the additional workloads and rely on the user that the `<custom dotnet directory>` contains the specific release installed. Would that be sufficient for the use case you have in mind?
created_at: 2025-03-25T09:01:43Z
author_association: MEMBER
diff_hunk: @@ -11,9 +11,14 @@
 from test import EXENAME
 
 setup_loggers(True)
+logger = getLogger(__name__)
+logger.info("Starting pre-command for Maui Android Default App")
 
 precommands = PreCommands()
-install_versioned_maui(precommands)
+
+precommands.setup_workload_update_mode("manifests")
path: src/scenarios/mauiandroid/pre.py
user: login: matouskozak
body: Update on that, we actually already support this scenario. If you use:
1. `. ./init.sh -dotnetdir <custom dotnet directory>`
2. `pre.py <other_args> --readonly-dotnet --has-workload` (https://github.com/dotnet/performance/blob/c3b4ec9be7d81bdbb2e765f2f454464ac1aba421/src/scenarios/shared/precommands.py#L176-L185)

This should ensure that the test app is build using the locally installed dotnet that has the specific configuration of SDKs.

I'll verify the code in `pre.py` to be compatible with this setup.
created_at: 2025-03-25T11:07:37Z
author_association: MEMBER
diff_hunk: @@ -25,19 +29,19 @@
   </ItemDefinitionGroup>
 
   <ItemGroup>
-    <MAUIAndroidScenario Include="Maui Android Default">
-      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
-      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
-      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
-      <PackageName>com.companyname.mauiandroiddefault</PackageName>
-    </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include=".NET Android Default">
+    <MAUIAndroidScenario Include=".NET Android Default $(RunConfigsString)">
       <ScenarioDirectoryName>netandroid</ScenarioDirectoryName>
       <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
       <ApkName>com.companyname.netandroiddefault-Signed</ApkName>
       <PackageName>com.companyname.NetAndroidDefault</PackageName>
     </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include="Maui Blazor Android Default">
+    <MAUIAndroidScenario Include="Maui Android Default $(RunConfigsString)" Condition="'$(RuntimeFlavor)' == 'mono'">
+      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
+      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
+      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
+      <PackageName>com.companyname.mauiandroiddefault</PackageName>
+    </MAUIAndroidScenario>
+    <MAUIAndroidScenario Include="Maui Blazor Android Default (TTFD) $(RunConfigsString)" Condition="'$(RuntimeFlavor)' == 'mono'">
path: eng/performance/maui_scenarios_android.proj
user: login: matouskozak
body: @LoopedBard3 do you remember why for blazor we are using `--use-fully-drawn-time` and not the default startup measurement config?
created_at: 2025-04-01T08:40:17Z
author_association: MEMBER
diff_hunk: @@ -25,19 +29,19 @@
   </ItemDefinitionGroup>
 
   <ItemGroup>
-    <MAUIAndroidScenario Include="Maui Android Default">
-      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
-      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
-      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
-      <PackageName>com.companyname.mauiandroiddefault</PackageName>
-    </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include=".NET Android Default">
+    <MAUIAndroidScenario Include=".NET Android Default $(RunConfigsString)">
path: eng/performance/maui_scenarios_android.proj
user: login: matouskozak
body: Added
created_at: 2025-04-01T16:14:53Z
author_association: MEMBER
diff_hunk: @@ -25,19 +29,19 @@
   </ItemDefinitionGroup>
 
   <ItemGroup>
-    <MAUIAndroidScenario Include="Maui Android Default">
-      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
-      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
-      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
-      <PackageName>com.companyname.mauiandroiddefault</PackageName>
-    </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include=".NET Android Default">
+    <MAUIAndroidScenario Include=".NET Android Default $(RunConfigsString)">
path: eng/performance/maui_scenarios_android.proj
user: login: matouskozak
body: Actually, I'm thinking about removing the `$(RunConfigsString)` all together from the scenario name as per 
https://github.com/dotnet/runtime/pull/114117#discussion_r2023567496 and offline discussion with @LoopedBard3 . 

We can already see the configuration used in the job name 
<img width="611" alt="image" src="https://github.com/user-attachments/assets/59cd5c4a-2864-4c7d-9130-7f9d453dd2b0" />
\+ everything we place in `configurations` https://github.com/dotnet/performance/blob/2659b9d72210c725f20e7a10499ff88d40bcc29f/scripts/run_performance_job.py#L550-L551 is going to flow to database so we can retrieve it for visualizations...
created_at: 2025-04-02T07:20:51Z
author_association: MEMBER
diff_hunk: @@ -25,19 +29,19 @@
   </ItemDefinitionGroup>
 
   <ItemGroup>
-    <MAUIAndroidScenario Include="Maui Android Default">
-      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
-      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
-      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
-      <PackageName>com.companyname.mauiandroiddefault</PackageName>
-    </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include=".NET Android Default">
+    <MAUIAndroidScenario Include=".NET Android Default $(RunConfigsString)">
       <ScenarioDirectoryName>netandroid</ScenarioDirectoryName>
       <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
       <ApkName>com.companyname.netandroiddefault-Signed</ApkName>
       <PackageName>com.companyname.NetAndroidDefault</PackageName>
     </MAUIAndroidScenario>
-    <MAUIAndroidScenario Include="Maui Blazor Android Default">
+    <MAUIAndroidScenario Include="Maui Android Default $(RunConfigsString)" Condition="'$(RuntimeFlavor)' == 'mono'">
+      <ScenarioDirectoryName>mauiandroid</ScenarioDirectoryName>
+      <PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
+      <ApkName>com.companyname.mauiandroiddefault-Signed</ApkName>
+      <PackageName>com.companyname.mauiandroiddefault</PackageName>
+    </MAUIAndroidScenario>
+    <MAUIAndroidScenario Include="Maui Blazor Android Default (TTFD) $(RunConfigsString)" Condition="'$(RuntimeFlavor)' == 'mono'">
path: eng/performance/maui_scenarios_android.proj
user: login: LoopedBard3
body: I don't recall anything specifically. Maybe TTID doesn't include some of the blazor startup. Regardless, I don't see any reason not to include both.
created_at: 2025-04-02T21:06:49Z
author_association: MEMBER
diff_hunk: @@ -11,6 +11,10 @@
     <PreparePayloadOutDirectoryName>scenarios_out</PreparePayloadOutDirectoryName>
     <PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' == 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)\</PreparePayloadWorkItemBaseDirectory>
     <PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' != 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)/</PreparePayloadWorkItemBaseDirectory>
+
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'mono'">/p:UseMonoRuntime=true</_MSBuildArgs>
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">/p:UseMonoRuntime=false;</_MSBuildArgs>
path: eng/performance/maui_scenarios_android.proj
user: login: ivanpovazan
body: ```suggestion
    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">/p:UseMonoRuntime=false</_MSBuildArgs>
```

Do we need to concat more than this property?
created_at: 2025-04-07T13:18:36Z
author_association: MEMBER
diff_hunk: @@ -531,16 +531,25 @@ def run_performance_job(args: RunPerformanceJobArgs):
 
     runtime_type = ""
 
+    # dotnet/runtime AndroidSampleApp using Mono runtime
     if android_mono:
         runtime_type = "Mono"
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    # dotnet/runtime AndroidSampleApp using CoreCLR runtime
     if android_coreclr:
         runtime_type = "CoreCLR"
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    # .NET Android and .NET MAUI Android sample app scenarios
+    if args.run_kind == "maui_scenarios_android":
+        if not args.runtime_flavor in ("mono", "coreclr"):
+            raise Exception("Runtime flavor must be specified for maui_scenarios_android")
+        configurations["CodegenType"] = str(args.codegen_type)
path: scripts/run_performance_job.py
user: login: ivanpovazan
body: Just for my understanding how is `CodegenType` different from `CompilationMode` used above?
created_at: 2025-04-07T13:20:52Z
author_association: MEMBER
diff_hunk: @@ -93,4 +94,80 @@ def install_versioned_maui(precommands: PreCommands):
         dump_dict_to_json_file(rollback_dict, f"rollback_{target_framework_wo_platform}.json")
         workload_install_args += ['--from-rollback-file', f'rollback_{target_framework_wo_platform}.json']
 
-    precommands.install_workload('maui', workload_install_args) 
+    precommands.install_workload('maui', workload_install_args)
+
+# TODO: the feed should be updated to the latest
path: src/scenarios/shared/mauisharedpython.py
user: login: ivanpovazan
body: Once we have darc setup to update NuGet feeds, this won't be needed anymore, correct?
created_at: 2025-04-07T13:22:33Z
author_association: MEMBER
diff_hunk: @@ -531,16 +531,25 @@ def run_performance_job(args: RunPerformanceJobArgs):
 
     runtime_type = ""
 
+    # dotnet/runtime AndroidSampleApp using Mono runtime
     if android_mono:
         runtime_type = "Mono"
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    # dotnet/runtime AndroidSampleApp using CoreCLR runtime
     if android_coreclr:
         runtime_type = "CoreCLR"
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    # .NET Android and .NET MAUI Android sample app scenarios
+    if args.run_kind == "maui_scenarios_android":
+        if not args.runtime_flavor in ("mono", "coreclr"):
+            raise Exception("Runtime flavor must be specified for maui_scenarios_android")
+        configurations["CodegenType"] = str(args.codegen_type)
path: scripts/run_performance_job.py
user: login: matouskozak
body: It is just naming, no difference. I think `CodegenType` aligns better with the rest of the naming in this and related files and I already changed the `CompilationMode` you are referencing to `CodegenType` (https://github.com/dotnet/performance/blob/bcf4b772911fd7cbcae18b351727f81c997617bf/scripts/run_performance_job.py#L532-L541), I need to resolve the merge conflict here, as going forward we only use `CodegenType`.
created_at: 2025-04-08T08:59:37Z
author_association: MEMBER
diff_hunk: @@ -11,6 +11,10 @@
     <PreparePayloadOutDirectoryName>scenarios_out</PreparePayloadOutDirectoryName>
     <PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' == 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)\</PreparePayloadWorkItemBaseDirectory>
     <PreparePayloadWorkItemBaseDirectory Condition="'$(TargetsWindows)' != 'true'">$(CorrelationPayloadDirectory)$(PreparePayloadOutDirectoryName)/</PreparePayloadWorkItemBaseDirectory>
+
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'mono'">/p:UseMonoRuntime=true</_MSBuildArgs>
+    <_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">/p:UseMonoRuntime=false;</_MSBuildArgs>
path: eng/performance/maui_scenarios_android.proj
user: login: matouskozak
body: Right, my mistake, not needed.
created_at: 2025-04-08T09:00:15Z
author_association: MEMBER
diff_hunk: @@ -531,16 +531,25 @@ def run_performance_job(args: RunPerformanceJobArgs):
 
     runtime_type = ""
 
+    # dotnet/runtime AndroidSampleApp using Mono runtime
     if android_mono:
         runtime_type = "Mono"
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    # dotnet/runtime AndroidSampleApp using CoreCLR runtime
     if android_coreclr:
         runtime_type = "CoreCLR"
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    # .NET Android and .NET MAUI Android sample app scenarios
+    if args.run_kind == "maui_scenarios_android":
+        if not args.runtime_flavor in ("mono", "coreclr"):
+            raise Exception("Runtime flavor must be specified for maui_scenarios_android")
+        configurations["CodegenType"] = str(args.codegen_type)
path: scripts/run_performance_job.py
user: login: ivanpovazan
body: Ok, I was referring to lines:
https://github.com/dotnet/performance/pull/4770/files#diff-75e8f68d6caa7704e8c72407bcfc108e368c9b3ea7e71ca1532033c8905bad46R543
https://github.com/dotnet/performance/pull/4770/files#diff-75e8f68d6caa7704e8c72407bcfc108e368c9b3ea7e71ca1532033c8905bad46R537

created_at: 2025-04-08T10:56:28Z
author_association: MEMBER
diff_hunk: @@ -531,16 +531,25 @@ def run_performance_job(args: RunPerformanceJobArgs):
 
     runtime_type = ""
 
+    # dotnet/runtime AndroidSampleApp using Mono runtime
     if android_mono:
         runtime_type = "Mono"
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    # dotnet/runtime AndroidSampleApp using CoreCLR runtime
     if android_coreclr:
         runtime_type = "CoreCLR"
         configurations["CompilationMode"] = "JIT"
         configurations["RuntimeType"] = str(runtime_type)
 
+    # .NET Android and .NET MAUI Android sample app scenarios
+    if args.run_kind == "maui_scenarios_android":
+        if not args.runtime_flavor in ("mono", "coreclr"):
+            raise Exception("Runtime flavor must be specified for maui_scenarios_android")
+        configurations["CodegenType"] = str(args.codegen_type)
path: scripts/run_performance_job.py
user: login: matouskozak
body: I think these are old version, `CompilationMode` is no longer being used there.
created_at: 2025-04-10T16:44:51Z
author_association: MEMBER
diff_hunk: @@ -93,4 +94,80 @@ def install_versioned_maui(precommands: PreCommands):
         dump_dict_to_json_file(rollback_dict, f"rollback_{target_framework_wo_platform}.json")
         workload_install_args += ['--from-rollback-file', f'rollback_{target_framework_wo_platform}.json']
 
-    precommands.install_workload('maui', workload_install_args) 
+    precommands.install_workload('maui', workload_install_args)
+
+# TODO: the feed should be updated to the latest
path: src/scenarios/shared/mauisharedpython.py
user: login: matouskozak
body: I wrote a new logic which extracts the latest feed from the repo's Nuget.config so we don't need this anymore.
created_at: 2025-04-10T17:00:30Z
author_association: MEMBER
diff_hunk: @@ -34,4 +34,63 @@ def get_version_from_dll_powershell(dll_path: str):
 
 def get_version_from_dll_powershell_ios(dll_path: str):
     result = subprocess.run(['pwsh', '-Command', rf'Get-ChildItem {dll_path} | Select-Object -ExpandProperty VersionInfo | Select-Object -ExpandProperty ProductVersion'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=False)
-    return result.stdout.decode('utf-8').strip()
\ No newline at end of file
+    return result.stdout.decode('utf-8').strip()
+
+def get_sdk_versions(dll_folder_path: str, windows_powershell: bool = True) -> dict[str, str]:
+    '''
+    Get the SDK versions from the used dlls
+    :param dll_folder_path: The folder path where the dlls are located
+    :return: A dictionary with the SDK version identifiers and commit hashes
+    '''
+
+    def parse_version_output(output: str) -> tuple[str, str]:
+        """
+        Parse the output of the PowerShell command to extract version and commit hash.
+        :param output: The output string from the PowerShell command.
+        :return: A tuple containing the version and commit hash.
+        """
+        version = None
+        commit = None
+
+        if '+' in output: # Handle "versi<version>+<commit>" format
+            parts = output.split('+')
+            version = parts[0].strip()
+            commit = parts[1].strip()
+        else: # Handle "<version>; git-rev-head:<commit>; git-branch:<branch>" format
+            parts = output.split(';')
+            version = parts[0].strip()
+
+            for part in parts:
+                if 'git-rev-head:' in part:
+                    commit = part.split(':', 1)[1].strip()
+                    break
+        
+        assert version is not None, "Version parsing failed"
+        assert commit is not None, "Commit parsing failed"
+
+        return version, commit
+
+    powershell_cmd = get_version_from_dll_powershell if windows_powershell else get_version_from_dll_powershell_ios
+
+    mobile_sdks = {
+        "net_android": "Mono.Android.dll",
+        "net_ios": "Mono.iOS.dll",
+        "net_maui": "Microsoft.Maui.dll",
+        "runtime": "System.Runtime.dll"
+    }
+    results = dict[str, str]()
+
+    for sdk, dll_name in mobile_sdks.items():
+        dll_path = os.path.join(dll_folder_path, dll_name)
+        print(f"Getting version from {dll_path}")
+        result = powershell_cmd(dll_path)
+        
+        if "Cannot find path" in result:
+            getLogger().warning(f"Cannot find {dll_name} in {dll_folder_path}. Skipping version extraction.")
+            continue
+
+        version, commit = parse_version_output(result)
+        results[f"{sdk}_version"] = version
+        results[f"{sdk}_commit_hash"] = commit
path: src/scenarios/shared/versionmanager.py
user: login: LoopedBard3
body: Rather than adding a new string to look for in the Reporter.cs, can this instead be `PERFLAB_DATA_{sdk}_commit_hash`? The PERFLAB_DATA_ is not included in the saved additional data so it should show up the same, but then we don't need to add the additional search string `_commit_hash`.
created_at: 2025-04-10T17:39:58Z
author_association: MEMBER
diff_hunk: @@ -93,4 +94,118 @@ def install_versioned_maui(precommands: PreCommands):
         dump_dict_to_json_file(rollback_dict, f"rollback_{target_framework_wo_platform}.json")
         workload_install_args += ['--from-rollback-file', f'rollback_{target_framework_wo_platform}.json']
 
-    precommands.install_workload('maui', workload_install_args) 
+    precommands.install_workload('maui', workload_install_args)
+
+def extract_latest_dotnet_feed_from_nuget_config(path: str) -> str:
+    '''
+        Extract the latest dotnet feed from the NuGet.config file.
+        The latest feed is the one that has the highest version number.
+        Supports only single number versioning (dotnet9, dotnet10, etc.)
+    '''
+    tree = ET.parse(path)
path: src/scenarios/shared/mauisharedpython.py
user: login: Copilot
body: The function 'extract_latest_dotnet_feed_from_nuget_config' uses 'ET' without an import. Please add 'import xml.etree.ElementTree as ET' at the top of the file.
created_at: 2025-04-10T21:03:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -93,4 +94,118 @@ def install_versioned_maui(precommands: PreCommands):
         dump_dict_to_json_file(rollback_dict, f"rollback_{target_framework_wo_platform}.json")
         workload_install_args += ['--from-rollback-file', f'rollback_{target_framework_wo_platform}.json']
 
-    precommands.install_workload('maui', workload_install_args) 
+    precommands.install_workload('maui', workload_install_args)
+
+def extract_latest_dotnet_feed_from_nuget_config(path: str) -> str:
+    '''
+        Extract the latest dotnet feed from the NuGet.config file.
+        The latest feed is the one that has the highest version number.
+        Supports only single number versioning (dotnet9, dotnet10, etc.)
+    '''
+    tree = ET.parse(path)
+    root = tree.getroot()
+
+    # Find the <packageSources> element
+    package_sources = root.find(".//packageSources")
+    if package_sources is None:
+        raise ValueError("No <packageSources> element found in NuGet.config")
+
+    # Extract all <add> elements with keys starting with "dotnet" followed by a number
+    dotnet_feeds = dict[int, str]()
+    for add_element in package_sources.findall("add"):
+        key = add_element.get("key")
+        value = add_element.get("value")
+        if key and value:
+            match = re.match(r"dotnet(\d+)$", key)
path: src/scenarios/shared/mauisharedpython.py
user: login: Copilot
body: The regex usage 're.match' requires the 're' module but no import is present. Please add 'import re' at the beginning of the file.
created_at: 2025-04-10T21:03:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -186,7 +186,7 @@ jobs:
               - '--is-scenario'
             - ${{ if ne(length(parameters.runEnvVars), 0) }}:
               - "--run-env-vars ${{ join(' ', parameters.runEnvVars)}}"
-            - ${{ if and(eq(parameters.runKind, 'maui_scenarios_ios'), ne(parameters.runtimeFlavor, '')) }}:
+            - ${{ if and(or(eq(parameters.runKind, 'maui_scenarios_ios'), eq(parameters.runKind, 'maui_scenarios_android')), ne(parameters.runtimeFlavor, '')) }}:
path: eng/pipelines/templates/run-performance-job.yml
user: login: caaavik-msft
body: ```suggestion
            - ${{ if and(in(parameters.runKind, 'maui_scenarios_ios', 'maui_scenarios_android'), ne(parameters.runtimeFlavor, '')) }}:
```
Just a minor change to make the condition simpler
created_at: 2025-04-12T13:00:36Z
author_association: CONTRIBUTOR

PR comments:
user: login: matouskozak
created_at: 2025-03-22T16:20:32Z
body: >  One question I have, are any of these changes (e.g. the _MSBuildArgs) being made because they are coupled to the runtime repository itself.

They are not coupled with the dotnet/runtime repository but rather with dotnet/android that use them to select/configure appropriate runtime (Mono vs CoreCLR).

> if we decided to run against an older version of the runtime

We won't be able to run CoreCLR Android perf jobs against older runtime because the runtime doesn't support it. For Mono, it would work as is.

> I'd like to know if any of the things we moved over were better living in the runtime repository.

I think it is ideal that all this now lives in dotnet/performance as it doesn't necessary depend on any custom/local builds of runtime. Also it is much simpler to debug now as these specific jobs all run on a single pipeline.

author_association: MEMBER
user: login: ivanpovazan
created_at: 2025-03-25T15:31:53Z
body: @matouskozak 

There might be some issues with the approach of `dotnet workload config --update-mode manifests` especially around servicing releases, because the manifests will include a reference to a new servicing release (like atm `9.0.4`) which is not publicly available and we don't know the exact feed, so the workload installation would fail with:
```
Workload installation failed: One or more errors occurred. (Version 9.0.4 of package microsoft.net.runtime.monoaotcompiler.task is not found in NuGet feeds ...
```
Will experiment a bit with this but I think we might need to the the `Option 2` from your proposal in: https://github.com/dotnet/performance/issues/4754
author_association: MEMBER
user: login: matouskozak
created_at: 2025-04-01T17:26:12Z
body: Refactored the PR to address some issues and changed the maui installation to use rollback file instead. The PR contains changes to Nuget.config which should be done maestro so should be reverted before merging.
author_association: MEMBER
user: login: matouskozak
created_at: 2025-04-10T18:03:18Z
body: >  I left one comment but am also wondering if this supports 9.0 and 8.0 mobile app runs.

Yes, currently works just on main. I think it could be extended to other branches. We would just search a different dotnet feed I think and make it more parametrized. However, I don't see much value now in having measurements on other branches but we could extend it in the future if needed.
author_association: MEMBER
user: login: matouskozak
created_at: 2025-04-11T16:30:18Z
body: > Think about a way of doing performance runs on specific releases, versions i.e., answering a question: what is the performance of our preview.X release - and with that freeze/tag such measurements for reference

We can do that locally by using manually installed dotnet and providing `--readonly-dotnet --has-workload` arguments to the `pre.py`. 
author_association: MEMBER