Repository: Azure/azureml-assets
Pull request #3946: Fix vulnerabilities in model evaluation environment
Author: login: guptha23
Created: 2025-03-15T05:59:24Z | Merged: 2025-03-28T10:23:06Z

Fix vulnerabilities in model evaluation environment.


Diff:
diff --git a/assets/training/model_evaluation/components/compute_metrics/spec.yaml b/assets/training/model_evaluation/components/compute_metrics/spec.yaml
index 9e2557f700..caeff6f5fd 100644
--- a/assets/training/model_evaluation/components/compute_metrics/spec.yaml
+++ b/assets/training/model_evaluation/components/compute_metrics/spec.yaml
@@ -80,6 +80,7 @@ code: ../../src
 environment: azureml://registries/azureml/environments/model-evaluation/labels/latest
 
 command: >-
+  python download_metrics_dependencies.py && 
   python compute_metrics.py
   --task '${{inputs.task}}'
   $[[--ground_truths '${{inputs.ground_truth}}']]
diff --git a/assets/training/model_evaluation/environment/context/Dockerfile b/assets/training/model_evaluation/environment/context/Dockerfile
index d1db377bfc..16f085ac6d 100644
--- a/assets/training/model_evaluation/environment/context/Dockerfile
+++ b/assets/training/model_evaluation/environment/context/Dockerfile
@@ -1,5 +1,5 @@
 # PTCA Image (enabling Singularity Support)
-FROM mcr.microsoft.com/aifx/acpt/stable-ubuntu2004-cu118-py310-torch222:{{latest-image-tag:biweekly\.\d{6}\.\d{1}.*}}
+FROM mcr.microsoft.com/aifx/acpt/stable-ubuntu2204-cu118-py310-torch222:{{latest-image-tag:biweekly\.\d{6}\.\d{1}.*}}
 
 RUN apt-get update && apt-get -y upgrade
 
diff --git a/assets/training/model_evaluation/environment/context/conda.yaml b/assets/training/model_evaluation/environment/context/conda.yaml
index d4bed9b2e3..e3d06f1748 100644
--- a/assets/training/model_evaluation/environment/context/conda.yaml
+++ b/assets/training/model_evaluation/environment/context/conda.yaml
@@ -25,18 +25,17 @@ dependencies:
   - azureml-metrics[all]=={{latest-pypi-version}}
   - xgboost>=1.6.1,<2.0.0
   - statsmodels>=0.11.0,<0.14.1
-  - lightgbm>=3.2.1,<4.2.0
+  - lightgbm=={{latest-pypi-version}}
   - msrest>=0.6.21,<0.7.1
   - mltable=={{latest-pypi-version}}
   - openai=={{latest-pypi-version}}
-  - azure-ai-ml=={{latest-pypi-version}}
   - mldesigner=={{latest-pypi-version:pre}}
   - tiktoken
   - idna==3.7
   - pydantic
   - mteb=={{latest-pypi-version}}
   - setuptools>=70.0.0
-  - cryptography==43.0.1
+  - cryptography=={{latest-pypi-version}}
   # batch score dependencies
   - opencensus-ext-azure=={{latest-pypi-version}}
   - StrEnum=={{latest-pypi-version}}
diff --git a/assets/training/model_evaluation/src/constants.py b/assets/training/model_evaluation/src/constants.py
index 50561a2310..b087437852 100644
--- a/assets/training/model_evaluation/src/constants.py
+++ b/assets/training/model_evaluation/src/constants.py
@@ -233,6 +233,7 @@ class TelemetryConstants:
     SCORE_NAME = "score"
     EVALUATE_MODEL_NAME = "evaluate_model"
     DOWNLOAD_MODEL_DEPENDENCIES = "download_model_dependencies"
+    DOWNLOAD_METRICS_DEPENDENCIES = "download_metrics_dependencies"
 
     MLFLOW_NAME = "mlflow_evaluate"
 
diff --git a/assets/training/model_evaluation/src/download_dependencies.py b/assets/training/model_evaluation/src/download_dependencies.py
index 72c861b9bc..116b51c146 100644
--- a/assets/training/model_evaluation/src/download_dependencies.py
+++ b/assets/training/model_evaluation/src/download_dependencies.py
@@ -49,7 +49,9 @@ def main():
             "azureml_metrics",
             "azureml-metrics",
             ]
-        install_deps = []
+        install_deps = [
+            "azure-ai-ml",
+        ]
         logger.info(f"Reading from {reqs_file}")
         with open(reqs_file, "r") as f:
             for line in f.readlines():
diff --git a/assets/training/model_evaluation/src/download_metrics_dependencies.py b/assets/training/model_evaluation/src/download_metrics_dependencies.py
new file mode 100644
index 0000000000..ba0a1c8daa
--- /dev/null
+++ b/assets/training/model_evaluation/src/download_metrics_dependencies.py
@@ -0,0 +1,60 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+
+"""Script for downloading and installing runtime dependencies for compute metrics."""
+
+from logging_utilities import (
+    custom_dimensions, get_logger, swallow_all_exceptions
+)
+from azureml.telemetry.activity import log_activity
+from constants import TelemetryConstants
+
+import sys
+import subprocess
+
+custom_dimensions.app_name = TelemetryConstants.DOWNLOAD_METRICS_DEPENDENCIES
+logger = get_logger(name=__name__)
+custom_dims_dict = vars(custom_dimensions)
+
+
+@swallow_all_exceptions(logger)
+def main():
+    """Download and Install compute metrics dependencies."""
+    with log_activity(logger, TelemetryConstants.DOWNLOAD_METRICS_DEPENDENCIES,
+                      custom_dimensions=custom_dims_dict):
+        install_deps = [
+            "azure-ai-ml",
+        ]
+
+        no_install = []
+        if len(install_deps) > 0:
+            try:
+                command = [sys.executable, '-m', 'pip', 'install'] + install_deps
+                command_str = ' '.join(command)
+                logger.info(f"Installing dependencies. Executing command: {command_str}")
+                subprocess.check_output(command, stderr=subprocess.STDOUT)
+            except Exception as e:
+                logger.warning(f"Installing dependencies failed with error:\n{e.stdout.decode('utf-8')}")
+                logger.info("Installing dependencies one by one.")
+
+                command = [sys.executable, '-m', 'pip', 'install', 'dep']
+                for dep in install_deps:
+                    try:
+                        command[-1] = dep
+                        command_str = ' '.join(command)
+                        logger.info(f"Installing dependencies. Executing command: {command_str}")
+                        subprocess.check_output(command, stderr=subprocess.STDOUT)
+                    except Exception as e:
+                        logger.warning(f"Installing dependency {dep} failed with error:\n{e.stdout.decode('utf-8')}")
+                        no_install += [dep]
+
+        if len(no_install):
+            no_install = "\n".join(no_install)
+            logger.warning(
+                f"Could not install following dependencies - \n{no_install}"
+                f"\nSome functionality might not work as expected."
+            )
+
+
+if __name__ == "__main__":
+    main()


Reviews:
user: login: avberaMS2
state: APPROVED
author_association: NONE
submitted_at: 2025-03-15T06:01:16Z
user: login: ShipraJain01
state: DISMISSED
author_association: MEMBER
submitted_at: 2025-03-17T06:08:26Z
user: login: ankushbhatia2
state: DISMISSED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T06:15:52Z
user: login: vizhur
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T12:10:59Z
user: login: guptha23
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T07:51:54Z
user: login: novaturient95
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-28T10:06:54Z
user: login: ghyadav
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-28T10:13:59Z

Review comments:
diff_hunk: @@ -36,7 +36,7 @@ dependencies:
   - pydantic
   - mteb=={{latest-pypi-version}}
   - setuptools>=70.0.0
-  - cryptography==43.0.1
+  - cryptography=={{latest-pypi-version}}
path: assets/training/model_evaluation/environment/context/conda.yaml
user: login: vizhur
body: you should not have cryptography and setuptools listed. Those should come from the dependencies that actually rely on those. 
created_at: 2025-03-17T12:10:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -36,7 +36,7 @@ dependencies:
   - pydantic
   - mteb=={{latest-pypi-version}}
   - setuptools>=70.0.0
-  - cryptography==43.0.1
+  - cryptography=={{latest-pypi-version}}
path: assets/training/model_evaluation/environment/context/conda.yaml
user: login: guptha23
body: thanks Vitaliy!!

we will try to fix them as part of this or next iteration by looking at the dependencies of the base packages.
created_at: 2025-03-18T07:51:54Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-15T05:59:57Z
body: ## Test Results for assets-test
0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit a6d7606b.

[test-results]:data:application/gzip;base64,H4sIAD9r5mcC/1WMywrCMBBFf6XM2sXQ4sT4MzLmAYNtI5lkJf67pRZNdvecC+cFUeagcB3wNIBWKT/wNXORtB64HUXbfdPqXC8e8uxEZJk7EXJO+TC5rtrMNvflf23nJrZz23JpWaRsAEzeENJ9xOiZEa0NdrJE0V04EDuDxOY8jfD+AD6qDw/7AAAA

:recycle: This comment has been updated with latest results.
author_association: NONE
user: login: github-actions[bot]
created_at: 2025-03-15T06:06:27Z
body: ## Test Results for training-model-eval-unittests
3 tests   3 ✅  9s ⏱️
1 suites  0 💤
1 files    0 ❌

Results for commit a6d7606b.

[test-results]:data:application/gzip;base64,H4sIAJVs5mcC/1WMyw6CMBBFf4V07WKkcWr9GTP2kUwEavpYEf/dgoJld8+5yZmF58ElcevOp06kwnkHWyJlDlNFXbEeebnktu+pGHMUT35VAbvwxMNBuBhD/JlYpq23zDb35X9t5Sa2ctsyYRw5VxCEViHgowdviQC0dlpqRG+u5JCMAiR1kb14fwCAAS4k+wAAAA==

:recycle: This comment has been updated with latest results.
author_association: NONE