Azure Machine Learning : Run Python Script Experiment | Towards Data Science

In Part 1 of this series I mostly covered an overview and introduction about Azure Machine Learning (AML) Service, in that post I covered setup, configuration, and various OOB capabilities on the offering by this cloud-managed service. I also discussed AML Workspace, AML Experiment, and AML service experiment’s dashboard in detail.
In this Part 2, I will discuss how we can create an independent experiment script to encapsulate the core implementation of an ML model on the cloud, i.e. by training and tracking through repetitive experiment run. Also, I will demonstrate how to set up a prerequisite environment and configurations required to execute an experiment script.
This particular post is extracted from the Kaggle notebook hosted — here. Use the link to setup and execute the experiment
Once connected to the Azure Machine Learning workspace, the next step is to define a model experiment script, this is a very general python script which is loaded and executed form the experiment’s ‘workspace’ compute context. This also requires that the data files’ root (folder) has to be at the same location where this script is loaded, as shown below —
Experiment Output Folder — Mostly, Run of an experiment generates some output files, e.g. a saved model, that are saved in the workspace’s outputs folder as shown below —
os.makedirs("outputs", exist_ok=True)joblib.dump(value=model, filename='outputs/iris_simple_model.pkl')# Complete the runrun.complete()
One can also use the Run object’s upload_file method in the experiment file code as shown below, this enables that any files written to the outputs folder in the compute context are automatically uploaded to the run’s outputs folder when the run completed, e.g. —
run.upload_file(name='outputs/IRIS.csv', path_or_stream='./sample.csv') # upload from local
After the script for implementing the ML model is ready, the next step is to define RunConfiguration object — which defines the Python environment in which the script will run, and ScriptRunConfig object— which associates the run environment with the script. The below code snippet instantiates the Python environment by calling the RunConfiguration() method and ScriptRunConfig() encapsulate the same environment for the script’s execution:
from azureml.core import Experiment, RunConfiguration, ScriptRunConfig# create a new RunConfig objectexperiment_run_config = RunConfiguration()experiment_run_config.environment.python.user_managed_dependencies = True# Create a script configsrc = ScriptRunConfig(source_directory=experiment_folder,                       script='iris_simple_experiment.py',                      run_config=experiment_run_config)
The RunConfig object also allows to additionally include the Python packages which are necessary for script execution.
Access the Azure Machine Learning Studio to navigate for the sample experiment run and verify the results in results on the dashboard. Here, the entire details of the experiment’s run history is displayed, as shown below — the details of run stats, history, results, metrics, logs, outputs, errors, diagnostics, etc.. are readily available from the dashboard:
In this part of the series, I tried to cover the most fundamental concept of Azure Machine Learning Service, i.e., to prepare and execute a machine learning experiment and generate a model binary. In the next article, I will cover a slightly more advanced way to set up and control the script’s execution environment, install or update the required dependencies & packages. So please stay tuned!
This is a series of blog posts encompassing a detailed overview of various Azure Machine Learning capabilities, the URLs for other posts are as follows:
Post 1: Azure Machine Learning Service: Part 1 — An Introduction
Post 2 (this): Azure Machine Learning Service — Run a Simple Experiment
Post 3: Azure Machine Learning Service — Train a model
Post 4: Azure Machine Learning Service — Where is My Data?
Post 5: Azure Machine Learning Service — What is the Target Environment?
Connect with me on LinkedIn to discuss further
[1] Notebook & Code — Azure Machine Learning — Introduction, Kaggle.[2] Azure Machine Learning Service Official Documentation, Microsoft Azure.