Azure Machine Learning Service — Train a model | by Pankaj Jainani | Towards Data Science

So far in this series of articles, I was able to cover a brief introduction of the Azure Machine Learning Service (AML) and also able to run a simple experiment. In this post, I am going to cover the parameterization of the experiment script, different ways to configure the script environment, framework, and dependencies. Also, how to register the model with AML service after the training is completed.
This particular post is extracted from the Kaggle notebook hosted — here. Use the link to setup to execute the experiment.
Before I jump into a discussion about data stores and data set pipelines in AML, here I will briefly touch upon some advanced configuration options to run a simple experiment as described earlier.
In the post# 2, I discussed implementing a simple ML experiment using the custom Python script, but that script file had one issue, it was plain and static, so how can we improve the script for the experiment so that it can accept a range of values and train a model effectively? Yes, the answer is to parameterized the script — so in the context of AML service the parameter values for any experiment script can be implemented as below:
### .... ###run = Run.get_context()##Adding the parameter: regularization rate parser = argparse.ArgumentParser()parser.add_argument('--reg_rate', type=float, dest='reg', default=0.01)args=parser.parse_args()r = args.regrun.log("model regularization", np.float(r))# load the data from a local filedata = pd.read_csv('IRIS.csv')X = data[['sepal_length', 'sepal_width','petal_length','petal_width']].valuesX=StandardScaler().fit_transform(X)Y= (data['species']).map(lambda x: 0 if x=='Iris-setosa' else (1 if x=='Iris-versicolor' else 2))#Split data into train and test setX_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size=0.25, random_state=1234)# fit the model: with extra parametermodel = LogisticRegression(C=1/r, solver='lbfgs', multi_class='multinomial').fit(X_train,Y_train)Y_pred = model.predict(X_test)accuracy = np.average(Y_test == Y_pred)print("accuracy: " + str(accuracy))run.log("Accuracy", np.float(accuracy))###....###
While Run Configuration and Script Run Configuration allows a developer to run a script-based experiment that trains a machine learning model, the AML service also comes with a better abstraction that encapsulates their capabilities into a single object, known as Estimator. The typical use case is when the Run Configuration may become too complex to configure for certain dependencies and framework. It is available for the most commonly used machine learning framework, i.e., Scikit-Learn, PyTorch, and Tensorflow.
The code below implements the generic Estimator class’ object to run the training the experiment. The Estimator()constructor parameterizes the references for compute resources for experiment execution, e.g. Container, VM, or local compute by using compute_targetparameter. The generic Estimator does not include the scikit-learn packages, therefore, we require to pass the value for the conda_packagesparameter. Also, see that we can pass the collection of script’s parameters by using the script_paramsparameter of Estimator()the constructor method.
from azureml.train.estimator import Estimatorfrom azureml.core import Experimentfrom azureml.widgets import RunDetails# Create an estimatorestimator = Estimator(source_directory=experiment_folder,                      entry_script='iris_simple_experiment.py',                      compute_target='local',                      use_docker=False,                      script_params = {'--reg_rate': 0.07},                      conda_packages=['scikit-learn']                      )# COMMENTED - SKLearn as an estimator#estimator = SKLearn(source_directory=experiment_folder, entry_script='iris_simple_experiment.py', compute_target='local', use_docker=False)# Create an experimentexperiment_name = 'iris-estimator-experiment'experiment = Experiment(workspace = ws, name = experiment_name)# Run the experiment based on the estimatorrun = experiment.submit(config=estimator)# Get Run DetailsRunDetails(run).show()# Wait to complete the experiment. In the Azure Portal we will find the experiment state as preparing --> finished.run.wait_for_completion(show_output=True)
The AML SDK includes framework-specific estmators Class — it allows developer to easily configure framework dependencies,e.g, for SKLearn or Tensorflow or PyTorch. The code also has a example for SKLearn Estimator (commented).
after completing the training of the ML model we have to use it for inference, therefore, with AML service we can register the model and consume it for inference whenever required.
Download Model: The run object’s download_file()ordownload_files() methods are used to download output files to the local file system.
# List the files generated by the experimentfor file in run.get_file_names():     print(file)# Download a named filerun.download_file(name='outputs/model.pkl', output_file_path='model.pkl')
To register the model and the associated meta-data — name, version, tags we can write the following:
from azureml.core import Modelrun.register_model(model_path='outputs/iris_simple_model.pkl', model_name='iris_estimator_model', tags={"Training Context":"Estimator", "Script Context":"Parameters"},                  properties={"Accuracy":run.get_metrics()["Accuracy"]})
Finally, to view the models registered with the AML service, one can use the following:-
for model in Model.list(ws):    print(model.name, ":", model.version)    print("\nTag List")    for tag_name in model.tags:        tag = model.tags[tag_name]        print(tag_name, tag)    print("\nProperties List")    for prop_name in model.properties:        prop = model.properties[prop_name]        print(prop_name,prop)Outputiris_estimator_model : 4Tag ListTraining Context EstimatorScript Context ParametersProperties ListAccuracy 0.9736842105263158IRIS_model : 1Tag ListTraining context Pipeline
With the end of this post, so far we are able to cover the most basic aspects of AML service. We are able to run a simple experiment and train an ML model. Also, using APIs like Run Configuration, Script Run Configuration, and Estimators allows us to create an environment with associated dependencies to dynamically run the experiments irrespective of the computing environment.
This is a series of blog posts encompassing a detailed overview of various Azure Machine Learning capabilities, the URLs for other posts are as follows:
Post 1: Azure Machine Learning Service: Part 1 — An Introduction
Post 2: Azure Machine Learning Service — Run a Simple Experiment
Post 3 (this): Azure Machine Learning Service — Train a model
Post 4: Azure Machine Learning Service — Where is My Data?
Post 5: Azure Machine Learning Service — What is the Target Environment?
Connect with me on LinkedIn to discuss further
[1] Notebook & Code — Azure Machine Learning — Model Training, Kaggle.[2] Azure Machine Learning — Estimator API, URL[3] Azure Machine Learning Service Official Documentation, Microsoft Azure.