Package: hdeeprm
Framework for evaluatingworkload managementpolicies based ondeep reinforcement learningforheterogeneousclusters.OverviewHDeepRMis a Python framework for evaluating workload management policies
based on deep reinforcement learning for heterogeneous clusters. It
leverages theBatsim ecosystemfor simulating a heterogeneous workload management context. This is composed
of thesimulator,Batsimand thedecision system,PyBatsim.HDeepRM provides a heterogeneity layer on top of PyBatsim, which adds support
for user-defined resource hierarchies. Memory capacity and bandwidth conflicts
are added along with interdependence when consolidating or scattering jobs across
the data centre.It offers a flexible API for developing deep reinforcement learning agents.
These may be trained by providing real workload traces inSWF formatalong
with platforms defined in the format specified inPlatforms. They can
be further evaluated and tested against classic policies.Installation PrerequisitesHDeepRM is distributed as a Python package onPyPi.
In order to download and install it, the following software is needed:Python3.6+, find your OS in thisinstallation guide.Pip, the Python package manager. If not already available with the Python
installation, follow theofficial guide.InstallationFor installing HDeepRM, just download the package from PyPi:pipinstall--upgrade--userhdeeprmIfpipis mapped to Python 2.x, try:pip3install--upgrade--userhdeeprmWhen working with multiple Python versions, use:python3.6-mpipinstall--upgrade--userhdeeprmThis should download thehdeeprmpackage with all its dependencies,
which are:defusedxml>= 0.5.0: secure XML generation and parsing.gym>= 0.12.0: environment, actions and observations definitions.lxml>= 4.3.2: generation of the XML tree. Backend fordefusedxml.numpy>= 1.16.2: efficient data structure operations.procset>= 1.0: closed-interval sets for resource selection.pybatsim>= 3.1.0: decision system and main interface to interact
with Batsim.torch>= 1.0.1.post2: deep learning library for agent definition.Usage PrerequisitesThe simulation side is done by Batsim, which is needed in order to run
HDeepRM experiments. Follow theofficial installation docsfor instructions.Launching experimentsIn order to experiment with HDeepRM, an integrated launcher is provided:hdeeprm-launch-a<agent.py>-cw<custom_workload.json>-im<saved_model.pt>-om<to_save_model.pt><options.json>Theoptions.jsonspecifies the experiment parameters. The JSON structure
is as follows:{"seed":0,"nb_resources":0,"nb_jobs":0,"workload_file_path":"","platform_file_path":"","pybatsim":{"log_level":"","env":{"objective":"","actions":{"selection":[{"":[]}],"void":false},"observation":"","queue_sensitivity":0.0,},"agent":{"type":"","run":"","hidden":0,"lr":0.0,"gamma":0.0}}}Global options:seed- The random seed for evaluation reproducibility.nb_resources- Total number of cores in the simulated platform.nb_jobs- Total number of jobs to generate in the workload.workload_file_path- Location of the original SWF formatted workload.platform_file_path- Location of the original
HDeepRM JSON formatted platform.PyBatsim options:log_level- Logging level for showing insights from the simulation. SeeLoggingfor reference on possible values.PyBatsim - Environment options:objective- Metric to be optimised by the agent. SeeObjectivesfor an explanation and recognised values.actions- Subset of actions for the simulation. If not specified, all 37 actions in HDeepRM are used.observation- Type of observation to use, one ofnormal,smallorminimal.queue_sensitivity- Sensitivity of the observation to variations in job queue size. SeeHyperparameters - Queue Sensitivity.PyBatsim - Common agent options:type- Type of the scheduling agent, one ofCLASSICorLEARNING.PyBatsim -Learningagent options:run- Type of run for the learning agent, one oftrainortest.
When training, the agent’s inner model is updated,
whereas testing is meant for evaluation purposes.hidden- Number of units in each hidden layer from the agent’s inner model. SeeHyperparameters - Hidden units.lr- Learning rate for updating the agent’s inner model. SeeHyperparameters - Learning rate.gamma- Discount factor for rewards. SeeHyperparameters - Reward Discount Factor.This is an example of anoptions.jsonfile
for a classic agent:{"seed":2009,"nb_resources":175,"nb_jobs":1000,"workload_file_path":"/workspace/workloads/my_workload.swf","platform_file_path":"/workspace/platforms/my_platform.json","pybatsim":{"log_level":"DEBUG","env":{"objective":"avg_utilization","actions":{"selection":[{"shortest":["high_mem_bw"]}],"void":false},"observation":"normal","queue_sensitivity":0.05},"agent":{"type":"CLASSIC"}}}This is another example of anoptions.jsonfile,
in this case for a learning agent:{"seed":1995,"nb_resources":175,"nb_jobs":1000,"workload_file_path":"/workspace/workloads/my_workload.swf","platform_file_path":"/workspace/platforms/my_platform.json","pybatsim":{"log_level":"WARNING","env":{"objective":"makespan","actions":{"selection":[{"first":["high_gflops","high_mem_bw"]},{"smallest":[""]}],"void":false},"queue_sensitivity":0.01},"agent":{"type":"LEARNING","run":"train","hidden":128,"lr":0.001,"gamma":0.99}}}Optional command line arguments are available:-a- The file containing your developed learning agent for evaluation.
Seeagent examplesfor reference.-cw- If you are thinking about proof-of-concept experiments, you
may need to define your own workload. Doing this in SWF is tedious, thus
this option allows for passing a custom workload defined in Batsim JSON format.-im- PyTorch trained models are usually saved in.ptfiles. This
option allows for loading a previously trained model to bootstrap the agent.-om- If you want to save the model after the simulation is finished, specify
the output file in this option. This is usually combined withtrainruns.