Package: dtuhpc
DTU HPCDTU HPC is a collection of scripts and tools for running jobs on the DTU HPC cluster.
It should help you to get started with running jobs on the cluster, and to make your life easier.InstallationTo install just run:pipinstalldtuhpcGetting startedTo get started you first need to run:dtuhpcauthIt will ask you for your username and password for DTU, and it will then
ask you for an encryption password. This password is used to encrypt your
DTU password, so that it can be stored on your computer. You will need to
remember this password, as it is used to decrypt your password when you
run commands.Afterwards, you should create a configuration file for your project. This
should be named.dtuhpc.tomland should be placed in the root of your project.
You can use the following template:[ssh]user="<username>"host="<login-host>"default_cwd="<default working directory>"key_filename="<path to ssh key>"[github]access_token="<github access token>"[project]name="<project name>"path="<path to project on cluster>"default_deploy_branch="master"Thesshsection is used to configure the ssh connection to the cluster.
The GitHub access token can be generated from the followingpage.Setup projectTo set up a project, you can run:dtuhpcinit[--poetry][--custom-job=<pathtojobscript>]This will dispatch a job to the cluster, which will clone your project, create a
virtual environment, and install the dependencies.
You can choose to use either poetry, pip, or a custom job script. How to define jobs
will be explained in the next section.Writing jobsJobs are defined as toml files. It contains numerous options:name = "<name of job>"
queue = "<queue name>"
single_host = <true/false>
walltime = { hours = <hours>, minutes = <minutes> }
standard_output = "<path to standard output file>"
error_output = "<path to error output file>"
memory = <memory to allocate>
memory_kill_limit = <memory kill limit>
cores = <number of cores to allocate>
email = "<email address>"
notification_start = <true/false>
notification_end = <true/false>
core_block_size = <core block size>
core_p_tile_size = <core p tile size>
use_gpu = { num_of_gpus = <number of gpus>, per_task = <true/false> }

commands = [
    "<bash command 1>",
    "<bash command 2>",
    ...
]An example of a script can be seen here:queue="hpc"name="init_${{ project_name }}"walltime={hours=0,minutes=15}single_host=truecpu=2memory=4standard_output="init_${{ project_name }}.out"error_output="init_${{ project_name }}.err"commands=["git clone ${{ git_url }} ${{ project_path }}","module load python3/3.10.7","cd ${{ project_path }}","python3 -m venv ${{ project_path }}/venv","source ${{ project_path }}/venv/bin/activate","pip3 install 'poetry==1.3.2'","poetry install",]In this script, we can see that we can use variables in the script. These variables
are some default ones that are only available for theinitjob.Deploying jobsTo deploy a job you just run the following command:dtuhpcdeploy<job_path>It will then ask you to pick from branches or PR's. It will then dispatch the job
to the cluster.Other commandsSome other commands:Exec commands on clusterTo execute commands on the cluster, you can run:dtuhpcexec'<command to run>'It will run in the default working directory, which is defined in the configuration file.SSH into clusterTo ssh into the cluster, you can run:dtuhpcsshIt will then open an ssh connection to the cluster. From here you can run commands
as you would normally.Predefined subcommandsThere are also some predefined subcommands, which are just wrappers around the
cluster commands. They are all prefixed bydtuhpc c <command_name>. To get the
full documentation for the commands, you can run:dtuhpcc<command_name>--helpbkillKill a job on the cluster.dtuhpccbkill<job_id>bqueuesList all queues on the cluster.dtuhpccbqueuesbstatGet the status of a job on the cluster.dtuhpccbstat<optionaljob_id>bsubSubmit a job to the cluster.dtuhpccbsub<pathtojobscript>nodestatGet the status of the nodes on the cluster.dtuhpccnodestatshowstartShow the start time of a job on the cluster.dtuhpccshowstart<job_id>