Package: rm-runner
Remote RunnerRemote Runner is an easy pythonic way to migrate your python training scripts from a local environment to a powerful cloud-backed instance to efficiently scale your training, save cost & time, and iterate quickly on experiments in a parallel containerized way.How does Remote Runner work?Creating all required cloud resourcesMigrating your script to the remote machineExecuting your scriptmaking sure the instance is terminated again.Getting startedpipinstallrm-runnerPermissonsTo useEC2RemoteRunneryou need to following permissions:create/delete keypairscreate/delete secruity groupsadd inbound/ingress rules to security groupscreate/start/terminate instances (with ebs)Habana Gaudi examplefromrm_runnerimportEC2RemoteRunnerrunner=EC2RemoteRunner(instance_type="dl1.24xlarge",profile="hf-sm",region="us-east-1")runner.launch(command="hl-smi")expected output2022-07-2110:00:09,898|INFO|Foundcredentialsinsharedcredentialsfile:~/.aws/credentials2022-07-2110:00:10,812|INFO|Createdkeypair:rm-runner-abdk2022-07-2110:00:11,621|INFO|Createdsecuritygroup:rm-runner-abdk2022-07-2110:00:13,227|INFO|Launchedinstance:i-03dcc3b5f53cb946a2022-07-2110:00:13,230|INFO|Waitingforinstancetobeready...2022-07-2110:00:29,252|INFO|Instanceisready.PublicDNS:ec2-3-93-4-123.compute-1.amazonaws.com2022-07-2110:00:29,267|INFO|Settingupsshconnection...2022-07-2110:01:49,292|INFO|Settingupsshconnection...2022-07-2110:02:05,434|INFO|Settingupsshconnection...2022-07-2110:02:10,542|INFO|Settingupsshconnection...2022-07-2110:02:10,766|INFO|Connected(version2.0,clientOpenSSH_8.2p1)2022-07-2110:02:11,840|INFO|Authentication(publickey)successful!2022-07-2110:02:11,840|INFO|Pullingcontainer:vault.habana.ai/gaudi-docker/1.4.1/ubuntu20.04/habanalabs/pytorch-installer-1.10.2:1.4.1-11...2022-07-2110:02:20,460|INFO|Executing:dockerrun--runtime=habana-eHABANA_VISIBLE_DEVICES=all-eOMPI_MCA_btl_vader_single_copy_mechanism=none--cap-add=sys_nice--net=host--ipc=host-v/home/ubuntu:/home/ubuntu/rm-runner--workdir=/home/ubuntu/rm-runnervault.habana.ai/gaudi-docker/1.4.1/ubuntu20.04/habanalabs/pytorch-installer-1.10.2:1.4.1-11hl-smi
+-----------------------------------------------------------------------------+|HL-SMIVersion:hl-1.4.1-rc-fw-35.0.2.0||DriverVersion:1.4.0-d8f95f4||-------------------------------+----------------------+----------------------+|AIPNamePersistence-M|Bus-IdDisp.A|VolatileUncorr.ECC||FanTempPerfPwr:Usage/Cap|Memory-Usage|AIP-UtilComputeM.||===============================+======================+======================||0HL-205N/A|0000:20:1d.0N/A|0||N/A35CN/A102W/350W|512Mib/32768Mib|2%N/A||-------------------------------+----------------------+----------------------+|1HL-205N/A|0000:a0:1d.0N/A|0||N/A36CN/A101W/350W|512Mib/32768Mib|1%N/A||-------------------------------+----------------------+----------------------+|2HL-205N/A|0000:a0:1e.0N/A|0||N/A33CN/A105W/350W|512Mib/32768Mib|3%N/A||-------------------------------+----------------------+----------------------+|3HL-205N/A|0000:90:1d.0N/A|0||N/A32CN/A97W/350W|512Mib/32768Mib|0%N/A||-------------------------------+----------------------+----------------------+|4HL-205N/A|0000:90:1e.0N/A|0||N/A35CN/A101W/350W|512Mib/32768Mib|1%N/A||-------------------------------+----------------------+----------------------+|5HL-205N/A|0000:10:1d.0N/A|0||N/A34CN/A93W/350W|512Mib/32768Mib|0%N/A||-------------------------------+----------------------+----------------------+|6HL-205N/A|0000:10:1e.0N/A|0||N/A36CN/A108W/350W|512Mib/32768Mib|4%N/A||-------------------------------+----------------------+----------------------+|7HL-205N/A|0000:20:1e.0N/A|0||N/A33CN/A101W/350W|512Mib/32768Mib|1%N/A||-------------------------------+----------------------+----------------------+|ComputeProcesses:AIPMemory||AIPPIDTypeProcessnameUsage||=============================================================================||0N/AN/AN/AN/A||1N/AN/AN/AN/A||2N/AN/AN/AN/A||3N/AN/AN/AN/A||4N/AN/AN/AN/A||5N/AN/AN/AN/A||6N/AN/AN/AN/A||7N/AN/AN/AN/A|+=============================================================================+2022-07-2110:04:00,641|INFO|Terminatinginstance:i-03dcc3b5f53cb946a2022-07-2110:05:48,297|INFO|Deletingsecuritygroup:rm-runner-abdk2022-07-2110:05:49,891|INFO|Deletingkey:rm-runner-abdk2022-07-2113:29:12,489|INFO|Totaltime:302s2022-07-2113:29:12,489|INFO|Startuptime:165s2022-07-2113:29:12,490|INFO|Executiontime:43s2022-07-2113:29:12,490|INFO|Terminationtime:94s2022-07-2113:29:12,490|INFO|Estimatedcost:$1.1