SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In Cheetah-Dir, the agent controls a robotic cheetah morphology by outputting a control torque for each of six joints on the morphology. The task is to run forward or backward with as large a velocity as possible. The agent observes the position, angle, and velocity of each body part (17 dimensions in total) and is given a positive reward for its velocity in the direction given by the task and a negative reward for control costs (specifically,  $5\%$  of the magnitude of the action vector).
In Cheetah-Vel, the agent controls the same robot, but instead of the positive reward component for running on the selected direction, the agent receives a negative reward component equal to the absolute difference to a target velocity, which is sampled uniformly from [0,3].
In And-Dir, the agent controls a robotic ant morphology by outputting a control torque for each of eight joints on the morphology. The task is to run forward or backward, with as large a velocity as possible. The observations are 27 dimensional in this case, and the agent is given a positive reward for its velocity in the direction given by the task and a negative reward for control costs (specifically,  $5\%$  of the magnitude of the action vector). Additionally, the agent is given a negative contact reward  $(.05\%)$  of the magnitude of the external forces on each joint) and a positive survival reward (1 per timestep). Episodes in this environment additionally terminate if the body falls outside of a predefined height.
In Walker, the agent controls a two-legged torso morphology and outputs a control torque for each of six joints on the morphology. The observations are 17 dimensional for this environment. The tasks are defined as uniform samples of 65 different physics coefficients (e.g. body mass and friction) for the simulation. The agent is given a positive reward for the forward velocity, a positive reward of one reward per timestep, and a negative reward for the control costs (specifically,  $0.1\%$  of the magnitude
of the action vector). Episodes in this environment additionally terminate if the body falls outside of a predefined height or rotation range.
8.7 Compute
Experiments were run on four to eight machines simultaneously each with eight GPUs, ranging from NVIDIA GeForce GTX 1080Ti to NVIDIA RTX A5000. Each model was tuned over five learning rates with three seeds per learning rate. Each experiment took between 4 hours for grid-worlds and 3 days for MuJoCo experiments. In total, there were approximately 360 experiments for MuJoCo and 270 for grid-worlds in the main results. With about 5 experiments on average per GPU, this constitutes roughly 5,400 GPU hours.