Package: lmn
🍋 LMN - A minimal launcherA lightweight tool to run your local project on a remote machine, with a single command.lmncan set up a container environment (Docker or Singularity) and can work with job schedulers (Slurm or PBS).🧑‍💻lmnin action:In your project directory, you can simply run$lmnrun<remote-machine>--pythongenerate_fancy_images.pyIt does:rsync your local project directorywith the remote machinessh into the remote machine, (optionally) set up the specified env (Docker or Singularity)Inside of the environment,lmnrunspython generate_fancy_images.pyIf files are created (such as images),lmnrsync the generated files back to the local project directory🚀 Quickstart▶️  InstallationYou can installlmnvia pip:$pipinstalllmn▶️  ConfigurationAll you need is to place a single configuration file atproject_dir/.lmn.json5(on your local machine).The config file looks like:Example Configuration{
    "project": {
        "name": "my_project",
        // What not to rsync with the remote machine:
        "exclude": [".git", ".venv", "wandb", "__pycache__"],
        // Project-specific environment variables:
        "environment": {
            "MUJOCO_GL": "egl"
        }
    },
    "machines": {
        "elm": {
            // Host information
            "host": "elm.ttic.edu",
            "user": "takuma",
            // Rsync target directory (on the remote host)
            "root_dir": "/scratch/takuma/lmn",
            // Mode: ["ssh", "docker", "slurm", "pbs", "slurm-sing", "pbs-sing"]
            "mode": "docker",
            // Docker configurations
            "docker": {
                "image": "ripl/my_transformer:latest",
                "network": "host",
                // Mount configurations (host -> container)
                "mount_from_host": {
                    "/ripl/user/takuma/project/": "/project",
                    "/dev/shm": "/dev/shm",
                },
            },
            // Host-specific environment variables
            "environment": {
                "PROJECT_DIR": "/project",
            },
        },
        "tticslurm": {
            "host": "slurm.ttic.edu",
            "user": "takuma",
            "mode": "slurm-sing",  // Running a Singularity container on a cluster with Slurm job scheduler
            "root_dir": "/share/data/ripl-takuma/lmn",
            // Slurm job configurations
            "slurm": {
                "partition": "contrib-gpu",
                "cpus_per_task": 1,
                "time": "04:00:00",
                "output": "slurm-%j.out.log",
                "error": "slurm-%j.error.log",
                "exclude": "gpu0,gpu18",
            },
            // Singularity configurations
            "singularity": {
                "sif_file": "/share/data/ripl-takuma/singularity/my_transformer.sif",
                "writable_tmpfs": true,
                "startup": "ldconfig /.singularity.d/libs",  // Command to run after starting up the container
                "mount_from_host": {
                    "/share/data/ripl-takuma/project/": "/project",
                },
            },
            "environment": {
                "PROJECT_DIR": "/project",
            }
        }
    }
}More example configurations can be found inthe example directory.▶️  Command examplesMake sure that you're in the project directory first.# Launch an interactive shell in the docker container (on elm):$lmnrunelm--bash# Run a job in the docker container (on elm):$lmnrunelm--pythontrain.py# Run a script on the host (on elm):$lmnrunelm--modessh--pythonhello.py# Run a command quickly on the host without syncing any files ("bare"-run; on elm)$lmnbrunelm--hostname# Check GPU usage on elm (This is equivalent to `lmn brun elm -- nvidia-smi`)$lmnnvelm# Launch an interactive shell in the Singularity container via Slurm scheduler (on tticslurm)$lmnruntticslurm--bash# Run a job in the Singularity container via Slurm scheduler (on tticslurm)$lmnruntticslurm--pythontrain.py# Submit a batch job that runs in the Singularity container via Slurm scheduler (on tticslurm)$lmnruntticslurm-d--pythontrain.py# Launching a sweep (batch jobs) that runs in the Singularity container via Slurm scheduler (on tticslurm)# This submits 10 batch jobs where `$LMN_RUN_SWEEP_IDX` is set from 0 to 9.$lmnruntticslurm--sweep0-10-d--pythontrain.py-l'$LMN_RUN_SWEEP_IDX'# Run a script on the login node (on tticslurm)$lmnruntticslurm--modessh--squeue-utakuma# Get help$lmn--help# Get help on `lmn run`$lmnrun--helpMore about `--sweep` format--sweep 0-10: ten jobs withLMN_RUN_SWEEP_IDX=0,1through9Internallylmnsimply runsrange(0, 10)--sweep 7: a single job withLMN_RUN_SWEEP_IDX=7--sweep 3,5,8:  three jobs withLMN_RUN_SWEEP_IDX=3and5and8Comparison with other packageslmnis inspired by the following great packages:geyang/jaynes,justinjfu/doodadandafdaniele/cpk.jaynesanddoodadfocus on launching a lot of jobs in non-interactive mode✅ support ssh, docker, slurm and AWS / GCP (but not PBS scheduler)😢 do not support Singularity😢 cannot launch interactive jobs😢 only work with Python project, and require (although small) modifications to the project codebasecpkfocuses on (though not limited to) ROS application and running programs in docker containers✅ supports X forwarding and more stuff that are helpful to run ROS applications on the container✅ provides more functionalities such as creating and deploying ssh keys on remote machines😢 does not support clusters with schedulers (Slurm or PBS), nor does it support SingularityTasksUse Pydantic for configurationsadd ssh-release subcommand (to drop the persistent ssh connection)Remove project.name from the global config, and get project name from the project root directory name