semvec/pypi-packages huggingface dataset:
jobs-supervisor 0.3.11
Tools to supervise slurm jobs
License: MIT License

## Introduction

A tool to supervise jobs status via Feishu bot messages.

Currently supportted job states are : 'fail', 'complete', 'unavailable', 'cancel', 'running', 'pending'. If your job does not belong to those states, the reportted status will be 'unknown'. 

Once you init a supervisor, it will be stopped if all its jobs are one of ['fail', 'complete', 'unavailable', 'cancel', 'unknown'], and will report all its jobs' status to you via your Feishu bot. 

## Setup Guide

1. Get webhook from your Feishu Bot. 

2. Export and set webhook 'JOBS_SUPERVISOR_WEBHOOK' in your ~/.basrc. For example:
``export JOBS_SUPERVISOR_WEBHOOK=https://open.feishu.cn/your-webhook``

3. Export and set the directory 'JOBS_SUPERVISOR_LOGDIR' to save your sbatch output in your ~/.basrc. For example:
``export JOBS_SUPERVISOR_LOGDIR=path_to_log_files``

3. Run `source ~/.basrc`

4. (Optional) Create an enviornment for the tools by `conda create -n env_tools python=3.8` 

5. Activate your enviornment by `conda activate env_tools`

6. Install this python package in the enviornment by `pip3 install -i https://pypi.org/project/ jobs-supervisor==0.3.11`


## Usage

Assume 'jobid_start' and 'jobid_end' are int.

1. Activate your enviornment by `conda activate env_tools`

2. In terminal:
``supervise_jobs jobid_start jobid_end``

3. You can run `supervise_jobs --help` for more configs. By default,

- it will skip to report 'unavailable' jobs (jobs that you cannot get status). Set the option `--show_unavailable` to report unavailable jobs. 

- it will only report once all jobs are stopped. If you want to report every time a job stops, set the option `--show_intermedia`