Saravanan666/pypi-org huggingface dataset:
Package: slurmray
SLURM_RAY👉Full documentationDescriptionSlurmRayis a module for effortlessly distributing tasks on aSlurmcluster using theRaylibrary.SlurmRaywas initially designed to work with theCurnaglcluster at theUniversity of Lausanne. However, it should be able to run on anySlurmcluster with a minimum of configuration.InstallationSlurmRayis designed to run both locally and on a cluster without any modification. This design is intended to allow work to be carried out on a local machine until the script seems to be working. It should then be possible to run it using all the resources of the cluster without having to modify the code.pipinstallslurmrayUsagefromslurmray.RayLauncherimportRayLauncherimportrayimporttorchdeffunction_inside_function():withopen("slurmray/RayLauncher.py","r")asf:returnf.read()[0:10]defexample_func(x):result=(ray.cluster_resources(),f"GPU is available :{torch.cuda.is_available()}",x+1,function_inside_function(),)returnresultlauncher=RayLauncher(project_name="example",# Name of the project (will create a directory with this name in the current directory)func=example_func,# Function to executeargs={"x":1},# Arguments of the functionfiles=["slurmray/RayLauncher.py"],# List of files to push to the cluster (file path will be recreated on the cluster)modules=[],# List of modules to load on the curnagl Cluster (CUDA & CUDNN are automatically added if use_gpu=True)node_nbr=1,# Number of nodes to useuse_gpu=True,# If you need A100 GPU, you can set it to Truememory=8,# In MegaBytesmax_running_time=5,# In minutesruntime_env={"env_vars":{"NCCL_SOCKET_IFNAME":"eno1"}},# Example of environment variableserver_run=True,# To run the code on the cluster and not locallyserver_ssh="curnagl.dcsr.unil.ch",# Address of the SLURM serverserver_username="hjamet",# Username to connect to the serverserver_password=None,# Will be asked in the terminal)result=launcher()print(result)Launcher documentationThe Launcher documentation is availablehere.
Package: slurm-sbalance
slurm-sbalancesbalanceis a Slurm utility for checking account balance. The utility calculates the remainingservice unitsorSUleft in the account.PrerequisitesSlurm with Accounting enabledCurrentlysbalancehas been tested with:Python 2.7.5 and Python 3.6.6Slurm 18.08.5Usageusage: sbalance [-h] [-V] [-k] [-m] [-v]

Query slurm account balance.

optional arguments:
  -h, --help     show this help message and exit
  -V, --version  show program's version number and exit
  -k             show output in kSU (1,000 SU)
  -m             show output in MSU (1,000,000 SU)
  -v, --verbose  verbose mode (multiple -v's increase verbosity)Setting up Slurmsbalancecurrently support following setup.Balance is limited per accountAccount limit is set through QoS withGrpTRESMinsandNoDecayflag.Following is an example setupCreating accounttutorialand QoStestwith billing balance of 1000000sacctmgr add qos test set GrpTRESMins=billing=1000000 Flags=NoDecay 
sacctmgr add account tutorial set QoS=test DefaultQoS=testAddtestuser to accounttutorialsacctmgr add user test set Account=tutorialChecking balance fortestuser[test@localhost ~]$ sbalance 
Account balances for user: test
tutorial:
	Allocation:               1000000.00 SU
	Remaining Balance:         991896.00 SU ( 99.19%)
	Used:                        8104.00 SU
Package: slurm-script
slurm_scriptPython Slurm command generatorThis package is designed to generate slurm script to submitmpijobs to a cluster.InstallationYou can install the package usingpip.python-mpipinstallslurm_scriptUsage$sjob--h
slurm_scriptversion:0.1.7
usage:slurm_script/sjob[-h][-nNPROC][-jJOB_NAME][-tTIME][-mMEM_PER_CPU][-cCOMMAND[COMMAND...]][-mailMAIL_TYPE][-ntNTASKS][-cntCPUS_PER_TASK][-aADDITIONAL_CMD[ADDITIONAL_CMD...]]pythoninterfacetogenerateandrunslurmcommand.

options:-h,--helpshowthishelpmessageandexit-nNPROC,--nprocNPROCNumberofprocessorstorunthejob.-jJOB_NAME,--job_nameJOB_NAMENameofthejob.-tTIME,--timeTIMEMaximumruntime[hours:minutes:second].-mMEM_PER_CPU,--mem_per_cpuMEM_PER_CPUMemorypercore[MB].-cCOMMAND[COMMAND...],--commandCOMMAND[COMMAND...]Programcommand.-mailMAIL_TYPE,--mail_typeMAIL_TYPEEmailnotificationateitherBEGIN,END,orFAIL.-ntNTASKS,--ntasksNTASKSNumberoftasks.-cntCPUS_PER_TASK,--cpus_per_taskCPUS_PER_TASKNumberofcpuspertask.-aADDITIONAL_CMD[ADDITIONAL_CMD...],--additional_cmdADDITIONAL_CMD[ADDITIONAL_CMD...]Additionalcommands.NoteIt seems like due to a security reason, it is not possible to usesjobcommand directly from the cluster.
Therefore, you can use the following command insteadpython-mslurm_script--hExample$sjob-n10-jtest_run-t10:00:00-m1024-mailBEGIN,END,FAIL-nt1-cnt10-amoduleaddpython-cpythontest.py
slurm_scriptversion:0.1.6

Previewofthegeneratedscript:
--------------------------------#!/bin/bash#SBATCH -n 10#SBATCH --job-name=test_run#SBATCH --time=10:00:00#SBATCH --mem-per-cpu=1024#SBATCH --mail-type=BEGIN,END,FAIL#SBATCH --ntasks=1#SBATCH --cpus-per-task=10moduleaddpython

mpirunpythontest.py
--------------------------------
Doyouwanttorunthescript?[y/n]
Package: slurm-time
slurm_timeShow slurm job time remaining in the jupyterlab interfaceThis extension is composed of a Python package namedslurm_timefor the server extension and a NPM package namedslurm-timefor the frontend extension.RequirementsJupyterLab >= 3.0InstallTo install the extension, execute:pipinstallslurm_timeUninstallTo remove the extension, execute:pipuninstallslurm_timeTroubleshootIf you are seeing the frontend extension, but it is not working, check
that the server extension is enabled:jupyterserverextensionlistIf the server extension is installed and enabled, but you are not seeing
the frontend extension, check the frontend extension is installed:jupyterlabextensionlistContributingDevelopment installNote: You will need NodeJS to build the extension package.Thejlpmcommand is JupyterLab's pinned version ofyarnthat is installed with JupyterLab. You may useyarnornpmin lieu ofjlpmbelow.# Clone the repo to your local environment# Change directory to the slurm_time directory# Install package in development modepipinstall-e.# Link your development version of the extension with JupyterLabjupyterlabextensiondevelop.--overwrite# Server extension must be manually installed in develop modejupyterserverextensionenableslurm_time# Rebuild extension Typescript source after making changesjlpmbuildYou can watch the source directory and run JupyterLab at the same time in different terminals to watch for changes in the extension's source and automatically rebuild the extension.# Watch the source directory in one terminal, automatically rebuilding when neededjlpmwatch# Run JupyterLab in another terminaljupyterlabWith the watch command running, every saved change will immediately be built locally and available in your running JupyterLab. Refresh JupyterLab to load the change in your browser (you may need to wait several seconds for the extension to be rebuilt).By default, thejlpm buildcommand generates the source maps for this extension to make it easier to debug using the browser dev tools. To also generate source maps for the JupyterLab core extensions, you can run the following command:jupyterlabbuild--minimize=FalseDevelopment uninstall# Server extension must be manually disabled in develop modejupyterserverextensiondisableslurm_time
pipuninstallslurm_timeIn development mode, you will also need to remove the symlink created byjupyter labextension developcommand. To find its location, you can runjupyter labextension listto figure out where thelabextensionsfolder is located. Then you can remove the symlink namedslurm-timewithin that folder.Testing the extensionServer testsThis extension is usingPytestfor Python code testing.Install test dependencies (needed only once):pipinstall-e".[test]"To execute them, run:pytest-vv-rap--covslurm_timeFrontend testsThis extension is usingJestfor JavaScript code testing.To execute them, execute:jlpm
jlpmtestIntegration testsThis extension uses Playwright for the integration tests (aka user level tests).
More precisely, the JupyterLab helperGalatais used to handle testing the extension in JupyterLab.More information are provided within theui-testsREADME.Packaging the extensionSeeRELEASE
Package: slurmtoppy
slurmtoppyA console-basedSLURMjob monitoring tool.Whathtopis forps,slurmtoppyis forsqueue.InstallationpippipinstallslurmtoppyThere are no dependencies, except of standard SLURM commands.RunningAfter installation:slurmtopUsing nix, without installation:nix run github:ischurov/slurmtoppyScreenshotFeaturesShow list of running jobs (a.k.a.watch squeue).Cancel the selected job (no job_id input needed!)View output of the selected job withtailorlessSSH to a node where the selected job runs (provided it's a one-node job)
Package: slurm-to-wandb
slurm-to-wandbMonitor SLURM jobs using Weights and Biases (wandb) 📊UsageOn your cluster, login to wandb using thecli, then run:pipinstallslurm-to-wandb
slurm_to_wandb<job_ids>... to upload allsacctinformation to wandb. Currently, uploads to a project called "peregrine" - support for configuring this is planned #4.<job_ids>can be multiple space-separated job ids. In the case multiple job id's match, all matched job ids are uploaded (useful, for example, when using job arrays).To construct a DataFrame with the information yourself, use the functionslurm_to_wandb.sacct_as_df(*job_ids, **sacct_args).sacct_argscan be any additional args to pass tosacct. Run it on the cluster:fromslurm_to_wandbimportsacct_as_dfdf=sacct_as_df("job_id_123","another_job_id",starttime="2021-05-20")dfYou can now upload the information to wandb however you like 🙌🏻AboutByJeroen Overschie.
Package: slurmtui
SlurmTUIA simple Terminal UI for monitoring SLURM jobs.Note: SlurmTUI requires slurm 21.08 or later for the Json output support.InstallationpipinstallslurmtuiUsageJust runslurmtuiin your terminal.Environment variables:UPDATE_INTERVAL: Refresh rate in seconds (default: 10)TAIL_LINES: Number of lines to show in the tail (default: -1, show all)FAQHow to select text in the App?JobUI is running a Textual app which puts your terminal in to application mode which disables clicking and dragging to select text. Most terminal emulators offer a modifier key which you can hold while you click and drag to restore the behavior you may expect from the command line. The exact modifier key depends on the terminal and platform you are running on.iTerm Hold the OPTION key.Gnome Terminal Hold the SHIFT key.Windows Terminal Hold the SHIFT key.Refer to the documentation for your terminal emulator, if it is not listed above.ContactWissam Antoun
Package: slurm-tui
Slurm TUISlurm Text User Interface using the Slurm REST API in Python.Free software: MIT licenseDocumentation:https://github.com/bihealth/slurm-tuiUsage Example# slurm-tui --server=http://hpc-slurmrestd scancel
[I 220401 14:43:28] args = {
    "verbose": false,
    "force_jwt_token": false,
    "api_version": "v0.0.37",
    "server": "http://hpc-slurmrestd",
    "command": "scancel"
    }
[I 220401 14:43:30] Which job to kill? Enter to select, Ctrl-C to cancel
    1317509 -- bash
    1317802 -- bash
    1320066 -- bash
    1320250 -- bash
    1320256 -- snakejob.ngs_mapping_bwa_run.86.sh
    1370797 -- foo
> 1370958 -- foo [Enter]
[I 220401 14:43:32] Deleted job 1370958
[I 220401 14:43:32] All done, have a nice day!Changelog0.1.0 (2022-04-01)Featuresadding simple graphical scancel (2bd6fb7)fetch JWT token on startup (6878fba)
Package: slurmutils
slurmutilsUtilities and APIs for interfacing with the Slurm workload manager.Featuresslurmutilsis a collection of various utilities and APIs to make it easier
for you and your friends to interface with the Slurm workload manager, especially if you
are orchestrating deployments of new and current Slurm clusters. Gone are the days of
seething over incomplete Jinja2 templates. Current utilities and APIs shipped in theslurmutilspackage include:from slurmutils.editors import ...slurmconfig:  An editorslurm.confandIncludefiles.slurmdbdconfig: An editor forslurmdbd.conffiles.InstallationOption 1: Install from PyPI$python3-mpipinstallslurmutilsOption 2: Install from sourceWe use thePoetrypackaging and dependency manager to
manage this project. It must be installed on your system if installingslurmutilsfrom source.$gitclonehttps://github.com/canonical/slurmutils.git
$cdslurmutils
$poetryinstallUsageEditorsslurmconfigThis module provides an API for editing bothslurm.confandIncludefiles,
and can create new configuration files if they do not exist. Here's some common Slurm
lifecycle management operators you can perform using this editor:Edit a pre-existingslurm.confconfiguration filefromslurmutils.editorsimportslurmconfig# Open, edit, and save the slurm.conf file located at _/etc/slurm/slurm.conf_.withslurmconfig.edit("/etc/slurm/slurm.conf")asconfig:delconfig.inactive_limitconfig.max_job_count=20000config.proctrack_type="proctrack/linuxproc"Add a new node to theslurm.conffilefromslurmutils.editorsimportslurmconfigfromslurmutils.modelsimportNodewithslurmconfig.edit("/etc/slurm/slurm.conf")asconfig:node=Node(NodeName="batch-[0-25]",NodeAddr="12.34.56.78",CPUs=1,RealMemory=1000,TmpDisk=10000,)config.nodes[node.node_name]=nodeslurmdbdconfigThis module provides and API for editingslurmdbd.conffiles, and can create newslurmdbd.conffiles if they do not exist. Here's some operations you can perform
on theslurmdbd.conffile using this editor:Edit a pre-existingslurmdbd.confconfiguration filefromslurmutils.editorsimportslurmdbdconfigwithslurmdbdconfig.edit("/etc/slurm/slurmdbd.conf")asconfig:config.archive_usage="yes"config.log_file="/var/spool/slurmdbd.log"config.debug_flags=["DB_EVENT","DB_JOB","DB_USAGE"]delconfig.auth_alt_typesdelconfig.auth_alt_parametersProject & CommunityTheslurmutilspackage is a project of theUbuntu HPCcommunity.
It is an open-source project that is welcome to community involvement, contributions, suggestions, fixes,
and constructive feedback. Interested in being involved with the development ofslurmutils?
Check out these links below:Join our online chatCode of ConductContributing guidelinesLicenseTheslurmutilspackage is free software, distributed under the GNU Lesser General Public License, v3.0.
See theLICENSEfile for more information.
Package: slurm-viewer
Slurm ViewerIntroductionView the nodes in a slurm cluster and view the queue.Settings{"debug":"True","server":"SERVER or NONE","nodelist":["List of nodes to view"],"node_columns":["short_name","gpu_tot","gpu_alloc","gpu_avail","gpu_type","gpu_mem","cpu_tot","cpu_alloc","cpu_avail","real_memory","alloc_mem","freemem"],"queue_columns":["nodelist","user","job_id","command","submit_time","start_delay","min_memory"]}
Package: slurm-workflow-utils
Slurm Workflow UtilsCollection of Python scripts to facilitate creation and dispatching of computational jobs via SLURM sbatch scripts.FeaturesTo following exported console scripts are available:workflow-builderworkflow-launcherCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2024-02-01)First release on PyPI.
Package: slurmworm
slurmwormThis is a rather simple tool to monitor SLURM emails and alert you via a telegram bot about your jobs.Note!This is under heavy development!Installationinstallimapclientdirectly from the github reponotpipclone this repo and runpythonsetupinstallcreate a directory~/.config/slurmwormin this directory you need to place two files.one is the telegram bot info for theyourbot that you mustcreatenamedaccess.ymlthe other is a UNIX conf file with your IMAP email info namedimap_monitor.iniwhich identifies the email account to which you will send slurm messagesTelegram bot setupIf you read the above guide to create your bot you will have nearly everything you need. You then need to get that info into youraccess.ymlfile:token:'<yourbotIDtoken>'chat_id:'<thegroupIDwhereyourbotwillsendmessages>'You should have your bot chat to a group you have created.email setupThis is what should go inimap_monitor.ini:[imap]host=<your imap server>username=<username>password=<password>ssl=Truefolder=INBOX[path]download=<where mail will be temporarily stored>RunningAfter you have installed and set everything up, just typeslurmwormin a shell and it will run and monitor your email.It is likely best to have this running in a tmux shell you can disconnect from.May all your jobs exit 0!
Package: slurmy
SLURMY - Special handLer for Universal Running of Multiple jobs, Yes!Slurmy is a general batch submission module, which allows to define very general jobs to be run on batch system setups on linux computing clusters. Currently, only slurm is supported as backend, but further backends can easily be added. The definition of the job execution is done with a general shell execution script, as is used by most batch systems. In addition to the batch definition, jobs can also be dynamically executed locally, which allows for an arbitrary combination of batch and local jobs.Please have a look at thedocumentation pagefor more information.
Package: slurpit_netbox
NetBox Discovery PluginNetboxplugin to automatically discover your network withSlurp'it.CompatibilityNetBox VersionPlugin VersionNetBox 3.7>= 0.8.0InstallationThe plugin is available as a Python package inpypiand can be installed with pippip install slurpit-netboxEnable the plugin in /opt/netbox/netbox/netbox/configuration.py:PLUGINS = ['slurpit_netbox']Restart NetBox and addslurpit-netboxto your requirements.txtSeeNetBox Documentationfor detailsGetting startedOn ourgetting started pageyou can take an Online Course to understand how the plugin works, or play with the plugin in a simulated network in our Sandbox.
Package: slurry
An async stream processing microframework for PythonIntroductionSlurrybuilds on the concepts of structured concurrency and memory channels, originating inTrio, and uses them to create a microframework for processing streaming data.The basic building blocks of Slurry includes:Pipelines- An asynchronous context manager which encapsulates a stream process.Sections- The individual processing steps.Taps- Output channels for the processed stream.Extensions- A way to add more processing steps to an existing pipeline.Slurry avoids using asynchronous generator functions, in favor of the pull-push programming style
of memory channels. It can be thought of as an asynchronous version ofitertools- on steroids!Included in the basic library are a number of basic stream processing building blocks, likeMap,Chain,MergeandZip, and it is easy to build your own!DemonstrationEnough talk! Time to see what’s up!asyncwithPipeline.create(Zip(produce_increasing_integers(1,max=3),produce_alphabet(0.9,max=3)))aspipeline,pipeline.tap()asaiter:results=[itemasyncforiteminaiter]assertresults==[(0,'a'),(1,'b'),(2,'c')]The example producers (which are not part of the framework) could look like this:asyncdefproduce_increasing_integers(interval,*,max=3):foriinrange(max):yieldiifi==max-1:breakawaittrio.sleep(interval)asyncdefproduce_alphabet(interval,*,max=3):fori,cinenumerate(string.ascii_lowercase):yieldcifi==max-1:breakawaittrio.sleep(interval)Further documentation is available onreadthedocs. Check out thesource codeongithub.InstallationStill here? Wanna try it out yourself? Install fromPyPI:pip install slurrySlurry is tested on Python 3.8 or greater and requires theTrioconcurrency and IO library.LicenseSlurry is licensed under theMIT license.
Package: slush-list
Программное средство работы со списками слушателейПрограмма получает на вход тестовый файл формата .txt, читает его и загружает.Основные функции проекта:Загрузка списка слушателей;Отображение на экране загруженных списков слушателей;Вывод на экран ссылок на репозитории и группы слушателей;Поиск слушателя по шаблону fullname.Функции редактирования списков:Изменение fullname и login cлушателя;Идаление загруженной группы.Построение docker образаКоманду необходимо выполнить внутри каталога src/slush_list/docker build -t slush_list:1.0 .Запуск веб-приложения:docker run -d -p 5000:5000 slush_list:1.0Далее необходимо прописать в браузере адрес: 0.0.0.0:5000Опции для запуска из командной строки:-l (--load) <путь_к файлу> - читает указанную группу-y (--year) <год_поступления> - год поступления (необходимо для создания имени группы)-n (--number) <номер_группы> - номер группы (необходимо для создания имени группы)Пример с пробрасыванием файла:docker run -v path/to/file:/slush_list/file.txt slush_list:1.0 -l ./file.txt -y <год_поступления> -n <порядковый_номер_группы>Пример с файлов, находящимся в /slush_list/group_lists/2018-3.txt:docker run -it slush_list:1.0 -l /slush_list/group_lists/2018-3.txt -y 2018 -n 3-rl (--relogin) <старый_логин> <новый_логин> - изменение login слушателяdocker run -it slush_list:1.0 -l /slush_list/group_lists/2018-3.txt -y 2018 -n 3 -rl nazar_lex nazarov-rn (--rename) <логин> <новый_fullname> - изменение fullname слушателяdocker run -it slush_list:1.0 -l /slush_list/group_lists/2018-3.txt -y 2018 -n 3 -rn nazar_lex 2018-3-33-nazarov-r (--repo) <номер_группы> <репозиторий> - отображение ссылок на репозитории слушателейdocker run -it slush_list:1.0 -l /slush_list/group_lists/2018-3.txt -y 2018 -n 3 -r 2018-3 timp-g (--group) <номер_группы> <группа> - отображение ссылок на группы слушателейdocker run -it slush_list:1.0 -l /slush_list/group_lists/2018-3.txt -y 2018 -n 3 -g 2018-3 prac-gs(--getstud)  - находит слушателя с указанным fullnamedocker run -it slush_list:1.0 -l /slush_list/group_lists/2018-3.txt -y 2018 -n 3 -gs 2018-3-20-nazЗапуск Pylintpylint main.py
pylint slush_list.py
pylint test_slush.pyЗапуск тестовpytest test_slush.pyПокрытие кодаpytest --cov=slush_listПостроение пакета для загрузки на PyPIНеобходимо выполнить данную команду из того же каталога, где находится pyproject.toml:python3 -m buildДанная команда генерирует два файла каталоге dist:dist/
  slush-list-VERSION.tar.gz
  slush_list-VERSION-py3-none-any.whlЗагрузка пакета на PyPIНеобходимо иметь учетную запись на PyPI для загрузки пакета.Выполните команду:python3 -m twine upload dist/*Установка пакетаВыполните команду:python3 -m pip install slush-listОбновление установленного пакетаpython3 -m pip install --upgrade slush-list
Package: slutil
slutilA modern slurm experience.A command line utility to view Slurm jobs. Written with rich and click.Usage: slutil [OPTIONS] COMMAND [ARGS]...

Commands:
  report  Get status of multiple jobs
  status  Get status of a slurm job.
  submit  Submit a slurm job.ContributingPushes tomainare forbidden, all changes must go through a PR before merging. All tests must pass for a PR to be merged. Code is to be formatted withblack. Built with poetry.submitAdd metadata to ansbatchcommand and store data in the databaseMustbe used to log jobs in the database.Usage: slutil submit [OPTIONS] SBATCH_FILE DESCRIPTION

  Submit a slurm job.

  SBATCH_FILE is a path to the .sbatch file for the job

  DESCRIPTION is a text field describing the job

Options:
  --help  Show this message and exit.reportView list of recent jobsCount parameter specifies the number of jobs to be displayed. Defaults to 10.Truncated to screen width by default,-vto enable word-wrap.Usage: slutil report [OPTIONS]

  Get status of multiple jobs

Options:
  -c, --count INTEGER
  -v, --verbose
  --help               Show this message and exit.statusDisplays the data on a specific jobUsage: slutil status [OPTIONS] SLURM_ID

  Get status of a slurm job.

  SLURM_ID is the id of the job to check.

Options:
  --help  Show this message and exit.