Saravanan666/pypi-org huggingface dataset:
Package: gpu-sentinel
GPU SentinelA Moonshine Labs toolOverviewIf you're automating training your large models in the cloud, cost control is critial. How many times have you accidentally left an expensive GPU instance running when the underlying job had crashed, costing you money or capacity with no benefit?GPU Sentinelis a simple tool that will watch your instance and automatically trigger when GPU utilization drops below a certain amount for a period of time. GPU Sentinel can automatically shutdown or reboot the instance, or simply end its own process so you can do an action yourself.Installation$ pip install gpu_sentinel
$ gpu_sentinel --helpUsageThe GPU sentinel has two states, IDLE and ARMED.When you start the program, it will wait for the GPU to be above a certain utilization for a set amount of time. Once this condition is met, the sentinel will be ARMED. This will let you set the sentinel at any point, and it will only trigger once the GPU has been running for a while.Once ARMED, the sentinel will wait for the GPU utilization to drop below a certain threshold for a set amount of time. Once this condition is met, thekill_actionwill occur immediately.Options:arm_duration: How many seconds of activity to wait before arming the sentinel.
arm_threshold: What level of utilization is considered activity
kill_duration: How many seconds of inactivity to wait before running the kill function.
kill_threshold: What level of utilization is considered inactivity
kill_action: What to do when the kill trigger is hit {end_process,shutdown,reboot}
gpu_devices: Which GPU devices to average (empty for all)APIIf you would prefer to use integrate this package into your own code, we provide a straightforward API to do so.fromgpu_sentinelimportSentinel,get_gpu_usagedefmy_callback_fn():print("Triggered!")exit()# Create the sentinel that watches the values.sentinel=Sentinel(arm_duration=10,arm_threshold=0.7,kill_duration=60,kill_threshold=0.7,kill_fn=my_callback_fn,)whileTrue:# This is the averaged GPU usage of the devices.gpu_usage=get_gpu_usage(device_ids=[0,1,2,3])# Add the GPU usage to the sentinel's next state.sentinel.tick(gpu_usage)# The sentinel operates on ticks, not seconds, so if we want to check every second# we must do the timer ourselves.time.sleep(1)Current LimitationsTo shutdown/reboot the machine, GPU Sentinel requires sudo permissions or sudo-less shutdown.Currently only working on Linux, can add Windows support if there's interest.
Package: gpush
gpushgpushis a command line utility for standardising commit messages usingConventional Commits. It is designed to be used in conjunction withpython-semantic-releaseto automate the release of your python package based on commits that are made on a branch when a PR is merged.Signing commitsgpushnatively supports signing commits using GPG. To enable this, you will need to have GPG installed and configured on your machine. You can find instructions on how to do thishere.InstallationThis package is available for installation via pypi using pip3:pip3 install gpushUsageWhen running gpush in a git directory, usegpushto replace your standardgit commit && git pushcommands.gpushwill ask a few questions to determine detail about your commit and generate a conventionally formatted git commit message.ExampleTo see additional options available, run gpush with the--helpflaggpush --helpLocal developmentTo develop locally, uninstall any existing version of gpush before executing the pip3 installation in the local directory, use Makefile command to do this:make devUnit testsTo run unit tests, execute the following command, you will need to have pytest installed and set environment
variableGITHUB_TOKENto a valid GitHub token that has permissions to create/delete and push to a repository
within your account.make testThe unit tests will create a test repository within your account for each unit test before testing functionality and deleting the repository.python-semantic-releasegpushgenerates commit messages that are compliant withpython-semantic-release. This allows you to automate the release of your python package using GitHub Actions.To use python-semantic-release, you will need to create a GitHub Action workflow that runs on themainbranch. The workflow will need to checkout the code, install python-semantic-release and then run the release command. The following is an example workflow that will run on themainbranch and release a new version of your package and push to PyPI when a PR is merged.You will need to create a PyPI token and add it to your GitHub repository secrets (AsPYPI_TOKEN). You can find instructions on how to do thishere..github/workflow/release.ymlname:Semver Bumpon:push:branches:-"main"jobs:auto-semver:runs-on:"ubuntu-latest"steps:-name:"Checkoutcode"uses:"actions/checkout@v3"with:fetch-depth:0-name:"PythonSemanticRelease"uses:"python-semantic-release/python-semantic-release@master"with:github_token:${{ secrets.GITHUB_TOKEN }}repository_username:__token__repository_password:${{ secrets.PYPI_TOKEN }}
Package: gpustat
gpustatJustlessthan nvidia-smi?NOTE: This works with NVIDIA Graphics Devices only, no AMD support as of now. Contributions are welcome!Self-Promotion: A web interface ofgpustatis available (in alpha)! Check outgpustat-web.Quick InstallationInstall fromPyPI:pip install gpustatIf you don't have root (sudo) privilege, please try installinggpustaton user namespace:pip install --user gpustat.To install the latest version (master branch) via pip:pip install git+https://github.com/wookayin/gpustat.git@masterNVIDIA Driver RequirementsgpustatusesNVIDIA's official python bindings for NVML library (pynvml). As of nowgpustatrequiresnvidia-ml-py >= 11.450.129, which is compatible with NVIDIA driver versions R450.00 or higher. Please upgrade the NVIDIA driver ifgpustatfails to display process information. If your NVIDIA driver is too old, you can use oldergpustatversions (pip install gpustat<1.0). See#107for more details.Python requirementsgpustat<1.0: Compatible with python 2.7 and >=3.4gpustat 1.0:Python >= 3.4gpustat 1.1: Python >= 3.6Usage$ gpustatOptions (Please seegpustat --helpfor more details):--color: Force colored output (even when stdout is not a tty)--no-color: Suppress colored output-u,--show-user: Display username of the process owner-c,--show-cmd: Display the process name-f,--show-full-cmd: Display full command and cpu stats of running process-p,--show-pid: Display PID of the process-F,--show-fan: Display GPU fan speed-e,--show-codec: Display encoder and/or decoder utilization-P,--show-power: Display GPU power usage and/or limit (drawordraw,limit)-a,--show-all: Display all gpu properties above--id: Target and query specific GPUs only with the specified indices (e.g.--id 0,1,2)--no-processes: Do not display process information (user, memory) (#133)--watch,-i,--interval: Run in watch mode (equivalent towatch gpustat) if given. Denotes interval between updates.--json: JSON Output (#10)--print-completion (bash|zsh|tcsh): Print a shell completion script. See#131for usage.TipsTrygpustat --debugif something goes wrong.To periodically watch, trygpustat --watchorgpustat -i(#41).For older versions, one may usewatch --color -n1.0 gpustat --color.Runningnvidia-smi daemon(root privilege required) will make querying GPUs muchfasterand use less CPU (#54).The GPU ID (index) shown bygpustat(andnvidia-smi) is PCI BUS ID,
while CUDA uses a different ordering (assigns the fastest GPU with the lowest ID) by default.
Therefore, in order to ensure CUDA andgpustatusesame GPU index,
configure theCUDA_DEVICE_ORDERenvironment variable toPCI_BUS_ID(before settingCUDA_VISIBLE_DEVICESfor your CUDA program):export CUDA_DEVICE_ORDER=PCI_BUS_ID.Default display[0] GeForce GTX Titan X | 77°C,  96 % | 11848 / 12287 MB | python/52046(11821M)[0]: GPU index (starts from 0) as PCI_BUS_IDGeForce GTX Titan X: GPU name77°C: GPU Temperature (in Celsius)96 %: GPU Utilization11848 / 12287 MB: GPU Memory Usage (Used / Total)python/...: Running processes on GPU, owner/cmdline/PID (and their GPU memory usage)ChangelogSeeCHANGELOG.mdLicenseMIT License
Package: gpustats
gpustats is a PyCUDA-based library implementing functionality similar to that
present in scipy.stats. It implements a simple framework for specifying new CUDA
kernels and extending existing ones. Here is a (partial) list of target
functionality:Probability density functions (pdfs). These are intended to speed up
likelihood calculations in particular in Bayesian inference applications, such
as in PyMCRandom variable generation using CURANDNotesRequires working PyCUDA installation
Package: gpustat-web
gpustat-webA web interface ofgpustat---
aggregategpustatacross multiple nodes.NOTE: This project is in alpha stage. Errors and exceptions are not well handled, and it might use much network resources. Please use at your own risk!Installationpip install gpustat-webPython 3.6+ is required.UsageLaunch the application as follows. A SSH connection will be established to each of the specified hosts.
Make suressh <host>works under a proper authentication scheme such as SSH authentication.gpustat-web --port 48109 HOST1 [... HOSTN]You might get "Host key is not trusted for<host>" errors. You'll have to accept and trust SSH keys of the host for the first time (it's stored in~/.ssh/known_hosts);
tryssh <host>in the command line, orssh -oStrictHostKeyChecking=accept-new <host>to automatically accept the host key. You can also use an optiongpustat-web --no-verify-hostto bypass SSH Host key validation (although not recommended).Note that asyncsshdoes NOT obeythe~/.ssh/configfile
(e.g. alias, username, keyfile), so any config in~/.ssh/configmight not be picked up.Endpointshttps://HOST:PORT/: A webpage that updates automatically through websocket.https://HOST:PORT/gpustat.html: Result as a static HTML page.https://HOST:PORT/gpustat.txt: Result as a static plain text.https://HOST:PORT/gpustat.ansi: Result as a static text with ANSI color codes. Trycurl https://.../gpustat.ansiQuery strings:?nodes=gpu001,gpu002: Select a subset of nodes to query and displayRunning as a HTTP (SSL/TLS) serverBy default the web server will run as a HTTP server.
If you want to run a secure SSL/TLS server over the HTTPS protocol, use--ssl-certfileand--ssl-keyfileoption.
You can use letsencrypt (certbot) to create a pair of SSL certificate and keyfile.Troubleshoothing: Verify SSL/TLS handshaking (if TLS connections cannot be established)openssl s_client -showcerts -connect YOUR_HOST.com:PORT < /dev/nullMore ExamplesTo see CPU usage as well:python -m gpustat_web --exec 'gpustat --color --gpuname-width 25 && echo -en "CPU : \033[0;31m" && cpu-usage | ascii-bar 27'LicenseMIT LicenseCopyright (c) 2018-2023 Jongwook Choi
Package: gpu-tester
gpu_testerGpu tester finds all your bad gpus.Works on slurm.Features:does a forward on each gpucheck for gpu returning incorrect resultscheck for gpu failing due to ECC errorsRoadmap:sanity check forward speedsanity check broadcast speedInstallCreate a venv:python3 -m venv .env
source .env/bin/activate
pip install -U pipThen:pip3 install torch --extra-index-url https://download.pytorch.org/whl/cu116
pip install gpu_testerPython examplesCheckout these examples to call this as a lib:example.pyOutputOutput looks like this:job succeeded
0 have incorrect results, 1 have gpu errors and 319 succeeded
incorrect results:
[]
gpu errors:
[['gpu_error', 'compute-od-gpu-st-p4d-24xlarge-156', '3']]Recommended testing strategyPair based strategyThe easiest way to quickly spot broken node is to do the pair-based strategy.
It will run many jobs in parallel and find which node can talk together
Here is one examplegpu_tester --nodes 2 --parallel-tests 50 --job_comment laion --partition "gpu" --test_kind "ddp" --job_timeout 45 --exclude 'gpu-st-p4d-24xlarge-[66]'All at once strategyOnce you validated this works, you may want to try the DDP strategy over all nodes, eg:gpu_tester --nodes 100 --parallel-tests 1 --job_comment laion --partition "gpu" --test_kind "ddp" --job_timeout 300 --exclude 'gpu-st-p4d-24xlarge-[66]'Simple forwardIf you want to only validate the forward functionality of gpus and not the communication, you may use:gpu_tester --nodes 100 --parallel-tests 1 --job_comment laion --partition "gpu" --test_kind "simple_forward" --job_timeout 50 --exclude 'gpu-st-p4d-24xlarge-[66]'APIThis module exposes a single functiongpu_testerwhich takes the same arguments as the command line tool:clusterthe cluster. (defaultslurm)job_nameslurm job name. (defaultgpu_tester)partitionslurm partition. (defaultcompute-od-gpu)gpu_per_nodenumbe of gpu per node. (default8)nodesnumber of gpu nodes. (default1)output_folderthe output folder. (defaultNonewhich means current folder / results)job_timeoutjob timeout (default150seconds)job_commentoptional comment arg given to slurm (defaultNone)job_accountoptional account arg given to slurm (defaultNone)test_kindsimple_forward or ddp. simple_forward is quick forward test. DDP uses pytorch ddp to check gpu interconnect (defaultsimple_forward)parallel_testsnumber of tests to run in parallel. Recommended to use that with nodes == 2 to test pair by pair (default1)nodelistnode whitelist, example 'gpu-st-p4d-24xlarge-[66-67]' (defaultNone)excludenode blacklist, example 'gpu-st-p4d-24xlarge-[66-67]' (defaultNone)For developmentEither locally, or ingitpod(doexport PIP_USER=falsethere)Setup a virtualenv:python3 -m venv .env
source .env/bin/activate
pip install -e .to run tests:pip install -r requirements-test.txtthenmake lint
make testYou can usemake blackto reformat the codepython -m pytest -x -s -v tests -k "dummy"to run a specific test
Package: gputils
gputilsis a library that contains a variety of utilities that may come handy in diverse projects.Installationpip install gputilsIndex of utilitiesdyn_mean: computes the mean based on a previous mean plus a new value. Useful when mean is built incrementally, it saves the usage of huge arrays.dyn_stdev: computes the stdev based on a previous stdev plus a new value.cosine_similarity: computes the cosine similarity between two vectors or matrix and vector.
Package: gpu-tpu-lab-server
gpu_tpu_lab_serverRunning a jupyter lab or vs code server environment from colab with google drive support.references:fns.colab. Most of the localtunnel colab code is taken from this repo.Motivation for creating this packageFor some reason infns.colabpython package version >=0.5, fns.colab supports jupyter notebook not jupyter lab, so here I am trying to use the previous version of that fns.colab code to support jupyter lab environment and to fix frequent time out and connection issues.> 1. It was boring to copy paste the whole code everytime to start the jupyter lab environment in colab, so I created a package of it.Also, I wanted to try how to create python package using nbdev, this sounded like a good excuse.Installpip install gpu_tpu_lab_serverHow to usefromgpu_tpu_lab_server.coreimportjupyter_lab,vscode,run_process_in_background,run_process_in_foregroundFor starting the server in google colab with google_drive supportTo enable jupyter_lab_environmentexecutejupyter_lab()an authorization window will open with a link, go to the link > select your google account > accept all the permissions > authorization code will be displayed > copy and paste the code in the windowan localtunnel link will be returned, go to that link, wait for 10 second, then press open tunnel button, it will open an jupyter lab environmentTo enable vscode() environmentexecutevscode()optional parameter any text :- the generated link will have the text or by default fastdaimaan authorization window will open with a link, go to the link > select your google account > accept all the permissions > authorization code will be displayed > copy and paste the code in the windowan localtunnel link will be returned, go to that link it will open an vscode server environment
Package: gputree
gputreeHosts configurationPATHIf no host is provided through the command line interface,gputreewill look for a configuration file. By default, it looks forgputree/config.ymlfile in$XDG_CONFIG_HOMEor~/.configdirectory. You can define a custom path to find the gputree configuration file by setting the$GPUTREE_CONFIG_FILEenvironement variable.exportGPUTREE_CONFIG_FILE=/path/to/gputree/config.yml`CONTENTIn the configuration file, list your default hosts to look at under thehostskey. You can specify hosts in two ways :Define a host with a unique name as key, and set its address with a user name as follows:my-gpu:
    hostname: 0.0.0.0
    user: my.usernameRefer to a host defined in your~/.ssh/configfile if exists:from-ssh-config:
    - gpu-1
    - gpu-2EXAMPLEFullgputreeconfiguration file example:hosts:
  my-gpu:
    hostname: 0.0.0.0
    user: my.username

  from-ssh-config:
    - gpu-1
    - gpu-2
Package: gpuutils
GpuUtilsWorking on a shared environment with multiple GPUs might be problematic. Advanced frameworks apply greedy approach and they tend to allocate all GPUs and all memory of your system. GpuUtils helps you to find the best GPU on your system to allocate. It also provides a gpu related information in a structure format.InstallationThe easiest way to install GpuUtils is to install it viaPyPI.pip install gpuutilsAnalyzing systemRunningnvidia-smicommand in the command prompt allows users to monitor GPU related information such as memory and utilization. Herein, system analysis function loads GPU related information into a pandas data frame or json array.fromgpuutilsimportGpuUtilsdf=GpuUtils.analyzeSystem()#this will return a pandas data frame#dict = GpuUtils.analyzeSystem(pandas_format = False) #this will return a json arrayDefault configuration of system analysis returns a Pandas data frame.gpu_indextotal_memories_in_mbavailable_memories_in_mbmemory_usage_percentageutilizationspower_usages_in_wattspower_capacities_in_watts132480324690.0339043300232480324690.0339043300332480324690.0339044300432480324690.0339043300532480324690.0339043300632480324690.0339043300732480324690.0339043300032480310314.4612756300AllocationGpuUtils can allocate GPUs as well. Calling allocation function directly finds the available GPUs and allocate based on your demand.fromgpuutilsimportGpuUtilsGpuUtils.allocate()#this tries to allocate a GPU having 1GB memory#GpuUtils.allocate(required_memory = 10000)#GpuUtils.allocate(required_memory = 10000, gpu_count=1)To avoid greedy approachAdvanced frameworks such as TensorFlow tend to allocate all memory. You can avoid this approach if you pass the framework argument in allocate function. In this way, the framework will use the gpu memory as much as needed. Currently, keras and tensorflow frameworks are supported in allocate function.GpuUtils.allocate(framework='keras')SupportThere are many ways to support a project - starring⭐️ the GitHub repos is just one.LicenceGpuUtils is licensed under the MIT License - seeLICENSEfor more details.
Package: gpu-utils
GPU UtilsA few small functions/scripts for working with GPUs.RequirementsPython 3.6+Linux OS for full functionality (only tested on Ubuntu; I usesubprocess.runforkillandlsof)Everything exceptkill_interrupted_processesshould work on any OSInstallationpip install gpu-utilsThe PyPI page ishere.Usagefromgpu_utilsimportgpu_init# sets GPU ids to use nvidia-smi ordering (CUDA_DEVICE_ORDER = PCI_BUS_ID)# finds the gpu with the most free utilization or memory# hides all other GPUs so you only use this one (CUDA_VISIBLE_DEVICES = <gpu_id>)gpu_id=gpu_init(best_gpu_metric="util")# could also use "mem"If you use TensorFlow or PyTorch,gpu_initcan take care of another couple of steps for you:# a torch.device for the selected GPUdevice=gpu_init(ml_library="torch")importtensorflowastf# a tf.ConfigProto to allow soft placement + GPU memory growthconfig=gpu_init(ml_library="tensorflow")session=tf.Session(config=config)Command Line Scriptsgpuis a more concise and prettier version ofnvidia-smi. It is similar togpustatbut with more control over the color configuration and the ability to show the full processes running on each GPU.kill_interrupted_processesis useful if you interrupt a process using a GPU but find that, even thoughnvidia-smino longer shows the process, the memory is still being held. It will sendkill -9to all such processes so you can reclaim your GPU memory.tmux_gpu_info.pyjust prints a list of the percent utilization of each GPU; you can, e.g., show this in the status bar oftmuxto keep an eye on your GPUs.AcknowledgementsUsingpynvmlinstead of parsingnvidia-smiwith regular expressions made this library a bit faster and much more robust than my previous regex parsing ofnivida-smi's output; thanks togpustatfor showing me this library and some ideas about the output format for thegpuscript.
Package: gpuv
gpuvgpuv (gpuvisual) is a python library that provides an interactive dashboard to monitor your gpu usage:InstallationInstall withpip:pip3 install gpuvUsageTo usegpuv, simply run the start command from the cli:gpuv startThis will launch the interface on another port. Navigate there to monitor your gpu usage live.EC2 UsageOn EC2, you will have to tunnel to the port5000. This can be done by spinning up th ec2 with port forwarding enabled, e.g.ssh -i <pemkey> -L 5000:127.0.0.1:5000 ubuntu@ec2<...>.comThen, just rungpuv startin the cli of your ec2, navigate to that port, and you're good to go!DemoHere's a quick demo video (for those who prefer watching vs reading):videoDev SetupTo set up gpuv for your own development/extension, usepoetry:pip3 install poetry
poetry install --no-root
python -m gpuv -sComing SoonSupport multiple gpu's in one view.Local data collection (that you can save to csv).
Package: gpvolve
gpvolveA python package for extracting tevolutionary trajectories from genotype-phenotype-mapsA Python API for the simulation and analysis of evolution in genotype-phenotype space.
You can use this library to:Build a markov state model from a genotype-phenotype-map.Find clusters of genotypes that represent metastable states of the system, using PCCA+.Compute fluxes and pathways between pairs of genotypes and/or clusters of interest, using Transition Path Theory.Visualize the outputs of all of the above.Basic ExampleBuild a Markov model from a genotype-phenotype map.# Import base class, Transition Path Theory class and functions for building Markov Model.fromgpvolveimportGenotypePhenotypeMSM,TransitionPathTheory,linear_skew,mccandlish,find_max# Import visualization tool.fromgpvolve.visualizationimportplot_network# Import GenotypePhenotypeMap class for handling genotype-phenotype data.fromgpmapimportGenotypePhenotypeMap# Helper functions.fromscipy.sparseimportdok_matrix# Genotype-phenotype map data.wildtype="AAA"genotypes=["AAA","AAT","ATA","TAA","ATT","TAT","TTA","TTT"]phenotypes=[0.8,0.81,0.88,0.89,0.82,0.82,0.95,1.0]# Instantiate Markov model class.gpm=GenotypePhenotypeMap(wildtype=wildtype,genotypes=genotypes,phenotypes=phenotypes)# Instantiate a evolutionary Markov State Model from the genotype-phenotype map.gpmsm=GenotypePhenotypeMSM(gpm)Apply an evolutionary model to describe transitions between genotypes.# Map fitnesses to phenotypes.gpmsm.apply_selection(fitness_function=linear_skew,selection_gradient=1)# Build Markov State Model based on 'mccandlish' fixation probability function.gpmsm.build_transition_matrix(fixation_model=mccandlish,population_size=100)# Find global fitness peak.fitness_peak=find_max(gpmsm=gpmsm,attribute='fitness')Calculate and plot the trajectory flux between the wildtype and triple mutant.# Compute fluxes from wildtype to fitness peak.fluxes=TransitionPathTheory(gpmsm,source=[0],target=[fitness_peak])# Normalize flux.norm_fluxes=fluxes.net_flux/fluxes.total_flux# Plot the network and the fluxesfig,ax=plot_network(gpmsm,flux=dok_matrix(norm_fluxes),edge_labels=True,colorbar=True)InstallTo install from PyPI:pip install gpvolveTo install a development version:git clone https://github.com/harmslab/gpvolve
cd gpvolve
pip install  -e .