semvec/pypi-packages huggingface dataset:
aws-resource-monitor 1.0.0
Easily monitor CPU and memory usage in AWS Batch jobs and report them to StatsD
License: MIT
Requires Python: >=3.5
Keywords: aws monitoring

# AWS Resource Monitor

Small Python 3.5+ convenience library to easily monitor CPU and memory usage in AWS Batch jobs and report them to StatsD. Since it uses [ECS Task Metadata V3 endpoints](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint-v3.html) it is perfectly suitable to do so for other type of ECS tasks as well.

## Setup

You can install this package by using `pip`:

	pip install aws-resource-monitor

If you fancy `pipenv` use:

	pipenv install aws-resource-monitor

To install from source, run:

	python setup.py install


To install via requirements file, add the following:

	-e git+https://github.com/luigisbox/aws-resource-monitor.git#egg=aws-resource-monitor

## Usage

First off, you need to require the library:

	from aws_resource_monitor import resource_monitoring

The `resource_monitoring` is a context manager accepting two positional arguments:

- `statsd_client` is your own instance of StatsD client (see [PythonStatsD library](https://statsd.readthedocs.io/en/latest/) which will be used to report metrics.
- `metrics_prefix` is a string which will prefix individual metrics.

Optionally, you can pass in keyword argument `interval` with `timedelta` object customizing polling period for new metrics (default is every five seconds).

	with resource_monitoring(statsd_client, metrics_prefix):
		fn()

Behind the scenes, `resource_monitoring` context manager first checks for existence of `ECS_CONTAINER_METADATA_URI` environment variable. If it is not found, the monitor does nothing thus working well in development.

In production or when the variable exists, it creates a separate thread from your main application that reads task UUID and polls the "stats" endpoint in the metadata URI to complete and send as gauge the following metrics:

- cpu.system
- cpu.user
- cpu.percent
- memory.limit
- memory.usage
- memory.max_usage

A more complete example would be:

	import statsd
	from aws_resource_monitor import resource_monitoring

	statsd = statsd.StatsClient('localhost', 8125)

	with resource_monitoring(statsd, "example.metrics.", interval=timedelta(seconds=10)):
		fn()

For example, for the task UUID assigned by AWS "afc92259-3a0c-4e14-8e23-a49cc41e7947" this will report gauge "example.metrics.afc92259-3a0c-4e14-8e23-a49cc41e7947.memory.usage" and in ten seconds check and report again.

## Contributing

1.  Check for open issues or open a new issue for a feature request or a bug.
2.  Fork the repository and make your changes to the master branch (or branch off of it).
3.  Send a pull request.

## Development

Run the linter with:

	make lint

The client library uses Black for code formatting. Code must be formatted with Black before PRs are submitted. Run the formatter with:

	make fmt

## Changelog

### v1.0.0: 08/06/2020

Initial version.
gpumonitor 0.1.2
GPU Monitoring Callbacks for TensorFlow and PyTorch Lightning
License: MIT
Requires Python: >=3.6

# gpumonitor

[![Pypi Version](https://img.shields.io/pypi/v/gpumonitor.svg)](https://pypi.org/project/gpumonitor/)
![Licence](https://img.shields.io/pypi/l/gpumonitor)
![Frameworks](https://img.shields.io/badge/Frameworks-PyTorchLightning%20|%20TensorFlow-blue.svg)


`gpumonitor` gives you **stats about GPU** usage during execution of your scripts and trainings,
as [TensorFlow](https://www.github.com/tensorflow/tensorflow) or 
[Pytorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning) callbacks.

<p align="center">
    <img src="./assets/callbacks.png" width="800" />
</p>


## Installation

Installation can be done directly from this repository:

```
pip install gpumonitor
```

## Getting started


<p align="center">
    <img src="./assets/gpumonitor.gif" width="1000" />
</p>

### Option 1: In your scripts

```python
monitor = gpumonitor.GPUStatMonitor(delay=1)

# Your instructions here
# [...]

monitor.stop()
monitor.display_average_stats_per_gpu()
```

It keeps track of the average of GPU statistics. To reset the average and start from fresh, you can also reset the monitor:

```python
monitor = gpumonitor.GPUStatMonitor(delay=1)

# Your instructions here
# [...]

monitor.display_average_stats_per_gpu()
monitor.reset()

# Some other instructions
# [...]

monitor.display_average_stats_per_gpu()
```

### Option 2: Callbacks

Add the following callback to your training loop:

For [TensorFlow](https://www.github.com/tensorflow/tensorflow),

```python
from gpumonitor.callbacks.tf import TFGpuMonitorCallback

model.fit(x, y, callbacks=[TFGpuMonitorCallback(delay=0.5)])
```

For [PyTorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning),

```python
from gpumonitor.callbacks.lightning import PyTorchGpuMonitorCallback

trainer = pl.Trainer(callbacks=[PyTorchGpuMonitorCallback(delay=0.5)])
trainer.fit(model)
```

## Display Format

You can customize the display format according to the `gpustat` options. For example, display of watts consumption,
fan speed are available. To know which options you can change, refer to:

- [TensorFlow callback example](https://github.com/sicara/gpumonitor/blob/42237f423254e8fc7ae21e8f2811533a4264064d/scripts/tf_training.py#L16)
- [`gpustat print_to()` docstring](https://github.com/wookayin/gpustat/blob/aba85f8eba9f7861022eb3dcc06ff771b451b3e1/gpustat/core.py#L178)

## Sources

- Built on top of [GPUStat](https://github.com/wookayin/gpustat)
- Separate thread loop coming from [gputil](https://github.com/anderskm/gputil)
epic-path 0.1.1
A simple high level library to work with Path
License: MIT

# :sparkles: EpicPath :sparkles: 
A high Level Python Library to work with Path

`EpicPath` is a SubClass of `Path` from `pathlib`.
This class aims to simplify high level operations with path

# Context and history

I'm a Computing Student tired of creating again and again the same functions through all my projects to do the exact same things... :tired_face: 
Indeed, in every of my projects I create files, delete them, create folders and delete them *blablabla* ...
To solve this boring problem, I decided to create my own library that I can create one and only one time to use and reuse as many times as I wish to.
And because I'm trying to be nice to this cruel world I share this *Epic* project with all of you :heart:
So feel free to download it, use it and give feedback about this ***epic***  creation. :+1:


# Description

You already know `Path` from the library `pathlib`. Well, `EpicPath` is all like `Path` but in a ***EPIC*** way :tada: . 

It is a *pseudo* subclass of `Path` but consider it as a subclass, it supports all the methods and options of the original library.
It is also a `os.PathLike` object which allows you to use as you would want to:

```python
with open(my_epic_path_object, 'w') as f:
    f.write('This is epic !!!')
```

 
# Use it



## Install it

```shell
pip install epic-path
```

## Use it

```python
from epicpath import EpicPath as EPath

path = EPath('f1', 'f2')        # EpicPath('f1/f2')
path2 = path / 'f3'             # EpicPath('f1/f2/f3')
path2 += '_0'                   # EpicPath('f1/f2/f3_0')
```

[See the documentation](doc/README.md)   

# :warning: Warnings :warning:

Because `Path` is build-in Python libraries, some other library
(like [PIL](https://github.com/python-pillow/Pillow) or [skopt](https://github.com/scikit-optimize/scikit-optimize))
can have a specific behavior when the given filename is a `str` or a `Path` object.
Unfortunately, they don't support an `EpicPath` object ... :cry: But no worries, it is still possible to give an `EpicPath` object as a `Path` object or a `str` object using the properties `.path` and `.str`:

```python
path = EPath('f1', 'f2')        # EpicPath('f1/f2')

weird_function(path.path)       # Gives Path('f1/f2') to the function
weird_function(path.str)        # Gives the string 'f1/f2' to the function
```