semvec/pypi-packages huggingface dataset:
nuse 0.1.8
A simple resource monitoring tool for SLURM jobs.

# nuse

nuse is a simple resource monitoring tool for SLURM jobs. It allows users to view the CPU and memory usage logs for individual jobs and the entire node. With nuse, when running a SLURM-based workflow, you can easily monitor resource consumption through command-line tools.

![nuse example](https://raw.githubusercontent.com/MolinDiscovery/nuse/main/images/nuse_example.png)

## Features

- **Job-Specific Monitoring:** Capture and view resource usage for each job (e.g., using cgroup filtering).
- **Node-Wide Monitoring:** Automatically collect a separate log for overall node usage.
- **Command-Line Interface:** Installed via pip, the `nuse` command lets you quickly view logs with a simple one-liner.
- **Custom Log Directory:** Easily configure where logs are stored by setting the `MONITOR_LOG_DIR` environment variable.

## Installation

Install nuse directly from PyPI:

```bash
pip install nuse
```

## Usage

In your script include.

```python
from nuse import start_monitoring

start_monitoring(filter_cgroup=True)

# your code
```
**INFO**: You can set filter_cgroup to `False` to watch the entire node and not just your own job.

**NOTE**: You should include `start_monitoring()` in the script you submit to SLRUM. Don't put it in a [submitit](https://github.com/facebookincubator/submitit) script.

Job-Specific Log:
To display the resource usage log for a specific job on a node, run in your CLI:

```bash
nuse node305 49847516
```

Here, node305 is the node's short name and 49847516 is the SLURM job ID.

## Configuration
Log Directory:
* By default, nuse stores logs in the directory ~/.monitoring.
To change the log directory, set the environment variable before running your jobs:

```bash
export MONITOR_LOG_DIR="/path/to/your/log_directory"
```

## How it Works

When included in your SLURM job pipeline (via start_monitoring(filter_cgroup=True) from the nuse package), nuse will:

* Create a job-specific log file with a naming convention like cpu_memory_usage_<hostname>.cluster_<SLURM_JOB_ID>.log.

Then, the nuse CLI tool uses these logs to provide an easy-to-read, real-time view of resource usage.

Happy monitoring!

## Planned improvements

* `nuse clear` to remove the monitoring directory. Until then just delete the ~/.monitoring folder.
flat-data 1.0.1
A Python library which provides a seamless solution for converting unstructured data into structured format, making data easier to process, analyze, and store. By organizing nested or inconsistent data into a standardized format, this library simplifies data handling for various applications.

# FlatData Python Library

## Overview
The library provides a seamless solution for converting unstructured data into structured format, making data easier to process, analyze, and store. By organizing nested or inconsistent data into a standardized format, this tool simplifies data handling for various applications.

## Use Cases
- **Data Processing:** Clean and structure raw JSON or dictionaries for easier analysis.
- **Machine Learning:** Prepare datasets for feature extraction and model training.
- **API Integration:** Structure API responses into consistent formats for frontend or backend use.
- **Database Operations:** Normalize data for storage in relational or NoSQL databases.
- **Scientific Research:** Organize experimental or biological data for statistical studies.
- **Business Applications:** Format data for reports, dashboards, or CRM systems.

## How to use it?

Install to library use the following command.
```pip install flat-data```

Import the library
```from flat_data import FlatData```

Pass the data that you want to structure and the records which you want to include.

```python
from flat_data import FlatData

input_data = [
    {
        "team": "Dot Net",
        "detail": [
            {
                "type": "Dot Net Dev",
                "members": ["Rohan", "Shriyash"],
                "salary": [25000, 35000]
            },
            {
                "type": "Dot Net Test",
                "members": ["Shravani", "Komal", "Satyen"],
                "salary": [30000, 40000, 50000]
            }
        ]
    },
    {
        "team": "Python",
        "detail": [
            {
                "type": "Python Dev",
                "members": ["Anushka", "Pranjal"],
                "salary": [22000, 36000]
            },
            {
                "type": "Python Test",
                "members": ["Pratham", "Amol", "Manjiri"],
                "salary": [52000, 67000, 24000]
            }
        ]
    }
]

fd = FlatData(data=input_data, records=['team', 'type', 'members', 'salary'])

flatten_data = fd.get_flat_data()
```

Recommended sequence for the records list is from top left to bottom right. In the below example, ```['team', 'type', 'members', 'salary']```

![alt text](images/record_direction.png)
openai-streamer 0.1.0
A package for streaming text completion using OpenAI's Responses API
Requires Python: >=3.8
Keywords: openai, ai, streaming, text completion, gpt

# OpenAI Streamer

A Python package for streaming text completion using OpenAI's Responses API.

## Installation

You can install the package directly from the source:

```bash
# Activate your conda environment
conda activate OpenRAPI

# Install the package in development mode
pip install -e .
```

Or you can build and install the package:

```bash
# Activate your conda environment
conda activate OpenRAPI

# Build the package
python setup.py sdist bdist_wheel

# Install the package
pip install dist/openai_streamer-0.1.0-py3-none-any.whl
```

## Usage

### Basic Usage

```python
from openai_streamer.streamer import stream_completion

# Simple example
prompt = "Write a short poem about artificial intelligence."
instructions = "You are a creative poet with a technical background."

response = stream_completion(prompt, instructions=instructions)
```

### Using the Class

```python
from openai_streamer.streamer import OpenAIStreamer

# Create an instance
streamer = OpenAIStreamer()

# Generate a completion
prompt = "Explain quantum computing in simple terms."
instructions = "You are a science educator explaining complex topics to beginners."

response = streamer.stream_completion(prompt, instructions=instructions)
```

## Configuration

The package uses environment variables for configuration. Create a `.env` file in your project root with:

```
OPENAI_API_KEY=your_api_key_here
```

## Requirements

- Python 3.8+
- openai>=1.0.0
- python-dotenv>=0.19.0

## License

MIT
binance-archiver 0.0.3
A package for archiving Binance data
Requires Python: >=3.11
Keywords: binance archiver quant data sprzedam opla

# Real-time market data listener and archiver in python. 
Saves raw binance data from:
limit orderbook difference depth stream
limit orderbook difference depth snapshot request
transaction stream

into .json / .json.zip / .json.zip on azure or backblaze

simply enter azure backblaze connection params into environment variables and import 

'launch_data_sink' - use as data sink

or 

'launch_data_listener' - Use in your algo trading as it just informs all observers without data save

NEW!!!:
Although Binance now limits each WebSocket connection to 24 hours, this code runs continuously (24/7) without missing any stream entries!


# Handles: 
spot, futures usd-m, futures coin-m
Level 2 orderbook difference depth stream
trade stream
orderbook snapshots with configured trigger interval 
24-hour WebSocket lifecycle. At the end of the WebSocket's lifespan, it initiates a new WebSocket to ensure the continuity of data flow is maintained seamlessly.

Configured to use as container on Azure with Azure blob / keyvault or backblaze

![image](https://github.com/user-attachments/assets/a9461c8d-b5a7-43de-b1cc-96ef5df72f40)

![image](https://github.com/user-attachments/assets/93a9cece-21fd-406c-8555-fbb774188265)

![Zrzut ekranu 2024-06-02 230137](https://github.com/DanielLasota/Binance-Archiver/assets/127039319/b400f859-60ef-4995-936d-d68ecab82ddf)



## Installation

```bash
# to be announced
```

## Usage

import the `run_stonks_analysis` function from the `stonks` module and run the script:

```python
import time
from dotenv import load_dotenv

from binance_archiver import load_config_from_json, DataSinkConfig, launch_data_sink

if __name__ == "__main__":

    load_dotenv('sample_env.env')

    data_sink_config = DataSinkConfig(
        instruments={
            'spot': ['BTCUSDT', 'XRPUSDT'],
            'usd_m_futures': ['BTCUSDT', 'XRPUSDT'],
            'coin_m_futures': ['BTCUSDT', 'XRPUSDT']
        },
        time_settings={
            "file_duration_seconds": 120,
            "snapshot_fetcher_interval_seconds": 300,
            "websocket_life_time_seconds": 120
        },
        data_save_target='azure_blob'
    )

    data_sink = launch_data_sink(data_sink_config=data_sink_config)
    
    time.sleep(99)
    
    data_sink.shutdown()
    

```