semvec/pypi-packages huggingface dataset:
flashnn 0.1.1
Flash: Triton Kernel Library for LLM Serving

# FLASHNN : A Triton-Powered Kernel Library for LLM Serving

FLASHNN is a pioneering kernel library for Large Language Models (LLMs), providing a high-performance implementation of GPU kernels optimized for LLM serving and inference, involving in comprehensive support for attention kernels and versatile quantization methods. 

By harnessing the power of Triton, FLASHNN is engineered to seamlessly integrate with multiple of hardware platforms, ensuring smooth operability and maximizing the utilization of hardware resources.



## Features

- **Comprehensive Support for Attention Kernels**: FLASHNN offers extensive support for various types of attention mechanisms, enabling it to handle a wide array of LLM architectures with ease. 
- **Multiple Quantization Methods**: FLASHNN incorporates multiple quantization techniques (int8, int4) aimed at optimizing both the computational overhead and the memory footprint of LLMs, making it easier to deploy LLMs in resource-constrained environments.
- **Low Runtime Overhead**：The primary contributor to the performance discrepancy observed with Triton kernels is the runtime overhead. To address this, we have implemented an ahead-of-time kernel cache for Triton kernels, which significantly mitigates this overhead.
- **Production-Ready Performance**: FLASHNN is meticulously optimized for production scenarios, which delivers state-of-art performance that meets the demanding requirements of real-world applications.
- **Smooth Portability on Multiple Hardware**: Facilitated by the inherent design of the Triton language, FLASHNN simplifying the process of adapting LLM serving solutions to diverse computing environments.


## **Compatibility**

### **Supported Operators**

| Type         | Operators                                               |
| ------------ | ------------------------------------------------------- |
| Gemm         | A8W8, A16W4, A16W8                                      |
| Attention    | PagedAttention V1, PagedAttention V2, FlashAttention V2 |
| Norm         | LayerNorm, RMSNorm                                      |
| Quantization | DynamicQuant, LayerNormDequant, RMSNormDequant          |
| Embedding    | RotaryEmbedding                                         |

### **Supported Platforms**

FlashNN is tested to work in Nvidia and AMD GPUs(e.g. A100, A10, H20, MI210, ...). 

| Platforms   | float16 | float32 | bfloat16 |
| ----------- | ------- | ------- | -------- |
| NVIDIA A100 |         |         |          |
| NVIDIA A10  |         |         |          |
| NVIDIA H20  |         |         |          |
| AMD MI210   |         |         |          |

## **Get Started**

### **Requirements**

FlashNN requires Pytorch and Triton.

### **Installation**

FlashNN operators can be customized to each function-equivalent PyTorch operators by simply replace the corresponding torch function.

The binary wheel distribution(whl) will be available soon.

## Benchmarks



## License

The FLASHNN project is based on [Apache 2.0](https://github.com/AlibabaPAI/FLASHNN/blob/main/LICENSE).
pyrealsense-multi 0.0.1
A small package for multiple Intel RealSense cameras
Requires Python: >=3.9

## Install
```shell
pip install multi-camera-recording
```

## Quick use
Start recording and then align frames
```python
from multi-camera-recording.record_sync import start_recording
from multi-camera-recording.align_frames import FrameAligner

path = start_recording()
fa = FrameAligner(path=path)
fa.align_frames()
```
wee-agent 0.1.3
A micro agent for open_ai llm model
Requires Python: >=3.10

# MICRO_AGENT

micro-agent 是一个轻量级的使用openAI模型服务的智能代理，将程序调用openAI API的过程进行了封装，使得用户可以调用函数一样与大模型进行对话。同时，micro-agent
包含了简单的会话上下文窗口管理，使得发送给大模型的对话历史可以根据token数量进行调整。同时，micro-agent
支持将本地函数注册到代理中，使得代理可以调用本地函数，使用最简单的方式构造agent。

## 安装

使用pip安装：

```shell
pip install wee_agent
```

## 使用

### 1. 使用前

使用前，应该将 OPEN_API_KEY 配置到环境变量中：

    ```shell
    export OPEN_API_KEY=your_open_api_key
    ```

或者，在代码的根目录中，创建一个`.env`文件，写入：

    ```shell
    OPEN_API_KEY=your_open_api_key
    ```

### 2. 基础使用

只需实例化一个代理，然后可以像调用函数一样直接调用openAI API的对话功能。

```python
from wee_agent import WeeAgent

agent = WeeAgent()
ret = agent("hello")
print(ret)
```

要实现多轮对话，只需要重复调用即可：

```python
from wee_agent import WeeAgent

agent = WeeAgent()
print(agent('hi，I am Bob. '))
# 显示：Hello, Bob! How can I assist you today?
print(agent('What is my name?'))
# 显示：Your name is Bob. How can I help you today, Bob?
```

代理会记录与用户沟通的历史对话，并能自动根据token数量调整发送给llm的对话历史窗口。

### 添加tool

micro_agent可以非常方便的将本地函数注册到agent中，使得micro_agent对象成为一个智能体。
注册的函数必须满足以下要求：

* 注册的函数，必须进行类型注释，指定参数的类型
* 函数的返回值，必须为文本格式，以便于大模型理解。目前不支持使用其他格式。

先使用set_tool装饰器，装饰本地函数，然后调用register_tool方法注册函数，最后调用agent对象，传入提示词，即可调用本地函数。

```python
from wee_agent import WeeAgent, set_tool


# set_tool是一个装饰器，用于注册函数
# 定义本地函数
@set_tool
def plus(a: int, b: int) -> str:
   """
    计算两个数的和
    :param a: 第一个数
    :param b: 第二个数
    :return: 两个数的和
    """
   return str(a + b)


agent = WeeAgent()

# 注册函数
agent.register_tool(name='plus', tool=plus)

# 调用函数
print(agent("计算1+2等于多少"))

# 输出：1+2等于3
```

### 3. 高级用法

#### 3.1 自定义提示词

在实例化agent时，可以传入写好的提示词作为参数：

```python
from wee_agent import WeeAgent

agent = WeeAgent(prompt="你是一名心里咨询师，善于解决心里问题...")
print(agent("我感到很烦躁"))
```

或者：

```python
from wee_agent import WeeAgent

agent = WeeAgent()
agent.set_prompts("你是一名心里咨询师，善于解决心里问题...")
print(agent("我感到很烦躁"))
```

#### 3.2 流式输出
在初始化类时，可以设置streaming参数为True，使得输出更加流畅：

```python
from wee_agent import WeeAgent

agent = WeeAgent(stream=True)

agent("hello")

# 在终端上逐token输出：Hello! How can I assist you today?
```



#### 3.3 控制对话窗口问答比例
micro_agent可以控制每次问答时，发送给大模型的对话历史占整个对话历史的比例。默认为0.9，即每次问答时，发送给大模型的对话历史占整个对话历史的90%。对话历史里包含了prompt。如果你需要大模型回答更多内容，可以将这个比例调低。同时，这也会导致对话历史信息降低。

```python
from wee_agent import WeeAgent

agent = WeeAgent(
   input_token_ratio=0.5  # 提问和回答各占对话窗口的50%
)
...
```
#### 3.4 重构本类
为了更加方便的使用，可以继承MicroAgent类，然后使用装饰器注册函数：

```python
from wee_agent import WeeAgent, set_tool
from datetime import datetime


class MyAgent(WeeAgent):

   @staticmethod
   @set_tool
   def get_time():
      """
      获取当前时间
      :return: 字符串格式的当前时间
      """
      return f"现在是{datetime.now().strftime('%Y年%m月%d日%H时%M分%S秒')}"


agent = MyAgent()
print(agent("请告诉我现在的时间"))

# 输出：现在是2024年3月19日18时39分24秒。如果您需要更准确的时间，请让我知道，我可以为您提供更具体的时分秒信息。
```

请注意，注册的函数，必须满足以下要求：

1. 注册的函数，必须进行类型注释，指定参数和返回值的类型，
2. 同时必须提供详细的，严格格式的函数文档，用于生成函数的schema信息，供大模型理解。其中：第一行必须为函数的功能说明。参数的说明，
    * 必须以:param 开头，
    * 返回值必须使用:return 开头。
3. 函数的返回值必须为文本格式，以便于大模型理解。目前不支持使用其他格式。

----

## 下一步计划

因为目前openAI的api格式几乎成为了大模型调用的标杆，修改本模块，让它可以支持其他开源模型，例如Ollama、vllm等。
Rythmix-dot-py 1.0.7
A simple CLI tool for playing music. (VLC is required!)

# Rythmix.py
Rythmix.py is a simple CLI tool used to play music.

To run rythmix, simply run "rythmix" in your CMD/Terminal and it will start.
Rythmix requires VLC to run. To automatically setup VLC, simply run "sudo rythmix_setup_vlc" in your CMD/Terminal.
quantumtrader 0.6
一个基于freqtrade的量化交易策略系统
Requires Python: >=3.8

# quantum-trader(仅供个人学习)


## 量化交易策略系统



### 主要特点



### 关于quantumtrader_ta指标库



### 如何使用

1. **安装依赖**：
2. **下载与配置**
3. **编写策略**：
4. **回测与交易**：

### 贡献与反馈



### 许可证
TorchPIV 0.1.17
Package for PIV data analysis
Requires Python: >=3.9

# PyTorch accelerated Particle Image Velocimetry
This program implements the basic algorithms of the PIV method, such as an iterative cross-correlation method based on FFT with an integer and continuous displacement __(DWS, CWS)__ of the interrogation windows, filtering and interpolation of the pair loss effect, and so on. At this stage, the __graphical interface__ is available, the ability to select PIV hyperparameters. The key feature of the project is the use of GPU due to the __torch__ library. PIV algorithm is completely vectorized, what results in a very high performance using the GPU, but still has a room for improvement.

__Parameters of the program:__
1. Interrogation window size
2. Window overlap size
3. Coordinate scale
4. Time between frames
5. Selection of the device (CPU or any of your GPUs)
6. Iteration method: DWS or CWS
7. The number of iterations of the algorithm
8. Interrogation window scale during iterations
9. Supported image formats (bmp, jpg, tiff, ect.)
10. Options for saving program results

__Installation:__    
It is easier to use conda environment.
1. Install nvidia CUDA Toolkit https://developer.nvidia.com/cuda-toolkit to ensure latest nvidia driver usage
2. Install pytorch with GPU support https://pytorch.org/ . Matching your CUDA version is not critical since PyTorch installs it's own cudatoolkit.  
3. In your environment or command line <code>pip install TorchPIV</code>

__Usage:__  
Start GUI version from terminal  
<code>python</code>  
```python
import torchPIV 
torchPIV.runGUI()
```
Use as a part of the script 
```python
from torchPIV import OfflinePIV

piv_gen = OfflinePIV(
    folder="test_images", # Path to experiment
    device=torch.cuda.get_device_name(), # Device name
    file_fmt="bmp",
    wind_size=64,
    overlap=32,
    dt=12, # Time between frames, mcs
    scale = 0.02, # mm/pix
    multipass=2,
    multipass_mode="CWS", # CWS or DWS
    multipass_scale=2.0, # Window downscale on each pass
    folder_mode="pairs" # Pairs or sequential frames 
)

results = []
for out in piv_gen():
    x, y, vx, vy = out
    results.append(out)

```


Tested on Windows

__Performance__:  
This method allows processing 4 thousand pairs of images with a size of 4 MP each with a search window of 64, overlap of 50%, two iterations with re-arranging (an increase in the number of vectors by 4 times) in less than 10 minutes. The first iteration of the algorithm (~[4000, 64, 64] subimage tensor) takes ~15 ms on a Geforce GTX 1660 Ti GPU.

__Futhure plans__:
Creating a version of the program for processing data during their recording. The ability to process stereo PIV data.

You can cite this work with:  
N. A. Nazarov and V. V Terekhov, “High level GPU-accelerated 2D PIV framework in Python,” Comput. Phys. Commun., p. 109009, 2023, doi: https://doi.org/10.1016/j.cpc.2023.109009.
photo-burst-detection 1.4.3
Flask App for photo burst detection
Requires Python: >=3.6

# photo_burst_detection

[![pypi version](https://img.shields.io/pypi/v/photo_burst_detection.svg)](https://pypi.org/project/photo_burst_detection/)

## install

```shell
pip install photo_burst_detection
```

### install from sources

```shell
git clone https://github.com/batou9150/photo_burst_detection.git
cd photo_burst_detection
python3 setup.py install
```

## run

### Run with gunicorn (Unix)

```shell
pip3 install gunicorn

export PHOTO_BURST_DETECTION_PATH=<start path>
export LDAP_HOST=ad.mydomain.com
export LDAP_BASE_DN=dc=mydomain,dc=com

gunicorn -b 0.0.0.0:8000 photo_burst_detection:app
```

### Run with waitress (Windows)

```shell
export PHOTO_BURST_DETECTION_PATH=<start path>
export LDAP_HOST=ad.mydomain.com
export LDAP_BASE_DN=dc=mydomain,dc=com

waitress-serve --listen=*:8000 photo_burst_detection:app
```

## configuration

| variable                     | description                             |
|------------------------------|-----------------------------------------|
| PHOTO_BURST_DETECTION_CONFIG | config file (optional)                  |
| PHOTO_BURST_DETECTION_PATH   | start path                              |
| SECRET_KEY                   | (default value = 'secret')              |
| LDAP_HOST                    |                                         |
| LDAP_PORT                    | (default value = '389')                 |
| LDAP_BASE_DN                 |                                         |
| LDAP_USER_DN                 |                                         |
| LDAP_GROUP_DN                |                                         |
| LDAP_USER_RDN_ATTR           | (default value = 'uid')                 |
| LDAP_USER_LOGIN_ATTR         | (default value = 'uid')                 |
| LDAP_BIND_USER_DN            |                                         |
| LDAP_BIND_USER_PASSWORD      |                                         |
| LDAP_GROUP_OBJECT_FILTER     | (default value = '(objectclass=group)') |
meritous 1.2.2
Simple Python Models
Requires Python: >=3.10
Keywords: model

[![PyPI](https://img.shields.io/pypi/v/meritous?style=for-the-badge)](https://pypi.org/project/meritous/)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/meritous?style=for-the-badge)
![PyPI - License](https://img.shields.io/pypi/l/meritous?style=for-the-badge)
[![Codecov](https://img.shields.io/codecov/c/github/errant/meritous?style=for-the-badge)](https://app.codecov.io/github/errant/meritous)
[![Code Climate maintainability](https://img.shields.io/codeclimate/maintainability/errant/meritous?style=for-the-badge)](https://codeclimate.com/github/errant/meritous)

# Meritous


Meritous is an absurdly simply approach to "Models" in Python.

It came about because there is no modern, framework agnostic approach to modelling data.

On the face of it the usage is very trivial; but the intent is that Meritous is a building block for more complex data models. Essentially, it provides a simple Model class which can contain data to be used in Python applications. It then sets out a standard practice for transforming that data for storage or transport.

- [Documentation](https://meritous.readthedocs.io/en/latest/)

## Installation

```bash
pip install meritous
```

## Basic Usage

```python
from meritous.core import Model
from meritous.core.properties import UUIDProperty, StrProperty, DateProperty

from datetime import date

class EventModel(Model):

    _schema = {
        "id"          : UUID4Property(),
        "title"       : StrProperty(),
        "date"        : DateProperty(),
        "description" : StrProperty(),
    }


event = EventModel()
event.title = 'Sample Event'
event.date = date.fromisoformat('2023-01-10')
print(event.id)
print(event.title)
print(event.date)
```
codable 0.1.1
A Python module for encoding and decoding objects using customizable serialization.
Requires Python: >=3.6

# Codable

Codable is a Python module for encoding and decoding objects using customizable serialization. It allows you to easily convert your Python objects to and from JSON, with support for custom encoders and decoders. This module is designed to be flexible and extensible, making it easy to work with complex data structures and custom object types.

## Features

- **Customizable Serialization**: Define your own encoders and decoders to handle specific object types.
- **Automatic Encoding/Decoding**: Use the built-in `AutoEncodable` and `AutoDecodable` classes to automatically handle common serialization tasks.
- **Registry-Based System**: Register your custom types with a global registry for easy access and management.
- **Integration with Django**: Includes a custom `JsonResponse` class that integrates with Django's HTTP response system.

## Installation

To install Codable, use pip:
```bash
    pip install codeble
```
llama-cmdline 0.0.1rc1
Llama CLI
Requires Python: >=3.10

# llama-toolchain

[![PyPI - Downloads](https://img.shields.io/pypi/dm/llama-toolchain)](https://pypi.org/project/llama-toolchain/)
[![Discord](https://img.shields.io/discord/1257833999603335178)](https://discord.gg/TZAAYNVtrU)

This repo contains the API specifications for various components of the Llama Stack as well implementations for some of those APIs like model inference.

The Llama Stack consists of toolchain-apis and agentic-apis. This repo contains the toolchain-apis.

## Installation

You can install this repository as a [package](https://pypi.org/project/llama-toolchain/) with `pip install llama-toolchain`

If you want to install from source:

```bash
mkdir -p ~/local
cd ~/local
git clone git@github.com:meta-llama/llama-toolchain.git

conda create -n toolchain python=3.10
conda activate toolchain

cd llama-toolchain
pip install -e .
```

## The Llama CLI

The `llama` CLI makes it easy to configure and run the Llama toolchain. Read the [CLI reference](docs/cli_reference.md) for details.

## Appendix: Running FP8

If you want to run FP8, you need the `fbgemm-gpu` package which requires `torch >= 2.4.0` (currently only in nightly, but releasing shortly...)

```bash
ENV=fp8_env
conda create -n $ENV python=3.10
conda activate $ENV

pip3 install -r fp8_requirements.txt
```
virtualhome-eval 0.1.1
Embodied agent interface evaluation for VirtualHome
Requires Python: >=3.8

# Installation and Usage Guide for virtualhome-eval


## Install dependencies
```
pip install virtualhome_eval
```

## Usage
To run `virtualhome_eval`
1. Use in python
```
from virtualhome_eval.agent_eval import agent_evaluation
agent_evaluation(mode=[generate_prompts, evaluate_results], eval_type=[goal_interpretation, action_sequence, transition_modeling], llm_response_path=[YOUR LLM OUTPUT DIR])
```
2. Use directly in the command line
```
virtualhome-eval --mode [generate_prompts, evaluate_results] --eval-type [goal_interpretation, action_sequence] --llm-response-path [YOUR LLM OUTPUT DIR] --output-dir [YOUR EVAL OUTPUT DIR]
```
### Parameters
- `mode`: Specifies either generate prompts or evaluate results. Options are:
  - `generate_prompts` 
  - `evaluate_results`
- `eval_type`: Specifies the evaluation task type. Options are:
  - `goal_interpretation`
  - `action_sequence`
  - `subgoal_decomposition`
  - `transition_model`
- `llm_response_path`: The path of LLM output directory to be evaluated. It is `""` by default, using the existing outputs at directory `virtualhome_eval/llm_response/`. The function will evaluate all LLM outputs under the directory.
- `dataset`: The dataset type. Options:
  - `virtualhome`
  - `behavior`
- `output_dir`: The directory to store the output results. By default, it is at `output/` of current path.

### Example usage in python
1. To generate prompts for `goal_interpretation`:
```
agent_evaluation(mode='generate_prompts',  eval_type='goal_interpretation')
```
2. To evaluate LLM outputs for `goal_interpretation`:
```
results = agent_evaluation(mode='evaluate_results', eval_type='goal_interpretation')
```
3. To generate prompts for `action_sequence`:
```
agent_evaluation(mode='generate_prompts',  eval_type='action_sequence')
```
4. To evaluate LLM outputs for `action_sequence`:
```
results = agent_evaluation(mode='evaluate_results', eval_type='action_sequence')
```
5. To generate Virtualhome prompts for `transition_model`:
```
agent_evaluation(mode='generate_prompts',  eval_type='transition_model')
```
6. To evaluate LLM outputs on Virtualhome for `transition_model`:
```
results = agent_evaluation(mode='evaluate_results', eval_type='transition_model')
```
7. To generate prompts for `subgoal_decomposition`:
```
agent_evaluation(mode='generate_prompts',  eval_type='subgoal_decomposition')
```
8. To evaluate LLM outputs for `subgoal_decomposition`:
```
results = agent_evaluation(mode='evaluate_results', eval_type='subgoal_decomposition')
ResearchTools 0.0.4
A package for generic tools that may be useful in scientific research projects. Python 3.10+

# ResearchTools

A package for various tools that can be useful in scientific research projects. Two design principles drive this project.



2. Everything should be as fast as possible, targetting high performance computing.
	- Numba-accelerated and vectorized functions where it makes sense.
	- Memoization module for caching the results of expensive functions.
	- Numpy ND-arrays for anything involving number-crunching.
	
1. Prioritize use of using functions/closures over custom Classes.
	- Closures typically serve a single purpose, and may expose 1-2 other fields for manipulating internal variables.
		- Keep it Simple and Modular, memory is in no shortage these days but CPU cycles are always costly.
	- When more structure is needed, we use the base Python datatypes as much as possible.
		- Lists for mutable integer-indexed Iterables, Tuples for immutable.
		- Dicts for key-value mappings.