semvec/pypi-packages huggingface dataset:
flex-head-fa 0.1.2
FlexHeadFA: Flash Attention with flexible head dimensions
Requires Python: >=3.8

# FlexHeadFA

This repository is a fork of the [FlashAttention](https://github.com/Dao-AILab/flash-attention) main repo. It extends the official implementation to support FlashAttention with flexible head dimensions.

All configurations in FlashAttention-2 are supported. Besides, we have supported:

- FlashAttention-2 with QKHeadDim=32, VHeadDim=64 
- FlashAttention-2 with QKHeadDim=64, VHeadDim=128
- FlashAttention-2 with QKHeadDim=96, VHeadDim=192
- FlashAttention-2 with QKHeadDim=128, VHeadDim=256
- FlashAttention-2 with QKHeadDim=192, VHeadDim=128
- FLashAttention-2 with not equal num_heads_k and num_heads_v, such as (num_heads_q, num_heads_k, num_heads_v) = (32, 4, 16)

For headdim not supported, you can use the autotuner to generate the implementation. Details are in `autotuner.md`.

Feel free to tell us what else you need. We might support it soon. :)


## Installation

The requirements is the same as [FlashAttention-2](https://github.com/Dao-AILab/flash-attention?tab=readme-ov-file#installation-and-features)

To install:
```sh
pip install flex-head-fa --no-build-isolation
```
Alternatively you can compile from source:

```sh
python setup.py install
```

The usage remains the same as FlashAttention-2. You only need to replace `flash_attn` with `flex_head_fa`, as shown below:


```python
from flex_head_fa import flash_attn_func, flash_attn_with_kvcache
```

We are also developing FlexHeadFA based on the lastest FLashAttention-3.
Currently, besides all configurations in FlashAttention-3, we also support

- FlashAttention-3 with QKHeadDim=32, VHeadDim=64

- FlashAttention-3 forward + FlashAttention-2 backward with QKHeadDim=128, VHeadDim=256 (FlashAttention-3 backward is under development)

Try it with:

```sh
cd hopper
python setup.py install
```

Usage:

```python
from flash_attn_interface import flash_attn_func # FlashAttention-3 forward+backward
from flash_attn_interface import flash_attn_f3b2_func as flash_attn_func # FlashAttention-3 forward + FlashAttention-2 backward 
```

## Performance of FlexHeadFA

We test the performance speedup compare to padding qk&v hidden_dim on A100.

We display FlexHeadFA speedup using these parameters:

- (qk dim, v_dim): (32,64), (64,128), (128,256); qk hidden dimension 2048 (i.e. 64, 32 or 16 heads).
- Sequence length 512, 1k, 2k, 4k, 8k, 16k.
- Batch size set to 16k / seqlen.

### Speedup

![Custom-flash-attn](assets/Customflash2_a100_fwd_bwd_benchmark.png)


## When you encounter issues

This new release of FlexHeadFA has been tested on several GPT-style
models, mostly on A100 GPUs.

If you encounter bugs, please open a GitHub Issue!
magic-maths 0.1.1
magically import evaluated math expressions
Requires Python: >=3.10

# magic-math

Magically import evaluated math expressions. Like so:

```py
from magic_math import TEN_TIMES_FIVE_DIVIDED_BY_TWO
from magic_math import FIVE_TIMES_FIVE
from magic_math import TWO_HUNDRED_MINUS_TEN
from magic_math import SIX_HUNDRED_AND_SIXTY_SIX_DIVIDED_BY_TWO_TIMES_EIGHT
from magic_math import OPEN_FIVE_MINUS_THREE_CLOSE_TIMES_TEN

ABC=7
from magic_math import OPEN_TEN_MINUS_OPEN_ABC_DIVIDED_BY_TWO_CLOSE_CLOSE

print(f"{TEN_TIMES_FIVE_DIVIDED_BY_TWO = }")  # 25.0
print(f"{FIVE_TIMES_FIVE = }")  # 25.0
print(f"{TWO_HUNDRED_MINUS_TEN = }")  # 190.0
print(f"{SIX_HUNDRED_AND_SIXTY_SIX_DIVIDED_BY_TWO_TIMES_EIGHT = }")  # 2664.0
print(f"{OPEN_FIVE_MINUS_THREE_CLOSE_TIMES_TEN = }")  # 20.0
print(f"{OPEN_TEN_MINUS_OPEN_ABC_DIVIDED_BY_TWO_CLOSE_CLOSE = }")  # 6.5
```

Notes:

- All returned values are in floating point form
- Floats can be specified using "point" or "dot"
- Variables are partially supported, currently only within scripts
- Uses basic postfix order of precendence
- Basic parenteses syntax using `OPEN` and `CLOSE` values

## Installation

```sh
python3 -m pip install magic-maths
```

Credits:

- Inspired by and uses code from David Buchanan's [`magic-numbers` package](https://github.com/DavidBuchanan314/magic-numbers)
My-question-rag 0.1
A pipeline for question answering and retrieval-augmented generation (RAG)
Requires Python: >=3.6

#readme.md

# QA RAG Pipeline

A Python package that implements a Question-Answering Retrieval-Augmented Generation (RAG) pipeline using OpenAI, FAISS, and Sentence Transformers.

## Features
- Retrieve relevant context using FAISS.
- Generate standalone questions using OpenAI's LLM.
- Classify intent to block certain queries.

## Installation
```bash
pip install .