semvec/pypi-packages huggingface dataset:
lm-infinite 0.0.2
Paper - Pytorch
License: MIT
Requires Python: >=3.6,<4.0
Keywords: artificial intelligence,deep learning,optimizers,Prompt Engineering

[![Multi-Modality](agorabanner.png)](https://discord.gg/qUtxnK2NMf)

# LM-INFINITE: SIMPLE ON-THE-FLY LENGTH GENERALIZATION FOR LARGE LANGUAGE MODELS

LM-Infinite is a solution proposed by Chi Han, Qifan Wang, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang to address the length generalization failure of Large Language Models (LLMs) on long sequences. LLMs, such as Transformer-based models, have shown impressive performance in various domains but struggle when it comes to longer reasoning processes or understanding larger contexts. Current pre-training schemes truncate training sequences to a fixed length, and even with relative positional encoding, LLMs struggle to generate coherent texts or perform downstream tasks after longer contexts.

The authors investigate the main out-of-distribution factors contributing to this problem and propose LM-Infinite as an efficient solution. LM-Infinite only requires a Λ-shaped attention mask and a distance limit, without any parameter updates or learning. It can be applied to different LLMs using relative-position encoding methods. LM-Infinite demonstrates consistent fluency and generation quality for sequences as long as 32k tokens on datasets like ArXiv and OpenWebText2, with a decoding speedup of 2.72x. Furthermore, it continues to perform well on inputs much longer than training lengths in downstream tasks like passkey retrieval, where vanilla models fail immediately.

[Paper Link](https://arxiv.org/pdf/2308.16137.pdf)

---

# Appreciation
* Lucidrains
* Agorians



# Install
`pip install lm-infinite`

# Usage
```python
import torch
from infinite.main import LMInfinite

d_model = 512
seq_len = 100
n_global = 100
l_pretrain = 50


#sample
q = torch.randn(1, seq_len, d_model)
k = torch.randn(1, seq_len, d_model)
v = torch.randn(1, seq_len, d_model)


#llm infinite mode
model = LMInfinite(
    d_model,
    n_global,
    l_pretrain
)

#forwad pass
output = model(q, k, v)
print(output.shape)
```
# Architecture

# Todo


# License
MIT

# Citations
```latex
@misc{2308.16137,
Author = {Chi Han and Qifan Wang and Wenhan Xiong and Yu Chen and Heng Ji and Sinong Wang},
Title = {LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models},
Year = {2023},
Eprint = {arXiv:2308.16137},
}
```
toxic-proxy 0.1.0
An asyncio tcp proxy for network resilience testing
License: MIT
Requires Python: >=3.9,<4.0
Keywords: resilience,asyncio,testing

# Toxic Proxy

Toxic Proxy is an asyncio-based TCP proxy designed for network resilience testing. This allows you to simulate network conditions like latency, bandwidth limitations, and other issues to test how your application behaves under different scenarios. Whether you're working with microservices or just need to test your network reliability, Toxic Proxy provides a flexible way to introduce network anomalies for better resilience testing.

## Features

- Simulate network latency
- Limit bandwidth
- Easy to integrate with Python applications
- Uses asyncio for high performance

## Installation

Install Toxic Proxy using pip.

```shell
pip install --upgrade toxic_proxy
```

## Quick Start

### Using with Cassandra

Here's an example that shows how to use Toxic Proxy with a Cassandra cluster.

```python
import asyncio
from aiocassandra import aiosession
from cassandra.cluster import Cluster
from toxic_proxy import toxic_proxy

# Create a proxy with latency
proxy = toxic_proxy(
    destination=('<cassandra-ip>', 9042),
    latency=5,
    port=9043
)

# Connect to Cassandra through the proxy
cluster = Cluster(['0.0.0.0:9043'])
session = cluster.connect()
# ... your code here ...


```

### Simplified Example

This example demonstrates a simple use case to limit bandwidth when connecting to google.com.

```python
import asyncio
from toxic_proxy import toxic_proxy

# Create a proxy with bandwidth limit
proxy = toxic_proxy(
    destination=('google.com', 80),
    bandwidth_rate_kb=0.5,
    port=8888
)

loop = asyncio.get_event_loop()
server = loop.run_until_complete(proxy)
print(f'Serving on {server.sockets[0].getsockname()}')

try:
    loop.run_forever()
except KeyboardInterrupt:
    pass

# Close the server
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()

```

### Testing

To test, you can use curl to make a request through the proxy.

```shell
curl -v 0.0.0.0:8888

```

## Contributing

We welcome contributions! If you find a bug or have a feature request, please open an issue. If you want to contribute code, feel free to open a pull request.

## License

This project is licensed under the MIT License. See the LICENSE file for details.