semvec/pypi-packages huggingface dataset:
amdsharktuner 3.9.0
IREE Model and Dispatch Tuner
Requires Python: >=3.10

# amdshark Tuner
amdshark Tuner automates the dispatch-level and model-level tuning for the IREE (Intermediate Representation Execution Environment) ML Compiler on AMD GPUs.

## Installation
Install amdshark Tuner from PyPI:
```shell
pip install amdsharktuner
```

This will install all required dependencies including IREE compiler and runtime.

You can use the latest nightly IREE python bindings:
```shell
pip install --upgrade --pre \
    iree-base-compiler \
    iree-base-runtime \
    --find-links https://iree.dev/pip-release-links.html
```

Verify that the environment is set up correctly:
```shell
python -m model_tuner --help
```

or

```shell
python -m dispatch_tuner --help
```

### Model Tuner
Use the Model Tuner to tune a dispatch and a model:
```shell
python -m model_tuner double_mmt.mlir mmt_benchmark.mlir \
    --compile-flags-file=compile_flags.txt \
    --model-benchmark-flags-file=model_benchmark_flags.txt \
    --devices=hip://0 \
    --num-candidates=30 \
    --model-tuner-num-dispatch-candidates=5 \
    --model-tuner-num-model-candidates=3`
```

Refer to [Mode Tuner README](https://github.com/nod-ai/amd-shark-ai/tree/main/amdsharktuner/model_tuner) for detailed information on flags and MLIR files.

### Dispatch Tuner
Use the Dispatch Tuner to tune a dispatch:
```shell
python -m dispatch_tuner dispatch_sample.mlir dispatch_sample_benchmark.mlir \
    --compile-flags-file=compile_flags.txt
    --devices=hip://0 \
    --num-candidates=30
```

Refer to [Dispatch Tuner README](https://github.com/nod-ai/amd-shark-ai/tree/main/amdsharktuner/dispatch_tuner) for detailed information on flags and MLIR files.
amdsharktank 3.9.0
amdshark layers and inference models for genai

# amdshark Tank

**WARNING: This is an early preview that is in progress. It is not ready for
general use.**

Light weight inference optimized layers and models for popular genai
applications.

This sub-project is a work in progress. It is intended to be a repository of
layers, model recipes, and conversion tools from popular LLM quantization
tooling.

## Project Status

[![CI - amdsharktank nightly](https://github.com/nod-ai/amd-shark-ai/actions/workflows/ci-amdsharktank-nightly.yml/badge.svg?branch=main)](https://github.com/nod-ai/amd-shark-ai/actions/workflows/ci-amdsharktank-nightly.yml)

## Examples

The repository will ultimately grow a curated set of models and tools for
constructing them, but for the moment, it largely contains some CLI examples.
These are all under active development and should not yet be expected to work.


### Perform batched inference in PyTorch on a paged llama derived LLM:

Note: Use `--device='cuda:0'` to run this inference on an AMD GPU.

```shell
python -m amdsharktank.examples.paged_llm_v1 \
  --hf-dataset=open_llama_3b_v2_f16_gguf \
  --device='cuda:0' \
  "Prompt 1" \
  "Prompt 2" ...
```

### Export an IREE compilable batched LLM for serving:

```shell
python -m amdsharktank.examples.export_paged_llm_v1 \
  --hf-dataset=open_llama_3b_v2_f16_gguf \
  --output-mlir=/tmp/open_llama_3b_v2_f16.mlir \
  --output-config=/tmp/open_llama_3b_v2_f16.json
```

### Generate sample input tokens for IREE inference/tracy:Add commentMore actions

```shell
python -m amdsharktank.examples.paged_llm_v1 \
  --irpa-file=open_llama_3b_v2_f16.irpa \
  --tokenizer-config-json=tokenizer_config.json \
  --prompt-seq-len=128 \
  --bs=4 \
  --dump-decode-steps=1 \
  --max-decode-steps=1 \
  --dump-path='/tmp' \
  --device='cuda:0'
```

### Dump parsed information about a model from a gguf file:

```shell
python -m amdsharktank.tools.dump_gguf --hf-dataset=open_llama_3b_v2_f16_gguf
```

## Package Python Release Builds

* To build wheels for Linux:

    ```bash
    ./build_tools/build_linux_package.sh
    ```

    That should produce
    `build_tools/wheelhouse/amdsharktank-{X.Y.Z}.dev0-py3-none-any.whl`, which can
    then be installed with

    ```bash
    python3 -m pip install build_tools/wheelhouse/amdsharktank-{X.Y.Z}.dev0-py3-none-any.whl
    ```

* To build a wheel for your host OS/arch manually:

    ```bash
    # Build amdsharktank.*.whl into the dist/ directory
    #   e.g. `amdsharktank-3.0.0.dev0-py3-none-any.whl`
    python3 -m pip wheel -v -w dist .

    # Install the built wheel.
    python3 -m pip install dist/*.whl
    ```
eth-abi 6.0.0b1
eth_abi: Python utilities for working with Ethereum ABI definitions, especially encoding and decoding
Requires Python: <4,>=3.10
Keywords: ethereum

# Ethereum Contract Interface (ABI) Utility

[![Join the conversation on Discord](https://img.shields.io/discord/809793915578089484?color=blue&label=chat&logo=discord&logoColor=white)](https://discord.gg/GHryRvPB84)
[![Build Status](https://circleci.com/gh/ethereum/eth-abi.svg?style=shield)](https://circleci.com/gh/ethereum/eth-abi)
[![PyPI version](https://badge.fury.io/py/eth-abi.svg)](https://badge.fury.io/py/eth-abi)
[![Python versions](https://img.shields.io/pypi/pyversions/eth-abi.svg)](https://pypi.python.org/pypi/eth-abi)
[![Docs build](https://readthedocs.org/projects/eth-abi/badge/?version=latest)](https://eth-abi.readthedocs.io/en/latest/?badge=latest)

Python utilities for working with Ethereum ABI definitions, especially encoding and decoding

Read the [documentation](https://eth-abi.readthedocs.io/).

View the [change log](https://eth-abi.readthedocs.io/en/latest/release_notes.html).

## Installation

```sh
python -m pip install eth-abi
```