semvec/pypi-packages huggingface dataset:
whale-triton-template 0.6
whale aml triton template
Requires Python: >=3.5

# Whale AML Triton

## Install
```bash
pip install whale-triton-template
```
## Usage 
```bash
wat my-service
```
## Generated Struct
```bash
my-service
├── config.pbtxt
├── hack
│   ├── start.sh
│   ├── pack.sh
│   └── upload.sh
├── manifest
│   ├── base
│   │   └── deployment.yaml
│   └── overlay
│       └── stage
│           └── deployment.yaml
├── model.py
└── models
    └── my-service-onnx
        └── config.pbtxt

```

## Quick Start

### pack your env

```bash
### Unix
pip install -r requirements.txt
conda pack -f -o env.tar.gz

### Mac
# docker run -it --rm -v $(pwd):/app -w /app --entrypoint bash  continuumio/miniconda3 ./hack/pack.sh
```

### start triton server
```bash

docker run -it --rm -p 8000:8000 -v $(pwd):/app -w /app nvcr.io/nvidia/tritonserver:22.05-py3 ./hack/start.sh
```

### test vie rest api
```bash
curl --location --request POST 'localhost:8000/v2/models/my-service/infer' \
--header 'Content-Type: application/json' \
--data-raw '{
    "inputs": [
        {
            "name": "INPUT_0",
            "shape": [
                1,
                1
            ],
            "data": [
                [
                    "XXX"
                ]
            ],
            "datatype": "BYTES"
        }
    ]
}'
```