semvec/pypi-packages huggingface dataset:
apstone 0.0.8
ai_power base stone

### Introduction

Base stone of AI_power, maintain all inference of AI_Power models.

#### Wrapper

- Supply different model infer wrapper, including ONNX/TensorRT/Torch JIT;
- Support onnx different Execution Providers (EP) , including cpu/gpu/trt/trt16/int8;
- High level mmlab model (converted) infer wrapper, including MMPose/MMDet;

#### Model Convert

- torch2jit torch2onnx etc.
- detectron2 to onnx
- modelscope to onnx
- onnx2simple2trt
- tf2pb2onnx

#### Model Tools

- torch model edit
- onnx model shape/speed test (different EP)
- common scripts from onnxruntime

### Usage

#### onnx model speed test
```python
from apstone import ONNXModel

onnx_p = 'pretrain_models/sr_lib/realesr-general-x4v3-dynamic.onnx'
input_dynamic_shape = (1, 3, 96, 72)  # None
# cpu gpu trt trt16 int8
ONNXModel(onnx_p, provider='cpu', debug=True, input_dynamic_shape=input_dynamic_shape).speed_test()
```

### Install

```sh
pip install apstone
```

#### Envs

| Execution Providers | Needs                                                       |
| ------------------- | ----------------------------------------------------------- |
| cpu                 | pip install onnxruntime                                     |
| gpu                 | pip install onnxruntime-gpu                                 |
| trt/trt16/int8      | onnxruntime-gpu compiled with tensorrt EP                   |
| TensorRT            | pip install tensorrt pycuda                                 |
| torch JIT           | install [pytorch](https://pytorch.org/get-started/locally/) |
llama-memory 0.0.1a1
Easy deployment of quantized llama models on cpu
Keywords: vector database,vector db,rag,long term ai memory,llama,ai,artificial intelligence,natural language processing,nlp,quantization,cpu,deployment,inference,model,models,model database,model repo,model repository,model library,model libraries,gguf,llm cpu,llm

# MEMORY FOR GGUF_LLAMA AND OTHER LLAMAs
cavasspy 3.0.1
CAVASS python APIs.
Requires Python: ~=3.11

# CAVASSPPY

CAVASS python APIs.

## Usage

from cavasspy.ops import *

Note that running cavass commands requires the following shell environment configs: PATH including CAVASS build path and VIEWNIX_ENV.
If the shell does not include these configs by default, set cavasspy.ops.CAVASS_PROFILE_PATH to your CAVASS build path (default path is ~/cavass-build in installation if you didn't make any change) before performing CAVASS commands.
convect-flow-sdk 0.0.6
convect flow sdk
Requires Python: >=3.9

# flow-sdk
flow sdk 

## Install

```bash
pip install convect-flow-sdk
```

## Usage

```bash
# set up environment variables
# FLOW_WORKSPACE_ID: your flow workspace id
# FLOW_API_TOKEN: your flow api token from the workspace
# FLOW_HOST: flow host, default to https://flow.convect.ai
from convect_flow_sdk import FlowAlgo
from pprint import pprint
# this is an example of how to use the flow_algo_sdk
flow_algo = FlowAlgo()
# this algo_id is will be the one defined in the algo project
algo_id = '2f1a1fa9-2958-4afe-bb48-4239a960986d'
pprint(flow_algo.list_algos())
pprint(flow_algo.list_algo_runs(algo_id))
# this submit will zip all the files in path_to_input folder and submit to flow
run_id =flow_algo.submit(algo_id,
                 "weekly_run",
                 {
                     "input_file": "input.csv",
                     "output_file": "output.csv",
                     "predict_start_week": "202348",
                     "predict_end_week": "202348",
                     "algo": "v03-percentile"
                 },
                 "path_to_input")
# flow_algo.terminate(run_id)
flow_algo.check_status(run_id)
flow_algo.log(run_id)
# this gather will download the output.tar.gz from flow and extract to path_to_output folder
flow_algo.gather(run_id,"./output")
# flow_algo_sdk.clear_local_algo_cache()
# clear local algo cache will delete the local history of submitted runs
```

## Development
### Regression Test
```bash
# set up environment variables
# FLOW_WORKSPACE_ID: your flow workspace id
# FLOW_API_TOKEN: your flow api token from the workspace
# FLOW_HOST: flow host
# you need to make sure the workspace has all the active apps
# you can run the following command to run the regression test
flow-cli regression-test
```