semvec/pypi-packages huggingface dataset:
caikit-tgis-backend 0.1.39
Caikit module backend for models run in TGIS
License: Apache-2.0
Requires Python: ~=3.8

# Caikit Text Generation Inference Service (TGIS) Backend

This project provides a Caikit module backend that manages models run in TGIS

## Configuration

Sample configuration using the `MULTI` finder and a remote `TGIS` backend):

```yaml
runtime:
  library: caikit_nlp
  local_models_dir: /path/to/models
  lazy_load_local_models: true

model_management:
  finders:
    default:
      type: MULTI
      config:
        finder_priority:
          - local
          - tgis-auto
  initializers:
    default:
      type: LOCAL
      config:
        backend_priority:
          - type: TGIS
            config:
              connection:
                hostname: "localhost:8033"
                connect_timeout: 30
              test_connections: true

log:
  formatter: pretty
```