zeblok 1.0.0
Zeblok AI SDK for accessing pretrained models
Requires Python: >=3.7

# Zeblok GenAI SDK 

The Zeblok SDK allows users to integrate their external MLOps workflows with the AI-MicroCloud platform. Specifically, this new release will add functionality and support for interactions with AI models.

## Components

### Auth

- API Authentication

### Resources

- AI-API
- Client
- Embedding Model
- GPT Model
- Inference Chat

## Interface

<h3> Authentication </h3>

- API Authentication: `APIKeyAuth()`

<h3> Client </h3>

- Create a new client: `Client(api_key)`
- Authenticate API key: `_authenticate_api_key()`
- Add a new model to the client: `_register_model(model_name, model_class)`
- List all models associated with the client: `list_models()`
- Make a request to Zeblok AI MicroCloud: `_request(method, endpoint, params)`

<h3> Models </h3>

- GPT Model: `GPTModel(model_name, client, api_key)`
- Embedding Mode: `EmbeddingModel(model_name, client, api_key)`
- Get completions (only for GPTModel): `completions(prompt, max_tokens=50, temperature=0.7, **kwargs)`
- Get chat history (only for GPTModel): `get_chat_history()`

# Usage

### Setup

To set up the environment, open a terminal and install `chromadb` and `numpy` with the following commands:
- `pip install numpy==1.26.3`
- `pip install chromadb==0.5.0`
Next, go to `utils/credentials.py` and enter your API key (retrieved from your account on Zeblok AI MicroCloud) and the name of the model you want to use. Remember that this is the model type, and NOT the display name.

### API Key Authentication
We use `API Key Authentication` to verify users' access to models within the MicroCloud. When the user creates a client, they must pass their API key as part of the instantiation process. This is then passed to the MicroCloud for verification.

If `api_key` is `empty`, `None`, or invalid, an `AuthenticationError` is raised.

### Model Interaction
Using the SDK, we can:
- Create a client
- Register models to the client
- Interact with those models and get responses to queries

#### Create a client

Exits sucessfully if the client is successfully created.

Uses
- `api_key`: User's API key (retrieved form AI MicroCloud)

Raises
- `AuthenticationError` if the user is not properly authenticated.
- `InvalidRequestError` if the user makes an invalid request.
- `APIError` if there is unintended or unexpected behavior by the application.

##### Create a client with API key
``` python
from client import Client
client = Client(api_key=f'<your-api-key>') # exits successfully if no errors, otherwise an error message is displayed
```

#### Register a model to a client

Exits successfully if the model is successfully registered.

Uses
- `model_name`: The name of the AI model being used. This is NOT the display name; it is the model type (e.g. `meta-llama/Meta-Llama-3-8B-Instruct`).
- `model-class`: The class of the AI model as per the SDK. This will either be `GPTModel` or `EmbeddingModel` (right now only `GPTModel` can be used, since embeddings are not yet fully integrated).

Raises
- `AuthenticationError` if the user is not properly authenticated.
- `InvalidRequestError` if the user makes an invalid request.
- `APIError` if there is unintended or unexpected behavior by the application.

##### Register a model to the client with a specified model name and class
```python
from client import *
client = Client(api_key=f'<your-api-key>')
client.register_model(f'<your-model-name>', f'<your-model-class>') 
```

#### Interact with the model
Given a prompt from the user, the API will be able to pass it to the LLM and retrieve an appropriate response. The implementation is detailed below, and can be found in `models.py`.

Uses
- `prompt`: The user's message/query for the AI model.
- `max_tokens`: Maximum amount of tokens for the model's response to use. Default to 50 is none is provided.
- `temperature`: Sampling temperature for the model. Default to 0.7 if none is provided.

Exact details of request and response
- Endpoint: `/api/v1/spawnedInferences/sdk/chat/completion/`
- Request body contains the following parameters. Of these five, ONLY the modelName and the api_key are strictly required.
    - `modelName`: the name of the model type (e.g. `meta-llama/Meta-Llama-3-8B-Instruct`)
    - `prompt`: prompt for the AI model from the user
    - `max_tokens`: maximum number of tokens (characters) to be generated and returned by the model
    - `temperature`: sampling temperature for the model
    - `api_key`: user's API key
- Response body contains the following important information:
    - `success`: Boolean value, determines whether the request was successful or not
    - `status_code`: Numeric status code of the request
    - `choices`: actual data responding to the user's query. This is a list object that has several elements, key among them being the message dictionary:
        - `message`: Dictionary object containing the model's answer, accessible via the `content` key

If the completion is successful, then the prompt, response, and timestamp are saved in a dictionary, formatted accordingly
- `prompt`: user prompt to the model
- `response`: model response
- `timestamp`: datetime object with the time that the request was made

Raises: 
- `AuthenticationError` if the user is not properly authenticated.
- `InvalidRequestError` if the user makes an invalid request.
- `RateLimitError` if the rate limit of the model is exceeded.
- `ServiceUnavailableError` if the model currently is not available, e.g. offline.
- `APIError` if there is generic, unintended, or unexpected behavior by the application.

##### Interact with a specific model through the client
```python
from client import *
client = Client(api_key=f'<your-api-key>')
client.register_model(f'<your-model-name>', f'<your-model-class>')
output = client.models[f'<your-model-name>'].completions(f'<your-prompt>', f'<your-max-tokens>', f'<your-temperature>')
# if successful, then the output variable will contain the model response
# additionally, when you retrieve the chat history of the model, you should see your query and model response
```
  
# Testing

Check out [TESTRESULTS.md](./zeblok/TESTRESULTS.md) for more information.