# Dynaseal

## Why this project was created

Imagine a future where LLM agents are widely deployed on edge devices. But currently, the way to call a large model is through an API key that anyone can use once they have it. What if users get the API key from the model on their device and can spam your services, skyrocketing your costs? If each user runs multiple agents on their phone, and you have a large user base, your servers could be overwhelmed, even worse than a DDoS attack! With DDoS, at least your service is disrupted, but in this case, your legitimate business is still running while your servers get flooded.

Thus, we propose [Dynaseal](https://github.com/IntelliKernel/Dynaseal), a future-oriented backend framework for managing LLM model calls from edge-side agents. It uses a dynamic token system similar to OSS, which restricts the models, parameters, and token lifetimes available to edge agents. It also supports direct communication between edge devices and large model service providers, and after each response, it triggers a callback to inform your backend of the transaction. We welcome any feedback and contributions; feel free to star our project!

> Please note that this project is just a demo. It implements our concept of dynamically distributing API keys for edge-side large model requests. The large model part is designed as a wrapper around an OpenAI-style large model server. On the client-side, we only implement basic integrity and parameter validation checks for keys. Callback functionality is limited to printing the results on the terminal.

## System Design

The architecture consists of three components: the LLM server, the backend, and the client.

1. The backend initializes by requesting the LLM server for an API key to access the large model.
2. The client initializes by requesting authentication from the backend. The backend returns a dynamic key specifying the model, token, request address, and other essential details.
3. The client uses the dynamic key to request the model from the LLM server.
4. The LLM server unpacks the dynamic key, verifies the identity, and returns a response.
5. The LLM server notifies the backend via a callback about the client’s request and the response.

![](https://cdn.studyinglover.com/pic/2024/10/63d54f64144c49f3a7...)

```mermaid
sequenceDiagram
participant Backend as Backend
participant Client as Client
participant LLM_Server as LLM Server

Note over Backend: Initialization
Backend->>LLM_Server: Request large model API key
LLM_Server-->>Backend: Return API key

Note over Client: Initialization
Client->>Backend: Request authentication
Backend-->>Client: Return dynamic key, including allowed model, token, etc.

Client->>LLM_Server: Use dynamic key to request model
LLM_Server->>LLM_Server: Unpack dynamic key, verify identity
LLM_Server-->>Client: Return response

LLM_Server->>Backend: Notify of client request and response

```

## Dynamic Key Design

The dynamic key is designed as a JWT token in the format: `header.payload.secret`.

- *Header*
- *Payload*: First, a JSON object like the one below is created and packed into the JWT payload:

```json
{
"api-key": 111, // The user ID registered by the backend on the LLM server. Note, it's the ID, not the key given by the LLM server.
"model": "deepseek-chat", // The model name that the client is allowed to call
"max_tokens": 100, // Maximum number of tokens that can be used
"expiring": 111, // Expiration time using Unix timestamp
"event_id": 111 // Event ID, used to identify this token. After the LLM server responds to the request, this ID will be included in the callback
}
```

- *Secret*: This is encrypted using the key that the backend registered with the LLM server, and the LLM server will use it to verify if the dynamic key is valid.