Package: llamakey
LlaMaKey: one master key for accessing all cloud LLM/GenAI APIsLlaMaKey is the easy and secure solution for key management of multiple users and multiple cloud LLM/GenAI APIs.
LlaMaKey gives each userone master keyto access multiple cloud AI APIs (OpenAI, Cohere, AnyScale, HuggingFace, Perplexity) through their official SDKs withno code changenor knowing the actual API keys. (How it works)For each user, one master key for multiple APIs.The master key is unique to each user. Granting and revoking a user's access won't impact other users.The actual API keys to authenticate with cloud APIs are never known to users.No code change! Keep using the official Python SDK of the cloud AI APIs.It solves the pain point of having too many keys to manage for both the user and the administrator and eliminates the risk of key leakage due to user carelessness.graph TD
   subgraph Your team
     A[User 1] -- Master Key 1 --> L["LlaMaKey server"]
     B[User 2] -- Master Key 2 --> L
     C[User 100] -- Master Key 100 --> L
   end
    L -- Actual <br> OPENAI_API_KEY --> O[OpenAI <br> endpoints]
    L -- Actual <br> CO_API_KEY--> P[Cohere <br> endpoints]
    L -- Actual <br> HF_API_KEY--> Q[HuggingFace  <br>endpoints]RoadmapSupported APIs:OpenAI (all endpoints)Cohere (all endpoints)AnyScale (AnyScale API is OpenAI-client compatible)Perplexity AI (Perplexity API is OpenAI-client compatible)HuggingFace Inference API (free tier)HuggingFace EndPoint APIAnthropicGoogle Vertex AIVectara AICurrently, authentication with the LlaMaKey server is not enabled. All users share the master keyLlaMaKey. If you want to see it, pleaseupvote here.InstallationStable version:pipinstallllamakeyNightly version: download fromhereFor building from source, seeBuild from source.UsageThe server endSet up the actual API keys as environment variables per their respective APIs, and then start the server, for example:# Step 1: Set the actual API keys as environment variablesexportOPENAI_API_KEY=sk-xxx# openaiexportCO_API_KEY=co-xxx# cohereexportHF_TOKEN=hf-xxx# huggingfaceexportANYSCALE_API_KEY=credential-xxx# anyscaleexportPERPLEXITY_API_KEY=pplx-xxx# perplexitylmk# Step 2: start the serverBy default, the server is started athttp://localhost:8000(8000 is the default port of FastAPI).Shell commands to activate proper environment variables on your client end will be printed, like this:exportOPENAI_BASE_URL="http://127.0.0.1:8000/openai"# direct OpenAI calls to the LlaMaKey serverexportCO_API_URL="http://127.0.0.1:8000/cohere"exportANYSCALE_BASE_URL="http://127.0.0.1:8000/anyscale"exportHF_INFERENCE_ENDPOINT="http://127.0.0.1:8000/huggingface"exportOPENAI_API_KEY="LlaMaKey"# One master key for all APIsexportCO_API_KEY="LlaMaKey"exportANYSCALE_API_KEY="LlaMaKey"exportHF_TOKEN="LlaMaKey"Such environment variables will direct the API calls to the LlaMaKey server. For your convenience, the commands are also dumped to the file./llamakey_client.env.The client endJust activate the environment variables generated above and then run your code as usual!
You may copy and paste the commands above or simply source thellamakey_client.envfile generated above, for example:# step 1: activate the environment variables that tell official SDKs to make requests to LlaMaKey serversourcellamakey_client.env# Step 2: Call offical Python SDKs as usual, for example, for OpenAI:python3-c'\from openai import OpenAI;client = OpenAI();print (\client.chat.completions.create(\model="gpt-3.5-turbo",\messages=[{"role": "user", "content": "How hot is the sun?"}]))'Build from sourceRequirements: git andRust Toolchain.gitclonegit@github.com:TexteaInc/LlaMasterKey.git# you can switch to a different branch:# git switch devcargobuild--release# binary at ./target/release/lmk# run it without installationcargorun# you can also install it system-widecargoinstall--path.# run itlmkHow LlaMaKey worksAs a proxy, LlaMaKey takes advantage of a feature in the Python SDK of most cloud LLM/GenAI APIs that they allow setting the base URL and API keys/tokens to and with which a request is sent and authenticated (OpenAI's,Cohere's). The base URL and API key can be set easily via environment variables. So a client just needs to set such environment variables (or manually configure in their code) and then call the APIs as usual -- seehow simple and easy. LlaMaKey will receive the request, authenticate the user (if authentication is enabled), and then forward the request to the corresponding actual cloud API with an actual API key (set by the administrator when starting a LlaMaKey server). The response will be passed back to the client after the LlaMaKey server hears back from a cloud API.LicenseAh, this is important. Let's say MIT for now?ContactFor usage, bugs, or feature requests, please open an issue on Github. For private inquiries, please emailhello@LlaMaKey.ai.