Package: openaoe
English |简体中文Latest Progress 🎉[February 2024] Add gemini-pro model[January 2024] refactor the config-template.yaml to control the backend and the frontend settings at the same time,clickto find more introduction about theconfig-template.yaml[January 2024] Add internlm2-chat-7b model[January 2024] Released version v0.0.1, officially open source！IntroductionWhat is OpenAOE?AOE, an acronym from DOTA2 for Area Of Effect, denotes an ability that can affect a group of targets within a certain area.
Here, AOE in AI implies that user can obtain parallel outputs from multiple LLMs with one single prompt at the same time.What problem does OpenAOE want to solve?Currently, there are many open-source frameworks based on the ChatGPT for chat, but the LGC(LLM Group Chat) framework is still not coming yet.The emergence of OpenAOE fills this gap:
OpenAOE can help LLM researchers, evaluators, engineering developers, and even non-professionals to quickly access the market's well-known commercial and open-source LLMs, providing both single model serial response mode and multi-models parallel response mode.What can you get from OpenAOE?OpenAOE can:return one or more LLMs' answersat the same timeby a single prompt.provide access to commercial LLM APIs, with default support for gpt3.5, gpt4, Google Palm, Minimax, Claude, Spark, etc., and also support user-defined access to other large model APIs. (API keys need to be prepared in advanced)provide access to open-source LLM APIs. ( We recommend to useLMDeployto deploy with one click)provide backend APIs and a WEB-UI to meet the needs of different requirements.Quick Run[!TIP]
Require python >= 3.9We provide three different ways to run OpenAOE:run by pip，run by dockerandrun by source codeas well.Run by pipInstallpipinstall-UopenaoeStartopenaoe-f/path/to/your/config-template.yamlRun by dockerInstallThere are two ways to get the OpenAOE docker image by:pull the OpenAOE docker imagedockerpullopensealion/openaoe:latestor build a docker imagegitclonehttps://github.com/internlm/OpenAOEcdOpenAOE
dockerbuild.-fdocker/Dockerfile-topensealion/openaoe:latestStartdockerrun-p10099:10099-v/path/to/your/config-template.yaml:/app/config.yaml--nameOpenAOEopensealion/openaoe:latestRun by source codeInstallclone this projectgitclonehttps://github.com/internlm/OpenAOE[optional] build the frontend project when the frontend codes are changedcdOpenAOE/openaoe/frontend
npminstall
npmrunbuildStartcdOpenAOE# this OpenAOE is the clone directorypipinstall-ropenaoe/backend/requirements.txt
python-mopenaoe.main-f/path/to/your/config-template.yaml[!TIP]/path/to/your/config-template.yamlis the configuration file loaded by OpenAOE at startup,
which contains the relevant configuration information for the LLMs,
including: API URLs, AKSKs, Tokens, etc.
A template configuration yaml file can be found inopenaoe/backend/config/config-template.yaml.Tech ReportYou are always welcome to fork this project to contribute your workand find theTODOs in furture.If you want to add more LLMs' APIs or features based on OpenAOE, the following info might be helpful.Tech StackThe technology stack we use includes:Backend framework based on python + fastapi;Frontend framework based on typescript + Sealion-Client (encapsulated based on React) + Sealion-UI.Build tools:conda: quickly create a virtual python env to install necessary packagesnpm: build the frontend project[!TIP]
The build tools can be installed quickly bypip install -U sealion-cliOrganization of the RepoFrontend codes are inopenaoe/frontendBackend codes are inopenaoe/backendProject entry-point isopenaoe/main.pyHow to add a new modelFrontendAdd new model info likename,avatar,provider, etc inopenaoe/frontend/src/config/model-config.tsAdd a new model basic API request payload configuration inopenaoe/frontend/src/config/api-config.tsModify your new model's payload specifically inopenaoe/frontend/src/services/fetch.ts, you may need to change the payload structure and handle corner cases according to your model's API definition.