Package: fastlmi
FastLMIFastLMI is a modern, fast (both in performance and development time) framework for creating LMIs, based on the belovedFastAPIlibrary.What is an LMI?and why not just call it an API?LMI stands for "Language Model Interface" -- it's a catch-all term for tools given to AI agents that can interact
with them, and how we define those tools (interfaces). We aren't providing an application programming interface;
rather than providing instructions (in the form of docs) for human developers to write apps, LMIs come bundled with
instructions for AI agents to interface with them.FastLMI is more than just a library for making ChatGPT plugins -- we believe the LMI ecosystem has the potential to be
and do much more without relying on a single authority to curate and provide "good" plugins. FastLMI is designed to be
ecosystem-agnostic with adapters for popular ecosystems, such as ChatGPT/OpenAI plugins.Cite UsDoing academic research on language models and their abilities to use tools using the FastLMI library? Cite us with
this BibTex entry!@software{Zhu_FastLMI_2023,author={Zhu, Andrew},doi={10.5281/zenodo.7916737},month=may,title={{FastLMI}},url={https://github.com/zhudotexe/fastlmi},version={0.1.0},year={2023}}RequirementsPython 3.8+Installation$pipinstallfastlmiJust as with FastAPI, you will need an ASGI server to run the app, such asUvicorn.$pipinstalluvicorn# or pip install "uvicorn[standard]" for Cython-based extrasExample (OpenAI/ChatGPT Plugin)NOTE: As of v0.2.0 FastLMI includes the AI Plugin (OpenAI, LangChain) interface as a default. This may change to an
extension-based system in the future as the library develops.To show off just how easy it is to create a plugin, let's make a ChatGPT plugin that gives it the ability to roll dice
in the d20 format (AIs playing D&D, anyone?).Example RequirementsFirst, you'll need to install thed20library:$pipinstalld20Create itThen, create amain.pyfile.importd20# pip install d20fromfastlmiimportFastLMI,utilsfrompydanticimportBaseModelapp=FastLMI(title="Dice Roller",name_for_model="DiceRoller",description="A simple plugin to roll dice.",description_for_model=("DiceRoller can roll dice in XdY format and do math.\n""Some dice examples are:\n""4d6kh3 :: highest 3 of 4 6-sided dice\n""2d6ro<3 :: roll 2d6s, then reroll any 1s or 2s once\n""8d6mi2 :: roll 8d6s, with each die having a minimum roll of 2\n""(1d4 + 1, 3, 2d6kl1)kh1 :: the highest of 1d4+1, 3, and the lower of 2 d6s\n""Normal math operations are also supported."),contact_email="foo@example.com",legal_url="https://example.com/legal",)# use this when developing localhost plugins to allow the browser to make the local requestutils.cors_allow_openai(app)classDiceRequest(BaseModel):dice:str# the dice to rollclassDiceResponse(BaseModel):result:str# the resulting dice stringtotal:int# the total numerical result of the roll (rounded down to nearest integer)@app.post("/roll")defroll(dice:DiceRequest)->DiceResponse:"""Roll the given dice and return a detailed result."""result=d20.roll(dice.dice)returnDiceResponse(result=result.result,total=result.total)(this example script is also available atexamples/dice_roller.py!)Run it... and run it with:$uvicornmain:app

INFO:Uvicornrunningonhttp://127.0.0.1:8000(PressCTRL+Ctoquit)INFO:Startedserverprocess[53532]INFO:Waitingforapplicationstartup.
INFO:Applicationstartupcomplete.Register itFinally, we need to tell ChatGPT about the new plugin.
In the ChatGPT interface, select the "Plugins" model, then head to the Plugin Store -> Develop your own plugin.Here, type in the address of your plugin. By default, it'slocalhost:8000.Click "Find manifest file," and you should see your plugin appear. FastLMI automatically handles generating all the
plugin metadata needed by OpenAI!Chat awayTo use your new plugin, select it from the list of plugins when starting a new chat:and start chatting. Congratulations! 🎉 You've just created a brand-new ChatGPT plugin - and we're excited to see what
else you'll make!Example (LangChain Tool)LangChain supportsinterfacing with AI plugins!
If you've followed the steps above (at least up through "Run it"), you can also expose your new LMI to a LangChain
agent.This example assumes that you
haveLangChaininstalled, and that your
LMI is running athttp://localhost:8000.fromlangchain.agentsimportAgentTypefromlangchain.agentsimportinitialize_agent,load_toolsfromlangchain.chat_modelsimportChatOpenAIfromlangchain.toolsimportAIPluginTooltool=AIPluginTool.from_plugin_url("http://localhost:8000/.well-known/ai-plugin.json")llm=ChatOpenAI(temperature=0)tools=load_tools(["requests_all"])tools+=[tool]agent_chain=initialize_agent(tools,llm,agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,verbose=True)agent_chain.run("Can you roll me stats for a D&D character?")AuthenticationAs of v0.2.0, FastLMI has built-in support for service-level auth, where the AI agent or LMI driver will send an
authorization token you decide as a header with each request.Enabling authentication is easy! First, define the authentication scheme you wish to use - this will be a subclass ofLMIAuth. For example, to provide service-level auth, you can define theLMIServiceAuthscheme:fromfastlmiimportDepends,FastLMIfromfastlmi.authimportLMIServiceAuthauth=LMIServiceAuth(access_tokens=["your_secret_token_here"],verification_tokens={"openai":"verification_token_generated_in_the_ChatGPT_UI"})This auth scheme allows defining a set of allowed access tokens (if one wanted to, for example, have a different
token for each plugin service). Then, when you define your app, all you have to do is add 2 parameters:app=FastLMI(...,auth=auth,dependencies=[Depends(auth)])Tada! 🎉 Your LMI now tells consumers that it uses theservice_httpauth scheme, and will validate that each request
to one of your defined routes provides a valid Bearer token.A complete example is inexamples/ai_plugin_auth.py. You can read more about OpenAI's service-level
authhere.Route-Level AuthIf you wanted to only require that certain routes use auth, you can also define the auth dependency on a route level:- app = FastLMI(..., auth=auth, dependencies=[Depends(auth)])+ app = FastLMI(..., auth=auth)...- @app.post("/hello")+ @app.post("/hello", dependencies=[Depends(auth)])def hello():...Read MoreBeing based on FastAPI, FastLMI can take full advantage of its superpowers. Check out
theFastAPI documentationfor more!Todoscopesscript to check for missing docs, over limits, etcloggingconfigure ecosystems