Package: semantic-router
Semantic RouterSemantic Router is a superfast decision-making layer for your LLMs and agents. Rather than waiting for slow LLM generations to make tool-use decisions, we use the magic of semantic vector space to make those decisions —routingour requests usingsemanticmeaning.QuickstartTo get started withsemantic-routerwe install it like so:pip install -qU semantic-router❗️If wanting to use a fully local version of semantic router you can useHuggingFaceEncoderandLlamaCppLLM(pip install -qU "semantic-router[local]", seehere). To use theHybridRouteLayeryou mustpip install -qU "semantic-router[hybrid]".We begin by defining a set ofRouteobjects. These are the decision paths that the semantic router can decide to use, let's try two simple routes for now — one for talk onpoliticsand another forchitchat:fromsemantic_routerimportRoute# we could use this as a guide for our chatbot to avoid political conversationspolitics=Route(name="politics",utterances=["isn't politics the best thing ever","why don't you tell me about your political opinions","don't you just love the president""don't you just hate the president","they're going to destroy this country!","they will save the country!",],)# this could be used as an indicator to our chatbot to switch to a more# conversational promptchitchat=Route(name="chitchat",utterances=["how's the weather today?","how are things going?","lovely weather today","the weather is horrendous","let's go to the chippy",],)# we place both of our decisions together into single listroutes=[politics,chitchat]We have our routes ready, now we initialize an embedding / encoder model. We currently support aCohereEncoderandOpenAIEncoder— more encoders will be added soon. To initialize them we do:importosfromsemantic_router.encodersimportCohereEncoder,OpenAIEncoder# for Cohereos.environ["COHERE_API_KEY"]="<YOUR_API_KEY>"encoder=CohereEncoder()# or for OpenAIos.environ["OPENAI_API_KEY"]="<YOUR_API_KEY>"encoder=OpenAIEncoder()With ourroutesandencoderdefined we now create aRouteLayer. The route layer handles our semantic decision making.fromsemantic_router.layerimportRouteLayerrl=RouteLayer(encoder=encoder,routes=routes)We can now use our route layer to make super fast decisions based on user queries. Let's try with two queries that should trigger our route decisions:rl("don't you love politics?").name[Out]: 'politics'Correct decision, let's try another:rl("how's the weather today?").name[Out]: 'chitchat'We get both decisions correct! Now lets try sending an unrelated query:rl("I'm interested in learning about llama 2").name[Out]:In this case, no decision could be made as we had no matches — so our route layer returnedNone!📚 ResourcesDocsNotebookDescriptionIntroductionIntroduction to Semantic Router and static routesDynamic RoutesDynamic routes for parameter generation and functionc callsSave/Load LayersHow to save and loadRouteLayerfrom fileLocal ExecutionFully local Semantic Router with dynamic routes —local models such as Mistral 7B outperform GPT-3.5 in most testsLangChain IntegrationHow to integrate Semantic Router with LangChain AgentsOnline CourseCommunityJulian Horsey,Semantic Router superfast decision layer for LLMs and AI agents, Geeky Gadgetsazhar,Beyond Basic Chatbots: How Semantic Router is Changing the Game, AI Insights @ MediumDaniel Avila,Semantic Router: Enhancing Control in LLM Conversations, CodeGPT @ MediumYogendra Sisodia,Stop Chat-GPT From Going Rogue In Production With Semantic Router, MediumAniket Hingane,LLM Apps: Why you Must Know Semantic Router in 2024: Part 1, Medium