Package: langstream
🪽🔗 LangStream⚠️ Heads up! LiteChain was renamed to LangStream, for more details, check outissue #4LangStream is a lighter alternative to LangChain for building LLMs application, instead of having a massive amount of features and classes, LangStream focuses on having a single small core, that is easy to learn, easy to adapt, well documented, fully typed and truly composable, with Streams instead of chains as the building block.DocumentationQuick Installpip install langstream openai🔗 The Stream building blockThe Stream is the building block for LangStream, an LLM is a Stream, an output parser is a Stream, a group of streams can be composed as another Stream, it'sStreams all the way down.Take a look atthe documentationfor guides on building on streams and building LLM applications, or go straight tothe referencefor the core concept and modules available.Quick ExampleHere is a ChatBot that answers anything you ask using only emojis:fromlangstream.contribimportOpenAIChatStream,OpenAIChatMessage,OpenAIChatDeltafromtypingimportIterable# Creating a GPT-4 EmojiStreamemoji_stream=OpenAIChatStream[str,OpenAIChatDelta]("EmojiStream",lambdauser_message:[OpenAIChatMessage(role="user",content=f"{user_message}. Reply in emojis")],model="gpt-4",temperature=0,)# Now interacting with itasyncforoutputinemoji_stream("Hey there, how is it going?"):print(output.data.content,end="")#=> 👋😊👍💻🌞asyncforoutputinemoji_stream("What is answer to the ultimate question of life, the universe, and everything?"):print(output.data.content,end="")#=> 4️⃣2️⃣In this simple example, we are creating aGPT4 Streamthat takes the user message and appends". Reply in emojis"to it for building the prompt, following theOpenAI chat structureand withzero temperature.Then, as you can see, we have an async loop going over each token output fromemoji_stream. In LangStream, everything is an async stream using Python'sAsyncGeneratorclass, and the most powerful part of it, is that you can connect those streams by composing two Streams together:# Creating another Stream to translate back from emojitranslator_stream=OpenAIChatStream[Iterable[OpenAIChatDelta],OpenAIChatDelta]("TranslatorStream",lambdaemoji_tokens:[OpenAIChatMessage(role="user",content=f"Translate this emoji message{[token.contentfortokeninemoji_tokens]}to plain english")],model="gpt-4",)# Connecting the two Streams togetherstream=emoji_stream.and_then(translator_stream)# Trying out the whole flowasyncforoutputinstream("Hey there, how is it going?"):print(output.data.content,end="")#=> 👋😊👍💻🌞"Hello, have a nice day working on your computer!"As you can see, it's easy enough to connect two Streams together using theand_thenfunction. There are other functions available for composition such asmap,collect,joinandgather, they form the small set of abstractions you need to learn to build complex Stream compositions for your application, and they behave as you would expect if you have Function Programming knowledge. You can read all about it in thereference. Once you learn those functions, any Stream will follow the same patterns, enabling you to build complex LLM applications.As you may also have noticed, Streams accept type signatures, EmojiStream has the type[str, OpenAIChatDelta], while TranslatorStream has the type[Iterable[OpenAIChatDelta], OpenAIChatDelta], those mean respectively theinputandoutputtypes of each Stream. Since the EmojiStream is taking user output, it simply takes astras input, and since it's using OpenAI Chat API with GPT-4, it producesOpenAIChatDelta, which isthe tokens that GPT-4 produces one at a time. TranslatorStream then takesIterable[OpenAIChatDelta]as input, since it's connected with the output from EmojiStream, it takes the full list of the generated tokens to later extract their content and form its own prompt.The type signatures are an important part of LangStream, having them can save a lot of time preventing bugs and debugging issues caused for example when Stream B is not expecting the output of Stream A. Using an editor like VSCode with PyLance allows you to get warned that Stream A doesn't fit into Stream B before you even try to run the code, you can read about LangStream typinghere.Last but not least, you may also have noticed that both the emojis and the translation got printed in the final output, this is by design. In LangStream, you always have access to everything that has gone through the whole stream in the final stream, this means that debugging it is very trivial, and adebugfunction is available to make it even easier. A propertyoutput.final : boolis availableto be checked if you want to print just the results of the final Stream, but there are also more utility functions available to help you work with output stream as you wish, check out more about it on ourWhy Streams? guideandthe reference.Prompts on the outsideIn our experience, when working with LLM applications, the main part you must spend tunning are your prompts, which are not always portable if you switch LLMs. The content one stream produces might change a lot how another stream should be written, the prompt carry the personality and the goal of your app, doing good prompt engineering can really make it or break it.That's why LangStream does not hide prompts away in agents, we will give examples in the documentation, but believe you should build your own agents, to be able to customize them and their prompts later. LangStream simply wants to facilitate and standardize the piping and connection between different parts, so you can focus on what is really important, we don't want you to spend time with LangStream itself.Bring your own integrationIn addition, as the name implies, LangStream wants to stay light, not embrace the world, the goal is that you really understand the Stream, making it very easy for your to add your own integration, without any additional layers in between.In our experience, wrappers can hurt more than they help, because instead of using the library or API you want to connect directly, now you need to learn another layer of indirection, which might not accept the same parameters to work the way you expect, it gets in the way.We do provide some integrations for OpenAI and GPT4All for example, but then we try to have a very thin layer, and to stay as close as possible to the original API, to the point that you can use the oficial documentation for it.📖 Learn moreTo continue developing with LangStream, take a look at ourdocumentationso you can find:Getting startedDetailed guidesHow-to examplesReference👥 CommunityJoin our discordcommunity to connect with other LangStream developers, ask questions, get support, and stay updated with the latest news and announcements.🚙 RoadmapAdd docs for debuggingAdd a simple default memory mechanismConsider RxPY as the underlying stream handlerAdd a compat adapter to reuse LangChain tools with LangStream🙋 ContributingAs a very new project in a rapidly developing field LangStream is extremely open to contributions, we need a lot of help with integrations, documentation and guides content, feel free to send MRs and open issues. The project is very easy to run (check out the Makefile, it's all you need), but more complete contibuting guidelines to be written (we need help with that too!)If you want to help me pay the bills and keep developing this project, you can: