Package: faststream
FastStreamEffortless event stream integration for your servicesFeaturesFastStreamsimplifies the process of writing producers and consumers for message queues, handling all the
parsing, networking and documentation generation automatically.Making streaming microservices has never been easier. Designed with junior developers in mind,FastStreamsimplifies your work while keeping the door open for more advanced use cases. Here's a look at the core features that makeFastStreama go-to framework for modern, data-centric microservices.Multiple Brokers:FastStreamprovides a unified API to work across multiple message brokers (Kafka,RabbitMQ,NATS,Redissupport)Pydantic Validation: LeveragePydantic'svalidation capabilities to serialize and validates incoming messagesAutomatic Docs: Stay ahead with automaticAsyncAPIdocumentationIntuitive: Full-typed editor support makes your development experience smooth, catching errors before they reach runtimePowerful Dependency Injection System: Manage your service dependencies efficiently withFastStream's built-in DI systemTestable: Supports in-memory tests, making your CI/CD pipeline faster and more reliableExtensible: Use extensions for lifespans, custom serialization and middlewareIntegrations:FastStreamis fully compatible with any HTTP framework you want (FastAPIespecially)Built for Automatic Code Generation:FastStreamis optimized for automatic code generation using advanced models like GPT and LlamaThat'sFastStreamin a nutshell—easy, efficient, and powerful. Whether you're just starting with streaming microservices or looking to scale,FastStreamhas got you covered.Documentation:https://faststream.airt.ai/latest/HistoryFastStreamis a new package based on the ideas and experiences gained fromFastKafkaandPropan. By joining our forces, we picked up the best from both packages and created a unified way to write services capable of processing streamed data regardless of the underlying protocol. We'll continue to maintain both packages, but new development will be in this project. If you are starting a new service, this package is the recommended way to do it.InstallFastStreamworks onLinux,macOS,Windowsand mostUnix-style operating systems.
You can install it withpipas usual:pipinstallfaststream[kafka]# orpipinstallfaststream[rabbit]# orpipinstallfaststream[nats]# orpipinstallfaststream[redis]By defaultFastStreamusesPydanticV2written inRust, but you can downgrade it manually, if your platform has noRustsupport -FastStreamwill work correctly withPydanticV1as well.Writing app codeFastStreambrokers provide convenient function decorators@broker.subscriberand@broker.publisherto allow you to delegate the actual process of:consuming and producing data to Event queues, anddecoding and encoding JSON-encoded messagesThese decorators make it easy to specify the processing logic for your consumers and producers, allowing you to focus on the core business logic of your application without worrying about the underlying integration.Also,FastStreamusesPydanticto parse input
JSON-encoded data into Python objects, making it easy to work with structured data in your applications, so you can serialize your input messages just using type annotations.Here is an example Python app usingFastStreamthat consumes data from an incoming data stream and outputs the data to another one:fromfaststreamimportFastStreamfromfaststream.kafkaimportKafkaBroker# from faststream.rabbit import RabbitBroker# from faststream.nats import NatsBroker# from faststream.redis import RedisBrokerbroker=KafkaBroker("localhost:9092")# broker = RabbitBroker("amqp://guest:guest@localhost:5672/")# broker = NatsBroker("nats://localhost:4222/")# broker = RedisBroker("redis://localhost:6379/")app=FastStream(broker)@broker.subscriber("in")@broker.publisher("out")asyncdefhandle_msg(user:str,user_id:int)->str:returnf"User:{user_id}-{user}registered"Also,Pydantic’sBaseModelclass allows you
to define messages using a declarative syntax, making it easy to specify the fields and types of your messages.frompydanticimportBaseModel,Field,PositiveIntfromfaststreamimportFastStreamfromfaststream.kafkaimportKafkaBrokerbroker=KafkaBroker("localhost:9092")app=FastStream(broker)classUser(BaseModel):user:str=Field(...,examples=["John"])user_id:PositiveInt=Field(...,examples=["1"])@broker.subscriber("in")@broker.publisher("out")asyncdefhandle_msg(data:User)->str:returnf"User:{data.user}-{data.user_id}registered"Testing the serviceThe service can be tested using theTestBrokercontext managers, which, by default, puts the Broker into "testing mode".The Tester will redirect yoursubscriberandpublisherdecorated functions to the InMemory brokers, allowing you to quickly test your app without the need for a running broker and all its dependencies.Using pytest, the test for our service would look like this:# Code above omitted 👆importpytestimportpydanticfromfaststream.kafkaimportTestKafkaBroker@pytest.mark.asyncioasyncdeftest_correct():asyncwithTestKafkaBroker(broker)asbr:awaitbr.publish({"user":"John","user_id":1,},"in")@pytest.mark.asyncioasyncdeftest_invalid():asyncwithTestKafkaBroker(broker)asbr:withpytest.raises(pydantic.ValidationError):awaitbr.publish("wrong message","in")Running the applicationThe application can be started using built-inFastStreamCLI command.To run the service, use theFastStream CLIcommand and pass the module (in this case, the file where the app implementation is located) and the app symbol to the command.faststreamrunbasic:appAfter running the command, you should see the following output:INFO-FastStreamappstarting...
INFO-input_data|-`HandleMsg`waitingformessages
INFO-FastStreamappstartedsuccessfully!ToexitpressCTRL+CAlso,FastStreamprovides you with a great hot reload feature to improve your Development Experiencefaststreamrunbasic:app--reloadAnd multiprocessing horizontal scaling feature as well:faststreamrunbasic:app--workers3You can learn more aboutCLIfeatureshereProject DocumentationFastStreamautomatically generates documentation for your project according to theAsyncAPIspecification. You can work with both generated artifacts and place a web view of your documentation on resources available to related teams.The availability of such documentation significantly simplifies the integration of services: you can immediately see what channels and message formats the application works with. And most importantly, it won't cost anything -FastStreamhas already created the docs for you!DependenciesFastStream(thanks toFastDepends) has a dependency management system similar topytest fixturesandFastAPI Dependsat the same time. Function arguments declare which dependencies you want are needed, and a special decorator delivers them from the global Context object.fromfaststreamimportDepends,Loggerasyncdefbase_dep(user_id:int)->bool:returnTrue@broker.subscriber("in-test")asyncdefbase_handler(user:str,logger:Logger,dep:bool=Depends(base_dep)):assertdepisTruelogger.info(user)HTTP Frameworks integrationsAny FrameworkYou can useFastStreamMQBrokerswithout aFastStreamapplication.
Juststartandstopthem according to your application's lifespan.fromaiohttpimportwebfromfaststream.kafkaimportKafkaBrokerbroker=KafkaBroker("localhost:9092")@broker.subscriber("test")asyncdefbase_handler(body):print(body)asyncdefstart_broker(app):awaitbroker.start()asyncdefstop_broker(app):awaitbroker.close()asyncdefhello(request):returnweb.Response(text="Hello, world")app=web.Application()app.add_routes([web.get("/",hello)])app.on_startup.append(start_broker)app.on_cleanup.append(stop_broker)if__name__=="__main__":web.run_app(app)FastAPIPluginAlso,FastStreamcan be used as part ofFastAPI.Just import aStreamRouteryou need and declare the message handler with the same@router.subscriber(...)and@router.publisher(...)decorators.fromfastapiimportFastAPIfrompydanticimportBaseModelfromfaststream.kafka.fastapiimportKafkaRouterrouter=KafkaRouter("localhost:9092")classIncoming(BaseModel):m:dict@router.subscriber("test")@router.publisher("response")asyncdefhello(m:Incoming):return{"response":"Hello, world!"}app=FastAPI(lifespan=router.lifespan_context)app.include_router(router)More integration features can be foundhereCode generatorAs evident,FastStreamis an incredibly user-friendly framework. However, we've taken it a step further and made it even more user-friendly! Introducingfaststream-gen, a Python library that harnesses the power of generative AI to effortlessly generateFastStreamapplications. Simply describe your application requirements, andfaststream-genwill generate a production-gradeFastStreamproject that is ready to deploy in no time.Save application description insidedescription.txt:Create a FastStream application using localhost broker for testing and use the
default port number.

It should consume messages from the 'input_data' topic, where each message is a
JSON encoded object containing a single attribute: 'data'.

While consuming from the topic, increment the value of the data attribute by 1.

Finally, send message to the 'output_data' topic.and run the following command to create a newFastStreamproject:faststream_gen-idescription.txt✨GeneratinganewFastStreamapplication!✔Applicationdescriptionvalidated.✔FastStreamappskeletoncodegenerated.akesaround15to45seconds)...✔Theappandthetestsaregenerated.around30to90seconds)...✔NewFastStreamprojectcreated.✔Integrationtestsweresuccessfullycompleted.Tokensused:10768TotalCost(USD):$0.03284
✨Allfilesweresuccessfullygenerated!TutorialWe also invite you to explore our tutorial, where we will guide you through the process of utilizing thefaststream-genPython library to effortlessly createFastStreamapplications:Cryptocurrency analysis with FastStreamStay in touchPlease show your support and stay in touch by:giving ourGitHub repositorya star, andjoining ourDiscord serverYour support helps us to stay in touch with you and encourages us to
continue developing and improving the framework. Thank you for your
support!ContributorsThanks to all of these amazing people who made the project better!