Package: vidgear
Releases|Gears|Documentation|Installation|LicenseVidGear is aHigh-Performance Video Processing Python Librarythat provides an easy-to-use, highly extensible, thoroughly optimisedMulti-Threaded + Asyncio API Frameworkon top of many state-of-the-art specialized libraries likeOpenCV,FFmpeg,ZeroMQ,picamera,starlette,yt_dlp,pyscreenshot,dxcam,aiortcandpython-mssserving at its backend, and enable us to flexibly exploit their internal parameters and methods, while silently deliveringrobust error-handling and real-time performance 🔥VidGear primarily focuses on simplicity, and thereby lets programmers and software developers to easily integrate and perform Complex Video Processing Tasks, in just a few lines of code.The followingfunctional block diagramclearly depicts the generalized functioning of VidGear APIs:Table of ContentsTL;DRGetting StartedGears: What are these?CamGearPiGearVideoGearScreenGearWriteGearStreamGearNetGearWebGearWebGear_RTCNetGear_AsyncContributionsDonationsCitationCopyrightTL;DRWhat is vidgear?"VidGear is a cross-platform High-Performance Framework that provides an one-stopVideo-Processingsolution for building complex real-time media applications in python."What does it do?"VidGear can read, write, process, send & receive video files/frames/streams from/to various devices in real-time, andfasterthan underline libraries."What is its purpose?"Write Less and Accomplish More"—VidGear's Motto"Built with simplicity in mind, VidGear lets programmers and software developers to easily integrate and performComplex Video-Processing Tasksin their existing or newer applications without going through hefty documentation and in just afew lines of code. Beneficial for both, if you're new to programming with Python language or already a pro at it."Getting StartedIf this is your first time using VidGear, head straight to theInstallation >>to install VidGear.Once you have VidGear installed,Checkout its Well-DocumentedFunction-Specific Gears >>Also, if you're already familiar withOpenCVlibrary, then seeSwitching from OpenCV Library >>Or, if you're just getting started with OpenCV-Python programming, then refer thisFAQ >>Gears: What are these?VidGear is built with multiple APIs a.k.aGears, each with some unique functionality.Each API is designed exclusively to handle/control/process different data-specific & device-specific video streams, network streams, and media encoders/decoders. These APIs provides the user an easy-to-use, dynamic, extensible, and exposed Multi-Threaded + Asyncio optimized internal layer above state-of-the-art libraries to work with, while silently delivering robust error-handling.These Gears can be classified as follows:A. Video-Capture Gears:CamGear:Multi-Threaded API targeting various IP-USB-Cameras/Network-Streams/Streaming-Sites-URLs.PiGear:Multi-Threaded API targeting various Raspberry-Pi Camera Modules.ScreenGear:High-performance API targeting rapid Screencasting Capabilities.VideoGear:Common Video-Capture API with internalVideo Stabilizerwrapper.B. Video-Writer Gears:WriteGear:Handles Lossless Video-Writer for file/stream/frames Encoding and Compression.C. Streaming Gears:StreamGear: Handles Transcoding of High-Quality, Dynamic & Adaptive Streaming Formats.Asynchronous I/O Streaming Gear:WebGear:ASGI Video-Server that broadcasts Live MJPEG-Frames to any web-browser on the network.WebGear_RTC:Real-time Asyncio WebRTC media server for streaming directly to peer clients over the network.D. Network Gears:NetGear:Handles High-Performance Video-Frames & Data Transfer between interconnecting systems over the network.Asynchronous I/O Network Gear:NetGear_Async:Immensely Memory-Efficient Asyncio Video-Frames Network Messaging Framework.CamGearCamGear can grab ultra-fast frames from a diverse range of file-formats/devices/streams, which includes almost any IP-USB Cameras, multimedia video file-formats (upto 4k tested), various network stream protocols such ashttp(s), rtp, rtsp, rtmp, mms, etc., and GStreamer's pipelines, plus direct support for live video streaming sites like YouTube, Twitch, LiveStream, Dailymotion etc.CamGear provides a flexible, high-level, multi-threaded framework around OpenCV'sVideoCapture classwith access almost all of its available parameters. CamGear internally implementsyt_dlpbackend class for seamlessly pipelining live video-frames and metadata from various streaming services likeYouTube,Twitch, andmany more >>. Furthermore, its framework relies exclusively onThreaded Queue modefor ultra-fast, error-free, and synchronized video-frame handling.CamGear API Guide:>>> Usage GuideVideoGearVideoGear API provides a special internal wrapper around VidGear's exclusiveVideo Stabilizerclass.VideoGear also acts as a Common Video-Capture API that provides internal access for bothCamGearandPiGearAPIs and their parameters with an exclusiveenablePiCameraboolean flag.VideoGear is ideal when you need to switch to different video sources without changing your code much. Also, it enables easy stabilization for various video-streams(real-time or not)with minimum effort and writing way fewer lines of code.Below is a snapshot of a VideoGear Stabilizer in action (See its detailed usagehere):Original Video Courtesy@SIGGRAPH2013Code to generate above result:# import required librariesfromvidgear.gearsimportVideoGearimportnumpyasnpimportcv2# open any valid video stream with stabilization enabled(`stabilize = True`)stream_stab=VideoGear(source="test.mp4",stabilize=True).start()# open same stream without stabilization for comparisonstream_org=VideoGear(source="test.mp4").start()# loop overwhileTrue:# read stabilized framesframe_stab=stream_stab.read()# check for stabilized frame if Nonetypeifframe_stabisNone:break# read un-stabilized frameframe_org=stream_org.read()# concatenate both framesoutput_frame=np.concatenate((frame_org,frame_stab),axis=1)# put text over concatenated framecv2.putText(output_frame,"Before",(10,output_frame.shape[0]-10),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,255,0),2,)cv2.putText(output_frame,"After",(output_frame.shape[1]//2+10,output_frame.shape[0]-10),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,255,0),2,)# Show output windowcv2.imshow("Stabilized Frame",output_frame)# check for 'q' key if pressedkey=cv2.waitKey(1)&0xFFifkey==ord("q"):break# close output windowcv2.destroyAllWindows()# safely close both video streamsstream_org.stop()stream_stab.stop()VideoGear API Guide:>>> Usage GuidePiGearPiGear is similar to CamGear but made to support various Raspberry Pi Camera Modules(such asOmniVision OV5647 Camera ModuleandSony IMX219 Camera Module).PiGear provides a flexible multi-threaded framework around completepicamerapython library, and provide us the ability to exploit almost all of its parameters likebrightness, saturation, sensor_mode, iso, exposure, etc.effortlessly. Furthermore, PiGear also supports multiple camera modules, such as in the case of Raspberry-Pi Compute Module IO boards.Best of all, PiGear containsThreaded Internal Timer- that silently keeps active track of any frozen-threads/hardware-failures and exit safely, if any does occur. That means that if you're running PiGear API in your script and someone accidentally pulls the Camera-Module cable out, instead of going into possible kernel panic, API will exit safely to save resources.Code to open picamera stream with variable parameters in PiGear API:# import required librariesfromvidgear.gearsimportPiGearimportcv2# add various Picamera tweak parameters to dictionaryoptions={"hflip":True,"exposure_mode":"auto","iso":800,"exposure_compensation":15,"awb_mode":"horizon","sensor_mode":0,}# open pi video stream with defined parametersstream=PiGear(resolution=(640,480),framerate=60,logging=True,**options).start()# loop overwhileTrue:# read frames from streamframe=stream.read()# check for frame if NonetypeifframeisNone:break# {do something with the frame here}# Show output windowcv2.imshow("Output Frame",frame)# check for 'q' key if pressedkey=cv2.waitKey(1)&0xFFifkey==ord("q"):break# close output windowcv2.destroyAllWindows()# safely close video streamstream.stop()PiGear API Guide:>>> Usage GuideScreenGearScreenGear is designed exclusively for targeting rapid Screencasting Capabilities, which means it can grab frames from your monitor in real-time, either by defining an area on the computer screen or full-screen, at the expense of inconsiderable latency. ScreenGear also seamlessly support frame capturing from multiple monitors as well as supports multiple backends.ScreenGear implements a Lightning-Fast API wrapper arounddxcam,pyscreenshot&python-msspython libraries and also supports an easy and flexible direct internal parameters manipulation.Below is a snapshot of a ScreenGear API in action:Code to generate the above results:# import required librariesfromvidgear.gearsimportScreenGearimportcv2# open video stream with default parametersstream=ScreenGear().start()# loop overwhileTrue:# read frames from streamframe=stream.read()# check for frame if NonetypeifframeisNone:break# {do something with the frame here}# Show output windowcv2.imshow("Output Frame",frame)# check for 'q' key if pressedkey=cv2.waitKey(1)&0xFFifkey==ord("q"):break# close output windowcv2.destroyAllWindows()# safely close video streamstream.stop()ScreenGear API Guide:>>> Usage GuideWriteGearWriteGear handles various powerful Video-Writer Tools that provide us the freedom to do almost anything imaginable with multimedia data.WriteGear API provides a complete, flexible, and robust wrapper aroundFFmpeg, a leading multimedia framework. WriteGear can process real-time frames into a lossless compressed video-file with any suitable specifications(such asbitrate, codec, framerate, resolution, subtitles,  etc.).WriteGear also supports streaming with traditional protocols such asRTSP/RTP, RTMP. It is powerful enough to perform complex tasks such asLive-Streaming(such as for Twitch, YouTube etc.)andMultiplexing Video-Audiowith real-time frames in just few lines of code.Best of all, WriteGear grants users the complete freedom to play with any FFmpeg parameter with its exclusiveCustom Commands function(see thisdoc)without relying on any third-party API.In addition to this, WriteGear also provides flexible access toOpenCV's VideoWriter APItools for video-frames encoding without compression.WriteGear primarily operates in the following two modes:Compression Mode:In this mode, WriteGear utilizes powerfulFFmpeginbuilt encoders to encode lossless multimedia files. This mode provides us the ability to exploit almost any parameter available within FFmpeg, effortlessly and flexibly, and while doing that it robustly handles all errors/warnings quietly.You can find more about this modehere >>Non-Compression Mode:In this mode, WriteGear utilizes basicOpenCV's inbuilt VideoWriter APItools. This mode also supports all parameter transformations available within OpenCV's VideoWriter API, but it lacks the ability to manipulate encoding parameters and other important features like video compression, audio encoding, etc.You can learn about this modehere >>WriteGear API Guide:>>> Usage GuideStreamGearStreamGear automates transcoding workflow for generating Ultra-Low Latency, High-Quality, Dynamic & Adaptive Streaming Formats (such as MPEG-DASH and Apple HLS) in just few lines of python code.StreamGear provides a standalone, highly extensible, and flexible wrapper aroundFFmpegmultimedia framework for generating chunked-encoded media segments of the content.SteamGear is an out-of-the-box solution for transcoding source videos/audio files & real-time video frames and breaking them into a sequence of multiple smaller chunks/segments of suitable lengths. These segments make it possible to stream videos at different quality levels(different bitrates or spatial resolutions)and can be switched in the middle of a video from one quality level to another – if bandwidth permits – on a per-segment basis. A user can serve these segments on a web server that makes it easier to download them through HTTP standard-compliant GET requests.SteamGear currently supportsMPEG-DASH(Dynamic Adaptive Streaming over HTTP, ISO/IEC 23009-1)andApple HLS(HTTP Live Streaming). But, Multiple DRM support is yet to be implemented.SteamGear also creates a Manifest file(such as MPD in-case of DASH)or a Master Playlist(such as M3U8 in-case of Apple HLS)besides segments that describe these segment information(timing, URL, media characteristics like video resolution and bit rates)and is provided to the client before the streaming session.StreamGear primarily works in two Independent Modes for transcoding which serves different purposes:Single-Source Mode:In this mode, StreamGeartranscodes entire video file(as opposed to frame-by-frame)into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well when you're transcoding long-duration lossless videos(with audio) for streaming that required no interruptions. But on the downside, the provided source cannot be flexibly manipulated or transformed before sending onto FFmpeg Pipeline for processing.Learn more about this modehere >>Real-time Frames Mode:In this mode, StreamGear directlytranscodes frame-by-frame(as opposed to a entire video file), into a sequence of multiple smaller chunks/segments for streaming. This mode works exceptionally well when you desire to flexibility manipulate or transformnumpy.ndarrayframes in real-time before sending them onto FFmpeg Pipeline for processing. But on the downside, audio has to added manually(as separate source)for streams.Learn more about this modehere >>StreamGear API Guide:>>> Usage GuideNetGearNetGear is exclusively designed to transfer video-frames & data synchronously between interconnecting systems over the network in real-time.NetGear implements a high-level wrapper aroundPyZmQpython library that contains python bindings forZeroMQ- a high-performance asynchronous distributed messaging library.NetGear seamlessly supports additionalbidirectional data transmissionbetween receiver(client) and sender(server) while transferring video-frames all in real-time.NetGear can also robustly handleMultiple Server-SystemsandMultiple Client-Systemsand at once, thereby providing access to a seamless exchange of video-frames & data between multiple devices across the network at the same time.NetGear allows remote connection overSSH Tunnelthat allows us to connect NetGear client and server via secure SSH connection over the untrusted network and access its intranet services across firewalls.NetGear also enables real-timeJPEG Frame Compressioncapabilities for boosting performance significantly while sending video-frames over the network in real-time.For security, NetGear implements easy access to ZeroMQ's powerful, smart & secure Security Layers that enableStrong encryption on dataand unbreakable authentication between the Server and the Client with the help of custom certificates.NetGear as of now seamlessly supports three ZeroMQ messaging patterns:zmq.PAIR(ZMQ Pair Pattern)zmq.REQ/zmq.REP(ZMQ Request/Reply Pattern)zmq.PUB/zmq.SUB(ZMQ Publish/Subscribe Pattern)Whereas supported protocol are:tcpandipc.NetGear API Guide:>>> Usage GuideWebGearWebGear is a powerfulASGIVideo-Broadcaster API ideal for transmittingMotion-JPEG-frames from a single source to multiple recipients via the browser.WebGear API works onStarlette's ASGI application and provides a highly extensible and flexible async wrapper around its complete framework. WebGear can flexibly interact with Starlette's ecosystem of shared middleware, mountable applications,Response classes,Routing tables,Static Files,Templating engine(with Jinja2), etc.WebGear API uses an intraframe-only compression scheme under the hood where the sequence of video-frames are first encoded as JPEG-DIB (JPEG with Device-Independent Bit compression) and then streamed over HTTP using Starlette's MultipartStreaming Responseand aUvicornASGI Server. This method imposes lower processing and memory requirements, but the quality is not the best, since JPEG compression is not very efficient for motion video.In layman's terms, WebGear acts as a powerfulVideo Broadcasterthat transmits live video-frames to any web-browser in the network. Additionally, WebGear API also provides a special internal wrapper aroundVideoGear, which itself provides internal access to bothCamGearandPiGearAPIs, thereby granting it exclusive power of broadcasting frames from any incoming stream. It also allows us to define our custom Server as source to transform frames easily before sending them across the network(see thisdocexample).Below is a snapshot of a WebGear Video Server in action on Chrome browser:WebGear Video Server athttp://localhost:8000/address.Code to generate the above result:# import required librariesimportuvicornfromvidgear.gears.asyncioimportWebGear# various performance tweaksoptions={"frame_size_reduction":40,"frame_jpeg_quality":80,"frame_jpeg_optimize":True,"frame_jpeg_progressive":False,}# initialize WebGear appweb=WebGear(source="foo.mp4",logging=True,**options)# run this app on Uvicorn server at address http://localhost:8000/uvicorn.run(web(),host="localhost",port=8000)# close app safelyweb.shutdown()WebGear API Guide:>>> Usage GuideWebGear_RTCWebGear_RTC is similar toWeGear APIin many aspects but utilizesWebRTCtechnology under the hood instead of Motion JPEG, which makes it suitable for building powerful video-streaming solutions for all modern browsers as well as native clients available on all major platforms.WebGear_RTC is implemented with the help ofaiortclibrary which is built on top of asynchronous I/O framework for Web Real-Time Communication (WebRTC) and Object Real-Time Communication (ORTC) and supports many features like SDP generation/parsing, Interactive Connectivity Establishment with half-trickle and mDNS support, DTLS key and certificate generation, DTLS handshake, etc.WebGear_RTC can handlemultiple consumersseamlessly and provides native support for ICE(Interactive Connectivity Establishment)protocol, STUN(Session Traversal Utilities for NAT), and TURN(Traversal Using Relays around NAT)servers that help us to seamlessly establish direct media connection with the remote peers for uninterrupted data flow. It also allows us to define our custom streaming class with suitable source to transform frames easily before sending them across the network(see thisdocexample).WebGear_RTC API works in conjunction withStarlette's ASGI application and provides easy access to its complete framework. WebGear_RTC can also flexibly interact with Starlette's ecosystem of shared middleware, mountable applications,Response classes,Routing tables,Static Files,Templating engine(with Jinja2), etc.Additionally, WebGear_RTC API also provides a special internal wrapper aroundVideoGear, which itself provides internal access to bothCamGearandPiGearAPIs.Below is a snapshot of a WebGear_RTC Media Server in action on Chrome browser:WebGear_RTC Video Server athttp://localhost:8000/address.Code to generate the above result:# import required librariesimportuvicornfromvidgear.gears.asyncioimportWebGear_RTC# various performance tweaksoptions={"frame_size_reduction":30,}# initialize WebGear_RTC appweb=WebGear_RTC(source="foo.mp4",logging=True,**options)# run this app on Uvicorn server at address http://localhost:8000/uvicorn.run(web(),host="localhost",port=8000)# close app safelyweb.shutdown()WebGear_RTC API Guide:>>> Usage GuideNetGear_Async.NetGear_Async can generate the same performance asNetGear APIat about one-third the memory consumption, and also provide complete server-client handling with various options to use variable protocols/patterns similar to NetGear, but lacks in term of flexibility as it supports only a fewNetGear's Exclusive Modes.NetGear_Async is built onzmq.asyncio, and powered by a high-performance asyncio event loop calleduvloopto achieve unmatchable high-speed and lag-free video streaming over the network with minimal resource constraints. NetGear_Async can transfer thousands of frames in just a few seconds without causing any significant load on your system.NetGear_Async provides complete server-client handling and options to use variable protocols/patterns similar toNetGear API. Furthermore, NetGear_Async allows us to define our custom Server as source to transform frames easily before sending them across the network(see thisdocexample).NetGear_Async now supports additionalbidirectional data transmissionbetween receiver(client) and sender(server) while transferring video-frames. Users can easily build complex applications such as likeReal-Time Video Chatin just few lines of code.NetGear_Async as of now supports all four ZeroMQ messaging patterns:zmq.PAIR(ZMQ Pair Pattern)zmq.REQ/zmq.REP(ZMQ Request/Reply Pattern)zmq.PUB/zmq.SUB(ZMQ Publish/Subscribe Pattern)zmq.PUSH/zmq.PULL(ZMQ Push/Pull Pattern)Whereas supported protocol are:tcpandipc.NetGear_Async API Guide:>>> Usage GuideContributions👑 Contributor Hall of Fame 👑We're happy to meet new contributors💗We welcome your contributions to help us improve and extend this project. If you want to get involved with VidGear development, checkout theContribution Guidelines ▶️We're offering support for VidGear onGitter Community Channel. Come and join the conversation over there!DonationsVidGear is free and open source and will always remain so. ❤️It is something I am doing with my own free time. But so much more needs to be done, and I need your help to do this. For just the price of a cup of coffee, you can make a difference 🙂CitationHere is a Bibtex entry you can use to cite this project in a publication:@software{vidgear,author={Abhishek Thakur andZoe Papakipos andChristian Clauss andChristian Hollinger andIan Max Andolina andVincent Boivin andenarche-ahn andfreol35241 andBenjamin Lowe andMickaël Schoentgen andRenaud Bouckenooghe},title={abhiTronix/vidgear: VidGear v0.3.1},month=jul,year=2023,publisher={Zenodo},version={vidgear-0.3.1},doi={10.5281/zenodo.8174694},url={https://doi.org/10.5281/zenodo.8174694}}CopyrightCopyright (c) abhiTronix 2019This library is released under theApache 2.0 License.