Package: pythonic-cv
Version: 1.2.2Author: ES AlexanderRelease Date: 26/May/2021AboutOpenCV is a fantastic tool for computer vision, with significant Python support
through automatically generated bindings. Unfortunately some basic functionality
is frustrating to use, and documentation is sparse and fragmented as to how best to
approach even simple tasks such as efficiently processing a webcam feed.This library aims to address frustrations in the OpenCV Python api that can be
fixed using pythonic constructs and methodologies. Solutions are not guaranteed to
be optimal, but every effort has been made to make the code as performant as
possible while ensuring ease of use and helpful errors/documentation.RequirementsThis library requires an existing version ofOpenCVwith Python bindings to be
installed (e.g.python3 -m pip install opencv-python). Some features (mainly
property access helpers) may not work for versions of OpenCV earlier than 4.2.0.
The library was tested using Python 3.8.5, and is expected to work down to at least
Python 3.4 (although the integrated advanced features example uses matmul (@) for
some processing, which was introduced in Python 3.5).Numpyis also used throughout, so a recent version is suggested (tested with 1.19.0).mssis used for cross-platform screen-capturing functionality (requires >= 3.0.0)InstallationThe library can be installed from pip, withpython3 -m pip install pythonic-cv.UsageNew functionality is provided in thepcvmodule, as described below. All other
opencv functionality should be accessed through the standardcv2import.Main FunctionalityThe main implemented functionality is handling video through a context manager,
while also enabling iteration over the input stream. While iterating, key-bindings
have been set up for play/pause (SPACE) and stopping playback (q). A dictionary
of pause_effects can be passed in to add additional key-bindings while paused without
needing to create a subclass. In addition, video playback can be sped up withw,
slowed down withs, and if enabled allows rewinding withaand returning to
forwards playback withd. Forwards playback at 1x speed can be restored withr.
While paused, video can be stepped backwards and forwards usingaandd. All
default key-bindings can be overwritten using the play_commands and pause_effects
dictionaries and the quit and play_pause variables on initialisation.Video I/O DetailsFor reading and writing video files, theVideoReaderandVideoWriterclasses should
be used. For streaming, the classesCamera,SlowCamera, andLockedCameraare
provided. The simplest of these isSlowCamera, which has slow iteration because image
grabbing is performed synchronously, with a blocking call while reading each frame.CameraextendsSlowCamerawith additional logic to perform repeated grabbing in a
separate thread, so processing and image grabbing can occur concurrently.LockedCamerasits between the two, providing thread based I/O but with more control over when each
image is taken. Analogues for all three camera classes are provided in thepcv.screenmodule -SlowScreen,Screen, andLockedScreen. The interface is the same as that
for the corresponding camera class, except that a monitor index (-1 for all) or screen
region is passed in instead of a camera id.Camerais most useful for applications with processing speeds that require the most
up to date information possible and don't want to waste time decoding frames that are
grabbed too early to be processed (frame grabbing occurs in a separate thread, and only
the latest frame is retrieved (decoded) on read).SlowCamerashould only be used where
power consumption or overall CPU usage are more important than fast processing, or in
hardware that is only capable of single-thread execution, in which case the
separate image-grabbing thread will only serve to slow things down.LockedCamerais intended to work asynchronously likeCamera, but with more control.
It allows the user to specify when the next image should be taken, which leads to less
wasted CPU and power usage on grabbing frames that aren't used, but with time for the
image to be grabbed and decoded before the next iteration needs to start. The locking
protocol adds a small amount of additional syntax, and starting the image
grabbing process too late in an iteration can result in waits similar to those inSlowCamera, while starting the process too early can result in images being somewhat
out of date. Tuning can be done using the 'preprocess' and 'process' keyword arguments,
with an in-depth usage example provided insomething_fishy.py. When used correctlyLockedCamerahas the fastest iteration times, or if delays are used to slow down the
process it can have CPU and power usage similar to that ofSlowCamera.If using a video file to simulate a live camera stream, useSlowCameraorLockedCamera-Camerawill skip frames.There is also aGuaranteedVideoWriterclass which guarantees the output framerate by
repeating frames when given input too slowly, and skipping frames when input is too
fast.OverviewExamplesBasic Camera Streamfrompcv.vidIOimportCamerafrompcv.processimportchannel_options,downsize# start streaming camera 0 (generally laptop webcam/primary camera), and destroy 'frame'#   window (default streaming window) when finished.# Auto-initialised to have 1ms waitKey between iterations, breaking on 'q' key-press,#   and play/pause using the spacebar.withCamera(0)ascam:cam.stream()# stream camera 0 on window 'channels', downsized and showing all available channels.withLockedCamera(0,display='channels',process=lambdaimg:channel_options(downsize(img,4)))ascam:cam.stream()Stream and Recordfrompcv.vidIOimportCamerawithCamera(0)ascam:print("press 'q' to quit and stop recording.")cam.record_stream('me.mp4')Screen FunctionalityMain Monitorimportcv2frompcv.screenimportLockedScreen# stream monitor 0, and record to 'screen-record.mp4' file.withLockedScreen(0,process=lambdaimg:\cv2.cvtColor(img,cv2.COLOR_BGRA2BGR)asscreen:# video-recording requires 3-channel (BGR) or single-channel#  (greyscale, isColor=False) to workscreen.record_stream('screen-record.mp4')Screen Regionfrompcv.ScreenimportScreenwithScreen({'left':-10,'top':50,'width':100,'height':200})asscreen:screen.stream()VideoReaderfrompcv.vidIOimportVideoReaderfrompcv.processimportdownsize# just play (simple)# Press 'b' to jump playback back to the beginning (only works if pressed#   before playback is finished)withVideoReader('my_vid.mp4')asvid:vid.stream()# start 15 seconds in, end at 1:32, downsize the video by a factor of 4withVideoReader('my_vid.mp4',start='15',end='1:32',preprocess=lambdaimg:downsize(img,4))asvid:vid.stream()# enable rewinding and super fast playback# Press 'a' to rewind, 'd' to go forwards, 'w' to speed up, 's' to slow down#    and 'r' to reset to forwards at 1x speed.withVideoReader('my_vid.mp4',skip_frames=0)asvid:vid.stream()# headless mode (no display), operating on every 10th framewithVideoReader('my_vid.mp4',auto_delay=False,skip_frames=10,process=my_processing_func)asvid:vid.headless_stream()Advanced ExamplesCheck thepcv/examplesfolder for some examples of full programs using this library.Something FishyThis example is a relatively basic augmented reality example, which creates a tank of
fish that swim around on top of a video/webbcam feed. You can catch the fish (click and
drag a 'net' over them with your mouse), or tickle them (move around in your webcam feed).
There are several generally useful processing techniques included, so take a look
through the code and find the functionality that's most interesting to you to explore
and modify.To run the example usepython3 -m pcv.examples.something_fishy, or optionally specify
the path to a newline separated text file of names (of your friends and family for
example), and run withpython3 -m pcv.examples.something_fishy path/to/names.txt.Video SwitcherThis was made in response to a question fromu/guillerubioon reddit, to show how to
efficiently read multiple videos simultaneously while only displaying one, and switching
between videos based on what's happening in a webcam feed.The current video is switched out when the camera is covered/uncovered. Switching is
performed intelligently by only actually reading frames from a single 'active' video
at a time. TheVideoSwitcherclass allows tracking along all the videos simultaneously
based on how many frames have occurred since they were last active, as well as just
resuming from where each one left off when it was last active. When a video ends it gets
started again (in an infinite loop).To run the example usepython3 -m pcv.examples.cam_video_switchwhile in a directory
with the.mp4files you want to switch between.Hand WriterThis uses Google'smediapipelibrary for hand detection, to show a relatively simple
AR workflow, where a pointed right hand index finger can be used to write/draw on a
camera feed. Once you've installed the library (python3 -m pip install mediapipe),
run withpython3 -m pcv.examples.hand_writeand start drawing.The pointing detection algorithm is quite rudimentary, so may require some angling of
your hand before it detects that the pointer finger is straight and the other fingers
are closed. It currently intentionally only detects right hands, so your left hand can
also be in the frame without causing issues. The side detection assumes a front-facing
(selfie) camera, as is commonly the case with webcams and similar.