Package: noddingpigeon
IntroductionTheNodding Pigeonlibrary provides a pre-trained model and
a simple inference API for detectinghead gesturesin short videos.
Under the hood, it uses GoogleMediaPipefor collecting the landmark features.InstallationTested for Python 3.8, 3.9, and 3.10.The best way to install this library with its dependencies is from PyPI:python3-mpipinstall--upgradenoddingpigeonAlternatively, to obtain the latest version from this repository:gitclonegit@github.com:bhky/nodding-pigeon.gitcdnodding-pigeon
python3-mpipinstall.UsageAn easy way to try the API and the pre-trained model is to
make a short video with your head gesture.WebcamThe code snippet below will perform the following:Search for the pre-trained weights file from$HOME/.noddingpigeon/weights/,
if not exists, the file will be downloaded from this repository.Start webcam.Collect the needed number of frames (default60) for the model.End webcam automatically (or you can pressqto end earlier).Make prediction of your head gesture and print the result to STDOUT.fromnoddingpigeon.inferenceimportpredict_videoresult=predict_video()print(result)# Example result:# {'gesture': 'nodding',#  'probabilities': {'has_motion': 1.0,#   'gestures': {'nodding': 0.9576354622840881,#    'turning': 0.042364541441202164}}}Video fileAlternatively, you could provide a pre-recorded video file:fromnoddingpigeon.inferenceimportpredict_videofromnoddingpigeon.videoimportVideoSegmentresult=predict_video("your_head_gesture_video.mp4",video_segment=VideoSegment.LAST,# Optionally change these parameters.motion_threshold=0.5,gesture_threshold=0.9)Note that no matter how long your video is, only the
pre-defined number of frames (60for the current model) are used for
prediction. Thevideo_segmentenum option controls how the frames
are obtained from the video,
e.g.,VideoSegment.LASTmeans the last (60) frames will be used.Thresholds can be adjusted as needed, see explanation in a later section.Result formatThe result is returned as a Python dictionary.{
  'gesture': 'turning',
  'probabilities': {
    'has_motion': 1.0,
    'gestures': {
      'nodding': 0.009188028052449226,
      'turning': 0.9908120036125183
    }
  }
}Head gesturesThe followinggesturetypes are available:nodding- Repeatedly tilt your head upward and downward.turning- Repeatedly turn your head leftward and rightward.stationary- Not tilting or turning your head; translation motion is still treated as stationary.undefined- Unrecognised gesture or no landmarks detected (usually means no face is shown).To determine the finalgesture:Ifhas_motionprobability is smaller thanmotion_threshold(default0.5),gestureisstationary. Other probabilities are irrelevant.Otherwise, we will look for the largest probability fromgestures:If it is smaller thangesture_threshold(default0.9),gestureisundefined,else, the corresponding gesture label is selected (e.g.,nodding).If no landmarks are detected in the video,gestureisundefined.
Theprobabilitiesdictionary is empty.APInoddingpigeon.inferencepredict_videoDetect head gesture shown in the input video either from webcam or file.Parameters:video_path(Optional[str], defaultNone):
File path to the video file, orNonefor starting a webcam.model(Optional[tf.keras.Model], defaultNone):
A TensorFlow-Keras model instance, orNonefor using the default model.max_num_frames(int, default60):
Maximum number of frames to be processed by the model.
Do not change when using the default model.video_segment(VideoSegmentenum, defaultVideoSegment.BEGINNING):
See explanation ofVideoSegment.end_padding(bool, defaultTrue):
IfTrueandmax_num_framesis set, when the input video has not enough
frames to form the feature tensor for the model, padding at the end will be
done using the features detected on the last frame.drop_consecutive_duplicates(bool, defaultTrue):
IfTrue, features from a certain frame will not be used to form the
feature tensor if they are considered to be the same as the previous frame.
This is a mechanism to prevent "fake" video created with static images.postprocessing(bool, defaultTrue):
IfTrue, the final result will be presented as the Python dictionary
described in theusagesection, otherwise the raw model output
is returned.motion_threshold(float, default0.5):
See thehead gesturessection.gesture_threshold(float, default0.9):
See thehead gesturessection.Return:A Python dictionary ifpostprocessingisTrue, otherwiseList[float]from the model output.noddingpigeon.videoVideoSegmentEnum class for video segment options.VideoSegment.BEGINNING: Collect the required frames for the model from the beginning of the video.VideoSegment.LAST: Collect the required frames for the model toward the end of the video.noddingpigeon.modelmake_modelCreate an instance of the model used in this library,
optionally with pre-trained weights loaded.Parameters:weights_path(Optional[str], default$HOME/.noddingpigeon/weights/*.h5):
Path to the weights in HDF5 format to be loaded by the model.
The weights file will be downloaded if not exists.
IfNone, no weights will be downloaded nor loaded to the model.
Users can provide path if the default is not preferred.
The environment variableNODDING_PIGEON_HOMEcan also be used to indicate
where the.noddingpigeon/directory should be located.Return:tf.keras.Modelobject.