Package: contentai-activity-classifier
activity-classifier-extractorGenerates activity classifications from low-level feature inputs in support
of analytic workflows within theContentAI Platform,
published as the extractordsai_activity_classifier.Getting StartedExecutionCreating ModelsTestingFuture DevelopmentChangesGetting StartedThis library is used as asingle-run executable.Runtime parameters can be passed for processing that configure the
returned results and can be examined in more detail in themainscript.verbose-(bool)- verbose input/output configuration printing (default=false)path_content-(str)- input video path for files to label (default=video.mp4)path_result-(str)- output path for samples (default=.)path_models-(str)- manifest path for model information (default=data/models/manifest.json)time_interval-(float)- time interval for predictions from models (default=3.0)average_predictions-(bool)- flatten predictions across time and class (default=false)round_decimals-(int)- rounding decimals for predictions (default=5)score_min-(float)- apply a minimum score threshold for classes (default=0.1)dependenciesTo install package dependencies in a fresh system, the recommended
technique is a set ofvanilla pip packages. The latest requirements should be validated from
therequirements.txtfile but at time of writing, they were the
following.pipinstall--no-cache-dir-rrequirements.txtExecution and DeploymentThis package is meant to be run as a one-off processing tool that
aggregates the insights of other extractors.command-line standaloneRun the code as if it is an extractor. In this mode, configure a few
environment variables to let the code know where to look for content.One can also run the command-line with a single argument as input and
optionally ad runtime configuration (seeruntime
variables) as part of theEXTRACTOR_METADATAvariable as JSON.EXTRACTOR_METADATA='{"compressed":True}'Locally Run Classifier on ResultsFor utility, the above line has been wrapped in the bash scriptrun_local.sh../run_local.sh<docker_image>[<source_directory><output_data_dir>[<json_args>]][<all_args>]-runclipextractiononsourcewithpriorprocessing<docker_image>=0IFlocalcommand-linebased(argsusingargparse)=1IFlocaldockeremulation=IMAGE_NAMEIFdockerimagenametorun./run_local.sh0--path_contentfeatures/--path_resultresults/--verbose./run_local.sh1features/results/0'{\"verbose\"true}'Through all of the above examples, the underlying command-line execution is
similar to this excution run on the testing data.python-uactivity_classifier/main.py--path_contenttesting/data/launch/video.mp4--path_resulttesting/class--path_modelsactivity_classifier/data/models/manifest.json--verboseFeature-Based SimilarityA helper script is also avaialble to compute the similarity of clips in
one or more feature files.(v1.1.0)python-uactivity_classifier/features.py--path_contenttesting/data/dummy.txt\\--feature_typedsai_videocnndsai_vggish--path_resulttesting/distContentAIDeploymentDeployment is easy and follows standard ContentAI steps.contentaideploydsai_activity_classifierDeploying...writingworkflow.dotdoneAlternatively, you can pass an image name to reduce rebuilding a docker
instance.dockerbuild-tdsai_activity_classifiercontentaideploymetadata-flattendsai_activity_classifierLocally Downloading ResultsYou can locally download data from a specific job for this extractor to
directly analyze.contentaidatawHaT3ver1t1s--dirdataRun as an Extractorcontentairunhttps://bucket/video.mp4-w'digraph { dsai_videocnn -> dsai_activity_classifier; dsai_vggish -> dsai_activity_classifier }'JOBID:1Tfb1vPPqTQ0lVD1JDPUilB8QNrCONTENT:s3://bucket/video.mp4STATE:completeSTART:FriFeb1504:38:05PM(6minutesago)UPDATED:1minuteagoEND:FriFeb1504:43:04PM(1minuteago)DURATION:4minutesEXTRACTORSmy_extractorTASKSTATESTARTDURATION724a493complete5minutesago1minuteOr run it via the docker image.  Please review therun_local.shfile for more information.View Extractor Logs (stdout)contentailogs-f<my_extractor>my_extractorFriNov1504:39:22PMwritingsomedataJobcompletein4m58.265737799sAdding New ModelsThere are two steps to adding new models.First, train the models and formulate
a well-known structure (this can be done exhaustively across a number of model types).
SeeMODELS.rstfor more details.Update the manifest according to the instructions below to indicate how the activity
classifier should load the model (e.g. theframework), the required features, and
a few fields for understanding other descriptions (e.g. thenameand theid).Updating The ManifestAdding models to the pre-determined set of models is as easy as editing a manifest file and
adding a model into git LFS.Archive the new model into a serialized fileset.  At time of writing, this was serializing
models fromsklearnwith simplepickle load/save serialization.Gather all of the relevant output files and compress them if you can.  Currently, the library
understands gzip compression extensions (e.g. “.gz”).Choose the appropriate sub-directory that corresponds to the upstream feature extractor.  For
example, models built on3dcnnfeatures may process new videos (viaextractor chaining)
to the extractordsai_3dcnn.  If one doesn’t exist yet, please create a new directory, but
remember what combination of audio and video features is required.Modify the manifest file inactivity_classifier/data/models/manifest.jsonfor your new entry.
Specifically, the input video and audio features must be defined as well as the serialization
library.  Below is an example block that indicates3dcnn` video and ``vggishaudio features for
a model crated withsklearnwhere prediction results will be nested with the nameRunning.[...{"path":"3dcnn-vggish/lr-Running.pkl.gz","name":"Running","id":"ugc","framework":"sklearn","video":"dsai_videocnn","audio":"dsai_vggish"},...]Prepare to add your model files to the repo.NOTE This repo uses `git-lfs <https://git-lfs.github.com/>`__
to store all binary files like models.  If your model is added with regular git tools alone, you will
get a sternly worded email (and friendly advice on how to re-add correctly).(fromthebasedirectoryonly)gitlfstrackactivity_classifier/data/models/3dcnn/moonwalk_model.pkl.gzgitaddactivity_classifier/data/models/3dcnn/moonwalk_model.pkl.gzgitaddactivity_classifier/data/models/manifest.jsonTest your model with the data in thetestingdirectory.  The CI/CD process should do this too
but it’s always easier to find and fix problems here than with a vague email.  The features in this
directory came from processing of theHBO Max Launch Video,
which is publicly available as a reference.(fromthebasedirectory)./run_local.sh0--path_contenttesting/data/test.mp4--time_interval1.5(checkforpredictionsfromyournewmodelindata.json)TestingTesting is included via tox.  To launch testing for the entire package, just runtoxat the command line.
Testing can also be run for a specific file within the package by setting the evironment variableTOX_ARGS.TOX_ARG=test_basic.pytoxFuture Developmentadditional training hooks?ChangesGenerates activity classifications from low-level feature inputs in support
of analytic workflows within theContentAI Platform.1.31.3.7fix run_local typosmore verbosity checks1.3.6modeling.py separatorsdocs reorg1.3.5contentai key request fix1.3.3docs updatemulticlass write1.3.2docker build update, run example update1.3.1docs fix for example of using packagebug fix for default location, change inputs to classify function1.3.0move models out of the primary packagebreaking change, rename input parampath_modelstopath_manifest1.21.2.2bump version for model migration to LFS1.2.1fix docker/deployed image run command1.2.0switch to package representation, push to pypiseveral updates for MANIFEST definition (id)inclusion of multi-parameter training and testing frameworksafety for model loading, catch exceptions, return gracefullyupdate documents to split for binary models1.11.1.1cosmetic change for reuse in other libraries1.1.0refactor feature code, add utility for difference computation among segmentsmin value thresholding to avoid low scoring results in output (default=0.1)refactor caching information for feature load (allow flatten, remove cache, allow multi-asset)allow recursive feature load for distance compute1.01.0.2fixes for output, modify to require other extractors as dependenciesfix order of paramters for local runs1.0.1updates for integration of other models, fixes for prediction outputadd l2norm after average/merge in time of source features1.0.0initial project merge from other sourcesgenerates json prediction dictcallable as packageincludes some testing routines with windowing comparison