Package: plastic-origins
Automated object counting on riverbanksProject Branches:release:This is the main/production branch (DO NOT PUSH DIRECTLY TO RELEASE)dev:This is the developpement branch (DO NOT PUSH DIRECTLY TO DEV)Feature BranchesThese are the branches where you can develop new features for the project. In order to create a feature branch:Make sure that yourdevbranch is up to dategitcheckoutdev
gitpulldevCreate a new branch fromdevwith the namefeature/name_of_your_featuregitcheckout-bfeature/name_of_your_featureOnce your feature developpement is complete, make a Pull Request of your feature branch todevResearch Branches:These are branches made for research purposes and they are namedresearch/name_of_your_subjectRelease Branch - InstallationFollow these steps in that order exactly:Clone the projectgitclonehttps://github.com/surfriderfoundationeurope/surfnet.git<folder-for-surfnet>-breleasecd<folder-for-surfnet>Install PoetrypipinstallpoetryCreate your virtual environmentHere we use python version 3.9poetryenvuse3.9Install dependenciespoetryinstallCode Linting and Formatting:pre-commits have been added to format and check the linting of the code before any commit. This process will run:PyUpgrade: to make sure that the code syntax is up to date with the latest python versionsBlack: which is a code formatterFlake8: to check that the code is properly formatted.All this process is automatic to ensure the commited code quality. So as a good measure, prior to committing any code it is highly recommended to run:poetryrunblackpath/to/the/changed/code/directory(ies)This will format the code that has been written and:poetryrunflake8path/to/the/changed/code/directory(ies)to check if there is any other issues to fix.Downloading pretrained modelsYou can download MobileNetV3 model with the following script:cdmodels
shdownload_pretrained_base.shThe file will be downloaded intomodels.Validation videosIf you want to download the 3 test videos on the 3 portions of the Auterrive riverbank, run:cd data
sh download_validation_videos.shThis will download the 3 videos in distinct folders ofdata/validation_videos.ServingDevelopmentSetting up the server and testing: from surfnet/ directory, you may run a local flask developement server with the following command:exportFLASK_APP=src/plasticorigins/serving/app.py
poetryrunflaskrunProductionSetting up the server and testing: from surfnet/ directory, you may run a local wsgi gunicorn production server with the following command:PYTHONPATH=./srcgunicorn-w5--threads2--bind0.0.0.0:8001--chdir./src/serving/wsgi:appTest surfnet APIThen, in order to test your local dev server, you may run:curl-XPOSThttp://127.0.0.1:5000/-F'file=@/path/to/video.mp4'# flaskChange port 5000 to 8001 to test on gunicorn or 8000 to test with Docker and gunicorn.DockerYou can build and run the surfnet AI API within a Docker container.Docker Build:dockerbuild-tsurfnet/surfnet:latest.Docker Run:dockerrun--envPYTHONPATH=/src-p8000:8000--namesurfnetapisurfnet/surfnet:latestMakefileYou can use the makefile for convenience purpose to launch the surfnet API:makesurfnet-dev-local# with flaskmakesurfnet-prod-local# with gunicornmakesurfnet-prod-build-docker# docker buildmakesurfnet-prod-run-docker# docker runKubernetesTo ease production operation, the surfnet API can be deployed on top of kubernetes (k8s) cluster. A pre-built Docker image is available on ghcr.io to be deployed using the surfnet.yaml k8s deployment file. To do so, change directory to k8s/, then once you are connected to your k8s cluster simply enter:kubectlapply-ysurfnet.yamlRemark: we use a specific surfnet k8s node pool label for our Azure production environment on aks. If you want to test deployment on a default k8s cluster using system nodes, you have either to use default surfnet.yaml file or remove the nodeSelector section from others deployment files (aks, gke).After the deployment is done, create a service to expose the surfnet API to be publicly accessible over the Internet.kubectlexposedeploymentsurfnet--type=LoadBalancer--name=surfnet-api
kubectlgetservicesurfnet-apiRelease plasticorigins to pypi:Prerelease: (Pull Request to Dev branch)Check or Bump version:Check the current version of the product:Docker Build:poetryversionBump the version to the product:poetryversion<bump-rule>bump rules can be found in :https://python-poetry.org/docs/cli/#:~:text=with%20concrete%20examples.-,RULE,-BEFOREchoose carefully the one that corresponds to your bump: for the prerelease we will use:prepatchpreminorpremajormake sure that in you pyproject.toml your version ends with-alpha.0Publish the prerelease to pypi:In order to publish your prerelease to PyPi, all you need to do is open a Pull Request of your current branch toDevbranch. Once the PR is approved and merged, the Prerelease will be done automatically with a github workflow.Release: (Pull Request to Release branch):In order to publish a release version to PyPi, all you have to do is open a Pull Request of theDevbranch into theReleasebranch. Once the PR is approved and merged, the Release will be done automatically with a github workflow.Testing:To launch the tests you can run this commandpoetryruncoveragerun-mpytest-s&&poetryruncoveragereport-mMkdocs Documentation:You need to install the following packages:pipinstallmkdocs
pipinstallmkdocstringsTo run the mkdocs documentation, you can run the following lines below:cdsrc
mkdocsserveThe documentation will be serving onhttp://127.0.0.1:8000/.Configurationsrc/serving/inference.pycontains a Configuration dictionary that you may change:skip_frames:3number of frames to skip. Increase to make the process faster and less accurate.kappa:7the moving average window.1prevents the average, avoid2which is ill-defined.tau:4the number of consecutive observations necessary to keep a track. If you increaseskip_frames, you should lowertau.Datasets and TrainingConsider other branches for that!