Package: sandpyper
Sandpyper- Sandy beaches SfM-UAV analysis tools -Sandpyperperforms an organised and automated extraction of color and elevation profiles from as many DSM and orthophotos as you like. It is thought to be used when a considerable number of DSMs and orthophotos from many different locations and coordinate reference systems need to be processed.
    Then, it computes volumetric and behavioural analysis of sandy beachfaces, speeding up an otherwise long and difficult to handle job.
    It has some specialised functions to help dealing with the common limitations found in working with Unoccupied Aerial Vehicles (UAVs) and Structure from Motion (SfM) in beach environments, which are:Swash zone: the water motion of waves washing in and out of the swash zone prevents SfM algorithm to reliably model elevation. It is commonly discarded.Vegetation: both dune vegetation and beach wracks (macroalgae, woody debris) should be removed or filtered as anything that is not sand would compromise sediment volumetric computation and behavioural analysis.File size: a few km long beach surveyed with a DJI Phantom 4-Advanced at 100 meters altitude creates roughly 10 Gb (uncompressed) of data, which can be cumbersome for some GIS to handle.From user-defined cross-shore transects,Sandpyperhelps with:cleaning profiles from unwanted non-sand pointscomputing period-specific limits of detection to obtain reliable estimates of changesdetecting statistically significant clusters of change (also referred to hotspots/coldspots) of beach changecomputing multiscale volumetric analysismodeling multiscale Beachface Cluster Dynamics indicesvisualising beach changes, limits of detection, transects and BCDs with a variety of in-built plotting methodsAdditionally,Sandpyperhas some useful functions that can come at hand, such as:
automatic transect creation from a vector line, grid creation along a line and subsequent tiles extraction and others.automatic transects creation from a vector linespatial grid creation of specified tile size along a linetiles extraction from spatial gridSandpyperis very easy to use.As the above image shows,Sandpyperprocessing pipeline is mainly composed of3 main components:Raw data extractionData correctionSediment dynamics analysisTo achieve this in an organised and coherent way,Sandpyperprovides two core objects, theProfileSetand theProfileDynamicsclasses.TheProfileSetclass sets up the monitoring global parameters, extracts the change data and Limit of Detections, implements iterative silhouette analysis with inflexion point search and facilitates point cleaning by using user-provided watermasks, shoremasks and class dictionaries.TheProfileDynamicsclass is the core processor for beach dynamics. It computes multitemporal and multiscale elevation change, performs Hotspot/Coldspot analysis at the location level, discretises the data into classes of magnitude of change, models multiscale behavioural dynamics and provides many different plotting options.After instantiation, both classes will gradually store more and more beach change information in their attributes.Follow theJupyter Notebook tutorialsto understand how it works!This code has supported the analysis and publication of the article"Citizen science for monitoring seasonal-scale beach erosion and behaviour with aerial drones", in the open access Nature Scientific Report journal.Explore the docs»Report Bug»Request FeatureTable of ContentsAbout The ProjectBackgroundModulesGetting StartedPrerequisitesInstallationUsageRoadmapContributingLicenseContactPublicationsAcknowledgementsAbout The ProjectBackgroundSandpyperhas been originally developed to facilitate the analysis of a large dataset coming from more than 300 Unoccupied Aerial Vehicles (UAV) surveys, performed by Citizen Scientist in Victoria, Australia. This is theEureka Award WinningWorld-firstbeach monitoring program powered by volunteers, who autonomously fly UAVs on 15 sensitive sites (erosional hotspots) across the Victorian coast, every 6 weeks for 3 years, since 2018. This project is part of a broader marine mapping program called TheVictorian Coastal Monitoring Program (VCMP), funded by theVictorian Department of Environment, Land, Water and Planning, co-funded byDeakin UniversityandThe University of Melbourne.Each survey creates Digital Surface Models (DSMs) and orthophotos of considerable size (5-10 Gb uncompressed), which can be troublesome for some GIS to render, let alone perform raster-based computations.Modulessandpyper: main module where theProfileSetandProfileDynamicsclasses and methods are defined.common: where all the functions are stored.Getting StartedCurrently Sandpyper is tested on Windows, MacOS and Ubuntu with Python 3.8 and 3.9. To get a local copy up and running follow these simple steps.PrerequisitesInstall Condain your local machine. We need it to create thesandpyper_envvirtual environment.then, if you do not have it already, installVisual Studio C++ build tools. You can download ithere.If you don't have it already, addconda-forgechannel to your anaconda config file, by typing this in your Anaconda Prompt terminal (base environment):condaconfig--addchannelsconda-forgeNow, always in the (base) environment, create a new environment calledsandpyper_envusingpython=3.9and install the required packages by typing:condacreate--namesandpyper_envpython=3.9geopandas=0.8.2matplotlib=3.3.4numpy=1.20.1pandas=1.2.2tqdm=4.56.2pysal=2.1rasterio=1.2.0richdem=0.3.4scikit-image=0.18.1scikit-learn=0.24.1scipy=1.6.0seaborn=0.11.1tqdm=4.56.2pooch=1.4.0fuzzywuzzyIfrasteriopackage cannot be installed due toGDAL binding issues, follow the instructions inrasterio installation webpage.If you want to test the package using the provided notebooks,download the test data (test_data.rar)HEREInstallationcondaactivatesandpyper_envpipinstallsandpyperInstall Jupyter Notebooks:condainstalljupyterOnce you open a Jupyter Notebook with the sandpyper_env, import it to test it works.importsandpyperUsageTo seeSandpyperin action, follow the Jupyter Notebooks providedhere.Download the test data file (test_data.rar)HERE.For the API reference, see theDocumentation.RoadmapUpdate CRS definition to new CRS object standard in order to upgrade Geopandas version.Relax all requirements to make it future-proof and available in Anaconda.org.Add raster support for Dems of Differences (DoDs) and LoDs.Add shoreline analysis from space.Add [leafmap](https://github.com/giswqs/leafmap) to better (interactive) plotting.Add automatic check for overlapping label correction polygons.See theopen issuesfor a list of proposed features (and known issues).ContributingContributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make aregreatly appreciated.Fork the ProjectCreate your Feature Branch (git checkout -b feature/AmazingFeature)Commit your Changes (git commit -m 'Add some AmazingFeature')Push to the Branch (git push origin feature/AmazingFeature)Open a Pull RequestLicenseDistributed under the MIT License. SeeLICENSEfor more information.ContactNicolas Pucino:@NicolasPucino-npucino@deakin.edu.auProject Link:https://github.com/npucino/sandpyperPublicationsPucino, N., Kennedy, D. M., Carvalho, R. C., Allan, B. & Ierodiaconou, D. Citizen science for monitoring seasonal-scale beach erosion and behaviour with aerial drones. Sci. Rep. 11, 3935 (2021)AcknowledgementsFunding provided by Deakin University and the Victorian Department of Environment, Land, Water and Planning.Deakin University Citizen Science program webpage.My PhD supervisorA/Prof Daniel Ierodiaconou,  and theDeakin Marine Mapping team.