Package: iceberg-rivers.search
Rivers (Arctic hydrology)We provide a classification algorithm for ice surface features from high-resolution imagery.  This algorithm was developed by convolutional neural network training to detect regions of large and small rivers and to distinguish them from crevasses and non-channelized slush. We also provide a detection algorithm to extract polyline water features from the classified high-probability river areas.Prerequisites - all available on bridges via the commands belowLinuxPython 3CPU and NVIDIA GPU + CUDA CuDNNSoftware Dependencies - these will be installed automatically with the installation below.numpyscipytifffilekeras==2.3.1opencv-pythonrasterioaffinepygdal==3.0.4tensorflow-gpu==1.15.0InstallationPreliminaries:These instructions are specific to XSEDE Bridges2 but other resources can be used if cuda, python3, and a NVIDIA P100 GPU are present, in which case 'module load' instructions can be skipped, which are specific to Bridges.For Unix or Mac Users:Login to Bridges 2 via ssh using a Unix or Mac command line terminal.  Login is available to bridges directly or through the XSEDE portal. Please see theBridges 2 User's Guide.For Windows Users:Many tools are available for ssh access to Bridges 2.  Please seeUbuntu,MobaXtermorPuTTYPSC Bridges 2Once you have logged into Bridges 2, you can follow one of two methods for installing iceberg-rivers.Method 1 (Recommended):The lines below following a '$' are commands to enter (or cut and paste) into your terminal (note that all commands are case-sensitive, meaning capital and lowercase letters are differentiated.)  Everything following '#' are comments to explain the reason for the command and should not be included in what you enter.  Lines that do not start with '$' or '[rivers_env] $' are output you should expect to see.$pwd/home/username
$cd$PROJECT# switch to your working space.$mkdirRivers# create a directory to work in.$cdRivers# move into your working directory.$moduleloadAI/anaconda3-tf1.2020.11
$exportPATH=/ocean/projects/mcb110096p/paraskev/gdal-3.0.4/bin:$PATH$exportLD_LIBRARY_PATH=/ocean/projects/mcb110096p/paraskev/gdal-3.0.4/lib:$LD_LIBRARY_PATH$exportGDAL_DATA=/ocean/projects/mcb110096p/paraskev/gdal-3.0.4/share/gdal
$condacreate--prefixiceberg_rivers--clone$AI_ENV$sourceactivateiceberg_rivers/[iceberg_rivers]$pwd/ocean/projects/group/username/Rivers[iceberg_rivers]$exportPYTHONPATH=/ocean/projects/group/username/iceberg_rivers/lib/python3.7/site-packages/[iceberg_rivers]$pipinstalliceberg_riversMethod 2 (Installing from source; recommended for developers only):$cd$PROJECT# switch to your working space.$mkdirRivers# create a directory to work in.$cdRivers# move into your working directory.$moduleloadAI/anaconda3-tf1.2020.11
$exportPATH=/ocean/projects/mcb110096p/paraskev/gdal-3.0.4/bin:$PATH$exportLD_LIBRARY_PATH=/ocean/projects/mcb110096p/paraskev/gdal-3.0.4/lib:$LD_LIBRARY_PATH$exportGDAL_DATA=/ocean/projects/mcb110096p/paraskev/gdal-3.0.4/share/gdal
$condacreate--prefixiceberg_rivers--clone$AI_ENV$sourceactivateiceberg_rivers/[rivers_env]$pwd/ocean/projects/group/username/Rivers
$gitclonehttps://github.com/iceberg-project/Rivers.git[iceberg_rivers]$exportPYTHONPATH=/ocean/projects/group/username/iceberg_rivers/lib/python3.7/site-packages/[iceberg_rivers]$pipinstall.To test[iceberg_rivers]$deactivate# exit your virtual environment.$interact--gpu# request a compute node.  This package has been tested on P100 GPUs on bridges, but that does not exclude any other resource that offers the same GPUs. (this may take a minute or two or more to receive an allocation).$cd$PROJECT/Rivers# make sure you are in the same directory where everything was set up before.$moduleloadAI/anaconda3-tf1.2020.11
$exportPATH=/ocean/projects/mcb110096p/paraskev/gdal-3.0.4/bin:$PATH$exportLD_LIBRARY_PATH=/ocean/projects/mcb110096p/paraskev/gdal-3.0.4/lib:$LD_LIBRARY_PATH$exportGDAL_DATA=/ocean/projects/mcb110096p/paraskev/gdal-3.0.4/share/gdal
$sourceactivateiceberg_rivers/# activate your environment, no need to create a new environment because the Rivers tools are installed and isolated here.[iceberg_rivers]$exportPYTHONPATH=/ocean/projects/group/username/iceberg_rivers/lib/python3.7/site-packages/[iceberg_rivers]$iceberg_rivers.tiling--help# this will display a help screen of available usage and parameters.PredictionDownload a pre-trained model at:You can download to your local machine and use scp, ftp, rsync, or Globus totransfer to bridges.Rivers predicting is executed in three steps:
First, follow the environment setup commands under 'To test' above. Then create tiles from an input GeoTiff image and write to the output_folder. The scale_bands parameter (in pixels) depends on the trained model being used.  The default scale_bands is 299 for the pre-trained model downloaded above.  If you use your own model the scale_bands may be different.[iceberg_rivers]$iceberg_rivers.tiling--tile_size=224--step=112--input=<image_abspath>--output=./test/Then, detect rivers on each tile and output counts and confidence for each tile.[iceberg_rivers]$iceberg_rivers.predict--input<tile_folder>-o<output_folder>-w<model>Finally, mosaic all the tiles back into one image[iceberg_rivers]$iceberg_rivers.mosaic--input_WVimage--input<masks_folder>--tile_size224--step112--output_folder./mosaic