DOI: 10.5281/zenodo.5895128
title: Cape Hatteras Landsat8 RGB Images and Labels for Image Segmentation using the program, Segmentation Zoo

description: <strong># Cape Hatteras Landsat8 RGB Images and Labels for Image Segmentation using the program, Segmentation Zoo</strong> 1 zipped file, total size 3GB <strong>## Overview</strong> * Test datasets and files for testing the [segmentation zoo](https://github.com/dbuscombe-usgs/segmentation_zoo) program for image segmentation * Data set by Daniel Buscombe, Marda Science LLC. * Dataset consists of a time-series of Landsat-8 images of Cape Hatteras National Seashore, courtesy of the U.S. Geological Survey. * Imagery spans the period February 2015 to September 2021. * Labels were created by Daniel Buscombe, Marda Science, using the labeling program [Doodler](https://github.com/dbuscombe-usgs/dash_doodler). Download this file and unzip to somewhere on your machine (although preferably <em>*not*</em> the `segmentation_zoo` folder), then see [segmentation zoo](https://github.com/dbuscombe-usgs/segmentation_zoo) for further explanation. <strong>## file structure</strong> ```{sh} /Users/Someone/my_segmentation_zoo_datasets │ ├── config │ | └── *.json │ ├── capehatteras_data | | ├── fromDoodler | | | ├──images │ | | └──labels | | ├──npzForModel │ | └──toPredict │ └── modelOut │ └── *.png │ └── weights │ └── *.h5 ``` <strong>## config</strong> There are 3 config files: `/config/hatteras_l8_resunet.json` and `/config/hatteras_l8_vanilla_unet.json` for res-unet and unet models respectively. They both contain the same essential information and differ as indicated below ``` { "TARGET_SIZE": [768,768], # the size of the imagery you wish the model to train on. This may not be the original size "MODEL": "resunet", # model name. Otherwise, "unet" "NCLASSES": 4, # number of classes "KERNEL":9, # horizontal size of convolution kernel in pixels "STRIDE":2, # stride in convolution kernel "BATCH_SIZE": 7, # number of images/labels per batch "FILTERS":6, # number of filters "N_DATA_BANDS": 3, # number of image bands "DROPOUT":0.1, # amount of dropout "DROPOUT_CHANGE_PER_LAYER":0.0, # change in dropout per layer "DROPOUT_TYPE":"standard", # type of dropout. Otherwise "spatial" "USE_DROPOUT_ON_UPSAMPLING":false, # if true, dropout is used on upsampling as well as downsampling "DO_TRAIN": false, # if false, the model will not train, but you will select this config file, data directory, and the program will load the model weights and test the model on the validation subset if true, the model will train from scratch (warning! this will overwrite the existing weights file in h5 format) "LOSS":"dice", # model training loss function, otherwise "cat" for categorical cross-entropy "PATIENCE": 10, # number of epochs of no model improvement before training is aborted "MAX_EPOCHS": 100, # maximum number of training epochs "VALIDATION_SPLIT": 0.6, #proportion to use for validation "RAMPUP_EPOCHS": 20, # [LR-scheduler] rampup to maximim "SUSTAIN_EPOCHS": 0.0, # [LR-scheduler] sustain at maximum "EXP_DECAY": 0.9, # [LR-scheduler] decay rate "START_LR": 1e-7, # [LR-scheduler] start lr "MIN_LR": 1e-7, # [LR-scheduler] min lr "MAX_LR": 1e-4, # [LR-scheduler] max lr "FILTER_VALUE": 0, #if &gt;0, the size of a median filter to apply on outputs (not recommended unless you have noisy outputs) "DOPLOT": true, #make plots "ROOT_STRING": "hatteras_l8_aug_768", #data file (npz) prefix string "USEMASK": false, # use the convention 'mask' in label image file names, instead of the preferred 'label' "AUG_ROT": 5, # [augmentation] amount of rotation in degrees "AUG_ZOOM": 0.05, # [augmentation] amount of zoom as a proportion "AUG_WIDTHSHIFT": 0.05, # [augmentation] amount of random width shift as a proportion "AUG_HEIGHTSHIFT": 0.05,# [augmentation] amount of random width shift as a proportion "AUG_HFLIP": true, # [augmentation] if true, randomly apply horizontal flips "AUG_VFLIP": false, # [augmentation] if true, randomly apply vertical flips "AUG_LOOPS": 10, #[augmentation] number of portions to split the data into (recommended &gt; 2 to save memory) "AUG_COPIES": 5 #[augmentation] number iof augmented copies to make } ``` The extra config file, called `/config/deploy_ct_E_resunet_allclasses_512_remap_cat.json`, is an example deployment file for the similar but larger Landsat-8 data record within the Coast Train data release (forthcoming) <strong>## capehatteras_data</strong> Folder containing all the model input data ```{sh} │ ├── capehatteras_data: folder containing all the model input data | | ├── fromDoodler: folder containing images and labels exported from Doodler using [this program](https://github.com/dbuscombe-usgs/dash_doodler/blob/main/utils/gen_images_and_labels_4_zoo.py) | | | ├──images: jpg format files, one per label image │ | | └──labels: jpg format files, one per image | | ├──npzForModel: npz format files for model training using [this program](https://github.com/dbuscombe-usgs/segmentation_zoo/blob/main/train_model.py) that have been created following the workflow [documented here](https://github.com/dbuscombe-usgs/segmentation_zoo/wiki/Create-a-model-ready-dataset) using [this program](https://github.com/dbuscombe-usgs/segmentation_zoo/blob/main/make_nd_dataset.py) │ | └──toPredict: a folder of images to test model prediction using [this program](https://github.com/dbuscombe-usgs/segmentation_zoo/blob/main/seg_images_in_folder.py) ``` <strong>## modelOut</strong> PNG format files containing example model outputs from the train ('<em>_train_</em>' in filename) and validation ('<em>_val_</em>' in filename) subsets as well as an image showing training loss and accuracy curves with `trainhist` in the filename. There are two sets of these files, those associated with the residual unet trained with dice loss contain `resunet` in their name, and those from the UNet are named with `vanilla_unet`. <strong>## weights</strong> There are 2 model weights files namely `/weights/hatteras_l8_resunet.h5` and `/weights/hatteras_l8_vanilla_unet.h5`, associated with `/config/hatteras_l8_resunet.json` and `/config/hatteras_l8_vanilla_unet.json` config files.