Package: tf-semantic-segmentation
TF Semantic SegmentationQuick StartSeeGETTING_STARTED, or theColab Notebook.Learn more at ourdocumentation.
See upcoming features on ourroadmap.FeaturesDistributed Training on Multiple GPUsHyper Parameter Optimization using WandBWandB IntegrationEasily create TFRecord from DirectoryTensorboard visualizationsEnsemble inferenceDatasetsAde20kCamvidCityscapesMappingChallengeMotsChallengeCocoPascalVoc2012TacoShapes (randomly creating triangles, rectangles and circles)Toy (Overlaying TinyImageNet with MNIST)ISIC2018CVC-ClinicDBModelsUnetErfnetMultiResUnetPSP(experimental)FCN(experimental)NestedUnet (Unet++)(experimental)U2Net / U2NetP(experimental)SatelliteUnetMobilenetUnet (unet with mobilenet encoder pre-trained on imagenet)InceptionResnetV2Unet (unet with inception-resnet v2 encoder pre-trained on imagenet)ResnetUnet (unet with resnet50 encoder pre-trained on imagenet)AttentionUnetLossesCatagorical CrossentropyBinary CrossentropyCrossentropy + SSIMDiceCrossentropy + DiceTverskyFocalFocal + TverskyActivationsmishswishrelu6OptimizersRangerRAdamNormalizationInstance NormBatch NormOn the fly Augmentationsflip left/rightflip up/downrot 180colorGetting StartedRequirementssudoapt-getinstalllibsm6libxext6libxrender-devlibyaml-devlibpython3-devTensorflow (2.x) & Tensorflow Addons (optional)pipinstalltensorflow-gpu==2.4.0--upgrade
pipinstalltensorflow-addons==0.12.0--upgradeInstallationpipinstalltf-semantic-segmentationRun tensorboardHint: To see train/test/val images you have to start tensorboard like thistensorboard--logdir=logs/--reload_multifile=trueTrain on inbuild datasets (generator)python-mtf_semantic_segmentation.bin.train-ds'tacobinary'-bs8-e100\-logdir'logs/taco-binary-test'-o'adam'-lr5e-3--size256,256\-l'binary_crossentropy'-fa'sigmoid'\--train_on_generator--gpus='0'\--tensorboard_train_images--tensorboard_val_imagesCreate a tfrecord from a dataset# create a tfrecord from the toy dataset and resize to 128x128tf-semantic-segmentation-tfrecord-writer-d'toy'-c/hdd/datasets/-s'128,128'Train using a fixed record pathpython-mtf_semantic_segmentation.bin.train--record_dir=records/cityscapes-512x256-rgb/\-bs4-e100-logdir'logs/cityscapes-bs8-e100-512x256'-o'adam'-lr1e-4-l'categorical_crossentropy'\-fa'softmax'-bufsize50--metrics='iou_score,f1_score'-m'erfnet'--gpus='0'-a'mish'\--tensorboard_train_images--tensorboard_val_imagesMulti GPU trainingpython-mtf_semantic_segmentation.bin.train--record_dir=records/cityscapes-512x256-rgb/\-bs4-e100-logdir'logs/cityscapes-bs8-e100-512x256'-o'adam'-lr1e-4-l'categorical_crossentropy'\-fa'softmax'-bufsize50--metrics='iou_score,f1_score'-m'erfnet'--gpus='0,1,2,3'-a'mish'Using Codefromtf_semantic_segmentation.bin.trainimporttrain_test_model,get_args# get the default argsargs=get_args({})# change some parameters# !rm -r logs/args.model='erfnet'# args['color_mode'] = 0args.batch_size=8args.size=[128,128]# resize input dataset to this sizeargs.epochs=10args.learning_rate=1e-4args.optimizer='adam'# ['adam', 'radam', 'ranger']args.loss='dice'args.logdir='logs'args.record_dir="datasets/shapes/records"args.final_activation='softmax'# train and testresults,model=train_test_model(args)ModelsErfnetUnetfromtf_semantic_segmentationimportmodels# print all available modelsprint(list(modes.models_by_name.keys()))# returns a model (without the final activation function)model=models.get_model_by_name('erfnet',{"input_shape":(128,128,3),"num_classes":5})# call models directlymodel=models.erfnet(input_shape=(128,128),num_classes=5)Use your own datasetAccepted file types are: jpg(jpeg) and pngIf you already have a train/test/val split then use the following data structure:dataset/
    labels.txt
    test/
        images/
        masks/
    train/
        images/
        masks/
    val/
        images/
        masks/or usedataset/
    labels.txt
    images/
    masks/The labels.txt should contain a list of labels separated by newline [/n]. For instance it looks like this:background
car
pedestrianTo create a tfrecord using the original image size and color use the script like this:INPUT_DIR=...
tf-semantic-segmentation-tfrecord-writer-dir$INPUT_DIR-r$INPUT_DIR/recordsThere are the following addition arguments:-s [--size] '$width,$height' (f.e. "512,512")-rm [--resize_method] ('resize', 'resize_with_pad', 'resize_with_crop_or_pad)cm [--color_mode] (0=RGB, 1=GRAY, 2=NONE (default))Datasetsfromtf_semantic_sementation.datasetsimportget_datasetbyname,datasets_by_name,DataType,get_cache_dir# print availiable dataset namesprint(list(datasets_by_name.keys()))# get the binary (waste or not) datasetdata_dir='/hdd/data/'name='tacobinary'cache_dir=get_cache_dir(data_dir,name.lower())ds=get_dataset_by_name(name,cache_dir)# print labels and classesprint(ds.labels)print(ds.num_classes)# print number of training examplesprint(ds.num_examples(DataType.TRAIN))# or simply print the summaryds.summary()Debug datasetspython-mtf_semantic_segmentation.debug.dataset_vis-dade20kTFRecordsThis library simplicifies the process of creating a tfrecord dataset for faster training.Write tfrecords:fromtf_semantic_segmentation.datasetsimportTFWriterds=...writer=TFWriter(record_dir)writer.write(ds)writer.validate(ds)or use simple with this script (will be save with size 128 x 128 (width x height)):tf-semantic-segmentation-tfrecord-writer-d'toy'-c/hdd/datasets/-s'128,128'Analyse already written tfrecord (with mean)python-mtf_semantic_segmentation.bin.tfrecord_analyser-rrecords/--meanDockerdockerbuild-ttf_semantic_segmentation-fdocker/Dockerfile./or pull the latest releasedockerpullbaudcode/tf_semantic_segmentation:latestPredictionpipinstallmatplotlibUsing Codefromtensorflow.keras.modelsimportload_modelimportnumpyasnpfromtf_semantic_segmentation.processingimportdatasetfromtf_semantic_segmentation.visualizationsimportshow,masksmodel=load_model('logs/model-best.h5',compile=False)# model parameterssize=tuple(model.input.shape[1:3])depth=model.input.shape[-1]color_mode=dataset.ColorMode.GRAYifdepth==1elsedataset.ColorMode.RGB# define an imageimage=np.zeros((256,256,3),np.uint8)# preprocessingimage=image.astype(np.float32)/255.image,_=dataset.resize_and_change_color(image,None,size,color_mode,resize_method='resize')image_batch=np.expand_dims(image,axis=0)# predict (returns probabilities)p=model.predict(image_batch)# draw segmentation mapnum_classes=p.shape[-1]ifp.shape[-1]>1else2predictions_rgb=masks.get_colored_segmentation_mask(p,num_classes,images=image_batch,binary_threshold=0.5)# show images using matplotlibshow.show_images([predictions_rgb[0],image_batch[0]])Using scriptsOn imagepython-mtf_semantic_segmentation.evaluation.predict-mmodel-best.h5-iimage.pngOn TFRecord (data type 'val' is default)python-mtf_semantic_segmentation.evaluation.predict-mmodel-best.h5-rrecords/camvid/On TFRecord (with export to directory)python-mtf_semantic_segmentation.evaluation.predict-mmodel-best.h5-rrecords/cubbinary/-oout/-rm'resize_with_pad'On Videopython-mtf_semantic_segmentation.evaluation.predict-mmodel-best.h5-vvideo.mp4On Video (with export to out/p-video.mp4)python-mtf_semantic_segmentation.evaluation.predict-mmodel-best.h5-vvideo.mp4-oout/Prediction using Tensorflow Model ServerInstallation# installecho"deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal"|sudotee/etc/apt/sources.list.d/tensorflow-serving.list&&\curlhttps://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg|sudoapt-keyadd-
sudoapt-getupdate&&apt-getinstalltensorflow-model-serverStart Model Server### using a single modeltensorflow_model_server--rest_api_port=8501--model_base_path=/home/user/models/mymodel/saved_model### or using an ensamble of multiple models# helper to write the ensamble config yaml file (models/ contains multiple logdirs/, logdir must contain the name 'unet')python-mtf_semantic_segmentation.bin.model_server_config_writer-dmodels/-c'unet'# start model server with written models.yamltensorflow_model_server--model_config_file=models.yaml--rest_api_port=8501Compare models and ensemnblepython-mtf_semantic_segmentation.evaluation.compare_models-ilogs/-c'taco'-data/hdd/datasets/-d'tacobinary'Parameters:-i(directory containing models)-c(model name (directory name) must contain this value)-data(data directory)-d(dataset name)Use--helpto get more helpUsing Codefromtf_semantic_segmentation.servingimportpredict,predict_on_batch,ensamble_prediction,get_models_from_directoryfromtf_semantic_segmentation.processing.datasetimportresize_and_change_colorimage=np.zeros((128,128,3))image_size=(256,256)color_mode=0# 0=RGB, 1=GRAYresize_method='resize'scale_mask=False# only scale mask when model output is scaled using sigmoid activationnum_classes=3# preprocess imageimage=image.astype(np.float32)/255.image,_=resize_and_change_color(image,None,image_size,color_mode,resize_method='resize')# prediction on 1 imagep=predict(image.numpy(),host='localhost',port=8501,input_name='input_1',model_name='0')############################################################################################################## if the image size should not match, the color mode does not match or the model_name does not match# you'll most likely get a `400 Client Error: Bad Request for url: http://localhost:8501/v1/models/0:predict`# hint: if you only started 1 model try using model_name 'default'############################################################################################################## prediction on batch (for faster prediction of multiple images)p=predict_on_batch([image],host='localhost',port=8501,input_name='input_1',model_name='0')# ensamble prediction (average the predictions of multiple models)# either specify models like this:models=[{"name":"0","path":"/home/user/models/mymodel/saved_model/","version":0,# optional"input_name":"input_1"},{"name":"1","path":"/home/user/models/mymodel2/saved_model/","input_name":"input_1"}]# or load from models in directory (models/) that contain the name 'unet'models=get_models_from_directory('models/',contains='unet')# returns the ensamble and all predictions madeensamble,predictions=ensamble_prediction(models,image.numpy(),host='localhost',port=8501)TFLite supportConvert the modelpython-mtf_semantic_segmentation.bin.convert_tflite-ilogs/mymodel/saved_model/0/-omodel.tfliteTest inference on the modelpython-mtf_semantic_segmentation.debug.tflite_test-mmodel.tflite-iHarris_Sparrow_0001_116398.jpg