Package: petrel-det
PetrelPyTorch has a very reliable implementation ofEfficientDetbut like all deep learning frameworks it requires a fair amount of work
to prepare the data for use in the model. Petrel streamlines this process
by providing a standard format for loading training and validation data
as well as a single class manage the whole training pipeline. The major
components are described below.Input Data Format.Petrel's Dataset classes take two dataframes as inputs, one consisting
of one row per image, the other consisting of one row per bounding box.The first table,metadata, is only required to contain one column:fileThis should be the absolute path name to the image file. Images
can be stored in any manner the user wishes.It is strongly recommended thatmetadatarows contain aheightand awidthcolumn for each image as well. Although
not strictly necessary, it is very likely the preprocessing pipeline
will require this information. Any other image level metadata should be
included in this table, but is not used by this package.Ensure that the indices formetadatarange from0tonumrows - 1.A sample table with the minimum recommended columns is below. This data
comes fromNABirds; the file
name consists of the species level directory number (encoded elsewhere)
and the random hex string identifying the image file.fileheightwidth00645/0001afd499a14a67b940d419413e23b3.jpg680102410900/0007181fa7274481ad89591200c61b9d.jpg819102420988/00071e2081564bd8b5ca6445c2560ee5.jpg768102430845/00081fce2a744a9fb52b9bb8871a48e2.jpg817102440698/00085a7befcc4c08a83038477e749101.jpg7251024The second table,boxes, contains the information about the
bounding boxes for each image. Bounding box coordinates are stored
in PASCAL VOC format with each coordinate a separate column. These are
converted into a single tensor by thePetrelmodule.
Six columns are required.fileThe name of the file containing the bounding box.labelsA numeric label for the class of the object contained in
the bounding box. Numbers should range from1tonum_clases. Zero is reserved for the empty class.xminUpper left horizontal coordinate of the bounding box.yminUpper left vertical coordinate of the boundingbox.xmaxLower right horizontal coordinate of the bounding box.ymaxLower right vertical coordinate of the bounding box.A seventh column containing class names, to go with thelabelcolumn,
is useful, but not required. Other columns can be added as the user
requires.A sample table, this time from theSweden Larch Casebearerdataset is below. In this tabledamageis the class name
corresponding to the label.filedamagelabelsxminyminxmaxymax0Jallasvag_20190527/Images/B03_0002.JPGHD22052882973801Jallasvag_20190527/Images/B03_0002.JPGHD22761864253992Kampe_20190527/Images/B04_0053.JPGLD328781747410173Kampe_20190527/Images/B04_0130.JPGHD25373016414114Kampe_20190527/Images/B04_0115.JPGH113615671455651Preparing Training and Validation DataThe classesTrainDatasetandValDatasetextend the basePetrelDatasetclass by adding functionality for separate training
and validation preprocessing. Preprocessing is done by a chained
series of transformations operating on both the image and the bounding
boxes.Preprocessing for training usually involves iterating through a series
of transformations, some of which are random (i.e. random cropping).
There is usually a requirement that at least one bounding box in the
image be preserved be contained in the final output, otherwise the
preprocessing steps are run again. If this fails after a certain number
of attempts another set of transformations (which may or may not be
the same) is done once in its place.TrainDatasetis designed to
be used withAlbumentationschained
together byCompose, but could work with another package with a
similar API.Since validation requires the same images for each epoch
ValDataset` only performs one transformation operation.Users are expected to provide their own transformation functions as
this is the part of model training that is most dependent on the
dataset in use. A sample transformation that only reshapes images to
a fixed size is provided indataset/___init__.pyA sampleTrainDatasetandValDatasetis construction is shown below.frompetrel.datasetimportTrainDataset,ValDatasettrain_dataset=TrainDataset(meta_data=TRAINING_METADATA,boxes=TRAINING_BOXES,image_root=f"{TRAIN_IMG_ROOT_DIR}",transform=TRAIN_TRANSFORM)val_dataset=ValDataset(meta_data=VAL_METADATA,boxes=VAL_BOXES,image_root=f"{VAL_IMG_ROOT_DIR}",transform=VAL_TRANSFORM,train_pipe=True#Set to false if not used for training.)Setting up and training a model is even simpler.frompetrel.modelimportload_edet,load_optimizer,load_scheduler,ModelTrainermodel=load_edet("tf_efficientdet_d0",image_size=512,num_classes=4)optimizer=load_optimizer("adamw",model,learning_rate=2.56e-3)scheduler=load_scheduler("exponential",optimizer=optimizer,gamma=0.94**0.25)model_trainer=ModelTrainer(model,optimizer,scheduler,base_dir=f"{BASE_MODEL_DIR}")Prediction and EvaluationThepredictmodule contains code for conputing predictions for
single batches, which can be size1, or whole datasets.
Thepredictfunction takes an image and an optional dictionary
of bounding boxes and labels and returns a dictionary of bounding
box predictions. When a full batch job is required,predict_dforval_predict_dfcan be used depending on whether ground truth data
is available.Theevalmodule computes evaluation metrics. For a range of IOU
match thresholds, defaulting to the COCO standard0.50:0.05:0.95,evalcomputes the PASCAL-VOC mAP for each class and an unweighted
average mAP over all classes. These can then be averaged over the IOU
thresholds to approximate the COCO mAP (COCO mAP averages over
regular recall values0.01:0.01:0.99while PASCAL-VOC mAP is a
continous integral over recall values; the difference will be slight.)Below is an example of evaluating a model on a dataset.frompetrel.modelimportload_edetfrompetrel.predictimportmodel_eval,val_prediction_dfmodel=load_edet("tf_efficientdet_d0",image_size=512,num_classes=4,checkpoint_path=f"{BASE_MODEL_DIR}",train=False)pred_df=val_prediction_df(model,val_loader,verbose=20)eval_df=model_eval(pred_df,categories=categories)The output ofmodel_evalfor a simple EfficientDet0 on the Larch
Casebearer Dataset is shown below.IOU ThresholdPrecisionHHDLDother0.5070.7454.0170.9186.0072.060.5569.6653.6969.7384.9470.290.6067.7252.6567.8383.2067.190.6564.7851.0164.5180.5163.090.7059.5548.5658.3775.3355.940.7550.1441.7247.2165.7445.870.8035.4130.7330.4248.6431.860.8517.2915.6812.0225.2816.160.903.623.611.835.183.870.950.080.060.030.110.11Averaged over the IOU Thresholds the (approximate) COCO mAP results areCategorymAPPrecision43.90Healthy35.17High Damage42.29Low Damage55.49Other42.64