deeplab2 / evaluator.proto
akhaliq3
spaces demo
506da10
// Copyright 2021 The Deeplab2 Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto2";
package deeplab2;
// Next ID: 22
message EvaluatorOptions {
// Set the number of steps to run evaluation. -1 corresponds to a run over the
// full dataset.
optional int32 eval_steps = 1 [default = -1];
// Set the number of train steps after which eval should run in interleaved
// mode.
optional int32 eval_interval = 2 [default = 5000];
// Set the number of seconds to wait at most for the next checkpoint. -1 means
// the job will wait forever.
optional int32 continuous_eval_timeout = 3 [default = -1];
// Set whether to run evaluation as a tf function.
optional bool use_tf_function = 4 [default = true];
// Set the area size of stuff segments to discard.
optional int32 stuff_area_limit = 6 [default = 0];
// Set the area size of thing segments to discard (set to ignore_label). Note
// that this option is currently only supported in MaX-DeepLab.
optional int32 thing_area_limit = 19 [default = 0];
// Set the threshold for the transformer class confidence.
optional float transformer_class_confidence_threshold = 20 [default = 0.7];
// Set the threshold for the per-pixel mask confidence. Note that this option
// is currently only supported in MaX-DeepLab.
optional float pixel_confidence_threshold = 21 [default = 0.4];
// Set the threshold of the center heatmap for post-processing.
optional float center_score_threshold = 7 [default = 0.1];
// Set the kernel size of the nms kernel for the center heatmap.
optional int32 nms_kernel = 8 [default = 3];
// Set the number of top centers to keep. -1 corresponds to keeping all
// centers.
optional int32 keep_k_centers = 9 [default = 400];
// Enable saving predictions to disk.
optional bool save_predictions = 10 [default = false];
// Override the storing location. By default, predictions are written to
// `experiment_root` + `experiment_name` + `vis`.
optional string override_save_dir = 11;
// Set the number of samples to visualize.
optional int32 num_vis_samples = 12 [default = 10];
// Enable saving raw predictions for the whole dataset. The output path is the
// save_dir + `raw_semantic`/`raw_panoptic`.
optional bool save_raw_predictions = 13 [default = false];
// The format of raw panoptic predictions. This flag is used together with
// `save_raw_predictions`. When save_raw_predictions is True, this field
// specifies the format of saved raw panoptic predictions. Supports:
// - 'two_channel_png': The popular format, also supported by the official
// COCO panoptic API (https://github.com/cocodataset/panopticapi), where
// the saved PNG image contains R-channel for semantic labels and
// G-channel for instance IDs.
// - 'three_channel_png': A simple extension of the 'two_channel_png' format,
// and is adopted in some video panoptic segmentation datasets (for
// example, KITTI-STEP and MOTChallenge-STEP), where the saved PNG image
// contains R-channel for semantic labels, G-channel for the values of
// (instance ID // 256), and B-channel for (instance ID % 256).
// - 'two_channel_numpy_array': A more flexible format (unconstrained by the
// PNG channel size), where the panoptic predictions are saved as a numpy
// array in the two channel format (i.e., first channel encodes the
// semantic class and the second channel the instance ID).
optional string raw_panoptic_format = 17 [default = 'two_channel_png'];
// Enable conversion of train IDs to eval IDs for raw predictions.
optional bool convert_raw_to_eval_ids = 14 [default = true];
// Add flipped images for evaluation or not. This is used for multi-scale
// inference (usually used together with `eval_scales`). If True, another
// flipped image will be used during inference.
optional bool add_flipped_images = 5 [default = false];
// The scales to resize images for inference. Change it to, e.g. [0.5, 0.75,
// 1.0, 1.25, 1.5, 1.75], for multi-scale inference.
repeated float eval_scales = 15 [packed = true];
// Boolean, if true, use TensorFlow operation (CUDA kernel) to merge
// semantic and instance segmentation (for the final panoptic segmentation).
// Defaults to true, as our GPU implementation is much faster. Set to false
// if you could not successfully compile TensorFlow with this operation.
optional bool merge_semantic_and_instance_with_tf_op = 16 [default = true];
// Displays detailed metrics on instance segmentation AP. This includes e.g.
// AP at a matching IoU threshold of 0.5, or the AP of small objects only,
// etc. If false, will only display a summary AP metric that's an average of
// IoU thresholds and over all objects.
optional bool detailed_ap_metrics = 18 [default = false];
}