Package: deepomatic-oef
Open-Experiment-Format (OEF)Open-Experiment-Format lets you define your model and experiment in a standardized and serializable format.InstallationOEF is PyPi package and can be installed simply by doing:pipinstalldeepomatic-oefUsing the API to launch a training on Deepomatic StudioIf you want to use training parameters which are not exposed on Studio, you can launch an experiment via the OEF API calllaunch_experimenton any dataset and view.Make sure that you have your Studio credentials for the given organisation. You can define directly theDEEPOMATIC_API_KEYenvironment variable:$exportDEEPOMATIC_API_KEY=abcdef0123456789or pass it as an keyword-argument to thelaunch_experimentfunction.The first step is to select the model you would like to use among the list of supported models (seethis section).fromdeepomatic.oef.utils.experiment_builderimportExperimentBuilderfromdeepomatic.oef.api.experimentimportlaunch_experimentbuilder=ExperimentBuilder('image_detection.pretraining_natural_rgb.faster_rcnn.resnet_101_v1')xp=builder.build()launch_experiment(xp,'organisation-slug','dataset-slug','view-uuid','Model name')Parametrizing your experimentThe.build()method of the builder can take many parameters. To get a complete documentation of the possible parameters, pleasevisit this page. You can pass as many parameters as you want from theExperimentandTrainerprotobuf messages, they will be set at the correct location automatically. For example, you can change theinitial_learning_ratefrom theExperimentprotobuf andbatch_sizefrom theTrainerprotobuf like this:xp=builder.build(initial_learning_rate=0.001,batch_size=32,)In the next subsections, you will find a few typical usages.Changing the input sizeYou may want to change the default input size of your model. Be aware that choosing sizes that are incompatible with the model architecture may lead to a crash. Refer tothis pageto get the default input sizes.Changing the input so that images are always resized to a fixed size can be done as follows:xp=builder.build(image_resizer={'fixed_shape_resizer':{'height':480,'width':640,# Those parameters are optional'convert_to_grayscale':False}})Refer to thedetailed documentationfor a description of all parameters. You may also want to resize input images in a way that respects the aspect ration of input images:xp=builder.build(image_resizer={'keep_aspect_ratio_resizer':{'min_dimension':600,'max_dimension':1024,# Those parameters are optional'convert_to_grayscale':False,'pad_to_max_dimension':False,'per_channel_pad_value':None}})Changing the learning rate intial value and policyControl on the learning rate can be done as follow. Refer to thedetailed documentationfor a description of all parameters.# Constant learning ratexp=builder.build(initial_learning_rate=0.0003,learning_rate_policy={'constant_learning_rate':{},})# Step learning rate, manually definedxp=builder.build(initial_learning_rate=0.0003,learning_rate_policy={'manual_step_learning_rate':{'warmup':False,'schedule':[{'step_pct':0.333,'learning_rate_factor':0.1},{'step_pct':0.666,'learning_rate_factor':0.01},],}})# Exponential decay policyxp=builder.build(initial_learning_rate=0.0003,learning_rate_policy={'exponential_decay_learning_rate':{# Those parameters are optional'decay_steps_pct':0.006,'decay_factor':0.95,'staircase':True,'burnin_learning_rate':0,'burnin_steps_pct':0,'min_learning_rate':0,}})# Cosine decay policyxp=builder.build(initial_learning_rate=0.0003,learning_rate_policy={'cosine_decay_learning_rate':{# Those parameters are optional'total_steps_pct':1.07,'warmup_learning_rate':0.0002,'warmup_steps_pct':0.0025,'hold_base_rate_steps_pct':0,}})Changing the optimizerControl on the optimizer can be done as follow. Refer to thedetailed documentationfor a description of all parameters.# Momentum optimizerxp=builder.build(optimizer={'momentum_optimizer':{'momentum_optimizer_value':0.9}},)# RMSprop optimizerxp=builder.build(optimizer={'rms_prop_optimizer':{'momentum_optimizer_value':0.9}},)# Adam optimizerxp=builder.build(optimizer={'adam_optimizer':{'momentum_optimizer_value':0.9}},)Short cutsIn the above examples, we explicitely gave the whole object with its nested parameters, but thebuildfunction descends the hierarchical structure tree until it finds the given parameter. One can hence directly define a single parameter easily without nesting.xp=builder.build(# Sets trainer.optimizer to adam_optimizeradam_optimizer={},# Sets trainer.learning_rate_policy to constant_learning_rateconstant_learning_rate={},# Sets trainer.image_task.backbone.input.image_resizer.default_resizer.convert_to_grayscale to Trueconvert_to_grayscale=True,)Nota Bene: This works when the given parameter name is unique. If not, the first occurence of a parameter with this name is set to the given value.Defining a hyper-parameter searchWe can also define a hyper-parameter search with its space definition and the maximum number of trials.To define a search space, you can add parameters with theadd_hyperparameterfunction of theExperimentBuilderclass by using its name as an input. The function raises an error if the name is incorrect. An optional keyword argument is thedistributionwhich is by defaultNone. If not given, the default distribution is looked up for the specific hyperparameter. If not found, the function raises an Exception. If an distribution is given, it is used instead of the default distribution.To run a hyper-parameter search, themax_hp_runsshould be set to an integer greater than 1 (default value).fromdeepomatic.oef.utils.experiment_builderimportExperimentBuildermodel='image_classification.pretraining_natural_rgb.softmax.efficientnet_b0'builder=ExperimentBuilder(model)builder.add_hyperparameter('trainer.optimizer')# Adds the optimizer as an hyperparameter with the default distributionbuilder.add_hyperparameter('trainer.optimizer.rms_prop_optimizer.momentum_optimizer_value')# Adds the momentum optimizer value as hyperparameter, but only for the rms_prop_optimizer and not for the momentum_optimizer# builder.add_hyperparameter('trainer.batch_size')  # This would raise an Exception since trainer.batch_size has no default distribution# But we can still add it as an hyperparameter with an user-defined distributionbuilder.add_hyperparameter('trainer.batch_size',distribution={'categorical':{'values':[{'integer_value':2},{'integer_value':4},{'integer_value':8}]}})# builder.add_hyperparameter('trainer.doesnot.exist')  # would also raise an Exception since the path is invalid# We can now set custom experiment values. For hyperparameters we check that they fall into the defined distribution rangexp_proto=builder.build(config_path='gs://dp-thoth/datasets/oxford-pets/config_mini_classif_v4.prototxt',num_train_steps=10,max_hp_runs=5,# Needs to be > 1 to activate the hyper-parameter searchbatch_size=2,# This works because 2 is in [2, 4, 8]momentum_optimizer_value=0.3,# this changes the momentum_optimizer_value for the momentum_optimizer and not for the rms_prop_optimizer, since the momentum_optimizer is the default# optimizer={'adam_optimizer': {}},  # this will raise an Exception since the adam_optimizer is not in the default distribution)Thehyperparameter_dump.pyscript looks recursively through all the protobuf message fields to find the hyperparameter definitions and dumps a JSON file with{full_message_name: {field_name: hyperparameter_definition_dict}}and can be used to see all the default distributions defined in OEF per protobuf message.Which models are available?You can import the dictionarymodel_listcontaining all model keys and there specific parameters. The keys of the dictionary have the following structuretask_type.pretraining_type.meta_architecture.backbone, wheretask_typecan beimage_classification,image_detection,image_ocr, and soonimage_segmentation, and the pretraining can be eitherpretraining_natural_rgborpretraining_none.If you want to get a list of all available pretrained multi-class classification models you can do that:fromdeepomatic.oef.configs.model_listimportmodel_listclassification=[kforkinmodel_list.keys()ifk.split[:-1]==('image_classification','pretraining_natural_rgb','softmax')]