Modular model definitions

A way to import a protobuf model into another one, by specifying a layer of type IMPORT. We went for a file system analogy, to allow referencing layers and blobs from different parts of the model:

* Imports are renamed to “name of the import layer/name”, e.g. “conv_pool1/relu”.
* Imported layers can reference layers and blobs from the importing network as a parent folder, e.g. “../data”, or using an absolute path: “/data”.
* By default names are relative references, so existing network definitions are fine. They will resolve all objects in the root folder.

Imports can be configured using ${variables}, that are applied during load using simple string replace.

We modified mnist/lenet_train_test.prototxt as an example, by exporting the Conv/Pool part as a module that is imported twice.

In lenet_train_test.prototxt:

<pre><code>
...
layers {
  name: "cp1"
  type: IMPORT
  import_param {
    net: "examples/mnist/lenet_conv_pool.prototxt"
    var { name: "bottom" value: "/data" }
    var { name: "num_output" value: "20" }
  }
}
layers {
  name: "cp2"
  type: IMPORT
  import_param {
    net: "examples/mnist/lenet_conv_pool.prototxt"
    var { name: "bottom" value: "../cp1/pool" }
    var { name: "num_output" value: "50" }
  }
}
…
</pre></code>

lenet_conv_pool.prototxt:

<pre><code>
layers {
  name: "conv"
  type: CONVOLUTION
  bottom: "${bottom}"
  top: "conv"
  blobs_lr: 1
  blobs_lr: 2
  convolution_param {
    num_output: ${num_output}
    kernel_size: 5
    stride: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layers {
  name: "pool"
  type: POOLING
  bottom: "conv"
  top: "pool"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
</pre></code>
The use case of this layer is very similar to the HTML template framework commonly used in web development where the variables are usually replaced by data fetched from the databases. Calling it TEMPLATE layer sounds more natural.