BVLC/caffe — pull request #1448 by mtamburrano (2014-11-18 17:41:21+00:00)
Title: Conditional layer

This Layer, developed by me and @bhack , adds the possibility to select a subset of the batch whose items
pass a conditional test. It includes test cases and doxygen in common_layers.hpp.

The layer consists of:

3 bottom blobs:

- bottom\[0\]: this is the "bottom_IF". For each item, an argmax is computed
and the index of the max value is compared with the layer parameter
*conditional_index*. If *index_argmax* == *conditional_index*, the index
of the item in the blob is stored in a vector, let's call this vector
*indices_to_forward*.
- bottom\[1\]: this is the "bottom_THEN". Only the items whose indices are contained
by the *indices_to_forward* vector will be forwarded.
- bottom\[2\]: this is the bottom which contains the labels. If the layer parameter
*output_type* is equal to FILTERED_LABELS, then only the labels whose indices
are contained by the *indices_to_forward* vector will be forwarded,
otherwise this bottom is ignored.

2 top blobs:

- top[0]: If the layer parameter *output_type* == *FILTERED_LABELS*, then this blob
will contains the filtered labels as explained above. If *output_type* == *FILTERED_INDICES*
this blob will contains the indices of items that passed the conditional tests
(the same indices contained in the *indices_to_forward* vector), especially
useful for predicting.
- top\[1\]: This blob will contains the filtered items from bottom[1]

Params:

- **int** *conditional_index*:
The index which will be compared with the argmax index of each item in the bottom[0]
- **int** *output_type*:
two different output types for top[0]. *FILTERED_LABELS*, useful for training/testing,
and *FILTERED_INDICES*, useful for predicting.

Example:
Let's say we want a net that distinguishes between "cat" and "NOT-cat", and tries to recognize
the breed.
If the net recognizes an input as "NOT-cat", is useless to perform the remaining forward passes that try
to discriminate among the cat breeds.
So we could have something like this:

INPUT_DATA
|
CONV_LAYER1
|
POOL1
/ \
/ \
INNER_PRODUCT1 CONV_LAYER2
/ |
SOFTMAX1 POOL2
//outputs 0 or 1 if |
// is "NON-cat" or "cat" INNER_PRODUCT2
|
SOFTMAX2
//outputs the cat's breed

In this net, no matter what SOFTMAX1 produces, all the data input will be always forwarded
to the other branch of the net. So with conditional layer we could build the following net:

INPUT_DATA
|
CONV_LAYER1
|
POOL1
|
INNER_PRODUCT1
|
SOFTMAX1
//outputs 0 or 1 if is "NON-cat" or "cat"
|
CONDITIONAL:
{
bottom[0]: SOFTMAX1
bottom[1]: POOL1
bottom[2]: INPUT_DATA_labels
top[0]: FILTERED_INPUT_DATA_labels
top[1]: CONDITIONAL_POOL1
conditional_parameters
{
conditional_index: 1
output_type: FILTERED_LABELS
}
}
|
CONV_LAYER2
|
INNER_PRODUCT2
|
POOL2
|
SOFTMAX2 //outputs the cat's breed

Now, when the net reaches the conditional layer, the output of SOFTMAX1 is evaluated
to find those items that have SOFTMAX\[1\] >= SOFTMAX\[0\] (because *conditional_index* == 1, if
SOFTMAX\[1\] > SOFTMAX\[0\] we can presume that the item is a cat). Then we store the indices
of these items and forward from POOL1 to CONV_LAYER2 only the items with those indices
(or else, the cats!).

In the train phase this allows to train only a part of the input data in a specific branch
of the net, maintaining the possibility to have shared layers. In the predict phase a conditional layer
could save a lot of time: in the previous example probably the most computional expensive part
of the net would be the one that discriminates among the breeds, so all the forward passes for
the NON-cats items are spared.
For predicting is useful to use *output_type* with *FILTERED_INDICES*, so you can distinguish which of
the input data are been forwarded until the end of the net and which stopped before.

In addition, the conditional_layer can be used as logic ports. For example:

GENERIC_LAYER1
/ \
CONDITIONAL_L1 CONDITIONAL_L2
\ /
GENERIC_LAYER2

simulates an OR, on the contrary:

GENERIC_LAYER1
|
CONDITIONAL_L1
|
CONDITIONAL_L2
|
GENERIC_LAYER2

simulates an AND.

#####NOTES:
If used for predicting, the conditional layer works fine, instead if used in the train/test phase, actually can't works in DEV. This is because each LOSS Layer has the following check:

if (propagate_down[1]) {
LOG(FATAL) << this->type_name()
<< " Layer cannot backpropagate to label inputs.";
}
into the backward function. The problem is that my layer needs to backpropagate Top[1] but not Top\[0\] (that contains labels) but actually seems this is not possible because a layer needs or not needs backpropagation on ALL its blobs. To test my layer I commented that check in the LOSS layer and everything seems to work fine, the net converges normally. So how can I fix this? It is possible to remove that check in the LOSS layers? I presume it is there just to be sure someone has not made up the net wrong, attaching a non-label blob to a loss layer, right?