fireml 0.1.1
fireml machine learning framework

# README #

Caffe-like machine learning framework in python


## layers types

### ImageData
layer to read images. Possible sources: txt file with path labels on each line, cifar archive.

**image_data_param**:

source: string - path to cifar archive or txt file

batch_size: int - how many images to process in each iteration

shuffle: bool - images will be shuffled when sampling for a batch

new_height: int - new image height(can be same as original)

new_width: int - new image width(can be same as original)

new_labels: int - expected number of labels(txt file may contain multiple labels for each path)

example:

     layer {
       top: "data"
       top: "label"
       name: "data"
       type: "ImageData"
       image_data_param {
         source: "../cifar/cifar-10-python.tar.gz"
         source: "data.txt.3"  # use cifar or txt file
         batch_size: 65
         shuffle: true
         new_height: 32
         new_width: 32
         n_labels: 10
       }
       transform_param {
         mean_value: 126 # r
         mean_value: 123 # g
         mean_value: 114 # b
         mirror: true
         scale: 0.02728125
         standard_params {
             var_average: 5000
             mean_average: 5000
             mean_per_channel: false
             var_per_channel: false
         }
       }
       include: { phase: TRAIN }
     }

**transform_param**

Parameters for data preprocessing

**standard_params**

Parameters for preprocessor for data standardization.
To achieve zero mean and unit variance the preprocessor will subtract iterative 
mean from each sample and divide the result by standard deviation.

    standard_params {
        var_average: 1
        mean_average: 1
        mean_per_channel: false
        var_per_channel: true
    }

var_average: int [default = 0] - use last var_average samples to compute variance and std  
disabled if var_average == 0  
mean_average: int [default = 0]  - use last var_average samples to compute mean  
disabled if mean_average == 0  
mean_per_channel: bool [default = false] - subtract from each channel mean for that channel  
var_per_channel: [default = false] - divide each channel by separate std value  

### Convolution

Convolution of 2-3 d images(matrices)

**convolution_param**

num_output: int   number of filters(output feature maps)

kernel_size:  int    size of receptive field of filters. Receptive field is kernel_size * kernel_size

stride: int - filter will be applied after stride pixels

weight_filler: see weight filler

example:

    layer {
      name: "conv1"
      type: "Convolution"
      bottom: "data"
      top: "conv1"
      convolution_param {
        num_output: 40
        kernel_size: 3
        stride: 2
        weight_filler {
          type: "xavier"
          variance_norm: AVERAGE
        }
      }
    }


### Pooling

Subsampling layer for max or average pooling

**pooling_param**

pool: MAX or AVE

kernel_size: int 
subsampling window size

stride: int
perform pooling each *stride* pixels

example:

    layer {
      name: "pool1"
      type: "Pooling"
      bottom: "conv1"
      top: "pool1"
      pooling_param {
        pool: MAX
        kernel_size: 3
        stride: 2
      }
    }

### Accuracy

Layer for computing accuracy
Accuracy of a classifier is defined as (true positive + true negative)/total  
In multilabel classification example counts as correctly classified iff **all** outputs
 are correct.  
Example:   

    layer {
      name: "accuracy"
      type: "Accuracy"
      bottom: "pool10"
      bottom: "label"
      top: "accuracy"
    }

## weight_filler

Weigth filler parameters are common for all layers with weights

type: string  
"xavier", "gaussian", "uniform"

mean: float  
mean value for gaussian initialization

std: float  
standard deviation for gaussian initialization

min: float  
lower bound for uniform initialization

max: float  
upper bound for uniform initialization

## Activation functions

### SeLU

Self-regularized linear unit:

example:  
    {
      name: "relu_conv1"
      type: "SeLU"
      bottom: "conv1"
      top: "conv1"
    }

## Loss layers

### SigmoidCrossEntropyLoss

Layer that applies sigmoid elementwise, followed by cross-entropy log loss
-mean(sum(y * log(p(y))  + (1 - y) * log(1 - p(y))))

where p(y) - sigmoid transformation of layer's input, that is vector of independent probabilities for each class.

    example:  
        layer { 
          name: "loss"
          type: "SigmoidCrossEntropyLoss"
          bottom: "pool1"
          bottom: "label"
          top: "loss"
          include {
              phase: TRAIN
          }
        }

## Maxout layer

Apply max operator for each *size* channels  

size: int [default = 0] - take max over each *size* channels  
lambda: int [default = 0.0] - apply probabilistic max if lambda != 0  

    layer {
      name: "maxout_1"
      type: "Maxout"
      maxout_param {
        lambda: 1
        size: 2
      }
      bottom: "conv1"
      top: "conv1"
    }