Feature Graph Architectures
Authors: Richard Davis, Sanjay Chawla, Philip Leong
ICLR 2014

Abstract: In this article we propose feature graph architectures (FGA), which are deep learning systems employing a structured initialisation and training method based on a feature graph which facilitates improved generalisation performance compared with a standard shallow architecture. The goal is to explore alternative perspectives on the problem of deep network training. We evaluate FGA performance for deep SVMs on some experimental datasets, and show how generalisation and stability results may be derived for these models. We describe the effect of permutations on the model accuracy, and give a criterion for the optimal permutation in terms of feature correlations. The experimental results show that the algorithm produces robust and significant test set improvements over a standard shallow SVM training method for a range of datasets. These gains are achieved with a moderate increase in time complexity.


Peer review:
[Review]:
* A brief summary of the paper's contributions, in the context of prior work.
Papers suggests to stack multiple machine learning modules on top of each other. Applies it to SVMs.

* An assessment of novelty and quality. 
        Not novel, quality is low.

* A list of pros and cons (reasons to accept/reject).
pros :
Exploration of non-standard deep architectures. Good direction of research.

cons :
Paper suggests randomly establish group of features to find correlated groups. This task might be extremely expensive and infeasible. 

Very poor experiments. One experiment on synthetic data, which is trivial to learn (sum xi) ^ 2, and others on unknown datasets.

It is unclear where is non-linearity coming from. If this are just SVMs stacked on top of each other, and there is no non-linearity in between ? Then entire procedure is just a linear classifier with regularization. 

Paper doesn’t state what is optimization objective of the entire system. It just brings an algorithm.

Dimensionality of data is extremely small ~25 dims.