Learning Transformations for Classification Forests
Authors: Qiang Qiu, Guillermo Sapiro
ICLR 2014

Abstract: This work introduces a transformation-based learner model for classification forests. The weak learner at each split node plays a crucial role in a classification tree. We propose to optimize the splitting objective by learning a linear transformation on subspaces using nuclear norm as the optimization criteria. The learned linear transformation restores a low-rank structure for data from the same class, and, at the same time, maximizes the separation between different classes, thereby improving the performance of the split function. Theoretical and experimental results support the proposed framework.


Peer review:
[Review]:
The manuscript presents a novel approach to random forests in which a linear discriminative transformation is learned at each split node in order to enhance class separation of the weak learners. The work offers a merger of existing ideas in an innovative way. The experimental results, which include synthetic and real-world datasets, adequately support the theoretical claims. Also noted is the comparison to other classification tree models, clearly differentiating the proposed scheme from prior state-of-the-art subspace learning methods. A core argument made is that the proposed method can perform better with fewer and more shallow trees than other popular, yet relatively simpler, classification tree methods.

Weaknesses:
(1) A single classifier type coupled with the linear transformation was described. It would be valuable if other classifier/transformation pairs were explored (e.g. SVNs). Does the framework easily extend to other classification techniques? 
(2) All dataset images used were scaled down to 16x16. How does this method perform on truly high-dimensional data? 
(3) Is there a reason that not all experiments were performed with each of the datasets?
(4) Timing and accuracy comparison results would be valuable if presented for additional datasets, thereby establishing the scalability properties of the algorithm; timing results aren't too informative when reported for a single dataset. 
(5) Experiments showing the effect of noise/mislabeld examples would be interesting as the impact of such imperfections on this scheme is unclear.
(6) Section 3.2, Paragraph 2, Sentence 1: What is the reason for mentioning that classes arriving at a node are randomly split into two categories? Why is it being introduced here as a new source of randomness?

Minor editorial comments:
 - Section 3.1, Paragraph 1, Sentence 1: Could be worded to flow better.
 - Section 3.1, Paragraph 2, Sentence 2: '... the paper ...' ->
   '...  this paper ...
 - Section 3.1, Paragraph 2, Sentence 4: Missing word (termination 'criteria'?)
 - Section 3.2, Paragraph 2, Sentence 2: Missing period.
 - Section 3.2, Paragraph 2, Sentence 3: '... the paper ...' ->
   '...  this paper ...
 - Section 3.2, Paragraph 2, Sentence 5: '... in details ...' ->
   ' ... in detail ...'
 - Section 3.2, Paragraph 2, Sentence 1: Not clear that its actually the
   samples from the dataset that the referenced figure contains.