The return of AdaBoost.MH: multi-class Hamming trees
Authors: Balázs Kégl
ICLR 2014

Abstract: Within the framework of AdaBoost.MH, we propose to train vector-valued decision trees to optimize the multi-class edge without reducing the multi-class problem to $K$ binary one-against-all classifications. The key element of the method is a vector-valued decision stump, factorized into an input-independent vector of length $K$ and label-independent scalar classifier. At inner tree nodes, the label-dependent vector is discarded and the binary classifier can be used for partitioning the input space into two regions. The algorithm retains the conceptual elegance, power, and computational efficiency of binary AdaBoost. In experiments it is on par with support vector machines and with the best existing multi-class boosting algorithm AOSOLogitBoost, and it is significantly better than other known implementations of AdaBoost.MH.


Peer review:
[Review]:
The author presents a way to train vector-valued decision trees (Hamming trees) in the context of multi-class AdaBoost.MH, as one way of not doing K binary one-vs-all classifications. The idea is a clever factorization that allows for efficient modeling and predictions.  The paper seems to be performing very thorough validation, comparisons and hyper-parameter selection, which is obviously a good thing.

Unfortunately, I cannot speak with any authority as to whether the algorithmic contributions of this paper are substantial or not. While the results do seem to support the idea that the proposed method, AdaBoost.MH + Hamming trees, works as well or better than SVMs/other AdaBoost methods, I cannot evaluate in earnestness whether this is a very novel contribution or incremental, since recent developments in boosting are not exactly my area of expertise.