By extending discriminative and large margin concepts to a probabilistic setting, MED permits many important generalizations to SVMs. We introduce feature selection as a particularly critical augmentation of the learning machine. MED derivations for both regression and classification cases are shown and lead to promising experimental results. Features are pruned simultaneously with parameter estimation to generate substantial improvements with relatively sparse training data. Furthermore, in the linear model case, complexity scales linearly with dimensionality and can remain tractable under explicit feature expansions of non-linear kernels.
Structured labels allow data examples to have unobserved degrees of freedom or invariants.
We discuss this class of problems and their solutions in a discriminative setting. Tony Jebara and Tommi Jaakkola. We present a general framework for discriminative estimation based on the maximum entropy principle and its extensions. This holds even when the data is not separable within the chosen parametric class, in the context of anomaly detection rather than classification, or when the labels in the training set are uncertain or incomplete. Support vector machines are naturally subsumed under this class and we provide several extensions.