Since we don't offer optimizers the ability to regularize different Weights
in the same WeightsSet differently, it did not seem worth the complication.
If we add some capability to differently regularize Weights (and I think we
should), I would be amenable to going back and adding explicit bias
Weights, but probably only in a parallel MultivariateWithBias or similarly
named set of examples/classifiers, for efficiency.


On Mon, Oct 14, 2013 at 5:20 PM, Andrew McCallum
<notifications@github.com>wrote:

> In src/main/scala/cc/factorie/app/classify/Classifier.scala:
>
> > +    for (f <- 0 until weightsValue.dim2; (l,t) <- (0 until numLabels).zip(weightTensor)) {
> > +      weightsValue(f,l) = t(f)
> > +    }
> > +    diagnostic(classifier)
> > +    classifier
> > +  }
> > +}
> > +
> > +/** Creates a trained naive Bayes classifier by counting feature occurrences, smoothed with pseudo-counts (m-Estimates).
> > +    Note that contrary to tradition, this naive Bayes classifier does not include a "bias" weight P(class); it only includes the feature weights, P(feature|class).
> > +    If you want a "bias" weight you must include in your data a feature that always has value 1.0. */
> > +class NaiveBayesClassifierTrainer(
> pseudoCount:Double = 0.1) extends LinearVectorClassifierTrainer {
> > +  def train[C<:LinearVectorClassifier[L,F],L<:LabeledDiscreteVar,F<:VectorVar](classifier:C, trainLabels:Iterable[L], l2f:L=>F): C = {
> > +    val labelSize = trainLabels.head.domain.size
> > +    val featureSize = l2f(trainLabels.head).domain.dimensionSize
> > +     // Collecting the bias weights afterall, in case we change our minds later, and store them in the classifier -akm
>
> Alex: So our policy is that if you want a "bias" factor then you have to
> add an always-on feature to your feature vector? I see the efficiency of
> this, but it seems sad to me.
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/factorie/factorie/pull/81/files#r6956103>
> .
>