Better performance  by not calculating objective gradient many times for LabeledOutputLayers