Finally found the bug in the gradient computation - wrong sign for positive samples. Optimization now converges, and results for the single free-moving part are already impressive for binary classification. Need to get results for multi-class classification.