Regularization is not uncommon as a hyperparameter, it's the learning rates that are "pesky".  As far as I can tell at a quick glance, this PR doesn't implement any of the "trendy" methods to set the learning rates schedule, right?