Now have a "satisfying" SGD. It is clear now that it **will** require grid search + CV to find out the best learning rate, and possibly batch size for a fixed number of iterations (which should be chosen to keep the running time acceptable).