That's up to you, I guess. It's what Optimizer._fit_sequence does under the hood. This is just for a cleaner API IMO. You don't have to write the loop yourself and overriding fit to support this didn't make sense to me. There may be some room for more code sharing.