pytorch-optimizer (pypi): optimizer & lr scheduler collections in PyTorch
Keywords: pytorch|deep-learning|optimizer|lr scheduler|A2Grad|ASGD|AccSGD|AdaBelief|AdaBound|AdaDelta|AdaFactor|AdaMax|AdaMod|AdaNorm|AdaPNM|AdaSmooth|AdaHessian|Adai|AdamP|AdamS|Adan|AggMo|AliG|Amos|Apollo|AvaGrad|DAdaptAdaGrad|DAdaptAdam|DAdaptAdan|DAdaptSGD|DiffGrad|Fromage|Gravity|LARS|Lamb|Lion|MADGRAD|MSVAG|Nero|NovoGrad|PID|PNM|QHAdam|QHM|RAdam|Ranger|Ranger21|SGDP|SGDW|SignSGD|SM3|SopihaH|SRMM|SWATS|ScalableShampoo|Shampoo|Yogi|SAM|GSAM|PCGrad|RotoGrad|adabelief|adabound|adai|adamd|adamp|adan|asam|chebyshev|diffgrad|gradient-centralization|gsam|lookahead|madgrad|nero|radam|ranger|sam