⚡ My PhD thesis, “Scalable Nested Optimization for Deep Learning,” is now on arXiv! ⚡
tl;dr: We develop various optimization tools with highlights, including: · Making the momentum coefficient complex for adversarial games like GANs. · Optimizing millions of hyperparameters using implicit differentiation. · Tuning hyperparameters using hypernetworks. · Differentiably finding bifurcations in optimization for diverse solutions.