Adam and then they use this decreasing learning rate schedule this cosine learning rate schedule so there seem to be a lot of things a lot of hyper parameters that are fairly important here and you can see that the gains are substantial sometimes but they aren't like through the roof substantial where you can make a good argument that it is unclear how much really comes from this clever combination that fit fix match proposes and how much also just comes from whether or not you set the hyper parameters correctly and exactly how