lines of work and then kind of things come together in exciting ways so this is work from Noma shows here really just pushing on maybe parameter counting the bottleneck you know maybe that's what's been holding back language models and so they really went crazy here and they they train models that have these what they call sparse again a mixture of experts layers so you have your standard LS teams and pink on top and bottom of this model and then in the middle you sandwich in what's called its mixture of experts layer and