methods that kind of allowed us to train with much larger state sizes that we would factorize the weight matrices such that they would be represented kind of as this two layered system of having a dense sub dense block and a lot of sparse blocks that are pruned away and we saw that these were slightly efficient more efficient in terms of parameters and they also worked better on things like set analysis when evaluated by these linear models which is like a standard probing for how good of a feature representation have I