that that is kind of not the shape of the true data manifold the input manifolds as you can see here the input manifold here isn't linear or flat it's actually very very tangled and we know that neural networks as you continue in the layers will flatten those representations because ultimately at the end it needs to classify the dataset linearly because the last layer is a softmax layer but the the idea that you could apply this to any layer seems a bit shady to me of course it works and they show it works and it's really nice