the direction of this sign in the function space that's essentially what Frank wolf algorithm will do but computing as Frank will sup is a exponential time so but instead if you had a ton of neurons and you add noise then there might be an exponentially small fraction of your neurons I see this Frank wolf direction but then great relu is homogeneous and then although it's exponentially small signal it goes up exponentially too and these things carefully balanced and at the end you do get a polynomial time result