New invariances stub in NNs