Vapid Shallow Models Must Die!