With glasses, you still have a blurry image because the light is still scattered when it passes through the cone.
With a soft contact lens, the lens drapes over the cone, which still is an irregular shape, so we still have blurred vision.
Rigid gas permeable contact lenses sit on top of the cornea and create a normal surface for light to go through. It is difficult to fit these contact lenses with the irregular corneal shape without having the lenses pop out with regular blinking or having them stuck onto the corneal surface, which can cause serious complications.