At some point, it stops behaving elastically and starts to develop permanent, irreversible strain. This change in the mode of deformation occurs at the elastic limit. For rocks, only small strains are reversible. Once strain exceeds the elastic limit, ductile deformation occurs. Ductile deformation is not reversible, release of stress results only in the cessation of strain increase.
Rocks can experience enormous ductile strains if subjected to high stresses for long times.
And like a rubber band, rocks can fracture or snap, exhibiting brittle deformation when the stress and strain grow too large. Fracturing, like ductile deformation, is irreversible. The answer depends on the confining pressure and temperature the rock is held at, the rate at which stress is increased, and the composition of the rock itself. The effect of increasing confining pressure and temperature is to increase the ductility of rock. Rocks above the transition respond to stress like brittle solids.