You do not need a very high level of physical fitness to be able to swim.
All you need to do is train your muscles with some in-water stretching and warm-up exercises before you start your swim strokes. It is a very gentle and healing form of physiotherapy as well.