As long as you can move consistently for a couple of hours during the day, your fitness level should be fine. However, reserach the retreat you want to join for any fitness requirements beforehand.