Slower swing speeds can effectively use higher torque shafts, and faster speeds need lower torque. Some ladies can use up to five degrees of torque effectively. Better players need less torque because their swing speeds are generally higher and they have less trouble squaring the clubface. On the whole, however, the most important thing regarding torque is that it's consistent throughout a set of shafts. While that's often true for Tour players, who put a premium on accuracy, I think the average or recreational player should use the most flexible shaft they can hit straight.
Softer shafts are more forgiving and make getting the ball airborne easier.
Trying to hit shots with an overly stiff shaft isn't easy. Generally, I'd say the vast majority of weekend golfers should use softer shafts. They're gentler on the muscles and joints and tend to play a lot easier, especially if you don't practice much. Hit a variety of shafts, and see what gives you the best results. If a ladies or seniors shaft works best, then use it.