A fleece vest gives core insulation, without creating unwanted bulk around and under the arms. Regardless, err on the side of warmth if there is any doubt, especially if conditions are likely to deteriorate. At lunch stops, damp paddlers will often replace a base layer with a dry one, which helps bring skin temperatures back up a degree or two. A quality paddling jacket is worth the investment. Drysuits with latex seals are best protected by neoprene overcuffs.
Seals do wear out and can tear, as can the suit's waterproof material.
On the other hand, many paddlers find out, after spending the money, that the full neck seal of a drysuit or drysuit top is just too warm for our coastal climates. At a minimum, a 3mm, full farmer john wetsuit is recommended. They will age and lose insulation value if you subject them to years of butt-numbing mileage, but generally they give many, many years of satisfactory service. Custom fitted wetsuits are very expensive, but may be necessary if standard sizes just don't fit. You can also wear thin polypro under a wetsuit if your metabolism runs on the colder side.