Quartering waves and wind make any boat difficult to hold on course and a rudder can provide just the right amount of course correction to ease the drudgery. The presence of a rudder does not suggest a poor design or a poor paddler - it is just a convenience. Adhering slavishly to a price limit is often false economy and a few extra dollars might buy a significantly better kayak. This is not a trivial matter. By definition a sea kayak should be seaworthy but don't let the deck and spray skirt fool you.
Seaworthiness is more than just keeping the water out.
The boat must be controllable in confused and large seas, fast enough to get you past dangerous shoals even when you have to paddle into the wind, and stable enough to keep you upright even when you are tired. Here is a good time to talk about the Eskimo roll. Experts differ as to the importance of the roll and whether a boat should be easy to roll or difficult to capsize. Nevertheless, thousands of miles of trouble free paddling have been logged in kayaks that roll poorly and the ability to roll is academic if one is right side up. Since kayaks that are easy to roll are also easy to capsize, that may be a good hint regarding the kind of boat you will want.