If it feels sluggish with you and your gear, it is too small even if there is room for twice as much gear. There is no law that says a kayak must be ugly to be inexpensive, durable, safe, or perform well. Sea kayaking is an aesthetic experience and you should never have to explain that the kayak on your BMW is not a bathroom fixture. This includes both tracking and maneuverability. The balance between the two depends upon how and where you paddle.
Exceptional maneuverability is less important in open than in confined waters.
This is also a good time to talk about rudders. A rudder can make an abominable kayak handle reasonably well but it should not be necessary. Well-designed kayaks track straight yet turn readily using just paddle strokes. Nevertheless, even the best hulls can profit from a rudder. Quartering waves and wind make any boat difficult to hold on course and a rudder can provide just the right amount of course correction to ease the drudgery.