Reeling it higher or faster will usually result in more blues than bass. Be alert on the pause as bass will often hit at that moment. Also prepare to go back in gear instantly if a hit is felt on the way down. Even when blues aren't a problem, it's generally a waste of time to reel all the way up before dropping down again. Unless fish are showing on the fishfinder at higher levels, restrict your retrieve to the bottom third of the water column and keep the diamond in the pay-off zone.
Most anglers employ a yo-yo motion off bottom, and sometimes that works even better.
Always be prepared to experiment, but my experience has been that the slow, short retrieve works best most of the time. Al Ristori has been writing national articles on saltwater fishing since his first for Salt Water Sportsman in 1965. Ristori has fished over much of the world, particularly in Latin America, and runs his charter boat Sheri Berri II from Point Pleasant, NJ for everything from stripers and blues up to tuna and sharks. This gives the offshore fisherman the choice of a gaff or harpoon while only taking up one space on the rack. This is very important to small boat fisherman where every bit of space matters.