Yes, you should pick flowers off your cucumber plants once a week to avoid a bitter-tasting cucumber. Flowers on cucumbers depend on the type of cucumber plant you have. If the flowers on cucumbers are not picked regularly, the fruits that develop will leave you with a nasty after taste and the vine will stop producing early.