Don't neglect them. Later on those fingers will be exercised individually with the stick. I suggest a set of warm up exercises designed to develop technique. One such exercise is a one hand exercise that develops stroke height, accent height and exercises the hand and wrist. In this exercise the student plays the following pattern in the right hand.
All large numbers are accents.
The accents are about 5-7 inches above the drum head. Small numbers are taps 2-5 inches above the drum head. Remember to play with a steady beat. There are many types of rudiments the percussionist must master. Probably the most important of these is the roll.