Simply put, yes.
Hand washing is the single most effective way to prevent the spread of infections.
The use of gloves is not a substitute for hand washing. You can also spread microbes by being in contact with or sharing items like food utensils, straws, etc. There is additional information in OSH Answers about how the common cold is transmitted. Please see the OSH Answers document Universal Precautions for more complete information. Hands should be washed for a minimum of 10 seconds.