There could be a variety of reasons why yellow jackets are flying around you. It is possible that they are attracted to your body heat, or they could be attracted to the carbon dioxide that you exhale. It is also possible that they are attracted to the food or drink that you are consuming.