Feeding them sugar water now so they can make enough honey to live through the winter. You don't have bees. You have yellowjackets. They're a kind of hornet, not a kind of bee. The thought of killing bees distresses me.
Hornets are different.
I just got stung on the neck by a bee about 30 minutes ago while pumping gas into my truck. It's been 35 years or so since I've been stung. The only real bee pests are 'killer' bees. Sorry about the long spiel on bees. Bees actually tend to stay away from people, and do their pollenating business, unless they are provoked or under stressed conditions.