Start by injecting a puff of aerosol inside the hole. For a long-term solution, spray an insecticide around the outside of the nest. Then, seal the hole so bees don't come back to reuse it.