First of all, check the trees in your yard whether they have any holes with ants, bees, or termites. If they are available, apply insecticide to the infested trees, making sure that you have used specifically made insecticide to kill only pests. When applying insecticide to them, keep your children and pets inside your house. After that, fill up the hole so that the insects die, and the others won't be able to enter into the hole. This way, you can prevent woodpeckers from returning.