Certain birds, such as woodpeckers and swallows, can help naturally control carpenter bees by feeding on them. Encouraging these bird species by providing suitable habitats like nesting boxes can aid in managing carpenter bee numbers and getting rid of them.