If those ants are actually carpenter ants, then they probably are making the tree their home and it could be causing structural integrity problems. Use an insecticide to abate the problem before it worsens or call your local tree service to diagnose and handle the problem for you. Call an arborist if the tree shows any signs of decline.