It depends on the type of flavoring that you are using, whether it is vanilla extract, vanilla syrup, or vanilla creamer. Since pure vanilla extract is not sweet, adding too much extract can make your coffee bitter. On the other hand, adding too much vanilla syrup or creamer will only make your coffee sweeter.