Oolong tea has been known to have some side effects and hence should be avoided in some instances. An example of such is if you are extremely sensitive to caffeine. In such a scenario, you should avoid oolong tea altogether. Women who are pregnant or breastfeeding are also advised against too much intake of oolong tea.