Taking supplements with milk can break fasting as it can stimulate insulin response and disrupt the metabolic state of fasting. It is recommended to take supplements with water or other non-caloric beverages during the fasting period.