If you are taking olive leaf for general health there is no harm to take a break and there may even be a benefit giving your body a break from olive leaf. We feel summer is the best time to break because of the lower chances of cold or flu infection.