While phentermine has not been shown to cause birth defects in studies, pregnant and breastfeeding women are advised against its use due to potential risks.