The challenges with LLMs include the high cost of training, ethical concerns related to bias and misinformation, and issues with explainability and interpretability.