LLMs are pre-trained on a large corpus of text data and then fine-tuned on a specific task. This allows them to leverage the knowledge learned during pre-training to perform well on a range of tasks.