Pre-Trained Language Models are models that have been previously trained on a large amount of data. They can be used as a starting point for specific tasks.