While autoregressive pre-training is a significant factor that may contribute to the observed limitations in LLMs' self-discrimination capabilities, there are other factors to consider as well. Some of these factors include:
Prompt Design: The design of prompts used for discrimination tasks can significantly impact the model's ability to discriminate among generated alternatives. Poorly designed prompts may confuse the model and hinder its discrimination capabilities.
Model Architecture: The specific architecture of the language model, including the number of layers, attention mechanisms, and training objectives, can influence its ability to discriminate among generated outputs. Complex architectures may introduce noise or make it harder for the model to differentiate between responses.
Training Data Quality: The quality and diversity of the training data used to pre-train the language model can affect its discrimination capabilities. Biases or inconsistencies in the training data may lead to suboptimal performance in discriminating among generated responses.
Fine-Tuning Strategies: The fine-tuning process after pre-training plays a crucial role in shaping the model's behavior. Inadequate fine-tuning strategies or insufficient fine-tuning data may limit the model's ability to improve its self-discrimination capabilities.
Task Complexity: The complexity of the tasks on which the model is evaluated can also impact its discrimination abilities. Tasks that require nuanced reasoning or understanding may pose challenges for the model to accurately discriminate among generated responses.
Considering these factors alongside autoregressive pre-training can provide a more comprehensive understanding of the limitations in LLMs' self-discrimination capabilities.