Post
🔍 Today's pick in Interpretability & Analysis of LMs: Show Me How It's Done: The Role of Explanations in Fine-Tuning Language Models by M. Ballout
@krumnack
et al.
Authors propose a fine-tuning procedure in which a classification task is framed as generation and augmented with a natural language explanation to clarify intermediate reasoning steps. The procedure is applied to fine-tune language models of various sizes on the ListOps dataset, containing synthetically-generated instructions on sequences of numbers.
Authors find that explanations contribute to improving model performances across all tested model sizes and explanations lengths. Smaller language models appear to benefit the most from this approach in terms of convergence speed, performance and input length generalisation, especially when given more exhaustive explanations.
📄 Paper: Show Me How It's Done: The Role of Explanations in Fine-Tuning Language Models (2402.07543)
💻 Code: https://github.com/BalloutAI/Fine-tuning-with-Explanation
🔍 All daily picks in LM interpretability: gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9
Authors propose a fine-tuning procedure in which a classification task is framed as generation and augmented with a natural language explanation to clarify intermediate reasoning steps. The procedure is applied to fine-tune language models of various sizes on the ListOps dataset, containing synthetically-generated instructions on sequences of numbers.
Authors find that explanations contribute to improving model performances across all tested model sizes and explanations lengths. Smaller language models appear to benefit the most from this approach in terms of convergence speed, performance and input length generalisation, especially when given more exhaustive explanations.
📄 Paper: Show Me How It's Done: The Role of Explanations in Fine-Tuning Language Models (2402.07543)
💻 Code: https://github.com/BalloutAI/Fine-tuning-with-Explanation
🔍 All daily picks in LM interpretability: gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9