Note Ultimately used in my full eval submission, with exclusion of dolly_hhrlhf. Included only in Mistral-7B-sft-v1.

Note Used both for Mistral-7B-sft-v0 and Mistral-7B-sft-v1 in my submissions.

Note This turned out to be one of the holdout tasks. mosaicml/instruct-v3 includes training problems from competition_math. This is a contributor to the high scores on GSM8k and MATH benchmarks.

Note Looked promising, but did not have time to explore.

Note Considered for improving ICL. Did not have time to explore.

Note Decided against training on MMLU data.

Note Avoided due to CC BY-NC-SA license, though it would have been allowed for the competition. Likely would have been a good resource otherwise.

Note The plan here would be to target robustness metrics by finetuning an expert model to correct perturbations and/or clarify the input. This could have paraphrasing or other text revision tasks if they appeared in the hidden eval. Did not have time to fully explore.

Note Similar use case as coedit.

Note Similar use case as coedit.

Note Now knowing that the holdout tasks had ethics questions, I wish I had used this.

Note Same as social_i_qa

Note Now knowing that the holdout tasks had ethics questions, I wish I had used this.

Note Could have been useful for RAG.

Note The thought was this could help with CNN/DM summarization, but some quality and license concerns combined with acceptable performance without it led to its exclusion.