Note The original SystemChat dataset for SFT, including a system prompt, and ending with either the assistant's or user's response

Note A subset of `abacusai/SystemChat-1.1` but removing the last user turn in the instances where the last message was from the user, around 800 (as those were "Thank you...", "I'll try that ...", etc.)

Note A subset of `distilabel-internal-testing/SystemChat-1.1-Clean` with alternative generations using `Nexusflow/Starling-LM-7B-beta`, `Qwen/Qwen1.5-14B-Chat`, and `01-ai/Yi-34B-Chat`; while preserving the existing ones with `Smaug-2-72B`, `dolphin-2.7-mixtral-8x7b` and Mistral-Medium (no reference on which is each)

Note A subset on top of `distilabel-internal-testing/SystemChat-1.1-Generations` but cleaning the empty and/or None generations, and fixing parsing issues within the `01-ai/Yi-34B-Chat` generations; while also checking some samples manually to ensure that the dataset is ready to be provided to a Reward Model (RM) for ranking / rating

Note A subset on top of `distilabel-internal-testing/SystemChat-1.1-Generations-Clean` running PairRM to generate the rewards for each response from the assistant

Note A subset on top of `distilabel-internal-testing/SystemChat-1.1-Generations-Clean` running EurusRM to generate the rewards for each response from the assistant

Note Contains the preferences generated by `OpenBMB/Eurus-RM-7b` on top of the generations in `distilabel-internal-testing/SystemChat-1.1-Generations-Clean`, and formatted to follow the chosen-rejected format for preference fine-tuning