content: html: <tr><td>Model</td><td>Size</td><td>Type</td><td>H6 (Avg.)</td><td>ARC</td><td>HellaSwag</td><td>MMLU</td><td>TruthfulQA</td><td>Winogrande</td><td>GSM8K</td></tr><tr><td>SOLAR 10.7B-Instruct</td><td>~ 11B</td><td>Alignment-tuned</td><td>74.20</td><td>71.08</td><td>88.16</td><td>66.21</td><td>71.43</td><td>83.58</td><td>64.75</td></tr><tr><td>Qwen 72B</td><td>~ 72B</td><td>Pretrained</td><td>73.60</td><td>65.19</td><td>85.94</td><td>77.37</td><td>60.19</td><td>82.48</td><td>70.43</td></tr><tr><td>Mixtral 8x7B-Instruct-v0.1</td><td>~ 47B</td><td>Instruction-tuned</td><td>72.62</td><td>70.22</td><td>87.63</td><td>71.16</td><td>64.58</td><td>81.37</td><td>60.73</td></tr><tr><td>Yi 34B-200K</td><td>~ 34B</td><td>Pretrained</td><td>70.81</td><td>65.36</td><td>85.58</td><td>76.06</td><td>53.64</td><td>82.56</td><td>61.64</td></tr><tr><td>Yi 34B</td><td>~34B</td><td>Pretrained</td><td>69.42</td><td>64.59</td><td>85.69</td><td>76.35</td><td>56.23</td><td>83.03</td><td>50.64</td></tr><tr><td>Mixtral 8x7B-v0.1</td><td>~ 47B</td><td>Pretrained</td><td>68.42</td><td>66.04</td><td>86.49</td><td>71.82</td><td>46.78</td><td>81.93</td><td>57.47</td></tr><tr><td>Llama 2 70B</td><td>~ 70B</td><td>Pretrained</td><td>67.87</td><td>67.32</td><td>87.33</td><td>69.83</td><td>44.92</td><td>83.74</td><td>54.06</td></tr><tr><td>Falcon 180B</td><td>~ 180B</td><td>Pretrained</td><td>67.85</td><td>69.45</td><td>88.86</td><td>70.50</td><td>45.47</td><td>86.90</td><td>45.94</td></tr><tr><td>SOLAR 10.7B</td><td>~ 11B</td><td>Pretrained</td><td>66.04</td><td>61.95</td><td>84.60</td><td>65.48</td><td>45.04</td><td>83.66</td><td>55.50</td></tr><tr><td>Qwen 14B</td><td>~ 14B</td><td>Pretrained</td><td>65.86</td><td>58.28</td><td>83.99</td><td>67.70</td><td>49.43</td><td>76.80</td><td>58.98</td></tr><tr><td>Mistral 7B-Instruct-v0.2</td><td>~ 7B</td><td>Instruction-tuned</td><td>65.71</td><td>63.14</td><td>84.88</td><td>60.78</td><td>68.26</td><td>77.19</td><td>40.03</td></tr><tr><td>Yi 34B-Chat</td><td>~34B</td><td>Instruction-tuned</td><td>65.32</td><td>65.44</td><td>84.16</td><td>74.90</td><td>55.37</td><td>80.11</td><td>31.92</td></tr><tr><td>Mistral 7B</td><td>~ 7B</td><td>Pretrained</td><td>60.97</td><td>59.98</td><td>83.31</td><td>64.16</td><td>42.15</td><td>78.37</td><td>37.83</td></tr>

content: text: Table 2: Evaluation results for SOLAR 10.7B and SOLAR 10.7B-Instruct along with other top-performing models.
We report the scores for the six tasks mentioned in Sec. 4.1 along with the H6 score (average of six tasks). We also
report the size of the models in units of billions of parameters. The type indicates the training stage of the model
and is chosen from {Pretrained, Instruction-tuned, Alignment-tuned}. Models based on SOLAR 10.7B are colored
purple. The best scores for H6 and the individual tasks are shown in bold.

content: text: We reformatted the instruction datasets with an
Alpaca-styled chat template. For datasets such as
OpenOrca, which are derived from FLAN (Long-
pre et al., 2023), we filter data that overlaps with
the benchmark datasets (see Tab. 8 in Appendix. C
for more information). The alignment datasets are
in the {prompt, chosen, rejected} triplet format.
We preprocess the alignment datasets following
Zephyr (Tunstall et al., 2023).

content: text: Evaluation. In the HuggingFace Open LLM
Leaderboard (Beeching et al., 2023), six types of
evaluation methods are presented: ARC (Clark
et al., 2018), HellaSWAG (Zellers et al., 2019),
MMLU (Hendrycks et al., 2020), TruthfulQA (Lin
et al., 2022), Winogrande (Sakaguchi et al., 2021),
and GSM8K (Cobbe et al., 2021). We utilize these
datasets as benchmarks for evaluation and also re-
port the average scores for the six tasks, e.g., H6.

content: text: Model merging. Model merging methods such
as Yadav et al. (2023) can boost model perfor-
mance without further training. We merge some
of the models that we trained in both the instruc-
tion and alignment tuning stages. We implement
our own merging methods although popular open
source also exist such as MergeKit3.

content: text: 4.2 Main Results

content: text: We present evaluation results for our SOLAR
10.7B and SOLAR 10.7B-Instruct models along
with other top-performing models in Tab. 2. SO-
LAR 10.7B outperforms other pretrained models
of similar sizes, such as Qwen 14B and Mistral
7B, which shows that DUS is an effective method
to up-scale base LLMs. Furthermore, despite the

content: text: smaller size, SOLAR 10.7B-Instruct scores the
highest in terms of H6, even surpassing the recent
top-performing open-source LLM Mixtral 8×7B-
Instruct-v0.1 or Qwen 72B. The above results indi-
cate DUS can up-scale models that are capable of
achieving state-of-the-art performance when fine-
tuned. We also report data contamination results
for SOLAR 10.7B-Instruct in Appendix C.

content: text: 4.3 Ablation Studies

content: text: We present ablation studies for both the instruction
and alignment tuning stages.

content: text: 4.3.1 Instruction Tuning

content: text: Ablation on the training datasets. We present
ablation studies using different training datasets
for the instruction tuning in Tab. 3. The ablated
models are prefixed with SFT for supervised fine-
tuning. 'SFT v1' only uses the Alpaca-GPT4
dataset, whereas 'SFT v2' also uses the OpenOrca
dataset. 'SFT v3' uses the Synth. Math-Instruct
dataset along with the datasets used in 'SFT v2'.
Similarly, 'SFT v4' uses the Synth. Math-Instruct
dataset along with the datasets used in 'SFT v1'.

content: text: First, we analyze how Alpaca-GPT4 and
OpenOrca affect the trained models. The first ab-
lated model, 'SFT v1', which used only the Alpaca-
GPT4 dataset for training, resulted in 69.15 for H6.
When we add the OpenOrca dataset to train the
second ablated model, 'SFT v2', the resulting H6
score is 69.21, which is little change from 69.15 of
'SFT v1'. However, the task scores vary more as
'SFT v2' gets a substantially higher GSM8K score
of 57.32 compared to 52.24 of 'SFT v1' but also
gets noticeably lower scores across the board for
ARC, HellaS wag, and TruthfulQA. This seems to

content: text: 3https://github.com/cg123/mergekit