"
+ ],
+ "text/plain": [
+ " chinese \\\n",
+ "0 老耿端起枪,眯缝起一只三角眼,一搂扳机响了枪,冰雹般的金麻雀劈哩啪啦往下落,铁砂子在柳枝间飞... \n",
+ "1 次日天未明时,刘姥姥便起来梳洗了, 又将板儿教了几句话; 五六岁的孩子,听见带了他进城逛去,... \n",
+ "\n",
+ " english \\\n",
+ "0 Old Geng picked up his shotgun, squinted, and ... \n",
+ "1 Next day Grannie Liu was up before dawn. As so... \n",
+ "\n",
+ " Qwen/Qwen2-0.5B-Instruct \\\n",
+ "0 Old Ge lifted his gun and lowered his eyes as ... \n",
+ "1 The next day, when the sun was still very earl... \n",
+ "\n",
+ " Qwen/Qwen2-0.5B-Instruct_checkpoint-560 \\\n",
+ "0 Old Goong cocked his gun and lowered his eyes ... \n",
+ "1 It was still night when she rose to dress up; ... \n",
+ "\n",
+ " Qwen/Qwen2-0.5B-Instruct_checkpoint-1120 \\\n",
+ "0 Old Geng held his gun up, his eyes narrowed. T... \n",
+ "1 It was still dark before her eyes when she got... \n",
+ "\n",
+ " Qwen/Qwen2-0.5B-Instruct_checkpoint-1680 \\\n",
+ "0 Old Geng took his pistol from his holster and ... \n",
+ "1 When she awoke again from her nap, she was alr... \n",
+ "\n",
+ " Qwen/Qwen2-0.5B-Instruct_checkpoint-2240 \\\n",
+ "0 Old Geng raised his rifle, narrowed his eyes, ... \n",
+ "1 By dawn's hour again the next day, Grannie Liu... \n",
+ "\n",
+ " Qwen/Qwen2-0.5B-Instruct_checkpoint-2800 \\\n",
+ "0 Old Geng raised his rifle,眯着眼睛,the trigger cli... \n",
+ "1 By day's dawn she was out of bed, having alrea... \n",
+ "\n",
+ " Qwen/Qwen2-0.5B-Instruct_checkpoint-3360 \\\n",
+ "0 Old Geng held his gun at someone's head, his e... \n",
+ "1 At about three o'clock in the morning the next... \n",
+ "\n",
+ " Qwen/Qwen2-1.5B-Instruct \\\n",
+ "0 Old耿端起枪,眯缝起一只三角眼,一搂扳机响了枪,冰雹般的金麻雀劈哩啪啦往下落,铁砂子在柳枝... \n",
+ "1 The next morning, when it was still dark, Liu ... \n",
+ "\n",
+ " Qwen/Qwen2-1.5B-Instruct_checkpoint-560 \\\n",
+ "0 Old Geng took up his rifle and squinted throug... \n",
+ "1 Misty rose at dawn the next morning, having dr... \n",
+ "\n",
+ " Qwen/Qwen2-1.5B-Instruct_checkpoint-1120 \\\n",
+ "0 Old Geng raised his gun, squinted at it throug... \n",
+ "1 When it was just dawn outside her room, Granni... \n",
+ "\n",
+ " Qwen/Qwen2-1.5B-Instruct_checkpoint-1680 \\\n",
+ "0 Old Geng took a step forward with his gun rais... \n",
+ "1 By daybreak she got up, dressed herself, and w... \n",
+ "\n",
+ " Qwen/Qwen2-1.5B-Instruct_checkpoint-2240 \\\n",
+ "0 Old Geng took up his weapon with a squinted lo... \n",
+ "1 When she got up the next morning before daybre... \n",
+ "\n",
+ " Qwen/Qwen2-1.5B-Instruct_checkpoint-2800 \\\n",
+ "0 Old Geng took a step forward, raised his pisto... \n",
+ "1 When she arose from bed at midnight, she had a... \n",
+ "\n",
+ " Qwen/Qwen2-1.5B-Instruct_checkpoint-3360 \\\n",
+ "0 Old Geng took a step forward, lifted his pisto... \n",
+ "1 At daybreak the old woman woke up from her slu... \n",
+ "\n",
+ " unsloth/qwen2-7b-instruct-bnb-4bit \n",
+ "0 Old Geng raised his gun, squinted one of his t... \n",
+ "1 The next morning, before daybreak, Mrs. Liu ro... "
+ ]
+ },
+ "execution_count": 51,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "import pandas as pd\n",
+ "\n",
+ "df = pd.read_csv(\"results/mac-results_lf-r2.csv\")\n",
+ "df.head(2)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 52,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['chinese',\n",
+ " 'english',\n",
+ " 'Qwen/Qwen2-0.5B-Instruct',\n",
+ " 'Qwen/Qwen2-0.5B-Instruct_checkpoint-560',\n",
+ " 'Qwen/Qwen2-0.5B-Instruct_checkpoint-1120',\n",
+ " 'Qwen/Qwen2-0.5B-Instruct_checkpoint-1680',\n",
+ " 'Qwen/Qwen2-0.5B-Instruct_checkpoint-2240',\n",
+ " 'Qwen/Qwen2-0.5B-Instruct_checkpoint-2800',\n",
+ " 'Qwen/Qwen2-0.5B-Instruct_checkpoint-3360',\n",
+ " 'Qwen/Qwen2-1.5B-Instruct',\n",
+ " 'Qwen/Qwen2-1.5B-Instruct_checkpoint-560',\n",
+ " 'Qwen/Qwen2-1.5B-Instruct_checkpoint-1120',\n",
+ " 'Qwen/Qwen2-1.5B-Instruct_checkpoint-1680',\n",
+ " 'Qwen/Qwen2-1.5B-Instruct_checkpoint-2240',\n",
+ " 'Qwen/Qwen2-1.5B-Instruct_checkpoint-2800',\n",
+ " 'Qwen/Qwen2-1.5B-Instruct_checkpoint-3360',\n",
+ " 'unsloth/qwen2-7b-instruct-bnb-4bit']"
+ ]
+ },
+ "execution_count": 52,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "df.columns.to_list()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 46,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from eval_modules.calc_repetitions import *\n",
+ "\n",
+ "import re # Import the re module for regex operations\n",
+ "\n",
+ "def calc_metrics_for_epochs(df, start_col=2, end_col=-3, alpha=0.1):\n",
+ " dict = {\n",
+ " \"epoch\": [],\n",
+ " }\n",
+ "\n",
+ " columns = df.columns[start_col:end_col]\n",
+ " # print(columns)\n",
+ "\n",
+ " for col in columns:\n",
+ " # Split using regex\n",
+ " parts = re.split(r\"[/|_]\", col) # Use re.split() to split by regex\n",
+ " model_name = parts[1]\n",
+ " if len(parts) == 3:\n",
+ " checkpoint = parts[2]\n",
+ " epoch = int(checkpoint.split(\"-\")[1]) // 560\n",
+ " else:\n",
+ " epoch = 0\n",
+ " checkpoint = \"base\"\n",
+ "\n",
+ " dict[\"epoch\"].append(epoch)\n",
+ "\n",
+ " if model_name not in dict:\n",
+ " dict[model_name] = []\n",
+ " dict[model_name + \"(RAP)\"] = []\n",
+ "\n",
+ " metrics = calc_metrics(df[\"english\"], df[col])\n",
+ " dict[model_name].append(metrics[\"meteor\"])\n",
+ " print(\"*****\", model_name, checkpoint, epoch)\n",
+ "\n",
+ " df[[\"ews_score\", \"repetition_score\", \"total_repetitions\"]] = df[col].apply(\n",
+ " detect_scores\n",
+ " )\n",
+ " print(\"ews_score:\", df[\"ews_score\"].mean())\n",
+ " print(\"repetition_score:\", df[\"repetition_score\"].mean())\n",
+ " print(\"total_repetitions:\", df[\"total_repetitions\"].mean())\n",
+ "\n",
+ " # find the record with the highest total_repetitions\n",
+ " print(\"highest total_repetitions:\", df[\"total_repetitions\"].max())\n",
+ "\n",
+ " index = df[\"total_repetitions\"].idxmax()\n",
+ " print(\"\\t@\", index, \":\", df[col][index])\n",
+ "\n",
+ " rap = dict[model_name][-1] / math.log10(10 + alpha * df[\"total_repetitions\"].mean())\n",
+ " dict[model_name + \"(RAP)\"].append(rap)\n",
+ "\n",
+ " print(\"meteor:\", dict[model_name][-1])\n",
+ " print(\"rap:\", rap)\n",
+ "\n",
+ " dict[\"epoch\"] = dict[\"epoch\"][: len(dict[model_name])]\n",
+ " # print(dict)\n",
+ " return pd.DataFrame(dict)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 47,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "***** Qwen2-0.5B-Instruct base 0\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 8.962047661076788\n",
+ "total_repetitions: 8.962047661076788\n",
+ "highest total_repetitions: 10070\n",
+ "\t@ 1079 : Peter said that he would not have dared to disturb him if it was not for his birthday on the 2nd of January; because they had been looking for him in various places; so he is thin and long, with a thick and long stalk, and fresh, sweet, juicy, yellow, and white flowers; large, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big, big\n",
+ "meteor: 0.26081317536040693\n",
+ "rap: 0.25144066035383883\n",
+ "***** Qwen2-0.5B-Instruct checkpoint-560 1\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 15.847308031774052\n",
+ "total_repetitions: 15.847308031774052\n",
+ "highest total_repetitions: 17851\n",
+ "\t@ 327 : A little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter, a little longer, a little shorter,\n",
+ "meteor: 0.29049497377443884\n",
+ "rap: 0.2730508610847356\n",
+ "***** Qwen2-0.5B-Instruct checkpoint-1120 2\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 0.07413945278022947\n",
+ "total_repetitions: 0.07413945278022947\n",
+ "highest total_repetitions: 27\n",
+ "\t@ 193 : Have... No... Have... No...'\n",
+ "meteor: 0.31049607994218004\n",
+ "rap: 0.31039617450937557\n",
+ "***** Qwen2-0.5B-Instruct checkpoint-1680 3\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 0.21359223300970873\n",
+ "total_repetitions: 0.21359223300970873\n",
+ "highest total_repetitions: 67\n",
+ "\t@ 1129 : Old man asked, kneeling on the ground: 'Give me mercy, eight-grade Devil! Give me mercy, eight-grade Devil!'\n",
+ "meteor: 0.3229970135854747\n",
+ "rap: 0.322697991935926\n",
+ "***** Qwen2-0.5B-Instruct checkpoint-2240 4\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 0.13062665489849956\n",
+ "total_repetitions: 0.13062665489849956\n",
+ "highest total_repetitions: 51\n",
+ "\t@ 798 : Oh yes! What's that? What's that?' said Pocky Cheng to them when they came up to him. 'What's that? What's that?'\n",
+ "meteor: 0.3162958975982396\n",
+ "rap: 0.31611668025646267\n",
+ "***** Qwen2-0.5B-Instruct checkpoint-2800 5\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 12.927625772285966\n",
+ "total_repetitions: 12.927625772285966\n",
+ "highest total_repetitions: 14322\n",
+ "\t@ 160 : Boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom - boom\n",
+ "meteor: 0.31283536857877553\n",
+ "rap: 0.2971460010416401\n",
+ "***** Qwen2-0.5B-Instruct checkpoint-3360 6\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 7.321270962047661\n",
+ "total_repetitions: 7.321270962047661\n",
+ "highest total_repetitions: 8180\n",
+ "\t@ 809 : I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I'm . . . I\n",
+ "meteor: 0.31152790941615477\n",
+ "rap: 0.3022530311750537\n",
+ "***** Qwen2-1.5B-Instruct base 0\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 0.12533097969991175\n",
+ "total_repetitions: 0.12533097969991175\n",
+ "highest total_repetitions: 66\n",
+ "\t@ 327 : Short, short, short, long, long, short, short, long, long, short, short. This is 1108:21:37.\n",
+ "meteor: 0.31463018790549185\n",
+ "rap: 0.3144591332305182\n",
+ "***** Qwen2-1.5B-Instruct checkpoint-560 1\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 0.22241835834068843\n",
+ "total_repetitions: 0.22241835834068843\n",
+ "highest total_repetitions: 164\n",
+ "\t@ 327 : Shorts and longs, short and long, short and long, short and long, short and long, shorts and longs, short and long, short and long, short and long, short and long, short and long. This is 1108:21:37.\n",
+ "meteor: 0.3601984358327376\n",
+ "rap: 0.3598512232963998\n",
+ "***** Qwen2-1.5B-Instruct checkpoint-1120 2\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 0.17917034421888792\n",
+ "total_repetitions: 0.17917034421888792\n",
+ "highest total_repetitions: 48\n",
+ "\t@ 327 : Short and long, short and long, short and long, short and long. This is 1108:21:37.\n",
+ "meteor: 0.3715145486123948\n",
+ "rap: 0.37122594617880744\n",
+ "***** Qwen2-1.5B-Instruct checkpoint-1680 3\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 18.94792586054722\n",
+ "total_repetitions: 18.94792586054722\n",
+ "highest total_repetitions: 21397\n",
+ "\t@ 327 : Short long short long long short long short long long short long long long short long short long long long short long short long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long long short long short long long long long long short long short long long long long long long short long short long long long long long short long short long long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long short long short long long long long long short long short long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long short long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long short long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long\n",
+ "meteor: 0.3720444789500285\n",
+ "rap: 0.3459730328822243\n",
+ "***** Qwen2-1.5B-Instruct checkpoint-2240 4\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 0.205648720211827\n",
+ "total_repetitions: 0.205648720211827\n",
+ "highest total_repetitions: 106\n",
+ "\t@ 327 : Short: Long, Short: Long, Short: Short, Long: Long, Long: Short, Short: Long, Short: Short, Long: Short, Short: Long, Short: Short, Long: Short, This is 1108:21:37.\n",
+ "meteor: 0.36344098311977724\n",
+ "rap: 0.3631170086884642\n",
+ "***** Qwen2-1.5B-Instruct checkpoint-2800 5\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 0.08737864077669903\n",
+ "total_repetitions: 0.08737864077669903\n",
+ "highest total_repetitions: 29\n",
+ "\t@ 62 : He said that I used to talk like Yangzhou people before, perhaps . . . perhaps . . . he might know what happened to my eyes?\n",
+ "meteor: 0.3590942297170212\n",
+ "rap: 0.3589580715650848\n",
+ "***** Qwen2-1.5B-Instruct checkpoint-3360 6\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 0.30714916151809357\n",
+ "total_repetitions: 0.30714916151809357\n",
+ "highest total_repetitions: 101\n",
+ "\t@ 327 : SHANG, SHANG, SHANG, ZONGZONG, ZONGZONG, ZONGZONG, ZONGZONG, ZONGZONG, ZONGZONG, ZONGZONG, ZONGZONG, ZONGZONG, ZONGZONG is the sequence of 1108:21:37.\n",
+ "meteor: 0.3486547719340687\n",
+ "rap: 0.34819102067834695\n"
+ ]
+ },
+ {
+ "data": {
+ "text/html": [
+ "
"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "import matplotlib.pyplot as plt\n",
+ "from numpy import argmax\n",
+ "\n",
+ "# Assuming perf_df.set_index(\"epoch\").plot() generates a line plot\n",
+ "ax = perf_df.set_index(\"epoch\").plot(\n",
+ " figsize=(10, 5), title=\"METEOR score vs epoch\", grid=True\n",
+ ")\n",
+ "\n",
+ "# Loop through each line to annotate the last point\n",
+ "for line in ax.lines:\n",
+ " # Get the data\n",
+ " xdata, ydata = line.get_data()\n",
+ " for index in [0, 1, argmax(ydata), -1]:\n",
+ " ax.annotate(\n",
+ " f\"{ydata[index]:.3f}\",\n",
+ " xy=(xdata[index], ydata[index]),\n",
+ " textcoords=\"offset points\",\n",
+ " xytext=(0, 1),\n",
+ " ha=\"center\",\n",
+ " )\n",
+ "\n",
+ "plt.tight_layout()\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 53,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "***** qwen2-7b-instruct-bnb-4bit base 0\n",
+ "ews_score: 0.0\n",
+ "repetition_score: 0.2621359223300971\n",
+ "total_repetitions: 0.2621359223300971\n",
+ "highest total_repetitions: 162\n",
+ "\t@ 327 : The sequence you provided appears to be a series of patterns or codes rather than a straightforward sentence or phrase. Here's the translation:\n",
+ "\n",
+ "\"Short long long, short long long, short short short, long long short short, long long short long, short short long, short short short long, long short long short, long short short short.\"\n",
+ "\n",
+ "If this sequence is meant to convey something specific (like a code or pattern), the context would help in understanding its meaning. As it stands, this is the literal translation of the given Chinese text into English.\n",
+ "meteor: 0.37148866818883547\n",
+ "rap: 0.3710667829506641\n"
+ ]
+ },
+ {
+ "data": {
+ "text/html": [
+ "