indiejoseph commited on
Commit
427ee7a
1 Parent(s): 458f2e0

Upload folder using huggingface_hub

Browse files
checkpoint-400/added_tokens.json ADDED
@@ -0,0 +1,3232 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "一來": 65791,
3
+ "一個人": 64042,
4
+ "一個市鎮": 64500,
5
+ "一個月": 64924,
6
+ "一動": 65385,
7
+ "一口氣": 64619,
8
+ "一啲": 64281,
9
+ "一場": 64387,
10
+ "一塊": 65225,
11
+ "一定係": 66125,
12
+ "一定會": 64236,
13
+ "一對": 65053,
14
+ "一層": 66082,
15
+ "一帶": 65128,
16
+ "一張": 64325,
17
+ "一拍": 66247,
18
+ "一拳": 65559,
19
+ "一擊": 65761,
20
+ "一旁": 64585,
21
+ "一早": 64707,
22
+ "一時": 64319,
23
+ "一時間": 66181,
24
+ "一晚": 66142,
25
+ "一會": 64328,
26
+ "一會兒": 65611,
27
+ "一望": 64573,
28
+ "一條": 64127,
29
+ "一樣的": 64930,
30
+ "一段時間": 65926,
31
+ "一班": 65195,
32
+ "一直都": 65634,
33
+ "一眾": 65346,
34
+ "一秒": 65829,
35
+ "一絲": 64750,
36
+ "一聲": 64089,
37
+ "一腳": 65636,
38
+ "一臉": 64163,
39
+ "一覽": 64116,
40
+ "一輪": 66263,
41
+ "一邊": 64024,
42
+ "一閃": 65961,
43
+ "一開始": 64767,
44
+ "一間": 64203,
45
+ "一陣": 64065,
46
+ "一隻": 64148,
47
+ "一雙": 65931,
48
+ "一頭": 65229,
49
+ "一齊": 64035,
50
+ "三個": 64096,
51
+ "上來": 64873,
52
+ "上嘅": 64569,
53
+ "上嚟": 64714,
54
+ "上堂": 66079,
55
+ "上學": 65739,
56
+ "上網": 65782,
57
+ "上課": 65423,
58
+ "上車": 65833,
59
+ "下一個": 65790,
60
+ "下意識": 65978,
61
+ "下頭": 65524,
62
+ "不動": 65093,
63
+ "不喜歡": 66213,
64
+ "不單": 65711,
65
+ "不夠": 65021,
66
+ "不對": 64840,
67
+ "不時": 64876,
68
+ "不滿": 64595,
69
+ "不知不覺": 65258,
70
+ "不自覺": 65620,
71
+ "不見": 64233,
72
+ "不覺": 64781,
73
+ "不說": 64934,
74
+ "不論": 64940,
75
+ "不過我": 66151,
76
+ "不過是": 66196,
77
+ "不錯": 64401,
78
+ "丐": 67017,
79
+ "丟": 66539,
80
+ "並不": 64268,
81
+ "並不是": 64408,
82
+ "並沒有": 64183,
83
+ "並無": 65309,
84
+ "並非": 64529,
85
+ "中嘅": 65233,
86
+ "中學": 64186,
87
+ "中意": 65062,
88
+ "中環": 65699,
89
+ "中間": 64332,
90
+ "主動": 64279,
91
+ "主教": 65677,
92
+ "之內": 65081,
93
+ "之後就": 64829,
94
+ "之時": 64746,
95
+ "之處": 65002,
96
+ "之間嘅": 65238,
97
+ "之間的": 64970,
98
+ "之際": 64405,
99
+ "之類": 64683,
100
+ "之餘": 65705,
101
+ "乜": 66318,
102
+ "乜嘢": 64314,
103
+ "九龍": 64382,
104
+ "也不會": 64641,
105
+ "也沒": 64302,
106
+ "也沒有": 64098,
107
+ "也許": 64269,
108
+ "乸": 67135,
109
+ "乾淨": 65573,
110
+ "了一個": 64124,
111
+ "了一口": 65612,
112
+ "了一口氣": 64715,
113
+ "了一會": 64604,
114
+ "了一聲": 64820,
115
+ "了一點": 65698,
116
+ "了下來": 64586,
117
+ "了個": 64360,
118
+ "了兩": 65060,
119
+ "了出來": 65005,
120
+ "了口氣": 65416,
121
+ "了嗎": 64231,
122
+ "了幾": 64435,
123
+ "了起來": 64245,
124
+ "事務": 64695,
125
+ "事實上": 65030,
126
+ "事後": 66201,
127
+ "五個": 65963,
128
+ "些什麼": 66140,
129
+ "交換": 65151,
130
+ "交給": 65256,
131
+ "亦係": 64918,
132
+ "亦會": 65132,
133
+ "亦都": 64445,
134
+ "人哋": 64357,
135
+ "人嘅": 64283,
136
+ "人手": 66021,
137
+ "人數": 64938,
138
+ "人會": 65946,
139
+ "人民共和國": 65857,
140
+ "人民幣": 65489,
141
+ "人間": 65304,
142
+ "什麼事": 64797,
143
+ "仆街": 64380,
144
+ "今次": 64077,
145
+ "介意": 64696,
146
+ "仔細": 64680,
147
+ "他便": 66118,
148
+ "他們的": 64118,
149
+ "他已經": 66274,
150
+ "他會": 65659,
151
+ "代價": 66123,
152
+ "令他": 65400,
153
+ "令佢": 66231,
154
+ "令到": 64531,
155
+ "以後": 64133,
156
+ "以為自己": 65890,
157
+ "仲係": 64648,
158
+ "仲可以": 66117,
159
+ "仲有": 64067,
160
+ "仲未": 64811,
161
+ "仲要": 64164,
162
+ "任永": 65008,
163
+ "伏熙": 65972,
164
+ "估唔到": 64568,
165
+ "估計": 64724,
166
+ "伶": 66865,
167
+ "似是": 65262,
168
+ "但佢": 65593,
169
+ "但係": 64027,
170
+ "但其實": 65807,
171
+ "但卻": 65354,
172
+ "但這": 65679,
173
+ "位置喺": 64426,
174
+ "低聲": 65421,
175
+ "低頭": 64522,
176
+ "住佢": 64639,
177
+ "住我": 64240,
178
+ "何時": 65169,
179
+ "何況": 65007,
180
+ "作戰": 65268,
181
+ "你仲": 65945,
182
+ "你係": 64278,
183
+ "你係咪": 65267,
184
+ "你個": 64392,
185
+ "你們的": 65455,
186
+ "你先": 65505,
187
+ "你又": 64406,
188
+ "你同": 65710,
189
+ "你呀": 64908,
190
+ "你呢": 65207,
191
+ "你咁": 65187,
192
+ "你哋": 64018,
193
+ "你唔": 64374,
194
+ "你唔係": 65544,
195
+ "你唔好": 64862,
196
+ "你啊": 65938,
197
+ "你啦": 65227,
198
+ "你嘅": 64299,
199
+ "你應該": 65953,
200
+ "你會": 64234,
201
+ "你真係": 65280,
202
+ "你睇": 66258,
203
+ "你知": 64600,
204
+ "你覺得": 65129,
205
+ "你話": 64881,
206
+ "你說": 64353,
207
+ "你講": 64620,
208
+ "你這": 65103,
209
+ "你還": 65246,
210
+ "你都": 64375,
211
+ "佢": 66301,
212
+ "佢係": 64169,
213
+ "佢個": 64480,
214
+ "佢又": 65470,
215
+ "佢同": 65259,
216
+ "佢哋": 64005,
217
+ "佢哋嘅": 64863,
218
+ "佢唔": 65924,
219
+ "佢喺": 64574,
220
+ "佢嘅": 64061,
221
+ "佢地": 65321,
222
+ "佢好": 65412,
223
+ "佢就": 64457,
224
+ "佢會": 65293,
225
+ "佢有": 65721,
226
+ "佢話": 64812,
227
+ "佢都": 65163,
228
+ "佳士": 65473,
229
+ "併": 66620,
230
+ "使出": 65301,
231
+ "使唔使": 66290,
232
+ "來不及": 66297,
233
+ "來了": 64201,
234
+ "來到": 64120,
235
+ "來到了": 65380,
236
+ "來得": 65916,
237
+ "來源": 64028,
238
+ "供應": 65475,
239
+ "依個": 65305,
240
+ "依家": 64045,
241
+ "依舊": 64577,
242
+ "侶": 66491,
243
+ "便會": 64722,
244
+ "係一": 64988,
245
+ "係一個": 64307,
246
+ "係一種": 66204,
247
+ "係中國": 64870,
248
+ "係你": 64572,
249
+ "係佢": 64617,
250
+ "係個": 65247,
251
+ "係呀": 64464,
252
+ "係呢": 65236,
253
+ "係咁": 65210,
254
+ "係咩": 64530,
255
+ "係咪": 64054,
256
+ "係唔係": 65182,
257
+ "係啊": 65477,
258
+ "係喎": 65874,
259
+ "係喺": 65731,
260
+ "係噉": 64503,
261
+ "係因為": 64891,
262
+ "係大": 66014,
263
+ "係好": 64844,
264
+ "係度": 64383,
265
+ "係我": 64094,
266
+ "係指": 64890,
267
+ "係日本": 64739,
268
+ "係有": 64793,
269
+ "係法國嘅": 64498,
270
+ "係法國嘅一個市鎮": 64502,
271
+ "係由": 65461,
272
+ "係要": 65868,
273
+ "係香港": 64237,
274
+ "係點": 65898,
275
+ "俄羅斯": 64606,
276
+ "俐": 67092,
277
+ "保證": 65592,
278
+ "俠": 66484,
279
+ "修煉": 65575,
280
+ "俾": 66323,
281
+ "俾人": 64304,
282
+ "倆": 66477,
283
+ "倉": 66530,
284
+ "個仔": 65707,
285
+ "個個": 64723,
286
+ "個名": 65484,
287
+ "個女": 66064,
288
+ "個案": 65297,
289
+ "個樣": 65057,
290
+ "個頭": 65167,
291
+ "倏": 67102,
292
+ "倒在地上": 65864,
293
+ "倖": 66761,
294
+ "倫敦": 65571,
295
+ "假設": 66244,
296
+ "偈": 66646,
297
+ "做乜": 64875,
298
+ "做什麼": 66293,
299
+ "做咗": 64807,
300
+ "做咩": 64112,
301
+ "做咩呀": 66169,
302
+ "做嘢": 64914,
303
+ "做過": 64717,
304
+ "停下來": 65561,
305
+ "停車": 66174,
306
+ "側": 66381,
307
+ "偵": 66614,
308
+ "偶爾": 65958,
309
+ "偽": 66582,
310
+ "傀": 67137,
311
+ "傘": 66629,
312
+ "傢": 66458,
313
+ "傢伙": 64388,
314
+ "傭": 66739,
315
+ "傳來": 64147,
316
+ "傳出": 65012,
317
+ "傳媒": 64616,
318
+ "傳說": 64517,
319
+ "傳送": 65193,
320
+ "傷口": 64523,
321
+ "傷害": 64286,
322
+ "傷心": 65122,
323
+ "傾": 66333,
324
+ "傾偈": 65556,
325
+ "僑": 66921,
326
+ "僕": 66708,
327
+ "僱": 66709,
328
+ "價錢": 65672,
329
+ "儀式": 65619,
330
+ "儘管": 64796,
331
+ "償": 66674,
332
+ "儡": 67123,
333
+ "優先": 64799,
334
+ "優勢": 65242,
335
+ "優雅": 66275,
336
+ "儲": 66482,
337
+ "兀": 67124,
338
+ "元朗": 65386,
339
+ "兇": 66466,
340
+ "兇手": 65894,
341
+ "先可以": 65613,
342
+ "先啦": 64584,
343
+ "先後": 66031,
344
+ "先會": 65846,
345
+ "先至": 64782,
346
+ "克蘭": 65254,
347
+ "兌": 66736,
348
+ "免費": 64973,
349
+ "兒子": 64545,
350
+ "兒的": 66202,
351
+ "入到": 65275,
352
+ "入去": 64338,
353
+ "入咗": 65150,
354
+ "入嚟": 64760,
355
+ "入面": 64029,
356
+ "入面嘅": 64931,
357
+ "內地": 64300,
358
+ "內心": 64219,
359
+ "內的": 64289,
360
+ "內部": 65156,
361
+ "兩人": 64059,
362
+ "兩位": 64352,
363
+ "兩個人": 64478,
364
+ "兩個月": 66175,
365
+ "兩名": 64740,
366
+ "兩天": 66241,
367
+ "兩年": 64866,
368
+ "兩日": 65523,
369
+ "兩條": 65632,
370
+ "兩次": 65510,
371
+ "兩者": 65813,
372
+ "兩邊": 65378,
373
+ "兩隻": 65117,
374
+ "公仔": 65035,
375
+ "公屋": 65895,
376
+ "公會": 65241,
377
+ "公眾": 64937,
378
+ "共和國": 64953,
379
+ "共產": 65431,
380
+ "共產黨": 65789,
381
+ "其中一": 65310,
382
+ "其中一個": 64363,
383
+ "其實係": 66107,
384
+ "其實我": 65219,
385
+ "其後": 64831,
386
+ "其餘": 64613,
387
+ "典禮": 66232,
388
+ "冇": 66312,
389
+ "冇事": 65177,
390
+ "冇人": 65200,
391
+ "冇咩": 66224,
392
+ "冇嘢": 65955,
393
+ "冇得": 66210,
394
+ "再見": 64681,
395
+ "再說": 64920,
396
+ "冒險": 64902,
397
+ "冚": 66892,
398
+ "冠軍": 64367,
399
+ "冧": 66889,
400
+ "冰冷": 65743,
401
+ "冷冷": 65075,
402
+ "冷笑": 65668,
403
+ "冷靜": 64276,
404
+ "凈": 67167,
405
+ "凍": 66495,
406
+ "凜": 66707,
407
+ "凝視": 65502,
408
+ "凱": 66353,
409
+ "出一個": 64916,
410
+ "出世": 64440,
411
+ "出世嘅人": 65795,
412
+ "出事": 66060,
413
+ "出來的": 64485,
414
+ "出咗": 64731,
415
+ "出嘅": 65968,
416
+ "出嚟": 64034,
417
+ "出嚟嘅": 66242,
418
+ "出現了": 65095,
419
+ "出現在": 64549,
420
+ "出發": 64399,
421
+ "出聲": 64771,
422
+ "出街": 65389,
423
+ "出門": 66026,
424
+ "出面": 64113,
425
+ "分為": 65776,
426
+ "分開": 64861,
427
+ "列車": 65048,
428
+ "判斷": 65657,
429
+ "別人的": 65658,
430
+ "別的": 65631,
431
+ "利亞": 64671,
432
+ "刪": 66780,
433
+ "到佢": 64997,
434
+ "到來": 65759,
435
+ "到去": 66272,
436
+ "到咗": 64867,
437
+ "到時": 64688,
438
+ "到最後": 66088,
439
+ "到處": 66065,
440
+ "到達": 64322,
441
+ "剌": 66905,
442
+ "前來": 65702,
443
+ "前輩": 65230,
444
+ "前進": 64823,
445
+ "剎": 66560,
446
+ "剎那": 65744,
447
+ "剛剛": 64159,
448
+ "剛好": 64588,
449
+ "剛才": 64040,
450
+ "剛才的": 65600,
451
+ "剝": 66841,
452
+ "創意": 66255,
453
+ "創辦": 66138,
454
+ "剷": 67108,
455
+ "剽": 67109,
456
+ "劇情": 64894,
457
+ "劇照": 65376,
458
+ "劇集": 65196,
459
+ "劍": 66316,
460
+ "劏": 66941,
461
+ "力氣": 65528,
462
+ "功課": 64776,
463
+ "加埋": 66003,
464
+ "勁": 66340,
465
+ "勇氣": 64765,
466
+ "勉強": 64603,
467
+ "動力": 65351,
468
+ "動畫": 65176,
469
+ "勝利": 65041,
470
+ "勞工": 66282,
471
+ "勢力": 64969,
472
+ "勳": 66928,
473
+ "勸": 66602,
474
+ "勻": 67118,
475
+ "包圍": 64828,
476
+ "包裝": 66179,
477
+ "匕": 66797,
478
+ "化妝": 65999,
479
+ "化學": 65496,
480
+ "化成": 65387,
481
+ "化為": 66146,
482
+ "匡": 66802,
483
+ "匣": 67175,
484
+ "區嘅": 65591,
485
+ "區議會": 65439,
486
+ "十分鐘": 65001,
487
+ "十多": 65369,
488
+ "十幾": 65901,
489
+ "千萬": 65031,
490
+ "升幅": 65597,
491
+ "升降機": 65067,
492
+ "半個": 65251,
493
+ "半島": 66164,
494
+ "半空": 65627,
495
+ "半身": 66144,
496
+ "半點": 65564,
497
+ "協議": 66090,
498
+ "博物館": 65885,
499
+ "危機": 64955,
500
+ "危險": 64168,
501
+ "即係": 64073,
502
+ "即刻": 64154,
503
+ "即將": 64989,
504
+ "即時": 64581,
505
+ "卻又": 65482,
506
+ "卻是": 64189,
507
+ "卻沒有": 65047,
508
+ "卻被": 66091,
509
+ "厄臨": 65988,
510
+ "原來": 64012,
511
+ "原來是": 65350,
512
+ "原則": 65635,
513
+ "原諒": 65460,
514
+ "厭": 66410,
515
+ "厲": 66406,
516
+ "厲害": 64539,
517
+ "去到": 64115,
518
+ "去咗": 64582,
519
+ "參考": 64125,
520
+ "參賽": 65440,
521
+ "參選": 65983,
522
+ "又係": 64326,
523
+ "又再": 64607,
524
+ "又叫": 65154,
525
+ "又唔": 65143,
526
+ "又唔係": 66183,
527
+ "又好": 65832,
528
+ "又或者": 65045,
529
+ "又指": 66260,
530
+ "又會": 64795,
531
+ "又說": 65110,
532
+ "及時": 65775,
533
+ "反對": 64512,
534
+ "反應": 64056,
535
+ "反擊": 65724,
536
+ "反駁": 65503,
537
+ "受傷": 64340,
538
+ "受歡迎": 65578,
539
+ "叢": 66692,
540
+ "口氣": 64095,
541
+ "古怪": 65735,
542
+ "句話": 64311,
543
+ "另一個": 64208,
544
+ "另一邊": 64743,
545
+ "叩": 66963,
546
+ "只不過": 64250,
547
+ "只係": 64038,
548
+ "只可以": 66163,
549
+ "只會": 64264,
550
+ "只有一": 65239,
551
+ "只見": 64101,
552
+ "叫你": 65390,
553
+ "叫佢": 65424,
554
+ "叫聲": 64764,
555
+ "叫道": 65960,
556
+ "召喚": 65736,
557
+ "可以係": 65353,
558
+ "可唔可以": 64704,
559
+ "可惡": 65873,
560
+ "可愛": 64272,
561
+ "可憐": 64848,
562
+ "可能係": 65966,
563
+ "可能會": 64400,
564
+ "可見": 64917,
565
+ "叱": 67071,
566
+ "右邊": 65754,
567
+ "司機": 64555,
568
+ "叻": 66662,
569
+ "吃飯": 64753,
570
+ "合照": 65952,
571
+ "合約": 66248,
572
+ "吋": 66869,
573
+ "同一個": 65042,
574
+ "同一時間": 66298,
575
+ "同人": 66178,
576
+ "同你": 64184,
577
+ "同你講": 65787,
578
+ "同佢": 64126,
579
+ "同佢講": 65886,
580
+ "同埋": 64032,
581
+ "同我": 64070,
582
+ "同我講": 64407,
583
+ "同阿": 64716,
584
+ "名單": 65260,
585
+ "名為": 64936,
586
+ "向她": 65900,
587
+ "向後": 65469,
588
+ "向著": 65407,
589
+ "吒": 66588,
590
+ "吔": 67160,
591
+ "吖": 66368,
592
+ "吖嘛": 65949,
593
+ "吠": 67219,
594
+ "否則": 64454,
595
+ "否認": 65554,
596
+ "吩": 66801,
597
+ "吱": 66638,
598
+ "吶": 66925,
599
+ "呀呀": 64711,
600
+ "呀嘛": 66226,
601
+ "呂": 66476,
602
+ "呃": 66404,
603
+ "呆呆": 66236,
604
+ "呈現": 66018,
605
+ "告訴我": 64888,
606
+ "呎": 66791,
607
+ "呢件事": 65300,
608
+ "呢位": 64922,
609
+ "呢個": 64003,
610
+ "呢個係": 65753,
611
+ "呢個時候": 65328,
612
+ "呢句": 65680,
613
+ "呢啲": 64049,
614
+ "呢度": 64102,
615
+ "呢條": 64783,
616
+ "呢次": 65449,
617
+ "呢班": 65994,
618
+ "呢種": 64417,
619
+ "呢間": 65534,
620
+ "呢隻": 65432,
621
+ "周圍": 64327,
622
+ "呯": 66776,
623
+ "呱": 67224,
624
+ "呷": 67024,
625
+ "呻": 66832,
626
+ "呼籲": 65766,
627
+ "命運": 64774,
628
+ "咁": 66308,
629
+ "咁啱": 65687,
630
+ "咁多": 64129,
631
+ "咁大": 64777,
632
+ "咁好": 65401,
633
+ "咁快": 65327,
634
+ "咁樣": 65278,
635
+ "咁耐": 65799,
636
+ "咆": 66835,
637
+ "咇": 67157,
638
+ "和阿": 65722,
639
+ "咔": 66729,
640
+ "咗": 66302,
641
+ "咗一": 64247,
642
+ "咗一個": 65887,
643
+ "咗你": 65434,
644
+ "咗佢": 64330,
645
+ "咗個": 64239,
646
+ "咗去": 65499,
647
+ "咗右": 65713,
648
+ "咗啲": 66129,
649
+ "咗喺": 65808,
650
+ "咗嘅": 65917,
651
+ "咗好多": 64893,
652
+ "咗幾": 66047,
653
+ "咗我": 64390,
654
+ "咦": 66435,
655
+ "咩": 66309,
656
+ "咩事": 64220,
657
+ "咩呀": 65307,
658
+ "咭": 67038,
659
+ "哂": 66395,
660
+ "哋": 66303,
661
+ "哧": 67198,
662
+ "哪裏": 64429,
663
+ "哽": 67055,
664
+ "唏": 67002,
665
+ "唔": 66300,
666
+ "唔住": 64202,
667
+ "唔使": 64068,
668
+ "唔係": 64001,
669
+ "唔再": 65728,
670
+ "唔到": 64026,
671
+ "唔可以": 64140,
672
+ "唔同": 64104,
673
+ "唔同嘅": 65445,
674
+ "唔多": 64335,
675
+ "唔夠": 64794,
676
+ "唔好": 64008,
677
+ "唔好再": 65696,
678
+ "唔好意思": 64996,
679
+ "唔少": 64261,
680
+ "唔得": 64404,
681
+ "唔想": 64271,
682
+ "唔敢": 66171,
683
+ "唔明": 65845,
684
+ "唔會": 64021,
685
+ "唔理": 66045,
686
+ "唔知": 64030,
687
+ "唔知點解": 65422,
688
+ "唔緊要": 65546,
689
+ "唔見": 66024,
690
+ "唔記得": 64758,
691
+ "唔該": 64596,
692
+ "唔講": 65863,
693
+ "唔識": 64679,
694
+ "唔通": 64623,
695
+ "唔錯": 66278,
696
+ "唔開心": 65548,
697
+ "唷": 67129,
698
+ "唸": 66651,
699
+ "商場": 64690,
700
+ "啊啊啊啊": 66217,
701
+ "問下": 66194,
702
+ "問你": 66008,
703
+ "問佢": 66099,
704
+ "問到": 66206,
705
+ "問我": 64266,
706
+ "問道": 64069,
707
+ "啓": 67158,
708
+ "啖": 66652,
709
+ "啜": 67014,
710
+ "啞": 66666,
711
+ "啟動": 65691,
712
+ "啦啦": 64883,
713
+ "啩": 67120,
714
+ "啪啪": 65538,
715
+ "啫": 66327,
716
+ "啫啫": 65827,
717
+ "啱": 66321,
718
+ "啱啱": 64200,
719
+ "啲": 66307,
720
+ "啲人": 64730,
721
+ "啲咩": 64489,
722
+ "啲嘢": 64662,
723
+ "啵": 67000,
724
+ "喂喂": 65831,
725
+ "喃喃": 65633,
726
+ "善容": 65914,
727
+ "喉嚨": 65765,
728
+ "喊道": 66077,
729
+ "喎": 66319,
730
+ "喔": 66432,
731
+ "喚": 66462,
732
+ "喪": 66344,
733
+ "喪屍": 64193,
734
+ "喬": 66397,
735
+ "單純": 65396,
736
+ "單車": 64817,
737
+ "喱": 66715,
738
+ "喲": 67003,
739
+ "喳": 67068,
740
+ "喺": 66304,
741
+ "喺佢": 65674,
742
+ "喺度": 64132,
743
+ "喺我": 64899,
744
+ "喺香港": 64778,
745
+ "嗄": 66714,
746
+ "嗌": 66565,
747
+ "嗖": 67162,
748
+ "嗚": 66391,
749
+ "嗚嗚": 65828,
750
+ "嗡": 66857,
751
+ "嗤": 67117,
752
+ "嗦": 67227,
753
+ "嗮": 66838,
754
+ "嗰": 66313,
755
+ "嗰個": 64152,
756
+ "嗰啲": 64515,
757
+ "嗰度": 64578,
758
+ "嗰日": 65379,
759
+ "嗰時": 64633,
760
+ "嗰種": 65942,
761
+ "嗰邊": 65133,
762
+ "嗰陣": 64076,
763
+ "嗱": 66436,
764
+ "嗲": 67020,
765
+ "嗶": 66947,
766
+ "嘅": 66299,
767
+ "嘅一": 64488,
768
+ "嘅一個": 64490,
769
+ "嘅事": 64192,
770
+ "嘅人": 64022,
771
+ "嘅佢": 65969,
772
+ "嘅係": 64284,
773
+ "嘅問題": 65357,
774
+ "嘅嘢": 64114,
775
+ "嘅地方": 64712,
776
+ "嘅大": 64837,
777
+ "嘅女仔": 65545,
778
+ "嘅市鎮一覽": 64525,
779
+ "嘅情況": 65118,
780
+ "嘅感覺": 65049,
781
+ "嘅我": 65427,
782
+ "嘅時候": 64091,
783
+ "嘅時間": 65653,
784
+ "嘅朋友": 66291,
785
+ "嘅話": 64151,
786
+ "嘅關係": 66074,
787
+ "嘆": 66357,
788
+ "嘈": 66606,
789
+ "嘉欣": 65758,
790
+ "嘉賓": 65922,
791
+ "嘔": 66618,
792
+ "嘖": 66784,
793
+ "嘗試": 64191,
794
+ "嘞": 66687,
795
+ "嘢": 66311,
796
+ "嘢食": 65442,
797
+ "嘥": 67089,
798
+ "嘩": 66423,
799
+ "嘭": 66757,
800
+ "嘯": 66752,
801
+ "嘰": 66994,
802
+ "嘶": 66764,
803
+ "嘻嘻": 64642,
804
+ "噁": 66927,
805
+ "噉": 66305,
806
+ "噉你": 64359,
807
+ "噉做": 65772,
808
+ "噉嘅": 64371,
809
+ "噉就": 65433,
810
+ "噉我": 65022,
811
+ "噉樣": 64145,
812
+ "噉講": 64242,
813
+ "噏": 67208,
814
+ "噓": 66906,
815
+ "噗": 66686,
816
+ "噠": 66742,
817
+ "噢": 66737,
818
+ "噴": 66464,
819
+ "噸": 67161,
820
+ "噹": 66596,
821
+ "嚇": 66330,
822
+ "嚇到": 66012,
823
+ "嚇得": 64686,
824
+ "嚎": 66919,
825
+ "嚐": 66812,
826
+ "嚓": 66826,
827
+ "嚕": 66856,
828
+ "嚟": 66306,
829
+ "嚟到": 65234,
830
+ "嚟嘅": 65141,
831
+ "嚟講": 64373,
832
+ "嚟越": 66069,
833
+ "嚥": 67007,
834
+ "嚨": 66733,
835
+ "嚮": 66839,
836
+ "嚴肅": 65678,
837
+ "嚿": 66846,
838
+ "囂": 66818,
839
+ "囉": 66325,
840
+ "囑": 66933,
841
+ "四個": 64557,
842
+ "四處": 65192,
843
+ "回來": 64084,
844
+ "回來了": 65618,
845
+ "回復": 64516,
846
+ "回想起": 66229,
847
+ "回憶": 64244,
848
+ "回歸": 65388,
849
+ "回覆": 64622,
850
+ "回過神": 65939,
851
+ "回頭": 64277,
852
+ "因為他": 65535,
853
+ "因為佢": 65565,
854
+ "因為我": 64347,
855
+ "囡": 66720,
856
+ "困難": 64763,
857
+ "國安": 65243,
858
+ "國家安全": 65651,
859
+ "國王": 65453,
860
+ "圖書": 64741,
861
+ "圖書館": 64869,
862
+ "圖案": 65543,
863
+ "圖片": 64033,
864
+ "圖片來源": 64043,
865
+ "在場": 65054,
866
+ "在於": 66234,
867
+ "在這": 64144,
868
+ "在這個": 64856,
869
+ "在這裏": 64256,
870
+ "地上的": 65842,
871
+ "地問": 64792,
872
+ "地圖": 65190,
873
+ "地獄": 64663,
874
+ "地產": 65871,
875
+ "地說": 64085,
876
+ "地說道": 66149,
877
+ "地鐵": 64462,
878
+ "地點": 64453,
879
+ "坐低": 65797,
880
+ "埋一齊": 65660,
881
+ "埋嚟": 65661,
882
+ "埗": 66904,
883
+ "埸": 67048,
884
+ "基於": 65608,
885
+ "堅定": 65155,
886
+ "堯": 66860,
887
+ "報仇": 66027,
888
+ "報紙": 65566,
889
+ "報警": 65908,
890
+ "報道": 64855,
891
+ "場地": 65897,
892
+ "場的": 66159,
893
+ "場面": 65336,
894
+ "塊面": 65866,
895
+ "塗": 66639,
896
+ "塵": 66429,
897
+ "墊": 66853,
898
+ "墜": 66657,
899
+ "墮": 66615,
900
+ "墳": 66805,
901
+ "壘": 67075,
902
+ "壯": 66475,
903
+ "壺": 66988,
904
+ "外國": 64508,
905
+ "多個": 64719,
906
+ "多啲": 64964,
907
+ "多數": 64459,
908
+ "多謝": 64216,
909
+ "多謝你": 65563,
910
+ "多麼": 65590,
911
+ "夢想": 64977,
912
+ "夥": 66704,
913
+ "大口": 65993,
914
+ "大叫": 64260,
915
+ "大嘅": 65995,
916
+ "大堂": 65295,
917
+ "大多數": 66032,
918
+ "大師": 65015,
919
+ "大廈": 64348,
920
+ "大戰": 65056,
921
+ "大樓": 65094,
922
+ "大約": 64362,
923
+ "大聲": 64305,
924
+ "大部份": 65277,
925
+ "大門": 64495,
926
+ "天佑": 64130,
927
+ "天佑同學": 65037,
928
+ "天台": 65393,
929
+ "天氣": 64932,
930
+ "天程": 65821,
931
+ "太過": 64864,
932
+ "太陽": 64343,
933
+ "失敗": 64455,
934
+ "失蹤": 65466,
935
+ "夾": 66371,
936
+ "奄": 67019,
937
+ "奧斯": 66009,
938
+ "奧運": 65950,
939
+ "奮": 66360,
940
+ "女仔": 64083,
941
+ "女兒": 64436,
942
+ "她們": 64119,
943
+ "她會": 66227,
944
+ "她的手": 66078,
945
+ "她說": 64334,
946
+ "好似": 64009,
947
+ "好呀": 64896,
948
+ "好啊": 65226,
949
+ "好啦": 64274,
950
+ "好嗎": 64842,
951
+ "好多人": 65185,
952
+ "好多時": 66135,
953
+ "好大": 64631,
954
+ "好彩": 65050,
955
+ "好快": 64468,
956
+ "好想": 64978,
957
+ "好有": 65237,
958
+ "好耐": 64609,
959
+ "好過": 65615,
960
+ "好難": 65464,
961
+ "如果唔係": 65284,
962
+ "妓": 67032,
963
+ "妝": 66527,
964
+ "妤": 66767,
965
+ "妲": 67072,
966
+ "妳的": 66176,
967
+ "始終": 64199,
968
+ "委會": 65822,
969
+ "姦": 66849,
970
+ "姿勢": 65481,
971
+ "姿態": 66167,
972
+ "威脅": 64850,
973
+ "娛": 66538,
974
+ "娛樂": 64703,
975
+ "婧": 67153,
976
+ "婭": 66902,
977
+ "媽咪": 64693,
978
+ "媽的": 65579,
979
+ "嫲": 66732,
980
+ "嫻": 67033,
981
+ "嬌": 66543,
982
+ "嬰": 66631,
983
+ "嬲": 66470,
984
+ "嬴": 67147,
985
+ "嬸": 66806,
986
+ "子彈": 64492,
987
+ "子瑜": 65629,
988
+ "子裏": 66240,
989
+ "孖": 66803,
990
+ "孤獨": 66233,
991
+ "孭": 67074,
992
+ "學家": 64926,
993
+ "學會": 64647,
994
+ "學生會": 65598,
995
+ "孽": 66956,
996
+ "它們": 64624,
997
+ "守衛": 65463,
998
+ "守護": 65063,
999
+ "安靜": 65269,
1000
+ "完全沒有": 65313,
1001
+ "完結": 64689,
1002
+ "官員": 65144,
1003
+ "定係": 65028,
1004
+ "定義": 65410,
1005
+ "宜家": 64857,
1006
+ "客廳": 65478,
1007
+ "客氣": 65283,
1008
+ "宣佈": 64738,
1009
+ "宣傳": 64747,
1010
+ "室內": 65539,
1011
+ "家姐": 64391,
1012
+ "家裏": 65157,
1013
+ "家超": 65170,
1014
+ "家長": 65090,
1015
+ "容許": 66055,
1016
+ "密碼": 66132,
1017
+ "察覺": 64552,
1018
+ "察覺到": 65506,
1019
+ "寢": 67229,
1020
+ "實力": 64209,
1021
+ "實在": 64082,
1022
+ "實在太": 65147,
1023
+ "實在是": 65706,
1024
+ "��際上": 65359,
1025
+ "實驗": 64444,
1026
+ "實體": 65415,
1027
+ "寧願": 65987,
1028
+ "寫真": 66134,
1029
+ "寵": 66589,
1030
+ "寵物": 66221,
1031
+ "射擊": 66062,
1032
+ "將佢": 65685,
1033
+ "將來": 64815,
1034
+ "將我": 65298,
1035
+ "將於": 65557,
1036
+ "專心": 65197,
1037
+ "專注": 65621,
1038
+ "專輯": 65149,
1039
+ "專門": 65816,
1040
+ "尋找": 64484,
1041
+ "對不起": 64452,
1042
+ "對了": 64737,
1043
+ "對他": 65017,
1044
+ "對付": 64762,
1045
+ "對住": 65366,
1046
+ "對你": 65511,
1047
+ "對佢": 66238,
1048
+ "對唔住": 64659,
1049
+ "對她": 65512,
1050
+ "對我": 64142,
1051
+ "對我說": 65773,
1052
+ "對手": 64267,
1053
+ "對抗": 65479,
1054
+ "對方": 64025,
1055
+ "對方的": 65096,
1056
+ "對此": 66294,
1057
+ "對着": 64593,
1058
+ "對自己": 65628,
1059
+ "對話": 64321,
1060
+ "對象": 64885,
1061
+ "對面": 64632,
1062
+ "導演": 64734,
1063
+ "小學": 64598,
1064
+ "小巴": 66279,
1065
+ "小弟": 65888,
1066
+ "小時候": 66223,
1067
+ "小組": 65930,
1068
+ "小說": 64301,
1069
+ "小隊": 65654,
1070
+ "少少": 64176,
1071
+ "少數": 66038,
1072
+ "少爺": 65750,
1073
+ "尖沙": 65921,
1074
+ "尖沙咀": 66251,
1075
+ "就係": 64007,
1076
+ "就只有": 64853,
1077
+ "就喺": 65374,
1078
+ "就噉": 66124,
1079
+ "就好似": 64575,
1080
+ "就已經": 64518,
1081
+ "就話": 65199,
1082
+ "就這樣": 64563,
1083
+ "就連": 64472,
1084
+ "就開始": 65605,
1085
+ "尷": 66422,
1086
+ "尷尬": 64265,
1087
+ "尼亞": 65281,
1088
+ "局局長": 65848,
1089
+ "局長": 64960,
1090
+ "屋企": 64037,
1091
+ "屋企人": 65446,
1092
+ "屌": 66408,
1093
+ "屌你": 66208,
1094
+ "屍": 66324,
1095
+ "屍體": 64394,
1096
+ "展現": 65417,
1097
+ "展覽": 65604,
1098
+ "屜": 67166,
1099
+ "屢": 67144,
1100
+ "屬性": 65967,
1101
+ "屯門": 65574,
1102
+ "岀": 66952,
1103
+ "岑": 66722,
1104
+ "岡": 66743,
1105
+ "峙": 67061,
1106
+ "峯": 66992,
1107
+ "峽": 66827,
1108
+ "崗": 66656,
1109
+ "崙": 67186,
1110
+ "崩潰": 65823,
1111
+ "嵐": 66451,
1112
+ "嶺": 66609,
1113
+ "嶼": 66877,
1114
+ "巔": 67027,
1115
+ "左邊": 66092,
1116
+ "差唔多": 64629,
1117
+ "差點": 64806,
1118
+ "已經係": 66143,
1119
+ "已經是": 65055,
1120
+ "已經有": 65472,
1121
+ "已經被": 65644,
1122
+ "巳": 67176,
1123
+ "市鎮": 64071,
1124
+ "市鎮一覽": 64155,
1125
+ "帝京": 66261,
1126
+ "帝國": 64721,
1127
+ "帥": 66529,
1128
+ "師傅": 64692,
1129
+ "師兄": 64602,
1130
+ "師父": 64536,
1131
+ "帶有": 65903,
1132
+ "帶着": 64448,
1133
+ "帶著": 64504,
1134
+ "帶領": 65249,
1135
+ "帶點": 65530,
1136
+ "常見": 65647,
1137
+ "幫你": 64364,
1138
+ "幫佢": 65362,
1139
+ "幫忙": 64510,
1140
+ "幫我": 64224,
1141
+ "幫手": 64532,
1142
+ "平時": 64180,
1143
+ "平靜": 64809,
1144
+ "年嘅": 64559,
1145
+ "年死": 64756,
1146
+ "年紀": 64634,
1147
+ "年輕人": 65802,
1148
+ "年青": 66292,
1149
+ "年齡": 65382,
1150
+ "幸運": 65291,
1151
+ "幾個": 64072,
1152
+ "幾個月": 66122,
1153
+ "幾多": 64528,
1154
+ "幾天": 64784,
1155
+ "幾好": 65436,
1156
+ "幾年": 64636,
1157
+ "幾日": 64845,
1158
+ "幾時": 64614,
1159
+ "幾次": 65109,
1160
+ "幾耐": 65409,
1161
+ "廁": 66447,
1162
+ "廁所": 64591,
1163
+ "廂": 66551,
1164
+ "廈": 66425,
1165
+ "廖永祥": 64694,
1166
+ "廚": 66417,
1167
+ "廚房": 64594,
1168
+ "廝": 67180,
1169
+ "廟": 66624,
1170
+ "廢話": 66160,
1171
+ "廣告": 64479,
1172
+ "廣場": 64658,
1173
+ "廣州": 64174,
1174
+ "廣州市": 66184,
1175
+ "廣播": 65134,
1176
+ "廣東": 64161,
1177
+ "廣東話": 64884,
1178
+ "建築物": 65500,
1179
+ "廿": 66479,
1180
+ "引發": 65985,
1181
+ "弩": 67116,
1182
+ "張開": 64786,
1183
+ "強化": 65883,
1184
+ "強大": 64789,
1185
+ "強大的": 65567,
1186
+ "強烈": 64565,
1187
+ "強的": 66252,
1188
+ "強者": 65794,
1189
+ "強行": 65349,
1190
+ "彌": 66528,
1191
+ "彎": 66504,
1192
+ "彙": 67008,
1193
+ "彥": 66580,
1194
+ "彷": 66365,
1195
+ "彷彿": 64196,
1196
+ "彿": 66392,
1197
+ "往後": 65662,
1198
+ "待續": 65553,
1199
+ "很想": 65255,
1200
+ "很難": 65347,
1201
+ "律師": 64773,
1202
+ "後便": 65077,
1203
+ "後就": 65755,
1204
+ "後悔": 64698,
1205
+ "後方": 64697,
1206
+ "後生": 65485,
1207
+ "後的": 64381,
1208
+ "後退": 65064,
1209
+ "後面": 64252,
1210
+ "徑": 66481,
1211
+ "得啦": 65206,
1212
+ "得我": 65695,
1213
+ "得返": 65648,
1214
+ "得閒": 66133,
1215
+ "從來": 64243,
1216
+ "從後": 65928,
1217
+ "從未": 65175,
1218
+ "從而": 66095,
1219
+ "復活": 65630,
1220
+ "徹": 66478,
1221
+ "徹底": 64958,
1222
+ "心臟": 64919,
1223
+ "心裏": 64081,
1224
+ "心靈": 65402,
1225
+ "心頭": 65465,
1226
+ "忍唔住": 65302,
1227
+ "忐": 67212,
1228
+ "忑": 67213,
1229
+ "忖": 67098,
1230
+ "忘記": 64262,
1231
+ "忘記了": 66023,
1232
+ "快啲": 64342,
1233
+ "快必": 64491,
1234
+ "快樂": 64372,
1235
+ "快點": 64843,
1236
+ "念頭": 65767,
1237
+ "忿": 66873,
1238
+ "怎會": 65333,
1239
+ "怎樣": 64087,
1240
+ "怎麼了": 64897,
1241
+ "怎麼會": 65774,
1242
+ "怎麼樣": 65762,
1243
+ "怎麼辦": 65257,
1244
+ "怔": 66723,
1245
+ "思緒": 65760,
1246
+ "怦": 67185,
1247
+ "恆": 66463,
1248
+ "恐懼": 64668,
1249
+ "恢復": 64473,
1250
+ "恥": 66626,
1251
+ "悅": 66501,
1252
+ "悕": 67171,
1253
+ "悚": 67041,
1254
+ "悲傷": 66084,
1255
+ "悶": 66450,
1256
+ "悸": 67138,
1257
+ "情侶": 64905,
1258
+ "情報": 65038,
1259
+ "情況下": 66136,
1260
+ "情緒": 64253,
1261
+ "惡魔": 64910,
1262
+ "惱": 66531,
1263
+ "想過": 64726,
1264
+ "惺": 67080,
1265
+ "愈來": 65204,
1266
+ "愈來愈": 65211,
1267
+ "意圖": 66257,
1268
+ "意識到": 65452,
1269
+ "愕": 66562,
1270
+ "愕然": 66186,
1271
+ "愛你": 65391,
1272
+ "愛情": 64139,
1273
+ "愛的": 64709,
1274
+ "愛麗絲": 65971,
1275
+ "感動": 65033,
1276
+ "感應": 65891,
1277
+ "感覺到": 64344,
1278
+ "愷": 67006,
1279
+ "慄": 67113,
1280
+ "慘": 66389,
1281
+ "慚": 67188,
1282
+ "慳": 66879,
1283
+ "慶祝": 65099,
1284
+ "慾": 66572,
1285
+ "憊": 67103,
1286
+ "憎": 66673,
1287
+ "憐": 66473,
1288
+ "憑": 66367,
1289
+ "憤": 66457,
1290
+ "憤怒": 64664,
1291
+ "憫": 67172,
1292
+ "憲": 66667,
1293
+ "懇": 66940,
1294
+ "應付": 65072,
1295
+ "應對": 65704,
1296
+ "應承": 65437,
1297
+ "應該係": 64874,
1298
+ "應該是": 64379,
1299
+ "應該會": 65819,
1300
+ "懲": 66627,
1301
+ "懲罰": 66034,
1302
+ "懶": 66488,
1303
+ "懷疑": 64228,
1304
+ "懸": 66594,
1305
+ "懼": 66448,
1306
+ "懾": 67127,
1307
+ "戀愛": 64945,
1308
+ "戇": 66823,
1309
+ "成個": 64376,
1310
+ "成日": 64135,
1311
+ "成為了": 65580,
1312
+ "我一眼": 66284,
1313
+ "我不會": 65855,
1314
+ "我亦": 65642,
1315
+ "我仲": 65406,
1316
+ "我來": 65876,
1317
+ "我係": 64108,
1318
+ "我個": 64486,
1319
+ "我們在": 65957,
1320
+ "我們要": 66054,
1321
+ "我們都": 65683,
1322
+ "我冇": 64852,
1323
+ "我卻": 65740,
1324
+ "我只係": 65700,
1325
+ "我可": 66205,
1326
+ "我同": 64066,
1327
+ "我同佢": 65751,
1328
+ "我呀": 65867,
1329
+ "我呢": 65979,
1330
+ "我哋": 64000,
1331
+ "我哋嘅": 65086,
1332
+ "我唔": 64100,
1333
+ "我唔係": 65323,
1334
+ "我唔想": 65276,
1335
+ "我唔會": 65798,
1336
+ "我唔知": 65770,
1337
+ "我問": 64249,
1338
+ "我啦": 65889,
1339
+ "我喺": 65852,
1340
+ "我嘅": 64079,
1341
+ "我好": 64181,
1342
+ "我將": 65865,
1343
+ "我對": 64290,
1344
+ "我已經": 64190,
1345
+ "我從": 65781,
1346
+ "我應該": 65610,
1347
+ "我明白": 66002,
1348
+ "我會": 64075,
1349
+ "我決定": 65991,
1350
+ "我沒": 65368,
1351
+ "我沒有": 64469,
1352
+ "我無": 64542,
1353
+ "我現在": 65098,
1354
+ "我的手": 65209,
1355
+ "我真係": 64458,
1356
+ "我睇": 66075,
1357
+ "我知": 64625,
1358
+ "我終於": 65652,
1359
+ "我聽": 66182,
1360
+ "我聽到": 66287,
1361
+ "我見": 66191,
1362
+ "我見到": 65910,
1363
+ "我覺得": 64235,
1364
+ "我記得": 65996,
1365
+ "我話": 65319,
1366
+ "我說": 64052,
1367
+ "我諗": 64317,
1368
+ "我講": 64645,
1369
+ "我還": 64463,
1370
+ "我還是": 64939,
1371
+ "我都係": 64904,
1372
+ "我都唔": 65796,
1373
+ "我都唔知": 65982,
1374
+ "我都會": 65923,
1375
+ "我開始": 65771,
1376
+ "或許": 64241,
1377
+ "戟": 67077,
1378
+ "戮": 66983,
1379
+ "戰場": 64993,
1380
+ "戰士": 64923,
1381
+ "戰爭": 64402,
1382
+ "戰略": 65929,
1383
+ "戰鬥": 64158,
1384
+ "房門": 65332,
1385
+ "房間": 64064,
1386
+ "所說": 65405,
1387
+ "所謂的": 65664,
1388
+ "手上的": 64548,
1389
+ "手提": 66265,
1390
+ "手槍": 65838,
1391
+ "手腳": 66115,
1392
+ "手術": 65419,
1393
+ "手袋": 64927,
1394
+ "手錶": 66119,
1395
+ "才會": 64597,
1396
+ "才發現": 65793,
1397
+ "打擊": 65181,
1398
+ "打斷": 64976,
1399
+ "打機": 65673,
1400
+ "打量": 65296,
1401
+ "打開": 64048,
1402
+ "打開了": 65516,
1403
+ "打電話": 65311,
1404
+ "批評": 65084,
1405
+ "承認": 64590,
1406
+ "承諾": 65073,
1407
+ "投訴": 65497,
1408
+ "抬起頭": 65697,
1409
+ "抬頭": 64507,
1410
+ "抱住": 66270,
1411
+ "抱著": 65498,
1412
+ "抿": 66986,
1413
+ "拉開": 65429,
1414
+ "拋": 66437,
1415
+ "拍拍": 65586,
1416
+ "拍拖": 64637,
1417
+ "拍攝": 64567,
1418
+ "拎住": 65430,
1419
+ "拎起": 65951,
1420
+ "拒絕": 64310,
1421
+ "拗": 67012,
1422
+ "拘捕": 66042,
1423
+ "拚": 67086,
1424
+ "拜託": 66083,
1425
+ "拳頭": 64682,
1426
+ "拿著": 64906,
1427
+ "指揮": 65019,
1428
+ "指數": 64476,
1429
+ "指著": 65051,
1430
+ "按年": 65856,
1431
+ "挾": 67170,
1432
+ "捉住": 65065,
1433
+ "捨": 66492,
1434
+ "捱": 66680,
1435
+ "捲": 66570,
1436
+ "捺": 67111,
1437
+ "掂": 66438,
1438
+ "掃": 66378,
1439
+ "掙": 66511,
1440
+ "掙扎": 64827,
1441
+ "掛": 66332,
1442
+ "掛住": 65581,
1443
+ "掟": 66990,
1444
+ "採取": 65468,
1445
+ "採用": 65408,
1446
+ "掣": 66842,
1447
+ "接下來": 64415,
1448
+ "接著": 64187,
1449
+ "接觸": 64361,
1450
+ "接過": 65079,
1451
+ "推廣": 66249,
1452
+ "推薦": 65741,
1453
+ "推進": 66277,
1454
+ "推開": 64742,
1455
+ "揀": 66446,
1456
+ "揍": 67084,
1457
+ "提議": 65088,
1458
+ "握著": 65203,
1459
+ "揮手": 65010,
1460
+ "揸": 66586,
1461
+ "揹": 67023,
1462
+ "揼": 67183,
1463
+ "搐": 67078,
1464
+ "搔": 66926,
1465
+ "搖": 66326,
1466
+ "搖了": 66036,
1467
+ "搖搖頭": 65173,
1468
+ "搖頭": 64167,
1469
+ "搗": 67058,
1470
+ "搞到": 64339,
1471
+ "搞掂": 65509,
1472
+ "搵": 66317,
1473
+ "搵到": 64409,
1474
+ "搵我": 65367,
1475
+ "摑": 67190,
1476
+ "摟": 66866,
1477
+ "摯": 67149,
1478
+ "摺": 66782,
1479
+ "撈": 66613,
1480
+ "撐": 66428,
1481
+ "撒旦": 65729,
1482
+ "撓": 66824,
1483
+ "撚": 66369,
1484
+ "撞到": 65622,
1485
+ "撥": 66503,
1486
+ "撫": 66498,
1487
+ "撲": 66420,
1488
+ "撳": 66871,
1489
+ "撻": 66955,
1490
+ "撿": 66825,
1491
+ "擁抱": 65617,
1492
+ "擂": 66996,
1493
+ "擊中": 65344,
1494
+ "擋": 66348,
1495
+ "擒": 66922,
1496
+ "擔心": 64088,
1497
+ "擔憂": 66168,
1498
+ "擘": 67216,
1499
+ "擠": 66564,
1500
+ "擰": 66706,
1501
+ "擱": 66981,
1502
+ "擲": 66676,
1503
+ "擴大": 66189,
1504
+ "擺": 66334,
1505
+ "擾": 66439,
1506
+ "攏": 66993,
1507
+ "攔": 66717,
1508
+ "攜": 66799,
1509
+ "攝影": 65069,
1510
+ "攞": 66346,
1511
+ "攞到": 65655,
1512
+ "攤": 66524,
1513
+ "攪": 67011,
1514
+ "攬": 66507,
1515
+ "支撐": 66285,
1516
+ "攰": 66655,
1517
+ "收市": 66000,
1518
+ "收費": 65569,
1519
+ "收起": 65643,
1520
+ "攷": 66442,
1521
+ "攻勢": 65896,
1522
+ "攻擊": 64047,
1523
+ "放低": 65690,
1524
+ "放學": 64446,
1525
+ "放棄": 64188,
1526
+ "放過": 65272,
1527
+ "放開": 65513,
1528
+ "放鬆": 65009,
1529
+ "故仔": 65932,
1530
+ "敘": 66885,
1531
+ "教練": 65273,
1532
+ "散發": 65024,
1533
+ "整個人": 64691,
1534
+ "敵人": 64324,
1535
+ "數十": 65549,
1536
+ "數字": 64337,
1537
+ "數學": 65083,
1538
+ "數碼": 66050,
1539
+ "數量": 64998,
1540
+ "斂": 67133,
1541
+ "斃": 66936,
1542
+ "斟": 67222,
1543
+ "斬": 66402,
1544
+ "新界": 65394,
1545
+ "新鮮": 65709,
1546
+ "旁邊": 64103,
1547
+ "旁邊的": 65004,
1548
+ "日後": 64991,
1549
+ "日日": 65191,
1550
+ "日本足球員": 66096,
1551
+ "日記": 66111,
1552
+ "旺角": 65245,
1553
+ "旻": 66622,
1554
+ "昇": 66694,
1555
+ "星期一": 66033,
1556
+ "星期六": 65331,
1557
+ "是一種": 65383,
1558
+ "是什麼": 64320,
1559
+ "是個": 64258,
1560
+ "是嗎": 65986,
1561
+ "是因為": 64293,
1562
+ "是為了": 65719,
1563
+ "是甚麼": 64933,
1564
+ "是誰": 64397,
1565
+ "是這樣": 65603,
1566
+ "時光": 65582,
1567
+ "時刻": 65810,
1568
+ "時尚": 65329,
1569
+ "時機": 66070,
1570
+ "時的": 64981,
1571
+ "時空": 65650,
1572
+ "時裝": 65104,
1573
+ "晉": 66525,
1574
+ "晌": 66886,
1575
+ "晚飯": 65577,
1576
+ "晝": 66663,
1577
+ "晞": 66636,
1578
+ "晟": 67201,
1579
+ "暈": 66468,
1580
+ "暉": 67191,
1581
+ "暢": 66642,
1582
+ "暫停": 65768,
1583
+ "暫時": 64259,
1584
+ "曆": 66593,
1585
+ "曉": 66342,
1586
+ "曖": 66912,
1587
+ "曠": 67081,
1588
+ "曬": 66703,
1589
+ "曳": 67150,
1590
+ "更何況": 65854,
1591
+ "更為": 65682,
1592
+ "書包": 65763,
1593
+ "書院": 66154,
1594
+ "最強": 65205,
1595
+ "最後一": 65023,
1596
+ "最後的": 65730,
1597
+ "會不會": 65111,
1598
+ "會否": 65384,
1599
+ "會唔會": 64423,
1600
+ "會在": 64761,
1601
+ "會被": 64971,
1602
+ "會議員": 66128,
1603
+ "會長": 64718,
1604
+ "月台": 65614,
1605
+ "月娥": 66211,
1606
+ "有一個": 64230,
1607
+ "有一種": 65059,
1608
+ "有一點": 66057,
1609
+ "有什麼": 64365,
1610
+ "有個": 64146,
1611
+ "有兩": 65339,
1612
+ "有兩個": 65975,
1613
+ "有冇": 64309,
1614
+ "有咩": 64312,
1615
+ "有問題": 66209,
1616
+ "有啲": 64053,
1617
+ "有好多": 64744,
1618
+ "有如": 65692,
1619
+ "有少少": 66141,
1620
+ "有幾": 64298,
1621
+ "有得": 65425,
1622
+ "有時": 64280,
1623
+ "有時候": 65727,
1624
+ "有機會": 64519,
1625
+ "有沒有": 64566,
1626
+ "有無": 64684,
1627
+ "有甚麼": 64928,
1628
+ "有種": 65000,
1629
+ "有興趣": 65514,
1630
+ "有著": 65113,
1631
+ "有錢": 65105,
1632
+ "有點": 64017,
1633
+ "有點不": 66152,
1634
+ "朕": 66870,
1635
+ "望一望": 65066,
1636
+ "望住": 64086,
1637
+ "望住我": 64907,
1638
+ "望向": 64227,
1639
+ "望著": 64729,
1640
+ "朧": 67065,
1641
+ "本來": 64078,
1642
+ "本書": 65443,
1643
+ "本港": 64841,
1644
+ "朱古": 65162,
1645
+ "朱古力": 65218,
1646
+ "李家超": 66101,
1647
+ "東京": 65330,
1648
+ "東南": 65712,
1649
+ "東方": 65777,
1650
+ "林顧": 65101,
1651
+ "果啲": 65726,
1652
+ "果陣": 65757,
1653
+ "枱": 66500,
1654
+ "某個": 65392,
1655
+ "柒": 66760,
1656
+ "校園": 64621,
1657
+ "校服": 65526,
1658
+ "校長": 64570,
1659
+ "根本就": 65420,
1660
+ "根本沒有": 66131,
1661
+ "格蘭": 65220,
1662
+ "桿": 66999,
1663
+ "梗係": 64652,
1664
+ "條例": 65070,
1665
+ "條友": 65180,
1666
+ "條女": 65956,
1667
+ "棟": 66650,
1668
+ "棧": 67207,
1669
+ "棲": 67139,
1670
+ "楓": 66549,
1671
+ "極度": 65920,
1672
+ "極為": 65850,
1673
+ "極限": 65904,
1674
+ "榮譽": 66203,
1675
+ "槍": 66322,
1676
+ "樂園": 66215,
1677
+ "樂甜": 66087,
1678
+ "樑": 66890,
1679
+ "樓下": 64994,
1680
+ "樓梯": 64354,
1681
+ "標誌": 65936,
1682
+ "樞": 66970,
1683
+ "模擬": 66198,
1684
+ "模樣": 64513,
1685
+ "模特兒": 65792,
1686
+ "樣嘢": 64975,
1687
+ "樣子": 64080,
1688
+ "樣貌": 66239,
1689
+ "樸": 66918,
1690
+ "樺": 66792,
1691
+ "樽": 66678,
1692
+ "機制": 65684,
1693
+ "機器": 65737,
1694
+ "機場": 64386,
1695
+ "機械": 64433,
1696
+ "機械人": 65859,
1697
+ "機甲": 65824,
1698
+ "機關": 65656,
1699
+ "橫": 66374,
1700
+ "檔案": 65970,
1701
+ "檯": 66654,
1702
+ "檳": 67018,
1703
+ "檸": 66785,
1704
+ "檻": 67220,
1705
+ "櫃": 66382,
1706
+ "櫈": 66982,
1707
+ "櫻": 66553,
1708
+ "欄": 66573,
1709
+ "權利": 65785,
1710
+ "權力": 65404,
1711
+ "欖": 66930,
1712
+ "欣賞": 64790,
1713
+ "欸": 66929,
1714
+ "欽": 67066,
1715
+ "歎": 66788,
1716
+ "歐陽": 66237,
1717
+ "正想": 65756,
1718
+ "正正": 65306,
1719
+ "正當": 65447,
1720
+ "正確": 64942,
1721
+ "正義": 66218,
1722
+ "此時": 64110,
1723
+ "武裝": 66076,
1724
+ "歲的": 65317,
1725
+ "死人": 65326,
1726
+ "死咗": 64804,
1727
+ "殘": 66354,
1728
+ "殤": 66903,
1729
+ "殭": 66854,
1730
+ "殲": 67063,
1731
+ "段時間": 64889,
1732
+ "殺了": 65487,
1733
+ "殺人": 64447,
1734
+ "殺手": 64801,
1735
+ "殺掉": 66039,
1736
+ "殺死": 64439,
1737
+ "殼": 66616,
1738
+ "毀": 66455,
1739
+ "毆": 67155,
1740
+ "每一個": 64968,
1741
+ "每個人": 65016,
1742
+ "毓": 67030,
1743
+ "毛毛": 64511,
1744
+ "毫無": 64315,
1745
+ "氣勢": 65989,
1746
+ "氣息": 65044,
1747
+ "氣氛": 64534,
1748
+ "氣的": 65011,
1749
+ "氫": 67152,
1750
+ "永祥": 64665,
1751
+ "永遠": 64182,
1752
+ "氹": 66880,
1753
+ "江河": 65142,
1754
+ "決心": 65395,
1755
+ "決賽": 65250,
1756
+ "沌": 66977,
1757
+ "沒事": 64428,
1758
+ "沒人": 66185,
1759
+ "沒什麼": 65488,
1760
+ "沒問題": 66264,
1761
+ "沒想到": 64580,
1762
+ "沒有人": 64443,
1763
+ "沒有任何": 64791,
1764
+ "沒法": 65962,
1765
+ "沒辦法": 65844,
1766
+ "沒錯": 64650,
1767
+ "沖": 66372,
1768
+ "沙田": 65717,
1769
+ "沙發": 65121,
1770
+ "法國": 64060,
1771
+ "法國嘅": 64150,
1772
+ "法國嘅市鎮一覽": 64496,
1773
+ "法師": 66145,
1774
+ "注視": 65783,
1775
+ "泰國": 65576,
1776
+ "洗手間": 65701,
1777
+ "洩": 66628,
1778
+ "洶": 66960,
1779
+ "流動": 65426,
1780
+ "浚": 67043,
1781
+ "浩然": 64346,
1782
+ "浪費": 64901,
1783
+ "浮現": 65363,
1784
+ "消滅": 66148,
1785
+ "涼": 66384,
1786
+ "淆": 67178,
1787
+ "淒": 66972,
1788
+ "淚": 66331,
1789
+ "淚水": 64860,
1790
+ "淨係": 64422,
1791
+ "淩": 67210,
1792
+ "淪": 66819,
1793
+ "深處": 65681,
1794
+ "淵": 66569,
1795
+ "混亂": 65018,
1796
+ "淺": 66471,
1797
+ "清潔": 65675,
1798
+ "渦": 66959,
1799
+ "測試": 64788,
1800
+ "港姐": 65342,
1801
+ "港鐵": 65520,
1802
+ "渾": 66558,
1803
+ "渾身": 65334,
1804
+ "湊": 66658,
1805
+ "湘渝": 65036,
1806
+ "湧": 66431,
1807
+ "準備好": 65570,
1808
+ "準確": 65693,
1809
+ "溝通": 64835,
1810
+ "溫度": 65862,
1811
+ "溫暖": 64962,
1812
+ "溫書": 65881,
1813
+ "溫柔": 64377,
1814
+ "滄": 67140,
1815
+ "滙": 67141,
1816
+ "滬": 67163,
1817
+ "滯": 66648,
1818
+ "滲": 66669,
1819
+ "滾": 66452,
1820
+ "滿了": 65665,
1821
+ "滿意": 64887,
1822
+ "滿臉": 65906,
1823
+ "滿足": 64547,
1824
+ "漁": 66710,
1825
+ "漆黑": 65052,
1826
+ "演員": 64172,
1827
+ "演唱會": 65270,
1828
+ "漩": 67145,
1829
+ "漫畫": 64909,
1830
+ "漬": 67228,
1831
+ "漸漸": 64207,
1832
+ "漿": 66833,
1833
+ "潑": 66775,
1834
+ "潤": 66557,
1835
+ "潰": 66659,
1836
+ "澀": 66753,
1837
+ "澳門": 64431,
1838
+ "激動": 64579,
1839
+ "濁": 67064,
1840
+ "濃": 66427,
1841
+ "濕": 66385,
1842
+ "濛": 67181,
1843
+ "濤": 66769,
1844
+ "濫": 66810,
1845
+ "濱": 66664,
1846
+ "濺": 66837,
1847
+ "濾": 67197,
1848
+ "瀅": 67142,
1849
+ "瀉": 66915,
1850
+ "瀏": 66891,
1851
+ "瀟": 66957,
1852
+ "瀨": 66897,
1853
+ "瀰": 67182,
1854
+ "瀾": 67136,
1855
+ "灑": 66630,
1856
+ "灘": 66587,
1857
+ "灣仔": 66273,
1858
+ "灣區": 66049,
1859
+ "炯": 67217,
1860
+ "炸彈": 65933,
1861
+ "為主": 65244,
1862
+ "為人": 65316,
1863
+ "為什麼要": 66259,
1864
+ "為何": 64134,
1865
+ "為你": 65902,
1866
+ "為咗": 64185,
1867
+ "為我": 65114,
1868
+ "為止": 64980,
1869
+ "為甚麼": 64329,
1870
+ "為自己": 65935,
1871
+ "烏克蘭": 65377,
1872
+ "焗": 66910,
1873
+ "無事": 64915,
1874
+ "無人": 64270,
1875
+ "無力": 64982,
1876
+ "無可": 65139,
1877
+ "無啦啦": 65860,
1878
+ "無奈": 64217,
1879
+ "無意": 66116,
1880
+ "無所": 66214,
1881
+ "無數": 64601,
1882
+ "無比": 64768,
1883
+ "無綫": 65034,
1884
+ "無綫電視": 65435,
1885
+ "無聊": 64946,
1886
+ "無聲": 65688,
1887
+ "無視": 65596,
1888
+ "無錯": 65965,
1889
+ "無限": 65135,
1890
+ "然後再": 66199,
1891
+ "然後就": 66276,
1892
+ "煉": 66373,
1893
+ "煒": 67031,
1894
+ "煥": 67104,
1895
+ "照顧": 64257,
1896
+ "煩": 66329,
1897
+ "煩惱": 65623,
1898
+ "煽": 66868,
1899
+ "熱情": 65909,
1900
+ "熱鬧": 66051,
1901
+ "熹": 66973,
1902
+ "熾": 66924,
1903
+ "燃燒": 65532,
1904
+ "燈光": 65282,
1905
+ "燙": 66863,
1906
+ "營運": 65825,
1907
+ "燦": 66726,
1908
+ "燭": 66748,
1909
+ "爆發": 64654,
1910
+ "爍": 66828,
1911
+ "爐": 66583,
1912
+ "爛": 66377,
1913
+ "爭取": 65145,
1914
+ "爭議": 65948,
1915
+ "爲": 66469,
1916
+ "爺": 66336,
1917
+ "爺爺": 64999,
1918
+ "牆": 66335,
1919
+ "牆上": 66220,
1920
+ "牆壁": 65020,
1921
+ "牠": 66341,
1922
+ "牠們": 64854,
1923
+ "物資": 66022,
1924
+ "物質": 65778,
1925
+ "物體": 65847,
1926
+ "特區": 65438,
1927
+ "特工": 66063,
1928
+ "特徵": 66262,
1929
+ "特登": 65872,
1930
+ "牽": 66444,
1931
+ "犧": 66677,
1932
+ "犧牲": 65288,
1933
+ "狙": 66763,
1934
+ "狙擊": 66187,
1935
+ "狡": 66935,
1936
+ "狩": 66939,
1937
+ "狹": 66847,
1938
+ "狽": 66896,
1939
+ "猙": 67015,
1940
+ "猛然": 66126,
1941
+ "猩": 66974,
1942
+ "猶": 66359,
1943
+ "猶如": 65026,
1944
+ "猶豫": 64752,
1945
+ "猿": 66913,
1946
+ "獄": 66383,
1947
+ "獅": 66517,
1948
+ "獅子": 65878,
1949
+ "獨特": 65365,
1950
+ "獨自": 64961,
1951
+ "獰": 66908,
1952
+ "獵": 66415,
1953
+ "獸": 66347,
1954
+ "王國": 65607,
1955
+ "玖": 67193,
1956
+ "玟": 67173,
1957
+ "玥": 66633,
1958
+ "玥嘉": 66150,
1959
+ "珈": 67151,
1960
+ "班人": 65480,
1961
+ "班房": 65224,
1962
+ "班牙": 65458,
1963
+ "班長": 66106,
1964
+ "珮": 67200,
1965
+ "現在的": 64966,
1966
+ "現實": 64204,
1967
+ "現時": 64412,
1968
+ "現象": 65609,
1969
+ "現身": 66048,
1970
+ "球員": 64640,
1971
+ "球場": 64947,
1972
+ "球隊": 64892,
1973
+ "理會": 64451,
1974
+ "琛": 66755,
1975
+ "瑋": 67036,
1976
+ "瑛": 67090,
1977
+ "瑣": 67076,
1978
+ "瑤": 66449,
1979
+ "瑩": 66554,
1980
+ "瑪": 66411,
1981
+ "璇": 66893,
1982
+ "環保": 65893,
1983
+ "瓊": 66976,
1984
+ "瓏": 67119,
1985
+ "甚麼": 64006,
1986
+ "甚麼事": 65686,
1987
+ "生氣": 64667,
1988
+ "甦": 67021,
1989
+ "用來": 65078,
1990
+ "用咗": 66243,
1991
+ "用嘅": 65348,
1992
+ "用嚟": 64900,
1993
+ "用盡": 65918,
1994
+ "男仔": 64369,
1995
+ "町": 66987,
1996
+ "畀": 66314,
1997
+ "畀人": 64635,
1998
+ "畀你": 64198,
1999
+ "畀佢": 64292,
2000
+ "畀我": 64137,
2001
+ "留低": 66004,
2002
+ "留意到": 65159,
2003
+ "畢竟": 64157,
2004
+ "畫面": 64246,
2005
+ "異常": 64935,
2006
+ "異能": 64954,
2007
+ "當下": 65869,
2008
+ "當作": 65552,
2009
+ "當你": 66137,
2010
+ "當初": 64660,
2011
+ "當哉": 65784,
2012
+ "當局": 64903,
2013
+ "當年": 64206,
2014
+ "當成": 65800,
2015
+ "當我": 64303,
2016
+ "當日": 64678,
2017
+ "當然是": 66110,
2018
+ "疇": 67035,
2019
+ "疊": 66571,
2020
+ "疑問": 65198,
2021
+ "痕跡": 66052,
2022
+ "痛楚": 65092,
2023
+ "痴線": 66121,
2024
+ "痺": 67194,
2025
+ "瘋": 66370,
2026
+ "瘋狂": 64583,
2027
+ "瘡": 67100,
2028
+ "癒": 66765,
2029
+ "癡": 66836,
2030
+ "癢": 66808,
2031
+ "癮": 66768,
2032
+ "癱": 67052,
2033
+ "癲": 66682,
2034
+ "登記": 65413,
2035
+ "發佈": 66266,
2036
+ "發光": 66108,
2037
+ "發出": 64092,
2038
+ "發動": 64830,
2039
+ "發射": 65136,
2040
+ "發現了": 65839,
2041
+ "發現自己": 66097,
2042
+ "發生了": 64886,
2043
+ "發生咩事": 65998,
2044
+ "發生過": 66071,
2045
+ "發行": 65448,
2046
+ "發覺": 64672,
2047
+ "發言": 64959,
2048
+ "發言人": 65980,
2049
+ "白痴": 65769,
2050
+ "百萬": 66155,
2051
+ "的一個": 65202,
2052
+ "的一刻": 66035,
2053
+ "的一聲": 65027,
2054
+ "的劍": 66109,
2055
+ "的動作": 65558,
2056
+ "的同時": 65371,
2057
+ "的吧": 65973,
2058
+ "的嗎": 64880,
2059
+ "的士": 64838,
2060
+ "的學生": 66281,
2061
+ "的後": 65899,
2062
+ "的情況下": 65641,
2063
+ "的感覺": 64136,
2064
+ "的房間": 65399,
2065
+ "的攻擊": 65370,
2066
+ "的時": 64921,
2067
+ "的時間": 64194,
2068
+ "的東西": 64210,
2069
+ "的模樣": 65486,
2070
+ "的樣子": 64178,
2071
+ "的機會": 64859,
2072
+ "的氣": 64263,
2073
+ "的氣氛": 66072,
2074
+ "的火": 66041,
2075
+ "的環境": 66013,
2076
+ "的畫面": 66120,
2077
+ "的眼": 65666,
2078
+ "的確": 64222,
2079
+ "的聲音": 64074,
2080
+ "的臉": 64306,
2081
+ "的語氣": 65345,
2082
+ "的說": 64225,
2083
+ "的說話": 64833,
2084
+ "的距離": 66102,
2085
+ "的身體": 64803,
2086
+ "的道": 64560,
2087
+ "的長": 65215,
2088
+ "的門": 65676,
2089
+ "的關係": 64520,
2090
+ "的阿": 65718,
2091
+ "的頭": 64759,
2092
+ "的風": 65879,
2093
+ "皎": 66964,
2094
+ "皮膚": 64706,
2095
+ "皺": 66412,
2096
+ "皺眉": 66046,
2097
+ "盃": 66536,
2098
+ "盜": 66607,
2099
+ "盞": 66958,
2100
+ "盡力": 65752,
2101
+ "盡快": 64985,
2102
+ "盡量": 64983,
2103
+ "盡頭": 65738,
2104
+ "監視": 66267,
2105
+ "盧": 66396,
2106
+ "盪": 66734,
2107
+ "盯著": 64822,
2108
+ "直覺": 66019,
2109
+ "相對": 64895,
2110
+ "相機": 66173,
2111
+ "相片": 65937,
2112
+ "相處": 64626,
2113
+ "相識": 65550,
2114
+ "眉頭": 64705,
2115
+ "看來": 64090,
2116
+ "看着我": 64749,
2117
+ "看著": 64062,
2118
+ "看見": 64063,
2119
+ "看起來": 64550,
2120
+ "看過": 65214,
2121
+ "真係": 64002,
2122
+ "真係好": 64160,
2123
+ "真實": 64618,
2124
+ "真鳳": 64995,
2125
+ "眼淚": 64410,
2126
+ "眼見": 65216,
2127
+ "眼鏡": 64533,
2128
+ "眾人": 64107,
2129
+ "眾多": 65959,
2130
+ "着我的": 65809,
2131
+ "着頭": 64780,
2132
+ "睇": 66310,
2133
+ "睇下": 64162,
2134
+ "睇住": 65715,
2135
+ "睇到": 64424,
2136
+ "睇嚟": 64769,
2137
+ "睜": 66516,
2138
+ "睜開": 66044,
2139
+ "睡覺": 65108,
2140
+ "瞇": 66749,
2141
+ "瞓": 66343,
2142
+ "瞞": 66660,
2143
+ "瞥": 66789,
2144
+ "瞬間": 64109,
2145
+ "石頭": 65364,
2146
+ "砰": 66493,
2147
+ "砲": 66946,
2148
+ "破壞": 64366,
2149
+ "確保": 65029,
2150
+ "確定": 64651,
2151
+ "確實": 64385,
2152
+ "確認": 64599,
2153
+ "碼頭": 65043,
2154
+ "磚": 66770,
2155
+ "磡": 67040,
2156
+ "礙": 66490,
2157
+ "礦": 66701,
2158
+ "社長": 65375,
2159
+ "祇": 67177,
2160
+ "祐": 67070,
2161
+ "祕": 67097,
2162
+ "神情": 65616,
2163
+ "神經": 64655,
2164
+ "神色": 65882,
2165
+ "祟": 67094,
2166
+ "祿": 66859,
2167
+ "禍": 66617,
2168
+ "禦": 66601,
2169
+ "禪": 67025,
2170
+ "禮拜": 64925,
2171
+ "禮物": 64413,
2172
+ "禮貌": 65315,
2173
+ "禱": 66864,
2174
+ "禿": 67044,
2175
+ "秋兒": 65418,
2176
+ "秘書": 66010,
2177
+ "移動": 64643,
2178
+ "稍為": 66040,
2179
+ "稟": 67226,
2180
+ "種族": 65849,
2181
+ "稱呼": 65483,
2182
+ "穀": 67016,
2183
+ "穌": 66644,
2184
+ "穎": 66430,
2185
+ "穢": 67206,
2186
+ "穫": 67221,
2187
+ "穹": 67165,
2188
+ "空氣": 64356,
2189
+ "穿過": 65228,
2190
+ "突然間": 65708,
2191
+ "窩": 66585,
2192
+ "窮": 66541,
2193
+ "窺": 66872,
2194
+ "窿": 66887,
2195
+ "竄": 66961,
2196
+ "竅": 67105,
2197
+ "竇": 66711,
2198
+ "竊": 66621,
2199
+ "立場": 65360,
2200
+ "立時": 65137,
2201
+ "立法會": 64992,
2202
+ "站起來": 64675,
2203
+ "笑了笑": 65324,
2204
+ "笑着說": 65976,
2205
+ "笑聲": 65201,
2206
+ "笑說": 64858,
2207
+ "笠": 66967,
2208
+ "第一個": 64471,
2209
+ "第二個": 65356,
2210
+ "筆記": 65583,
2211
+ "等於": 65812,
2212
+ "答應": 64487,
2213
+ "答道": 66015,
2214
+ "算係": 66080,
2215
+ "算啦": 65381,
2216
+ "節奏": 66104,
2217
+ "範圍": 64331,
2218
+ "篤": 66807,
2219
+ "簡直": 64537,
2220
+ "簾": 66705,
2221
+ "籃": 66421,
2222
+ "籃球": 64509,
2223
+ "籠": 66599,
2224
+ "籤": 66766,
2225
+ "籬": 66876,
2226
+ "籲": 66693,
2227
+ "粉絲": 65669,
2228
+ "粵": 66349,
2229
+ "粵語": 64524,
2230
+ "精靈": 64481,
2231
+ "糧": 66619,
2232
+ "糾": 66681,
2233
+ "紀京": 66025,
2234
+ "紀念": 64984,
2235
+ "約定": 65892,
2236
+ "約會": 65411,
2237
+ "紅的": 66225,
2238
+ "紅色": 64323,
2239
+ "紅色的": 65318,
2240
+ "紋": 66454,
2241
+ "紐約": 65851,
2242
+ "純粹": 65444,
2243
+ "紗": 66637,
2244
+ "紙巾": 66086,
2245
+ "級的": 65555,
2246
+ "紛紛": 64757,
2247
+ "紮": 66643,
2248
+ "細佬": 65252,
2249
+ "細個": 66180,
2250
+ "細心": 65640,
2251
+ "細節": 65522,
2252
+ "細細": 65504,
2253
+ "細胞": 65172,
2254
+ "細路": 64775,
2255
+ "紳": 67057,
2256
+ "終於": 64020,
2257
+ "組合": 64878,
2258
+ "組成": 64657,
2259
+ "絆": 67083,
2260
+ "結婚": 64345,
2261
+ "結局": 65186,
2262
+ "結界": 65492,
2263
+ "絕對": 64097,
2264
+ "絕望": 65314,
2265
+ "絢": 67159,
2266
+ "給予": 64948,
2267
+ "給他": 64834,
2268
+ "給你": 64470,
2269
+ "給她": 64972,
2270
+ "給我": 64117,
2271
+ "絨": 67101,
2272
+ "統一": 66295,
2273
+ "絲毫": 65940,
2274
+ "綁": 66519,
2275
+ "經典": 64687,
2276
+ "經理": 64674,
2277
+ "綠色": 64949,
2278
+ "綫": 66394,
2279
+ "維護": 65779,
2280
+ "綱": 66762,
2281
+ "網上": 64414,
2282
+ "網友": 65667,
2283
+ "網民": 64506,
2284
+ "網絡": 64358,
2285
+ "網頁": 64141,
2286
+ "綴": 67046,
2287
+ "綺": 66875,
2288
+ "綻": 66724,
2289
+ "綽": 66965,
2290
+ "綾": 66796,
2291
+ "綿": 66610,
2292
+ "緊張": 64128,
2293
+ "緊急": 65541,
2294
+ "緊緊": 64708,
2295
+ "緊要": 64438,
2296
+ "緋": 66998,
2297
+ "線索": 66006,
2298
+ "緝": 66978,
2299
+ "編輯": 65551,
2300
+ "緩緩": 64350,
2301
+ "緬": 66917,
2302
+ "緯": 67096,
2303
+ "練習": 64627,
2304
+ "緻": 66645,
2305
+ "縛": 66884,
2306
+ "縫": 66604,
2307
+ "縱": 66453,
2308
+ "縷": 67107,
2309
+ "總之": 64499,
2310
+ "總是": 64275,
2311
+ "總會": 64913,
2312
+ "總理": 65745,
2313
+ "總算": 65125,
2314
+ "總部": 65261,
2315
+ "繃": 66830,
2316
+ "繆": 67218,
2317
+ "繞": 66460,
2318
+ "繡": 67060,
2319
+ "繩": 66540,
2320
+ "繪": 66689,
2321
+ "繫": 66592,
2322
+ "繳": 67131,
2323
+ "繹": 66950,
2324
+ "繼承": 66200,
2325
+ "繼續說": 65911,
2326
+ "繽": 67115,
2327
+ "纏": 66486,
2328
+ "纖": 66697,
2329
+ "纜": 67045,
2330
+ "置信": 66271,
2331
+ "罵": 66426,
2332
+ "罷": 66355,
2333
+ "罷���": 65274,
2334
+ "羅斯": 64467,
2335
+ "羅馬": 65508,
2336
+ "美麗": 64877,
2337
+ "羚": 67042,
2338
+ "羣": 66932,
2339
+ "羨": 66738,
2340
+ "羲": 66850,
2341
+ "翌": 66813,
2342
+ "習近平": 65490,
2343
+ "翹": 66499,
2344
+ "老實": 65397,
2345
+ "老母": 65039,
2346
+ "老細": 65663,
2347
+ "老豆": 64483,
2348
+ "老闆": 64179,
2349
+ "老頭": 66268,
2350
+ "考試": 64676,
2351
+ "而來": 65061,
2352
+ "而係": 64813,
2353
+ "而家": 64031,
2354
+ "耐煩": 65954,
2355
+ "耳機": 65308,
2356
+ "耳邊": 64589,
2357
+ "耶穌": 65152,
2358
+ "耷": 67174,
2359
+ "聖誕": 64389,
2360
+ "聞言": 65670,
2361
+ "聯合國": 66288,
2362
+ "聯絡": 64441,
2363
+ "聯繫": 66043,
2364
+ "聯賽": 65474,
2365
+ "聯邦": 65335,
2366
+ "聰": 66456,
2367
+ "聰明": 64882,
2368
+ "聲明": 65562,
2369
+ "聲線": 64865,
2370
+ "聲音": 64015,
2371
+ "聲響": 65977,
2372
+ "聳": 66670,
2373
+ "聶": 67121,
2374
+ "職員": 64824,
2375
+ "聽到": 64014,
2376
+ "聽完": 65568,
2377
+ "聽得": 65320,
2378
+ "聽日": 64497,
2379
+ "聽着": 65594,
2380
+ "聽見": 64745,
2381
+ "聽說": 65834,
2382
+ "聽過": 64814,
2383
+ "聾": 67125,
2384
+ "肅": 66612,
2385
+ "肌膚": 66007,
2386
+ "背後": 64156,
2387
+ "胸口": 64872,
2388
+ "脅": 66545,
2389
+ "脫離": 66283,
2390
+ "脷": 67050,
2391
+ "脹": 66685,
2392
+ "脾氣": 65927,
2393
+ "腎": 67054,
2394
+ "腦海": 64297,
2395
+ "腦海中": 65638,
2396
+ "腦袋": 64521,
2397
+ "腩": 67067,
2398
+ "腫": 66740,
2399
+ "腮": 67039,
2400
+ "腰間": 65742,
2401
+ "腳步": 64316,
2402
+ "腳步聲": 66058,
2403
+ "腸": 66552,
2404
+ "膊頭": 65343,
2405
+ "膚": 66401,
2406
+ "膛": 66814,
2407
+ "膠": 66363,
2408
+ "膩": 66931,
2409
+ "膽": 66393,
2410
+ "臉上": 64166,
2411
+ "臉色": 65138,
2412
+ "臉頰": 65279,
2413
+ "臘": 66699,
2414
+ "臟": 66496,
2415
+ "臥": 66756,
2416
+ "臨時": 65467,
2417
+ "自動": 64571,
2418
+ "自己嘅": 64287,
2419
+ "自從": 64501,
2420
+ "自殺": 64611,
2421
+ "自語": 65235,
2422
+ "與他": 66245,
2423
+ "與我": 66166,
2424
+ "興奮": 64229,
2425
+ "興趣": 64177,
2426
+ "舉動": 65166,
2427
+ "舉起": 64646,
2428
+ "舌頭": 66188,
2429
+ "舖": 66526,
2430
+ "艙": 66790,
2431
+ "艦": 66512,
2432
+ "艱": 66781,
2433
+ "艷": 66671,
2434
+ "芊": 66661,
2435
+ "花園": 64871,
2436
+ "若果": 65213,
2437
+ "苦笑": 65130,
2438
+ "英語": 66192,
2439
+ "茗": 66804,
2440
+ "茱": 67069,
2441
+ "茲": 66625,
2442
+ "荃": 66730,
2443
+ "荊": 66829,
2444
+ "莉絲": 65338,
2445
+ "莉莉": 66170,
2446
+ "莖": 67195,
2447
+ "莘": 66831,
2448
+ "萊": 66424,
2449
+ "萬一": 66011,
2450
+ "萬人": 65107,
2451
+ "落去": 64226,
2452
+ "落嚟": 64514,
2453
+ "著他": 65913,
2454
+ "著我": 65232,
2455
+ "著的": 65102,
2456
+ "葛亦": 64720,
2457
+ "葛亦民": 64779,
2458
+ "蒼": 66520,
2459
+ "蓓": 67004,
2460
+ "蔔": 67179,
2461
+ "蔣": 66561,
2462
+ "蔥": 66878,
2463
+ "蔭": 66989,
2464
+ "蕃": 67026,
2465
+ "蕎": 67205,
2466
+ "蕙": 67189,
2467
+ "蕩": 66598,
2468
+ "蕭": 66502,
2469
+ "薑": 67028,
2470
+ "薰": 66945,
2471
+ "藉口": 66112,
2472
+ "藍色": 64755,
2473
+ "藝人": 64702,
2474
+ "藹": 67013,
2475
+ "蘆": 67029,
2476
+ "蘊": 66858,
2477
+ "蘿": 66647,
2478
+ "處於": 64911,
2479
+ "號碼": 65217,
2480
+ "虧": 66696,
2481
+ "蜓": 67154,
2482
+ "蜥": 66907,
2483
+ "蜻": 67132,
2484
+ "蝕": 66690,
2485
+ "蝙": 66909,
2486
+ "蝠": 66771,
2487
+ "蝦": 66590,
2488
+ "螂": 67110,
2489
+ "螞": 67211,
2490
+ "螢": 66518,
2491
+ "螢幕": 65533,
2492
+ "蟬": 67087,
2493
+ "蟲": 66380,
2494
+ "蟻": 66773,
2495
+ "蠅": 67143,
2496
+ "蠍": 66962,
2497
+ "蠟": 66822,
2498
+ "蠱": 66899,
2499
+ "蠻": 66695,
2500
+ "行到": 64963,
2501
+ "行政區": 65912,
2502
+ "行政長官": 65734,
2503
+ "街頭": 66230,
2504
+ "衝動": 65476,
2505
+ "衝擊": 65032,
2506
+ "衝突": 65271,
2507
+ "衞": 66597,
2508
+ "衣人": 65126,
2509
+ "被我": 65231,
2510
+ "裏的": 64288,
2511
+ "裏面": 64106,
2512
+ "裏頭": 65639,
2513
+ "補充": 65076,
2514
+ "補習": 65830,
2515
+ "裝作": 65801,
2516
+ "裝備": 65160,
2517
+ "裝甲": 66093,
2518
+ "裝置": 65494,
2519
+ "複雜": 64677,
2520
+ "褲": 66366,
2521
+ "褸": 66888,
2522
+ "襪": 66949,
2523
+ "襯": 66535,
2524
+ "襲": 66350,
2525
+ "襲擊": 64710,
2526
+ "西亞": 64653,
2527
+ "西裝": 64770,
2528
+ "要我": 65013,
2529
+ "覆蓋": 66212,
2530
+ "見你": 65625,
2531
+ "見到": 64010,
2532
+ "見到佢": 65358,
2533
+ "見我": 65725,
2534
+ "見狀": 65014,
2535
+ "見過": 64221,
2536
+ "見面": 64398,
2537
+ "規則": 65153,
2538
+ "覓": 66938,
2539
+ "視線": 64291,
2540
+ "視覺": 66020,
2541
+ "親自": 64967,
2542
+ "覷": 67128,
2543
+ "覺得好": 65915,
2544
+ "覺得自己": 64951,
2545
+ "觀看": 66061,
2546
+ "解開": 65934,
2547
+ "言論": 65714,
2548
+ "訂閱": 64735,
2549
+ "訂閱讀者": 65127,
2550
+ "訂閱讀者優先": 65361,
2551
+ "計算": 65171,
2552
+ "訊息": 64313,
2553
+ "訊號": 65925,
2554
+ "討厭": 64576,
2555
+ "訕": 67164,
2556
+ "託": 66514,
2557
+ "記住": 64879,
2558
+ "記得": 64036,
2559
+ "記憶": 64205,
2560
+ "記錄": 65068,
2561
+ "訝": 66379,
2562
+ "訟": 67199,
2563
+ "訣": 66731,
2564
+ "訪問": 64685,
2565
+ "設定": 65943,
2566
+ "設施": 64821,
2567
+ "設立": 66289,
2568
+ "証": 66883,
2569
+ "詐": 66758,
2570
+ "評論": 66037,
2571
+ "詛": 66911,
2572
+ "詠": 66416,
2573
+ "詢問": 65804,
2574
+ "試下": 64943,
2575
+ "試圖": 65588,
2576
+ "試試": 66098,
2577
+ "試過": 64551,
2578
+ "詫": 66900,
2579
+ "詭": 66577,
2580
+ "詭異": 65161,
2581
+ "話係": 66059,
2582
+ "話要": 66197,
2583
+ "話說": 65222,
2584
+ "話題": 64544,
2585
+ "詳細": 65671,
2586
+ "誅": 67187,
2587
+ "誇": 66581,
2588
+ "誇張": 65337,
2589
+ "認同": 65091,
2590
+ "認真": 64197,
2591
+ "認知": 66254,
2592
+ "誒": 66953,
2593
+ "誕": 66409,
2594
+ "誘": 66584,
2595
+ "語氣": 64254,
2596
+ "誤會": 64990,
2597
+ "說不": 64349,
2598
+ "說不定": 65493,
2599
+ "說了": 64527,
2600
+ "說什麼": 65223,
2601
+ "說出": 64701,
2602
+ "說到": 65454,
2603
+ "說完": 64419,
2604
+ "說得": 65373,
2605
+ "說是": 64411,
2606
+ "說的": 64461,
2607
+ "說着": 64418,
2608
+ "說罷": 65003,
2609
+ "說話": 64023,
2610
+ "說過": 64556,
2611
+ "說道": 64041,
2612
+ "説": 66489,
2613
+ "課室": 64465,
2614
+ "誼": 66759,
2615
+ "請你": 65131,
2616
+ "請問": 65299,
2617
+ "諒": 66591,
2618
+ "諗": 66315,
2619
+ "諗下": 66081,
2620
+ "諗住": 64432,
2621
+ "諗到": 65560,
2622
+ "諗起": 65501,
2623
+ "諗過": 65240,
2624
+ "諜": 66995,
2625
+ "諧": 66943,
2626
+ "諭": 66914,
2627
+ "諮": 66786,
2628
+ "諷": 66653,
2629
+ "謀": 66445,
2630
+ "謊": 66595,
2631
+ "謎": 66716,
2632
+ "謙": 66467,
2633
+ "講到": 64748,
2634
+ "講咗": 65089,
2635
+ "講嘅": 65624,
2636
+ "講嘢": 64562,
2637
+ "講完": 64802,
2638
+ "講過": 65080,
2639
+ "謝謝": 64427,
2640
+ "謝謝你": 66153,
2641
+ "謠": 67062,
2642
+ "謬": 66844,
2643
+ "謹": 66713,
2644
+ "證實": 66094,
2645
+ "證據": 65123,
2646
+ "證明": 64378,
2647
+ "識得": 66158,
2648
+ "譚": 66480,
2649
+ "譚默": 65462,
2650
+ "譜": 66683,
2651
+ "警員": 65115,
2652
+ "議會": 64666,
2653
+ "譴": 67047,
2654
+ "護士": 65208,
2655
+ "讀書": 64546,
2656
+ "讀者": 64294,
2657
+ "變咗": 65085,
2658
+ "變得": 64044,
2659
+ "變態": 65025,
2660
+ "變成了": 65531,
2661
+ "讓他": 64341,
2662
+ "讓他們": 65540,
2663
+ "讓你": 64941,
2664
+ "讓她": 64610,
2665
+ "讓我": 64122,
2666
+ "讓我們": 65517,
2667
+ "讓自己": 65870,
2668
+ "讚": 66443,
2669
+ "豈": 66611,
2670
+ "豎": 66795,
2671
+ "豔": 66820,
2672
+ "豬": 66362,
2673
+ "貓": 66338,
2674
+ "貞": 66688,
2675
+ "財政": 65428,
2676
+ "貢獻": 66157,
2677
+ "販": 66744,
2678
+ "貪": 66568,
2679
+ "貫": 66521,
2680
+ "貴族": 65723,
2681
+ "貶": 67169,
2682
+ "貸": 66774,
2683
+ "賀": 66608,
2684
+ "資料來源": 64987,
2685
+ "資格": 64587,
2686
+ "賈": 66834,
2687
+ "賊": 66534,
2688
+ "賜": 66668,
2689
+ "賠": 66750,
2690
+ "賤": 66634,
2691
+ "賦": 66702,
2692
+ "質疑": 65164,
2693
+ "質素": 65974,
2694
+ "賬": 67106,
2695
+ "賭": 66472,
2696
+ "賺": 66542,
2697
+ "購買": 65507,
2698
+ "賽事": 65146,
2699
+ "贈": 66772,
2700
+ "贊": 66649,
2701
+ "贏": 66375,
2702
+ "贖": 66971,
2703
+ "走出來": 66228,
2704
+ "走吧": 65602,
2705
+ "走咗": 64535,
2706
+ "走啦": 65817,
2707
+ "走進": 64333,
2708
+ "走過": 65880,
2709
+ "走過來": 65788,
2710
+ "起嚟": 66253,
2711
+ "起碼": 65165,
2712
+ "超級": 64612,
2713
+ "越來": 64416,
2714
+ "越來越": 64425,
2715
+ "越嚟越": 66089,
2716
+ "趙": 66403,
2717
+ "足夠": 64393,
2718
+ "足球員": 64543,
2719
+ "距離": 64099,
2720
+ "跟住": 64165,
2721
+ "跟我說": 65341,
2722
+ "跟著": 64403,
2723
+ "跟隨": 65312,
2724
+ "跡": 66398,
2725
+ "路線": 64772,
2726
+ "踏入": 66001,
2727
+ "踐": 66969,
2728
+ "蹟": 66712,
2729
+ "蹤": 66399,
2730
+ "蹬": 67215,
2731
+ "躍": 66440,
2732
+ "身前": 65840,
2733
+ "身形": 65941,
2734
+ "身後": 64131,
2735
+ "身後的": 66068,
2736
+ "身為": 64661,
2737
+ "身處": 64766,
2738
+ "身軀": 65097,
2739
+ "身邊": 64058,
2740
+ "躲在": 66165,
2741
+ "軀": 66523,
2742
+ "車廂": 65818,
2743
+ "車站": 64494,
2744
+ "車輛": 66296,
2745
+ "軌": 66509,
2746
+ "軍事": 65183,
2747
+ "軍人": 65905,
2748
+ "軍隊": 65116,
2749
+ "軒": 66356,
2750
+ "軸": 66778,
2751
+ "輕易": 64868,
2752
+ "輕聲": 65414,
2753
+ "輕輕": 64105,
2754
+ "輕鬆": 64434,
2755
+ "輝": 66358,
2756
+ "輩": 66418,
2757
+ "輩子": 66147,
2758
+ "輪迴": 66073,
2759
+ "輸入": 65112,
2760
+ "輻": 67196,
2761
+ "輾": 67034,
2762
+ "轄": 66700,
2763
+ "轉向": 66177,
2764
+ "轉眼": 66016,
2765
+ "轉移": 65046,
2766
+ "轉變": 66172,
2767
+ "轉身": 64121,
2768
+ "轉頭": 64460,
2769
+ "轟": 66400,
2770
+ "辦公": 64541,
2771
+ "辦公室": 64751,
2772
+ "辮": 67223,
2773
+ "辯": 66566,
2774
+ "返個": 66280,
2775
+ "返到": 64466,
2776
+ "返去": 64248,
2777
+ "返嚟": 64175,
2778
+ "返學": 65303,
2779
+ "返屋企": 64475,
2780
+ "返工": 64851,
2781
+ "迴": 66433,
2782
+ "追問": 65290,
2783
+ "退後": 66219,
2784
+ "逃走": 65450,
2785
+ "逐漸": 64628,
2786
+ "逕": 67079,
2787
+ "這一切": 65992,
2788
+ "這一刻": 65780,
2789
+ "這不是": 65837,
2790
+ "這件事": 64810,
2791
+ "這份": 65944,
2792
+ "這位": 64493,
2793
+ "這個世界": 66085,
2794
+ "這個人": 65521,
2795
+ "這句": 65861,
2796
+ "這句話": 65221,
2797
+ "這場": 65536,
2798
+ "這就是": 64846,
2799
+ "這才": 65457,
2800
+ "這時": 64505,
2801
+ "這條": 65877,
2802
+ "這樣做": 65189,
2803
+ "這樣說": 65286,
2804
+ "這段": 65919,
2805
+ "這裏": 64019,
2806
+ "這裏的": 66127,
2807
+ "這邊": 65179,
2808
+ "這麼多": 65529,
2809
+ "通訊": 65518,
2810
+ "逞": 67082,
2811
+ "連忙": 64368,
2812
+ "連接": 65140,
2813
+ "連結": 65537,
2814
+ "連續": 64785,
2815
+ "進了": 65100,
2816
+ "進來": 64608,
2817
+ "進去": 64421,
2818
+ "進攻": 65074,
2819
+ "進步": 65456,
2820
+ "運作": 64956,
2821
+ "運動員": 66190,
2822
+ "運用": 65587,
2823
+ "運輸": 65194,
2824
+ "過了": 64138,
2825
+ "過來": 64039,
2826
+ "過咗": 64732,
2827
+ "過嘅": 65984,
2828
+ "過嚟": 64251,
2829
+ "過往": 65703,
2830
+ "過後": 64442,
2831
+ "過得": 66250,
2832
+ "過我": 64787,
2833
+ "過的": 64370,
2834
+ "過神": 65720,
2835
+ "過身": 65814,
2836
+ "過頭": 64849,
2837
+ "道別": 66207,
2838
+ "達成": 66139,
2839
+ "違反": 65964,
2840
+ "遙": 66434,
2841
+ "遜": 66691,
2842
+ "遞": 66419,
2843
+ "遠處": 64395,
2844
+ "遠遠": 66130,
2845
+ "適合": 64456,
2846
+ "適應": 65572,
2847
+ "遲": 66337,
2848
+ "遲到": 65645,
2849
+ "遺憾": 65841,
2850
+ "遼": 66942,
2851
+ "避開": 64826,
2852
+ "邁": 66640,
2853
+ "還不": 65835,
2854
+ "還可以": 66216,
2855
+ "還在": 64308,
2856
+ "還好": 65689,
2857
+ "還會": 65858,
2858
+ "還有一": 65325,
2859
+ "還未": 64285,
2860
+ "還沒": 64670,
2861
+ "還沒有": 65058,
2862
+ "還能": 65525,
2863
+ "還要": 64223,
2864
+ "邊個": 64123,
2865
+ "邊度": 64713,
2866
+ "邊的": 64798,
2867
+ "邊緣": 66053,
2868
+ "邏": 66684,
2869
+ "那個": 64013,
2870
+ "那個人": 65372,
2871
+ "那名": 66286,
2872
+ "那時": 64819,
2873
+ "那時候": 66113,
2874
+ "那樣": 64669,
2875
+ "那種": 64474,
2876
+ "那裏": 64232,
2877
+ "那邊": 64355,
2878
+ "那麼多": 66269,
2879
+ "邨": 66559,
2880
+ "部長": 65749,
2881
+ "部隊": 64727,
2882
+ "郵": 66548,
2883
+ "都不會": 65875,
2884
+ "都係": 64004,
2885
+ "都冇": 64143,
2886
+ "都唔": 64011,
2887
+ "都唔係": 64974,
2888
+ "都唔會": 64558,
2889
+ "都唔知": 64318,
2890
+ "都好": 64093,
2891
+ "都已經": 64728,
2892
+ "都幾": 64836,
2893
+ "都未": 64437,
2894
+ "都沒": 64944,
2895
+ "都沒有": 64214,
2896
+ "都無": 64173,
2897
+ "都話": 65694,
2898
+ "鄒": 67192,
2899
+ "鄢": 67209,
2900
+ "鄧": 66487,
2901
+ "鄭吒": 64950,
2902
+ "鄭月娥": 66246,
2903
+ "鄰": 66556,
2904
+ "鄺": 67010,
2905
+ "配音": 65746,
2906
+ "醒來": 64979,
2907
+ "醜": 66563,
2908
+ "醬": 66605,
2909
+ "釀": 66843,
2910
+ "釁": 66901,
2911
+ "釋放": 65589,
2912
+ "重複": 65805,
2913
+ "重視": 65491,
2914
+ "金屬": 64818,
2915
+ "金錢": 65716,
2916
+ "釘": 66675,
2917
+ "釣": 66898,
2918
+ "鈍": 67126,
2919
+ "鈔": 67168,
2920
+ "鈕": 66783,
2921
+ "鈞": 66777,
2922
+ "鈴": 66388,
2923
+ "鉛": 67059,
2924
+ "銅": 66515,
2925
+ "銘": 66483,
2926
+ "銜": 66934,
2927
+ "銳": 66576,
2928
+ "鋒": 66351,
2929
+ "鋪": 66465,
2930
+ "鋸": 67022,
2931
+ "錄音": 65747,
2932
+ "錐": 67095,
2933
+ "錘": 67051,
2934
+ "錦": 66555,
2935
+ "錫": 66508,
2936
+ "錯了": 65398,
2937
+ "錯誤": 65265,
2938
+ "錶": 66387,
2939
+ "鍊": 66632,
2940
+ "鍋": 66751,
2941
+ "鍛": 66923,
2942
+ "鍾": 66320,
2943
+ "鍾意": 64046,
2944
+ "鎊": 67148,
2945
+ "鎖": 66345,
2946
+ "鎖匙": 65907,
2947
+ "鎗": 67001,
2948
+ "鎚": 66980,
2949
+ "鎧": 66852,
2950
+ "鏗": 67073,
2951
+ "鏘": 67056,
2952
+ "鏟": 67093,
2953
+ "鏡頭": 65178,
2954
+ "鏢": 66851,
2955
+ "鐮": 67053,
2956
+ "鐵路": 65071,
2957
+ "鑄": 67037,
2958
+ "鑊": 66728,
2959
+ "鑑": 66867,
2960
+ "鑒": 67204,
2961
+ "鑣": 67112,
2962
+ "鑰": 66811,
2963
+ "鑲": 67005,
2964
+ "鑼": 66809,
2965
+ "鑽": 66506,
2966
+ "鑽石": 66067,
2967
+ "長大": 65289,
2968
+ "長官": 65158,
2969
+ "長的": 64351,
2970
+ "長老": 65990,
2971
+ "長髮": 65997,
2972
+ "門前": 64965,
2973
+ "門口": 64153,
2974
+ "門外": 64538,
2975
+ "門的": 66193,
2976
+ "閂": 66954,
2977
+ "閃": 66328,
2978
+ "閃過": 65732,
2979
+ "閉上": 65248,
2980
+ "開了": 64195,
2981
+ "開口": 64218,
2982
+ "開咗": 65646,
2983
+ "開啟": 65981,
2984
+ "開心": 64051,
2985
+ "開槍": 66114,
2986
+ "開玩笑": 66161,
2987
+ "開門": 64649,
2988
+ "閏": 66966,
2989
+ "閑": 67085,
2990
+ "閒": 66364,
2991
+ "間房": 65294,
2992
+ "閘": 66533,
2993
+ "閣": 66544,
2994
+ "閨": 67202,
2995
+ "閩": 67122,
2996
+ "閻": 66894,
2997
+ "闆": 66386,
2998
+ "闇": 67203,
2999
+ "闊": 66532,
3000
+ "闖": 66641,
3001
+ "關上": 65253,
3002
+ "關心": 64554,
3003
+ "阪": 67009,
3004
+ "防禦": 65292,
3005
+ "阿俊": 66100,
3006
+ "阿哥": 65820,
3007
+ "阿妹": 65495,
3008
+ "阿媽": 64450,
3009
+ "阿明": 65884,
3010
+ "阿爸": 65082,
3011
+ "阿爾": 65547,
3012
+ "陛": 66741,
3013
+ "陛下": 66030,
3014
+ "除咗": 64171,
3015
+ "陪我": 65515,
3016
+ "陳浩然": 64839,
3017
+ "陸續": 66195,
3018
+ "陽光": 64615,
3019
+ "隊伍": 65184,
3020
+ "隊友": 65441,
3021
+ "隊員": 65322,
3022
+ "隊的": 66222,
3023
+ "隊長": 64564,
3024
+ "隔離": 64526,
3025
+ "隨便": 64449,
3026
+ "隨即": 64170,
3027
+ "隨後": 65174,
3028
+ "隨意": 65826,
3029
+ "隨時": 64540,
3030
+ "隱約": 65340,
3031
+ "隱藏": 65148,
3032
+ "隸": 66735,
3033
+ "隻手": 64273,
3034
+ "雅克": 65040,
3035
+ "雙手": 64050,
3036
+ "雙目": 66017,
3037
+ "雙眼": 64111,
3038
+ "雙腳": 65188,
3039
+ "雙腿": 66056,
3040
+ "雜誌": 65585,
3041
+ "離去": 64699,
3042
+ "離開了": 64561,
3043
+ "難以": 64211,
3044
+ "難度": 65733,
3045
+ "難得": 64736,
3046
+ "難怪": 65264,
3047
+ "難道": 64149,
3048
+ "雪琪": 66029,
3049
+ "雪糕": 66028,
3050
+ "電台": 65403,
3051
+ "電梯": 65087,
3052
+ "電視劇": 64808,
3053
+ "電視台": 65748,
3054
+ "震動": 66005,
3055
+ "震驚": 65815,
3056
+ "霎": 66874,
3057
+ "霧": 66413,
3058
+ "靈力": 65626,
3059
+ "靈魂": 64296,
3060
+ "靚": 66339,
3061
+ "靚仔": 65601,
3062
+ "靚女": 65168,
3063
+ "靜靜": 64816,
3064
+ "面具": 65120,
3065
+ "面積": 64952,
3066
+ "面色": 65649,
3067
+ "鞏": 67130,
3068
+ "鞘": 67134,
3069
+ "韋": 66578,
3070
+ "韌": 67049,
3071
+ "韻": 66474,
3072
+ "響起": 64295,
3073
+ "頃": 66979,
3074
+ "順便": 65542,
3075
+ "順利": 64700,
3076
+ "頌": 66746,
3077
+ "預告": 66235,
3078
+ "預期": 64957,
3079
+ "預測": 65584,
3080
+ "頑": 66920,
3081
+ "頒": 66574,
3082
+ "頒獎": 65451,
3083
+ "頓時": 64255,
3084
+ "頗": 66603,
3085
+ "領袖": 65806,
3086
+ "頤": 67099,
3087
+ "頭上": 64656,
3088
+ "頭先": 64238,
3089
+ "頭的": 64725,
3090
+ "頭部": 66156,
3091
+ "頭頂": 65947,
3092
+ "頭髮": 64282,
3093
+ "頰": 66546,
3094
+ "頴": 66895,
3095
+ "頸": 66414,
3096
+ "頹": 66815,
3097
+ "頻道": 65595,
3098
+ "顆": 66459,
3099
+ "額頭": 65285,
3100
+ "顎": 67214,
3101
+ "顏": 66376,
3102
+ "顏色": 64832,
3103
+ "願望": 65803,
3104
+ "顛": 66779,
3105
+ "類似": 64825,
3106
+ "類型": 65471,
3107
+ "顥": 66848,
3108
+ "顫": 66497,
3109
+ "顫抖": 65124,
3110
+ "顯得": 64605,
3111
+ "顯然": 64898,
3112
+ "顱": 66754,
3113
+ "風俗": 65637,
3114
+ "風俗習慣": 65853,
3115
+ "風吹": 66256,
3116
+ "風格": 64644,
3117
+ "颱": 66794,
3118
+ "飄": 66461,
3119
+ "飆": 66991,
3120
+ "飛機": 64482,
3121
+ "飛行": 65212,
3122
+ "食嘢": 65786,
3123
+ "食完": 65352,
3124
+ "食飯": 64396,
3125
+ "飢": 67114,
3126
+ "飽": 66567,
3127
+ "飾演": 64847,
3128
+ "餅": 66550,
3129
+ "餐廳": 64212,
3130
+ "餓": 66494,
3131
+ "餵": 66793,
3132
+ "餸": 66840,
3133
+ "饒": 66787,
3134
+ "首領": 65527,
3135
+ "香港人": 64477,
3136
+ "香港的": 65811,
3137
+ "馬上": 64057,
3138
+ "馬來": 64754,
3139
+ "馬來西亞": 65287,
3140
+ "馬路": 64805,
3141
+ "馮": 66575,
3142
+ "馳": 66817,
3143
+ "馴": 67225,
3144
+ "駁": 66522,
3145
+ "駒": 66975,
3146
+ "駕駛": 64929,
3147
+ "駛": 66407,
3148
+ "駭": 66882,
3149
+ "駱": 66665,
3150
+ "駿": 66721,
3151
+ "騎士": 64638,
3152
+ "騙": 66441,
3153
+ "騰": 66485,
3154
+ "騷": 66600,
3155
+ "驀": 66944,
3156
+ "驅": 66513,
3157
+ "驕": 66862,
3158
+ "驚喜": 65266,
3159
+ "驚訝": 64213,
3160
+ "驟": 66635,
3161
+ "骰": 67184,
3162
+ "骷": 66951,
3163
+ "骸": 66916,
3164
+ "髀": 67146,
3165
+ "髏": 66968,
3166
+ "髒": 66821,
3167
+ "體內": 65006,
3168
+ "體力": 65519,
3169
+ "體育": 64553,
3170
+ "高層": 65843,
3171
+ "高級": 64912,
3172
+ "高興": 64420,
3173
+ "高達": 66105,
3174
+ "鬍": 66861,
3175
+ "鬚": 66845,
3176
+ "鬧": 66352,
3177
+ "鬱": 66698,
3178
+ "魔力": 64430,
3179
+ "魔法師": 66162,
3180
+ "魔物": 65764,
3181
+ "魔王": 65263,
3182
+ "鮑": 66985,
3183
+ "鮮血": 64733,
3184
+ "鯉": 67088,
3185
+ "鯊": 66937,
3186
+ "鯨": 66798,
3187
+ "鱗": 66984,
3188
+ "鱷": 66997,
3189
+ "鳥": 66405,
3190
+ "鳩": 66623,
3191
+ "鳳": 66361,
3192
+ "鳳凰": 65606,
3193
+ "鳴": 66505,
3194
+ "鴉": 66745,
3195
+ "鴨": 66719,
3196
+ "鴻": 66672,
3197
+ "鴿": 66948,
3198
+ "鵝": 66718,
3199
+ "鵠": 67091,
3200
+ "鵬": 66727,
3201
+ "鶴": 66881,
3202
+ "鷹": 66547,
3203
+ "鹹": 66747,
3204
+ "鹽": 66725,
3205
+ "麗絲": 65119,
3206
+ "麵": 66390,
3207
+ "麵包": 65459,
3208
+ "麻煩": 64215,
3209
+ "麽": 66579,
3210
+ "黃色": 65599,
3211
+ "黃金": 64800,
3212
+ "黐": 66800,
3213
+ "黑影": 66103,
3214
+ "點了": 64630,
3215
+ "點了點頭": 65106,
3216
+ "點呀": 64986,
3217
+ "點會": 64673,
3218
+ "點樣": 64384,
3219
+ "點知": 64336,
3220
+ "點算": 65355,
3221
+ "點解": 64016,
3222
+ "點解會": 65836,
3223
+ "點解要": 66066,
3224
+ "點頭": 64055,
3225
+ "點點頭": 64592,
3226
+ "黯": 66855,
3227
+ "齋": 66816,
3228
+ "齒": 66537,
3229
+ "齣": 67156,
3230
+ "龐": 66510,
3231
+ "龜": 66679
3232
+ }
checkpoint-400/config.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "hon9kon9ize/yi-1.5-6b-yub-vocab-expanded",
3
+ "architectures": [
4
+ "LlamaForCausalLM"
5
+ ],
6
+ "attention_bias": false,
7
+ "attention_dropout": 0.0,
8
+ "bos_token_id": 1,
9
+ "eos_token_id": 2,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 4096,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 11008,
14
+ "max_position_embeddings": 4096,
15
+ "model_type": "llama",
16
+ "num_attention_heads": 32,
17
+ "num_hidden_layers": 32,
18
+ "num_key_value_heads": 4,
19
+ "pad_token_id": 0,
20
+ "pretraining_tp": 1,
21
+ "rms_norm_eps": 1e-06,
22
+ "rope_scaling": null,
23
+ "rope_theta": 5000000.0,
24
+ "tie_word_embeddings": false,
25
+ "torch_dtype": "bfloat16",
26
+ "transformers_version": "4.40.2",
27
+ "use_cache": false,
28
+ "vocab_size": 67264
29
+ }
checkpoint-400/generation_config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 1,
4
+ "eos_token_id": 2,
5
+ "pad_token_id": 0,
6
+ "transformers_version": "4.40.2"
7
+ }
checkpoint-400/model-00001-of-00003.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4592ad6466a51e270851d6eeb4fa795bfdee3f68616110e0538ac9425c2fce59
3
+ size 4959450112
checkpoint-400/model-00002-of-00003.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:075f8006a67ffaf18f17240b4ac87d2bd82aac94a6aa74659416cd80ad8b7362
3
+ size 4976802816
checkpoint-400/model-00003-of-00003.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5df7be45282d79297a2fd594e100fa06c59742d379f40d5d119699bd799bd6a7
3
+ size 2239329256
checkpoint-400/model.safetensors.index.json ADDED
@@ -0,0 +1,298 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_size": 12175548416
4
+ },
5
+ "weight_map": {
6
+ "lm_head.weight": "model-00003-of-00003.safetensors",
7
+ "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
8
+ "model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
9
+ "model.layers.0.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
10
+ "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
11
+ "model.layers.0.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
12
+ "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
13
+ "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
14
+ "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
15
+ "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
16
+ "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
17
+ "model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
18
+ "model.layers.1.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
19
+ "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
20
+ "model.layers.1.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
21
+ "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
22
+ "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
23
+ "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
24
+ "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
25
+ "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
26
+ "model.layers.10.input_layernorm.weight": "model-00001-of-00003.safetensors",
27
+ "model.layers.10.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
28
+ "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
29
+ "model.layers.10.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
30
+ "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
31
+ "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
32
+ "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
33
+ "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
34
+ "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
35
+ "model.layers.11.input_layernorm.weight": "model-00001-of-00003.safetensors",
36
+ "model.layers.11.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
37
+ "model.layers.11.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
38
+ "model.layers.11.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
39
+ "model.layers.11.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
40
+ "model.layers.11.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
41
+ "model.layers.11.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
42
+ "model.layers.11.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
43
+ "model.layers.11.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
44
+ "model.layers.12.input_layernorm.weight": "model-00002-of-00003.safetensors",
45
+ "model.layers.12.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
46
+ "model.layers.12.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
47
+ "model.layers.12.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
48
+ "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
49
+ "model.layers.12.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
50
+ "model.layers.12.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
51
+ "model.layers.12.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
52
+ "model.layers.12.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
53
+ "model.layers.13.input_layernorm.weight": "model-00002-of-00003.safetensors",
54
+ "model.layers.13.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
55
+ "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
56
+ "model.layers.13.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
57
+ "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
58
+ "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
59
+ "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
60
+ "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
61
+ "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
62
+ "model.layers.14.input_layernorm.weight": "model-00002-of-00003.safetensors",
63
+ "model.layers.14.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
64
+ "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
65
+ "model.layers.14.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
66
+ "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
67
+ "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
68
+ "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
69
+ "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
70
+ "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
71
+ "model.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
72
+ "model.layers.15.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
73
+ "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
74
+ "model.layers.15.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
75
+ "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
76
+ "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
77
+ "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
78
+ "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
79
+ "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
80
+ "model.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
81
+ "model.layers.16.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
82
+ "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
83
+ "model.layers.16.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
84
+ "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
85
+ "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
86
+ "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
87
+ "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
88
+ "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
89
+ "model.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
90
+ "model.layers.17.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
91
+ "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
92
+ "model.layers.17.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
93
+ "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
94
+ "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
95
+ "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
96
+ "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
97
+ "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
98
+ "model.layers.18.input_layernorm.weight": "model-00002-of-00003.safetensors",
99
+ "model.layers.18.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
100
+ "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
101
+ "model.layers.18.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
102
+ "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
103
+ "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
104
+ "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
105
+ "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
106
+ "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
107
+ "model.layers.19.input_layernorm.weight": "model-00002-of-00003.safetensors",
108
+ "model.layers.19.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
109
+ "model.layers.19.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
110
+ "model.layers.19.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
111
+ "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
112
+ "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
113
+ "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
114
+ "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
115
+ "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
116
+ "model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
117
+ "model.layers.2.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
118
+ "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
119
+ "model.layers.2.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
120
+ "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
121
+ "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
122
+ "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
123
+ "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
124
+ "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
125
+ "model.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
126
+ "model.layers.20.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
127
+ "model.layers.20.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
128
+ "model.layers.20.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
129
+ "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
130
+ "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
131
+ "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
132
+ "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
133
+ "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
134
+ "model.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
135
+ "model.layers.21.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
136
+ "model.layers.21.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
137
+ "model.layers.21.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
138
+ "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
139
+ "model.layers.21.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
140
+ "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
141
+ "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
142
+ "model.layers.21.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
143
+ "model.layers.22.input_layernorm.weight": "model-00002-of-00003.safetensors",
144
+ "model.layers.22.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
145
+ "model.layers.22.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
146
+ "model.layers.22.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
147
+ "model.layers.22.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
148
+ "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
149
+ "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
150
+ "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
151
+ "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
152
+ "model.layers.23.input_layernorm.weight": "model-00002-of-00003.safetensors",
153
+ "model.layers.23.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
154
+ "model.layers.23.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
155
+ "model.layers.23.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
156
+ "model.layers.23.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
157
+ "model.layers.23.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
158
+ "model.layers.23.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
159
+ "model.layers.23.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
160
+ "model.layers.23.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
161
+ "model.layers.24.input_layernorm.weight": "model-00002-of-00003.safetensors",
162
+ "model.layers.24.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
163
+ "model.layers.24.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
164
+ "model.layers.24.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
165
+ "model.layers.24.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
166
+ "model.layers.24.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
167
+ "model.layers.24.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
168
+ "model.layers.24.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
169
+ "model.layers.24.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
170
+ "model.layers.25.input_layernorm.weight": "model-00002-of-00003.safetensors",
171
+ "model.layers.25.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
172
+ "model.layers.25.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
173
+ "model.layers.25.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
174
+ "model.layers.25.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
175
+ "model.layers.25.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
176
+ "model.layers.25.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
177
+ "model.layers.25.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
178
+ "model.layers.25.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
179
+ "model.layers.26.input_layernorm.weight": "model-00002-of-00003.safetensors",
180
+ "model.layers.26.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
181
+ "model.layers.26.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
182
+ "model.layers.26.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
183
+ "model.layers.26.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
184
+ "model.layers.26.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
185
+ "model.layers.26.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
186
+ "model.layers.26.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
187
+ "model.layers.26.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
188
+ "model.layers.27.input_layernorm.weight": "model-00003-of-00003.safetensors",
189
+ "model.layers.27.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
190
+ "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
191
+ "model.layers.27.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
192
+ "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
193
+ "model.layers.27.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
194
+ "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
195
+ "model.layers.27.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
196
+ "model.layers.27.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
197
+ "model.layers.28.input_layernorm.weight": "model-00003-of-00003.safetensors",
198
+ "model.layers.28.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
199
+ "model.layers.28.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
200
+ "model.layers.28.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
201
+ "model.layers.28.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
202
+ "model.layers.28.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
203
+ "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
204
+ "model.layers.28.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
205
+ "model.layers.28.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
206
+ "model.layers.29.input_layernorm.weight": "model-00003-of-00003.safetensors",
207
+ "model.layers.29.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
208
+ "model.layers.29.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
209
+ "model.layers.29.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
210
+ "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
211
+ "model.layers.29.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
212
+ "model.layers.29.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
213
+ "model.layers.29.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
214
+ "model.layers.29.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
215
+ "model.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
216
+ "model.layers.3.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
217
+ "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
218
+ "model.layers.3.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
219
+ "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
220
+ "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
221
+ "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
222
+ "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
223
+ "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
224
+ "model.layers.30.input_layernorm.weight": "model-00003-of-00003.safetensors",
225
+ "model.layers.30.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
226
+ "model.layers.30.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
227
+ "model.layers.30.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
228
+ "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
229
+ "model.layers.30.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
230
+ "model.layers.30.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
231
+ "model.layers.30.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
232
+ "model.layers.30.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
233
+ "model.layers.31.input_layernorm.weight": "model-00003-of-00003.safetensors",
234
+ "model.layers.31.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
235
+ "model.layers.31.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
236
+ "model.layers.31.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
237
+ "model.layers.31.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
238
+ "model.layers.31.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
239
+ "model.layers.31.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
240
+ "model.layers.31.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
241
+ "model.layers.31.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
242
+ "model.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
243
+ "model.layers.4.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
244
+ "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
245
+ "model.layers.4.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
246
+ "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
247
+ "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
248
+ "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
249
+ "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
250
+ "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
251
+ "model.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
252
+ "model.layers.5.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
253
+ "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
254
+ "model.layers.5.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
255
+ "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
256
+ "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
257
+ "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
258
+ "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
259
+ "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
260
+ "model.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
261
+ "model.layers.6.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
262
+ "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
263
+ "model.layers.6.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
264
+ "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
265
+ "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
266
+ "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
267
+ "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
268
+ "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
269
+ "model.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
270
+ "model.layers.7.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
271
+ "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
272
+ "model.layers.7.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
273
+ "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
274
+ "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
275
+ "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
276
+ "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
277
+ "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
278
+ "model.layers.8.input_layernorm.weight": "model-00001-of-00003.safetensors",
279
+ "model.layers.8.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
280
+ "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
281
+ "model.layers.8.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
282
+ "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
283
+ "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
284
+ "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
285
+ "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
286
+ "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
287
+ "model.layers.9.input_layernorm.weight": "model-00001-of-00003.safetensors",
288
+ "model.layers.9.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
289
+ "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
290
+ "model.layers.9.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
291
+ "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
292
+ "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
293
+ "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
294
+ "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
295
+ "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
296
+ "model.norm.weight": "model-00003-of-00003.safetensors"
297
+ }
298
+ }
checkpoint-400/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8dcec3964548ca1973fbfdc2d6896d0d995c26a070c0e25e1044610c5837dcc
3
+ size 24351351162
checkpoint-400/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:386fcc8cc1089aade9450d86fb239ea3483f455fd2d78d8378645feecfec9d69
3
+ size 14244
checkpoint-400/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7b0c6bc12d12de3f863924e496a6cdacb77547187243d9d71ed5386cad916673
3
+ size 1064
checkpoint-400/special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<|startoftext|>",
4
+ "lstrip": false,
5
+ "normalized": true,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "<|endoftext|>",
11
+ "lstrip": false,
12
+ "normalized": true,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<unk>",
18
+ "lstrip": false,
19
+ "normalized": true,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": true,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
checkpoint-400/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-400/tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:386c49cf943d71aa110361135338c50e38beeff0a66593480421f37b319e1a39
3
+ size 1033105
checkpoint-400/tokenizer_config.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-400/trainer_state.json ADDED
@@ -0,0 +1,2821 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 0.6075564837668502,
5
+ "eval_steps": 500,
6
+ "global_step": 400,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0015188912094171254,
13
+ "grad_norm": 494.0,
14
+ "learning_rate": 5.000000000000001e-07,
15
+ "loss": 21.3162,
16
+ "step": 1
17
+ },
18
+ {
19
+ "epoch": 0.003037782418834251,
20
+ "grad_norm": 440.0,
21
+ "learning_rate": 1.0000000000000002e-06,
22
+ "loss": 19.5214,
23
+ "step": 2
24
+ },
25
+ {
26
+ "epoch": 0.004556673628251376,
27
+ "grad_norm": 462.0,
28
+ "learning_rate": 1.5e-06,
29
+ "loss": 20.3955,
30
+ "step": 3
31
+ },
32
+ {
33
+ "epoch": 0.006075564837668502,
34
+ "grad_norm": 446.0,
35
+ "learning_rate": 2.0000000000000003e-06,
36
+ "loss": 19.8595,
37
+ "step": 4
38
+ },
39
+ {
40
+ "epoch": 0.007594456047085627,
41
+ "grad_norm": 440.0,
42
+ "learning_rate": 2.5e-06,
43
+ "loss": 19.3269,
44
+ "step": 5
45
+ },
46
+ {
47
+ "epoch": 0.009113347256502752,
48
+ "grad_norm": 432.0,
49
+ "learning_rate": 3e-06,
50
+ "loss": 20.3605,
51
+ "step": 6
52
+ },
53
+ {
54
+ "epoch": 0.010632238465919878,
55
+ "grad_norm": 408.0,
56
+ "learning_rate": 3.5e-06,
57
+ "loss": 18.9548,
58
+ "step": 7
59
+ },
60
+ {
61
+ "epoch": 0.012151129675337003,
62
+ "grad_norm": 366.0,
63
+ "learning_rate": 4.000000000000001e-06,
64
+ "loss": 17.6873,
65
+ "step": 8
66
+ },
67
+ {
68
+ "epoch": 0.013670020884754129,
69
+ "grad_norm": 346.0,
70
+ "learning_rate": 4.5e-06,
71
+ "loss": 17.031,
72
+ "step": 9
73
+ },
74
+ {
75
+ "epoch": 0.015188912094171255,
76
+ "grad_norm": 284.0,
77
+ "learning_rate": 5e-06,
78
+ "loss": 15.499,
79
+ "step": 10
80
+ },
81
+ {
82
+ "epoch": 0.01670780330358838,
83
+ "grad_norm": 238.0,
84
+ "learning_rate": 5.500000000000001e-06,
85
+ "loss": 13.135,
86
+ "step": 11
87
+ },
88
+ {
89
+ "epoch": 0.018226694513005504,
90
+ "grad_norm": 248.0,
91
+ "learning_rate": 6e-06,
92
+ "loss": 12.58,
93
+ "step": 12
94
+ },
95
+ {
96
+ "epoch": 0.01974558572242263,
97
+ "grad_norm": 251.0,
98
+ "learning_rate": 6.5000000000000004e-06,
99
+ "loss": 10.6039,
100
+ "step": 13
101
+ },
102
+ {
103
+ "epoch": 0.021264476931839756,
104
+ "grad_norm": 180.0,
105
+ "learning_rate": 7e-06,
106
+ "loss": 9.4173,
107
+ "step": 14
108
+ },
109
+ {
110
+ "epoch": 0.022783368141256883,
111
+ "grad_norm": 108.5,
112
+ "learning_rate": 7.500000000000001e-06,
113
+ "loss": 8.5445,
114
+ "step": 15
115
+ },
116
+ {
117
+ "epoch": 0.024302259350674007,
118
+ "grad_norm": 81.5,
119
+ "learning_rate": 8.000000000000001e-06,
120
+ "loss": 8.3828,
121
+ "step": 16
122
+ },
123
+ {
124
+ "epoch": 0.025821150560091134,
125
+ "grad_norm": 67.0,
126
+ "learning_rate": 8.5e-06,
127
+ "loss": 7.3555,
128
+ "step": 17
129
+ },
130
+ {
131
+ "epoch": 0.027340041769508258,
132
+ "grad_norm": 46.25,
133
+ "learning_rate": 9e-06,
134
+ "loss": 6.5261,
135
+ "step": 18
136
+ },
137
+ {
138
+ "epoch": 0.028858932978925386,
139
+ "grad_norm": 62.5,
140
+ "learning_rate": 9.5e-06,
141
+ "loss": 6.4529,
142
+ "step": 19
143
+ },
144
+ {
145
+ "epoch": 0.03037782418834251,
146
+ "grad_norm": 57.25,
147
+ "learning_rate": 1e-05,
148
+ "loss": 6.1533,
149
+ "step": 20
150
+ },
151
+ {
152
+ "epoch": 0.03189671539775964,
153
+ "grad_norm": 36.75,
154
+ "learning_rate": 9.999939382570075e-06,
155
+ "loss": 5.6474,
156
+ "step": 21
157
+ },
158
+ {
159
+ "epoch": 0.03341560660717676,
160
+ "grad_norm": 20.375,
161
+ "learning_rate": 9.999757531750086e-06,
162
+ "loss": 5.379,
163
+ "step": 22
164
+ },
165
+ {
166
+ "epoch": 0.034934497816593885,
167
+ "grad_norm": 17.125,
168
+ "learning_rate": 9.999454451949364e-06,
169
+ "loss": 5.1134,
170
+ "step": 23
171
+ },
172
+ {
173
+ "epoch": 0.03645338902601101,
174
+ "grad_norm": 11.6875,
175
+ "learning_rate": 9.999030150516681e-06,
176
+ "loss": 4.8157,
177
+ "step": 24
178
+ },
179
+ {
180
+ "epoch": 0.03797228023542814,
181
+ "grad_norm": 10.1875,
182
+ "learning_rate": 9.998484637740058e-06,
183
+ "loss": 4.7658,
184
+ "step": 25
185
+ },
186
+ {
187
+ "epoch": 0.03949117144484526,
188
+ "grad_norm": 8.625,
189
+ "learning_rate": 9.997817926846528e-06,
190
+ "loss": 4.6094,
191
+ "step": 26
192
+ },
193
+ {
194
+ "epoch": 0.04101006265426239,
195
+ "grad_norm": 7.25,
196
+ "learning_rate": 9.997030034001815e-06,
197
+ "loss": 4.389,
198
+ "step": 27
199
+ },
200
+ {
201
+ "epoch": 0.04252895386367951,
202
+ "grad_norm": 6.78125,
203
+ "learning_rate": 9.99612097830993e-06,
204
+ "loss": 4.3941,
205
+ "step": 28
206
+ },
207
+ {
208
+ "epoch": 0.04404784507309664,
209
+ "grad_norm": 6.9375,
210
+ "learning_rate": 9.995090781812724e-06,
211
+ "loss": 4.3693,
212
+ "step": 29
213
+ },
214
+ {
215
+ "epoch": 0.045566736282513766,
216
+ "grad_norm": 6.84375,
217
+ "learning_rate": 9.993939469489342e-06,
218
+ "loss": 4.2591,
219
+ "step": 30
220
+ },
221
+ {
222
+ "epoch": 0.04708562749193089,
223
+ "grad_norm": 5.53125,
224
+ "learning_rate": 9.99266706925562e-06,
225
+ "loss": 4.2106,
226
+ "step": 31
227
+ },
228
+ {
229
+ "epoch": 0.048604518701348014,
230
+ "grad_norm": 5.09375,
231
+ "learning_rate": 9.991273611963413e-06,
232
+ "loss": 4.131,
233
+ "step": 32
234
+ },
235
+ {
236
+ "epoch": 0.050123409910765145,
237
+ "grad_norm": 6.09375,
238
+ "learning_rate": 9.98975913139984e-06,
239
+ "loss": 4.0936,
240
+ "step": 33
241
+ },
242
+ {
243
+ "epoch": 0.05164230112018227,
244
+ "grad_norm": 7.96875,
245
+ "learning_rate": 9.98812366428647e-06,
246
+ "loss": 4.0474,
247
+ "step": 34
248
+ },
249
+ {
250
+ "epoch": 0.05316119232959939,
251
+ "grad_norm": 6.375,
252
+ "learning_rate": 9.986367250278423e-06,
253
+ "loss": 4.0911,
254
+ "step": 35
255
+ },
256
+ {
257
+ "epoch": 0.054680083539016516,
258
+ "grad_norm": 8.0,
259
+ "learning_rate": 9.984489931963429e-06,
260
+ "loss": 3.8623,
261
+ "step": 36
262
+ },
263
+ {
264
+ "epoch": 0.05619897474843364,
265
+ "grad_norm": 4.46875,
266
+ "learning_rate": 9.982491754860763e-06,
267
+ "loss": 3.7118,
268
+ "step": 37
269
+ },
270
+ {
271
+ "epoch": 0.05771786595785077,
272
+ "grad_norm": 5.8125,
273
+ "learning_rate": 9.980372767420179e-06,
274
+ "loss": 3.9689,
275
+ "step": 38
276
+ },
277
+ {
278
+ "epoch": 0.059236757167267895,
279
+ "grad_norm": 4.3125,
280
+ "learning_rate": 9.978133021020697e-06,
281
+ "loss": 3.782,
282
+ "step": 39
283
+ },
284
+ {
285
+ "epoch": 0.06075564837668502,
286
+ "grad_norm": 5.0,
287
+ "learning_rate": 9.97577256996939e-06,
288
+ "loss": 3.7709,
289
+ "step": 40
290
+ },
291
+ {
292
+ "epoch": 0.06227453958610214,
293
+ "grad_norm": 5.34375,
294
+ "learning_rate": 9.97329147150005e-06,
295
+ "loss": 3.6426,
296
+ "step": 41
297
+ },
298
+ {
299
+ "epoch": 0.06379343079551927,
300
+ "grad_norm": 5.09375,
301
+ "learning_rate": 9.970689785771798e-06,
302
+ "loss": 3.5875,
303
+ "step": 42
304
+ },
305
+ {
306
+ "epoch": 0.06531232200493639,
307
+ "grad_norm": 5.90625,
308
+ "learning_rate": 9.96796757586764e-06,
309
+ "loss": 3.7528,
310
+ "step": 43
311
+ },
312
+ {
313
+ "epoch": 0.06683121321435352,
314
+ "grad_norm": 4.0,
315
+ "learning_rate": 9.965124907792916e-06,
316
+ "loss": 3.5526,
317
+ "step": 44
318
+ },
319
+ {
320
+ "epoch": 0.06835010442377065,
321
+ "grad_norm": 5.15625,
322
+ "learning_rate": 9.962161850473723e-06,
323
+ "loss": 3.4634,
324
+ "step": 45
325
+ },
326
+ {
327
+ "epoch": 0.06986899563318777,
328
+ "grad_norm": 4.25,
329
+ "learning_rate": 9.95907847575523e-06,
330
+ "loss": 3.534,
331
+ "step": 46
332
+ },
333
+ {
334
+ "epoch": 0.0713878868426049,
335
+ "grad_norm": 4.09375,
336
+ "learning_rate": 9.955874858399936e-06,
337
+ "loss": 3.4982,
338
+ "step": 47
339
+ },
340
+ {
341
+ "epoch": 0.07290677805202202,
342
+ "grad_norm": 4.40625,
343
+ "learning_rate": 9.952551076085864e-06,
344
+ "loss": 3.5063,
345
+ "step": 48
346
+ },
347
+ {
348
+ "epoch": 0.07442566926143915,
349
+ "grad_norm": 4.4375,
350
+ "learning_rate": 9.949107209404664e-06,
351
+ "loss": 3.4372,
352
+ "step": 49
353
+ },
354
+ {
355
+ "epoch": 0.07594456047085628,
356
+ "grad_norm": 4.46875,
357
+ "learning_rate": 9.945543341859681e-06,
358
+ "loss": 3.4007,
359
+ "step": 50
360
+ },
361
+ {
362
+ "epoch": 0.0774634516802734,
363
+ "grad_norm": 4.5,
364
+ "learning_rate": 9.94185955986391e-06,
365
+ "loss": 3.2577,
366
+ "step": 51
367
+ },
368
+ {
369
+ "epoch": 0.07898234288969053,
370
+ "grad_norm": 4.0625,
371
+ "learning_rate": 9.938055952737908e-06,
372
+ "loss": 3.2997,
373
+ "step": 52
374
+ },
375
+ {
376
+ "epoch": 0.08050123409910766,
377
+ "grad_norm": 3.984375,
378
+ "learning_rate": 9.934132612707631e-06,
379
+ "loss": 3.3514,
380
+ "step": 53
381
+ },
382
+ {
383
+ "epoch": 0.08202012530852477,
384
+ "grad_norm": 3.390625,
385
+ "learning_rate": 9.930089634902197e-06,
386
+ "loss": 3.1966,
387
+ "step": 54
388
+ },
389
+ {
390
+ "epoch": 0.0835390165179419,
391
+ "grad_norm": 5.78125,
392
+ "learning_rate": 9.925927117351573e-06,
393
+ "loss": 3.1777,
394
+ "step": 55
395
+ },
396
+ {
397
+ "epoch": 0.08505790772735902,
398
+ "grad_norm": 4.125,
399
+ "learning_rate": 9.921645160984205e-06,
400
+ "loss": 3.0858,
401
+ "step": 56
402
+ },
403
+ {
404
+ "epoch": 0.08657679893677615,
405
+ "grad_norm": 4.90625,
406
+ "learning_rate": 9.917243869624573e-06,
407
+ "loss": 3.1884,
408
+ "step": 57
409
+ },
410
+ {
411
+ "epoch": 0.08809569014619328,
412
+ "grad_norm": 4.6875,
413
+ "learning_rate": 9.91272334999066e-06,
414
+ "loss": 3.1061,
415
+ "step": 58
416
+ },
417
+ {
418
+ "epoch": 0.0896145813556104,
419
+ "grad_norm": 4.15625,
420
+ "learning_rate": 9.908083711691383e-06,
421
+ "loss": 3.1701,
422
+ "step": 59
423
+ },
424
+ {
425
+ "epoch": 0.09113347256502753,
426
+ "grad_norm": 5.125,
427
+ "learning_rate": 9.903325067223918e-06,
428
+ "loss": 3.1281,
429
+ "step": 60
430
+ },
431
+ {
432
+ "epoch": 0.09265236377444465,
433
+ "grad_norm": 4.59375,
434
+ "learning_rate": 9.898447531970989e-06,
435
+ "loss": 3.0717,
436
+ "step": 61
437
+ },
438
+ {
439
+ "epoch": 0.09417125498386178,
440
+ "grad_norm": 5.9375,
441
+ "learning_rate": 9.893451224198051e-06,
442
+ "loss": 3.0151,
443
+ "step": 62
444
+ },
445
+ {
446
+ "epoch": 0.09569014619327891,
447
+ "grad_norm": 4.25,
448
+ "learning_rate": 9.888336265050443e-06,
449
+ "loss": 3.0585,
450
+ "step": 63
451
+ },
452
+ {
453
+ "epoch": 0.09720903740269603,
454
+ "grad_norm": 5.28125,
455
+ "learning_rate": 9.883102778550434e-06,
456
+ "loss": 3.0649,
457
+ "step": 64
458
+ },
459
+ {
460
+ "epoch": 0.09872792861211316,
461
+ "grad_norm": 3.53125,
462
+ "learning_rate": 9.877750891594224e-06,
463
+ "loss": 3.0054,
464
+ "step": 65
465
+ },
466
+ {
467
+ "epoch": 0.10024681982153029,
468
+ "grad_norm": 4.3125,
469
+ "learning_rate": 9.872280733948867e-06,
470
+ "loss": 2.911,
471
+ "step": 66
472
+ },
473
+ {
474
+ "epoch": 0.1017657110309474,
475
+ "grad_norm": 4.375,
476
+ "learning_rate": 9.866692438249124e-06,
477
+ "loss": 3.0651,
478
+ "step": 67
479
+ },
480
+ {
481
+ "epoch": 0.10328460224036454,
482
+ "grad_norm": 4.34375,
483
+ "learning_rate": 9.86098613999424e-06,
484
+ "loss": 2.9609,
485
+ "step": 68
486
+ },
487
+ {
488
+ "epoch": 0.10480349344978165,
489
+ "grad_norm": 3.75,
490
+ "learning_rate": 9.855161977544672e-06,
491
+ "loss": 3.0308,
492
+ "step": 69
493
+ },
494
+ {
495
+ "epoch": 0.10632238465919878,
496
+ "grad_norm": 3.078125,
497
+ "learning_rate": 9.849220092118721e-06,
498
+ "loss": 2.97,
499
+ "step": 70
500
+ },
501
+ {
502
+ "epoch": 0.10784127586861592,
503
+ "grad_norm": 3.390625,
504
+ "learning_rate": 9.84316062778912e-06,
505
+ "loss": 2.9543,
506
+ "step": 71
507
+ },
508
+ {
509
+ "epoch": 0.10936016707803303,
510
+ "grad_norm": 2.6875,
511
+ "learning_rate": 9.836983731479526e-06,
512
+ "loss": 2.8797,
513
+ "step": 72
514
+ },
515
+ {
516
+ "epoch": 0.11087905828745016,
517
+ "grad_norm": 2.84375,
518
+ "learning_rate": 9.830689552960974e-06,
519
+ "loss": 2.8188,
520
+ "step": 73
521
+ },
522
+ {
523
+ "epoch": 0.11239794949686728,
524
+ "grad_norm": 3.171875,
525
+ "learning_rate": 9.824278244848236e-06,
526
+ "loss": 2.9654,
527
+ "step": 74
528
+ },
529
+ {
530
+ "epoch": 0.11391684070628441,
531
+ "grad_norm": 2.1875,
532
+ "learning_rate": 9.817749962596115e-06,
533
+ "loss": 2.8417,
534
+ "step": 75
535
+ },
536
+ {
537
+ "epoch": 0.11543573191570154,
538
+ "grad_norm": 2.671875,
539
+ "learning_rate": 9.811104864495691e-06,
540
+ "loss": 2.9101,
541
+ "step": 76
542
+ },
543
+ {
544
+ "epoch": 0.11695462312511866,
545
+ "grad_norm": 2.6875,
546
+ "learning_rate": 9.804343111670472e-06,
547
+ "loss": 2.8579,
548
+ "step": 77
549
+ },
550
+ {
551
+ "epoch": 0.11847351433453579,
552
+ "grad_norm": 2.53125,
553
+ "learning_rate": 9.797464868072489e-06,
554
+ "loss": 2.8388,
555
+ "step": 78
556
+ },
557
+ {
558
+ "epoch": 0.11999240554395292,
559
+ "grad_norm": 3.078125,
560
+ "learning_rate": 9.790470300478318e-06,
561
+ "loss": 2.8716,
562
+ "step": 79
563
+ },
564
+ {
565
+ "epoch": 0.12151129675337004,
566
+ "grad_norm": 2.046875,
567
+ "learning_rate": 9.783359578485047e-06,
568
+ "loss": 2.8199,
569
+ "step": 80
570
+ },
571
+ {
572
+ "epoch": 0.12303018796278717,
573
+ "grad_norm": 2.1875,
574
+ "learning_rate": 9.776132874506153e-06,
575
+ "loss": 2.8302,
576
+ "step": 81
577
+ },
578
+ {
579
+ "epoch": 0.12454907917220429,
580
+ "grad_norm": 2.140625,
581
+ "learning_rate": 9.768790363767321e-06,
582
+ "loss": 2.795,
583
+ "step": 82
584
+ },
585
+ {
586
+ "epoch": 0.1260679703816214,
587
+ "grad_norm": 2.125,
588
+ "learning_rate": 9.761332224302209e-06,
589
+ "loss": 2.7479,
590
+ "step": 83
591
+ },
592
+ {
593
+ "epoch": 0.12758686159103855,
594
+ "grad_norm": 2.765625,
595
+ "learning_rate": 9.753758636948112e-06,
596
+ "loss": 2.7841,
597
+ "step": 84
598
+ },
599
+ {
600
+ "epoch": 0.12910575280045566,
601
+ "grad_norm": 2.125,
602
+ "learning_rate": 9.74606978534159e-06,
603
+ "loss": 2.8815,
604
+ "step": 85
605
+ },
606
+ {
607
+ "epoch": 0.13062464400987278,
608
+ "grad_norm": 3.328125,
609
+ "learning_rate": 9.738265855914014e-06,
610
+ "loss": 2.7458,
611
+ "step": 86
612
+ },
613
+ {
614
+ "epoch": 0.13214353521928993,
615
+ "grad_norm": 2.3125,
616
+ "learning_rate": 9.730347037887041e-06,
617
+ "loss": 2.8145,
618
+ "step": 87
619
+ },
620
+ {
621
+ "epoch": 0.13366242642870704,
622
+ "grad_norm": 2.9375,
623
+ "learning_rate": 9.722313523268028e-06,
624
+ "loss": 2.7804,
625
+ "step": 88
626
+ },
627
+ {
628
+ "epoch": 0.13518131763812416,
629
+ "grad_norm": 2.578125,
630
+ "learning_rate": 9.714165506845381e-06,
631
+ "loss": 2.7899,
632
+ "step": 89
633
+ },
634
+ {
635
+ "epoch": 0.1367002088475413,
636
+ "grad_norm": 3.359375,
637
+ "learning_rate": 9.705903186183828e-06,
638
+ "loss": 2.7296,
639
+ "step": 90
640
+ },
641
+ {
642
+ "epoch": 0.13821910005695842,
643
+ "grad_norm": 2.96875,
644
+ "learning_rate": 9.697526761619621e-06,
645
+ "loss": 2.7233,
646
+ "step": 91
647
+ },
648
+ {
649
+ "epoch": 0.13973799126637554,
650
+ "grad_norm": 2.09375,
651
+ "learning_rate": 9.689036436255698e-06,
652
+ "loss": 2.7421,
653
+ "step": 92
654
+ },
655
+ {
656
+ "epoch": 0.14125688247579268,
657
+ "grad_norm": 2.828125,
658
+ "learning_rate": 9.680432415956736e-06,
659
+ "loss": 2.7627,
660
+ "step": 93
661
+ },
662
+ {
663
+ "epoch": 0.1427757736852098,
664
+ "grad_norm": 1.9765625,
665
+ "learning_rate": 9.671714909344175e-06,
666
+ "loss": 2.7611,
667
+ "step": 94
668
+ },
669
+ {
670
+ "epoch": 0.14429466489462692,
671
+ "grad_norm": 1.9140625,
672
+ "learning_rate": 9.66288412779115e-06,
673
+ "loss": 2.753,
674
+ "step": 95
675
+ },
676
+ {
677
+ "epoch": 0.14581355610404403,
678
+ "grad_norm": 2.0625,
679
+ "learning_rate": 9.653940285417381e-06,
680
+ "loss": 2.7191,
681
+ "step": 96
682
+ },
683
+ {
684
+ "epoch": 0.14733244731346118,
685
+ "grad_norm": 1.4765625,
686
+ "learning_rate": 9.644883599083959e-06,
687
+ "loss": 2.7194,
688
+ "step": 97
689
+ },
690
+ {
691
+ "epoch": 0.1488513385228783,
692
+ "grad_norm": 2.078125,
693
+ "learning_rate": 9.635714288388103e-06,
694
+ "loss": 2.6886,
695
+ "step": 98
696
+ },
697
+ {
698
+ "epoch": 0.1503702297322954,
699
+ "grad_norm": 1.7265625,
700
+ "learning_rate": 9.626432575657834e-06,
701
+ "loss": 2.7631,
702
+ "step": 99
703
+ },
704
+ {
705
+ "epoch": 0.15188912094171256,
706
+ "grad_norm": 1.6796875,
707
+ "learning_rate": 9.617038685946578e-06,
708
+ "loss": 2.6749,
709
+ "step": 100
710
+ },
711
+ {
712
+ "epoch": 0.15340801215112967,
713
+ "grad_norm": 1.59375,
714
+ "learning_rate": 9.60753284702772e-06,
715
+ "loss": 2.6299,
716
+ "step": 101
717
+ },
718
+ {
719
+ "epoch": 0.1549269033605468,
720
+ "grad_norm": 1.421875,
721
+ "learning_rate": 9.597915289389067e-06,
722
+ "loss": 2.6723,
723
+ "step": 102
724
+ },
725
+ {
726
+ "epoch": 0.15644579456996394,
727
+ "grad_norm": 1.5625,
728
+ "learning_rate": 9.58818624622727e-06,
729
+ "loss": 2.7104,
730
+ "step": 103
731
+ },
732
+ {
733
+ "epoch": 0.15796468577938105,
734
+ "grad_norm": 1.640625,
735
+ "learning_rate": 9.578345953442163e-06,
736
+ "loss": 2.6862,
737
+ "step": 104
738
+ },
739
+ {
740
+ "epoch": 0.15948357698879817,
741
+ "grad_norm": 1.8515625,
742
+ "learning_rate": 9.568394649631055e-06,
743
+ "loss": 2.7183,
744
+ "step": 105
745
+ },
746
+ {
747
+ "epoch": 0.16100246819821531,
748
+ "grad_norm": 1.9453125,
749
+ "learning_rate": 9.558332576082925e-06,
750
+ "loss": 2.6991,
751
+ "step": 106
752
+ },
753
+ {
754
+ "epoch": 0.16252135940763243,
755
+ "grad_norm": 1.5078125,
756
+ "learning_rate": 9.548159976772593e-06,
757
+ "loss": 2.591,
758
+ "step": 107
759
+ },
760
+ {
761
+ "epoch": 0.16404025061704955,
762
+ "grad_norm": 1.703125,
763
+ "learning_rate": 9.537877098354787e-06,
764
+ "loss": 2.6819,
765
+ "step": 108
766
+ },
767
+ {
768
+ "epoch": 0.16555914182646667,
769
+ "grad_norm": 1.765625,
770
+ "learning_rate": 9.527484190158171e-06,
771
+ "loss": 2.6957,
772
+ "step": 109
773
+ },
774
+ {
775
+ "epoch": 0.1670780330358838,
776
+ "grad_norm": 2.265625,
777
+ "learning_rate": 9.5169815041793e-06,
778
+ "loss": 2.7064,
779
+ "step": 110
780
+ },
781
+ {
782
+ "epoch": 0.16859692424530093,
783
+ "grad_norm": 2.21875,
784
+ "learning_rate": 9.506369295076505e-06,
785
+ "loss": 2.6742,
786
+ "step": 111
787
+ },
788
+ {
789
+ "epoch": 0.17011581545471804,
790
+ "grad_norm": 1.640625,
791
+ "learning_rate": 9.495647820163725e-06,
792
+ "loss": 2.6757,
793
+ "step": 112
794
+ },
795
+ {
796
+ "epoch": 0.1716347066641352,
797
+ "grad_norm": 2.03125,
798
+ "learning_rate": 9.484817339404261e-06,
799
+ "loss": 2.6503,
800
+ "step": 113
801
+ },
802
+ {
803
+ "epoch": 0.1731535978735523,
804
+ "grad_norm": 1.6640625,
805
+ "learning_rate": 9.473878115404477e-06,
806
+ "loss": 2.6608,
807
+ "step": 114
808
+ },
809
+ {
810
+ "epoch": 0.17467248908296942,
811
+ "grad_norm": 1.8125,
812
+ "learning_rate": 9.462830413407427e-06,
813
+ "loss": 2.6217,
814
+ "step": 115
815
+ },
816
+ {
817
+ "epoch": 0.17619138029238657,
818
+ "grad_norm": 1.578125,
819
+ "learning_rate": 9.451674501286436e-06,
820
+ "loss": 2.6885,
821
+ "step": 116
822
+ },
823
+ {
824
+ "epoch": 0.17771027150180368,
825
+ "grad_norm": 1.71875,
826
+ "learning_rate": 9.440410649538592e-06,
827
+ "loss": 2.6041,
828
+ "step": 117
829
+ },
830
+ {
831
+ "epoch": 0.1792291627112208,
832
+ "grad_norm": 1.4453125,
833
+ "learning_rate": 9.42903913127819e-06,
834
+ "loss": 2.6156,
835
+ "step": 118
836
+ },
837
+ {
838
+ "epoch": 0.18074805392063795,
839
+ "grad_norm": 1.796875,
840
+ "learning_rate": 9.417560222230115e-06,
841
+ "loss": 2.5769,
842
+ "step": 119
843
+ },
844
+ {
845
+ "epoch": 0.18226694513005506,
846
+ "grad_norm": 1.46875,
847
+ "learning_rate": 9.405974200723156e-06,
848
+ "loss": 2.5831,
849
+ "step": 120
850
+ },
851
+ {
852
+ "epoch": 0.18378583633947218,
853
+ "grad_norm": 1.59375,
854
+ "learning_rate": 9.394281347683247e-06,
855
+ "loss": 2.7265,
856
+ "step": 121
857
+ },
858
+ {
859
+ "epoch": 0.1853047275488893,
860
+ "grad_norm": 1.34375,
861
+ "learning_rate": 9.382481946626673e-06,
862
+ "loss": 2.6134,
863
+ "step": 122
864
+ },
865
+ {
866
+ "epoch": 0.18682361875830644,
867
+ "grad_norm": 1.6015625,
868
+ "learning_rate": 9.370576283653178e-06,
869
+ "loss": 2.5806,
870
+ "step": 123
871
+ },
872
+ {
873
+ "epoch": 0.18834250996772356,
874
+ "grad_norm": 1.5078125,
875
+ "learning_rate": 9.358564647439037e-06,
876
+ "loss": 2.5152,
877
+ "step": 124
878
+ },
879
+ {
880
+ "epoch": 0.18986140117714068,
881
+ "grad_norm": 1.546875,
882
+ "learning_rate": 9.34644732923006e-06,
883
+ "loss": 2.6123,
884
+ "step": 125
885
+ },
886
+ {
887
+ "epoch": 0.19138029238655782,
888
+ "grad_norm": 1.3984375,
889
+ "learning_rate": 9.33422462283452e-06,
890
+ "loss": 2.6245,
891
+ "step": 126
892
+ },
893
+ {
894
+ "epoch": 0.19289918359597494,
895
+ "grad_norm": 1.2109375,
896
+ "learning_rate": 9.321896824616036e-06,
897
+ "loss": 2.5982,
898
+ "step": 127
899
+ },
900
+ {
901
+ "epoch": 0.19441807480539205,
902
+ "grad_norm": 2.1875,
903
+ "learning_rate": 9.309464233486386e-06,
904
+ "loss": 2.61,
905
+ "step": 128
906
+ },
907
+ {
908
+ "epoch": 0.1959369660148092,
909
+ "grad_norm": 1.3984375,
910
+ "learning_rate": 9.29692715089826e-06,
911
+ "loss": 2.5773,
912
+ "step": 129
913
+ },
914
+ {
915
+ "epoch": 0.19745585722422632,
916
+ "grad_norm": 1.4296875,
917
+ "learning_rate": 9.284285880837947e-06,
918
+ "loss": 2.5853,
919
+ "step": 130
920
+ },
921
+ {
922
+ "epoch": 0.19897474843364343,
923
+ "grad_norm": 1.46875,
924
+ "learning_rate": 9.271540729817969e-06,
925
+ "loss": 2.5594,
926
+ "step": 131
927
+ },
928
+ {
929
+ "epoch": 0.20049363964306058,
930
+ "grad_norm": 1.765625,
931
+ "learning_rate": 9.258692006869644e-06,
932
+ "loss": 2.5041,
933
+ "step": 132
934
+ },
935
+ {
936
+ "epoch": 0.2020125308524777,
937
+ "grad_norm": 1.265625,
938
+ "learning_rate": 9.245740023535596e-06,
939
+ "loss": 2.5043,
940
+ "step": 133
941
+ },
942
+ {
943
+ "epoch": 0.2035314220618948,
944
+ "grad_norm": 1.53125,
945
+ "learning_rate": 9.232685093862206e-06,
946
+ "loss": 2.5629,
947
+ "step": 134
948
+ },
949
+ {
950
+ "epoch": 0.20505031327131193,
951
+ "grad_norm": 1.34375,
952
+ "learning_rate": 9.219527534391983e-06,
953
+ "loss": 2.5121,
954
+ "step": 135
955
+ },
956
+ {
957
+ "epoch": 0.20656920448072907,
958
+ "grad_norm": 1.7890625,
959
+ "learning_rate": 9.206267664155906e-06,
960
+ "loss": 2.5578,
961
+ "step": 136
962
+ },
963
+ {
964
+ "epoch": 0.2080880956901462,
965
+ "grad_norm": 1.203125,
966
+ "learning_rate": 9.192905804665677e-06,
967
+ "loss": 2.5341,
968
+ "step": 137
969
+ },
970
+ {
971
+ "epoch": 0.2096069868995633,
972
+ "grad_norm": 2.203125,
973
+ "learning_rate": 9.179442279905927e-06,
974
+ "loss": 2.5867,
975
+ "step": 138
976
+ },
977
+ {
978
+ "epoch": 0.21112587810898045,
979
+ "grad_norm": 1.2890625,
980
+ "learning_rate": 9.165877416326365e-06,
981
+ "loss": 2.6066,
982
+ "step": 139
983
+ },
984
+ {
985
+ "epoch": 0.21264476931839757,
986
+ "grad_norm": 1.8828125,
987
+ "learning_rate": 9.152211542833856e-06,
988
+ "loss": 2.6332,
989
+ "step": 140
990
+ },
991
+ {
992
+ "epoch": 0.2141636605278147,
993
+ "grad_norm": 1.6640625,
994
+ "learning_rate": 9.138444990784455e-06,
995
+ "loss": 2.5835,
996
+ "step": 141
997
+ },
998
+ {
999
+ "epoch": 0.21568255173723183,
1000
+ "grad_norm": 1.6484375,
1001
+ "learning_rate": 9.124578093975358e-06,
1002
+ "loss": 2.622,
1003
+ "step": 142
1004
+ },
1005
+ {
1006
+ "epoch": 0.21720144294664895,
1007
+ "grad_norm": 1.4453125,
1008
+ "learning_rate": 9.110611188636828e-06,
1009
+ "loss": 2.5668,
1010
+ "step": 143
1011
+ },
1012
+ {
1013
+ "epoch": 0.21872033415606607,
1014
+ "grad_norm": 1.1640625,
1015
+ "learning_rate": 9.096544613424026e-06,
1016
+ "loss": 2.6229,
1017
+ "step": 144
1018
+ },
1019
+ {
1020
+ "epoch": 0.2202392253654832,
1021
+ "grad_norm": 1.515625,
1022
+ "learning_rate": 9.082378709408805e-06,
1023
+ "loss": 2.5456,
1024
+ "step": 145
1025
+ },
1026
+ {
1027
+ "epoch": 0.22175811657490033,
1028
+ "grad_norm": 1.296875,
1029
+ "learning_rate": 9.068113820071447e-06,
1030
+ "loss": 2.4778,
1031
+ "step": 146
1032
+ },
1033
+ {
1034
+ "epoch": 0.22327700778431744,
1035
+ "grad_norm": 1.53125,
1036
+ "learning_rate": 9.053750291292321e-06,
1037
+ "loss": 2.4628,
1038
+ "step": 147
1039
+ },
1040
+ {
1041
+ "epoch": 0.22479589899373456,
1042
+ "grad_norm": 1.265625,
1043
+ "learning_rate": 9.039288471343505e-06,
1044
+ "loss": 2.5147,
1045
+ "step": 148
1046
+ },
1047
+ {
1048
+ "epoch": 0.2263147902031517,
1049
+ "grad_norm": 1.2109375,
1050
+ "learning_rate": 9.024728710880345e-06,
1051
+ "loss": 2.5673,
1052
+ "step": 149
1053
+ },
1054
+ {
1055
+ "epoch": 0.22783368141256882,
1056
+ "grad_norm": 1.2265625,
1057
+ "learning_rate": 9.010071362932945e-06,
1058
+ "loss": 2.5198,
1059
+ "step": 150
1060
+ },
1061
+ {
1062
+ "epoch": 0.22935257262198594,
1063
+ "grad_norm": 1.40625,
1064
+ "learning_rate": 8.995316782897605e-06,
1065
+ "loss": 2.5247,
1066
+ "step": 151
1067
+ },
1068
+ {
1069
+ "epoch": 0.23087146383140308,
1070
+ "grad_norm": 1.3046875,
1071
+ "learning_rate": 8.98046532852822e-06,
1072
+ "loss": 2.5068,
1073
+ "step": 152
1074
+ },
1075
+ {
1076
+ "epoch": 0.2323903550408202,
1077
+ "grad_norm": 1.2421875,
1078
+ "learning_rate": 8.965517359927583e-06,
1079
+ "loss": 2.5362,
1080
+ "step": 153
1081
+ },
1082
+ {
1083
+ "epoch": 0.23390924625023732,
1084
+ "grad_norm": 1.2265625,
1085
+ "learning_rate": 8.950473239538672e-06,
1086
+ "loss": 2.5166,
1087
+ "step": 154
1088
+ },
1089
+ {
1090
+ "epoch": 0.23542813745965446,
1091
+ "grad_norm": 1.421875,
1092
+ "learning_rate": 8.935333332135853e-06,
1093
+ "loss": 2.502,
1094
+ "step": 155
1095
+ },
1096
+ {
1097
+ "epoch": 0.23694702866907158,
1098
+ "grad_norm": 1.6796875,
1099
+ "learning_rate": 8.920098004816035e-06,
1100
+ "loss": 2.5186,
1101
+ "step": 156
1102
+ },
1103
+ {
1104
+ "epoch": 0.2384659198784887,
1105
+ "grad_norm": 1.0859375,
1106
+ "learning_rate": 8.904767626989774e-06,
1107
+ "loss": 2.5439,
1108
+ "step": 157
1109
+ },
1110
+ {
1111
+ "epoch": 0.23998481108790584,
1112
+ "grad_norm": 1.1640625,
1113
+ "learning_rate": 8.88934257037231e-06,
1114
+ "loss": 2.5026,
1115
+ "step": 158
1116
+ },
1117
+ {
1118
+ "epoch": 0.24150370229732296,
1119
+ "grad_norm": 1.109375,
1120
+ "learning_rate": 8.873823208974557e-06,
1121
+ "loss": 2.5948,
1122
+ "step": 159
1123
+ },
1124
+ {
1125
+ "epoch": 0.24302259350674008,
1126
+ "grad_norm": 1.203125,
1127
+ "learning_rate": 8.85820991909404e-06,
1128
+ "loss": 2.5613,
1129
+ "step": 160
1130
+ },
1131
+ {
1132
+ "epoch": 0.2445414847161572,
1133
+ "grad_norm": 1.03125,
1134
+ "learning_rate": 8.842503079305757e-06,
1135
+ "loss": 2.5053,
1136
+ "step": 161
1137
+ },
1138
+ {
1139
+ "epoch": 0.24606037592557434,
1140
+ "grad_norm": 1.109375,
1141
+ "learning_rate": 8.826703070453014e-06,
1142
+ "loss": 2.4929,
1143
+ "step": 162
1144
+ },
1145
+ {
1146
+ "epoch": 0.24757926713499145,
1147
+ "grad_norm": 1.09375,
1148
+ "learning_rate": 8.810810275638183e-06,
1149
+ "loss": 2.4863,
1150
+ "step": 163
1151
+ },
1152
+ {
1153
+ "epoch": 0.24909815834440857,
1154
+ "grad_norm": 1.765625,
1155
+ "learning_rate": 8.794825080213415e-06,
1156
+ "loss": 2.5587,
1157
+ "step": 164
1158
+ },
1159
+ {
1160
+ "epoch": 0.2506170495538257,
1161
+ "grad_norm": 1.15625,
1162
+ "learning_rate": 8.778747871771293e-06,
1163
+ "loss": 2.5292,
1164
+ "step": 165
1165
+ },
1166
+ {
1167
+ "epoch": 0.2521359407632428,
1168
+ "grad_norm": 1.375,
1169
+ "learning_rate": 8.76257904013544e-06,
1170
+ "loss": 2.5414,
1171
+ "step": 166
1172
+ },
1173
+ {
1174
+ "epoch": 0.25365483197266,
1175
+ "grad_norm": 1.234375,
1176
+ "learning_rate": 8.746318977351066e-06,
1177
+ "loss": 2.5376,
1178
+ "step": 167
1179
+ },
1180
+ {
1181
+ "epoch": 0.2551737231820771,
1182
+ "grad_norm": 1.390625,
1183
+ "learning_rate": 8.729968077675454e-06,
1184
+ "loss": 2.5962,
1185
+ "step": 168
1186
+ },
1187
+ {
1188
+ "epoch": 0.2566926143914942,
1189
+ "grad_norm": 1.265625,
1190
+ "learning_rate": 8.713526737568415e-06,
1191
+ "loss": 2.5141,
1192
+ "step": 169
1193
+ },
1194
+ {
1195
+ "epoch": 0.25821150560091133,
1196
+ "grad_norm": 1.375,
1197
+ "learning_rate": 8.696995355682656e-06,
1198
+ "loss": 2.6006,
1199
+ "step": 170
1200
+ },
1201
+ {
1202
+ "epoch": 0.25973039681032845,
1203
+ "grad_norm": 1.5390625,
1204
+ "learning_rate": 8.680374332854134e-06,
1205
+ "loss": 2.4407,
1206
+ "step": 171
1207
+ },
1208
+ {
1209
+ "epoch": 0.26124928801974556,
1210
+ "grad_norm": 1.359375,
1211
+ "learning_rate": 8.663664072092324e-06,
1212
+ "loss": 2.4783,
1213
+ "step": 172
1214
+ },
1215
+ {
1216
+ "epoch": 0.26276817922916273,
1217
+ "grad_norm": 1.421875,
1218
+ "learning_rate": 8.646864978570445e-06,
1219
+ "loss": 2.513,
1220
+ "step": 173
1221
+ },
1222
+ {
1223
+ "epoch": 0.26428707043857985,
1224
+ "grad_norm": 1.453125,
1225
+ "learning_rate": 8.629977459615655e-06,
1226
+ "loss": 2.4805,
1227
+ "step": 174
1228
+ },
1229
+ {
1230
+ "epoch": 0.26580596164799697,
1231
+ "grad_norm": 1.390625,
1232
+ "learning_rate": 8.613001924699146e-06,
1233
+ "loss": 2.5176,
1234
+ "step": 175
1235
+ },
1236
+ {
1237
+ "epoch": 0.2673248528574141,
1238
+ "grad_norm": 1.3125,
1239
+ "learning_rate": 8.595938785426241e-06,
1240
+ "loss": 2.5586,
1241
+ "step": 176
1242
+ },
1243
+ {
1244
+ "epoch": 0.2688437440668312,
1245
+ "grad_norm": 1.3671875,
1246
+ "learning_rate": 8.578788455526398e-06,
1247
+ "loss": 2.4918,
1248
+ "step": 177
1249
+ },
1250
+ {
1251
+ "epoch": 0.2703626352762483,
1252
+ "grad_norm": 1.203125,
1253
+ "learning_rate": 8.561551350843185e-06,
1254
+ "loss": 2.5574,
1255
+ "step": 178
1256
+ },
1257
+ {
1258
+ "epoch": 0.27188152648566544,
1259
+ "grad_norm": 1.3828125,
1260
+ "learning_rate": 8.544227889324199e-06,
1261
+ "loss": 2.4831,
1262
+ "step": 179
1263
+ },
1264
+ {
1265
+ "epoch": 0.2734004176950826,
1266
+ "grad_norm": 1.4296875,
1267
+ "learning_rate": 8.526818491010922e-06,
1268
+ "loss": 2.5496,
1269
+ "step": 180
1270
+ },
1271
+ {
1272
+ "epoch": 0.2749193089044997,
1273
+ "grad_norm": 1.5078125,
1274
+ "learning_rate": 8.509323578028547e-06,
1275
+ "loss": 2.5047,
1276
+ "step": 181
1277
+ },
1278
+ {
1279
+ "epoch": 0.27643820011391684,
1280
+ "grad_norm": 1.5234375,
1281
+ "learning_rate": 8.491743574575743e-06,
1282
+ "loss": 2.4502,
1283
+ "step": 182
1284
+ },
1285
+ {
1286
+ "epoch": 0.27795709132333396,
1287
+ "grad_norm": 1.46875,
1288
+ "learning_rate": 8.474078906914359e-06,
1289
+ "loss": 2.5617,
1290
+ "step": 183
1291
+ },
1292
+ {
1293
+ "epoch": 0.2794759825327511,
1294
+ "grad_norm": 1.234375,
1295
+ "learning_rate": 8.456330003359093e-06,
1296
+ "loss": 2.4827,
1297
+ "step": 184
1298
+ },
1299
+ {
1300
+ "epoch": 0.2809948737421682,
1301
+ "grad_norm": 1.828125,
1302
+ "learning_rate": 8.438497294267117e-06,
1303
+ "loss": 2.5726,
1304
+ "step": 185
1305
+ },
1306
+ {
1307
+ "epoch": 0.28251376495158537,
1308
+ "grad_norm": 1.453125,
1309
+ "learning_rate": 8.420581212027625e-06,
1310
+ "loss": 2.4644,
1311
+ "step": 186
1312
+ },
1313
+ {
1314
+ "epoch": 0.2840326561610025,
1315
+ "grad_norm": 1.25,
1316
+ "learning_rate": 8.402582191051365e-06,
1317
+ "loss": 2.4287,
1318
+ "step": 187
1319
+ },
1320
+ {
1321
+ "epoch": 0.2855515473704196,
1322
+ "grad_norm": 1.640625,
1323
+ "learning_rate": 8.38450066776009e-06,
1324
+ "loss": 2.5467,
1325
+ "step": 188
1326
+ },
1327
+ {
1328
+ "epoch": 0.2870704385798367,
1329
+ "grad_norm": 1.359375,
1330
+ "learning_rate": 8.36633708057599e-06,
1331
+ "loss": 2.4555,
1332
+ "step": 189
1333
+ },
1334
+ {
1335
+ "epoch": 0.28858932978925383,
1336
+ "grad_norm": 1.3203125,
1337
+ "learning_rate": 8.348091869911054e-06,
1338
+ "loss": 2.5255,
1339
+ "step": 190
1340
+ },
1341
+ {
1342
+ "epoch": 0.29010822099867095,
1343
+ "grad_norm": 1.34375,
1344
+ "learning_rate": 8.329765478156394e-06,
1345
+ "loss": 2.5443,
1346
+ "step": 191
1347
+ },
1348
+ {
1349
+ "epoch": 0.29162711220808807,
1350
+ "grad_norm": 1.2890625,
1351
+ "learning_rate": 8.311358349671516e-06,
1352
+ "loss": 2.5705,
1353
+ "step": 192
1354
+ },
1355
+ {
1356
+ "epoch": 0.29314600341750524,
1357
+ "grad_norm": 1.203125,
1358
+ "learning_rate": 8.292870930773551e-06,
1359
+ "loss": 2.4817,
1360
+ "step": 193
1361
+ },
1362
+ {
1363
+ "epoch": 0.29466489462692236,
1364
+ "grad_norm": 1.3671875,
1365
+ "learning_rate": 8.274303669726427e-06,
1366
+ "loss": 2.5036,
1367
+ "step": 194
1368
+ },
1369
+ {
1370
+ "epoch": 0.2961837858363395,
1371
+ "grad_norm": 1.1484375,
1372
+ "learning_rate": 8.255657016729997e-06,
1373
+ "loss": 2.5086,
1374
+ "step": 195
1375
+ },
1376
+ {
1377
+ "epoch": 0.2977026770457566,
1378
+ "grad_norm": 1.078125,
1379
+ "learning_rate": 8.23693142390914e-06,
1380
+ "loss": 2.4332,
1381
+ "step": 196
1382
+ },
1383
+ {
1384
+ "epoch": 0.2992215682551737,
1385
+ "grad_norm": 1.1015625,
1386
+ "learning_rate": 8.218127345302775e-06,
1387
+ "loss": 2.4879,
1388
+ "step": 197
1389
+ },
1390
+ {
1391
+ "epoch": 0.3007404594645908,
1392
+ "grad_norm": 1.109375,
1393
+ "learning_rate": 8.199245236852871e-06,
1394
+ "loss": 2.5324,
1395
+ "step": 198
1396
+ },
1397
+ {
1398
+ "epoch": 0.302259350674008,
1399
+ "grad_norm": 1.328125,
1400
+ "learning_rate": 8.180285556393384e-06,
1401
+ "loss": 2.4826,
1402
+ "step": 199
1403
+ },
1404
+ {
1405
+ "epoch": 0.3037782418834251,
1406
+ "grad_norm": 1.109375,
1407
+ "learning_rate": 8.161248763639154e-06,
1408
+ "loss": 2.4943,
1409
+ "step": 200
1410
+ },
1411
+ {
1412
+ "epoch": 0.30529713309284223,
1413
+ "grad_norm": 1.140625,
1414
+ "learning_rate": 8.142135320174758e-06,
1415
+ "loss": 2.4246,
1416
+ "step": 201
1417
+ },
1418
+ {
1419
+ "epoch": 0.30681602430225935,
1420
+ "grad_norm": 1.296875,
1421
+ "learning_rate": 8.122945689443328e-06,
1422
+ "loss": 2.4313,
1423
+ "step": 202
1424
+ },
1425
+ {
1426
+ "epoch": 0.30833491551167647,
1427
+ "grad_norm": 1.09375,
1428
+ "learning_rate": 8.1036803367353e-06,
1429
+ "loss": 2.559,
1430
+ "step": 203
1431
+ },
1432
+ {
1433
+ "epoch": 0.3098538067210936,
1434
+ "grad_norm": 1.1328125,
1435
+ "learning_rate": 8.084339729177142e-06,
1436
+ "loss": 2.514,
1437
+ "step": 204
1438
+ },
1439
+ {
1440
+ "epoch": 0.3113726979305107,
1441
+ "grad_norm": 1.2578125,
1442
+ "learning_rate": 8.064924335720023e-06,
1443
+ "loss": 2.4889,
1444
+ "step": 205
1445
+ },
1446
+ {
1447
+ "epoch": 0.3128915891399279,
1448
+ "grad_norm": 1.0703125,
1449
+ "learning_rate": 8.045434627128446e-06,
1450
+ "loss": 2.4603,
1451
+ "step": 206
1452
+ },
1453
+ {
1454
+ "epoch": 0.314410480349345,
1455
+ "grad_norm": 1.2734375,
1456
+ "learning_rate": 8.025871075968828e-06,
1457
+ "loss": 2.4971,
1458
+ "step": 207
1459
+ },
1460
+ {
1461
+ "epoch": 0.3159293715587621,
1462
+ "grad_norm": 0.9765625,
1463
+ "learning_rate": 8.006234156598043e-06,
1464
+ "loss": 2.5571,
1465
+ "step": 208
1466
+ },
1467
+ {
1468
+ "epoch": 0.3174482627681792,
1469
+ "grad_norm": 1.0625,
1470
+ "learning_rate": 7.986524345151924e-06,
1471
+ "loss": 2.5384,
1472
+ "step": 209
1473
+ },
1474
+ {
1475
+ "epoch": 0.31896715397759634,
1476
+ "grad_norm": 1.09375,
1477
+ "learning_rate": 7.966742119533724e-06,
1478
+ "loss": 2.4885,
1479
+ "step": 210
1480
+ },
1481
+ {
1482
+ "epoch": 0.32048604518701346,
1483
+ "grad_norm": 1.0546875,
1484
+ "learning_rate": 7.946887959402504e-06,
1485
+ "loss": 2.4381,
1486
+ "step": 211
1487
+ },
1488
+ {
1489
+ "epoch": 0.32200493639643063,
1490
+ "grad_norm": 0.984375,
1491
+ "learning_rate": 7.926962346161535e-06,
1492
+ "loss": 2.5199,
1493
+ "step": 212
1494
+ },
1495
+ {
1496
+ "epoch": 0.32352382760584775,
1497
+ "grad_norm": 1.078125,
1498
+ "learning_rate": 7.9069657629466e-06,
1499
+ "loss": 2.5338,
1500
+ "step": 213
1501
+ },
1502
+ {
1503
+ "epoch": 0.32504271881526486,
1504
+ "grad_norm": 0.9375,
1505
+ "learning_rate": 7.886898694614292e-06,
1506
+ "loss": 2.4292,
1507
+ "step": 214
1508
+ },
1509
+ {
1510
+ "epoch": 0.326561610024682,
1511
+ "grad_norm": 1.2890625,
1512
+ "learning_rate": 7.866761627730253e-06,
1513
+ "loss": 2.4925,
1514
+ "step": 215
1515
+ },
1516
+ {
1517
+ "epoch": 0.3280805012340991,
1518
+ "grad_norm": 1.0859375,
1519
+ "learning_rate": 7.846555050557381e-06,
1520
+ "loss": 2.4482,
1521
+ "step": 216
1522
+ },
1523
+ {
1524
+ "epoch": 0.3295993924435162,
1525
+ "grad_norm": 1.1640625,
1526
+ "learning_rate": 7.826279453043985e-06,
1527
+ "loss": 2.5279,
1528
+ "step": 217
1529
+ },
1530
+ {
1531
+ "epoch": 0.33111828365293333,
1532
+ "grad_norm": 0.99609375,
1533
+ "learning_rate": 7.805935326811913e-06,
1534
+ "loss": 2.4002,
1535
+ "step": 218
1536
+ },
1537
+ {
1538
+ "epoch": 0.3326371748623505,
1539
+ "grad_norm": 1.21875,
1540
+ "learning_rate": 7.78552316514462e-06,
1541
+ "loss": 2.3698,
1542
+ "step": 219
1543
+ },
1544
+ {
1545
+ "epoch": 0.3341560660717676,
1546
+ "grad_norm": 1.0859375,
1547
+ "learning_rate": 7.765043462975217e-06,
1548
+ "loss": 2.4626,
1549
+ "step": 220
1550
+ },
1551
+ {
1552
+ "epoch": 0.33567495728118474,
1553
+ "grad_norm": 1.15625,
1554
+ "learning_rate": 7.744496716874472e-06,
1555
+ "loss": 2.5538,
1556
+ "step": 221
1557
+ },
1558
+ {
1559
+ "epoch": 0.33719384849060186,
1560
+ "grad_norm": 1.109375,
1561
+ "learning_rate": 7.723883425038759e-06,
1562
+ "loss": 2.4648,
1563
+ "step": 222
1564
+ },
1565
+ {
1566
+ "epoch": 0.33871273970001897,
1567
+ "grad_norm": 0.984375,
1568
+ "learning_rate": 7.703204087277989e-06,
1569
+ "loss": 2.3859,
1570
+ "step": 223
1571
+ },
1572
+ {
1573
+ "epoch": 0.3402316309094361,
1574
+ "grad_norm": 1.015625,
1575
+ "learning_rate": 7.682459205003484e-06,
1576
+ "loss": 2.4553,
1577
+ "step": 224
1578
+ },
1579
+ {
1580
+ "epoch": 0.34175052211885326,
1581
+ "grad_norm": 1.2578125,
1582
+ "learning_rate": 7.661649281215823e-06,
1583
+ "loss": 2.5044,
1584
+ "step": 225
1585
+ },
1586
+ {
1587
+ "epoch": 0.3432694133282704,
1588
+ "grad_norm": 1.3828125,
1589
+ "learning_rate": 7.640774820492647e-06,
1590
+ "loss": 2.4061,
1591
+ "step": 226
1592
+ },
1593
+ {
1594
+ "epoch": 0.3447883045376875,
1595
+ "grad_norm": 1.21875,
1596
+ "learning_rate": 7.619836328976416e-06,
1597
+ "loss": 2.4957,
1598
+ "step": 227
1599
+ },
1600
+ {
1601
+ "epoch": 0.3463071957471046,
1602
+ "grad_norm": 1.09375,
1603
+ "learning_rate": 7.598834314362151e-06,
1604
+ "loss": 2.383,
1605
+ "step": 228
1606
+ },
1607
+ {
1608
+ "epoch": 0.34782608695652173,
1609
+ "grad_norm": 1.0546875,
1610
+ "learning_rate": 7.57776928588511e-06,
1611
+ "loss": 2.4998,
1612
+ "step": 229
1613
+ },
1614
+ {
1615
+ "epoch": 0.34934497816593885,
1616
+ "grad_norm": 1.1171875,
1617
+ "learning_rate": 7.556641754308447e-06,
1618
+ "loss": 2.4103,
1619
+ "step": 230
1620
+ },
1621
+ {
1622
+ "epoch": 0.35086386937535596,
1623
+ "grad_norm": 1.0390625,
1624
+ "learning_rate": 7.535452231910829e-06,
1625
+ "loss": 2.4448,
1626
+ "step": 231
1627
+ },
1628
+ {
1629
+ "epoch": 0.35238276058477314,
1630
+ "grad_norm": 1.2421875,
1631
+ "learning_rate": 7.514201232474012e-06,
1632
+ "loss": 2.4017,
1633
+ "step": 232
1634
+ },
1635
+ {
1636
+ "epoch": 0.35390165179419025,
1637
+ "grad_norm": 1.03125,
1638
+ "learning_rate": 7.492889271270382e-06,
1639
+ "loss": 2.4629,
1640
+ "step": 233
1641
+ },
1642
+ {
1643
+ "epoch": 0.35542054300360737,
1644
+ "grad_norm": 1.296875,
1645
+ "learning_rate": 7.471516865050468e-06,
1646
+ "loss": 2.4281,
1647
+ "step": 234
1648
+ },
1649
+ {
1650
+ "epoch": 0.3569394342130245,
1651
+ "grad_norm": 1.125,
1652
+ "learning_rate": 7.450084532030402e-06,
1653
+ "loss": 2.4231,
1654
+ "step": 235
1655
+ },
1656
+ {
1657
+ "epoch": 0.3584583254224416,
1658
+ "grad_norm": 1.0,
1659
+ "learning_rate": 7.428592791879361e-06,
1660
+ "loss": 2.4429,
1661
+ "step": 236
1662
+ },
1663
+ {
1664
+ "epoch": 0.3599772166318587,
1665
+ "grad_norm": 1.015625,
1666
+ "learning_rate": 7.407042165706969e-06,
1667
+ "loss": 2.469,
1668
+ "step": 237
1669
+ },
1670
+ {
1671
+ "epoch": 0.3614961078412759,
1672
+ "grad_norm": 1.0234375,
1673
+ "learning_rate": 7.385433176050654e-06,
1674
+ "loss": 2.5211,
1675
+ "step": 238
1676
+ },
1677
+ {
1678
+ "epoch": 0.363014999050693,
1679
+ "grad_norm": 1.2109375,
1680
+ "learning_rate": 7.36376634686298e-06,
1681
+ "loss": 2.4338,
1682
+ "step": 239
1683
+ },
1684
+ {
1685
+ "epoch": 0.3645338902601101,
1686
+ "grad_norm": 1.1171875,
1687
+ "learning_rate": 7.342042203498952e-06,
1688
+ "loss": 2.5161,
1689
+ "step": 240
1690
+ },
1691
+ {
1692
+ "epoch": 0.36605278146952724,
1693
+ "grad_norm": 1.171875,
1694
+ "learning_rate": 7.320261272703259e-06,
1695
+ "loss": 2.4728,
1696
+ "step": 241
1697
+ },
1698
+ {
1699
+ "epoch": 0.36757167267894436,
1700
+ "grad_norm": 1.203125,
1701
+ "learning_rate": 7.298424082597526e-06,
1702
+ "loss": 2.4609,
1703
+ "step": 242
1704
+ },
1705
+ {
1706
+ "epoch": 0.3690905638883615,
1707
+ "grad_norm": 1.21875,
1708
+ "learning_rate": 7.276531162667484e-06,
1709
+ "loss": 2.4977,
1710
+ "step": 243
1711
+ },
1712
+ {
1713
+ "epoch": 0.3706094550977786,
1714
+ "grad_norm": 1.203125,
1715
+ "learning_rate": 7.254583043750152e-06,
1716
+ "loss": 2.4324,
1717
+ "step": 244
1718
+ },
1719
+ {
1720
+ "epoch": 0.37212834630719577,
1721
+ "grad_norm": 1.1171875,
1722
+ "learning_rate": 7.232580258020952e-06,
1723
+ "loss": 2.4407,
1724
+ "step": 245
1725
+ },
1726
+ {
1727
+ "epoch": 0.3736472375166129,
1728
+ "grad_norm": 0.91796875,
1729
+ "learning_rate": 7.210523338980814e-06,
1730
+ "loss": 2.5021,
1731
+ "step": 246
1732
+ },
1733
+ {
1734
+ "epoch": 0.37516612872603,
1735
+ "grad_norm": 0.9453125,
1736
+ "learning_rate": 7.1884128214432366e-06,
1737
+ "loss": 2.4268,
1738
+ "step": 247
1739
+ },
1740
+ {
1741
+ "epoch": 0.3766850199354471,
1742
+ "grad_norm": 1.0390625,
1743
+ "learning_rate": 7.1662492415213194e-06,
1744
+ "loss": 2.4063,
1745
+ "step": 248
1746
+ },
1747
+ {
1748
+ "epoch": 0.37820391114486424,
1749
+ "grad_norm": 1.0078125,
1750
+ "learning_rate": 7.14403313661476e-06,
1751
+ "loss": 2.4253,
1752
+ "step": 249
1753
+ },
1754
+ {
1755
+ "epoch": 0.37972280235428135,
1756
+ "grad_norm": 1.265625,
1757
+ "learning_rate": 7.1217650453968335e-06,
1758
+ "loss": 2.461,
1759
+ "step": 250
1760
+ },
1761
+ {
1762
+ "epoch": 0.3812416935636985,
1763
+ "grad_norm": 1.109375,
1764
+ "learning_rate": 7.099445507801324e-06,
1765
+ "loss": 2.5152,
1766
+ "step": 251
1767
+ },
1768
+ {
1769
+ "epoch": 0.38276058477311564,
1770
+ "grad_norm": 1.03125,
1771
+ "learning_rate": 7.0770750650094335e-06,
1772
+ "loss": 2.4101,
1773
+ "step": 252
1774
+ },
1775
+ {
1776
+ "epoch": 0.38427947598253276,
1777
+ "grad_norm": 1.0234375,
1778
+ "learning_rate": 7.0546542594366605e-06,
1779
+ "loss": 2.5015,
1780
+ "step": 253
1781
+ },
1782
+ {
1783
+ "epoch": 0.3857983671919499,
1784
+ "grad_norm": 0.96875,
1785
+ "learning_rate": 7.03218363471965e-06,
1786
+ "loss": 2.4964,
1787
+ "step": 254
1788
+ },
1789
+ {
1790
+ "epoch": 0.387317258401367,
1791
+ "grad_norm": 1.03125,
1792
+ "learning_rate": 7.0096637357030105e-06,
1793
+ "loss": 2.4427,
1794
+ "step": 255
1795
+ },
1796
+ {
1797
+ "epoch": 0.3888361496107841,
1798
+ "grad_norm": 1.390625,
1799
+ "learning_rate": 6.987095108426102e-06,
1800
+ "loss": 2.3514,
1801
+ "step": 256
1802
+ },
1803
+ {
1804
+ "epoch": 0.3903550408202012,
1805
+ "grad_norm": 1.0390625,
1806
+ "learning_rate": 6.964478300109796e-06,
1807
+ "loss": 2.4921,
1808
+ "step": 257
1809
+ },
1810
+ {
1811
+ "epoch": 0.3918739320296184,
1812
+ "grad_norm": 0.9453125,
1813
+ "learning_rate": 6.94181385914321e-06,
1814
+ "loss": 2.4503,
1815
+ "step": 258
1816
+ },
1817
+ {
1818
+ "epoch": 0.3933928232390355,
1819
+ "grad_norm": 0.9375,
1820
+ "learning_rate": 6.91910233507041e-06,
1821
+ "loss": 2.3622,
1822
+ "step": 259
1823
+ },
1824
+ {
1825
+ "epoch": 0.39491171444845263,
1826
+ "grad_norm": 1.0703125,
1827
+ "learning_rate": 6.896344278577083e-06,
1828
+ "loss": 2.4076,
1829
+ "step": 260
1830
+ },
1831
+ {
1832
+ "epoch": 0.39643060565786975,
1833
+ "grad_norm": 0.9453125,
1834
+ "learning_rate": 6.873540241477189e-06,
1835
+ "loss": 2.4053,
1836
+ "step": 261
1837
+ },
1838
+ {
1839
+ "epoch": 0.39794949686728687,
1840
+ "grad_norm": 0.90625,
1841
+ "learning_rate": 6.850690776699574e-06,
1842
+ "loss": 2.4828,
1843
+ "step": 262
1844
+ },
1845
+ {
1846
+ "epoch": 0.399468388076704,
1847
+ "grad_norm": 1.0390625,
1848
+ "learning_rate": 6.8277964382745675e-06,
1849
+ "loss": 2.4118,
1850
+ "step": 263
1851
+ },
1852
+ {
1853
+ "epoch": 0.40098727928612116,
1854
+ "grad_norm": 1.1015625,
1855
+ "learning_rate": 6.804857781320558e-06,
1856
+ "loss": 2.4467,
1857
+ "step": 264
1858
+ },
1859
+ {
1860
+ "epoch": 0.4025061704955383,
1861
+ "grad_norm": 0.96875,
1862
+ "learning_rate": 6.781875362030512e-06,
1863
+ "loss": 2.5718,
1864
+ "step": 265
1865
+ },
1866
+ {
1867
+ "epoch": 0.4040250617049554,
1868
+ "grad_norm": 0.96875,
1869
+ "learning_rate": 6.758849737658508e-06,
1870
+ "loss": 2.5041,
1871
+ "step": 266
1872
+ },
1873
+ {
1874
+ "epoch": 0.4055439529143725,
1875
+ "grad_norm": 1.0625,
1876
+ "learning_rate": 6.735781466506216e-06,
1877
+ "loss": 2.4383,
1878
+ "step": 267
1879
+ },
1880
+ {
1881
+ "epoch": 0.4070628441237896,
1882
+ "grad_norm": 1.0625,
1883
+ "learning_rate": 6.712671107909359e-06,
1884
+ "loss": 2.478,
1885
+ "step": 268
1886
+ },
1887
+ {
1888
+ "epoch": 0.40858173533320674,
1889
+ "grad_norm": 1.046875,
1890
+ "learning_rate": 6.6895192222241534e-06,
1891
+ "loss": 2.4207,
1892
+ "step": 269
1893
+ },
1894
+ {
1895
+ "epoch": 0.41010062654262386,
1896
+ "grad_norm": 0.98828125,
1897
+ "learning_rate": 6.666326370813722e-06,
1898
+ "loss": 2.4511,
1899
+ "step": 270
1900
+ },
1901
+ {
1902
+ "epoch": 0.41161951775204103,
1903
+ "grad_norm": 1.0078125,
1904
+ "learning_rate": 6.643093116034486e-06,
1905
+ "loss": 2.4288,
1906
+ "step": 271
1907
+ },
1908
+ {
1909
+ "epoch": 0.41313840896145815,
1910
+ "grad_norm": 0.9609375,
1911
+ "learning_rate": 6.619820021222518e-06,
1912
+ "loss": 2.437,
1913
+ "step": 272
1914
+ },
1915
+ {
1916
+ "epoch": 0.41465730017087526,
1917
+ "grad_norm": 1.0625,
1918
+ "learning_rate": 6.5965076506799e-06,
1919
+ "loss": 2.4671,
1920
+ "step": 273
1921
+ },
1922
+ {
1923
+ "epoch": 0.4161761913802924,
1924
+ "grad_norm": 1.1171875,
1925
+ "learning_rate": 6.573156569661026e-06,
1926
+ "loss": 2.5018,
1927
+ "step": 274
1928
+ },
1929
+ {
1930
+ "epoch": 0.4176950825897095,
1931
+ "grad_norm": 0.890625,
1932
+ "learning_rate": 6.549767344358903e-06,
1933
+ "loss": 2.3747,
1934
+ "step": 275
1935
+ },
1936
+ {
1937
+ "epoch": 0.4192139737991266,
1938
+ "grad_norm": 1.140625,
1939
+ "learning_rate": 6.526340541891418e-06,
1940
+ "loss": 2.4567,
1941
+ "step": 276
1942
+ },
1943
+ {
1944
+ "epoch": 0.4207328650085438,
1945
+ "grad_norm": 1.1484375,
1946
+ "learning_rate": 6.5028767302875974e-06,
1947
+ "loss": 2.4371,
1948
+ "step": 277
1949
+ },
1950
+ {
1951
+ "epoch": 0.4222517562179609,
1952
+ "grad_norm": 0.984375,
1953
+ "learning_rate": 6.479376478473822e-06,
1954
+ "loss": 2.3193,
1955
+ "step": 278
1956
+ },
1957
+ {
1958
+ "epoch": 0.423770647427378,
1959
+ "grad_norm": 1.1171875,
1960
+ "learning_rate": 6.455840356260041e-06,
1961
+ "loss": 2.4587,
1962
+ "step": 279
1963
+ },
1964
+ {
1965
+ "epoch": 0.42528953863679514,
1966
+ "grad_norm": 1.1484375,
1967
+ "learning_rate": 6.432268934325947e-06,
1968
+ "loss": 2.4221,
1969
+ "step": 280
1970
+ },
1971
+ {
1972
+ "epoch": 0.42680842984621226,
1973
+ "grad_norm": 0.99609375,
1974
+ "learning_rate": 6.408662784207149e-06,
1975
+ "loss": 2.4534,
1976
+ "step": 281
1977
+ },
1978
+ {
1979
+ "epoch": 0.4283273210556294,
1980
+ "grad_norm": 1.1640625,
1981
+ "learning_rate": 6.385022478281307e-06,
1982
+ "loss": 2.4384,
1983
+ "step": 282
1984
+ },
1985
+ {
1986
+ "epoch": 0.4298462122650465,
1987
+ "grad_norm": 1.15625,
1988
+ "learning_rate": 6.361348589754255e-06,
1989
+ "loss": 2.4464,
1990
+ "step": 283
1991
+ },
1992
+ {
1993
+ "epoch": 0.43136510347446366,
1994
+ "grad_norm": 0.98828125,
1995
+ "learning_rate": 6.337641692646106e-06,
1996
+ "loss": 2.4387,
1997
+ "step": 284
1998
+ },
1999
+ {
2000
+ "epoch": 0.4328839946838808,
2001
+ "grad_norm": 1.2734375,
2002
+ "learning_rate": 6.313902361777327e-06,
2003
+ "loss": 2.5006,
2004
+ "step": 285
2005
+ },
2006
+ {
2007
+ "epoch": 0.4344028858932979,
2008
+ "grad_norm": 1.359375,
2009
+ "learning_rate": 6.290131172754811e-06,
2010
+ "loss": 2.5047,
2011
+ "step": 286
2012
+ },
2013
+ {
2014
+ "epoch": 0.435921777102715,
2015
+ "grad_norm": 0.94140625,
2016
+ "learning_rate": 6.266328701957911e-06,
2017
+ "loss": 2.3543,
2018
+ "step": 287
2019
+ },
2020
+ {
2021
+ "epoch": 0.43744066831213213,
2022
+ "grad_norm": 1.171875,
2023
+ "learning_rate": 6.24249552652447e-06,
2024
+ "loss": 2.5089,
2025
+ "step": 288
2026
+ },
2027
+ {
2028
+ "epoch": 0.43895955952154925,
2029
+ "grad_norm": 1.09375,
2030
+ "learning_rate": 6.2186322243368236e-06,
2031
+ "loss": 2.4545,
2032
+ "step": 289
2033
+ },
2034
+ {
2035
+ "epoch": 0.4404784507309664,
2036
+ "grad_norm": 0.96484375,
2037
+ "learning_rate": 6.194739374007792e-06,
2038
+ "loss": 2.4691,
2039
+ "step": 290
2040
+ },
2041
+ {
2042
+ "epoch": 0.44199734194038354,
2043
+ "grad_norm": 1.0625,
2044
+ "learning_rate": 6.170817554866646e-06,
2045
+ "loss": 2.5019,
2046
+ "step": 291
2047
+ },
2048
+ {
2049
+ "epoch": 0.44351623314980065,
2050
+ "grad_norm": 1.03125,
2051
+ "learning_rate": 6.1468673469450655e-06,
2052
+ "loss": 2.4309,
2053
+ "step": 292
2054
+ },
2055
+ {
2056
+ "epoch": 0.44503512435921777,
2057
+ "grad_norm": 0.91796875,
2058
+ "learning_rate": 6.122889330963069e-06,
2059
+ "loss": 2.4174,
2060
+ "step": 293
2061
+ },
2062
+ {
2063
+ "epoch": 0.4465540155686349,
2064
+ "grad_norm": 1.140625,
2065
+ "learning_rate": 6.098884088314938e-06,
2066
+ "loss": 2.4323,
2067
+ "step": 294
2068
+ },
2069
+ {
2070
+ "epoch": 0.448072906778052,
2071
+ "grad_norm": 1.1171875,
2072
+ "learning_rate": 6.074852201055121e-06,
2073
+ "loss": 2.4597,
2074
+ "step": 295
2075
+ },
2076
+ {
2077
+ "epoch": 0.4495917979874691,
2078
+ "grad_norm": 1.046875,
2079
+ "learning_rate": 6.050794251884112e-06,
2080
+ "loss": 2.4288,
2081
+ "step": 296
2082
+ },
2083
+ {
2084
+ "epoch": 0.4511106891968863,
2085
+ "grad_norm": 0.96875,
2086
+ "learning_rate": 6.026710824134331e-06,
2087
+ "loss": 2.4338,
2088
+ "step": 297
2089
+ },
2090
+ {
2091
+ "epoch": 0.4526295804063034,
2092
+ "grad_norm": 0.94140625,
2093
+ "learning_rate": 6.002602501755974e-06,
2094
+ "loss": 2.4154,
2095
+ "step": 298
2096
+ },
2097
+ {
2098
+ "epoch": 0.45414847161572053,
2099
+ "grad_norm": 0.9765625,
2100
+ "learning_rate": 5.978469869302861e-06,
2101
+ "loss": 2.4921,
2102
+ "step": 299
2103
+ },
2104
+ {
2105
+ "epoch": 0.45566736282513765,
2106
+ "grad_norm": 1.0625,
2107
+ "learning_rate": 5.954313511918252e-06,
2108
+ "loss": 2.4616,
2109
+ "step": 300
2110
+ },
2111
+ {
2112
+ "epoch": 0.45718625403455476,
2113
+ "grad_norm": 0.9296875,
2114
+ "learning_rate": 5.9301340153206685e-06,
2115
+ "loss": 2.4872,
2116
+ "step": 301
2117
+ },
2118
+ {
2119
+ "epoch": 0.4587051452439719,
2120
+ "grad_norm": 1.0390625,
2121
+ "learning_rate": 5.905931965789688e-06,
2122
+ "loss": 2.4421,
2123
+ "step": 302
2124
+ },
2125
+ {
2126
+ "epoch": 0.46022403645338905,
2127
+ "grad_norm": 0.9375,
2128
+ "learning_rate": 5.881707950151725e-06,
2129
+ "loss": 2.4822,
2130
+ "step": 303
2131
+ },
2132
+ {
2133
+ "epoch": 0.46174292766280617,
2134
+ "grad_norm": 1.140625,
2135
+ "learning_rate": 5.857462555765809e-06,
2136
+ "loss": 2.4562,
2137
+ "step": 304
2138
+ },
2139
+ {
2140
+ "epoch": 0.4632618188722233,
2141
+ "grad_norm": 1.0234375,
2142
+ "learning_rate": 5.8331963705093375e-06,
2143
+ "loss": 2.5125,
2144
+ "step": 305
2145
+ },
2146
+ {
2147
+ "epoch": 0.4647807100816404,
2148
+ "grad_norm": 1.078125,
2149
+ "learning_rate": 5.808909982763825e-06,
2150
+ "loss": 2.4263,
2151
+ "step": 306
2152
+ },
2153
+ {
2154
+ "epoch": 0.4662996012910575,
2155
+ "grad_norm": 1.15625,
2156
+ "learning_rate": 5.784603981400632e-06,
2157
+ "loss": 2.4522,
2158
+ "step": 307
2159
+ },
2160
+ {
2161
+ "epoch": 0.46781849250047464,
2162
+ "grad_norm": 1.09375,
2163
+ "learning_rate": 5.760278955766695e-06,
2164
+ "loss": 2.4475,
2165
+ "step": 308
2166
+ },
2167
+ {
2168
+ "epoch": 0.46933738370989175,
2169
+ "grad_norm": 0.91796875,
2170
+ "learning_rate": 5.735935495670229e-06,
2171
+ "loss": 2.4194,
2172
+ "step": 309
2173
+ },
2174
+ {
2175
+ "epoch": 0.4708562749193089,
2176
+ "grad_norm": 0.9921875,
2177
+ "learning_rate": 5.711574191366427e-06,
2178
+ "loss": 2.3649,
2179
+ "step": 310
2180
+ },
2181
+ {
2182
+ "epoch": 0.47237516612872604,
2183
+ "grad_norm": 1.046875,
2184
+ "learning_rate": 5.687195633543151e-06,
2185
+ "loss": 2.4821,
2186
+ "step": 311
2187
+ },
2188
+ {
2189
+ "epoch": 0.47389405733814316,
2190
+ "grad_norm": 0.9296875,
2191
+ "learning_rate": 5.662800413306611e-06,
2192
+ "loss": 2.475,
2193
+ "step": 312
2194
+ },
2195
+ {
2196
+ "epoch": 0.4754129485475603,
2197
+ "grad_norm": 1.0078125,
2198
+ "learning_rate": 5.6383891221670275e-06,
2199
+ "loss": 2.3078,
2200
+ "step": 313
2201
+ },
2202
+ {
2203
+ "epoch": 0.4769318397569774,
2204
+ "grad_norm": 1.1171875,
2205
+ "learning_rate": 5.613962352024293e-06,
2206
+ "loss": 2.3925,
2207
+ "step": 314
2208
+ },
2209
+ {
2210
+ "epoch": 0.4784507309663945,
2211
+ "grad_norm": 1.09375,
2212
+ "learning_rate": 5.589520695153618e-06,
2213
+ "loss": 2.4178,
2214
+ "step": 315
2215
+ },
2216
+ {
2217
+ "epoch": 0.4799696221758117,
2218
+ "grad_norm": 0.890625,
2219
+ "learning_rate": 5.5650647441911706e-06,
2220
+ "loss": 2.5059,
2221
+ "step": 316
2222
+ },
2223
+ {
2224
+ "epoch": 0.4814885133852288,
2225
+ "grad_norm": 1.078125,
2226
+ "learning_rate": 5.540595092119709e-06,
2227
+ "loss": 2.5159,
2228
+ "step": 317
2229
+ },
2230
+ {
2231
+ "epoch": 0.4830074045946459,
2232
+ "grad_norm": 1.1328125,
2233
+ "learning_rate": 5.516112332254203e-06,
2234
+ "loss": 2.334,
2235
+ "step": 318
2236
+ },
2237
+ {
2238
+ "epoch": 0.48452629580406303,
2239
+ "grad_norm": 0.953125,
2240
+ "learning_rate": 5.491617058227443e-06,
2241
+ "loss": 2.4316,
2242
+ "step": 319
2243
+ },
2244
+ {
2245
+ "epoch": 0.48604518701348015,
2246
+ "grad_norm": 1.046875,
2247
+ "learning_rate": 5.46710986397565e-06,
2248
+ "loss": 2.5104,
2249
+ "step": 320
2250
+ },
2251
+ {
2252
+ "epoch": 0.48756407822289727,
2253
+ "grad_norm": 1.046875,
2254
+ "learning_rate": 5.442591343724081e-06,
2255
+ "loss": 2.4709,
2256
+ "step": 321
2257
+ },
2258
+ {
2259
+ "epoch": 0.4890829694323144,
2260
+ "grad_norm": 0.93359375,
2261
+ "learning_rate": 5.418062091972604e-06,
2262
+ "loss": 2.4407,
2263
+ "step": 322
2264
+ },
2265
+ {
2266
+ "epoch": 0.49060186064173156,
2267
+ "grad_norm": 0.8828125,
2268
+ "learning_rate": 5.393522703481303e-06,
2269
+ "loss": 2.3959,
2270
+ "step": 323
2271
+ },
2272
+ {
2273
+ "epoch": 0.4921207518511487,
2274
+ "grad_norm": 1.1015625,
2275
+ "learning_rate": 5.36897377325604e-06,
2276
+ "loss": 2.4172,
2277
+ "step": 324
2278
+ },
2279
+ {
2280
+ "epoch": 0.4936396430605658,
2281
+ "grad_norm": 1.0078125,
2282
+ "learning_rate": 5.344415896534039e-06,
2283
+ "loss": 2.3965,
2284
+ "step": 325
2285
+ },
2286
+ {
2287
+ "epoch": 0.4951585342699829,
2288
+ "grad_norm": 0.90625,
2289
+ "learning_rate": 5.319849668769449e-06,
2290
+ "loss": 2.4589,
2291
+ "step": 326
2292
+ },
2293
+ {
2294
+ "epoch": 0.4966774254794,
2295
+ "grad_norm": 0.9375,
2296
+ "learning_rate": 5.295275685618905e-06,
2297
+ "loss": 2.3558,
2298
+ "step": 327
2299
+ },
2300
+ {
2301
+ "epoch": 0.49819631668881714,
2302
+ "grad_norm": 0.99609375,
2303
+ "learning_rate": 5.270694542927089e-06,
2304
+ "loss": 2.5089,
2305
+ "step": 328
2306
+ },
2307
+ {
2308
+ "epoch": 0.4997152078982343,
2309
+ "grad_norm": 1.015625,
2310
+ "learning_rate": 5.246106836712277e-06,
2311
+ "loss": 2.4541,
2312
+ "step": 329
2313
+ },
2314
+ {
2315
+ "epoch": 0.5012340991076514,
2316
+ "grad_norm": 0.91796875,
2317
+ "learning_rate": 5.2215131631518945e-06,
2318
+ "loss": 2.3858,
2319
+ "step": 330
2320
+ },
2321
+ {
2322
+ "epoch": 0.5027529903170685,
2323
+ "grad_norm": 1.0078125,
2324
+ "learning_rate": 5.196914118568054e-06,
2325
+ "loss": 2.4189,
2326
+ "step": 331
2327
+ },
2328
+ {
2329
+ "epoch": 0.5042718815264856,
2330
+ "grad_norm": 1.03125,
2331
+ "learning_rate": 5.1723102994130994e-06,
2332
+ "loss": 2.4426,
2333
+ "step": 332
2334
+ },
2335
+ {
2336
+ "epoch": 0.5057907727359028,
2337
+ "grad_norm": 1.0625,
2338
+ "learning_rate": 5.147702302255143e-06,
2339
+ "loss": 2.4418,
2340
+ "step": 333
2341
+ },
2342
+ {
2343
+ "epoch": 0.50730966394532,
2344
+ "grad_norm": 0.97265625,
2345
+ "learning_rate": 5.123090723763607e-06,
2346
+ "loss": 2.3768,
2347
+ "step": 334
2348
+ },
2349
+ {
2350
+ "epoch": 0.508828555154737,
2351
+ "grad_norm": 0.9296875,
2352
+ "learning_rate": 5.098476160694741e-06,
2353
+ "loss": 2.4267,
2354
+ "step": 335
2355
+ },
2356
+ {
2357
+ "epoch": 0.5103474463641542,
2358
+ "grad_norm": 1.046875,
2359
+ "learning_rate": 5.073859209877167e-06,
2360
+ "loss": 2.4311,
2361
+ "step": 336
2362
+ },
2363
+ {
2364
+ "epoch": 0.5118663375735713,
2365
+ "grad_norm": 0.99609375,
2366
+ "learning_rate": 5.049240468197401e-06,
2367
+ "loss": 2.5122,
2368
+ "step": 337
2369
+ },
2370
+ {
2371
+ "epoch": 0.5133852287829884,
2372
+ "grad_norm": 0.8828125,
2373
+ "learning_rate": 5.0246205325853824e-06,
2374
+ "loss": 2.384,
2375
+ "step": 338
2376
+ },
2377
+ {
2378
+ "epoch": 0.5149041199924056,
2379
+ "grad_norm": 0.9375,
2380
+ "learning_rate": 5e-06,
2381
+ "loss": 2.4921,
2382
+ "step": 339
2383
+ },
2384
+ {
2385
+ "epoch": 0.5164230112018227,
2386
+ "grad_norm": 1.0390625,
2387
+ "learning_rate": 4.975379467414621e-06,
2388
+ "loss": 2.4624,
2389
+ "step": 340
2390
+ },
2391
+ {
2392
+ "epoch": 0.5179419024112398,
2393
+ "grad_norm": 0.92578125,
2394
+ "learning_rate": 4.950759531802602e-06,
2395
+ "loss": 2.402,
2396
+ "step": 341
2397
+ },
2398
+ {
2399
+ "epoch": 0.5194607936206569,
2400
+ "grad_norm": 0.90625,
2401
+ "learning_rate": 4.926140790122835e-06,
2402
+ "loss": 2.4924,
2403
+ "step": 342
2404
+ },
2405
+ {
2406
+ "epoch": 0.5209796848300741,
2407
+ "grad_norm": 0.89453125,
2408
+ "learning_rate": 4.90152383930526e-06,
2409
+ "loss": 2.4247,
2410
+ "step": 343
2411
+ },
2412
+ {
2413
+ "epoch": 0.5224985760394911,
2414
+ "grad_norm": 0.9609375,
2415
+ "learning_rate": 4.876909276236395e-06,
2416
+ "loss": 2.3653,
2417
+ "step": 344
2418
+ },
2419
+ {
2420
+ "epoch": 0.5240174672489083,
2421
+ "grad_norm": 0.96484375,
2422
+ "learning_rate": 4.852297697744857e-06,
2423
+ "loss": 2.4583,
2424
+ "step": 345
2425
+ },
2426
+ {
2427
+ "epoch": 0.5255363584583255,
2428
+ "grad_norm": 1.0078125,
2429
+ "learning_rate": 4.827689700586902e-06,
2430
+ "loss": 2.4109,
2431
+ "step": 346
2432
+ },
2433
+ {
2434
+ "epoch": 0.5270552496677425,
2435
+ "grad_norm": 0.86328125,
2436
+ "learning_rate": 4.803085881431949e-06,
2437
+ "loss": 2.4321,
2438
+ "step": 347
2439
+ },
2440
+ {
2441
+ "epoch": 0.5285741408771597,
2442
+ "grad_norm": 0.98046875,
2443
+ "learning_rate": 4.778486836848107e-06,
2444
+ "loss": 2.5462,
2445
+ "step": 348
2446
+ },
2447
+ {
2448
+ "epoch": 0.5300930320865768,
2449
+ "grad_norm": 0.9921875,
2450
+ "learning_rate": 4.7538931632877254e-06,
2451
+ "loss": 2.3965,
2452
+ "step": 349
2453
+ },
2454
+ {
2455
+ "epoch": 0.5316119232959939,
2456
+ "grad_norm": 1.03125,
2457
+ "learning_rate": 4.729305457072913e-06,
2458
+ "loss": 2.4898,
2459
+ "step": 350
2460
+ },
2461
+ {
2462
+ "epoch": 0.533130814505411,
2463
+ "grad_norm": 0.9140625,
2464
+ "learning_rate": 4.704724314381097e-06,
2465
+ "loss": 2.4746,
2466
+ "step": 351
2467
+ },
2468
+ {
2469
+ "epoch": 0.5346497057148282,
2470
+ "grad_norm": 0.875,
2471
+ "learning_rate": 4.680150331230552e-06,
2472
+ "loss": 2.3629,
2473
+ "step": 352
2474
+ },
2475
+ {
2476
+ "epoch": 0.5361685969242453,
2477
+ "grad_norm": 1.0234375,
2478
+ "learning_rate": 4.6555841034659625e-06,
2479
+ "loss": 2.3738,
2480
+ "step": 353
2481
+ },
2482
+ {
2483
+ "epoch": 0.5376874881336624,
2484
+ "grad_norm": 0.9140625,
2485
+ "learning_rate": 4.631026226743962e-06,
2486
+ "loss": 2.4069,
2487
+ "step": 354
2488
+ },
2489
+ {
2490
+ "epoch": 0.5392063793430796,
2491
+ "grad_norm": 0.921875,
2492
+ "learning_rate": 4.606477296518698e-06,
2493
+ "loss": 2.344,
2494
+ "step": 355
2495
+ },
2496
+ {
2497
+ "epoch": 0.5407252705524966,
2498
+ "grad_norm": 0.91015625,
2499
+ "learning_rate": 4.581937908027397e-06,
2500
+ "loss": 2.4081,
2501
+ "step": 356
2502
+ },
2503
+ {
2504
+ "epoch": 0.5422441617619138,
2505
+ "grad_norm": 0.921875,
2506
+ "learning_rate": 4.55740865627592e-06,
2507
+ "loss": 2.4161,
2508
+ "step": 357
2509
+ },
2510
+ {
2511
+ "epoch": 0.5437630529713309,
2512
+ "grad_norm": 0.91796875,
2513
+ "learning_rate": 4.532890136024351e-06,
2514
+ "loss": 2.4184,
2515
+ "step": 358
2516
+ },
2517
+ {
2518
+ "epoch": 0.545281944180748,
2519
+ "grad_norm": 0.93359375,
2520
+ "learning_rate": 4.508382941772558e-06,
2521
+ "loss": 2.4924,
2522
+ "step": 359
2523
+ },
2524
+ {
2525
+ "epoch": 0.5468008353901652,
2526
+ "grad_norm": 0.90234375,
2527
+ "learning_rate": 4.483887667745798e-06,
2528
+ "loss": 2.3855,
2529
+ "step": 360
2530
+ },
2531
+ {
2532
+ "epoch": 0.5483197265995823,
2533
+ "grad_norm": 0.96484375,
2534
+ "learning_rate": 4.459404907880293e-06,
2535
+ "loss": 2.4468,
2536
+ "step": 361
2537
+ },
2538
+ {
2539
+ "epoch": 0.5498386178089995,
2540
+ "grad_norm": 0.953125,
2541
+ "learning_rate": 4.434935255808831e-06,
2542
+ "loss": 2.3802,
2543
+ "step": 362
2544
+ },
2545
+ {
2546
+ "epoch": 0.5513575090184165,
2547
+ "grad_norm": 1.0078125,
2548
+ "learning_rate": 4.410479304846385e-06,
2549
+ "loss": 2.35,
2550
+ "step": 363
2551
+ },
2552
+ {
2553
+ "epoch": 0.5528764002278337,
2554
+ "grad_norm": 0.91796875,
2555
+ "learning_rate": 4.386037647975708e-06,
2556
+ "loss": 2.3869,
2557
+ "step": 364
2558
+ },
2559
+ {
2560
+ "epoch": 0.5543952914372509,
2561
+ "grad_norm": 1.265625,
2562
+ "learning_rate": 4.361610877832974e-06,
2563
+ "loss": 2.41,
2564
+ "step": 365
2565
+ },
2566
+ {
2567
+ "epoch": 0.5559141826466679,
2568
+ "grad_norm": 0.9296875,
2569
+ "learning_rate": 4.337199586693389e-06,
2570
+ "loss": 2.3689,
2571
+ "step": 366
2572
+ },
2573
+ {
2574
+ "epoch": 0.5574330738560851,
2575
+ "grad_norm": 1.0078125,
2576
+ "learning_rate": 4.312804366456851e-06,
2577
+ "loss": 2.4504,
2578
+ "step": 367
2579
+ },
2580
+ {
2581
+ "epoch": 0.5589519650655022,
2582
+ "grad_norm": 0.98046875,
2583
+ "learning_rate": 4.2884258086335755e-06,
2584
+ "loss": 2.4652,
2585
+ "step": 368
2586
+ },
2587
+ {
2588
+ "epoch": 0.5604708562749193,
2589
+ "grad_norm": 0.9140625,
2590
+ "learning_rate": 4.2640645043297715e-06,
2591
+ "loss": 2.4019,
2592
+ "step": 369
2593
+ },
2594
+ {
2595
+ "epoch": 0.5619897474843364,
2596
+ "grad_norm": 0.8828125,
2597
+ "learning_rate": 4.239721044233306e-06,
2598
+ "loss": 2.306,
2599
+ "step": 370
2600
+ },
2601
+ {
2602
+ "epoch": 0.5635086386937536,
2603
+ "grad_norm": 0.85546875,
2604
+ "learning_rate": 4.215396018599369e-06,
2605
+ "loss": 2.3859,
2606
+ "step": 371
2607
+ },
2608
+ {
2609
+ "epoch": 0.5650275299031707,
2610
+ "grad_norm": 0.890625,
2611
+ "learning_rate": 4.191090017236177e-06,
2612
+ "loss": 2.3734,
2613
+ "step": 372
2614
+ },
2615
+ {
2616
+ "epoch": 0.5665464211125878,
2617
+ "grad_norm": 1.0078125,
2618
+ "learning_rate": 4.166803629490664e-06,
2619
+ "loss": 2.4194,
2620
+ "step": 373
2621
+ },
2622
+ {
2623
+ "epoch": 0.568065312322005,
2624
+ "grad_norm": 0.93359375,
2625
+ "learning_rate": 4.142537444234192e-06,
2626
+ "loss": 2.4815,
2627
+ "step": 374
2628
+ },
2629
+ {
2630
+ "epoch": 0.569584203531422,
2631
+ "grad_norm": 0.8828125,
2632
+ "learning_rate": 4.118292049848277e-06,
2633
+ "loss": 2.4493,
2634
+ "step": 375
2635
+ },
2636
+ {
2637
+ "epoch": 0.5711030947408392,
2638
+ "grad_norm": 0.8828125,
2639
+ "learning_rate": 4.094068034210313e-06,
2640
+ "loss": 2.4458,
2641
+ "step": 376
2642
+ },
2643
+ {
2644
+ "epoch": 0.5726219859502563,
2645
+ "grad_norm": 0.91796875,
2646
+ "learning_rate": 4.069865984679332e-06,
2647
+ "loss": 2.4004,
2648
+ "step": 377
2649
+ },
2650
+ {
2651
+ "epoch": 0.5741408771596734,
2652
+ "grad_norm": 0.87109375,
2653
+ "learning_rate": 4.045686488081748e-06,
2654
+ "loss": 2.4643,
2655
+ "step": 378
2656
+ },
2657
+ {
2658
+ "epoch": 0.5756597683690906,
2659
+ "grad_norm": 0.91796875,
2660
+ "learning_rate": 4.021530130697141e-06,
2661
+ "loss": 2.4709,
2662
+ "step": 379
2663
+ },
2664
+ {
2665
+ "epoch": 0.5771786595785077,
2666
+ "grad_norm": 0.84765625,
2667
+ "learning_rate": 3.997397498244028e-06,
2668
+ "loss": 2.4853,
2669
+ "step": 380
2670
+ },
2671
+ {
2672
+ "epoch": 0.5786975507879248,
2673
+ "grad_norm": 0.9453125,
2674
+ "learning_rate": 3.97328917586567e-06,
2675
+ "loss": 2.5029,
2676
+ "step": 381
2677
+ },
2678
+ {
2679
+ "epoch": 0.5802164419973419,
2680
+ "grad_norm": 0.95703125,
2681
+ "learning_rate": 3.9492057481158905e-06,
2682
+ "loss": 2.4853,
2683
+ "step": 382
2684
+ },
2685
+ {
2686
+ "epoch": 0.5817353332067591,
2687
+ "grad_norm": 1.046875,
2688
+ "learning_rate": 3.92514779894488e-06,
2689
+ "loss": 2.4058,
2690
+ "step": 383
2691
+ },
2692
+ {
2693
+ "epoch": 0.5832542244161761,
2694
+ "grad_norm": 1.2578125,
2695
+ "learning_rate": 3.901115911685063e-06,
2696
+ "loss": 2.4426,
2697
+ "step": 384
2698
+ },
2699
+ {
2700
+ "epoch": 0.5847731156255933,
2701
+ "grad_norm": 1.25,
2702
+ "learning_rate": 3.877110669036932e-06,
2703
+ "loss": 2.4151,
2704
+ "step": 385
2705
+ },
2706
+ {
2707
+ "epoch": 0.5862920068350105,
2708
+ "grad_norm": 0.89453125,
2709
+ "learning_rate": 3.853132653054936e-06,
2710
+ "loss": 2.4061,
2711
+ "step": 386
2712
+ },
2713
+ {
2714
+ "epoch": 0.5878108980444275,
2715
+ "grad_norm": 0.89453125,
2716
+ "learning_rate": 3.829182445133356e-06,
2717
+ "loss": 2.4192,
2718
+ "step": 387
2719
+ },
2720
+ {
2721
+ "epoch": 0.5893297892538447,
2722
+ "grad_norm": 0.95703125,
2723
+ "learning_rate": 3.8052606259922097e-06,
2724
+ "loss": 2.3988,
2725
+ "step": 388
2726
+ },
2727
+ {
2728
+ "epoch": 0.5908486804632618,
2729
+ "grad_norm": 0.9140625,
2730
+ "learning_rate": 3.7813677756631773e-06,
2731
+ "loss": 2.4392,
2732
+ "step": 389
2733
+ },
2734
+ {
2735
+ "epoch": 0.592367571672679,
2736
+ "grad_norm": 0.94140625,
2737
+ "learning_rate": 3.75750447347553e-06,
2738
+ "loss": 2.3739,
2739
+ "step": 390
2740
+ },
2741
+ {
2742
+ "epoch": 0.5938864628820961,
2743
+ "grad_norm": 0.91796875,
2744
+ "learning_rate": 3.7336712980420897e-06,
2745
+ "loss": 2.4588,
2746
+ "step": 391
2747
+ },
2748
+ {
2749
+ "epoch": 0.5954053540915132,
2750
+ "grad_norm": 0.91015625,
2751
+ "learning_rate": 3.7098688272451893e-06,
2752
+ "loss": 2.4266,
2753
+ "step": 392
2754
+ },
2755
+ {
2756
+ "epoch": 0.5969242453009304,
2757
+ "grad_norm": 0.97265625,
2758
+ "learning_rate": 3.6860976382226747e-06,
2759
+ "loss": 2.3617,
2760
+ "step": 393
2761
+ },
2762
+ {
2763
+ "epoch": 0.5984431365103474,
2764
+ "grad_norm": 1.109375,
2765
+ "learning_rate": 3.662358307353897e-06,
2766
+ "loss": 2.4095,
2767
+ "step": 394
2768
+ },
2769
+ {
2770
+ "epoch": 0.5999620277197646,
2771
+ "grad_norm": 0.921875,
2772
+ "learning_rate": 3.638651410245746e-06,
2773
+ "loss": 2.5,
2774
+ "step": 395
2775
+ },
2776
+ {
2777
+ "epoch": 0.6014809189291817,
2778
+ "grad_norm": 0.96484375,
2779
+ "learning_rate": 3.6149775217186954e-06,
2780
+ "loss": 2.4601,
2781
+ "step": 396
2782
+ },
2783
+ {
2784
+ "epoch": 0.6029998101385988,
2785
+ "grad_norm": 0.89453125,
2786
+ "learning_rate": 3.5913372157928515e-06,
2787
+ "loss": 2.4496,
2788
+ "step": 397
2789
+ },
2790
+ {
2791
+ "epoch": 0.604518701348016,
2792
+ "grad_norm": 0.90234375,
2793
+ "learning_rate": 3.5677310656740537e-06,
2794
+ "loss": 2.367,
2795
+ "step": 398
2796
+ },
2797
+ {
2798
+ "epoch": 0.6060375925574331,
2799
+ "grad_norm": 0.85546875,
2800
+ "learning_rate": 3.5441596437399596e-06,
2801
+ "loss": 2.4182,
2802
+ "step": 399
2803
+ },
2804
+ {
2805
+ "epoch": 0.6075564837668502,
2806
+ "grad_norm": 0.91015625,
2807
+ "learning_rate": 3.5206235215261785e-06,
2808
+ "loss": 2.4485,
2809
+ "step": 400
2810
+ }
2811
+ ],
2812
+ "logging_steps": 1,
2813
+ "max_steps": 658,
2814
+ "num_input_tokens_seen": 0,
2815
+ "num_train_epochs": 1,
2816
+ "save_steps": 200,
2817
+ "total_flos": 7.313516697275597e+18,
2818
+ "train_batch_size": 4,
2819
+ "trial_name": null,
2820
+ "trial_params": null
2821
+ }
checkpoint-400/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e854ad22a418f911f565f571c2d76df063735273533ddc9fc799dc2a35352fe0
3
+ size 5112
trainer_log.jsonl ADDED
The diff for this file is too large to render. See raw diff