# An Investigation of Applying Large Language Models to Spoken Language Learning
## 3 Experiments and Results
### Zero-Shot and Few-Shot Benchmarks
 & \(\Delta\) & **Overall** \\ \hline LLAMM2-7B-Chat & Direct & 0 & 72.2\% (104) & & \(31.2\%\) (94) & & \(44.5\%\) \\ LLAM2-7B-Chat & Direct & 3 & 70.8\% (102) & \(-1.4\%\) & \(31.0\%\) (93) & \(-0.2\%\) & \(43.8\%\) \\ LLAM2-7B-Chat & GT & 0 & \(86.1\%\) (124) & \(+13.9\%\) & \(28.6\%\) (66) & \(-2.6\%\) & \(47.2\%\) \\ LLAM2-7B-Chat & CoT & 3 & \(77.1\%\) (111) & \(+4.9\%\) & \(33.2\%\) (100) & \(+2.0\%\) & \(47.4\%\) \\ \hline Pythia-7B & Direct & 0 & \(2.8\%\) (4) & & \(18.9\%\) (57) & & \(13.7\%\) \\ Pythia-7B & Direct & 3 & \(15.3\%\) (22) & \(+12.5\%\) & \(15.3\%\) (46) & \(-3.6\%\) & \(15.3\%\) \\ Pythia-7B & GT & 0 & \(8.3\%\) (12) & \(+5.5\%\) & \(12.0\%\) (66) & \(-6.9\%\) & \(10.8\%\) \\ Pythia-7B & CoT & 3 & \(14.6\%\) (21) & \(+11.8\%\) & \(26.9\%\) (81) & \(+8.0\%\) & \(22.9\%\) \\ \hline Flam-75-XLL-11B & Direct & 0 & \(88.2\%\) (127) & & \(46.2\%\) (139) & & \(59.8\%\) \\ Flam-75-XLL-11B & Direct & 3 & \(88.9\%\) (128) & \(+0.7\%\) & \(47.8\%\) (144) & \(+1.6\%\) & \(61.1\%\) \\ Flam-75-XLL-11B & OGT & 0 & \(79.2\%\) (114) & \(-9.0\%\) & \(37.9\%\) (114) & \(-8.3\%\) & \(51.2\%\) \\ Flam-75-XLL-11B & CoT & 3 & \(86.1\%\) (124) & \(-2.1\%\) & \(40.5\%\) (122) & \(-5.7\%\) & \(55.3\%\) \\ \hline Pythia-12B & Direct & 0 & \(20.1\%\) (29) & & \(24.9\%\) (75) & & \(23.4\%\) \\ Pythia-12B & Direct & 3 & \(20.1\%\) (29) & \(+0.0\%\) & \(26.6\%\) (80) & \(+1.7\%\) & \(24.5\%\) \\ Pythia-12B & CoT & 0 & \(18.1\%\) (26) & \(-2.0\%\) & \(19.3\%\) (88) & \(-5.6\%\) & \(18.9\%\) \\ Pythia-12B & CoT & 3 & \(22.2\%\) (32) & \(+2.1\%\) & \(2.3\%\) (67) & \(-2.6\%\) & \(22.2\%\) \\ \hline LLAM1-13B & Direct & 0 & \(33.3\%\) (48) & & \(7.6\%\) (21) & & \(15.9\%\) \\ LLAM1-13B & Direct & 3 & \(66.7\%\) (96) & \(+33.4\%\) & \(26.2\%\) (79) & \(+18.6\%\) & \(39.3\%\) \\ LLAM1-13B & CoT & 0 & \(25.0\%\) (36) & \(-8.5\%\) & \(11.3\%\) (34) & \(+3.7\%\) & \(15.7\%\) \\ LLAM1-13B & CoT & 3 & \(65.3\%\) (94) & \(+32.0\%\) & \(33.6\%\) (101) & \(+26.0\%\) & \(43.8\%\) \\ \hline LLAM2-13B & Direct & 0 & \(72.2\%\) (104) & & \(30.2\%\) (91) & & \(43.8\%\) \\ LLAM2-13B & Direct & 3 & \(81.3\%\) (117) & \(+9.1\%\) & \(16.3\%\) (49) & \(-9.3\%\) & \(37.3\%\) \\ LLAM2-13B & CoT & 0 & \(25.7\%\) (37) & \(+46.5\%\) & \(10.6\%\) (32) & \(-19.6\%\) & \(15.5\%\) \\ LLAM2-13B & CoT & 3 & \(83.3\%\) (120) & \(+11.1\%\) & \(42.2\%\) (127) & \(+19.9\%\) & \(55.5\%\) \\ LLAM2-13B & Chatt & 0 & \(88.2\%\) (127) & & \(35.2\%\) (106) & & \(52.4\%\) \\ LLAM2-13B & Chatt & 3 & \(74.3\%\) (107) & \(-13.9\%\) & \(36.2\%\) (109) & \(-1.0\%\) & \(48.5\%\) \\ LLAM2-13B & Chatt & 0 & \(84.7\%\) (122) & \(-3.5\%\) & \(38.9\%\) (117) & \(+3.7\%\) & \(53.7\%\) \\ LLAM2-13B & CaT & 3 & \(85.4\%\) (123) & \(-2.8\%\) & \(40.2\%\) (121) & \(+5.0\%\) & \(54.8\%\) \\ \hline Vicuma-13B & Direct & 0 & \(63.2\%\) (91) & & \(19.9\%\) (60) & & \(3.9\%\) \\ Vicuma-13B & Direct & 3 & \(6.9\%\) (10) & \(-56.3\%\) & \(10.6\%\) (32) & \(-9.3\%\) & \(9.4\%\) \\ Vicuma-13B & CoT & 0 & \(72.9\%\) (105) & \(+9.7\%\) & \(30.6\%\) (92) & \(+10.7\%\) & \(44.3\%\) \\ Vicuma-13B & CoT & 3 & \(83.3\%\) (120) & \(+20.1\%\) & \(38.9\%\) (117) & \(+9.0\%\) & \(53.3\%\) \\ \hline Flan-ULE-20B & Direct & 0 & \(87.5\%\) (126) & & \(41.9\%\) (126) & & \(56.6\%\) \\ Flam-ULE-20B & Direct & 3 & \(88.9\%\) (128) & \(+1.4\%\) & \(44.5\%\) (134) & \(+2.6\%\) & \(58.9\%\) \\ Flan-ULE-20B & CoT & 0 & \(86.8\%\) (125) & \(-0.2\%\) & \(38.9\%\) (117) & \(-3.0\%\) & \(54.4\%\) \\ Flam-ULE-20B & CoT & 3 & \(88.9\%\) (128) & \(+1.4\%\) & \(38.5\%\) (116) & \(-3.4\%\) & \(54.8\%\) \\ \hline LLAM1-30B & Direct & 0 & \(78.5\%\) (113) & & \(0.7\%\) (2) & & \(25.8\%\) \\ LLAM1-30B & Direct & 3 & \(86.8\%\) (125) & \(+8.3\%\) & \(38.5\%\) (116) & \(+37.8\%\) & \(54.2\%\) \\ LLAM1-30B & CoT & 0 & \(25.7\%\) (37) & \(-52.8\%\) & \(31.6\%\) (41) & \(+18.9\%\subsets, which are plotted in Figure 4. The values used to create the box plots were taken from the \"Concept (144)\" and \"Applied Questions (301)\" columns in Table 3, respectively.