results/results.txt

|    |   accuracy |   bits |   clip | dataset   | method   | model   | params    |   peak_mem_alloc_mb |   peak_mem_reserv_mb |   perplexity |   time_ms |   wer |
|---:|-----------:|-------:|-------:|:----------|:---------|:--------|:----------|--------------------:|---------------------:|-------------:|----------:|------:|
|  0 |     0      |      6 |      0 | wikitext  | comb_2   | llama3  | nlr_t_cos |            14178.7  |             15367.9  |       1.1456 |  121.684  |     0 |
|  1 |     0      |      6 |      0 | wikitext  | comb_6   | llama3  | nlr_t_cos |            14178.7  |             15367.9  |       1.1248 |  145.985  |     0 |
|  2 |     0.4383 |      6 |      0 | wikitext  | comb_15  | llama3  | nlr_t_cos |            14179.2  |             15367.9  |       5.2511 |   68.6528 |     0 |
|  3 |     0.4409 |      6 |      0 | wikitext  | woq      | llama3  | fixed     |            14179.3  |             15367.9  |       5.1933 |   95.7922 |     0 |
|  4 |     0.0079 |      6 |      0 | wikitext  | comb_16  | llama3  | nlr_t_cos |            14185.3  |             15374.2  |       1.1336 |  148.303  |     0 |
|  5 |     0      |      6 |      0 | wikitext  | comb_9   | llama3  | nlr_t_cos |            14185.3  |             15374.2  |       1.1548 |  201.816  |     0 |
|  6 |     0.4387 |      6 |      0 | wikitext  | comb_11  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2293 |  147.834  |     0 |
|  7 |     0.4391 |      6 |      0 | wikitext  | comb_17  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2298 |  147.986  |     0 |
|  8 |     0.4384 |      6 |      0 | wikitext  | comb_18  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2482 |  148.162  |     0 |
|  9 |     0.4383 |      6 |      0 | wikitext  | comb_1   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2489 |  148.4    |     0 |
| 10 |     0.4385 |      6 |      0 | wikitext  | comb_5   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2621 |  175.837  |     0 |
| 11 |     0.4379 |      6 |      0 | wikitext  | comb_8   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2538 |  201.555  |     0 |
| 12 |     0      |      6 |      0 | wikitext  | comb_14  | llama3  | nlr_t_cos |            11051.9  |             18748.5  |       1.1556 |    4.1952 |     0 |
|  0 |     0      |      7 |      0 | wikitext  | comb_2   | llama3  | nlr_t_cos |            14178.7  |             15367.9  |       1.1227 |  121.439  |     0 |
|  1 |     0      |      7 |      0 | wikitext  | comb_6   | llama3  | nlr_t_cos |            14178.7  |             15367.9  |       1.1248 |  146.225  |     0 |
|  2 |     0.4421 |      7 |      0 | wikitext  | comb_15  | llama3  | nlr_t_cos |            14179.2  |             15367.9  |       5.2372 |   68.8236 |     0 |
|  3 |     0.4428 |      7 |      0 | wikitext  | woq      | llama3  | fixed     |            14179.3  |             15367.9  |       5.2489 |   95.747  |     0 |
|  4 |     0      |      7 |      0 | wikitext  | comb_16  | llama3  | nlr_t_cos |            14185.3  |             15374.2  |       1.1507 |  147.971  |     0 |
|  5 |     0.0111 |      7 |      0 | wikitext  | comb_9   | llama3  | nlr_t_cos |            14185.3  |             15374.2  |       1.1087 |  201.585  |     0 |
|  6 |     0.4423 |      7 |      0 | wikitext  | comb_17  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2384 |  147.836  |     0 |
|  7 |     0.4427 |      7 |      0 | wikitext  | comb_11  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2469 |  147.862  |     0 |
|  8 |     0.4433 |      7 |      0 | wikitext  | comb_18  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2537 |  148.004  |     0 |
|  9 |     0.4436 |      7 |      0 | wikitext  | comb_1   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2402 |  148.318  |     0 |
| 10 |     0.4427 |      7 |      0 | wikitext  | comb_5   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2546 |  175.422  |     0 |
| 11 |     0.4434 |      7 |      0 | wikitext  | comb_8   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2389 |  201.752  |     0 |
| 12 |     0      |      7 |      0 | wikitext  | comb_14  | llama3  | nlr_t_cos |            11051.9  |             18748.5  |       1.1359 |    4.4018 |     0 |
|  0 |     0.4278 |      4 |      0 | wikitext  | ao_woq   | llama3  | fixed     |             5542.62 |              7163.87 |       5.3584 |   38.8297 |     0 |
|  1 |     0      |      4 |      0 | wikitext  | comb_2   | llama3  | nlr_t_cos |            14178.7  |             15367.9  |       1.2404 |  121.642  |     0 |
|  2 |     0      |      4 |      0 | wikitext  | comb_6   | llama3  | nlr_t_cos |            14178.7  |             15367.9  |       1.1248 |  145.886  |     0 |
|  3 |     0.3747 |      4 |      0 | wikitext  | comb_15  | llama3  | nlr_t_cos |            14179.2  |             15367.9  |       6.6227 |   68.6559 |     0 |
|  4 |     0.3798 |      4 |      0 | wikitext  | woq      | llama3  | fixed     |            14179.3  |             15367.9  |       6.1632 |   95.7422 |     0 |
|  5 |     0.0032 |      4 |      0 | wikitext  | comb_16  | llama3  | nlr_t_cos |            14185.3  |             15374.2  |       1.1276 |  147.916  |     0 |
|  6 |     0      |      4 |      0 | wikitext  | comb_9   | llama3  | nlr_t_cos |            14185.3  |             15374.2  |       1.136  |  201.328  |     0 |
|  7 |     0.3766 |      4 |      0 | wikitext  | comb_11  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       6.5586 |  147.472  |     0 |
|  8 |     0.3757 |      4 |      0 | wikitext  | comb_18  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       6.5769 |  147.926  |     0 |
|  9 |     0.3764 |      4 |      0 | wikitext  | comb_17  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       6.5346 |  148.008  |     0 |
| 10 |     0.3754 |      4 |      0 | wikitext  | comb_1   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       6.6261 |  148.223  |     0 |
| 11 |     0.3713 |      4 |      0 | wikitext  | comb_5   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       6.7536 |  175.571  |     0 |
| 12 |     0.3745 |      4 |      0 | wikitext  | comb_8   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       6.5784 |  201.374  |     0 |
| 13 |     0      |      4 |      0 | wikitext  | comb_14  | llama3  | nlr_t_cos |            11051.9  |             18748.5  |       1.1425 |    4.3992 |     0 |
|  0 |     0.4437 |      8 |      0 | wikitext  | ao_woq   | llama3  | fixed     |            12110.4  |             14040.4  |       5.2086 |   52.4845 |     0 |
|  1 |     0      |      8 |      0 | wikitext  | comb_2   | llama3  | nlr_t_cos |            14178.7  |             15367.9  |       1.1235 |  122.586  |     0 |
|  2 |     0      |      8 |      0 | wikitext  | comb_6   | llama3  | nlr_t_cos |            14178.7  |             15367.9  |       1.1248 |  146.344  |     0 |
|  3 |     0.4436 |      8 |      0 | wikitext  | comb_15  | llama3  | nlr_t_cos |            14179.2  |             15367.9  |       5.2049 |   69.0691 |     0 |
|  4 |     0.4439 |      8 |      0 | wikitext  | woq      | llama3  | fixed     |            14179.3  |             15367.9  |       5.2082 |   95.9166 |     0 |
|  5 |     0      |      8 |      0 | wikitext  | comb_4   | llama3  | nlr_t_cos |            14185.3  |             15374.2  |       1.2091 |  148.202  |     0 |
|  6 |     0.4438 |      8 |      0 | wikitext  | comb_17  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2125 |  147.821  |     0 |
|  7 |     0.4438 |      8 |      0 | wikitext  | comb_18  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2044 |  147.877  |     0 |
|  8 |     0.4432 |      8 |      0 | wikitext  | comb_11  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2164 |  147.889  |     0 |
|  9 |     0.4431 |      8 |      0 | wikitext  | comb_1   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2058 |  148.012  |     0 |
| 10 |     0.1085 |      8 |      0 | wikitext  | comb_16  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |      33.6885 |  148.312  |     0 |
| 11 |     0.4436 |      8 |      0 | wikitext  | comb_5   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2039 |  175.21   |     0 |
| 12 |     0.4436 |      8 |      0 | wikitext  | comb_8   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.2028 |  201.585  |     0 |
| 13 |     0.3125 |      8 |      0 | wikitext  | comb_9   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       8.284  |  201.68   |     0 |
| 14 |     0      |      8 |      0 | wikitext  | comb_14  | llama3  | nlr_t_cos |            11051.9  |             18748.5  |       1.1416 |    4.4107 |     0 |
|  0 |     0      |      5 |      0 | wikitext  | comb_2   | llama3  | nlr_t_cos |            14178.7  |             15367.9  |       1.22   |  121.859  |     0 |
|  1 |     0      |      5 |      0 | wikitext  | comb_6   | llama3  | nlr_t_cos |            14178.7  |             15367.9  |       1.1248 |  146.231  |     0 |
|  2 |     0.4124 |      5 |      0 | wikitext  | comb_15  | llama3  | nlr_t_cos |            14179.2  |             15367.9  |       5.5999 |   68.7519 |     0 |
|  3 |     0.4356 |      5 |      0 | wikitext  | woq      | llama3  | fixed     |            14179.3  |             15367.9  |       5.449  |   95.6505 |     0 |
|  4 |     0      |      5 |      0 | wikitext  | comb_9   | llama3  | nlr_t_cos |            14185.3  |             15374.2  |       1.1338 |  201.643  |     0 |
|  5 |     0.0159 |      5 |      0 | wikitext  | comb_16  | llama3  | nlr_t_cos |            14185.3  |             15374.2  |       1.4766 |  148.071  |     0 |
|  6 |     0.4134 |      5 |      0 | wikitext  | comb_11  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.6022 |  148.017  |     0 |
|  7 |     0.4105 |      5 |      0 | wikitext  | comb_18  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.5852 |  148.104  |     0 |
|  8 |     0.4139 |      5 |      0 | wikitext  | comb_1   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.5405 |  148.131  |     0 |
|  9 |     0.4129 |      5 |      0 | wikitext  | comb_17  | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.6089 |  148.177  |     0 |
| 10 |     0.4107 |      5 |      0 | wikitext  | comb_5   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.5705 |  175.83   |     0 |
| 11 |     0.4095 |      5 |      0 | wikitext  | comb_8   | llama3  | nlr_t_cos |            14185.7  |             15374.2  |       5.6048 |  201.444  |     0 |
| 12 |     0      |      5 |      0 | wikitext  | comb_14  | llama3  | nlr_t_cos |            11051.9  |             18748.5  |       1.1452 |    4.4253 |     0 |
| 13 |     0.4437 |     32 |      0 | wikitext  | base     | llama3  | ----      |            33003.5  |             33156    |       5.2238 |   27.3757 |     0 |