Update README.md
Browse files
README.md
CHANGED
@@ -49,7 +49,7 @@ Our 8B parameter model achieves competitive or superior performance compared to
|
|
49 |
|
50 |
The resulting model is competitive with state of the art LLMs despite its small size. We evaluate our model on KernelBench which is an open-source benchmark to evaluate the ability of LLMs to write efficient GPU kernels. It contains 250 selected PyTorch modules organized into difficulty levels, from single torch operators such as Conv2D or Swish (level 1), to full model architectures (level 3). The benchmark measures both correctness (by comparing against reference PyTorch outputs) and performance (by measuring speedup over baseline implementations). We implemented a new KernelBench-Triton variant that evaluates an LLMs ability to generate Triton kernels, making it an ideal benchmark for evaluating KernelLLM's capabilities. All our measurements were done on Nvidia H100 GPUs.
|
51 |

|
52 |
-
|
53 |
|
54 |
|
55 |
For more information, please see [Project Popcorn](https://gpu-mode.github.io/popcorn/).
|
|
|
49 |
|
50 |
The resulting model is competitive with state of the art LLMs despite its small size. We evaluate our model on KernelBench which is an open-source benchmark to evaluate the ability of LLMs to write efficient GPU kernels. It contains 250 selected PyTorch modules organized into difficulty levels, from single torch operators such as Conv2D or Swish (level 1), to full model architectures (level 3). The benchmark measures both correctness (by comparing against reference PyTorch outputs) and performance (by measuring speedup over baseline implementations). We implemented a new KernelBench-Triton variant that evaluates an LLMs ability to generate Triton kernels, making it an ideal benchmark for evaluating KernelLLM's capabilities. All our measurements were done on Nvidia H100 GPUs.
|
51 |

|
52 |
+
*KernelLLM shows quasi log-linear scaling behavior during pass@k analysis.*
|
53 |
|
54 |
|
55 |
For more information, please see [Project Popcorn](https://gpu-mode.github.io/popcorn/).
|