Update evaluation/demo_humaneval.md
Browse files
evaluation/demo_humaneval.md
CHANGED
@@ -1,11 +1,12 @@
|
|
1 |
|
2 |
-
We can load HumanEval dataset and pass@k metric from 🤗 [`datasets`](https://huggingface.co/docs/datasets/index)
|
3 |
|
4 |
```python
|
5 |
-
from datasets import load_dataset
|
|
|
6 |
|
7 |
human_eval = load_dataset("openai_humaneval")
|
8 |
-
code_eval_metric =
|
9 |
```
|
10 |
|
11 |
We can easily compute the pass@k for a problem that asks for the implementation of a function that sums two integers:
|
|
|
1 |
|
2 |
+
We can load HumanEval dataset and pass@k metric from 🤗 [`datasets`](https://huggingface.co/docs/datasets/index) and 🤗 [`evaluate`](https://huggingface.co/docs/evaluate/index)
|
3 |
|
4 |
```python
|
5 |
+
from datasets import load_dataset
|
6 |
+
from evaluate import load
|
7 |
|
8 |
human_eval = load_dataset("openai_humaneval")
|
9 |
+
code_eval_metric = load("code_eval")
|
10 |
```
|
11 |
|
12 |
We can easily compute the pass@k for a problem that asks for the implementation of a function that sums two integers:
|