Update README.md
Browse files
README.md
CHANGED
@@ -95,6 +95,14 @@ SQL Query:
|
|
95 |
SELECT avg(age), min(age), max(age) FROM singer WHERE country = 'France'
|
96 |
```
|
97 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
98 |
|
99 |
## Training
|
100 |
|
|
|
95 |
SELECT avg(age), min(age), max(age) FROM singer WHERE country = 'France'
|
96 |
```
|
97 |
|
98 |
+
## Evaluation
|
99 |
+
Evaluation was done on the dev split of the Spider and Spider-syn dataset. The databases present in the dev split have no intersection with the databases of the train split.
|
100 |
+
This way we ensure, that the model was not exposed to the evaluated databases during training.
|
101 |
+
The evaluation was done by comparing the results of querying the database using the generated query and reference.
|
102 |
+
Both Spider and Spider-Syn dev splits have 1032 samples.
|
103 |
+
* **Spider dev accuracy:** 49.2%
|
104 |
+
* **Spider Syn dev accuracy:** 39.5%
|
105 |
+
|
106 |
|
107 |
## Training
|
108 |
|