NGrov commited on
Commit
57fed75
1 Parent(s): 23ceea2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -95,6 +95,14 @@ SQL Query:
95
  SELECT avg(age), min(age), max(age) FROM singer WHERE country = 'France'
96
  ```
97
 
 
 
 
 
 
 
 
 
98
 
99
  ## Training
100
 
 
95
  SELECT avg(age), min(age), max(age) FROM singer WHERE country = 'France'
96
  ```
97
 
98
+ ## Evaluation
99
+ Evaluation was done on the dev split of the Spider and Spider-syn dataset. The databases present in the dev split have no intersection with the databases of the train split.
100
+ This way we ensure, that the model was not exposed to the evaluated databases during training.
101
+ The evaluation was done by comparing the results of querying the database using the generated query and reference.
102
+ Both Spider and Spider-Syn dev splits have 1032 samples.
103
+ * **Spider dev accuracy:** 49.2%
104
+ * **Spider Syn dev accuracy:** 39.5%
105
+
106
 
107
  ## Training
108