tscholak commited on
Commit
749931d
1 Parent(s): edfd83e

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +75 -0
README.md ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ thumbnail: "https://repository-images.githubusercontent.com/401779782/c2f46be5-b74b-4620-ad64-57487be3b1ab"
5
+ tags:
6
+ - text2sql
7
+ widget:
8
+ - "How many singers do we have? | concert_singer | stadium : stadium_id, location, name, capacity, highest, lowest, average | singer : singer_id, name, country, song_name, song_release_year, age, is_male | concert : concert_id, concert_name, theme, stadium_id, year | singer_in_concert : concert_id, singer_id"
9
+ license: "Apache 2.0"
10
+ datasets:
11
+ - spider
12
+ metrics:
13
+ - spider
14
+ ---
15
+
16
+ ## tscholak/3vnuv1vf
17
+
18
+ Fine-tuned weights for [PICARD - Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models](https://arxiv.org/abs/2109.05093) based on [t5.1.1.lm100k.base](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k).
19
+
20
+
21
+ ### Training Data
22
+
23
+ The model has been fine-tuned on the 7000 training examples in the [Spider text-to-SQL dataset](https://yale-lily.github.io/spider). The model solves Spider's zero-shot text-to-SQL translation task, and that means that it can generalize to unseen SQL databases.
24
+
25
+
26
+ ### Training Objective
27
+
28
+ This model was initialized with [t5.1.1.lm100k.base](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) and fine-tuned with the text-to-text generation objective.
29
+
30
+ Questions are always grounded in a database schema, and the model is trained to predict the SQL query that would be used to answer the question. The input to the model is composed of the user's natural language question, the database identifier, and a list of tables and their columns:
31
+
32
+ ```
33
+ [question] | [db_id] | [table] : [column] ( [content] , [content] ) , [column] ( ... ) , [...] | [table] : ... | ...
34
+ ```
35
+
36
+ The model outputs the database identifier and the SQL query that will be executed on the database to answer the user's question:
37
+
38
+ ```
39
+ [db_id] | [sql]
40
+ ```
41
+
42
+
43
+ ### Performance
44
+
45
+ Out of the box, this model achieves 59.4 % exact-set match accuracy and 60.0 % execution accuracy on the Spider development set.
46
+
47
+ Using the PICARD constrained decoding method (see [the official PICARD implementation](https://github.com/ElementAI/picard)), the model's performance can be improved to **66.6 %** exact-set match accuracy and **68.4 %** execution accuracy on the Spider development set.
48
+
49
+
50
+ ### Usage
51
+
52
+ Please see [the official repository](https://github.com/ElementAI/picard) for scripts and docker images that support evaluation and serving of this model.
53
+
54
+
55
+ ### References
56
+
57
+ 1. [PICARD - Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models](https://arxiv.org/abs/2109.05093)
58
+
59
+ 2. [Official PICARD code](https://github.com/ElementAI/picard)
60
+
61
+
62
+ ### Citation
63
+
64
+ ```bibtex
65
+ @inproceedings{Scholak2021:PICARD,
66
+ author = {Torsten Scholak and Nathan Schucher and Dzmitry Bahdanau},
67
+ title = "{PICARD}: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models",
68
+ booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
69
+ month = nov,
70
+ year = "2021",
71
+ publisher = "Association for Computational Linguistics",
72
+ url = "https://aclanthology.org/2021.emnlp-main.779",
73
+ pages = "9895--9901",
74
+ }
75
+ ```