rishdotblog commited on
Commit
a88aee5
1 Parent(s): a742c27

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -20,7 +20,7 @@ SQLCoder is fine-tuned on a base StarCoder model.
20
  | model | perc_correct |
21
  |-|-|
22
  | gpt4-2023-10-04 | 82.0 |
23
- | defog-sqlcoder2 | 74.5 |
24
  | gpt4-2023-08-28 | 74.0 |
25
  | defog-sqlcoder-7b | 71.0 |
26
  | gpt-3.5-2023-10-04 | 66.0 |
@@ -41,11 +41,11 @@ You can read more about our [training approach](https://defog.ai/blog/open-sourc
41
  We classified each generated question into one of 5 categories. The table displays the percentage of questions answered correctly by each model, broken down by category.
42
  | query_category | gpt-4 | sqlcoder2-15b | sqlcoder-7b | gpt-3.5 | claude-2 | claude-instant | gpt-3 |
43
  |:-----------------|--------:|----------------:|--------------:|----------:|-----------:|-----------------:|--------:|
44
- | date | 72 | 76 | 64 | 68 | 52 | 48 | 32 |
45
- | group_by | 91.4 | 80 | 82.9 | 77.1 | 71.4 | 71.4 | 71.4 |
46
  | order_by | 82.9 | 77.1 | 74.3 | 68.6 | 74.3 | 74.3 | 68.6 |
47
- | ratio | 80 | 60 | 54.3 | 37.1 | 57.1 | 45.7 | 25.7 |
48
- | join | 82.9 | 77.1 | 74.3 | 71.4 | 65.7 | 62.9 | 57.1 |
49
  | where | 80 | 77.1 | 74.3 | 74.3 | 62.9 | 60 | 54.3 |
50
 
51
  ## Using SQLCoder
 
20
  | model | perc_correct |
21
  |-|-|
22
  | gpt4-2023-10-04 | 82.0 |
23
+ | defog-sqlcoder2 | 77.5 |
24
  | gpt4-2023-08-28 | 74.0 |
25
  | defog-sqlcoder-7b | 71.0 |
26
  | gpt-3.5-2023-10-04 | 66.0 |
 
41
  We classified each generated question into one of 5 categories. The table displays the percentage of questions answered correctly by each model, broken down by category.
42
  | query_category | gpt-4 | sqlcoder2-15b | sqlcoder-7b | gpt-3.5 | claude-2 | claude-instant | gpt-3 |
43
  |:-----------------|--------:|----------------:|--------------:|----------:|-----------:|-----------------:|--------:|
44
+ | date | 72 | 80 | 64 | 68 | 52 | 48 | 32 |
45
+ | group_by | 91.4 | 82.9 | 82.9 | 77.1 | 71.4 | 71.4 | 71.4 |
46
  | order_by | 82.9 | 77.1 | 74.3 | 68.6 | 74.3 | 74.3 | 68.6 |
47
+ | ratio | 80 | 74.3 | 54.3 | 37.1 | 57.1 | 45.7 | 25.7 |
48
+ | join | 82.9 | 74.3 | 74.3 | 71.4 | 65.7 | 62.9 | 57.1 |
49
  | where | 80 | 77.1 | 74.3 | 74.3 | 62.9 | 60 | 54.3 |
50
 
51
  ## Using SQLCoder