Add FreebaseQA dataset
Opened by anaerobeth (closed)

This PR adds the FreebaseQA dataset: A Trivia-type QA Data Set over the Freebase Knowledge Graph

Repo: https://github.com/kelvin-jiang/FreebaseQA

Paper: https://www.aclweb.org/anthology/N19-1028.pdf


## TODO: create dummy data

Error encountered when running `python datasets-cli dummy_data datasets/freebase_qa --auto_generate`
```
    f"Couldn't parse columns {list(json_data.keys())}. "
ValueError: Couldn't parse columns ['Dataset', 'Version', 'Questions']. Maybe specify which json field must be used to read the data with --json_field <my_field>.
```


Comments: @yjernite @lhoestq Any suggestions on how to get the dummy data generator to recognize the columns? The structure of the json is:
```
{
  "Dataset": "FreebaseQA-eval", 
  "Version": "1.0", 
  "Questions": [
    {
      "Question-ID": "FreebaseQA-eval-0", 
      "RawQuestion": "Who is the female presenter of the Channel 4 quiz show '1001 things you should know'?", 
      "ProcessedQuestion": "who is the female presenter of the channel 4 quiz show '1001 things you should know'", 
      "Parses": [
        {
          "Parse-Id": "FreebaseQA-eval-0.P0", 
          "PotentialTopicEntityMention": "1001 things you should know", 
          "TopicEntityName": "1001 things you should know", 
          "TopicEntityMid": "m.0nd3t34", 
          "InferentialChain": "tv.tv_program.regular_personal_appearances..tv.tv_regular_personal_appearance.person", 
          "Answers": [
            {
              "AnswersMid": "m.0216y_", 
              "AnswersName": [
                "sandi toksvig"
              ]
            }
          ]
        }
      ]
    }, 
   ...
  ]
}
```

Thanks!

Unfortunately this json structure is not recognized by the auto-generation yet, so you'd have to create the dummy data manually. 
You can get some instructions on how to do that with: `python datasets-cli dummy_data datasets/freebase_qa`
We can definitely help you with that if there are too many files! 

@yjernite Thanks for the instructions. I manually added dummy data and created the zip file but one of the splits seem to return an empty list.

```
tests/test_dataset_common.py F                        [100%]

========================= FAILURES ==========================
_ LocalDatasetTest.test_load_dataset_all_configs_freebase_qa _

self = <tests.test_dataset_common.LocalDatasetTest testMethod=test_load_dataset_all_configs_freebase_qa>
dataset_name = 'freebase_qa'

    @slow
    def test_load_dataset_all_configs(self, dataset_name):
        configs = self.dataset_tester.load_all_configs(dataset_name, is_local=True)
>       self.dataset_tester.check_load_dataset(dataset_name, configs, is_local=True)

tests/test_dataset_common.py:237:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_dataset_common.py:198: in check_load_dataset
    self.parent.assertTrue(len(dataset[split]) > 0)
E   AssertionError: False is not true
```

Note that the dataset has `train`, `eval`, and `dev` (no test split). I am not sure if I am mapping them correctly when I called the Split Generator.


The dummy json files must follow the exact same structure as the original json files.

However it looks like the dummy json files you have in your dummy_data.zip file are not structured the same way.
For example the original json is a dict with a field "Questions" that is a list of items.
However your dummy json is simply a list of items.

Can you update your dummy json files to follow the same structure ?

And I'm pretty sure that this structure is supported by the dummy data auto-generation tool
```
python datasets-cli dummy_data ./datasets/freebase_qa --json_field "Questions"
```

Hi @anaerobeth did you manage to get the dummy data right ?

Feel free to ping me if you have questions or when you're ready for a review

Thanks for your help! I am able to create the dummy data with the dict structure as suggested. I'll add the tags and update this PR shortly.

Also don't forget to run `make style` to fix the code formatting check in the CI :)

Hi @anaerobeth ! Have you had a chance to consider updating the dataset script to yield one example per question ?

Feel free to ping me if you have questions or if I can help :) 

Hi @lhoestq,

I am willing to take this forward if you and @anaerobeth don't mind.


Hi @gchhablani thanks for proposing your help :) 
Sure if you want to take this forward feel free to do so.
Also pinging @anaerobeth to make sure that you both don't work on the same thing at the same time

Hi ! Closing this one since the dataset was added in #1814 

Thanks you two @anaerobeth and @gchhablani for adding this dataset !