SlowGuess/NAACL_2022_OCR huggingface dataset:
Due to the nature of KBQA tasks, LC-QuAD 1.0 involves questions about certain real-world entities usually including persons, organizations or objects (which is exactly the conditions where KBQA is applied to real-world applications). However, most information about the individuals (e.g. name, team, etc.) are publicly available (since the dataset utilizes DBpedia as background KB while DBpedia mainly collects data from publicly available Wikipedia). Further, when annotating the dataset, we perform a brief manual check on potential offensive or biased contents, to the best of our efforts, we find no apparent offensive hints in the questions and SPARQL queries. Hence, we believe that LC-QuAD 1.0 under intended KBQA use has minor potential to offend others or cause privacy issues.
D Details of Data Annotation
Annotation Guidelines We adopt the same annotation format as Zhang et al., 2021 to annotate the LC-QuAD 1.0 dataset. Specifically, for each node in the query graph corresponding to the SPARQLs in the dataset, the mention of such node in the question is annotated. All annotated mentions are required as whole-words (e.g. including the 's' for plural words), the mention is left as "None" when no mention of a node can be found. There are
certain cases where multiple mentions co-refer a node, we encourage annotators to choose a mention containing more concrete semantics, while all of these mentions are acceptable (e.g. for the question "Who is Jack's dad?", both "Who" and "dad" are correct mentions but the latter is encouraged since it indicates more semantics of the node). We provide a detailed guideline<sup>3</sup> to annotators with extra discussions on marginal cases to further aid the annotation. Also, we discuss the potential risks and the overall usage of such annotations to get agreements from the annotators in the guideline.
Annotation Process We recruit 9 annotators with necessary background knowledge from school, consisting of 5 undergraduate and 4 graduate students, to fulfil the annotation task. By completing the annotation, we provide essential payments for each annotator. Finally, we use a script to auto-check the collected annotations and perform basic corrections (e.g. align all mentions to whole-words).
Table 5 details our hyperparameter settings.
Name	Description	Setting
hrb	Hidden size of the RoBERTa encoder	1024
hbi	Hidden size of the biaffine model	256
hle	Dimension of the label embedding	256
τ	Gumbel-softmax temperature	0.05
optim	Optimizer to train both GSG and RE models	AdamW
β1/β2	Betas of the AdamW optimizer	0.9 / 0.9
wd	Weight decay rate of the AdamW optimizer	1e-5
lrrb	Learning rate of the RoBERTa encoder in GSG	1e-5
lrsgs	Learning rate of other parameters in GSG	5e-5
batchsgs	Batch size of the GSG model	64
lrre	Learning rate of the RE model	1e-5
batchre	Batch size of the RE model	100
b	Beam width in RE	4
Table 5: Detailed hyperparameter settings in this work.
F Ethical Statements
Considering the nature of NLP-based QA systems, our method keeps the risk to output false (e.g. incorrect answers to factoid questions) or biased (e.g. imprecise count of answer numbers) answers, which might cause issues in trustworthy or practical uses. However, we'd like to clarify that this work is intended for discovering more accurate and efficient systems on KBQA regardless of the exact content in a KB, the answers to specific questions given by our method does not reflect the authors' point of view.
See the guideline in supplementary materials.