We present a method for inducing taxonomic trees from pretrained transformers. Given a set of input terms, we assign a score for the likelihood that each pair of terms forms a parent-child relation . To produce a tree from pairwise parent-child edge scores, we treat this as a graph optimization problem and output the maximum spanning tree  . We train the model by finetuning it on parent-child relations from subtrees of WordNet and test on non-overlapping subtrees. In addition, we incorporate semi-structured definitions from the web to further improve performance. On the task of inducing subtrees of WordNet, the model achieves 66.0 ancestor F_1, a 10.4 point absolute increase over the previous best published result on this task  .
We present a method for constructing taxonomic trees (e.g., WordNet) using pretrained language models. Our approach is composed of two modules, one that predicts parenthood relations and another that reconciles those predictions into trees. The parenthood prediction module produces likelihood scores for each potential parent-child pair, creating a graph of parent-child relation scores. The tree reconciliation module treats the task as a graph optimization problem and outputs the maximum spanning tree of this graph . We train our model on subtrees sampled from WordNet, and test on non-overlapping WordNet subtrees. We show that incorporating web-retrieved glosses can further improve performance. On the task of constructing subtrees of English WordNet, the model achieves 66.7 ancestor F1, a 20.0\% relative increase over the previous best published result on this task . In addition, we convert the original English dataset into nine other languages using Open Multilingual WordNet and extend our results across these languages .
R
inducing taxonomic trees from pretrained transformers. Given a set of input terms, we assign a score for the likelihood that each pair of terms forms a
constructing taxonomic trees (e.g., WordNet) using pretrained language models. Our approach is composed of two modules, one that predicts parenthood relations and another that reconciles those predictions into trees. The parenthood prediction module produces likelihood scores for each potential parent-child pair, creating a graph of
R
. To produce a tree from pairwise parent-child edge scores, we treat this
scores. The tree reconciliation module treats the task
R
output
outputs
A
of this graph
R
the model by finetuning it on parent-child relations from subtrees of WordNet
our model on subtrees sampled from WordNet,
R
subtrees. In addition, we incorporate semi-structured definitions from the web to
WordNet subtrees. We show that incorporating web-retrieved glosses can
R
inducing subtrees of
constructing subtrees of English
R
66.0 ancestor F_1, a 10.4 point absolute
66.7 ancestor F1, a 20.0\% relative
A
. In addition, we convert the original English dataset into nine other languages using Open Multilingual WordNet and extend our results across these languages