  A biomedical relation statement is commonly expressed in multiple sentences
and consists of many concepts, including gene, disease, chemical, and mutation.
To automatically extract information from biomedical literature, existing
biomedical text-mining approaches typically formulate the problem as a
cross-sentence n-ary relation-extraction task that detects relations among n
entities across multiple sentences, and use either a graph neural network (GNN)
with long short-term memory (LSTM) or an attention mechanism. Recently,
Transformer has been shown to outperform LSTM on many natural language
processing (NLP) tasks. In this work, we propose a novel architecture that
combines Bidirectional Encoder Representations from Transformers with Graph
Transformer (BERT-GT), through integrating a neighbor-attention mechanism into
the BERT architecture. Unlike the original Transformer architecture, which
utilizes the whole sentence(s) to calculate the attention of the current token,
the neighbor-attention mechanism in our method calculates its attention
utilizing only its neighbor tokens. Thus, each token can pay attention to its
neighbor information with little noise. We show that this is critically
important when the text is very long, as in cross-sentence or abstract-level
relation-extraction tasks. Our benchmarking results show improvements of 5.44%
and 3.89% in accuracy and F1-measure over the state-of-the-art on n-ary and
chemical-protein relation datasets, suggesting BERT-GT is a robust approach
that is applicable to other biomedical relation extraction tasks or datasets.
