Before revision:  This creates a gap when fusing knowledge graphs into language modeling, especially in the scenario of insufficient paired text-knowledge data. In this paper , we propose to utilize external entity description to provide contextual information for graph entities.
After revision:  This creates a gap when fusing knowledge graphs into language modeling, especially when there is insufficient labeled data. Thus , we propose to utilize external entity description to provide contextual information for graph entities.