  Recently, BERT realized significant progress for sentence matching via
word-level cross sentence attention. However, the performance significantly
drops when using siamese BERT-networks to derive two sentence embeddings, which
fall short in capturing the global semantic since the word-level attention
between two sentences is absent. In this paper, we propose a Dual-view
distilled BERT~(DvBERT) for sentence matching with sentence embeddings. Our
method deals with a sentence pair from two distinct views, i.e., Siamese View
and Interaction View. Siamese View is the backbone where we generate sentence
embeddings. Interaction View integrates the cross sentence interaction as
multiple teachers to boost the representation ability of sentence embeddings.
Experiments on six STS tasks show that our method outperforms the
state-of-the-art sentence embedding methods significantly.
