Question: Is their approach similar to making an averaged weighted sum of word vectors, where weights reflect word frequencies?
Answer: Different from their work, we choose to corrupt the original document by randomly removing significant portion of words, and represent the document using only the embeddings of the words remained.