Hypertext documents , such as web pages and academic papers , are of great importance in delivering information in our daily life . Although being effective on plain documents , conventional text embedding methods suffer from information loss if directly adapted to hyper-documents . In this paper , we propose a general embedding approach for hyper-documents , namely , hyperdoc2vec , along with four criteria characterizing necessary information that hyper-document embedding models should preserve . Systematic comparisons are conducted between hyperdoc2vec and several competitors on two tasks , i.e. , paper classification and citation recommendation , in the academic paper domain . Analyses and experiments both validate the superiority of hyperdoc2vec to other models w.r.t . the four criteria . The ubiquitous World Wide Web has boosted research interests on hypertext documents , e.g. , personal webpages ( Lu and Getoor , 2003 ) , Wikipedia pages ( Gabrilovich and Markovitch , 2007 ) , as well as academic papers ( Sugiyama and Kan , 2010 ) . Unlike independent plain documents , a hypertext document ( hyper-doc for short ) links to another hyper-doc by a hyperlink or citation mark in its textual content . Given this essential distinction , hyperlinks or citations are worth specific modeling in many tasks such as link-based classification ( Lu and Getoor , 2003 ) , web retrieval ( Page et al . , 1999 ) , entity linking ( Cucerzan , 2007 ) , and citation recommendation ( He et al . , 2010 ) . To model hypertext documents , various efforts ( Cohn and Hofmann , 2000 ; Kataria et al . , 2010 ; Perozzi et al . , 2014 ; Zwicklbauer et al . , 2016 ; Wang et al . , 2016 ) have been made to depict networks of hyper-docs as well as their content . Among potential techniques , distributed representation ( Mikolov et al . , 2013 ; Le and Mikolov , 2014 ) tends to be promising since its validity and effectiveness are proven for plain documents on many natural language processing ( NLP ) tasks . Conventional attempts on utilizing embedding techniques in hyper-doc-related tasks generally fall into two types . The first type ( Berger et al . , 2017 ; Zwicklbauer et al . , 2016 ) simply downcasts hyper-docs to plain documents and feeds them into word2vec ( Mikolov et al . , 2013 ) ( w2v for short ) or doc2vec ( Le and Mikolov , 2014 ) ( d2v for short ) . These approaches involve downgrading hyperlinks and inevitably omit certain information in hyper-docs . However , no previous work investigates the information loss , and how it affects the performance of such downcasting-based adaptations . The second type designs sophisticated embedding models to fulfill certain tasks , e.g. , citation recommendation ( Huang et al . , 2015b ) , paper classification ( Wang et al . , 2016 ) , and entity linking ( Yamada et al . , 2016 ) , etc . These models are limited to specific tasks , and it is yet unknown whether embeddings learned for those particular tasks can generalize to others . Based on the above facts , we are interested in two questions : • What information should hyper-doc embedding models preserve , and what nice property should they possess ? • Is there a general approach to learning taskindependent embeddings of hyper-docs ? To answer the two questions , we formalize the hyper-doc embedding task , and propose four criteria , i.e. , content awareness , context awareness , newcomer friendliness , and context intent aware-ness , to assess different models . Then we discuss simple downcasting-based adaptations of existing approaches w.r.t . the above criteria , and demonstrate that none of them satisfy all four . To this end , we propose hyperdoc2vec ( h-d2v for short ) , a general embedding approach for hyperdocs . Different from most existing approaches , h-d2v learns two vectors for each hyper-doc to characterize its roles of citing others and being cited . Owning to this , h-d2v is able to directly model hyperlinks or citations without downgrading them . To evaluate the learned embeddings , we employ two tasks in the academic paper domain1 , i.e. , paper classification and citation recommendation . Experimental results demonstrate the superiority of h-d2v . Comparative studies and controlled experiments also confirm that h-d2v benefits from satisfying the above four criteria . We summarize our contributions as follows : • We propose four criteria to assess different hyper-document embedding models . • We propose hyperdoc2vec , a general embedding approach for hyper-documents . • We systematically conduct comparisons with competing approaches , validating the superiority of h-d2v in terms of the four criteria . We focus on the hyper-doc embedding problem . We propose that hyper-doc embedding algorithms should be content aware , context aware , newcomer friendly , and context intent aware . To meet all four criteria , we propose a general approach , hyperdoc2vec , which assigns two vectors to each hyper-doc and models citations in a straightforward manner . In doing so , the learned embeddings satisfy all criteria , which no existing model is able to . For evaluation , paper classification and citation recommendation are conducted on three academic paper datasets . Results confirm the effectiveness of our approach . Further analyses also demonstrate that possessing the four properties helps h-d2v outperform other models .

Challenge: Existing text embedding methods do not take structures of hyper-documents into account losing useful properties for downstream tasks.
Approach: They propose an embedding method for hyper-documents that learns citation information along with four criteria to assess the properties the models should preserve.
Outcome: The proposed model satisfies all of the introduced criteria and performs two tasks in the academic domain better than existing models.