Owing to the need for a deep understanding of linguistic items, semantic representation is considered to be one of the fundamental components of several applications in Natural Language Processing and Artificial Intelligence. As a result, semantic representation has been one of the prominent research areas in lexical semantics over the past decades. However, due mainly to the lack of large sense-annotated corpora, most existing representation techniques are limited to the lexical level and thus cannot be effectively applied to individual word senses. In this paper we put forward a novel multilingual vector representation, called Nasari, which not only enables accurate representation of word senses in different languages, but it also provides two main advantages over existing approaches: (1) high coverage, including both concepts and named entities, (2) comparability across languages and linguistic levels (i.e., words, senses and concepts), thanks to the representation of linguistic items in a single unified semantic space and in a joint embedded space, respectively. Moreover, our representations are flexible, can be applied to multiple applications and are freely available at http://lcl.uniroma1.it/nasari/. As evaluation benchmark, we opted for four different tasks, namely, word similarity, sense clustering, domain labeling, and Word Sense Disambiguation, for each of which we report state-of-the-art performance on several standard datasets across different languages. as its dimensions, permitting direct comparison of different representations across languages and hence enabling cross-lingual applications. In this article, we improve our approach, referred to as Nasari (Novel Approach to a Semantically-Aware Representation of Items) henceforth, and extend their application to a wider range of tasks in lexical semantics. Specifically, the novel contributions are as follow: 50 1. We propose a new formulation for fast computation of lexical specificity (Section 3.1.1). 2. We propose a new flexible way to get continuous embedded vector representations, with the added benefit of obtaining a semantic space shared by BabelNet synsets, words and texts (Section 3.3). 3. We put forward a technique for improved computation of weights in the unified vectors and show how it can improve the accuracy and efficiency of the representations (Section 3.4). 55 4. We compute and assign weights to individual edges in our semantic network (Section 4.1) and show by means of different experiments the advantage we gain when using this new weighted graph (Section 10). 5. We release the lexical and unified vector representations for five different languages (English, French, German, Italian and Spanish) and the embedded vector representations for the English language at http://lcl. uniroma1.it/nasari/ . In addition to these contributions, we also devised robust frameworks that enable direct application of our representations to four different tasks: Semantic Similarity (Section 6), Sense Clustering (Section 7), Domain Labeling The rest of this article is structured as follows. We first provide an introduction of some of the most widely used knowledge resources in lexical semantics, in Section 2. After which, in Section 3 we describe our methodology to convert text into lexical, embedded and unified vectors. The process to obtain vector representations for synset vectors by leveraging the knowledge resources described in Section 2, and the methodology to obtain vectors from text described in Section 3, is presented in Section 4. We present a summary of the experiments and the performance of 70 Nasari across tasks in Section 5. Then, we describe some applications of the vectors with their respective frameworks and experiments in Sections 6 (Semantic Similarity), 7 (Sense Clustering), 8 (Domain Labeling), and 9 (Word Sense Disambiguation). We analyze the performance of different components of our model in Section 10. Finally, we discuss the related work in Section 11 and provide the concluding remarks in Section 12. Knowledge Resources Knowledge resources can be divided into two general categories: expert made and collaboratively constructed. Each type has its own advantages and limits. Manually-annotated resources feature highly-accurate encoding of concepts and semantic relationships between them but, with a few exceptions, are usually limited in their lexical coverage, and are typically focused on a specific language only. A good example is WordNet [84], a semantic network whose basic units are synsets. A synset represents a concept which may be expressed through nouns, verbs, 80 adjectives or adverbs and is composed of the different lexicalizations (i.e., synonyms that are used to express it). For example, the synset of the middle of the day concept comprises six lexicalizations: noon, twelve noon, high noon, midday, noonday, noontide. Synsets may also be seen as nodes in a semantic network. These nodes are connected to each other by means of lexical or semantic relations (hypernymy, meronymy, etc.). These relations are seen as the edges in the WordNet semantic network. Despite being one of the largest and most complete manually-made lexical 85 resources, WordNet still lacks coverage of lemmas and senses from domain specific lexicons (e.g., law or medicine), named entities, creative slang usages, or those for technology that came into existence only recently. On the other hand, collaboratively-constructed resources, such as Wikipedia, provide features such as multilinguality, wide coverage and up-to-dateness. As of September 2015, Wikipedia provides more than 100K articles in over fifty languages. This coverage is steadily increasing. For instance, the English Wikipedia alone receives 750 90 new articles per day. Each of these articles provides, for its corresponding concept, a great deal of information in the form textual information, tables, infoboxes, and various relations (such as redirections, disambiguations, and categories). These features have persuaded many researchers over the past few years to exploit the huge amounts of semi-structured knowledge available in such collaborative resources for different NLP applications [46, 126] . The types of knowledge available in the expert-based and collaboratively-constructed resources make them com-95 plementary. This has motivated researchers to combine various lexical resources across the two categories [102, 109]. A prominent example is BabelNet [99], which provides a mapping of WordNet to a number of collaborativelyconstructed resources, including Wikipedia. The structure of BabelNet 3 is similar to that of WordNet. Synsets are the main linguistic units and are connected to other semantically related synsets, whose lexicalizations are multilingual in this case. For instance, the synset corresponding to United States is represented with a set of multilin-100 gual lexicalizations including United States EN , and U.S.A. IT in Italian. The relations between synsets are the ones coming from WordNet (hypernyms, hyponyms, etc.), plus new relations coming from other resources such as Wikipedia hyperlinks and WikiData 4 relations (e.g. Madrid capital of Spain). BabelNet is the largest multilingual 105 semantic network available, containing 13,789,332 synsets (6,418,418 concepts and 7,370,914 named entities) and 354,538,633 relations for 271 languages 5 . For the English language, BabelNet contains 4,403,148 synsets with at 3 http://babelnet.org/ 200 205 420 the SimLex-999 dataset as our last evaluation benchmark. The complete SimLex-999 dataset is composed of 999 word pairs, 666 of which are noun pairs. Comparison systems. We selected state-of-the-art approaches which are available online as comparison systems. These systems can be split into two categories: knowledge-based and corpus-based. As knowledge-based, we selected two approaches based on the WordNet semantic graph: [107, ADW] 13 and [69, Lin] 14 . Another knowledge-based 425 approach is [35, ESA] 15 , which represents a word in a semantic space of Wikipedia articles. We also compared our systems with four corpus-based approaches 16 . Firstly, we took the pre-trained word embeddings of Word2Vec[82] 17 , the same used for our Nasari embed system (see Section 4.2). Then, we took the best predictive and count-based 13 ADW implementation available at https://github.com/pilehvar/ADW 14 Results for Lin were obtained from the WS4J implementation available at https://code.google.com/p/ws4j/ 15 ESA implementation available at DKProSimilarity package [7] . 16 All the corpus-based approaches mentioned in the paper use cosine as comparison measure. 17 The pre-trained models are available at https://code.google.com/p/word2vec/. They were trained on a Google News corpus of about 100 billion words.
Finally, NASARI REF combines text-based and knowledge-based methods, yielding multilingual vector representations.
Nasari : Integrating explicit knowledge and corpus statistics for a multilingual representation of concepts and entities
Artif. Intell.
Artif. Intell.
Computer Science