# Vietnamese Sentence Similarity Based on Concepts

Hien T. Nguyen, Phuc H. Duong, Vinh T. Vo

<figure><img src="image_1.png" /></figure>

► To cite this version:

Hien T. Nguyen, Phuc H. Duong, Vinh T. Vo. Vietnamese Sentence Similarity Based on Concepts. 13th IFIP International Conference on Computer Information Systems and Industrial Management (CISIM), Nov 2014, Ho Chi Minh City, Vietnam. pp.243-253, 10.1007/978-3-662-45237-0_24. hal-01405592

HAL Id: hal-01405592

https://inria.hal.science/hal-01405592

Submitted on 30 Nov 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Vietnamese Sentence Similarity Based on Concepts

Hien T. Nguyen, Phuc H. Duong, Vinh T. Vo

Faculty of Information Technology, Ton Duc Thang University, Vietnam
hien@tdt.edu.vn, huuphucduong@gmail.com, vothanhvinh@tdt.edu.vn

**Abstract.** We propose a novel method for measuring semantic similarity of two sentences. The originality of the method is the way that it explores the similarity of concepts referred to in the sentences using Wikipedia. The method also exploits Wiktionary to measure word-to-word similarity. The overall semantic similarity is a linear combination of word-to-word similarity, word-order similarity, and concept similarity. We build datasets consisting of 45 Vietnamese sentence pairs and then evaluate the method on these datasets. The results show that in the best cases, concept similarity help improving the performance of our method more than 15% point. The proposed method is language-independent and quite easy to employ. Therefore, one can readily adopt our method to measure semantic similarity for sentences written in other languages.

**Keywords:** Paraphrase Identification; Text Similarity; Semantic Similarity

## 1 Introduction

We study the task of measuring semantic similarity of short texts, i.e., sentence, text segments or very short text snippets. With the development of natural language applications recently, this task has been playing an increasingly important role in plagiarism detection, question answering, machine translation, text summarization, information retrieval, etc. This is a challenging task. Considering the following two sentence pairs, two sentences in the first pair are more likely similarity in meaning even though not many words they contain are common; while all most all words in two sentences in the second pair are the same but their meanings are different.

* "She has to pass the exam." and "She must get through the exam"

* "To gain admission to UCLA, you need to present an academic profile much stronger than represented by the minimum UC admission requirements below."<fnref n="1" /> and "To gain admission to Berkeley, you need to present an academic profile much stronger than that represented by the minimum UC admission requirements"<fnref n="2" />

Until now, there have been many methods proposed for measuring text similarity. Most of them use either knowledge-based or corpus-based word-to-word semantic

<footnote n="1">¹ www.admissions.ucla.edu/prospect/adm_fr/fracadrq.htm</footnote>

<footnote n="2">² admissions.berkeley.edu/sites/default/files/docs/Freshman_Flier.pdf</footnote>
similarity measures [1], in combination with syntactic information [4], [5], [13], [14] or string matching algorithm [2]. Some works exploited machine translation metrics [7], discourse information [6], or graph subsumption [12] for paraphrase identification. In summary, the work in literature did not exploit concepts, i.e. named entities and common concepts, including coreference and disambiguation, for measuring similarity of short texts. A concept we mean in this work is a named entity such as a person, a location, or an organization in particular, etc. or a common concept such as computer science or information technology.

We propose a novel method to compute semantic similarity between sentence pairs. We evaluate the method on a dataset consisting of Vietnamese sentences. The method explores concepts to determine how they contribute to the performance of measuring semantic similarity of sentences. It also exploits word-to-word similarity and word-order similarity as proposed by Li et al. in [4]. An intuition shows that exploring named entities (as well as concepts) and their features is potentially in improving the performance of semantic similarity of sentences, especially for those with the same meaning containing named entities but few words in common. For example, with these two sentences “I am currently working at IBM” and “I am a developer at International Business Machines”, if we only compute the similarity based on words and word-order, the similarity score may not be high as it would be even though the entities IBM and International Business Machines are the same in the contexts of the two sentences.

One of the challenging problem when dealing with Vietnamese texts is that Vietnamese is a language with a deficient natural language processing support, such as no Vietnamese WordNet or corpus like Brown Corpus of American English for measuring semantic similarity between Vietnamese words. To overcome that limitation, we exploit Wikipedia<fnref n="3" /> to recognize which concepts referred to in compared sentences exist in Wikipedia to expand the context of those sentences by different surface forms of the concepts and exploit Wiktionary<fnref n="4" /> to estimate the similarity of words.

Wikipedia is a free online encyclopedia whose content is contributed by a large number of volunteer users. It consists of a large collection of articles, each of which defines and describes a concept. In reality, a concept may have several surface forms and one surface form may be used to refer to different concepts in different contexts. In Wikipedia, many-to-many correspondence between names and entities can be captured by utilizing *redirect pages* and *disambiguation pages*. A redirect page typically contains only a reference to an article. The title of a redirect page is an alternative surface form of the described entity or concept in that article. For example, from redirect pages of the United States, we extract alternative surface forms of the United States such as “US”, “USA”, “United States of America”, etc. A disambiguation page is created for an ambiguous surface form which may use to denote two or more entities in Wikipedia. It consists of links to articles that define the different concepts having the same surface form. Wiktionary is a free-content multilingual dictionary, designed as the lexical companion to Wikipedia and opened for volunteers to edit all the

<footnote n="3">³ http://vi.wikipedia.org</footnote>

<footnote n="4">⁴ http://vi.wiktionary.org</footnote>
contents. It provides the meaning of vocabulary, not includes the encyclopedic information – an advantage of Wikipedia.

The contributions of this paper is three-fold as follows: (i) we propose a novel method that based on concepts for measuring similarity of Vietnamese sentences, (ii) we build a dataset consisting of 45 Vietnamese sentence pairs, each of which was estimate by human subjects if it is paraphrase or not and evaluate our proposed method on this dataset. The method exploits Wiktionary to measure similarity of words, exploits Wikipedia to identify which concepts referred to in the compared sentences existing in Wikipedia for expanding the contexts of those sentences by different surface forms of the concepts. The originality of this work is the way that our method expands the contexts of compared sentences using Wikipedia.

The rest of this paper is organized as follows. Section 2 presents related work. Section 3 presents our proposed approach. Section 4 presents datasets, experiments and results. Finally, we draw conclusion in Section 5.

## 2 Related work

There is much research on sentence similarity measurement. In [1], the authors proposed a method that estimates the semantic similarity between two short texts using both corpus-based and knowledge-based similarity measures of words. Given two short text segments, the method finds for each word in the first one the most similar matching word in the second one and then the similarity between those word pairs are included in the overall semantic similarity of the two text segments. In [2], the authors introduced a method that computes the text similarity by combining a corpus-based similarity measure of words and a modified version of the Longest Common Subsequence string matching algorithm. The method proposed in [9] uses word-to-word similarity derived from WordNet to identify paraphrase.

The methods proposed in [4], [5], [13], [14] measure text similarity based on semantic and syntactic information contained in the compared texts. In addition to using word-to-word similarity, the method in [4] presents the important role of word-order in improvement of sentence similarity measure, the method in [5] exploited adjectives and adverbs in two sentences, the method in [13] takes nouns and verbs in consideration, and the method in [14] exploited interdependent between word-to-word similarity and sentence similarity and computed both of them simultaneously by an iterative algorithm. In [12], the authors use graph subsumption (originally developed for recognizing entailment) with lexical, syntactic, synonymy and antonymy information to identify paraphrase.

The method proposed in [6] combines machine translation metrics and the ordered similarity between elementary discourse units (EDUs). An EDU are blocks of words playing an important role in sentence similarity. In [3], the authors proposed a method that takes advantage of web search results to extend context of short texts and in [8] the authors uses unfolding recursive auto-encoder method for measuring the similarity. In [11], instead of identifying the similarity, the authors propose a method detecting dissimilarities. The method proposed in [14] combines semantic and statistical
information within short texts to compute the similarity. The method proposed in [13] takes advantage of corpus-based ontology to overcome the problem that evaluates the semantic similarity between irregular sentences.

In this paper, we propose a novel method that explores concepts to measure the similarity of sentences. In contrast with related methods, our method use Wiktionary instead of WordNet to measure the similarity of words. Moreover, it identifies which concepts referred to in the compared sentences existing in Wikipedia for expanding the contexts of those sentences by different surface forms of the concepts.

# 3 Proposed method

We propose a novel method that computes overall semantic similarity of two sentences, let say $S_1$ and $S_2$, as follows:

$$Sim(S_1, S_2) = \alpha \times Sim_{word} + \beta \times Sim_{word-order} + \gamma \times Sim_{concept} \quad (1)$$

where $Sim_{word}$, $Sim_{word-order}$, $Sim_{concept}$ are sentence similarity measures based exclusively on word-to-word similarity, word-order similarity, and concept similarity respectively; $\alpha + \beta + \gamma = 1$; and the coefficients {$\alpha, \beta, \gamma$} decide the contribution of word-to-word similarity, word-order similarity and concept similarity to the whole sentences. In following three sub-sections, we respectively present the similarity measures.

## 3.1 Model

Our method contains four main steps for computing semantic similarity between two sentences. The first step performs tokenizer. The second step calculates the similarity between sentences by using word-to-word similarity based on Wiktionary. The third computes similarity between sentences by using word-order similarity of the two sentences. The fourth step recognizes concepts in the two sentences using Wikipedia and expands the contexts of the sentences to compute the concept similarity; then the similarity between sentences is computed using the concept similarity. Finally, the overall sentence similarity score is derived by combining the word-to-word similarity, the word-order similarity and the concept similarity.

## 3.2 Sentence similarity based on word similarity

The essence of sentence similarity is word-to-word similarity. Thus, we propose to use the Text Overlap method to compute the word-to-word similarity between two words. To our knowledge, this is the most possible method for computing the word-to-word similarity based on Vietnamese Wiktionary. Text Overlap method was first introduced in 1986 by Michael Lesk [16]. The main idea of the method is based on the level of the intersection of gloss texts. The higher level of intersection is, the more similar two words are and otherwise.

We first have to split two input sentences into tokens, then, we will create semantic vectors base on those tokens. Now, we will go through all steps in word-to-word Sim-
ilarity. Given two sentences S<sub>1</sub> and S<sub>2</sub>, a joint word set is defined by S = S<sub>1</sub> ∪ S<sub>2</sub> = {w<sub>1</sub>; w<sub>2</sub>; ...; w<sub>n</sub>} . The word set S contains all distinct words from the two sentences and includes all inflectional morphology words. For example, word and words, thesis and theses are considered as distinct words and must appear in S . For example, given the following sentences:

• S<sub>1</sub> = I am a developer at International Business Machines.

• S<sub>2</sub> = I am currently working at IBM.

• S = {I, am, a, developer, at, International, Business, Machines, currently, work- ing, IBM}.

Because the word set S is derived from two sentences, we should use it as a stand-
ard semantic vector for comparing, denoted by V . The semantic vector of two sen-
tences ( V<sub>1</sub> and V<sub>2</sub> ) must have the same dimension of V and each value of an entry of
the semantic vector ( V<sub>1</sub> and V<sub>2</sub> ), denoted by s<sub>i</sub> ( i = 1, 2, ..., m ), is determined by word-
to-word similarity method and should lie between [0,1]. Taking S<sub>1</sub> as an example:

• Case 1: if w<sub>i</sub> appears in S<sub>1</sub>, set s<sub>i</sub> to 1.

* Case 2: if wᵢ not appears in S₁, word-to-word similarity method will be used to compute the similarity between wᵢ and each word in S₁. Thus, the word-to-word similarity score should be the highest number k. If k exceeds a standard threshold, then sᵢ = k, otherwise, set sᵢ = 0. Suppose that the highest number k = 0.01, the value of sᵢ should be 0, because 0.01 is closer to zero, and that's why we should have a standard threshold.

Unlike other text similarity methods, this approach keeps all function words, since
these words contain syntactic information if the sentence is too short. Although they
appear in the joint word set S, they can't affect the whole meaning of the sentence, as
well as the semantic vectors, because we use a threshold to eliminate them. After we
have two semantic vectors V<sub>1</sub> and V<sub>2</sub>, the semantic similarity between two sentences is
computed by cosine coefficient of those vectors:

$$
Sim_{word} = \frac{V_1 \cdot V_2}{|V_1| \cdot |V_2|} \qquad (2)
$$

3.3 Sentence similarity based on word order similarity

Before starting this section, we should look over the two example sentences below,
they contain the same words in each sentence but differ from the position of two
words *boy* and *girl* as follows:

• S<sub>1</sub> = A boy buys the girl a gift.

• S<sub>2</sub> = A girl buys the boy a gift.

We can see that if we only consider word-to-word similarity on that candidate pair,
the similarity score will be maximum because all words in that pair are the same. The
dissimilarity between S<sub>1</sub> and S<sub>2</sub> is caused by the different word-order. To resolve this
problem, we form a joint word set S, each entry value denoted by w<sub>i</sub> (i = 1, 2, ..., m).
S = {A, boy, buys, the, girl, a, gift}. We assign an index number for each word in S<sub>1</sub> and S<sub>2</sub>. To compute the similarity score, a word-order vector R<sub>i</sub> is formed for each sentence with the length equals to the size of S and each value of R<sub>i</sub> is simply an index number in the sentence. Taking S<sub>1</sub> as an example, for each word w<sub>i</sub> in S, we will consider its position or the most similar between it and the others, look the cases as follows:

* Case 1: if w<sub>i</sub> appears in S<sub>1</sub>, entry value of this word in R<sub>i</sub> is the index number from S<sub>1</sub>.

* Case 2: if w<sub>i</sub> doesn't appear in S<sub>1</sub>, entry value of this word in R<sub>i</sub> is zero.

For example of S<sub>1</sub> and S<sub>2</sub>, two word-order vectors R<sub>1</sub> and R<sub>2</sub> should be:

* R<sub>1</sub> = {1, 2, 3, 4, 5, 6, 7}

* R<sub>2</sub> = {1, 5, 3, 4, 5, 2, 7}

Through this approach, we can see that R<sub>1</sub> and R<sub>2</sub> clearly show the basic structure of the sentence. Eventually, a measure for computing word-order similarity of the two input sentences is:

$$Sim_{\text{word-order}} = 1 - \frac{\|R_1 - R_2\|}{\|R_1 + R_2\|} \quad (3)$$

## 3.4 Sentence similarity based on concept similarity

In order to improve the quality of assessment of similarity between two sentences, we determine the similarity between concepts occurring in sentences base on Wikipedia. To better understand the purpose of applying concept similarity, considering the following example with two sentences S<sub>1</sub> and S<sub>2</sub>. If we only compute semantic similarity base on words and word-order, we can't detect two entities *Ho Chi Minh City* and *Sai Gon* are the same. As a result, it may lead to the similarity between two sentences not accurate.

* S<sub>1</sub> = I am living in Sai Gon.

* S<sub>2</sub> = I am living in Ho Chi Minh City.

We adopt the mention recognition of Huy *et al.* [19] to identify surface forms of Wikipedia concepts in a sentence. If an identified surface form has only one candidate concept, our method collects all of its surface forms - extracted from its title and its redirect page titles in Wikipedia. If an identified surface form is ambiguous, our method does not disambiguate it, but collects all surface forms of all of its candidate concepts based on the disambiguation page of the identified surface form in Wikipedia.

Given a pair of sentences, let SF<sub>1</sub> be a set of surface forms identified in the first sentence and SF<sub>2</sub> be a set of surface forms identified in the second sentence. The sentence similarity based on concept similarity is computed as follows:

$$Sim_{\text{concept}} = \frac{|SF_1 \cap SF_2|}{\text{Min}(|SF_1|, |SF_2|)} \quad (4)$$
# 4 Dataset and Evaluation

## 4.1 Dataset

To evaluate the performance of our method, we build a dataset consisting of 45 pairs of sentences. Then the dataset is sent to different persons to estimate if each pair is paraphrase or not. If a pair is paraphrase, result will be 1, otherwise result will be 0. After collecting seven survey results, we analyze and create two datasets to serve the assessment process. The results show that there are 19 pairs getting the same agreement by 7 persons, 6 pairs getting the same agreement by 6 persons, 6 pairs getting the same agreement by 5 persons. In total 31 pairs get the same agreement by at least 5 persons. We give 7 persons a chance to discuss on these 31 pairs and get a dataset consisting of 31 sentence pairs with agreement by 7 persons, namely Dataset-1. We build Dataset-2 as follows: the assessments which have at least four number of 1 (the same agreement by at least 4 persons) will unify to 1, the others will be 0. It means that Dataset-2 consists of 45 pairs of sentences.

## 4.2 Evaluation

In the fields of science, engineering, industry, and statistics, "the accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity's actual (true) value". In our case, we use a binary classification to measure of how well a test correctly identifies or excludes a condition. Let's take a look the problem we have: the output of computational process lies in [0;1], thus, which value of threshold $\tau$ give the similarity of two sentences is 1 or 0; and with that $\tau$, how well the accuracy of the approach is. To solve the problem, we will initially set $\tau = 0.5$, each loop $\tau$ will increase a value $t = 0.01$, if the output of computational process is larger than $\tau$, the similarity will be 1, otherwise, will be 0. The value of accuracy calculated by applying the formula:

$$ accuracy = \frac{TP+TN}{TP+FP+TN+FN} \qquad (5) $$

where TP denotes true positive, TN denotes true negative, FP denotes false positive, FN denotes false negative.

We evaluate our method on Dataset-1 and Dataset-2 respectively. Table 1 shows the results after running our method on Dataset-1. The column "$\tau$" shows the $\tau$ values, The column "accuracy with concepts" shows overall sentence similarity of our method in the term of accuracy, and the column "accuracy without concepts" shows overall sentence similarity of our method in the case we do not use concept similarity. The results shown in Table 2 are explained as the same as doing for the results shown in Table 1.

As shown in Table 1, for Dataset-1, the accuracy is maximized at threshold {0.50 to 0.55, 0.62, 0.63} and minimized at {0.94, 0.95}. As shown in Table 2, for Dataset-2, the accuracy is maximized at threshold {0.50 to 0.55, 0.62, 0.63} and minimized at {0.94, 0.95}.
imized at {0.94, 0.95}. Fig.1 and Fig 2 respectively show the curves comparing the performance of our method on Dataset-1 and Dataset-2 with and without using concept similarity.

We can see that, the higher threshold is, the smaller accuracy is, because the value of τ at low level is closer to people judgment. All in all, the results show that in the best cases, concept similarity help improving the performance of our method more than 15% point; which prove that concept similarity significantly contribute to the performance of our method.

<table><caption>Table 1. The overall accuracy of our method on Datatset-1</caption><thead><tr><th>τ</th><th>Accuracy with Concepts</th><th>Accuracy without Concepts</th><th>τ</th><th>Accuracy with Concepts</th><th>Accuracy without Concepts</th></tr></thead><tbody><tr><td>0.50</td><td>93.55</td><td>77.42</td><td>0.73</td><td>64.52</td><td>38.71</td></tr><tr><td>0.51</td><td>93.55</td><td>77.42</td><td>0.74</td><td>64.52</td><td>38.71</td></tr><tr><td>0.52</td><td>93.55</td><td>77.42</td><td>0.75</td><td>64.52</td><td>38.71</td></tr><tr><td>0.53</td><td>93.55</td><td>77.42</td><td>0.76</td><td>64.52</td><td>38.71</td></tr><tr><td>0.54</td><td>93.55</td><td>77.42</td><td>0.77</td><td>64.52</td><td>38.71</td></tr><tr><td>0.55</td><td>93.55</td><td>77.42</td><td>0.78</td><td>61.29</td><td>38.71</td></tr><tr><td>0.56</td><td>90.32</td><td>74.19</td><td>0.79</td><td>54.84</td><td>35.48</td></tr><tr><td>0.57</td><td>90.32</td><td>74.19</td><td>0.80</td><td>48.39</td><td>35.48</td></tr><tr><td>0.58</td><td>90.32</td><td>74.19</td><td>0.81</td><td>41.94</td><td>35.48</td></tr><tr><td>0.59</td><td>90.32</td><td>74.19</td><td>0.82</td><td>41.94</td><td>35.48</td></tr><tr><td>0.60</td><td>90.32</td><td>70.97</td><td>0.83</td><td>38.71</td><td>35.48</td></tr><tr><td>0.61</td><td>90.32</td><td>70.97</td><td>0.84</td><td>38.71</td><td>35.48</td></tr><tr><td>0.62</td><td>93.55</td><td>70.97</td><td>0.85</td><td>38.71</td><td>32.26</td></tr><tr><td>0.63</td><td>93.55</td><td>67.74</td><td>0.86</td><td>38.71</td><td>29.03</td></tr><tr><td>0.64</td><td>90.32</td><td>64.52</td><td>0.87</td><td>35.48</td><td>29.03</td></tr><tr><td>0.65</td><td>87.1</td><td>64.52</td><td>0.88</td><td>32.26</td><td>29.03</td></tr><tr><td>0.66</td><td>83.87</td><td>58.06</td><td>0.89</td><td>32.26</td><td>29.03</td></tr><tr><td>0.67</td><td>83.87</td><td>58.06</td><td>0.90</td><td>32.26</td><td>25.81</td></tr><tr><td>0.68</td><td>80.65</td><td>58.06</td><td>0.91</td><td>29.03</td><td>25.81</td></tr><tr><td>0.69</td><td>80.65</td><td>51.61</td><td>0.92</td><td>29.03</td><td>22.58</td></tr><tr><td>0.70</td><td>77.42</td><td>45.16</td><td>0.93</td><td>29.03</td><td>22.58</td></tr><tr><td>0.71</td><td>74.19</td><td>45.16</td><td>0.94</td><td>25.81</td><td>22.58</td></tr><tr><td>0.72</td><td>67.74</td><td>41.94</td><td>0.95</td><td>25.81</td><td>22.58</td></tr></tbody></table>
<figure><img src="image_3.png" /><figcaption>Fig. 1. The curves comparing the performance on Dataset-1 with and without using concept similarity</figcaption></figure>

<table><caption>Table 2. The overall accuracy of our method on Dataset-2</caption><thead><tr><th>τ</th><th>Accuracy with Concepts</th><th>Accuracy without Concepts</th><th>τ</th><th>Accuracy with Concepts</th><th>Accuracy without Concepts</th></tr></thead><tbody><tr><td>0.50</td><td>86.67</td><td>73.33</td><td>0.73</td><td>64.44</td><td>46.67</td></tr><tr><td>0.51</td><td>86.67</td><td>73.33</td><td>0.74</td><td>64.44</td><td>46.67</td></tr><tr><td>0.52</td><td>86.67</td><td>73.33</td><td>0.75</td><td>64.44</td><td>46.67</td></tr><tr><td>0.53</td><td>86.67</td><td>73.33</td><td>0.76</td><td>66.67</td><td>46.67</td></tr><tr><td>0.54</td><td>86.67</td><td>73.33</td><td>0.77</td><td>66.67</td><td>44.44</td></tr><tr><td>0.55</td><td>86.67</td><td>73.33</td><td>0.78</td><td>62.22</td><td>44.44</td></tr><tr><td>0.56</td><td>84.44</td><td>71.11</td><td>0.79</td><td>57.78</td><td>44.44</td></tr><tr><td>0.57</td><td>84.44</td><td>71.11</td><td>0.80</td><td>55.56</td><td>44.44</td></tr><tr><td>0.58</td><td>84.44</td><td>71.11</td><td>0.81</td><td>51.11</td><td>44.44</td></tr><tr><td>0.59</td><td>84.44</td><td>71.11</td><td>0.82</td><td>48.89</td><td>44.44</td></tr><tr><td>0.60</td><td>84.44</td><td>68.89</td><td>0.83</td><td>46.67</td><td>44.44</td></tr><tr><td>0.61</td><td>84.44</td><td>68.89</td><td>0.84</td><td>46.67</td><td>44.44</td></tr><tr><td>0.62</td><td>86.67</td><td>68.89</td><td>0.85</td><td>44.44</td><td>42.22</td></tr><tr><td>0.63</td><td>86.67</td><td>68.89</td><td>0.86</td><td>46.67</td><td>40.00</td></tr><tr><td>0.64</td><td>82.22</td><td>66.67</td><td>0.87</td><td>44.44</td><td>40</td></tr><tr><td>0.65</td><td>80</td><td>64.44</td><td>0.88</td><td>42.22</td><td>40</td></tr><tr><td>0.66</td><td>77.78</td><td>60</td><td>0.89</td><td>42.22</td><td>40</td></tr><tr><td>0.67</td><td>77.78</td><td>60</td><td>0.90</td><td>42.22</td><td>37.78</td></tr><tr><td>0.68</td><td>75.56</td><td>60</td><td>0.91</td><td>40</td><td>37.78</td></tr><tr><td>0.69</td><td>75.56</td><td>55.56</td><td>0.92</td><td>40</td><td>33.33</td></tr><tr><td>0.70</td><td>73.33</td><td>53.33</td><td>0.93</td><td>40</td><td>33.33</td></tr><tr><td>0.71</td><td>71.11</td><td>51.11</td><td>0.94</td><td>37.78</td><td>33.33</td></tr><tr><td>0.72</td><td>66.67</td><td>48.89</td><td>0.95</td><td>37.78</td><td>33.33</td></tr></tbody></table>
<figure><img src="image_4.png" /><figcaption>**Fig. 2.** The curves comparing the performance on Dataset-2 with and without using concept similarity</figcaption></figure>

# 5 Conclusion

In this paper, we present a novel method for measuring semantic similarity of two sentences. The overall semantic similarity is a linear combination of word-to-word similarity, word-order similarity, and concept similarity. The method exploits Wiktionary to measure similarity of words, exploits Wikipedia to identify which concepts referred to in the compared sentences existing in Wikipedia for expanding the contexts of those sentences by different surface forms of the concepts in order to compute the concept similarity. We evaluate the method on the datasets consisting of Vietnamese sentence pairs. The results show that in the best cases, concept similarity help improving the performance of our method more than 15% point. The proposed method is language-independent and quite easy to employ. Therefore, one can readily adopt our method to measure semantic similarity for sentences written in other languages.

# References

1. Mihalcea, R., Corley, C., & Strapparava, C.: Corpus-based and knowledge-based measures of text semantic similarity. In AAAI, Vol. 6, pp. 775-780 (2006)

2. Islam, A., & Inkpen, D.: Semantic text similarity using corpus-based word similarity and string similarity. In: ACM Transactions on Knowledge Discovery from Data (TKDD), 2(2), Article 10 (2008).

3. Sahami, M., & Heilman, T. D.: A web-based kernel function for measuring the similarity of short text snippets. In: Proceedings of the 15th international conference on World Wide Web, pp. 377-386 (2006)

4. Li, Y., McLean, D., Bandar, Z. A., O'shea, J. D., & Crockett, K.: Sentence similarity based on semantic nets and corpus statistics. In: IEEE Transactions on Knowledge and Data Engineering, 18(8), 1138-1150 (2006)

5. Oliva, J., Serrano, J. I., del Castillo, M. D., & Iglesias, Á.: SyMSS: A syntax-based measure for short-text semantic similarity. In: Data & Knowledge Engineering, 70(4), pp. 390-405 (2011)
6. Bach, N. X., Minh, N. L., & Shimazu, A.: Exploiting discourse information to identify paraphrases. In: Expert Systems with Applications, 41(6), 2832-2841 (2014).

7. Madnani, N., Tetreault, J., and Chodorow, M.: Re-examining Machine Translation Metrics for Paraphrase Identification. In: Proceedings of 2012 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2012), pp. 182-190 (2012)

8. Socher, R., Huang, E. H., Pennington, J., Ng, A. Y., & Manning, C. D.: Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection. In NIPS (Vol. 24, pp. 801-809) (2011)

9. Fernando, S., & Stevenson, M.: A semantic similarity approach to paraphrase detection. In: Proceedings of the 11th Annual Research Colloquium of the UK Special Interest Group for Computational Linguistics (pp. 45-52) (2008).

10. Das, D., and Smith, N.: Paraphrase identification as probabilistic quasi-synchronous recognition. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp. 468-476, (2009)

11. Qiu, L. and Kan, M.Y. and Chua, T.S.: Paraphrase recognition via dissimilarity significance classification. In: Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pp. 18-26 (2006)

12. Rus, V. and McCarthy, P.M. and Lintean, M.C. and McNamara, D.S. and Graesser, A.C.: Paraphrase identification with lexico-syntactic graph subsumption. In: FLAIRS 2008, pp. 201-206.

13. Lee, M. C.: A novel sentence similarity measure for semantic-based expert systems. In: Expert Systems with Applications, 38(5), 6392-6399 (2011)

14. Wenyin, L., Quan, X., Feng, M., & Qiu, B.: A short text modeling method combining semantic and statistical information. In: Information Sciences, 180(20), 4031-4041 (2010)

15. Blacoe, W., & Lapata, M.: A comparison of vector-based representations for semantic composition. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 546-556

16. Lesk, M.: Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In: Proceedings of the 5th annual international conference on Systems documentation, pp. 24-26 (1986)

17. Tsatsaronis, G., Varlamis, I., & Vazirgiannis, M.: Text relatedness based on a word thesaurus. Journal of Artificial Intelligence Research, 37(1), 1-40 (2010)

18. Rubenstein, H., & Goodenough, J. B.: Contextual correlates of synonymy. Communications of the ACM, 8(10), 627-633 (1965)

19. Huynh, H. M., Nguyen, T. T., & Cao, T. H.: Using coreference and surrounding contexts for entity linking. In 2013 IEEE RIVF International Conference on Computing and Communication Technologies, Research, Innovation, and Vision for the Future (RIVF 2013), pp. 1-5, (2013)