Journal of Robotics, Networking and Artificial Life, Vol. 1, No. 2 (September 2014), 103-107

On Parameter Setting in Identifying the Same Languages Involved inDifferent Language DataRen WuYamaguchi Junior College
1346-2, Oaza Daido, Hofu city, Yamaguchi prefecture, 747-1232, Japan
Hiroshi Matsuno

Graduate School of Science and Engineering, Yamaguchi University
1677-1, Yoshida, Yamaguchi city, Yamaguchi prefecture, 753-8511, Japan
Tel/Fax : +81-83-933-5697
E-mail: matsuno@sci.yamaguchi-u.ac.jp

Abstract
We have introduced several kinds of similarity measure and proposed a method for identifying the same languages
involved in two language classification trees. Several unknown parameters are used there and need to be set to constantvalues. This paper aims to determine all the values of these parameters and get the identification results in order toconfirm the usefulness and effectiveness of our proposed similarity measures. As the result, we obtained reasonable
good values for the parameters throughout the experiments.
Keywords: world languages tree, language name similarity, language classification similarity

1. Introduction
We have proposed a method based on tree structure andstring alignment technique for identifying the samelanguages involved in two world languages trees,denoted by WLTs, which are two language
classification trees provided by different linguists. We
have named these two WLTs as T Y and T S 1-4 .
In our previous work, we have quantified several
kinds of similarity measure, such as language namesimilarity, language classification similarity andlanguage general similarity and so forth. Language
name similarity is defined as string similarity between
two language names. On the other hand, language
classification similarity is defined as a weighted
average of three kinds of similarities (family namesimilarity, parent name similarity and brother namesimilarity) that are based on language name similarity.
Furthermore, language general similarity is defined asa weighted average of language name similarity andlanguage classification similarity. At the same time, wehave developed an algorithm for finding out samelanguage pairs involved in T Y and T S by applying above

Fig. 1. Two WLTs (T Y and T S )defined similarity measures. In this algorithm, several
unknown parameters, (e, f, g), (a, b), Δ and threshold ρ,
are used and need to be set to constant values firstly.
This paper aims to determine all the values of theseparameters and then get the identification results in

Published by Atlantis Press
Copyright: the authors
103

R. Wu and H. Matsuno

order to confirm that our proposed similarity measuresand the algorithm are useful and effective. This paper isorganized as follows. First, we give the definitions ofour proposed similarity measures and the algorithm forfinding same language pairs. Then, we give the process
for setting the parameters (e,f,g), (a,b), Δ and threshold
ρ. Finally, we give the experimental results by applying
the parameter values that are set.
Fig. 2. The same language that needs to be identified

2. Preliminary
2.1. Similarity Measure
Fig. 1(1) shows an image of WLT and Fig. 1(2) shows
two WLTs called T Y and T S . A language y included in T Y
is denoted by y ∊ V leaf (T Y ). Language y and s are
assumed to be the same language included in T Y and T S
respectively.
Fig. 2 shows an example of the same language pair(y,s) involved in T Y and T S . The language names of yand s are “ARABIC, SUDANESE CREOLE” and“Arabic, Sudanese Creole” respectively and the same.
Here, all alphabetical notations are not case-sensitive.
Finding out such same language pairs like (y,s)
automatically is our purpose. In order to solve thisproblem, measure of language name similarity has been
introduced and defined. In the following definition, aterm WORD is used, which is a string consisting of
only alphabets, such as “Arabic”. Language name is alist of WORD(s), just like language names “ARABIC,SUDANESE CREOLE” and “Arabic, Sudanese Creole”.
If two WORDs are individually included in twolanguage names and are most similar, for example“ARABIC” and “Arabic”, they are called WORD pair.
[Definition 1] Let v and w be two WORDs. WORD
similarity between v and w, denoted by sd_w(v, w), isdefined bywhere ed (v, w) and l A (v, w) respectively represent edit
distance and length of optimal alignment between v andw, under the condition that the operations of insertions,deletions and substitutions all cost 1 4 .
□Language name similarity, denoted by sd_ln(L 1 ,L 2 ),between L 1 and L 2 is defined by□[Definition 3] Let y ∊ V lea f (T Y ) and s ∊ V leaf (T S ) be
languages and L T Y y and L T S s be the primary language
names of y and s. Let m be the number of alternatenames with s, furthermore let all the alternate names beA s 1 , A s 2 , ∙∙∙, A s m if m > 0. Then language namesimilarity of y and s, denoted by sd_ln node (y, s), isdefined by□
Based on language name similarity, family namesimilarity denoted by sd_fn(y,s), parent name similarity
denoted by sd_pn(y,s) and brother name similarity
denoted by sd_bn(y,s) are defined as follows. These
three similarities are designed for language
classification similarity, which is intended to be adifferent similarity measure, from the aspect oflanguage classification.
[Definition 4] Let FN y and FN s be the family names oflanguages y ∊ V leaf (T Y ) and s ∊ V leaf (T S ), respectively.
Then family name similarity between y and s, denotedby sd_ fn(y, s), is defined by[Definition 2] Let L 1 = { v 1 , v 2 , ∙∙∙ , v m } and L 2 = { w 1 ,w 2 , ∙∙∙ , w n } (m ≧ n) be language names, (v i , w′ i ) be
WORD pair. Note that for any v i ∊ L 1 , if L 2 does notcontain w′ i that corresponds to v i , then let w′ i = Null.
Published by Atlantis Press
Copyright: the authors
104□

On Parameter Setting

Fig. 3. Process of finding same language pairs
Fig. 5. True-false judgment for outputted same language pairs

similarity between x 1 and x 2 , denoted by sd_bn(x 1 , x 2 ),is defined by□
Then language classification similarity, and further
language general similarity that integrates language
name similarity and language classification similarity,
are defined as follows.
[Definition 7] language classification similarity
between y ∊ V l eaf (T Y ) and s ∊ V leaf (T S ), denoted by sd _lc(y,s), is defined byFig. 4. Algorithm: FSLV[Definition 5] Let PN y and PN s be the parent’s names oflanguages y ∊ V leaf (T Y ) and s ∊ V leaf (T S ), respectively.
Then parent name similarity between y and s, denotedby sd_ pn(y,s), is defined by□
Languages “NUBI” and “Nubi” shown in Fig. 2 arebrothers of “ARABIC, SUDANESE CREOLE” and“Arabic, Sudanese Creole”, respectively. A brothercombination from two sets of brothers, such as(“NUBI”, “Nubi”), is called brother language pair.

where e, f and g satisfy 1 ≧ e ≧ 0, 1 ≧ f ≧ 0, 1 ≧ g ≧ 0and e + f + g=1.
□
[Definition 8] language general similarity betweeny ∊ V leaf (T Y ) and s ∊ V leaf (T S ), denoted by sd_gen(y, s), isdefined bywhere a and b satisfy 1 ≧ a ≧ 0, 1 ≧ b ≧ 0 and a + b=1.
□
2.2 Finding same language pairs[Definition 6] Let x 1 (x 2 ) be a language included in T Yor T S (T S or T Y ), BL x 1 ={bl x 1 1 , bl x 1 2, ∙∙∙} and BL x 2 ={bl x 2 1 ,bl x 2 2, ∙∙∙} (|BL x 1 | ≧ |BL x 2 | > 0) be respectively the sets ofbrother languages of x 1 and x 2 , and further BP(x 1 ,x 2 ) bethe set of brother language pair. Then brother nameSame language pairs such as (y,s) shown in Fig. 2 canbe found out by applying language name similarity andlanguage general similarity as follows. Two dummy
WLTs with 3 languages in T Y and 4 in T Y are used todescribe how to find them.

Published by Atlantis Press
Copyright: the authors
105

R. Wu and H. Matsuno

Table 1. Experimental results

Fig. 6. Ups and downs of the evaluation values(1) First, generate a matrix Ψ with y i , s j = sd_ln node (y i ,s j ) as its element for y i ∊ V leaf (T Y ) and s j ∊ V l eaf (T S ) as
shown in Fig. 3 (a).
(2) Then select all the combinations of y i and s jsatisfying sd_ln node (y i ,s j ) > γ − Δ. The selectedcombinations are boxed as shown in Fig. 3 (b). Here,the initial value of and Δ are set to 1 and 0.1,respectively.
(3) Calculate the values of sd_gen(y i ,s j ) for the selectedcombinations as shown in Fig. 3 (c). Values ofcoefficients (e, f, g) and (a, b) will be determined byexperiments.
(4) Select the pair (y 3 ,s 2 ) with the highest language
general similarity value from all the values ofsd_gen(y i , s j ). If we set ρ=0.5, this language pair (y 3 ,s 2 ) will be passed over and identified as the samelanguage. Then the language general similarity
values as shown Fig. 3 (d) are deleted. Similarly, (y 1 ,s 4 ) is identified as well as (y 3 ,s 2 ) as shown inFig. 3 (e).
(5) Delete the values related to the identified samelanguage pairs as shown in Fig. 3 (f).
(6) Update γ as γ = γ − Δ and repeat this process from(2) ～ (5) till γ ≦ 0. The algorithm of thisprocessing is shown in Fig. 4.
3. Parameter Setting

set at first. These three parameters are individually thecoefficients of the three similarities. Take (y,s) shownin Fig. 2 as the example, we consider that, if there aresame language pairs such as (“NUBI”, “Nubi”) within
the brothers, then it is most probably that y and s may
also be the same language. Hence, we lay weight on gthat is the coefficient of brother language similarity,and consider that e=0.25, f=0.25, g=0.5 should beappropriate.
On the other hand, parameters (a,b), Δ and thresholdρ will be determined throughout experiments. We use
test data of 200 languages which were selected out of
all 2,869 languages included in T Y in a random manner.
In advance of setting parameters, we investigate thecorresponding languages in T S for all the languages ofthis test data. However, it does not mean that thecorresponding same languages can always be found in
T S for all the languages.
For the test data, we repeat experiments according
to algorithm FSLV shown in Fig. 4 by changing thevalues of parameters (a,b), Δ and threshold ρ, and thencheck the results with the investigated facts to get thevalues of TP, FP, TN and FN as shown in Fig. 5. Here,TP + FP + TN + FN = 200 should be satisfied.
We use evaluation measures F-measure, Recall,Precision 5 to choose the best (a,b), Δ and threshold ρ.
These measures are defined as follows.

In algorithm FSLV shown in Fig. 4, parameters (e, f, g),(a,b), Δ and threshold ρ are used as input arguments,
and are unknown. What should be done first is setting
their values. In this section, we are going to give asolution for parameter setting.
3.1 Evaluation method and test dataParameters (e, f, g) are used to calculate language
classification similarity which is defined as a weighted
average of family name similarity, parent namesimilarity and brother name similarity, and need to beWe consider that the values of parameters (a,b), Δ
and threshold ρ, with the highest values of F-measureand Precision, should be the best ones.

Published by Atlantis Press
Copyright: the authors
106

On Parameter Setting

3.2 Process of parameter setting
Here, we determine the values of parameters (a, b), Δ
and threshold ρ. Firstly, we deal with parameters (a, b).
Repeat executing FSLV with Δ=0.01 and ρ=0.5 bychanging the values of parameters (a, b) by steps of 0.1
as follows. (1.0, 0.0), (0.9, 0.1), ∙∙∙ , (0.1, 0.9), (0.0,1.0). Then do the same thing as well by increasing thevalues of Δ from 0.01 to 0.25 by steps of 0.01 such as
Δ=0.02, 0.03, ∙∙∙, 0.25. Here, threshold ρ is set as ρ=0.5.
After tallying the values of TP, FP, TN and FN for
every execution of algorithm FS LV, compute thevalues of Recall，Precision and F-measure and use
them to estimate the trend of parameters (a, b). We get
to notice that the values of F-measure and Precision
have the best values when a = 0.5 and b = 0.5. So we
set the values of parameters (a, b) to (0.5, 0.5).
What need to be set continuously is the value ofparameter Δ. As described above, we repeat executing
algorithm FSLV by increasing the values of Δ from 0.01
to 0.25 by steps of 0.01. The values of F-measure andPrecision have ups and downs due to the change of Δ.
We show a part of values of F-measure and Precision inFig. 6(1) and Fig. 6(2) respectively by setting Δ as 0.01,0.05, 0.10, 0.15, 0.20, 0.25, 0.13 under the condition ofa=0.5, b=0.5, ρ=0.5. Fig. 6(1) indicates that F-measure
has a peak at Δ=0.10 and 0.13. At the two points, Fig.
6(2) has the same highest values. We cannot
differentiate the effect between the cases when Δ=0.10
and Δ=0.13, So we set Δ=0.10 or Δ=0.13.
Finally, we set the value of threshold ρ. Under thecondition of Δ=0.10, a=0.5, b=0.5 or Δ=0.13, a=0.5,b=0.5, we repeat execution of algorithm FSLV by
increasing the value of ρ from 0.50 to 0.80 by steps of0.05, then we found that F-measure has a peak at
ρ=0.55. So we repeat the execution additionally by
setting ρ as 0.51, 0.52, 0.53 or 0.54 individually.
F-measure shows the same value at ρ=0.54 or ρ=0.55 as
well as Precision. Under the condition of the same
F-measure and Precision, the threshold with lower
value is better. Therefore, we set ρ=0.54.
Up to here, we have gotten the best values ofparameters (a, b), Δ and threshold ρ under the conditionof e=0.25, f=0.25, g=0.5 by using the test data. As theresult, Δ=0.10, a=0.5, b=0.5, ρ=0.54 or Δ=0.13, a=0.5,
b=0.5, ρ=0.54 are obtained.

accuracy of parameter setting and the usefulness andeffectiveness of our proposed method. By setting theparameters to (i) Δ=0.10, a=0.5, b=0.5, ρ=0.54 and (ii)Δ=0.13, a=0.5, b=0.5, ρ=0.54 (both under e=0.25,
f=0.25, g=0.5), we have executed algorithm FSLV for
T Y and T S and gotten the results shown in Table 1. in all
the 2,869 languages of T Y , we have gotten 2,687 samelanguage pairs as the output of algorithm FSLV, inwhich 2,648 languages (92%) are the true cases.
5. Concluding Remarks
We have given a way of parameter setting for our
previously proposed method. Using the parametervalues, about 92% languages of T Y have been
identified, and the precision of identification is up to98%. These results imply that our parameter setting
described above is reasonably good. And at the sametime, we have confirmed that our proposed method
previously is useful and effective.
It should be noticed that the values of parameters (a,b), Δ and threshold ρ are set under the condition of
e=0.25, f=0.25, g=0.5. As the future work, we need todo further tests to confirm the parameter setting.
Acknowledgement
This work was supported by JSPS KAKENHI
Grant-in-Aid for Challenging Exploratory Research
Numbers 23650129.
References
1. R. Wu and H. Matsuno, Identifying Same Languages byConsidering

Similarities

ofLanguage

NameandLanguage Classification, Proceedings of The 2010
International Technical Conference on Circuits/Systems,Computers and Communications(ITC-CSCC 2010),

pp.516-519.
2. R. Wu, H. Inui and H. Matsuno, New Measurement of
Similarity of Language Classification, Proceedings ofThe2012

International

Circuits/Systems,

Technical

ComputersandConferenceonCommunications(ITC-CSCC 2012), in CD-ROM (F-M2-05).
3. Euzenat, J. and Shvaiko, P.: Ontology Matching(Springer-Verlag, 2007)
4. G. Navarro, A guided tour to approximate string
matching,ACM

Computing

Surveys(CSUR),33(1)(2001), pp.31-88.

4. Experimental Results

5. V. Vapnik, The nature of statistical learning theory

We have done experiment by applying the obtained
parameters to original data T Y and T S to confirm the(Springer, New York, 1995)

Published by Atlantis Press
Copyright: the authors
107