2019 8th International Congress on Advanced Applied Informatics (IIAI-AAI)

Book Recommender System for Wikipedia Article
Readers in a University Library
Keita Tsuji
Faculty of Library, Information and Media Science,
University of Tsukuba
Tsukuba-city, Japankeita@slis.tsukuba.ac.jp

automatically assigned to Wikipedia articles? (b) how can booksof the same NDC categories as Wikipedia articles be selected?

Abstract—Wikipedia has emerged as an important source of
information for university students. It has been reported that
university students tend to read and do research with Wikipedia
articles more than they do with books, even when within a library.
To encourage students to read library books as a more reliable
source of information, a library system was developed forrecommending library books to Wikipedia readers within aparticular university library. The proposed system assigns a
Nippon Decimal Classification (NDC) category to each Wikipedia
article and recommends library books in the same NDC category
to readers of the article. In the test implementation of the system,
the precision of assigning NDC categories to Wikipedia articles
using a convolutional neural network was as high as 87.4%, while
the precision of selecting books for recommendation using a
support vector machine reached 99.8%.

The reason for assigning NDC categories to Wikipedia
articles is as follows. As would be further discussed, 1,070,202
Japanese Wikipedia articles were considered in this study, andthe average number of printed books held by Japanese university
libraries in 2017 was approximately 227,244 (= 323,595,000
books divided by 1,424 libraries reported by the Japan Library
Association in 2018 [2]). If all the books in a library are
considered as potential candidates for recommendation for eachWikipedia article, the total possible combination of articles andbooks would be 1,070,202 × 227,244. A system based on thisfigure would require a time-consuming and unrealistic operation.
It thus becomes rational to adopt a process for reducing thenumber of candidate books for a particular article. The NDC isvery handy for this purpose. Almost all Japanese books inJapanese libraries are assigned an NDC category as part of theirrespective call numbers. If NDC categories are similarly
assigned to Wikipedia articles based on their content and only
books of the same category are considered as recommendation
candidates for a particular article, the number of possiblecombinations of books and articles would be significantly
reduced. For instance, if a Wikipedia article is assigned NDC
category 324 (i.e., Civil Law), only books in the same NDC
category would be considered as recommendation candidates
for that article.

Keywords—Book Recommendation; Wikipedia; University
Library; Convolutional Neural Network; Support Vector Machine;
Nippon Decimal Classification

I. INTRODUCTION
Wikipedia has become an important source of information
for university students. It has been reported that university
students tend to read and do research with Wikipedia articles
than with books, even when within a library [1]. Within thiscontext, a system was developed for recommending librarybooks to Wikipedia readers when they are within a university
library. The system is addable to the Web browser of a librarydesktop PC. The recommended books are displayed to the
Wikipedia article reader with their covers and call numbers
within that library. The system is aimed at encouraging students
to read library books as a more reliable source of information
rather than relying on Wikipedia articles. Although thedemonstration of the proposed system in the present study
considered readers of Japanese Wikipedia articles in Japanese
university libraries, the system is applicable to any language.

A convolutional neural network (CNN) and support vector
machine (SVM) were respectively used to answer the abovementioned questions (1) and (2) in the development of the
proposed system, namely, to automatically assign NDC
categories to Wikipedia articles and to select books forrecommendation to readers of a particular article. These twotools are representative machine learning methods for textclassification [3, 4, 5, 6].

The methodology of the proposed library system involves
the following two steps:

II. RELATED STUDIES(1) Assigning a Nippon Decimal Classification (NDC) category
to each Japanese Wikipedia article.

There have been some previous works on book
recommendation to library users. Mikawa et al. [7] proposed theuse of silhouette images captured by cameras installed inlibraries and recommended books based on the gender and ageof the library users. Jomsri [8] proposed the use of an association
rule based on the faculty and profile of the user, their bookloaning data, and book categories. The target users of the system(2) Selecting books to recommend to readers of each article.
In the context of these two steps, the present study
considered two questions: (a) how can NDC categories be

978-1-7281-2627-2/19/$31.00 ©2019 IEEE
DOI 10.1109/IIAI-AAI.2019.00034

121are somewhat similar to those of the present system, although
the present target users are further narrowed to readers of
Wikipedia articles in university libraries. In addition, theinformation used for book recommendation in the present
system mainly consists of the words in the Wikipedia article.
This particularly distinguishes the system from those of Mikawa
et al. [7] and Jomsri [8].

Development of Training and Testing Data
The steps below were used to develop the training andtesting data that were used to assign NDC categories to the
Wikipedia articles.
(b-1) From the Wikipedia articles extracted in Step (a-2) above,
the articles that contained “References” and cited bookswith their ISBNs were extracted. There were 50,375 sucharticles, including 45,151 unique ISBNs and 117,219 books(including duplicates). Only books with specified ISBNs
were considered because this enabled obtaining the bookbibliographies without ambiguity in Step (b-2). A
procedure that utilizes book titles or author names is likely
to produce differing bibliographies.

The author has previously investigated the use of books citedin Wikipedia articles and their NDC categories in making book
recommendations to readers [9], and attempted to assign NDC
categories to Wikipedia articles based on their setting titles,
categories, and main texts using one single CNN channel [10].

(b-2) Based on the identified book ISBNs, the unique andconsistent bibliographies of the cited books were obtainedusing the OpenSearch tool of the National Diet Library ofJapan [15], which is the largest and most comprehensive
library in Japan. Bibliographies that did not contain threedigit NDCs (for instance, the three digits such as 324
indicate the book’s main class 3, division 2, and section 4)
or book titles were excluded. Overall, 40,647
bibliographies were obtained.

III. METHODThis section explains some key aspects of the proposed
library book recommendations system. The first aspect is how
Wikipedia articles are accessed and the distributed
representation of nouns in them are determined. The second is
how an NDC category is assigned to an article. The third is how
books are selected for recommendation to an article reader. The
fourth is obtaining the user’s evaluation of the recommended
books. It should be emphasized that the objective of the second
aspect.

(b-3) The most popular NDC categories among the books citedin each article were identified. Regarding the level or depthof the NDC categories, the main class and division of thebooks were employed. For example, if an article cited fivebooks with NDC categories 324, 324, 324, 325, and 369,
respectively, the NDC category 32 (main class 3 anddivision 2) was considered to be the most popular, having
a frequency of four. Incidentally, NDC category 32 and 36
indicate Law and Society, respectively.

A. Accessing Wikipedia Articles and Determining theDistributed Representation of Nouns
The steps below were used to access Wikipedia articles anddetermine the distributed representation of nouns.
(a-1) The Japanese Wikipedia dump as of August 1, 2017 wasdownloaded from https://dumps.wikimedia.org/jawiki/.
The file size was approximately 14 GB.

(b-4) An article was extracted if the frequency of the mostpopular NDC category was ı3 and its ratio relative to thetotal number of categories were ı 0.8. The thresholdvalues of 3 and 0.8 were chosen empirically. Thus, theexample article mentioned in Step (b-3) was extracted
because the frequency of the NDC category 32 is 4 (> 3)
and its ratio is 0.8 (= 4/5). In this way, 5,385 articles wereextracted. This set of articles is henceforth referred to as the
LC set.

(a-2) Pages (i) (“ns” Į 0, indicating that it is not a“Main/Article”) and (ii) (it is only a redirection page) of thearticles were removed from the download (here, “ns” is thenamespace tag). The number of pages that remained was
1,070,202, which are implied in subsequent references to
Wikipedia articles.
(a-3) Mecab ver. 0.996 [11] with the mecab-ipadic-NEologd
dictionary (as of August 10, 2017) [12] was used for amorphological analysis of the downloaded Wikipedia
articles. A total of 5,512,620 types of nouns were identified
in the articles.

(b-5) It was assumed that the most popular NDC category in anarticle in the LC set was the one that should be assigned tothat article. For instance, the NDC category 32 should be
assigned to the example article in Step (b-4). Accordingly,
5,385 pairs of the above-mentioned articles and NDC
categories were extracted from the LC set. These were usedas training and testing data for the CNN and SVM, as will
be explained in the next subsection. The number of NDC
categories that were to be assigned to the LC set of articles
are presented in Table I, where C, N, and T respectively
represent the NDC category, number of articles, and theirrespective total values. It can be seen from Table I that, forinstance, NDC category 32 should be assigned to 107
articles in the LC set. In other words, 107 articles wereobtained for the training and testing of the CNN for NDC(a-4) Word2Vec [13] in gensim [14] was used to obtain a 200dimensional distributed representation of each noun in the
Wikipedia articles. The parameters window and min_countin the Word2Vec module of gensim were set to 5 and 10,
respectively.
B. Assigning NDC Categories to Each Wikipedia Article
This subsection explains the development of the training andtesting data that were used to assign NDC categories to the
Wikipedia articles. The data were inputted to the CNN and SVM.

122

Finally, ten nouns within the main text of each LC set article
with the highest TF-IDFs (term frequency-inverse document
frequency) were extracted. The TF-IDF of noun X in article Y
is defined as “frequency of X in the main text of Wikipedia
article Y” multiplied by “log (number of Wikipedia
articles/number of articles with main text containing X).” These
were represented by a 50 × 200 matrix. The {(i−1)*10+1}-th
row contained the 200-dimensional vector (obtained throughStep (a-4) in subsection III.A) of the noun whose TF-IDF was
the i-th highest in the main text (i İ 10). The other rows were
set to zero vectors. This matrix is henceforth referred to as tx.

category 32. Although the CNN and SVM could not be
assigned NDC categories that were not included in the
training data, these were somewhat exceptional cases.
Actually, only 16 NDC categories (05, 06,…97) were notincluded in the LC set.
TABLE I.
㻯
㻜㻜
㻜㻝
㻜㻞
㻜㻟
㻜㻠
㻜㻡
㻜㻢
㻜㻣
㻜㻤
㻜㻥
㼀

㻺
㻟㻞
㻞㻞
㻝㻜
㻝
㻞
㻜
㻜
㻠
㻜
㻜
㻣㻝

㻯
㻝㻜
㻝㻝
㻝㻞
㻝㻟
㻝㻠
㻝㻡
㻝㻢
㻝㻣
㻝㻤
㻝㻥
㼀

㻺
㻝
㻝
㻝㻠
㻝㻠
㻡㻞
㻟
㻡㻟
㻝㻣
㻣㻥
㻢㻜
㻞㻥㻠

NUMBERS OF NDC CATEGORIES TO BE ASSIGNED TO
ARTICLES IN LC SET
㻯
㻺
㻞㻜
㻠
㻞㻝
㻣㻡㻠
㻞㻞
㻣㻠
㻞㻟
㻝㻝㻠
㻞㻠
㻡
㻞㻡
㻢
㻞㻢
㻜
㻞㻣
㻝
㻞㻤
㻝㻠㻡
㻞㻥
㻝㻡㻥
㼀 㻝㻘㻞㻢㻞

㻯
㻟㻜
㻟㻝
㻟㻞
㻟㻟
㻟㻠
㻟㻡
㻟㻢
㻟㻣
㻟㻤
㻟㻥
㼀

㻺
㻡
㻟㻢
㻝㻜㻣
㻠㻞
㻜
㻜
㻠㻢
㻞㻟
㻣㻟
㻢㻞
㻟㻥㻠

㻯
㻠㻜
㻠㻝
㻠㻞
㻠㻟
㻠㻠
㻠㻡
㻠㻢
㻠㻣
㻠㻤
㻠㻥
㼀

㻺
㻢
㻝㻟㻥
㻟㻣
㻝㻝
㻝㻟
㻝㻠㻞
㻝㻣
㻝㻝㻠
㻞㻣㻥
㻞㻝㻥
㻥㻣㻣

㻯
㻡㻜
㻡㻝
㻡㻞
㻡㻟
㻡㻠
㻡㻡
㻡㻢
㻡㻣
㻡㻤
㻡㻥
㼀

㻺
㻤
㻝㻟
㻞㻢
㻡㻣
㻟㻞
㻞㻟㻠
㻠
㻞
㻤
㻤㻝
㻠㻢㻡

㻯
㻢㻜
㻢㻝
㻢㻞
㻢㻟
㻢㻠
㻢㻡
㻢㻢
㻢㻣
㻢㻤
㻢㻥
㼀

㻺
㻜
㻣
㻞
㻜
㻠㻠
㻞㻜
㻢
㻞㻜
㻝㻜㻝
㻣
㻞㻜㻣

㻯
㻺
㻣㻜
㻝㻝
㻣㻝
㻜
㻣㻞
㻝㻝㻤
㻣㻟
㻜
㻣㻠
㻝㻠
㻣㻡
㻠㻡
㻣㻢
㻝㻥㻜
㻣㻣
㻡㻣㻢
㻣㻤
㻟㻟㻜
㻣㻥
㻥㻜
㼀 㻝㻘㻟㻣㻠

㻯
㻤㻜
㻤㻝
㻤㻞
㻤㻟
㻤㻠
㻤㻡
㻤㻢
㻤㻣
㻤㻤
㻤㻥
㼀

㻺
㻠
㻞㻝
㻝
㻞
㻜
㻜
㻜
㻜
㻜
㻝
㻞㻥

㻯
㻥㻜
㻥㻝
㻥㻞
㻥㻟
㻥㻠
㻥㻡
㻥㻢
㻥㻣
㻥㻤
㻥㻥
㼀

㻺
㻠
㻞㻣㻞
㻤
㻥
㻝㻜
㻢
㻜
㻜
㻝
㻞
㻟㻝㻞

Hence, tl, cg, and tx constituted the input channels to thepresent CNN. All seven possible combinations of these channels
were tested, namely, (a) tl, (b) cg, (c) tx (i.e. only one channel),(d) tl and cg, (e) tl and tx, (f) cg and tx (two channels), and (g)tl, cg, and tx (all three channels).

(b-6) The above-mentioned 5,385 pairs were randomly divided
into 4,835 training data and 500 testing data. The former
was used to train the CNN and SVM to assign NDC
categories to each Wikipedia article, while the latter wasused to compare the performances of the CNN and SVM.
The better of CNN and SVM was then used to assign NDC
categories to all the Wikipedia articles. As will be
discussed later, the assigned NDC categories were used to
select books for recommendation to the Wikipedia article
readers.

CNN Settings
The CNN is well-known for its excellent ability for visual
recognition. The above-mentioned matrices can be considered
as two-dimensional (2D) images. The present CNN utilized fivekinds of filters of heights 1, 2, 3, 4, and 5, respectively. All thefilters had a width of 200. Furthermore, for each filter size, six
different numbers of filters were used in the tests, namely, 50,
100, 150, 200, 250, and 300.
The Tensorflow ver. 1.0 [16] was used to create and train the
CNN. The strides and padding of the tf.nn.conv2d function were
set to [1, 1, 1, 1] and VALID, respectively. Rectified linear units(tf.nn.relu) were used after tf.nn.conv2d, and max pooling was
employed. The ksize, strides, and padding of the tf.nn.max_pool
function were set to [1, IH-FH+1, 1, 1], [1, 1, 1, 1], and VALID,
respectively, where IH and FH respectively represent the heightsof the input channel and filter. The tf.nn.softmax function wasused in the final output layer. The size of the mini batches,
number of epochs, dropout rate, and learning rate were 100, 300,
0.5, and 0.0001, respectively. The data flow is shown in Figure
1.

Input to CNN
The input to the employed CNN utilized three channels,
which correspond to the title, category, and main text of the
Wikipedia articles. The structures of the channels and how the
input was implemented are discussed below.
First, the nouns in the title of each Wikipedia article in the
LC set were extracted and represented by their distributed
representation (i.e., 200-dimensional vectors obtained throughStep (a-4) in subsection III.A). The vectors and other zero
vectors constitute a 50 (rows) by 200 (columns) matrix, referred
to as the title channel for the CNN. In the title channel matrix,
the i-th row contains (a) the 200-dimensional vector for the i-th
noun in the title, when i İ 5 (the threshold was chosenempirically), and (b) a zero vector, when i > 5. If the number ofnouns in a certain title is less than five, the corresponding row
would contain a zero vector. If the number of nouns is greaterthan five, only the first five nouns would be adopted in the title
channel. This matrix is henceforth referred to as tl.
The first five categories in each LC set article were extracted
and represented by a 50 × 200 matrix. The {(i−1)*10+j}-th rowof the matrix contained the 200-dimensional vector (obtained by
Step (a-4) in subsection III.A) for the j-th noun in the i-th
category (i İ 5 and j İ 5). The threshold 5 was chosenempirically. The other rows contained zero vectors. If thenumber of nouns in a certain category was less than five, thecorresponding row was set to a zero vector. If the number ofnouns was greater than five, only the first five nouns in eachcategory was adopted. This matrix is henceforth referred to as
cg.

Fig. 1. Data Flow of the Developed CNN

123

Input to SVM(c-3) The author, together with others, has previously reported
that the recommendation of frequently borrowed books
produces better results [8]. To promote therecommendation of such books, the loan frequencies of thelibrary books were adopted as a feature in the CNN and
SVM, specifically the number of times that each book had
been borrowed between January 2, 2006 and March 31,
2017. This parameter is henceforth referred to as N. Eachbook was then assigned a value ln(N/Y+1), where Y is (a)2018 minus the publication year of the book, if the book
was published in or after 2006, or (b) 12, if the book was
published before 2006. For instance, ln(20/4+1) = ln(6) was
assigned to a book published in 2014 and had beenborrowed 20 times during the above-mentioned period.
Henceforth, ln(N/Y+1) is referred to as LF. The LF is thenatural logarithm of a rough approximation of the numberof times the book had been borrowed per year during theabove-mentioned period. The initial idea was to use N/Y,
but the above natural logarithm seemed to be more
effective in the pilot experiment. Further study is planned
to confirm this observation based on the responses of thesubjects to questions presented to them.

The channels mentioned in the previous subsections wereconverted into the vector V, where the V's {(i−1)*200+j}-thelement is the (i, j) element of the channel and was inputted to
the SVM. When multiple channels were used for the input to the
SVM in this study, the corresponding vectors were concatenatedfor the input. For instance, when all the three channels (title,
category, and main text) were used, the dimension of the input
vector was 30,000 (= 50 × 200 × 3).
SVM Settings
The LIBSVM ver. 3.23 [17] was used with the SVM toassign NDC categories to the Wikipedia articles. The radialbasis function (RBF) kernel of the SVM was employed for thispurpose. The easy.py of the LIBSVM package was used todetermine the optimal values of the parameters C and γ.
C. Selecting Books to Recommend
This subsection describes the selection of books torecommend to a Wikipedia article reader. The training andtesting data were generated and inputted to the CNN and SVM.

(c-4) The sets {ai}, {xi}, {yi}, and {zi} were defined as
distributed representations (the 200-dimensional vectors
obtained in Step (a-4) in subsection III.A) of the first fivenouns in the book titles (A), the first five nouns in the titlesof the Wikipedia articles (X), the first five nouns in the first
five categories of the Wikipedia articles (Y), and ten nouns
in the main texts of the Wikipedia articles (Z) with thehighest TF-IDFs, respectively. Calculations were thenimplemented to obtain a set of values {dot(ai, xj)}i,j, whichwere arranged in descending order, where dot(p, q) is theinner product of vectors p and q. If the number of values
was less than a threshold M, 0’s were appropriately added
to the end of the sequence. This sequence of values (i.e., avector) is henceforth referred to as SAX. The vectors SAY
and SAZ were similarly obtained from {dot(ai, yj)}i,j and{dot(ai, zj)}i,j, respectively.

Generation of Training and Testing Data
The following steps were used to generate the training andtesting data for selecting books to recommend to Wikipedia
article readers.
(c-1) As mentioned in section I, the aim of the present study was
the development of a system for recommending librarybooks to students who read Wikipedia articles in university
libraries. The experimental implementation of the systemin this study was conducted at T University. There were621,129 candidate books for recommendation in the library,
the books having been borrowed at least once between
January 2, 2006 and March 31, 2017. It was assumed thatbooks that had not been borrowed in more than 10 years
would not be suitable for recommendation. The exclusionof such books also had the additional benefit of reducing
the computational cost of the system.

(c-5) Pairs of Wikipedia articles and the books they cited wereextracted as training data (i.e., pairs of Wikipedia articles
and books that should/should not be recommended to thearticle readers). More concretely, from the combination of
50,375 articles and 117,219 books they cited (mentioned in(b-1) in subsection III.B), pairs of articles and books wereextracted if they satisfied all the following conditions: (1)the assigned NDC categories of the former and the NDC
categories of the latter were identical, (2) LF of the books
in Step (c-3) were no less than ln2 (implying that the books
were borrowed at least once a year on average), and (3) the
maximum values of the elements of SAX, SAY, and SAZ
in Step (c-4) were ı0.9. The threshold 0.9 was chosen
heuristically through preliminary experiments. The pairs
were regarded as positive examples (i.e., examples of
Wikipedia articles that the corresponding books should berecommended for). A total of 5,444 such pairs of articles
and books were extracted. Among these were 3,943 articles,
which implies that some articles were paired with more(c-2) The distribution of the NDC categories of the 621,129
candidate books is presented in Table II, where C and N
represent NDC category and number of corresponding
samples, respectively. It can be seen from the Table II that,for instance, the number of books with NDC category 32 is
21,276. Incidentally, the 621,129 candidate books had
3,830,241 loan records.
TABLE II.
㻯
㻜㻜
㻜㻝
㻜㻞
㻜㻟
㻜㻠
㻜㻡
㻜㻢
㻜㻣
㻜㻤
㻜㻥
㼀

㻺
㻝㻠㻘㻡㻠㻠
㻤㻘㻡㻢㻞
㻟㻘㻡㻤㻠
㻡㻣㻞
㻤㻞㻣
㻢㻤㻣
㻣㻡㻠
㻝㻘㻜㻤㻤
㻞㻠㻘㻤㻥㻡
㻝㻘㻣㻢㻝㻡㻣㻘㻞㻣㻠

㻯
㻝㻜
㻝㻝
㻝㻞
㻝㻟
㻝㻠
㻝㻡
㻝㻢
㻝㻣
㻝㻤
㻝㻥
㼀

㻺
㻟㻘㻤㻥㻝
㻞㻘㻟㻤㻤
㻡㻘㻡㻥㻞
㻝㻜㻘㻟㻢㻞
㻝㻟㻘㻟㻜㻡
㻝㻘㻠㻝㻣
㻠㻘㻡㻢㻣
㻝㻘㻠㻞㻟
㻠㻘㻢㻥㻠
㻟㻘㻣㻡㻜
㻡㻝㻘㻟㻤㻥

DISTRIBUTION OF NDC CATEGORIES OF BOOKS BORROWEDFROM T UNIVERSITY LIBRARY
㻯
㻞㻜
㻞㻝
㻞㻞
㻞㻟
㻞㻠
㻞㻡
㻞㻢
㻞㻣
㻞㻤
㻞㻥
㼀

㻺
㻡㻘㻞㻠㻠
㻞㻝㻘㻥㻤㻣
㻤㻘㻤㻣㻣
㻠㻘㻟㻡㻢
㻝㻘㻟㻡㻡
㻝㻘㻝㻠㻝
㻠㻝㻤
㻞㻞㻡
㻠㻘㻟㻤㻟
㻠㻘㻣㻠㻜
㻡㻞㻘㻣㻞㻢㻯㻺㻟㻜 㻝㻜㻘㻟㻥㻥㻟㻝 㻝㻤㻘㻝㻣㻣㻟㻞 㻞㻝㻘㻞㻣㻢㻟㻟 㻞㻡㻘㻢㻟㻥
㻟㻠
㻞㻘㻢㻥㻢
㻟㻡
㻞㻘㻠㻤㻞
㻟㻢 㻞㻡㻘㻣㻝㻝㻟㻣 㻟㻢㻘㻝㻢㻤㻟㻤 㻝㻝㻘㻜㻞㻠㻟㻥㻝㻘㻟㻥㻥㼀 㻝㻡㻠㻘㻥㻣㻝

㻯
㻺
㻠㻜
㻢㻘㻜㻤㻤
㻠㻝 㻞㻢㻘㻝㻠㻣㻠㻞 㻝㻝㻘㻜㻞㻜
㻠㻟
㻣㻘㻜㻜㻞
㻠㻠
㻝㻘㻠㻥㻡
㻠㻡
㻢㻘㻞㻠㻟
㻠㻢
㻤㻘㻞㻞㻢
㻠㻣
㻞㻘㻜㻤㻥
㻠㻤
㻞㻘㻤㻠㻣
㻠㻥 㻠㻟㻘㻠㻣㻥㼀 㻝㻝㻠㻘㻢㻟㻢

㻯
㻡㻜
㻡㻝
㻡㻞
㻡㻟
㻡㻠
㻡㻡
㻡㻢
㻡㻣
㻡㻤
㻡㻥
㼀

㻺
㻤㻘㻣㻣㻜
㻤㻘㻜㻢㻟
㻤㻘㻣㻡㻝
㻞㻘㻣㻣㻠
㻤㻘㻢㻣㻞
㻡㻢㻥
㻝㻘㻟㻞㻥
㻝㻘㻣㻢㻠
㻝㻘㻞㻥㻣
㻣㻝㻣
㻠㻞㻘㻣㻜㻢

㻯
㻢㻜
㻢㻝
㻢㻞
㻢㻟
㻢㻠
㻢㻡
㻢㻢
㻢㻣
㻢㻤
㻢㻥
㼀

㻺
㻠㻘㻜㻢㻜
㻣㻘㻣㻝㻥
㻝㻘㻣㻢㻥
㻝㻠㻢
㻝㻘㻜㻝㻝
㻞㻘㻟㻥㻝
㻢㻝㻣
㻠㻘㻜㻜㻞
㻞㻘㻜㻡㻣
㻣㻥㻟
㻞㻠㻘㻡㻢㻡

㻯
㻣㻜
㻣㻝
㻣㻞
㻣㻟
㻣㻠
㻣㻡
㻣㻢
㻣㻣
㻣㻤
㻣㻥
㼀

㻺
㻝㻜㻘㻡㻞㻝
㻝㻘㻢㻤㻡㻝㻡㻘㻥㻡㻤
㻣㻣㻡
㻝㻘㻣㻣㻠
㻠㻘㻝㻟㻡
㻞㻘㻟㻝㻣
㻞㻘㻥㻞㻜
㻝㻡㻘㻜㻣㻠
㻢㻥㻞㻡㻡㻘㻤㻡㻝

㻯
㻤㻜
㻤㻝
㻤㻞
㻤㻟
㻤㻠
㻤㻡
㻤㻢
㻤㻣
㻤㻤
㻤㻥
㼀

㻺
㻤㻘㻜㻤㻝
㻣㻘㻝㻤㻤
㻞㻘㻤㻤㻢
㻠㻘㻡㻜㻣
㻤㻜㻥
㻞㻘㻟㻟㻜
㻝㻠㻣
㻤㻢
㻢㻣㻡
㻢㻝㻝
㻞㻣㻘㻟㻞㻜

㻯
㻥㻜
㻥㻝
㻥㻞
㻥㻟
㻥㻠
㻥㻡
㻥㻢
㻥㻣
㻥㻤
㻥㻥
㼀

㻺
㻟㻘㻣㻠㻢
㻝㻥㻘㻢㻥㻣
㻡㻘㻜㻤㻠
㻡㻘㻝㻡㻞
㻞㻘㻝㻝㻝
㻞㻘㻡㻟㻤
㻝㻣㻞
㻝㻝㻢
㻢㻤㻣
㻟㻤㻤
㻟㻥㻘㻢㻥㻝

124

and channels does not necessarily improve the results, owing to
the problem of overfitting.

than one book. Articles on Leukemia and Galois Theory
were the most represented and appeared in 14 pairs.
(c-6) For each of the 3,943, the books were chosen when (1) their
NDC categories were identical to the NDC categories
assigned to the articles and (2) they were not cited in thearticles. For instance, when a certain article appeared inthree positive examples, three books with the same NDC
category as that assigned to the article and which were notcited in the article were randomly chosen. In this way,
another 5,444 pairs of articles and books were generated
and regarded as negative examples. A total of 10,888
training data was thus obtained.

TABLE III.

PRECISION OF NDC CATEGORY ASSIGNMENT USING CNN

㼠㼘
㼏㼓
㼎㼐
㼠㼘㻗㼏㼓
㼠㼘㻗㼎㼐
㼏㼓㻗㼎㼐
㼠㼘㻗㼏㼓㻗㼎㼐

㻡㻜
㻠㻣㻚㻠
㻣㻟㻚㻠
㻣㻡㻚㻞
㻣㻟㻚㻢
㻤㻜㻚㻞
㻤㻠㻚㻤
㻤㻡㻚㻤

㻝㻜㻜
㻠㻥㻚㻜
㻢㻣㻚㻢
㻤㻜㻚㻜
㻣㻡㻚㻞
㻤㻜㻚㻤
㻤㻠㻚㻠
㻤㻡㻚㻜

㻺㼡㼙㼎㼑㼞㻌㼛㼒㻌㻲㼕㼘㼠㼑㼞㼟
㻝㻡㻜
㻞㻜㻜
㻠㻣㻚㻠
㻠㻤㻚㻢
㻢㻣㻚㻤
㻢㻥㻚㻞
㻤㻜㻚㻜
㻤㻜㻚㻤
㻣㻟㻚㻜
㻣㻠㻚㻜
㻤㻞㻚㻜
㻤㻞㻚㻠
㻤㻠㻚㻤
㻤㻡㻚㻤
㻤㻢㻚㻞
㻤㻢㻚㻜

㻞㻡㻜
㻠㻥㻚㻠
㻢㻟㻚㻠
㻤㻜㻚㻠
㻣㻡㻚㻜
㻤㻝㻚㻞
㻤㻢㻚㻜
㻤㻣㻚㻠

㻟㻜㻜
㻠㻥㻚㻞
㻢㻤㻚㻜
㻤㻜㻚㻤
㻣㻣㻚㻞
㻤㻞㻚㻞
㻤㻢㻚㻞
㻤㻡㻚㻠

Table IV presents the SVM-based NDC category assignment
results for the LC set of Wikipedia articles. It can be seen fromthe table that, for example, the use of the distributed
representation of only the titles produces a precision of just
67.2%, corresponding to C and γ (subsection III.B) values of 128
and 0.031250000, respectively. The highest precision of 84.4%
is achieved using the distributed representations of the titles,
categories, and main texts, with C and γ values of 512 and
0.000122070. This precision is lower than the highest value of
87.4% achieved by the CNN (Table III).

Input to and Settings of CNN and SVM
The scalar LF in Step (c-3) and vectors SAX, SAY, and SAZ
in Step (c-4) were concatenated and inputted to the CNN and
SVM. Regarding M in Step (c-4), values of 10, 20, and 30 (innerproducts of vectors) were considered in the tests. Therefore, forexample, for M = 30, a vector of length 91 (= 1 + (30 × 3)) was
inputted to the CNN and SVM. In this case, the number of the
CNN input channels was one, and the height and width of thefilter were 1 and 1+M*3, respectively. The tests considered 50,
100, 200, and 400 filters, respectively. Regarding the SVM, theeasy.py was once again used to determine the optimal values of
C and γ for the radial basis function kernel.

TABLE IV.

PRECISION OF NDC CATEGORY ASSIGNMENT USING SVM

㼠㼘
㼏㼓
㼎㼐
㼠㼘㻗㼏㼓
㼠㼘㻗㼎㼐
㼏㼓㻗㼎㼐
㼠㼘㻗㼏㼓㻗㼎㼐

In the tests, the present CNN (or SVM) was used to assign
the NDC category (main class and division) of each Wikipedia
article, as described in subsection III.B. Subsequently, for eachWikipedia article, all the books in the T University Library with
NDCs identical with that of the article were identified. For
example, for a Wikipedia article with an assigned NDC of 32,
all the books in the T University Library with an NDC of 32
were identified. The three books with the highest probabilities
in the CNN and SVM were then recommended for that article.

㻼㼞㼑㼏㼕㼟㼕㼛㼚
㻢㻣㻚㻞
㻣㻥㻚㻟
㻣㻥㻚㻝
㻤㻜㻚㻢
㻤㻜㻚㻝
㻤㻠㻚㻜
㻤㻠㻚㻠

㻯
㻝㻞㻤
㻟㻞
㻤
㻝㻞㻤
㻡㻝㻞
㻡㻝㻞
㻡㻝㻞

γ
㻜㻚㻜㻟㻝㻞㻡㻜㻜㻜㻜㻜㻚㻜㻜㻝㻥㻡㻟㻝㻞㻡㻜㻚㻜㻜㻝㻥㻡㻟㻝㻞㻡㻜㻚㻜㻜㻝㻥㻡㻟㻝㻞㻡㻜㻚㻜㻜㻜㻠㻤㻤㻞㻤㻝㻜㻚㻜㻜㻜㻜㻜㻟㻜㻡㻞㻜㻚㻜㻜㻜㻝㻞㻞㻜㻣㻜

Based on the above results, the developed CNN was used toassign NDC categories to all the Wikipedia articles using theirtitles, categories, and main texts and 250 filters. The number of
Wikipedia articles with respect to the NDC category obtained bythis means is presented in Table V. Here, C and N represent the
NDC category and number of articles, respectively. It can beseen that, for example, the number of Wikipedia articles
assigned NDC category 32 is 9,447.

IV. RESULTS
This section presents the results of (1) the assignment of
NDC categories to Wikipedia articles, (2) the selection andrecommendation of books to articles readers, and (3) theevaluation of the recommended books by articles readers.

TABLE V.
㻯㻺㻜㻜 㻝㻠㻘㻠㻟㻠㻜㻝 㻝㻘㻤㻥㻞㻜㻞 㻝㻘㻣㻟㻡
㻜㻟
㻜
㻜㻠
㻥㻤
㻜㻡
㻜
㻜㻢
㻜
㻜㻣
㻟㻜㻠
㻜㻤
㻜
㻜㻥
㻜
㼀 㻝㻤㻘㻠㻢㻟

A. Assigning NDC Categories to Wikipedia Articles
The precision of the NDC category assignment to the
Wikipedia articles was defined as the ratio of the number ofarticles to which correct categories were assigned to the total
number of articles, expressed as a percentage. The CNN-based
NDC category assignment results are presented in Table III,
where tl, cg, ..., tl+cg+bd respectively represent channel
combinations (1)–(7) mentioned in subsection III.B. It can beseen from the table that, for example, when 100 filters and only
the titles of the Wikipedia articles (i.e., tl) are used, the NDC
category assignment precision is 49.0%. The use of article titles,
categories, and main texts with 250 filters produces the highest
precision of 87.4%. It is also obvious that the use of more filters

㻯
㻺
㻝㻜
㻜
㻝㻝
㻣㻟
㻝㻞 㻝㻘㻥㻠㻡㻝㻟 㻞㻘㻡㻥㻠㻝㻠 㻟㻘㻡㻥㻥㻝㻡㻜㻝㻢 㻟㻘㻣㻤㻣㻝㻣 㻠㻘㻠㻥㻠㻝㻤 㻥㻘㻡㻣㻤㻝㻥 㻣㻘㻝㻥㻣㼀 㻟㻟㻘㻞㻢㻣NUMBER OF WIKIPEDIA ARTICLES WITH RESPECT TO
ASSIGNED NDC CATEGORY
㻯
㻺
㻞㻜
㻝㻤㻝
㻞㻝 㻢㻤㻘㻜㻢㻢㻞㻞 㻞㻣㻘㻡㻤㻠㻞㻟 㻠㻝㻘㻜㻣㻜
㻞㻠
㻤㻜㻤
㻞㻡
㻥㻜㻞
㻞㻢
㻜
㻞㻣
㻜
㻞㻤 㻠㻘㻠㻝㻜㻞㻥 㻥㻞㻘㻝㻤㻡㼀 㻞㻟㻡㻘㻞㻜㻢㻯㻺㻟㻜 㻟㻘㻟㻣㻟㻟㻝 㻝㻞㻘㻣㻜㻞㻟㻞 㻥㻘㻠㻠㻣㻟㻟 㻝㻝㻘㻞㻢㻟㻟㻠㻜㻟㻡㻜㻟㻢 㻝㻜㻘㻢㻠㻞㻟㻣 㻟㻠㻘㻟㻟㻝㻟㻤 㻠㻘㻤㻤㻜㻟㻥 㻟㻞㻘㻤㻤㻠㼀 㻝㻝㻥㻘㻡㻞㻞

㻯
㻺
㻠㻜
㻥㻢㻟
㻠㻝 㻣㻘㻟㻟㻤㻠㻞 㻠㻘㻟㻜㻤㻠㻟 㻠㻘㻡㻣㻜㻠㻠 㻤㻘㻝㻞㻟㻠㻡 㻣㻘㻟㻡㻞㻠㻢 㻝㻘㻤㻠㻝㻠㻣 㻡㻘㻠㻥㻟㻠㻤 㻝㻜㻘㻣㻡㻞㻠㻥 㻝㻥㻘㻢㻥㻞㼀 㻣㻜㻘㻠㻟㻞

㻯
㻡㻜
㻡㻝
㻡㻞
㻡㻟
㻡㻠
㻡㻡
㻡㻢
㻡㻣
㻡㻤
㻡㻥
㼀

㻺
㻝㻘㻡㻞㻞
㻞㻘㻝㻞㻜
㻥㻘㻟㻣㻥
㻝㻡㻘㻢㻣㻝
㻝㻟㻘㻣㻜㻠
㻝㻟㻘㻣㻟㻢
㻞㻝㻜
㻤㻟
㻥㻢㻟
㻡㻘㻢㻢㻤
㻢㻟㻘㻜㻡㻢㻯㻺㻢㻜㻜㻢㻝 㻞㻟㻘㻣㻣㻠㻢㻞㻜㻢㻟㻜㻢㻠 㻠㻘㻡㻜㻢
㻢㻡
㻞㻝㻞
㻢㻢
㻟
㻢㻣 㻡㻘㻠㻥㻜㻢㻤 㻠㻥㻘㻝㻣㻝㻢㻥 㻞㻞㻘㻜㻠㻥㼀 㻝㻜㻡㻘㻞㻜㻡㻯㻺㻣㻜 㻞㻘㻣㻢㻟㻣㻝㻜㻣㻞 㻞㻥㻘㻥㻤㻠㻣㻟㻜㻣㻠 㻞㻘㻡㻤㻥㻣㻡 㻝㻘㻤㻜㻢㻣㻢 㻥㻡㻘㻥㻣㻝㻣㻣 㻝㻜㻝㻘㻥㻡㻣㻣㻤 㻝㻞㻣㻘㻤㻢㻜㻣㻥 㻞㻞㻘㻤㻢㻡㼀 㻟㻤㻡㻘㻣㻥㻡

㻯
㻤㻜
㻤㻝
㻤㻞
㻤㻟
㻤㻠
㻤㻡
㻤㻢
㻤㻣
㻤㻤
㻤㻥
㼀

㻺
㻝㻘㻞㻞㻥
㻠㻘㻢㻝㻠
㻜
㻝㻘㻝㻡㻣
㻜
㻜
㻜
㻜
㻜
㻜
㻣㻘㻜㻜㻜

㻯
㻺
㻥㻜 㻝㻘㻞㻥㻞㻥㻝 㻞㻟㻘㻡㻟㻥㻥㻞 㻝㻘㻠㻟㻜㻥㻟 㻠㻘㻜㻜㻣㻥㻠 㻝㻘㻥㻤㻤
㻥㻡
㻜
㻥㻢
㻜
㻥㻣
㻜
㻥㻤
㻜
㻥㻥
㻜
㼀 㻟㻞㻘㻞㻡㻢

B. Selecting Books to Recommend
The precisions of the CNN and SVM for classifying books
to recommend or not recommend are presented in Tables VI andVII. A comparison of the tables reveals that the SVM produces
better results, with its precision reaching 99.8%. The SVM wasthus used to select books for recommendation to the Wikipedia
articles readers. The three books with the highest probabilities

125

extension. It would be interesting to further implement thesystem as an extension of other Web browsers such as Microsoft
Edge and Mozilla Firefox, and investigate whether the books
recommended to the Wikipedia article readers are actuallyborrowed from the library.

of belonging to the group of books to be recommended for eachWikipedia article were selected and recommended to thesubjects of the test implementation of the propose system.
TABLE VI.

PRECISION OF BOOK CLASSIFICATION USING CNN
㻹
㻝㻜
㻞㻜
㻟㻜

TABLE VII.

㻡㻜
㻥㻤㻚㻟
㻥㻤㻚㻠
㻥㻤㻚㻢

㻺㼡㼙㼎㼑㼞㻌㼛㼒㻌㻲㼕㼘㼠㼑㼞㼟
㻝㻜㻜
㻞㻜㻜
㻥㻤㻚㻠
㻥㻤㻚㻟
㻥㻤㻚㻣
㻥㻤㻚㻢
㻥㻤㻚㻢
㻥㻤㻚㻢

㻠㻜㻜
㻥㻤㻚㻠
㻥㻤㻚㻡
㻥㻤㻚㻡

REFERENCESPRECISION OF BOOK CLASSIFICATION USING SVM
㻹 㻼㼞㼑㼏㼕㼟㼕㼛㼚
㻝㻜
㻥㻥㻚㻤
㻞㻜
㻥㻥㻚㻤
㻟㻜
㻥㻥㻚㻣

㻯
㻡㻝㻞
㻟㻞
㻟㻞

γ
㻜㻚㻜㻟㻝㻞㻡
㻜㻚㻝㻞㻡㻜㻜
㻜㻚㻝㻞㻡㻜㻜

V. DISCUSSION
This section considers some potential error sources in thepresent work and how such may be addressed for further study.
Among the 500 test data that were used for NDC category
assignment, 63 Wikipedia articles were wrongly assigned NDC
categories by the CNN. Among these 63, articles with proper
nouns in their titles were predominant, with 12 containing thenames of persons and six the names of organizations. Threearticles with the names of historic Japanese personalities in theirtitles were assigned NDC category 21 (History of Japan),although the “correct” category was 28 (Biography). In thisstudy, the correct NDC category was determined based on thebooks cited in the Wikipedia articles. These persons werementioned in articles that cited their biographies. However,considering that the personalities are actually of historicsignificance, the above-mentioned assignment of History ofJapan to the relevant articles is not entirely a failure. Other“incorrect” categorizations were observed for articles thatbelonged to multiple categories. For example, while the “correct”
NDC category of an article on a historic Chinese calligrapher
and philosopher is 72 (Painting and Calligraphy), the proposedsystem assigned category 12 (Oriental Philosophy) to it. Thisapparent failure may be difficult to correct, and it may not evenbe necessary to attempt a correction for the purpose of book
recommendation because recommending a book for a specificaspect of the article may be acceptable.
The book selection precision of the SVM reached 99.8% andfew errors were observed here.

VI. CONCLUSIONSTo encourage students to read library books as a morereliable source of information, a system was developed forrecommending library books to Wikipedia article readers inuniversity libraries. The system assigns NDC categories to
Wikipedia articles and recommends library books in the same
categories as the respective articles. The CNN-based NDC
category assignment precision of the system for Wikipedia
articles was determined to be as high as 87.4%, while the SVMbased book selection precision reached 99.8%. In the present
study, the system was implemented as a Google Chrome

126[1] Anbiru, T. et al. (2010) Information seeking behavior. Proceedings of theSpring Meeting of the Japan Society of Library and Information Science,
87-90. (Text in Japanese.)[2] Japan Library Association (2018) Statistics on library in Japan. Japan
Library Association. 515p.
[3] Kim, Y. (2014) Convolutional neural networks for sentence classification.
Proceedings of the 2014 Conference on Empirical Methods in Natural
Language Processing, 1746-1751.
[4] Arai, S. and Tsuji, K. (2015) Automatically assigning NDC categories toreference service records by using machine learning methods. J Jpn Soc
Inf Knowl 25(1):23-40. (Text in Japanese.)[5] Johnson, R. and Zhang, T. (2015) Effective use of word order for text
categorization with convolutional neural networks. Proceedings of
Human Language Technologies: The 2015 Annual Conference of theNorth American Chapter of the ACL, 103-112.
[6] Wang, P. et al. (2015) Semantic clustering and convolutional neural network
for short text categorization. Proceedings of the 53rd Annual Meeting of
the Association for Computational Linguistics and the 7th International
Joint Conference on Natural Language Processing, 352-357.
[7] Mikawa, M. et al. (2011) Book recommendation signage system using
silhouette-based gait classification. Proceedings or the 10th International
Conference on Machine Learning and Applications, 416-419.
[8] Jomsri, P. (2018) FUCL Mining technique for book recommender system
in library service. Proceedings of the 11th International Conference
Interdisciplinarity in Engineering, 550-557.
[9] Tsuji, K. (2016) Books cited in Wikipedia: possibility to use their Nippon
decimal classification categories for book recommendation. Proceedingsof the 7th International Conference on E-Service and Knowledge
Management, 1196-1197.
[10] Tsuji, K. (2017) Automatic classification of Wikipedia articles by using
convolutional neural network. Proceedings of the 9th Qualitative andQuantitative Methods in Libraries International Conference, 8p. (NoPagination).
[11] Mecab. http://taku910.github.io/mecab/ [Last Access: 2019-01-14][12]
mecab-ipadic-NEologd:
neologism
dictionary for
MeCab.
https://github.com/neologd/mecab-ipadic-neologd [Last Access: 201901-14][13] Word2vec. https://radimrehurek.com/gensim/models/word2vec.html[Last Access: 2019-01-14][14] gensim: topic modelling for humans. https://radimrehurek.com/gensim/[Last Access: 2019-01-14][15] OpenSearch by the National Diet Library of Japan.
http://iss.ndl.go.jp/information/api/ [Last Access: 2019-01-14][16] Tensorflow. https://www.tensorflow.org/ [Last Access: 2019-01-14][17]
LIBSVM:
libraryforsupportvectormachines.
https://www.csie.ntu.edu.tw/~cjlin/libsvm/ [Last Access: 2019-01-14]