DEIM Forum 2019 B8-1

A Lossless Irregular Tensor Factorization for Cross-domain Recommender
System with Genetic Algorithm on Spark
Guodong XUE†

Seiki MIYAMOTO††

Takumi ZAMAMI††† and†Graduate School of Fundamental Science and Engineering, Waseda University

Hayato YAMANA††††

3-4-1 Okubo, Shinjuku-ku, Tokyo,169-8555 Japan†† Faculty of Science and Engineering, Waseda University††† Ceo’s office of DMM.com

3-2-1 Roppongi, Minato-ku, Tokyo, Japan†††† Faculty of Science and Engineering, Waseda University
E-mail:3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555 Japan

3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555 Japan†xueguodong@fuji.waseda.jp, †† smiyamoto14@yama.info.waseda.ac.jp,†††zamami-takumi@dmm.com †††† yamana@waseda.jp

Abstract Users may have sufficient experience in their focused domains but lack experience in unfamiliar domains,
leading to a drop in the accuracies of recommender systems owing to the unavailability of data with which to train suchdomains. Therefore, the cross-domain recommender system, which analyzes user information from both the user’s familiar andunfamiliar domains, has become an important emerging subject of research. Weighted irregular tensor factorization (WITF) has
been proposed as an algorithm to leverage cross-domain feedback across all users and considers the dataset as an irregulartensor containing different items for each domain. According to tensor theory, an irregular tensor must be transferred into aregular tensor (in which each domain contains the same items), before executing tensor factorization to obtain the user's latent
factors. Therefore, WITF must assign a weight on each domain to ensure minimal data loss during the transformation.
Searching for the optimal domain weight configuration is a complex and time-consuming procedure. Thus, WITF adopts anempirical domain weight configuration. In this study, we propose a model that combines Apache Spark® and a genetic
algorithm to search efficiently for the optimal domain weight configurations for WITF, and then uses the optimal configuration
obtained to generate more precise users’ latent factors to make recommendations. Experimental evaluation using two datasetsof different sizes show 4.2% and 6.3% higher accuracies in comparison with the WITF alone

1. Intorduction

the cross-domain recommender systems, which leverageWith the development of the Internet, the number andcross-domain feedback data across all users to learn thevariety of content on the World Wide Web continue touser's latent factors, have become an important emerging

increase, and user demand is higher for searches andresearch topic.

recommendations. Recommender systems, which make

Weighted irregular tensor factorization (WITF) [2] is aaccurate and personalized recommendations using users’

model

that

considerscross-domain

informationbypreferences, interests, and experiences, have become moreconsidering the multiple domains dataset as an irregularand more important and are widely used in many fields.

tensor. In this model, a weight must be assigned to each

Collaborative filtering (CF) [1] is the algorithm mostdomain to ensure minimal data loss after transferring into

utilized in recommender systems, and it has been used ina regular tensor, followed by processing the tensor

manyfactorization to make recommendations. However, theapplications,

includingAmazonand

Netflix.

However, the data sparsity problem (many users do not

WITF

have data in the CF recommender systems) always affects

configuration that decides the weight based on the number

the performance of CF recommender systems. In general,of items in each domain. Therefore, a method for selecting

users always have sufficient experience in their focused

the optimal weight configuration for the WITF model isdomains. In some researches, the recommender system

necessary to improve its performance.

uses data from other domains to supply more data, andthus generates more accurate recommendations. Therefore,

model

adoptsanempirical

domainweightGenetic algorithms [3] are commonly used to generate
high-quality

solutionsfor

optimization

andsearchproblems. Furthermore, the WITF model and genetic

algorithm are suitable for parallel processing. For this,factorization has two models: Candecomp/Parafac (CP) [7]

Apache Spark [4] is a powerful distributed computing

decomposition and Tucker [7] decomposition. As shown inframework that has become an important part of the big

Figure 1, CP decomposition decomposes a tensor into a

data analytics ecosystem.

sum of component vectors, e.g., 𝑼𝒊,: , 𝑪𝒊,: , and 𝑽𝒊,: , of theThis study focuses on combining the WITF model withthree latent factor matrices U, V, and C. Compared to CP

a genetic algorithm on Spark to search for the optimal

decomposition,weighttensor into one core tensor 𝒈 and three latent factor

configurationsformakingmore

accurate

recommendations and reducing the execution time.

Tuckerdecomposition

decomposesamatrices, e.g., U, V, and C.

2. Background
2.1 Notations
Table 1 describes notations referred to in this study.

𝑿
𝑿𝒌
𝑿𝒌,𝒊,:
𝑿𝒌,:,𝒋
𝑿𝒊,:
𝑿:,𝒋
𝑼
𝑽
𝑪
𝑰
{ω, }
𝑿 1

Table 1. Notations
Description
A tensor
A matrix (user-item rating matrix)
The matrix of domain
A row (vector) of domain matrix 𝑿𝒌
A column (vector) of domain matrix 𝑿𝒌
A row (vector) of matrix X
A column (vector) of matrix X
User latent factor matrix
Item latent factor matrix
Domain latent factor matrix
Identity matrix
A set of domain weights, 1 ≤ k ≤ 𝑲
Frobenius norm of matrix X⊛

Hadamard product

Notationx

2.1 Cross-domain Recommender System
In the real-world environment, users have sufficientexperience in their focused domains but lack experience inother domains. When information from users’ familiar
domains is used as auxiliary data, recommender systems
perform better at recommending potentially desirable
items in domains unfamiliar to users. Thus, cross-domain
recommender systems have become an important emerging
subject of research. Cross-domain collaborative filtering(CDCF) [5] is an important subject of research that
focuses on product domains. Product domains, time
domains, spatial domains, and other domains are also
adopted in research related to CDCF. Because matrix
factorization (MF) is a widely used model of CF
recommender systems, cross-domain matrix factorization

Figure 1.

Two models of tensor factorization

To adopt the two models, the tensor must be a regulartensor with each of its domains containing the same items.
Unfortunately, almost all real-world datasets cannot be
considered as regular tensors with the same items in eachdomain. These datasets can be considered as irregulartensors, each of whose domains contain specific items. As
shown in Figure 2, it is necessary to transfer the irregulartensor into the regular tensor that has the same set ofvirtual items. However, during the transfer, data loss isinevitable [2].
Therefore, it must be ensured that the data loss is minimal
because the origin data from the irregular tensor is to be
utilized maximally, and the most accurate latent factors
are then obtained by processing the factorization for the
regular tensor that has been transferred from the irregulartensor.
We define this kind of irregular tensor factorization as
lossless irregular tensor factorization, where lossless
represents the minimal data loss while transferring anirregular tensor to a regular tensor.

(CDMF) [6] has been proposed as an improvement
technique for the CDCF method.
2.2 Irregular Tensor Factorization
In CF recommender systems, a dyadic user–item
relationship is considered a core relationship. To use
additional data, such as tags and time, some studies adopt
tensor factorization (TF) [7] to process the triadic
relationships, e.g., user–item–tag, user–item–time. Tensor

Figure 2.

Irregular Tensor Factorization

2.3 Genetic Algorithm

factorization to obtain the user's latent factors. According

Genetic algorithms are commonly used to generate
high-quality

solutionsfor

optimization

andsearchproblems by relying on bio-inspired genetic operations
such

as

mutation,crossover,andselection.

to tensor theory, the irregular tensor must be transferred
into a regular tensor, in which each domain contains thesame items, to execute the tensor factorization.

These

Due to the inevitable data loss in irregular tensor

bio-inspired operators can be processed in parallel totransformation, the WITF model utilizes a set of domain

speed up the operation. To improve the algorithm,

weights to reduce the data loss as shown in Figure 3.

researchers have refined each bio-inspired operator. For
example, roulette wheel selection [3] and tournament
selection [8] have been proposed to improve the selection
operator. Binary mutation and Gaussian mutation [9] havebeen proposed to improve the mutation operator. Decades
after the genetic algorithm was proposed, it is still widelyused.
2.4 Apache Spark
Although

MapReduce

has

been

successfully

implemented in large-scale data-intensive applications oncommodity clusters, some other applications are notsuitable for MapReduce. These applications contain many

Figure 3.

Domain Weight Configuration

iterative machine learning algorithms that reuse a working

The WITF model provides an objective function, based

set of data across multiple parallel operations. Thus, theon CP decomposition, to describe the data loss. It is

new Spark framework distributed by Apache, to support

presented as formula 1 [2], where 𝜔7 is the domain

these applications. Spark contains two main new features,

weight for each domain 𝑿𝒌 , 𝐖𝒌 is the weight for each

which

entry

retainthescalability

and

fault

toleranceofindomain

𝑿𝒌 ,

𝚺𝒌

is

the

diagonal

matrix

MapReduce. They are resilient distributed datasets (RDDs)

constructed based on the latent factor vector 𝑪𝒌 that[4] that save data on memory and various parallel

belongs to domain matrix C, and 𝑷𝒌 is the additional

operations.

constraint [11] for obtaining a unique value of the

The architecture of a Spark cluster contains threecomponents: Spark driver program, cluster manager, andSpark workers. The Spark driver program converts the

Frobenius norm ( …
𝐽 = 𝑎𝑟𝑔𝑚𝑖𝑛
𝑼,𝑽,𝑪

1
2

1 ) in the objective function.

E

𝜔7 𝑾7 ⊛ (𝑿7 − 𝑼𝚺7 (𝑷7 𝑽)L ) M1

𝑠. 𝑡. 𝑃7L 𝑃7 = 𝑰

(1)

7FG

user ’s applications into tasks and schedules, then sends

The WITF model adopts a specific set of domain

these tasks to Spark workers. The cluster manager

weights {ω, }, then iterates while updating U, V, C, andmanages the resources of whole clusters as a Spark plugin,𝑷𝒌 in each iteration until the objective function converges.

and Spark can run different external cluster managers. A

The convergence value of the objective function can be

Spark worker contains several executors, which haveconsidered as the minimal data loss of the irregular tensor

multiple cores ( ≥ 1 ), to execute tasks and return thefactorization using a specific set of domain weights {ω, }.

results to the Spark driver program.

Furthermore, the WITF model also proves that the optimal

3. Related Workdomain weight configuration can minimize the data loss.

3.1 Weighted Irregular Tensor Factorization (WITF)

Searching for the optimal domain weight configuration

tensoris a complex and time-consuming procedure. Therefore,

factorization (WITF) model [2] to leverage cross-domain

the WITF model adopts an empirical domain weight

feedback data across all users to learn users' latent factors

configuration that assigns the domain weight based on thethat are more accurate than users' latent factors only in aratiosparticular domain. Epinions dataset [10], which is used toperformance of WITF can be optimized by searching forevaluate WITF, contains multiple domains, with eachthe optimal weight configuration on each domain, adomain containing different items. WITF considers themethod to search for the optimal weight configuration fordataset as an irregular tensor in which each domain

the WITF model is necessary.

Huetcontains

al.

proposeddifferent

items

a

weighted

and

then

irregular

executes

tensoroftheratingsineachdomain.

Since

the

4. Proposed Method

C is the latent factor matrix for domains

4.1 Overview

V, 𝑷𝒌 are the latent factor matrices for items

We propose a model that combines Apache Spark and a
genetic algorithm to search efficiently for the optimal
domain weight configurations for the WITF model; then,we use the optimal configuration obtained to generate

Begin
Initialization:
1: Randomly Initialize U,Cmore precise users’ latent factors with which to make

2: V = I

recommendations. In this respect, Spark is an efficient and3: 𝑷𝒌 = 𝑨𝑩𝑻 , with the SVD: 𝑿𝑻𝒌 𝑼𝚺𝒌 𝑽𝑻 ≈ 𝑨𝚺𝑩𝑻

powerful large-scale data processing engine. A genetic

Iteration:

algorithm is easily parallelized and widely used to searchfor solutions to complex problems. It is suitable forcomplex computing problems such as searching for the
optimal domain weight configuration for the WITF model.
Furthermore,

the

WITF

model

is

also

suitablefor

4: Generate tensor with each 𝑿𝒌
5: Update 𝑼𝒊 of each user i in parallel by WITF theory
6: Update 𝑪𝒌 of each domain k in parallel by WITF theory
7: Update V as a whole by WITF theory

parallelization. Therefore, parallelization of the genetic

8: Update 𝑷𝒌 of each domain k in parallel by WITF theory

algorithm and the WITF can speed up computation when

Repeat 4-8 until the objective function convergence

using a genetic algorithm to search for the optimal domain

9: Return U,V,C, {𝑷𝒌 }weight configurations for the WITF model.

End

4.2 Architecture
Although the genetic algorithm is easy to parallelize
and widely used to search for the solutions to a complex
problem, a genetic algorithm is not suitable for problems

The

secondphase

parallelization

is

for

genetic

algorithm operators. We use the root mean square error

with a complex fitness evaluation. However, the proposed(RMSE) [12] metric, which is calculated using themethod utilizes the WITF model as a fitness evaluation.

returned U,V,C, {𝑷𝒌 } from the WITF model as a fitness

Therefore, we propose a two-phase Spark parallelization

value for each individual ({ω, }). The RMSE is defined as:

scheme to process the WITF model and genetic algorithm
efficiently.

𝑅𝑀𝑆𝐸 =The Spark RDDs are one of the core components ofb (] _] )a
`
^cd ^

e

(2)

Spark. The RDDs store the data in memory to reduce thewhere 𝑌g is the user ’s real rating for item i, and 𝑌h is thetime for data input/output (I/O) and have various efficient

predicted rating of item i for that user. We store themethods available to users. Therefore, Spark users always

individuals in RDDs and process these operators inseparate large data into partitions and store them in RDDs,

parallel on Spark. This phase is presented as Algorithm 2.

then process these data on Spark in parallel.
For our proposed method, WITF and genetic algorithm
operators, such as crossover and mutation, are thetime-consuming procedures. To speed up these operations,
the

WITF

model

is

processed

as

the

first

Algorithm 2: Parallel Genetic Algorithm with WITF

Input: {ω, } is the initial set of domain weights
𝑿𝒌 is the data matrix for each domain

phase

𝑮𝒎𝒂𝒙 is the maximum number of generations

parallelization. In the first phase, the data of the WITF,
e.g., 𝑿𝒌 , U, V, and C, are stored in RDDs and computed in

Output: {𝝍𝒌 }is the optimal set of domain weights

parallel inside the WITF model. This phase is presented as

Begin

Algorithm 1.
Algorithm 1: Parallelization of WITF Model [2][U,V,C,{𝑷𝒌 }] = (function) Parallel_WITF({𝑿𝒌 }, {ω, })Input: {ω, } is set of domain weights
𝑿𝒌 is the data matrix for each domain
Output: U is the latent factor matrix for users

Initial Population:
1:

Make initial population by {ω, }Generation Iteration of Genetic Algorithm:
2:for each individual {ω, } in population do

3:[U,C,V, 𝑷𝒌 ] = Parallel_WITF({𝑿𝒌 }, {ω, })4:

Calculate RMSE by [U,C,V, 𝑷𝒌 ]5:

Save the RMSE as the fitness value of {ω, }5.2 Genetic Algorithm Encoding and Operators

6:

end for7:

Selection of the population in parallel

The individuals (chromosomes) of the initial population

8:

Crossover of the population in parallelin a genetic algorithm are a set of domain weights {ω, }.

9:Mutation of the population in parallel

Each weight ω, is considered as a gene inside an
individual (chromosome). This initial encoding method is

Repeat 2-9 for 𝑮𝒎𝒂𝒙 times:

called a float-encoding genetic algorithm [13].

10:{𝝍𝒌 } is the {ω, } that obtains minimal RMSE

11:

Return {𝝍𝒌 }A genetic algorithm has three operators: selection,
crossover, and mutation. For the selection operator,
tournament selection [8] was adopted. It executes several

End

tournaments among a few individuals chosen at random
from the population, and the individual with the best

5. Experimental Evaluation

fitness value becomes the winner to be selected for

5.1 Datasets and Environment
The Epinions dataset [10] is extracted from the Epinions
website, which is an e-commerce website with multiple
domains of products. The users can provide reviews andratings (which are integers from 1 to 5) to products ofdifferent domains, and products on each domain are
unique to that domain. Five domains from the Epinions
dataset [10] that have the most ratings were selected, andthe data were pre-processed to discard users with less than
five and ten ratings on the five selected domains. In thesmall dataset, first, the users who have at least ten ratingsin each selected domain are retained, and then the items
that have been rated by at least ten of the retained users,
are retained. Similarly, in the large dataset, users who
have at least five ratings in each selected domain are
retained, and then the items that have been rated by atleast five of the retained users are retained. The twodatasets are presented in Table 2.
Table 2.

Sizewhen processing the crossover operator for which wholearithmetic recombination [14] was adopted. It is the mostcommonly used crossover operator for the float-encoding
genetic algorithm and works by taking the weighted sumof the two parental alleles for each gene in the children.
After

processing

thecrossover

operator,

Gaussianmutation [9] was adopted. It adds a random value from a
Gaussian distribution to a randomly selected gene of anindividual to create a new offspring.
5.3 Efficiency Evaluation
5.3.1 Parallel WITF model on Spark
A Spark cluster contains many computing resources thatare considered as workers. To manage these workers in thecluster, Spark provides a driver program to dispatch tasks,
which contain data and instructions, and to receive thetask results when workers finish their assigned tasks. An
efficient program on Spark should execute its codes on theDataset Statistic

Smallcrossover. A pair of individuals will cross their genes

Spark workers for as long as possible.

Large#Users#Items#Ratings#Users#Items#Ratings

Among the procedures of the proposed method, the most

Domain 1

2,403

1,104

14,960

6,682

1,771

27,786

Domain 2

2,403

320

7,350

6,682

702

13,802time-consuming procedures are the fitness evaluation,

Domain 3

2,403

1,362

30,490

6,682

2,318

47,972

Domain 4

2,403

1,050

10,223

6,682

2,459

22,448domain weight configuration of each individual. As

Domain 5

2,403

761

8,733

6,682

1,786

18,925

discussed in section 3.1, the WITF model executes

which computes the RMSE of the WITF model using theiterations to update its parameters, e.g., U, C, V, and 𝑷𝒌 ,The experiment was processed on four servers, and theuntil reaching convergence. Therefore, the efficiency ofenvironment of each server is presented in Table 3. In theeach iteration of the WITF model is important for theexperimental Spark cluster, one server was configured as

overallthe Spark master server to process the Spark driver

implementing the WITF model on Spark using the Sparkprogram and Spark cluster manager. The other three

Python API and web user interface to monitor the program,servers were configured as Spark workers (also called

the timeline of an implemented iteration of the WITF

Spark slave servers) to process the Spark tasks.

model can be analyzed. The timeline of a WITF iteration

OS
CentOS
7.5

Table 3. Experiment Environment
CPU#CPU(Core)Intel Xeon CPUE5-2620
v4 @ 2.10GHz

2 Physical16 Logicalefficiency

of

the

proposedusing the small dataset is shown in Figure 4.
Memory128GBmethod.

By

Figure 4.

Timeline of a WITF iteration on SparkThe gray bars in Figure 4 represent the period of Spark

Figure 5.

Execution Time of a WITF Iteration

workers executing the tasks. The white spaces in the

Because the experiment results from setting the Sparkfigure represent the period during which the Spark driver

executors to only one core show that the executor ’s

program is executing, and the workers are idle. The timecomputing power is limited, another series of experiments

ratio between the workers executing time and a WITF

on the two datasets were processed with the Sparkiteration executing time is 67.9%.

executors set to multiple cores (48 cores in total). The

5.3.2 Efficiency of the WITF model on a Spark cluster

execution time results of a WITF iteration for the two

A series of experiments were processed using two

datasets are presented in Figure 6. The execution time ofdatasets of different sizes to evaluate the implementation

the WITF iteration decreased with the increase in theof the WITF model on a Spark cluster, which contains one

number of cores in each executor, and the decrease in themaster server and three slave servers (16 cores in eachnumber of executors. The execution time tends to be stable

server, 48 cores in total). Experiments with different

when each executor has more than eight cores (number of

numbers of Spark executors with one core were processed

executors is less than six, with 48 cores in total). When

first. Then experiments with different numbers of Sparkusing the large dataset, the best execution time, which was

executors with multiple cores were processed, and all 48

obtained with three executors (16 cores in each), is 59.96%

cores were used.

faster than the worst execution time obtained with 48

First, three group experiments for the two datasets ofexecutors (one core in each). When using the small dataset,different sizes were processed by setting 16, 32, and 48

the best execution time, which was obtained with three

executors with one core. The results of the execution time

executors (16 cores in each), is 40.04% faster than the(in minutes) of the WITF iteration is shown in Figure 5.

worst execution time obtained with 48 executors (one core

According to the results, the execution time of the WITF

in each).

iteration shows a slight increase with an increase in thenumber of Spark executors used while processing either
the small or large dataset. The reason the execution timeincreased when more executors (cores) were utilized isthat these executors contain only one core and thecomputing

power

of

each

executorwaslimited.

Furthermore, communication between the Spark driverprogram and the Spark executors, e.g., data broadcast andtask results return, was increased by utilizing more
executors. Thus, the execution time of the WITF iteration
increased,

althoughmore

computingresources

were

utilized.
Figure 6.

Execution Time of a WITF Iteration withdifferent multiple-core Spark executors

Regarding the ratio between the executing time perTable 4.

Genetic Algorithm Execution Time

executor and the entire execution time of a WITF iteration,(a).

the results for the two datasets are shown in Figure 7. Like

Procedure

the results in Figure 6, the execution time ratios also

1 Generation
1 Individual

decreased with an increase in the number of cores in each(b).

executor and a decrease in the number of the executors. It
also tends to be stable when the number of cores in each
executor is more than eight (and the number of executors
is less than six, with 48 cores in total).

Small Dataset

Average Execution time (min.)172.62
10.63

Procedure

Large Dataset

Average Execution time (min.)

1 Generation
382.93
1 Individual
23.93
5.5 Genetic Algorithm Generation Evaluation
The mean fitness (RMSE) value of all 16 individuals ineach generation was calculated, and the minimum fitness(RMSE) value was recorded as the best RMSE of all 16
individuals in each generation. Furthermore, the RMSE
value was calculated using the WITF model’s empirical
domainweightconfiguration,

which

determinesthedomain weights by adopting the number of ratings in eachdomain as the baseline.

Figure 7.

Execution Time Ratios of a WITF Iteration

with different multiple-core Spark executors
The results in Figure 6 and 7 show that more cores ineach Spark executor increases the computing power of
each Spark executor and reduces the execution time for thetasks processed on each Spark executor. Furthermore,
reducingthenumberofexecutors

decreases

communication between the Spark driver program and the

Figure 8.

Genetic Algorithm Generations Evaluation

Spark executors, consequently decreasing the execution

using the small dataset

time of the Spark driver program. The execution time ratio,

For the small dataset, the comparison of those threewhich is between the time of the Spark executors and thevalues in each generation is presented in Figure 8. The red

total execution time, also decreases. This means that theline

decrease in the Spark driver program execution time isrepresents the mean fitness (RMSE) value of eachmuch less than the decrease in the Spark executors

generation, and the green polyline represents the best

execution time while setting more cores in each executor

fitness (RMSE) value of each generation. Figure 8 shows

and deploying fewer executors, as in our experiments. In

that the mean fitness and best fitness value of eachother words, the implementation of the WITF model would

generation tend to be stable and better than the initialhave a more efficient performance in a Spark cluster withindividual’s mean and best fitness values as the generation

represents

thebaseline.

The

goldenpolyline

executors that have high computing power.

grows. After the 22 nd generation, the best fitness values

5.4 Efficiency of the Proposed Method on Sparkare better than the baseline and tend to be stable after the

The population of the genetic algorithm was initialized

41 st generation. Before the 34 th generation, the mean

with 16 individuals and three executors set to 16 cores infitness values are worse than the baseline. Then after theeach and 48 in total. An implementation of the proposed

39 th generation, the mean fitness values are better than themethod was then executed for 50 generations using thebaseline and approach to the best fitness RMSE. Thistwo datasets. Based on the execution time, the execution

demonstrates

time statistics shown in Table 4 were obtained.

algorithm reaches convergence in the proposed method.

that

the

searchresultofthegenetic

Furthermore, the best RMSE using the small dataset

RMSE improvement. For the large dataset, the proposed

among all generations is improved by 4.2% compared withmethod searched for a domain weight configuration with athe baseline.

6.3% RMSE improvement. The efficiency evaluation
results show that parallelization implementation of the
WITF model has a more efficient performance in a Sparkcluster with executors that have high computing power.

References

Figure 9.

Genetic Algorithm Generations Evaluation
using the large dataset

For the large dataset, the result, which is shown in
Figure 9, is similar to the results of the small dataset.
Because the large dataset is sparser than the small dataset,the baseline, mean fitness RMSE, and best fitness RMSE
are worse than the fitness value for the small dataset, andthe mean fitness RMSE is worse than the baseline among
all the 50 generations. However, like the result of thesmall dataset, the mean fitness RMSE would approach tothe best fitness RMSE after enough generations. After the
14 th generation, the best fitness values are better than thebaseline, and then tend to be stable after the 38 th
generation. Furthermore, the best RMSE using the large
dataset among all generations is improved by 6.3%
compared with the baseline.
The results show that the proposed method can searchand obtain the optimal domain weight configuration for
WITF to make recommendations that have the best RMSE.

6. Conclusion
In this study, we proposed a method that combines the
WITF model and a genetic algorithm to search for the
optimal domain weight configuration of a WITF model for
more accurate recommendations. To make the computation
efficiently, we adopted parallelization in both the WITF
model and the genetic algorithm, then implemented andevaluated the proposed method on a Spark platform.
We evaluated two different sizes of datasets on a Sparkcluster containing 48 cores and analyzed the experimental
results for each dataset. The results of the evaluation showthat the proposed method and its implementation cansearch for the optimal domain weight configuration for the
WITF model. For the small dataset, the proposed method
searched for a domain weight configuration with a 4.2%

[1] P. Resnick, N. Iacovou, M.Suchak, P.Bergstrom, andJ.Riedl, GroupLens: An Open Architecture forCollaborative Filtering of Netnews, Proceedings of ACM
Conference on Computer Supported Cooperative Work(CSCW1994), pp.175-186, 1994.
[2] L. Hu, L. Cao, J. Cao, Z. Gu, G. Xu, and D. Yang.
Learning informative priors from heterogeneous domains
to improve recommendation in cold-start user domains,
ACM Trans. Inf. Syst. Vol.35, No.2, Article 13, 2016.
[3] D.E. Goldberg, Genetic Algorithms in Search,
Optimization, and Machine Learning, Addison-Wesley.
1989[4] Zaharia M, Chowdhury M, Das T, Dave A, Ma j,
McCauley M, Franklin M, Shenker S, Stoica I, Resilient
distributed datasets: a fault-tolerant abstraction forin-memory cluster computing, Proceedings of the 9th
USENIX Conference on Networked Systems Design and
Implementation, Berkeley, 2012.
[5] B. Li. Cross-domain collaborative filtering: A brief
survey, Proceedings of the 2011 IEEE 23rd nternational
Conference on Tools with Artificial Intelligence, 2011.
[6] W. Pan, E. W. Xiang, N. N. Liu, and Q. Yang, Transfer
learning in collaborative filtering for sparsity reduction.
Proceedings of the 24th AAAI Conference on Artificial
Intelligence, 2010.
[7] T. G. Kolda and B. W. Bader, Tensor decompositions
and applications, SIAM Rev., Vol.51, No.3, pp.455–500,
2009.
[8] D. E. Goldberg and K. Deb, A comparative analysis of
selection
schemesusedingenetic
algorithms,
Foundations of Genetic Algorithms, Vol.1, pp.69-93,
1991.
[9] T. Back and H.-P. Schwefel, An Overview of
Evolutionary Algorithms for Parameter Optimization,
Evolutionary Computation,Vol.1, No.1, pp.1-23, 1993.
[10] S. Meyffret , E. Guillot , L. Médini, and F. Laforest,
RED: a Rich Epinions Dataset for Recommender Systems,
Dataset for Recommender Systems, 2014.
[11] H. A. L. Kiers, J. M. F. Ten Berge, and R. Bro.
Parafac2—part I. A direct fitting algorithm for theparafac2 model, Journal of Chemometrics, Vol.13,
pp.275-294, 1999.
[12] CJ. Willmott, K. Matsuura, Advantages of the mean
absolute error (MAE) over the root mean square error(RMSE)
inassessingaveragemodel
performance, Climate Research, Vol.30, No.1, pp.79-82,
2005.
[13] C. T. Su and W. T. Tyen, A Genetic Algorithm
Approach Employing Systems,
Proceedings ofInternational Congress on Modeling and Simulation, pp.
1444-1449, 1997.
[14] Z. Michalewicz. Genetic Algorithms + Data
Structures = Evolution Programs. Springer, 3rd edition,
1996.