IEICE TRANS. INF. & SYST., VOL.E101–D, NO.4 APRIL 2018

1096

PAPER

Special Section on Data Engineering and Information Management

Songrium Derivation Factor Analysis: A Web Service for Browsing
Derivation Factors by Modeling N-th Order Derivative Creation∗
Kosetsu TSUKUDA†a) , Keisuke ISHIDA†b) , Masahiro HAMASAKI†c) , Nonmembers,
and Masataka GOTO†d) , Member

SUMMARY
Creating new content based on existing original work is
becoming popular especially among amateur creators. Such new content
is called derivative work and can be transformed into the next new derivative work. Such derivative work creation is called “N-th order derivative
creation.” Although derivative creation is popular, the reason an individual derivative work was created is not observable. To infer the factors thattrigger derivative work creation, we have proposed a model that incorporates three factors: (1) original work’s attractiveness, (2) original work’s
popularity, and (3) derivative work’s popularity. Based on this model, in
this paper, we describe a public web service for browsing derivation factors
called Songrium Derivation Factor Analysis. Our service is implemented
by applying our model to original works and derivative works uploaded to avideo sharing service. Songrium Derivation Factor Analysis provides various visualization functions: Original Works Map, Derivation Tree, Popularity Influence Transition Graph, Creator Distribution Map, and Creator Profile. By displaying such information when users browse and watch
videos, we aim to enable them to find new content and understand the N-thorder derivative creation activity at a deeper level.
key words: user generated content, derivative creation, latent variable
model, web service

1.

Introduction

The era when only professional creators were able to create and provide content on the web has passed; now amateur creators who used to be just consumers can also easily create and provide content. Such content is known as
user generated content (UGC). Since not all amateur creators can create new content from scratch, it is common touse existing original (1st generation) work as the basis for
new content. Such content is called derivative work [1] or
2nd generation work. For example, on YouTube∗∗ , there aremany videos in which amateur creators dance to an existingsong or perform a cover of it [2], [3]. Thingiverse∗∗∗ is a
web service that facilitates derivative work creation, where
amateur creators can share 3D model data intended for a 3D
printer. On Thingiverse, creators can download original 3D
model data created by others, modify it, and upload their
Manuscript received June 27, 2017.
Manuscript revised November 6, 2017.
Manuscript publicized January 18, 2018.
†
The authors are with National Institute of Advanced Industrial
Science and Technology (AIST), Tsukuba-shi, 305–8568 Japan.
∗
This paper is recommended by Technical Committee on Data
Engineering.
a) E-mail: k.tsukuda@aist.go.jpb) E-mail: ksuke-ishida@aist.go.jpc) E-mail: masahiro.hamasaki@aist.go.jpd) E-mail: m.goto@aist.go.jp
DOI: 10.1587/transinf.2017DAT0001

new version [4]. In this kind of derivative work creation activity, a creator influenced by 2nd generation content cancreate 3rd generation content. Similarly, N-th generation
content can be transformed into N+1-th generation content.
Such derivative work creation activity is called “N-th orderderivative creation [5].”We know that derivative creation is popular, but why
are individual derivative works created? There are various
factors that inspire the creation of derivative works. However, since the factors that trigger derivative creation cannot
usually be observed on the web, they are diﬃcult to detect.
To get around this problem, we assume that when a creator creates a derivative work, there are three triggering factors: (1) original work’s attractiveness, (2) original work’s
popularity, and (3) derivative work’s popularity. Based on
this assumption, we have proposed a model to estimate thefactors that trigger derivative work creation [6]. Since therelative influence of the three factors varies among creators,
our model also takes into account the latent relationships between creators and each of the three factors. Moreover, ourmodel uses content ranking information to take into accountthe popularity of the original and derivative works. By referring to the examination model of a web search result [7], [8],we model popularity based on the hypothesis that higher
ranked content has a larger influence because such content
has probably been viewed by many creators. We quantitatively evaluated our proposed model by using music-relatedN-th order derivative creation datasets and showed that the
proposed model considering all of the three factors outperformed the models considering one or two of them [6]. We
also showed that when we considered the content popularity
based on popularity ranking, the method reflecting creators’browsing behavior was the most eﬀective to model derivative creation activity.
In this paper, we describe a public web service,called Songrium Derivation Factor Analysis (http://factor.
songrium.jp), for browsing derivation factors estimated byusing our model. To implement the web service, we applied our model to VOCALOID [9]-related original works
and derivative works uploaded to Niconico∗∗∗∗ , which is oneof the most popular video sharing web services in Japan.
Songrium Derivation Factor Analysis mainly provides the∗∗

http://www.youtube.com
http://www.thingiverse.com∗∗∗∗
http://www.nicovideo.jp∗∗∗

c 2018 The Institute of Electronics, Information and Communication Engineers
Copyright TSUKUDA et al.: SONGRIUM DERIVATION FACTOR ANALYSIS

1097

following visualization functions:• Original Works Map shows the characteristics of theoriginal work in the N-th order derivative creation.
• Derivation Tree shows the N-th order derivative creation process.
• Popularity Influence Transition Graph shows the temporal development regarding the impact of content
popularity that triggered the derivative work creation.
• Creator Distribution Map shows the degree to which acreator is influenced by each of the three factors.
• Creator Profile shows the creator’s influence information.
In terms of consumers, by displaying such information when
they browse and watch videos, we aim to enable them tofind new content and understand the N-th order derivative
creation activity at a deeper level. In terms of creators, suchinformation would be useful to decide which content they
use to create a new derivative work.
Our main contributions in this paper are as follows.
• We applied our proposed model to music-related realworld datasets in three categories: (1) singing: covering an original song, (2) dancing: dancing to an original song, and (3) playing: playing an original songon a musical instrument. They include approximately
160,000, 90,000, and 100,000 derivative works, respectively.
• We implemented a web service Songrium Derivation
Factor Analysis that has five main functions: Original Works Map, Derivation Tree, Popularity Influence
Transition Graph, Creator Distribution Map, and Creator Profile.
• We made Songrium Derivation Factor Analysis open to
the public to support both consumers and creators.
The remainder of this paper is organized as follows.
Section 2 describes related work in two areas: (1) analysis
of derivative creation activity and (2) modeling influencesin social communities. Section 3 describes the model that
adopts the aforementioned three factors. Section 4 presents
the functions of Songrium Derivation Factor Analysis. Finally, Sect. 6 concludes this paper.
2.

Related Work

2.1

Analysis of Derivative Creation Activity

Eto et al. [10] developed a 3D modeling application and amodel sharing web service called Modulobe, which allows
users to create 3D models from scratch or based on the workof other creators. They reported that 10.4% of the models were parents of other models and that the chains of creation reached up to four generations. Cheliotic and Yew [11]examined remixing activities in the ccMixter online music
community† . They reported that derivative creation greatly†

http://ccmixter.org

boosted the output of a community as well as increasing thediversity of the output. Hamasaki et al. [1] analyzed derivative creation activity on Niconico. They used explicit citation information between an original work and its derivative
works and discussed certain statistics (e.g., the number ofderivative works of an original work). Hamasaki et al. [12]also developed a web service called Songrium†† that helps
a user browse original songs and their derivative works byvisualizing their relationships.
All the studies mentioned above analyzed how derivative works had been created by using a network based on
the relationships between the original work and derivative
works. In this work, we focus on why derivative works werecreated and implement a web service for browsing derivation factors.
2.2 Modeling Influences in Social Communities
Since estimating influences among users in social activities is useful for various applications, such as influentialuser detection [13] and personalized recommendation [14],many methods for estimating such influences have been proposed. One major approach is to use an information diffusion model such as the independent cascade model [15].
Although discrete time is assumed with this model, Saito
et al. [16] proposed a model based on Poisson processes, allowing for continuous time modeling. However, their model
requires a network structure of users in which a node corresponds to a user and an edge between users represents theexistence of influence. To overcome this limitation, Iwata etal. [17] proposed a model that discovers latent influences between users without a network structure. Although the cascade Poisson process [18] models a sequence of cascading
events, the model proposed by Iwata et al., which is called
the Shared Cascade Poisson Process (SCPP), can handle
multiple sequences of adoption events for multiple items bysharing parameters. Iwata et al. used a Bayesian approachto discourage overfitting during parameter inference. Theyevaluated the model by using social bookmark data, whereadopting an item corresponds to bookmarking a web page.
Tanaka et al. [19] extended SCPP to estimate the factors thattrigger item purchase events. They considered the users’view histories for TV advertisements in addition to influences between users and showed that SCPP is also eﬀective
in modeling purchase events.
Our model extends SCPP and the model proposed by
Tanaka et al. [19], diﬀering from them in the following two
respects. First, in the other models, there is no need to consider the eﬀect of adopted items such as bookmarked web
pages and purchased items. However, in derivative creation
activity, adopted items (i.e., derivative works) also influenceother creators’ creation activity. Therefore, we extended
SCPP so that we can handle the eﬀect of both original works
and derivative works. Second, although the other models assume that the popularity of items is constant regardless of††

http://songrium.jp

IEICE TRANS. INF. & SYST., VOL.E101–D, NO.4 APRIL 2018

1098

time, we assume that content popularity depends on time.
Hence, our model incorporates the time-dependent popularity of both original works and derivative works by considering content ranking data and the creators’ ranking browsing
behavior.
3.

Model

Our proposed model [6] assumes that the following threefactors play an important role in modeling derivative creation activity: (1) original work’s attractiveness, (2) original
work’s popularity, and (3) derivative work’s popularity. Below, we describe notations, followed by each of the threefactors.
3.1

Notations

Given a category (e.g., “3D models of chairs” or “music
videos covering songs”) and observation time period T , let
I be a set of original works posted to a web service (e.g.,
Thingiverse or YouTube) between time 0 and time T . Let(tipj , uipj ) denote the jth derivative work posting event of original work i. More specifically, creator uipj ∈ U posts i’sderivative work at time tipj . Here, U is the set of creators.
Without loss of generality, we assume that derivative work
posting events are sorted in ascending order of their timestamps: tipj ≤ tipj for j < j . When Ji represents the total
number of i’s derivative works posted during the observation time period, the set of derivative work posting events ofi. Hence, the set of derivai is given by Di = {(tipj , uipj )} Jj=1
tive work posting events of all original works is given by
D = {Di }i∈I .
Suppose creators can see the ranking of original works
on the web service, where original works are ranked basedon the popularity computed using statistics such as viewcount. Let (tiko , riko ) denote the kth ranked event of i ∈ I.
That is, i is ranked at the riko th place at time tiko . We also assume that the events are sorted in ascending order of theirtimestamps without loss of generality: tiko ≤ tiko  for k < k .
Let Kio be the total number of i’s ranked events between time
0 and time T . Then, the set of ranked events of i is given by
Kio. Therefore, the set of ranked events of all
Oi = {(tiko , riko )}k=1
original works is given by O = {Oi }i∈I .
Similarly, suppose creators can also see the ranking ofderivative works. In the same manner as with the ranked
event of the original work, let (tikc , rikc ) denote the kth ranked
event of i’s derivative work. Let Kic be the total number ofranked events of i’s derivative works between time 0 andtime T ; then the set of ranked
events of i’s derivative works
Kic. Note that Ci includes ranked
is given by Ci = {(tikc , rikc )}k=1
events of all i’s derivative works: (tikc , rikc ) and (tikc  , rikc  ) can
be ranked events of diﬀerent derivative works. Finally, theset of ranked events of all derivative works of all original
works is given by C = {Ci }i∈I .

Fig. 1time t.

Rate at which creator u posts original work i’s derivative work at

3.2 Factors
3.2.1

Original Work Attractiveness (Oatt)

A creator may create original work i’s derivative work because he/she thinks that i is attractive. We assume that eachcreator has a diﬀerent preference for original content attractiveness. We also assume that the post rate based on original
work attractiveness is constant in the time period from 0 to
T as described in Fig. 1 (a). Here, the rate at time t represents the instantaneous probability of a creator posting i’s
derivative work at t. Based on these assumptions, we model
the rate at which creator u posts i’s derivative work triggeredby i’s attractiveness as follows:fi (u) = αi θ0u ,(1)

where αi ≥ 0 is the original work attractiveness, θ0u ≥ 0
represents the probability that u is influenced by original
workattractiveness when he/she creates a derivative work,and u∈U θ0u = 1. In Fig. 1 (a), the height of the blue line
corresponds to αi θ0u .
3.2.2

Original Work Popularity (Opop)

If original work i is popular among consumers, creator u
may create i’s derivative work because his/her derivativeTSUKUDA et al.: SONGRIUM DERIVATION FACTOR ANALYSIS

1099

work might also become popular. As mentioned in Sect. 3.1,
we assume creators can see the popularity ranking of original works. When two original works are ranked, we hypothesize that the higher ranked one has a larger influencethan the lower ranked one based on the user’s search resultbrowsing behavior [7], [8]. In addition, we assume that eachcreator has a diﬀerent preference for original work popularity and that the influence of original work popularity on acreator decays over time. Based on these assumptions, we
model the rate at which u posts i’s derivative work at time t
based on the influence of i’s popularity as follows:⎧⎪⎪⎨rb(r )ωi θ−1u e−γo (t−t ) if t < t(2)ho(i,t ,r ) (t, u) = ⎪⎪⎩0
otherwise,
where r represents the rank of i at time t , and rb(r ) = r1 ,which computes the rank bias. The term ωi ≥ 0 represents
the influence of i’s popularity, θ−1u ≥ 0 represents the probability that u is influenced by original work popularity when
he/she creates a derivative work, and u∈U θ−1u = 1. Finally, e−γo (t−t ) models the decay of influence over time withdecay parameter γo ≥ 0.
In Fig. 1 (b), the original work i appears four times in
the popularity ranking. Let r be the rank of the first ranked
event. The influence of the event is rb(r )ωi θ−1u , which coro
. Then, the influence deresponds to h1 in Fig. 1 (b) at ti1
creases as time proceeds.
3.2.3

Derivative Work Popularity (Dpop)

If original work i’s derivative work is popular among consumers, creator u may also create i’s derivative work because his/her derivative work might also become popular.
As mentioned in Sect. 3.1, we assume creators can see thepopularity ranking of derivative works. Based on similar assumptions and the hypothesis described in Sect. 3.2.2, wheni’s derivative work was ranked r th at time t , we model therate at which u posts i’s derivative work at time t based on
the influence of i’s derivative work popularity as follows:⎧⎪⎪⎨rb(r )σi θ−2u e−γd (t−t ) if t < t
hd(i,t ,r ) (t, u) = ⎪(3)⎪⎩0otherwise,
where σi ≥ 0 represents the influence of the popularity of i’sderivative work, θ−2u ≥ 0 represents the probability that u is
influenced by derivative work popularity when he/she cre
ates a derivative work, and u∈U θ−2u = 1. Finally, e−γd (t−t )models the decay of influence over time with decay parameter γd ≥ 0.
Figures 1 (c) and (d) show the influences of i’s first andsecond derivative work popularity, respectively. The first
derivative work appears three times in the ranking, while
the second one appears once. Let r be the rank of the firstranked event in Fig. 1 (c). The influence of the first ranked
event is rb(r )σi θ−2u , which corresponds to h2 in Fig. 1 (c) at
c. Then, the influence decreases as time proceeds.
ti1

3.3 Derivative Work Post Rate
Based on the factors described in Sects. 3.2.1 to 3.2.3, therate at which u posts i’s derivative work at t is given by:ho(i,t ,r ) (t, u)λi (t, u) = fi (u) +(t ,r )∈Oit+

hd(i,t ,r ) (t, u),(4)(t ,r )∈Cit

where Oit = {(t , r )|(t , r ) ∈ Oi and t < t} is the setof ranked events of i before t and Cit = {(t , r )|(t , r ) ∈Ci and t < t} is the set of ranked events of i’s derivative
works before t. Here, λi (t, u) corresponds to h3 in Fig. 1 (e).
Given D, O, and C, the likelihood of the function of D
is given by:
P(D|O, C, α, ω, σ, Θ, γ)
⎛
⎞ J
i
T 

⎜⎜⎜
⎟⎟⎟ 
⎜
=
exp ⎜⎜⎝−λi (t, u)dt⎟⎟⎟⎠λi (tipj , uipj ),
i∈I

0

u∈U

(5)

j=1

where α = {αi }i∈I , ω = {ωi }i∈I , σ = {σi }i∈I , Θ = {θu }u ∈U+ ,θu = {θu u }u∈U , and γ = {γo , γd }. Here, U+ denotes{0, −1, −2}, where 0, −1, and −2 represent virtual creators
for Oatt, Opop, and Dpop, respectively. By using eﬃcient Bayesian inference based on a stochastic expectationmaximization (EM) algorithm [17], we can obtain the latent
triggers for derivative work posts. More details of the inference process can be found in Tsukuda et al.’s study [6].
4.

Songrium Derivation Factor Analysis

By using our proposed model, we implemented a public web
service called Songrium Derivation Factor Analysis, which
specializes in derivative videos on Niconico. Unlike ordinary video sharing services including Niconico, our service
enables users to understand more about N-th order derivative creation and find original videos, derivative videos, andcreators based on the factors estimated by our model. Morespecifically, we aim to realize the following three goals:• Enable consumers to explore original and derivative
videos based on the estimated results.
• Enable consumers to explore creators based on the estimated results.
• Enable creators to explore original videos based on theestimated results.
To achieve the first goal, we enable consumers to findoriginal videos according to the original videos’ characteristics in the derivative creation activity (the Original Works
Map function). Moreover, our service visualizes the derivative creation process so that consumers can explore derivative videos based on the influence in the derivative creation
activity (the Derivation Tree function).
In terms of the second goal, given an original video,

IEICE TRANS. INF. & SYST., VOL.E101–D, NO.4 APRIL 2018

1100

Fig. 2

Original works map.

we enable consumers to see the derivative creators’ characteristics by visualizing the degree to which each creator is
influenced by each of the three factors (the Creator Distribution Map function). When a consumer finds an interesting creator, our service shows the creator’s profile so that theconsumer can see the creator’s influence (the Creator Profile
function).
To achieve the third goal, displaying the originalvideos’ characteristics mentioned in the first goal also enables creators to find an original video to create a derivative
video from a new viewpoint. In addition, our service alsovisualizes the popularity influence in the derivative creation
activity so that creators who put a high priority on content
popularity can find a desirable original video (the Popularity
Influence Transition Graph function).
This section describes the dataset used for Songrium
Derivation Factor Analysis followed by the functions of the
web service.
4.1

DatasetFor Songrium Derivation Factor Analysis, we used derivative creation activity data of music content on Niconico. On
Niconico, any user can upload and view videos, and music
derivative work is frequently created. As of June 2017, morethan 140,000 original song videos and more than 630,000
derivative videos had been uploaded to Niconico. Most original songs are created by using singing synthesizer software
called VOCALOID [9]; we restricted ourselves to original
song videos of this type. With respect to derivative works,
Niconico maintains three categories of derivative works:• Singing: covering an original song. Creators typicallyuse the original video’s image and record their singing
voice by using the karaoke version of the original song.
• Dancing: dancing to an original song. Most creators
create dancing videos by recording their entire bodies including their faces. Since a limited number ofcreators can compose original choreography, most creators imitate the existing choreography.
• Playing: playing an original song on a musical instrument such as a guitar or piano. Creators often record
their music instruments and their bodies excluding theirfaces.

Singing
Dancing
Playing

Table 1 Statistics of dataset.
|I|
|O|
|D|
|C|
5,632 157,086 402,406 152,086
784
93,82831,67776,457658
95,024
9,316
52,514|U|
31,782
3,095
1,066

In all categories, the derivative creation activity is quite popular. In the “Singing” category, for example, the most popular (in terms of the number of derivative videos) originalvideo has more than 4,500 derivative videos as of November 2017. Among them, the most played one has a viewcount over 1.5 million. Similarly, in the “Dancing” (“Playing”) category, the most popular original video has about
3,000 (700) derivative videos and the most played derivative
video created from the original video has a view count over
2.8 million (2.8 million). We crawled original songs (i.e.,
original works) and their derivative works posted between
11/1/2009 and 12/31/2016. In each category, we eliminatedoriginal works that had fewer than three derivative works
and creators who posted fewer than four derivative works
during the period.
We also collected ranking data. On Niconico, users
can see the top 100 daily ranking for original songs and the
top 100 daily ranking for derivatives in each of the singing,
dancing, and playing categories. Ranking data on one day iscreated based on several statistics of the previous day (e.g.,view count and comment count) so that the ranking data
represents the work’s aggregated popularity. We crawled
the top 100 ranking data in each of the original song andthree derivative content categories between 11/1/2009 and12/31/2016. Since only daily ranking data is available on
Niconico, the timestamp in all our model is measured indays. Table 1 lists the statistics of the dataset used in ourservice.
4.2 Original Works Map
In Songrium Derivation Factor Analysis, original songs ineach category are embedded in a two-dimensional space(Fig. 2). Each circle icon corresponds to an original song;
the size of the icon varies in proportion to the number of its
derivative works. Users can change the category by clicking
on the category name on the upper left corner of the screen.

TSUKUDA et al.: SONGRIUM DERIVATION FACTOR ANALYSIS

1101

Algorithm 1 Calculating degree of content attractiveness(Eatt ) and content popularity (Epop ) for jth derivative work
posting event of original work i
Require: P(zi j |D, Z\i j , O, C, γ, β, a, b)
1: Eatt ⇐ P(zi j = 0|D, Z\i j , O, C, γ, β, a, b), Epop ⇐ 0
2: y ⇐ 1
3: while y ≤ |Oit | + |Cit | do4:
Epop ⇐ Epop + P(zi j = y|D, Z\i j , O, C, γ, β, a, b)
5:y⇐y+1
6: end while
7: return Eatt , Epop

Fig. 3Meanings of axes and quadrants.

As shown in Fig. 3, the vertical axis represents whether an
original work’s derivative works are created because of content attractiveness (i.e., Oatt in Sect. 3.2.1) or content popularity (i.e., Opop in Sect. 3.2.2 or Dpop in Sect. 3.2.3). The
horizontal axis represents whether the derivative work creators put a high priority on content attractiveness or content popularity when they regularly create an original work’s
derivative work.
By considering these two axes, we can characterizeoriginal songs as follows. The original songs in the first
quadrant are the ones used by many derivative creators who
put a high priority on content popularity, so they can be regarded as “standard” original songs. When an original songappears in the original song ranking but it is not easy formost creators to create its derivative work, such an original song is mapped to the second quadrant; it is regarded asa “challenging” original song. Original songs in the third
quadrant are the ones that do not frequently appear in the
ranking and are used by many creators who put a high priority on content attractiveness, so such original songs are
regarded as “for experts” original songs. Original songs in
the fourth quadrant are the ones that do not frequently appear in the ranking; they are used by many creators who put
a high priority on content popularity. Such original songs
are likely to be used by many creators from that time, sothey are regarded as “trend candidate” original songs. Below, we describe the way to compute the coordinates of agiven original song.
4.2.1

Vertical Axis Value

Our model has latent variables zi j ∈ {0, 1, · · · , |Oit | + |Cit |}to indicate the index of the latent trigger of the jth derivative work posting event of original work i (hereafter, the jthderivative work of original work i is denoted by vi j ). The
terms zi j = 0, 1 ≤ zi j ≤ |Oit |, |Oit | + 1 ≤ zi j ≤ |Oit | + |Cit |indicate that the event was triggered due to the influence ofOatt, Opop, and Dpop, respectively. By using our model,
we can compute the probability at which each latent variable influences the creation of vi j . Algorithm 1 shows thepseudo-code for computing the following two kinds of val-ues. The first one, denoted by Eatt , is the degree to which
vi j was created because of content attractiveness of i (i.e.,Oatt). The second one, denoted by Epop , is the degree towhich vi j was created because the content popularity (i.e.,
Opop or Dpop). By summing Eatt of all derivative works
of i, the influence degree of the content attractiveness regarding i’s derivative work creation can be computed (letSUMEatt denote the sum). Similarly, by summing Epop , theinfluence degree of the content popularity can be computed(let SUMEpop denote the sum). We compute the vertical axis
value V(i) of the original song i as follows.
SUMEpop
V(i) = 2− 0.5 .
(6)
SUMEpop + SUMEatt
V(i) ranges from −1 to 1, and a larger V(i) represents ahigher influence of the content popularity.
4.2.2

Horizontal Axis Value

Given creator u’s all derivative works, we compute CEatt andCEpop , which are the sum of each derivative work’s Eatt andEpop obtained by Algorithm 1, respectively. CEatt (CEpop )represents the degree to which u has put a high priority on
the content attractiveness (the content popularity). Given allof original song i’s derivative works, we compute the sumof CEatt and CEpop of each creator who created i’s derivative
work. The sum of CEatt , denoted by CSUMEatt , is the degree
to which i’s derivative work creators put a high priority on
the content attractiveness, while the sum of CEpop , denotedby CSUMEpop , is the degree to which i’s derivative work
creators put a high priority on the content popularity. We
compute the horizontal axis value H(i) of original song i asfollows.

CSUMEpop
H(i) = 2− 0.5 .
(7)
CSUMEpop + CSUMEatt
H(i) ranges from −1 to 1, and a larger H(i) represents ahigher influence of the content popularity.
4.3 Detailed Informaion of Derivative Works
When a user clicks on an original song in the twodimensional space in Sect. 4.2, our service shows detailed
information of the original song’s derivative works. Figure 4

IEICE TRANS. INF. & SYST., VOL.E101–D, NO.4 APRIL 2018

1102

Fig. 4

Fig. 5

Detailed information of the selected original song’s derivative works.

Derivation tree.

shows an example result when a user clicks on an original
song “Torinoko City† ” in the “Singing” category. The information consists of the following three parts: (1) Derivation
Tree, (2) Popularity Influence Transition Graph, and (3) Creator Distribution Map. Below, we describe each of them.
4.3.1

Derivation Tree

The Derivation Tree visualizes the derivative creation process of an original song. Figure 5 shows the enlarged
Derivation Tree in Fig. 4. Each circle represents an original
song or its derivative work. The horizontal axis is the timefrom the day when the original song was uploaded to the
day when the latest derivative work was uploaded. In the
Derivation Tree, the original song and its derivative works
have a hierarchical structure. The original song is displayedon the first layer, and the derivative works that were created from the influence of the original song are displayed on
the second layer as the second generation derivative works.
†

http://www.nicovideo.jp/watch/sm11559163

Moreover, derivative works created due to the influence ofthe second generation ones are displayed on the third layer
as third generation derivative works. The third generation
derivative work v is linked to the derivative work in the second layer that has the highest influence on v. The kth (k ≥ 4)
layer is also displayed in the same manner. Although a web
service that visualizes the set of derivative works of an original song has been proposed by Hamasaki et al. [1], by using our model, our service can visualize the relationship between derivative works.
When a user clicks on a circle icon, he/she can watch
the corresponding derivative work video. In addition, as
shown in Fig. 6, when the user hovers the cursor over a circle icon (i.e., derivative work v), derivative works that directly or indirectly influenced v and those directly or indirectly influenced by v are highlighted. This enables the user
to browse the paths related to his/her interesting derivative
work in the Derivation Tree.
To create the Derivation Tree of the original song i, foreach derivative work, we detect y, which is the index thatgives the maximum value of P(zi j ). When y = 0 or 1 ≤ y ≤|Oit |, the derivative work vi j was created by Oatt or Opop. In
this case, vi j is linked to the original work on the Derivation
Tree. When y corresponds to a ranked event of derivative
work vi j ( j < j), vi j was created from the influence of vi j .
In this case, vi j is linked to vi j and displayed on the nextlayer below vi j .
4.3.2

Popularity Influence Transition Graph

This function visualizes the transition of the popularity in-

TSUKUDA et al.: SONGRIUM DERIVATION FACTOR ANALYSIS

1103

Fig. 8Fig. 6

Paths related to hovered derivative work is highlighted.

Fig. 7

Popularity influence transition graph.

fluence in N-th order derivative creation activity. Figure 7 isan enlargement of the Popularity Influence Transition Graphin Fig. 4. The horizontal axis is the time from the monthwhen the original song was uploaded to the month when thelatest derivative work was uploaded. The vertical axis represents the popularity influence in each month. A low valuemeans that most derivative works in the month were createdfrom the influence of content attractiveness.
When we create the graph for original song i, computing the popularity influence in each month is required.
Hence, given all derivative works uploaded in a month, we
compute SUMEatt and SUMEpop as in Sect. 4.2.1. The value
SUMEpopof the vertical axis is given by SUMEatt +SUME
.
pop
4.3.3

Creator Distribution Map

The Creator Distribution Map visualizes the degree to which
each creator is influenced by each of the three factors in
Sect. 3.2. Figure 8 shows an enlargement of the Creator
Distribution Map in Fig. 4. One circle corresponds to a creator. “Original attractiveness,” “Original popularity,” and“Derivative popularity” represent Oatt, Opop, and Dpop, respectively. In Fig. 8, it can be observed that many creators
who put a high priority on Oatt and/or Opop uploaded thisoriginal work’s derivative works.
To create the map, given a creator, we need to computethe influence of each of the three factors for him/her basedon all of his/her derivative works. When creator u creates
the jth derivative work of original song i, the influence ofeach factor is computed by the pseudo-code in Algorithm 2.
E f , Eho , and Ehd represent the influence of Oatt, Opop, andCreator distribution map.

Algorithm 2 Calculating degree of three factors for jthderivative work of i
Require: P(zi j |D, Z\i j , O, C, γ, β, a, b)
1: E f ⇐ P(zi j = 0|D, Z\i j , O, C, γ, β, a, b), Eho ⇐ 0, Ehd ⇐ 0
2: y ⇐ 1
3: while y ≤ |Oit | + |Cit | do
4:
if y ≤ |Oit | then5:
Eho ⇐ Eho + P(zi j = y|D, Z\i j , O, C, γ, β, a, b)
6:
else
7:
Ehd ⇐ Ehd + P(zi j = y|D, Z\i j , O, C, γ, β, a, b)
8:
end if
9:y⇐y+1
10: end while
11: return E f , Eho , Ehd

Dpop, respectively. Given all of u’s derivative works, let
SUME f , SUMEho , and SUMEhd denote the sum of E f , Eho ,and Ehd , respectively. By normalizing their sum to 1, thecoordinates of u on the triangle can be detected.
4.4 Creator Profile
When a user clicks on a circle icon on the Creator Distribution Map in Sect. 4.3.3, the detailed information of thecreator is displayed. Figure 9 shows the example results ofa creator “Yamaneko Sanae.” The user can browse the listof derivative works posted by the creator. In addition, foreach derivative work v, an original work or a derivative workthat influenced v (denoted by “Influenced by”) and derivative works that are influenced by v (denoted by “Influencedon”) are displayed. By clicking on “Derivation Graph,” theuser can see the derivative work’s detailed information as
described in Sect. 4.3.
5.

Discussion

In this section, we describe how Songrium Derivation Factor
Analysis is useful for both consumers and creators. In particular, we mention new findings for them by using concrete
examples.
With ordinary video sharing services, consumers usually search for videos according to metrics such as the view

IEICE TRANS. INF. & SYST., VOL.E101–D, NO.4 APRIL 20181104

Fig. 9Fig. 10

Derivation tree of an original video in the “Singing” category.

Fig. 11

Derivation tree of an original video in the “Dancing” category.

Creator profile.

count and uploaded date; while the Original Works Map enables consumers to explore original videos based on thircharacteristics in the derivative creation activity. For example, by selecting the “Dancing” category, consumers canfind original videos that are standard for dancers from the“Standard” area on the map. By selecting the “Singing” category, they can also find original videos that are diﬃcult tosing from the “Challenging” area.
This function is also useful for creators to find an original video to create a derivative video from a new viewpoint.
For example, for novice creators, it would be useful to beable to search for original songs in the “Standard” area;
while for skillful creators, original songs in the “Challenging” area would be fascinating. In addition, novice creators
can find out that original videos having a high view countand/or many derivative works are not always appropriate touse for creating derivative videos. For example, an original video has over one million view counts and as many as
199 derivative videos in the “Dancing” category. Looking at
only such statistics, this original video may seem to be appropriate for novice creators. However, this original video
is mapped to the “For expert” area. By using our service,novice creators can avoid original videos that are popular
but inappropriate for them. On the other hand, skillful creators can find out that original videos with low view counts
and a handful of derivative videos do not always require ahigh level of skill because many of such videos are mapped
to the “Standard” area. Our service enables skillful creators
to find original videos that really require high skill from the“Challenging” area.
The Derivation Tree enables consumers to explore
derivative videos from a new viewpoint: the influence ofeach derivative video. For example, in Fig. 6, the derivative
video that the cursor is hovering over has influence because
our model estimates that the video triggered eight derivative videos. Without our service, it is diﬃcult to find suchan influential derivative video. Derivative videos that havehigh view counts are not always influential videos, as shown

Fig. 12

Examples of popularity influence transition graph.

in Fig. 10. Hence, video exploration based on influence canonly be realized by using our model. In another example,
Fig. 11 shows the tree for an original video in the “Dancing” category. In the tree, one of the most influential derivative videos was the video in which the creator danced tothe original song with her own choreography. Since othercreators imitate the choreography, the derivative video does
have a big influence. Although our model did not know
which derivative video had the original choreography, ourmodel was able to estimate that the derivative video had a
big influence.
In terms of the Popularity Influence Transition Graph,as shown in Fig. 12, the popularity influence transition isvery diﬀerent from one original video to another. For example, Fig. 12 (a) shows that the popularity influence is highonly during the earliest stage of the derivative creation activity. In Figs. 12 (b) and (c), the popularity has a big influenceTSUKUDA et al.: SONGRIUM DERIVATION FACTOR ANALYSIS

1105

Fig. 13

Examples of creator distribution map.

during the early part and the latter part, respectively. Hence,for creators who put a high priority on popularity, the original video of (c) is more appropriate to use for creating aderivative video than those of (a) and (b) as of December
2016. By using the Popularity Influence Transition Graph,creators can find out if now is the appropriate time to create
the original video’s derivative video.
By viewing the Creator Distribution Map of an original video, consumers can find out which of the three factors creators tend to put a high priority on. In Fig. 13, (a),(b), and (c) represent the maps of original videos in the“Singing,” “Dancing,” and “Playing,” respectively. The tendency is very diﬀerent according to the categories and original videos. This function also enables consumers to explore
creators. For example, in Fig. 13 (c), since most creators put
a high priority on Opop, consumers can find a typical creatorin this original video’s derivative creation activity around theOpop corner of the map; while they can also find an atypicalcreator around the Dpop corner.
Finally, by using the Creator Profile, consumers caneasily find the characteristics of a creator in terms of whether
he/she tends to be influenced by an original video or otherderivative videos and whether he/she tends to influenceother creators’ creation activity. For example, as the creatorin Fig. 9 puts a high priority on Oatt, three of the four derivative videos are influenced by the original video. Moreover,since two derivative videos created by this creator triggers
other derivative videos, consumers can find out that this creator has influence in the derivative creation activity.
6.

Conclusion

In this paper, we described a public web service called
Songrium Derivation Factor Analysis that was implemented
by applying our proposed model, which infers latent factors and their influences in derivative creation activity, todatasets obtained from the video sharing service Niconico.
Songrium Derivation Factor Analysis has several functions
that could enable users to browse and watch videos from a
new viewpoint and decide which content they use to create
a new derivative work. We have to carry out evaluations
such as a user study to clarify the suﬃciency of the proposed functions; we leave this as future work. Although weapplied our model to the data on Niconico, our model can beapplied to any data if the conditions mentioned in Sect. 3.1

are satisfied. For future work, we plan to analyze users’
click logs on our web service and discuss if our service influences users’ video watching behaviors (e.g., if videos thathave a larger influence are watched more often). We are alsointerested in deeper analysis regarding how creators use ourservice to find an original work that they want to use forcreating a new derivative work.
Acknowledgments
This work was supported in part by JSPS KAKENHI Grant
Number 17K12688 and JST ACCEL Grant Number JPMJAC1602, Japan.
References[1] M. Hamasaki, H. Takeda, and T. Nishimura, “Network analysis of
massively collaborative creation of multimedia contents: Case studyof hatsune miku videos on nico nico douga,” Proc. 1st International
Conference on Designing Interactive User Experiences for TV andVideo, UXTV ’08, pp.165–168, 2008.
[2] C. Cayari, “The YouTube eﬀect: How YouTube has provided new
ways to consume, create, and share music,” International Journal of
Education & the Arts, vol.12, no.6, pp.1–28, 2011.
[3] L.A. Liikkanen and A. Salovaara, “Music on YouTube: User engagement with traditional, user-appropriated and derivative videos,”Computers in Human Behavior, vol.50, pp.108–124, 2015.
[4] S. Papadimitriou and E.E. Papalexakis, “Towards laws of the
3d-printable design web,” Proc. 2014 ACM Conference on Web Science, WebSci ’14, pp.255–256, 2014.
[5] M. Goto, “Grand challenges in music information research,”
Dagstuhl Follow-Ups: Multimodal Music Processing, vol.3, pp.217–
225, 2012.
[6] K. Tsukuda, M. Hamasaki, and M. Goto, “Why did you cover thatsong?: Modeling n-th order derivative creation with content popularity,” Proc. 25th ACM International on Conference on Information
and Knowledge Management, CIKM ’16, pp.2239–2244, 2016.
[7] L.A. Granka, T. Joachims, and G. Gay, “Eye-tracking analysis ofuser behavior in WWW search,” Proc. 27th Annual International
ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’04, pp.478–479, 2004.
[8] T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay, “Accurately interpreting clickthrough data as implicit feedback,” Proc.
28th Annual International ACM SIGIR Conference on Researchand Development in Information Retrieval, SIGIR ’05, pp.154–161,
2005.
[9] H. Kenmochi and H. Ohshita, “Vocaloid – Commercial singing synthesizer based on sample concatenation,” Proc. INTERSPEECH,
pp.4009–4010, 2007.
[10] K. Eto, M. Hamasaki, K. Watanabe, Y. Kawasaki, and T. Nishimura,IEICE TRANS. INF. & SYST., VOL.E101–D, NO.4 APRIL 2018

1106

“Modulobe: A creation and sharing platform for articulated models with complex motion,” Proc. 2008 International Conference
on Advances in Computer Entertainment Technology, ACE ’08,
pp.305–308, 2008.
[11] G. Cheliotis and J. Yew, “An analysis of the social structure of remix
culture,” Proc. Fourth International Conference on Communities andTechnologies, C&T ’09, pp.165–174, 2009.
[12] M. Hamasaki and M. Goto, “Songrium: A music browsing assistance service based on visualization of massive open collaboration
within music content creation community,” Proc. 9th International
Symposium on Open Collaboration, WikiSym ’13, pp.4:1–4:10,
2013.
[13] X. Song, Y. Chi, K. Hino, and B.L. Tseng, “Information flow modeling based on diﬀusion rate for prediction and ranking,” Proc.
16th International Conference on World Wide Web, WWW ’07,
pp.191–200, 2007.
[14] X. Song, B.L. Tseng, C.-Y. Lin, and M.-T. Sun, “Personalized recommendation driven by information flow,” Proc. 29th Annual International ACM SIGIR Conference on Research and Development inInformation Retrieval, SIGIR ’06, pp.509–516, 2006.
[15] J. Yang and J. Leskovec, “Modeling information diﬀusion in implicit networks,” Proc. 2010 IEEE International Conference on Data
Mining, ICDM ’10, pp.599–608, 2010.
[16] K. Saito, M. Kimura, K. Ohara, and H. Motoda, “Learning
continuous-time information diﬀusion model for social behavioral
data analysis,” Proc. 1st Asian Conference on Machine Learning:
Advances in Machine Learning, ACML ’09, pp.322–337, 2009.
[17] T. Iwata, A. Shah, and Z. Ghahramani, “Discovering latent influence
in online social activities via shared cascade Poisson processes,”
Proc. 19th ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining, KDD ’13, pp.266–274, 2013.
[18] A. Simma and M.I. Jordan, “Modeling events with cascades ofPoisson processes,” Proc. 26th Conference on Uncertainty in Artificial Intelligence, UAI ’10, pp.546–555, 2010.
[19] Y. Tanaka, T. Kurashima, Y. Fujiwara, T. Iwata, and H. Sawada,“Inferring latent triggers of purchases with consideration of social eﬀects and media advertisements,” Proc. Ninth ACM International Conference on Web Search and Data Mining, WSDM ’16,
pp.543–552, 2016.

Kosetsu Tsukuda
received the M.S. and
Ph.D. degrees in Informatics from Kyoto University, Japan, in 2011 and 2014, respectively.
He is currently a Researcher at the National Institute of Advanced Industrial Science and Technology (AIST), Japan. His research interests
include web information retrieval, web mining,and data mining.

Keisuke Ishida
was the CEO of Ohma Inc.
and managed an online social network mining
system, SPYSEE, until 2011. Since 2012, hehas been a Technical Staﬀ (Creative Engineers)
at the National Institute of Advanced Industrial
Science and Technology (AIST), Japan. He has
implemented web applications regarding musical information processing and information visualization.

Masahiro Hamasaki
received the Ph.D. degree in informatics from the Graduate University for Advanced Studies (SOKENDAI), Japan
in 2005. He is currently the Leader of the Media Interaction Group at the National Instituteof Advance Industrial Science and Technology(AIST), Japan. His research interests include
web mining, semantic web, and social media
analysis. He is a member of the JSAI, the IPSJ,
and ACM.

Masataka Goto
received the Doctor ofEngineering degree from Waseda University in
1998. He is currently a Prime Senior Researcher
at the National Institute of Advanced Industrial
Science and Technology (AIST), Japan. Overthe past 25 years, he has published more than
250 papers in refereed journals and international
conferences and has received 46 awards, including several best paper awards, best presentation
awards, the Tenth Japan Academy Medal, andthe Tenth JSPS PRIZE. In 2016, as the Research
Director he began OngaACCEL Project, a 5-year JST-funded research
project (ACCEL) on music technologies.