NIPS2016

Multi-view Anomaly Detection
via Robust Probabilistic Latent
Variable Models
Tomoharu Iwata NTT Communication Science Labs
Makoto Yamada RIKEN, AIPCopyright©2016 NTT corp. All Rights Reserved.

Multi-view anomaly• Instances that have inconsistent views• Application– information disparity management• find documents that contain different information
across multilingual Wikipedia documents– malicious insider detection– purchase behavior analysis• find movies inconsistently purchased by users
based on the genre (animation by purchased bygrown-ups)
Multi-view
imagedataannotation

textURLs

audiovideo

Japanese text
English text2

Single-view/multi-view anomaly
View1
BDMCA

View2
SF GI
H EJI EH
MJGSFADCB• single-view anomaly is an instance that does notconform to expected behavior• S is single-view anomaly, but not multi-view anomaly• M is not single-view anomaly, but multi-view
3anomaly

Existing method: HOAD (1)• HOrizontal Anomaly Detection (HOAD)– A Spectral Framework for Detecting Inconsistency
across Multi-source Object Relationships, Gao et.
al. ICDM 2011

• Step1: soft clustering two views together withthe constraint that an instance should beassigned to the same cluster
view1 view2
two-view
combinedsimilarity
graph

diagonal

view1

spectral clustering
diagonal

view2

4

Existing method: HOAD (2)• Step2: quantify the difference between thetwo clustering solutions• weak points– anomalous instances also have the constraint tobe assigned to the same cluster– require hyper-parameter tuning (e.g. weight forthe constraint)

5

Proposed model• Normal (non-anomalous) instance– all views are generated from a single latent vector• Anomaly– different views are generated from different latentvectors
BAMCD𝑾1BDMCA

F GI MJHELatent space

S𝑾2

SF GIH EJI
M EHJGS
Observed view 1FADC
B

Observed view 2

6

Proposed model• Each instance has potentially a countably infinite numberof latent vectors• Each view of an instance is generated depending on a
view-specific projection matrix and a latent vector

AA

AA

𝑾1

Observed view 1A
Latent space

𝑾2

Observed view 2

𝑾𝐷

Observed view D7

Proposed model• Each instance has potentially a countably infinite numberof latent vectors• Each view of an instance is generated depending on aview-specific projection matrix and a latent vector

AAA
A

𝑾1

Observed view 1A
Latent space

𝑾2

Observed view 2

𝑾𝐷

Observed view D

8Proposed model• Each instance has potentially a countably infinite numberof latent vectors• Each view of an instance is generated depending on a
view-specific projection matrix and a latent vector

AA

AA𝑾1

ALatent space

𝑾2

𝑾𝐷A

Observed view 1

Observed view 2

Observed view D9

Proposed model• Each instance has potentially a countably infinite numberof latent vectors• Each view of an instance is generated depending on aview-specific projection matrix and a latent vector

AA

A
A

𝑾1

A
Latent space

𝑾2

𝑾𝐷A

Observed view 1

Observed view 2

Observed view D10

Proposed model• Each instance has potentially a countably infinite numberof latent vectors• Each view of an instance is generated depending on aview-specific projection matrix and a latent vector

AA

A

A
Latent space

A𝑾1
A

Observed view 1

𝑾2

𝑾𝐷A

Observed view 2

Observed view D11

Proposed model• Each instance has potentially a countably infinite numberof latent vectors• Each view of an instance is generated depending on a
view-specific projection matrix and a latent vector

AA

A

A
Latent space

A

𝑾1
A

Observed view 1

𝑾2

𝑾𝐷A

Observed view 2

Observed view D12

Proposed model• Each instance has potentially a countably infinite numberof latent vectors• Each view of an instance is generated depending on aview-specific projection matrix and a latent vector

AAAALatent spaceA

𝑾1
AObserved view 1

𝑾2
AObserved view 2𝑾𝐷A

Observed view D13

Proposed probabilistic model
projection matrix for view d∞

precision𝑝 𝒙𝑛𝑛 𝒛𝑛 , 𝑾𝑑 , 𝜽𝑛 , 𝛼 = � 𝜃𝑛𝑛 𝑁(𝒙𝑛𝑛 |𝑾𝑑 𝒛𝑛𝑛 , 𝛼 −1 𝑰)view d ofinstance n

𝑗=1

mixture weight for j-th latent vector

j-th latent vectors of instance n• By using Dirichlet processes prior for each instance, the number oflatent vectors is automatically determined from the given data
– if views are consistent, they are clustered; otherwise, theyneed different latent vectors• By using view-dependent projection matrices, we can handle
different properties across different views
Anomaly score:
probability that theinstance uses more than
one latent vector

𝐻#latent vectors of instance n at the h-th
iteration after burn-in

1
𝑣𝑛 = � 𝐼(𝐽𝑛 ℎ > 1)
𝐻ℎ=1

indicator function14

Proposed model• clustering views for each instance in the latent space

A3A1 A4A5

Latent space

A2𝑾1𝑾2𝑾5A

AA
Observed view 1

Observed view 2Observed view 515

Relation with other latent variable models• Proposed model−1– 𝑝 𝒙𝑛𝑛 𝒛𝑛 , 𝑾𝑑 , 𝜽𝑛 , 𝛼 = ∑∞𝑗=1 𝜃𝑗 𝑁(𝒙𝑛𝑛 |𝑾𝑑 𝒛𝑛𝑛 , 𝛼 𝑰)

• Probabilistic PCA– 𝑝 𝒙𝑛𝑛 𝒁, 𝑾 = 𝑁(𝒙𝑛𝑑 |𝑾𝒛𝑛𝑑 , 𝛼 −1 𝑰)• Probabilistic CCA– 𝑝 𝒙𝑛𝑑 𝒁, 𝑾 = 𝑁(𝒙𝑛𝑑 |𝑾𝑑 𝒛𝑛𝑑 , 𝚺)

• Infinite Gaussian mixture−1– 𝑝 𝒙𝑛𝑑 𝝁, 𝜽 = ∑∞𝑗=1 𝜃𝑗 𝑁(𝒙𝑛𝑑 |𝝁𝑗 , 𝛼 𝑰)
16

Generative process

17

Inference based on stochastic EM• Analytically integrate out the latent vectors Z, mixture
weights Θ, precision 𝛼• E-step: collapsed Gibbs sampling of latent vector
assignment s for each view of each instance ℓ = (𝑛, 𝑑)• M-step: maximum joint likelihood estimation ofprojection matrices W

18

Experiments• Data– 11 data sets from LIBSVM data
– generated multiple views by randomly splitting
the features– anomalies were added by swapping views of two
randomly selected instances• Comparing methods– PCCA: probabilistic canonical correlation analysis– HOAD: horizontal anomaly detection– CC: consensus clustering based anomaly detection– OCSVM: one-class support vector machine
19

Multi-view anomaly detection withdifferent anomaly rate

20
The proposed model achieved the best with 8 of the 11 data sets

Multi-view anomaly detection withdifferent latent dimensionality

21

MovieLens data analysis
instance: movie, view1: user list who rated, view2: genre

high anomaly score movies

low anomaly score movies‘The Full Monty’ and ‘Liar Liar’ were ‘Comedy’ genre. They are rated by not
only users who likes ‘Comedy’, but also who likes ‘Romance’ and ‘ActionThriller’.
‘The Professional’ was anomaly because it was rated by two different usergroups, where a group prefers ‘Romance’ and the other prefers ‘Action’.
Since ‘Star Trek’ series are typical Sci-Fi and liked by specific users, its
anomaly score was low.
22

Conclusion• We proposed a generative model approach for multiview anomaly detection, which finds instances thathave inconsistent views.
• In the experiments, we confirmed that the proposedmodel could perform much better than existing
methods for detecting multi-view anomalies• Future work
– nonlinear projection using Gaussian processes ordeep neural nets

23