HumanDiffusion:diffusion model using perceptual gradients

Session: Wed-P9
Paper number: 1680

Yota Ueda, Shinnosuke Takamichi, Yuki Saito, Norihiro Takamune, and Hiroshi Saruwatari
The University of Tokyo, Japan

1. SYNOPSIS
1.
2.

Proposes a diffusion model that represents a human-acceptable distribution.
Demonstrates that the proposed algorithm successfully samples data that follow a human-acceptable distribution.

2. RESEARCH BACKGROUND

4. PROPOSED METHOD: HumanDiffusion

Human-acceptable distribution
Method• A probability distr. in which data humans can accept as natural. • Sampling data 𝒙 ~ # 𝐷 𝒙 with score ∇log 𝐷 𝒙∫&• A human-acceptable distribution is typically wider than a realApproximation of scoredata distribution.
• Approximating score because it cannot be calculated analytically.
Human-acceptable distribution• Neighbor data is sampled from real data for gradient around them.
Real-data•
Evaluates
perturbed
datainnaturalness
acceptabilityforNES.
distribution
Modeling the distribution with DNN (deep neural network)• Can generate speech in a human-acceptable distribution.
• Can generate speech that do not exist.

%#𝒙!,+ − 𝛥#𝒙!,+𝐷%#!,+ − 𝛥𝒙#!,+𝒙−𝐷'$!,# − 𝛥$𝒙𝒙!,#0 𝛥$𝒙!,#+𝐷'$!,# − 𝛥$𝒙𝒙!,#/2*1𝒙!,# ) =, 𝐷
∇𝐷(#𝒙!,+ ) ∇𝐷($2𝜎$%& 𝑅'$!,# + 𝛥$𝒙𝒙!,##!,+𝐷 𝒙'$!,# + 𝛥$𝒙𝒙!,#'()$!,# = argmax+ 𝑝+𝐷 𝒙𝑆𝒙)

𝑆∇3 𝒙)
𝑆3 𝒙)

𝑆∇3
𝑆3

Discriminator 𝐷#!
𝒙
Generated
updatedata

𝑅 times
Evaluation in naturalness%
Humans#𝐷 𝒙!,+ + 𝛥#𝒙!,+

Perturbation%,𝛥#𝒙!,+ ∼ 𝑁(𝟎, 𝜎"#$𝑰)%#!,+ + 𝛥𝒙#!,+𝒙

𝐷Training score network and sampling

Real/Fake∇𝐷𝒙)4'(estimated∇ log 𝐷(#𝒙!,+ ))56(𝑛: data index)•+&' !&'𝜖 , 𝑆∇3 𝒙)
Sampling: 𝒙)4' = 𝒙) +2 𝑆3 𝒙)

𝒙1+ +𝒓#!,+ − ∇𝐷 𝒙#!,+
• Loss function: 𝐿 = - - 𝑆∇3 𝒙

HumanGAN[2],𝒙'𝒙) 𝒙)4' 𝒙 0#!,+ − 𝐷 𝒙#!,++ 𝑆3 𝒙,+ 𝜖𝒓Consists of a DNN-generator and a humans-discriminator.
The generator outputs data in a human-acceptable distribution.
Gradients ∇𝐷 cannot be calculated analytically.
Purpose[4]
Approximates gradients by natural evolution strategy (NES) .
•Confirm
HumanDiffusion
modelshuman-acceptable
distribution.
%𝐺
Humans#! + Δ𝒙!
𝒙
Prior
Preparation
ofdataspace
Data#𝒙

5. EXPERIMENTAL EVALUATIONS/a/𝒙! ) ≈• Gradients are approximated as ∇𝐷(#2𝜎1"#$ 𝑅%
#!
𝒙

- ∆𝐷%
/ ∆𝒙!

%&'

Human-acceptable distribution
Real-data
distribution

Diffusion model[3]• Consists of a score network 𝑆 only.
• The score network models score ∇ log 𝑝 𝒙 of the real-data
distribution 𝑝 𝒙 .
• Data is sampled with Langevin dynamics using estimatedscores and it follows the real-data distribution.

𝒙1𝑆𝒙)𝒙)𝒙0

From: https://yang-song.net/blog/2021/score/Priordata 𝒙1

Estimated∇ log 𝑝 𝒙)𝒙'++15

𝑆73 (𝒙)

∇𝐷(𝒙)

𝐷(𝒙)

0-15 -150Sampleddata𝒙) 𝒙)4' 𝒙 0
𝑇: sampling step𝑡: sampling index
𝜖: small constant𝒓~𝑁(𝟎, 𝑰): random value• Sampling with Langevin dynamics$𝜖𝒙!"# = 𝒙! + 𝑆(𝒙! ) + 𝜖𝒓
2
• If score of humansʼ evaluation function 𝐷(𝒙) is modeled insteadof 𝑝(𝒙), sampled data follows a human-acceptable distribution.

𝑆3 (𝒙)
0.5: real-data distribution

First principal component

𝒓

1 sec

15

1st principal component𝒙)4'

1st principalcomponent

Hyper-parameters• 𝑛: 100, 𝑚: 3, #humans: 105, 𝜎()* : 10, 𝜎+,- : 1.0, 𝜖: 0.003, 𝑇: 10000, 𝑅: 20• Score network: 3 fully-connected layers, optimizer: Adam
Evaluation 1: confirm score network learns 𝐷 𝒙 and ∇𝐷(𝒙)• 𝑆& 𝑥 got higher values around the real-data distribution.
• 𝑆∇& (𝑥) pointed to the real-data distribution.
• Score network was trained successfully.
1

2nd principal
component

HumanGAN(training truncated)
HumanGAN(training converged)

Applying PCA tolog spectral envelopes

Average of F0, aperiodicities(• Problem: needs heuristic training truncation:𝑝 𝒙: ∇ log 𝑝 𝒙

corpus of many
speakersʼ phonemes

2nd principal
component

perturbation
𝑅 times

Evaluation innaturalness ∆𝐷(#𝒙! )Data space Synthesizing speechusing WORLD

Second principal component#! )∇𝐷(𝒙𝜎"#$ : standard deviation
𝑅: perturbation number
𝑟: perturbation index
𝑰: identity matrix

WORLD[6] analysis

JVPD[5]: speech0

First principal component

Evaluation 2: perceptual evaluation of sampled data
• Humans evaluated sampled data and real data in naturalness.
• The evaluation value was almost same although the distributionof sampled data was wider than real-data distribution.
• HumanDiffusion samples data in human-acceptable distribution.
15

Real-data
distribution
Real-data
distribution

0-15

Sampleddata-150

1.0

averageNatural

0.5

0.0 Unnatural

15

1st principal component
First principal component

Perceptualevaluation

update%#! − Δ𝒙!
𝒙%,Δ𝒙! ~𝑁(𝟎, 𝜎"#$𝑰)

Second principal component!

Second principal component

distribution

2nd principalcomponent

••••

𝒙!

#!,+𝒙,∼ 𝑁 𝒙! , 𝜎-./𝑰∇ log 𝐷(#𝒙!,+ ) =GAN (generative adversarial network)[1]• Consists of a DNN-generator and a DNN-discriminator.
• The generator outputs data in a real-data distribution.
• Gradients ∇𝐷 for updating generatorʼs parameters can becalculated analytically.
Prior
distribution

Neighbordata• Score is calculated as

3. RELATED WORKSGenerator 𝐺RealdataSampled data Real data

References [1] Goodfellow et al., Proc. NIPS, 2014. [2] Fujii et al., Proc. ICASSP, 2020. [3] Song, Yang, et al. arXiv:2011.13456 (2020).
[4] Ilyas et al., Proc. ICML, 2018. [5] ”http://research.nii.ac.jp/src/en/JVPD.html [6] Morise et al., IEICE, 2016.

1