Reflection-based Word Attribute Transfer
Yoichi Ishibashi, Katsuhito Sudoh, Koichiro Yoshino, Satoshi Nakamura
SNL-2019

Nara Institute of Science and Technology

Conclusion

Motivationp Word attribute transfer can be used for data argumentationp Analogy-based word attribute transfer requires theknowledge whether the input word is for male or female

explicitp We propose Reflection-based word attribute transfer, a method
without such explicit knowledge

Approachü Reflection-based word attribute transfer can transform word
attributes without explicit knowledge
p E.g., girlboy, boygirlü Reflection has high stability (99.9% of non-attribute words were not changed)
p E.g., apple
apple, humanhumanü Reflection has a property similar to logical negation

Relationship with Symbolic Logic

What is this task?

p Reflection is similar to logical negation ¬¬ man = woman
Ref (man) = woman

Transform word attributes on a word embedding space¬ ¬ man = man
Ref (Ref (man)) = man

Experiment

Analogy-based Word Attribute Transfer

p E.g. 1 if the nearest neighbor of f (boy) is “girl”, otherwise 0(24 words for evaluation)p Stability : Stability of words without gender attribute

We can transform an attributeby using analogy, but…

p E.g. 1 if the nearest neighbor of f (apple) is “apple”, otherwise 0(1000 words for evaluation)

Ref
Ref + PM
Diff
AvgDiff

p Problem : Analogy method requires theexplicit knowledge whether the input
word is for male or femalex

fgender (x) =(x + (queenx (queenking)king)(x 2 Male)(x 2 Female)fgen (fgen (man)) = man
We need this function

Reflection-based Word Attribute Transfer
What is Reflection?

Refa,c (x) = x

p A mapping that transfers
vector x to y with ahyperplane (Mirror)2(xc)T aa
Ta al Refa,c (x) : Reflection
l: Input vector
l a, c : Parameters of mirrorlx

p An identity mapping is
obtained when Ref(x) is
applied twice

Ref (Ref (man)) = man

How to apply to word attribute transfer?
p Learn a mirror to transform an attribute(e.g., gender)y = Refa,c (x)1
L=|A|X(yia = M LP (attr id)
c = M LP (attr id)
X1ti ) 2 +(yj|N |(man, woman) 2 A
p Problem : Linear inseparable
p Idea : Parameterized mirrorxØ Estimate a mirror from each input wordxj ) 2xj 2N(xi ,ti )2A

Reflection-based word attribute transfer
Reflection-based transfer with parameterized mirror
Analogy-based transfer with one differential vector
Analogy-based transfer with average of differential vectors

Resultsp Goal : No knowledge = Transform with same function

fgen (man) = woman
fgen (woman) = manx

Input: word vector,output: word vector
target attribute ID (e.g., gender)p Dataset : 106 pairs of gender words (train/val/test = 58/24/24)
p |A| = 58, |N | = 4 (in the training)p Add random noise tobecause the train data size is smallp Accuracy : Transformation accuracy of words with gender attribute

apple 2 N

a = M LP (attr id, x )c = M LP (attr id, x )p Reflection can transform a word attribute without explicit
knowledge (Transformation accuracy is 55.55%)
p Reflection is very stable (99.9% of non-attribute words were not changed)

Accuracy (%)
Stability (%)
knowMethodledge Mean@3 @1@3 Mean@3 @1@3Ref
40.27 25.00 54.16
99.53 99.50 99.60
Ref + PM
96.90
96.50 97.30
55.55 45.83 62.50
MLP
19.44
8.33 33.00
0.00
0.00
0.00
Diff (-)
21.31
7.61 30.74
83.29
79.36 85.87
AvgDiff (-)
23.61
4.16 33.33
98.13
98.10 98.20✓
Diff
40.65 15.94 57.67✓
AvgDiff
47.20 12.50 66.66
How many non-attribute words |N | do we need when training?
p Reflection has high stability even only |N | = 10

Accuracy @1 (%)
Stability @1 (%)
Method|N | = 041050
041050
Ref20.83 25.00 25.00 25.00 97.10 99.50 98.40 95.60

Ref + PM 45.83 45.83 37.50 29.16 35.80 96.90 99.90 99.30
MLP

4.16

8.33

0.00

0.00

0.00

0.00

0.00

0.00

Reflection-based transfer examplesx

Ref (x)Ref (Ref (x))

when my father was a boy,he had liked the lady
who is an actress

Apply to other attributes
Original

she is my motherwhen my mother was a girl,she had liked the gentleman + Gender
he is my father
who is an actor+ Age
he is my grandfather
when my father was a boy,he had liked the lady
who is an actress

Reflection-based Word Attribute Transfer+ Tense

he was my grandfather

SNL-2019