19
Bull. Soc. Photogr. Imag. Japan. (2015) Vol. 25 No. 2: 19–24

Original Paper

Toward Image Emotion Space: Evaluation of Image Impression Shifts with Kansei
Words
Kenji Kagitani1, Deng Pei2, Ei Matsuda2, Tomoyasu Aizaki1, Satoshi Nakamura1, Toshihiro Suzuki1,
Michiyoshi Tachikawa1, Naokazu Aoki2, Hiroyuki Kobayashi2

Abstract:

 n image emotion space defining the relationship between the manipulation of a photographic image and the resulting change inA
the impression it produces was constructed by presenting original images and their manipulated versions in pairs to a panel of observers and asking them to identify the one giving the stronger impression for each of 59 emotion words. Picture-independent results
could be obtained by comparing raw images of various pictures with those having various effects applied at random. Components
governing the responses were extracted from the results by principal component analysis to construct a two-dimensional space withthe components lightness and contrast and chroma as axes. Furthermore, it was shown that the differences in impressions can beclustered into 12 clusters by cluster analysis. A Kansei space model showing pathways to “preferred” was obtained by multiple linear
regression analysis using “preferred” as the dependent variable and the representative words of their respective clusters as the independent variable. It consisted of two pathways, one for lightness information and one for color information.

Key words:

image emotion space, effect, Kansei space model, pathways to “preferred”

1. Introduction
One of the main objectives of photograph retouching is to manipulate the photograph to provide a predetermined impression to
the viewer. For example, in printing workflow, a request or instruction for photo retouching is transmitted using emotion words. Now
that photo retouching applications for digital cameras and smartphones are widespread and we can share manipulated images, it isvery important to know the change in the impression produced byan effect. It is of interest to construct an image emotion space1) defining the relationship between the manipulation of a photographic
image and the resulting change in the impression it produces, notonly from a practical point of view but also from a cognitive science
point of view.
In this study, we presented original images and their manipulated
versions in pairs to a panel of observers and asked them to identify
the one giving the stronger impression for each of 59 emotion words.
Components governing the responses were extracted from the results by statistical analysis to construct a two-dimensional spacewith the components as axes. Another goal was to construct a Kansei structure including a causal relationship between the word “preference” and the other emotion words.
An image includes not only the image quality but also a picture,and picture dependency must always be taken into consideration in
Received: 24th, April 2015; Accepted: 9st, September 2015

1

Ricoh Institute of Technology, Ricoh Co., Ltd.

16-1 Shineicho, Tsuzuki-ku, Yokohama 224-0035, Japan

2

Graduate School of Advanced Integration Science, Chiba University
1-33 Yayoi-cho, Inage-ku, Chiba 263-8522, Japansubjective evaluations. Therefore, it would be ideal to understand thepicture dependence of our impressions. This is difficult, however, because types of pictures are infinite in number and impossible toquantify. Bernat et al. measured physiological responses such as
blinking and skin conductance as measures of affect for themes including erotica, adventure, victim, and threat2), but this is a rare case.
In order to avoid picture-dependence, many image studies using the
Kansei model3) have used a single color patch4) or a combination of
two color patches5) as a stimulus. In contrast, in this study the aim
was to obtain a picture-independent result by comparing raw imagesof various pictures with those having various effects applied at random and, further, to investigate the differences between the impressions created by the raw images and those created by the processed
images.

2. Experiment
2.1 Stimuli
We obtained 60 photos depicting a variety of scenes and objects
from the photo-sharing site stock.xchng (now called Freeimages)6),
and we applied an image manipulation effect selected at random
from 16 different types of Adobe Photoshop actions to each photo.
Each effect (action) is a series of image manipulation proceduresthat combines the color tone and sharpness of change. Multiple ap-

20

Bull. Soc. Photogr. Imag. Japan. Vol. 25 No. 2 (2015)

Fig. 2A sample pair (Image9)).

Table 1

Emotion words (59 words)

natural, lonely, quiet, tasteful, real, dense, open, solemn, light, crisp, transparent, mild, brilliant, reasonable, clear, unstable, preferred, vital, hard, pure,cold, deep, cool and bitter, giving a moist feeling, giving a you-are-there
feeling, giving a profound feeling, calm, rich, with texture, strange, sophisticated, swinging, well-modulated, unique, voluminous, smooth, smoky,
pale, large-scaled, evil, sharp, impressive, peaceful, relaxed, three-dimensional, obscure, refreshing, magnificent, eerie, shining, healthy, evoking afeeling of the season, warm, garish, fresh, giving a sizzle feeling, giving anaerial feeling, simple and clear, dry

Fig. 1 T wo original images and their versions after processing with each of
the 16 actions.
The number and colored mark for each action will be used in the figures
shown later; refer to this figure when viewing those figures.

plications of each action were allowed. The subjective evaluation experiment was performed using 60 pairs, each consisting of an original image and its processed version.
Further, in order to confirm the presence or absence of picture
dependence, each of the sixteen effects was applied to two images ofsignificantly different picture type, and the same subjective evaluation experiment was performed using these pairs. The original andprocessed images are shown in Fig.1.
2.2 Procedure
The panel of observers was shown an image pair consisting of theoriginal and its processed image, arranged side by side or one above
the other on the display, to evaluate them. The positional relationship of the pair was changed randomly every time to avoid prejudg-

ment by the panel. The viewing time was not limited.
For each of 59 emotion words (Table 1), the panel was asked to
select one of three choices: which image of the two presented wasmore appropriate for that emotion or (the third choice) neither ofthem. The adjectives were selected from a vast number of words andwere chosen for their availability and their correlation with otherwords as determined through a preliminary experiment. Because theevaluation was performed using the emotion words in Japanese, Japanese ones are put with their English words.
Forty-three subjects in their 20s through 60s participated in thisexperiment, including students studying image technology and engineers working in an imaging-technology company.
Subjects in the experiment viewed the samples on their own liquid crystal display of 17 to 24 inches. The room illumination and theviewing distance were not specified.
2.3 Data analysis
Subject evaluations were scored by assigning -1 when the original
image was deemed more appropriate for the emotion word, +1 whenthe manipulated one was chosen, and 0 when neither was selected as
appropriate, and the data were statistically analyzed using the IBM
SPSS Statistics (IBM Co.) software package10).
The correlations between the score profiles for each emotion word
for the 60 image pairs and the synonymity or antonymy between thewords as represented by plus and minus, respectively, were reasonable for each word, using their dictionary meanings. This confirms
the validity of the scoring method.

Kenji Kagitani, Deng Pei, Ei Matsuda, Tomoyasu Aizaki, Satoshi Nakamura, Toshihiro Suzuki, Michiyoshi Tachikawa, Naokazu Aoki, Hiroyuki Kobayashi

3. Results and Discussion
3.1 Principal component analysis (PCA)11)
Five principal components were extracted from the experimental
data by PCA. Contribution ratios of the first and the second principal component are 33.2 and 23.0, respectively. The cumulative contribution ratio of the two principle components exceeds 56%. The
words of large component loading for the first principle component
are “voluminous”, “impressive”, “magnificent”, “pale”, and “dense”,
and those for the second principle component are “cool and bitter”,“quiet”, “lonely”, “calm”, and “cold”. It is interpreted that the respective components summarize “dignity” and “vividness”.
The loadings for each principal component were obtained foreach emotion word. The component score for each principal component was calculated for each image pair by adding loading for the
principal component multiplied by average score of each subject together. As shown in Fig.3, plots for image pairs manipulated by thesame effect (shown as plots of the same color) were grouped in acertain area in the scatter plot with axes of the first and the second

Fig. 3Fig. 4

Toward Image Emotion Space: Evaluation of Image Impression Shifts with Kansei Words

21

principal component. Impressions produced by the images wereclassified roughly according to the effects; the results suggested that
impressions were independent of the pictures in the images. This was
confirmed by applying each effect to images of the same picture andplotting the results of PCA in the same space, as shown in Fig.4. The
results for the two pictures fell in almost the same areas as in Fig.3
for each respective effect.
The changes produced by the actions were investigated by applying them to color patches of the Macbeth Color Checker. The results
are shown in Fig.5 comparing with the color chart without the actions. Changes in lightness (L*), chroma (C*), and tone brought
about by applying the actions are shown in Fig.6. The actions aregrouped into classes by the results of principal component analysis.
The L* values were obtained by averaging lightness of six grayscalecolor patches (White, Neutral 8, Neutral 6.5, Neutral 5, Neutral 3.5,
and Black) and the C* values were obtained by averaging chroma of
the primary colors patches (Red, Green, and Blue). Tone curves wereobtained by plotting the L* values of the six grayscale colors patches
against those of the original. They are shown in the same four groups
with Fig.6 in Fig.7.
Actions located in negative score region of the first principle component increase L* and those located in positive score region decrease or do not change L*. Clear differences in C* were not ob-

A scatter plot of the primary and secondary ingredients.
Color marks: See Fig.1.

Validation of the results shown in Fig.3 by applying each of the actions to an image.
Color marks and numbers: See Fig.1.

Fig. 5

 olor patches of the Macbeth Color Checker. after processing withCeach of the 16 actions (left) compared with the original (right)

22

Fig. 6

Bull. Soc. Photogr. Imag. Japan. Vol. 25 No. 2 (2015)

 hange in lightness(L*), chroma (C*), and tone brought about by
C
applying the actions. The actions are grouped into classes by the results of principal component analysis. The marks + and – for tone
indicate “become harder” and “become softer” by application of actions, respectively (see Fig.7).

Fig. 8Fig. 7

Dendrogram obtained by cluster analysis (Ward’s method).

 hange of tone curve by application of the actions. The actions areCgrouped into classes by the results of principal component analysis
as in Fig.6.

served between the two action groups. With regard to tone an
obvious difference was observed that actions in positive score region
make harder and those in negative score region make softer.
With regard to the second principle component no obvious difference in change of L* and tone was observed between the positivescore group and the negative score group. On the other hand, C* ofthe positive score group increases much larger than the negativescore group.
From all of these results it is suggested that the first principle
component axis is for lightness and contrast and the second principle component axis is for chroma and that the combination of lightness with contrast and chroma control “dignity” and “vividness”, respectively.
3.2 Multiple linear regression analysis (MLRA)12) and covariancestructure analysis (CSA)13)
The emotion words were clustered according to similarities in theevaluation results, and we used MLRA to obtain a structural modelfor the emotional response that would show causal connections be-

Fig. 9

 model representing two-way causal connections between 13 words
A(the representative words of the 12 clusters plus the word “preferred”).
Independent variables are those at arrow tails, and dependent variables arethose at arrowheads. Magnitudes of standard partial regression coefficientsare represented by the thicknesses of the arrows.

tween the clusters. First, we investigated clustering methods14) appropriate for improving the fit of models obtained by MLRA andselection methods for identifying a word representative of each cluster. It was found that dendrogram clustering according to distanceand choosing the word with the largest dispersion as the representative one of the cluster were effective. In addition, performing theanalysis after omitting words with a high frequency of “neither” resulted in an improved model fit, the best result being obtained at
50% of the frequency. In this case, 28 words were omitted, and 31

Kenji Kagitani, Deng Pei, Ei Matsuda, Tomoyasu Aizaki, Satoshi Nakamura, Toshihiro Suzuki, Michiyoshi Tachikawa, Naokazu Aoki, Hiroyuki Kobayashi

Acronym
CMNI(Minimum of
discrepancy
function)

Table 2

Goodness-of-fit indices

Explanation

chi-square value

DF (Degrees of
freedom)
P (Probabilityof CMIN)GFI (Goodness-of-fit
index)

Critical values andpolarity

47.892 CMIN/DF = 1.29
(Acceptable fit: 1~2)37
0.108 Good fit: >0.1
proportion of variancethat is accounted for by
the estimated population covariance

0.900 Good fit: ~1

measure of how well themodel, with unknown
but optimally chosenparameter estimates,
would fit the population’s covariance matrix

0.071 Good fit: <0.1

AGFI (Adjust- GFI adjusted fored goodnessdegrees of freedom,
of-fit index)
with more saturated
models reducing fit.
Thus, more parsimonious models are preferred, while complicated models are penalized.

RMSEA(Root-meansquare error of
approximation)

Value

0.755

Fig. 10 A model representing causal connections.
Arrows with standard partial regression coefficients greater than 0.3 andpointing toward “preferred” are retained from Fig.7.

Fig. 11 Images that gave “strange” at high rates.
1 : With Action 12 applied; 216): With Action 12 applied; 39): With Action
9 applied.
15)

words were clustered into 12 clusters (Fig.8). We executed stepwise
MLRA for 13 words (the 12 words having the largest dispersion ofeach cluster plus the word “preferred”) and constructed a model ofhigh goodness of fit including two-way causal connections by using
every word as the dependent variable and all other words as a candi-

Toward Image Emotion Space: Evaluation of Image Impression Shifts with Kansei Words

23

date independent variable. Next, we used CSA to construct a model
including all 13 words, and a model of high goodness of fit was obtained (Fig.9). Indices of model fit are listed in Table 2. The redsolid line is the standard for clustering, and the broken lines are thecluster borders. The words with a red sphere are the representative
words of their respective clusters.
A complex model including bidirectional causal connections was
obtained as a significant one. However, pathways toward “preferred”
emerged when connections with standard partial regression coefficients smaller than 0.3 were removed, leaving arrows pointing toward “preferred” as shown in Fig.10. That is, the pathways that startwith words related to lightness, such as “dense” and “pale”, go
through “light” or “three-dimensional”, “well-modulated” and “brilliant” and reach “preferred”. This suggests that from the viewpoint ofimage quality, the enhancement of tone and sharpness achieved by achange in lightness results in “preferred”. On the other hand, thepathways that start with “calm” and “cold” go through “warm” or“strange”, and “real”, and reach “preferred”. We can connect “cold”
and “warm” with cool and warm coloration and interpret this pathway as one for color information. Images that gave “strange” at highrates are of very high or very low chroma (Fig.11), suggesting that
this pathway is also one of color information.

4. Conclusion
Differences in impressions produced by the existence or nonexistence of various effects were evaluated by their aptness for 59 emotion words. A two-dimensional Kansei space having lightness andcontrast axes and chroma axes was constructed from the result by
PCA. It was verified that this result is picture-independent. Furthermore, it was shown that the differences in impressions can be clustered into 12 clusters.
A Kansei space model showing pathways to “preferred” was obtained by MLRA using “preferred” as the dependent variable. It consists of two pathways, one for lightness information and one forcolor information. It is very interesting that these two pathways areanalogous to those of our visual system from retina to brain, which
are the “Where” system and the “What” system18). The former carries information about lightness and is used for motion perception,
depth perception, spatial organization, and figure/ground segregation, and the latter carries information about color and is used for
object recognition, face recognition, and color perception.

References
1) B. Oicherman, C. Staelin, Kansei Imaging: First Steps, HPL-2009-12,20092) E. Bernat, C. J. Patrick, S. D. Benning, A. Tellegan, Effects of Picture
Content and Intensity on Affective Physiological Response, Psychophysiology, 43(1): 93(2006)
3) L. C. Ou, M. R. Luo, A. Woodcock, A. Wright, A Study of Colour
Emotion and Colour Preference. Part III: Colour Preference Modelling,
Color Research and Application, 29(5), 381(2004)

24

Bull. Soc. Photogr. Imag. Japan. Vol. 25 No. 2 (2015)4) L. C. Ou, M. R. Luo, A. Woodcock, A. Wright, A Study of Colour
Emotion and Colour Preference. Part I: Colour Emotions for Single
Colours, Color Research and Application, 29(3), 232(2004)
5) L. C. Ou, M. R. Luo, A. Woodcock, A. Wright, A Study of Colour
Emotion and Colour Preference. Part II: Colour Emotions for Two-Colour Combinations, Color Research and Application, 29(4), 292(2004)6) http://www.sxc.hu/
7) M. Richert, http://www.sxc.hu/photo/1330703
8) Sheepy, http://www.sxc.hu/photo/225852
9) T. Gulick, http://www.sxc.hu/photo/36013
10) S. Landau, B. S. Everitt, A Handbook of Statistical Analyses using

SPSS, Chapman & Hall/CRC, 2004
11) I. T. Jolliffe, Principal Component Analysis, 2nd Ed., Springer, 2002
12) J. Higgins, The Radical Statistician, 5th Ed., Chap. 4, The Management
Advantage, 2005
13) J. L. Arbuckle, Amos 5.0 Update to the AMOS User’s Guide (S. 77-85),
Chicago: Small Waters Corp., 2003
14) B. S. Everitt, S. Landau, M. Leese, D. Stahl, Cluster Analysis, 5th Ed.,
Wiley, 201115) A. Kovacs, http://www.sxc.hu/photo/98822
16) M. Alberto, http://www.sxc.hu/photo/1349619
17) M. Livingstone, Vision and Art, Chap. 4, Abrams, 2002