2019 IEEE International Conference on Consumer Electronics(ICCE)

Texture Synthesis with Desired Visual Impressions
Using Deep Correlation Feature
Atsushi Takemoto, Kensuke Tobitani, Yusuke Tani, Taishi Fujiwara, Yoichi Yamazaki and Noriko Nagata
Kwansei Gakuin University
Sanda, Hyogo, Japan
E-mail: {a.takemoto, nagata}@kwansei.ac.jp

Abstract—We propose a texture synthesis method with desired
visual impressions, to realize product designs suitable for individual needs. First, we conducted an evaluation experiment andquantified visual impressions of texture images. After that, wemodeled the relationship between the quantified visual impressions and the image feature extracted from the convolutional
neural network. Based on the obtained model, we updated thefeatures to have the desired visual impressions and generated
texture image. Finally, we confirmed the effectiveness of ourmethod by conducting a verification experiment.

I. I NTRODUCTION
Manufacture of mass production and mass consumption has
so far been carried out. However, manufacturing products that
better suits individual needs is nowadays regarded as important
to differentiate products.
Texture is the surface properties of objects, and the impressions of a texture, such as luxury and smoothness, areimportant factors in the preferences or quality judgments ofobjects [1]. Therefore, texture control techniques are neededin various manufacturing scenes to create the ideal impression.
Many texture synthesis methods have so far been made[2]. An algorithm for transferring an image style by using
VGG-19 which is a convolutional neural network for object
recognition [3] has been proposed [4]. However, there iscurrently no research that manipulates the visual impression
felt from images in texture synthesis.
In this research, we propose a texture synthesis method thatcan control the visual impressions felt from images. To thisend, we conduct the following tasks: quantification of visual
impressions on texture images; modeling of the relationships
between visual impression and image features extracted froma convolutional neural network; and texture synthesis with thedesired visual impression. Finally, we verify the effectivenessof our method by conducting an evaluation experiment using
images generated by our method. Fig. 1 shows an overview
of our approach.
II. Q UANTIFICATION OF VISUAL IMPRESSIONS
We conducted a subjective evaluation experiment to quantify
visual impressions of texture images. Prior to this experiment,
we collected 25 Japanese adjectives to evaluate visual impressions, and 29 texture images, including images collected fromthe PerTex texture database [5] (V01-V29). The participantsin this experiment were 20 Japanese in their 20s (15 men; 5women; average=22.2; SD=1.01). They evaluated 25 adjective
scales about 29 images in 5 stages.
We performed factor analysis to rate the data using theunweighted least squares method and the Promax rotation.
As a result, 4 factors were extracted (Fig 1 upper left). We
interpreted each factor as follows: Factor 1 is “freshness”
because “youthful” and “sporty” have high factor loading;
Factor 2 is “roughness” because “slick” and “smooth” havehigh negative factor loading; Factor 3 is “robustness” because“firm” and “heavy” have high factor loading; and Factor 4
is “crudeness” because “fine” has high negative factor loadingand “unlikable” has factor loading. We quantified the obtainedfactor scores as the visual impressions of the texture images.
III. M ODELING
We modeled the relationship between the quantified visual
impressions and the image features. We extracted deep correlation features [6] using a pre-trained VGG-19 [3], and weused them as image features of texture images. They consistof 4 elements: Gram matrix, Deep correlations, Diversity, andSmoothness. Because the Gram matrix represents the imagestyle [4], it can be considered to be highly related to visual
impressions of texture images. For this reason, we used theGram matrices extracted in pooling layers 1, 2, 3, and 4 as thefeatures to be modeled. The dimensions of them were 64×64,128×128, 256×256 and 512×512, respectively.
In order to model the relationships, we performed a Lasso
regression by setting the factor scores as dependent variables and the Gram matrices as independent variables. In
this research, since the Gram matrices are high-dimensionalfeatures with respect to the number of texture images, overfitting was expected. In Lasso regression, significant parts areselected from independent variables, and regression model isconstructed while preventing over-fitting, which is why weused this method.
First, as a result of a regression using all Gram matrices as
independent variables, the Gram matrix extracted in poolinglayer 4 was not selected. This suggests that features extracted
in deep layers are important for object recognition and are
not related to the visual impressions of the texture images.
Therefore, we focused on the Gram matrix extracted in poolinglayer 1, 2 and 3 and again performed regression. TABLE I
shows the results of the Lasso regression. We constructed the978-1-5386-7910-4/19/$31.00 ©2019 IEEEFig. 1: Overview of our proposed method.

regression models with high precision and modeled the relationships between the visual impressions and image features.
TABLE I: Results of Lasso regression.
Independent variables
Coefficient of determination

64×640.65

128×1280.83

256×256
0.91

IV. T EXTURE SYNTHESISBased on the obtained model, we updated the Gram matrix
to have the desired visual impressions. By using the Grammatrix of an original image as an initial value, we searchedfor optimal features with desired (promotion or suppression)
factor scores near the initial value using a gradient method.
Based on the redesigned Gram matrix and the other features ofthe original image, we generated a texture image from white
noise image using the method of Sendik et al. [6].
Figure 2 shows the results of generating a V07 image withpromoted “freshness.” An image was generated that emphasized the horizontal and vertical edges while maintaining theentire structure of the original images. These results suggest
the effectiveness of our method.

(a) Original image.

(b) Generated image.

Fig. 2: Results of generation from updated features.
V. E FFECT VERIFICATION EXPERIMENT
We conducted an evaluation experiment to verify whether
the impressions of the image generated by our method changed
as designed. We used 8 texture images as stimuli: the original
images of V07 and V12, images with promoted “freshness,”
images with suppressed “crudeness,” and images with both
impressions changed. We used the same 25 adjectives usedin the previous experiment as the evaluation words. The

participants in this experiment were 15 Japanese in their 20s(7 men; 8 women; average=21.7; SD=1.29). They evaluated
25 adjective scales about 8 images in 5 stages.
After comparing the evaluation scores of the original images
and the generated images, we confirmed that the impression
had changed as designed in V07 with both impressionschanged, V12 with promoted “freshness,” and V12 with both
impressions changed. We also confirmed an expansion effectoutside the range of the impression space obtained in theprevious experiment (Fig. 1 lower left). From these results,
we confirmed the effectiveness of our method.
VI. C ONCLUSION
We proposed a texture synthesis method that can control visual impressions. First, we modeled the relationships between
the visual impressions and image features of texture images.
After that, based on the obtained model, we updated image
features to have desired visual impressions , and generated
texture images. In addition, we confirmed the effectiveness of
our method by conducting a verification experiment.
In the future, we will generate texture images with various
visual impressions changed and verify the effects. As a result, our method will contribute to manufacturing that better
satisfies individual needs.
R EFERENCES[1] A. Takemoto, T. Fujiwara, K. Tobitani, Y. Tani, and N. Nagata, “Camparison of visual impression given by texture of real surfaces and synthesizedimages,” in The 11th IEEE Pacific Visualization Symposium, no. 124,
2018.
[2] L. Liu, J. Chen, P. W. Fieguth, G. Zhao, R. Chellappa, and M. Pietikäinen,“A survey of recent advances in texture representation,” CoRR, vol.
abs/1801.10324, 2018.
[3] K. Simonyan and A. Zisserman, “Very deep convolutional networks forlarge-scale image recognition,” CoRR, vol. abs/1409.1556, 2014.
[4] L. Gatys, A. S. Ecker, and M. Bethge, “Image style transfer using
convolutional neural networks,” in Conference on Computer Vision andPattern Recognition, 2016, pp. 2414–2423.
[5] F. Halley, “Perceptually relevant browsing environments for large texture
databases,” Ph.D. dissertation, Heriot Watt University, 2012.
[6] O. Sendik and D. Cohen-Or, “Deep correlations for texture synthesis,”
ACM Trans. Graph., vol. 36, no. 4, 2017.