CGI2015 manuscript No.
(will be inserted by the editor)

Direction and Scale Preserving Image Analogies
Hiromu Seki · Masahiro Toyoura · Xiaoyang Mao

Abstract Image Analogies is an eﬀective technique fortransferring the relationship between the texture features of a source image pair onto a target image forprocessing. Conventional Image Analogies may fail topreserve the local texture directions or texture scales of
the target image if the source image does not have thetextures of similar directions or scales. This paper proposes a new technique that involves linking the regions
between source image and target image based on texture scale and then rotating the regions in the source
image so as to align its texture direction with that of thecorresponding region in the target image. This enables
the reproduction of textures of the proper directions
and scale even when there are no common elements
linking the scale and directions of the textures in thesource and target images.

important. In addition, the scale of texture is also an
important feature of a ﬁlter. An artist usually changes
the size of brush strokes to reﬂect the local texture size.
However, how to preserving the local texture direction
and size is not considered in the existing Image Analogies approach either.
To solve such problem, this paper proposes a new
method that involves automatically changing the directions of the textures in a source image to match thoseof the target image. This makes it possible to reproduce textures in the proper directions even when there
are no common elements linking the directions of thetextures in the source and target images. By automatically identifying the regions with textures of similar
scale, the proposed method also maintains the scale ofthe textures in the resulting image.

Keywords Image Analogies · Texture transferring ·Style transferring · Gabor ﬁlter

2 Related research

1 Introduction
The Image Analogies [5] approach is one topic of research in the ﬁelds of image ﬁltering technology andrelated texture synthesis. The framework transfers therelationship between the texture features of an example image pair A and A’ (called source image pair hereafter) onto an image B (called unﬁltered target image) for processing to create an image B’ (called ﬁlteredtarget image ).
For a ﬁlter producing images with directional texture, such as the brush strokes in a painterly ﬁlter, preserving the texture direction in the input image is very
University of Yamanashi
Takeda 4–3–11, Kofu, Yamanashi, Japan

Given a pair of images A and A′ , the unﬁltered andﬁltered source image pair, along with an unﬁltered target image B, Image Analogies synthesizes a new ﬁlteredtarget image B ′ to complete the analogy A : A′ :: B: B ′ . It is achieved by ﬁnding the best matching pixels
between the source and target images. For the pixel qof B being synthesized, the pixel p in the source image
A that having a neighborhood best matches that of q isfound. Then, the pixel in the corresponding position ofp in A′ is copied to the pixel in the corresponding position of q in B ′ . Multi-scale representations of images arealso constructed to match pixels with successive sizesof the neighborhood.
Several research works have been conducted to extend Image analogies or other texture synthesis techniques for preserving the directional or structural features in the resulting image. Xie et al. [12, 13] pro-2

Hiromu Seki et al.
Obtain texture direc!onand scale (Sec!on 4.1)

Pair regions withmatching scales

…

…

…

…

Unﬁltered target image

Obtain texture direc!onand scale (Sec!on 4.1)

Gaborﬁlters

Gaborﬁlters

Filtered source image

RotateMatch the angles of the scaleand texture regions (Sec!on 5)
Filtered target image

Unﬁltered source image

Image Analogies

Fig. 1 Framework of the proposed method

posed a technique to better preserve the structure in
the synthesized target image by adaptively varying thecoherence parameter according to their distance fromthe structural features such edges and boundaries. WeiHan Chang et al. [2] proposed to better preserve thestructure information of the resulting image by adaptive changing the patch size when performing patch
based texture synthesis. Tong-Yee Lee et al. [9] tried
to improve the quality of texture synthesis by using thestructure information in similarity searching. Johannes
Kopf et al. [6] developed a technique for generating directional texture on 3D surface by learning from examples. H. Lee et al. [7,8] extended Ashikhmin’s fast texture synthesis technique [1] to generate directional effect in the resulting image. Either in Image analogies or
Ashikhmin’s algorithm, two diﬀerent searches are combined to ﬁnd the best matching pixels. B. Wang et al.
[11] and Yan-wen Guo et al. [4] also proposed methods
that involved superimposing several user-speciﬁed features of an example painting over the brightness gradient of a photograph to synthesize the directions oftextures. However, these two methods require the user
to specify regions manually.
3 Proposed method
Under the proposed method, the ﬁrst step involves obtaining information on texture direction and scale fromthe unﬁltered target image and the source image. After
dividing the images into regions with the similar texture
directions and scales, the method aligns the texture directions of regions of the same scale and then applies
Image Analogies approach.
Figure 1 shows the framework of the proposed method.
First, a set of Gabor ﬁlters are used to obtain the texture direction and texture scale of each pixel in boththe ﬁltered source image and unﬁltered target image.
Next, both images are segmented into multiple texture
regions based on the texture directions and scales. Thenfor each region in the unﬁltered target image, a regionof ﬁltered source image with the same scale is selected.
The selected region is rotated so that its texture direction is aligned with that of the corresponding region
in the target image. Finally, Image Analogies framework is applied to transfer the ﬁltering eﬀect across thematching region pairs.
The ﬁrst step of the algorithm is to detect the directions and scales of local textures in both the ﬁlteredsource image and the unﬁltered target image and divide
the images into regions of similar texture directions andscales. We use K-means clustering for the segmentation
in a high-dimensional feature space deﬁning the texture direction and scale. Response of Gabor ﬁlter bank,color in Lab color space and pixel location are used for
K-means clustering.
Gabor ﬁlter is known to be a powerful texture descriptor[3]. To capture the texture directions and scales,
we use a ﬁlter bank consisting of Gabor ﬁlters given
with the following equations [10]:g(x, y : λ, θ, ψ, σ, γ) =x′2 + γ 2 y ′2x′exp(−)cos(2π+ ψ)2σ 2λx′ = xcosθ + ysinθ, y ′ = ycosθ − xsinθ.

(1)Here ψ is the phase shift. γ is the parameter forcontrolling the aspect ratio of the 2D Gaussian ﬁlter
kernel. λ is the parameter for controlling the frequency.
Varying λ makes it possible to create Gabor ﬁlters for
capturing the scales of textures. θ is the parameter forcontrolling orientation. Varying this value makes it pos-

Direction and Scale Preserving Image Analogies

sible to create Gabor ﬁlters for capturing local texture
directions.
Our method use 30 Gabor ﬁlters in total with 6
diﬀerent orientations ranging from 0 degrees to 150 degrees at 30-degree intervals, and 5 diﬀerent frequencies.
The texture descriptor is a 30 dimension feature vector
with each dimension being the strength of response to
one of the Gabor ﬁlter, which is computed as the sumof all pixel values in the ﬁltered image.
Given that regions are essentially groups of pixels
with similar color patterns, we also include the colorof texture as the feature in clustering. The color of then × n neighborhood is used as the color feature for apixel. Lab color space is used from a perspective of beingclose to human perception. In current implementation,
we set n = 5 in the current implementation and hence
the dimension of color feature is 75(5 × 5 × 3). Finally,
we also include pixel location (x, y) as the feature forclustering so as to deal with the images with multiple
texture regions that have similar directions and colors
but not adjacent to each other.
Given the segmentation results of the ﬁltered source
image A′ and unﬁltered target image B, we ﬁrst segment the unﬁltered source image A in the same way as
A′ . Then the ﬁltered target image B ′ is generated region
by region by applying Image Analogies to the matchingregion pairs between source and target image.
Denoting the representative direction and scale of a→region R as −v (R) and S(R). For each region Bi (i =1, 2, · · · , n) of B, its ﬁltered target region Bi′ is generated with the following algorithm.
1. Find a set of regions {A′j }(j = 1, 2, · · · , k) in A′
satisfying |S(A′j ) − S(Bi )| < ϵ.
2. For each A′j−→
2.1 compute θ(A′j , Bi ) = arccos→v (Aj ) · −v (Bi )′′−→→
2.2 rotate Aj , Aj by θ(Aj , Bi ) to align v (Aj ), −v (A′j )−→with v (Bj )3. Apply Image Analogies with {Aj }, {A′j } as the source
image pair and Bi as the unﬁltered target region.
Here, ϵ is a user given threshold for matching regions between source and target images based on texture scale.
Diﬀering from the conventional Image Analogies, here
{Aj } and {A′j } are a set of regions. Step 3 performs thesimilarity search in all regions of {Aj } and {A′j }. The
regions of {Aj }, {A′j } and Bi are usually no longer ina rectangular shape. However, by taking the advantageof Image Analogies as a pixel based approach, we canperform the similarity and coherence search for eachpixel in the same way as the conventional algorithm.
To establish local texture directions and scales in animage, we assign a representative texture direction andscale to each texture region.

3Multidirectional Gabor ﬁltering provides directional
reaction strength values for six directions. The direction with the strongest reaction strength becomes thetexture direction for each region. To determine texture
scale, we use multiresolutional Gabor ﬁltering to determine reaction strength values by frequency level. Of
the ﬁve possible frequency levels, the level with thestrongest reaction strength becomes the texture scalefor each region.
The proposed method searches for pixels in image
regions whose texture scales match those of the regionsin the target image. Compared to searching the entire
source image, this approach can make it more diﬃcult
to reproduce the original intensity of the target unﬁltered image with precision. To solve this problem, we
add a post processing step which transfers the intensityof the unﬁltered target image B to the resulting ﬁlteredtarget image B ′ . Each pixel PB ′ in B ′ is modiﬁed via
following rule.
PB′ ′ ← PB′ ′ + (mB − mB ′ ).

(2)Here, mB and mB ′ are the average intensity of B andB ′ . Such correction makes it possible to retain the texture conﬁguration created by the intensity variance but
make the overall brightness closer to that of the originalimage.
4 Result
For verifying the capability of dealing with directional
texture, we used oil paintings with directional stroke
texture as the ﬁltered source images. To produce thecorresponding unﬁltered source images, we applied bilateral ﬁltering to eliminate the characteristic brushwork that emerges in the images as a high-frequency
component while retaining the region boundaries.
Figure 2(c) is a photograph of chestnuts in burrs.
The photograph shows the burrs growing on cracked
chestnut shells, with the spines radiating out in diﬀerent directions. We applied the source image pair shownin Figure 2(a) and (b) to the target image in Figure2(c) and then generated images using the two methods. Figure 2(d) shows the result of the conventional
method, while Figure 2(e) shows the result of the proposed method. two methods. The results of Figure 2exhibit the same diﬀerences. Whereas the conventional
method performs its transfer processing without taking
the directional and scale properties of the textures in
the photograph, the proposed method retains the photograph’s various texture directions and scale better.
Figure 3 shows an example of applying a small painting example (Figure 3(a)) to a target image with textures of varying directions and scales(Figure 3(c)). Fig-4

Hiromu Seki et al.

(a) Unﬁltered source image A(c) Unﬁltered target image B(b) Filtered source image A′(d) Conventional method(e) Proposed method

Fig. 2 A comparison of the generated images(a) Unﬁlteredsource image A(b) Filteredsource image A′
which is a smallpainting example(c) Unﬁltered target image B(d) Conventional method(e) Proposed methodFig. 3 An example of using a small source image

ure 3(d) shows the results of conventional method, while
Figure 3(e) shows the results of the proposed method.
Despite having just a small painting example with only
one directional property, the proposed method successfully rendered the photograph’s texture directions.
5 Concluding remarks
This paper proposed a method that involves ﬁnd texture regions of similar scale in the source image androtating the textures to match those of the target image. This enables the reproduction of textures in theproper directions scale even when there are no common
elements linking the directions and scale of the texturesin the source and target images.
Acknowledgements
This research was supported by JSPS KAKENHI Grant
Number 26560006.
References
1. Ashikhmin, M.: Fast texture transfer. IEEE Computer
Graphics and Applications 23(4), 38–43 (2003)
2. Chang, W.H., Cheng, M.C., Kuo, C.M., Huang, G.D.:
Feature-oriented artistic styles transfer based on eﬀective texture synthesis. Journal of Information Hiding andMultimedia Signal Processing 6(1), 29–46 (2015)

3. Gogel, I., Sagi, D.: Gabor ﬁlters as texture discriminator.
Biological Cybernetics 61(2), 103–113 (1989)
4. wen Guo, Y., hui Yu, J., dong Xu, X., Wang, J., shengPeng, Q.: Example based painting generation. Journal ofZhejiang University SCIENCE A 7(7), 1152–1159 (2006)
5. Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., ,Salesin, D.: Image analogies. In: Proc. SIGGRAPH, pp.
327–340 (2001)
6. Kopf, J., Fu, C.W., Cohen-Or, D., Deussen, O., Lischinski, D., Wong, T.T.: Solid texture synthesis from 2d exemplars. ACM Transactions on Graphics 26(3) (2007)
7. Lee, H., Seo, S., Ryoo, S., Yoon, K.: Directional texturetransfer. In: Proc. International Symposium on NonPhotorealistic Animation and Rendering (2010)
8. Lee, H., Seo, S., Yoon, K.: Extended papers from npar
2010: Directional texture transfer with edge enhancement. Computers and Graphics 35(1), 81–91 (2011)
9. Lee, T.Y., Yan, C.R.: Feature-based texture synthesis. In:Proc. Computational Science and Its Applications (2005)
10. Perez, P., Gangnet, M., Blake, A.: Poisson image editing.
ACM Transaction on Graphics 22(3), 313–318 (2003)
11. Wang, B., Wang, W., Yang, H., Sun, J.: Eﬃcient
example-based painting and synthesis of 2d directional
texture. IEEE Transactions on Visualization and Computer Graphics 10(3), 266–277 (2004)
12. Xie, X., Tian, F., Seah, H.S.: Feature guided texture synthesis (fgts) for artistic style transfer. In: Proc. Digitalinteractive media in entertainment and arts (2007)
13. Xie, X., Tian, F., Seah, H.S.: Style learning with featurebased texture synthesis. Theoretical and Practical Computer Applications in Entertainment 6(4) (2008)