# A Natural Image Pointillism with Controlled Ellipse Dots

Authors: Dongxiang Chi
Journal: Advances in Multimedia
 (2014)

---

## Abstract

This paper presents an image-based artistic rendering algorithm for the automatic Pointillism style. At first, ellipse dot locations are randomly generated based on a source image; then dot orientations are precalculated with help of a direction map; a saliency map of the source image decides long and short radius of the ellipse dot. At last, the rendering runs layer-by-layer from large size dots to small size dots so as to reserve the detailed parts of the image. Although only ellipse dot shape is adopted, the final Pointillism style performs well because of variable characteristics of the dot.


---

## Body

## 1. Introduction

With rapid development of computer graphics, it diversifies itself gradually into two opposite branches, photorealistic rendering and nonphotorealistic rendering (NPR) in the last two decades. In contrast to the traditional computer graphics, which focuses on photorealism, NPR is inspired by artistic styles such as painting, drawing, technical illustration, and animated cartoons. In one word, NPR focuses on presenting a new world scene in an artistic view. NPR has been applied to movies and video games in the form of cartoon, data visualization, and so forth.Unlike the photorealistic rendering which is engaged mainly by computer graphics scientists, NPR attracts many researchers who love arts and science as a cross-disciplinary research domain. The further NPR developed, the closely the art and science are integrated. One of the most important branch of NPR is “image-based artistic rendering (IBAR),” which deals with the two-dimensional content in photographs and video with artistic stylization techniques [1]. The aim of IBAR is to transfer aesthetic feeling and effective information to people instead of simply describing every details of an image. It is like an artistic filter which turns a natural image into an artistic work. Too many details in the natural image often conceive the most important things. A simplified image makes people grasp the primary information as much as possible. Similarly, IBAR strengthens the effective part of the image in a painting style instead of overlooking its details.According to [1], IBAR is roughly classified into stroke based rendering (SBR), region-based techniques, example-based rendering, image processing, and filtering techniques. (For convenience, stroke, point, brush, and dot will not be discriminated with each other in the following parts of this paper. Different papers used different terminology for their conveniences.) As the most prevalent form of IBAR [1], SBR algorithm renders a 2D image with atomic primitives to simulate a particular style in a local or global manner. The primitives involved in SBR are virtual painting brush [2, 3], tiles [4], points (stippling, Halftoning, and hatching) [5–13], and even images [14–17]. Some also simulated water-color style [18], pencil and ink style [19, 20], line drawing [21, 22], and Pointillism [23–27].As basic primitive, dots are adopted in Pointillism, stippling, and so forth and could be extended in various ways. In stippling, it is important for dots to be placed in a denser or sparser ways to express dark or bright tones with only black dots. When simulating Pointillism, the dots colors are usually restricted to about ten colors and similar strokes to mimic the style. Classic painters like Georges Seurat and Paul Signac or contemporary artists like Yayoi Kusama adopted dots as main elements to transfer their artistic ideas.In fact, dots here can be strong expressive elements with different size, color, shape, texture, and arrangement. It has evolved into a new form in comparison to the classic Pointillism and stippling. In this paper, not like the traditional Pointillism with juxtaposed dots and limited colors and stippling with black dots, we focus on natural images (with characteristics of more colors and intensities) Pointillism with multilayer dots overlapping with each other and where colors are based on the source image itself. We transfer a natural image to Pointillism-like style. The connotation of the term “Pointillism” here progresses further from the classic painting style.The rest of this paper is organized as follows. Section2 reviews some works related to this study. Section 3 gives the proposed method and Section 4, the rendering results and discussion. Section 5 concludes the paper and works in the future.The main contributions of this paper include the following:(a)
a single dot shape with variable characteristics to render a Pointillism style image,
(b)
saliency map-guided dots to enhance the main body of the source image,
(c)
dot orientation with help of a direction map to follow object edges in the source image,
(d)
randomly generated dots positions to increase the variance of the final Pointillism,
(e)
layer-by-layer strategy to keep the detailed information of the source image,
(f)
an equalized saliency map to adjust the long-short-axis ratios to further enhance the painting effect.Figure 1 shows the main points of the proposed method.(a) Source image: cat; (b) saliency map; (c) direction map details of the cat’s nose; (d) Pointillism of the source image with the proposed method,N
=
19000.
(a)
(b)
(c)
(d)


## 2. Related Works

In recent years, it has developed, step by step, from mimicking simple painting techniques to painting style in NPR.Pointillism, being a technique of painting with small, distinct dots of pure color applied in patterns to form an image, is derived from neoimpressionism, which was developed by Georges Seurat and Paul Signac in 1886. Some classic works, like “A Sunday Afternoon on the Island of La Grande Jatte,” “Bathing at Asnieres” by Georges Seurat, “The Windmills at Overschie” by Paul Signac, and so forth, were created. The practice of Pointillism is in sharp contrast to the traditional methods of blending pigments on a palette. Pointillism is analogous to the four-color CMYK printing process used by some color printers and large presses that place dots of Cyan (blue), Magenta (red), Yellow, and Key (black).To simulate Seurat’s painting style, C.-K. Yang and H.-L. Yang, and so forth, went through all accessible Seurat’s paintings and extracted some important features, such as the 22 primitive colors, color juxtaposition, point sizes, and, in particular, the effects of complementary colors and halos [23]. They diffused points with poisson disks and kept different dot sizes and dot distances in background layer and second layer in an empirical manner. Every parameter in this paper was set faithfully according to Seurat’s Pointillism, from dot color to dot size and dot shape. This imitation increases the similarity to Pointillism style to some extent while losing novelty of the proposed method. And the adjustment of the four parameters is inconvenient for users. Further, there is not high level understanding of an input image. This increases the difficulty of automating the simulating process. Comparing to [2], Wu et al. also proposed their method for Seurat’s Pointillism style. They obtained a statistical color model through analyzing the color distributions of Seurat’s paintings. Then, the model was combined with a modified multiclass blue noise sampling [12] to stylize an input image with characteristics of color composition in Seurat’s paintings [28]. They made a quantitative comparison and a subjective user study to verify that their results are closer to Seurat’s painting style than the results of previous approaches [28]. However, the learning process was too time-expensive besides segmentation, sampling, and drawing processes of several hundred to thousand seconds. Hertzmann, and so forth, painted an image with a series of spline brush strokes. Brush strokes are chosen to match colors in a source image. A painting is built up in a series of layers, starting with a rough sketch drawn with a large brush. The sketch is painted over with progressively smaller brushes, but only in areas where the sketch differs from the blurred source image. Thus, visual emphasis in the painting corresponds roughly to the spatial energy present in the source image [2].Paper [7] emulated a coarse to fine painting process customary to styles like impressionism by extending the concept of layers [2]. Instead of orienting each brush stroke largely based on the local gradient estimates as suggested by [2, 24], they oriented the brush strokes along the strongest gradients. Each brush stroke’s color is determined by averaging the color of the pixels underneath it in the source frame [7]. Shiraishi, and so forth [29], introduced a painterly style which generated rectangular brush strokes approximating the local regions of the source image with suitable locations, orientations, and sizes. The resulting image is composited with smaller strokes at the details while its flat regions are painted with larger ones.Sugita and Takahashi [25], presented a pointillistic Halftoning method for color Halftoning on random dots by utilizing a spatial data structure of boundary sampling algorithm. In addition, they implemented complementary color contrast and halo effect according to actual Seurat’s painting steps [25]. But too many parameter settings prevent the method from automated implementation.In this paper, the Pointillism style is expanded to a more general manner which includes dots of different sizes, colors, shapes, and arrangement order. With these effective expressive elements and very simple parameter settings, we can deploy a different painting style extended from Pointillism.


## 3. Proposed Method

The proposed algorithm is based on basic image feature: image color. For a canvas imageΩ and source image I, Ω is rendered with ellipse dot Φ
i
(
P
,
D
,
C
,
O
) which sources from source image I:
(1)
Ω
=
∑
∅
i
(
P
,
D
,
C
,
O
)
,
i
∈
1
,
N
.
N is the total number of dots.For an ellipse dotΦ
i, it is decided by P
i
(
x
,
y
), the location of the dot, D
i
(
r
L
,
r
S
), the two axis radiuses of the ellipse dot, C
i
(
r
,
g
,
b
), the three color components of the dot, and O
i, orientation of the dot. Ω andI are of the same size X
×
Y.When considering Pointillism style, there are several effective factors affecting the appearances of a painting, including size, shape, texture, and color for any one dot and arrangement order for a group of dots.


### 3.1. Shape, Size, Color, and Location

For a classic Pointillism, the shape, size, color and location of dots are very important factors affecting the final appearance of the painting. When simulating a Pointillism style in a digital manner, these elements should not be neglected.


#### 3.1.1. Shape of Dots

To simulate a classic Seurat’s Pointillism, strokes were usually derived from a single stroke pattern [25, 29] or several stroke patterns [23]. Others, like curved spline brush stroke [2], a transparent circle [30], an ellipse with user-defined transverse and conjugate diameters [28], and an antialiased line with a circular brush [2, 7, 24], are all simplified digital components of the Pointillism style.Considering the balance between painting style variance and dot control, an ellipse dot shape is selected in this paper because of its simplicity and two controllable, orthogonal axes: long axisand short axis. An ellipse dot’s shape is decided by its long axis and short axis. A small long-short-axis ratio leads to a circular ellipse and vice versa a large ratio makes a narrow ellipse. Based on the color and gradient values of the source image, the ellipse dots with different axis lengths are placed in different locations of the canvas image. These shape variances and placement strategy strengthen the edges of the source image. The long-short-axis ratio depends on an equalized saliency map. The saliency map is an explicit two-dimensional map that expresses the saliency or conspicuity of objects in the visual environment [31]. For a simulated Pointillism style, there should be always something to be enhanced or strengthened in the source image. Objects inside a source image can be described in a saliency map. The saliency information affects the ellipse shape and size so as to enhance the saliency part. See Section 3.3 for more discussions.


#### 3.1.2. Size of Dots

The dot size in this paper not only is adaptable to source image color but also is affected by gradient value of the source image. This means that the proposed method simulates the Pointillism style instead of faithfully following it. In those areas with slow-changed colors and gradients, large dots are rendered and areas with fast-changed colors and gradients small dots are rendered. In this paper, the ellipse dot size is defined as the long axis radiusr
L and short axis radius r
S which depends on two saliency-related maps, R
L
Map and R
S
Map:
(2)
r
L
=
round
R
max
⁡
-
R
L
Map
x
,
y
×
R
max
⁡
-
R
min
⁡
.The short axis lengthr
S is empirically dependent on r
L and R
S
Map:
(3)
r
S
=
round
r
L
×
1
-
α
×
R
S
Map
x
,
y
.
The R
L
Map and R
S
Map are normalized to range [
0
,
1
]. They are comprised of color and gradient information and decide the long axis radius (dot size) and the short axis radius, respectively. See Section 3.3 for more discussions. α shows how much an ellipse is stretched or reduced. It is empirically set as 0.95. That means if a R
S
Map value of an ellipse location is 1, the relationship between short radius and long radius of the ellipse is r
S
=
round
(
0.05
×
r
L
). r
S ranges in [
0.05
×
r
L
,
r
L
].


#### 3.1.3. Color of Dots

The dot colorC
i is set by the average color of the dot area c
z:
(4)
C
i
=
1
m
∑
m
c
z
,
where m is the pixel number of the zone the dot covers in the source image. For a RGB color image, i
∈
{
1,2
,
3
}. In the canvas image, a dot is rendered with an average color of the source image area covered by the dot. According to (2) and (3), the more salient the area is, the smaller the dot size is. This means that those salient areas are substituted with small dots and the detailed areas are kept deliberately. And to some extent, this method also keeps some characteristics of the Pointillism style.


#### 3.1.4. Location of Dots

For each dot locationP
i
(
x
,
y
), x and y are randomly selected along the columns and rows of the source image and every dot falls on the canvas image plane. For the image contents are not taken into account, a dot location is selected without any guidance.


#### 3.1.5. Total Dot Number

According to the proposed algorithm, the total dot number isN. The larger the N is, the denser the dots are. So, the dot number N affects the ultimate Pointillism effect to some extent. The N is set empirically according to the size of the source image. To diminish the hole-effect of the canvas image Ω in case of small N, Ω may be predefined as a blurred source image or an average color image of the source image.


### 3.2. Orientation and Edge Enhancement with Direction Map

Besides the dot size, color, location, and shape, the dot orientationO
i also plays an important role in the last appearance of the painting style. The dot orientation here is defined as the orientation of the long axis. Generally, a group of ellipse dots with similar orientation along their long axis enhance an edge of a source image. The edges in a source image provide important information to readers. On enhancing edges, [2, 24] oriented each brush stroke largely based on the estimated local gradient. Paper [7] oriented the brush strokes along the strongest gradients however. Paper [29] introduced a painterly style which is composite with smaller strokes at the details while its flat regions are painted with larger ones to enhance the edges (details).In this paper, a direction map is constructed to decide the orientation of every ellipse dot. The dot always places its long axis along the orientation of the direction map of the source image.According to Kang et al. [21, 32], for a given image I
(
x
) in this paper, the vector field ofedge tangent flow, that is, a direction map t
(
x
), is used as a guiding map of the ellipse orientations.The direction map is defined as an iterative process:(5)
t
x
=
1
k
∬
Ω
μ
e
∅
x
,
y
t
y
w
s
x
,
y
w
m
x
,
y
w
d
x
,
y
d
y
,
where Ω
μ
e
(
x
,
y
) denotes the kernel of radius μ
e at pixel x, and k is the vector normalizing term. The tangent vector t
(
·
) is assumed to be π
-periodic. For the spatial weight function w
s, we use a box filter of radius μ
e:
(6)
w
s
x
,
y
=
1
if
x
-
y
<
μ
e
0
otherwise
.The magnitude weight functionw
m is defined to be a monotonically increasing function:
(7)
w
m
=
g
^
y
-
g
^
x
+
1
2
.
This ensures the preservation of the dominant edge directions.The direction weight functionw
d is defined to promote smoothing among similar orientations:
(8)
w
d
x
,
y
=
t
x
·
t
y
,
where t
(
z
) denotes the normalized tangent vector at z. This weight function increases as the two vectors align closely.For tight alignment of vectors, we temporarily reverse the direction oft
(
y
) using the sign function ∅
(
x
,
y
)
∈
1
,
-
1, in case that θ is bigger than π
/
2:
(9)
∅
x
,
y
=
1
if
t
x
·
t
y
>
0
-
1
otherwise
.The initial flow map, denoted ast
0
(
x
), is obtained by taking perpendicular vectors (in the counterclockwise sense) from the initial gradient map g
0
(
x
) of the input imageI. t
0
(
x
) is then normalized before use. The initial gradient map g
0
(
x
) is computed by employing a Sobel operator. The input image may be optionally Gaussian-blurred before gradient computation. Figure 2(b) shows direction map with 2 iterations. The map preserves edge directions well around important features while keeping them smooth elsewhere. For any dot position x
(
x
,
y
) and a direction map t, the orientation O
i of the dot Φ
i is t
(
x
). We note from the maps that the edges of the source images are kept and enhanced. Some “weak” flows are adjusted to follow the strong flows.One source image, its direction map, saliency map, and Pointillism ((a), (b), (c), and (d)).
(a)
(b)
(c)
(d)


### 3.3. Arrangement: Layer-by-Layer according to Saliency Map

With the attributes of the dot shape, size, color, location, and orientation, a Pointillism style image is ready to be rendered. To keep the detailed areas uncovered, a layer-by-layer-strategy is adopted. According to the dot size, small size dots are painted after the large size dots. That is to say, larger dots are always below smaller dots. For the smaller dots always show the detailed information in salient regions and large dots cover nonsalient regions; this layer-by-layer-strategy makes the detailed parts stand out of the source image. Also, readers pay more attention to those salient areas than to those nonsalient areas. So, a saliency map here can be constructed to decide dot size. Usually, the salient areas are those areas where there exist color contrasts, intensity changes, or edges. These salient areas provide finer details and important contents which attract the readers’ attention more than the other areas do.A saliency map is a salient metric of an image. A high salient value attracts readers’ attention more than a small salient value does. The saliency map helps the proposed rendering Pointillism to enhance the salient parts. To deal with a source image where no a priori knowledge available, Itti et al. introduced their bottom-up visual attention model [31] inspired by the behavior and the neuronal architecture of the early primate visual system. In their method, multiscale image feature maps of color, intensity, and orientation are extracted, and local spatial contrast is estimated for each feature at each location, providing a separate conspicuity map for each feature. These maps are combined to a single topographical saliency map that guides the attention focus in a bottom-up manner. In this paper, a modified saliency map by [33] is applied to the source image. The map is comprised of intensity, color, and edge information of five pyramid images of the source image.At first, intensity imageI is obtained through r, g, and b components of the source image. I is calculated as I
=
r
+
g
+
b
/
3. Those pixels with intensity less than 10% of its maximum over the entire image have zero r, g, and b values, because hue variations are not perceivable at very low luminance (and hence not salient). R, G, B, and Y are formed in this way: R
=
r
-
(
g
+
b
)
/
2; G
=
g
-
(
r
+
b
)
/
2; B
=
b
-
(
g
+
r
)
/
2; Y
=
(
r
+
g
)
/
2
-
r
-
g
/
2
-
b if (
r
+
g
)
/
2
-
r
-
g
/
2
-
b
>
0, else Y
=
0. Five groups of Gaussian pyramid level I
(
σ
), R
(
σ
), G
(
σ
), B
(
σ
), and Y
(
σ
) are generated from I, R, G, B, and Y, respectively, where σ is Gaussian pyramid level index, σ
∈
[
1,2
,
3,4
,
5
]. Then, according to a contrast-based saliency map [34], the center-neighborhood distances of I, R, G, B, and Y, which include the luminant and chromatic saliency information, are calculated within a 3
×
3 window by Euclidean distance. As for orientations, a different approach of four line detection masks R0, R45, R90, and R135 [35] is proposed to filter the intensity image I to simplify the calculation of Itti-Koch’s Gabor filters. Each level of intensity image is filtered by four direction masks; contrast saliency maps are calculated and combined to be an orientation saliency map O
(
σ
). At last, all saliency maps are resized to the original image size (level 0) and combined together to be a saliency map. To enhance the high salient region and restrain the low salient region, the saliency map is square power transformed:
(10)
S
=
I
(
σ
)
+
R
(
σ
)
+
G
(
σ
)
+
B
(
σ
)
+
Y
(
σ
)
+
O
(
σ
)
6
2
.Experiments prove the proposed method to be efficient and effective in several aspects. At first, the luminance, chrominance, and orientation factors are taken into account to find the salient region; secondly, the simplified orientation calculation does not decrease the oriental salience; thirdly, the power transformation makes the salient region more significant without changing the smooth tone of the saliency map. Figure2(c) shows the saliency maps of source images. Every salient area, like color differences, gradient changes, and intensity variances, gives support to the saliency map. In this paper, a coarse to fine strategy is proposed that the dots are rendered from large dots to small dots. Further, the dot size (long axis length) is decided by the saliency map in an inverse proportional manner as (2) shows. R
L
Map here is derived from the normalized saliency map S:
(11)
R
L
Map
x
=
1
n
∑
Ω
μ
s
S
.
The R
L
Map value of a dot position x is mean value of a block zone Ω with block size μ
s in the normalized saliency map S. For the dot size is relative to the saliency of the source image, smaller dots mean larger salient values of the source image and are thus on the top of larger dots which are relative to the smaller salient values. Figure 3 shows the dot-placement-strategy from large dots in the nonsalient areas to small dots in salient areas, one by one, layer-by-layer.Figure 3
Dot-placement strategy and rendering process: from large dots to small dots, from nonsalient areas to salient areas, one by one, layer-by-layer. (a) Source image; (b) saliency image; (c) final Pointillism with 10000 dots; (c1)~(c10) rendering process from 100 dots to 1000 dots, 2000 dots until 9000 dots, taking the saliency image as canvas image so as to check the placement strategy. In real rendering process, the blurred source image is taken as a canvas image.


### 3.4. Equalized Standard Deviation Saliency Map to Decide the Short Axis of an Ellipse

Once we have decided the long axis length of the ellipse dot, the short axis length and orientation are the last two things to be taken into account. To enhance the edges of the painting image, the long axis of the ellipse has to follow its orientation (Section3.2). At the same time, the short axis should also be variable based on the saliency map. As we have discussed before, the ellipse dots help to enhance the edges of the source image. Compared to an ellipse with small long-short-axis ratio (near to 1), the ellipse with large ratio (larger than 1) stretches itself to a long ellipse to follow the edges. This shape deformation achieves a further enhancement as the orientation does. The shape deformation depends on saliency variance with block zone Ω within block size μ
s. A high variance value means there exsited a sharp edge while a low variance value means slow-changed edges. So the standard deviation map R
S
Map
x
0 is defined as:
(12)
R
S
Map
x
0
=
STD
Ω
μ
s
S
.TheR
S
Map
0 value of a dot position x is a standard deviation value of zone Ω
μ
s in the normalized saliency map. As we can see from the histogram of the block zones deviation within saliency map (Figure 4(b)), the deviation values do not distribute in a balanced way. Thus the high and low deviation values are not distributed evenly. This makes these dots to be either stretched too much or not stretched. That is to say, these unbalanced deviation values lead to stretched dots or circular dots and there are not “midlevel” dots. Nearly all dots tend to be too circular to describe edges in most areas in our experiments because of too many low deviation values (Figure 4(d)).Comparisons of two Pointillisms with equalized/unequalizedR
S
Map, N
=
40000.
(a)
(b)
(c)
(d)Apparently, the unbalanced deviation map has to be balanced. In this paper, it is equalized with a LOGSIG function, that is, to make the distribution contour fit a LOGSIG function:(13)
R
S
Map
x
=
HISTEQ
R
S
Map
x
0
,
LOGSIG
n
.The target function is(14)
LOGSIG
n
=
1
1
+
e
-
n
,
n
=
0
:
bin
.The bin is empirically set and up-rounded to be an integer number:(15)
bin
=
N
1000
.
N is the total dot number.As (13) shows, the new R
S
Map value of a dot position x is an equalized standard deviation value of zone Ω
μ
s in the normalized saliency map. Figure 4(a) shows its equalized histogram of Figure 4(b), which are histograms of R
S
Map. This equalization generates R
S
Map which decides short-axis-length of the ellipse dot. The Pointillism (Figure 4(c)) with equalized R
S
Map is rendered with ellipses with different long-short-axis ratios, while the long-short-axis ratios of ellipses in the Pointillism (Figure 4(d)) with unequalized R
S
Map are nearly similar to each other. Apparently, the equalization to the R
S
Map works. The equalized R
S
Map makes the dots’ short axis radii distribute so flatly that dots with different long-short-axis ratios render the Pointillism more clearly compared to the unequalized version.Besides several main parameters discussed above, some other parameters are set empirically in Table1.Table 1
Pointillism settings.
Parameters
Settings
Total dot number
N
=
X
×
Y/20, constrained from 10000 to 40000
Minimal dot size (pixels)
R
min
⁡
= R
max
⁡/10, minimal to 3
Maximal dot size (pixels)
R
max
⁡
=
e
×
min
⁡
(
X
,
Y
), maximal to 99
e
0.05~0.1,e controls the maximal dot size according to the source image size
Block radius (pixels) for direction map
μ
e = 4
Block size (pixels) when deciding dot size with saliency values
μ
s
=
2
×
R
min
⁡ + 1, constrained from 9 to 21With these settings, the proposed method is implemented with MATLAB. We will show the aforementioned and several other natural source images and their Pointillism in the next section.


## 3.1. Shape, Size, Color, and Location

For a classic Pointillism, the shape, size, color and location of dots are very important factors affecting the final appearance of the painting. When simulating a Pointillism style in a digital manner, these elements should not be neglected.


### 3.1.1. Shape of Dots

To simulate a classic Seurat’s Pointillism, strokes were usually derived from a single stroke pattern [25, 29] or several stroke patterns [23]. Others, like curved spline brush stroke [2], a transparent circle [30], an ellipse with user-defined transverse and conjugate diameters [28], and an antialiased line with a circular brush [2, 7, 24], are all simplified digital components of the Pointillism style.Considering the balance between painting style variance and dot control, an ellipse dot shape is selected in this paper because of its simplicity and two controllable, orthogonal axes: long axisand short axis. An ellipse dot’s shape is decided by its long axis and short axis. A small long-short-axis ratio leads to a circular ellipse and vice versa a large ratio makes a narrow ellipse. Based on the color and gradient values of the source image, the ellipse dots with different axis lengths are placed in different locations of the canvas image. These shape variances and placement strategy strengthen the edges of the source image. The long-short-axis ratio depends on an equalized saliency map. The saliency map is an explicit two-dimensional map that expresses the saliency or conspicuity of objects in the visual environment [31]. For a simulated Pointillism style, there should be always something to be enhanced or strengthened in the source image. Objects inside a source image can be described in a saliency map. The saliency information affects the ellipse shape and size so as to enhance the saliency part. See Section 3.3 for more discussions.


### 3.1.2. Size of Dots

The dot size in this paper not only is adaptable to source image color but also is affected by gradient value of the source image. This means that the proposed method simulates the Pointillism style instead of faithfully following it. In those areas with slow-changed colors and gradients, large dots are rendered and areas with fast-changed colors and gradients small dots are rendered. In this paper, the ellipse dot size is defined as the long axis radiusr
L and short axis radius r
S which depends on two saliency-related maps, R
L
Map and R
S
Map:
(2)
r
L
=
round
R
max
⁡
-
R
L
Map
x
,
y
×
R
max
⁡
-
R
min
⁡
.The short axis lengthr
S is empirically dependent on r
L and R
S
Map:
(3)
r
S
=
round
r
L
×
1
-
α
×
R
S
Map
x
,
y
.
The R
L
Map and R
S
Map are normalized to range [
0
,
1
]. They are comprised of color and gradient information and decide the long axis radius (dot size) and the short axis radius, respectively. See Section 3.3 for more discussions. α shows how much an ellipse is stretched or reduced. It is empirically set as 0.95. That means if a R
S
Map value of an ellipse location is 1, the relationship between short radius and long radius of the ellipse is r
S
=
round
(
0.05
×
r
L
). r
S ranges in [
0.05
×
r
L
,
r
L
].


### 3.1.3. Color of Dots

The dot colorC
i is set by the average color of the dot area c
z:
(4)
C
i
=
1
m
∑
m
c
z
,
where m is the pixel number of the zone the dot covers in the source image. For a RGB color image, i
∈
{
1,2
,
3
}. In the canvas image, a dot is rendered with an average color of the source image area covered by the dot. According to (2) and (3), the more salient the area is, the smaller the dot size is. This means that those salient areas are substituted with small dots and the detailed areas are kept deliberately. And to some extent, this method also keeps some characteristics of the Pointillism style.


### 3.1.4. Location of Dots

For each dot locationP
i
(
x
,
y
), x and y are randomly selected along the columns and rows of the source image and every dot falls on the canvas image plane. For the image contents are not taken into account, a dot location is selected without any guidance.


### 3.1.5. Total Dot Number

According to the proposed algorithm, the total dot number isN. The larger the N is, the denser the dots are. So, the dot number N affects the ultimate Pointillism effect to some extent. The N is set empirically according to the size of the source image. To diminish the hole-effect of the canvas image Ω in case of small N, Ω may be predefined as a blurred source image or an average color image of the source image.


## 3.1.1. Shape of Dots

To simulate a classic Seurat’s Pointillism, strokes were usually derived from a single stroke pattern [25, 29] or several stroke patterns [23]. Others, like curved spline brush stroke [2], a transparent circle [30], an ellipse with user-defined transverse and conjugate diameters [28], and an antialiased line with a circular brush [2, 7, 24], are all simplified digital components of the Pointillism style.Considering the balance between painting style variance and dot control, an ellipse dot shape is selected in this paper because of its simplicity and two controllable, orthogonal axes: long axisand short axis. An ellipse dot’s shape is decided by its long axis and short axis. A small long-short-axis ratio leads to a circular ellipse and vice versa a large ratio makes a narrow ellipse. Based on the color and gradient values of the source image, the ellipse dots with different axis lengths are placed in different locations of the canvas image. These shape variances and placement strategy strengthen the edges of the source image. The long-short-axis ratio depends on an equalized saliency map. The saliency map is an explicit two-dimensional map that expresses the saliency or conspicuity of objects in the visual environment [31]. For a simulated Pointillism style, there should be always something to be enhanced or strengthened in the source image. Objects inside a source image can be described in a saliency map. The saliency information affects the ellipse shape and size so as to enhance the saliency part. See Section 3.3 for more discussions.


## 3.1.2. Size of Dots

The dot size in this paper not only is adaptable to source image color but also is affected by gradient value of the source image. This means that the proposed method simulates the Pointillism style instead of faithfully following it. In those areas with slow-changed colors and gradients, large dots are rendered and areas with fast-changed colors and gradients small dots are rendered. In this paper, the ellipse dot size is defined as the long axis radiusr
L and short axis radius r
S which depends on two saliency-related maps, R
L
Map and R
S
Map:
(2)
r
L
=
round
R
max
⁡
-
R
L
Map
x
,
y
×
R
max
⁡
-
R
min
⁡
.The short axis lengthr
S is empirically dependent on r
L and R
S
Map:
(3)
r
S
=
round
r
L
×
1
-
α
×
R
S
Map
x
,
y
.
The R
L
Map and R
S
Map are normalized to range [
0
,
1
]. They are comprised of color and gradient information and decide the long axis radius (dot size) and the short axis radius, respectively. See Section 3.3 for more discussions. α shows how much an ellipse is stretched or reduced. It is empirically set as 0.95. That means if a R
S
Map value of an ellipse location is 1, the relationship between short radius and long radius of the ellipse is r
S
=
round
(
0.05
×
r
L
). r
S ranges in [
0.05
×
r
L
,
r
L
].


## 3.1.3. Color of Dots

The dot colorC
i is set by the average color of the dot area c
z:
(4)
C
i
=
1
m
∑
m
c
z
,
where m is the pixel number of the zone the dot covers in the source image. For a RGB color image, i
∈
{
1,2
,
3
}. In the canvas image, a dot is rendered with an average color of the source image area covered by the dot. According to (2) and (3), the more salient the area is, the smaller the dot size is. This means that those salient areas are substituted with small dots and the detailed areas are kept deliberately. And to some extent, this method also keeps some characteristics of the Pointillism style.


## 3.1.4. Location of Dots

For each dot locationP
i
(
x
,
y
), x and y are randomly selected along the columns and rows of the source image and every dot falls on the canvas image plane. For the image contents are not taken into account, a dot location is selected without any guidance.


## 3.1.5. Total Dot Number

According to the proposed algorithm, the total dot number isN. The larger the N is, the denser the dots are. So, the dot number N affects the ultimate Pointillism effect to some extent. The N is set empirically according to the size of the source image. To diminish the hole-effect of the canvas image Ω in case of small N, Ω may be predefined as a blurred source image or an average color image of the source image.


## 3.2. Orientation and Edge Enhancement with Direction Map

Besides the dot size, color, location, and shape, the dot orientationO
i also plays an important role in the last appearance of the painting style. The dot orientation here is defined as the orientation of the long axis. Generally, a group of ellipse dots with similar orientation along their long axis enhance an edge of a source image. The edges in a source image provide important information to readers. On enhancing edges, [2, 24] oriented each brush stroke largely based on the estimated local gradient. Paper [7] oriented the brush strokes along the strongest gradients however. Paper [29] introduced a painterly style which is composite with smaller strokes at the details while its flat regions are painted with larger ones to enhance the edges (details).In this paper, a direction map is constructed to decide the orientation of every ellipse dot. The dot always places its long axis along the orientation of the direction map of the source image.According to Kang et al. [21, 32], for a given image I
(
x
) in this paper, the vector field ofedge tangent flow, that is, a direction map t
(
x
), is used as a guiding map of the ellipse orientations.The direction map is defined as an iterative process:(5)
t
x
=
1
k
∬
Ω
μ
e
∅
x
,
y
t
y
w
s
x
,
y
w
m
x
,
y
w
d
x
,
y
d
y
,
where Ω
μ
e
(
x
,
y
) denotes the kernel of radius μ
e at pixel x, and k is the vector normalizing term. The tangent vector t
(
·
) is assumed to be π
-periodic. For the spatial weight function w
s, we use a box filter of radius μ
e:
(6)
w
s
x
,
y
=
1
if
x
-
y
<
μ
e
0
otherwise
.The magnitude weight functionw
m is defined to be a monotonically increasing function:
(7)
w
m
=
g
^
y
-
g
^
x
+
1
2
.
This ensures the preservation of the dominant edge directions.The direction weight functionw
d is defined to promote smoothing among similar orientations:
(8)
w
d
x
,
y
=
t
x
·
t
y
,
where t
(
z
) denotes the normalized tangent vector at z. This weight function increases as the two vectors align closely.For tight alignment of vectors, we temporarily reverse the direction oft
(
y
) using the sign function ∅
(
x
,
y
)
∈
1
,
-
1, in case that θ is bigger than π
/
2:
(9)
∅
x
,
y
=
1
if
t
x
·
t
y
>
0
-
1
otherwise
.The initial flow map, denoted ast
0
(
x
), is obtained by taking perpendicular vectors (in the counterclockwise sense) from the initial gradient map g
0
(
x
) of the input imageI. t
0
(
x
) is then normalized before use. The initial gradient map g
0
(
x
) is computed by employing a Sobel operator. The input image may be optionally Gaussian-blurred before gradient computation. Figure 2(b) shows direction map with 2 iterations. The map preserves edge directions well around important features while keeping them smooth elsewhere. For any dot position x
(
x
,
y
) and a direction map t, the orientation O
i of the dot Φ
i is t
(
x
). We note from the maps that the edges of the source images are kept and enhanced. Some “weak” flows are adjusted to follow the strong flows.One source image, its direction map, saliency map, and Pointillism ((a), (b), (c), and (d)).
(a)
(b)
(c)
(d)


## 3.3. Arrangement: Layer-by-Layer according to Saliency Map

With the attributes of the dot shape, size, color, location, and orientation, a Pointillism style image is ready to be rendered. To keep the detailed areas uncovered, a layer-by-layer-strategy is adopted. According to the dot size, small size dots are painted after the large size dots. That is to say, larger dots are always below smaller dots. For the smaller dots always show the detailed information in salient regions and large dots cover nonsalient regions; this layer-by-layer-strategy makes the detailed parts stand out of the source image. Also, readers pay more attention to those salient areas than to those nonsalient areas. So, a saliency map here can be constructed to decide dot size. Usually, the salient areas are those areas where there exist color contrasts, intensity changes, or edges. These salient areas provide finer details and important contents which attract the readers’ attention more than the other areas do.A saliency map is a salient metric of an image. A high salient value attracts readers’ attention more than a small salient value does. The saliency map helps the proposed rendering Pointillism to enhance the salient parts. To deal with a source image where no a priori knowledge available, Itti et al. introduced their bottom-up visual attention model [31] inspired by the behavior and the neuronal architecture of the early primate visual system. In their method, multiscale image feature maps of color, intensity, and orientation are extracted, and local spatial contrast is estimated for each feature at each location, providing a separate conspicuity map for each feature. These maps are combined to a single topographical saliency map that guides the attention focus in a bottom-up manner. In this paper, a modified saliency map by [33] is applied to the source image. The map is comprised of intensity, color, and edge information of five pyramid images of the source image.At first, intensity imageI is obtained through r, g, and b components of the source image. I is calculated as I
=
r
+
g
+
b
/
3. Those pixels with intensity less than 10% of its maximum over the entire image have zero r, g, and b values, because hue variations are not perceivable at very low luminance (and hence not salient). R, G, B, and Y are formed in this way: R
=
r
-
(
g
+
b
)
/
2; G
=
g
-
(
r
+
b
)
/
2; B
=
b
-
(
g
+
r
)
/
2; Y
=
(
r
+
g
)
/
2
-
r
-
g
/
2
-
b if (
r
+
g
)
/
2
-
r
-
g
/
2
-
b
>
0, else Y
=
0. Five groups of Gaussian pyramid level I
(
σ
), R
(
σ
), G
(
σ
), B
(
σ
), and Y
(
σ
) are generated from I, R, G, B, and Y, respectively, where σ is Gaussian pyramid level index, σ
∈
[
1,2
,
3,4
,
5
]. Then, according to a contrast-based saliency map [34], the center-neighborhood distances of I, R, G, B, and Y, which include the luminant and chromatic saliency information, are calculated within a 3
×
3 window by Euclidean distance. As for orientations, a different approach of four line detection masks R0, R45, R90, and R135 [35] is proposed to filter the intensity image I to simplify the calculation of Itti-Koch’s Gabor filters. Each level of intensity image is filtered by four direction masks; contrast saliency maps are calculated and combined to be an orientation saliency map O
(
σ
). At last, all saliency maps are resized to the original image size (level 0) and combined together to be a saliency map. To enhance the high salient region and restrain the low salient region, the saliency map is square power transformed:
(10)
S
=
I
(
σ
)
+
R
(
σ
)
+
G
(
σ
)
+
B
(
σ
)
+
Y
(
σ
)
+
O
(
σ
)
6
2
.Experiments prove the proposed method to be efficient and effective in several aspects. At first, the luminance, chrominance, and orientation factors are taken into account to find the salient region; secondly, the simplified orientation calculation does not decrease the oriental salience; thirdly, the power transformation makes the salient region more significant without changing the smooth tone of the saliency map. Figure2(c) shows the saliency maps of source images. Every salient area, like color differences, gradient changes, and intensity variances, gives support to the saliency map. In this paper, a coarse to fine strategy is proposed that the dots are rendered from large dots to small dots. Further, the dot size (long axis length) is decided by the saliency map in an inverse proportional manner as (2) shows. R
L
Map here is derived from the normalized saliency map S:
(11)
R
L
Map
x
=
1
n
∑
Ω
μ
s
S
.
The R
L
Map value of a dot position x is mean value of a block zone Ω with block size μ
s in the normalized saliency map S. For the dot size is relative to the saliency of the source image, smaller dots mean larger salient values of the source image and are thus on the top of larger dots which are relative to the smaller salient values. Figure 3 shows the dot-placement-strategy from large dots in the nonsalient areas to small dots in salient areas, one by one, layer-by-layer.Figure 3
Dot-placement strategy and rendering process: from large dots to small dots, from nonsalient areas to salient areas, one by one, layer-by-layer. (a) Source image; (b) saliency image; (c) final Pointillism with 10000 dots; (c1)~(c10) rendering process from 100 dots to 1000 dots, 2000 dots until 9000 dots, taking the saliency image as canvas image so as to check the placement strategy. In real rendering process, the blurred source image is taken as a canvas image.


## 3.4. Equalized Standard Deviation Saliency Map to Decide the Short Axis of an Ellipse

Once we have decided the long axis length of the ellipse dot, the short axis length and orientation are the last two things to be taken into account. To enhance the edges of the painting image, the long axis of the ellipse has to follow its orientation (Section3.2). At the same time, the short axis should also be variable based on the saliency map. As we have discussed before, the ellipse dots help to enhance the edges of the source image. Compared to an ellipse with small long-short-axis ratio (near to 1), the ellipse with large ratio (larger than 1) stretches itself to a long ellipse to follow the edges. This shape deformation achieves a further enhancement as the orientation does. The shape deformation depends on saliency variance with block zone Ω within block size μ
s. A high variance value means there exsited a sharp edge while a low variance value means slow-changed edges. So the standard deviation map R
S
Map
x
0 is defined as:
(12)
R
S
Map
x
0
=
STD
Ω
μ
s
S
.TheR
S
Map
0 value of a dot position x is a standard deviation value of zone Ω
μ
s in the normalized saliency map. As we can see from the histogram of the block zones deviation within saliency map (Figure 4(b)), the deviation values do not distribute in a balanced way. Thus the high and low deviation values are not distributed evenly. This makes these dots to be either stretched too much or not stretched. That is to say, these unbalanced deviation values lead to stretched dots or circular dots and there are not “midlevel” dots. Nearly all dots tend to be too circular to describe edges in most areas in our experiments because of too many low deviation values (Figure 4(d)).Comparisons of two Pointillisms with equalized/unequalizedR
S
Map, N
=
40000.
(a)
(b)
(c)
(d)Apparently, the unbalanced deviation map has to be balanced. In this paper, it is equalized with a LOGSIG function, that is, to make the distribution contour fit a LOGSIG function:(13)
R
S
Map
x
=
HISTEQ
R
S
Map
x
0
,
LOGSIG
n
.The target function is(14)
LOGSIG
n
=
1
1
+
e
-
n
,
n
=
0
:
bin
.The bin is empirically set and up-rounded to be an integer number:(15)
bin
=
N
1000
.
N is the total dot number.As (13) shows, the new R
S
Map value of a dot position x is an equalized standard deviation value of zone Ω
μ
s in the normalized saliency map. Figure 4(a) shows its equalized histogram of Figure 4(b), which are histograms of R
S
Map. This equalization generates R
S
Map which decides short-axis-length of the ellipse dot. The Pointillism (Figure 4(c)) with equalized R
S
Map is rendered with ellipses with different long-short-axis ratios, while the long-short-axis ratios of ellipses in the Pointillism (Figure 4(d)) with unequalized R
S
Map are nearly similar to each other. Apparently, the equalization to the R
S
Map works. The equalized R
S
Map makes the dots’ short axis radii distribute so flatly that dots with different long-short-axis ratios render the Pointillism more clearly compared to the unequalized version.Besides several main parameters discussed above, some other parameters are set empirically in Table1.Table 1
Pointillism settings.
Parameters
Settings
Total dot number
N
=
X
×
Y/20, constrained from 10000 to 40000
Minimal dot size (pixels)
R
min
⁡
= R
max
⁡/10, minimal to 3
Maximal dot size (pixels)
R
max
⁡
=
e
×
min
⁡
(
X
,
Y
), maximal to 99
e
0.05~0.1,e controls the maximal dot size according to the source image size
Block radius (pixels) for direction map
μ
e = 4
Block size (pixels) when deciding dot size with saliency values
μ
s
=
2
×
R
min
⁡ + 1, constrained from 9 to 21With these settings, the proposed method is implemented with MATLAB. We will show the aforementioned and several other natural source images and their Pointillism in the next section.


## 4. Results and Discussion

### 4.1. Pointillism Style

Pointillism results of source images from Figure1(a) are shown in Figure 1(d). Figure 2(d) shows the results of the aforementioned source images of Figure 2(a). Figures 5, 6, and 7 show several other natural images like portraits, flowers, and birds and their Pointillism with different dot-size settings.Two portrait images’ Pointillisms.N
=
40000 (a) and N
=
39000 (b). ((a1), (b1)) Source image with size: 1024
×
1024 and 768
×
1024; ((a2), (a3)) with maximal/minimal dot size: 103/11 and 51/5; ((b2), (b3)) with maximal/minimal dot size: 77/9 and 39/5.
(a)
(b)Flower and bird images’ Pointillism.N
=
21000 (a) and N
=
39000 (b). ((a1), (b1)) Source image with size: 1024
×
425 and 1024
×
777; ((a2), (a3)) with maximal/minimal dot size: 43/7 and 21/3; ((b2), (b3)) with maximal/minimal dot size: 79/9 and 39/5.
(a)
(b)((a), (c)) Pointillism without direction map and with saliency map to decide the dot size; ((b), (d)) Pointillism without direction map and with equalized standard deviation map of the saliency map to decide the dot size. All other settings are as same as those in Figures2(d) and 6(a3).
(a)
(b)
(c)
(d)Several factors, including dot size, saliency map, and direction map, should be taken into account when considering the whole generative process.(
1
) Dot size: a rough observation could disclose that the dot size, especially minimal dot size, is a primary setting affecting the style. The smaller the minimal dot size is, the clearer the Pointillism image is. That is of course because a smaller size dot zone contains average color information in a small zone while a larger dot zone contains average color information in a large zone. If the minimal dot size shrinks to one pixel, the target image becomes the source image itself. So, it should be noted that there is a balance between the minimal dot size and the Pointillism style. As we have declared in Section 3.1.2, the long axis radius is defined as dot size. At the same time, the short axis radius also plays important roles in generating the style. Distributions of short and long radii of dots size corresponding to different dot number (in Figure 9) are shown in Figure 10. On one hand, the flat distribution of the short radius benefits from the equalized R
S
Map. On the other hand, the unbalanced long radius distribution keep the target image covered as much as possible if the number of dots is fixed (further discussion in later part).(
2
) Direction map: further, a close research shows that the detailed parts in the source image are kept deliberately with help of the direction map. For example, the hairs in Figure 2(d) seemed to be rendered in a random direction. Comparison of its source image in Figure 2(a) shows that every hair on the eagle’s head has its own direction. The detailed random hairs in the Pointillism in fact originate from the source image; that is, the direction information is kept in the generated Pointillism. Comparisons between source images of Figures 6(a1) and 6(b1) and target images of Figures 6(a3) and 6(b3) also give similar conclusion. Fluffs on the flower and fine feather in bird’s neck keep the detailed information while generating Pointillism.Several target images without the help of direction map are shown in Figure7. It shows two different conditions: at first, Figures 7(a) and 7(c) give Pointillism whose dot size is decided by a saliency map; secondly, Figures 7(b) and 7(d) show Pointillism whose dot size is decided by an equalized standard deviation map of the saliency map. All other settings are the same as those in Figures 2(d) and 6(a3). With the saliency map deciding the dot size, the generated target images show Pointillism style with many similar size dots (for Figures 7(a) and 7(c)) or non-similar-size dots (for Figures 7(b) and 7(d)). Those non-similar-size dots come from an equalized standard deviation map of the saliency map while the similar-size dots from the saliency map itself. Compared to those target images in Figures 2(d) and 6(a3), these target images without the guidance of direction map could not keep clear edges. For example in Figure 7(a), the head of the eagle shows ghost edges because of the similar-size dots around the head and every dot does not rotate itself to enhance the edge of the head without the direction map. In Figure 7(b) however, the edge of the eagle head is clearer than that in Figure 7(a) because of the equalized standard deviation map. All dots have a flat distribution. So the edge of the head is rendered with smaller dots which keep the detailed head edges compared to Figure 7(a). A further observation around the edge of the eagle head and its eye in Figure 7(b) indicates that there are many holes around these edges compared to Figure 2(d).(
3
) Saliency map: a saliency map is comprised of several factors like gradients, color differences, and edge information. So, it is reasonable for a saliency map to decide the dot size of a source image.Several target images without the help of direction map and saliency map are shown in Figure8. It shows two different conditions: at first, Figures 8(a) and 8(c) give Pointillism whose dot size is decided by an edge map (source image filtered by vertical and horizontal Sobel operators); secondly, Figures 8(b) and 8(d) show Pointillism whose dot size is decided by an equalized standard deviation map of the edge map. All other settings are the same as those in Figures 2(d) and 6(a3). Compared to target images in Figures 2(d), 1(d), and 6(a3), they are not acceptable at all considering the prominent dots, holes, and blurred edges. Even if compared to target images in Figure 7, those target images also give even poor results. For example in Figure 8(a), only edge-map-guided dot size makes the target image show a rough look of the source image, while in Figure 8(b), an equalized standard deviation map of the edge map makes the target image look well with a flat-dot-size distribution. But the holes and ghost edges around the target image could not be ignored.((a), (c)) Pointillism without direction map and with edge map to decide the dot size; ((b), (d)) Pointillism without direction map and with equalized standard deviation map of the edge map to decide the dot size. All other settings are as same as those in Figures2(d) and 6(a3).
(a)
(b)
(c)
(d)Figure 9
Pointillism results with different dot number settings from left to right, top to down: 500, 1000, 5000, 10000, 20000, 30000, and 40000 dots. The maximal and minimal dot size are 51 and 5, respectively.Dot radius distributions (left column: long radius; right column: short radius) from 500, 1000, 5000, and 10000 to 40000 dots in Figure9.
(a)
(b)(
4
) Number of dots: each dot occupies its own zone and overlapped with each other if there are enough dots. We compare different dot number settings in Figure 9. The maximal dot size and minimal dot size are set to 51 and 5. Intuitively, enough dots cover the whole target image. In Figure 9, we can see that it is enough for only 5000 dots to cover most part of the target image. The coverage increases rapidly from 500 dots to 5000 dots. Increasing dot number from 10000 to 40000 does not give a considerable coverage increase, however. There are minor holes in the target image even with 40000 dots. In the proposed method, this problem is avoided by taking a blurred source image as the canvas image (see Section 3.1.5).


### 4.2. Comparisons

As we had mentioned in the former part, we expand the Pointillism style to a more general manner. That is to say, the proposed method has something in common with a typical Pointillism. For example, similar dot shape is adopted as rendering element and dot color is taken from the averaged source image color. The proposed method “borrows” ideas from the process of simulating Pointillism and tries to expand to a new field. The aim of the proposed method is to expand the Pointillism instead of simulating the style. Figure11 demonstrates the comparisons of the proposed method and Wu’s method [28]. We adopt similar dot size as shown in Wu’s results. However, these two results have different aim and show different style. Wu’s style is more close to Seurat’s Pointillism in dot size and placement strategy, while ours consisted of different dot size and shape and layer-by-layer placement strategy. It is not easy for the proposed method to be judged or compared with other simulated results.Comparisons between our method and Wu’s method. The source images are in the first row; the second row and third row are Wu’s results and our results. The maximal and minimal dot sizes of our method are 13 and 5, respectively, which are similar to Wu’s dot size.
(a)
(b)
(c)


### 4.3. Computational Complexity

Computational complexity is a key characteristic in assessing a method. The computational complexity of the proposed method mainly depends on calculations of the direction map, the saliency map, and the dot placement process, which respond toO
(
X
·
Y
·
(
2
μ
e
-
1
)
2
), O
(
X
·
Y
), and O
(
N
·
X
·
Y
). The proposed method runs on MATLAB platform without optimization under Microsoft Window-XP Professional 2002 operating system and Intel Duo CPU P8700@ 2.53 GHz, 2.89 GB Memory. A typical running time of the above three processes is 3270 seconds, 63 seconds, and 212 seconds for 1024
×
1024 image of Figure 5(a1) with 40000 dots. The running time for 512
×
512 image with 12000 dots is 137 seconds, 15 seconds, and 44 seconds, respectively. In general, the image size X, Y, and block radius μ
e are primary factors affecting the computation time.


## 4.1. Pointillism Style

Pointillism results of source images from Figure1(a) are shown in Figure 1(d). Figure 2(d) shows the results of the aforementioned source images of Figure 2(a). Figures 5, 6, and 7 show several other natural images like portraits, flowers, and birds and their Pointillism with different dot-size settings.Two portrait images’ Pointillisms.N
=
40000 (a) and N
=
39000 (b). ((a1), (b1)) Source image with size: 1024
×
1024 and 768
×
1024; ((a2), (a3)) with maximal/minimal dot size: 103/11 and 51/5; ((b2), (b3)) with maximal/minimal dot size: 77/9 and 39/5.
(a)
(b)Flower and bird images’ Pointillism.N
=
21000 (a) and N
=
39000 (b). ((a1), (b1)) Source image with size: 1024
×
425 and 1024
×
777; ((a2), (a3)) with maximal/minimal dot size: 43/7 and 21/3; ((b2), (b3)) with maximal/minimal dot size: 79/9 and 39/5.
(a)
(b)((a), (c)) Pointillism without direction map and with saliency map to decide the dot size; ((b), (d)) Pointillism without direction map and with equalized standard deviation map of the saliency map to decide the dot size. All other settings are as same as those in Figures2(d) and 6(a3).
(a)
(b)
(c)
(d)Several factors, including dot size, saliency map, and direction map, should be taken into account when considering the whole generative process.(
1
) Dot size: a rough observation could disclose that the dot size, especially minimal dot size, is a primary setting affecting the style. The smaller the minimal dot size is, the clearer the Pointillism image is. That is of course because a smaller size dot zone contains average color information in a small zone while a larger dot zone contains average color information in a large zone. If the minimal dot size shrinks to one pixel, the target image becomes the source image itself. So, it should be noted that there is a balance between the minimal dot size and the Pointillism style. As we have declared in Section 3.1.2, the long axis radius is defined as dot size. At the same time, the short axis radius also plays important roles in generating the style. Distributions of short and long radii of dots size corresponding to different dot number (in Figure 9) are shown in Figure 10. On one hand, the flat distribution of the short radius benefits from the equalized R
S
Map. On the other hand, the unbalanced long radius distribution keep the target image covered as much as possible if the number of dots is fixed (further discussion in later part).(
2
) Direction map: further, a close research shows that the detailed parts in the source image are kept deliberately with help of the direction map. For example, the hairs in Figure 2(d) seemed to be rendered in a random direction. Comparison of its source image in Figure 2(a) shows that every hair on the eagle’s head has its own direction. The detailed random hairs in the Pointillism in fact originate from the source image; that is, the direction information is kept in the generated Pointillism. Comparisons between source images of Figures 6(a1) and 6(b1) and target images of Figures 6(a3) and 6(b3) also give similar conclusion. Fluffs on the flower and fine feather in bird’s neck keep the detailed information while generating Pointillism.Several target images without the help of direction map are shown in Figure7. It shows two different conditions: at first, Figures 7(a) and 7(c) give Pointillism whose dot size is decided by a saliency map; secondly, Figures 7(b) and 7(d) show Pointillism whose dot size is decided by an equalized standard deviation map of the saliency map. All other settings are the same as those in Figures 2(d) and 6(a3). With the saliency map deciding the dot size, the generated target images show Pointillism style with many similar size dots (for Figures 7(a) and 7(c)) or non-similar-size dots (for Figures 7(b) and 7(d)). Those non-similar-size dots come from an equalized standard deviation map of the saliency map while the similar-size dots from the saliency map itself. Compared to those target images in Figures 2(d) and 6(a3), these target images without the guidance of direction map could not keep clear edges. For example in Figure 7(a), the head of the eagle shows ghost edges because of the similar-size dots around the head and every dot does not rotate itself to enhance the edge of the head without the direction map. In Figure 7(b) however, the edge of the eagle head is clearer than that in Figure 7(a) because of the equalized standard deviation map. All dots have a flat distribution. So the edge of the head is rendered with smaller dots which keep the detailed head edges compared to Figure 7(a). A further observation around the edge of the eagle head and its eye in Figure 7(b) indicates that there are many holes around these edges compared to Figure 2(d).(
3
) Saliency map: a saliency map is comprised of several factors like gradients, color differences, and edge information. So, it is reasonable for a saliency map to decide the dot size of a source image.Several target images without the help of direction map and saliency map are shown in Figure8. It shows two different conditions: at first, Figures 8(a) and 8(c) give Pointillism whose dot size is decided by an edge map (source image filtered by vertical and horizontal Sobel operators); secondly, Figures 8(b) and 8(d) show Pointillism whose dot size is decided by an equalized standard deviation map of the edge map. All other settings are the same as those in Figures 2(d) and 6(a3). Compared to target images in Figures 2(d), 1(d), and 6(a3), they are not acceptable at all considering the prominent dots, holes, and blurred edges. Even if compared to target images in Figure 7, those target images also give even poor results. For example in Figure 8(a), only edge-map-guided dot size makes the target image show a rough look of the source image, while in Figure 8(b), an equalized standard deviation map of the edge map makes the target image look well with a flat-dot-size distribution. But the holes and ghost edges around the target image could not be ignored.((a), (c)) Pointillism without direction map and with edge map to decide the dot size; ((b), (d)) Pointillism without direction map and with equalized standard deviation map of the edge map to decide the dot size. All other settings are as same as those in Figures2(d) and 6(a3).
(a)
(b)
(c)
(d)Figure 9
Pointillism results with different dot number settings from left to right, top to down: 500, 1000, 5000, 10000, 20000, 30000, and 40000 dots. The maximal and minimal dot size are 51 and 5, respectively.Dot radius distributions (left column: long radius; right column: short radius) from 500, 1000, 5000, and 10000 to 40000 dots in Figure9.
(a)
(b)(
4
) Number of dots: each dot occupies its own zone and overlapped with each other if there are enough dots. We compare different dot number settings in Figure 9. The maximal dot size and minimal dot size are set to 51 and 5. Intuitively, enough dots cover the whole target image. In Figure 9, we can see that it is enough for only 5000 dots to cover most part of the target image. The coverage increases rapidly from 500 dots to 5000 dots. Increasing dot number from 10000 to 40000 does not give a considerable coverage increase, however. There are minor holes in the target image even with 40000 dots. In the proposed method, this problem is avoided by taking a blurred source image as the canvas image (see Section 3.1.5).


## 4.2. Comparisons

As we had mentioned in the former part, we expand the Pointillism style to a more general manner. That is to say, the proposed method has something in common with a typical Pointillism. For example, similar dot shape is adopted as rendering element and dot color is taken from the averaged source image color. The proposed method “borrows” ideas from the process of simulating Pointillism and tries to expand to a new field. The aim of the proposed method is to expand the Pointillism instead of simulating the style. Figure11 demonstrates the comparisons of the proposed method and Wu’s method [28]. We adopt similar dot size as shown in Wu’s results. However, these two results have different aim and show different style. Wu’s style is more close to Seurat’s Pointillism in dot size and placement strategy, while ours consisted of different dot size and shape and layer-by-layer placement strategy. It is not easy for the proposed method to be judged or compared with other simulated results.Comparisons between our method and Wu’s method. The source images are in the first row; the second row and third row are Wu’s results and our results. The maximal and minimal dot sizes of our method are 13 and 5, respectively, which are similar to Wu’s dot size.
(a)
(b)
(c)


## 4.3. Computational Complexity

Computational complexity is a key characteristic in assessing a method. The computational complexity of the proposed method mainly depends on calculations of the direction map, the saliency map, and the dot placement process, which respond toO
(
X
·
Y
·
(
2
μ
e
-
1
)
2
), O
(
X
·
Y
), and O
(
N
·
X
·
Y
). The proposed method runs on MATLAB platform without optimization under Microsoft Window-XP Professional 2002 operating system and Intel Duo CPU P8700@ 2.53 GHz, 2.89 GB Memory. A typical running time of the above three processes is 3270 seconds, 63 seconds, and 212 seconds for 1024
×
1024 image of Figure 5(a1) with 40000 dots. The running time for 512
×
512 image with 12000 dots is 137 seconds, 15 seconds, and 44 seconds, respectively. In general, the image size X, Y, and block radius μ
e are primary factors affecting the computation time.


## 5. Conclusion and Future Work

Generally, the proposed method gives a plausible Pointillism of natural images like portraits, flowers, and birds and so forth. This kind of Pointillism is not as same as the classic pointillism in that all dots in the proposed method are rendered in a random way and overlapped with each other layer-by-layer, from large dots to small dots so as to keep the details of the source image. The proposed method not only reserves the classic Pointillism style, but also increases the variances of the style with different settings in a digital way by simply controlling the ellipse parameters including the dot size (long axis length), long-short-axis ratios, and direction of the ellipse. All these settings are adjusted according to the direction map and saliency map of the source image. Further, the equalized standard deviation saliency map smoothens the dot size distribution, which describes the Pointillism in a vivid way. In most cases, the parameters are preset and the whole process is automated.Portraits, flowers, and birds are all rendered carefully dots by dots, one after another. The detailed parts of the Pointillism are clear enough and at the same time keep the Pointillism style.Currently, there is not an objective method to evaluate the aesthetics of an art work. It is even difficult to determine whether it is an art work or meaningless scribbles in an objective way. Further, everyone has his/her own point of views because of different culture backgrounds and personal experiences, and so forth. In the future, we will focus on evaluation method of digital art works like Pointillism.



---

*Source: 567846-2014-12-24.xml*