Journal of Robotics, Networking and Artificial Life, Vol. 3, No. 2 (September 2016), 96-101

Mobile Camera based Motion Segmentation by Image ResizingChunyu Yu1, Fengzhi Dai2*12*

Department of Engineering and Computer Science, Syracuse University,
900 South Crouse Ave
Syracuse, NY 13244, UScyu07@syr.edu

College of Electronic Information and Automation, Tianjin University of Science and Technology,
1038 Dagunanlu Road,Tianjin, 300222, China
E-mail: daifz@tust.edu.cn

Abstract
The need of detecting moving object like human and vehicle by mobile camera is increasing in commerce andindustry. Therefore, tradition method of abstracting moving object from still background cannot solve the problem.
Though Histogram of Oriented Gradients (HOG) have been widely used for human detection, due to relying on
sliding windows and multi-scale resizing method, it is computational expensive to perform on ordinary equipment.
In this paper, an image-resize methodology which can abstract motion segmentation and detect moving object frommoving background is proposed. First, edges images are computed. Then movement vector between frame images
are computed and the relative background motion is compensated. By adjusting the parameters of resize algorithm,
human liked object or vehicle liked object can be segmented separately and the segmentation can be used forfurther detection. Experiments have been performed under three different environments for human detection andvehicle detection. The results show that the running time is highly reduced and the accuracy can reach as high as
93.04%.
Keywords: Mobile cameras, motion segmentation, edge-based alignment, object detection.

1. Introduction
Real-time human detection is a key study field incomputer vision. This technique is widely used in
automotive engineering, traffic monitoring, robotics,
human-computer interaction, security, military andmany other applications. In literature, there are a lot ofresearchers focused on stationary camera based object
detection. Main technique used is image difference tofind the difference of two image pixel by pixel.
However, when camera is moving during the recording,the whole background is changing. To resolve thisproblem, developed different algorithms to against thelight changes1-4.
Another widely used method is statistical theory whichclassifies pixels into background and foreground. C.
Rasmussen and G. D. Hager combine Maximum A
posteriori Probability (MAP) with a Probabilistic Data
Association Filter 5 . N. Vaswani, A. Tannenbaum, andA. Yezzi unified geometric active contours and levelsets within Particle Filter 6 and R. Stolkin, A. Greig, M.
Hodgetts, and J. Gilby proposed an algorithm which
combines Expectation Maximization and extendedMarkov Random Field 7. Another method is wavelets
based. The main idea is to analysis the difference part’s

Published by Atlantis Press
Copyright: the authors96

Chunyu Yu, Fengzhi Dai

frequency. For instance, presented a motion
segmentation based on Galilean wavelets 8. In 1980
Horn and Schunch first proposed a new method named
optical flow 9. After that, many researchers madeimprovement on it such as J. Zhang, F. Shi, J. Wang,and Y. Liu 10.
In this paper, an image resizing based motion
segmentation method is proposed. Unlike othercomputational expensive method, this algorithm only
relies on a new edge alignment method and fast image
resizing according to object morphology. First, edges in
two consecutive frame images are detected. Then thenew edge-based alignment method is adopted to find themotion vector between two images. Motion
compensation is performed before motion segmentation.
Then the image is resized according to object
morphology. After filtering pixels lower than thresholdvalue, object is stand out of the background. The
experiment was performed by MATLAB 2014a on i52450m and 6GB RAM in three different scenarios forhuman liked object detection and vehicle liked object
detection. The results illustrate that the accuracy rate
can reach as high as 93.04%, and the execution time ishighly speeded up (approximately 0.82s per frame).
The paper is organized as follows. The proposed
method for motion segmentation and object detection isdescribed in section 2 and section 3 presents theexperiment results and the conclusion part is in section
4.

Fig.1. One of eight directions of relative movement between
two images.

compared. Specifically, fill zero into the central area of
two images and find out the relative movement between
the background in 8 directions (vertical, horizontal anddiagonal) (Fig. 1). By the algorithm in Table 1, relative
motion direction and distance are achieved. Base onexperiments, the movement between two frames isalways 3 to 6 pixels, so 10 pixels are sufficient forsearching in our algorithm.
Table 1. Algorithm for finding relative motion
direction and distance.
1 frame[] m = frame[80]
2 for 1 to 8 directions
3
for 1 to 10 pixels
4m[i] = abs (tailored (f1, -direction, pixel) –tailored (f2, direction, pixel))
5 direction = minima (m).direction
6 distance = minima (m).pixel

According to the direction and movement distance,tailor pixels from two images respectively. After that,beside the people, backgrounds are almost in the same
position in two images (say, reach the global minima).

2. Proposed Method
2.1. Finding global minima
Take human liked object detection as an example. Twoconsecutive frame images f1 and f2 are loaded first. By
using Canny edge detector, two binary edge images areobtained. Ideally, the camera is stable and thebackground is not moved, so in the difference image(f2-f1), only moving object is visible and background isoffset. Virtually, the camera may slide a very small
distance in a direction, so the background and people
are moved simultaneously when two consecutive
images are compared. Therefore, eliminate themovement of background is essential.
Assume people are most likely appear in the central
area of an image, when detecting backgroundmovement, only edges at the margin of images are

2.2. Resizing image algorithm

Fig. 2. Difference image acquired without resizing algorithm.

Published by Atlantis Press
Copyright: the authors
97

Mobile Camera based Motion

Table 2. Difference image acquired without dilating (1-value pixels: 15).

Row

391392393
394395396
397
398399400

258
0110000000259
0110000000Column
261 26200111100000000000000

260
01
10000000

263
0000000000

2640000000000

2650000000000

2660000000
110

267
00000
100
1
1

eliminated by deleting pixels which value is lower thanspecific threshold value. According to the environmentand object, 0.5 is adopted in this experiment. Therefore,
all the pixels in Table 4 will be 0. Compare to Fig. 2, theroad line in Fig. 3 is eliminated (yellow rectangle) andhuman is out stand of background.
The following step are dilating image, filling image

Even global minima is reached, virtually the edges in
two images cannot be overlapped exactly. As a result,when difference image (f2-f1) is calculated, backgroundedges of two images can hardly offset each other. The
background may become even more complicated thanhuman outline. As a result, the human cannot stand outof background (Fig. 2).

Table 3. Difference image acquired after dilating (1-value pixels: 13).

Row

391
392393394395396
397398
399
400

258
100
1000000259
100
1000000260
100
1000000Column261 262

To resolve the problem, an image resizing algorithm isadopted. Table 2 is the matrix of yellow rectangle inFig. 2 which is achieved without dilating images. Table
3 is the result acquired after dilating images. By
compare two tables, we can notice that the number ofpixels which value is 1 is decreased and they are more
scattered in Table 3. In terms of the whole image,dilating images results to more mottled background.
As moving human most likely look like verticalrectangles, they are more robust than noise (remainedbackground) in terms of vertical image resizing.
Therefore, based on the result in Table 3, the image isresized in vertical direction and then restored. After thatpixels’ value are between -1 and 1.
Due to the distortion, mottled background faded
dramatically compared to human outline. It can be10010000001000000000263
1000000000264
0000000000265
0000001100266
0000100000267
0000000000Fig. 3. Difference image acquired by resizing algorithm.

hole and eroding image. The largest block in the imageis the object.

Published by Atlantis Press
Copyright: the authors
98Chunyu Yu, Fengzhi Dai

Table 4. Matrix after image resizing.

Row

391
392
393
394
395
396
397
398
399
400

258

0.311
0.340
0.312
0.256
0.195
0.146
0.092
0.038
-0.005-0.026

259

0.312
0.343
0.315
0.258
0.196
0.146
0.092
0.038
-0.005-0.026

260

0.312
0.343
0.315
0.258
0.196
0.146
0.092
0.038
-0.005-0.026

Column
261
262

0.312
0.343
0.315
0.258
0.196
0.146
0.092
0.038
-0.005-0.026

0.235
0.230
0.169
0.084
0.010
-0.023-0.026-0.015-0.0030.001

2630.235
0.230
0.170
0.086
0.012
-0.028-0.050-0.057-0.044-0.004264

0.001-0.005-0.024-0.042-0.042-0.005
0.075
0.171
0.253
0.296

265

-0.042-0.005
0.075
0.171
0.253
0.296
0.316
0.316
0.296
0.253

266

0.010
0.044
0.111
0.185
0.235
0.230
0.168
0.082
0.008
-0.027

267

0.000
0.000
0.001
0.001
0.001
-0.003-0.015-0.026-0.0230.010

3. Experiment Results
Without loss of generality, the experiment was
performed to test vertical resizing and horizontalresizing in three different scenarios: (1) human walkindoor, (2) human walk outdoor and (3) human walkoutdoor with moving vehicle in background. The datasetof Zheng et al.’s experiment was adopted 11.
Fig. 4. Segmentation achieved without resizing algorithm.

3.1. Human walk indoor without moving object
231 frames of a video were tested in this experiment.
Fig. 6 as part of results illustrates that the movement
segmentation is highly accurate and almost overlaps
with the human body. Thus the segmentation will
decrease the running time largely for further HOG
detection. Even though illumination sometimes disturbs
the detection, the overall successful detection rate forwhole frames is 77.56%.
3.2. Human walk outdoor without moving vehicle

Fig. 5. Segmentation achieved with resizing algorithm.

The following assumptions are made in proposed
algorithm. Firstly, there must be sufficient edges in themargin of the images as they are used to find motion
vector. Secondly, there must be sufficient movement of
the object. If the object is stationary, it will beconsidered as background. Finally, the object should
large enough in terms of the image scale as the largest
block is selected as object in the final step. By
contrasting Fig. 4 and Fig. 5, we can see that thealgorithm makes large difference on segmentation.

In this experiment, 273 frames were tested and theaccurate detection rate is 93.04%. Similar to the resultsof indoor experiment, the segmentation part is highly
overlapped with the human body (Fig. 7).
3.3. Human walk outdoor with moving vehicle inthe background
100 frames were tested for human and vehicle. The
detection rate is 86% and 83.1% respectively. As shownin Fig. 8 and Fig. 9, the moving vehicle can be
distinguished from stopped bus.

Published by Atlantis Press
Copyright: the authors
99

Mobile Camera based Motion

Fig. 6. Indoor sequence. The green line outlines the largest block and the yellow box indicates the possible human region.

Fig. 7. Outdoor sequence. The green line outlines the largest block and yellow box indicates the possible human region.

region, further detection such as HOG in this region canbe speed up largely.

4. ConclusionsIn this paper, a fast and simple motion segmentation
method for moving background is proposed. Zheng etal.’s dataset was tested in the experiment. The results
show that the proposed method is highly accurate
especially in complex environment. In the future, priorknowledge will be considered. The position of yellow
box can be limited in a range according to its position inprior frame. Thus the algorithm can be more robust to
fast light changing. Due to pretty accurate segmentation

References
1. A. Cavallaro, O. Steiger, and T. Ebrahimi, TrackingVideo Objects in Cluttered Background, IEEE
Transactions on Circuits and Systems for Video
Technology 15(4) (2005) 575–584.
2. F.-H. Cheng and Y.-L. Chen, Real time multiple objects
tracking and identification based on discrete wavelet
transform, Pattern Recognition 39(6) (2006) 1126–1139.
3. R. Li, S. Yu, and X. Yang, Efficient spatio-temporalsegmentation for extracting moving objects in video

Published by Atlantis Press
Copyright: the authors
100Chunyu Yu, Fengzhi Dai

Fig. 8. Outdoor sequence with moving car in background. The green line outlines the largest block and yellow box
indicates the possible human region.

Fig. 9. Outdoor sequence with moving car in background. The green line outlines the largest block and yellow box indicatesthe possible vehicle region.

4.
5.

6.

7.

sequences, IEEE Transactions on Consumer Electronics
53(3) (2007) 1161–1167.
A. Colombari, A. Fusiello, and V. Murino, Segmentationand tracking of multiple video objects, Pattern
Recognition 40(4) (2007) 1307–1317.
C. Rasmussen and G. D. Hager, Probabilistic dataassociation methods for tracking complex visual objects,
IEEE Transactions on Pattern Analysis and MachineIntelligence 23(6) (2001) 560–576.
N. Vaswani, A. Tannenbaum, and A. Yezzi, Tracking
deforming objects using particle filtering for geometricactive contours, IEEE Transactions on Pattern Analysis
and Machine Intelligence 29(8) (2007) 1470–1475.
R. Stolkin, A. Greig, M. Hodgetts, and J. Gilby, Tracking
deforming objects using particle filtering for geometricactive contours, An em/e-mrf algorithm for adaptive

8.

9.
10.
11.

model based tracking in extremely poor visibility 26(4)(2008) 480–495.
M. Kong, J.-P. Leduc, B. Ghosh, and V. Wickerhauser,
Spatio-temporal continuous wavelet transforms formotion-based segmentation in real image sequences,Proceedings of the International Conference on Image
Processing 2 (2008) 662–666.
B. K. Horn and B. G. Schunck, Determining optical flow
(Artificial Intelligence Laboratory, Cambridge, MA,
1980).
J. Zhang, F. Shi, J. Wang, and Y. Liu, 3d motion
segmentation from straight-line optical flow, Multimedia
Content Analysis and Mining (2007) 85–94.
Y. Zheng, A. Mahabalagiri, and S. Velipasalar, Detectionof moving people with mobile cameras by fast motion
segmentation, Distributed Smart Cameras (2013).

Published by Atlantis Press
Copyright: the authors
101