# Instant First-Person Posture Estimation

Takafumi Serizawa, Yasuyuki Yanagida

► To cite this version:

Takafumi Serizawa, Yasuyuki Yanagida. Instant First-Person Posture Estimation. 9th International Conference on Entertainment Computing (ICEC), Sep 2010, Seoul, South Korea. pp.508-510, 10.1007/978-3-642-15399-0_75. hal-01055585

HAL Id: hal-01055585

https://inria.hal.science/hal-01055585

Submitted on 13 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Instant first-person posture estimation

Takafumi Serizawa¹ and Yasuyuki Yanagida²

¹ Graduate School of Science & Technology, Meijo University

² Faculty of Science and Engineering, Meijo University
1-501 Shiogamaguchi, Tempaku-ku, Nagoya 468-8502 Japan

¹ m0941501@ccalumni.meijo-u.ac.jp

² yanagida@.meijo-u.ac.jp

**Abstract.** We propose an instant posture estimation technique, which can operate only by a stereo image with small (6 cm) baseline. It does not require a priori information of the target user, background information, and markers. This system allows cameras to move freely, because it operates only with a small stereo camera unit. Moreover, if the input image is replaced with a movie or real-time video, this system can be used as a real-time motion tracker. With our proposed technique, robots and computers will be able to non-verbally communicate with unspecified people as well as pre-registered people. Moreover, this system can output not only the posture but also the body size and clothes. Therefore, proposed technique can also be used as a calibration procedure for other motion tracking algorithms.

**Keywords:** posture estimation, motion tracking, Marker-less, Calibration-free,

# 1 Introduction

The motion tracking technology is necessary to achieve a natural interaction of human and computer/robot as will as recognition of voice and facial expression. Current “motion tracking” technologies based on volume matching or image matching have achieved high-speed and high-accuracy. However, existing techniques require a priori information of the person being tracked [1]. When the target is a specific user, a priori information of the target user can be input before the tracking session. But, when the user is unspecified, it is impossible to input a priori information. To solve this problem, it is necessary to develop a new algorithm to estimate initial information based on the input of single frame.

This paper explains the “posture estimation” algorithm to acquire initial information and the entertainment application using our algorithm.
## 2 Proposed Technique

We propose a robust algorithm of posture estimation (Fig. 1), by avoiding error propagation from body trunk to ends of limbs and by using reliable 3D body ridge line detection in estimating the position/orientation of each part of the body. To avoid the error propagation, we introduced loose coupling of bones and a mechanism to feedback the estimation result of upper arms to the rotation angle of body trunk (Fig. 2). Unlike existing methods that use rigid joint-bone structure and estimate the direction/rotation of each part serially, estimation error in the rotation angle of body trunk does not propagate to the errors at the end of limbs (Fig. 3).

<figure><img src="image_2.png" /><figcaption>Fig. 1. System</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Fig. 2. Proposed</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3. Related</figcaption></figure>

## 3 System Performance

Fig. 4-7 shows the case that parts of the body (lower legs) are out of the camera view, with complex background. Fig. 8-11 shows the case that entire body is visible with simple background.

<figure><img src="image_5.png" /><figcaption>Fig. 4. Left input A</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5. Right Input A</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 6. 2D Output A</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 7. 3D Output A</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Fig. 8. Left input B</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Fig. 9. Right Input B</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 10. 2D Output B</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. 11. 3D Output B</figcaption></figure>

## 4 Our demonstration

In our demonstrations, a visitor (A) is asked to put on a headgear composed of an HMD and a stereo camera. When another person (B) enters the visitor's view, the appearance of B is replaced by a 3D character (Fig. 12).

<figure><img src="image_13.png" /><figcaption>Fig. 12. Demonstration (3D model and character (C) Teatime/inc)</figcaption></figure>

## References

1. Olivier, B., Pascal C.M.C., Arnaud, B.: Fast nonparametric belief propagation for real-time stereo articulated body tracking. In: Computer Vision and Image Understanding Volume 113, Issue 1, pp. 29–47. Computer Vision and Image Understanding, Elsevier (2009)