# Motion Patterns: Signal Interpretation towards the Laban Movement Analysis Semantics

Luís Santos, Jorge Dias

<figure><img src="image_1.png" /></figure>

► To cite this version:

Luís Santos, Jorge Dias. Motion Patterns: Signal Interpretation towards the Laban Movement Analysis Semantics. 2nd Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), Feb 2011, Costa de Caparica, Portugal. pp.333-340, 10.1007/978-3-642-19170-1_36. hal-01566557

HAL Id: hal-01566557

https://inria.hal.science/hal-01566557

Submitted on 21 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Motion Patterns:
Signal Interpretation Towards the
Laban Movement Analysis Semantics

Luís Santos¹, Jorge Dias¹,

¹ Instituto de Sistemas e Robótica
Departamento de Engenharia Electrotécnica e de Computadores
Universidade de Coimbra, Portugal
3030-290 Pólo II
{luis,jorge}@isr.uc.pt

**Abstract.** This work studies the performance of different signal features regarding the qualitative meaning of Laban Movement Analysis semantics. Motion modeling is becoming a prominent scientific area, with research towards multiple applications. The theoretical representation of movements is a valuable tool when developing such models. One representation growing particular relevance in the community is Laban Movement Analysis (LMA). LMA is a movement descriptive language which was developed with underlying semantics. Divided in components, its qualities are mostly divided in binomial extreme states. One relevant issue to this problem is the interpretation of signal features into Laban semantics. There are multiple signal processing algorithms for feature generation, each providing different characteristics. We implemented some, covering a range of those measure categories. The results for method comparison are provided in terms of class separability of the LMA space state.

**Keywords:** Laban Movement Analysis, Motion Pattern, Signal Processing, Feature Generation.

# 1 Introduction

This paper sheds light on the interpretation of a human motion signal into a set of characteristics belonging to the Laban Movement Analysis semantics [1]. Despite the existence of multiple solutions for sensing human motion, this work is based on the study of body part trajectories, independent of the acquisition method. The objective is to apply multiple feature generation algorithms to segment the signals according to LMA theory, in order to find patterns and define the most prominent features in each of the descriptors defined in Labanotation [2]. This work can be seen as an important issue in human motion modeling, in the sense that feature generation strongly influences the model performance.

By definition, model is an abstract representation that reflects the characteristics of a given *entity*, either physical or conceptual. Thus, one issue of paramount importance is the establishment of the relation between sets of variables belonging to different abstraction levels.
The *entity* to be modeled sometimes has a theoretical representation/formalism that can be used as a basis for model development. In the specific case of human motion, LMA can be defined as a language to describe human motion in general and its application to human movement modeling is increasing [2],[4],[5]. LMA is divided in four<fnref n="1" /> main components [3], each of them described through specific semantics that quantify and qualify different aspects of the human motion. Apart from mathematical sciences, LMA is a widely used tool in areas like physiotherapy, individual sports analysis, dancing.

We propose to study different feature generation/signal processing algorithms (e.g. Principal Component Analysis [6] or the analysis in the frequency domain [7]) to segment body part trajectories. The purpose of applying multiple segmentation techniques is to have a broad range of algorithms that provide different characteristics. Thus, this work provides a variety of features allowing finding suitable patterns that characterize each of the states of the Laban semantic space. We will evaluate the selected algorithms using a method based on Scatter Matrices to quantify the class separability, i.e. how each algorithm performs in terms of discretizing the variables in the LMA space-state.

This paper will be organized as follows: the next chapter will comment the contribution emerging from our work. Section 3 covers LMA theory, introducing a contextualization to Laban semantics. The subsequent section 4 will present the different segmentation algorithms, followed by the results in section 5. We will conclude in section 6 with the final remarks and future work.

## 2 Technological Innovation for Sustainability

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** The path from trajectory to the feature space to the classifier.</figcaption></figure>

A research on state of the art shows that most of features selected are very specific to the objectives of each study. Some work verses on general feature selection; however, most focuses on joint angle information and kinematics. During our research, we found residual contributions applying other techniques. It becomes even more noticeable when it comes to the use of Laban Movement Analysis as intermediate mid-level descriptor. The conducted research verified that most approaches use a set of theoretically defined features, rather than testing multiple methods towards the selection of good features relating to LMA semantics. This will help to improve motion/behavior model performance. Applicable to areas

like surveillance, monitoring or physiotherapy, the performance improvement of such systems might have significant scientific impact with reflex on social and even economic sustainability.

<footnote n="1">¹ Laban theorists are not in unison regarding the number of main components. The two mainstreams divide themselves between four and five components respectively.</footnote>
## 3 Interpreting with Laban Movement Analysis

Laban Movement Analysis has been described in previous works [4],[5], however it will be briefly introduced, in order to contextualize this work.

Developed in the early 20th Century by Rudolf Laban, Laban Movement Analysis has evolved throughout the years as a language to describe human motion, using a specific notation (Labanotation). Its semantic allows qualifying human motion in its different aspects, and introduces the fundamentals of our space-state definition.

Laban components are divided in two main groups, kinematic and non-kinematic. The kinematic components, body and space, deal with more quantitative aspects of the movement, and previous works [5],[8] demonstrate they are easily extracted. Consequently we found no value trying to apply those components the processing methods. The decision was to place emphasis on the qualitative (non-kinematic) components, Shape and Effort, as these pose a more relevant and interesting problem. Non-Kinematic components are described in theory by a rich and consequently complex semantic, thus constituting a very useful characterizing framework for human motion modeling. This work will not describe these two components in detail; rather it will make a very short overview and present the resulting space state. The Effort component is divided in 4 qualities lying between 2 extreme states. Each quality is associated to a cognitive process, a subject and lies between extreme states (see Table 1).

<table><caption>**Table 1.** Effort qualities, cognitive process, subject and space state.</caption><thead><tr><td>Quality</td><td>Cognitive process</td><td>Subject</td><td>Space State</td></tr></thead><tbody><tr><td>Space</td><td>Attention</td><td>Spatial Orientation</td><td>[Direct, Indirect]</td></tr><tr><td>Weight</td><td>Intention</td><td>Impact</td><td>[Strong, Light]</td></tr><tr><td>Time</td><td>Decision</td><td>Urgency</td><td>[Sudden, Sustained]</td></tr><tr><td>Flow</td><td>Progression</td><td>How to keep going</td><td>[Free, Careful]</td></tr></tbody></table>

Bartenieff and Lewis [3] does not define Shape as a component of its own, but rather a set of qualities emerging from Body and Space components. Shape is also divided into two qualities, which are summarized in Table 2, defining the space-state.

<table><caption>**Table 2.** Shape qualities and correspondent space state.</caption><thead><tr><td>Quality</td><td>Space State</td></tr></thead><tbody><tr><td>Flow</td><td>[Rising, Sinking]<br/>[Spreading, Enclosing]</td></tr><tr><td>Spatial</td><td>[Advancing, Retreating]</td></tr></tbody></table>

## 4 Feature Space

In the previous section, we have presented the space state of Laban Movement Analysis, which has only two states (binomial) for each quality. Its semantic carries
meaningful qualitative characteristics which we seek to interpret using different
signal processing algorithms.

The core of feature generation is to transform the available set of data features
into another. If the transform is suitable, the transformation domain features may
exhibit characteristics that yield a lot of meaningful information about the original
signal.

In the feature generation area are domains which are more recurrent than others.
Some algorithms aim data reduction, such as Principal Component Analysis and
Single Value Decomposition [9], which belong to a class of methods known as
Linear Discriminant Analysis. Within Nonlinear methods there are some focusing
on the geometric characteristics of the signal in graph based approaches like
Isometric Mapping. The Fourier Transform and others alike study the signal in the
frequency domain. And there are a wide range of methods that study the signal
regarding its derivative characteristics, the first, second and higher order moments
methods.

**4.1 Feature Generation Methods**

Since the implementation of all methods is an intractable task, we selected a group
of methods for this first approach, enough to cover the previously described domains
in feature generation. The objective is to get a first evaluation on how each domain
and correspondent methods behave in the task of discriminating the binomial LMA
space state.

**4.1.1 Karhunen-Loève Transform**

The computation of the Karhunen-Loève (KL) transformation matrix will exploit the
statistical information describing the data. The first assumption is that the data
values have zero mean. The goal is to obtain mutually uncorrelated features in an
effort to avoid information redundancies.

The method computes the data correlation matrix, which by its symmetric
properties generates a set of mutually orthogonal eigenvectors V, known as the KL
transform. As it turns out, KL has a number of other important properties, which
provide different ways for its interpretation. One is the actually generated orthogonal
eigenvectors, which encompass the principal directions of the spanned data, as well
as the variance along each its directions. Thus we will use this information to
represent trajectories in the resultant component space. We decided to use this
information rather than the original purpose of the KL (re-project data in a
dimensional space smaller the original), because data reduction methods are not
optimized regarding class separability, and they do not assure that the principal
components provide the best discriminatory properties.

KL transform, is a widely recognized technique, hence, more information on the
method and its properties can be found in [6].

**4.1.2 Local Linear Embedding**

The starting point of this method is the assumption that the data points lie on a
smooth manifold (hyper-surface). The main philosophy behind Local Linear
Embedding [10] is to compute a low-dimensional representation of the data however preserving the local neighborhood information. The outcome of this algorithm attempts to reflect the geometric structure of the data. This algorithm can be resumed in its basic form with the following three steps:

(1) Select the nearest neighbors for each of the data points $x_i$, $i=1,2,...,n$. Some common techniques are Euclidean distances or the K-nearest neighbors.

(2) Compute the weights $W(i,j)$ that best reconstruct the point $x_i$ from its neighbors minimizing the cost function

$$ \arg\min_{W} E_{W} = \sum_{i=1}^{n} \left\| x_i - \sum_{j=1}^{n} W(i,j) x_j \right\|^2 \quad (1) $$

A typical weight function is

$$ W(i,j) = \begin{cases} \exp\left(-\frac{\|x_i - x_j\|^2}{\sigma^2}\right), & \text{if points correspond to neighbors} \\ 0, & \text{otherwise} \end{cases} \quad (2) $$

where $\sigma^2$ is a user-defined parameter. The weights are constrained such that the rows of the weight matrix, i.e., the sum of the weights over all neighbors equals to 1. (3) Use the weights obtained from the previous step to compute the corresponding points $y_i \in R^m$, $i=1,2,...,n$, to minimize the cost with respect to the unknown points $Y=\{y_i, i=1,2,...,n\}$

$$ \arg\min_{W} E_{W} = \sum_{i=1}^{n} \left\| y_i - \sum_{j=1}^{n} W(i,j) y_j \right\|^2 \quad (3) $$

This method explores the local linearity of the data and tries to predict each point through its neighbors using the least squares error criterion. Minimizing the cost regarding to the constraint given in (2) results in a solution that satisfies the following interesting properties: Scale, rotation and translation invariance. Solving (3) for the unknown points $y_i$, $i=1,2,...,n$, is equivalent to:

* Performing the eigen-decomposition of the matrix $(I - W)^T (I - W)$.

* Discarding the eigenvector corresponding to the smallest eigenvalue.

* The remaining eigenvectors corresponding to the other eigenvalues yield the low-dimensional outputs $y_i$, $i=1,2,...,n-1$.

### 4.1.3 Discrete Fourier Transform

The Discrete Fourier Transform (DFT) [11] transforms a function into a sum of functions that represent it in the frequency domain. There is an assumption that the signal must be finite, which is accomplished in our case due to signal nature. The aim of this technique is to quantify how much of the signal lies in a determined
frequency, i.e. to determine the dominant frequencies in a signal. For this work, we use the dominant frequencies and their coefficients to define the feature space state. We will not explain the theory, as this is probably one of the most well-known techniques around. However, we suggest the reader, if needed, to learn more or familiarize with method [11].

### 4.1.4 7 Moments of Hu

Under the scope of geometric moments, which are used to characterize data such as areas or information about orientation, we have the known 7 moments of Hu [10]. Within this class of methods, we have opted for Hu's moments because this technique intrinsically encompasses invariance to rotation, translation and scale. These are important properties because of the assumption that trajectory contours can be performed at different scales and orientations or space, depending on the physical structure of performer. The moments of Hu base themselves in the definition of central moments

$$ \mu_{pq} = \iint (x - \bar{x})^p (y - \bar{y})^q f(x, y) dx dy \quad (4) $$

which are then normalized. We will not describe the mathematics of Hu's 7 moments, as they are somewhat cumbersome to this article and are readily available in [10] for the interested reader. An important remark is the statement that the first six moments are also invariant under the action of reflection, while the seventh moment changes signal. This property is interesting in the sense that it allows both left and right handed performers to be considered indifferent in terms of generated data. The values of these quantities can be quite different. In practice, to avoid precision problems, the logarithms of their absolute values are usually used as features.

## 4.2 Method Comparison and Evaluation

To establish a comparison criterion to evaluate the class separability capability of each method, we will use a method based on Scatter Matrix (SM) [10]. The reader might be familiar with the known Fisher Discriminant Ratio, which is a particular case of SM methods for 1 dimension and 2 classes. We selected SM due to the fact that other methods such as Divergence or Bhattacharyya Distance turn to be computationally demanding if a Gaussian assumption of data distribution is not employed. We should aim to select features leading to large between-class distance and small within-class variance in the feature vector space.

SM is built upon information related to the way feature vector samples are scattered in the l-dimensional space. The method defines the following matrices:

$$ S_w = \sum_{i=1}^{n} P_i \Sigma_i \quad (5) $$

Which is known as within class scatter matrix, and $\Sigma_i$ is the covariance matrix for class $w_i$ and $P_i$ is the a priori probability of class $w_i$, i.e. $P_i \approx n_i/N$, where $n_i$ is the
number of samples of class $w_i$ out of a total $N$ samples. Then defining the *Between-class scatter matrix*

$$ S_b = \sum_{l=1}^{M} R_l (\mu_l - \mu_0) (\mu_l - \mu_0)^T \quad (6) $$

where $\mu_0$ is the global mean vector. The simplified computation for the *Mixture scatter matrix* turns out

$$ S_{m} = S_{w} + S_{b} \quad (7) $$

with $S_m$ the covariance matrix of the feature vector with respect to the global mean. Its trace<fnref n="2" /> is the sum of variances of the features around their global mean. From these definitions we define the criterion as

$$ J_l = \frac{\text{trace}(S_{m,l})}{\text{trace}(S_w)} \quad (8) $$

The ratio $J_l$ takes large values when samples are well clustered around their mean and the clusters are well separated.

# 5 Experimental Considerations

Most of the experimental process is an undergoing work as we aim to have a large enough database to encompass movements with all different LMA characteristics. We present our preliminary results using 5 different movements: Punch, Write, Waving Bye-Bye, Point and Lift. These movements have been hand labeled with different Effort Time and Effort Space characteristics. Punch, Point, Lift belong to *Direct* movements, whereas Write and Bye-Bye to *Indirect*. In the case of Effort Time, Punch and Point have been considered *Sudden* while the remaining three are considered *Sustained*. We have performed feature generation with all described techniques. Table 3 presents the separability ratio resulting from the application of the Scatter Matrix approach.

<table><caption>**Table 3.** The table shows the value of the separability ratio for the Effort Time and Effort Space qualities for each of the presented techniques.</caption><thead><tr><th></th><th>PCA</th><th>DFT</th><th>LLE</th><th>Hu</th></tr></thead><tbody><tr><td>Effort.Time</td><td>246,7</td><td>36,6</td><td>190,3</td><td>143,2</td></tr><tr><td>Effort.Space</td><td>210,2</td><td>29,1</td><td>229,6</td><td>134,3</td></tr></tbody></table>

<footnote n="2">² Trace is defined to be the sum of the elements on the main diagonal.</footnote>
# 6 Conclusions and Future Work

From the observation of the presented results, one concludes there is not a single perfect method for feature extraction. Different LMA qualities exhibit better separability performances for different methods. If we chose one method only, then we need to select one whose average performance is better. However if the computational cost of having different algorithms performing data processing is not an issue, then the choice must fall on the best method for the specific characteristic to be modeled. In the future there we will (it is an ongoing work) augment the database into a comprehensive set, which will encompass movements with all different LMA characteristics. The development of software to allow testing any desirable method is on the horizon, as well as doing efficiency vs. separability tests. Also, simple models for LMA classification should be done, for classification tests. The goal is to verify the true impact of the separability ratio vs. positive classification rate.

# References

1. Liwei Zhao: Synthesis and Acquisition of Laban Movement Analysis Qualitative Parameters for Communicative Gestures. PhD Thesis, University of Pennsylvania (2002)

2. Diane Chi, Mónica Costa, Liwei Zhao, and Norman Badler: The emote model for effort and shape. In: SIGGRAPH 00, Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, 173--182. ACM Press (July 2000)

3. I. Bartenieff and D. Lewis: Body Movement: Coping with the Environment. New York, Gordon and Breach Science (1980)

4. Rett J.: Robot Human Interface Using Laban Movement Analysis Inside a Bayesian Framework. PhD Thesis, University of Coimbra (2009)

5. Joerg Rett, Luís Santos and Jorge Dias: Laban Movement Analysis using Multi-Ocular System. In: International Conference on Intelligent RObots and Systems, IROS (2008)

6. Jolliffe I.: Principal Component Analysis, Series: Springer Series in Statistics, 2nd ed., Springer, NY, (2002)

7. Broughton, S.A., Bryan, K.: Discrete Fourier analysis and Wavelets: Applications to Signal and Image Processing. New York: Wiley (2008)

8. José Prado, Luis Santos and Jorge Dias: Horopter based Dynamic Background Segmentation applied to an Interactive Mobile Robot. In: 14th International Conference on Advanced Robotics, ICAR (2009)

9. Horn, Roger A., Johnson, Charles R.: Matrix Analysis. Cambridge University Press (1985)

10. Theodoridis S., Koutroumbas K.: Pattern Recognition. 4th Edition, Elsevier (2009)

11. Oppenheim, Alan V., Schafer R., Buck J.: Discrete-time signal processing. Upper Saddle River, N.J.: Prentice Hall (1999)