# Spatiotemporal Relations and Modeling Motion Classes by Combined Topological and Directional Relations Method

Authors: Nadeem Salamat; El-hadi Zahzah
Journal: ISRN Machine Vision
 (2012)

---

## Abstract

Defining spatiotemporal relations and modeling motion events are emerging issues of current research. Motion events are the subclasses of spatiotemporal relations, where stable and unstable spatio-temporal topological relations and temporal order of occurrence of a primitive event play an important role. In this paper, we proposed a theory of spatio-temporal relations based on topological and orientation perspective. This theory characterized the spatiotemporal relations into different classes according to the application domain and topological stability. This proposes a common sense reasoning and modeling motion events in diverse application with the motion classes as primitives, which describe change in orientation and topological relations model. Orientation information is added to remove the locative symmetry of topological relations from motion events, and these events are defined as a systematic way. This will help to improve the understanding of spatial scenario in spatiotemporal applications.


---

## Body

## 1. Introduction

Automatic event detection is gaining more and more attention in computer vision and video researchers community. Visual scene description takes into account ontological viewpoint of relative object positions. It is sufficient to emphasize the model of moving object’s spatial relations such as modeling video events [1, 2]. Modeling spatiotemporal relations between moving objects involves the modeling of motion events such as durative events. These events are the union of primitive events, which hold for each snapshot during the interval with a particular temporal order.Spatiotemporal features could be used for modeling the spatiotemporal events [3]. Defining spatiotemporal relations have two main domains of research: spatiotemporal object and spatiotemporal relations modeling. Cuboid object approximation or three-dimensional geometry is used to model the former and for lateral two-dimensional objects occupy different spatial locations at different time points [4]. Several types of logics for mechanizing the spatiotemporal relations and reasoning process are used like interval temporal logic [5, 6], point temporal logic [7], and propositional model logic [8].The point temporal logic supports instantaneous snapshots of the world. A snapshot represents the current situation, and a spatiotemporal relation is defined if a particular spatial relation holds for every snapshot during that interval. It is considered that time and space are bounded to each other. Spatiotemporal relations are modeled between moving objects by taking transaction from one snapshot to the next snapshot.Dimiter Vakarelov, in [9, 10] provided strong mathematical and logical bases for defining the general spatiotemporal topological relations and divided them into two categories: (i) stable spatiotemporal topological relations and (ii) unstable spatiotemporal topological relations. A stable spatiotemporal topological relation is a relation which holds for every frame or snapshot in the interval and unstable spatiotemporal topological relations are those which hold at least for one snapshot during the temporal interval. Some spatiotemporal relations are strictly stable such as disjoint (D), nontangent proper part(NTPP), nontangent proper part inverse(NTPPI), equal (EQ), and others may be stable or unstable like meet(M), partially overlap(PO), tangent proper part(TPP), and tangent proper part inverse(TPPI). This provides a way to use the spatiotemporal relations in linguistics and their use as motion events modeling.Most of the existing theories of spatiotemporal relations are domain-based, where domain knowledge imposes the conditions to a spatiotemporal relation to be topologically stable or unstable. A domain where the spatiotemporal relation is topologically stable, there the directional or distance relations are unstable such as spatiotemporal relations on the road networks. In defining motion events or verbs which represents the transitive movement, stability or unstability of topological relations plays important role and directional relations along with the topological relations remove certain symmetries and helps the user to understand real-scene situation. Consider the following two examples.(1)
Mr. John (object, name of a person) crosses (relation) the football ground (object);(2)
Mr. John (object, name of a person) crosses (relation) the football ground (object) from north to south.In proposition 1, there is no confusion about the topological relation that objectA (John) has certain topological relation with object B (football ground). But there is a symmetry about the directions, and user did not know the exact direction from object A to object B before and after the occurrence of cross event. But in proposition 2, when directional constraints are added, they remove the confusion about directions and symmetry of topological relations that object A (john) crosses the object B (ground) from north to south. It justifies that how the topological relations in two objects change, and what was the temporal order of occurrence the primitive events.In our approach, we used a method of combined topological and directional (CTD) relations [11], more suitable for reasoning about the moving objects and developing the motion events. Situation is represented by relationship between the considered entities. It is natural to represent the information using relations. Events can be expressed by interpreting collective behavior of physical objects over a certain period of time. The main focus of this work is to formalize the spatiotemporal relations and the spatiotemporal events in a systematic way.This paper is arranged as follows. Related work is discussed in Section2 and Section 3 composed of preliminary definitions. Section 4 explains the combined topological and directional relations method, spatiotemporal relations are defined in Sections 5 and 6 compose on geometric representation of some motion events and these motion events are defined in Section 7. Section 8 concludes the paper.


## 2. Related Work

A moving object occupies different positions at different time instants. Relative motion means that the object changes its position with respect to another object. This relative motion can be studied through different aspects of space, and spatial relations are one of them. A set of spatial relations which hold for one snapshot is considered as a primitive event, then spatial relations between moving objects for an interval are characterized as spatiotemporal relations or spatiotemporal events.Commonly adopted approaches are qualitative and domain-based such as qualitative trajectory calculus (QTC) [12, 13]. This describes relation between moving point objects. Hornsby and Egenhofer [14] modeled the different spatiotemporal relations between moving objects on road network. All these relations represent certain class of motion and objects are approximated as point objects. When objects are under motion, especially on road networks, the relations are purely directional relations, where the objects change their position, but do not change the topological structure of scene.A mereotopological approach is extend to define spatiotemporal relations and a notation of temporal slice is used, where temporal slice is called an episode of history for a given interval [15, 16]. The primitive events can be defined using the Allen’s temporal logic and defining relation holds(P,i) (property Pholdsduring the time interval i) [17]. In this method, interval temporal logic is used, and a primitive temporal interval is defined, the smallest interval where the relation does not change. For composite events another property “occurs,” defined as occurs(e,i)= evente,occurs during the intervali, and different hold predicates are combined through logical connectors in a sequential order.Ma and Mc Kevitt [18] described a method based on continuous transection from one state to another state. In this approach, topological relations are computed by the 9-intersection method [19]. This method supports instantaneous point temporal logic, which detects only changes in topological structure of scene at different instants of time. This method is based on point set topology and uses the snapshot model for spatiotemporal data [20–22]. This model of topological relations is used by Muller [23, 24] to model the motion events or motion classes. They model the different motion events which involve the topological changes at each analysis frame. Spatiotemporal relations between moving objects are also effected by the environment regarding its application domain such as modeling the relation cross, enter, leave shows that one object is only on concept level, that is, a region of interest. They are defined for a network, visual tracking, image understanding and activity recognition, or freely moving objects like modeling movement behavior of animals.In [10] provided the strong mathematical and logical bases for defining the general spatiotemporal topological relations and divided them into two categories, namely, stable and unstable spatiotemporal relations. Both stable and unstable spatiotemporal relations play an important role in modeling the spatiotemporal events. World is represented as situation (a primitive event), and action is simply a function from one situation to another. A single snapshot represents a primitive event at an instant t. Events are embedded in time, they have temporal boundaries, they have their relationship to time. They do not occupy space, but they are related to space.Spatiotemporal events are defined as composite events, how their different parts (primitive events) are interrelated. A propertyholds_at(P,t) (property P holds at time instant t) is used along with the instantaneous temporal logic. The primitive events are defined for each snapshot during an interval T using the Allen’s temporal logic and defining the relation holds(P,T) (property P holds during the time interval T) [17], and a relationship between holds and holds_at can be represented as holds(P,T)=forallt∈T, holds_at(P,t). This provides us that a property P holds for an interval T if it holds for every point during the interval. In this method, interval temporal logic is used, and a primitive temporal interval is defined, if T is a zero duration interval, then it represents a snapshot.Motion events are the subclasses of spatiotemporal relations with a temporal ordering in a primitive actions. Motion events, they do not formulate the necessitate of a calculus, they are only logical representation and temporal ordering of existence of different primitive events. Modeling the motion events, where property (P) changes at each instant, it is more suitable to use the sequential logic and use relation, seq_eve(t,e1,e2,s) (event e1 occurs before e2 in S during time t). Composite events are the initial conditions dependant, when an initial primitive event occurs at a certain time point t0, it set up the superclass and name of the possible composite event to be happening.Topological relations have a certain type of locative symmetries, they do not explain the symmetric location of path and motion direction of argument objects. To remove the symmetry of spatial relations about the locative perspective, relevant spatial orientation is added. In language semantics, motion events are divided into three classes: an initial, the median, and terminal [25]. Some sentences can be explained with the help of a single directional relation such as enter, release, touch, and some need two directions like crossandgraze.We used CTD method [11] to develop such motion events, where topological components play role for defining the motion events, directional components are used to overcome the locative symmetries and locative perspective and for other class of motion events, topological components can be used for controlling variables, and directional components become important. We hope this paper will create a bridge between the two approaches of modeling the spatiotemporal events, approach based on interval logic, and that of point logic.


## 3. Preliminary Definitions

In this section, we recall some basic definitions which are frequently used throughout the remainder of the paper.Fuzzy set: a fuzzy setA in a set X is a set of pairs(X,μA(x))suchthat(1)A={(x,μA(x)∣x∈X)}.Fuzzy membership function: a membership functionμ in a set X is a function μ:X→[0,1]. Different fuzzy membership functions are proposed according to the requirements of the applications. For instance, Trapezoidal membership function is defined as(2)μ(x;α,β,γ,δ)=max⁡(min⁡(x-αβ-α,1,δ-xδ-γ),0),
it is written as μ(α,β,γ,δ)(x), where x,α,β,γ,δ∈ℝ∧α<β≤γ<δ.Force histogram: the force histogram attaches a weight to the argument objectA that this lies afterB in direction θ, it is defined as
(3)FAB(θ)=∫-∞+∞F(θ,Aθ(v),Bθ(v))dv.
The definition of force histogram FAB(θ), directly depends upon the definition of real-valued functions ϕ,f, and F used for the treatment of points, segments, and longitudinal sections, respectively [26]. These functions are defined as(4)ϕr(y)={1yrify>00otherwise,f(xI,yIJθ,zJ)=∫xI+yIJθxI+yIJθ+zJ∫0zJϕ(u-w)dwdu,F(θ,Aθ(v),Bθ(v))=∑i=1⋯n,j=1⋯mf(xIi,yIiJjθ,zJj),
where nand m represent the number of segments of object A and object B, respectively, and variables (x, y, z) are explained in Figure 1. These are the definitions of Force histograms, directly depending upon the definition of function ϕ. FAB(θ) is actually a real-valued function.Figure 1
Oriented lineΔθ(v), segment as in case of object B, longitudinal section as in case of object A[27].


## 4. Combined Topological and Directional Relations Method

In this section, we explain different steps of the combined topological and directional relations method. This explains different terms used in computation of combined topological and directional relations.


### 4.1. Oriented Lines, Segments, and Longitudinal Sections

LetA and B be two spatial objects and (v,θ)∈ℝ, where v is any real number and θ∈[0,2π]. Δθ(v) is an oriented line at orientation angle θ, and A∩Δθ(v) is the intersection of object A and oriented line Δθ(v). It is denoted by Aθ(v), called segment of object A and length of its projection interval on x-axis is x. Similarly for object B, where B∩Δθ(v)=Bθ(v) is segment and length of its projection interval on x-axis is z, y is the difference between the minimum of projection points of A∩Δθ(v) and maximum value of projection points of B∩Δθ(v)(for details [27]).In case of polygonal object approximation(x,y,z) can be calculated from intersecting points of line and object boundary, oriented lines are considered which passes through at least one vertex of two polygons. If there exist more than one segment, then it is called longitudinal section as in case of Aθ(v) in Figure 1.


### 4.2. Allen Temporal Relations in Spatial Domain and Fuzziness

Allen [5] introduced the 13 jointly exhaustive and pairwise disjoint (JEPD) interval relations. These relations are 𝒜={<,m,o,s,f,d,eq,di,fi,si,oi,mi,>} with meanings before, meet, overlap, start, finish, during, equal, during_by, finish_by, start_by, overlap_by, meet_by,andafter. All the Allen relations in space are conceptually illustrated in Figure 2.Figure 2
Black segment represents the reference object, and gray segment represents argument object.These relations have a rich support for the topological relations and represent the eight topological relations in one-dimensional spatial domain. Fuzzy Allen relations are used to represent the fuzzy topological relations, where vagueness or fuzziness is represented at the relation’s level.Fuzzification process of Allen relations do not depend upon particular choice of fuzzy membership function. Trapezoidal membership function is used due to flexibility in shape. Letr(I,J) be an Allen relation between segments I (segment of an argument object) and J (segment of an reference object), r′ is the distance between r(I,J) and its conceptional neighborhood. We consider a fuzzy membership function μ:r′→[01]. The fuzzy Allen relations defined in [28] as(5)f<(I,J)=μ(-∞,-∞,-b-3a/2,-b-a)(y)f>(I,J)=μ(0,a/2,∞,∞)(y)fm(I,J)=μ(-b-3a/2,-b-a,-b-a,-b-a/2)(y)fmi(I,J)=μ(-a/2,0,0,a/2)(y)fO(I,J)=μ(-b-a,-b-a/2,-b-a/2,-b)(y)foi(I,J)=μ(-a,-a/2,-a/2,0)(y)ff(I,J)=min⁡(μ(-(b+a)/2,-a,-a,+∞)(y),μ(-3a/2,-a,-a,-a/2)(y),μ(-∞,-∞,z/2,z)(x))ffi(I,J)=min⁡(μ-b-a/2,-b,-b,-b+a/2(y),μ(-∞,-∞,-b,-(b+a)/2)(y),μ(z,2z,+∞,+∞)(x))fs(I,J)=min⁡(μ-b-a/2,-b,-b,-b+a/2(y),μ(-∞,-∞-b,-(b+a)/2)(y),μ(-∞,-∞,z/2,z)(x))fsi(I,J)=min⁡(μ(-(b+a)/2,-a,-a,+∞)(y),μ(-3a/2,-a,-a,-a/2)(y),μ(z,2z,+∞,+∞)(x))fd(I,J)=min⁡(μ(-b,-b+a/2,-3a/2,-a)(y),μ(-∞,-∞,z/2,z)(x))fdi(I,J)=min⁡(μ(-b,-b+a/2,-3a/2,-a)(y),μ(z,2z,+∞,+∞)(x)),
where a=min(x,z),b=max(x,z), x is the length of segment (I), z is the length of segment (J), and (x,y,z) are computed as described in Section 4.1.Most of relations are defined by one membership liked(during), di(during_by), f(finish), and fi(finished_by). In fuzzy set theory, sum of all the relations is one, this gives the definition for fuzzy relation equal. These are the topological relations which represent the fuzziness at relation’s level, for example, here Meet topological relation is represented based on nearness, and length of the smaller interval defines the smooth transition between the Meet(Meet_by) and before(after)relation. In spatial domain, before(after) are called the disjoint topological relations. These relations have the following properties:(6)f<(θ)=f>(θ+π),fm(θ)=fmi(θ+π),fo(θ)=foi(θ+π),ff(θ)=fs(θ+π),ffi(θ)=fsi(θ+π),fd(θ)=fd(θ+π),fdi(θ)=fdi(θ+π),f=(θ)=f=(θ+π).Eight relations are possible combination of eight independent Allen relations in one-dimensional spatial domain. These relations and their reorientation show that the whole 2D space can be explored with the help of 1D Allen relations using the oriented lines varying from (0,π).


### 4.3. Combining Topological and Directional Relations

Eight topological relations represented in point set topology or point less topology between object pair are represented in one-dimensional space by the Allen’s temporal relations in spatial domain. We extend these Allen relations for two-dimensional objects through logical implication, where a2D object is decomposed into parallel segments of a 1D lines in a given direction, and the relation between each pair of line segments is computed.The process of object decomposition is repeated for each direction varying from 0 toπ, two-dimensional topological relations are then defined as it provides us with information about how the objects are relatively distributed. These relations are not jointly exhaustive and pairwise disjoint (JEPD), to obtain JEPD set of topological and directional relations an algorithm was advocated in [11], it provides us with the JEPD set of relations. Objects are approximated through the polygon object approximation. Different steps of computing the combined topological and directional relations are(i)
fix angleθ and draw lines passing through the vertices of polygons representing the objects;(ii)
for each line, compute the variables(x,y,z) as depicted in Section 4.1 and compute Allen relation for each segment as given by (5). In case of longitudinal sections, use fuzzy operators to integrate the information, usually the disjunction operators are suitable. These relations are computed for each line in a direction, then obtained information is integrated into a single value. Normalize these relations for a direction θ by dividing sum of all Allen relations to each Allen relation;(iii)
these normalized fuzzy Allen relation is then multiplied to a fuzzy directional set to find the degree of an Allen relation in a direction;(iv)
for qualitative directions, this information is summarized, and different topological relations with directional contents are defined, such as(7)fE=∑θ=0π/4Ar2×cos⁡2⁡(2θ)+∑θ=3π/4πAr1×cos⁡2⁡(2θ),
where f represents a topological relation, and E represents the direction Ar1 is the reorientation of Ar2;(v)
this information is represented in a matrix, this matrix represents fuzzy spatial information;(vi)
these fuzzy spatial relations are defuzzified by an algorithm, this provides us with final topological and directional relations between the object pair. These topological and directional relations are JEPD.This model describes well the possible topological relations between every sort of objects.


## 4.1. Oriented Lines, Segments, and Longitudinal Sections

LetA and B be two spatial objects and (v,θ)∈ℝ, where v is any real number and θ∈[0,2π]. Δθ(v) is an oriented line at orientation angle θ, and A∩Δθ(v) is the intersection of object A and oriented line Δθ(v). It is denoted by Aθ(v), called segment of object A and length of its projection interval on x-axis is x. Similarly for object B, where B∩Δθ(v)=Bθ(v) is segment and length of its projection interval on x-axis is z, y is the difference between the minimum of projection points of A∩Δθ(v) and maximum value of projection points of B∩Δθ(v)(for details [27]).In case of polygonal object approximation(x,y,z) can be calculated from intersecting points of line and object boundary, oriented lines are considered which passes through at least one vertex of two polygons. If there exist more than one segment, then it is called longitudinal section as in case of Aθ(v) in Figure 1.


## 4.2. Allen Temporal Relations in Spatial Domain and Fuzziness

Allen [5] introduced the 13 jointly exhaustive and pairwise disjoint (JEPD) interval relations. These relations are 𝒜={<,m,o,s,f,d,eq,di,fi,si,oi,mi,>} with meanings before, meet, overlap, start, finish, during, equal, during_by, finish_by, start_by, overlap_by, meet_by,andafter. All the Allen relations in space are conceptually illustrated in Figure 2.Figure 2
Black segment represents the reference object, and gray segment represents argument object.These relations have a rich support for the topological relations and represent the eight topological relations in one-dimensional spatial domain. Fuzzy Allen relations are used to represent the fuzzy topological relations, where vagueness or fuzziness is represented at the relation’s level.Fuzzification process of Allen relations do not depend upon particular choice of fuzzy membership function. Trapezoidal membership function is used due to flexibility in shape. Letr(I,J) be an Allen relation between segments I (segment of an argument object) and J (segment of an reference object), r′ is the distance between r(I,J) and its conceptional neighborhood. We consider a fuzzy membership function μ:r′→[01]. The fuzzy Allen relations defined in [28] as(5)f<(I,J)=μ(-∞,-∞,-b-3a/2,-b-a)(y)f>(I,J)=μ(0,a/2,∞,∞)(y)fm(I,J)=μ(-b-3a/2,-b-a,-b-a,-b-a/2)(y)fmi(I,J)=μ(-a/2,0,0,a/2)(y)fO(I,J)=μ(-b-a,-b-a/2,-b-a/2,-b)(y)foi(I,J)=μ(-a,-a/2,-a/2,0)(y)ff(I,J)=min⁡(μ(-(b+a)/2,-a,-a,+∞)(y),μ(-3a/2,-a,-a,-a/2)(y),μ(-∞,-∞,z/2,z)(x))ffi(I,J)=min⁡(μ-b-a/2,-b,-b,-b+a/2(y),μ(-∞,-∞,-b,-(b+a)/2)(y),μ(z,2z,+∞,+∞)(x))fs(I,J)=min⁡(μ-b-a/2,-b,-b,-b+a/2(y),μ(-∞,-∞-b,-(b+a)/2)(y),μ(-∞,-∞,z/2,z)(x))fsi(I,J)=min⁡(μ(-(b+a)/2,-a,-a,+∞)(y),μ(-3a/2,-a,-a,-a/2)(y),μ(z,2z,+∞,+∞)(x))fd(I,J)=min⁡(μ(-b,-b+a/2,-3a/2,-a)(y),μ(-∞,-∞,z/2,z)(x))fdi(I,J)=min⁡(μ(-b,-b+a/2,-3a/2,-a)(y),μ(z,2z,+∞,+∞)(x)),
where a=min(x,z),b=max(x,z), x is the length of segment (I), z is the length of segment (J), and (x,y,z) are computed as described in Section 4.1.Most of relations are defined by one membership liked(during), di(during_by), f(finish), and fi(finished_by). In fuzzy set theory, sum of all the relations is one, this gives the definition for fuzzy relation equal. These are the topological relations which represent the fuzziness at relation’s level, for example, here Meet topological relation is represented based on nearness, and length of the smaller interval defines the smooth transition between the Meet(Meet_by) and before(after)relation. In spatial domain, before(after) are called the disjoint topological relations. These relations have the following properties:(6)f<(θ)=f>(θ+π),fm(θ)=fmi(θ+π),fo(θ)=foi(θ+π),ff(θ)=fs(θ+π),ffi(θ)=fsi(θ+π),fd(θ)=fd(θ+π),fdi(θ)=fdi(θ+π),f=(θ)=f=(θ+π).Eight relations are possible combination of eight independent Allen relations in one-dimensional spatial domain. These relations and their reorientation show that the whole 2D space can be explored with the help of 1D Allen relations using the oriented lines varying from (0,π).


## 4.3. Combining Topological and Directional Relations

Eight topological relations represented in point set topology or point less topology between object pair are represented in one-dimensional space by the Allen’s temporal relations in spatial domain. We extend these Allen relations for two-dimensional objects through logical implication, where a2D object is decomposed into parallel segments of a 1D lines in a given direction, and the relation between each pair of line segments is computed.The process of object decomposition is repeated for each direction varying from 0 toπ, two-dimensional topological relations are then defined as it provides us with information about how the objects are relatively distributed. These relations are not jointly exhaustive and pairwise disjoint (JEPD), to obtain JEPD set of topological and directional relations an algorithm was advocated in [11], it provides us with the JEPD set of relations. Objects are approximated through the polygon object approximation. Different steps of computing the combined topological and directional relations are(i)
fix angleθ and draw lines passing through the vertices of polygons representing the objects;(ii)
for each line, compute the variables(x,y,z) as depicted in Section 4.1 and compute Allen relation for each segment as given by (5). In case of longitudinal sections, use fuzzy operators to integrate the information, usually the disjunction operators are suitable. These relations are computed for each line in a direction, then obtained information is integrated into a single value. Normalize these relations for a direction θ by dividing sum of all Allen relations to each Allen relation;(iii)
these normalized fuzzy Allen relation is then multiplied to a fuzzy directional set to find the degree of an Allen relation in a direction;(iv)
for qualitative directions, this information is summarized, and different topological relations with directional contents are defined, such as(7)fE=∑θ=0π/4Ar2×cos⁡2⁡(2θ)+∑θ=3π/4πAr1×cos⁡2⁡(2θ),
where f represents a topological relation, and E represents the direction Ar1 is the reorientation of Ar2;(v)
this information is represented in a matrix, this matrix represents fuzzy spatial information;(vi)
these fuzzy spatial relations are defuzzified by an algorithm, this provides us with final topological and directional relations between the object pair. These topological and directional relations are JEPD.This model describes well the possible topological relations between every sort of objects.


## 5. Spatiotemporal Relations

Spatiotemporal relations can be defined as spatial relation holds for an interval, that is, relation holds for a certain time interval, and it does not change. In spatiotemporal object theory it is defined as(P) a spatial relation is a relation holding between all temporal slices of two entities during the relevant period. All eight spatiotemporal relations are defined in terms of theorems.


### 5.1. Spatiotemporal Relation

Theorem 1.
A spatiotemporal disjoint relation between object pair(X,Y) holds  during the interval T, that is, D(XY,T)⇔forallt∈T,D(XY,t) holds.Proof.
(⇒) A spatiotemporal disjointD(XY,T) relation is defined as object pair (X,Y) are disjoint during the interval T if X≡tY(X is temporally equivalent to Y). Now let us consider the partition of interval T=[tatb], then its partition can be taken as ta=t1<t2<t3⋯<tn=tb. Each ti∈T,i=1,2,…,n represents discrete points of interval T, and this representation is equivalent to a snapshot. Typically a snapshot is a sampling process, which represents zero duration temporal slice of a spatiotemporal object. There are n snapshots in interval T, as a result a disjoint topological relation exists for each snapshot separately. Thus, for all t∈T,D(XY,t) holds.
(⇐) Let us consider n snapshots where the temporal ordering holds, that is, t1,t2,⋯,tn such that t1<t2<t3⋯<tn and all these points form partition of an interval T. If the disjoint topological relation holds at discrete points, it means that D(XY,t1)∧D(XY,t2)→D(XY,[t1t2]). If the disjoint topological relation holds between object pair, it means that both the objects are temporally equivalent (x≡ty). Hence, D(XY,T) holds during the whole interval T.Theorem 2.
A spatiotemporal relation Meet M(XY,T)  holds⇔∃ti∈T such that M(XY,ti)∧for all tj∈T∧tj≠ti⇒D(XY,tj) holds.Proof.
(⇒) A spatiotemporal relation meetM(XY,T) holds between object pair (X,Y) over interval T, where X≡tY. Let ta=t1<t2<t3<⋯<tn=tb be partition of interval T=[tatb], if forallti∈T,M(XY,ti) holds, then a stable topological relation M(XY,T) holds. We consider on contrary, that ∃tj, where the topological relation M(XY,tj) does not hold, but it holds at M(XY,tj-1), then according to the temporal logic and continuity of topological relations ◯(M(XY,tj-1))⇒(D(XY,tj)∨M(XY,tj)∨PO(XY,tj). This shows that any of the three relations is possible (◯stands for future position). If PO(XY,tj) holds, then the whole spatiotemporal relation is changed, and it becomes the spatiotemporal partial overlap relation. This possibility is ruled out. In other case, spatiotemporal relation remains meet, as j is an arbitrary variable, this shows the minimum condition. Hence, ∃t∈T,s.t.M(XY,t) holds.
(⇐) Let us consider that there are n snapshots in an order, which construct an interval T. Now consider that there exists at least one snapshot during whole interval, where spatial meet relation holds, and for all other snapshots, the spatial relation is disjoint. This shows that during temporal interval T, the unstable spatiotemporal meet relation holds. It satisfies the minimum conditions for a spatiotemporal meet relation, hence M(XY,T) holds during interval T.Theorem 3.
A spatiotemporal partial overlap  (PO) relation holds over intervalT, that is, PO(XY,T)⇔∃t∈T,s.t.PO(XY,t).Proof.
Spatiotemporal relations have the spatial and temporal boundaries. A stable spatiotemporal relation holds during the temporal slice, If it holds at every point of the interval. As temporal slice is the union of finite points of temporal domain, spatiotemporalpartial overlap holds during the whole slice, if this relation holds at least one sampling point (snapshot), at remaining points any of the spatial relation may exist. Hence∃t∈T,s.t.,PO(XY,t)∧(t1≠t,CO(XY,t1)∨M(XY,t1)∨D(XY,t1)) (CO(XY) stands for complete overlap of objects (XY), s.t., CO(XY)=TPP(XY)∨NTPP(XY)∨TPPI(XY)∨NTPPI(XY)). If there does not exist such a t1, then the relation holds for every t∈T, which shows that a stable PO topological relation holds.
(⇐) We suppose on contrary that ∄t∈T,s.t.PO(XY,t) holds. It means that at all points either the binary topological relations are complete overlap or disjoint and meet. If the relations are complete overlap, that is, for all t∈T,CO(XY,t) holds, then the spatiotemporal relation will be a part of complete overlap. In case of other choice that ∃t∈T, s.t.,M(XY,t) or for all t∈T,s.t.,M(XY,t) holds, then the spatiotemporal topological relation will be stable or unstable meet and for case for all t∈T,s.t.,D(XY,t), the topological relation will be disjoint. The choice, that ∃t1∈T,s.t.,M(XY,t) and ∃t2∈T,s.t.,CO(XY,t2 holds is impossible because in a such a case ∃t∈T∧t1<t<t2, s.t.,PO(XY,t) holds (continuity of spatial relations).Theorem 4.
A spatiotemporal tangent proper part (TPP) relation holds over intervalT, that is,TPP(XY,T)⇔∃t1∈T,such thatTPP(XY,t1)for all t2∧t2≠t1, NTPP(XY,t2) holds.Proof.
(⇒) A spatiotemporal topological relation TPP(XY) holds between the object pair X,Y during the interval T. Now let us consider that interval T consist of n snapshots, if this relation holds for every snapshot then a spatiotemporal stable topological relation holds. In other case, there are two possibilities that for all t1≠t2, there exists a topological relation TPP(XY,t1), and for t2 either the topological relation is PO(XY,t2) or NTPP(XY,t2) due to continuity of topological relations between moving objects. For PO(XY,t2), the spatiotemporal topological relation is changed, and it becomes the spatiotemporal PO topological relation, this possibility is ruled out. It remains that NTPP(XY,t2), if this relation holds and t2 is an arbitrary point, so the relation becomes the unstable spatiotemporal TPP(XY,T).
(⇐) Consider that ∃ti such that TPP(XY,ti) holds. We consider on contrary that ∃ti+1∨ti-1 such that NTPP(XY,ti-1) or NTPP(XY,ti+1) does not holds. Then, possible topological relations at ti-1 are TPP(XY,ti-1),PO(XY,ti-1) similarly for ti+1. Other possibilities are ruled out due to the continuity of topological relations, and EQ(XY,ti-1) does not hold because objects are considered under motion, and expansion or zooming of one object is not allowed.
In case the topological relationPO(XY,ti-1) holds, then the whole spatiotemporal relation over the interval T becomes partial overlap. Similarly for instant ti+1 and i is an arbitrary point, so this is impossible for whole the interval T. For the topological relation TPP(XY,ti-1), the spatiotemporal relation becomes the stable spatiotemporal TPP.Theorem 5.
A spatiotemporal nontangent proper part  (NTPP) relation holds over intervalT, that is, NTPP(XY,T)⇔ for allt∈T,NTPP(XY,t) holds.Proof.
(⇒) Let us suppose on contrary that ∃ti∈Ts.t.NTPP(XY,ti) does not hold, and at temporal points ti-1,ti+1 the relation NTPP(XY,ti-1),NTPP(XY,ti+1), holds. Then continuity of spatial relations forces the existence of TPP(XY,ti) or EQ(XY,ti) spatial relations. This contradicts the existence of the spatiotemporal NTPP(XY,T) relation. Hence, forallt∈T,NTPP(XY,t) holds.
(⇐) It is given that forallt∈T,NTPP(XY,t) holds. If a spatial relation between object pair holds at every point of the interval, then it means that it holds throughout the interval, that is, NTPP(XY,T) holds.Theorem 6.
A spatiotemporal tangent proper part inverse  (TPP) relation holds over intervalT, that is, TPPI(XY,T)⇔∃t∈T,s.t.TPPI(XY,t)∧forallt1≠t,NTPPI(XY,t1) holds.Proof.
Proof is similar to theTPP(XY,T), just replace TPP by TPPI and NTPP by NTPPI.Theorem 7.
A spatiotemporal nontangent proper part inverse (NTPPI) relation holds over temporal intervalT, that is, NTPPI(XY,T)⇔forallt∈TandNTPPI(XY,t) holds.Proof.
Proof is similar to theNTPP(XY,T).Theorem 8.
A spatiotemporal relation equal(EQ) holds between the object pairXY, EQ(XY,T)⇔forallt∈T, s.t.,EQ(XY,t) holds.Proof.
(⇒) We suppose on contrary that there exists a t∈T, where the EQ(XY,t) relation does not hold. It shows that there are two possibilities that either the relation at t is a complete overlap or partial overlap. If the relation at t is complete overlap, then the spatiotemporal relation becomes TPP or TPPI. In the second case, the spatiotemporal relation becomes the PO(XY,T) during the whole interval. Thus, both cases prove the contrary conditions, hence ∄ts.t.EQ(XY,t) does not hold, that is, for allt∈T,EQ(XY,t) holds.
(⇐) Converse of this proof is very simple and straight forward. Let T be the interval for which we have to define the spatiotemporal relation, both the objects are temporally comparable (X≡tY). Let t∈T be an arbitrary point of the interval and relation EQ(XY,t) holds. Since t is an arbitrary point so, the relation holds throughout the interval T, that is, EQ(XY,T) holds.


## 5.1. Spatiotemporal Relation

Theorem 1.
A spatiotemporal disjoint relation between object pair(X,Y) holds  during the interval T, that is, D(XY,T)⇔forallt∈T,D(XY,t) holds.Proof.
(⇒) A spatiotemporal disjointD(XY,T) relation is defined as object pair (X,Y) are disjoint during the interval T if X≡tY(X is temporally equivalent to Y). Now let us consider the partition of interval T=[tatb], then its partition can be taken as ta=t1<t2<t3⋯<tn=tb. Each ti∈T,i=1,2,…,n represents discrete points of interval T, and this representation is equivalent to a snapshot. Typically a snapshot is a sampling process, which represents zero duration temporal slice of a spatiotemporal object. There are n snapshots in interval T, as a result a disjoint topological relation exists for each snapshot separately. Thus, for all t∈T,D(XY,t) holds.
(⇐) Let us consider n snapshots where the temporal ordering holds, that is, t1,t2,⋯,tn such that t1<t2<t3⋯<tn and all these points form partition of an interval T. If the disjoint topological relation holds at discrete points, it means that D(XY,t1)∧D(XY,t2)→D(XY,[t1t2]). If the disjoint topological relation holds between object pair, it means that both the objects are temporally equivalent (x≡ty). Hence, D(XY,T) holds during the whole interval T.Theorem 2.
A spatiotemporal relation Meet M(XY,T)  holds⇔∃ti∈T such that M(XY,ti)∧for all tj∈T∧tj≠ti⇒D(XY,tj) holds.Proof.
(⇒) A spatiotemporal relation meetM(XY,T) holds between object pair (X,Y) over interval T, where X≡tY. Let ta=t1<t2<t3<⋯<tn=tb be partition of interval T=[tatb], if forallti∈T,M(XY,ti) holds, then a stable topological relation M(XY,T) holds. We consider on contrary, that ∃tj, where the topological relation M(XY,tj) does not hold, but it holds at M(XY,tj-1), then according to the temporal logic and continuity of topological relations ◯(M(XY,tj-1))⇒(D(XY,tj)∨M(XY,tj)∨PO(XY,tj). This shows that any of the three relations is possible (◯stands for future position). If PO(XY,tj) holds, then the whole spatiotemporal relation is changed, and it becomes the spatiotemporal partial overlap relation. This possibility is ruled out. In other case, spatiotemporal relation remains meet, as j is an arbitrary variable, this shows the minimum condition. Hence, ∃t∈T,s.t.M(XY,t) holds.
(⇐) Let us consider that there are n snapshots in an order, which construct an interval T. Now consider that there exists at least one snapshot during whole interval, where spatial meet relation holds, and for all other snapshots, the spatial relation is disjoint. This shows that during temporal interval T, the unstable spatiotemporal meet relation holds. It satisfies the minimum conditions for a spatiotemporal meet relation, hence M(XY,T) holds during interval T.Theorem 3.
A spatiotemporal partial overlap  (PO) relation holds over intervalT, that is, PO(XY,T)⇔∃t∈T,s.t.PO(XY,t).Proof.
Spatiotemporal relations have the spatial and temporal boundaries. A stable spatiotemporal relation holds during the temporal slice, If it holds at every point of the interval. As temporal slice is the union of finite points of temporal domain, spatiotemporalpartial overlap holds during the whole slice, if this relation holds at least one sampling point (snapshot), at remaining points any of the spatial relation may exist. Hence∃t∈T,s.t.,PO(XY,t)∧(t1≠t,CO(XY,t1)∨M(XY,t1)∨D(XY,t1)) (CO(XY) stands for complete overlap of objects (XY), s.t., CO(XY)=TPP(XY)∨NTPP(XY)∨TPPI(XY)∨NTPPI(XY)). If there does not exist such a t1, then the relation holds for every t∈T, which shows that a stable PO topological relation holds.
(⇐) We suppose on contrary that ∄t∈T,s.t.PO(XY,t) holds. It means that at all points either the binary topological relations are complete overlap or disjoint and meet. If the relations are complete overlap, that is, for all t∈T,CO(XY,t) holds, then the spatiotemporal relation will be a part of complete overlap. In case of other choice that ∃t∈T, s.t.,M(XY,t) or for all t∈T,s.t.,M(XY,t) holds, then the spatiotemporal topological relation will be stable or unstable meet and for case for all t∈T,s.t.,D(XY,t), the topological relation will be disjoint. The choice, that ∃t1∈T,s.t.,M(XY,t) and ∃t2∈T,s.t.,CO(XY,t2 holds is impossible because in a such a case ∃t∈T∧t1<t<t2, s.t.,PO(XY,t) holds (continuity of spatial relations).Theorem 4.
A spatiotemporal tangent proper part (TPP) relation holds over intervalT, that is,TPP(XY,T)⇔∃t1∈T,such thatTPP(XY,t1)for all t2∧t2≠t1, NTPP(XY,t2) holds.Proof.
(⇒) A spatiotemporal topological relation TPP(XY) holds between the object pair X,Y during the interval T. Now let us consider that interval T consist of n snapshots, if this relation holds for every snapshot then a spatiotemporal stable topological relation holds. In other case, there are two possibilities that for all t1≠t2, there exists a topological relation TPP(XY,t1), and for t2 either the topological relation is PO(XY,t2) or NTPP(XY,t2) due to continuity of topological relations between moving objects. For PO(XY,t2), the spatiotemporal topological relation is changed, and it becomes the spatiotemporal PO topological relation, this possibility is ruled out. It remains that NTPP(XY,t2), if this relation holds and t2 is an arbitrary point, so the relation becomes the unstable spatiotemporal TPP(XY,T).
(⇐) Consider that ∃ti such that TPP(XY,ti) holds. We consider on contrary that ∃ti+1∨ti-1 such that NTPP(XY,ti-1) or NTPP(XY,ti+1) does not holds. Then, possible topological relations at ti-1 are TPP(XY,ti-1),PO(XY,ti-1) similarly for ti+1. Other possibilities are ruled out due to the continuity of topological relations, and EQ(XY,ti-1) does not hold because objects are considered under motion, and expansion or zooming of one object is not allowed.
In case the topological relationPO(XY,ti-1) holds, then the whole spatiotemporal relation over the interval T becomes partial overlap. Similarly for instant ti+1 and i is an arbitrary point, so this is impossible for whole the interval T. For the topological relation TPP(XY,ti-1), the spatiotemporal relation becomes the stable spatiotemporal TPP.Theorem 5.
A spatiotemporal nontangent proper part  (NTPP) relation holds over intervalT, that is, NTPP(XY,T)⇔ for allt∈T,NTPP(XY,t) holds.Proof.
(⇒) Let us suppose on contrary that ∃ti∈Ts.t.NTPP(XY,ti) does not hold, and at temporal points ti-1,ti+1 the relation NTPP(XY,ti-1),NTPP(XY,ti+1), holds. Then continuity of spatial relations forces the existence of TPP(XY,ti) or EQ(XY,ti) spatial relations. This contradicts the existence of the spatiotemporal NTPP(XY,T) relation. Hence, forallt∈T,NTPP(XY,t) holds.
(⇐) It is given that forallt∈T,NTPP(XY,t) holds. If a spatial relation between object pair holds at every point of the interval, then it means that it holds throughout the interval, that is, NTPP(XY,T) holds.Theorem 6.
A spatiotemporal tangent proper part inverse  (TPP) relation holds over intervalT, that is, TPPI(XY,T)⇔∃t∈T,s.t.TPPI(XY,t)∧forallt1≠t,NTPPI(XY,t1) holds.Proof.
Proof is similar to theTPP(XY,T), just replace TPP by TPPI and NTPP by NTPPI.Theorem 7.
A spatiotemporal nontangent proper part inverse (NTPPI) relation holds over temporal intervalT, that is, NTPPI(XY,T)⇔forallt∈TandNTPPI(XY,t) holds.Proof.
Proof is similar to theNTPP(XY,T).Theorem 8.
A spatiotemporal relation equal(EQ) holds between the object pairXY, EQ(XY,T)⇔forallt∈T, s.t.,EQ(XY,t) holds.Proof.
(⇒) We suppose on contrary that there exists a t∈T, where the EQ(XY,t) relation does not hold. It shows that there are two possibilities that either the relation at t is a complete overlap or partial overlap. If the relation at t is complete overlap, then the spatiotemporal relation becomes TPP or TPPI. In the second case, the spatiotemporal relation becomes the PO(XY,T) during the whole interval. Thus, both cases prove the contrary conditions, hence ∄ts.t.EQ(XY,t) does not hold, that is, for allt∈T,EQ(XY,t) holds.
(⇐) Converse of this proof is very simple and straight forward. Let T be the interval for which we have to define the spatiotemporal relation, both the objects are temporally comparable (X≡tY). Let t∈T be an arbitrary point of the interval and relation EQ(XY,t) holds. Since t is an arbitrary point so, the relation holds throughout the interval T, that is, EQ(XY,T) holds.


## 6. Visual Interpretation: A Three-Dimensional View

Geometrical figures can better elaborate concepts, a moving object changes its position at each instantt. These objects in a spatiotemporal domain can be represented by their envelops, a two-dimensional object becomes volume. Here, spatiotemporal meet and partially overlap relations are represented by their envelops in Figures 3(a)–3(d) and 4(a)–4(h). These are possible representation of motion events. Spatial relations between moving objects are used in modeling the motion verbs or motion events in natural language processing. A set of motion relations is introduced that capture semantic between pairs of moving objects. This information is useful about reasoning the moving objects.SpatiotemporalMeet relation (unstable meet).
(a)
touch(b)
excurse(c)
snap(d)
releaseSpatiotemporalpartial_Overlap relation (unstable overlap).
(a)
enter(b)
leave(c)
cross(d)
graze(e)
into(f)
out of(g)
melt(h)
spring


## 7. Modeling Motion Classes

Visual images may illustrate cases of a definition, giving us a more visual grasp of its applications. They may help us understanding the description of a mathematical situation or steps in reasoning. These relations can be defined as the transection of relations at timet1 to t2. This change may be in topological or metric relations, and different classes of spatial relations, between moving objects have been defined [12, 13, 29, 30]. Motion classes based on intuitive logics or motion verbs have been defined in [31] by Phillipe Muller and Ralf H. Güting and Markus Schneider used in database. We define in this paper only the motion events, where topological relations capture changes between situations. These motion events can be defined using the predicates holds_at, holds,occurs_at,   occurs and sequence. In next section, seq_eve represents the sequence event.


### 7.1. Unstable Meet Spatiotemporal Relation

Unstable spatiotemporal relation is a relation where objects changes their states at each time instant. A spatiotemporal meet relation is characterized by different motion events depending upon the logical and temporal order of different states or primitive events.Touch(XY,T): A spatiotemporal meet relation can be characterized as a motion event Touch, s.t. ∃t1,t2,t3,∈Tandt1<t2<t3, where primitive events occur in an order and defined as(8)Touch(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(D(XY,t3))).An institutive view of this spatiotemporal relation is shown in Figure3(a). This relation can be expressed by a single direction, where a meet topological relation holds. It means(9)Dir(Touch(XY,T))=holds(Dir(XY,t2)).Snap(XY,T): A spatiotemporal meet relation is called a Snap if ∃t1,t2∈Tandt1<t2 such that(10)Snap(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))).A geometric representation is shown in Figure3(c). This relation can be expressed by a single direction, where a meet topological relation holds. It means(11)Dir(Snap(XY,T))=holds(Dir(XY,t2)).Release(XY,T): A spatiotemporal Meet(XY,T) is called Release(XY,T), read as X releases Y during interval T if it has a certain temporal ordering, ∃t1,t2∈Tandt1<t2such that(12)Release(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t1))).A three-dimensional geometric view of this relation is shown in Figure3(d). This relation can be expressed by a single direction, which is the destination direction. For example, object X  releases (motion event) object Y towards East(destination direction). Direction for such a relation is defined as(13)Dir(Release(XY,T))=holds(Dir(XY,t2)).Bypass(XY,T): A spatiotemporal Meet(XY,T) is called a Bypass(XY,T), read as X bypasses Y during interval T if it has a certain temporal ordering, that is, ∃t1,t2,t3,t4∈T such thatt1<t2<t3<t4(14)Bypass(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(M(XY,t3))∧holds(D(XY,t4))).This relation can be expressed by a single direction, where a meet topological relation holds. It means(15)Dir(Touch(XY,T))=holds(Dir(XY,t2)).Excurse(XY,T): A spatiotemporal Meet(XY,T) is called a Excurse(XY,T), read as X excurse Y during interval T if it has a certain temporal ordering, an intuitive view of this relation is shown in Figure 3(b). ∃t1,t2,t3∈T,s.t.,t1<t2<t3(16)Excurse(XY,T)=seq_eve(holds(M(XY,t1))∧holds(D(XY,t2))∧holds(M(XY,t3))).This relation is expressed by an initial and destination directions, the direction for this relation can be defined as(17)Dir(Excurse(XY,T))=seq_eve(holds(Dir(XY,t1))∧holds(Dir(XY,t3))).


### 7.2. Unstable Overlap Spatiotemporal Relation

Enter(XY,T): An unstable spatiotemporal overlap relation is calledEnter, generally denoted by Enter(XY,T) and read as “X enters in Y during interval T.” If ∃t1,t2,t3,t4∈Tsuchthatt1<t2<t3<t4, then relation is defined as(18)Enter(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(PO(XY,t3))∧holds(TPP(XY,t4))).An intuitive view of this relation is shown in Figure4(a). This relation can be expressed by a single direction because the destination point is inside and can be expressed without direction, a direction for the Enter spatiotemporal event is the direction where a meet topological relation holds, that is,(19)Dir(Enter(XY,T))=holds(Dir(XY,t2)).Leave(XY,T): A spatiotemporal partial overlap relation is called Leave, denoted as Leave(XY,T) “X leaves Y during interval T”. If ∃t1,t2,t3,t4∈Tsuchthatt1<t2<t3<t4, then relation is defined as(20)Leave(XY,T)=seq_eve(holds(NTPP(XY,t1))∧holds(TPP(XY,t2))∧holds(PO(XY,t3))∧holds(M(XY,t4))∧holds(D(XY,t5))).An intuitive view of this relation is shown in Figure4(b). This relation can be expressed by a single direction which is the destination point, that is(21)Dir(Leave(XY,T))=holds(Dir(XY,t4)).Cross(XY,T): A spatiotemporal relation Cross(XY,T) “X crosses Y during the interval T.” Its geometric view is given in Figure 4(c). If ∃t1,t2,t3,…,t9∈Tsuchthatt1<t2<⋯<t9, then relation is defined as(22)Cross(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(PO(XY,t3))∧holds(TPP(XY,t4))∧holds(NTPP(XY,t5))∧holds(TPP(XY,t6))∧holds(PO(XY,t7))∧holds(M(XY,t8))∧holds(D(XY,t9))).This spatiotemporal relation is expressed by a initial as well as destination direction such as objectX crosses (motion event) objectY from north (direction) towards east(direction) during the interval T:(23)Dir(Cross(XY,T))=seq_eve(holds(Dir(XY,t1))∧holds(Dir(XY,t9))).Into(XY,T): A spatiotemporal relationInto(XY,T) read as “X get into Y during the interval T.” If ∃t1,t2,t3∈Tsuchthatt1<t2<t3, then relation is defined as(24)Into(XY,T)=seq_eve(holds(M(XY,t1))∧holds(PO(XY,t2))∧holds(TPP(XY,t3))).Its three-dimensional geometric view is given in Figure4(e). This relation can be expressed by a single direction in language semantics, where a meet topological relation holds. For example, object A get into(spatiotemporal event) object B from north (direction). It means.(25)Dir(Into(XY,T))=holds(Dir(XY,t1)).Out_of(XY,T): A spatiotemporal relation Outof(XY,T) read as “X comes out of Y during the interval T,” its intuitive view is considered in Figure 4(f). If ∃t1,t2,t3,t4∈Tsuchthatt1<t2<t3<t4, then relation is defined as(26)Out_of(XY,T)=seq_eve(holds(TPP(XY,t1))∧holds(PO(XY,t2))∧holds(D(XY,t3))).This relation can be expressed by a single direction. ObjectX go out_of(motion event) object Y towards east(direction), where a meet topological relation holds. It means
(27)Dir(out_of(XY,T))=holds(Dir(XY,t3)).Melt(XY,T): A spatiotemporal relation Melt(XY,T) read as “X,Y melts during the interval T”. If ∃t1,t2,t3,t4∈Tsuchthatt1<t2<t3<t4, then relation is defined as(28)Melt(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(PO(XY,t3))∧holds(EQ(XY,t4))).An intuitive view of this relation is shown in Figure4(g). This relation can be expressed by a single direction because its destination point is dimensionless. This can be its direction, where initial spatial relation holds:(29)Dir(Melt(XY,T))=holds(Dir(XY,t1)).Spring(XY,T): A spatiotemporal relation Spring(XY,T) also called Separate(XY,T) read as “X separates Y during the interval T.” If ∃t1,t2,t3,t4∈Tsuchthatt1<t2<t3<t4, then relation is defined as(30)Spring(XY,T)=seq_eve(holds(EQ(XY,t1))∧holds(PO(XY,t2))∧holds(M(XY,t3))∧holds(D(XY,t4))).Its three-dimensional geometric view is given in Figure4(h). This relation can be expressed by a single direction because its destination point is dimensionless. This can be its direction, where terminal spatial relation holds:(31)Dir(Spring(XY,T))=holds(Dir(XY,t4)).Graze(XY,T): A spatiotemporal relation Graze(XY,T) read as “X grazes Y during the interval T.” If ∃t1,t2,t3,t4,t5∈Tsuchthatt1<t2<t3<t4<t5, then relation is defined as(32)Graze(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(PO(XY,t3))∧holds(M(XY,t4))∧holds(D(XY,t5))).This relation is represented in a three-dimensional perspective in Figure4(d). This spatiotemporal relation is expressed by an initial as well as destination direction such as object X grazes (motion event) object Y from north(direction) toward east(direction):(33)Dir(Graze(XY,T))=seq_eve(holds(Dir(XY,t1))∧holds(Dir(XY,t4))).


## 7.1. Unstable Meet Spatiotemporal Relation

Unstable spatiotemporal relation is a relation where objects changes their states at each time instant. A spatiotemporal meet relation is characterized by different motion events depending upon the logical and temporal order of different states or primitive events.Touch(XY,T): A spatiotemporal meet relation can be characterized as a motion event Touch, s.t. ∃t1,t2,t3,∈Tandt1<t2<t3, where primitive events occur in an order and defined as(8)Touch(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(D(XY,t3))).An institutive view of this spatiotemporal relation is shown in Figure3(a). This relation can be expressed by a single direction, where a meet topological relation holds. It means(9)Dir(Touch(XY,T))=holds(Dir(XY,t2)).Snap(XY,T): A spatiotemporal meet relation is called a Snap if ∃t1,t2∈Tandt1<t2 such that(10)Snap(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))).A geometric representation is shown in Figure3(c). This relation can be expressed by a single direction, where a meet topological relation holds. It means(11)Dir(Snap(XY,T))=holds(Dir(XY,t2)).Release(XY,T): A spatiotemporal Meet(XY,T) is called Release(XY,T), read as X releases Y during interval T if it has a certain temporal ordering, ∃t1,t2∈Tandt1<t2such that(12)Release(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t1))).A three-dimensional geometric view of this relation is shown in Figure3(d). This relation can be expressed by a single direction, which is the destination direction. For example, object X  releases (motion event) object Y towards East(destination direction). Direction for such a relation is defined as(13)Dir(Release(XY,T))=holds(Dir(XY,t2)).Bypass(XY,T): A spatiotemporal Meet(XY,T) is called a Bypass(XY,T), read as X bypasses Y during interval T if it has a certain temporal ordering, that is, ∃t1,t2,t3,t4∈T such thatt1<t2<t3<t4(14)Bypass(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(M(XY,t3))∧holds(D(XY,t4))).This relation can be expressed by a single direction, where a meet topological relation holds. It means(15)Dir(Touch(XY,T))=holds(Dir(XY,t2)).Excurse(XY,T): A spatiotemporal Meet(XY,T) is called a Excurse(XY,T), read as X excurse Y during interval T if it has a certain temporal ordering, an intuitive view of this relation is shown in Figure 3(b). ∃t1,t2,t3∈T,s.t.,t1<t2<t3(16)Excurse(XY,T)=seq_eve(holds(M(XY,t1))∧holds(D(XY,t2))∧holds(M(XY,t3))).This relation is expressed by an initial and destination directions, the direction for this relation can be defined as(17)Dir(Excurse(XY,T))=seq_eve(holds(Dir(XY,t1))∧holds(Dir(XY,t3))).


## 7.2. Unstable Overlap Spatiotemporal Relation

Enter(XY,T): An unstable spatiotemporal overlap relation is calledEnter, generally denoted by Enter(XY,T) and read as “X enters in Y during interval T.” If ∃t1,t2,t3,t4∈Tsuchthatt1<t2<t3<t4, then relation is defined as(18)Enter(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(PO(XY,t3))∧holds(TPP(XY,t4))).An intuitive view of this relation is shown in Figure4(a). This relation can be expressed by a single direction because the destination point is inside and can be expressed without direction, a direction for the Enter spatiotemporal event is the direction where a meet topological relation holds, that is,(19)Dir(Enter(XY,T))=holds(Dir(XY,t2)).Leave(XY,T): A spatiotemporal partial overlap relation is called Leave, denoted as Leave(XY,T) “X leaves Y during interval T”. If ∃t1,t2,t3,t4∈Tsuchthatt1<t2<t3<t4, then relation is defined as(20)Leave(XY,T)=seq_eve(holds(NTPP(XY,t1))∧holds(TPP(XY,t2))∧holds(PO(XY,t3))∧holds(M(XY,t4))∧holds(D(XY,t5))).An intuitive view of this relation is shown in Figure4(b). This relation can be expressed by a single direction which is the destination point, that is(21)Dir(Leave(XY,T))=holds(Dir(XY,t4)).Cross(XY,T): A spatiotemporal relation Cross(XY,T) “X crosses Y during the interval T.” Its geometric view is given in Figure 4(c). If ∃t1,t2,t3,…,t9∈Tsuchthatt1<t2<⋯<t9, then relation is defined as(22)Cross(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(PO(XY,t3))∧holds(TPP(XY,t4))∧holds(NTPP(XY,t5))∧holds(TPP(XY,t6))∧holds(PO(XY,t7))∧holds(M(XY,t8))∧holds(D(XY,t9))).This spatiotemporal relation is expressed by a initial as well as destination direction such as objectX crosses (motion event) objectY from north (direction) towards east(direction) during the interval T:(23)Dir(Cross(XY,T))=seq_eve(holds(Dir(XY,t1))∧holds(Dir(XY,t9))).Into(XY,T): A spatiotemporal relationInto(XY,T) read as “X get into Y during the interval T.” If ∃t1,t2,t3∈Tsuchthatt1<t2<t3, then relation is defined as(24)Into(XY,T)=seq_eve(holds(M(XY,t1))∧holds(PO(XY,t2))∧holds(TPP(XY,t3))).Its three-dimensional geometric view is given in Figure4(e). This relation can be expressed by a single direction in language semantics, where a meet topological relation holds. For example, object A get into(spatiotemporal event) object B from north (direction). It means.(25)Dir(Into(XY,T))=holds(Dir(XY,t1)).Out_of(XY,T): A spatiotemporal relation Outof(XY,T) read as “X comes out of Y during the interval T,” its intuitive view is considered in Figure 4(f). If ∃t1,t2,t3,t4∈Tsuchthatt1<t2<t3<t4, then relation is defined as(26)Out_of(XY,T)=seq_eve(holds(TPP(XY,t1))∧holds(PO(XY,t2))∧holds(D(XY,t3))).This relation can be expressed by a single direction. ObjectX go out_of(motion event) object Y towards east(direction), where a meet topological relation holds. It means
(27)Dir(out_of(XY,T))=holds(Dir(XY,t3)).Melt(XY,T): A spatiotemporal relation Melt(XY,T) read as “X,Y melts during the interval T”. If ∃t1,t2,t3,t4∈Tsuchthatt1<t2<t3<t4, then relation is defined as(28)Melt(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(PO(XY,t3))∧holds(EQ(XY,t4))).An intuitive view of this relation is shown in Figure4(g). This relation can be expressed by a single direction because its destination point is dimensionless. This can be its direction, where initial spatial relation holds:(29)Dir(Melt(XY,T))=holds(Dir(XY,t1)).Spring(XY,T): A spatiotemporal relation Spring(XY,T) also called Separate(XY,T) read as “X separates Y during the interval T.” If ∃t1,t2,t3,t4∈Tsuchthatt1<t2<t3<t4, then relation is defined as(30)Spring(XY,T)=seq_eve(holds(EQ(XY,t1))∧holds(PO(XY,t2))∧holds(M(XY,t3))∧holds(D(XY,t4))).Its three-dimensional geometric view is given in Figure4(h). This relation can be expressed by a single direction because its destination point is dimensionless. This can be its direction, where terminal spatial relation holds:(31)Dir(Spring(XY,T))=holds(Dir(XY,t4)).Graze(XY,T): A spatiotemporal relation Graze(XY,T) read as “X grazes Y during the interval T.” If ∃t1,t2,t3,t4,t5∈Tsuchthatt1<t2<t3<t4<t5, then relation is defined as(32)Graze(XY,T)=seq_eve(holds(D(XY,t1))∧holds(M(XY,t2))∧holds(PO(XY,t3))∧holds(M(XY,t4))∧holds(D(XY,t5))).This relation is represented in a three-dimensional perspective in Figure4(d). This spatiotemporal relation is expressed by an initial as well as destination direction such as object X grazes (motion event) object Y from north(direction) toward east(direction):(33)Dir(Graze(XY,T))=seq_eve(holds(Dir(XY,t1))∧holds(Dir(XY,t4))).


## 8. Conclusion and Future Work

In this paper, we define spatiotemporal relations, where the discrete time space is used. These spatiotemporal relations are topologically stable or unstable. Motion events represent the subclass of spatiotemporal relations, and certain number of motion events represent the class of a topologically unstable and stable spatiotemporal relations. In these spatiotemporal relations temporal order of holding a primitive event is more important, and this order has a pivotal role in natural language semantics. Topological relations have a locative symmetries, to remove these symmetries we add a directional components. In this paper, CTD method [11] is used to model the motion events, where topological and directional information for a snapshot are captured at the same abstract level. Hopefully this work will bring a significant change in video understanding, modeling video events, and other related areas of research.



---

*Source: 872687-2012-05-16.xml*