Tracking method and tracking apparatus

According to an embodiment of the present invention, a tracking method includes detecting a mobile unit within a space, tracking the detected mobile unit, making a position determination of the mobile unit to be tracked to obtain positional data, and making a movement prediction of the mobile unit, based on a high frequency component of positional data.

CROSS REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2007-29424, filed on Feb. 8, 2007, the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a tracking method and a tracking apparatus.

2. Background Art

In recent years, the introduction of an autonomous mobile unit such as a robot into the general environment such as a shop or facility has been examined. When a robot is introduced into the general environment, it is a problem whether the safety of the robot is assured. Particularly, under the environment where the robot and a person are coexistent, it is required that the robot recognizes the person and takes a safe operation for the person. On the other hand, it is important to assure the basic performance of the robot such as a smooth movement operation. Accordingly, when the robot is introduced into the general environment, a tradeoff between the assured safety and the assured basic performance is a problem.

Conventionally, when the robot recognizes a person, the robot stops at the site and waits for the person to leave away, or avoids the person while keeping an enough interval from the person. However, at the small shop or small scale facility, it is difficult to secure the interval between the robot and the person sufficiently. Accordingly, the robot has to stop at the site and let the person go away in such a case. However, the robot taking such a motion strategy has a bad work efficiency and is unusable for practical applications.

One of the reasons why such motion strategy must be taken is that the robot has less movement information of other mobile units. Accordingly, to realize the introduction of the robot into the general environment, it is important to improve techniques, such as detection technique for a mobile unit, tracking technique for a mobile unit, feature analysis technique for a mobile unit, and interaction technique between robots.

As examples of a system for detecting a mobile unit within a space, there are proposed a system using a surveillance camera and a system using a transmitter such as an ID tag. Furthermore, processes using these systems have been researched. Examples of such processes include person detection, position estimation, traffic line detection for a moving person, and action estimation for a moving person. Also, as an example of a method for detecting a mobile unit, there is proposed a method using optical flow. This method is effective for detecting the motion of a rigid single body such as one automobile or one ball, but this method easily causes a false detection for an object of multi-link structure such as a person, because it is difficult to treat its internal motion and interaction with other persons. JP-A 2006-127157 (KOKAI) describes a method for extracting the features of a mobile unit by time averaging of flow vectors. However, with this method, there is possibility that the features to be extracted may disappear by time average.

It is also researched that a point extracted from a camera image is translated from a camera screen onto a movement plane to estimate the movement amount of an object on the movement plane. JP-A 2006-33188 (KOKAI) describes a method for tracking an object by estimating the movement amount of the object on the movement plane using a speed vector. However, with this method, there is strong possibility that a meaningless motion contained in the significant motion of the person may be also detected. Also, if the speed vector is simply averaged over time to prevent this as previously described, there is strong possibility that the significant motion may also disappear.

Main purposes of the methods described in JP-A 2006-127157 (KOKAI) and JP-A 2006-33188 (KOKAI) are to investigate guest trend and to take crime prevention. With the method described in JP-A 2006-127157 (KOKAI) or JP-A 2006-33188 (KOKAI), information such as the position coordinate, moving speed, and residence time of a detected mobile unit is acquired. However, these kinds of information are not enough to support the motion of the autonomous mobile unit that autonomously moves within the shop or facility. Also, if only a client refers to these kinds of information exclusively, there is possibility that the movement of the autonomous mobile unit becomes movement based on the absolute viewpoint of the client.

SUMMARY OF THE INVENTION

An embodiment of the invention is, for example, a tracking method including, detecting a mobile unit within a space, tracking the detected mobile unit, making a position determination of the mobile unit to be tracked to obtain positional data, and making a movement prediction of the mobile unit, based on a high frequency component of positional data.

Another embodiment of the invention is, for example, a tracking apparatus including, a detection unit configured to detect a mobile unit within a space, a tracking unit configured to track the detected mobile unit, a determination unit configured to make a position determination of the mobile unit to be tracked, and a prediction unit configured to make a movement prediction of the mobile unit, based on a high frequency component of positional data obtained by the position determination.

DESCRIPTION OF THE EMBODIMENTS

FIG. 1is a system configuration diagram of a tracking system101. The tracking system101is constructed in a space where various mobile units111such as a person111A and a robot (autonomous mobile unit)111B come and go. Here, the space is supposed to be a shop floor having a flat floor face where a person, shopping cart, and robot come and go.

The tracking system101ofFIG. 1includes a tracking apparatus121, which detects a mobile unit111within the space and tracks the mobile unit111. The tracking apparatus121includes an input unit131, a memory unit132, an output unit133, a mobile unit detection unit141, a position determination using a signal, and movement prediction using a position determination result.

(1) Position Determination Using Image

The tracking apparatus121is supplied with camera image information. Here, the image information which is divided in pixel units is inputted into the tracking apparatus121at a certain periodical interval. As the image information, the tracking apparatus121is supplied with an image within an angle of view of a camera. The image may be a still image or a moving image. Here, the image obtained by imaging a mobile unit111within the space is inputted into the tracking apparatus121. The tracking apparatus121detects the mobile unit111from the image, and tracks the mobile unit111while the mobile unit111exists within a detectable area (angle of visibility of the camera). The mobile unit detection using the image has an advantage that various kinds of information such as shape, pattern, and color of the mobile unit111can be acquired, compared with mobile unit detections by other methods.

In this embodiment, one or more wide-angle cameras151are mounted on the ceiling within the shop, as shown inFIG. 2. Here, there are mounted a plurality of cameras151A having the optical axes directed to the vertically downward direction, and one or more cameras151B having the optical axes inclined from the vertically downward direction. The cameras151A are arranged so that the overlap of the fields of view is as little as possible. Each of the cameras151B is arranged so that the inclination θ from the vertically downward direction is 0°<θ<90°.

The tracking apparatus121includes a user interface for allowing a user to input the mounted position, height, and direction of each camera151within the shop, as known information, into the tracking apparatus121. Examples of the user interface include a keyboard, a mouse, and a monitor. The known information is saved in a storage of the tracking apparatus121. Examples of the storage include a hard disk and a flash memory. Examples of the known information settable by the user include the mounted coordinate and posture of each camera151, the correction value for distortion and the breadth for angle of view in the lenses of each camera151, and shop structure information such as position, height, and size of each wall and pillar within the shop.

A specific example of the user who inputs the known information into the tracking apparatus121is a system administrator of the tracking system101. The tracking apparatus121can recognize the structure of the shop where the tracking apparatus121is installed, according to the shop structure information inputted and saved. Further, the tracking apparatus121can grasp by which camera151each image is taken and in which part each image is taken within the shop, according to mounting data and lens data of each camera151.

After mounting a camera151, the tracking apparatus121can acquire a basic background image which is a reference image for difference process. Here, the basic background image is taken in a state where there is no coming and going of mobile units111such as person111A and robot111B, after the lighting situation of illumination and the exhibition situation of commodities within the shop is placed in the state during business hours. The taken basic background image is recorded as the reference image for the difference process.

During normal business hours of the shop, an image shot by the camera151is inputted at a certain periodical interval into the tracking apparatus121. The tracking apparatus121loads the basic background image corresponding to the inputted image, and performs a difference process (correlation comparison) between the image and the basic background image.FIGS. 3A and 3Bshow specific examples of the image and the basic background image. Through the difference process, mobile units111such as person111A and robot111B are detected from the image, as shown inFIG. 3C. The tracking apparatus121detects an area where the correlation value between the image and the basic background image is less than or equal to a threshold (unmatched area), as an image unit142, and a motion feature amount extraction unit143. The mobile unit detection unit141is a block for detecting a mobile unit111within the space and tracking the mobile unit111. The mobile unit detection unit141is an example of a detection unit and tracking unit. The position determination unit142is a block for making a position determination of the mobile unit111. The position determination unit142is an example of a determination unit and prediction unit. The motion feature amount extraction unit143is a block for extracting a motion feature amount of the mobile unit111. The motion feature amount extraction unit143is an example of an extraction unit, filtering unit, and distinction unit.

Information for detecting a mobile unit111within the space is inputted into the input unit131. The mobile unit detection unit141detects and tracks the mobile unit111using this information. The position determination unit142determines the position of the mobile unit111. The motion feature amount extraction unit143extracts a motion feature amount of the mobile unit111. A determination result of the position and an extraction result of the motion feature amount are stored in the memory unit132, and used at the next mobile unit detection. The output unit133outputs, for example, a prediction result of a movement prediction for the mobile unit111.

The tracking system101ofFIG. 1is provided with one or more cameras151for imaging a mobile unit111within the space, and one or more antennas152for receiving a signal transmitted from a mobile unit111within the space. The tracking apparatus121can acquire an image taken by each camera151, and a signal received at each antenna152, via the input unit131. Each of the cameras151and antennas152may be equipped in the tracking apparatus121or installed within the shop.

In the following description, a tracking method performed by the tracking apparatus121will be described, in the order of position determination using an image, position determination area where a mobile unit111exists (mobile unit detection area), as shown inFIG. 3D.

The tracking apparatus121provides, regarding a mobile unit111detected through the difference process, a feature point on the geometrical pattern of the texture, as shown inFIG. 4A. Further, the tracking apparatus121extracts a motion feature amount which is a feature amount regarding the motion of the mobile unit111. Examples of the motion feature amount include speed and acceleration. Here, the tracking apparatus121observes and extracts, as the motion feature amount of the mobile unit111, the motion of the feature point on a time continuous image, i.e., an optical flow of the mobile unit111, as shown inFIG. 4B.

Moreover, the tracking apparatus121distinguishes each mobile unit111, based on the angle of the flow vector of the optical flow, as shown inFIG. 4C. Regarding one feature point and another feature point that are proximate, if the angle between the flow vector of the former feature point and the flow vector of the latter feature point is within a certain threshold, the former feature point and the latter feature point are judged to belong to the same mobile unit111. Conversely, if the angle is not within the threshold, the former feature point and the latter feature point are judged to belong to different mobile units111. In this manner, different mobile units111in the same image are distinguished. If the feature points belonging to the same mobile unit detection area are judged to belong to different mobile units111, the mobile unit detection area is divided for each mobile unit111, and each mobile unit detection area produced by division is managed by its own identification name. Mobile unit detection areas produced by the division are managed by respective identification names.

Particularly, in this embodiment, the tracking apparatus121filters the flow vector of the optical flow through a frequency filter161(FIG. 1). Each ofFIGS. 4D and 4Eshows time variations of X component and Y component of a flow vector before filtering. Here, the X direction and the Y direction correspond to the transverse direction and the longitudinal direction of image. The flow vector ofFIG. 4Dcorresponds to a flow vector of a feature point in person's arm, and the flow vector ofFIG. 4Ecorresponds to a flow vector of a feature point in person's trunk.

The frequency filter161is here a low pass filter having a cut-off frequency ωc. The cut-off frequency ωcis greater than the frequency component of the center of gravity, ω1, conceivably contained in the normal movement of a person within the shop, and smaller than the vibration frequency component, ω2, conceivably contained in the pitch motion of arm swing and leg movement when a person walks and moves. That is, ω1<ωc<ω2. In this manner, the frequency filter161is a low pass filter in consideration of operation frequencies of person.

An example of the value of ω1is the frequency from 0.1 Hz to 1.0 Hz. An example of the value of ω2is the frequency from 2.0 Hz to 5.0 Hz. Accordingly, an example of the value of ωcis the frequency from 0.1 Hz to 5.0 Hz, preferably from 1.0 Hz to 2.0 Hz. The filtering action of the low pass filter161is represented by the expression (1). Findenotes the flow vector before filtering, Foutdenotes the flow vector after filtering, and L(ωc) denotes the low pass filter161.
Fout=L(ωc)Fin(1)

Each ofFIGS. 4F and 4Gshows time variations of X component and Y component of the flow vector after filtering. It can be found that the high frequency component of the flow vector is removed by the frequency filter161. In this embodiment, the tracking apparatus121distinguishes each mobile unit111, based on the optical flow filtered through the frequency filter161. That is, in this embodiment, parts of the same mobile unit111are recognized as the same mobile unit111, and parts of different mobile units111are recognized as different mobile units111, based on optical flows filtered through the frequency filter161. Thereby, in this embodiment, it is suppressed that the trunk (low frequency component) and the hand and foot (high frequency component) of one person are falsely recognized as different mobile units111, and a plurality of mobile units111can be distinguished. The former effect is mainly caused by removal of higher frequency component than the cut-off frequency, and the latter effect is caused by remnant of lower frequency component than the cut-off frequency. As described above, a method for distinguishing each mobile unit111is performed based on the angle of the flow vector of the optical flow.

Here, the distinction between each mobile unit111is not necessarily perfect. In practice, the mobile unit111recognized as one mobile unit may correspond to one person, or a group of plural persons, as will be described later. To discriminate between the former case and the latter case, a matching process is performed.

Also, in this embodiment, the image taken by the camera151is filtered through a frequency filter. The tracking apparatus121filters the image through a low pass filter162(FIG. 1) having a cut-off frequency ωclower than the frequency ω1. Then, the tracking apparatus121updates the basic background image by combining the image filtered through the low pass filter162and the basic background image. Thereby, a background removal process (FIG. 3) for a strong background variation such as a varying commodity exhibition situation is implemented.

In this embodiment, the basic background image and the image are combined at a predetermined weighting. Specifically, each pixel value component of the basic background image and each pixel value component of the image are added at a ratio of 1-W to W, as in the expression (2). Gnowdenotes the current image, Golddenotes the basic background image before update in current use, Gnewdenotes the basic background image after update for use at the next period, and L(ωc) denotes the low pass filter162. For W (weight factor for time weighting), 0<W<1 holds, and for ωc(cut-off frequency), ωc<ω1holds. In this manner, the tracking apparatus121repeatedly updates the basic background image.
Gnew=(1−W)·Gold+W·L(ωc)Gnow(2)

Also, the tracking apparatus121performs image processing such as binarization and edge extraction (contour extraction), for an image area where a mobile unit is detected (mobile unit detection area,FIG. 3D).FIG. 5Ashows an image area before image processing, andFIG. 5Bshows an image area after image processing.

Next, the tracking apparatus121performs filtering through a frequency filter for the image area, prior to a collation process (matching process) of the image area with a sample image for the collation process. The tracking apparatus121filters the image area through a low pass filter163(FIG. 1) whose cut-off frequency is the maximum frequency component for the contour line of a person in the sample image. Thereby, noise removal is performed from the image area where the edge extraction is made, as shown inFIG. 5C. Further, the tracking apparatus121normalizes the size of the image area, as shown inFIG. 5D. Thereby, the size of the image area is scaled up or down to the same size as the sample image.

Then, the tracking apparatus121performs the matching process of the image area with the sample image. As the sample image, a template having the contour line representing the shape feature of a person (person shape feature template) is used. Accordingly, through the matching process of the image area with the sample image, the tracking apparatus121can judge whether or not the mobile unit111in the image area is one person.

For the tracking apparatus121, a number of sample images are prepared as a person shape feature database (FIG. 5E). The tracking apparatus121performs the matching process (correlation comparison) of each image area with at least one kind of sample image. If the correlation value with any sample image is greater than or equal to a threshold, the mobile unit111in the image area is recognized as one person.

FIG. 6shows a processing flow of this matching process. The tracking apparatus121performs a matching process of an image area with a sample image (template) (S101). If the correlation value between the image area and the sample image is greater than or equal to a threshold (S102), the mobile unit111in the image area is judged as one person (S103).

The matching process at S101is repeated until the sample images (templates) to be used are exhausted (S111, S112). If the correlation value with any sample image is greater than or equal to the threshold (S102), the mobile unit111in the image area is judged as one person (S103). On the other hand, if the correlation values with all the sample images are less than the threshold, the mobile unit111in the image area is not judged as one person, and the procedure goes to S121. The process from S121will be described later.

If the mobile unit111in the image area is judged as one person through the matching process, the tracking apparatus121gives an inherent ID (identification) to the mobile unit111. The tracking apparatus121further acquires a character feature amount which is a feature amount characterizing the character (mobile unit111). Examples of the character feature amount include color features, such as absolute color information for each part of the character, and relative color relation between each part of the character and its vicinity. Also, examples of the character feature amount include geometrical features, such as the image size of the image having the character, and the coordinates of the image in the entire image. Also, examples of the character feature amount include texture features, such as coordinates of each feature point in the image having the character. The character feature amounts such as the color features, geometrical features, and texture features exemplified herein can be acquired from the image or image area, through image processing or analysis process by the tracking apparatus121.

Then, the tracking apparatus121manages the character feature amount acquired for the mobile unit111, associated with the mobile unit111. Specifically, the character feature amount of the mobile unit111is stored and managed associated with the ID of the mobile unit111. The character feature amount is used as identification information for tracking the mobile unit111. That is, the tracking apparatus121judges that a newly detected mobile unit and a stored mobile unit are identical, if the correlation value between the character feature amount of the detected mobile unit and the stored character feature amount is greater than a certain threshold. The detected mobile unit is given the same ID as the stored mobile unit. In this embodiment, the same mobile unit can be tracked due to this mobile unit identification. Thereafter, the character feature amount is updated by a weight function and a time dependent forgetfulness factor.

On the other hand, if the mobile unit111in the image area is not judged as one person in the matching process, the tracking apparatus121calculates the image size of the image area, regarding the image area before normalization ofFIG. 5D(S121ofFIG. 6). Then, the tracking apparatus121judges whether or not the image size is greater than or equal to a threshold (S122). Thereby, the tracking apparatus121can judge whether or not the mobile unit111in the image area is a group of plural persons.

If the image size is greater than or equal to the threshold, the tracking apparatus121judges the mobile unit111as a group of plural persons (S123), or if the image size is less than the threshold, the tracking apparatus121judges the mobile unit111as one unknown mobile unit (S124). The threshold is set to the image size corresponding to the size of one standard person. It is noted that the image size changes depending on mounted situation of the camera, detection position of the image area within the camera screen, and the like.

For example, in a tilted camera such as a camera151B ofFIG. 2, the mobile unit at the lower side of the camera screen is located closer to the camera and reflected larger on the screen, and the mobile unit at the upper side of the camera screen is farther away to the camera and reflected smaller on the screen. Accordingly, in this case, the threshold is set smaller according to the height of the grasped mobile unit. The changing ratio of the threshold can be set based on the perspective property of the image.

Similarly, in a vertically downward faced camera such as a camera151A ofFIG. 2, the mobile unit near the center of the camera screen is located closer to the camera and reflected larger on the screen, and the mobile unit near the periphery of the camera screen is located farther away to the camera and reflected smaller on the screen. Accordingly, in this case, the threshold is set smaller according to the distance of the grasped mobile unit from the center of the screen. The changing ratio of the threshold can be set based on the perspective property of the image, in the same manner as above.

If the mobile unit111is judged as a group of plural persons, the tracking apparatus121calculates the ratio of the image size calculated at S121and the threshold used at S122, as a congestion degree indicating the degree that the group is congested. The congestion degree is stored and managed as a state quantity at the specific time (detection time of the image area) and the specific place (detection position of the image area). The congestion degree is usable as a character feature amount (group feature amount in this case) of the mobile unit111, for example.

Also, the tracking apparatus121determines the position of the mobile unit111, using the image area where the mobile unit111is detected (mobile unit detection area,FIG. 3D). When the tracking apparatus121determines the position of the mobile unit111which is judged as one person, it decides one representative point within the mobile unit detection area of the mobile unit111, and determines the representative point as the current position of the mobile unit111.

FIG. 7Ashows an example of an image taken by the camera151B (obliquely looking down camera) ofFIG. 2. The lower end of the image is shown by “B”. In an image area within the image, a point nearest to the lower end B within the image area is decided as a representative point of the image area. The representative point P of each image area R is shown inFIG. 7A.

FIG. 8Ashows an example of an image taken by the camera151A (vertically looking down camera) ofFIG. 2. The center of the image is shown by “A”. In an image area within the image, a point nearest to the center A within the image area is decided as a representative point of the image area. The representative point P of each image area R is shown inFIG. 8A.

With the methods ofFIGS. 7A and 8A, the point corresponding to the foot of the mobile unit111is decided as the representative point of the image area. This is because, in the image ofFIG. 7A, the foot of each mobile unit111is closer to the lower end B of the image, and in the image ofFIG. 8A, the foot of each mobile unit111is closer to the center A of the image. In this embodiment, since a danger of approach or collision between mobile units111is a problem, it may be appropriate to represent the position of each mobile unit111with the position of the foot of each mobile unit111.

If there are a plurality of points nearest to the lower end B or the center A of the image within one image area, the point nearest to the center of gravity of the image area among those points is decided as the representative point. Instead of making the point nearest to the center of gravity of the image area the representative point, the point nearest to the center line of the image area may be decided as the representative point.

Also, regarding the method ofFIG. 8A, it is desirable to correct an image distortion of the wide-angle lens, before the position determination. Also, regarding the method ofFIG. 8A, for an image area overlapping with the center A of the image, or for an image area near the center A of the image, the center of gravity of the image area may be decided as the representative point of the image area.

The tracking apparatus121creates a two-dimensional virtual movement plane map, based on a determination result of the position of each mobile unit111, and in consideration of the mounted position, posture, and lens characteristics of the camera, as shown inFIG. 7Bor8B. That is, the coordinate transformation (Fb) from the screen ofFIG. 7Ato the map ofFIG. 7B, or the coordinate transformation (Fa) from the screen ofFIG. 8Ato the map ofFIG. 8B, is performed by the tracking apparatus121.

The tracking apparatus121stores and manages the position and congestion degree of each mobile unit111, as state quantities at the specific time (detection time of the image area) and the specific place (detection position of the image area), in time series.

In this embodiment, each robot and shopping cart may be provided with an illuminant or a marker. In this case, a flash signal of the illuminant, or the color, pattern, or geometrical design of the marker is grasped by the camera. Thereby, the mobile unit-detection process is facilitated.

Though the position determination using an image has been described in this chapter, the position determination using a signal will be described in the next chapter.

(2) Position Determination Using Signal

The tracking system101ofFIG. 1is provided with a receiver for receiving a radio wave signal transmitted from a transmitter of each mobile unit111. The tracking apparatus121can analyze a signal code of the radio wave signal, and can measure the strength of the radio wave signal. The tracking system101is provided with one or more antennas152for receiving the radio wave signal, as shown inFIG. 1. The tracking apparatus121allows a user to input the mounted position of each antenna152as known information, as in the case of the cameras151. The tracking apparatus121detects a mobile unit111within the space, using the radio wave signal transmitted from the mobile unit111. The tracking apparatus121tracks the mobile unit111, while the mobile unit111exists within a detectable area (area capable of receiving the radio wave signal). The mobile unit detection using the signal has an advantage that identifiable information of the mobile unit111can be contained in the signal, compared with mobile unit detections by other methods. An example of this information is an ID code as will be described later.

In this embodiment, each robot and shopping cart among the mobile units111within the shop is provided with a transmitter. Also, in this embodiment, each mobile unit111is allocated an ID code (mobile unit identification code) which is different for each mobile unit111. The ID code of a mobile unit111is appended to the radio wave signal transmitted from the mobile unit111. The ID code is used as identification information for tracking the mobile unit111. The tracking apparatus121tracks the mobile unit111by associating the mobile unit111with the radio wave signal, based on the ID code.

The tracking apparatus121creates a two-dimensional virtual movement plane map, based on the strength of the received radio wave signal, as shown inFIG. 7Bor8B. Specifically, the tracking apparatus121firstly draws a circle having a radius of the strength of the received radio wave signal, and centered at the position of an antenna152receiving the radio wave signal. Then, the tracking apparatus121draws a straight line from the center to the direction of receiving the radio wave signal, and determines an intersection point between the circle and the straight line as the position of the mobile unit111. The tracking apparatus121creates the two-dimensional virtual movement plane map, using such position determination result.

The relationship between the signal strength and the distance can be experimentally decided beforehand. Also, the relationship between the signal strength and the distance for each antenna152may be experimentally measured in detail beforehand, which allows the tracking apparatus121to correct a distortion from the ideal concentric circular strength distribution.

Though the position determination using an image has been described in the previous chapter and the position determination using a signal has been described in this chapter, the tracking apparatus121of this embodiment may use these position determination methods together.

(3) Movement Prediction Using Position Determination Result

The tracking apparatus121ofFIG. 1can acquire positional data of each mobile unit111to be tracked, by the position determination for each mobile unit111. With the position determination as described in the first chapter (position determination using an image), the coordinate data of the representative point of each mobile unit111is obtained as the positional data of each mobile unit111. With the position determination as described in the second chapter (position determination using a signal), the coordinate data of the position where the signal is transmitted from each mobile unit111is obtained as the positional data of each mobile unit111. In this embodiment, the tracking apparatus121ofFIG. 1makes a movement prediction of each mobile unit111, based on the positional data obtained by any of these position determinations. Here, the movement prediction is made based on a high frequency component and low frequency component of the positional data.

In making the movement prediction for a mobile unit111, the tracking apparatus121firstly acquires the time series data of the positional data of the mobile unit111. Specifically, the time series data of the X coordinate and the Y coordinate of the mobile unit111is acquired. The axial directions of the X coordinate and the Y coordinate are identical with the axial directions of the X coordinate and the Y coordinate on the two-dimensional virtual movement plane map, respectively.

Next, the tracking apparatus121performs a filtering of the time series data through a high pass filter164(FIG. 1), and a filtering of the time series data through a low pass filter165(FIG. 1). Each of the high pass filter164and the low pass filter165is a frequency filter whose cut-off frequency ωcis the maximum frequency component contained in the typical walking motion of a person (excluding noise component). That is, the cut-off frequency of the high pass filter164and the cut-off frequency of the low pass filter165are equal. Here, each of the high pass filter164and the low pass filter165is of a sequential or recursive computation type (first order lag type).

In this embodiment, the tracking apparatus121makes the movement prediction for each mobile unit111, based on high pass time series data (high pass positional data) obtained by filtering the time series data (positional data) through the high pass filter164, and low pass time series data (low pass positional data) obtained by filtering the time series data (positional data) through the low pass filter165. The movement prediction may be performed based on the high pass time series data and the low pass time series data themselves, or based on data derived from the high pass time series data and the low pass time series data, as will be described later. The movement prediction will be described below in more detail.

Each of the high pass positional data and the low pass positional data is a two-dimensional vector including X component and Y component. The tracking apparatus121calculates a local moving speed vector Vmicrorepresented by the expression (3), and a global moving speed vector Vmacrorepresented by the expression (4), as movement feature amounts which are feature amounts regarding the movement of the mobile unit111. Examples of a movement feature amount include speed and acceleration. In the expressions (3) and (4), P denotes the positional data of the mobile unit111, H(ωc) denotes the high pass filter164, and L(ωc) denotes the low pass filter165.

The local moving speed vector Vmicrois a differential amount of the high pass positional data H(ωc)P. The local moving speed vector Vmicrois a movement feature amount indicating the microscopic movement feature of the mobile unit111. On the other hand, the global moving speed vector Vmacrois a differential amount of the low pass positional data L(ωc)P. The global moving speed vector Vmacrois a movement feature amount indicating the macroscopic movement feature of the mobile unit111. Since these speed vectors are time differential amounts of the positional data, the time series data of the positional data is required to calculate these speed vectors. The tracking apparatus121stores and manages the calculated global moving speed vector Vmacroand local moving speed vector Vmicroin time series, associated with the mobile unit111, like the positional data.

FIG. 9shows an angle θvmade by the local moving speed vector Vmicrorelative to the global moving speed vector Vmacro. The angle θvcan be regarded as the moving direction caused by a microscopic motion that the mobile unit111performing a macroscopic motion conducts at a certain moment. The angle θvis referred to as a local movement azimuth angle. The tracking apparatus121calculates the angle θv, and stores and manages the angle θvas a movement feature amount of the mobile unit111in time series, associated with the mobile unit111. A calculation formula for the local movement azimuth angle θvis the following expression (5).

The tracking apparatus121further calculates a value T that is the absolute value of the local moving speed vector Vmicromultiplied by a certain weight factor W. This T is referred to as an azimuth momentary movement amount in the direction of the relative azimuth angle θ from the global moving speed direction. A calculation formula for the azimuth momentary movement amount T is the following expression (6). The tracking apparatus121stores and manages the azimuth momentary movement amount T as a movement feature amount of the mobile unit111in time series, associated with the mobile unit111.
T=W·|Vmicro|  (6)

The tracking apparatus121further calculates a value P(θ) of integrating the product of the azimuth momentary movement amount T and the time dependent forgetfulness factor D(t) over time. This value P(θ) is referred to as a local movement prediction range in the direction of the relative azimuth angle θ from the global moving speed direction. A calculation formula for the local movement prediction range P(θ) is the following expression (7). The local movement prediction range P(θ) is shown inFIG. 10. The tracking apparatus121stores and manages the local movement prediction range P(θ) as a movement feature amount of the mobile unit111, associated with the mobile unit111.
P(θ)=∫D(t)·Tdt(7)

The tracking apparatus121further calculates a value U of integrating the local movement prediction range P(θ) all around the relative azimuth angle θ. This value U is referred to as a complex mobility. A calculation formula for the complex mobility U is the following expression (8). The tracking apparatus121stores and manages the complex mobility U as a movement feature amount of the mobile unit111, associated with the mobile unit111.
U=∫P(θ)dθ=∫∫D(t)·Tdtdθ(8)

The complex mobility U indicates the possibility that the mobile unit111suddenly moves in another direction from a large flow of movement. If the complex mobility U is greater, the possibility that the mobile unit111suddenly moves in another direction is high, so that another mobile unit111that is located near the mobile unit111has a greater danger. Conversely, if the complex mobility U is smaller, the possibility that the mobile unit111suddenly moves in another direction is low, so that another mobile unit111that is located near the mobile unit111has a smaller danger. In this manner, the complex mobility U is a parameter indicating the height of possibility that the mobile unit111suddenly moves in another direction.

Accordingly, the tracking apparatus121obtains a prediction result of the movement prediction such as how much possibility there is that the mobile unit111suddenly moves in another direction, as a result of the complex mobility U being derived. That is, the tracking apparatus121makes the movement prediction in the form of predicting how much possibility there is that the mobile unit111suddenly moves in another direction, by deriving the complex mobility U. It is possible to know the height of possibility that the mobile unit111suddenly moves in another direction, from the complex mobility U (prediction data) obtained by the movement prediction. The complex mobility U is an example of the prediction data.

The tracking apparatus121uses the current positional data and a few pieces of the previous positional data (i.e., positional data in the past), as positional data for calculating the complex mobility U. Thereby, the tracking apparatus121predicts the next movement (i.e., movement in the future) of the mobile unit111. The positional data in the past may be the positional data stored in time series in the memory unit132(FIG. 1). The number how many pieces of the positional data in the past are to be used may be arbitrary. In this embodiment, a few pieces of the positional data in the past are to be used. Accordingly, a high pass filter and a low pass filter of sequential or recursive computation type are employed as the high pass filter164and the low pass filter165, in this embodiment.

The tracking apparatus121notifies the complex mobility U of each mobile unit111within the shop, to the robot111B within the shop. Then, when the robot111B passes near another mobile unit111, the robot111B refers to the complex mobility U of the mobile unit111. Then, if the complex mobility U is great, the robot111B takes a greater interval from the mobile unit111in passing near the mobile unit111, to avoid a danger of collision with the mobile unit111. Conversely, if the complex mobility U is small, the robot111B does not need to take a greater interval from the mobile unit111in passing near the mobile unit111, whereby the movement efficiency of the robot111B is not impeded.

In this manner, since the prediction data obtained by the movement prediction is notified to the robot111B in this embodiment, it is possible to avoid a danger of approach or collision between the robot111B and another mobile unit111, while keeping the movement efficiency of the robot111B. An example of the notification method from the tracking apparatus121to the robot111B is wireless communication via the output unit133(FIG. 1).

The tracking apparatus121may make the movement prediction in another way from the above movement prediction. Here, as an example of a movement feature amount instead of the complex mobility U, a movement prediction area S(θ) is adopted.

As shown inFIG. 11, the movement prediction area S(θ) is provided to be largest in the global moving speed direction, and open to both sides across the direction. The movement prediction area S(θ) is provided such that as the angle θ from the global moving speed direction is larger, the absolute value of its directional component is smaller.

The tracking apparatus121uses the movement prediction area S(θ) as information for searching mobile units.FIG. 12shows a flowchart for a mobile unit searching process. In detecting mobile units, if the tracking apparatus121detects a mobile unit111within the movement prediction area S(θ) of a previously recognized mobile unit111(S201), the tracking apparatus121preferentially makes a correlation comparison between the feature amount of the recognized mobile unit111and the feature amount of the detected mobile unit111(S202). This is because there is strong possibility that they are the same mobile unit111. If the correlation value between these feature amounts is greater than or equal to a threshold (S203), it is judged that they are the same mobile unit111(S204). If the correlation value between these feature amounts is less than the threshold, the correlation comparison is made again with decreasing the threshold (S212) unless the threshold is less than a permissible threshold (S211).

In the above-mentioned mobile unit searching process, the tracking apparatus121may use, instead of the movement prediction area S(θ), a corrected movement prediction area S′(θ), which is the movement prediction area S(θ) corrected by the local movement prediction range P(θ). Thereby, the mobile unit searching process is implemented by further considering the movement feature of each mobile unit111. A calculation formula for the corrected movement prediction area S′(θ) is the following expression (9).FIG. 13shows the corrected movement prediction area S′(θ).
S′(θ)=S(θ)+P(θ)  (9)

The tracking apparatus121can graphically display a behavior of movement of each mobile unit111, on a monitor171ofFIG. 1, using the time series data of positional data obtained by the position determination for each mobile unit111. According to a displaying scheme, the two-dimensional virtual movement plane is displayed as shown inFIG. 7Bor8B, and the symbol (e.g., “point”) denoting each mobile unit111is displayed at the position derived from the positional data of each mobile unit111. A supervisor of this system can graphically grasp the trend of guests, robots, and shopping carts within the shop by monitoring on the monitor171.

At this time, the tracking apparatus121can display the calculated local moving speed vector Vmicro, global moving speed vector Vmacro, complex mobility U, local movement prediction range P(θ), and movement prediction area S(θ), along with the behavior of movement of each mobile unit111, on the monitor171ofFIG. 1. The displaying scheme of them may be simple numerical indication, or graphical representation as described above. According to an example of the graphical representation, they are displayed in the form as shown inFIGS. 10 and 11. The supervisor of this system can predict how each mobile unit111moves within the shop by monitoring on the monitor171. Thereby, it is possible to prevent collision of a guest with a robot owing to the action of the supervisor, for example.

The tracking apparatus121here outputs the behavior of movement of each mobile unit111and the movement feature amounts (prediction data) to a local user interface (the monitor171of the tracking apparatus121), but it may output them to a remote user interface.

The tracking apparatus121can output the time series data of positional data obtained by the position determination for each mobile unit111, and the calculated local moving speed vector Vmicro, global moving speed vector Vmacro, complex mobility U, local movement prediction range P(θ), and movement prediction area S(θ), to the outside via a communication interface. The notification to the robot and the output to the remote user interface correspond to examples thereof. Examples of the communication interface include a wire LAN interface, a wireless LAN interface, and a USB interface. As an example of the communication path on which the positional data and the movement feature amounts (prediction data) are outputted, a network172is shown inFIG. 1.

Each robot111B moving autonomously within the shop is provided with a plurality of infrared radiation emitting devices for repeatedly emitting light in specific flashing pattern, and a plurality of markers drawn in specific color and pattern. Each robot111B is provided with an enough number of infrared radiation emitting devices and markers to ascertain the robot111B in any direction within the shop. The tracking apparatus121possesses the light emitting pattern and the marker feature of each robot111B as known information. When a certain light emitting pattern or marker feature is detected, the tracking apparatus121specifies the corresponding robot111B, and judges that intervention into the moving operation of the robot111B is possible. For an intervention command and intervention method for intervening in the motion, it is supposed to use a command and method which are made consistent between the tracking apparatus121and the robot111B beforehand.

Referring toFIG. 14, an intervention process performed by the tracking apparatus121will be described below.FIG. 14is a flowchart for the intervention process.

The tracking apparatus121selects mobile units111located adjacently, using the stored positional data, and manages them as a mobile unit pair (S301). Then, if the distance between these mobile units111is smaller than or equal to a threshold (S302), and if the global moving speed vectors of these mobile units111cross each other (S303), the tracking apparatus121judges that these mobile units111are on a collision course (S304). At this time, if at least one of these mobile units111is capable of operation intervention (S311), the tracking apparatus121outputs an emergency stop command to at least one of these mobile units111(S312). That is, the operation intervention is performed. The mobile unit111receiving the emergency stop command is decelerated promptly and stops. Thereby, the collision between the mobile units111is avoided.

As described above, the tracking apparatus121intervenes in the movement operation of each robot111B, according to a movement feature amount (prediction data) obtained for each robot111B. In this embodiment, the intervention is performed through a socket communication via the network. Also, an emergency stop intervention may be performed by means such as an infrared remote controller with cord. Also, the intervention process of this embodiment is applicable to a mobile unit111such as a shopping cart heteronomously moving within the shop. For example, the shopping cart may be provided with a brake system with a wheel lock mechanism operating in response to an intervention signal. Also, to intervene in a person such as a clerk or worker, the person may carry an intercom or earphone generating a warning sound in response to an intervention signal. Further, a rotating lamp that flashes in response to an intervention signal may be provided at an essential point such as an intersection within the shop.

Both of the notification to the robot (autonomous mobile unit) and the operation intervention can suppress a danger of approach or collision between the robot and another mobile unit while keeping the movement efficiency of the robot. A difference between the notification and the operation intervention is that the subject for judging the danger is typically the robot in the former case, whereas the subject for judging the danger is typically the tracking apparatus in the latter case. The processing load of the tracking apparatus is reduced in the former case, whereas the processing load of the robot is reduced in the latter case.