# Patent

## Title
Information processing device, information processing method, and program

## Abstract

An information processing device includes: a foreground state estimating unit configured to estimate a foreground state of an image using an actual image which is an image to be actually observed; and a visible model updating unit configured to update a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state.

## Background
BACKGROUND
The present disclosure relates to an information processing device, an information processing method, and a program, and specifically relates to an information processing device, an information processing method, and a program, which allow an agent capable of performing action such as a robot or the like to readily perform learning of an object and so forth within an environment where the agent performs action.
Heretofore, in order to use an image obtained by shooting a certain environment with a camera to perform learning (or recognition) of an object within the environment thereof, trimming of an image area of the object to be learned (or object to be recognized) has had to be performed.
As for a method to trimming an image area to be learned, there have principally been approach employing previous knowledge regarding appearance of an object to be learned (Japanese Unexamined Patent Application Publication No. 7-88791), and approach taking advantage of motion of an object (Japanese Unexamined Patent Application Publication Nos. 5-282275, 7-29081, and 2005-128959).
With the approach employing previous knowledge regarding appearance, creation of a recognition model is performed by performing marking to identify an object, or learning regarding an object to be learned (object to be processed) beforehand.
With the approach taking advantage of motion of an object to be processed, an image area alone where the object is moving using image difference or optical flow or the like is extracted.
Incidentally, for example, with an object operating task wherein a robot operates an object, there has to be distinguished between an object to be operated, and the hand (of the robot itself) which the robot uses to operate the object.
Therefore, with the approach employing previous knowledge regarding appearance, the robot has to identify the label thereof after marking is made on the object and hand with a label to distinguish each thereof. Also, with the approach taking advantage of motion of an object to be processed, there has to be recognized whether or not an image area trimmed from an image shot with a camera is an image area of the object.
Further, at the time of recognizing whether or not an image area trimmed from an image shot with a camera is an image area of the object, a recognizer which performs recognition thereof has to be specified with the hand (has to be given with the knowledge of the hand) so as to distinguish the hand and object.
Also, with technology disclosed in Japanese Unexamined Patent Application Publication No. 2005-128959, there is created beforehand a geometry model such as how a robot arm including a hand is reflected within an image shot with a camera, where the hand position of the robot arm moves when outputting what kind of command to the robot arm, and so forth, and object operations are performed in accordance with the geometry model thereof.
With the technology disclosed in Japanese Unexamined Patent Application Publication No. 2005-128959, object operations are performed in accordance with the geometry model as described above, and accordingly, when changing a relative position between the camera and robot arm, when exchanging the lens of the camera, when changing the size of the robot arm, or the like, the geometry model has to be corrected by hand each time thereof.
SUMMARY
In the event of performing learning of an object within an environment where an agent who enables to perform action such as a robot or the like performs action, there have had to be performed making to identify the object, and extraction of an image area where the object is being moved, which has been troublesome.
It has been found to be desirable to facilitate learning of an object or the like within an environment where an agent who is allowed to perform action such as a robot or the like performs action.
An information processing device or program according to an embodiment of the present technology is an information processing device or program causing a computer serving as an information processing device including: a foreground state estimating unit configured to estimate a foreground state of an image using an actual image which is an image to be actually observed; and a visible model updating unit configured to update a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state.
An information processing method according to an embodiment of the present technology is an information processing method including: estimating a foreground state of an image using an actual image which is an image to be actually observed; and updating a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state.
With an embodiment of the present technology, a foreground state of an image is estimated using an actual image which is an image to be actually observed, and a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground is updated using an estimation result of the foreground state.
Note that the information processing device may be a standalone device, or may be an internal block which makes up one device.
Also, the program may be provided by being transmitted via a transmission medium or by being recorded in a recording medium.
According to an embodiment of the present technology, learning of an object or the like within an environment may be facilitated.



BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a diagram for describing overview of the present technology;
FIG. 2 is a diagram for describing an example of an environment to which the present technology is applied;
FIG. 3 is a diagram illustrating an example of a foreground model and a background model, and an observation image to be generated using the foreground model and background model;
FIG. 4 is a diagram for describing a state of a foreground;
FIG. 5 is a diagram for describing a motion model of the foreground;
FIG. 6 is a diagram for describing action of the foreground;
FIG. 7 is a diagram illustrating an example of a graphical model of a generating model which generates an observation image using the foreground model and background model;
FIG. 8 is a flowchart for describing the EM algorithm;
FIG. 9 is a diagram illustrating an example of a graphical model of a model;
FIG. 10 is a diagram for describing a particle filter;
FIG. 11 is a diagram for describing the roulette method;
FIG. 12 is a diagram for describing equidistant sampling;
FIG. 13 is a diagram illustrating an example of implementation of a generating model which generates an observation image using the foreground model and background model;
FIG. 14 is a diagram for describing updating of a foreground texture model using weighting of a particle;
FIG. 15 is a diagram for describing updating of a background texture model;
FIG. 16 is a diagram for describing object operations by an agent;
FIG. 17 is a diagram illustrating the current state and a target state of an object to be operated;
FIG. 18 is a diagram for describing translation motion of an object;
FIG. 19 is a diagram for describing rotation motion of an object;
FIG. 20 is a diagram for describing the initial position of a hand;
FIG. 21 is a block diagram illustrating a configuration example of an embodiment of an agent to which the present technology has been applied;
FIG. 22 is a block diagram illustrating a configuration example of a foreground state estimating unit;
FIG. 23 is a block diagram illustrating a configuration example of a motion model estimating unit;
FIG. 24 is a block diagram illustrating a configuration example of an action generating unit;
FIG. 25 is a flowchart for describing overview of learning processing for a foreground model, a background model, and a motion model that an agent performs;
FIG. 26 is a flowchart for describing details of the learning processing for a foreground model, a background model, and a motion model that an agent performs;
FIG. 27 is a flowchart for describing overview of action control processing that an agent performs for object operations;
FIG. 28 is a flowchart for describing details of the action control processing that an agent performs for object operations;
FIGS. 29A to 29G are diagrams schematically illustrating simulation results; and
FIG. 30 is a block diagram illustrating a configuration example of an embodiment of a computer to which the present technology has been applied.



DETAILED DESCRIPTION OF EMBODIMENTS
Chapter 0. Overview of Present Technology
FIG. 1 is a diagram for describing an overview of the present technology.
The present technology is technology relating to an algorithm by an agent capable of performing action, such as a robot or the like which includes a hand or the like serving as a manipulator which enables to operate an object within an environment, and a camera or the like which enables to observe (shoot) an environment, learning a model of an environment, object, and a body (a movable portion of the agent) such as the hand or the like, to realize operating an object under an environment, from an image observed at the camera, and performing state recognition using the model thereof.
With the present technology, particularly, a designer obtains a model relating to an object within an environment (according to appearance of an object to be operated, and applied acceleration, kinetics parameters for deciding how the object thereof moves under an environment, etc.) by learning in an self-organizing manner from experience that an agent obtained through interaction with an environment without creating the model beforehand.
Further, with the present technology, for example, in the event that an image observed by the camera has become an image with high resolution, or in the event that the particle size of a state variable to be expressed becomes fine due to the attitude of an object within an environment changing or the like, and even in the event that the number of dimensions increases, learning is performed with high speed.
With the present technology, experience of object operations obtained through interaction between an agent and an environment is input to a learning device through a camera mounted on the agent as image time series.
Here, it is commonly difficult to start learning in a state in which no explicit marker has been appended to the hand and object, and no preliminary knowledge regarding the hand and the appearance of the object has been given, and to learn a model regarding the hand and the appearance of the object from an image on which a scene where the hand and object are excising in a mixed manner is reflected. Also, in the event that the particle size of a state variable to be expressed becomes fine, or in the event that the number of dimensions increases, when calculation amount for learning exponentially increases, this causes a problem on practicality under a condition for expecting real-time requirement such as control of an agent under the actual environment.
With the present technology, there are prepared frameworks for describing image observation series (image observation value series) by expressing an image observed with the camera by separating the image into the hand of an agent, a model regarding appearance and motion of an object to be operated (foreground), and a model of an environment where object operations are performed (background).
Further, with the present technology, parameters of a model are successively estimated using the probability model most-likely estimation method (EM (Expectation Maximization) algorithm) based on image observation series on which experience of object operations and so forth are reflected, thereby learning the model from experience in a self-organizing manner.
Note that, as for image observation series, a moving image taken with a video camera, a moving image captured from screens of a computer game, or the like may be employed, for example.
Also, with the present technology, though learning of a model may be performed using image observation series observed from a three-dimensional environment by extending frameworks of observation models (later-described foreground visible model and background visible model), hereinafter, for simplification of description, description will be made assuming that image observation series observed from a two-dimensional environment are employed for learning of a model.
Further, with the following description, though an image template is employed as implementation of an observation model, local feature amount and so forth of an image may additionally be employed as an observation model, for example.
Chapter 1. Environmental Setting
FIG. 2 is a diagram for describing an example of an environment to which the present technology is applied.
In FIG. 2, with an environment serving as a two-dimensional plane, a L-shaped object and an hand of an agent are allocated.
The hand of the agent is attached to the tip of an arm equivalent to the arm, and the hand attached to the tip of the arm thereof is also moved by the agent moving the arm. Accordingly, the agent may autonomously move the hand.
On the other hand, the L-shaped object does not autonomously move, and when force is externally applied such as the hand or the like, moves in accordance with the applied force thereof.
With the agent, an image is observed wherein an environment is shot by a camera, the environment is reflected as the background, and the L-shaped object which moves while changing the position and attitude and the hand are reflected as the foreground overlapped with the background thereof.
Note that, though FIG. 2 illustrates a two-dimensional environment, the present technology may be applied to a three-dimensional environment as described above.
Also, in FIG. 2, though the image observed at the agent includes two foregrounds of a foreground corresponding to the L-shaped object and a foreground corresponding to the hand, the number of foregrounds existing in the image may be one or may be three or more.
With the present technology, regarding the i-th foreground #i of the foregrounds included in an image, a state such as the position or attitude or the like of center-of-gravity at point-in-time t (center-of-gravity of inertial mass) is represented as zit, a motion model is represented as ai, and action performed by the foreground at the point-in-time t for movement or the like, or action applied to the foreground is represented as uit, respectively.
Also, a model representing the background of an image will also be referred to as a background model, and also, a model representing the foreground #i of the image will also be referred to as a foreground model #i.
The foreground model #i includes a foreground visible model μi which is a visible model of the foreground #i.
The foreground visible model μi includes a foreground texture model τi representing the texture of the foreground #i, and a foreground shape model σi representing the shape of the foreground #i, and is expressed with Expression (1).




  
    
      
        
          μ
          i
        
        =
        
          [
          
            
              
                
                  τ
                  i
                
              
            
            
              
                
                  σ
                  i
                
              
            
          
          ]
        
      
    
    
      
        (
        1
        )
      
    
  




The background model includes a background visible model μw which is a visible model of the background.
The background visible model μw includes a background texture model τw representing the texture of the entire background.

μw=τw  (2)

With the present technology, a generating model which generates an image observation value includes the foreground model #i and the background model as described above, and generates an image observation value (hereinafter, also referred to as observation image) xt at each point-in-time t.
FIG. 3 is a diagram illustrating an example of the foreground model #i, and background model, and the observation image xt generated at the generating model including the foreground model #i and background model.
With the foreground model #i, the foreground texture model τi and foreground shape model σi of the foreground visible model μi are, for example, images with the same size, and the maximum size assumed as the size of the foreground may be employed as the size thereof.
The image serving as the foreground texture model τi includes values serving as the texture of the foreground #i (e.g., RGB or the like) as the pixel values of the pixels (pixels, voxels).
The image serving as the foreground shape model σi includes consecutive values in a range of 0 through 1 as the pixel values of the pixels. The pixel value of each pixel of the foreground shape model σi represents probability that the pixel thereof will be the foreground (a pixel on which the foreground is reflected).
Now, if we say that product of the pixel values of pixels in the same position is represented as *, visibility of the foreground #i is expressed with the product of the pixel values of pixels in the same position as with the foreground texture model τi included in the foreground visible model μi, and the foreground shape model σi (i.e., τi*σi).
In FIG. 3, two foreground visible models μ1 and μ2 are superimposed on the background visible model (μw=τw, and the observation image xt is generated.
FIG. 4 is a diagram for describing a state zit of the foreground #i.
With regard to the state zit of the foreground #i, there are defined a world coordinates system O serving as a coordinates system for expressing, so to speak, an absolute state of the foreground #i in the environment (background), and a local coordinates system O′ serving as a coordinates system for expressing the state of the foreground in the foreground model (foreground visible model) regardless of the state of the foreground #i on the world coordinates system O. The local coordinates system O′ is defined for each foreground model.
As for the world coordinates system O, for example, there may be employed a two-dimensional coordinates system wherein from the upward to downward direction of the background visible model μw=τw is taken as an s1 axis, and from the left to right direction is taken as an s2 axis, and also, a point on the upper left of the background visible model μw=τw is taken as the origin.
As for the local coordinates system O′, for example, there may be employed a two-dimensional coordinates system wherein from the upward to downward direction of the foreground visible model μ(foreground texture model τi, foreground shape model τi) is taken as an r1 axis, and from the left to right direction is taken as an r2 axis, and also, a point on the upper left of the foreground visible model μ is taken as the origin.
Note that, with the local coordinates system O′, let us say that the rx axis is parallel to the s1 axis, and the r2 axis is parallel to the s2 axis as default (e.g., t=0), for example. In FIG. 4, a two-dimensional coordinates system with a dotted line represents the default local coordinates system O′.
The state zit of the foreground #i includes, for example, a position sit on the world coordinates system O of the mass center of gravity of the foreground #i (foreground visible model μi) at the point-in-time t, speed s−it of change in the position sit, attitude of the foreground #i, and speed q−it of change in attitude qit, and is expressed with Expression (3).




  
    
      
        
          z
          t
          i
        
        =
        
          [
          
            
              
                
                  s
                  t
                  i
                
              
            
            
              
                
                  s
                  t
                  
                    ′
                    ⁢
                    
                      
                    
                    ⁢
                    i
                  
                
              
            
            
              
                
                  q
                  t
                  i
                
              
            
            
              
                
                  q
                  t
                  
                    ′
                    ⁢
                    
                      
                    
                    ⁢
                    i
                  
                
              
            
          
          ]
        
      
    
    
      
        (
        3
        )
      
    
  




Here, as for the attitude qit of the foreground #i, for example, there may be employed an angle (rotation angle) that the axis r1 (r2) of the local coordinates system O′ makes up against the axis s1 (s2) of the world coordinates system O.
FIG. 5 is a diagram for describing a motion model ai of the foreground #i.
The motion model ai includes a position ric on the local coordinates system O′ of the mass center of gravity (inertial mass) mi that decides the characteristic of motion (translation motion) of the translational component of the foreground #i, and inertial moment IIc around center of gravity that decides the characteristic of motion (rotation motion) of the rotation component of the foreground, and is represented with Expression (4).




  
    
      
        
          a
          i
        
        =
        
          [
          
            
              
                
                  r
                  c
                  i
                
              
            
            
              
                
                  m
                  i
                
              
            
            
              
                
                  I
                  c
                  i
                
              
            
          
          ]
        
      
    
    
      
        (
        4
        )
      
    
  




FIG. 6 is a diagram for describing action uit of the foreground #i.
The action uit of the foreground #i includes, at the point-in-time t, a position (point of load) riF on the local coordinates system O′ where force is applied to the foreground #i, and force Fi applied to the position riF thereof, and is represented with Expression (5).




  
    
      
        
          u
          t
          i
        
        =
        
          [
          
            
              
                
                  r
                  F
                  i
                
              
            
            
              
                
                  F
                  i
                
              
            
          
          ]
        
      
    
    
      
        (
        5
        )
      
    
  




Note that the image observation value xit observed as the foreground #i at the point-in-time t is expressed with Expression (6) using an image expected value λt to be observed at the generating model at the point-in-time t.

xti=λt  (6)

Chapter 2. Learning of Foreground Model and Background Model
Section 2-1. Frameworks of Generation Model
FIG. 7 is a diagram illustrating an example of a graphical model of a generating model which generates an observation image.
FIG. 7 is a diagram illustrating an example of a graphical model of a generating model of an image including two foregrounds #1 and #2.
According to the generating model in FIG. 7, a state z1t at the point-in-time t of the foreground #1 is decided with a motion model a2 of the foreground #1, a state z1t−1 at the last point-in-time t−1, and action u1t−1 of the foreground #1 at the last point-in-time t#1.
Similarly, a state z2t at the point-in-time t of the foreground #2 is decided with a motion model a2 of the foreground #2, a state z2t−1 at the last point-in-time t−1, and action u2t−1 of the foreground #2 at the last point-in-time t−1.
With the generating model, (an image serving as) the foreground visible model μi is superimposed on (an image serving as) the background visible model μw in accordance with the state z1t at the point-in-time t of the foreground #1, and also, (an image serving as) the foreground visible model μ2 is superimposed on (an image serving as) the background visible model μw in accordance with the state z2t at the point-in-time t of the foreground #2, thereby generating an observation image xt to be observed at the point-in-time t.
In FIG. 7, a function f( ) is a function to generate an observation image xt using the foreground visible model (μi, state z1t at the point-in-time t of the foreground #1, foreground visible model μ2, state z2t at the point-in-time t of the foreground #2, and background visible model μw, and represents the generating model.
Here, the observation image xt at the point-in-time t generated at the generating model is a random variable (an expected value for an observation value), and the likelihood of the observation image xt may be obtained by comparison with the observed value x ̃t of an actually observed image.
Also, let us say that action u1t of the foreground #i is a decisive parameter that may be observed at the agent. This is because, in the event that the foreground #i is the hand of the agent, the agent has to sense action of the hand which is a portion of the own body, and in the event that the foreground #i is an object to be operated, the agent has to be able to sense action of the object to be operated which is in contact with the hand as reaction.
The foreground visible model μi, background visible model μw, and the state zi of the foreground #i, and further as appropriate the motion model ai are, with the present technology, parameters of the generating model which generates an image observation value, and are obtained by learning.
Also, the state zi of the foreground #i is a latent variable that changes from time to time, the motion model ai, foreground visible model μi, and background visible model μw are permanent parameters (model parameters) that do not depend on time.
Now, let us say that the total number of foregrounds is represented with N. In FIG. 7, though two is set as the total number of foregrounds N, a value of one or three or more may be employed as the total number of foregrounds N.
The observation image xt to be generated at the generating model is decided with the state zit of the foreground #i serving as a latent variable, the foreground visible model μit and background visible model μw serving as model parameters, and accordingly, the observation image xt may be represented with Expression (7) using the function f( ) serving as the generating model.

xt=f(zt1, . . . ,ztN, . . . ,μN,μw)  (7)

Section 2-2. EM Algorithm

FIG. 8 is a flowchart for describing the EM algorithm.
As with the generating model illustrated in FIG. 7, examples of a method for obtaining the most-likely solution for the model parameters using an observation value to be actually observed regarding a model including the latent variable (zit) include the Expectation-Maximization (EM) algorithm.
Now, let us say that a group of all of the observation values is represented as X, and a group of latent (hidden) variables is represented as Z, respectively.
With observation of the observation value X, the latent variable Z fails to be observed, but the group {X, Z} of the observation value X and latent variable Z is referred to as complete data, and the observation value X is referred to as incomplete data.
Let us say that the complete data {X, Z} is able to be modeled with a model that is a probability distribution, and (a model parameter of) the model thereof is represented with θ.
With the model θ, logarithmic likelihood In(p(X|θ)) where the observation value X is observed may be obtained by marginalizing likelihood p(X, Z|θ) where the complete data {X, Z} is observed with the latent variable Z, and taking logarithm therefrom, and is represented with Expression (8).




  
    
      
        
          In
          ⁢
          
            
          
          ⁢
          
            p
            ⁡
            
              (
              
                X
                |
                θ
              
              )
            
          
        
        =
        
          ln
          ⁢
          
            {
            
              
                ∑
                z
              
              ⁢
              
                p
                ⁡
                
                  (
                  
                    X
                    ,
                    
                      Z
                      |
                      θ
                    
                  
                  )
                
              
            
            }
          
        
      
    
    
      
        (
        8
        )
      
    
  




Here, ln represents a natural logarithm.
Also, hereinafter, the same discussion holds regarding a case where the latent variable Z is a consecutive value, by replacing summation (Σ) regarding the latent variable Z with a product.
The incomplete data X alone may be observed, and all of the complete data {X, Z} fail to be observed. Accordingly, the likelihood (function) p(X, Z|θ) of the complete data {X, Z} fails to be used, and accordingly, with the EM algorithm, an expected value for posterior probability regarding the latent variable Z is alternatively considered.
A process to consider an expected value for posterior probability regarding the latent variable Z is the E step of the EM algorithm.
With the EM algorithm, with the model θ, the current model parameter θold is employed for calculation of the posterior distribution p(Z|X, θ) of the latent variable Z in the event that the observation value X has been observed, and this posterior distribution p(Z|X, θold) is employed for calculation of an expected value Q(θ, θold) of the logarithmic likelihood (hereinafter, also referred to as complete data logarithmic likelihood) ln(p(X, Z|θ) where the complete data {X, Z} is observed with an optional model parameter θ.
Accordingly, the expected value Q(θ, θold) is represented with Expression (9).




  
    
      
        
          Q
          ⁡
          
            (
            
              θ
              ,
              
                θ
                old
              
            
            )
          
        
        =
        
          
            ∑
            z
          
          ⁢
          
            
              p
              ⁡
              
                (
                
                  
                    Z
                    |
                    X
                  
                  ,
                  
                    θ
                    old
                  
                
                )
              
            
            ⁢
            ln
            ⁢
            
              
            
            ⁢
            
              p
              ⁡
              
                (
                
                  X
                  ,
                  
                    Z
                    |
                    θ
                  
                
                )
              
            
          
        
      
    
    
      
        (
        9
        )
      
    
  




With Expression (9), the expected value Q(θ, θold) of the complete data logarithmic likelihood ln(p(X, Z|θ) is calculated using the posterior distribution p(Z|X, θold) as a probability distribution of the complete data logarithmic likelihood ln(p(X, Z|θ).
With the EM algorithm, in the E step, as described above, after the expected value Q(θ, θold) is obtained, in the M step, the model parameter θ is updated from the current model parameter θold to a new model parameter θnew so as to maximize the expected value Q(θ, θold) in accordance with Expression (10).




  
    
      
        
          θ
          new
        
        =
        
          
            argmax
            θ
          
          ⁢
          
            
          
          ⁢
          
            Q
            ⁡
            
              (
              
                θ
                ,
                
                  θ
                  old
                
              
              )
            
          
        
      
    
    
      
        (
        10
        )
      
    
  




Here, argmax in Expression (10) represents the model parameter θ that maximizes Q(θ, θold).
Note that, before the first E step is performed, the model parameter θ=θold is initialized to a suitable value, for example, using a random number or the like.
The EM algorithm will further be described with reference to FIG. 8.
With the EM algorithm, in step S11, the model parameter θ is initialized to a suitable initial value θold, and the processing proceeds to step S12.
In step S12, the processing in the E step is performed, and the processing proceeds to step S13. Specifically, in step S12, the posterior distribution p(Z|X, θold) is obtained using the incomplete data X.
In step S13, processing in the M step is performed, and the processing proceeds to step S14. Specifically, in step S13, a new model parameter θnew is obtained so as to maximize the expected value Q(θ, θold) of the complete data logarithmic likelihood ln(p(X, Z|θ) to be obtained with the posterior distribution p(Z|X, θold).
In step S14, determination is made whether or not predetermined convergence conditions for ending the EM algorithm are satisfied.
Here, examples of the convergence conditions for ending the EM algorithm may include that the E step and M step have been repeated a predetermined number of times, that the new model parameter θnew has not scarcely changed from the last model parameter θold, and that the likelihood of the observation value X observed with the new model parameter θnew has not scarcely changed from the likelihood of the observation value X observed with the last model parameter θold.
In the event that determination is made in step S14 that the convergence conditions are not satisfied, the processing proceeds to step S15, where the model parameter θ is updated from the current model parameter θold to the new model parameter θnew.
The processing then returns from step S15 to step S12, and the same processing will hereinafter be repeated.
Also, in the event that determination is made in step S14 that the convergence conditions are satisfied, the processing ends.
Section 2-3. Approximation with Particle Filter
FIG. 9 is a diagram illustrating an example of a graphical model of the model 9 which expresses the complete data {X, Z}.
In FIG. 9, series x1, x2, . . . , xt+1 of the observation value X are represented with linkages z1, z2, . . . , zt+1 of the latent variable Z. The observation values xt are conditioned with the value (state) of the corresponding latent variable zt.
In the event that the model θ is a model which makes it difficult to analytically calculate the E step of the EM algorithm, the E step may be calculated with the approximation reasoning method based on numerical sampling (Monte Carlo sampling).
The Monte Carlo sampling has been described in CM. Bishop, “Pattern Recognition and Machine Learning” Second volume Statistical Prediction by Bayers' Theorem, Springer Japan, 2008, pp. 364-365, Michael Isard and Andrew Blake, CONDENSATION-conditional density propagation for visual tracking, Int. J. Computer Vision, 29, 1, 5-28, 1988, and so forth.
In the event that the latent variable Z is a consecutive value, the expected value Q(θ, θold) of the complete data logarithmic likelihood ln(p(X, Z|θ) is represented with Expression (11) where the summation (S) in Expression (9) has been repressed with a product.

Q(θ,θold)=∫p(Z|X,θold)ln p(Z,X|θ)dZ  (11)

The expected value Q(θ, θold) in Expression (11) may be approximated as indicted in Expression (12) with the finite sum of L samples Z(l) (l=1, 2, . . . L) of the latent variable Z sampled from the current posterior distribution p(Z|X, θold) using a sampling method.




  
    
      
        
          Q
          ⁡
          
            (
            
              θ
              ,
              
                θ
                old
              
            
            )
          
        
        ≅
        
          
            ∑
            
              l
              =
              1
            
            L
          
          ⁢
          
            ln
            ⁢
            
              
            
            ⁢
            
              p
              ⁡
              
                (
                
                  
                    Z
                    
                      (
                      l
                      )
                    
                  
                  ,
                  
                    X
                    |
                    θ
                  
                
                )
              
            
          
        
      
    
    
      
        (
        12
        )
      
    
  




The expected value Q(θ, θold) obtained in accordance with Expression (12) may be optimized by the normal M step (the model parameter θ=θnew to maximize the expected value Q(θ, θold) may be obtained).
As described above, the EM algorithm to obtain the model parameter θ=θnew that maximizes the expected value Q(θ, θold) obtained in accordance with Expression (12) is called the Monte Carlo EM algorithm.
On the other hand, as a technique for successively performing sampling for the series x1, x2, . . . , xt of the observation value X, there is the sequential Monte Carlo algorithm called a particle filter.
FIG. 10 is a diagram for describing a particle filter.
With a particle filter, for example, with the model θ expressed with the graphical model illustrated in FIG. 9, series Xt={x1, x2, . . . , xt−1} of the observation value X are observed until point-in-time t−1, and L states z(1)t (z(1)t, z(2)t, . . . z(L)t) are sampled from the posterior distribution p(zt|Xt−1) in the state zt (becoming the state zt) at the point-in-time t. This sampled sample (here, the state z(1)t) is called a particle.
With the state zt at the point-in-time t, the expected value E[f(zt)] of the observation value xt=f(zt) observed in accordance with FIG. 7 and the function f( ) serving as the generating model in Expression (7) may be approximated with Expression (13) using the particle in the state zt (state particle) z(1)t.




  
    
      
        
          E
          ⁡
          
            [
            
              f
              ⁡
              
                (
                
                  z
                  t
                
                )
              
            
            ]
          
        
        ≅
        
          
            ∑
            
              l
              =
              1
            
            L
          
          ⁢
          
            
              w
              t
              
                (
                l
                )
              
            
            ⁢
            
              f
              ⁡
              
                (
                
                  z
                  t
                  
                    (
                    l
                    )
                  
                
                )
              
            
          
        
      
    
    
      
        (
        13
        )
      
    
  




Here, w(1)t represents weighting of the particle z(1)t. The weighting w(1)t is a value corresponding to the (observation) likelihood p(xt|z(1)t) where the observation value xt is observed in the state z(1)t (serving as a particle), and is defined with Expression (14).




  
    
      
        
          w
          t
          
            (
            l
            )
          
        
        =
        
          
            p
            ⁡
            
              (
              
                
                  x
                  t
                
                |
                
                  z
                  t
                  
                    (
                    l
                    )
                  
                
              
              )
            
          
          
            
              ∑
              
                m
                =
                1
              
              L
            
            ⁢
            
              p
              ⁡
              
                (
                
                  
                    x
                    t
                  
                  |
                  
                    z
                    t
                    
                      (
                      m
                      )
                    
                  
                
                )
              
            
          
        
      
    
    
      
        (
        14
        )
      
    
  




Note that the weighting w(1)t is a value within a range represented with Expression 0≦w(1)t≦1, and satisfies Expression w(1)t+w(2)t+ . . . +w(L)t=1.
Now, in the event that L particles z(1)t to z(L)t and the weighting w(1)t to w(L)t were obtained at the point-in-time t, when the observation value xt+1 was observed at the point-in-time t+1, the series Xt={x1, x2, . . . xt} of the observation value X are observed until the point-in-time t, and the posterior distribution p(zt+1|Xt) that is in the state zt+1 at the point-in-time t+1 may be obtained with Expression (15) using transition probability p(zt+1|z(1)t) that the state z(1)t at the point-in-time t will change to the state zt+1 at the point-in-time t+1, and the weighting w(1)t of the particle z(1)t.




  
    
      
        
          p
          ⁡
          
            (
            
              
                z
                
                  t
                  +
                  1
                
              
              |
              
                X
                t
              
            
            )
          
        
        ≅
        
          
            ∑
            
              l
              =
              1
            
            L
          
          ⁢
          
            
              w
              t
              
                (
                l
                )
              
            
            ⁢
            
              p
              ⁡
              
                (
                
                  
                    z
                    
                      t
                      +
                      1
                    
                  
                  |
                  
                    z
                    t
                    
                      (
                      l
                      )
                    
                  
                
                )
              
            
          
        
      
    
    
      
        (
        15
        )
      
    
  




The posterior distribution p(zt+1|Xt) in Expression (15) is a mixed distribution with the weighting w(1)t as a mixed coefficient, and with the particle filter, re-sampling to select the particle z(1)t is performed with probability corresponding to the weighting w(1)t serving as a mixed coefficient.
The particle filter will further be described with reference to FIG. 10.
With the particle filter, the series Xt−1={x1, x2, . . . , Xt−1} of the observation value X are observed until the point-in-time t−1, the posterior distribution p(zt|Xt−1) that is in the state zt at the point-in-time t is represented with samples by the L particles z(1)t−1 to z(L)t−1 and weighting w(1)t−1 to w(L)t−1.
Here, in FIG. 10, the number L of the particles z(1)t is 5.
With the particle filter, in order to obtain sample expression at the next point-in-time t, based on the transition probability (zt|z(1)t−1) of the mixed distribution (posterior distribution) p(zt|Xt−1) in Expression (15), a transition destination at the next point-in-time t of the L particles z(1)t−1 to z(L)t−1 at the point-in-time t−1 is calculated, i.e., the L particles z(1)t to z(L)t at the next point-in-time t are predicted (generated).
Here, in FIG. 10, round marks represent particles. Also, a number within a round mark representing a particle indicates an example of the value of the state z(1)t serving as the particle thereof.
In FIG. 10, for example, though the number of a round mark representing the state z(5)t−1 serving as a particle at the point-in-time t−1 is 5.0, the number of a round mark representing the state z(5)t−1 serving as a particle at the point-in-time t predicted based on the transition probability p(zt|z(5)t−1) is 5.4.
After the L particles z(1)t to z(L)t at the next point-in-time t are predicted, at the particle filter, the observation likelihood p(xt|z(1)t) where the observation value xt is observed at the particle z(1)t, is calculated with the observation value xt at the point-in-time t.
Further, with the particle filter, the weighting w(1)t of the particle z(1)t is obtained in accordance with Expression (14) using the observation likelihood p(xt|z(1)t). In FIG. 10, a round mark representing the weighting w(1)t is illustrated with a size equivalent to the value of the weighting w(1)t thereof.
Thereafter, with the particle filter, the particle z(1)t is sampled with probability corresponding to the weighting w(1)t.
With the particle filter, similar processing is repeated using the re-sampled particle z(1)t thereof along with the weighting w(1)t as sample expression of the posterior distribution p(zt+1|Xt) existing in the state zt+1 at the point-in-time t+1 after the series Xt={x1, x2, . . . , xt} of the observation value X are observed until the point-in-time t.
Note that, with re-sampling, the weighting w(1)t is small since the particle z(1)t is sampled with probability corresponding to the weighting w(1)t, and accordingly, the particle z(1)t which has not sampled at all disappears.
Also, with re-sampling, the particle z(1)t of which the weighting w(1)t is great may be sampled more than once, and in this case, the particle having the same value (state) as with the particle z(1)t is copied by the same number as with the number of times of sampling.
As described above, with re-sampling, though there may be generated multiple particles having the same value V as with the particle z(1)t, the value of each of the multiple particles is scattered around the same value V when prediction (calculation of a transition destination) of the particles at the next point-in-time t is performed, based on the transition probability p(zt+1|z(1)t) in Expression (15), and becomes a different value.
Here, examples of a re-sampling method for sampling the particle z(1)t with probability corresponding to the weighting w(1)t include the roulette method, and a method called equidistant sampling.
FIG. 11 is a diagram for describing the roulette method.
With the roulette method, a range of 0 to 1 is sequentially assigned to the L weighting w(1)t, w(2)t, . . . , w(L)t. Width proportional to the size of the weighting w(1)t of a range of 0 to 1 is assigned to the weighting w(1)t.
With the roulette method, it is repeated until the number of times of sampling becomes the same as the total number L of particles that a value within a range of 0 to 1 is sampled with a random number, and a particle corresponding to the weighting w(1)t to which the sampled value thereof has been assigned is copied.
FIG. 12 is a diagram for describing equidistant sampling.
With equidistant sampling, as with the roulette method, a range of 0 to 1 is assigned to the L weighting w(1)t, w(2)t, . . . , w(L)t.
With equidistant sampling, a value within a range of 0 to 1 is sampled with a random number, and a particle corresponding to the weighting w(1)t to which the sampled value thereof has been assigned is copied.
Thereafter, with equidistant sampling, it is repeated until the number of times of sampling becomes the same as the total number L of particles that of a value within a range of 0 to 1, a value obtained by adding a predetermined value to the last sampled value is sampled, and a particle corresponding to the weighting w(1)t to which the sampled value thereof has been assigned is copied.
Note that, in the event that the value obtained by adding a predetermined value to the last sampled value exceeds 1, a value below a decimal point of the value thereof is sampled.
With the roulette method, L random numbers have to be calculated, and on the other hand, with equidistant sampling, only one random number has to be calculated, and accordingly, equidistant sampling is small in random number calculation cost as compared to the roulette method.
Chapter 3. Specific Implementation
Section 3-1. Implementation of Generation Model
FIG. 13 is a diagram illustrating an example of implementation of the generating model θ which generates an observation image, i.e., the function f( ) in Expression (7) using the foreground model #i and background model.
With the generating model, the foreground model μi is superimposed on the background visible model μw with a position sit and attitude qit that the state z1t represents.
With superimposition of the foreground visible model μi on the background visible model μw, α blending is performed between (texture represented with) a background texture model τw serving as the background visible model μw, and a foreground texture model τi of the foreground visible model μi using a foreground shape model σi of the foreground visible model μi as an α value.
For example, now, if we say that the first foreground #1 alone exists in the observation image xt, there is represented with Expression (16) an expected value λ1(z1t, μ1, μw) of the observation image Xt where the single foreground #1 alone exists, to be observed at the generating model where model parameters (foreground visible model μi, background visible model μw, the state zi of the foreground #i, and if necessary motion model ai) have been obtained.




  
    
      
        
          
            λ
            1
          
          ⁡
          
            (
            
              
                z
                t
                1
              
              ,
              
                μ
                1
              
              ,
              
                μ
                w
              
            
            )
          
        
        =
        
          
            
              λ
              t
              1
            
            ⁡
            
              (
              
                r
                t
                
                  ′
                  ⁢
                  
                    
                  
                  ⁢
                  1
                
              
              )
            
          
          =
          
            {
            
              
                
                  
                    
                      
                        (
                        
                          1
                          -
                          
                            
                              σ
                              1
                            
                            ⁡
                            
                              (
                              
                                r
                                1
                              
                              )
                            
                          
                        
                        )
                      
                      ⁢
                      
                        
                          τ
                          w
                        
                        ⁡
                        
                          (
                          
                            r
                            t
                            
                              ′
                              ⁢
                              
                                
                              
                              ⁢
                              1
                            
                          
                          )
                        
                      
                    
                    +
                    
                      
                        
                          σ
                          1
                        
                        ⁡
                        
                          (
                          
                            r
                            1
                          
                          )
                        
                      
                      ⁢
                      
                        
                          τ
                          1
                        
                        ⁡
                        
                          (
                          
                            r
                            1
                          
                          )
                        
                      
                    
                  
                
                
                  
                    
                      if
                      ⁢
                      
                        
                      
                      ⁢
                      
                        r
                        1
                      
                    
                    ∈
                    
                      μ
                      1
                    
                  
                
              
              
                
                  
                    
                      τ
                      w
                    
                    ⁡
                    
                      (
                      
                        r
                        t
                        
                          ′
                          ⁢
                          
                            
                          
                          ⁢
                          1
                        
                      
                      )
                    
                  
                
                
                  else
                
              
            
          
        
      
    
    
      
        (
        16
        )
      
    
  




Here, σ1(r1) represents a pixel value of a point r1 on the local coordinates system O′ of the foreground shape model σ1, τ1(r1) represents a pixel value of a point r1 on the local coordinates system O′ of the foreground texture model τ1. Further, τw(r′1t) represents a pixel value of a point r′1t on the world coordinates system O of the background texture model τw.
Also, a point r′it obtained by adding a dash (′) and an index t of point-in-time to a point ri on the local coordinates system O′ represents a point (corresponding point) on the world coordinates system O corresponding to the point ri on the local coordinates system O′ at the point-in-time t.
For example, if we say that a mapping function for mapping a point A on the local coordinates system O′ in the event that the local coordinates system O′ is rotated around the origin of the world coordinates system O by an angle B in the corresponding point on the world coordinates system O is represented with R(A, B), the point r′it on the world coordinates system O corresponding to the point ri on the local coordinates system O′ is represented with Expression (17) using the mapping function R( ).

r′ti=sti+R(ri−rcu,qti)  (17)

According to Expression (16), in the event that the point r1 on the local coordinates system O′ is a point on the foreground visible model μ1 (the image serving as the foreground texture model τi or foreground shape model σ1) (r1εμ1), as for the pixel value λ1t(r′1) of the corresponding point r′1t of the point r1 thereof, there is employed a value (1−σ1(r1)) τw(r′1t)+σ1(r1) τ1(r1) obtained by performing α blending between the pixel value τw(r′1t) of the corresponding point r′1t of the background texture model τw and the pixel value τ1(r1) of the point r1 of the foreground texture model τ1 using the pixel value σ1(r1) of the point r1 of the foreground shape model σ1 as an α value.
Further, according to Expression (16), in the event that the point r1 on the local coordinates system O′ is not a point on the foreground visible model μ1, as for the pixel value λ1t(r′1t) of the corresponding point r′1t of the point r1, the pixel value τw(r′1t) of the corresponding point r′1t of the background texture model τw is employed.
In the event that there are multiple foregrounds (in the event that there are multiple foreground models) in the observation image xt, the same processing as with a case where the first foreground #1 alone exists in the observation image xt is repeatedly performed on each of the multiple foregrounds, an expected value of the observation image xt where the multiple foregrounds exist to be observed at the generating model may be obtained.
However, in the event that there are multiple foregrounds in the observation image xt, the expected value of the observation image xt obtained regarding the last foreground is employed instead of the background texture model xw (background visible model μw) regarding the second foreground and thereafter.
Accordingly, for example, now, if we say that two foregrounds #1 and #2 exist in the observation image xt, as illustrated in FIG. 13, first, the expected value λ1(z1t, μ1, μw) of the observation image xt where the first foreground #1 alone exists to be observed at the generating model is obtained in accordance with Expression (16).
An expected value λ1,2(z1t, z2t, μ1, μ2, μw) of the observation image xt where the first foreground #1 and second foreground #2 exist is obtained, as illustrated in FIG. 13, using the expected value λ1(z1t, μ1, μw) of the observation image xt obtained regarding the foreground #1 in Expression (16) instead of the background texture model τw in the same way as with the expected value λ1(z1t, μ1, μw) of the observation image xt.
Specifically, the expected value λ1,2(z1t, z2t, μ1, p2, pw) of the observation image xt where the first foreground #1 and second foreground #2 exist is obtained in accordance with Expression (18).




  
    
      
        
          
            λ
            
              1
              ,
              2
            
          
          ⁡
          
            (
            
              
                z
                t
                1
              
              ,
              
                z
                t
                2
              
              ,
              
                μ
                1
              
              ,
              
                μ
                2
              
              ,
              
                μ
                w
              
            
            )
          
        
        =
        
          
            
              λ
              t
              
                1
                ,
                2
              
            
            ⁡
            
              (
              
                r
                t
                
                  ′
                  ⁢
                  
                    
                  
                  ⁢
                  2
                
              
              )
            
          
          =
          
            {
            
              
                
                  
                    
                      
                        (
                        
                          1
                          -
                          
                            
                              σ
                              2
                            
                            ⁡
                            
                              (
                              
                                r
                                2
                              
                              )
                            
                          
                        
                        )
                      
                      ⁢
                      
                        
                          λ
                          t
                          1
                        
                        ⁡
                        
                          (
                          
                            r
                            t
                            
                              ′
                              ⁢
                              
                                
                              
                              ⁢
                              2
                            
                          
                          )
                        
                      
                    
                    +
                    
                      
                        
                          σ
                          2
                        
                        ⁡
                        
                          (
                          
                            r
                            2
                          
                          )
                        
                      
                      ⁢
                      
                        
                          τ
                          2
                        
                        ⁡
                        
                          (
                          
                            r
                            2
                          
                          )
                        
                      
                    
                  
                
                
                  
                    
                      if
                      ⁢
                      
                        
                      
                      ⁢
                      
                        r
                        2
                      
                    
                    ∈
                    
                      μ
                      2
                    
                  
                
              
              
                
                  
                    
                      λ
                      t
                      1
                    
                    ⁡
                    
                      (
                      
                        r
                        t
                        
                          ′
                          ⁢
                          
                            
                          
                          ⁢
                          2
                        
                      
                      )
                    
                  
                
                
                  else
                
              
            
          
        
      
    
    
      
        (
        18
        )
      
    
  




Here, of the pixel values of images serving as the expected value λ1,2(z1t, z2t, μ1, p2, pw) of the observation image xt where the first foreground #1 and second foreground #2 exist, λ1,2t(r′2t) represents the pixel value of the corresponding point r′2t on the world coordinates system O of the point r2 on the local coordinates system O′ (of the foreground #2).
Even when there are N foregrounds equal to or greater than three foregrounds, an expected value λ1,2, . . . , N(z1t, z2t, . . . , zNt, μ1, μ2, . . . , μN, μw) of the observation image xt where the N foregrounds exist may be obtained in the same way.
Section 3-2. Estimation of State of Foreground When One Foreground Exists (E step)
Description will be made regarding specific implementation of the E step using the particle filter described in Section 2-3 in the event that there is one foreground.
Note that the number of foregrounds is here one, and accordingly, description of the index i representing the i-th foreground will be omitted.
With the E step in the EM algorithm, the posterior distribution p(Z|X, θold) of the latent variable Z is obtained using the model parameter θ=θold updated in the last M step, and the expected value Q(θ, θold) of the complete data logarithmic likelihood ln(p(X, Z|θ) is calculated in accordance with Expression (9) using this posterior distribution p(Z|X, θold).
In the event that the particle filter has been applied to this E step, the posterior distribution p(zt+1|Xt) of the latent variable zt+1 at the point-in-time t+1 when the series Xt={x1, x2, . . . , xt} of the observation value X are observed by the point-in-time t may be calculated in accordance with Expression (15).
Here, according to Expression (14), the weighting w(1)t in Expression (15) is proportional to the observation likelihood p(xt|z(1)t) where the observation value xt is observed in the sate z(1)t serving as a particle.
Accordingly, in order to obtain the weighting w(1)t, the observation likelihood p(xt|z(1)t) has to be obtained.
With the present technology, assuming that the state of the foreground is in the state z(1)t serving as a particle, the expected value λt of the observation image xt to be observed at the generating model is generated in accordance with the state z(1)t serving as the particle thereof.
With the present technology, error for an image λ ̃t actually observed of the expected value λt of the observation image xt to be observed at the generating model (distance between the expected value λt and the image λ ̃t) is obtained as the observation likelihood p(xt|z(1)t).
Specifically, with the present technology, the observation likelihood p(xt|z(1)t) is obtained, for example, in accordance with Expression (19).




  
    
      
        
          p
          ⁡
          
            (
            
              
                x
                t
              
              |
              
                z
                t
                
                  (
                  l
                  )
                
              
            
            )
          
        
        =
        
          K
          ⁢
          
            
          
          ⁢
          
            exp
            (
            
              -
              
                
              
              ⁢
              
                
                  
                    ∑
                    s
                  
                  ⁢
                  
                    
                      
                      
                        
                          
                            λ
                            t
                          
                          ⁡
                          
                            (
                            s
                            )
                          
                        
                        -
                        
                          
                            λ
                            t
                            ~
                          
                          ⁡
                          
                            (
                            s
                            )
                          
                        
                      
                      
                    
                    2
                  
                
                
                  σ
                  2
                
              
            
            )
          
        
      
    
    
      
        (
        19
        )
      
    
  




Note that λt(s) represents the pixel value of a point s on the world coordinates system O of an image with the expected value λt of the observation image xt to be observed at the generating model θ, and λ ̃t(s) represents the pixel value of a point s on the world coordinates system O of the image λ ̃t actually observed at the point-in-time t.
Also, with Expression (19), K and σ2 are predetermined constants, and are determined beforehand.
On the other hand, probability that the state z(1)t serving as a particle at the point-in-time t will become the state zt+1 at the next point-in-time t+1, i.e., if we say that the transition probability p(zt+1|z(1)t) serving as a particle follows a Gauss distribution (normal distribution), the transition model of the state z(1)t serving as a particle may be represented with Expression (20), for example.




  
    
      
        
          z
          
            t
            +
            1
          
        
        =
        
          
            [
            
              
                
                  
                    s
                    
                      t
                      +
                      1
                    
                  
                
              
              
                
                  
                    s
                    
                      t
                      +
                      1
                    
                    ·
                  
                
              
              
                
                  
                    q
                    
                      t
                      +
                      1
                    
                  
                
              
              
                
                  
                    q
                    
                      t
                      +
                      1
                    
                    ·
                  
                
              
            
            ]
          
          =
          
            
              
                [
                
                  
                    
                      1
                    
                    
                      
                        Δ
                        ⁢
                        
                          
                        
                        ⁢
                        t
                      
                    
                    
                      0
                    
                    
                      0
                    
                  
                  
                    
                      0
                    
                    
                      1
                    
                    
                      0
                    
                    
                      0
                    
                  
                  
                    
                      0
                    
                    
                      0
                    
                    
                      1
                    
                    
                      
                        Δ
                        ⁢
                        
                          
                        
                        ⁢
                        t
                      
                    
                  
                  
                    
                      0
                    
                    
                      0
                    
                    
                      0
                    
                    
                      1
                    
                  
                
                ]
              
              ⁡
              
                [
                
                  
                    
                      
                        s
                        t
                      
                    
                  
                  
                    
                      
                        s
                        t
                        
                          ·
                          
                            
                          
                        
                      
                    
                  
                  
                    
                      
                        q
                        t
                      
                    
                  
                  
                    
                      
                        q
                        t
                        ·
                      
                    
                  
                
                ]
              
            
            +
            
              [
              
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            Σ
                            s
                          
                        
                        )
                      
                    
                  
                
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            Σ
                            
                              s
                              ·
                            
                          
                        
                        )
                      
                    
                  
                
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            Σ
                            q
                          
                        
                        )
                      
                    
                  
                
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            Σ
                            
                              q
                              ·
                            
                          
                        
                        )
                      
                    
                  
                
              
              ]
            
          
        
      
    
    
      
        (
        20
        )
      
    
  




Here, Δt represents interval (time) between the point-in-time t and the next point-in-time t+1, and N(A, B) represents a Gauss distribution where the average vector (average value) is A, and a variance covariance matrix (variance) is B.
In Expression (20), any of Gauss distributions N (0, Σs), N (0, Σs•), N (0, Σq), and N (0, Σq•) is a Gauss distribution where the average vector is 0. Also, the variance covariance matrices Σs, Σs•, Σq, and Σq• are determined beforehand.
According to the transition model in Expression (20), the state zt+1 at the point-in-time t+1 is obtained (predicted) (a transition destination is calculated) assuming that the state serving as a particle has a component that makes the transition in accordance with the position st and attitude qt serving as the state at the pervious point-in-time t, and the one-time derivative value (speed) s•t and q•t thereof, and also makes the transition with stochastic fluctuation in accordance with the Gauss distribution of a predetermined variance covariance matrix where the average vector is 0.
Here, with the transition model in Expression (20), it is not taken into consideration that foreground motion follows a kinetics model for rigid motion.
In the event of considering that foreground motion follows a kinetics model for rigid motion, the transition model of the state z(1)t serving as a particle may be represented with Expression (21) using a foreground motion model a and foreground action ut, for example.




  
    
      
        
          z
          
            t
            +
            1
          
        
        =
        
          
            [
            
              
                
                  
                    s
                    
                      t
                      +
                      1
                    
                  
                
              
              
                
                  
                    s
                    
                      t
                      +
                      1
                    
                    ·
                  
                
              
              
                
                  
                    q
                    
                      t
                      +
                      1
                    
                  
                
              
              
                
                  
                    q
                    
                      t
                      +
                      1
                    
                    ·
                  
                
              
            
            ]
          
          =
          
            
              
                [
                
                  
                    
                      1
                    
                    
                      
                        Δ
                        ⁢
                        
                          
                        
                        ⁢
                        t
                      
                    
                    
                      0
                    
                    
                      0
                    
                  
                  
                    
                      0
                    
                    
                      1
                    
                    
                      0
                    
                    
                      0
                    
                  
                  
                    
                      0
                    
                    
                      0
                    
                    
                      1
                    
                    
                      
                        Δ
                        ⁢
                        
                          
                        
                        ⁢
                        t
                      
                    
                  
                  
                    
                      0
                    
                    
                      0
                    
                    
                      0
                    
                    
                      1
                    
                  
                
                ]
              
              ⁡
              
                [
                
                  
                    
                      
                        s
                        t
                      
                    
                  
                  
                    
                      
                        s
                        t
                        ·
                      
                    
                  
                  
                    
                      
                        q
                        t
                      
                    
                  
                  
                    
                      
                        q
                        t
                        ·
                      
                    
                  
                
                ]
              
            
            +
            
              Δ
              ⁢
              
                
              
              ⁢
              
                t
                ⁡
                
                  [
                  
                    
                      
                        0
                      
                    
                    
                      
                        
                          γ
                          /
                          m
                        
                      
                    
                    
                      
                        1
                      
                    
                    
                      
                        
                          φ
                          /
                          
                            I
                            
                              c
                              ⁢
                              
                                
                              
                            
                          
                        
                      
                    
                  
                  ]
                
              
            
            +
            
              [
              
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            Σ
                            s
                          
                        
                        )
                      
                    
                  
                
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            Σ
                            
                              s
                              ·
                            
                          
                        
                        )
                      
                    
                  
                
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            Σ
                            q
                          
                        
                        )
                      
                    
                  
                
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            Σ
                            
                              q
                              ·
                            
                          
                        
                        )
                      
                    
                  
                
              
              ]
            
          
        
      
    
    
      
        (
        21
        )
      
    
  




Here, γ and Φ represent, in the event that force F serving as a component of the foreground action ut (Expression (5)) has been applied to a position rF of a foreground similarly serving as a component of the foreground action ut, translation force at the position of the center of gravity rc of the foreground and torque of the circumference of the center of gravity rc.
The translation force γ is equal to the force F serving as a component of the foreground action ut. The torque Φ is may be obtained in accordance with Expression Φ=(rF−rc)×F using the force F serving as a component of the foreground action ut, the foreground position rF serving as a component of the foreground action ut, and (the position of) the foreground center of gravity rc serving as a component of the foreground motion model a.
Also, in Expression (21), m and Ic represent foreground mass m (of the corresponding object) serving as a component of the foreground motion model a (Expression (4)), and moment of inertia Ic around the center of gravity.
The transition model in Expression (21) may strictly express the transition (change) of the foreground state since it is considered that the foreground action follows a kinetics model for rigid motion.
In the event that it may be assumed as premise knowledge that the foreground consecutively moves on the background, i.e., in the event that it may be assumed that the foreground state consecutively changes within an environment, preliminary knowledge that the transition models in Expressions (20) and (21) takes as a premise effectively functions such that based on information of the previous point-in-time, neighborhood is randomly searched with noise of a Gauss distribution while conclusively predicting the state of the next point-in-time using the motion model.
As a result thereof, with a process of re-sampling of a particle, calculation of state estimation for a state that a foreground becomes with low possibility may be omitted, e.g., as compared to a method for sectioning a background serving as an environment in a grid shape, and estimating, regarding all of the grid points, whether or not there is a foreground in the positions of the gird points thereof, scalability may be increased.
Section 3-3. Estimation of Foreground State When Multiple Foregrounds Exist (E step)
Description will be made regarding specific implementation of the E step using the particle filter described in Section 2-3 in the event that multiple foregrounds exist.
In the event that multiple foregrounds exist, for each foreground #i there is the state zit of that foreground #i. The l-th particle of the state zit of the foreground #i is represented as zi(1)t. Also, the weighting of the particle zi(1)t is represented as wi(1)t.
With the present technology, (the expected value λt of) the observation image xt observed at the generating model is generated as described in Expression (16) to Expression (18).
In order to calculate the observation likelihood p(xt|zi(1)t) used for calculation of Expression (14) to obtain the weighting wi(1)t of a particle at the particle filter, the expected value λt of the observation image xt to be observed at the generating model has to be calculated using, with regard to the foreground #i of interest, the state zi(1)t serving as a particle to obtain the weighting wi(1)t, and also, with regard to a foreground other than the foreground #i of interest, using a distribution of the particle of the state of the foreground thereof, and the weighting of each particle.
However, in addition to the particle zi(1)t of the state zit of the foreground #i of interest, in the event of calculating the expected value λt of the observation image xt to be observed at the generating model using a distribution of the particle of the state of a foreground other than the foreground #i of interest and the weighting of each particle, this increases calculation cost.
Therefore, with the present embodiment, with regard to multiple foregrounds, independently, i.e., without considering a foreground other than the foreground #i of interest, assuming that the foreground #i of interest alone exists on the background, thereby realizing speed-up of calculation while sacrificing stringency with a range having no practical problem.
In the event of assuming that the foreground #i of interest alone exists on the background, the observation likelihood p(xit|zi(1)t) when the foreground #i of interest is in the state zi(1)t serving as a particle may be calculated in accordance with Expression (22).




  
    
      
        
          p
          ⁡
          
            (
            
              
                x
                t
                i
              
              |
              
                z
                t
                
                  i
                  ⁡
                  
                    (
                    l
                    )
                  
                
              
            
            )
          
        
        =
        
          K
          ⁢
          
            
          
          ⁢
          
            exp
            (
            
              -
              
                
                  
                    ∑
                    s
                  
                  ⁢
                  
                    
                      
                      
                        
                          
                            λ
                            t
                            i
                          
                          ⁡
                          
                            (
                            s
                            )
                          
                        
                        -
                        
                          
                            λ
                            t
                            ~
                          
                          ⁡
                          
                            (
                            s
                            )
                          
                        
                      
                      
                    
                    2
                  
                
                
                  σ
                  2
                
              
            
            )
          
        
      
    
    
      
        (
        22
        )
      
    
  




Note that λit(s) represents the pixel value of a point s on the world coordinates system O of an image where the expected value λt of the observation image xt to be observed at point-in-time t at the generating model θ in the event of assuming that the foreground #i of interest alone exists on the background.
Also, in Expression (22), λ ̃t(s) represents, in the same way as with the case of Expression (19), the pixel value of a point s on the world coordinates system O of the image λ ̃t actually observed at the point-in-time t.
Further, in Expression (22), K and σ2 are, in the same way as with the case of Expression (19), predetermined constants, and are determined beforehand.
The expected value λit=λi(zit, μi, μw) of the observation image xt at the point-in-time t to be observed at the generating model θ in the event of assuming that the foreground #i of interest alone exists on the background may be obtained in accordance with Expression (23).




  
    
      
        
          
            λ
            i
          
          ⁡
          
            (
            
              
                z
                t
                i
              
              ,
              
                μ
                i
              
              ,
              
                μ
                w
              
            
            )
          
        
        =
        
          
            
              λ
              t
              i
            
            ⁡
            
              (
              
                r
                t
                
                  ′
                  ⁢
                  
                    
                  
                  ⁢
                  i
                
              
              )
            
          
          =
          
            {
            
              
                
                  
                    
                      
                        (
                        
                          1
                          -
                          
                            
                              σ
                              i
                            
                            ⁡
                            
                              (
                              
                                r
                                i
                              
                              )
                            
                          
                        
                        )
                      
                      ⁢
                      
                        
                          τ
                          w
                        
                        ⁡
                        
                          (
                          
                            r
                            t
                            
                              ′
                              ⁢
                              
                                
                              
                              ⁢
                              i
                            
                          
                          )
                        
                      
                    
                    +
                    
                      
                        
                          σ
                          i
                        
                        ⁡
                        
                          (
                          
                            r
                            i
                          
                          )
                        
                      
                      ⁢
                      
                        τ
                        i
                      
                      ⁢
                      
                        
                      
                      ⁢
                      
                        (
                        
                          r
                          i
                        
                        )
                      
                    
                  
                
                
                  
                    
                      if
                      ⁢
                      
                        
                      
                      ⁢
                      
                        r
                        i
                      
                    
                    ∈
                    
                      μ
                      i
                    
                  
                
              
              
                
                  
                    
                      τ
                      w
                    
                    ⁡
                    
                      (
                      
                        r
                        t
                        
                          ′
                          ⁢
                          
                            
                          
                          ⁢
                          i
                        
                      
                      )
                    
                  
                
                
                  else
                
              
            
          
        
      
    
    
      
        (
        23
        )
      
    
  




Here, σi(ri) represents the pixel value of a point ri on the local coordinates system O′ of the foreground shape model σi, and τi(ri) represents the pixel value of a point ri on the local coordinates system O′ of the foreground texture model τi. Further, τw(r′it) represents the pixel value of a point r′it on the world coordinates system O of the foreground texture model τw.
Also, a point r′it obtained by adding a dash (′) and an index t of point-in-time to a point ri on the local coordinates system O′ represents a point (corresponding point) on the world coordinates system O corresponding to the point ri on the local coordinates system O′ at the point-in-time t, as described with Expression (16), and is obtained in accordance with Expression (17).
Section 3-4. Updating of Foreground Texture Model (M Step)
Description will be made regarding specific implementation of updating of the foreground texture model τi to be performed in the E step using the particle filter described in Section 2-3.
With the M step in the EM algorithm, the model parameter θ is updated to a new model parameter θnew from the current model parameter θold so as to maximize the expected value Q(θ, θold) of the complete data logarithmic likelihood ln(p(X, Z|θ) to be calculated with the posterior distribution p(Z|X, θold) of the latent variable Z in the E step.
In the event of having applied the particle filter to this M step, the model parameter θ is updated with the particle weighting wi(1)t obtained in the E step.
FIG. 14 is a diagram for describing updating of the foreground texture model τi using the particle weighting wi(1)t.
With updating of the foreground texture model τi, at the point-in-time t, in order to obtain the updated data τ ̃it for one point-in-time worth of the point-in-time t thereof, as illustrated in FIG. 14, assuming that the state of the foreground #i is in the state zi(1)t serving as a particle, an image of an area estimated as the foreground #i which is in the state zi(1)t thereof is trimmed from (the observation value of) the image λ ̃t actually observed at the point-in-time t, and projected on the local coordinates system O′ of the foreground #i.
Here, the image trimmed from the image (hereinafter, also referred to as actual image) λ ̃t actually observed at the point-in-time t and projected on the local coordinates system O′ of the foreground #i will also referred to as trimmed image τ ̃i(1)t.
The trimmed image τ ̃i(1)t is an image having the same size as with the foreground texture model τi (and foreground shape model σi), and is represented with Expression (24).

τt ̃i(1)(st′i(1))=λt ̃(s),st′i(1)εμi  (24)

Here, τ ̃i(1)t(s′i(1)t) represents the pixel value of a point s′i(1)t on the local coordinates system O′ of the trimmed image τ ̃i(1)t, and λ ̃t(s) represents the pixel value of a point s on the world coordinates system O of the actual image λ ̃t.
Also, a point s′i(1)t obtained by adding a dash (′), an index i of the foreground, and an index (1) of a particle, and an index t of point-in-time to the point s on the world coordinates system O represents a point (corresponding point) on the local coordinates system O′ of the foreground #i corresponding to the point s on the world coordinates system O.
The point s′i(1)t on the world coordinates system O corresponding to the point s on the local coordinates system O′ of the foreground #i is represented with Expression (25) by employing the mapping function R(A, B) described in Expression (17).

st′i=R(s−sti,−αti)+rci  (25)

Note that, in Expression (25), the index (1) of a particle is omitted.
According to Expression (25), in the event that the corresponding point s′i(1)t on the local coordinates system O′ of the point s on the world coordinates system O is a point on the foreground visible model μi (an image serving as the foreground texture model τi or the foreground shape model σi) at the point-in-time t (s′i(1)tεμi), the pixel value λ ̃t(s) of the point s of the actual image λ ̃t is employed as the pixel value τ ̃i(1)t(s′i(1)t) of the corresponding point s′i(1)t of the trimmed image τ ̃i(1)t.
With updating of the foreground texture model τi, trimmed images τ ̃i(1)t, τ ̃i(2)t, . . . , τ ̃i(L)t are obtained regarding the L particles zi(1)t, zi(2)t, . . . zi(L)t of the foreground #i. A weighed mean value of the L trimmed images τ ̃i(1)t to τ ̃i(L)t is obtained using the weighting wi(1)t to wi(L)t of the particles zi(1)t to zi(L)t as the updated data τ ̃it for one point-in-time worth of the point-in-time t regarding the foreground texture model τi in accordance with Expression (26).




  
    
      
        
          τ
          t
          
            ~
            i
          
        
        =
        
          
            ∑
            
              l
              =
              1
            
            L
          
          ⁢
          
            
              w
              t
              
                i
                ⁡
                
                  (
                  l
                  )
                
              
            
            ⁢
            
              τ
              t
              
                ~
                
                  i
                  ⁡
                  
                    (
                    l
                    )
                  
                
              
            
          
        
      
    
    
      
        (
        26
        )
      
    
  




Further, with updating of the foreground texture model τi, an average value (averaging for predetermined time worth) of the updated data τ ̃it, τ ̃it−1, . . . , τ ̃it−TFG+1, for the past TFG−1 point-in-time worth from the current point-in-time t is obtained as a new foreground texture model τi,new serving as a new model parameter θnew in accordance with Expression (27).




  
    
      
        
          τ
          
            i
            ,
            new
          
        
        =
        
          
            1
            
              T
              FG
            
          
          ⁢
          
            
              ∑
              
                t
                =
                
                  t
                  -
                  
                    T
                    FG
                  
                  +
                  1
                
              
              t
            
            ⁢
            
              τ
              t
              
                ~
                i
              
            
          
        
      
    
    
      
        (
        27
        )
      
    
  




Note that, though a weighed mean value of the trimmed images τ ̃i(1)t to τ ̃i(L)t each obtained regarding the L particles zi(1)t to zi(L)t has been obtained here as the updated data τ ̃it for one point-in-time worth of the point-in-time t, additionally, for example, the most-likely state zit is obtained from the L particles zi(1)t to zi(L)t as will be described later, and the trimmed image τ ̃it obtained regarding the most-likely state zit may be employed as the updated data τ ̃it for one point-in-time worth of the point-in-time t.
Section 3-5. Updating of Background Texture Model (M Step)
Description will be made regarding specific implementation of updating of the background texture model τw to be performed in the E step using the particle filter described in Section 2-3.
FIG. 15 is a diagram for describing updating of the background texture model τw.
With the updating processing of the background texture model τw, the updated data τ ̃wt for one point-in-time worth of the point-in-time t is obtained using the states zi(1)t to z1(L)t, z2(L)t to z2(L)t, . . . , zN(L)t to zN(L)t serving as all of the particles of all of the foreground models #1 to #N.
Specifically, with the updating processing of the background texture model τw, as illustrated in FIG. 15, of the actual image λ ̃t at the point-in-time t, it is performed on the state zi(1)t serving as each particle of the foreground model #i to replace the pixel value of an area estimated as a foreground with the current background texture model τw,old, and an image obtained as a result thereof is taken as the updated data τ ̃wt for one point-in-time worth of the point-in-time t.
Specifically, first, the foreground #1 is taken as the foreground of interest. Next, assuming that the state of the foreground #1 which is the foreground of interest is the state z1(1)t serving as a particle, there is obtained an image (hereinafter, also referred to as foreground removed image) τ ̃w,1(1) wherein image components of the foreground model #i (the foreground visible model μi) have been removed from an area estimated as the foreground #i which is in the state of zi(1)t, and also, the image components of the current foreground model (the current background texture model τw,old) have been added.
Of the foreground #1, the foreground removed image obtained regarding the state z1(1)t serving as a particle is represented with Expression (28).




  
    
      
        
          
            τ
            t
            
              
                ~
                w
              
              ,
              
                1
                ⁢
                
                  (
                  l
                  )
                
              
            
          
          ⁡
          
            (
            s
            )
          
        
        =
        
          {
          
            
              
                
                  
                    
                      λ
                      t
                      ~
                    
                    ⁡
                    
                      (
                      s
                      )
                    
                  
                  -
                  
                    
                      
                        σ
                        1
                      
                      ⁡
                      
                        (
                        
                          s
                          t
                          
                            ′1
                            ⁡
                            
                              (
                              l
                              )
                            
                          
                        
                        )
                      
                    
                    ⁢
                    
                      
                        τ
                        1
                      
                      ⁡
                      
                        (
                        
                          s
                          t
                          
                            ′1
                            ⁡
                            
                              (
                              l
                              )
                            
                          
                        
                        )
                      
                    
                  
                  +
                  
                    
                      (
                      
                        1
                        -
                        
                          
                            σ
                            1
                          
                          ⁡
                          
                            (
                            
                              s
                              t
                              
                                ′1
                                ⁡
                                
                                  (
                                  l
                                  )
                                
                              
                            
                            )
                          
                        
                      
                      )
                    
                    ⁢
                    
                      
                        τ
                        
                          w
                          ,
                          old
                        
                      
                      ⁡
                      
                        (
                        s
                        )
                      
                    
                  
                
              
              
                
                  
                    if
                    ⁢
                    
                      
                    
                    ⁢
                    
                      s
                      t
                      
                        ′1
                        ⁡
                        
                          (
                          l
                          )
                        
                      
                    
                  
                  ∈
                  
                    μ
                    1
                  
                
              
            
            
              
                
                  
                    λ
                    t
                    ~
                  
                  ⁡
                  
                    (
                    s
                    )
                  
                
              
              
                else
              
            
          
        
      
    
    
      
        (
        28
        )
      
    
  




According to Expression (28), in the event that, of the point s on the world coordinates system O, the corresponding point s′1(1)t on the local coordinates system O′ of the foreground #1 is a point on the foreground visible model μ1 (an image serving as the foreground texture model τ1 or the foreground shape model σ1) (s′1(1)tεμ1), as the pixel value τ ̃w,1(1)(s) the point s on the foreground removed image τ ̃w,1(1), there is employed a value λ ̃t(s)−σ1(s′1(1)t) τ1(s′1(1)t)+(1−σ1(s′1(1)t)τw,old(s) obtained by multiplying the pixel value τ1(s′1(1)t) of the corresponding point s′1(1)t on the foreground texture model τ1 by σ1(s′1(1)t) and subtracting this from the pixel value λ ̃t(s) of the point s on the actual image λ ̃t, and also multiplying the pixel value τw,old(s) of the point s on the current background texture model τw,old by (1−σ1(s′1(1)t)) and adding this to the subtraction result thereof.
Further, according to Expression (28), in the event that of the point s on the world coordinates system O, the corresponding point s′1(1)t on the local coordinates system O′ of the foreground #1 is not a point on the foreground visible model ν1, the pixel value λ ̃t(s) of the point s on the actual image λ ̃t is employed.
As a result thereof, as the foreground removed image τ ̃w,1(1) there is obtained an image such that the foreground #1 which is in the state z1(1)t serving as a particle is removed from the actual image λ ̃t, and background hidden in the removed foreground #1 thereof becomes visible.
With the updating processing of the background texture model τw, the foreground removed images τ ̃w,1(1) to τ ̃w,1(L are obtained regarding the L particles z1(1)t to z1(L)t of the foreground #1, respectively.
Next, a weighed mean value π ̃w,1 of the foreground removed images τ ̃w,1(1) to τ ̃w,1(L) is obtained as a final image where the foreground #1 is removed from the actual image λ ̃t (hereinafter, also referred to as the final removed image regarding the foreground #1) using the weighting w1(1)t to w1(L)t of the L particles z1(1)t to z1(L)t of the foreground #1.
The final removed image # ̃w,1 regarding the foreground #1 is represented with Expression (29).




  
    
      
        
          τ
          t
          
            
              ~
              w
            
            ,
            1
          
        
        =
        
          
            ∑
            
              l
              =
              1
            
            L
          
          ⁢
          
            
              w
              t
              
                1
                ⁢
                
                  (
                  l
                  )
                
              
            
            ⁢
            
              τ
              t
              
                
                  ~
                  w
                
                ,
                
                  1
                  ⁢
                  
                    (
                    l
                    )
                  
                
              
            
          
        
      
    
    
      
        (
        29
        )
      
    
  




With the updating processing of the background texture model τw, other foregrounds #2 to #N are sequentially taken as the foreground of interest to perform the same processing.
However, with regard to the foreground #i of the foreground #2 and thereafter, the final removed image τ ̃w,1−1 regarding the foreground #(i−1) which is the last foreground of interest is employed instead of the actual image λ ̃t.
Accordingly, of the foreground #i, the foreground removed image τ ̃w,i(1) obtained regarding the state zi(1)t serving as a particle is obtained in accordance with Expression (30) which is Expression (28) of which the actual image λ ̃t in Expression 28 has been replaced with the final removed image τ ̃w,i−1 regarding the foreground #(i−1).




  
    
      
        
          
            τ
            t
            
              
                ~
                w
              
              ,
              
                i
                ⁡
                
                  (
                  l
                  )
                
              
            
          
          ⁡
          
            (
            s
            )
          
        
        =
        
          {
          
            
              
                
                  
                    
                      τ
                      t
                      
                        
                          ~
                          w
                        
                        ,
                        
                          i
                          -
                          
                            1
                            ⁢
                            
                              (
                              l
                              )
                            
                          
                        
                      
                    
                    ⁡
                    
                      (
                      s
                      )
                    
                  
                  -
                  
                    
                      
                        σ
                        i
                      
                      ⁡
                      
                        (
                        
                          s
                          t
                          
                            ′
                            ⁢
                            
                              
                            
                            ⁢
                            
                              i
                              ⁡
                              
                                (
                                l
                                )
                              
                            
                          
                        
                        )
                      
                    
                    ⁢
                    
                      
                        τ
                        i
                      
                      ⁡
                      
                        (
                        
                          s
                          t
                          
                            ′
                            ⁢
                            
                              
                            
                            ⁢
                            
                              i
                              ⁡
                              
                                (
                                l
                                )
                              
                            
                          
                        
                        )
                      
                    
                  
                  +
                  
                    
                      (
                      
                        1
                        -
                        
                          
                            σ
                            i
                          
                          ⁡
                          
                            (
                            
                              s
                              t
                              
                                ′
                                ⁢
                                
                                  
                                
                                ⁢
                                
                                  i
                                  ⁡
                                  
                                    (
                                    l
                                    )
                                  
                                
                              
                            
                            )
                          
                        
                      
                      )
                    
                    ⁢
                    
                      
                        τ
                        
                          w
                          ,
                          old
                        
                      
                      ⁡
                      
                        (
                        s
                        )
                      
                    
                  
                
              
              
                
                  
                    if
                    ⁢
                    
                      
                    
                    ⁢
                    
                      s
                      t
                      
                        ′
                        ⁢
                        
                          
                        
                        ⁢
                        
                          i
                          ⁡
                          
                            (
                            l
                            )
                          
                        
                      
                    
                  
                  ∈
                  
                    μ
                    i
                  
                
              
            
            
              
                
                  
                    τ
                    t
                    
                      
                        ~
                        w
                      
                      ,
                      
                        i
                        -
                        
                          1
                          ⁢
                          
                            (
                            l
                            )
                          
                        
                      
                    
                  
                  ⁡
                  
                    (
                    s
                    )
                  
                
              
              
                else
              
            
          
        
      
    
    
      
        (
        30
        )
      
    
  




Next, the final removed image τ ̃w,i regarding the foreground #i is obtained in accordance with Expression (31) in same way as with Expression (29) using the foreground removed image τ ̃w,i(1).




  
    
      
        
          τ
          t
          
            
              ~
              w
            
            ,
            i
          
        
        =
        
          
            ∑
            
              l
              =
              1
            
            L
          
          ⁢
          
            
              w
              t
              
                i
                ⁡
                
                  (
                  l
                  )
                
              
            
            ⁢
            
              τ
              t
              
                
                  ~
                  w
                
                ,
                
                  i
                  ⁡
                  
                    (
                    l
                    )
                  
                
              
            
          
        
      
    
    
      
        (
        31
        )
      
    
  




The final removed image τ ̃w,N regarding the foreground #N is an image where all of N foregrounds #1 to #N are removed from the actual image λ ̃t, and also an area corresponding to the current background texture model τw,old is added to areas where the foregrounds #1 to #N have been included respectively. As indicated in Expression (32), this image τ ̃w,N is taken as updated data τ ̃wt for one point-in-time worth of the point-in-time t regarding the background texture model τw.

τt ̃w=τt ̃w,N  (32)

FIG. 15 illustrates a scene of calculation of the updated data τ ̃wt for one point-in-time worth of the point-in-time t regarding the background texture mode τw in the event that the number of foregrounds N is two.
Thereafter, with updating of the background texture model τw, an average value (averaging for predetermined time worth) of the updated data τ ̃wt, τ ̃wt−1, . . . , τ ̃wt−TBG+1, for the past TBG−1 point-in-time worth from the current point-in-time t is obtained as a new background texture model τw,new serving as a new model parameter θnew in accordance with Expression (33).




  
    
      
        
          τ
          
            i
            ,
            new
          
        
        =
        
          
            1
            
              T
              FG
            
          
          ⁢
          
            
              ∑
              
                t
                =
                
                  t
                  -
                  
                    T
                    FG
                  
                  +
                  1
                
              
              t
            
            ⁢
            
              τ
              t
              
                ~
                i
              
            
          
        
      
    
    
      
        (
        33
        )
      
    
  




With updating of the background texture model τw, with regard to the foreground #i of the foreground #2 and thereafter, the foreground removed image τ ̃w,i(1) is obtained using the final removed image τ ̃w,i−1 instead of the actual image λ ̃t regarding the foreground #(i−1) which had been the foreground of interest immediately before, and accordingly, exclusive control is performed on multiple foreground models so as to suppress simultaneous modeling (capturing) of the same foreground.
Section 3-6. Updating of Foreground Shape Model (M Step)
With updating of the foreground shape model σi, evaluation regarding which area belongs to the foreground #i, and which area does not belong to the foreground #i is performed, for example, in increments of pixels regarding an image serving as the foreground texture model τi (new foreground texture model τi,new) after updating.
Specifically, with updating of the foreground shape model σi, regarding each of the pixels of an image serving as the foreground texture model τi after updating, evaluation regarding whether or not the pixel thereof is a pixel of the foreground #i is performed.
With evaluation regarding whether or not a pixel of the image serving as the foreground texture model τi is a pixel of the foreground #i, regarding each pixel of the image serving as the foreground texture model τi, likelihood that the pixel thereof is a pixel of the foreground #i (hereinafter, also referred to as foreground likelihood) is expected.
As for foreground likelihood, for example, observation likelihood may be employed wherein the image serving as the foreground texture model τi is observed as the updated data τ ̃it for one point-in-time worth of the point-in-time t regarding the foreground texture model τi in Expression (26) serving as an expected value of the foreground #i.
In this case, the observation likelihood serving as the foreground likelihood may be obtained in accordance with Expression (34), for example.




  
    
      
        
          
            L
            ⁡
            
              (
              
                FG
                |
                
                  
                    τ
                    i
                  
                  ⁡
                  
                    (
                    
                      r
                      i
                    
                    )
                  
                
              
              )
            
          
          =
          
            K
            ⁢
            
              
            
            ⁢
            
              exp
              (
              
                -
                
                  
                    
                      
                      
                        
                          
                            τ
                            i
                          
                          ⁡
                          
                            (
                            
                              r
                              i
                            
                            )
                          
                        
                        -
                        
                          
                            τ
                            t
                            
                              ~
                              i
                            
                          
                          ⁡
                          
                            (
                            
                              r
                              i
                            
                            )
                          
                        
                      
                      
                    
                    2
                  
                  
                    σ
                    2
                  
                
              
              )
            
          
        
        ,
        
          
            r
            i
          
          ∈
          
            μ
            i
          
        
      
    
    
      
        (
        34
        )
      
    
  




Here, τi(ri) represents the pixel value of a point ri on the local coordinates system O′ of the foreground texture model τi, and τ ̃it (ri) represents the pixel value of the point ri on the local coordinates system O′ serving as the updated data τ ̃it for one point-in-time worth of the point-in-time t regarding the foreground texture model τi.
Further, L(FG|τi(ri)) represents likelihood (foreground likelihood) that the pixel value τi(ri) of the point ri of the foreground texture model τ1 will be a pixel of the foreground #i.
Also, in Expression (34), K and σ2 are predetermined constants, and are determined beforehand.
With evaluation regarding whether or not a pixel of the image serving as the foreground texture model τi is a pixel of the foreground #i, in addition to the foreground likelihood L(FG|τi(ri)) as described above, regarding each pixel of the image serving as foreground texture model τi, there is expected likelihood that the pixel thereof will be a pixel of the foreground (hereinafter, also referred to as background likelihood).
As for the background likelihood, for example, observation likelihood may be employed wherein the image serving as the foreground texture model τi will be observed as the final removed image τ ̃w,it regarding the foreground #i in Expression (31) serving as an expected value of the background.
In this case, the observation likelihood serving as the background likelihood may be obtained in accordance with Expression (35), for example.




  
    
      
        
          
            L
            ⁡
            
              (
              
                BG
                |
                
                  
                    τ
                    i
                  
                  ⁡
                  
                    (
                    
                      r
                      i
                    
                    )
                  
                
              
              )
            
          
          =
          
            K
            ⁢
            
              
            
            ⁢
            
              exp
              (
              
                -
                
                  
                    
                      
                      
                        
                          
                            τ
                            i
                          
                          ⁡
                          
                            (
                            
                              r
                              i
                            
                            )
                          
                        
                        -
                        
                          
                            τ
                            t
                            
                              
                                
                                  ~
                                  ′
                                
                                ⁢
                                
                                  
                                
                                ⁢
                                w
                              
                              ,
                              i
                            
                          
                          ⁡
                          
                            (
                            
                              r
                              i
                            
                            )
                          
                        
                      
                      
                    
                    2
                  
                  
                    σ
                    2
                  
                
              
              )
            
          
        
        ,
        
          
            r
            i
          
          ∈
          
            μ
            i
          
        
      
    
    
      
        (
        35
        )
      
    
  




Here, τ ̃′w,it obtained by adding a dash (′) to τ ̃w,it representing the final removed image regarding the foreground #i represents (an expected value of) a projection image where the area of the foreground visible model μi is trimmed from the final removed image τ ̃w,it regarding the foreground #i, and is projected on the local coordinates system O′ of the foreground #i.
In Expression (35), τ ̃w,it(ri) represents the pixel value of the point ri on the local coordinates system O′ of the projection image τ ̃w,it.
Further, L(BG|τi(ri)) represents likelihood (background likelihood) that the pixel value τi(ri) of the point ri of the foreground texture model τi will be a pixel of the background.
Also, in Expression (35), K and σ2 are predetermined constants, and are determined beforehand.
The projection image τ ̃w,i may be obtained in accordance with Expressions (36) and (37) using the foreground removed image τ ̃w,i(1)t in Expression (30) (Expression (28)) to be used for obtaining the final removed image τ ̃w,it regarding the foreground #i in Expression (31).




  
    
      
        
          
            
              τ
              t
              
                
                  
                    ~
                    ′
                  
                  ⁢
                  
                    
                  
                  ⁢
                  w
                
                ,
                
                  i
                  ⁡
                  
                    (
                    l
                    )
                  
                
              
            
            ⁡
            
              (
              
                s
                t
                
                  ′
                  ⁢
                  
                    
                  
                  ⁢
                  
                    i
                    ⁡
                    
                      (
                      l
                      )
                    
                  
                
              
              )
            
          
          =
          
            
              τ
              t
              
                
                  ~
                  w
                
                ,
                
                  i
                  ⁡
                  
                    (
                    l
                    )
                  
                
              
            
            ⁡
            
              (
              s
              )
            
          
        
        ,
        
          
            s
            t
            
              ′
              ⁢
              
                
              
              ⁢
              
                i
                ⁡
                
                  (
                  l
                  )
                
              
            
          
          ∈
          
            μ
            i
          
        
      
    
    
      
        (
        36
        )
      
    
  
  
    
      
        
          τ
          t
          
            
              
                ~
                ′
              
              ⁢
              
                
              
              ⁢
              w
            
            ,
            i
          
        
        =
        
          
            ∑
            
              l
              =
              1
            
            L
          
          ⁢
          
            
              w
              t
              
                i
                ⁡
                
                  (
                  l
                  )
                
              
            
            ⁢
            
              τ
              t
              
                
                  
                    ~
                    ′
                  
                  ⁢
                  
                    
                  
                  ⁢
                  w
                
                ,
                
                  i
                  ⁡
                  
                    (
                    l
                    )
                  
                
              
            
          
        
      
    
    
      
        (
        37
        )
      
    
  




Here, τ ̃w,i(1)t represents a projection image in the event of assuming that the foreground #i is in the state zi(1)t serving as a particle, and τ ̃w,i(1)t(s′i(1)t) represents the pixel value of the corresponding point s′i(1)t of the projection image τ ̃w,i(1)t corresponding to the particle zi(1)t.
According to Expression (36), in the event that of the point s on the world coordinates system O, the corresponding point s′i(1)t on the local coordinates system O′ of the foreground #i is a point on the foreground visible model μi, the pixel value τ ̃w,i(1)t(s) of the point s of the foreground removed image τ ̃w,i(1)t(s) is employed as the pixel value τ ̃w,i(1)t(s′i(1)t) of the corresponding point s′i(1)t of the projection image τ ̃w,i(1)t corresponding to the particle zi(1)t.
According to Expression (37), a weighed mean value of the projection image τ ̃w,i(1)t corresponding to each particle zi(1)t using the weighting wi(1)t of a particle is obtained as a projection image τ ̃w,it.
With updating of the foreground shape model σi, each of the pixels of the image serving as the foreground texture model τi, probability that the pixel thereof will be a pixel of the foreground #i is obtained in accordance with Expression (38) using the foreground likelihood L(FG|τi(ri)) in Expression (34), and the background likelihood L(BG|τi(ri)) in Expression (35).




  
    
      
        
          
            p
            ⁡
            
              (
              
                FG
                |
                
                  
                    τ
                    i
                  
                  ⁡
                  
                    (
                    
                      r
                      i
                    
                    )
                  
                
              
              )
            
          
          =
          
            
              L
              ⁡
              
                (
                
                  FG
                  |
                  
                    
                      τ
                      i
                    
                    ⁡
                    
                      (
                      
                        r
                        i
                      
                      )
                    
                  
                
                )
              
            
            
              
                L
                ⁡
                
                  (
                  
                    FG
                    |
                    
                      
                        τ
                        i
                      
                      ⁡
                      
                        (
                        
                          r
                          i
                        
                        )
                      
                    
                  
                  )
                
              
              +
              
                L
                ⁡
                
                  (
                  
                    BG
                    |
                    
                      
                        τ
                        i
                      
                      ⁡
                      
                        (
                        
                          r
                          i
                        
                        )
                      
                    
                  
                  )
                
              
            
          
        
        ,
        
          
            r
            i
          
          ∈
          
            μ
            i
          
        
      
    
    
      
        (
        38
        )
      
    
  




Here, p(FG|τi(ri)) represents probability that of the foreground shape model σi, the point (pixel) ri on the local coordinates system O′ of the foreground #i will be a pixel of the foreground #i. With updating of the foreground shape model σi, the pixel value σi(ri) of the point ri is updated to probability p(FG|τi(ri)).
Chapter 4. Updating of Motion Model
Kinetics Parameter
When observing the observation value xt+1 at the point-in-time t+1, series Xt={x1, x2, . . . , xt} of the observation value are observed until the point-in-time t, and the posterior distribution p(zt+1|Xt) which is in the state Zt+1 may be obtained in accordance with Expression (15) at the point-in-time t+1.
A most-likely state Zit+1 having the highest possibility that the foreground #i has been at the point-in-time t+1 may be obtained in accordance with Expression (39) since the posterior distribution p(zit+1|Xt) at the point-in-time t+1 is in the maximum state.




  
    
      
        
          z
          
            t
            +
            1
          
          i
        
        =
        
          
            
              max
              z
            
            ⁢
            
              p
              ⁡
              
                (
                
                  
                    z
                    
                      t
                      +
                      1
                    
                    i
                  
                  |
                  
                    X
                    t
                  
                
                )
              
            
          
          ≅
          
            
              max
              z
            
            ⁢
            
              
                ∑
                
                  l
                  =
                  1
                
                L
              
              ⁢
              
                
                  w
                  t
                  
                    i
                    ⁡
                    
                      (
                      l
                      )
                    
                  
                
                ⁢
                
                  p
                  ⁡
                  
                    (
                    
                      
                        z
                        
                          t
                          +
                          1
                        
                        i
                      
                      |
                      
                        z
                        t
                        
                          i
                          ⁡
                          
                            (
                            l
                            )
                          
                        
                      
                    
                    )
                  
                
              
            
          
        
      
    
    
      
        (
        39
        )
      
    
  




With updating of the motion model, the most-likely state Zit of the foreground #i of each point-in-time t is successively obtained, and most-likely estimation is performed using series (most-likely series) Zit={zi1, zi2, . . . , zit} of the most-likely state Zit thereof as an observation value that the agent is allowed to observe, and accordingly, kinetics parameters ric, mi, and Iic of rigid motion in Expression (4) serving as the motion model ai of the foreground #i are updated.
With most-likely estimation, from a point-in-time going back in the past from the current point-in-time t by a certain time to a state until the current point-in-time are estimated, and error for the most-likely series Zit of the state series obtained as a result thereof is employed as observation likelihood that the state zit will be observed at the point-in-time t in the motion model ai.
Specifically, with the present technology, the most-likely state zit−TDYN at point-in-time t-TDYN going back a certain time TDYN from the current point-in-time t is taken as an initial value, and state series Z^it−TDYN+1,t={z^it−TDYN+1, z^it−TDYN+2, . . . , z^it} from the point-in-time t−TDYN+1 to the current point-in-time t of the foreground #i are estimated in accordance with a time development equation in Expression (40), using the motion model ai, and an action series Uit−1={uit−TDYN, uit−TDYN+1, . . . , uit−1} of the foreground #i from point-in-time t−TDYN to point-in-time t−1.

zt+1i=d(zti,ai)  (40)

As for the time development equation in Expression (40), the rigid motion equations in Expressions (41) and (42) may be employed, for example.




  
    
      
        
          
            m
            i
          
          ⁢
          
            d
            
              d
              t
            
          
          ⁢
          
            s
            t
            
              ·
              i
            
          
        
        =
        
          F
          i
        
      
    
    
      
        (
        41
        )
      
    
  
  
    
      
        
          
            I
            c
            i
          
          ⁢
          
            d
            
              d
              t
            
          
          ⁢
          
            q
            t
            
              ·
              i
            
          
        
        =
        
          
            (
            
              
                r
                F
                i
              
              -
              
                r
                c
                i
              
            
            )
          
          ×
          
            F
            i
          
        
      
    
    
      
        (
        42
        )
      
    
  




As for the motion equations in Expressions (41) and (42), numerical computation is performed, for example, using the Euler's method or Runge Kutta methods, and accordingly, there may be obtained the state series Z^it−TDYN+1,t={z^it−TDYN+1, z^it−TDYN+2, . . . , z^it} of the foreground #i to be moved in accordance with the motion equations in Expressions (41) and (42).
With the present technology, a value corresponding to error between the most-likely series Zit={zit−TDYN+1, zit−TDYN+2, . . . , zit} from the point-in-time t−TDYN+1 in time from the current point-in-time t by time TDYN−1, to the current point-in-time t, and the state series Z^it−TDYN+1,t={z^id−TDYN+1, z^it−TDYN+2, . . . , z^it} of the foreground #i obtained in accordance with the motion equations in Expressions (41) and (42) is obtained as observation likelihood (observation likelihood that the state zit will be observed at the point-in-time t as the state of the foreground #i of the motion model ai) p(zit|ai) that the state zit will be observed at the point-in-time t at the motion model ai.
Specifically, the observation likelihood p(zit|ai) is obtained in accordance with Expression (43).




  
    
      
        
          p
          ⁡
          
            (
            
              
                z
                t
                i
              
              |
              
                a
                i
              
            
            )
          
        
        =
        
          K
          ⁢
          
            
          
          ⁢
          
            exp
            (
            
              -
              
                
                  
                    1
                    
                      T
                      DYN
                    
                  
                  ⁢
                  
                    
                      ∑
                      
                        t
                        =
                        
                          t
                          -
                          
                            T
                            DYN
                          
                          +
                          1
                        
                      
                      
                        t
                        =
                        t
                      
                    
                    ⁢
                    
                      
                        
                        
                          
                            z
                            t
                            
                              
                                
                              
                              ^
                            
                          
                          -
                          
                            z
                            t
                            i
                          
                        
                        
                      
                      2
                    
                  
                
                
                  σ
                  2
                
              
            
            )
          
        
      
    
    
      
        (
        43
        )
      
    
  




Note that, in Expression (43), K and σ2 are predetermined constants and are determined d beforehand.
With the present technology, the motion model ai is employed as a particle, and updating of the motion model ai is successively performed using the particle filter in parallel with updating of the model parameters of the foreground model and background model.
A basic calculation method for the particle filter in the event of employing the motion model ai as a particle is as described in Chapter 2.
However, the weighting wi(1)t of the particle of the motion model ai is obtained in the same way as with Expression (14) using the observation likelihood p(zit|ai) to be obtained in accordance with Expression (43).
Also, with the present technology, assuming that probability that the motion model ai(1)t serving as a particle at the point-in-time t will change to the motion model ait+1 at the next point-in-time t+1, i.e., the transition probability p(ait+1|ai(1)t) of the motion model ai(1)t serving as a particle follows a Gauss distribution (normal distribution), the transition model of the motion model ai(1)t serving as a particle is represented with Expression (44), for example.




  
    
      
        
          a
          
            t
            +
            1
          
          i
        
        =
        
          
            [
            
              
                
                  
                    r
                    
                      c
                      ,
                      
                        t
                        +
                        1
                      
                    
                    i
                  
                
              
              
                
                  
                    m
                    
                      t
                      +
                      1
                    
                    i
                  
                
              
              
                
                  
                    I
                    
                      c
                      ,
                      
                        t
                        +
                        1
                      
                    
                    i
                  
                
              
            
            ]
          
          =
          
            
              [
              
                
                  
                    
                      r
                      
                        c
                        ,
                        t
                      
                      i
                    
                  
                
                
                  
                    
                      m
                      t
                      i
                    
                  
                
                
                  
                    
                      I
                      
                        c
                        ,
                        t
                      
                      i
                    
                  
                
              
              ]
            
            +
            
              [
              
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            Σ
                            
                              r
                              c
                            
                          
                        
                        )
                      
                    
                  
                
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            σ
                            m
                            2
                          
                        
                        )
                      
                    
                  
                
                
                  
                    
                      N
                      ⁡
                      
                        (
                        
                          0
                          ,
                          
                            Σ
                            
                              I
                              c
                            
                          
                        
                        )
                      
                    
                  
                
              
              ]
            
          
        
      
    
    
      
        (
        44
        )
      
    
  




Here, as described with Expression (20), N (A, B) represents a Gauss distribution wherein the average vector (average value) is A, and the variance covariance matrix (covariance) is B.
In Expression (44), any of Gauss distributions N (0, Σrc), N (0, σ2m), and N (0, ΣIc) is a Gauss distribution where the average vector is 0. Also, the variance covariance matrices (covariance) Σrc, σ2m, and ΣIc are determined beforehand.
According to the transition model in Expression (44), assuming that the motion model serving as a particle makes the transition with stochastic fluctuation in accordance with the Gauss distribution of a predetermined variance covariance matrix where the average vector is 0, the motion model ait+1 at the point-in-time t+1 which is a transition destination is obtained (predicted).
Also, with the present technology, with regard to the motion model ai, a weighed mean value between the motion model ai(1)t serving as a particle at the point-in-time t and the weighting wi(1)t thereof is obtained in accordance with Expression (45), and the motion model ait+1 at the point-in-time t+1 is updated to the weighed mean value.




  
    
      
        
          a
          
            t
            +
            1
          
          i
        
        =
        
          
            ∑
            
              l
              =
              1
            
            L
          
          ⁢
          
            
              w
              t
              
                i
                ⁡
                
                  (
                  l
                  )
                
              
            
            ⁢
            
              a
              
                i
                ⁡
                
                  (
                  l
                  )
                
              
            
          
        
      
    
    
      
        (
        45
        )
      
    
  




Chapter 5. Object Operations Using Estimated Model
FIG. 16 is a diagram for describing object operations by the agent.
In the event of having obtained the foreground model (foreground visible model μi), background model (background visible model μw), and the motion model ai of the foreground #i, the agent may serve as a manipulator which operates an object within an environment using these foreground model, background model, and motion model ai.
In FIG. 16, a L-shaped object is allocated as an object to be operated within an environment serving as a two-dimensional plane, and the object to be operated moves by the hand of the agent illustrated with a circle being in contact therewith.
Here, in FIG. 16, a L-shape illustrated with a solid line represents the current state of the object to be operated, and a L-shape illustrated with a dotted line represents the target state of the object to be operated.
Now, let us say that the object to be operated moves in accordance with a rigid motion equation within an environment, but unless external force is applied thereto, motion energy is lost over time due to friction with the environment, and speed decreases. That is to say, let us say that the object to be operated immediately stops if application of external force is stopped without applying force to stop.
Also, the object to be operated itself does not autonomously move, and moves only by the circular hand being in contact therewith in the same way as with a case where a human being transports an object by hand.
Further, let us say that the agent may optionally specify a position and speed regarding the hand.
FIG. 17 is a diagram illustrating the current state and target state of the object to be operated.
In FIG. 17, sobjt represents the position (current position) of center-of-gravity of the object at the current point-in-time t on the world coordinates system O, and qobjt represents the attitude (current attitude) of target of the object on the world coordinates system.
Also, in FIG. 17, stgt represents the position (target position) of center-of-gravity of target of the object on the world coordinates system O, and qtgt represents the attitude (target attitude) of target of the object on the world coordinates system O.
Rigid motion may be divide into translation motion and rotation motion. The present technology makes the object to be operated perform translation motion and also perform rotation motion as appropriate, thereby changing the object to the target state.
Specifically, with the present technology, the agent first makes the object to be operated perform translation motion, and makes the object perform rotation motion after error of the position of the object for the target state becomes sufficiently small. In the event that error of the position of the object for the target state becomes great during rotation motion, the agent makes the object perform translation motion again, and hereinafter, the same processing is performed.
On the other hand, in the event that error of the position and attitude of the object for the target state becomes sufficiently small during rotation motion, the agent ends object operations assuming that the object has changed to the target state.
Note that, with the above-mentioned object operations, though description has been made wherein translation motion is preferentially performed, and then rotation motion is performed, and additionally, for example, an arrangement may be made wherein rotation motion is preferentially performed, and then translation motion is performed.
FIG. 18 is a diagram for describing translation motion of the object.
At the time of making the object perform translation motion, in order to translate the object without rotating the object as much as possible, the agent applies force to the object which is a rigid body so as to suppress occurrence of torque.
FIG. 18 illustrates, as described above, in the event of applying force so as to suppress the object from occurrence of torque, a method for deciding a point of load of the force thereof, and a method for deciding a vector of the force thereof (force vector).
First, the point of load of the force is determined to be an intersection between a straight line L1 which connects a position stgt of a target state of the object and the current position (center-of-gravity) sobjt, and edges of the current object.
Here, the edges of the object may be obtained, for example, by performing image processing to detect edges on a foreground shape model of the foreground corresponding to the object thereof.
Also, in FIG. 18, there are two points scontact,0 and scontact,1 as intersections between the straight line L1 and the edges of the current object.
In this manner, in the event that there are multiple points as intersections between the straight line L1 and the edges of the current object, for example, when operating the object by pressing the object by hand, of multiple points serving as intersections between the straight line L1 and the edges of the current object, a point farthest from the position stgt of the target state of the object is determined to be a point of load of the force.
With regard to the two points scontact,0 and scontact,1 in FIG. 18, the point scontact,0 is a point farthest from the position stgt of the target state of the object, and accordingly determined to be a point of load of the force.
Note that, when operating the object by dragging the object by hand, of the multiple points serving as intersections between the straight line L1 and the edges of the current object, a point closest from the position stgt of the target state of the object is determined to be a point of load of the force.
The direction of a force vector Fcontrolt to be applied to the object at the point-in-time t is determined to be a direction advancing from the point of load scontact,0 to the position stgt in the target state.
Further, the force vector Fcontrolt is determined, for example, in accordance with Expression (46) so that the greater distance |stgt−sobjt| from the current position sobjt of the object to the position stgt in the target state is, the greater the force vector Fcontrolt becomes, and also, the greater the current speed s•objt of the object is, the smaller the force vector Fcontrolt becomes.

Ftcontrol=K0(stgt−stobj)−k1st•obj  (46)

Here, in Expression (46), K0 and K1 are predetermined constants, and are determined beforehand.
FIG. 19 is a diagram for describing rotation motion of the object.
At the time of making the object perform rotation motion, in order to rotate the object without translating the object as much as possible, the agent applies force to the object which is a rigid body so as to generate torque around center-of-gravity.
FIG. 19 illustrates, as described above, in the event of applying force so as to generate torque around center-of-gravity, a method for deciding a point of load of the force thereof, and a method for deciding a vector of the force thereof (force vector).
First, the point of load of the force is determined to be an intersection between one or more straight lines which pass through the position (center-of-gravity) sobjt of the current state of the object and the edges of the current object.
Here, in FIG. 19, three straight lines L1, L2, and L3 exist as one or more straight lines which pass through the position sobjt of the current state of the object.
Also, there are two points scontact,0 and scontact,3 as intersections between the straight line L1 and the edges of the current object, two points scontact,2 and scontact,4 as intersections between the straight line L2 and the edges of the current object, and two points scontact,2 and scontact,5 as intersections between the straight line L3 and the edges of the current object.
Now, for example, as with the case of translation motion described with FIG. 18, if we say that the object is made to perform rotation motion by pressing the object by hand, in the event that there are multiple points as intersections between a straight line passing through the current position sobjt of the object and the edges of the current object, a point where a direction of torque that occurs when applying force so as to press the edges of the object agrees with a direction intended to rotate the object is selected from the multiple points as a point-of-load candidate.
Here, as for the direction intended to rotate the object, for example, a direction having a small rotational angle when rotating the object to attitude agreeing with the target state may be employed of rotational directions around the center-of-gravity of the object. In FIG. 19, the counterclockwise direction is the direction intended to rotate the object.
After a point-of-load candidate is selected, of point-of-load candidates thereof, a candidate farthest from the current position sobjt of the object is determined to be a point of load.
In FIG. 19, the point scontact,0 is determined to be the point of load of the force.
The direction of the force vector Fcontrolt to be applied to the object at the point-in-time t is determined to be a direction to rotate the object in a direction intended to rotate the object, orthogonal to a straight line passing through the current position sobjt of the object and the point of load.
In FIG. 19, a direction to rotate the object in the counterclockwise direction, orthogonal to the straight line L1 passing through the current position sobjt of the object and the point of load scontact,0 is determined to be the direction of the force vector Fcontrolt.
The rotation motion is determined, for example, in accordance with Expression (47) so that according to the force vector Fcontrolt, the greater a rotational angle |qtgt−qobjt| from the current attitude qobjt of the object to the attitude qtgt of the target state, the greater torque Tcontrolt to rotate the object around the center-of-gravity sobjt becomes, and also, the greater the current angular velocity q•objt of the object is, the smaller the torque Tcontrolt becomes.

ttcontrol=K2(qtgt−qtobj)−K3qt•obj  (47)

Here, in Expression (47), K2 and K3 are predetermined constants, and are determined beforehand.
The torque Tcontrolt is an exterior product r between a distance vector r representing distance from the center-of-gravity sobjt of the object to the point of load and the force vector Fcontrolt×Fcontrolt, and accordingly, the torque Tcontrolt is determined, whereby the force vector Fcontrolt to generate the torque Tcontrolt thereof may also be determined.
After determining the point of load of the force, and the force vector of the force thereof, the agent moves the hand to a predetermined initial position, and moves the hand to the point of load of the force of the object to be operated so as to apply the force vector.
FIG. 20 is a diagram for describing the initial position of the hand.
The agent determines a point smanip0 sufficiently far from the point of load scontact to be the initial position of the hand (the position of the hand with the point-in-time t=0), which passes through the force point-of-load scontact, on the straight line L1 parallel to the force vector Fcontrolt, in the opposite direction of the direction of the force vector Fcontrolt with the force point-of-load scontact as a base point, and also not interfering with the object to be operated.
The agent then moves the hand to the initial position smanip0, following which moves the hand at velocity s•manipt corresponding to the force vector Fcontrolt.
The velocity (velocity vector) s•manipt corresponding to the force vector Fcontrolt is obtained, for example, in accordance with Expression (48).

st•manip=K4Ftcontrol  (48)

Here, in Expression (48), K4 is a predetermined constant, and is determined beforehand.
Note that the agent moves the hand to the initial position smanip0 when starting translation motion and when starting rotation motion.
Also, the agent consecutively moves the hand at velocity s•manipt in Expression (48) obtained using the force vector Fcontrolt between up to starting rotation motion after starting translation motion, and between up to starting translation motion after starting rotation motion.
Embodiment of Agent to Which Present Technology Has Been Applied
FIG. 21 is a block diagram illustrating a configuration example of an embodiment of the agent to which the present technology has been applied.
In FIG. 21, the agent includes an actuator control unit 11, a sensor 12, a foreground state estimating unit 13, a visible model updating unit 14, a visible model storage unit 15, a most-likely state calculating unit 16, a most-likely state series storage unit 17, a motion model estimating unit 18, a motion model updating unit 19, a motion model storage unit 20, a target generating unit 21, an action generating unit 22, and an action series storage unit 23.
In FIG. 21, the foreground state estimating unit 13 to motion model storage unit 20 make up a learning device which learns the foreground visible model μi, background visible model μw, and motion model ai.
Action uit (action that the foreground #i is made to perform or action that the foreground #i performs at point-in-time t) of the foreground #i is supplied from the action generating unit 22 to the actuator control unit 11.
The actuator control unit 11 controls the actuator which drives the hand or the like not illustrated of the agent in accordance with the action uit from the action generating unit 22, and thus, the hand is moved, for example.
The sensor 12 is a camera or the like, shoots an image with a predetermined frame rate, and supplies an observation value of the image thereof (an observation value of the image that the agent may observe) (actual image) λ ̃t to the foreground state estimating unit 13 and visible model updating unit 14.
Note that the sensor 12 includes, in addition to the camera, for example, a pressure sensor or the like mounted on the hand. According to the pressure sensor mounted on the hand, action of the object may be acquired (observed) by reaction at the time of the hand coming into contact with the object.
The action uit of the foreground #i corresponding to the object to be acquired by the sensor 12 is supplied to the action series storage unit 23.
In addition to supply of the observation value (actual image) λ ̃t of an image from the sensor 12, the foreground visible model μi and background visible model μw stored in the visible model storage unit 15 are supplied to the foreground state estimating unit 13.
The foreground state estimating unit 13 estimates the state zit of the foreground #i using the actual image λ ̃t from the sensor 12, and the foreground visible model μi and background visible model μw stored in the visible model storage unit 15, and supplies the particle zi(1)t and weighting wi(1)t of the state zit to the visible model updating unit 14 and most-likely state calculating unit 16 as results of the estimation thereof.
Note that, with the foreground state estimating unit 13, as appropriate, the state zit of the foreground #i may be estimated using the motion model ai stored in the motion model storage unit 20 and the action uit stored in the action series storage unit 23.
The visible model updating unit 14 updates the foreground visible model μi and background visible model μw stored in the visible model storage unit 15 using the actual image λ ̃t from the sensor 12, and the particle zi(1)t and weighting wi(1)t of the state zit serving as estimation results of the state zit of the foreground #i supplied from the foreground state estimating unit 13.
Specifically, the visible model updating unit 14 updates a foreground texture model τi of the foreground visible model μi in accordance with Expressions (24) to (27), and updates a foreground shape model σi of the foreground visible model μi in accordance with Expressions (34) to (38).
Also, the visible model updating unit 14 updates the background visible model μw (background texture model τw) in accordance with Expressions (28) to (33).
The visible model storage unit 15 stores the foreground visible model μi and background visible model μw.
The most-likely state calculating unit 16 obtains the most-likely state zit using the particle zi(1)t and weighting wi(1)t of the state zit serving as estimation results of the state zit of the foreground #i supplied from the foreground state estimating unit 13 in accordance with Expression (39), and supplies this to the most-likely state series storage unit 17.
The most-likely state series storage unit 17 sequentially stores the most-likely state zit from the most-likely state calculating unit 16. With the most-likely state series storage unit 17, the most-likely state zit from the most-likely state calculating unit 16 is sequentially stored, and consequently, most-likely series Zit={zi1, zi2, . . . , zit} are stored.
The motion model estimating unit 18 estimates the motion model ai using the most-likely series Zit={zit−TDYN+1, zit−TDYN+2, . . . , zit} for the past TDYN−1 time worth stored in the most-likely state series storage unit 17, the most-likely state zit−TDYN at the point-in-time t-TDYN, and action series Uit−1={uit−TDYN, uit−TDYN+1, . . . , uit−1} stored in the action series storage unit 23, and supplies the particle ai(1)t and weighting wi(1)t of the motion model ai serving as estimation results to the motion model updating unit 19.
The motion model updating unit 19 updates the motion model ai stored in the motion model storage unit 20 using the particle ai(1)t and weighting wi(1)t of the motion model ai serving as estimation results of the motion model ai supplied from the motion model estimating unit 18.
Specifically, the motion model updating unit 19 updates the motion model ai in accordance with Expression (45).
The motion model storage unit 20 stores the motion model ai.
The target generating unit 21 generates a position stgt and attitude qtgt serving as the target state of the foreground, and supplies these to the action generating unit 22. Note that the target generating unit 21 may generate a target state, for example, in accordance with a user's operations, an optional algorithm to set a target state, or the like.
The action generating unit 22 generates action uit to change the states of the foreground #i to the target state supplied from the target generating unit 21 using the foreground visible model μi stored in the visible model storage unit 15, the most-likely state zit stored in the most-likely state series storage unit 17, and the motion model ai stored in the motion model storage unit 20, and supplies this to the actuator control unit 11 and action series storage unit 23.
The action series storage unit 23 stores the action uit supplied from the sensor 12, and the action uit supplied from the action generating unit 22.
FIG. 22 is a block diagram illustrating a configuration example of the foreground state estimating unit 13 in FIG. 21.
The foreground state estimating unit 13 includes a visible model likelihood calculating unit 41, a state particle updating unit 42, a state particle storage unit 43, and an estimated observation value generating unit 44.
With the visible model likelihood calculating unit 41, the observation value (actual image) λ ̃t of an image is supplied from the sensor 12, and also, the observation image λi(1)t observed at the generating model is supplied from the estimated observation value generating unit 44.
The visible model likelihood calculating unit 41 obtains observation likelihood p(xit|zi(1)t) of the observation value xit of the foreground #i when the foreground #i of interest is in the state zi(1)t serving as a particle using the actual image λ ̃t from the sensor 12 and the observation image λi(1)t from the estimated observation value generating unit 44 in accordance with Expression (22), and supplies this to the state particle updating unit 42.
The state particle updating unit 42 performs prediction (calculation) of a transition destination of a particle to obtain a particle zi(1)t serving as the state zit at the next point-in-time t from the state zi(1)t−1 serving as a particle at the point-in-time t−1 stored in the state particle storage unit 43 in accordance with Expression (20) or Expression (21), as estimation of the state zit of the foreground #i serving as a particle.
The state particle updating unit 42 supplies the particle zi(1)t serving as the state zit at the next point-in-time t to the state particle storage unit 43, and stores this therein instead of the state zi(1)−1 serving as a particle at the point-in-time t−1.
Further, the state particle updating unit 42 obtains the weighting wi(1)t of the particle zi(1)t using the observation likelihood p(xit|z(1)t) from the visible model likelihood calculating unit 41 in accordance with Expression (14), and supplies this to the state particle storage unit 43 to store this therein in the same way as with the particle zi(1)t.
Also, the state particle updating unit 42 performs re-sampling to select the particle zi(1)t with probability corresponding to the weighting wi(1)t of the particle zi(1)t, and rewrites the particle zi(1)t stored in the state particle storage unit 43 with the re-sampling results.
The particle zi(1)t serving as re-sampling results is used at the time of predicting the particle zi(1)t+1 serving as the state zit+1 at the point-in-time t+1 in accordance with Expression (20) or Expression (21) after the observation likelihood p(xi(1)t+1|zi(1)t+1) at the next point-in-time t+1 is obtained.
Note that, with the state particle updating unit 42, when predicting the particle zi(1)t serving as the state zit at the point-in-time t in accordance with Expression (21) of Expressions (20) and (21), the motion model ai stored in the motion model storage unit 20 and the action uit stored in the action series storage unit 23 are employed. That is to say, γ/m and Φ/Ic in Expression (21) are obtained using the motion model ai and action uit.
The state particle storage unit 43 stores the particle zi(1)t serving as the foreground state zit and the weighting wi(1)t thereof that the state particle updating unit 42 updates.
The estimated observation value generating unit 44 generates an observation image (image to be observed at the generating model) λi(1)t where the foreground state has become the sate zi(1)t serving as each particle stored in the state particle storage unit 43 using the foreground visible model μi and background visible model μw stored in the visible model storage unit 15 in accordance with Expression (23), and supplies this to the visible model likelihood calculating unit 41.
FIG. 23 is a block diagram illustrating a configuration example of the motion model estimating unit 18 in FIG. 21.
The motion model estimating unit 18 includes a motion model likelihood calculating unit 51, a motion model particle updating unit 52, a motion model particle storage unit 53, and an estimated observation value generating unit 54.
With the motion model likelihood calculating unit 51, the most-likely series Zit={zit−TDYN+1, zit−TDYN+2, . . . , zit} of the state of the foreground #i are supplied from the most-likely state series storage unit 17. Further, with the motion model likelihood calculating unit 51, state series Z^it−TDYN:t={z^it−TDYN+1, z^it−TDYN+2, . . . , z^it} of the foreground #i obtained in accordance with the motion equations in Expressions (41) and (42) are supplied from the estimated observation value generating unit 54.
The motion model likelihood calculating unit 51 obtains observation likelihood of the state series Z^it−TDYN+1:t (likelihood that the state series Z^it−TDYN+1:t will be observed under the particle ai(1)t serving as the motion model) p(zit|ai)=p(Zit|ai(1)t) in the event that the motion model ait is the particle ai(1)t serving as a motion model in accordance with Expression (43) using the most-likely series Zit={zit−TDYN+1, zit−TDYN+2, . . . , zit} from the most-likely state series storage unit 17 and the state series Z^i(1)t−TDYN+1:t={z^i(1)t−TDYN+1, z^i(1)t−TDYN+2, . . . , z^it} obtained from the estimated observation value generating unit 54 following the motion equations in Expressions (41) and (42), and supplies this to the motion model particle updating unit 52.
The motion model particle updating unit 52 performs prediction (calculation) of a transition destination of a particle to obtain the particle ai(1)t serving as the motion model ait at the next point-in-time t from the motion model ai(1)t−1 serving as a particle at the point-in-time t−1 stored in the motion model particle storage unit 53 in accordance with Expression (44), as estimation of the motion model ait serving as a particle.
The motion model particle updating unit 52 supplies the particle ai(1)t serving as the motion model ait at the point-in-time t to the motion model particle storage unit 53, and stores this therein instead of the motion model ai(1)t−1 serving as a particle at the point-in-time t−1.
Further, the motion model particle updating unit 52 obtains the weighting wi(1)t of the particle ai(1)t in accordance with Expression (14) using the observation likelihood p(Zit|ai(1)t) from the motion model likelihood calculating unit 51 instead of the observation likelihood p(xit|zi(1)t) in Expression (14), and supplies this to the motion model particle storage unit 53 to store this therein in the same way as with the particle ai(1)t.
Also, the motion model particle updating unit 52 performs re-sampling to select the particle ai(1)t with probability corresponding to the weighting wi(1)t of the particle ai(1)t, and rewrites the particle ai(1)t stored in the motion model particle storage unit 53 with the re-sampling results.
The particle ai(1)t serving as the re-sampling results is used when predicting the particle ai(1)t+1 serving as the motion model ait+1 at the next point-in-time t+1 in accordance with Expression (44) after the observation likelihood p(Zit+1|ai(1)t+1) at the next point-in-time t+1 is obtained.
The motion model particle storage unit 53 stores the particle ai(1)t serving as the foreground motion model ait and the weighting wi(1)t thereof that the motion model particle updating unit 52 updates.
The estimated observation value generating unit 54 takes the most-likely state zit−TDYN at the point-in-time t−TDYN as an initial value, and calculates the motion equations in Expressions (41) and (42) using the particle ai(1)t serving as the motion model stored in the motion model particle storage unit 53, and action series Uit−1={uit−TDYN, uit−TDYN+1, . . . , uit−1} stored in the action series storage unit 23, thereby obtaining the state series Z^i(1)t−TDYN+1:t={z^i(1)t−TDYN+1, z^i(1)t−TDYN+2, . . . , z^i(1)t} of the foreground #i to be moved in accordance with the motion equations in Expressions (41) and (42), and supplying these to the motion model likelihood calculating unit 51.
FIG. 24 is a block diagram illustrating a configuration example of the action generating unit 22 in FIG. 21.
In FIG. 24, the action generating unit 22 includes a translation motion control unit 61, a rotation motion control unit 62, and an action output unit 63, and generates action such as the hand serving as the other foreground or the like for changing the state of the object to be operated or the like serving as one foreground to a predetermined target state using the foreground visible model μi stored in the visible model storage unit 15 and the motion model ai stored in the motion model storage unit 20, and so forth.
Specifically, with the translation motion control unit 61, there are supplied the foreground visible model μi stored in the visible model storage unit 15, the most-likely state zit stored in the most-likely state series storage unit 17, the motion model ai stored in the motion model storage unit 20, and the target position stgt and target attitude qtgt serving as the target state generated at the target generating unit 21.
The translation motion control unit 61 recognizes the edges of the foreground (object) #i to be operated from the foreground visible model μi and most-likely state zit, and obtains, as described with FIG. 18, a point of load for performing translation motion to bring a position sobjt=sit serving the current state (most-likely state zit) of the foreground #i close to the target position stgt, and a force vector to be applied to the point of load thereof.
The translation motion control unit 61 supplies the point of load and force vector to the action output unit 63.
With the rotation motion control unit 62, there are supplied the foreground visible model μi stored in the visible model storage unit 15, the most-likely state zit stored in the most-likely state series storage unit 17, the motion model ai stored in the motion model storage unit 20, and the target position stgt and target attitude qtgt serving as the target state generated at the target generating unit 21.
The rotation motion control unit 62 recognizes the edges of the foreground (object) #i to be operated from the foreground visible model μi and most-likely state zit, and obtains, as described with FIG. 19, a point of load for performing rotation motion to bring attitude qobjt=qit serving the current state (most-likely state zit) of the foreground #i close to the target attitude qtgt, and a force vector to be applied to the point of load thereof.
The rotation motion control unit 62 supplies the point of load and force vector to the action output unit 63.
The action output unit 63 generates, in accordance with the point of load and force vector from the translation motion control unit 61 or rotation motion control unit 62, a velocity vector s•manipt for applying the force of the force vector thereof to the point of load thereof in accordance with Expression (48), and outputs this to the actuator control unit 11 and action series storage unit 23 as action of the hand.
Note that the action output unit 63 obtains the initial position smanip0 described with FIG. 20 when starting translation motion and when starting rotation motion, and outputs action to move the hand to the initial position smanip0 thereof.
Also, with the action output unit 63, there are supplied the most-likely state zit stored in the most-likely state series storage unit 17, and the target position stgt and target attitude qtgt serving as the target state generated at the target generating unit 21.
The action output unit 63 obtains error of the position sit serving as the most-likely state zit of the object to be operated with respect to the target position stgt, and error of the attitude serving as the most-likely state zit with respect to the target attitude qtgt, and controls output of action to move the hand according to the error of the position sit and the error of the attitude sit.
FIG. 25 is a flowchart for describing overview of learning processing of a foreground model, background model, and motion model to be performed by the agent in FIG. 21.
With the agent, an image is shot at the sensor 12, and the image (actual image) λ ̃t thereof is supplied from the sensor 12 to the foreground state estimating unit 13 and visible model updating unit 14.
Upon the actual image λ ̃t being supplied from the sensor 12 to the foreground state estimating unit 13, in step S31 the agent selects the foreground model #i which has not been selected as the model of interest, as the model of interest with respect to the actual image λ ̃t r and the processing proceeds to step S32.
In step S32, the foreground state estimating unit 13 estimates the state zit of the foreground #i using the observation value λ ̃t of the image from the sensor 12, the foreground visible model μi which the model of interest stored in the visible model storage unit 15 and the background visible model μw, and supplies the particle zi(1)t of the state zit of the foreground #i that the model of interest represents, and the weighting wi(1)t thereof to the visible model updating unit 14 and most-likely state calculating unit 16 as results of the estimation thereof, and the processing proceeds to step S33.
Here, the most-likely state calculating unit 16 obtains the most-likely state zit in accordance with Expression (39) using the particle zi(1)t and weighting wi(1)t of the state zit serving as the estimation results of the state zit of the foreground #i supplied from the foreground state estimating unit 13, and supplies this to the most-likely state series storage unit 17 to store this therein.
In step S33, the visible model updating unit 14 updates the foreground visible model μi stored in the visible model storage unit 15 using the observation value λ ̃t of the image from the sensor 12, and the particle zi(1)t and weighting wi(1)t of the state zit serving as the estimation results of the state zit of the foreground #i supplied from the foreground state estimating unit 13, and the processing proceeds to step S34.
In step S34, the visible model updating unit 14 updates the background visible model μw stored in the visible model storage unit 15 using the observation value λ ̃t of the image from the sensor 12, and the particle zi(1)t and weighting wi(1)t of the state zit serving as the estimation results of the state zit of the foreground #i supplied from the foreground state estimating unit 13, and the processing proceeds to step S35.
In step S35, the motion model estimating unit 18 estimates the motion model ai of the foreground #i that the model of interest represents using the most-likely series Zit=(zit−TDYN+1, zit−TDYN+2, . . . , zit) for the past TDYN−1 time worth stored in the most-likely state series storage unit 17, the most-likely state zit−TDYN, and the action series Uit−1={uit−TDYN, uit−TDYN+1, . . . , uit−1} stored in the action series storage unit 23, and supplies the particle ai(1)t of the motion model ai and weighting wi(1)t thereof to the motion model updating unit 19 as estimation results, and the processing proceeds to step S36.
In step S36, the motion model updating unit 19 updates the motion model ai stored in the motion model storage unit 20 using the particle ai(1)t and weighting wi(1)t of the motion model ai serving as the estimation results of the motion model ai supplied from the motion model estimating unit 18.
Thereafter, the processing returns from step S36 to step S31, where of the N foreground models #1 to #N, one foreground model which has not been selected as the model of interest with respect to the actual image λ ̃t is selected as the model of interest, and hereinafter, the same processing is repeated.
In the event that the N foreground models #1 to #N have been selected as the model of interest with respect to the actual image λ ̃t, the processing in steps S31 to S36 is repeated in wait for the actual image λ ̃t+1 at the next point-in-time t+1 being supplied from the sensor 12 to the foreground state estimating unit 13.
Note that, in FIG. 25, the processing in steps S32 to S34 is learning of a foreground model and background model, and the processing in steps S35 and S36 is learning of a motion model.
That is to say, with the agent, learning of a foreground model and background model, and learning of a motion model are alternately performed.
FIG. 26 is a flowchart for describing details of the learning processing of a foreground model, background model, and motion model to be performed by the agent in FIG. 21.
With the agent, an image is shot at the sensor 12, and the image (actual image) λ ̃t thereof is supplied from the sensor 12 to the foreground state estimating unit 13 and visible model updating unit 14.
Upon the actual image λ ̃t being supplied from the sensor 12 to the foreground state estimating unit 13, in step S51 the agent selects the foreground model #i which has not been selected as the model of interest, as the model of interest with respect to the actual image λ ̃t f and the processing proceeds to step S52.
In step S52, with the foreground state estimating unit 13 (FIG. 22), the state particle updating unit 42 selects one state particle zi(1)t−1 which has not been selected as the state particle of interest out of the states (hereinafter, also referred to as state particles) zi(1)t−1 to zi(L)t−1 of the foreground #i serving as L particles at the point-in-time t−1 stored in the state particle storage unit 43, as the state particle of interest, and the processing proceeds to step S53.
In step S53, the state particle updating unit 42 predicts the state particle zi(1)t of interest at the point-in-time t from the state particle zi(1)t−1 of interest stored in the state particle storage unit 43 in accordance with Expression (20) or Expression (21).
Further, the state particle updating unit 42 supplies the state particle zi(1)t of interest at the point-in-time t to the state particle storage unit 43 to store this therein instead of the state particle zi(1)t−1 of interest at the point-in-time t−1, and the processing proceeds from step S53 to step S54.
In step S54, with the foreground state estimating unit 13 (FIG. 22), the estimated observation value generating unit 44 generates an observation image (an image to be observed at the generating model) λi(1)t where the foreground state becomes the state particle zi(1)t of interest stored in the state particle storage unit 43, using the foreground visible model μi and background visible model μw stored in the visible model storage unit 15 in accordance with Expression (23).
The estimated observation value generating unit 44 supplies the observation image λi(1)t to the visible model likelihood calculating unit 41, and the processing proceeds from step S54 to step S55.
In step S55, the visible model likelihood calculating unit 41 (FIG. 22) obtains observation likelihood p(xit|zi(1)t) of the observation value xit of the foreground #i when the foreground #i of interest becomes the state zi(1)t serving as the state particle of interest in accordance with Expression (22) using the actual image λ ̃t from the sensor 12 and the observation image λi(1)t from the estimated observation value generating unit 44.
The visible model likelihood calculating unit 41 supplies the observation likelihood p(xit|zi(1)t) to the state particle updating unit 42, and the processing proceeds from step S55 to step S56.
In step S56, the state particle updating unit 42 obtains weighting wi(1)t of the state particle zi(1)t of interest in accordance with Expression (14) using the observation likelihood p(xit|zi(1)t) from the visible model likelihood calculating unit 41, and supplies, in the same way as with the state particle zi(1)t, this to the state particle storage unit 43 to store this therein, and the processing proceeds to step S57.
In step S57, the state particle updating unit 42 determines whether or not all of the L state particles zi(1)t−1 to zi(L)t−1 at the point-in-time t−1 stored in the state particle storage unit 43 have been selected as the state particle of interest.
In the event that determination is made in step S57 that the L state particles zi(1)t−1 to zi(L)t−1 have not been selected as the state particle of interest, the processing returns to step S52, where the state particle updating unit 42 newly selects one state particle which has not been selected as the state particle of interest of the L state particles zi(1)t−1 to zi(L)t−1, as the state particle of interest, and hereinafter, the same processing is repeated.
Also, in the event that determination is made in step S57 that all of the L state particles zi(1)t−1 to zi(L)t−1 have been selected as the state particle of interest, the processing proceeds to step S58, where the visible model updating unit 14 updates the foreground visible model μi stored in the visible model storage unit 15 using the actual image λ ̃t supplied from the sensor 12, the state particle zi(1)t at the point-in-time t stored in the state particle storage unit 43 in step S53, and the weighting wi(1)t stored in the state particle storage unit 43 in step S56, and the processing proceeds to step S59.
Specifically, the visible model updating unit 14 updates the foreground texture model τi of the foreground visible model μi in accordance with Expressions (24) to (27), and updates the foreground shape model σi of the foreground visible model μi in accordance with Expressions (34) to (38).
Note that, as described above, the most-likely state calculating unit 16 obtains the most-likely state zit in accordance with Expression (39) using the state particle zi(1)t stored in the state particle storage unit 43 and weighting wi(1)t thereof, and supplies this to the most-likely state series storage unit 17 to store this therein in parallel with the visible model updating unit 14 updating the foreground visible model μi.
In step S59, the visible model updating unit 14 updates the background visible model μw stored in the visible model storage unit 15 using the actual image λ ̃t from the sensor 12, the state particle zi(1)t at the point-in-time t stored in the state particle storage unit 43 in step S53, and the weighting wi(1)t stored in the state particle storage unit 43 in step S56, and the processing proceeds to step S60.
Specifically, the visible model updating unit 14 updates the background visible model μw (background texture model τw) in accordance with Expressions (28) to (33).
In step S60, with the motion model estimating unit 18 (FIG. 23), the motion model particle updating unit 52 selects one motion model particle ai(1)t−1 which has not been selected as the motion model particle of interest yet out of the motion models (hereinafter, also referred to as motion model particles) ai(1)t−1 to ai(L)t−1 of the foreground #i serving as the L particles at the point-in-time t−1 stored in the motion model particle storage unit 53, as the motion model particle of interest, and the processing proceeds to step S61.
In step S61, the motion model particle updating unit 52 predicts the motion model particle ai(1)t of interest at the next point-in-time t from the motion model particle ai(1)t−1 of interest at the point-in-time t−1 stored in the motion model particle storage unit 53 in accordance with Expression (44).
Further, in step S61, the motion model particle updating unit 52 supplies the motion model particle ai(1)t of interest at the point-in-time t to the motion model particle storage unit 53 to store this therein instead of the motion model ai(1)t−i serving as the particle at the point-in-time t−1, and the processing proceeds to step S62.
In step S62, the estimated observation value generating unit 54 (FIG. 23) takes the most-likely state zit−TDYN at the point-in-time t−TDYN stored in the most-likely state series storage unit 17 as an initial value, and calculates the motion equations in Expressions (41) and (42) using the motion model particle ai(1)t of interest stored in the motion model particle storage unit 53 and the action series Uit−1={uit−TDYN, uit−TDYN+1, . . . , uit−1} stored in the action series storage unit 23, thereby obtaining the state series Z^i(1)t−TDYN+1:t={z^i(1)t−TDYN+1, z^i(1)t−TDYN+2, . . . , z^i(1)t} of the foreground #i which moves in accordance with the motion equations in Expressions (41) and (42) with respect to the motion model particle ai(1)t of interest.
The estimated observation value generating unit 54 supplies the state series Z^i(1)t−TDYN+1:t={z^i(1)t−TDYN+1, z^i(1)t−TDYN+2, . . . , z^i(1)t} obtained with respect to the motion model particle ai(1)t of interest to the motion model likelihood calculating unit 51, and the processing proceeds from step S62 to step S63.
In step S63, the motion model likelihood calculating unit 51 obtains observation likelihood p(zit|ai)=p(Zit|ai(1)t) of the state series z^i(1)t−TDYN+1:t in the event that the motion model ait is the motion model particle ai(1)t of interest, using the most-likely series Zit={zit−TDYN+1, zit−TDYN+2, . . . , zit} stored in the most-likely state series storage unit 17 and the state series Z^i(1)t−TDYN+1:t={z^i(1)t−TDYN+2, z^i(1)t−TDYN+2, . . . , z^i(1)t} with respect to the motion model particle ai(1)t of interest from the estimated observation value generating unit 54 in accordance with Expression (43).
The motion model likelihood calculating unit 51 then supplies the observation likelihood p(zit|ai)=p(Zit|ai(1)t) to the motion model particle updating unit 52, and the processing proceeds from step S63 to step S64.
In step S64, the motion model particle updating unit 52 obtains weighting wi(1)t of the motion model particle ai(1)t of interest using the observation likelihood p(Zit|ai(1)t) from the motion model likelihood calculating unit 51 instead of the observation likelihood p(xit|zi(1)t) in Expression (14).
The motion model particle updating unit 52 supplies the weighting wi(1)t of the motion model particle ai(1)t of interest to the motion model particle storage unit 53 to store this therein in the same way as with the motion model particle ai(1)t, and the processing proceeds from step S64 to step S65.
In step S65, the motion model particle updating unit 52 determines whether or not all of the L motion model particles ai(1)t−1 to ai(L)t−1 at the point-in-time t−1 stored in the motion model particle storage unit 53 have been selected as the motion model particle of interest.
In the event that determination is made in step S65 that the L motion model particles ai(1)t−1 to ai(L)t−1 have not been selected yet as the motion model particle of interest, the processing returns to step S60, where the motion model particle updating unit 52 newly selects one motion model particle which has not been selected yet as the motion model particle of interest, of the L motion model particles ai(1)t−1 to ai(L)t−1, as the motion model particle of interest, and hereinafter, the same processing is repeated.
Also, in the event that determination is made in step S65 that all of the L motion model particles ai(1)t−1 to ai(L)t−1 have been selected as the motion model particle of interest, the processing proceeds to step S66, where the motion model updating unit 19 (FIG. 21) updates the motion model ai stored in the motion model storage unit 20 in accordance with Expression (45) using the motion model particle ai(1)t and weighting wi(1)t stored in the motion model particle storage unit 53, and the processing proceeds to step S67.
In step S67, the agent determines whether or not all of the N foreground models #1 to #N have been selected as the model of interest with respect to the actual image λ ̃t.
In the event that determination is made in step S67 that the N foreground models #1 to #N have not been selected as the model of interest yet, the processing returns to step S51. In step S51, the agent newly selects the foreground model #i which has not been selected as the model of interest with respect to the actual image λ ̃t, as the model of interest, and hereinafter, the same processing is repeated.
Also, in the event that determination is made in step S67 that all of the N foreground models #1 to #N have been selected as the model of interest, the processing proceeds to step S68, where the state particle updating unit 42 (FIG. 22) performs re-sampling to select the state particle zi(1)t stored in the state particle storage unit 43 with probability corresponding to the weighting wi(1)t similarly stored in the state particle storage unit 43, rewrites the state particle zi(1)t stored in the state particle storage unit 43 with the re-sampling result, and the processing proceeds to step S69.
The state particle zi(1)t serving as the re-sampling result is used at the time of predicting the state particle zi(1)t+1 serving as the state zit+1 at the point-in-time t+1 in accordance with Expression (20) or Expression (21) after the observation likelihood p(xit+1|zi(1)t+1) at the next point-in-time t+1 is obtained.
In step S69, the motion model particle updating unit 52 (FIG. 23) performs re-sampling to select the motion model particle ai(1)t stored in the motion model particle storage unit 53 with probability corresponding to the weighting wi(1)t similarly stored in the motion model particle storage unit 53, rewrites the motion model particle ai(1)t stored in the motion model particle storage unit 53 with the re-sampling result.
The motion model particle ai(1)t serving as the re-sampling result is used at the time of predicting the motion model particle ai(1)t+1 serving as the motion model ait+1 at the point-in-time t+1 in accordance with Expression (44) after the observation likelihood p(Zit+1|ai(1)t+1) at the next point-in-time t+1 is obtained.
After the processing in step S69, an actual image λ ̃t+1 at the next point-in-time is shot at the sensor 12, the processing returns to step S51 in wait for the actual image λ ̃t+1 thereof being supplied from the sensor 12 to the foreground state estimating unit 13, and hereinafter, the same processing is repeated.
Note that, in FIG. 26, steps S52 to S57 and steps S60 to S65 are equivalent to the E step in the EM algorithm, and steps S58 and S59 and step S66 are equivalent to the M step in the EM algorithm.
FIG. 27 is a flowchart for describing overview of action control processing that the agent performs for object operations.
In step S71, the action output unit 63 (FIG. 24) determines, based on the most-likely state zit at the current point-in-time t stored in the most-likely state series storage unit 17, and the target position stgt and target attitude qtgt serving as a target state generated at the target generating unit 21, whether or not both of error of the position sit serving as the most-likely state zit of the object to be operated with respect to the target position stgt (hereinafter, also referred to as position error), and error of the attitude sit serving as the most-likely state zit with respect to the target attitude qtgt (hereinafter, also referred to as attitude error) are sufficiently small.
In the event that determination is made in step S71 that one or both of the position error and attitude error are not sufficiently small, i.e., in the event that the position error or attitude error is not equal to nor smaller than a predetermined threshold, the processing proceeds to step S72, where the action output unit 63 determines, in the same way as with step S71, whether or not the position error is sufficiently small.
In the event that determination is made in step S72 that the position error is not sufficiently small, i.e., in the event that the position error is not equal to nor smaller than a predetermined threshold, the processing proceeds to step S73, where the action generating unit 22 (FIG. 24) performs translation motion control.
Specifically, in step S73, the translation motion control unit 61 recognizes the edges of the foreground (object) #i to be operated from the foreground visible model μi stored in the visible model storage unit 15 and the most-likely state zit stored in the most-likely state series storage unit 17, and obtains, as described with FIG. 18, a point of load for performing translation motion to bring the position sit=sobj serving as the current state (most-likely state zit) of the foreground #i close to the target position stgt serving as the target state generated at the target generating unit 21, and a force vector of the force to be applied to the point of load thereof, and supplies these to the action output unit 63.
The action output unit 63 generates, in accordance with the point of load and force vector from the translation motion control unit 61, a velocity vector s•manipt for applying the force of the force vector thereof to the point of load thereof, in accordance with Expression (48), and outputs this to the actuator control unit 11 and action series storage unit 23 as action of the hand.
Thereafter, the processing returns from step S73 to step S71, and hereinafter, the same processing is repeated.
On the other hand, in the event that determination is made in step S72 that the position error is sufficiently small, i.e., in the event that the position error is equal to nor smaller than a predetermined threshold, the processing proceeds to step S73, where the action generating unit 22 performs rotation motion control.
Specifically, in step S74, the rotation motion control unit 62 recognizes the edges of the foreground (object) #i to be operated from the foreground visible model μi stored in the visible model storage unit 15 and the most-likely state zit stored in the most-likely state series storage unit 17, and obtains, as described with FIG. 19, a point of load for performing rotation motion to bring the attitude qit=qobjt serving as the current state (most-likely state zit) of the foreground #i close to the target attitude qtgt, and a force vector of the force to be applied to the point of load thereof.
The rotation motion control unit 62 supplies the point of load and force vector to the action output unit 63.
The action output unit 63 generates, in accordance with the point of load and force vector from the rotation motion control unit 62, a velocity vector s•manipt for applying the force of the force vector thereof to the point of load thereof, in accordance with Expression (48), and outputs this to the actuator control unit 11 and action series storage unit 23 as action of the hand.
Thereafter, the processing returns from step S74 to step S71, and hereinafter, the same processing is repeated.
In the event that determination is made in step S71 that both of the position error and attitude error are sufficiently small, the action control processing is ended.
Note that, with the action control processing in FIG. 27 (the same is true in later-described FIG. 28), an arrangement may be made wherein in the event that translation motion is preferentially performed, and the position error decreases, rotation motion is performed, but conversely, in the event that rotation motion is preferentially performed, and the attitude error decreases, translation motion is performed.
FIG. 28 is a flowchart for describing details of the action control processing for the agent performing object operations.
In step S81, the target generating unit 21 generates (determines) the target position stgt and target attitude qtgt serving as a target state, and supplies these to the translation motion control unit 61, rotation motion control unit 62, and action output unit 63 of the action generating unit 22 (FIG. 24), and the processing proceeds to step S82.
In step S82, the action output unit 63 determines, based on the most-likely state zit stored in the most-likely state series storage unit 17, and the target position stgt and target attitude qtgt serving as the target state Zit from the target generating unit 21, whether or not both of error of the position sit=sobjt serving as the most-likely state of the object to be operated (position error) with respect to the target position stgt, and error of the attitude sit serving as the most-likely state zit (attitude error) with respect to the target attitude qtgt are sufficiently small.
In the event that determination is made in step S82 that one or both of the position error and attitude error are not sufficiently small, i.e., in the event that the position error or attitude error is not equal to nor smaller than a predetermined threshold, the processing proceeds to step S83, where the action output unit 63 determines, in the same way as with step S82, whether or not the position error is sufficiently small.
In the event that determination is made in step S83 that the position error is not sufficiently small, i.e., in the event that the position error is not equal to nor smaller than a predetermined threshold, the processing sequentially proceeds to steps S84 to S86, where the translation motion control unit 61 performs translation motion control.
Specifically, in step S84, the translation motion control unit 61 recognizes the edges of the foreground (object) #i to be operated from the foreground visible model μi stored in the visible model storage unit 15, and the most-likely state zit stored in the most-likely state series storage unit 17.
Further, the translation motion control unit 61 determines, as described with FIG. 18, intersections between the straight line L1 which connects the target position stgt of the object, and the current position (center-of-gravity) sobjt, and the edges of the current object to be candidates of the point of load, and the processing proceeds from step S84 to step S85.
In step S85, the translation motion control unit 61 selects (determines), of candidates for the point of load obtained in step S84, a candidate farthest from the target position stgt to be the point of load of the force, and the processing proceeds to step S86.
In step S86, the translation motion control unit 61 obtains a force vector of the force to be applied to the point of load in accordance with Expression (46), and the processing proceeds to step S87.
In step S87, the translation motion control unit 61 outputs the point of load and force vector obtained in steps S84 to S86 to the action output unit 63, and the processing proceeds to step S92.
On the other hand, in the event that determination is made in step S83 that the position error is sufficiently small, i.e., in the event that the position error is equal to or smaller than a predetermined threshold, the processing sequentially proceeds to steps S88 to S91, where the rotation motion control unit 62 performs rotation motion control.
Specifically, in step S88, the rotation motion control unit 62 recognizes the edges of the foreground (object) #i to be operated from the foreground visible model μi stored in the visible model storage unit 15, and the most-likely state zit stored in the most-likely state series storage unit 17.
Further, the rotation motion control unit 62 determines, as described with FIG. 19, an intersection between each of the three straight lines L1, L2, and L3 serving as one or more straight lines passed through the position (center-of-gravity) sobjt of the current state of the object to be a candidate of the point of load, and the processing proceeds from step S88 to step S89.
In step S89, the rotation motion control unit 62 selects (determines) of candidates for the point of load obtained in step S88, a candidate farthest from the center-of-gravity sobjt of the object which is a candidate where the direction of torque generated at the time of applying the force so as to press the edges of the object agrees with a direction intended to rotate the object, to be the point of load of the force, and the processing proceeds to step S90.
In step S90, the rotation motion control unit 62 obtains torque to be generated by the force vector of the force to be applied to the point of load in accordance with Expression (47), and the processing proceeds to step S91.
In step S91, the rotation motion control unit 62 obtains the force vector of the force to be applied to the point of load to generate the torque obtained in step S90, and the processing proceeds to step S87.
In step S87, the rotation motion control unit 62 outputs the point of load and force vector obtained in steps S88 to S91 to the action output unit 63, and the processing proceeds to step S92.
In step S92, the action output unit 63 determines whether or not motion control for object operations is immediately after change from one to the other of translation motion control by the translation motion control unit 61, and rotation motion control by the rotation motion control unit 62.
In the event that determination is made in step S92 that motion control is immediately after change from one to the other of translation motion control and rotation motion control, the processing proceeds to step S93, where the action output unit 63 outputs action to move the hand to the initial position described with FIG. 20.
In wait for the hand moving to the initial position, the processing proceeds from step S93 to step S94.
On the other hand, in the event that determination is made in step S92 that motion control is not immediately after change from one to the other of translation motion control and rotation motion control, the processing skips step S93 to proceed to step S94, where the action output unit 63 generates, in accordance with the point of load and force vector from the translation motion control unit 61 or rotation motion control unit 62, a velocity vector s•manipt for applying the force of the force vector thereof to the point of load thereof in accordance with Expression (48).
Further, the action output unit 63 outputs the velocity vector s•manipt generated in accordance with Expression (48) to the actuator control unit 11 and action series storage unit 23, and the processing returns to step S82.
In the event that determination is made in step S82 that both of the position error and attitude error are sufficiently small, i.e., in the event that both of the position error and attitude error are equal to or smaller than a predetermined threshold, the action control processing is ended determining that the state of the object to be operated becomes the target state generated at the target generating unit 21.
Simulation
FIGS. 29A to 29G are diagrams schematically illustrating simulation results of the learning processing and action control processing performed at a computer by employing an image created by synthesizing two foregrounds #1 and #2 as the actual image λ ̃t.
With the simulation, the learning processing of a foreground model, background model, and motion model was performed by employing an image on which a scene where the foreground #1 serving as a L-shaped object is operated by the foreground #2 serving as a circular hand is reflected, as the actual image λ ̃t.
FIG. 29A illustrates the actual image λ ̃t at certain point-in-time t.
FIG. 29B illustrates an observation image λt generated by superimposing a foreground visible model μ1 (foreground texture model τ1 and foreground shape model σ1) obtained at the point-in-time t on a background visible model μw=τw with the most-likely state z1t, and also superimposing a foreground visible model μ2 (foreground texture model τ2 and foreground shape model σ2) on the background visible model μw with a most-likely state z2t.
FIG. 29C illustrates a scene when operating the L-shaped object by the circular hand, and specifically illustrates a force vector to be applied to the L-shaped object to be operated, and a point of load to which the force vector thereof is applied.
FIG. 29D illustrates the foreground texture model τ1 and foreground shape model σ1 of the foreground visible model μ1 of the foreground #1, and the foreground texture model τ2 and foreground shape model σ2 of the foreground visible model μ2of the foreground #2.
FIG. 29E illustrates the background model μw=τw.
FIG. 29F illustrates center-of-gravity r1c of each of two certain motion model particles (illustrated with a circle mark and a star mark) at certain point-in-time t serving as a motion model a1 of the foreground #1 to be operated.
FIG. 29G illustrates mass m1c and inertial moment I1c of each of two certain motion model particles (illustrated with a circle mark and a star mark) at certain point-in-time t serving as the motion model a1 of the foreground #1 to be operated.
According to the simulation performed by the present inventor, there was able to be confirmed a scene where as the learning processing advances, the foreground and background of the actual image λ ̃t are acquired separately with the foreground models and background model respectively, and further, the two foregrounds #1 and #2 of the actual image λ ̃t are exclusively acquired for each of the individual foreground models respectively, and the foreground visible models μ1 and μ2 and background visible model μw converge on the foregrounds #1 and #2 and background that are reflected on the actual image λ ̃t, respectively.
Also, with the simulation, it was able to be confirmed that as the foreground visible models μ1 and μ2 and background visible model μw converge, an observation image λt to be generated with the foreground visible models μ1 and μ2 and background visible model μw thereof approximate the actual image λ ̃t.
As described above, according to the present technology, series of the actual images λ ̃t to be observed at an agent capable of performing action are employed as learned data for learning, and accordingly, there may be readily performed foreground and background learning to simultaneously learn foreground visibility and a state (spatial) thereof and background visibility within an environment where the agent is allocated without an explicit tutor label to distinguish the fore ground and background.
Specifically, according to the present technology, in the event that multiple foregrounds are moving on the background while changing their states such as a position, attitude, or the like, foreground visibility and state such as a position and attitude, and background visibility are simultaneously able to be acquired using times series of an actual image where the foregrounds are superimposed on the background as learned data even without teaching information of a foreground visibility and state and so forth beforehand.
Also, according to the present technology, in addition to an actual image, action series of a foreground are employed as learned data, and accordingly, a motion model of the foreground is able to be acquired, and further, the state of the foreground is able to be estimated with high precision by considering that action of a foreground follows a kinetics model for rigid-body motion, and consequently, foreground visibility and background visibility are able to be captured with high precision.
Additionally, according to the present technology, processing is performed with only a state serving as the particle zi(1)t having high foreground observation likelihood p(xt|zi(1)t) as an object by applying the particle filter, and accordingly, calculation cost may exponentially be reduced as compared to, for example, a case where processing is performed by considering that, with regard to all of the pixels in the background, a foreground exists in the positions of the pixels thereof, and consequently, scalability may be improved.
For example, in the event that, with regard to each pixel of the background, processing is performed by taking that a foreground exists in the position of the pixel thereof, in order to execute 1-fps (Frames Per Second) calculation processing at a normal household PC, though there has been a limitation wherein an image with resolution of around horizontally 30 by vertically 40 pixels may be employed as an actual image, according to the present technology, an image with resolution of around horizontally 128 by vertically 128 pixels to horizontally 640 by vertically 480 pixels may be employed as an actual image.
Also, according to the present technology, in addition to the position sit of the foreground #i, velocity s•it of change in the attitude qit or position sit, velocity q•it of change in the attitude qit may be handled as the state zit of the foreground #i. Specifically, with the present technology, with regard to a two-dimensional actual image, there may be handled six degrees of freedom of the x coordinate and y coordinate serving as the position sit, a rotation angle serving as the attitude qit, an x component and y component serving as the velocity s•it of change in the position sit, and an angular velocity serving as the velocity q•it of change in the attitude qit.
Further, with the present technology, when the observation value xt+1 at the point-in-time t+1, series Xt={x1, x2, . . . , xt} of the observation value are observed until the point-in-time t, and a posterior distribution p(zt+1|Xt) that is in the state zt+1 at the point-in-time t+1 is a mixed distribution in Expression (15) using transition probability p(zt+1|z(1)t) that is in the state z(1)t at the point-in-time t will change to the state zt+1 at the point-in-time t+1, and the weighting w(1)t of the particle z(1)t, and accordingly, consecutive amount may be handled as a foreground state of such a posterior distribution p(zt+1|Xt).
Also, with the present technology, the foreground visible model μi includes the foreground texture model τi representing the texture of the foreground #i, and the foreground shape model σi representing probability that a pixel value will be a pixel of the foreground, and accordingly, according to the foreground visible model μi, even with a foreground having a complicated shape, the foreground thereof may be expressed with high precision.
Further, according to the present technology, the state of a foreground is estimated, and the foreground visible model μi and background visible model μw are updated using the estimation result of the foreground thereof, and also, a motion model which is a kinetics parameter for expressing spatial transition of a foreground is estimated using the estimation result of the foreground, and the motion model is updated using the estimation result of the motion model thereof, and accordingly, the center-of-gravity ric, mass mi, and inertial moment Iic, serving as the motion model σi of the foreground #i, may be acquired with high precision.
Also, according to the present technology, there may be performed object operations to change an object within an environment to an optional state (target state) using (the foreground shape model σi of) the foreground visible model μi and motion model.
Further, with the present technology, as the state zit of the foreground #i, in addition to the position sit and attitude qit of the foreground #i, velocity s•it of change in the position sit and velocity q•it of change in the attitude qit which are the primary differentials of the attitude sit and attitude qit are handled, and accordingly, a foreground which performs (motion approximate to) motion with constant velocity or motion with constant angular velocity may be modeled with high precision in accordance with the law of inertia, for example, like a rigid body which slides on an ice top.
Also, according to the present technology, an object which moves following an operation to change the attitude of the object or the rule of kinetics may be operated such as an object operation at the real world.
For example, with the present technology, object operations may be performed by considering that in the event of operating a slender object, translation motion may be performed in a stable manner by applying force a position closer to center-of-gravity, that in order to change the attitude of an object, the attitude may be changed with smaller force by applying force a position farther from center-of-gravity, and so forth.
Accordingly, the present technology may be expected to be applied to environment recognition of an agent which is implemented under a three-dimensional environment, and operates under the actual environment, or action control technique in the future.
Further, with the present technology, only by providing actual image series and action series as learned data, foregrounds and background are individually learned, and also a foreground motion model is learned, and finally, object operations may be performed, and accordingly, time and effort used for creating an environment model and a database like before may be omitted. Accordingly, improvement in adaptation capability under an optional environment may be expected.
Also, with the present technology, action control for object operations is executed based on the foreground visible model μi and motion model ai obtained by successive learning using an actual image where the object to be operated and the hand are reflected as foregrounds.
Accordingly, with the initial stage of learning, learning for the foreground visible model μi and motion model ai does not sufficiently converge, and accordingly, it is difficult to recognize a foreground such as the hand or object or the like within an environment, with high precision, and further, it is also difficult to perform suitable action control for object operations.
However, in the event that time has elapsed, and learning has advanced, recognition performance for a foreground has improved, and accompanied therewith, suitable action control for object operations has been enabled. As a result thereof, with the agent, the hand may be moved so as to effectively come into contact with the object to be operated, and it becomes easy to observe an actual image for accurately learning the foreground visible model μi and motion model ai of a foreground such as the hand or object or the like, and accordingly, as time elapses, learning capability is expected to improve.
Here, according to the present technology, with recognition of a foreground, for example, observation likelihood p(xt|zit) that the observation value xt of the foreground #i which is in the most-likely state zit on an observation image λt(s) to be generated from the generating model will be observed is calculated regarding each of the foreground models #i in accordance with Expression (19), and the foreground model #i of which the observation likelihood p(xt|zit) is equal to or greater than a predetermined threshold is recognized to exist on the background in the most-likely sate zit.
Description of Computer with Present Technology Being Applied
Next, the above-mentioned series of processing may be performed by hardware, or may be performed by software. In the event of performing the series of processing by software, a program making up the software thereof is installed into a general-purpose computer or the like.
Accordingly, FIG. 30 illustrates a configuration example of an embodiment of the computer into which the program that executes the above-mentioned series of processing is installed.
The program may be recorded in a hard disk 105 or ROM 103 serving as recording media housed in the computer beforehand.
Alternatively, the program may be stored (recorded) in a removable recording medium 111. Such a removable recording medium 111 may be provided as a so-called package software. Here, examples of the removable recording medium 111 include a flexible disk, Compact Disc Read Only Memory (CD-ROM), Magneto Optical (MO) disk, Digital Versatile Disc (DVD), magnet disk, and semiconductor memory.
Note that, in addition to installing from the removable recording medium 111 to the computer as described above, the program may be downloaded to the computer via a communication network or broadcast network, and installed into a built-in hard disk 105. That is to say, the program may be transferred from a download site to the computer by radio via a satellite for digital satellite broadcasting, or may be transferred to the computer by cable via a network such as a Local Area Network (LAN) or the Internet.
The computer houses a Central Processing Unit (CPU) 102, and the CPU 102 is connected to an input/output interface 110 via a bus 101.
In the event that a command has been input via the input/output interface 110 by a user operating an input unit 107 or the like, in response to this, the CPU 102 executes the program stored in the Read Only Memory (ROM) 103. Alternatively, the CPU 102 loads the program stored in the hard disk 105 to Random Access Memory (RAM) 104 and executes this.
Thus, the CPU 102 performs processing following the above-mentioned flowchart, or processing to be performed by the configuration of the above-mentioned block diagram. For example, the CPU 102 outputs the processing results thereof from an output unit 106 via the input/output interface 110 or transmits from a communication unit 108, further records in the hard disk 105, and so forth as appropriate.
Note that the input unit 107 is configured of a keyboard, a mouse, a microphone, and so forth. Also, the output unit 106 is configured of a Liquid Crystal Display (LCD), a speaker, and so forth.
Here, with the present Specification, processing that the computer performs in accordance with the program does not necessarily have to be processed in time sequence along the sequence described as the flowchart. That is to say, the processing that the computer performs in accordance with the program also encompasses processing to be executed in parallel or individually (e.g., parallel processing or processing according to an object).
Also, the program may be processed by one computer (processor), or may be processed in a distributed manner by multiple computers. Further, the program may be transferred to a remote computer for execution.
Further, with the present Specification, the term “system” means a group of multiple components (devices, modules (parts), etc.), and it does not matter whether or not all of the components are included in the same casing. Accordingly, both of multiple devices housed in separates casings and connected via a network, and a single device in which multiple modules are housed in one casing are systems.
Note that an embodiment of the present technology is not restricted to the above-mentioned embodiment, and various modifications may be made without departing from the essence of the present technology.
For example, the present technology may have a configuration of cloud computing to process one function in a manner shared and collaborated with multiple devices.
Also, the steps described in the above-mentioned flowchart may be executed not only with a single device but also with multiple devices in a shared manner.
Further, in the event that multiple processes are included in one step, the multiple processes included in the one step may be executed not only with a single device but also with multiple devices in a shared manner.
Note that the present technology may have a configuration as will be described below.
[1] An information processing device including: a foreground state estimating unit configured to estimate a foreground state of an image using an actual image which is an image to be actually observed; and a visible model updating unit configured to update a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state.
[2] The information processing device according to [1], further including: a motion model estimating unit configured to estimate a motion model of the foreground using action of the foreground state, and state series of most-likely state of the foreground to be obtained from an estimation result of the foreground state; and a motion model updating unit configured to update the motion model using an estimation result of the motion model.
[3] The information processing device according to [2], wherein the foreground visible model includes a background texture model which represents texture of the background; and wherein the foreground visible model includes a foreground texture model which represents texture of the foreground, and a foreground shape model which represents a shape of the foreground; and wherein the foreground state includes a position of the foreground, speed of change in position, attitude, and speed of change in attitude; and wherein the foreground motion model includes mass of the foreground, center-of-gravity of mass, and moment of inertia working around center-of-gravity; and wherein the foreground action includes force to be applied to the foreground, and a position to which the force is applied.
[4] The information processing device according to [3], wherein the background visible model, the foreground visible model, and the motion model are updated in accordance with the EM algorithm.
[5] The information processing device according to [3] or [4], wherein the background visible model, the foreground visible model, and the motion model are updated in accordance with the EM algorithm using a particle filter.
[6] The information processing device according to [5], wherein the foreground state estimating unit performs processing to calculate weighting of a state particle which is a particle which represents the foreground state, and a transition destination of the state particle as the E step of the EM algorithm using the particle filter.
[7] The information processing device according to [6], wherein the foreground state estimating unit calculates a value corresponding to observation most-likelihood of an observation value to be observed at a generating model which generates an observation value of an image in the event that the foreground state is a state that the state particle represents, as weighting of the state particle, and also calculates a transition destination of the state particle assuming that the state particle makes the transition with stochastic fluctuation in accordance with the speed of change in position and speed of change in attitude of the foreground state that the state particle represents; and wherein with the generating model, the image observation value observed at the generating model is generated by subjecting, regarding points on an image that the foreground visible model represents, texture that the foreground texture model represents and texture that the background texture model represents to a blending with the foreground shape model as an α value, and employing, regarding points other than the points on the image that the foreground visible model represents, texture that the background texture model represents.
[8] The information processing device according to [7], wherein the foreground state estimating unit calculates a transition destination of the state particle assuming that the state particle makes the transition further in accordance with the action and the motion model.
[9] The information processing device according to any of [6] to [8], wherein the visible model updating unit performs as the M step of the EM algorithm using the particle filter: processing to update, in accordance with a state that the state particle represents, the foreground texture model by trimming an image from the actual image and subjecting the trimmed image which is an image trimmed from the actual image to weighting addition in accordance with weighting of the state particle; processing to update, in accordance with a state that the state particle represents, the background texture model by removing a foreground corresponding to the state from the actual image, and subjects the actual image from which the foreground has been removed to weighting addition in accordance with weighting of the state particle; and processing to update the foreground shape model by obtaining likelihood that each pixel of texture that the foreground texture model represents will be a foreground, and likelihood that each pixel will be a background, and obtaining probability that each pixel of texture that the foreground texture model represents will be a foreground using the likelihood that each pixel will be the foreground, and the likelihood that each pixel will be a background.
[10] The information processing device according to [9], wherein the visible model updating unit updates the foreground texture model by averaging a weighting addition value obtained by subjecting the trimmed image to weighting addition in accordance with weighting of the state particle, for an amount equivalent to a predetermined period of time, and also updates the background texture model by averaging a weighting addition value obtained by subjecting the actual image from which the foreground has been removed to weighting addition in accordance with weighting of the state particle, for an amount equivalent to a predetermined period of time.
[11] The information processing device according to [9] or [10], wherein the visible model updating unit updates, regarding a plurality of foreground visible models, the background texture model by repeating that the plurality of foreground visible models are sequentially taken as a model of interest, and in accordance with a state that the state particle represents, a foreground corresponding to the state is removed from a removed image from which a foreground that a foreground visible model which has been a model of interest immediately before represents has been removed, and the removed image from which the foreground has been removed is subjected to weighting addition in accordance with weighting of the state particle.
[12] The information processing device according to [5], wherein the motion model estimating unit performs processing to calculate weighting of a motion model particle which is a particle which represents the motion model, and a transition destination of the motion model particle as the E step of the EM algorithm using the particle filter.
[13] The information processing device according to [12], wherein the motion model estimating unit calculates a value corresponding to observation likelihood of state series of the foreground state to be observed at a motion model that the motion model particle represents as weighting of the motion model particle, and also calculates a transition destination of the motion model particle assuming that the motion model particle makes the transition with stochastic fluctuation.
[14] The information processing device according to [13], further including: a most-likely state calculating unit configured to obtain a most-likely state of the foreground; wherein the motion model estimating unit obtains a value corresponding to error between a state series of the foreground state to be generated with the motion model particle, and a state series of the most-likely state of the foreground as observation likelihood of a state series of the foreground state.
[15] The information processing device according to any of [12] to [14], wherein the motion model updating unit performs processing to update the motion model by subjecting a motion model that the motion model particle represents to weighting addition in accordance with weighting of the motion model particle as the M step of the EM algorithm using the particle filter.
[16] The information processing device according to any of [12] to [15], further including: an action generating unit configured to generate another foreground action which allows one foreground to be in a predetermined target state using the foreground visible model and the motion model.
[17] The information processing device according to [16], wherein the action generating unit generates action to translate the one foreground, and action to rotate the one foreground.
[18] The information processing device according to [17], wherein the action generating unit continuously generates one of action to translate the one foreground, and action to rotate the one foreground until error between the one foreground state and the target state becomes small, and then generates the other action.
[19] An information processing method including:
estimating a foreground state of an image using an actual image which is an image to be actually observed; and
updating a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state.
[20] A program causing a computer serving as:
a foreground state estimating unit configured to estimate a foreground state of an image using an actual image which is an image to be actually observed; and a visible model updating unit configured to update a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state.
The present disclosure contains subject matter related to that disclosed in Japanese Priority Patent Application JP 2012-058679 filed in the Japan Patent Office on Mar. 15, 2012, the entire contents of which are hereby incorporated by reference.
It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.

## Claims

1. An information processing device comprising:
one or more processors configured to:
estimate a foreground state of an image using an actual image which is an image to be actually observed;
update a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state;
estimate a motion model of the foreground using action of the foreground state, and state series of most-likely state of the foreground to be obtained from the estimation result of the foreground state; and
generate another foreground action which allows the foreground to be in a predetermined target state using the foreground visible model and the estimated motion model.

2. The information processing device according to claim 1, further comprising:
one or more processors configured to:
update the motion model using an estimation result of the motion model.

3. The information processing device according to claim 1, wherein the foreground visible model includes a background texture model which represents texture of the background;
and wherein the foreground visible model includes
a foreground texture model which represents texture of the foreground, and
a foreground shape model which represents a shape of the foreground;

and wherein the foreground state includes
a position of the foreground,
a speed of change in the position,
an attitude, and
a speed of change in the attitude;

and wherein the foreground motion model includes
a mass of the foreground,
a center-of-gravity of the mass, and
a moment of inertia working around the center-of-gravity;

and wherein the foreground action includes
a force to be applied to the foreground, and
a position to which the force is applied.

4. The information processing device according to claim 1, wherein the background visible model, the foreground visible model, and the motion model are updated in accordance with an Expectation Maximization (EM) algorithm.

5. The information processing device according to claim 1, wherein the background visible model, the foreground visible model, and the motion model are updated in accordance with an Expectation Maximization (EM) algorithm using a particle filter.

6. The information processing device according to claim 5, wherein the estimated foreground state of the image is processed to calculate weighting of a state particle which is a particle which represents the foreground state, and a transition destination of the state particle as the E step of the EM algorithm using the particle filter.

7. The information processing device according to claim 3, wherein the one or more processors are configured to calculate a value corresponding to observation most-likelihood of an observation value to be observed at a generating model which generates an observation value of an image in an event that the foreground state is a state that a state particle represents, as weighting of the state particle, and calculate a transition destination of the state particle assuming that the state particle makes a transition with stochastic fluctuation in accordance with the speed of change in the position and speed of change in the attitude of the foreground state that the state particle represents;
and wherein with the generating model, the observation value of the image observed at the generating model is generated by subjecting, regarding points on an image that the foreground visible model represents, texture that the foreground texture model represents and texture that the background texture model represents to a blending with the foreground shape model as an α value, and employing, regarding points other than the points on the image that the foreground visible model represents, texture that the background texture model represents.

8. The information processing device according to claim 7, wherein the one or more processors are configured to calculate a transition destination of the state particle assuming that the state particle makes the transition further in accordance with the action and the motion model.

9. The information processing device according to claim 3, wherein the one or more processors are configured to perform the M step of an Expectation Maximization (EM) algorithm using the particle filter:
processing to update, in accordance with a state that the state particle represents, the foreground texture model by trimming an image from the actual image and subjecting the trimmed image which is an image trimmed from the actual image to weighting addition in accordance with weighting of the state particle;
processing to update, in accordance with a state that the state particle represents, the background texture model by removing a foreground corresponding to the state from the actual image, and subjects the actual image from which the foreground has been removed to weighting addition in accordance with weighting of the state particle; and
processing to update the foreground shape model by obtaining likelihood that each pixel of texture that the foreground texture model represents will be a foreground, and likelihood that each pixel will be a background, and obtaining probability that each pixel of texture that the foreground texture model represents will be a foreground using the likelihood that each pixel will be the foreground, and the likelihood that each pixel will be a background.

10. The information processing device according to claim 9, wherein the one or more processors are configured to update the foreground texture model by averaging a weighting addition value obtained by subjecting the trimmed image to weighting addition in accordance with weighting of the state particle, for an amount equivalent to a predetermined period of time, and also updates the background texture model by averaging a weighting addition value obtained by subjecting the actual image from which the foreground has been removed to weighting addition in accordance with weighting of the state particle, for an amount equivalent to a predetermined period of time.

11. The information processing device according to claim 9, wherein the one or more processors are configured to update regarding a plurality of foreground visible models, the background texture model by repeating that the plurality of foreground visible models are sequentially taken as a model of interest, and in accordance with a state that the state particle represents, a foreground corresponding to the state is removed from a removed image from which a foreground that a foreground visible model which has been a model of interest immediately before represents has been removed, and the removed image from which the foreground has been removed is subjected to weighting addition in accordance with weighting of the state particle.

12. The information processing device according to claim 5, wherein the one or more processors are configured to perform processing to calculate weighting of a motion model particle which is a particle which represents the motion model, and a transition destination of the motion model particle as the E step of the EM algorithm using the particle filter.

13. The information processing device according to claim 12, wherein the one or more processors are configured to calculate a value corresponding to observation likelihood of state series of the foreground state to be observed at a motion model that the motion model particle represents as weighting of the motion model particle, and calculate a transition destination of the motion model particle assuming that the motion model particle makes a transition with stochastic fluctuation.

14. The information processing device according to claim 13, wherein the one or more processors are configured to:
obtain a most-likely state of the foreground, and
obtain a value corresponding to error between a state series of the foreground state to be generated with the motion model particle, and a state series of the most-likely state of the foreground as observation likelihood of a state series of the foreground state.

15. The information processing device according to claim 12, wherein the one or more processors are configured to perform processing to update the motion model by subjecting a motion model that the motion model particle represents to weighting addition in accordance with weighting of the motion model particle as the M step of the EM algorithm using the particle filter.

16. An information processing device comprising:
one or more processors configured to:
estimate a foreground state of an image using an actual image which is an image to be actually observed;
update a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state;
estimate a motion model of the foreground using action of the foreground state, and state series of most-likely state of the foreground to be obtained from the estimation result of the foreground state;
update the motion model using the estimation result of the motion model,
wherein the foreground visible model includes a background texture model which represents texture of the background;
and wherein the foreground visible model includes
a foreground texture model which represents texture of the foreground, and
a foreground shape model which represents a shape of the foreground;

and wherein the foreground state includes
a position of the foreground,
a speed of change in the position,
an attitude, and
a speed of change in the attitude;

and wherein the foreground motion model includes
a mass of the foreground,
a center-of-gravity of the mass, and
a moment of inertia working around the center-of-gravity;

and wherein the foreground action includes
a force to be applied to the foreground, and
a position to which the force is applied;

wherein the background visible model, the foreground visible model, and the motion model are updated in accordance with an Expectation Maximization (EM) algorithm using a particle filter; and

generate another foreground action which allows one foreground to be in a predetermined target state using the foreground visible model and the motion model.

17. The information processing device according to claim 16, wherein the one or more processors are configured to generate a first action to translate the one foreground, and a second action to rotate the one foreground.

18. The information processing device according to claim 17, wherein the one or more processors are configured to generate one of the first action to translate the one foreground, and the second action to rotate the one foreground continuously until error between the one foreground state and the target state becomes small, and then generate another action.

19. An information processing method comprising:
estimating a foreground state of an image using an actual image which is an image to be actually observed;
updating a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state, wherein at least one of the background visible model and the foreground visible model is updated in accordance with an Expectation Maximization (EM) algorithm using a particle filter; and
generating a foreground action which allows the foreground to be in a predetermined target state using the foreground visible model and a motion model.

20. A non-transitory computer readable medium having stored thereon, a set of computer-executable instructions for causing a computer to perform steps comprising:
estimating a foreground state of an image using an actual image which is an image to be actually observed;
updating a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state;
estimating a motion model of the foreground using action of the foreground state, and state series of most-likely state of the foreground to be obtained from the estimation result of the foreground state; and
generating another foreground action which allows the foreground to be in a predetermined target state using the foreground visible model and the estimated motion model.