Patent Publication Number: US-2023141610-A1

Title: Accuracy prior and diversity prior based future prediction

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application claims the benefit of U.S. Provisional Patent Application, Ser. No. 63/278,208 (Attorney Docket No. H1213860US01) entitled “MULTI-OBJECTIVE DIVERSE HUMAN MOTION PREDICTION WITH KNOWLEDGE DISTILLATION”, filed on Nov. 11, 2021; the entirety of the above-noted application(s) is incorporated by reference herein. 
    
    
     BACKGROUND 
     Research has explored several techniques to enhance diversity and maintain the accuracy of human motion prediction at the same time. However, most of the research needs to define a combined loss, such as the sum of likelihood loss and diversity loss and needs to determine the weights of the loss prior training. Some research has investigated how to increase the diversity of human motion prediction based on deep generative models or diverse sampling techniques. However, these efforts may suffer from mode collapse problems. Other challenges include being forced to choose a hyperparameter during training to balance the likelihood and diversity sampling. 
     Future prediction has been investigated with many different approaches in the computer vision community. At the early stage, several methods without deep learning techniques have been proposed, such as Gaussian process, Hidden Markov Model, and latent variable models. However, such methods may not be suitable for more complicated human motions. Other works are based on a graph neural network (GNN) to capture both the temporal and spatial information, but may fail when the predicted horizon is large. 
     Deep generative models may be used to estimate the data distribution. However, existing models typically need to pre-define a hyperparameter to balance the likelihood and diversity, which may not be desirable. 
     BRIEF DESCRIPTION 
     According to one aspect, a system for accuracy prior and diversity prior based future prediction may include an accuracy prior, a diversity prior, a decoder, an oracle, and a processor. The accuracy prior may generate an accuracy latent representation based on a feature extracted history portion of a time series of information. The diversity prior may generate a diversity latent representation based on the feature extracted history portion. The decoder may generate an accuracy loss based on an accuracy decoder input including the accuracy latent representation. The decoder may generate a preliminary prediction based on a diversity decoder input including the diversity latent representation. The oracle may generate one or more samples based on the preliminary prediction. The processor may generate a diversity loss based on the preliminary prediction, the accuracy loss, and one or more of the samples. The processor may perform training and updating of the diversity prior based on the diversity loss. 
     The decoder may include a recurrent neural network (RNN) and a multi-layer perceptron (MLP). The oracle may generate one or more of the samples based on the preliminary prediction across a time horizon including one or more time steps. The oracle may perform sampling of one or more of the samples based on a k-determinantal point process (k-DPP). The diversity prior may be implemented without the accuracy prior and without the future portion of the time series of information after training of the diversity prior is complete. The oracle may include a variational autoencoder. The time series of information may include a time series of images of poses associated with a moving object. The time series of information may include a time series of images of locations associated with a moving object. The time series of information may include a time series of behaviors associated with an object. The processor may perform training and updating of the accuracy prior based on the accuracy loss. 
     According to one aspect, a computer-implemented method for accuracy prior and diversity prior based future prediction may include generating an accuracy latent representation based on a feature extracted history portion of a time series of information, generating a diversity latent representation based on the feature extracted history portion, generating an accuracy loss based on the accuracy latent representation, generating a preliminary prediction based on the diversity latent representation, generating one or more samples based on the preliminary prediction, generating a diversity loss based on the preliminary prediction, the accuracy loss, and one or more of the samples, and training and updating of a diversity prior based on the diversity loss. 
     The generating the accuracy loss may be based on a recurrent neural network (RNN) and a multi-layer perceptron (MLP). The generating one or more samples may be based on the preliminary prediction across a time horizon including one or more time steps. The computer-implemented method for accuracy prior and diversity prior based future prediction may include sampling of one or more of the samples based on a k-determinantal point process (k-DPP). The computer-implemented method for accuracy prior and diversity prior based future prediction may include implementing the diversity prior without an accuracy prior and without the future portion of the time series of information after training of the diversity prior is complete. The receiving the preliminary prediction and generating one or more samples may be based on a variational autoencoder. 
     According to one aspect, a system for accuracy prior and diversity prior based future prediction may include an accuracy prior, a diversity prior, a decoder, an oracle, and a processor. The accuracy prior may generate an accuracy latent representation based on a feature extracted history portion of a time series of information. The diversity prior may generate a diversity latent representation based on the feature extracted history portion. The decoder may generate a first output based on a first decoder input including the accuracy latent representation. The decoder may generate a second output based on a second, diversity decoder input including the diversity latent representation. The processor may generate an accuracy prior and diversity prior based future prediction based on the first output and the second output. The diversity prior may be trained during a training stage utilizing the accuracy prior. 
     The decoder may include a recurrent neural network (RNN) and a multi-layer perceptron (MLP). The time series of information may include a time series of images of poses associated with a moving object. The time series of information may include a time series of images of locations associated with a moving object. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG.  1    is an exemplary component diagram of a system for accuracy prior and diversity prior based future prediction, according to one aspect. 
         FIG.  2    is an exemplary component diagram of a system for accuracy prior and diversity prior based future prediction implemented during a training phase, according to one aspect. 
         FIG.  3    is an exemplary component diagram of a system for accuracy prior and diversity prior based future prediction implemented during an execution phase, according to one aspect. 
         FIG.  4    is an exemplary component diagram of an oracle for the system for accuracy prior and diversity prior based future prediction of  FIGS.  1 - 2   , according to one aspect. 
         FIG.  5    is an exemplary illustration of the diversity possible during pose prediction, according to one aspect. 
         FIGS.  6 A- 6 B  are exemplary flow diagrams of a method for accuracy prior and diversity prior based future prediction, according to one aspect. 
         FIG.  7    is an illustration of an example computer-readable medium or computer-readable device including processor-executable instructions configured to embody one or more of the provisions set forth herein, according to one aspect. 
         FIG.  8    is an illustration of an example computing environment where one or more of the provisions set forth herein are implemented, according to one aspect. 
     
    
    
     DETAILED DESCRIPTION 
     The following includes definitions of selected terms employed herein. The definitions include various examples and/or forms of components that fall within the scope of a term and that may be used for implementation. The examples are not intended to be limiting. Further, one having ordinary skill in the art will appreciate that the components discussed herein, may be combined, omitted or organized with other components or organized into different architectures. 
     A “processor”, as used herein, processes signals and performs general computing and arithmetic functions. Signals processed by the processor may include digital signals, data signals, computer instructions, processor instructions, messages, a bit, a bit stream, or other means that may be received, transmitted, and/or detected. Generally, the processor may be a variety of various processors including multiple single and multi-core processors and co-processors and other multiple single and multi-core processor and co-processor architectures. The processor may include various modules to execute various functions. 
     A “memory”, as used herein, may include volatile memory and/or non-volatile memory. Non-volatile memory may include, for example, ROM (read only memory), PROM (programmable read only memory), EPROM (erasable PROM), and EEPROM (electrically erasable PROM). Volatile memory may include, for example, RAM (random access memory), synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), and direct RAM bus RAM (DRRAM). The memory may store an operating system that controls or allocates resources of a computing device. 
     A “disk” or “drive”, as used herein, may be a magnetic disk drive, a solid state disk drive, a floppy disk drive, a tape drive, a Zip drive, a flash memory card, and/or a memory stick. Furthermore, the disk may be a CD-ROM (compact disk ROM), a CD recordable drive (CD-R drive), a CD rewritable drive (CD-RW drive), and/or a digital video ROM drive (DVD-ROM). The disk may store an operating system that controls or allocates resources of a computing device. 
     A “bus”, as used herein, refers to an interconnected architecture that is operably connected to other computer components inside a computer or between computers. The bus may transfer data between the computer components. The bus may be a memory bus, a memory controller, a peripheral bus, an external bus, a crossbar switch, and/or a local bus, among others. The bus may also be a vehicle bus that interconnects components inside a vehicle using protocols such as Media Oriented Systems Transport (MOST), Controller Area network (CAN), Local Interconnect Network (LIN), among others. 
     A “database”, as used herein, may refer to a table, a set of tables, and a set of data stores (e.g., disks) and/or methods for accessing and/or manipulating those data stores. 
     An “operable connection”, or a connection by which entities are “operably connected”, is one in which signals, physical communications, and/or logical communications may be sent and/or received. An operable connection may include a wireless interface, a physical interface, a data interface, and/or an electrical interface. 
     A “computer communication”, as used herein, refers to a communication between two or more computing devices (e.g., computer, personal digital assistant, cellular telephone, network device) and may be, for example, a network transfer, a file transfer, an applet transfer, an email, a hypertext transfer protocol (HTTP) transfer, and so on. A computer communication may occur across, for example, a wireless system (e.g., IEEE 802.11), an Ethernet system (e.g., IEEE 802.3), a token ring system (e.g., IEEE 802.5), a local area network (LAN), a wide area network (WAN), a point-to-point system, a circuit switching system, a packet switching system, among others. 
     The aspects discussed herein may be described and implemented in the context of non-transitory computer-readable storage medium storing computer-executable instructions. Non-transitory computer-readable storage media include computer storage media and communication media. For example, flash memory drives, digital versatile discs (DVDs), compact discs (CDs), floppy disks, and tape cassettes. Non-transitory computer-readable storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, modules, or other data. 
     Predicting accurate and diverse accuracy prior and diversity prior based future prediction is useful to many industrial applications, such as robotics and autonomous driving. A prediction framework that may balance likelihood sampling and diversity sampling during the testing phase is presented. A multi-objective conditional variational inference prediction model is defined herein. A short-term oracle may be implemented to encourage the prediction framework to explore more diverse future motions or branches. 
     Several works have attempt to obtain an accurate accuracy prior and diversity prior based future prediction without considering diversity, based on graph neural networks (GNNs) and based on recurrent neural networks (RNNs). Other research has investigated how to increase the diversity of accuracy prior and diversity prior based future prediction based on deep generative models or diverse sampling techniques. Deep generative models, such as variational autoencoder and generative adversarial net-work may naturally capture stochastic behaviors, while suffering from mode collapse problems. However, even if it is assumed that the generative models capture the actual data distribution, this captured data distribution may still be imbalanced and skewed, which makes sampling the minor modes challenging within a limited number of samples. 
     Other works have proposed new losses to increase the diversity meanwhile keeping the prediction naturally and accurate. A multiple sampling function may be designed to explicitly capture the different modes of the distribution based on a pre-trained conditional variational autoencoder. By using this pre-trained variational autoencoder, such methods may control the likelihood of predicted motion with a training hyperparameter. A generative model may learn the distribution implicitly. However, hyperparameter may be selected during training to balance the likelihood and diversity sampling. This selection implies that such approaches cannot be adjusted and controlled during testing. 
     Considering the real application, such as pedestrian motion prediction in autonomous driving, it may be useful to know all the different possible modes of motion, but also to know which modes most likely happen. Further, it may be more practical to decide the balance of likelihood sampling and diversity sampling during the testing phase for the purpose of designing the risk-averse or risk-seeking planner of autonomous vehicles rather than during the training phase. To address this problem, a multi-objective variational inference framework with two different priors is introduced. 
     The proposed structure makes adjusting the ratio between likelihood and diversity sampling during testing time possible. Meanwhile, since there is only one ground-truth future motion trajectory given a historical observation, a similarity-clustered based technique to obtain the multi-modal ground-truth future may be used. Similar initial poses may be grouped, and their corresponding future poses may be viewed as the pseudo-possible future motions for each initial pose in the group. Such logic may be applied recursively. At certain steps, similar poses may be grouped again and the shared futures obtained. This algorithm may boost the diversity of future motions. However, the sampling number may exponentially increase due to the recursive queries during training and make such direct implementation intractable. In order to solve this issue, a variational-autoencoder-based oracle may be introduced, which makes a short-term prediction and provides several possible future motions to supervise the prediction framework. 
       FIG.  1    is an exemplary component diagram of a system  100  for accuracy prior and diversity prior based future prediction, according to one aspect. The system  100  for accuracy prior and diversity prior based future prediction may include a processor  102 , a memory  104 , a disk drive  106 , one or more sensors  110 , a feature extractor  112 , an encoder  120 , an accuracy prior  132 , a diversity prior  134 , a concatenator  136 , a decoder  140 , and an oracle  150 . The accuracy prior  132  and the diversity prior  134  may be functions or neural networks. Together, these components of the system  100  for accuracy prior and diversity prior based future prediction may perform training of a diversity prior  134 , training of an accuracy prior  132 , and may perform accuracy prior and diversity prior based future prediction utilizing an architecture based on the trained diversity prior  134 . 
     The architecture of  FIGS.  1 - 4    may enable accuracy prior and diversity prior based future prediction with accuracy for many diverse scenarios. A multi-objective conditional variational autoencoder based accuracy prior and diversity prior based future prediction framework is discussed herein. This framework may adjust the likelihood and diversity sampling ratio during the testing phases. An oracle  150  or short-term oracle may be learned and the system may distill the oracle&#39;s knowledge into the prediction framework of the system to increase the diversity of accuracy prior and diversity prior based future prediction. In order to achieve such a goal, a sample-based accuracy loss may be introduced. Although described with respect to human motion, the framework of  FIGS.  1 - 4    may be implemented to predict motion, behaviors, trajectories, etc. 
     In this way, the multi-task generative prediction framework which is based on conditional variational inference is introduced. Additionally, the short-term oracle, which provides the multi-modality supervision to the prediction framework may be implemented, along with a training strategy and a testing procedure for the system. 
     Problem Formulation 
     A goal may be to predict the possible accuracy prior and diversity prior based future prediction for a dataset D. A human motion trajectory with time horizon T may be defined as X t:t+T−1 =[X t , X t+1 , . . . , X t+T−1 ], where X t ∈R d  is the human joints Cartesian coordinates at time step t. Given an observation C═X t−T     H     +1:t , the future trajectories&#39; distribution P(X t+1:t+T     f   |C,ρ) may be obtained. Since such conditional probabilistic distribution may have one dominant mode, it may be difficult to sample the other modes given a fixed sampling number. If the diversity of the sampling is increased, the likelihood or prediction accuracy may be influenced. A variable ρ∈[0, 1] may control the degree of diversity of prediction (e.g., diverse samples X t+1:t+T     f     i ˜P(X t+1:t+T     f   |C,ρ), i=1, . . . , n) may be obtained. In other words, the larger ρ becomes, the more diverse samples may be generated and focused on the rarer cases. Conversely, the smaller ρ becomes, the prediction may be more focused on the most likely modes. 
     Multi-Objective Predictor 
     A multi-objective predictor may be implemented by the system  100  for accuracy prior and diversity prior based future prediction and may include the processor  102 , the memory  104 , the disk drive  106 , one or more of the sensors  110 , the feature extractor  112 , the accuracy prior  132 , the diversity prior  134 , the concatenator  136 , and the decoder  140 . 
     According to one aspect, a probabilistic distribution may be represented via a latent variable model: 
         P ( X|C,Q )= E   Z˜Q(Z|C) [ P ( X|C,Z )]  (1)
 
     where Q(Z|C) may be a conditional prior distribution of latent variable Z E R d     z    whose dimension is d z , and P(X|C, Z) may be defined as the conditional likelihood given the observation information C and latent variable Z. The prior distribution may be varied to achieve different distribution of X given the same observation C. In the system, two different prior distributions Q acc (Z|C) and Q div (Z|C) may be introduced. To obtain the most similar distribution with the data distribution P D  sampling from Q acc (Z|C) may be performed, and obtain the most diverse distribution which mainly focuses on the minor modes, sampling from Q div (Z|C) may be performed. The historical observation encoder and future information encoder may be defined as: 
         e   h ( C )=[ MLP∘RNN ]( C ), 
         e   f ( X )=[ MLP∘RNN ]( X )  (2)
 
     where the temporal information of trajectories may be first encoded by using a recurrent neural network (RNN) and then using a forward neural network to map the states of RNN to the trajectory space R d·T     f   . Here d may be the dimension of joints Cartesian coordinates and T f  may be the prediction horizon. The decoder function d θ (X|C, Z) may be defined as: 
         d   θ ( X|C,Z )=[ MLP∘RNN ]( e   h ( C )∥ Z )  (3)
 
     where θ is the parameter of the decoder  140 , “∥” may represent the concatenate operator of two vectors. Similar neural network structure may be utilized for the decoder  140  as for the encoders. 
     Accuracy Sampler 
     An accuracy sampler may be implemented by the system  100  for accuracy prior and diversity prior based future prediction and may include the processor  102 , the memory  104 , the disk drive  106 , and the accuracy prior  132 . 
     A first task may be to inference the accuracy prior distribution Q acc (Z|C). The data distribution may be approximated by sampling from the accuracy prior distribution. Hence, the variational inference may be applied, which may minimize the evidence lower bound of the likelihood: 
       £ ELBO   =E   Qψ(Z|X,C) [log  P   θ ( X|Z,C )]− D   KL [ Q   ψ ( Z|X,C )∥ Q   acc ( Z|C )]  (4)
 
     where Qψ(Z|X, C) may be a posterior distribution of latent variable Z given the historical observation and future information. Here, Qacc(Z|C) may be modeled as a Gaussian distribution N(μ ϕ     acc   (C), Σ ϕ     acc   (C)). Therefore, the KLD KL [Q ψ ∥Q acc ] divergence may become: 
     
       
         
           
             
               
                 
                   
                     1 
                     2 
                   
                   [ 
                   
                     
                       log 
                       ⁢ 
                       
                         
                           
                             ∑ 
                             
                               
                                 ϕ 
                                   
                               
                               acc 
                             
                           
                           
                             ∑ 
                             ψ 
                           
                         
                         
                           - 
                           
                             d 
                             z 
                           
                         
                       
                     
                     + 
                     
                       
                         
                           T 
                           r 
                         
                         ( 
                         
                           
                             ∑ 
                             
                               ϕ 
                               acc 
                             
                             
                               - 
                               1 
                             
                           
                           
                             ∑ 
                             ψ 
                           
                         
                         ) 
                       
                       ⁢ 
                       
                         
                            
                           
                             
                               µ 
                               
                                 ϕ 
                                 acc 
                               
                             
                             - 
                             
                               µ 
                               ψ 
                             
                           
                            
                         
                         
                           ∑ 
                           
                             ϕ 
                             acc 
                           
                           
                             - 
                             1 
                           
                         
                         2 
                       
                     
                   
                   ] 
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
     However, since there may be no control over the distribution Q acc (Z|C), it could be arbitrarily distributed, thereby increasing the difficulty of training. Some research suggests investigating the collapse problems for conditional variational inference and that using a universal prior distribution, e.g., an isotropic Gaussian distribution, may not be a good choice for conditional distribution estimation since it is difficult to capture complex conditional multimodal data and introduce strong model bias resulting in missing modes. In order to constrain the prior distribution, a best-of-many loss may be implemented as the regularization the model: 
     
       
         
           
             
               
                 
                   
                     £ 
                     
                       Q 
                       acc 
                     
                   
                   = 
                   
                     
                       
                         
                           min 
                         
                       
                       
                         
                           i 
                         
                       
                     
                     ⁢ 
                     
                       
                          
                         
                           
                             X 
                             i 
                           
                           - 
                           X 
                         
                          
                       
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
       
         
           
             
               z 
               i 
             
             ~ 
             
               Q 
               ⁡ 
               ( 
               
                 Z 
                 ❘ 
                 C 
               
               ) 
             
           
         
       
       
         
           
             
               
                 X 
                 i 
               
               = 
               
                 
                   d 
                   θ 
                 
                 ( 
                 
                   
                     X 
                     ❘ 
                     C 
                   
                   , 
                   
                     z 
                     i 
                   
                 
                 ) 
               
             
             , 
             
               i 
               = 
               1 
             
             , 
             … 
                 
             , 
             n 
           
         
       
     
     where n is the number of samples. Thus, the overall loss for the accuracy sampler may be: 
       £ acc (θ,ψ)=λ elbo £ ELBO +λ acc £ Q     acc     (7)
 
     where λ elbo  and λ acc  may be used to balance two losses. 
     Diversity Sampler 
     A diversity sampler may be implemented by the system  100  for accuracy prior and diversity prior based future prediction and may include the processor  102 , the memory  104 , the disk drive  106 , and the diversity prior  134 . 
     In order to explore the minor modes of possible future trajectories, the system may learn another prior distribution Q div (Z|C). A diversity definition may be: 
     
       
         
           
             
               DIV 
               ⁡ 
               ( 
               
                 X 
                 , 
                 Y 
               
               ) 
             
             = 
             
               
                 
                   
                     1 
                   
                 
                 
                   
                     
                       
                         N 
                         x 
                       
                       ⁢ 
                       
                         N 
                         y 
                       
                     
                   
                 
               
               ⁢ 
               
                 
                   ∑ 
                   
                     i 
                     , 
                     j 
                   
                 
                 
                   e 
                   
                     - 
                     
                       d 
                       ⁡ 
                       ( 
                       
                         
                           X 
                           i 
                         
                         , 
                         
                           Y 
                           j 
                         
                       
                       ) 
                     
                   
                 
               
             
           
         
       
       
         
           
             
               
                 
                   
                     X 
                     i 
                   
                   , 
                   
                     
                       Y 
                       j 
                     
                     ∈ 
                     X 
                   
                   , 
                   Y 
                   , 
                   
                     i 
                     = 
                     1 
                   
                   , 
                   … 
                       
                   , 
                   
                     N 
                     x 
                   
                   , 
                   
                     j 
                     = 
                     1 
                   
                   , 
                   … 
                       
                   , 
                   
                     N 
                     y 
                   
                 
               
               
                 
                   
                     ( 
                     8 
                     ) 
                   
                 
               
             
           
         
       
     
     where X and Y represent two sets of samples with size N x  and N y ·d(⋅,⋅) may be a metric defined in the Euclidean space. Here, the metric may be defined as d(x,y)=η∥x−y∥2, where η may be a parameter to adjust the sensitivity of diversity. The set of the samples which are generated by the major modes sampler may be denoted as X acc , and the set of samples generated by the diversity samplers may be denoted as X div . The diversity loss may be defined as: 
       £ div =α div   DIV ( X   div   ,X   div )+(1−α div ) DIV ( X   div   ,X   acc )  (9)
 
     where DIV(X div , X div ) may represent the diversity of samples generated by the diversity sampler and DIV(X div , X acc ) may represent the average pairwise distance between the samples from the accuracy sampler and the diversity sampler. The system may evaluate the differences between samples from the accuracy sampler and the diversity sampler. When the weight of diversity loss is large, this may have a negative influence on the accuracy sampler to approximate the data distribution. A goal or intention of the system may be to disentangle the accuracy objective and diversity objective, and thus, the pairwise distances between samples from the diversity sampler may be increased using the first term in Equation (9). Samples from the diversity sampler may be made dissimilar to the samples from the accuracy sampler using the second term in Equation (9). The relative importance of the two items in Equation (9) may be determined by a weight α div . A larger α div  may mean to focus on making the samples from Q div  more different. 
     Only using the diversity loss may not be enough to obtain realistic prediction since it not only increases the differences between the samples, meanwhile, it may also make the sampler generate arbitrarily unrealistic poses. Hence, human motion from the data may be used to constrain the prediction. In order to constrain each generated trajectories, it may be assumed that there exists an oracle  150 , defined as: 
         {tilde over (X)}   t+1:t+τ   ˜O ( X   t ,τ)  (10)
 
     where O(X t ) may be the probabilistic distribution of future distribution with horizon τ given the current initial pose X t . The oracle  150  may serves as a guide to provide supervision of the predictor. A sample-based loss may be defined as: 
     
       
         
           
             
               
                 
                   
                     
                       
                         £ 
                         ref 
                       
                       ( 
                       τ 
                       ) 
                     
                     = 
                     
                       
                         1 
                         n 
                       
                       ⁢ 
                       
                         
                           ∑ 
                           
                             i 
                             = 
                             1 
                           
                           n 
                         
                         
                           
                             ∑ 
                             
                               t 
                               = 
                               0 
                             
                             
                               
                                 [ 
                                 
                                   T 
                                   r 
                                 
                                 ] 
                               
                               - 
                               1 
                             
                           
                           
                             
                               
                                 
                                   min 
                                 
                               
                               
                                 
                                   j 
                                 
                               
                             
                             ⁢ 
                             
                               
                                  
                                 
                                   
                                     
                                       X 
                                       ~ 
                                     
                                     
                                       tr 
                                       : 
                                           
                                       
                                         ( 
                                         
                                           t 
                                           + 
                                           1 
                                         
                                         ) 
                                       
                                       ⁢ 
                                       r 
                                     
                                     i 
                                   
                                   - 
                                   
                                     
                                       X 
                                       ~ 
                                     
                                     
                                       tr 
                                       - 
                                       
                                         
                                           ( 
                                           
                                             t 
                                             + 
                                             1 
                                           
                                           ) 
                                         
                                         ⁢ 
                                         r 
                                       
                                     
                                     j 
                                   
                                 
                                  
                               
                               2 
                             
                           
                         
                       
                     
                   
                   , 
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
       
         
           
             s 
             . 
             t 
             . 
                
             
               
                 Z 
                 i 
               
               ~ 
               
                 
                   Q 
                   div 
                 
                 ( 
                 
                   Z 
                   ❘ 
                   C 
                 
                 ) 
               
             
           
         
       
       
         
           
             
               
                 
                   X 
                   ^ 
                 
                 
                   1 
                   : 
                   T 
                 
                 i 
               
               = 
               
                 d 
                 ⁡ 
                 ( 
                 
                   C 
                   , 
                   
                     Z 
                     i 
                   
                 
                 ) 
               
             
             , 
           
         
       
       
         
           
             
               
                 
                   X 
                   ~ 
                 
                 
                   tr 
                   : 
                       
                   
                     ( 
                     
                       t 
                       + 
                       1 
                     
                     ) 
                   
                   ⁢ 
                   τ 
                 
                 j 
               
               ~ 
               
                 O 
                 ⁡ 
                 ( 
                 
                   
                     X 
                     ~ 
                   
                   
                     
                       t 
                       ⁢ 
                       τ 
                     
                     - 
                     1 
                   
                 
                 ) 
               
             
             , 
           
         
       
       
         
           
             
               i 
               = 
               1 
             
             , 
             … 
                 
             , 
             
               n 
               div 
             
             , 
             
               j 
               = 
               1 
             
             , 
             … 
                 
             , 
             
               n 
               o 
             
             , 
             
               
                 ∀ 
                 t 
               
               = 
               0 
             
             , 
             … 
                 
             , 
             
               T 
               / 
               τ 
             
           
         
       
     
     where τ may represent the time horizon of predicted poses from the oracle  150 , n acc  may be the number of samples of predictor, and n o  may be the number of samples which the oracle  150  provides. Given one sample {circumflex over (X)} tτ:(t+1)τ   i , the oracle  150  may provide several possible short-term futures {tilde over (X)} tτ:(t+1)τ   j , given the current predicted pose {circumflex over (X)} tτ−1   i . It may be desirable for the subsequence {circumflex over (X)} tτ(t+1)r   i  to be similar to one of the provided futures {tilde over (X)} tτ(t+1)r   j . The diversity loss defined in Equation (9) may encourage the predictor to choose which provided accuracy prior and diversity prior based future prediction is useful to increase the diversity of all the samples. Physical constraints of predicted poses may be adopted, such as the angle likelihood, the limbs&#39; length, and velocity to make the prediction from the diversity sampler more realistic: 
       £ pose =λ vel £ vel +λ angle £ angle +λ limb £ limb   (12)
 
     where the velocity loss defined as the average difference between each two successive poses may be: 
     
       
         
           
             
               
                 
                   
                     
                       £ 
                       vel 
                     
                     ( 
                     X 
                     ) 
                   
                   = 
                   
                     
                       1 
                       T 
                     
                     ⁢ 
                     
                       
                         ∑ 
                         
                           t 
                           = 
                           0 
                         
                         
                           T 
                           - 
                           1 
                         
                       
                       
                         
                            
                           
                             
                               X 
                               
                                 t 
                                 + 
                                 1 
                               
                             
                             - 
                             
                               X 
                               t 
                             
                           
                            
                         
                         2 
                       
                     
                   
                 
               
               
                 
                   ( 
                   13 
                   ) 
                 
               
             
           
         
       
     
     and the pose loss is defined as the log-likelihood of the angles respect to the origin may be: 
       £ ang ( X )=log  P ( n )  (14)
 
     where the likelihood may be approximated by a normalizing flow. n may be the normalized direction of each joint. The predicted limbs&#39; length  1  should be same as the ground truth, as enforced: 
     
       
         
           
             
               
                 
                   
                     
                       £ 
                       limb 
                     
                     ( 
                     X 
                     ) 
                   
                   = 
                   
                     
                       1 
                       
                         n 
                         l 
                       
                     
                     ⁢ 
                     
                       
                         ∑ 
                         j 
                       
                       
                         
                            
                           
                             
                               
                                 I 
                                 ^ 
                               
                               j 
                             
                             - 
                             
                               I 
                               j 
                             
                           
                            
                         
                         2 
                       
                     
                   
                 
               
               
                 
                   ( 
                   15 
                   ) 
                 
               
             
           
         
       
     
     Therefore, an overall loss for the diversity sampler may be: 
       £ D =λ ref £ ref +λ div £ div +£ ang   (16)
 
     Oracle 
     The short-term oracle O(X) may be learned by using another conditional variational autoencoder to capture the ground-truth multi-modality. This method may also be used as the multimodality evaluation metrics. In the system, the following may be defined: 
       Ω( X   t )= S ( X;τ,K )
 
         X   o   ={X   t+1,t+r   j    . . . X   t+1:t+τ   |X| } 
         d ( X   t   j   ,X   t )≤δ,∀ j∈|X   o |  (17)
 
     where X o  may represent the set of all future poses whose corresponding initial poses X t   j  are in a ball with radius δ which centered at the given initial pose X t . The ball may be defined by metric d(⋅,⋅). Ω(X t ) may represent the set of K selected future poses which has time horizon T given the initial pose X t   i . Since there might be many similar poses to the given initial poses and most of the corresponding future poses may be very similar, a proper fixed number of future poses may be selected in X in order to capture the different modes. A k-determinantal point process (k-DPP) may be used as the selection function S to select the future poses. 
     k-Determinantal Point Process (k-DPP) 
     The k-determinantal point process may be used to sample the diverse points given a fixed number of samples. Given a set X={X 1 , X 2 , . . . , X n }, a k-determinantal point process P defined on the set X may be a probability measure on 2 X : 
     
       
         
           
             
               
                 
                   
                     Pr 
                     ⁡ 
                     ( 
                     S 
                     ) 
                   
                   = 
                   
                     
                       
                         det 
                         ⁡ 
                         ( 
                         
                           L 
                           S 
                         
                         ) 
                       
                       ⁢ 
                       1 
                       ⁢ 
                       
                         ( 
                         
                           
                             ❘ 
                             &#34;\[LeftBracketingBar]&#34; 
                           
                           
                             S 
                             = 
                             k 
                           
                           
                             ❘ 
                             &#34;\[RightBracketingBar]&#34; 
                           
                         
                         ) 
                       
                     
                     
                       
                         
                           ∑ 
                           S 
                         
                         ⊂ 
                         X 
                       
                       , 
                       
                         
                           
                             ❘ 
                             &#34;\[LeftBracketingBar]&#34; 
                           
                           S 
                           
                             ❘ 
                             &#34;\[RightBracketingBar]&#34; 
                           
                         
                         = 
                         
                           k 
                           ^ 
                           
                             det 
                             ⁡ 
                             ( 
                             
                               L 
                               S 
                             
                             ) 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   18 
                   ) 
                 
               
             
           
         
       
     
     where S may be denoted as a subset of X and L S ∈R |S|×|S| as the similarity matrix: 
       { L   s } ij = e   −d(X     t       i     +1:t+τ,X     t       j     +1:t+r)   (19)
 
     The training data may be pre-processed to augment each cases with K futures poses. 
     Oracle Neural Network 
     After obtaining the augmented datasets, a variational autoencoder which is similar to the accuracy sampler defined above may be used to capture the multi-modality. Since the augmented data may be relatively balanced by the k-determinantal point process, there are less extremely minor modes and hence mitigation of trouble of rare-case sampling is provided. 
     Training Algorithm 
     Input:
         N: number of epochs   n acc : number of samples for major modes sampler   n div : number of samples for diversity sampler   n o : number of samples for oracles       

     Output:
         θ, ϕ acc , ϕ div          

     Data:
         training dataset D train          

     Training Algorithm
         Σ i=1   ∞ :=0 while epoch ≤N do
           Sample B={X i , C i } i ˜D train      foreach X, C∈B do
               Generate n acc  samples:   {circumflex over (X)} acc   i =d(C i , Z i ), Z i ˜Q acc (Z|C)   Generate n div  samples   {circumflex over (X)} div   i =d(C i , Z i ), Z i ˜Q div (Z|C)   for t=0, . . . , T f /τ do
                   Generate n o  samples:   {tilde over (X)} tτ:t(τ+1)   i ˜O({circumflex over (X)} tτ−1 )   
                   
               Update θ, ψ, ϕ acc  with £ acc ({X}, {{circumflex over (X)} acc })   Update θ, ϕ div  with £ div ({X}, {{circumflex over (X)} div }, {{tilde over (X)}})   
               

     Training Phase 
     One or more of the sensors  110  may detect, sense, or observe a time series set of information. One or more of the sensors  110  may include image capture devices, charge-coupled device (CCD), or active-pixel sensors fabricated using complementary MOS (CMOS) or N-type MOS (e.g., NMOS or live MOS) technologies. In any event, the sensor may detect the time series of information. Examples of the time series of information may include a time series of images of poses associated with a moving object, a time series of images of locations associated with a moving object (e.g., a trajectory of the object), a time series of behaviors associated with an object. 
     The feature extractor  112  may perform feature extraction on the time series of information. During the training phase, the time series of information may be treated as training data and be divided into multiple portions, such as a history portion and a future portion. For example, the time series of information may be broken into or divided into a training history portion and a training future portion, which may serve as ground-truth during the training phase. This division may be performed because the future may be very diverse. The feature extractor  112  may perform feature extraction on these portions. For example, the feature extractor  112  may perform feature extraction on the training history portion to generate a feature extracted history portion. Similarly, the feature extractor  112  may perform feature extraction on the training future portion to generate a feature extracted future portion. 
     The encoder  120  may receive the feature extracted future portion and generate an encoded feature extracted future portion based on the feature extracted future portion. 
     The accuracy prior  132  may receive the feature extracted history portion of a time series of information and generate an accuracy latent representation based on the feature extracted history portion. The accuracy latent representation may be a low dimensional representation or a latent variable. 
     The diversity prior  134  may receive the feature extracted history portion of the time series of information and generate a diversity latent representation based on the feature extracted history portion. The diversity latent representation may be a low dimensional representation or a latent variable. 
     Unlike most prediction systems and techniques, the use of two different, distinct priors (e.g., the accuracy prior  132  and the diversity prior  134 ) within the framework of the system  100  for accuracy prior and diversity prior based future prediction enables a more accurate prediction. 
     The concatenator  136  may concatenate the accuracy latent representation and an encoded feature extracted future portion of the time series of information to generate an accuracy decoder input. The concatenator  136  may concatenate the diversity latent representation and the feature extracted history portion to generate a diversity decoder input. 
     The decoder  140  may receive the accuracy decoder input and the diversity decoder input, generate an accuracy loss based on the accuracy decoder input, and generate a preliminary prediction based on the diversity decoder input. The decoder  140  may include a recurrent neural network (RNN) and a multi-layer perceptron (MLP). 
     The oracle  150  may be a short-term oracle and may receive the preliminary prediction and generate one or more samples based on the preliminary prediction. The oracle  150  may generate one or more samples based on the preliminary prediction associated with a time across a time horizon including one or more time steps. The oracle  150  is described in greater detail herein at  FIG.  4   . 
     The oracle  150  may include a variational autoencoder and may learn short-term oracle data. The short-term oracle data may be oracle knowledge associated with the prediction framework to increase the diversity of future time series of information, such as future motion. The oracle  150  may sample of one or more of the samples based on a k-determinantal point process (k-DPP). 
     The processor  102  may generate a diversity loss based on the preliminary prediction, the accuracy loss, and one or more of the samples for a given time horizon. For example, the diversity loss may be calculated at least in part based on a comparison between one or more of the samples and the preliminary prediction. This process may be repeated across one or more times steps. For example, after the first time step, a second preliminary prediction may be provided to the oracle  150 , which may one or more second samples based on the second preliminary prediction associated with the second time step across the time horizon. 
     The processor  102  may perform training and updating of the diversity prior  134  based on the diversity loss. Thus, the oracle  150  may supervise the training of the diversity prior  134  because the diversity prior  134  may be trained based on the diversity loss and because the diversity loss is determined based on one or more of the samples from the oracle  150 . The processor  102  may perform training and updating of the accuracy prior  132  based on the accuracy loss. In this way, the processor  102  may train two different, distinct prior functions (e.g., diversity prior  134  and accuracy prior  132 ). 
     Testing Procedure 
     Input:
         ρ: the ratio of sampling numbers between Q acc  and Q div      M: the total number of samples       

     Output:
         {circumflex over (X)} i , the predicted poses       

     Data:
         Testing Dataset: D test          

     Testing Procedure
         foreach X, C∈D test  do   Generate ρM samples from Q acc      Generate (1−ρ)M samples from Q div          

     Execution Phase 
     During the execution phase, after training of the diversity prior  134  is complete, the diversity prior  134  may be implemented without the accuracy prior  132  and without the future portion of the time series of information. According to one aspect, the architecture associated with the execution phase may be included within the architecture of the training phase. In other words, the architecture associated with execution may be a subset of the architecture associated with training. 
     One or more of the sensors  110  may detect the time series set of information during execution and the feature extractor  112  may perform feature extraction on the time series of information. Unlike the training phase, the time series of information may be treated as execution data and is not divided into multiple portions. In this regard, the time series of information may merely include an execution history portion. In any event, the feature extractor  112  may perform feature extraction on the execution history portion. 
     As previously indicated, the diversity prior  134  may be trained during the training phase or training stage utilizing the accuracy prior  132  distinct from the diversity prior  134 . During the execution phase, this pre-trained diversity prior  134  may receive a feature extracted history portion of a time series of information and generate a diversity latent representation based on the feature extracted history portion. 
     During the execution phase, the concatenator  136  may concatenate the feature extracted history portion and the feature extracted history portion to generate a first decoder input. Additionally, the concatenator  136  may concatenate the diversity latent representation and the feature extracted history portion to generate a second, diversity decoder input. 
     During the execution phase, the decoder  140  may receive the first decoder input and the second, diversity decoder input, generate a first output based on the first decoder input, and generate a second output based on the second, diversity decoder input. Thereafter, the processor  102  may generate an accuracy prior and diversity prior based future prediction based on the first output and the second output. 
       FIG.  2    is an exemplary component diagram of a system  100  for accuracy prior and diversity prior based future prediction implemented during a training phase, such as a phase for training and/or testing the diversity prior  134  and/or the accuracy prior  132 , according to one aspect. The architectures of  FIGS.  2 - 4    may include a multi-objective conditional variational encoder based framework.  FIG.  2    may be directed toward training of the diversity prior  134 .  FIG.  3    may be directed toward using the trained diversity prior  134 .  FIG.  4    includes a detailed component diagram of the oracle  150  for the system  100  for accuracy prior and diversity prior based future prediction of  FIGS.  1 - 2   . 
       FIG.  3    is an exemplary component diagram of a system  100  for accuracy prior and diversity prior based future prediction implemented during an execution phase, such as a phase for executing the accuracy prior and diversity prior based future prediction based on the trained diversity prior  134 , according to one aspect. It should be noted that the architecture of  FIG.  3    is included within the architecture of  FIG.  2   . In other words, the architecture associated with execution may be a subset of the architecture associated with training. 
       FIG.  4    is an exemplary component diagram of an oracle  150  for the system  100  for accuracy prior and diversity prior based future prediction of  FIGS.  1 - 2   , according to one aspect. During training, several predicted motions may be obtained. For each motion (e.g., indicated by the Sample i arrow), the poses may be fed to oracle  150  after each time step τ. The oracle  150  may provide several possible future poses as one or more options (e.g., one or more of the samples generated). The predicted future motions (e.g., the preliminary prediction along the Sample i arrow) may be similar with one or more of the options or samples generated by the oracle  150  within each short time horizon. In the example depicted in  FIG.  4   , the time horizon may be 2τ, and thus, include two time steps τ. 
       FIG.  5    is an exemplary illustration of the diversity possible during pose prediction, according to one aspect.  FIG.  5    depicts an illustration associated with receiving multi-modal future motion from a dataset. It may be possible for the processor  102  or the system to cluster one or more similar initial poses (e.g., first time group similar pose dashed circle) and share their future poses as common ground-truth data. According to one aspect, poses of the middle row may be ground-truth poses and poses from the other rows may be augmented poses. This approach may be applied recursively (e.g., second time group similar pose dashed circle), which may lead to additional, different modes of motions. 
       FIGS.  6 A- 6 B  are exemplary flow diagrams of a method  600  for accuracy prior and diversity prior based future prediction, according to one aspect. The method  600  for accuracy prior and diversity prior based future prediction may be computer-implemented and may be implemented via a processor, a memory, a disk drive or storage drive, etc. The method  600  for accuracy prior and diversity prior based future prediction may include a method  610  for training and testing an accuracy prior and a diversity prior and a method  650  for executing the accuracy prior and diversity prior based future prediction based on the trained diversity prior. 
     According to one aspect depicted in  FIG.  6 A , the method  610  for training and testing the accuracy prior and the diversity prior may include generating  612  an accuracy latent representation based on a training feature extracted history portion, generating  614  a diversity latent representation based on the training feature extracted history portion, concatenating  616  the accuracy latent representation and a training encoded feature extracted future portion of time series of information to generate an accuracy decoder input, concatenating  618  the diversity latent representation and the feature extracted history portion to generate a diversity decoder input, generating  620  an accuracy loss based on the accuracy decoder input, generating  622  a preliminary prediction based on the diversity decoder input, generating  624  one or more samples based on the preliminary prediction, generating  626  a diversity loss based on the preliminary prediction, the accuracy loss, and one or more of the samples, and training  628  and updating the diversity prior based on the diversity loss. 
     According to one aspect depicted in  FIG.  6 B , the method  650  for executing the accuracy prior and diversity prior based future prediction based on the trained diversity prior and accuracy prior may include generating  652  a diversity latent representation based on a feature extracted history portion, concatenating  654  the feature extracted history portion and the feature extracted history portion to generate a first decoder input, concatenating  656  the diversity latent representation and the feature extracted history portion to generate a second decoder input, receiving  658  the first decoder input and the second decoder input, generating  660  the first output based on the first decoder input, generating  662  the second output based on the second decoder input, and generating  664  a accuracy prior and diversity prior based future prediction based on the first output and the second output. 
     Still another aspect involves a computer-readable medium including processor-executable instructions configured to implement one aspect of the techniques presented herein. An aspect of a computer-readable medium or a computer-readable device devised in these ways is illustrated in  FIG.  7   , wherein an implementation  700  includes a computer-readable medium  708 , such as a CD-R, DVD-R, flash drive, a platter of a hard disk drive, etc., on which is encoded computer-readable data  706 . This encoded computer-readable data  706 , such as binary data including a plurality of zero&#39;s and one&#39;s as shown in  706 , in turn includes a set of processor-executable computer instructions  704  configured to operate according to one or more of the principles set forth herein. In this implementation  700 , the processor-executable computer instructions  704  may be configured to perform a method  702 , such as the method  600  of  FIG.  6   . In another aspect, the processor-executable computer instructions  704  may be configured to implement a system, such as the system  100  of  FIG.  1   , the system  200  of  FIG.  2   , or the system  300  of  FIG.  3   . Many such computer-readable media may be devised by those of ordinary skill in the art that are configured to operate in accordance with the techniques presented herein. 
     As used in this application, the terms “component”, “module,” “system”, “interface”, and the like are generally intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processing unit, an object, an executable, a thread of execution, a program, or a computer. By way of illustration, both an application running on a controller and the controller may be a component. One or more components residing within a process or thread of execution and a component may be localized on one computer or distributed between two or more computers. 
     Further, the claimed subject matter is implemented as a method, apparatus, or article of manufacture using standard programming or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media. Of course, many modifications may be made to this configuration without departing from the scope or spirit of the claimed subject matter. 
       FIG.  8    and the following discussion provide a description of a suitable computing environment to implement aspects of one or more of the provisions set forth herein. The operating environment of  FIG.  8    is merely one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality of the operating environment. Example computing devices include, but are not limited to, personal computers, server computers, hand-held or laptop devices, mobile devices, such as mobile phones, Personal Digital Assistants (PDAs), media players, and the like, multiprocessor systems, consumer electronics, mini computers, mainframe computers, distributed computing environments that include any of the above systems or devices, etc. 
     Generally, aspects are described in the general context of “computer readable instructions” being executed by one or more computing devices. Computer readable instructions may be distributed via computer readable media as will be discussed below. Computer readable instructions may be implemented as program modules, such as functions, objects, Application Programming Interfaces (APIs), data structures, and the like, that perform one or more tasks or implement one or more abstract data types. Typically, the functionality of the computer readable instructions are combined or distributed as desired in various environments. 
       FIG.  8    illustrates a system  800  including a computing device  812  configured to implement one aspect provided herein. In one configuration, the computing device  812  includes at least one processing unit  816  and memory  818 . Depending on the exact configuration and type of computing device, memory  818  may be volatile, such as RAM, non-volatile, such as ROM, flash memory, etc., or a combination of the two. This configuration is illustrated in  FIG.  8    by dashed line  814 . 
     In other aspects, the computing device  812  includes additional features or functionality. For example, the computing device  812  may include additional storage such as removable storage or non-removable storage, including, but not limited to, magnetic storage, optical storage, etc. Such additional storage is illustrated in  FIG.  8    by storage  820 . In one aspect, computer readable instructions to implement one aspect provided herein are in storage  820 . Storage  820  may store other computer readable instructions to implement an operating system, an application program, etc. Computer readable instructions may be loaded in memory  818  for execution by the processing unit  816 , for example. 
     The term “computer readable media” as used herein includes computer storage media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions or other data. Memory  818  and storage  820  are examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which may be used to store the desired information and which may be accessed by the computing device  812 . Any such computer storage media is part of the computing device  812 . 
     The term “computer readable media” includes communication media. Communication media typically embodies computer readable instructions or other data in a “modulated data signal” such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” includes a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. 
     The computing device  812  includes input device(s)  824  such as keyboard, mouse, pen, voice input device, touch input device, infrared cameras, video input devices, or any other input device. Output device(s)  822  such as one or more displays, speakers, printers, or any other output device may be included with the computing device  812 . Input device(s)  824  and output device(s)  822  may be connected to the computing device  812  via a wired connection, wireless connection, or any combination thereof. In one aspect, an input device or an output device from another computing device may be used as input device(s)  824  or output device(s)  822  for the computing device  812 . The computing device  812  may include communication connection(s)  826  to facilitate communications with one or more other devices  830 , such as through network  828 , for example. 
     Although the subject matter has been described in language specific to structural features or methodological acts, it is to be understood that the subject matter of the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example aspects. 
     Various operations of aspects are provided herein. The order in which one or more or all of the operations are described should not be construed as to imply that these operations are necessarily order dependent. Alternative ordering will be appreciated based on this description. Further, not all operations may necessarily be present in each aspect provided herein. 
     As used in this application, “or” is intended to mean an inclusive “or” rather than an exclusive “or”. Further, an inclusive “or” may include any combination thereof (e.g., A, B, or any combination thereof). In addition, “a” and “an” as used in this application are generally construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form. Additionally, at least one of A and B and/or the like generally means A or B or both A and B. Further, to the extent that “includes”, “having”, “has”, “with”, or variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term “comprising”. 
     Further, unless specified otherwise, “first”, “second”, or the like are not intended to imply a temporal aspect, a spatial aspect, an ordering, etc. Rather, such terms are merely used as identifiers, names, etc. for features, elements, items, etc. For example, a first channel and a second channel generally correspond to channel A and channel B or two different or two identical channels or the same channel. Additionally, “comprising”, “comprises”, “including”, “includes”, or the like generally means comprising or including, but not limited to. 
     It will be appreciated that various of the above-disclosed and other features and functions, or alternatives or varieties thereof, may be desirably combined into many other different systems or applications. Also, that various presently unforeseen or unanticipated alternatives, modifications, variations or improvements therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the following claims.