Abstract:
Disclosed are a system and method for constructing and using a predictive model to generate a prediction signal, also referred to as a classification signal when the signal indicates one of a plurality of distinct classes. In various embodiments, the disclosed technique reduces a size of a predictive Support Vector Model by extracting certain values beforehand and storing only weighting values. The technique does not sacrifice generalization performance but does significantly reduce the model size and accelerate prediction performance. The described system applies to most kernel functions, whether linear or nonlinear.

Description:
RELATED APPLICATION 
       [0001]    The present application claims priority to U.S. Provisional Patent Application 61/857,285, filed on Jul. 23, 2013, which is incorporated herein by reference in its entirety. 
     
    
     TECHNICAL FIELD 
       [0002]    The present disclosure is related generally to prediction based on a predictive model and, more particularly, to a system and method for providing a compact prediction model. 
       BACKGROUND 
       [0003]    Modern electronic devices such as cellular telephones, smart phones, tablets, and so on are increasingly capable of customizing their behavior to accommodate particular environments or situations. For example, a device that is being used while in a moving vehicle may behave differently than a device that is being used in a stationary location. However, such devices seldom have direct knowledge of their usage environment, and as such, the environment may need to be inferred or predicted based on other factors. 
         [0004]    While current prediction models may allow some predictions to be made, such models are generally insufficient. For example, while it may be possible to efficiently encode models to solve linear problems, e.g., via linear regression, there is no suitable general solution to efficiently encode models to be used at the prediction phase in both linear and nonlinear systems. 
         [0005]    The present disclosure is directed to a system that may exhibit numerous distinctions or advantages over prior systems. However, it should be appreciated that any particular distinction or advantage is not a limitation on the scope of the disclosed principles nor of the attached claims, except to the extent expressly noted. Additionally, the discussion of any problem in this Background section is not an indication that the problem represents known prior art. 
     
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
         [0006]    While the appended claims set forth the features of the present techniques with particularity, these techniques, together with their objects and advantages, may be best understood from the following detailed description taken in conjunction with the accompanying drawings of which: 
           [0007]      FIG. 1  is a schematic illustration of an example device within which embodiments of the disclosed principles may be implemented; 
           [0008]      FIG. 2  is a schematic illustration of an environment within which embodiments of the disclosed principles may be implemented; 
           [0009]      FIG. 3  is an architectural illustration showing an arrangement of functional modules in accordance with an embodiment of the disclosed principles; and 
           [0010]      FIG. 4  is a flowchart showing a process of predictive modeling and application in accordance with the disclosed principles. 
       
    
    
     DETAILED DESCRIPTION 
       [0011]    Turning to the drawings, wherein like reference numerals refer to like elements, techniques of the present disclosure are illustrated as being implemented in a suitable environment. The following description is based on embodiments of the claims and should not be taken as limiting the claims with regard to alternative embodiments that are not explicitly described herein. 
         [0012]    As noted above, there is currently no suitable general solution to efficiently encode models for use at the prediction phase in both linear and nonlinear systems. The most widely used technique to solve complex optimization is to construct Support Vector Machines (“SVMs”). SVMs have been used for many applications due to their generalization performance and strong theoretical foundation. While the topic of efficient algorithms for training SVMs has been touched on in the literature, utilization of SVMs in the testing and prediction phase has not been resolved. 
         [0013]    Currently, a saved SVM model is used to predict subsequent incoming new data in the testing phase. However, if the model is very large due to the large number of support vectors (the data points representing the classification boundary) and features, then there is a high space requirement, prohibiting the use of SVMs in resource-constrained environments. Moreover, a large model leads to longer prediction time, imposing another obstacle with respect to real-world SVM applications. While it may be possible in certain circumstances to reduce the number of support vectors in the model, a prohibitive number of support vectors may nonetheless remain in the model (e.g., when the number of training samples is large). Moreover, such approaches sacrifice the SVM generalization accuracy in order to reduce the model size. The resulting SVM performance greatly depends on the extent of support vector reduction. 
         [0014]    The disclosed principles provide a technique for efficiently encoding a model for use in prediction in both linear and non-linear problem spaces. In general terms, this technique does not sacrifice generalization performance while still significantly reducing the model size and making accelerating prediction performance. Essentially, the described techniques ensure generalization performance by maintaining all features while making the model size significantly smaller. Furthermore, while many existing approaches are limited to linear SVM, the described system functions with most kernel functions, whether linear or nonlinear. 
         [0015]    The model in many embodiments does not store any support vectors and associated coefficients; rather, it only stores the weight vector coefficients in the kernel space, making the model size very small. In addition, the technique can be applied even to more complex kernels by adding approximation methods, making the technique suitable even for advanced SVM algorithms. Thus, the disclosed principles are capable of providing reduced model size and accelerated prediction. 
         [0016]    While the disclosed principles are discussed by reference to SVMs, it will be appreciated that the disclosed approach is applicable to any linear, or kernel-based nonlinear, regression task, since the prediction function in this case is the same as that in SVM classification. Some exemplary alternative applications include ridge regression, support vector regression, and so on. 
         [0017]    Turning now to a more detailed description in view of the attached figures, the schematic diagram of  FIG. 1  shows an exemplary device within which aspects of the present disclosure may be implemented. In particular, the schematic diagram  100  illustrates exemplary internal components of a mobile smart phone implementation of a small touch-screen device. These components can include wireless transceivers  102 , a processor  104 , a memory  106 , one or more output components  108 , one or more input components  110 , and one or more sensors  128 . The processor  104  may be any of a microprocessor, microcomputer, application-specific integrated circuit, or the like. Similarly, the memory  106  may, but need not, reside on the same integrated circuit as the processor  104 . 
         [0018]    The device can also include a component interface  112  to provide a direct connection to auxiliary components or accessories for additional or enhanced functionality and a power supply  114 , such as a battery, for providing power to the device components. All or some of the internal components may be coupled to each other, and may be in communication with one another, by way of one or more internal communication links  132 , such as an internal bus. 
         [0019]    The memory  106  can encompass one or more memory devices of any of a variety of forms, such as read-only memory, random-access memory, static random-access memory, dynamic random-access memory, etc., and may be used by the processor  104  to store and retrieve data. The data that are stored by the memory  106  can include one or more operating systems or applications as well informational data. Each operating system is implemented via executable instructions stored in a storage medium in the device that controls basic functions of the electronic device, such as interaction among the various internal components, communication with external devices via the wireless transceivers  102  or the component interface  112 , and storage and retrieval of applications and data to and from the memory  106 . 
         [0020]    With respect to programs, sometimes also referred to as applications, each program is implemented via executable code that utilizes the operating system to provide more specific functionality, such as file system service and handling of protected and unprotected data stored in the memory  106 . Although many such programs govern standard or required functionality of the small touch-screen device, in many cases the programs include applications governing optional or specialized functionality, which can be provided in some cases by third-party vendors unrelated to the device manufacturer. 
         [0021]    Finally, with respect to informational data, this non-executable code or information can be referenced, manipulated, or written by an operating system or program for performing functions of the device. Such informational data can include, for example, data that are preprogrammed into the device during manufacture or any of a variety of types of information that are uploaded to, downloaded from, or otherwise accessed at servers or other devices with which the device is in communication during its ongoing operation. 
         [0022]    The device can be programmed such that the processor  104  and memory  106  interact with the other components of the device to perform a variety of functions, including interaction with the touch-detecting surface to receive signals indicative of gestures therefrom, evaluation of these signals to identify various gestures, and control of the device in the manners described below. The processor  104  may include various modules and execute programs for initiating different activities such as launching an application, data-transfer functions, and the toggling through various graphical user interface objects (e.g., toggling through various icons that are linked to executable applications). 
         [0023]    The wireless transceivers  102  can include, for example as shown, both a cellular transceiver  103  and a wireless local area network transceiver  105 . Each of the wireless transceivers  102  utilizes a wireless technology for communication, such as cellular-based communication technologies including analog communications, digital communications, and next generation communications or variants thereof, peer-to-peer or ad hoc communication technologies such as HomeRF, Bluetooth and IEEE 802.11 (a, b, g, or n), or other wireless communication technologies. 
         [0024]    Exemplary operation of the wireless transceivers  102  in conjunction with other internal components of the device can take a variety of forms and can include, for example, operation in which, upon reception of wireless signals, the internal components detect communication signals and one of the transceivers  102  demodulates the communication signals to recover incoming information, such as voice or data, transmitted by the wireless signals. After receiving the incoming information from one of the transceivers  102 , the processor  104  formats the incoming information for the one or more output components  108 . Likewise, for transmission of wireless signals, the processor  104  formats outgoing information, which may or may not be activated by the input components  110 , and conveys the outgoing information to one or more of the wireless transceivers  102  for modulation as communication signals. The wireless transceivers  102  convey the modulated signals to a remote device, such as a cell tower or an access point (not shown). 
         [0025]    The output components  108  can include a variety of visual, audio, or mechanical outputs. For example, the output components  108  can include one or more visual-output components  116  such as a display screen. One or more audio-output components  118  can include a speaker, alarm, or buzzer, and one or more mechanical-output components  120  can include a vibrating mechanism, for example. Similarly, the input components  110  can include one or more visual-input components  122  such as an optical sensor of a camera, one or more audio-input components  124  such as a microphone, and one or more mechanical-input components  126  such as a touch-detecting surface and a keypad. 
         [0026]    The sensors  128  can include an accelerometer, a gyroscope, any haptic, light, temperature, biological, chemical, or humidity sensor, or any other sensor that can provide pertinent information, such as to identify a current location of the device. Actions that can actuate one or more input components  110  can include for example, powering on, opening, unlocking, moving, or operating the device. For example, upon power on, a “home screen” with a predetermined set of application icons can be displayed on the touch screen. 
         [0027]    One task that may be accomplished for various reasons at various times by a device such as that shown in  FIG. 1  is predictive modeling, e.g., to predict a variable or class of interest given collected data known or believed to be result-effective. For example, it may be undesirable to force a user to indicate manually whenever the device environment changes, e.g., from being handheld during walking to being pocketed during driving and so on. Nonetheless, the device designer or an application developer may wish to change device or application behavior based on which of these two situations accurately describes the current device environment. In such a case, it is beneficial to obtain the values of result-effective variables such as speed, noise, and so on, and to predict the device environment on that basis. 
         [0028]    However, mobile electronic devices are significantly resource-constrained with respect to memory and processing power. Although device capabilities continue to advance rapidly, the requirements placed on such devices are also expanding rapidly. Thus, accurate and rapid predictive modeling remains difficult to implement in such devices. 
         [0029]    In an embodiment of the disclosed principles, these hurdles to effective predictive modeling in small mobile devices are overcome via a technique that precomputes values that are needed during the prediction phase. This technique can be used for both linear and non-linear systems and significantly reduces model size since only single values for features need be stored, eliminating the need to store thousands of examples with values for each feature. 
         [0030]    Moreover, for problems that can be optimally decided with linear or polynomial kernels, the precomputation is lossless, producing the same optimal accuracy as a non-precomputed solution. For problems requiring more complicated kernels, such as radial basis function (“RBF”) kernels, an approximation technique is employed, which for many problem spaces will provide a very close approximation to the optimal solution. This technique allows one to use significant computation resources, data, and techniques to create prediction technologies (e.g., classifiers), that are very accurate (even optimal in some cases), and then deploy them in severely resource-constrained environments. 
         [0031]    Before discussing the disclosed technique in detail, an example of an implementation environment is given to provide the reader with context within which to understand the subsequent examples. The simplified plan view shown in  FIG. 2  represents an example environment via which the presently disclosed techniques may be implemented. The illustrated example environment  200  includes a resource-constrained electronic device  201 , which may be, for example, a cellular telephone, a smart phone, a tablet, and so on. The resource-constrained electronic device  201  includes an input-sensor group  205 , a processor  206 , and an application or module  207  that makes use of the prediction results. 
         [0032]    The input-sensor group includes one or more sensors including, for example, a location sensor (e.g., GPS), one or more movement sensors (e.g., accelerometers, gyroscopes, and so on), a light sensor, and an audible sensor. It will be appreciated that the sensor group  205  may include additional or alternative sensors and that the given sensors may provide multiple sensed values. For example, a GPS module may provide both location and movement data. 
         [0033]    A resource-rich computing device  202  is shown in communication with the resource-constrained electronic device  201 . The resource-rich computing device  202  may be, for example, a laptop computer, a desktop computer, a workstation, and so on. In practice, the resource-rich computing device  202  is used to precompute model values for a computationally expensive and accurate prediction model. In an embodiment, the computationally expensive and accurate prediction model is one which makes use of SVMs, although other model types are usable as well. As a reminder, SVMs are support vector machines, which use support vectors to define a decision or classification surface. Support vectors can be viewed as elements of the training set whose removal would change the position of the decision surface. 
         [0034]    In the illustrated environment, the resource-rich computing device  202  receives a training set  203  of training vectors, which may be calculated or empirical. The resource-rich computing device  202  processes the training set  203  to generate model values  204  and provides these model values  204  to the processor  206  of the resource-constrained electronic device  201 . 
         [0035]    In operation, the processor  206  of the resource-constrained electronic device  201  receives sensed environment and device-condition parameters from the sensor group  205 . In order to classify the received data as indicating a first condition (e.g., phone in use while walking) or a second condition (e.g., phone idle and moving in vehicle), the processor  206  of the resource-constrained electronic device  201  implements a prediction algorithm such as one employing SVMs, while applying the model values  204  received from the resource-rich computing device  202 . The result of the model is output as a prediction or classification value  208  to the application or module  207  that makes use of the prediction results. 
         [0036]    The application or module  207  that makes use of the prediction results may be, for example, an operating system function that manages device power usage based on the predicted environment. In another example, the application or module  207  that makes use of the prediction results is an application that provides a service, such as alert muting or volume management based on the predicted environment. 
         [0037]    The SVM prediction function is: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       y 
                        
                       
                         ( 
                         z 
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         n 
                       
                        
                       
                         
                           w 
                           i 
                           ϕ 
                         
                          
                         
                           z 
                           i 
                           ϕ 
                         
                       
                     
                   
                   , 
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where w is the ith feature of the weight vector in a kernel space induced by the kernel function φ(.) and z i   φ  is the ith feature of the test sample in the kernel space. 
         [0038]    Since after the kernel mapping, the weight vector and test sample become implicit, the prediction function may be rewritten as 
         [0000]    
       
         
           
             
               
                 y 
                  
                 
                   ( 
                   z 
                   ) 
                 
               
               = 
               
                 
                   ∑ 
                   
                     i 
                     = 
                     1 
                   
                   n 
                 
                  
                 
                   
                     α 
                     i 
                   
                    
                   
                     y 
                     i 
                   
                    
                   
                     k 
                      
                     
                       ( 
                       
                         
                           x 
                           i 
                         
                          
                         z 
                       
                       ) 
                     
                   
                 
               
             
             , 
           
         
       
     
         [0000]    where αi is the coefficient, xi is the support vector, yi is the label of x i , and k(.,.) is the kernel function. If the prediction model is saved at this point by storing all the support vectors, coefficients, and class labels, then this leads to a large model size. 
         [0039]    However, in an embodiment, the weight vector is explicitly calculated in the kernel space rather than implicitly reformulating the prediction function. By obtaining the explicit weight vector coefficients in the kernel space, only these coefficients need be stored, yielding a small and efficient model. 
         [0040]    For example, consider the case of a typical kernel function. For a linear kernel, i.e., k(x i ,z)=x i   T z, this yields: 
         [0000]    
       
         
           
             
               
                 
                   
                     y 
                      
                     
                       ( 
                       z 
                       ) 
                     
                   
                   = 
                     
                    
                   
                     
                       ∑ 
                       i 
                     
                      
                     
                       
                         α 
                         i 
                       
                        
                       
                         y 
                         i 
                       
                        
                       
                         x 
                         i 
                         T 
                       
                        
                       z 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       ∑ 
                       i 
                     
                      
                     
                       
                         α 
                         i 
                       
                        
                       
                         
                           y 
                           i 
                         
                         ( 
                         
                           
                             ∑ 
                             j 
                           
                            
                           
                             
                               x 
                               ij 
                             
                              
                             
                               z 
                               j 
                             
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       ∑ 
                       j 
                     
                      
                     
                       
                         z 
                         j 
                       
                       ( 
                       
                         
                           ∑ 
                           i 
                         
                          
                         
                           
                             α 
                             i 
                           
                            
                           
                             y 
                             i 
                           
                            
                           
                             x 
                             ij 
                           
                         
                       
                       ) 
                     
                   
                 
               
             
           
         
       
     
       So 
       [0041]    
       
         
           
             ( 
             
               
                 ∑ 
                 i 
               
                
               
                 
                   α 
                   i 
                 
                  
                 
                   y 
                   i 
                 
                  
                 
                   x 
                   ij 
                 
               
             
             ) 
           
         
       
     
         [0000]    can be precalculated and stored for each zj, i.e., each feature. 
         [0042]    In the case of a polynomial kernel, the general form is k(x i ,z)=a(x i   T z+b) p  We may assume, for the sake of example, that a=1, b=0. For simplicity, let p=2, with the understanding that the reduction in model size also holds for general p. 
         [0000]    
       
         
           
             
               y 
                
               
                 ( 
                 z 
                 ) 
               
             
             = 
             
               
                 ∑ 
                 i 
               
                
               
                 
                   α 
                   i 
                 
                  
                 
                   
                     
                       y 
                       i 
                     
                      
                     
                       ( 
                       
                         
                           x 
                           i 
                           T 
                         
                          
                         z 
                       
                       ) 
                     
                   
                   2 
                 
               
             
           
         
       
     
         [0000]    And assume two features of interest, then 
         [0000]    
       
         
           
             
               
                 
                   
                     y 
                      
                     
                       ( 
                       z 
                       ) 
                     
                   
                   = 
                     
                    
                   
                     
                       ∑ 
                       i 
                     
                      
                     
                       
                         α 
                         i 
                       
                        
                       
                         
                           
                             y 
                             i 
                           
                            
                           
                             ( 
                             
                               
                                 
                                   x 
                                   
                                     i 
                                      
                                     
                                         
                                     
                                      
                                     1 
                                   
                                 
                                  
                                 
                                   z 
                                   1 
                                 
                               
                               + 
                               
                                 
                                   x 
                                   
                                     i 
                                      
                                     
                                         
                                     
                                      
                                     2 
                                   
                                 
                                  
                                 
                                   z 
                                   2 
                                 
                               
                             
                             ) 
                           
                         
                         2 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       ∑ 
                       i 
                     
                      
                     
                       
                         α 
                         i 
                       
                        
                       
                         
                           y 
                           i 
                         
                          
                         
                           ( 
                           
                             
                               
                                 x 
                                 
                                   i 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                                 2 
                               
                                
                               
                                 z 
                                 1 
                                 2 
                               
                             
                             + 
                             
                               2 
                                
                               
                                 x 
                                 
                                   i 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                                
                               
                                 x 
                                 
                                   i 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                                
                               
                                 z 
                                 1 
                               
                                
                               
                                 z 
                                 2 
                               
                             
                             + 
                             
                               
                                 x 
                                 
                                   i 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                                 2 
                               
                                
                               
                                 z 
                                 2 
                                 2 
                               
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       
                         z 
                         1 
                         2 
                       
                        
                       
                         
                           ∑ 
                           i 
                         
                          
                         
                           
                             α 
                             i 
                           
                            
                           
                             y 
                             i 
                           
                            
                           
                             x 
                             
                               i 
                                
                               
                                   
                               
                                
                               1 
                             
                             2 
                           
                         
                       
                     
                     + 
                     
                       
                         z 
                         2 
                         2 
                       
                        
                       
                         
                           ∑ 
                           i 
                         
                          
                         
                           
                             α 
                             i 
                           
                            
                           
                             y 
                             i 
                           
                            
                           
                             x 
                             
                               i 
                                
                               
                                   
                               
                                
                               2 
                             
                             2 
                           
                         
                       
                     
                     + 
                     
                       2 
                        
                       
                         z 
                         1 
                       
                        
                       
                         z 
                         2 
                       
                        
                       
                         
                           ∑ 
                           i 
                         
                          
                         
                           
                             α 
                             i 
                           
                            
                           
                             y 
                             i 
                           
                            
                           
                             x 
                             
                               i 
                                
                               
                                   
                               
                                
                               1 
                             
                           
                            
                           
                             x 
                             
                               i 
                                
                               
                                   
                               
                                
                               2 
                             
                           
                         
                       
                     
                   
                 
               
             
           
         
       
     
         [0000]    Given this, 
         [0000]    
       
         
           
             
               
                 ∑ 
                 i 
               
                
               
                 
                   α 
                   i 
                 
                  
                 
                   y 
                   i 
                 
                  
                 
                   x 
                   
                     i 
                      
                     
                         
                     
                      
                     1 
                   
                   2 
                 
               
             
             , 
             
               
                 ∑ 
                 i 
               
                
               
                 
                   α 
                   i 
                 
                  
                 
                   y 
                   i 
                 
                  
                 
                   x 
                   
                     i 
                      
                     
                         
                     
                      
                     2 
                   
                   2 
                 
                  
                 
                     
                 
                  
                 and 
                  
                 
                     
                 
                  
                 
                   
                     ∑ 
                     i 
                   
                    
                   
                     
                       α 
                       i 
                     
                      
                     
                       y 
                       i 
                     
                      
                     
                       x 
                       
                         i 
                          
                         
                             
                         
                          
                         1 
                       
                     
                      
                     
                       x 
                       
                         i 
                          
                         
                             
                         
                          
                         2 
                       
                     
                   
                 
               
             
           
         
       
     
         [0000]    can be precalculated and stored for each term. As a result, only three values for each of two features need be stored. For a greater number of features, the Multinomial theorem may be used, but the end result is the same, that is, a dramatic reduction in model size due to the need to store fewer values with the model. 
         [0043]    In the case of an RBF kernel, 
         [0000]    
       
         
           
             
               k 
                
               
                 ( 
                 
                   
                     x 
                     i 
                   
                   , 
                   z 
                 
                 ) 
               
             
             = 
             
               
                  
                 
                   
                     - 
                     r 
                   
                    
                   
                     
                        
                       
                         
                           x 
                           i 
                         
                         - 
                         z 
                       
                        
                     
                     2 
                   
                 
               
               = 
               
                 
                    
                   
                     - 
                     
                       r 
                        
                       
                         ( 
                         
                           
                             
                               x 
                               i 
                               T 
                             
                              
                             
                               x 
                               i 
                             
                           
                           - 
                           
                             2 
                              
                             
                               x 
                               i 
                               T 
                             
                              
                             z 
                           
                           + 
                           
                             
                               z 
                               T 
                             
                              
                             z 
                           
                         
                         ) 
                       
                     
                   
                 
                 . 
               
             
           
         
       
     
         [0000]    Note that it is difficult to factor out z due to the exponential term, but a Taylor expansion may be applied in this case to approximate the original function and to remove the exponential term. The Taylor expansion for e−rx is: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                        
                       
                         - 
                         rx 
                       
                     
                     = 
                     
                       1 
                       - 
                       rx 
                       + 
                       
                         
                           
                             r 
                             2 
                           
                            
                           
                             x 
                             2 
                           
                         
                         2 
                       
                       - 
                       
                         
                           
                             r 
                             3 
                           
                            
                           
                             x 
                             3 
                           
                         
                         6 
                       
                       + 
                       
                         O 
                          
                         
                           ( 
                           
                             x 
                             4 
                           
                           ) 
                         
                       
                     
                   
                   , 
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
         [0000]    and the term O(x k ) may be ignored since it denotes the higher order terms which are generally very small. 
         [0044]    For most applications, a second order or third order solution is a sufficient approximation. Substituting Equation (2) into the kernel function defined above, the problem reverts to one that is similar to that associated with a polynomial kernel, and the model can also now be simplified in a similar fashion. 
         [0045]    As can be seen, the disclosed principles are applicable to simplify models based on linear, polynomial, and RBF kernels. In each case, the model need not store any support vectors, and as such, the model size depends only on the number of features of the data points in the mapped space. Although an approximation is used in the case of RBF kernels, the loss of accuracy is very small, and the described approach does not sacrifice any generalization performance while reducing the model size and making the prediction faster. 
         [0046]    In essence, the described approach assures that generalization performance is maintained despite the model reduction process. Furthermore, while most existing approaches only work for linear SVM, the described principles apply to most popularly used kernel functions, whether linear or nonlinear. As noted above, although the disclosed principles are exemplified herein by the use of SVMs, the described approach is applicable to any linear or kernel-based nonlinear regression task, since the prediction function in such cases is the same as that in SVM classification. Some such alternative applications include ridge regression, support vector regression, and so on. 
         [0047]    The described technique may be used to provide solutions to any of a number of prediction problems for the mobile device  201 . While the device may not always be in a full power state, the described prediction technique can be used for classification and prediction running on low-powered sensors, chipsets, and processors. For example, the described technique may provide an efficient classifier to run on a digital signal processor (“DSP”), employing constant passive audio sensing to predict whether the device is in a running automobile, e.g., via the gathered noise data. This example can be extended to a number of other use cases and may also utilize sensor signals in addition to audio. For example, more generally, the described techniques may be used to predict a user&#39;s context and state, such as mood, activity, etc. 
         [0048]    Because of the compact model size provided by the disclosed principles, a number of such classifiers may be deployed simultaneously without incurring substantial computational overhead. This makes the described technique readily applicable for use in the DSP, sensor hub, watch, glass, and similar resource-constrained environments. 
         [0049]    Turning to  FIG. 3 , a schematic illustration of the mobile electronic device  201  is shown in greater detail, in an environment in which the device  201  is running multiple predictive models according to the described principles. In particular, the processor  206  of the device  201  is shown running multiple reduced predictive models  301 ,  302  of the type described above to provide a respective prediction or classification  303 ,  304  to the subscriber modules or applications  305 ,  306 . 
         [0050]    The sensor group  205  provides input data for each reduced predictive model  301 ,  302 . Although not required, some or all sensed data may be shared by the reduced predictive models  301 ,  302 . In the illustrated example, one sensed quantity, e.g., noise, light, location, etc., is an input to both reduced predictive models  301 ,  302 , while the remaining sensed quantities are input to one or the other of the reduced predictive models  301 ,  302 . 
         [0051]    Although the precise process for applying the reduced predictive model architecture disclosed herein may vary from implementation to implementation, an exemplary process is illustrated in the flowchart of  FIG. 4 . The illustrated process  400  operates in the context of a device as shown in  FIGS. 2 and 3 , however it will be appreciated that the process  400  may be applied as is or with hardware-dependent variations to numerous other types of resource-constrained electronic devices. 
         [0052]    At stage  401  of the process  400 , the weight vector coefficients for support vectors of a predictive model are precalculated on a resource-rich machine such as a desktop or laptop computer. The model is then stored on a resource-constrained device at stage  402 . The stored model includes the precalculated weight vector coefficients for the support vectors but does not include the support vectors themselves. 
         [0053]    At stage  403 , the resource-constrained device runs the predictive model. The model may be run on demand or continuously depending upon its purpose and upon device constraints. While running, the predictive model receives data regarding one or more features from a sensor group of the device at stage  404 . As noted above, features may pertain to any sensed quantity that is result-effective within the model, e.g., sound, light, vibration, location, and so on. 
         [0054]    Based on the sensed data, the predictive model produces, at stage  405 , a prediction or classification. For example, the predictive model may be one that predicts device location with respect to a vehicle or a user, or that predicts any other condition that is not directly readable by the device but that may be predicted or inferred based on selected features. For example, the described techniques may be used to predict a user&#39;s context and state, such as mood, activity, etc. 
         [0055]    While the foregoing example focuses on SVMs, it will be appreciated that the described system and process are applicable to a wide variety of predictive applications as noted above. In view of the many possible embodiments to which the principles of the present discussion may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of the claims. Therefore, the techniques as described herein contemplate all such embodiments as may come within the scope of the following claims and equivalents thereof.