Patent Document

RELATED APPLICATIONS 
       [0001]    This is a continuation-in-part (CIP) of U.S. application Ser. No. 14/670,313 filed Mar. 26, 2015, and incorporated herein by this reference. 
     
    
     COPYRIGHT NOTICE 
       [0002]    © 2015 IFWIZARD CORPORATION. A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. 37 CFR §1.71(d). 
       TECHNICAL FIELD 
       [0003]    The present disclosure relates to computerized statistical analysis. In particular, the present invention relates to high performance model optimization and analytics. 
       BACKGROUND OF THE INVENTION 
       [0004]    The largest drawbacks in traditional optimization, especially for applications that rely on accessing large datasets, remain significant barriers needing solutions: namely, the sophisticated programmer knowledge that traditional optimization requires, the fragility of tuned solutions in dynamic analyses, and the abundant software development time needed to create an optimized solution. The need remains for improvements in automatically optimizing database analytics operations to improve performance and flexibility while reducing reliance on application programmer skills. 
       SUMMARY OF THE INVENTION 
       [0005]    The following is a summary of the invention in order to provide a basic understanding of some aspects of the invention. This summary is not intended to identify key/critical elements of the invention or to delineate the scope of the invention. Its sole purpose is to present some concepts of the invention in a simplified form as a prelude to the more detailed description that is presented later. 
         [0006]    It is therefore an object of the present invention to provide a computerized analytics system which overcomes the aforementioned problems and shortcomings of the prior art. It is a further object of the present invention to provide a computerized programmatic mechanism for enumerating, querying, and analyzing stored statistical data. It is a further object of the present invention to provide the elaboration of functional definitions of statistical data and their interactions. 
         [0007]    Another object is to provide a computer-implemented database analytics system comprising: an API access layer arranged to enable a client to access the system, wherein the API access layer includes a base client library implemented in a selected target programming language; a data model manager component for storing at least one data model in the system, wherein the stored data model includes a programmatically defined structure of the model, and at least one dependent analytical function associated with the model; a data model source code generator configured to convert the stored data model to an internal source code file; an interface generator component, configured to generate a model specific interface library that is conformant with a syntax of the target programming language and that matches the defined structure of the stored data model; and further wherein the interface generator component is configured to generate a model specific interface library that is conformant with a syntax of the target programming language and that implements the dependent analytical function; and a compiler and linker component arranged to compile the internal source code file into native machine code for linking into a function table associated with the data model so that operations expressed by the client using the model specific interface library of the API access layer may be executed utilizing native machine code on the server. 
         [0008]    Another object of the present invention to provide an evaluating monitor of changes to said functional definitions for those which require a compilation action. It is still a further object of the present invention to provide a compilation subsystem that allows the computerized generation of optimized analytics program code, source compilation, and dynamic linking into the interaction mechanism. It is still a further object of the present invention to provide a set of feedback mechanisms that allow for automatic continuous optimization of computerized models. 
         [0009]    Additional aspects and advantages of this invention will be apparent from the following detailed description of preferred embodiments, which proceeds with reference to the accompanying drawings. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0010]      FIG. 1  is a block diagram of an example server system consistent with the present disclosure. 
           [0011]      FIG. 2  is a block diagram showing an example of a model subsystem consistent with the present disclosure. 
           [0012]      FIG. 3  is a block diagram showing an example of structure subsystem detail consistent with the present disclosure. 
           [0013]      FIG. 4  is a block diagram showing an example of mapping subsystem detail consistent with the present disclosure. 
           [0014]      FIG. 5  is a block diagram showing an example of an example of generator subsystem detail consistent with the present disclosure. 
           [0015]      FIG. 6  is a block diagram showing an example of the event subsystem detail consistent with the present disclosure. 
           [0016]      FIG. 7  is a block diagram showing an example of the filter subsystem detail consistent with the present disclosure. 
           [0017]      FIG. 8  is a block diagram showing an example of the analysis subsystem detail consistent with the present disclosure. 
           [0018]      FIG. 9  is a flow chart of an example of a client access process consistent with the present disclosure. 
           [0019]      FIG. 10  is a flow chart of an example of model compilation process consistent with the present disclosure. 
           [0020]      FIG. 11  is a process diagram of an example of a generic optimization pattern consistent with the present disclosure. 
           [0021]      FIG. 12  is a flow chart of an exemplary experimental optimization process consistent with the present disclosure. 
           [0022]      FIG. 13  is a flow chart of an exemplary predictive tuning optimization process. 
           [0023]      FIG. 14  is a flow chart of an exemplary genetic algorithm optimization process. 
           [0024]      FIG. 15  is a flow chart illustrating one example of the native source code generation process. 
       
    
    
     DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
       [0025]    The following description is intended to provide a detailed description of examples of some embodiments and aspects of the invention and should not be taken to be limiting of the invention itself. Rather, any number of variations may fall within the scope of the invention, which is defined in the claims following the description. 
         [0026]    The present disclosure includes a method, a data processing system and a computer program product that compiles a source code program into a several object code files where the method includes the steps of defining a set of computerized statistical models and their functional interactions; determining changes that require an optimizing compilation and linking action; dynamically vending programmatic libraries for accessing and interacting with the models; executing model interaction in a performant, scalable, and simultaneous manner; and optionally propagating experimental model results as feedback for continuous and automatic optimization either in terms of execution performance or accuracy. 
         [0027]    As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon. 
         [0028]    Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which are implemented via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. 
         [0029]    These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks. 
         [0030]    The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which are implemented on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. 
         [0031]    Referring to the system overview diagram of  FIG. 1 , the server  100  is the central mechanism for providing the functionality described herein. Accessed by one or more clients  101 , the server  100  allows access to the analytics system through the API (Application Programming Interface) access layer  102 . It should be understood that a typical computer system providing the server  100  or a client  101  contains many other components not shown, which are not essential to an understanding of the present invention. 
         [0032]    In one embodiment, the access layer  102  is presented through an HTTP (Hypertext Transfer Protocol) communication over an IP (Internet Protocol) computer network connection. In this particular embodiment, the HTTP communication, which may be optionally cryptographically secured through a TLS (Transport Layer Security) application layer implementation, provides access through a REST (Representational State Transfer) API with JSON (Javascript Object Notation) elaboration of structures and Base64 encoded raw data. 
         [0033]    The client  101  access is built on top of a base client library available in multiple programming languages. This base client library provides authentication, RBAC (Role Based Access Control), transparent error recovery, data marshalling, and a “native” interface for the programming language of choice. The access layer  102  interfaces with the base client library to provide a model specific programmatic interface through the interface generator  103 . This generator creates a programming library in the client&#39;s language of choice (the “target programming language”) that directly matches the model&#39;s attributes and functionality for optimal convenience and efficiency of the client developer. Illustrative target programming languages may include without limitation any or all of the following: 
         [0034]    Ada (GNAT sockets) 
         [0035]    C (BSD/POSIX sockets or Winsock) 
         [0036]    C++ (Qt or Boost ASIO) 
         [0037]    C# 
         [0038]    Clojure 
         [0039]    D 
         [0040]    Go 
         [0041]    Java 
         [0042]    Javascript 
         [0043]    Object Pascal (Delphi-compatible) 
         [0044]    Objective-C 
         [0045]    PHP 
         [0046]    Python 
         [0047]    Ruby 
         [0048]    The interface generator  103  generates the interface based on the data model  200  stored within the data model manager  104 , which is a dynamic container of meta-information about all analytical models in the server  100 . Preferably the model manager  104  provides for the creation, deletion, enumeration, and use of the models in their various subsystems. 
         [0049]    When a specific model task, such as the creation of a new model, is specified by the access layer  102  to the data model manager  104 , the task may be passed to a task dispatcher  105  which enables parallelized task completion while maintaining consistent, known model state. This standard goal of parallel programming is thereby provided upstream of the client  101 , greatly simplifying and streamlining a traditionally complex area of optimization. 
         [0050]    A wide variety of data models can be implemented. Models may contain programming scripts as part of their definition to provide complex functionality. The server&#39;s  100  scripting system  106  preferably provides a large set of mathematical, statistical, textual, and logical functions available in a deterministic and highly efficient context. In one particular embodiment of the present invention, this efficiency is gained by trans-compiling the scripts to the highly performant “C” programming language as part of the source generation of the models by the model source generator  107 , which converts the model to “C” source files. These source files are then compiled by an optimizing “C” compiler and dynamically linked back into the server  100  process space by the compiler and linker  108  system. 
         [0051]    The scripting language has a strictly deterministic operation allowing real-time performance characteristics primarily through preventing all looping outside of list enumeration. This intentional limitation includes “while”-style conditional loops, “goto”, and recursion. As a purely functional language, no side-effects or modifiable variables are permitted within the script functions. Outside of the special case of analysis reduction functions  804 , there is intentionally no facility for defining functions visible across scripts, nor common programming-in-the-large idioms such as object-oriented programming, complex type definition, namespaces, etc. 
         [0052]    Though without sophisticated facilities for developing reusable libraries, the scripting language includes a large set of mathematical, statistical, textual, and logical functions built-in and these are automatically available to all scripts. Similar to the Lisp programming language, scripts are composed entirely of nested programmatic s-expressions each with a function name and its arguments, and each returning a result. As such, the syntax is minimal and there are no operator symbols such as “+”, order of operation rules, and other complexities contained in most other programming languages. 
         [0053]    The scripting language may be written in either parenthesized prefix notation (Lisp-style) or function notation (Fortran-style). Prefix and function notation may also be freely mixed, although this is generally not advisable due to reducing programmer readability. The following two example scripts, though using different notational styles, do the same function (i.e. return the boolean value, ‘true’, if var_x and var_y add up to 10; else return ‘false’). 
         [0054]    Prefix notation: (if (eq (add var_x var_y) 10) true false) 
         [0055]    Function notation: if(eq(add(var_x, var_y), 10), true, false) 
         [0056]    As discussed further below, scripts may be used in defining mapping transforms  402 , defining filter criteria  702 , defining data reductions for analysis  802 , and as noted in otherwise defining a data model structure and its initialization  502 . 
         [0057]      FIG. 2  is a block diagram showing a presently preferred set of model subsystems available in the model manager&#39;s  104  data models  200 . The model  200  provides for the lookup of specific model instance events which particular embodiments or instantiations of the model structure through the event lookup index  202 . The memory mapped storage  203  contains the disk-backed memory access to the events so indexed. The interface generator&#39;s  103  source may be cached within the model&#39;s  200  source interface cache  205 . 
         [0058]    The access layer  102  preferably interfaces directly with the compiled and linked machine code for each of the model&#39;s  200  API subsystems  201 . These APIs  201  are directly executable through the dynamically linked function table  204  and include the model&#39;s  200  structures  300 , mappings  400 , generators  500 , events  600 , filters  700 , and analyses  800 , described in more detail in the subsequent figures. 
         [0059]      FIG. 3  illustrates a structure subsystem  300 . Before a model  200  can be used by a client  101 , it must be defined and these foundational definitions are known as structures  300 . Internally, a structure  300  may comprise a list of named variables and their formats. For example, a structure  300 , “tire”, might have variables such as “mileage”, “wet_grip”, “retreadability”, “dry_handling”, and so on. Variables may include the following types with their corresponding constraints: 
         [0060]    TEXT: A sequence of bytes that are frequently UTF-8 or ISO-8859 encoded characters with a fixed maximum length. The given length preferably is pre-allocated for each instance (event) of this data. A text variable&#39;s constraints must include its length. 
         [0061]    BLOB: A sequence of data bytes or encoded characters with a potentially unbound length. A structure that contains blobs will be much slower for certain operations than one that only contains the other variable types due to optimizations available with a fixed structure size. A blob variable&#39;s constraints may include its length. 
         [0062]    INTEGER: A signed integer number without a fractional component such as “771” or “0”. The potential range depends on the “size” constraint which selects between 32-bit, 64-bit, and arbitrary precision. Arbitrary precision integers are much slower than other numerical formats. Integer constraints include its minimum, maximum, and size (“32”, “64”, or “arbitrary”). 
         [0063]    REAL: A floating point number with fraction and exponent components such as “3.14” or “2.5 −68 ”. The precision is selectable from single- and double-precision with moderately faster performance for single-precision. Real constraints include minimum, maximum, and precision (“single” or “double”). 
         [0064]    BOOLEAN: A simple binary flag with only “true” (1) or “false” (0) as possible values. 
         [0065]    ENUM: An efficient and easily testable selection between [among??] a list of given named labels. 
         [0066]    Structures may have many variables to track the different data occurring in each recorded event. These data form the basis for the other functions and features in the model  200  as further described below. 
         [0067]    The API for the structure  300  may include the command to list structures  301 , which returns a list of all structures  300  available in the server  100 . Additionally, the client  101  may create a new structure  302 , which requires the structure name and a list of variables. The client  101  may also delete a previously created structure  303 . Finally, the client  101 , through the interface generator  103 , may expand out the particular functionality of a given structure  300  with a specific target programming language through the command to interface with the structure  304 . The command returns the programming library for that specific structure and language. In other words, the interface generator component preferably is configured to generate a model specific interface library that is conformant with a syntax of the target programming language and that matches the defined structure of the stored data model. This is one of several features designed to improve performance while enabling the application programmer (client  101 ) to operate in a target programming language of her choosing. Creating  302  or deleting  303  a structure  300  triggers the compilation and linking  108  system and modifies the interface models created by the interface generator  103 . 
         [0068]      FIG. 4  shows an example of a mapping subsystem  400 . Once a structure  300  has been defined, a transformed or partial view of it may be created through a mapping  400  belonging to the structure  300 . Additionally, mappings  400  can be derived from another mapping  400 , except for circular references. Mappings  400  may provide their own set of transformed variables through pure functions using the scripting system  106 . In an embodiment, mapping functions may use any of the following to generate a mapped transform variable: 
         [0069]    Any numeric, textual, Boolean, or enum constant 
         [0070]    Any variable of the parent structure or transform of the parent mapping 
         [0071]    Any idempotent scripting function. 
         [0072]    The mapped transform&#39;s type may be set as with structure variables, but must be the same or castable from the return value. Mapped transforms have no constraint pre-conditions as they are not directly set by the client or generated. Mapped transform names may be different or identical to parent variable/transform names, but the mapping name itself must be unique among all structures and mappings. 
         [0073]    The API for the mapping  400  may include the command to list mappings  401 , which returns a list of all mappings  400  available in the server  100  for a given structure  300 . Additionally, the client  101  may create a new structure  402 , which requires the mapping name, the name of the parent structure  300  or mapping  400 , and a list of mapping transforms. The client  101  may also delete a previously created mapping  403 . Creating  402  or deleting  403  a mapping  400  triggers the compilation and linking  108  system and modifies the interface models created by the interface generator  103 . 
         [0074]      FIG. 5  shows the generator subsystem  500 . When the client  101  wishes to store data, it needs to first generate an instance of the target structure  300 . This newly generated instance, known as an “event”  600 , is generated according to rules of a previously defined generator  500 . A generator acts similar to a constructor in OOP (Object-Oriented Programming), providing initialization that can potentially be programmatically complex. While initialization can also be done in client code, the use of generators can improve efficiency, convenience, and consistency across multiple client environments. 
         [0075]    Generators  500  contain a list of initializers for a set of the structure&#39;s  300  variables. Each initializer does the work of automatically setting the starting value of its corresponding variable using a programming script. Unlike mapping  400  transforms, the generator&#39;s  500  initializers do not need to be idempotent (the property of certain operations in mathematics and computer science that can be applied multiple times without changing the result beyond the initial application). This allows for generators  500  that can, for example, use random values for some variables. 
         [0076]    If a generator  500  does not initialize one of the event&#39;s variables, the client  101  should set it explicitly before saving the event. Using custom defined generators  500  is optional as each structure  300  automatically has an implicit generator  500  created for it which simply requires all variables to be explicitly set. 
         [0077]    The API for the generator  500  may include a command to list generators  501 , which returns a list of all generators  500  available in the server  100  for a given structure  300 . Additionally, the client  101  may create a new generator  502 , which requires the generator name, the name of the parent structure  300 , a list of generator initializers, and a list of settings which are input variables with name, type, and constraints that are to be provided during generation and which may be used by the initializers. The client  101  may also delete a previously created generator  503 . Finally, the client  101  may generate  504  a new event  600  for the structure  300  according to the rules of the generator  500 . If settings were provided when the generator  500  was created  502 , they must be provided during generation  504 . The generated event  600  must be explicitly saved in order to be visible and persisted within the server  100 . Creating  502  or deleting  503  a generator  500  triggers the compilation and linking  108  system and modifies the interface models created by the interface generator  103 . 
         [0078]      FIG. 6  shows an example of the event subsystem  600 . An event  600  is a particular instantiation of a structure&#39;s  300  model. All data in the server  100  is stored and processed as recorded events. Events  600  may be interacted with as “Plain Old Objects” within the client language, providing an easy and natural interface to the underlying data model. Saveable events  600  can only exist for structures  300  while unsaveable events can be returned by filters  700  applied to mappings  400 . Events  600  must have all variables set either explicitly or through a generator  500  in order to be saved. Events  600  may not be saved more than once or modified after saving; a new event  600  must be created or an existing event  600  cloned. 
         [0079]    In addition to their variable values, events  600  preferably have the following intrinsic meta-information which may be accessed: 
         [0080]    Structure name 
         [0081]    Time of generation 
         [0082]    Whether all variables have been set 
         [0083]    If and when saved 
         [0084]    A globally unique ID 
         [0085]    The API for the event  600  includes the command to store the event  601  to the server  100 . Additionally, a previously saved event may be deleted  602  given a particular globally unique event ID. 
         [0086]      FIG. 7  shows an example of a filter subsystem  700 . Filters  700  are used for querying the server  100  and enumerating events  600 . Filters  700  have one or more criteria which preferably are themselves script functions with access to all structure  300  variables or mapping  400  transforms, and return boolean values. All criteria must return “true” in order for the filter  700  to accept an event  600 ; if a single criterion returns “false”, the filter will skip the tested event. Each structure  300  or mapping  400  may have an implicit filter  700  with no criteria which returns all events  600  for the structure  300  or mapping  400 . 
         [0087]    The API for the filter  700  may include a command to list filters  701 , which returns a list of all filters  700  available in the server  100  for a given structure  300  or mapping  400 . Additionally, the client  101  may create a new filter  702 , which requires the filter name, the name of the parent structure  300  or mapping  400 , a list of filter criteria, and a list of settings which are input variables with name, type, and constraints that are to be provided during filtering and which may be used by the criteria. The client  101  may also delete a previously created filter  703 . Further, the client  101  may count  704  all events  600  for a specified filter  700 . If settings were provided when the filter  700  was created, they must also be provided when counting  704 . 
         [0088]    Finally, the client  101  may enumerate  705  a given filter  700 , i.e. enumerate the filtered results, returning entire event instances  600  up to a given fetch size along with a cursor. The cursor may be passed back on an additional call to the filter  700  in order to continue the enumeration from the last returned event. If no cursor is returned, all events have been retrieved. If settings were provided when the filter  700  was created, they must also be provided when enumerating  705 . Creating  702  or deleting  703  a filter  700  triggers the compilation and linking  108  system and modifies the interface models created by the interface generator  103 . 
         [0089]      FIG. 8  shows the analysis subsystem  800 . An analysis  800  is a process that iteratively analyzes multiple events  600 , reducing the data to a normally much smaller set of outputs. Functional scripts used to provide the reducing functions, each passed the enumerated  705  set of filtered events  600 . If no event data  600  has been added or deleted since the last analyzing with given settings, a cached result may be returned. 
         [0090]    In a preferred embodiment, reduction functions must be pure, while analyses  800  have the special property of being available to generator  500  and filter  700  scripts as callable functions. This allows for easy creation of incremental optimization and adjustment mechanisms. Furthermore, unlike all other scriptual references, analyses  800  of other structures  300  or their mappings  400  may be used as functions, permitting complex scenarios where variant experiments or data models can be related automatically. 
         [0091]    The API for the analysis  800  may include a command to list analyses  801 , which returns a list of all analyses  800  available in the server  100  for a given structure  300  or mapping  400 . Additionally, the client  101  may create a new analysis  802 , which requires the analysis name; the name of the parent structure  300  or mapping  400 ; a list of analysis reductions including the name, type, and script of each reduction; and a list of settings which are input variables with name, type, and constraints that are to be provided during analyzing and which may be used by the outputs. The client  101  may also delete a previously created analysis  803 . Finally, the client  101  may analyze  804  a given analysis  800 , returning a set of reductions given the analysis name, settings if they were provided during analysis creation  802 , the filter  700  to use with the analysis  800 , and settings for the given filter  700  if any are required. Creating  802  or deleting  803  an analysis  800  triggers the compilation and linking  108  system and modifies the interface models created by the interface generator  103 . 
         [0092]    Thus the fundamental data management process may be summarized as comprising:
       Accept high level structural and processing definitions through the API   Convert these definitions into efficient (for example, C) source code representations   Compile and dynamically link this source code forming internal models-as-libraries   Provide clients with native programmatic access to these model libraries for storing, filtering, and analyzing their data.       
 
         [0097]      FIG. 9  illustrates flow of a client access process in a preferred embodiment. Upon start  900 , the client  101  loads the base client libraries  901  and authenticates its connection  902  to the server  100 . After completing connection and authentication, model access is provided by checking whether the programmatic model interface is available  903 . If it is not available, the client  101  acquires and loads  904  it from the interface generator  103 . 
         [0098]    The client  101  may then select which action  905  it would like to perform on the model  200 . The client may wish to manage the model  910 , by which it will access the various model subsystem APIs  201  to manage the structures  911 , mappings  912 , generators  913 , filters  914 , and analyses  915 . 
         [0099]    Alternatively, the client may wish to directly programmatically interact with the event data  600 . In this case it may wish to generate a new event instance  906 , enumerate a given filter  907 , perform analysis reductions  908 , or store a generated instance  909 . This sequence may be continued via decision  916  until the client  101  has completed access  917  to the server  100 . 
         [0100]      FIG. 10  shows the flow of the model compilation process. Upon start  1000  the server  100  receives a compilation command  1001  such as following a structure generation task  302 . The server verifies  1002  that the model does, indeed, need compilation by comparing the changes with the previous meta-data for the model  200  and terminates compilation if it is not needed  1003 . 
         [0101]    If compilation is needed, the model is locked  1004 . In one preferred embodiment this is done using a mutual exclusion thread lock. The model is then prepared for compilation  1005  including validation and consistency checking. The native source code for the model is then generated  1006  as further described below. This source code is then compiled and relinked  1007  dynamically into the server&#39;s  100  process space. The model&#39;s references are suitably updated  1008  including the function table  204  and the model is unlocked  1009 , completing  1010  the compilation and linking process. 
         [0102]      FIG. 11  shows an illustrative process diagram of a generic optimization process whereby the optimization is target is setup  1100  with a set of controlled and experimental variable, a test  1101  is conducted to produce the results of that configuration, the results  1102  measured and analyzed according to some metric or metrics of success or error, and these data are fed back  1103  into the setup process. As a simple example of this type of generic optimization, consider an artillery cannon designed to fire at a remote target with a variably stronger or weaker gunpowder charge. The given amount of gunpowder is the setup  1100 , the firing of the cannon is the test  1101 , establishing the delta between the intended target and the place of impact is the measurement  1102 , and the adjustment of the following charge is the feedback  1103 . 
         [0103]      FIG. 12  shows the flow of an exemplary experimental optimization process. The present invention enables this kind of optimization that is oriented towards a relatively free-form optimization that may be human guided or completely automated, using a basic pattern of experimentation and analysis. Upon start  1200 , the client  101  generates initial control events  1201  using seed initializers  1202 . Analysis on these control events  1203  yields control reductions  1204  which are used in reference to experimental initializers  1206  when generating newly-effected events  1205 . These newly-effected events are further analyzed  1207  and their reductions  1208  measured  1209  for the target effect. This process if iteratively  1210  repeated until the desired optimization is complete  1211 . 
         [0104]      FIG. 13  shows the flow of an exemplary predictive tuning optimization process. The present invention enables this kind of optimization that is oriented towards weighted prediction graphs and backpropagation, such as for an online product recommender. Upon start  1300 , the client  101  sets up  1301  the initial weightings  1302  for the prediction variables. These weightings are used as settings with an analysis  1303  of some set of sample data to further derive a reduced set  1304  of weightings (this analysis may be skipped for a less “trained” version of this algorithm). These weightings are then used to generate a prediction  1307  based on a test case sample  1306 . This is compared  1309  with the actual outcome  1308  and further tuning  1310  may be done, potentially using the resulting comparison, until the tuning is complete  1311 . 
         [0105]      FIG. 14  shows the flow of an exemplary genetic algorithm optimization process. The present invention enables this kind of optimization that is oriented towards the gradual, unguided evolution of optimization using genetic algorithms with fully or partially random mutation and fitness criteria. Unlike the other example optimization algorithms, the genetic algorithm approach involves changing programmatic scripts. The present invention&#39;s high efficiency allows for such algorithmically designed scripts to execute with particularly high performance, making evolutionary optimization applicable to wider range of scenarios than otherwise. Upon start  1400 , an initial seed algorithm is randomly generated  1401  and used to create the desired subsystems and test  1402 . The test is performed and the actual outcome  1403  is compared  1405  against the desired outcome  1404 . If further tuning  1406  is sought, the results are analyzed  1407  and reduced  1408  into a descendant algorithm  1409  for further testing  1402 . Otherwise, the optimization is complete  1410 . 
         [0106]      FIG. 15  shows the flow of the native source code generation process. This process is performed for each structure  300  in the model manager  104  and for all of the structure&#39;s  300  dependent objects such as mappings  400  or filters  700 . Upon start  1500 , initial boilerplate code is generated and the structure  300  source code is created  1501  as a C “struct” element in the preferred embodiment, shown in the pseudo-code output  1502  with all of the dependent properties of an event  600  as this forms the raw memory storage for all events. In the preferred embodiment, variables are directly named and included in the “struct” element as var_NAME where NAME is replaced with the variable identifier rather than the array representation shown in the pseudo-code  1502 . 
         [0107]    For each  1503  mapping  400  the preferred embodiment creates  1504  a new mapping C “struct” element and a new mapping function, as shown in the pseudo-code  1505 . The mapping “struct” transforms are directly named and includes as transform_NAME where NAME is replaced with the transform identifier rather than the array representation shown in the pseudo-code  1505 . The mapping function takes the parent structure or map (either or, depending on whether the parent is a structure or map) and calls the corresponding transform script for each transform in the mapping  400 , returning a new mapping_type instance. 
         [0108]    For each  1506  generator  500  the preferred embodiment creates  1507  a new generation function  504 , as shown in the pseudo-code  1508 . This function accepts the generator&#39;s  500  settings and calls the corresponding initializer script with the settings for each initializer, returning a new event instance  600  of the structure_type upon completion. 
         [0109]    For each  1509  filter  700  the preferred embodiment creates  1510  a new counting function  704  and a new enumeration function  705 , as shown in the pseudo-code  1511 . The counting function  704  accepts the filter&#39;s  700  settings and calls the criterion script for each criterion on each structure_type event instance  600  in the database. The count of how many event instances  600  matched the criteria is returned. The enumeration function accepts the filter&#39;s  700  settings, an optional cursor object to page through results, and a fetch size parameter. The enumeration function  705  calls the criterion script for each criterion on each structure_type event instance  600  within the cursor and fetch size page window, returning the actual event data of structure_type for all event instances  600  matching the criteria. 
         [0110]    For each  1512  analysis  800  the preferred embodiment creates  1513  an analysis reduction type to hold analysis results and an analysis function  804 , as shown in the pseudo-code  1514 . The analysis function  804  accepts the analysis&#39;  800  settings, a filter  700 , and settings for the filter. This function  804  then calls the reduction script with the setting, filter, and filter settings for each of the analysis&#39;  800  reductions, reducing across the parameterized filter events for each reduction variable. Once reduction is complete, the analysis function  804  returns the resulting analysis_type object. Having completed generation of all “structs” and functions, the finishing boilerplate code is created  1515 , source generation  1006  is complete, and compilation and relinking  1007  may commence. 
       Computer Program Code and Machine-Readable Storage Media 
       [0111]    Any combination of one or more computer readable medium(s) may be utilized in connection with implementation of various embodiments. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. 
         [0112]    More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction implementation system, apparatus, or device. 
         [0113]    A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction implementation system, apparatus, or device. 
         [0114]    Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc. or any suitable combination of the foregoing. 
         [0115]    Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may be implemented entirely on the user&#39;s computer, partly on the user&#39;s computer, as a stand-alone software package, partly on the user&#39;s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user&#39;s computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). 
         [0116]    It will be obvious to those having skill in the art that many changes may be made to the details of the above-described embodiments without departing from the underlying principles of the invention. The scope of the present invention should, therefore, be determined only by the following claims.

Technology Category: 3