Determining threshold values based on sampled data

Some embodiments provide a program that queries a database for a subset of a plurality of records in the database. Each record in the plurality of records includes a value for a field. The program further samples the subset of the plurality of records to identify a set of records in the subset of the plurality of records. The program also sorts the set of records based on the value for the field in each record in the set of records. The program further determines a first value for the field of a first record in the sorted set of records and a second value for the field of a second record in the sorted set of records forms a slope that is greater than or equal to a defined slope. The program determines a threshold value for the subset of the plurality of records based on the first record.

BACKGROUND

In data processing, many different techniques and approaches may be used to analyze sets of data and discover useful information from the sets of data. For example, data cleansing operations can be performed on sets of data in order to identify incorrect or corrupt data and either fix remove the identified data. Data transformation may be performed on sets of data in order to convert the data into a desirable format or structure that is easier or more efficient to work with. As another example, various operations can be performed on the sets of data in order to calculate descriptive statistics that describe or summarize the sets of data. Examples of descriptive statistics include medians, means, modes, standard deviations, variances, etc.

SUMMARY

In some embodiments, a non-transitory machine-readable medium stores a program executable by at least one processing unit of a device. The program queries a database for a subset of a plurality of records in the database. Each record in the plurality of records includes a value for a field. The program further samples the subset of the plurality of records to identify a set of records in the subset of the plurality of records. The program also sorts the set of records based on the value for the field in each record in the set of records. The program further determines a first value for the field of a first record in the sorted set of records and a second value for the field of a second record in the sorted set of records forms a slope that is greater than or equal to a defined slope. The program also determines a threshold value for the subset of the plurality of records based on the first record.

In some embodiments, determining that the first value for the field of the first record and the second value for the field of the second record forms the slope greater than the defined slope may include calculating a gradient value based on the first value for the field of the first record and the second value for the field of the second record and determine that the gradient value is greater than or equal to a defined gradient value. Sorting the set of records based on the value for the field in each record in the set of records may include sorting the set of records from least to greatest based on the value for the field in each record in the set of records.

In some embodiments, the program may further store the threshold value in a storage for later processing of newly received records. The program may further receive from a first client device a third record that includes a third value for the field; determine that the third value for the field of the third record is greater than or equal to the threshold value; and, based on the determination that the third value for the field of the third record is greater than or equal to the threshold value, send a notification to a second client device indicating that the third record is to be reviewed.

In some embodiments, the subset of the plurality of records may be a first subset of the plurality of records, the set of records may be a first set of records, the slope may be a first slope, and the threshold value may be a first threshold value. The program may further query the database for a second subset of the plurality of records in the database; sample the second subset of the plurality of records to identify a second set of records in the second subset of the plurality of records; sort the second set of records based on the value for the field in each record in the second set of records; determine a third value for the field of a third record in the sorted set of records and a fourth value for the field of a fourth record in the sorted set of records forms a second slope that is greater than or equal to the defined slope; and determine a second threshold value for the second subset of the plurality of records based on the third record. A first number of records in the first set of records may be equal to a second number of records in the second set of records.

In some embodiments, a method queries a database for a subset of a plurality of records in the database. Each record in the plurality of records includes a value for a field. The method further samples the subset of the plurality of records to identify a set of records in the subset of the plurality of records. The method also sorts the set of records based on the value for the field in each record in the set of records. The method further determines a first value for the field of a first record in the sorted set of records and a second value for the field of a second record in the sorted set of records forms a slope that is greater than or equal to a defined slope. The method also determines a threshold value for the subset of the plurality of records based on the first record.

In some embodiments, determining that the first value for the field of the first record and the second value for the field of the second record forms the slope greater than the defined slope may include calculating a gradient value based on the first value for the field of the first record and the second value for the field of the second record and determine that the gradient value is greater than or equal to a defined gradient value. Sorting the set of records based on the value for the field in each record in the set of records may include sorting the set of records from least to greatest based on the value for the field in each record in the set of records.

In some embodiments, the method may further store the threshold value in a storage for later processing of newly received records. The method may further receive from a first client device a third record that includes a third value for the field; determine that the third value for the field of the third record is greater than or equal to the threshold value; and, based on the determination that the third value for the field of the third record is greater than or equal to the threshold value, send a notification to a second client device indicating that the third record is to be reviewed.

In some embodiments, the subset of the plurality of records may be a first subset of the plurality of records, the set of records may be a first set of records, the slope may be a first slope, and the threshold value may be a first threshold value. The method may further query the database for a second subset of the plurality of records in the database; sample the second subset of the plurality of records to identify a second set of records in the second subset of the plurality of records; sort the second set of records based on the value for the field in each record in the second set of records; determine a third value for the field of a third record in the sorted set of records and a fourth value for the field of a fourth record in the sorted set of records forms a second slope that is greater than or equal to the defined slope; and determine a second threshold value for the second subset of the plurality of records based on the third record. A first number of records in the first set of records may be equal to a second number of records in the second set of records.

In some embodiments, a system includes a set of processing units and a non-transitory machine-readable medium that stores instructions. The instructions cause the at least one processing unit to query a database for a subset of a plurality of records in the database. Each record in the plurality of records comprising a value for a field. The instructions further cause the at least one processing unit to sample the subset of the plurality of records to identify a set of records in the subset of the plurality of records. The instructions also cause the at least one processing unit to sort the set of records based on the value for the field in each record in the set of records. The instructions further cause the at least one processing unit to determine a first value for the field of a first record in the sorted set of records and a second value for the field of a second record in the sorted set of records forms a slope that is greater than or equal to a defined slope. The instructions also cause the at least one processing unit to determine a threshold value for the subset of the plurality of records based on the first record.

In some embodiments, determining that the first value for the field of the first record and the second value for the field of the second record forms the slope greater than the defined slope may include calculating a gradient value based on the first value for the field of the first record and the second value for the field of the second record and determine that the gradient value is greater than or equal to a defined gradient value. Sorting the set of records based on the value for the field in each record in the set of records may include sorting the set of records from least to greatest based on the value for the field in each record in the set of records.

In some embodiments, the instructions may further cause the at least one processing unit to store the threshold value in a storage for later processing of newly received records. The instructions may further cause the at least one processing unit to receive from a first client device a third record that includes a third value for the field; determine that the third value for the field of the third record is greater than or equal to the threshold value; and, based on the determination that the third value for the field of the third record is greater than or equal to the threshold value, send a notification to a second client device indicating that the third record is to be reviewed.

In some embodiments, the subset of the plurality of records may be a first subset of the plurality of records, the set of records may be a first set of records, the slope may be a first slope, and the threshold value may be a first threshold value. The instructions may further cause the at least one processing unit to query the database for a second subset of the plurality of records in the database; sample the second subset of the plurality of records to identify a second set of records in the second subset of the plurality of records; sort the second set of records based on the value for the field in each record in the second set of records; determine a third value for the field of a third record in the sorted set of records and a fourth value for the field of a fourth record in the sorted set of records forms a second slope that is greater than or equal to the defined slope; and determine a second threshold value for the second subset of the plurality of records based on the third record. A first number of records in the first set of records may be equal to a second number of records in the second set of records.

The following detailed description and accompanying drawings provide a better understanding of the nature and advantages of various embodiments of the present disclosure.

DETAILED DESCRIPTION

Described herein are techniques for determining threshold values based on sampled data. In some embodiments, a computing system may receive data records (also referred to as records) from any number of different client devices and store them in a storage. Each of the received records includes a value for a particular field. The computing system periodically determines threshold values for different subsets of records stored in the storage. To determine a threshold value for a particular subset of records, the computing system queries the storage for the subset of records. Next, the computing system can sample a set of records from the subset of records. Then, the computing system sorts the set of records based on the values for the particular field from least to greatest. The computing system iterates through each pair of successive records (also referred to as adjacent records or neighboring records) in the sorted set of records and calculates the slope formed by the pair of successive records. When the computing system encounters a slope for a particular pair of successive records that is greater than or equal to a defined slope, the computing system determines the value for the particular field of one of the records in the pair to be the threshold value for the subset of records.

The techniques described in the present application provide a number of benefits and advantages over conventional methods of determining threshold values for sets of data. For instance, by sampling the set of data first and then determining a threshold value for the set of data based on the sampled data, the computing system is able to determine threshold values quickly and efficiently since determining threshold values in such a manner is utilizes less processing resources. Moreover, the computing system can employ techniques where only a portion of the sampled data is processed to determine threshold values. This allows the computing system to determine threshold values even faster and more efficient as even less processing resources are utilized with these techniques.

As mentioned above, the techniques described here are used to determine threshold values based on sampled data. There are many applications to which the techniques may be applied. For example, the computing system may be an expense management system that manages and processes expense reports received from client devices. Each expense report includes one or more line items. Each line item can include a number of different fields such as, for example, a date, a vendor name, a location, an expense type, an amount of the expense, etc. In some instances, the particular field of interest is the amount of an expense. The techniques described here can be applied to determine a threshold amount for a given set of line items that have the same type, the same location, the same vendor name, etc. or any combination thereof. The determined threshold amounts may be used to determine whether newly received expense reports have line items that need to be reviewed or manually approved because such line items contain amounts that are identified as unreasonable (e.g., too high).

FIG.1illustrates a system100for determining threshold values based on sampled data according to some embodiments. As shown, system100includes client devices105a-n, computing system115, records storage140, and threshold values storage145. Records storage140is configured to store records (e.g., line items in expense reports) received from client devices105a-n. Threshold values storage145can store threshold values determined for different subsets of records stored in records storage140. In some embodiments, storages140and145are implemented in a single physical storage while, in other embodiments, storages140and145may be implemented across several physical storages. WhileFIG.1shows storages140and145as external to computing system115, one of ordinary skill in the art will appreciate that records storage140and/or threshold values storage145may be part of computing system115in some embodiments.

Client devices105a-ncan be configured to communicate and interact with computing system115. For example, each client device100includes a client application (e.g., a desktop application, a mobile application, a web browser, etc.) operating on the client device110. A user of a client device110may use the client application to access computing system115and transmit records to computing system115. In some embodiments, the records are line items that the user of the client device110sends to computing system115in the form of expense reports. An expense report can include one or more line items. Each line item in an expense report can include a set of fields. Examples of fields in a line item include a date on which the expense was incurred, a name of a vendor associated with the expense, a location where the expense was incurred, a type of the expense (e.g., meals, lodging, transport, parking, entertainment, etc.), an amount of the expense, etc. A user of a client device110can also receive notifications from computing system115indicating that one or more records is to be reviewed (e.g., the records have a value for a field that is greater than or equal to a threshold value determined by computing system115).

Computing system115is configured to manage records and determine threshold values for different subsets of records. For instance, computing system115may receive records from client devices105a-n. Upon receiving records from a client device105, computing system115stores them in records storage140. As mentioned above, in some embodiments, the records received from client devices105a-nare line items. In some such embodiments, the line items that computing system115receives from client devices105a-nare in the form of expense reports.

As shown inFIG.1, computing system115includes query manager120, query processor125, data preprocessor130, and threshold manager135. Query manager120is configured to manage queries for subsets of records stored in records storage140. In some embodiments, the queries are predefined queries for a particular subset of records stored in records storage140. For example, in cases where the records are line items, examples of predefined queries include a query for line items having a meals expense type, a query for line items having a meals expense type that were incurred in San Francisco, Calif., a query for line items having a meals expense type that were incurred in Los Angeles, Calif., a query for line items having a hotel expense type, a query for line items having a hotel expense type that were incurred in Seattle, Wash., a query for line items having a hotel expense type that were incurred in Las Vegas, Nev., etc. One of ordinary skill in the art will realize that query manager120can be configured with any number of additional and/or different predefined queries.

At defined intervals (e.g., once per day, once per week, once a month, etc.), query manager125sends out for processing queries for different sets of records based on the predefined queries. For example, for each predefined query, query manager120generates the query specified by the predefined query and sends it to query processor125for further processing. When query manager120receives the set of records back from query processor125, query manager120sends the set of records to data preprocessor130for further processing.

Query processor125handles the processing of queries. For instance, when query processor125receives a query from query manager120, query processor125accesses records storage140and retrieves a set of records specified by the query. Then, query processor125sends the set of records to query manager125.

Data preprocessor130is responsible for performing various preprocessing operations on sets of records. For example, data preprocessor130may receive a set of records from query manager120. In response, data preprocessor130samples the set of records to identify a subset of the set of records. In some embodiments, data preprocessor130samples the set of records by randomly selecting a defined number (e.g., one hundred, one thousand, two thousand, five thousand, etc.) of samples from the set of records. In some instances, the defined number of samples is used for sampling all the different subsets of records. In other instances, different defined numbers of samples are used for sampling different subsets of records. Then, data preprocessor130sorts the sampled subset of records. As an example, in cases where records are line items, data preprocessor130sorts the sampled subset of records based on the amount field of each record from least to greatest. That is, the starting end of the sorted subset of records has the lowest value for the amount field and the ending end of the sorted subset of records has the highest value for the amount field.

In some embodiments, data preprocessor130can sort the set of records first and then sample the sorted set of records to identify a subset of the set of records. For instance, in some such embodiments, when data preprocessor130receives the set of records from query manager120, data preprocessor130sorts the set of records based on the amount field of each record from least to greatest in a similar manner described above. Then, data preprocessor130selects a defined number (e.g., one hundred, one thousand, two thousand, five thousand, etc.) of samples from the sorted set of records. In some embodiments, data preprocessor130samples the defined number of samples from the sorted set of records by dividing the number of records in the set of records by the defined number of samples and using the quotient as an offset to select records in the sorted set of records. For example, if there are 140,000 records in the set of records and the defined number of samples is 2,000, the quotient is 70 (140,000/2,000). Then, data preprocessor130selects the 70th record in the sorted set of records as the first sampled record, the 140th record in the sorted set of records as the second sampled record, the 210th record in the sorted set of records as the third sampled record, and so on and so forth, until data preprocessor130selects the 140,000th record in the sorted set of records as the 2,000th sampled record. Once data preprocessor130has preprocessed the set of records, data preprocessor130sends the processed set of records to threshold manager135.

FIG.2illustrates an example chart200representing sampled and sorted data according to some embodiments. In this example, the set of records from which the records represented by the points in chart200were sampled and sorted is 1,000 line items having a meal expense type that were incurred in San Francisco, Calif. The defined number of samples for this example is 47 samples. As shown inFIG.2, the range of the values for the amount field in the sampled and sorted set of records is approximately between S9and S94.

Threshold manager135is configured to determine threshold values for sets of records. For instance, upon receiving a set of records (e.g., a set of sampled records that has been sorted based on the amount field) from data preprocessor130, threshold manager135iterates through each pair of successive records in the sorted set of records and calculates the slope formed by the pair of successive records (e.g., based on values for fields in the record). Next, threshold manager135determines whether the calculated slope is greater than or equal to a defined slope (e.g., 30 degrees, 45 degrees, 55 degrees, etc.). If the calculated slope is not greater than or equal to the defined slope, threshold manager135continues to process pairs of successive records in the set of records. When threshold manager135reaches a pair of successive records that has a calculated slope that is greater than or equal to the defined slope, threshold manager135uses the value of the amount field of one of the records in the pair of successive records as the threshold value for the set of records.

Threshold manager135may use any number of different techniques to determine the slope formed by a pair of successive records. For instance, in some embodiments, threshold manager135can calculate a gradient value based on values for a field (e.g., an amount field) of the pair of successive records and determine whether the gradient value is greater than or equal to a defined gradient value. The defined gradient value can correspond to a defined slope. For example, a defined gradient value of 1.73 corresponds to a slope of 30 degrees, a defined gradient value of 1 corresponds to a slope of 45 degrees, a defined gradient value of 0.6 corresponds to a slope of 60 degrees, etc.

FIG.3illustrates a threshold value determined based on the example chart illustrated inFIG.2according to some embodiments. For this example, the defined slope that threshold manager135uses to determine the threshold value is 45 degrees. To determine the threshold value, threshold manager135iterated through each pair of successive records in the sorted set of records and calculates the slope formed by the pair of successive records until a slope formed by a particular pair of successive records is greater than or equal to the defined slope. For instance, threshold manager135may start by calculating the slope formed by the leftmost two points in chart200. Then, threshold manager135can calculate the slope formed by the second leftmost two points in chart200(i.e., the second and third points starting from the left side of chart200), the slope formed by the third leftmost two points in chart200(i.e., the third and fourth points starting from the left side of chart200), and so on and so forth until threshold manager135reaches the pair of successive records represented by points305and310. Here, threshold manager135calculated that the slope formed by the pair of successive records represented by points305and310is 45 degrees. Since this slope is the first slope that threshold manager135encountered that is greater than or equal to the defined slope, threshold manager135determines the value for the amount field of the record represented by point305as the threshold value. In this example, the value for the amount field of the record represented by point305is approximately S41. As such, threshold manager135determines that S41is the threshold value for line items having a meal expense type that were incurred in San Francisco, Calif.

In some embodiments, threshold manager135starts processing pairs of successive records from the beginning of the sorted set of records. Referring toFIG.3as an example, threshold manager135can start from the left hand side of chart200. In some embodiments, threshold manager135may start at a different starting point. For instance, threshold manager135may determine the median record in the sorted set of records and start from the point represented by the median record. Referring toFIG.3as an example, threshold manager135determines that the record represented by point315is the median record in the set of records. As such, threshold manager starts processing the pair of successive records formed by the record represented by point315and the record represented by the point to the right of point315and continues processing pairs of successive records towards the right of point315in a similar fashion described above. As another example, threshold manager135can start from the right hand side of chart200. Then, threshold manager135may iterate through and process pairs of successive records towards the left. In some such cases, the slope of the points represented by pairs of successive on the right hand side may be greater than or equal to the defined slope. Thus, when threshold manager135encounters the first slope formed by a pair of successive records that is not greater than or equal to (i.e., less than) the defined slope, threshold manager135determines the value for the amount field of the record represented by one of the two points (e.g., the left point or the right point) representing the records as the threshold value for the set of data. Referring toFIG.3as an example, the first slope that threshold manager135encounters that is less than the defined slope is the slope formed by the pair of successive record represented by points320and305. Thus, threshold manager135determines the value for the amount field of the record represented by point305as the threshold value. In this example chart200, the two approaches (e.g., starting from the left/middle of chart200and iterating right and starting from the right and iterating left) arrive at the same threshold value. In some instances where the two approaches result in two different threshold values, threshold manager135may select one of the threshold values or calculate an average of the two threshold values as being the threshold value for the set of data (e.g., line items having a meal expense type that were incurred in San Francisco, Calif. for the example inFIG.3).

In some embodiments, threshold manager135does not calculate slopes formed by pairs of successive records. Rather, in some such embodiments, threshold manager135uses an extrapolation technique to determine a function based on the set of records. The function that takes the relative position of a record in sorted set of records as an input and outputs the corresponding value for the amount field of the record. Examples of such functions include a polynomial function. After determining the function, threshold manager135determines the derivative of the function. In some embodiments, threshold manager135calculates uses inputs values (e.g., the relative position of a record in the sorted set of records) for the derivative function starting from the beginning of the sorted set of records and then calculates the first value that the derivative function outputs above the defined slope. In other embodiments, threshold manager135uses input values (e.g., the relative position of a record in the sorted set of records) for the derivative function starting from the median record in the set of records and then calculates the first value that the derivative function outputs above the defined slope. Threshold manager125then uses the first value as the input to the determined function and uses the output from the determined function as the determined threshold value for the set of data.

After threshold manager135determines a threshold value for a set of data (i.e., a set of data that is the result set of a predefined query), threshold manager135stores the threshold value in threshold values storage145for later processing. For example, when computing system115receives an expense report that includes a line item that belongs to a particular set of data, computing system115accesses threshold values storage145and retrieves the threshold value determined for the particular set of data and determines, based on the threshold value, whether the line item needs to be reviewed. For example, if the value of the amount field for the line item is greater than or equal to the threshold value, computing system115marks the line item for review. In some embodiments, computing system115sends a notification to a user (e.g., a user in charge with reviewing and/or approving line items) of a client device105indicating that the line item is to be reviewed.

FIG.4illustrates a process400for determining a threshold values for a sampled set of data according to some embodiments. In some embodiments, computing system115performs process400. Process400begins by querying, at410a database for a subset of a plurality of records in the database, each record in the plurality of records comprising a value for a field. Referring toFIGS.1and2as an example, query manager120may generated a predefined query for line items having a meal expense type that were incurred in San Francisco, Calif. Query manager120sends the generated query to query processor125, which performs operation410to retrieve a subset of a plurality of records stored in records storage140. For this example, the subset of the plurality of records includes line items having a meal expense type that were incurred in San Francisco, Calif.

Next, process400samples, at420, the subset of the plurality of records to identify a set of records in the subset of the plurality of records. Referring toFIGS.1and2as an example, once data preprocessor130receives the subset of the plurality of records from query manager120, data preprocessor130samples the subset of the plurality of records to identify a set of the records. As mentioned above, in some embodiments, data preprocessor130may select a defined number of samples from the subset of the plurality of records to be include in the set of records.

Process400then sorts, at430, the set of records based on the value for the field in each record in the set of records. Referring toFIGS.1and2as an example, data preprocessor130sorts the set of records based on the value of the amount field in each of the records in the set of records. In this example, data preprocessor130sorts the records in the set of records according to the value of the amount field from least to greatest. As shown in chart200ofFIG.2, the set of records (represented by points) are sorted from the lowest value for the amount field to the largest value for the amount field.

After operation430, process400determines, at440, a first value for the field of a first record in the sorted set of records and a second value for the field of a second record in the sorted set of records forms a slope that is greater than or equal to a defined slope. Referring toFIGS.1and3as an example, upon receiving the sampled and sorted set of records from data preprocessor130, threshold manager135starts iterating through pairs of successive records in the set of records and calculating slopes formed by the pairs of successive records.FIG.3shows threshold manager135encountering the first pair of successive records that has a calculated slope that is greater than or equal to the defined slope. These records are represented by points305and310.

Finally, process400determines, at450, a threshold value for the subset of the plurality of records based on the first record. Referring toFIGS.1and3as an example, when threshold manager135reaches the pair of successive records represented by points305and310, which has a calculated slope that is greater than or equal to the defined slope, threshold manager135determines that the value of the amount field of one of the records in the pair of successive records as the threshold value for the subset of the plurality of records. For this example, threshold manager135determines that the value of the amount field for the record represented by point305as the threshold value for the subset of the plurality of records comprised of line items having a meal expense type that were incurred in San Francisco, Calif.

FIG.5illustrates an exemplary computer system500for implementing various embodiments described above. For example, computer system500may be used to implement systems client devices105a-nand computing system115. Computer system500may be a desktop computer, a laptop, a server computer, or any other type of computer system or combination thereof. Some or all elements of client applications110a-n, query manager120, query processor125, data preprocessor130, threshold manager135, or combinations thereof can be included or implemented in computer system500. In addition, computer system500can implement many of the operations, methods, and/or processes described above (e.g., process400). As shown inFIG.5, computer system500includes processing subsystem502, which communicates, via bus subsystem526, with input/output (I/O) subsystem508, storage subsystem510and communication subsystem524.

Bus subsystem526is configured to facilitate communication among the various components and subsystems of computer system500. While bus subsystem526is illustrated inFIG.5as a single bus, one of ordinary skill in the art will understand that bus subsystem526may be implemented as multiple buses. Bus subsystem526may be any of several types of bus structures (e.g., a memory bus or memory controller, a peripheral bus, a local bus, etc.) using any of a variety of bus architectures. Examples of bus architectures may include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, a Peripheral Component Interconnect (PCI) bus, a Universal Serial Bus (USB), etc.

Processing subsystem502, which can be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of computer system500. Processing subsystem502may include one or more processors504. Each processor504may include one processing unit506(e.g., a single core processor such as processor504-1) or several processing units506(e.g., a multicore processor such as processor504-2). In some embodiments, processors504of processing subsystem502may be implemented as independent processors while, in other embodiments, processors504of processing subsystem502may be implemented as multiple processors integrate into a single chip or multiple chips. Still, in some embodiments, processors504of processing subsystem502may be implemented as a combination of independent processors and multiple processors integrated into a single chip or multiple chips.

In some embodiments, processing subsystem502can execute a variety of programs or processes in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in processing subsystem502and/or in storage subsystem510. Through suitable programming, processing subsystem502can provide various functionalities, such as the functionalities described above by reference to process400, etc.

I/O subsystem508may include any number of user interface input devices and/or user interface output devices. User interface input devices may include a keyboard, pointing devices (e.g., a mouse, a trackball, etc.), a touchpad, a touch screen incorporated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, audio input devices with voice recognition systems, microphones, image/video capture devices (e.g., webcams, image scanners, barcode readers, etc.), motion sensing devices, gesture recognition devices, eye gesture (e.g., blinking) recognition devices, biometric input devices, and/or any other types of input devices.

User interface output devices may include visual output devices (e.g., a display subsystem, indicator lights, etc.), audio output devices (e.g., speakers, headphones, etc.), etc. Examples of a display subsystem may include a cathode ray tube (CRT), a flat-panel device (e.g., a liquid crystal display (LCD), a plasma display, etc.), a projection device, a touch screen, and/or any other types of devices and mechanisms for outputting information from computer system500to a user or another device (e.g., a printer).

As illustrated inFIG.5, storage subsystem510includes system memory512, computer-readable storage medium520, and computer-readable storage medium reader522. System memory512may be configured to store software in the form of program instructions that are loadable and executable by processing subsystem502as well as data generated during the execution of program instructions. In some embodiments, system memory512may include volatile memory (e.g., random access memory (RAM)) and/or non-volatile memory (e.g., read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, etc.). System memory512may include different types of memory, such as static random access memory (SRAM) and/or dynamic random access memory (DRAM). System memory512may include a basic input/output system (BIOS), in some embodiments, that is configured to store basic routines to facilitate transferring information between elements within computer system500(e.g., during start-up). Such a BIOS may be stored in ROM (e.g., a ROM chip), flash memory, or any other type of memory that may be configured to store the BIOS.

As shown inFIG.5, system memory512includes application programs514(e.g., client application110a-n), program data516, and operating system (OS)518. OS518may be one of various versions of Microsoft Windows, Apple Mac OS, Apple OS X, Apple macOS, and/or Linux operating systems, a variety of commercially-available UNIX or UNIX-like operating systems (including without limitation the variety of GNU/Linux operating systems, the Google Chrome® OS, and the like) and/or mobile operating systems such as Apple iOS, Windows Phone, Windows Mobile, Android, BlackBerry OS, Blackberry 10, and Palm OS, WebOS operating systems.

Computer-readable storage medium520may be a non-transitory computer-readable medium configured to store software (e.g., programs, code modules, data constructs, instructions, etc.). Many of the components (e.g., client applications110a-n, query manager120, query processor125, data preprocessor130, and threshold manager135) and/or processes (e.g., process400) described above may be implemented as software that when executed by a processor or processing unit (e.g., a processor or processing unit of processing subsystem502) performs the operations of such components and/or processes. Storage subsystem510may also store data used for, or generated during, the execution of the software.

Storage subsystem510may also include computer-readable storage medium reader522that is configured to communicate with computer-readable storage medium520. Together and, optionally, in combination with system memory512, computer-readable storage medium520may comprehensively represent remote, local, fixed, and/or removable storage devices plus storage media for temporarily and/or more permanently containing, storing, transmitting, and retrieving computer-readable information.

Computer-readable storage medium520may be any appropriate media known or used in the art, including storage media such as volatile, non-volatile, removable, non-removable media implemented in any method or technology for storage and/or transmission of information. Examples of such storage media includes RAM, ROM, EEPROM, flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray Disc (BD), magnetic cassettes, magnetic tape, magnetic disk storage (e.g., hard disk drives), Zip drives, solid-state drives (SSD), flash memory card (e.g., secure digital (SD) cards, CompactFlash cards, etc.), USB flash drives, or any other type of computer-readable storage media or device.

Communication subsystem524serves as an interface for receiving data from, and transmitting data to, other devices, computer systems, and networks. For example, communication subsystem524may allow computer system500to connect to one or more devices via a network (e.g., a personal area network (PAN), a local area network (LAN), a storage area network (SAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a global area network (GAN), an intranet, the Internet, a network of any number of different types of networks, etc.). Communication subsystem524can include any number of different communication components. Examples of such components may include radio frequency (RF) transceiver components for accessing wireless voice and/or data networks (e.g., using cellular technologies such as 2G, 3G, 4G, 5G, etc., wireless data technologies such as Wi-Fi, Bluetooth, ZigBee, etc., or any combination thereof), global positioning system (GPS) receiver components, and/or other components. In some embodiments, communication subsystem524may provide components configured for wired communication (e.g., Ethernet) in addition to or instead of components configured for wireless communication.

One of ordinary skill in the art will realize that the architecture shown inFIG.5is only an example architecture of computer system500, and that computer system500may have additional or fewer components than shown, or a different configuration of components. The various components shown inFIG.5may be implemented in hardware, software, firmware or any combination thereof, including one or more signal processing and/or application specific integrated circuits.

FIG.6illustrates an exemplary computing device600for implementing various embodiments described above. For example, computing device600may be used to implement devices client devices105a-n. Computing device600may be a cellphone, a smartphone, a wearable device, an activity tracker or manager, a tablet, a personal digital assistant (PDA), a media player, or any other type of mobile computing device or combination thereof. Some or all elements of client applications60a-nor combinations thereof can be included or implemented in computing device600. As shown inFIG.6, computing device600includes processing system602, input/output (I/O) system608, communication system618, and storage system620. These components may be coupled by one or more communication buses or signal lines.

Processing system602, which can be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of computing device600. As shown, processing system602includes one or more processors604and memory606. Processors604are configured to run or execute various software and/or sets of instructions stored in memory606to perform various functions for computing device600and to process data.

Each processor of processors604may include one processing unit (e.g., a single core processor) or several processing units (e.g., a multicore processor). In some embodiments, processors604of processing system602may be implemented as independent processors while, in other embodiments, processors604of processing system602may be implemented as multiple processors integrate into a single chip. Still, in some embodiments, processors604of processing system602may be implemented as a combination of independent processors and multiple processors integrated into a single chip.

Memory606may be configured to receive and store software (e.g., operating system622, applications624, I/O module626, communication module628, etc. from storage system620) in the form of program instructions that are loadable and executable by processors604as well as data generated during the execution of program instructions. In some embodiments, memory606may include volatile memory (e.g., random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, etc.), or a combination thereof.

I/O system608is responsible for receiving input through various components and providing output through various components. As shown for this example, I/O system608includes display610, one or more sensors612, speaker614, and microphone616. Display610is configured to output visual information (e.g., a graphical user interface (GUI) generated and/or rendered by processors604). In some embodiments, display610is a touch screen that is configured to also receive touch-based input. Display610may be implemented using liquid crystal display (LCD) technology, light-emitting diode (LED) technology, organic LED (OLED) technology, organic electro luminescence (OEL) technology, or any other type of display technologies. Sensors612may include any number of different types of sensors for measuring a physical quantity (e.g., temperature, force, pressure, acceleration, orientation, light, radiation, etc.). Speaker614is configured to output audio information and microphone616is configured to receive audio input. One of ordinary skill in the art will appreciate that I/O system608may include any number of additional, fewer, and/or different components. For instance, I/O system608may include a keypad or keyboard for receiving input, a port for transmitting data, receiving data and/or power, and/or communicating with another device or component, an image capture component for capturing photos and/or videos, etc.

Communication system618serves as an interface for receiving data from, and transmitting data to, other devices, computer systems, and networks. For example, communication system618may allow computing device600to connect to one or more devices via a network (e.g., a personal area network (PAN), a local area network (LAN), a storage area network (SAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a global area network (GAN), an intranet, the Internet, a network of any number of different types of networks, etc.). Communication system618can include any number of different communication components. Examples of such components may include radio frequency (RF) transceiver components for accessing wireless voice and/or data networks (e.g., using cellular technologies such as 2G, 3G, 4G, 5G, etc., wireless data technologies such as Wi-Fi, Bluetooth, ZigBee, etc., or any combination thereof), global positioning system (GPS) receiver components, and/or other components. In some embodiments, communication system618may provide components configured for wired communication (e.g., Ethernet) in addition to or instead of components configured for wireless communication.

Storage system620handles the storage and management of data for computing device600. Storage system620may be implemented by one or more non-transitory machine-readable mediums that are configured to store software (e.g., programs, code modules, data constructs, instructions, etc.) and store data used for, or generated during, the execution of the software. Many of the components (e.g., client applications60a-n) described above may be implemented as software that when executed by a processor or processing unit (e.g., processors604of processing system602) performs the operations of such components and/or processes.

In this example, storage system620includes operating system622, one or more applications624, I/O module626, and communication module628. Operating system622includes various procedures, sets of instructions, software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components. Operating system622may be one of various versions of Microsoft Windows, Apple Mac OS, Apple OS X, Apple macOS, and/or Linux operating systems, a variety of commercially-available UNIX or UNIX-like operating systems (including without limitation the variety of GNU/Linux operating systems, the Google Chrome® OS, and the like) and/or mobile operating systems such as Apple iOS, Windows Phone, Windows Mobile, Android, BlackBerry OS, Blackberry 10, and Palm OS, WebOS operating systems.

Applications624can include any number of different applications installed on computing device600. For example, client applications60a-nmay be installed on computing device600. Other examples of such applications may include a browser application, an address book application, a contact list application, an email application, an instant messaging application, a word processing application, JAVA-enabled applications, an encryption application, a digital rights management application, a voice recognition application, location determination application, a mapping application, a music player application, etc.

I/O module626manages information received via input components (e.g., display610, sensors612, and microphone616) and information to be outputted via output components (e.g., display610and speaker614). Communication module628facilitates communication with other devices via communication system618and includes various software components for handling data received from communication system618.

One of ordinary skill in the art will realize that the architecture shown inFIG.6is only an example architecture of computing device600, and that computing device600may have additional or fewer components than shown, or a different configuration of components. The various components shown inFIG.6may be implemented in hardware, software, firmware or any combination thereof, including one or more signal processing and/or application specific integrated circuits.

FIG.7illustrates an exemplary system700for implementing various embodiments described above. For example, cloud computing system612may be used to implement computing system115and client devices702-708may be used to implement client devices105a-n. As shown, system700includes client devices702-708, one or more networks710, and cloud computing system712. Cloud computing system712is configured to provide resources and data to client devices702-708via networks710. In some embodiments, cloud computing system700provides resources to any number of different users (e.g., customers, tenants, organizations, etc.). Cloud computing system712may be implemented by one or more computer systems (e.g., servers), virtual machines operating on a computer system, or a combination thereof.

As shown, cloud computing system712includes one or more applications714, one or more services716, and one or more databases718. Cloud computing system700may provide applications714, services716, and databases718to any number of different customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner.

In some embodiments, cloud computing system700may be adapted to automatically provision, manage, and track a customer's subscriptions to services offered by cloud computing system700. Cloud computing system700may provide cloud services via different deployment models. For example, cloud services may be provided under a public cloud model in which cloud computing system700is owned by an organization selling cloud services and the cloud services are made available to the general public or different industry enterprises. As another example, cloud services may be provided under a private cloud model in which cloud computing system700is operated solely for a single organization and may provide cloud services for one or more entities within the organization. The cloud services may also be provided under a community cloud model in which cloud computing system700and the cloud services provided by cloud computing system700are shared by several organizations in a related community. The cloud services may also be provided under a hybrid cloud model, which is a combination of two or more of the aforementioned different models.

In some instances, any one of applications714, services716, and databases718made available to client devices702-708via networks710from cloud computing system700is referred to as a “cloud service.” Typically, servers and systems that make up cloud computing system700are different from the on-premises servers and systems of a customer. For example, cloud computing system700may host an application and a user of one of client devices702-708may order and use the application via networks710.

Applications714may include software applications that are configured to execute on cloud computing system712(e.g., a computer system or a virtual machine operating on a computer system) and be accessed, controlled, managed, etc. via client devices702-708. In some embodiments, applications714may include server applications and/or mid-tier applications (e.g., HTTP (hypertext transport protocol) server applications, FTP (file transfer protocol) server applications, CGI (common gateway interface) server applications, JAVA server applications, etc.). Services716are software components, modules, application, etc. that are configured to execute on cloud computing system712and provide functionalities to client devices702-708via networks710. Services716may be web-based services or on-demand cloud services.

Databases718are configured to store and/or manage data that is accessed by applications714, services716, and/or client devices702-708. For instance, storages140and145may be stored in databases718. Databases718may reside on a non-transitory storage medium local to (and/or resident in) cloud computing system712, in a storage-area network (SAN), on a non-transitory storage medium local located remotely from cloud computing system712. In some embodiments, databases718may include relational databases that are managed by a relational database management system (RDBMS). Databases718may be a column-oriented databases, row-oriented databases, or a combination thereof. In some embodiments, some or all of databases718are in-memory databases. That is, in some such embodiments, data for databases718are stored and managed in memory (e.g., random access memory (RAM)).

Client devices702-708are configured to execute and operate a client application (e.g., a web browser, a proprietary client application, etc.) that communicates with applications714, services716, and/or databases718via networks710. This way, client devices702-708may access the various functionalities provided by applications714, services716, and databases718while applications714, services716, and databases718are operating (e.g., hosted) on cloud computing system700. Client devices702-708may be computer system500or computing device600, as described above by reference toFIGS.5and6, respectively. Although system700is shown with four client devices, any number of client devices may be supported.

Networks710may be any type of network configured to facilitate data communications among client devices702-708and cloud computing system712using any of a variety of network protocols. Networks710may be a personal area network (PAN), a local area network (LAN), a storage area network (SAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a global area network (GAN), an intranet, the Internet, a network of any number of different types of networks, etc.