Patent Description:
Information retrieval typically requires a user to have a minimum level of skill with a particular query language. Furthermore, in order to properly construct a query, the user must often know how stored data is structured. For example, a user who wishes to retrieve information from a Structured Query Language (SQL) database may need to be familiar with both SQL and the database schema.

Although user interfaces may be used to push some of the requisite skill for users to programmers, the user interfaces may be limited in the complexity and efficiency of the queries they can be used to generate. For example, web interfaces that employ drop-down menus can be used to specify a limited number of predetermined query parameters. Furthermore, the predetermined query parameters may be used to construct a query that is executed in an inefficient order.

<CIT> discloses methods and apparatus for processing a query joining tables stored at different data sources. <CIT> discloses systems and user interfaces for dynamic and interactive simultaneous querying of multiple data stores using a mathematical graph in which nodes represent data sources and/or portions of data sources and edges represent relationships among data sources and/or portions of data sources.

The example embodiment(s) of the present invention are illustrated, by way of example and not limitation, in the accompanying drawings and in which like reference numerals refer to similar elements and in which:.

In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the example embodiment(s) the present Application. It will be apparent, however, that the example embodiment(s) may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the example embodiment(s). Modifiers such as "first" and "second" may be used to differentiate elements, but the modifiers do not necessarily indicate any particular order.

Computer-implemented techniques for automatic generation of composite datasets based on hierarchical fields are described. In some example embodiments, datasets are annotated with metadata that indicates the interrelationships between/among the datasets. More specifically, the metadata associates categories with dataset fields. Furthermore, a hierarchy mapping indicates hierarchical relationships between/among different categories.

In some example embodiments, the techniques encompass a computer-implemented method performed at one or more computing devices comprising one or more processors and storage media storing one or more computer programs executed by the one or more processors to perform the method. Performance of the method involves performing operations related to query processing that automatically determines which data to use for efficient query execution.

In some example embodiments, the method includes interpreting a natural language query that specifies a first granularity level. The first granularity level indicates a particular category and one or more field values that correspond to the particular category. One or more dataset filters are generated based on the first granularity level. The one or more dataset filters are used to discard, from one or more datasets, any data that is unnecessary for query processing and/or execution.

In some example embodiments, a query result can be returned based on retrieving data from a pre-existing dataset that is determined to be the most relevant dataset for query processing and/or execution. However, in other example embodiments, the method includes generating a composite dataset from which to retrieve the data for returning a query result. The composite dataset is generated based on joining two or more datasets that are among the most relevant datasets for query processing and/or execution. Relevancy is determined based on one or more predetermined criteria. For example, the hierarchy mapping can be used to identify one or more hierarchically related categories of the particular category that was indicated by the query. The two or more datasets can be selected based, at least in part, on their having at least one hierarchically related category in common.

In some example embodiments, the method includes modifying the one or more dataset filters to discard unnecessary data from at least one dataset of the two or more datasets. Modification includes translating the one or more dataset filters to a second granularity level that indicates the at least one hierarchically related category.

In some example embodiments, the two or more datasets are joined after applying any translated dataset filters. Data can be retrieved from the resulting composite dataset to return a query result. A query result can include the data and/or a derivative of the data. For example, any of a number of computations can be performed on the data to generate the derivative.

<FIG> depicts a plurality of example datasets. Referring to <FIG>, dataset 100A includes fields 102A-C, and dataset 100N includes fields <NUM>-N. Each field includes field values. For example, field 102A includes field values 104A-C, and field <NUM> includes field values 104D-F.

Datasets 100A-N can be any structured information stored in volatile and/or non-volatile memory. Example datasets 100A-N include Structured Query Language (SQL) database tables and Spark SQL Resilient Distributed Datasets (RDDs). Each dataset of datasets 100A-N can be organized according to fields 102A-N. Each field of fields 102A-N can be a column, a row, or any other data structure within a dataset. Each field of fields 102A-N can store any of a number of field values 104A-N. With the exception of null field values, all field values that are included in a particular field are typically of the same type. At a minimum, the field values in a particular field typically share a common data type, such as integer, string, etc. However, they also typically share a common field type. For example, the field values "San Jose", "Palo Alto", and "San Francisco" share the common field type of "City". The common field type of the field values in a particular field is referred to hereinafter as the "category" of the field. Other geospatial categories can include "ZIP code", "State", "Region", etc..

For ease of explanation, the datasets of <FIG> can be represented by the following example tables:.

Each table column stores values that share a common category. For example, the "ZIP Code" columns store values of the "ZIP Code" category, and the "State" column stores values of the "State" category. As illustrated in the examples above, each category may correspond to one or more fields 102A-N. Annotating datasets 100A-N with categories that are included in the datasets 100A-N can be useful for determining relationships between the datasets 100A-N.

<FIG> depicts example annotated datasets. Referring to <FIG>, dataset 100A is associated with metadata 200A, and dataset 100N is associated with metadata 200N. Metadata 200A associates field 102B with category 202A. Metadata 200A also associates measure <NUM> with category 202B and field 102C. Metadata 200N associates field <NUM> and 102N with category 202A and category 202N, respectively.

Each dataset of datasets 100A-N can be annotated with respective metadata. The respective metadata can be stored with a corresponding dataset and/or separately from a corresponding dataset (e.g., in a database dictionary). Metadata 200A-N associates at least some of fields 102A-N with categories 202A-N. Metadata 200A-N can also associate a measure with a category and/or a field.

Measures include any of a number of computations that can be performed using data stored in datasets 100A-N. For example, measures can include demographic statistics, revenue statistics, etc. Each measure can be associated with a category that describes a type of computation that is performed. For example, a measure can correspond to a "sum" category, a "weighted mean" category, etc. Each measure can also be associated with at least one field upon which the measure is dependent. A measure is dependent upon one or more fields in a dataset that are necessary to compute the measure for the dataset. In the example of Dataset A above, measure <NUM> can be a "total sales revenue" measure of the "sum" category. The "total sales revenue" measure is dependent on the values in the "Sales Revenue" column. A measure can also be dependent upon another measure. For example, a "November percentage of total sales revenue" measure is dependent on the "total sales revenue" measure.

<FIG> depicts an example hierarchy mapping. Referring to <FIG>, hierarchy mapping <NUM> includes category 202A, category 202B, and category 202N. Category 202A and category 202N exhibit a hierarchical relationship <NUM> with respect to each other.

Hierarchy mapping <NUM> indicates a hierarchical relationship <NUM> between different categories. In <FIG>, category 202A is a subcategory of category 202N, and category 202N is a supercategory of category 202A. For example, in Dataset N above, category 202A is a "ZIP Code" category that is a subcategory of category 202N, which is a "State" category. The closeness of a hierarchical relationship <NUM> between two categories is hereinafter referenced as a "distance" between the two categories. For example, the distance between a "ZIP Code" category and a "City" category is smaller than the distance between a "ZIP Code" category and a "State" category.

Hierarchy mapping <NUM> can be stored with a dataset or separately from a dataset. For example, hierarchy mapping <NUM> can be stored in metadata 200A. Hierarchy mapping <NUM> can indicate all categories that are hierarchically related to a particular category, regardless of whether all the categories are included in a single dataset. For example, even if category 202N does not correspond to any of fields 102A-C in dataset 100A, metadata 200A can include a hierarchy mapping <NUM> that indicates category 202N is hierarchically related to category 202A in dataset 100A. Additionally or alternatively, hierarchy mapping <NUM> can indicate all possible field values that correspond to a particular category. For example, hierarchy mapping <NUM> can indicate all fifty states that correspond to the "State" category.

<FIG> depicts an example query. Referring to <FIG>, natural language query <NUM> specifies measure <NUM> and first granularity level <NUM>. First granularity level <NUM> indicates category 202N and field value 104N.

Natural language query <NUM> can be any information request that is composed in a human language (e.g., conversational English) instead of a computer language (e.g., SQL). For example, the SQL query "SELECT Sales Revenue FROM Dataset A WHERE ZIP Code = <NUM>" can be composed as the natural language query <NUM> "Sales revenue for <NUM>". In other words, natural language query <NUM> can refer to a query that consists only of normal terms in a user's language, without any special syntax or format. For example, natural language query <NUM> can be a statement, a question, or a simple list of keywords.

Note that natural language query <NUM> can indicate a category based on implicitly or explicitly referencing the category. In other words, it is unnecessary for the natural language query <NUM> to be composed as "Sales revenue for the ZIP code <NUM>". As will be described in greater detail in <FIG>, natural language query <NUM> can be constructed in any of a number of different ways. For example, a drop-down menu corresponding to ZIP codes can list "<NUM>" among the selectable choices. As another example, "<NUM>" can be automatically recognized as a ZIP code based on natural language processing. Thus, a computer can automatically associate a specified field value with an unspecified category.

<FIG> depicts example dataset filters. Referring to <FIG>, dataset filter <NUM> includes first granularity level <NUM>, which further includes category 202N and field value 104N. Translated dataset filter <NUM> includes second granularity level <NUM>, which further includes category 202A, field value 104E, and field value 104F.

A dataset filter (e.g., dataset filter <NUM>, translated dataset filter <NUM>) can be a function, method, procedure, or any other set of instructions that can cause discarding data that is unnecessary for query processing and/or execution. A dataset filter can be composed in a data manipulation language (DML) to cause discarding unnecessary data from a particular dataset. For example, the Spark SQL "filter" verb can be used to construct a dataset filter. Dataset filters can be constructed from a query parameter or any other query portion that implicitly or explicitly indicates a category and one or more corresponding field values. In some example embodiments, dataset filters can be used to perform dataset translation, which will be described in greater detail hereinafter.

For example, the natural language query <NUM> "Sales revenue by ZIP code in California" specifies a measure <NUM>, a category for the query result, and a first granularity level <NUM>. The measure <NUM> is "Sales revenue". The category for the query result is "ZIP code". The first granularity level <NUM> is "California". The first granularity level implicitly indicates a category of "State" and explicitly indicates a field value of "California". Thus, a dataset filter can be composed to cause discarding, from a dataset, any rows that have a value other than "California" in the "State" column.

In some example embodiments, further efficiency can be realized based on minimizing the number of datasets to which dataset filters will be applied. Minimizing the number of datasets can be achieved based on selecting datasets that are among the most relevant for query processing and/or execution. Relevancy can be determined based on any of a number of predetermined criteria, such as the following:.

In some example embodiments, dataset selection can proceed according to a selection algorithm. For example, the selection algorithm causes generating a set of relevant categories based on causing identifying the category for the query result and all subcategories of that category. Based on the set of relevant categories, the selection algorithm causes identifying datasets that include at least one of the subcategories in the set of relevant categories. Furthermore, the selection algorithm causes determining the number of dataset filters that can be applied to each dataset of the identified datasets. The selection algorithm can cause generating a ranking of the identified datasets according to the number of dataset filters that can be applied. Based on the ranking, the selection algorithm can cause selecting one or more datasets to which the largest number of dataset filters can be applied.

In some example embodiments, the selection algorithm can further reduce the number of selected datasets. For example, the selection algorithm can cause generating one or more distance rankings. The one or more distance rankings can be determined based on the distance between the category for the query result and a category in a particular dataset. Additionally or alternatively, the one or more distance rankings can be determined based on the distance between the category for the query result and a category indicated in a dataset filter.

In some example embodiments, the number of selected datasets can be further reduced based on any of a number of arbitrary rankings. For example, a lexicographic ranking can be generated based on the name of a dataset. Thus, the dataset with the name "Sales" can be selected over the dataset with the name "State".

In some example embodiments, data in the selected datasets can be filtered based on necessity for query processing and/or query execution. For example, a filtration algorithm can cause determining which fields in a particular dataset are unnecessary fields and discarding the unnecessary fields. Necessity can be determined based on any of a number of predetermined filtration criteria, including whether a category that corresponds to a particular field is hierarchically related to a category for the query result, a category for a measure <NUM>, and/or a category indicated in a dataset filter. Additionally or alternatively, the filtration algorithm can cause applying one or more dataset filters to the selected datasets.

For example, natural language query <NUM> can be "Sales revenue by ZIP code in California". Unnecessary columns can be discarded based on predetermined filtration criteria. Referring to the example of Dataset A above, the "Date" column can be discarded, because it is not hierarchically related to a category for the query result (e.g., ZIP code), a category for a measure <NUM> (e.g., Sales revenue), and/or a category indicated in a dataset filter (e.g., State). Additionally or alternatively, unnecessary rows can be discarded based on applying one or more dataset filters to the selected datasets. Referring to the example of Dataset N above, the row that includes "Washington" can be discarded, because it does not include the field value "California". Thus, a filtered version of Dataset N can be represented as the following table:.

Applying a dataset filter to a dataset can be simple if, as in the previous example, a category indicated in the dataset filter matches a category that corresponds to a field in the dataset. However, if the category indicated in the dataset filter fails to match any categories associated with the dataset, the dataset filter can be translated (e.g., demoted, promoted) to match a category that corresponds to a field in the dataset.

<FIG> depicts translated dataset filter <NUM> is generated to apply dataset filter <NUM> to dataset 100A. Dataset filter <NUM> includes category 202N, which is absent from dataset 100A. However, dataset 100A includes category 202A, which is hierarchically related to category 202N according to hierarchy mapping <NUM>. Based on hierarchy mapping <NUM>, first granularity level <NUM> can be translated to second granularity level <NUM>. Referring to the example of Dataset A above, dataset filter <NUM> and translated dataset filter <NUM> can be represented as {State → ["California"]} and {ZIP Code → ["<NUM>", "<NUM>"]}, respectively.

<FIG> depicts an example filtered dataset. Referring to <FIG>, translated dataset filter <NUM> is applied to dataset 100A to generate filtered dataset <NUM>. Filtered dataset <NUM> includes fields 102B-C. Field 102B includes field values 104E-F, and field 102C includes field values <NUM>-I. In terms of the example of Dataset A above, the abstraction of filtered dataset <NUM> can be represented as the following table:.

In some example embodiments, a particular field included in filtered dataset <NUM> can subsequently be promoted to a category that matches the category for the query result. For example, if natural language query <NUM> had been "Sales Revenue in California", the example of Filtered Dataset A above can be promoted to include a "State" column to replace the "ZIP Code" column.

<FIG> depicts an example composite dataset generated based on a filtered dataset. Referring to <FIG>, composite dataset <NUM> includes fields 102B, 102C, and 102N.

Field 102B includes field values 104E-F, field 102C includes field values <NUM>-I, and field 102N includes two instances of field value 104N.

Generating a composite dataset <NUM> is unnecessary when all necessary data is included in a single dataset. However, when the necessary data is spread across multiple datasets, a composite dataset <NUM> can be generated based on performing a join operation. In some example embodiments, each dataset of the multiple datasets can be keyed based on the field that corresponds to the category for the query result. Additionally or alternatively, prior to performing the join operation, at least one dataset of the multiple datasets can include a field that is demoted to a level of a common subcategory that is shared by the multiple datasets.

Referring to the example of <FIG>, composite dataset <NUM> is generated based on joining filtered dataset <NUM> to a filtered version of dataset 100N. In terms of the examples of Filtered Dataset A and Filtered Dataset N above, the abstraction of composite dataset <NUM> can be represented as the following table:.

In some example embodiments, the "State" column can be considered as unnecessary data, and generated composite dataset <NUM> can be avoided.

<FIG> depicts an example query result. Referring to <FIG>, result <NUM> includes measure <NUM> and field values <NUM>-I. Result <NUM> is an abstraction that represents data (e.g., field values <NUM>-I) that is retrieved from a dataset and that represents a computation (e.g., measure <NUM>) that is performed on the data. In some example embodiments, a computation is unnecessary, so the data retrieved from a dataset can be a query result.

For example, natural language query <NUM> can be "Sales Revenue for California". In terms of the example of Composite Dataset A above, data from the "Sales Revenue" column can be retrieved and summed to generate a query result. Thus, "$<NUM>" and "$<NUM>" can be added together to generate the query result "$<NUM>".

<FIG> depicts an example user interface that constructs a query. Referring to <FIG>, user interface <NUM> receives input <NUM> and text input <NUM>. User interface <NUM> includes predetermined query parameters <NUM>, histogram <NUM>, and line graph <NUM>.

Natural language query <NUM> can be constructed based on one or more inputs received at a user interface <NUM>. The one or more inputs can be a hovering, a typing, and/or a clicking action at a selectable element of the user interface <NUM>. For example, input <NUM> can be a hovering action over a map to select a particular region of the map. Text input <NUM> can be a string that is typed into a search field of the user interface <NUM>. In some example embodiments, text input <NUM> can be autocompleted. Auto-completion can be based, at least in part, on the categories and measures included in hierarchy mapping <NUM>. Predetermined query parameters <NUM> can include options in a vertically arranged menu, a horizontally arranged menu, etc..

In some example embodiments, user interface <NUM> can provide one or more visualizations of a query result. The visualizations can include a histogram <NUM>, a line graph <NUM>, a summary table, etc..

Further efficiency can be realized based on pre-computing one or more intermediate datasets that are stored in volatile and/or non-volatile memory. For example, the one or more intermediate datasets can include data that is frequently accessed, and the one or more intermediate datasets can be stored in a memory cache.

In some example embodiments, an intermediate dataset can include a frequently accessed dataset that is promoted to a level of a frequently indicated category. Hereinafter, this intermediate dataset is referenced as "promoted dataset". <FIG> depicts an example promoted dataset. Referring to <FIG>, translated dataset <NUM> is generated based on promoting field 102B of dataset 100A to field 102N. Field 102N includes field values <NUM>-N, which exhibit a translated category level <NUM>. In other words, field 102B's category 202A has been promoted to field 102N's category 202N.

In terms of the example of Dataset A above, translated dataset <NUM> can be represented as the following table:.

Thus, if subsequently received queries include request for data in Dataset A at the State level, the data can simply be retrieved from Promoted Dataset A.

In some example embodiments, an intermediate dataset can include all available data that is related to a frequently indicated category. <FIG> depicts an example composite dataset generated based on all available data related to a particular category. Referring to <FIG>, available related data <NUM> includes dataset 100A and dataset 100N. Composite dataset <NUM> is generated based on available related data <NUM>.

In terms of the examples of Dataset A and Dataset N above, composite dataset <NUM> can be represented as the following table:.

Composite Dataset B can be a "view" table that represents the universe of available data related to ZIP codes.

<FIG> is a flow diagram that depicts an approach for automatically generating composite datasets, according to some example embodiments of the present Application. At block <NUM>, datasets are annotated with metadata that include categories corresponding to fields. Each dataset can include multiple fields, and each category can correspond to one or more fields. Additionally or alternatively, the metadata can include one or more measures. Each measure of the one or more measures is dependent on at least one field.

At block <NUM>, a hierarchy mapping is generated. The hierarchy mapping indicates a hierarchical relationship between different categories. Block <NUM> can be performed concurrently with or subsequent to block <NUM>.

At block <NUM>, a natural language query is received. The natural language query specifies a first granularity level. The first granularity level implicitly or explicitly indicates a particular category and one or more field values that correspond to the particular category. Additionally, the natural language query can specify a particular measure and/or a category that corresponds to a result for the natural language query.

The natural language query can be constructed based on one or more inputs received at a user interface. The user interface can include predetermined query parameters that are selectable. Additionally or alternatively, the natural language query can be constructed based on text input that is autocompleted. Additionally or alternatively, the natural language query can be constructed based on a mouse pointer that hovers over a map. For example, this hovering action can cause data to be retrieved from a particular composite dataset that is generated based on all available data related to the particular category.

At block <NUM>, one or more hierarchically related categories of the particular category are identified. This identification can be based on the hierarchy mapping.

At block <NUM>, two or more datasets that include at least one hierarchically related category are selected. This selection can include ranking at least some of the datasets based on the hierarchical relationship between the particular category and the at least one hierarchically related category. Additionally or alternatively, this selection can include ranking at least some of the datasets based on a number of the one or more hierarchically related categories that are included in a particular dataset.

At block <NUM>, one or more dataset filters are generated based on the first granularity level. Block <NUM> can be performed prior to, concurrently with, or subsequent to block <NUM>.

At block <NUM>, the one or more dataset filters are translated to a second granularity level that corresponds to the at least one hierarchically related category. For example, the one or more dataset filters can be demoted to a subcategory of the particular category.

At block <NUM>, the one or more translated dataset filters are applied to at least one dataset of the two or more datasets. Optionally, the at least one dataset can be subsequently translated (e.g., promoted) to a level that corresponds to the particular category.

At block <NUM>, a composite dataset that includes the particular category can be generated. The composite dataset can be generated based on joining the two or more datasets. Block <NUM> can be performed prior to, concurrently with, or subsequent to block <NUM>.

At block <NUM>, data is retrieve from the composite dataset to return a result for the natural language query. The result can be returned based on displaying the result as a line graph, summary table, or a histogram.

At any point in the example process of <FIG>, if the at least one dataset is determined to be a frequently accessed dataset and if the particular category is determined to be a frequently indicated category, the at least one dataset can be translated (e.g., promoted) to a level of the particular category. After being translated, the at least one dataset can be stored for use with subsequently received queries.

Referring now to <FIG>, it is a block diagram that illustrates a basic computing device <NUM> in which the example embodiment(s) of the present Application may be embodied. Computing device <NUM> and its components, including their connections, relationships, and functions, is meant to be exemplary only, and not meant to limit implementations of the example embodiment(s). Other computing devices suitable for implementing the example embodiment(s) may have different components, including components with different connections, relationships, and functions.

Computing device <NUM> may include a bus <NUM> or other communication mechanism for addressing main memory <NUM> and for transferring data between and among the various components of device <NUM>.

Computing device <NUM> may also include one or more hardware processors <NUM> coupled with bus <NUM> for processing information. A hardware processor <NUM> may be a general purpose microprocessor, a system on a chip (SoC), or other processor.

Main memory <NUM>, such as a random access memory (RAM) or other dynamic storage device, also may be coupled to bus <NUM> for storing information and software instructions to be executed by processor(s) <NUM>. Main memory <NUM> also may be used for storing temporary variables or other intermediate information during execution of software instructions to be executed by processor(s) <NUM>.

Software instructions, when stored in storage media accessible to processor(s) <NUM>, render computing device <NUM> into a special-purpose computing device that is customized to perform the operations specified in the software instructions. The terms "software", "software instructions", "computer program", "computer-executable instructions", and "processor-executable instructions" are to be broadly construed to cover any machine-readable information, whether or not human-readable, for instructing a computing device to perform specific operations, and including, but not limited to, application software, desktop applications, scripts, binaries, operating systems, device drivers, boot loaders, shells, utilities, system software, JAVASCRIPT, web pages, web applications, plugins, embedded software, microcode, compilers, debuggers, interpreters, virtual machines, linkers, and text editors.

Computing device <NUM> also may include read only memory (ROM) <NUM> or other static storage device coupled to bus <NUM> for storing static information and software instructions for processor(s) <NUM>.

One or more mass storage devices <NUM> may be coupled to bus <NUM> for persistently storing information and software instructions on fixed or removable media, such as magnetic, optical, solid-state, magnetic-optical, flash memory, or any other available mass storage technology. The mass storage may be shared on a network, or it may be dedicated mass storage. Typically, at least one of the mass storage devices <NUM> (e.g., the main hard disk for the device) stores a body of program and data for directing operation of the computing device, including an operating system, user application programs, driver and other support files, as well as other data files of all sorts.

Computing device <NUM> may be coupled via bus <NUM> to display <NUM>, such as a liquid crystal display (LCD) or other electronic visual display, for displaying information to a computer user. In some configurations, a touch sensitive surface incorporating touch detection technology (e.g., resistive, capacitive, etc.) may be overlaid on display <NUM> to form a touch sensitive display for communicating touch gesture (e.g., finger or stylus) input to processor(s) <NUM>.

An input device <NUM>, including alphanumeric and other keys, may be coupled to bus <NUM> for communicating information and command selections to processor <NUM>. In addition to or instead of alphanumeric and other keys, input device <NUM> may include one or more physical buttons or switches such as, for example, a power (on/off) button, a "home" button, volume control buttons, or the like.

Another type of user input device may be a cursor control <NUM>, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor <NUM> and for controlling cursor movement on display <NUM>.

While in some configurations, such as the configuration depicted in <FIG>, one or more of display <NUM>, input device <NUM>, and cursor control <NUM> are external components (i.e., peripheral devices) of computing device <NUM>, some or all of display <NUM>, input device <NUM>, and cursor control <NUM> are integrated as part of the form factor of computing device <NUM> in other configurations.

Functions of the disclosed systems, methods, and modules may be performed by computing device <NUM> in response to processor(s) <NUM> executing one or more programs of software instructions contained in main memory <NUM>. Such software instructions may be read into main memory <NUM> from another storage medium, such as storage device(s) <NUM>. Execution of the software instructions contained in main memory <NUM> cause processor(s) <NUM> to perform the functions of the example embodiment(s).

While functions and operations of the example embodiment(s) may be implemented entirely with software instructions, hard-wired or programmable circuitry of computing device <NUM> (e.g., an ASIC, a FPGA, or the like) may be used in other embodiments in place of or in combination with software instructions to perform the functions, according to the requirements of the particular implementation at hand.

The term "storage media" as used herein refers to any non-transitory media that store data and/or software instructions that cause a computing device to operate in a specific fashion. Non-volatile media includes, for example, non-volatile random access memory (NVRAM), flash memory, optical disks, magnetic disks, or solid-state drives, such as storage device <NUM>. Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid-state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, flash memory, any other memory chip or cartridge.

Various forms of media may be involved in carrying one or more sequences of one or more software instructions to processor(s) <NUM> for execution. For example, the software instructions may initially be carried on a magnetic disk or solid-state drive of a remote computer. The remote computer can load the software instructions into its dynamic memory and send the software instructions over a telephone line using a modem. A modem local to computing device <NUM> can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. Bus <NUM> carries the data to main memory <NUM>, from which processor(s) <NUM> retrieves and executes the software instructions.

The software instructions received by main memory <NUM> may optionally be stored on storage device(s) <NUM> either before or after execution by processor(s) <NUM>.

Computing device <NUM> also may include one or more communication interface(s) <NUM> coupled to bus <NUM>. A communication interface <NUM> provides a two-way data communication coupling to a wired or wireless network link <NUM> that is connected to a local network <NUM> (e.g., Ethernet network, Wireless Local Area Network, cellular phone network, Bluetooth wireless network, or the like). Communication interface <NUM> sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information. For example, communication interface <NUM> may be a wired network interface card, a wireless network interface card with an integrated radio antenna, or a modem (e.g., ISDN, DSL, or cable modem).

Network link(s) <NUM> typically provide data communication through one or more networks to other data devices. For example, a network link <NUM> may provide a connection through a local network <NUM> to a host computer <NUM> or to data equipment operated by an Internet Service Provider (ISP) <NUM>. Local network(s) <NUM> and Internet <NUM> use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link(s) <NUM> and through communication interface(s) <NUM>, which carry the digital data to and from computing device <NUM>, are example forms of transmission media.

Computing device <NUM> can send messages and receive data, including program code, through the network(s), network link(s) <NUM> and communication interface(s) <NUM>. In the Internet example, a server <NUM> might transmit a requested code for an application program through Internet <NUM>, ISP <NUM>, local network(s) <NUM> and communication interface(s) <NUM>.

<FIG> is a block diagram of a basic software system <NUM> that may be employed for controlling the operation of computing device <NUM>. Software system <NUM> and its components, including their connections, relationships, and functions, is meant to be exemplary only, and not meant to limit implementations of the example embodiment(s). Other software systems suitable for implementing the example embodiment(s) may have different components, including components with different connections, relationships, and functions.

Software system <NUM> is provided for directing the operation of computing device <NUM>. Software system <NUM>, which may be stored in system memory (RAM) <NUM> and on fixed storage (e.g., hard disk or flash memory) <NUM>, includes a kernel or operating system (OS) <NUM>.

The OS <NUM> manages low-level aspects of computer operation, including managing execution of processes, memory allocation, file input and output (I/O), and device I/O. One or more application programs, represented as 1402A, 1402B, 1402C. 1402N, may be "loaded" (e.g., transferred from fixed storage <NUM> into memory <NUM>) for execution by the system <NUM>. The applications or other software intended for use on device <NUM> may also be stored as a set of downloadable computer-executable instructions, for example, for downloading and installation from an Internet location (e.g., a Web server, an app store, or other online service).

Software system <NUM> includes a graphical user interface (GUI) <NUM>, for receiving user commands and data in a graphical (e.g., "point-and-click" or "touch gesture") fashion. These inputs, in turn, may be acted upon by the system <NUM> in accordance with instructions from operating system <NUM> and/or application(s) <NUM>. The GUI <NUM> also serves to display the results of operation from the OS <NUM> and application(s) <NUM>, whereupon the user may supply additional inputs or terminate the session (e.g., log off).

OS <NUM> can execute directly on the bare hardware <NUM> (e.g., processor(s) <NUM>) of device <NUM>. Alternatively, a hypervisor or virtual machine monitor (VMM) <NUM> may be interposed between the bare hardware <NUM> and the OS <NUM>. In this configuration, VMM <NUM> acts as a software "cushion" or virtualization layer between the OS <NUM> and the bare hardware <NUM> of the device <NUM>.

VMM <NUM> instantiates and runs one or more virtual machine instances ("guest machines"). Each guest machine comprises a "guest" operating system, such as OS <NUM>, and one or more applications, such as application(s) <NUM>, designed to execute on the guest operating system. The VMM <NUM> presents the guest operating systems with a virtual operating platform and manages the execution of the guest operating systems.

In some instances, the VMM <NUM> may allow a guest operating system to run as if it is running on the bare hardware <NUM> of device <NUM> directly. In these instances, the same version of the guest operating system configured to execute on the bare hardware <NUM> directly may also execute on VMM <NUM> without modification or reconfiguration. In other words, VMM <NUM> may provide full hardware and CPU virtualization to a guest operating system in some instances.

In other instances, a guest operating system may be specially designed or configured to execute on VMM <NUM> for efficiency. In these instances, the guest operating system is "aware" that it executes on a virtual machine monitor. In other words, VMM <NUM> may provide para-virtualization to a guest operating system in some instances.

The above-described basic computer hardware and software is presented for purpose of illustrating the basic underlying computer components that may be employed for implementing the example embodiment(s). The example embodiment(s), however, are not necessarily limited to any particular computing environment or computing device configuration. Instead, the example embodiment(s) may be implemented in any type of system architecture or processing environment that one skilled in the art, in light of this disclosure, would understand as capable of supporting the features and functions of the example embodiment(s) presented herein.

Claim 1:
A computer-implemented method performed by one or more computer systems, the method comprising:
annotating (<NUM>) a plurality of datasets with metadata, wherein the plurality of datasets includes a plurality of fields, wherein the metadata includes a plurality of categories in which each category corresponds to one or more fields of the plurality of fields;
generating (<NUM>) a hierarchy mapping that indicates a hierarchical relationship between different categories of the plurality of categories;
receiving (<NUM>) a natural language query that specifies a first granularity level indicating a particular category of the plurality of categories and one or more field values that correspond to the particular category;
identifying (<NUM>), based on the hierarchy mapping, one or more categories of the plurality of categories that are hierarchically related to the particular category, wherein at least one of the one or more categories correspond to at least a second granularity level hierarchically related to the first granularity level;
selecting (<NUM>), based on the metadata, two or more datasets of the plurality of datasets that include at least one category of the one or more categories;
translating (<NUM>) at least one dataset of the two or more datasets to the first granularity level;
joining (<NUM>) the two or more datasets to generate a composite dataset that includes the particular category; and
executing (<NUM>) the natural language query against the composite dataset to return a result for the natural language query.