System and method for facilitating data visualization via a map-reduce framework

In one embodiment, a method includes receiving a request for data visualization. The request specifies input data, at least one user query, and a data-visualization type. The method further includes inferring at least one additional query based on a user-interface (UI) range of freedom associated with the request. In addition, the method includes causing a map-reduce framework to process the input data according to the at least one user query and the at least one additional query. The causing yields resultant data. Further, the method includes storing in a cache a portion of the resultant data that relates to the at least one additional query. The method also includes generating the requested data visualization based on a portion of the resultant data that relates to the at least one user query.

BACKGROUND

1. Technical Field

The present invention relates generally to data visualization and more particularly, but not by way of limitation, to systems and methods for data visualization.

2. History Of Related Art

Distributed computing systems that use processing units to work on parts of large-scale computing problems are increasingly common. The amounts of data that are produced by them are also growing in scale, thereby making standard data and information visualization capabilities work inefficiently or not at all when confronted with the scale of the data results.

Approaches for visualizing geospatial relationships and trees have been developed based on the idea of so-called semantic zooming, where a type and form of a visual representation changes depending on an amount of information and/or a granularity of a viewing window. For semantic zooming to work effectively, a user interface must retrieve appropriate data from a remote server incrementally depending on a scale or resolution of the visual representation. This is dependent upon the data being available at multiple scales of resolution, which is not always the case.

SUMMARY OF THE INVENTION

In one embodiment, a method includes, on at least one server computer, receiving a request for data visualization. The request specifies input data, at least one user query, and a data-visualization type. The method further includes inferring, by the at least one server computer, at least one additional query based on a user-interface (UI) range of freedom associated with the request. In addition, the method includes the at least one server computer causing a map-reduce framework to process the input data according to the at least one user query and the at least one additional query. The causing yields resultant data. The method also includes the at least one server computer storing in a cache a portion of the resultant data that relates to the at least one additional query. Additionally, the method includes the at least one server computer generating the requested data visualization based on a portion of the resultant data that relates to the at least one user query.

In one embodiment, an information handling system is operable to perform a method. The method includes receiving a request for data visualization. The request specifies input data, at least one user query, and a data-visualization type. The method further includes inferring at least one additional query based on a user-interface (UI) range of freedom associated with the request. In addition, the method includes causing a map-reduce framework to process the input data according to the at least one user query and the at least one additional query. The causing yields resultant data. Further, the method includes storing in a cache a portion of the resultant data that relates to the at least one additional query. The method also includes generating the requested data visualization based on a portion of the resultant data that relates to the at least one user query.

In one embodiment, a computer-program product includes a computer-usable medium having computer-readable program code embodied therein. The computer-readable program code is adapted to be executed to implement a method. The method includes receiving a request for data visualization. The request specifies input data, at least one user query, and a data-visualization type. The method further includes inferring at least one additional query based on a user-interface (UI) range of freedom associated with the request. In addition, the method includes causing a map-reduce framework to process the input data according to the at least one user query and the at least one additional query. The causing yields resultant data. Further, the method includes storing in a cache a portion of the resultant data that relates to the at least one additional query. The method also includes generating the requested data visualization based on a portion of the resultant data that relates to the at least one user query.

DETAILED DESCRIPTION OF ILLUSTRATIVE

Embodiments of the Invention

In various embodiments, computer systems work together to process large collections of data and produce aggregate measures or profiles of the data. In a typical embodiment, a map-reduce framework sends parts of the data to different processing units for processing that can include, for example, counting, outlier detection, collation, extraction of additional metadata from the data, and the like.

For purposes of this disclosure, the term map-reduce framework refers to a system that is configured to perform: (i) at least one map operation that generates one or more intermediate key/value pairs from input data; and (ii) at least one reduce operation that generates one or more output values from the intermediate key/value pairs. The map-reduce framework typically divides the map operation among multiple program components (sometimes called “workers”) and assigns one or more map tasks to each worker. A map-reduce framework divides the reduce operation into reduce tasks and assigns them to workers. In some implementations, the map-reduce framework is run on a cluster of processing devices such as a network of information handling systems. For example, a data center can be established that has racks with tens of thousands of processing modules (e.g., a processing module can be an information handling system, or part thereof), and the map-reduce procedure can be performed as a massive data job distributed among the thousands (or more) processing units. A map-reduce job can involve mapping and reducing several terabytes of data or more, to name just one example.

FIG. 1illustrates an example of a system100for performing data visualization. The system100includes a map-reduce framework102, a client device104, and a server computer122. The client device104may be an information handling system such as, for example, a server, desktop computer, laptop computer, tablet computer, mobile device, and the like. The server computer122is operable to communicate with the client device104and the map-reduce framework102over a network106. The network106can include, for example, the Internet, a public or private intranet, a network for mobile devices (e.g., a cellular phone network), and/or other communications networks. The client device104has a client application108such as, for example, a web browser, executing thereon. The server computer122has visualization software124resident and executing thereon. As described in greater detail below, a user can employ the client application108to access the visualization software124and request a data visualization, for example, of large volumes of data.

In a typical embodiment, the client application108is configured to allow the user to specify the request for data visualization. The request for data visualization typically specifies input data110, at least one user query on the input data110, and a data-visualization type. The input data110is representative of a data source to which the request for visualization is applicable. The input data110can include any suitable information that can be processed in the map-reduce framework102. For example, and without limitation, the input data110can correspond to network data, sales data, observational data, scientific data, random data, population data, artistic data, statistical data, and combinations thereof. The input data110can have any data format that is suitable for the map-reduce operations, including, but not limited to, a binary data format, a plain text format, a markup-language format (e.g., XML), or an image format. In a typical embodiment, the input data110is accessible to the server computer122and/or the map-reduce framework102over the network106.

The at least one user query is typically representative of data operations, data relationships, and the like that underlie the requested visualization. In some embodiments, the client application108is configured so that the user can formulate the at least one user query (e.g., using Structured Query Language (SQL) or another query language) to be performed on the input data110. In other embodiments, the at least one user query may be graphically selectable in the client application108. The visualization type of the request generally specifies how returned data should be graphically presented. The visualization type can include, for example, scatterplots, pie charts, tables, bar charts, geospatial representations, heat map, chord charts, interactive graphs, bubble charts, candlestick charts, stoplight charts, spring graphs, and/or other types of charts, graphs, or manners of displaying data. The visualization type is typically presented on the client application108via a visualization interface provided by the visualization software124.

The visualization software124is operable to receive the request for data visualization and, in response thereto, infer at least one additional query on the input data110that the user is deemed likely to want in the near future. Thereafter, the visualization software124causes the map-reduce framework102to process the input data110according to the at least one user query and the at least one additional query. In particular, as described with respect toFIG. 2, the visualization software prepares one or more map-reduce jobs that are processed by the map-reduce framework. The processing yields resultant data (i.e., output values). The visualization software124generates the requested data visualization based on a portion of the resultant data that relates to the at least one user query. Subsequently, the visualization software can serve the requested visualization to the client application108on the visualization interface.

In typical embodiment, the visualization software124stores in a cache a portion of the resultant data that relates to the at least one additional query. The cache may be located, for example, on the map-reduce framework102, on the server computer122, and/or on the client device104. In a typical embodiment, the cache allows the visualization software124to efficiently handle user modifications to the data visualization. Specifically, when the user modifies the request via the visualization interface, the visualization software124may be able serve a modified data visualization by accessing the cache and without having to wait for further processing by the map-reduce framework102.

Operation of the map-reduce framework102will now be described in more detail. The map-reduce framework102includes a map operation112and a reduce operation114. In some implementations, the map operation112is configured for processing some or all of the input data110as instructed by the visualization software124and generating at least one intermediate key/value pair therefrom. In some implementations, the reduce operation114is configured for processing at least part of the intermediate key/value pair(s) and generating at least one output value therefrom. In general, the map operation112can detect the occurrences, in the input data110, of each of multiple keys, and the reduce operation114can sum or otherwise aggregate data from such detections into useful output information (e.g., a table of occurrence frequencies).

The map operation112and/or the reduce operation114can be performed as tasks distributed to one or more processors. In some implementations, the map-reduce framework102includes, or otherwise works in connection with, a cluster116of processing units, such as machines118. For example, each machine118can be an information handling system (e.g., a PC) and they can be networked using any suitable communication protocol (e.g., Ethernet). Each of the machines118has at least one memory120, either integrated in the device or communicatively coupled thereto by an appropriate connection (e.g., a system bus). For example, the memory120is used in detecting occurrences of keys in the input data110, and/or in summing data for generating the output information.

In this fashion, an organization can use the client application108to visualize volumes of business data. For example, a telecommunications company (such as a wireless service provider) can have hundreds of millions of customers who engage in many transactions with the company (for example, each use of the wireless service can be registered as a transaction). In such situations, the cumulative business data for some period of time (e.g., a week, month or year) can reach a billion records or more. Assuming the request for data visualization requests visualization of such data, the map-reduce procedure can first be used in analyzing the business data, for example to count the number of transactions originating in a specific country, or the number of unsuccessful transactions. In such a situation, the client application108identifies the volume of business data to be used (i.e., the input data110), and defines a criterion or characteristic that is being sought (i.e., as part of the at least one user query). After processing by the map-reduce framework102, the visualization software124is operable to generate and provide to the client application108a visualization interface that includes the requested data visualization.

FIG. 2illustrates an example200of how visualization software224can interact with a map-reduce framework202. In a typical embodiment, the visualization software224and the map-reduce framework202operate as described with respect to the visualization software124and the map-reduce framework202, respectively, ofFIG. 2. The visualization software224is operable to select, configure, and sequence map-reduce filter226(1), map-reduce filter226(2), and map-reduce filter226(3) (collectively, map-reduce filters226) based on input data such as, for example, the input data110, at least one user query, and at least one addition query that is inferred as described with respect toFIG. 1.

The map-reduce filters226can be selected, configured, and sequenced based on, for example, types of data in the input data110and based on individual data operations required by the at least one user query and the at least one additional query. Each of the one or more map-reduce filters226can encompass a single map-reduce job or a composition of map-reduce jobs. In various embodiments, the one or more map-reduce filters226may be selected from a library of map-reduce filters. In this manner, the visualization software is operable to prepare one or more map-reduce jobs represented by the map-reduce filters226. Subsequently, the visualization software124causes the map-reduce filters226to be applied on the map-reduce framework202.

FIG. 3illustrates an example300of a visualization interface328. The visualization interface328includes a data visualization330and a plurality of user-interface (UI) controls332. The data visualization330may be requested and served as described, for example, with respect toFIG. 1. The plurality of UI controls332are elements of the visualization interface328that are designed to be selectable, modifiable, or otherwise interacted with by a user. The plurality of UI controls332can include, for example, buttons, text boxes, sliders, list boxes, menus, menu bars, navigation controls (e.g., links, tabs, and scroll bars), containers (e.g. windows), zooming controls, and other similar GUI elements.

In general, the plurality of UI controls332allow the user to modify an original request for the data visualization330. Modifications may involve supplementing the data visualization330with new or different data operations, new or different data relationships, and/or the like. For example, the plurality of UI controls332may enable the user to change a level of granularity of the data visualization330(i.e., semantic zooming). An example of semantic zooming will be described with respect toFIG. 5.

The UI range of freedom334encompasses all modifications to the data visualization330collectively enabled by the plurality of UI controls332. As described above with respect toFIG. 1, the UI range of freedom334can serve as the basis for inferring at least one additional query that has not been specified by the user. The at least one additional query can be inferred by identifying each additional data operation and/or data relationship that is included within the UI range of freedom334. In that way, the at least one additional query is designed to specify additional data that has not yet been requested by the user but may soon be requested via a modification made via the plurality of UI controls332. In some embodiments, the at least one additional query may be scoped, for example, to include only a subset of the UI range of freedom. For example, in various embodiments, the at least one additional query may be scoped to a smaller set of data operations that are predicted to be more likely based on, for example, a machine-learning model. In addition, depending on available memory and capacity of the map-reduce framework, the at least one additional query may also be scoped according to pre-defined rules and policies.

The visualization interface328and, in particular, the data visualization330, are typically responsive to either a user request or a user modification to an existing request. Therefore, for ease of description, the UI range of freedom334may be considered to be associated with either a request or a modification to a request that resulted in the data visualization330. In various embodiments, the visualization interface328may be a pre-packaged interface such that the UI range of freedom334is pre-defined. In various other embodiments, the visualization interface328may be customized via, for example, UI widgets that provide at least a portion of the plurality of UI controls332. In these embodiments, the visualization software typically analyzes the plurality of UI controls in order to dynamically ascertain the UI range of freedom334.

FIG. 4illustrates a process400that may be implemented via visualization software loaded and executing on a server computer. The server computer may be similar to the server computer122ofFIG. 1. The visualization software may be similar to the visualization software124ofFIG. 1and the visualization software224ofFIG. 2. The process400begins at step402.

At step402, the visualization software receives a request for data visualization from a user. The request for data visualization typically specifies input data such as, for example, the input data110ofFIG. 1, at least one user query on the input data, and a data-visualization type. From step402, the process400proceeds to step404. At step404, the visualization software infers at least one additional query based on a UI range of freedom associated with the request. In a typical embodiment, the UI range of freedom corresponds to all modifications to the request that will be enabled by a visualization interface provided in response to the request. The at least one additional query may be inferred as described with respect toFIG. 3. From step404, the process400proceeds to step405.

At step405, the visualization software prepares one or more map-reduce jobs based on the input data, the at least one user query, and the least one additional query. In various embodiments, the one or more map-reduce jobs may be prepared as described with respect toFIG. 2. From step405, the process400proceeds to step406. At step406, the visualization software causes a map-reduce framework such as, for example, the map-reduce framework102ofFIG. 1or the map-reduce framework202ofFIG. 2, to process the one or more map-reduce jobs as described with respect toFIGS. 1-2. From step406, the process400proceeds to step408. At step408, the visualization software receives resultant data from the map-reduce framework. The resultant data generally includes data responsive to the at least one user query and the at least one additional query. From step408, the process400proceeds to step410.

At step410, the visualization software stores in a cache a portion of the resultant data that is responsive to the at least one additional query. As described with respect toFIG. 1, the cache may be located on the server computer, on the map-reduce framework, or on a client device operated by the user. From step410, the process400proceeds to step412. At step412, the visualization software generates the data visualization based on a portion of the resultant data that is responsive to the at least one user query. From step412, the process400proceeds to step414.

At step414, the visualization software serves the data visualization on a visualization interface that affords the UI range of freedom. For example, the visualization interface may be served to a client device such as, for example, the client device104ofFIG. 1, so that the visualization interface can be presented on a client application such as, for example, the client application108ofFIG. 1. From step414, the process400proceeds to step416. At step416, the visualization software allows the user to navigate the visualization interface via a plurality of UI controls provided thereon. From step416, the process400proceeds to step418.

At step418, the visualization software determines whether a modification to the request has been made by the user via the visualization interface. If not, the process400returns to step416and proceeds as described above. If it is determined at step418that a modification to the request has been made, the process400proceeds to step420. The modification typically specifies at least one new user query. At step420, the visualization software determines whether the modification can be fulfilled by the cache. If it is determined at step420that the modification can be fulfilled by the cache, the process400proceeds to step422.

At step422, the visualization software retrieves data from the cache that fulfills the at least one new user query. From step422, the process400proceeds to step424. At step424, the visualization software generates an updated data visualization based on the retrieved data. From step424, the process400proceeds to step426. At step426, the visualization software serves the updated data visualization on the visualization interface to the client device104. From step426, the process400returns to step416and, in parallel, performs steps428-436as described below. At step428, the visualization software infers at least one new additional query based on a UI range of freedom associated with the modification. In a typical embodiment, the at least one new additional query may be inferred as described with respect to step404. From step428, the process400proceeds to step430.

At step430, the visualization software prepares one or more new map-reduce jobs based on the at least one new additional query. From step430, the process400proceeds to step432. At step432, the visualization software causes the map-reduce framework to process the one or more new map-reduce jobs. From step432, the process400proceeds to step434. At step434, the visualization software receives new resultant data from the map-reduce framework. From step434, the process400proceeds to step436. At step436, the visualization software stores in the cache the new resultant data, which data relates to the at least one new additional query inferred at step428.

If it is determined at step420that the modification cannot be fulfilled by the cache, the process400proceeds to step438. At step438, the visualization software infers at least one new additional query based on the UI range of freedom associated with the modification as described with respect to428. From step438, the process400proceeds to step440. At step440, the visualization software prepares one or more new map-reduce jobs based on the at least one new user query and the at least one new additional query. From step440, the process400proceeds to step442. At step442, the visualization software causes the map-reduce framework to process the one or more new map-reduce jobs. From step442, the process400proceeds to step444.

At step444, the visualization software receives new resultant data from the map-reduce framework. From step444, the process400proceeds to step446. At step446, the visualization software stores in the cache a portion of the new resultant data that is responsive to the at least one new additional query. From step446, the process400proceeds to step448. At step448, the visualization software generates an updated data visualization based on a portion of the new resultant data that is responsive to the at least one new user query. From step448, the process400proceeds to step450. At step450, the visualization software serves the updated data visualization on the visualization interface to the client device. In a typical embodiment, the process400continues until the ends a visualization session on the client application.

FIG. 5illustrates a process500for semantic zooming. In various embodiments, the process500may be performed, for example, as all or part of steps404-410,428-436, and/or steps438-446ofFIG. 4. It should be appreciated that semantic zooming and, more particularly, the process400, is a specific example of a class of modifications that a user may be allowed to make on a visualization interface. In typical embodiment, the process500is implemented via visualization software such as, for example, the visualization software124ofFIG. 1or the visualization software224ofFIG. 2. The visualization software is typically loaded and executing on a server computer such as, for example, the server computer122ofFIG. 1. The process500begins at step502.

At step502, the visualization software determines at least one level of granularity that is beyond what was specified in a user request for data visualization. The at least one level of granularity may be either a level of greater specificity or a level of lesser specificity. Stated somewhat differently, the at least one level of granularity may drill down on (i.e., zoom-in on) the data visualization to include greater specificity or, alternatively, may drill up on (i.e. zoom-out on) the data visualization to provide a higher-level view.

In various embodiments, the at least one level of granularity may include one or more zoom-in next steps and/or one or more zoom-out next steps. The zoom-in next steps typically include next levels of greater specificity supported by the visualization software for a set of input data. The zoom-out next steps typically include next levels of less specificity supported by the visualization software for the set of input data. In some embodiments, the zoom-in next steps and the zoom-out next steps may be pre-configured based on data types included in the set of input data and/or data operations involved. In various other embodiments, determining the at least one level of granularity may involve an analysis of zooming velocity by the user. For example, if the user repeatedly zooms-in on the set of input data within a short period of time, the visualization software can infer that much greater granularity is necessary and specify the at least one level of granularity accordingly. From step502, the process500proceeds to step504.

At step504, the visualization software fashions at least one query based on the at least one level of granularity. From step504, the process500proceeds to step506. At step506, the visualization software prepares one or more map-reduce jobs based on the at least one query. From step506, the process500proceeds to step508. At step508, the visualization software causes a map-reduce framework such as, for example, the map-reduce framework102ofFIG. 1or the map-reduce framework202ofFIG. 2, to process the one or more map-reduce jobs. From step508, the process500proceeds to step510. At step510, the visualization software receives resultant data from the map-reduce framework. From step510, the process500proceeds to step512. At step512, the visualization software stores in a cache the resultant data. After step512, the process500ends.