English:
[0001] This invention relates to information processing, and more particularly, to analytics.
[0003] FIG. 1 illustrates an example method for providing analytics on data sources within an organization.
[0004] FIG. 2 illustrates one example of a system for providing analytics.
[0005] FIG. 3 illustrates another example of a system for providing analytics.
[0006] FIG. 4 illustrates an example method for providing analytics on data sources within an organization.
[0007] FIG. 5 illustrates an example of a computer system that can be employed to implement the systems and methods illustrated in FIGS. 1-4.
[0008] FIG. 6 illustrates an example of a clustered computer system that can be used in conjunction with the systems and methods illustrated in FIGS. 1-4.
[0009] Information management in the enterprise is the new trend in research to enrich the value of structured data in the enterprise by the added value of unstructured data. In this invention, we present a model for performing analytics over structured and unstructured data in the enterprise and in real-time. In the following description, Structured Query Language (SQL) data in relational database systems (RDBMS) is described as structured data, and the term "unstructured data" is intended to extend to any data that is not structured according to an organizational scheme associated with the structured data source, and should be read to include both traditional unstructured data as well as semi-structured data. It will be appreciated that the description throughout should be read as inclusive, and thus the recitation of a given element should be read to include implementations containing either one of that element or more than one of that element. In general, the systems described herein can be represented as a plurality of functional components, each of which can be implemented as any appropriate combination of hardware and programming configured to perform their associated function. In the illustrated example, each functional component is described as a software module stored on a non-transitory computer readable medium and executed by an associated processor, but it will be appreciated that other implementations of the functional components, for example, as dedicated hardware or as a combination of hardware and machine readable instructions, could be used.
[0010] FIG. 1 illustrates an example method 10 for providing real-time analytics on data sources within an organization. At 12, a query of a structured data source is generated. For example, the query can be generated during the execution of an analytics function to retrieve relevant data from an associated enterprise data warehouse. At 14, a query of an unstructured data source is generated at a federation component in response to the query of the structured data source. In one example, the federation component can include a uniform information access layer local to the analytics function that receives the query directly from the analytics function. In another example, the federation component can include a table-valued user defined function at the structured data source, and generating the query of the unstructured data source includes calling the table-valued user defined function as part of the query of the structured data source. In this example, the table-valued user defined function can map the results of the query of the unstructured data source to a virtual SQL table to facilitate the return of the results to the analytics function. To further simplify handling of the results, a call to the table-valued user defined function can include any predicates associated with the query of the structured data source, such that the query of the unstructured data source includes the predicate and the returned data is limited by the predicate.
[0011] At 16, the results of the query of the unstructured data source and the query of the structured data source are merged, in one example, the uniform information access layer can simply combine the results of the query of the unstructured data source and the query of the structured data source into a single SQL query to provide to the analytics function. In another example, an SQL table representing the results of the query of the structured data source can be joined to the virtual SQL table containing the results of the query of the unstructured data source via an SQL join operation at the structured data source before the results are returned to the analytics function.
[0012] At 18, the merged results are stored at an in-memory database that is local to the analytics function. The in-memory database maintains data identified as relevant to the analysis performed by the analytics function, including historical data, incremental updates of the structured data, and real-time data provided from the federation component described above. At 20, the analytics function is executed to provide a real-time analytics output representing the contents of the structured data source and the unstructured data source from the data stored in the in-memory database. The real-time analytics output is displayed to a user at 22.
[0013] FIG. 2 illustrates one example of a system 30 for providing real-time analytics. The system 30 includes a computer system 31 interconnected with set of data sources including a structured data source 32 and an unstructured data source 34. The computer system 31 comprises a processor 36 and a memory module 38, and can be connected to the structured and unstructured data sources 32 and 34 via a communication interface 39. It will be appreciated that the memory 38 can be a removable memory, connected to the processor 36 and the communications interface 39 through an appropriate port or drive, such as an optical drive, a USB port, or other appropriate interface. The memory 38 can be remote from the processor 36, with machine readable instructions stored on the memory provided to the processor via a communications link. The communication interface 36 can comprise any appropriate hardware to communicate with the different data sources 32 and 34 in the enterprise. Further, it will be appreciated that what is described as computer system is not limited to a single computer system, but can also include a clustered system for scalability purposes. An example of such a system is provided below as FIG. 7.
[0014] The memory 34 stores a Virtual Cache (VC) component 42 comprising an In-Memory Database (IMDB) 44 and a Uniform Information Access Layer (UIAL) 46. An analytic software component 48 is configured to run over the Virtual Cache 42. The UIAL 46 is a software component that provides uniform interface to all data sources in the enterprise, such that differences between structured and unstructured data are not apparent at the analytic component 48. The analytic component 48 issues a query against the UIAL component 46 of the VC 42 and receives the answers from the different data sources 32 and 34 in the enterprise in a structured format and stored in the IMDB 44. The analytics component 48 uses data from the virtual cache 42 to provide a report from the relevant contents of the different data sources or to create a mathematical model to predict future behavior based on the past and current data. It will be understood that the term "ln-memory Database - IMDB", as used herein, can include both a true in-memory Database (IMDB) or a large, clustered cache such as a Hadoop cluster. The IMDB 44 maintains (1 ) historical data from the various data sources in the organization identified as relevant to the analysis performed at the analytics component 48 as well as (2) infrequent incremental updates of dynamic data from the various sources, and finally (3) any relevant real-time data provided by the UIAL 46 in response to any queries issued by the analytics component 48.
[0015] A significant difference in this architecture from traditional extraction tools such as Extraction, Transforming, and Loading (ETL) with a RDBMS is that with traditional ETL approach we extract the updated data in a form of the deltas from the different data sources in the enterprise, including those within the data warehouse, and pass it to the destination (IMDB) and that makes it difficult to update the IMDB in real-time with all changes of the different data sources in the enterprise which may or may not be relevant to the analytics function at hand. Instead, in the illustrated system, the deltas are updated from the different data sources, but infrequently. Instead, on demand, SQL and search queries are issued against the different data sources to return only relevant data to the analytics query in real-time since the last delta update. This approach has much better chance in securing relevant data to the analytics component 48 in real-time.
[0016] In the illustrated system 30, the IMDB 44 functions as a local cache to the analytics component 48, aggregating relevant data from all data sources in the organization. The IMDB 44 is implemented with the ability to overflow tables to disk and acquire incremental data from the relevant data sources on a regular basis (e.g., every few minutes). The IMDB can support SQL OLAP windows capability as well as tight integration between table valued user defined function (TVUDF) and the SQL OLAP Windows for the use of the analytics component 48.
[0018] The virtual cache 42 allows for relevant data to be brought together under a common interface transparent to the analytic component 48. Further, the federation performed by the UIAL 46 allows for the consideration of real-time data. In general, frequent updating of a structured data source 32, such as may be found in a data warehouse, from all data sources in the enterprise as being done today, can greatly impact the performance of the data warehouse for query processing, which is the primary purpose of a data warehouse. Accordingly today, the data warehouse is updated overnight, when usage is light, but the consequence of such updating is that information in the data source 32 becomes increasingly out of date between updates. By federating the data locally at the UIAL 46, real time data can be provided to the IMDB 44 for analysis by the analytics component 48. Accordingly, the virtual cache 42 provides a scalable approach to allow the analytics component 48 to operate on real-time data by maintaining a local store of relevant data in the IMDB 44 and providing new data directly from structured and unstructured data sources through the uniform interface provided by the UIAL 46. The access to realtime data can provide a significant increase in the accuracy of predictions made at the analytics component 48.
[0019] FIG. 3 illustrates another example of a system 50 for providing realtime analytics. The system 50 includes a computer system 51 comprising a processor 52 and a memory 54. The computer system 51 further includes an analytics component 56, configured to produce an analytic output from data stored in a virtual cache 60. The analytics component 56 can comprise a hardware or software component that performs an analysis of the data stored in the virtual cache 60 to provide an output comprehensible to a human operator. In one example, the analytics component 56 is implemented as a software program on the memory 54. The virtual cache 60 includes an in-memory database (IMDB) 62 that serves as a local cache for the analytics component 56, and a uniform interface access layer 64 is a software component that provides uniform interface to all data sources in the enterprise, such that differences between structured and unstructured data are not apparent at the analytic component 56.
[0020] The computer system 51 uses a communication interface 66, which can comprise any appropriate hardware, to communicate with a second computer system 67, comprising a processor 68 and a memory 69. The memory 69 of the second computer system 67 stores a data warehouse 70 comprising a data table 72 storing data relevant to the analytics component 56, and a database engine 74 configured to provide a SQL table representing data responsive to a SQL query. The data warehouse 70 is operatively connected with a plurality of enterprise relevant data sources, referred to herein as unstructured data sources 80. As used herein, the term "unstructured data" is intended to extend to any data that is not structured according to an organization scheme associated with the structured data source, and should be read to include both traditional unstructured data and semi- structured data. For example, the unstructured data sources can include a
[0021] In FIG. 3, it is assumed that the vast majority of the data relevant for analysis will be located within a structured data source, such as the data table 72 within the data warehouse 70. Accordingly, the illustrated system 50 performs the federation in the data warehouse 70, and the data warehouse returns a table representing the desired result in a structured format. To this end, the system allows for integration (i.e., federation) of data from structured data sources (e.g., 72) and unstructured data sources 60 to be performed in the data warehouse 70, specifically utilizing a table-valued User Defined Function (TVUDF) 92. As used here, the table- valued user defined function 92 is a user defined function stored at the data warehouse that, when called as part of a query of the data warehouse, provides an output relevant to the query in the form of a table.
[0022] During operation, the UIAL 64 invokes the TVUDF 92 indirectly, passing enough information to enable the TVUDF to invoke a remote federated query to the unstructured data sources 80. The TVUDF 92 is invoked in the data warehouse 70 and, in turn, it invokes remotely a web services request that performs a federated query to the unstructured data sources 80. The TVUDF 92 maps the returned results from the unstructured data sources 80 into a virtual table and instructs the database engine 74 to join the virtual table with a table representing relevant data from the data table 72, resulting in a new virtual table that is returned to the UIAL 64 to be stored in the I DB 62. The TVUDF 92 provides the query results from the unstructured data source 80 as a virtual table, allowing the data warehouse
[0023] In the illustrated system 50, an analytics component 56 generates a traditional SQL query with an embedded call to the TVUDF 92. In the illustrated example, the TVUDF 92 can query multiple unstructured data sources, and the call to the TVUDF 92 includes a query to the unstructured data sources 80, the TVUDF functions as a federation engine 94 between the data table 72 in the data warehouse 70 and the unstructured data sources 80. The returned virtual table becomes part of the original SQL query and gets executed, effectively joining the virtual table from the unstructured data sources with the relevant tables from the data table 72 in the data warehouse 70. The joined results are provided to the analytics component 56 as a single SQL table to be saved in the IMDB 62 within the virtual cache 60 associated with the analytics component 56.
[0024] in one example, an SQL compiler 96 is configured to format the SQL query provided by the analytics component 56 for execution on the data table. In the illustrated system, the SQL compiler is configured to pass any predicates in the SQL query to the TVUDF function 92 at runtime. The TVUDF 92 supplements the search query to the unstructured federation engine 94 with only the relevant predicates. This, in turn, optimizes the amount of data returned back over the network from the unstructured federation engine 94 to the data warehouse 70. In other words, the predicates passing by the SQL compiler 96 to the TVUDF 92 will limit extraneous data returned to the table-valued UDF.
[0025] The illustrated system 50 thus provides real-time data to the analytics component 56 while providing a number of advantages. The system simplifies the integration of structured and unstructured data in a given query and hides the complexities from the U1AL 64 and eliminates non-relevant data from the unstructured data sources 80 before joining the structured data. Finally, the system 50 leverages the existing SQL join capability at the data warehouse 70 to return SQL data types back to the IMDB 62, placing the data in an appropriate form for use at the analytics component 56.
[0026] In view of the foregoing structural and functional features described above in FIG. 3, an example methodology wiil be better appreciated with reference to FIG. 4. While, for purposes of simplicity of explanation, the methodology of FIG. 4 is shown and described as executing serially, it is to be understood and appreciated that the present invention is not limited by the illustrated order, as some actions could in other examples occur in different orders and/or concurrently from that shown and described herein.
[0027] FIG. 4 illustrates an example method 100 for providing real-time analytics on data sources within the enterprise. At 102, an analytics component composes a SQL query against a structured data source embedding a special table- valued user defined function (TVUDF) in the SQL query to handle unstructured data. At 104, the TVUDF generates a query to an unstructured federation engine, which, in turn, issues a web services request. The federation engine issues the web services request as a query against inverted indexes representing various unstructured data sources and returns the results back to the TVUDF as a stream. To further simplify handling of the results from the unstructured data sources, the web services request discussed above can include any relevant predicates from the SQL query so the unstructured federation engine would filter the returned data before sending the data over the network back to the TVUDF.
[0028] At 106, the TVUDF maps the returned stream into a virtual table. The TVUDF instructs an engine associated with the structured data source to execute a JOIN operation between the relevant structured tables and the virtual table representing the result from the unstructured data sources. The SQL query result, a table, is returned back to a uniform information access layer (UIAL) component. At 108, the UIAL component stores the returned query result into an in-memory database (!MDB). The in-memory database maintains historical data, infrequent incremental updates, and any rea!-time data returned in the IMDB from the federated structured and unstructured data sources. At 110, the analytics component processes the relevant data in the IMDB component to provide an output. This output can, for example, comprise a display of the results of the analytics function performed y the analytics component to a user.
[0029] FIG. 5 is a schematic block diagram illustrating an exemplary system 200 of hardware components capable of implementing examples of the present disclosed in FIGS. 1-4, such as the real-time analytics systems illustrated in FIGS. 2 and 3. The system 200 can include various systems and subsystems. The system 200 can be a personal computer, a laptop computer, a workstation, a computer system, an appliance, an application-specific integrated circuit (ASIC), a server, a server blade center, a server farm, etc.
[0030] The system 200 can includes a system bus 202, a processing unit 204, a system memory 206, memory devices 208 and 210, a communication interface 212 (e.g., a network interface), a communication link 214, a display 216 (e.g., a video screen), and an input device 218 (e.g., a keyboard and/or a mouse). The system bus 202 can be in communication with the processing unit 204 and the system memory 206. The additional memory devices 208 and 210, such as a hard disk drive, server, stand alone database, or other non-volatile memory, can also be in communication with the system bus 202. The system bus 202 interconnects the processing unit 204, the memory devices 206-210, the communication interface 212, the display 216, and the input device 218. in some examples, the system bus 202 also interconnects an additional port (not shown), such as a universal serial bus (USB) port.
[0031] The processing unit 204 can be a computing device and can include an application-specific integrated circuit (ASIC). The processing unit 204 executes a set of instructions to implement the operations of examples disclosed herein. The processing unit can include a processing core.
[0032] The additional memory devices 206, 208 and 210 can store data, programs, instructions, database queries in text or compiled form, and any other information that can be needed to operate a computer. The memories 206, 208 and 210 can be implemented as computer-readable media (integrated or removable) such as a memory card, disk drive, compact disk (CD), or server accessible over a network. In certain examples, the memories 206, 208 and 210 can comprise text, images, video, and/or audio, portions of which can be available in different human.
[0033] Additionally, the memory devices 208 and 210 can serve as databases or data storage such as the in-memory databases 46 and 62 illustrated in FIGS. 2 and 3. Additionally or alternatively, the system 200 can access an external data source or query source through the communication interface 212, which can communicate with the system bus 202 and the communication link 214.
[0034] In operation, the system 200 can be used to implement a real-time analytics system that produces a report based on queries of structured and unstructured data sources. The queries can be formatted in accordance with various query database protocols, including SQL. Computer executable logic for impiementing the real-time analytics system resides on one or more of the system memory 206, and the memory devices 208, 210 in accordance with certain examples. The processing unit 204 executes one or more computer executable instructions originating from the system memory 206 and the memory devices 208 and 210. The term "computer readable medium" as used herein refers to a medium that participates in providing instructions to the processing unit 204 for execution.
[0035] F!G. 6 is a schematic block diagram illustrating an exemplary system 300 of clustered scalable hardware components. The system 300 comprises a plurality of clustered hardware components 301-303 interconnected by a fast network 310 allowing collaboration of software components running on these nodes toward impiementing a scalable Virtual Cache and efficient analytics component over the Virtual Cache. In one implementation, each node 301-303 can comprise a computer system similar to that illustrated in FIG. 5.

中文:
[0001]本发明涉及信息处理,并且更具体地涉及分析。
[0003]图1图示用于对组织中的数据源提供分析的示例方法。
[0004]图2图示用于提供分析的系统的一个示例。
[0005]图3图示用于提供分析的系统的另一示例。
[0006]图4图示用于对组织中的数据源提供分析的示例方法。
[0007]图5图示能够用于实现图1-图4中图示的系统及方法的计算机系统的示例。
[0008]图6图示能够与图1-图4中图示的系统和方法结合使用的集群化计算机系统的示例。
[0009]企业中的信息管理是研究中的新趋势,以通过非结构化数据的添加值使该企业中的结构化数据的值充实。在本发明中,我们提出一种用于对企业中的结构化数据和非结构化数据执行实时分析的模型。在以下描述中,将关系数据库系统(RDBMS)中的结构化查询语言(SQL)数据描述为结构化数据,并且术语“非结构化数据”意在扩展至未根据与结构化数据源关联的组织规划(schema)被结构化的任何数据,并且应当被解读为包括传统的非结构化数据以及半结构化数据。将理解,全文描述应当被解读为包括性的,因此,对特定元素的记载应当被解读为包括包含该元素之一或不止该元素之一的实现。通常,能够将本文描述的系统表现为多个功能组件,能够将每个功能组件实现为被配置为执行其关联功能的硬件和编程的任何合适组合。在所图示的示例中,将每个功能组件描述为存储在非瞬态计算机可读介质上且由关联处理器运行的软件模块,但是将理解,能够使用功能组件的其它实现,例如作为专用硬件或硬件和机器可读指令的结合。
[0010]图1图示用于对组织中的数据源提供实时分析的示例方法10。在12处,生成结构化数据源的查询。例如,在分析函数的运行期间能够生成查询,以从关联的企业数据仓库中检索相关数据。在14处,响应于结构化数据源的查询,在联邦(federation)组件处生成非结构化数据源的查询。在一个示例中,联邦组件能够包括位于分析函数本地的统一信息访问层,该统一信息访问层直接从分析函数接收查询。在另一示例中,联邦组件能够包括位于结构化数据源处的表值用户定义函数,并且生成非结构化数据源的查询包括调用表值用户定义函数作为结构化数据源的查询的一部分。在该示例中,表值用户定义函数能够将非结构化数据源的查询的结果映射到虚拟SQL表,以促进将结果返回至分析函数。为进一步简化结果的处理,对表值用户定义函数的调用能够包括与结构化数据源的查询关联的任何谓词(predicate),使得非结构化数据源的查询包括该谓词,并且由该谓词限制所返回的数据。
[0011]在16处,合并非结构化数据源的查询的结果和结构化数据源的查询的结果。在一个示例中,统一信息访问层能够简单地将非结构化数据源的查询的结果和结构化数据源的查询的结果合并到单个SQL查询内,以提供至分析函数。在另一示例中,在将这些结果返回至分析函数以前,能够在结构化数据源处经由SQL连接操作将代表结构化数据的查询的结果的SQL表联接至包含非结构化数据源的查询的结果的虚拟SQL表。
[0012]在18处,将所合并的结果存储在对于分析函数本地的内存数据库中。该内存数据库维持被标识为与由分析函数执行的分析相关的数据,该数据包括历史数据、结构化数据的增量更新以及从上面描述的联邦组件中提供的实时数据。在20处,运行分析函数,以根据存储在内存数据库中的数据提供代表结构化数据源的内容和非结构化数据源的内容的实时分析输出。在22处,将该实时分析输出显示给用户。
[0013]图2图示用于提供实时分析的系统30的一个示例。系统30包括与包括结构化数据源32和非结构化数据源34的一组数据源互连的计算机系统31。计算机系统31包括处理器36以及存储器模块38,并且能够经由通信接口39连接至结构化数据源32和非结构化数据源34。将理解,存储器38能够是可移除的存储器,能够通过诸如光学驱动器、USB端口或其它适合接口之类的合适的端口或驱动器连接至处理器36以及通信接口39。存储器38能够远离处理器36,此时存储在该存储器上的机器可读指令经由通信链接提供至处理器。通信接口36能够包括任何合适的硬件,以与企业中不同的数据源32和34通信。进一步,将理解,被描述作为计算机系统的不局限于单个计算机系统,而是为可扩展目的还能够包括集群化系统。下面提供这样的系统的示例作为图7。
[0014]存储器34存储虚拟缓存(VC)组件42,虚拟缓存组件42包括内存数据库(IMDB)44以及统一信息访问层(UIAL)46。将分析软件组件48配置为在虚拟缓存42之上运行。UIAL46是软件组件,其提供对企业中的全部数据源的统一接口,使得结构化数据和非结构化数据之间的区别在分析组件48处不明显。分析组件48对VC42的UIAL组件46发出查询并且从企业中的不同的数据源32和34接收呈结构化格式的且存储在IMDB44中的应答。分析组件48使用来自虚拟缓存42的数据,以提供来自不同数据源的相关内容的报告或创建数学模型来基于过去的或当前的数据预测未来行为。将理解,本文使用的术语“内存数据库-IMDB”既能够包括真实的内存数据库(IMDB)又能够包括大型集群化缓存,例如Hadoop集群。IMDB44维持(1)被标识为与在分析组件48处执行的分析相关的来自组织中各数据源的历史数据以及(2)来自各个源的动态数据的不频繁增量更新,以及最终(3)响应于由分析组件48发出的任何查询由UIAL46提供的任何相关实时数据。
[0015]此架构与传统提取工具(例如对RDBMS的提取、转换和加载(ETL))的显著不同之处在于:用传统ETL方法,我们从企业中的不同数据源提取呈增量(delta)形式的更新数据(包括数据仓库中的那些数据),并且将更新数据传送至目的地(IMDB),并且这使得利用与手边的分析函数可能相关或可能不相关的企业中不同数据源的全部改变实时更新IMDB变得困难。作为替代,在图示的系统中,更新来自不同数据源的增量,但这不频繁发生。作为替代,经要求,从上次增量更新起对不同数据源发出SQL查询和搜索查询,以仅将相关数据实时返回给分析查询。此方法在将相关数据实时地使输送至分析组件48时具有更好机会。
[0016]在所图示的系统30中,IMDB44起到对分析组件48的本地缓存的作用,聚集来自组织中的全部数据源的相关数据。IMDB44被实现为具有将表充溢到磁盘上以及定期(例如每数分钟)从相关数据源获取增量数据的能力。IMDB能够支持SQL OLAP窗能力以及表值用户定义函数(TVUDF)和SQL OLAP窗之间的紧密集成,以使用分析组件48。
[0018]虚拟缓存42允许在对分析组件48透明的公共接口下将相关数据聚集在一起。进一步,由UIAL46执行的联邦允许考虑实时数据。通常,白天正在进行的企业中全部数据源中结构化数据源32的频繁更新(例如可以在数据仓库中发现的)能够大大影响数据仓库对查询处理的性能,这是数据仓库的主要用途。相应地如今,数据仓库夜间更新,此时使用较少,但是这样更新的结果是数据源32中的信息在更新期间变得越来越过时。通过对位于UIAL46本地的数据进行联邦,能够向IMDB44提供实时数据,以供分析组件48进行分析。相应地,虚拟缓存42提供可扩展的方法,以通过维持相关数据在IMDB44中的本地存储以及通过凭借由UIAL46提供的统一接口提供直接来自结构化数据源和非结构化数据源的新数据,允许分析组件48对实时数据进行操作。对实时数据的访问能够提供在分析组件48处做出的预测的精度方面的显著增加。
[0019]图3图示用于提供实时分析的系统50的另一示例。系统50包括计算机系统51,计算机系统51包括处理器52及存储器54。计算机系统51进一步包括分析组件56,将分析组件56配置为从存储在虚拟缓存60中的数据中产生分析输出。分析组件56能够包括硬件或软件组件,该硬件或软件组件执行对存储在虚拟缓存60中的数据的分析,以提供人类操作者可理解的输出。在一个示例中,将分析组件56实现为存储器54上的软件程序。虚拟缓存60包括内存数据库(IMDB)62,内存数据库(IMDB)62担当用于分析组件56的本地缓存,并且统一接口访问层64是向企业中的全部数据源提供统一接口的软件组件,使得结构化数据和非结构化数据之间的差异在分析组件56处不明显。
[0020]计算机系统51使用通信接口66来与第二计算机系统67通信,通信接口66能够包括任何合适的硬件,第二计算机系统67包括处理器68及存储器69。第二计算机系统67的存储器69存储数据仓库70,数据仓库70包括数据表72和数据库引擎74,数据表72存储与分析组件56相关的数据,将数据库引擎74配置为提供代表对SQL查询有响应的数据的SQL表。数据仓库70可操作地与多个与企业相关的数据源连接,该与企业相关的数据源在本文被称为非结构化数据源80。如本文所用的,术语“非结构化数据”旨在扩展至未根据与结构化数据源关联的组织计划而被结构化的任何数据,并且应被解读为既包括传统的非结构化数据还包括半结构化数据。例如,非结构化数据源能够包括客户关系管理(CRM)组件82、文档藏库84以及经由例如互联网连接的实时馈给86,客户关系管理组件82包含来自客户的未分类反馈,文档藏库84包含原始文本文档。进一步,将理解,被描述为计算机系统的不局限于单个计算机系统,而是为了扩展目的还能够包括集群化系统。
[0021]在图3中,假设与分析相关的数据中绝大多数将位于结构化数据源中,如在数据仓库中的数据表72中。相应地,所图示的系统50执行数据仓库70中的联邦,并且数据仓库返回呈结构化格式的代表期望结果的表。为此,该系统允许在数据库仓库70中执行来自结构化数据源(例如72)和非结构化数据源60的数据的集成(例如联邦),特别地利用表值用户定义函数(TVUDF)92执行的集成。如在此使用的,表值用户定义函数92是存储在数据仓库处的用户定义函数,当调用该用户定义函数作为数据仓库的查询的一部分时,该用户定义函数以表的形式提供与查询相关的输出。
[0022]在操作期间,UIAL64间接唤起TVUDF92,传递足够信息来使TVUDF能够唤起对非结构化数据源80的远程联邦化查询。TVUDF92在数据仓库70中被唤起,接着其远程地唤起网络服务请求,该网络服务请求执行对非结构化数据源80的联邦查询。TVUDF92将来自非结构化数据源80的返回结果映射到虚拟表中并且指示数据库引擎74将虚拟表与代表来自数据表72的相关数据的表联接,这生成新的虚拟表,将该新的虚拟表返回至UIAL64去存储在IMDB62中。TVUDF92提供来自非结构化数据源80的查询结果作为虚拟表,由于其成为SQL联接操作,这允许数据仓库70有效地执行结构化数据和非结构化数据的查询之间的联邦。
[0023]在所图示的系统50中,分析组件56生成具有对TVUDF92的嵌入调用的传统SQL查询。在所图示的示例中,TVUDF92能够查询多个非结构化数据源,并且对TVUDF92的调用包括对非结构化数据源80的查询,TVUDF充当数据仓库70中的数据表72和非结构化数据源80之间的联邦引擎94。被返回的虚拟表成为原始SQL查询的一部分且得到运行,这有效地将来自非结构化数据源的虚拟表与来自数据仓库70中的数据表72的相关表联接。将所联接的结果提供至分析组件56作为单个SQL表,以保存在与分析组件56关联的虚拟缓存60内的IMDB62中。
[0024]在一个示例中,将SQL编译器96配置为对由分析组件56提供的SQL查询进行格式编排,以在数据表上的运行。在所图示的系统中,将SQL编译器配置为在运行时将SQL查询中的任何谓词传递至TVUDF函数92。TVUDF92仅将相关谓词补充给对非结构化联邦引擎94的搜索查询。然后,这优化通过网络从非结构化联邦引擎94返回给数据仓库70的数据量。换句话说,SQL编译器96向TVUDF92传递的谓词将限制向表值UDF返回的无关数据。
[0025]因此,所图示的系统50向分析组件56提供实时数据,同时提供多个优势。该系统简化结构化数据和非结构化数据在特定查询中的集成,并且对UIAL64隐藏复杂性,并且在联接结构化数据以前消除来自非结构化数据源80的不相关数据。最终,系统50利用数据仓库70处的现有SQL联接能力来将SQL数据类型返回至IMDB62,这放置适当形式的数据来供在分析组件56处使用。
[0026]鉴于上面在图3中描述的前述结构特征和功能特征,将对照图4更好地理解示例方法。虽然为简化解释目的,将图4的方法示出和描述为顺序地执行,但是要理解和领会,本发明不受所图示的顺序限制,因为一些动作可能在其它示例中以不同的顺序发生和/或与本文示出和描述的顺序同时发生。
[0027]图4图示用于对企业内的数据源提供实时分析的示例方法100。在102处,分析组件构造对结构化数据源的SQL查询,该SQL查询在该SQL查询中嵌有特殊的表值用户定义函数(TVUDF)来处理非结构化数据。在104处,TVUDF生成对非结构化联邦引擎的查询,该非结构化联邦引擎接着发出网络服务请求。该联邦引擎发出网络服务请求作为对代表各种非结构化数据源的反向索引的查询,并且将结果返回至TVUDF作为流。为进一步简化来自非结构化数据源的结果的处理,上面讨论的网络服务请求能够包括来自SQL查询的任何相关谓词,因此非结构化联邦引擎在通过网络将被返回的数据发送回至TVUDF以前会过滤被返回的数据。
[0028]在106处,TVUDF将被返回的流映射到虚拟表中。TVUDF指示与结构化数据源关联的引擎运行相关结构化表和代表来自非结构化数据源的结果的虚拟表之间的JOIN操作。将SQL查询结果(一种表)返回至统一信息访问层(UIAL)组件。在108处,UIAL组件将被返回的查询结果存储到内存数据库(IMDB)中。该内存数据库维持历史数据、不频繁的增量更新以及从联邦的结构化数据源和非结构化数据源返回的在IMDB中的任何实时数据。在110处,分析组件处理IMDB组件中的相关数据以提供输出。该输出能够例如包括向用户显示由分析组件执行的分析函数的结果。
[0029]图5是图示能够实现图1至图4中公开的本发明示例的由硬件组件组成的示例性系统200(如图2和图3中图示的实时分析系统)的示意性框图。系统200能够包括各种系统和子系统。系统200能够是个人计算机、膝上型计算机、工作站、计算机系统、仪器、专用集成电路(ASIC)、服务器、刀片式服务器中心、服务器群等等。
[0030]系统200能够包括系统总线202、处理单元204、系统存储器206、存储器设备208和210、通信接口212(例如网络接口)、通信链接214、显示器216(例如显示屏)以及输入设备218(例如键盘和/或鼠标)。系统总线202能够与处理单元204和系统存储器206通信。附加存储器设备208和210,如硬盘驱动器、服务器、独立数据库或其它非易失性存储器,也能够与系统总线202通信。系统总线202与处理单元204,存储器设备206-210、通信接口212、显示器216以及输入设备218互连。在一些示例中,系统总线202还与附加端口(未示出)互连,如与通用串行总线(USB)端口互连。
[0031]处理单元204能够是计算设备且能够包括专用集成电路(ASIC)。处理单元204运行一组指令,以实现本文公开的示例的操作。处理单元能够包括处理核。
[0032]附加存储器设备206、208和210能够存储呈文本形式或呈编译形式的数据、程序、指令、数据库查询以及操作计算机可能需要的任何其它信息。能够将存储器206、208及210实现为(集成的或可移除的)计算机可读介质,如存储卡、磁盘驱动器、光盘(CD)或可通过网络访问的服务器。在特定示例中,存储器206、208和210能够包括文本、图像、视频和/或音频,文本、图像、视频和/或音频的各部分能够用于不同人。
[0033]另外,存储器设备208和210能够担当数据库或数据存储,如图2及图3中图示的内存数据库46和62。另外或可替代地,系统200能够通过通信接口212访问外部数据源或查询源,通信接口212能够与系统总线202及通信链接214通信。
[0034]在操作中,系统200能够用于实现实时分析系统,该实时分析系统在结构化数据源和非结构化数据源的查询基础上产生报告。能够根据包括SQL在内的各种查询数据库协议对查询进行格式编排。根据特定示例,用于实现实时分析系统的计算机可运行逻辑驻留在系统存储器206及存储器设备208和210中的一个或多个上。处理单元204运行源于系统存储器206及存储器设备208和210的一个或多个计算机可运行指令。如本文使用的术语“计算机可读介质”指参与向处理单元204提供用于运行的指令的介质。
[0035]图6是图示具有集群化的可扩展硬件组件的示例性系统300的示意框图。系统300包括通过快速网络310互连的多个集群化硬件组件301-303,多个集群化硬件组件301-303允许在这些节点上运行的软件组件的合作,以用于实现可扩展的虚拟缓存和该虚拟缓存上的有效分析组件。在一个实现中,每个节点301-303能够包括与图5中图示的计算机系统类似的计算机系统。