Source: http://patents.com/us-10198459.html
Timestamp: 2019-02-21 04:18:07
Document Index: 83197812

Matched Legal Cases: ['Application No. 14897734', 'Application No. 14897860', 'Application No. 201480048972', 'Application No. 201480048975', 'Application No. 201480048969', 'Application No. 201480048973']

US Patent # 1,019,8459. Data model change management - Patents.com
United States Patent 10,198,459
A version manager that manages multiple versions of a logical data model. Each version of the logical data model exposes a data storage system using a corresponding semantic mapping set that for multiple logical data model entities, maps at least the logical data model entity to at least a corresponding physical data model entity of the data storage system. The logical data model itself may consistent of multiple constituent logical data models, each used for a different vertical to expose the same underlying storage system. The version manager corresponds each of one or more data consumers of the data storage system with a version of the logical data model. Accordingly, when requests come in from a particular data consumer, the corresponding version of the logical data model is used.
Liu; Zhen (Tarrytown, NY), Mak; Chiu Chun Bobby (Beijing, CN), Deng; Xiaomin (Beijing, CN), Huynh; Yet Luong (Beijing, CN), Zhou; Chunxing (Beijing, CN)
Family ID: 1000003806759
14/549,452
US 20160019244 A1 Jan 21, 2016
PCT/CN2014/082209 Jul 15, 2014
Current CPC Class: G06F 16/212 (20190101)
Current International Class: G06F 16/84 (20060101)
Field of Search: ;707/999.1,999.102,803,999.101,E17.125,769,802,809,806,791,695,601,999.203
5550971 August 1996 Brunner et al.
5842212 November 1998 Ballurio et al.
6785689 August 2004 Daniel
7085757 August 2006 Dettinger et al.
7185317 February 2007 Fish et al.
7359906 April 2008 Hargett et al.
7533107 May 2009 Gupta et al.
7730056 June 2010 Kaiser et al.
7849065 December 2010 Kamani et al.
7849104 December 2010 Sun et al.
7895568 February 2011 Goodwin et al.
7899837 March 2011 Polo-Malouvier et al.
7949654 May 2011 Das et al.
8346811 January 2013 Wyatt et al.
8352432 January 2013 Prahlad et al.
8442982 May 2013 Jacobson et al.
8510732 August 2013 Chevrette et al.
8577927 November 2013 Fabret et al.
8838654 September 2014 Hannah et al.
9298857 March 2016 Mineno
2002/0107840 August 2002 Rishe
2003/0046292 March 2003 Subramanian
2003/0115035 June 2003 Kulshreshtha et al.
2003/0154191 August 2003 Fish et al.
2003/0191727 October 2003 Yao et al.
2004/0230571 January 2004 Robertson
2005/0091249 April 2005 Hanson et al.
2005/0102538 May 2005 Hunt et al.
2005/0149552 July 2005 Chan
2005/0234889 October 2005 Fox et al.
2005/0289174 December 2005 Kolli
2006/0089941 April 2006 Jenni et al.
2006/0172906 August 2006 Wu et al.
2006/0173906 August 2006 Chu et al.
2006/0271885 November 2006 Pittendrigh
2006/0277224 December 2006 Aftab et al.
2006/0282470 December 2006 Yu et al.
2006/0287899 December 2006 Balmelli et al.
2006/0294181 December 2006 Barzacanos et al.
2007/0118540 May 2007 Guo
2007/0240109 October 2007 Chandra et al.
2007/0299858 December 2007 Norcott
2008/0077598 March 2008 Wilmering et al.
2008/0154745 June 2008 Chan et al.
2008/0288510 November 2008 Bloesch
2008/0320012 December 2008 Loving
2010/0004603 January 2010 Kristensen et al.
2010/0063880 March 2010 Atsmon et al.
2010/0076947 March 2010 Kurapat
2012/0078877 March 2012 Copperman
2012/0110021 May 2012 Hryniewicki
2012/0260265 October 2012 Doganata et al.
2013/0080349 March 2013 Bhola et al.
2013/0254199 September 2013 Surendran et al.
2014/0013297 January 2014 Cook et al.
2014/0040533 February 2014 Huang et al.
2014/0046936 February 2014 John
2014/0149369 May 2014 Fan et al.
2014/0324472 October 2014 Delaney et al.
2015/0142836 May 2015 Borges et al.
2015/0347621 December 2015 Hu et al.
2016/0055225 February 2016 Xu et al.
2016/0171511 June 2016 Goel et al.
101079736 Nov 2007 CN
101617290 Dec 2009 CN
101794315 Aug 2010 CN
101808109 Aug 2010 CN
101916299 Dec 2010 CN
105138526 Dec 2015 CN
0838771 Apr 1998 EP
1081611 Mar 2001 EP
International Preliminary Report on Patentability issued in PCT Application No. PCT/CN2014/082210, dated Jan. 26, 2017. cited by applicant .
International Preliminary Report on Patentability issued in PCT Application No. PCT/CN2014/082209 dated Jan. 26, 2017. cited by applicant .
International Preliminary Report on Patentability issued in PCT Application No. PCT/CN2014/082212 dated Jan. 26, 2017. cited by applicant .
International Preliminary Report on Patentability issued in PCT Application No. PCT/CN2014/082213 dated Jan. 26, 2017. cited by applicant .
U.S. Appl. No. 14/549,463, filed Nov. 20, 2014, Liu, et al. cited by applicant .
U.S. Appl. No. 14/550,123, filed Nov. 21, 2014, Liu, et al. cited by applicant .
U.S. Appl. No. 14/549,468, filed Nov. 20, 2014, Liu, et al. cited by applicant .
"International Search Report & Written Opinion Received for PCT Patent Application No. PCT/CN2014/082210", dated Apr. 20, 2015, 13 pages. cited by applicant .
"International Search Report and Written Opinion Issued in PCT Application No. PCT/CN2014/082212", dated Mar. 27, 2015, 12 pages. cited by applicant .
"International Search Report and Written Opinion Issued in PCT Patent Application No. PCT/CN2014/082213", dated Apr. 29, 2015, 14 pages. cited by applicant .
"International Search Report and Written Opinion Issued for PCT Patent Application No. PCT/CN2014/082209", dated Mar. 27, 2015, 11 Pages. cited by applicant .
Braunschweig, et al., "A flexible graph-based data model supporting incremental schema design and evolution Services", In Proceedings of the 11th international conference on Current Trends in Web Engineering, Jun. 20, 2011, 2 pages. cited by applicant .
Zaman, et al., "Information Integration for Heterogeneous Data Sources Services", In Proceedings of IOSR Journal of Engineering, vol. 2 No. 4, Apr. 2012, 4 pages. cited by applicant .
Dong, et al., "Indexing Dataspaces", In Proceedings of ACM SIGMOD International Conference on Management of Data, Jun. 12, 2007, pp. 43-54. cited by applicant .
Stuckenschmidt, et al., "Exploring Large Document Repositories with RDF Technology: The DOPE Project", In IEEE Intelligent Systems, vol. 19, Issue 3, May 2004, pp. 34-40. cited by applicant .
Branson, et al., "A Data Model for Integrating Heterogeneous Medical Data in the Health-e-Child Project", In Proceedings of Studies in Health Technology and Informatics, Jun. 16, 2014, 10 pages. cited by applicant .
Hema, et al., "Service Oriented Ontology Based Data Federation for Heterogeneous Data Sources", In Journal of Theoretical and Applied Information Technology, vol. 55, No. 1, Sep. 10, 2013, pp. 126-136. cited by applicant .
Chirathamjaree, Chaiyaporn., "A Data Model for Heterogeneous Data Sources", In IEEE International Conference on e-Business Engineering, Oct. 22, 2008, pp. 121-127. cited by applicant .
Smit, et al., "A Unified Representation for the Model-based Visualization of Heterogeneous Anatomy Data", In Proceedings of Eurographics Conference on Visualization, Jul. 15, 2014, 5 pages. cited by applicant .
Liu, et al., "Data Model Indexing for Model Queries", unfiled US Patent Application. cited by applicant .
Liu, et al., "Managing Multiple Data Models Over Data Storage System", unfiled US Patent Application. cited by applicant .
Liu, et al., "Data Retrieval Across Multiple Models", unfiled US Patent Application. cited by applicant .
Extended European Search Report issued in European Application No. EP109779RK, dated Nov. 8, 2017 (Copy Attached). cited by applicant .
Office Action issued in U.S. Appl. No. 14/459,123 dated Sep. 21, 2017 (Copy Attached). cited by applicant .
Office Action issued in U.S. Appl. No. 14/549,452 dated Sep. 21, 2017 (Copy Attached). cited by applicant .
Office Action issued in U.S. Appl. No. 14/549,463 dated Jan. 30, 2018 (Copy Attached). cited by applicant .
"Office Action Issued in European Patent Application No. 14897734.1", dated Mar. 5, 2018, 7 Pages. cited by applicant .
"Non Final Office Action Issued in U.S. Appl. No. 14/549,468", dated Sep. 21, 2017, 13 Pages. cited by applicant .
"Final Office Action Issued in U.S. Appl. No. 14/550,123", dated Apr. 19, 2018, 6 Pages. cited by applicant .
"Notice of Allowance Issued in U.S. Appl. No. 14/549,468", dated Apr. 23, 2018, 8 Pages. cited by applicant .
"Search Report Issued in European Patent Application No. 14897860.4", dated Nov. 8, 2017, 7 Pages. cited by applicant .
"First Office Action and Search Report Issued in Chinese Patent Application No. 201480048972.6", dated Apr. 4, 2018, 12 Pages. cited by applicant .
"First Office Action and Search Report Issued in Chinese Patent Application No. 201480048975.X", dated May 3, 2018, 15 Pages. cited by applicant .
Luozi, et al., "Database System and its Application Foundation", In Journal of Tsinghua University Press, Jun. 30, 2005, pp. 7-12. cited by applicant .
"Non Final Office Action Issued in U.S. Appl. No. 14/550,123", dated Jul. 27, 2018, 12 Pages. cited by applicant .
"First Office Action Issued in Chinese Patent Application No. 201480048969.4", dated Aug. 16, 2018, 14 Pages. cited by applicant .
"First Office Action Issued in Chinese Patent Application No. 201480048973.0", dated Jul. 2, 2018, 12 Pages. cited by applicant.
This application is a Continuation of PCT International Application PCT/CN2014/082209 filed Jul. 15, 2014, which is expressly incorporated herein by reference in its entirety.
1. A computer system comprising a plurality of logical data models configured to expose a data storage system, the system comprising: one or more processors; the data storage system; a plurality of logical data model versions, wherein each of the plurality of logical data model versions includes at least a corresponding semantic mapping set that correlates a corresponding plurality of data entities in the logical data model version to at least a corresponding set of data entities in a corresponding physical data model version of the data storage system; and a version manager that associates each of the plurality of logical data model versions to a different set of one or more identified consumers, such that the data storage system is exposed to a plurality of identified consumers through different logical data model versions, wherein the version manager is further configured to perform the following in response to detecting a change to the physical data model of the data storage system: create a new version of the physical data model; create a new logical data model version corresponding to the new version of the physical data model; assign an identified set of one or more identified consumers to the new logical data model version; and expose the data storage system to the identified set of one or more identified consumers according to the new logical data model version, the plurality of logical data model versions including (i) a first logical data model version that is associated with a first set of one or more identified consumers and that corresponds to a first physical data model version, and (ii) a second logical data model version that is associated with a second set of one or more identified consumers and that corresponds to a second physical data model version, the version manager concurrently exposing the first and second logical data model versions to the first and second sets of one or more identified consumers.
2. The system in accordance with claim 1, wherein at least one of the plurality of logical data models comprises a logical data model comprising a plurality of constituent logical data models, each constituent logical data model configured to expose the data storage system using a different constituent semantic mapping.
3. The system in accordance with claim 2, further comprising: a logical data modelling notation component configured to use a corresponding modelling notation when communicating with a constituent logical data model.
4. The system in accordance with claim 2, each constituent logical data model used by a different vertical.
5. The system in accordance with claim 1, wherein at least one of the plurality of versions of the logical data model maps a plurality of logical data model entities to the corresponding entity in the physical data model.
6. The system in accordance with claim 1, wherein at least one of the plurality of versions of the logic data model maps the logical data model entity to a plurality of entities on in the physical data model.
7. A method for responding to queries, the method comprising: maintaining a plurality of logical data model versions, wherein each of the plurality of logical data model versions includes at least a corresponding semantic mapping set that correlates a corresponding plurality of data entities in the logical data model version to at least a corresponding set of data entities in a corresponding physical data model version of the data storage system; and associating each of the plurality of logical data model versions to a different set of one or more identified consumers, such that the data storage system is exposed to a plurality of identified consumers through different logical data model versions, wherein the version manager is further configured to perform the following in response to detecting a change to the physical data model of the data storage system: creating a new version of the physical data model; creating a new logical data model version corresponding to the new version of the physical data model; assigning an identified set of one or more identified consumers to the new logical data model version; and exposing the data storage system to the set of one or more identified consumers according to the new logical data model version, the plurality of logical data model versions including (i) a first logical data model version that is associated with a first set of one or more identified consumers and that corresponds to a first physical data model version, and (ii) a second logical data model version that is associated with a second set of one or more identified consumers and that corresponds to a second physical data model version, the version manager concurrently exposing the first and second logical data model versions to the first and second sets of one or more identified consumers.
8. The method in accordance with claim 7, further comprising: creating a new version of the logical data model to add to the plurality of versions of the logical data model at least under some circumstances when a schema of the data storage system changes.
9. The method in accordance with claim 7, wherein at least one of the plurality of logical data models comprises a logical data model comprising a plurality of constituent logical data models, each constituent logical data model configured to expose the data storage system using a different constituent semantic mapping.
10. The method in accordance with claim 9, further comprising: using a corresponding modelling notation when communicating with a constituent logical data model.
11. The method in accordance with claim 9, each constituent logical data model used by a different vertical.
12. The method in accordance with claim 7, wherein at least one of the plurality of versions of the logical data model maps a plurality of logical data model entities to the corresponding entity in the physical data model.
13. The method in accordance with claim 7, wherein at least one of the plurality of versions of the logical data model maps the logical data model entity to a plurality of entities on in the physical data model.
Computing systems and associated networks have revolutionized the way human beings work, play, and communicate. Nearly every aspect of our lives is affected in some way by computing systems. The proliferation of networks has allowed computing systems to share data and communicate thereby vastly increasing information access. For this reason, the present age is often referred to as the "information age".
Data models describe the structure (i.e., the definition, format, relationships, constraints, and so forth) of structured data for storage in data management systems, such as relational databases. During operation, various executing software may reference the data model when determining data to retrieve, the meaning of the data, the relationship of the data with other data, attributes or properties of the data, and so forth.
When designing data systems for an organization, it is common to first design a logical data model that is relatively independent of the underlying data management system (e.g., that is relatively independent of an underlying database management system). The logical data model often includes data objects that are descriptive of business entities. For instance, if the organization was a shaved ice company, the business entities might include shaved ice stands, employees, flavors, sizes, prices, revenue, sales volume, and so forth. Of course, these terms are independent of any underlying physical data management system.
Once the logical data model is designed, the remainder of the data management system is designed to meet the needs of the logical data model. For instance, the physical data model underlies the logical data model, and also describes entities, relationships, attributes, and so forth, but in a manner that more closely aligns to the facilities and constraints of the underlying data management system. Entities and attributes within the logical data model are mapped to entities and attributes within the physical data model.
At least some embodiments described herein relate to the use of a version manager that manages multiple versions of a logical data model. Each version of the logical data model exposes a data storage system using a corresponding semantic mapping set that for multiple logical data model entities, maps at least the logical data model entity to at least a corresponding physical data model entity of the data storage system. In some embodiments, the logical data model itself may consistent of multiple constituent logical data models, each used for a different vertical to expose the same underlying storage system.
The version manager corresponds each of one or more data consumers of the data storage system with a version of the logical data model. Accordingly, when requests come in from a particular data consumer, the corresponding version of the logical data model is used. Thus, the data model may be updated with new versions without impacting performance on existing data consumers, or requiring existing data consumers to change their understanding of the logical data model before continuing with queries.
FIG. 2 illustrates a data management environment that includes a data storage system, a physical data model layer, a logical data model layer that includes multiple logical data models, and a mapping layer the intermediates between the physical data model layer and the logical data model layer;
FIG. 3 illustrates examples of four different logical data model structural alteration types performed on example logical data models;
FIG. 4 illustrates a flowchart of a method for augmenting a data model that already includes one or more logical data models that expose a data storage system;
FIG. 5 illustrates a data management environment that is similar to the data management environment of FIG. 2, and in which a model query engine acts as a hub for handling a model query;
FIG. 6 illustrates a flowchart of a method for a model query engine to operate upon received model queries;
FIG. 7 illustrates a data management environment that is similar to that of FIG. 2, except that additional logical data models are illustrated, each of the logical and physical data models have an associated index, and there is a cache for model queries;
FIG. 8 illustrates a version management environment in which multiple data consumers are to access data from a data source system via respective versions of a logical data model and a physical data model; and
FIG. 9 illustrates a method for responding to a model query in the context of there being multiple versions of a logical data model or a unified data model.
At least some embodiments described herein relate to the use of a version manager that manages multiple versions of a logical data model. Each version of the logical data model exposes a data storage system using a corresponding semantic mapping set that for multiple logical data model entities, maps at least the logical data model entity to at least a corresponding physical data model entity of the data storage system. In some embodiments, the logical data model itself may consistent of multiple constituent logical data models, each used for a different vertical to expose the same underlying storage system. In those embodiments, the logical data model might be referred to as a "unified data model".
Some introductory discussion of a computing system will be described with respect to FIG. 1. Then, the structure and use of multiple logical data models (also called hereinafter a "unified data model") to expose a data storage system will be described with respect to FIGS. 2 through 7. Then, the principles of versioning of a logical data model or unified data model will be described with respect to FIG. 8 and subsequent figures.
Computing systems are now increasingly taking a wide variety of forms. Computing systems may, for example, be handheld devices, appliances, laptop computers, desktop computers, mainframes, distributed computing systems, datacenters, or even devices that have not conventionally been considered a computing system, such as wearables (e.g., glasses). In this description and in the claims, the term "computing system" is defined broadly as including any device or system (or combination thereof) that includes at least one physical and tangible processor, and a physical and tangible memory capable of having thereon computer-executable instructions that may be executed by the processor. The memory may take any form and may depend on the nature and form of the computing system. A computing system may be distributed over a network environment and may include multiple constituent computing systems.
As illustrated in FIG. 1, in its most basic configuration, a computing system 100 typically includes at least one hardware processing unit 102 and memory 104. The memory 104 may be physical system memory, which may be volatile, non-volatile, or some combination of the two. The term "memory" may also be used herein to refer to non-volatile mass storage such as physical storage media. If the computing system is distributed, the processing, memory and/or storage capability may be distributed as well. As used herein, the term "executable module" or "executable component" can refer to software objects, routines, or methods that may be executed on the computing system. The different components, modules, engines, and services described herein may be implemented as objects or processes that execute on the computing system (e.g., as separate threads).
FIG. 2 illustrates a data management environment 200 that includes a data storage system 210 that contains data. The data management environment 200 may be implemented using the computing system 100 of FIG. 1. The data storage system 210 might have a single constituent data storage system 211 (also referred to as a "store"). For instance, a store might be a database or other unified storage. Alternatively, the ellipses 212 represents that there may be multiple stores. For instance, the data storage system 210 might also be a hybrid data storage system in which cases there are multiple stores, at least some of which being of different types. For instance, one of the constituent data storage systems might be structured data, and another of the constituent data storage systems might be unstructured data.
The physical data model level 220 includes zero or more physical data models. For instance, the physical data model level 220 is illustrated as including one physical data model 221, although the ellipses 222 represent that there may be multiple physical data models (or even zero physical data models) within the physical data model level 220. In this description and in the claims, a "physical data model" is a data model that takes into account the facilities and constraints of a specific data management system, such as a database management system. In the case of the data storage system 210 being a hybrid data storage system, there might be multiple physical data models, each taking into account the facilities and constraints of a respective data management system that facilitates the respective constituent data storage system.
Conventionally, there is only one logical data model used to map to a particular physical data model or data storage system. However, in accordance with the principles described herein, there may be multiple logical data models that expose a particular data storage system. Accordingly, the logical data model level 230 is illustrated as including two logical data models 231 and 232 that expose the underlying data storage system 210 (either directly or via one or more of the physical data models in the physical data model layer 220). However, the ellipses 233 symbolically represents that there might be a single logical data model, or potentially three or more logical data models that each expose the data storage system 210.
In this description and in the claims, a "logical data model" is a data model that has at least one set of entities or attributes that is mapped to at least one set of entities or attributes of a physical data model in the physical data model layer 220. In the remainder of this description and in the claims, an "entity" with respect to a data model is either a node in the data model (e.g., an object), or an attribute of a node in the data model (e.g., a property).
The data management environment 200 also includes a mapping layer 240 between the logical data model level 230 and the physical data model level 220. For each of at least some of the logical data models within the logical data model level 230, a semantic mapping set maps sets of one or more entities of the corresponding logical data model in the logical data model level 230 with one or more entities in one or more of the physical data models in the physical data model level 220.
For instance, the semantic mapping set 241 corresponds to the logical data model 231 in that the logical data model 231 uses the semantic mapping set 241 to expose the data storage system 210 (through mapping to one or more entities of one or more of the physical data models in the physical data model layer 220, and/or by direct mapping to one or more entities of the data storage system 210 itself). Likewise, the semantic mapping set 242 corresponds to the logical data model 232 in that the logical data model 232 uses the semantic mapping set 242 to expose the data storage system 210 (either directly, or indirectly via one or more physical data models). The ellipses 243 illustrates that there might be more or less semantic mapping sets related roughly to the number of logical data models in the logical data model layer 230.
Each logical data model may be designed for a different vertical solution (also sometimes referred to as a "domain" or simply as a "vertical"). For instance, the first logical data model 231 is shown as exposing data to a first vertical A, and the second logical data model 232 is shown as exposing data to a second vertical B. Accordingly, different semantics are applied to data for different verticals. For instance, one piece of data may mean different things in different domains. For instance, in the medical domain, a name field have the value "John Doe" might have a slightly different meaning (e.g., a meaning of "patient") that it might in the domain associated with the place where John Doe is employed (in which the same data would have the meaning "employee"). Of course, the terms "patient" and "employee" have significantly different semantics. The semantic mapping sets for each logical data model provides the appropriate mappings to support different semantics being applied to the same data.
The data management environment 200 also includes a common notation component 250 that is configured to allow each logical data model (and semantic mapping sets) to have a different modelling notation. At a higher level, the common notation component 250 allows users to use whatever modelling notation (also called a "representation framework") they desire to access the data via their logical data model. The common notation component 250 does this by translating all or portions of different modelling notations into a common modelling notation for use at the physical data model level 220 and/or at the data storage system 210. Accordingly, any given physical data model may rely on the use of a common modelling notation, regardless of the logical data models that use that physical data model having different modelling notations.
The data management environment 200 also includes a logical data model lifecycle tracking component 260 configured to track logical data model structural alterations of a plurality of logical data model structural alteration types. When a particular alteration of a particular logical data model structural alteration type is performed, the logical data model lifecycle tracking component 260 causes appropriate reflection to be made in the appropriate semantic mapping set(s). For instance, FIG. 3 illustrates four different types of logical data model structural alteration types. In each case, the alteration is represented by dashed lines.
A first logical data model structural alteration type is represented generally by arrow 301 in which a new component entity of a logical data model (in this case logical data model 300A) is created (also referred to as "birth", "propagation" or "enhancement"). The tracking component 260 would thus potentially create (as represented by arrow 311) a representation of the new entity within the semantic mapping set for the logical data model 300A for use in future mapping to corresponding one or more entities in a physical data model 221 within the physical data model level 220 and/or for direct mapping to one or more entities in the data storage system 210.
A second logical data model structural alteration type is represented generally by arrow 302 in which an entity of a logical data model (in this case logical data model 300D) is retired (or removed) from the logical data model. The tracking component 260 would thus remove (as represented by arrow 312) a representation of the retired entity within the semantic mapping set for the logical data model 300D, thus eliminating or rendering moot any mappings involving the retired entity.
A third logical data model structural alteration type is represented generally by arrow 303 in which two logical data models (in this case, logical data models 300A and 300B) are merged into a single data model. The tracking component 260 would thus appropriately merge (as represented by arrow 313) two semantic mapping sets to reflect this merging option.
A fourth logical data model structural alteration type is represented generally by arrow 304 in which one logical data model (in this case, logical data model 300C) is spit from another logical data model (in this case, logical data model 300B). The tracking component 260 would thus appropriately split (as represented by arrow 314) a semantic mapping set into two semantic mapping sets to reflect this splitting operation.
Referring back to FIG. 2, the data management system 200 is also illustrated as including an augmentation component 270, which allows new logical data models to be added to the logical data model level 230. FIG. 4 illustrates a flowchart of a method 400 for augmenting a data management system that includes a one or more logical data models that expose a data storage system with an additional logical data model that also exposes the data storage system. The method 400 might be performed by the augmentation component 270 of FIG. 2.
First, the additional logical data model is associated with a data storage system that already has at least one previous logical data model associated with the data storage system (act 401). For instance, a template for the semantic mapping set might be created for later population. Then, over time, the semantic mapping set is defined for the additional logical data model (act 402) such that the additional logical data model may also expose the data storage system. Note that the semantic mapping sets may be different for each logical data model, and thus this new semantic mapping set may be different than any of the other semantic mapping set(s) used for the other logical data model(s). In addition, the user might specify a modelling notation to use (act 403) either generally when referring to data of the particular additional logical data model, or with respect to using a particular logical data model to reference data.
Accordingly, a data management environment has been described in which a flexible number of logical data models may be used to expose a data storage system. Now, a mechanism for using the data management environment will be described. The primary use for using any data management environment is to perform a query against the data management environment. Thus, this description will now turn to mechanisms for performing queries against the data management environment 200 to thereby have returned data from the data storage system 210.
The data management environment 200 also includes a model query engine 280. FIG. 5 illustrates a data management environment 500 that is similar to the data management environment 200 of FIG. 2, except that the model query engine 501 is shown as an example of the model query engine 280 of FIG. 2. Furthermore, the model query engine 501 is shown more centrally located acting as a hub for handling model queries. Furthermore, various flows associated with handling a model query are illustrated. In addition, though there may be various logical data models present, only one logical data model 231 is illustrated. Though there may be various physical data models present (or no physical data models present), only one physical data model 221 is illustrated. As the model query engine 501 may use, contain, or be the mapping level 240, the mapping level 240 is also shown as contained within the model query engine 501.
FIG. 6 illustrates a flowchart of a method 600 for a model query engine to operate upon received model queries. The method 600 may be performed by the model query engine 280 of FIG. 2, and/or by the model query engine 501 of FIG. 5. Accordingly, the method 600 will now be described with respect to FIGS. 2 and 5.
The model query engine 280 and/or 501 may perform the method 600 upon receiving a model query. In this description and in the claims, a "model query" is query that specifies a logical data model identity and potentially the identity of one or more logical data model entities (e.g., nodes) and/or the identity of one or more logical data model entity attributes (e.g., properties). Thus, rather than have any awareness about the schema of the underlying data storage system, the user is able to query directly against one or more of the logical data models.
The principles described herein are not limited to any particular format for the model queries. However, the following are examples of generalized model queries, and are provided as they follow similar structure to table-based queries, but are directed towards models, rather than tables:
TABLE-US-00001 SELECT Attribute(1), Attribute(2), . . . FROM Model(1)
TABLE-US-00002 SELECT Attribute(1), Attribute(2), . . . FROM Model(1) WHERE Attribute(1) = `{criteria}`
TABLE-US-00003 SELECT Model(1).Attribute(1), Model(1).Attribute(2), Model(2).Attribute(1), . . . FROM Model(1) JOIN Model(2) ON Model(1).Attribute(1), Model(2) Attribute(1)
TABLE-US-00004 SELECT Attribute(1), Attribute(2), . . . FROM Model(1) UNION {ALL} SELECT Attribute(1), Attribute(2), . . . FROM Model(2)
Thus, the model query language may, but need not, be similar to traditional query language, but refer to logical data model entities and/or attributes, instead of data storage system tables and/or attributes. For instance, the model query language may have provisions for set expressions (such as "UNION", "JOIN", "DIFFERENCE", "INTERSECTION" and the like). Each of the model queries might have a common schema across all of the plurality of logical data models. Thus, the querier need not have any knowledge at all about the data storage system in order to generate a model query, but instead just reference logical data models, logical data model entities and/or logical data model entity attributes, and as appropriate, apply constraints, set expressions, and the like to the model query. If, however, the querier is familiar with the modelling notation of a particular logical data model, the querier might still generate a logical data model query directly against that logical data model using that modelling notation.
The model query engine receives the generated model query (act 601). For instance, in FIG. 5, the model query engine 501 receives a model query 502 (as represented by arrow 511).
The model query engine then interprets the model query (act 602). The model query engine may then check a cache (act 603) to see if all or parts of the same model query have been made in the past and recently enough that the associated results in the cache may be considered fresh.
For instance, FIG. 7 illustrates a data management environment 700 that is similar to the data management environment 200 of FIG. 2. Here, however, there are four logical data models 731 through 734 illustrated. Accordingly, logical data models 731 and 732 might represent examples of the logical data models 231 and 232 of FIG. 2. The logical data models 733 and 734 might represent by the ellipses 233 in FIG. 3. Each logical data model is illustrated as having different shaped elements, which symbolically represents that the logical data models may each be represented by different modelling notation. There is a cache 710 illustrated where results of previously made model queries (or portions of such query results) are kept at least until such results are no longer considered fresh. The model query engine 501 of FIG. 5 may thus consult with the cache 710 to see if part or all of the model query already has fresh results within the cache 710.
The model query engine then identifies one or more logical data models identified in the query (act 604). For instance, in the example of FIG. 5, suppose that the only logical data model identified is the logical data model 231. Interpretation of the model query also involves identifying any logical data model entities and/or attributes specified in the model query.
For each of the identified logical data models, the model query engine interacts with the identified logical data model(s) (act 610). In the example of FIG. 5, there was only one logical data model 231 identified in the model query, and thus the model query engine 501 interacts with the logical data model 231 over the appropriate channel as represented by arrows 512 and 513.
As part of this interaction, the model query engine 501 identifies a modelling notation used by the logical data model (act 611). For instance, the logical data model provider may have specified this modelling notation in act 403 of FIG. 4. Alternatively or in addition, the model query engine 501 may automatically detect the modelling notation used by the logical data model.
The model query engine 501 then formulates a logical data model query (act 612) that complies with the modelling notation for the logical data model and that corresponds to that portion of the model query that corresponds to the specific logical data model.
The model query engine 501 may then check an index associated with the logical data model (act 613), to see if the results of that query have previously been returned and/or if there are indices that will assist with simplifying the interaction with the logical data model. In one example, each logical data model has an associated distinct index. For instance, in the example of FIG. 7, note that each of the logical data models 731 through 734 have an associated index 701 through 704. In the example of FIG. 5, since there is only one logical data model 231 being interacted with, only the associated logical data model index 521 is shown. For instance, if the logical data model 231 of FIG. 5 were the logical data model 731 of FIG. 7, the logical data model index 521 of FIG. 5 would be the logical data model index 701 of FIG. 7. Accordingly, the principles described herein may be expedited by used of a multi-model index.
If necessary (because the results of the logical data model query are not completely cached in the index), the model query engine 501 then submits (act 614) the logical data model query (or a simplified version of the logical data model query permitted by the index for that logical data model) to the corresponding logical data model. This submission is represented by arrow 512 in FIG. 2.
Whether from the index for the logical data model, or from concurrent interaction with the logical data model, the model query engine 501 receives results of the logical data model query (act 615) from the corresponding logical data model. This receipt of logical date model query results is represented in FIG. 5 by the arrow 513 in FIG. 5.
The model query engine then converts these results from the modelling notation used by the logical data model into a common modelling notation (act 616), if the modelling notation of the logical data model is not already the common modelling notation. An example of the common modelling notation is the Uniformed Modelling Language (UML), although this is just an example. Accordingly, when interacting with the physical data model(s), the model query engine may use the common modelling notation, while still allowing the providers of the logical data models to specify the modelling notation to be used when referencing the respective logical data model.
The results of this logical data model query may potentially be indexed (act 617) in the index of the corresponding logical data model. For instance, in FIG. 5, the results returned (as represented by arrow 513) from the logical data model 231 might be indexed within the corresponding index 521 for the logical data model 231. Referring to FIG. 7, each of the logical data model indexes 701 through 704 may likewise index results of previous model queries made to respective logical data models 731 through 734.
Based on the results, the model query engine also identifies (act 618) one or more physical data models (or the data storage system directly) that are to be interacted with in order to respond to the model query. To do so, the model query engine might reference the semantic mapping set for the corresponding logical data model from which the results were obtained.
Then, the model query engine interacts with each of the resulting one or more physical data models (act 620). Accordingly, the contents of act 620 are performed for each of one or more physical data models, or for the data storage system directly, as the mappings might direct. In the remaining description of the content of act 620, the term "physical data model" will be used interchangeably with the data source system, since the contents of act 620 may be performed to interact with either a particular physical data model that operates above the data source system, or directly with the data source system itself.
To do so, the model query engine 501 consults the semantic mapping set for the logical data model (act 621). Using this consultation, and the results of the logical data model queries, the model query engine 501 then formulates a physical data model query (act 622) that complies with the common modelling notation used at the physical data model layer (or by the data source system itself).
The model query engine 501 may then check an index associated with the physical data model (act 623), to see if the results of that query have previously been returned and/or if there are indices that will assist with simplifying the interaction with the physical data model (or by the data source system as the case may be). For instance, in FIG. 5, the physical data model index 522 serves the physical data model 221. For instance, in FIG. 7, this physical data model index 705 is also illustrated.
If necessary (because the results of the physical data model query are not completely cached in the index 522), the model query engine 501 then submits (act 624) the physical data model query (or a simplified version of the physical data model query permitted by the index for that physical data model) to the corresponding physical data model. This submission is represented by arrow 524 in FIG. 5.
Whether from the index 522 for the physical data model, or from concurrent interaction with the physical data model 221, the model query engine 501 receives results of the physical data model query (act 625) from the corresponding physical data model. This receipt of physical date model query results is represented in FIG. 5 by the arrow 515.
The results of this physical data model query may potentially be indexed (act 626) in the index of the corresponding physical data model. For instance, in FIG. 5, the results returned (as represented by arrow 515) from the physical data model 221 might be indexed within the corresponding index 522 for the physical data model 221. In the context of FIG. 7, the results might be indexed in the physical data model index 705.
The model query engine then assembles all of the physical data model query results into a result set (act 631) and returns the result set to the querier (act 632). This return of the result set is represented in FIG. 5 using arrow 516. The result set is represented by element 503 in FIG. 5.
The principles described herein may be particularly useful for enormous volumes of data that potentially change over time (sometimes referred to as "big data"). Such data is not often spoken about in terms of Terabytes, but often in terms of petabytes (1024 terabytes), exabytes (1024 petabytes), and in the future perhaps zettabytes (1024 exabytes) and yottabytes (1024 zettabytes). Such data is often more global and distributed in nature, and involves numerous data sources, often being changed, added, and replaced dynamically. Thus, "big data" may be viewed as more of an organic and evolving data source system.
With big data, it may be quite beneficial to have multiple vertical solutions using that kind of volume of data, since such big data represents a valuable resource to many organizations. Furthermore, since the data is so large, it is helpful to perform queries on the data by referencing the logical data model that the particular vertical solution is familiar with, rather than referencing the schema of the data source directly. Accordingly, the principles described herein allow multiple vertical solutions to use distinct logical data models to access the same data, and allow them to query on that same big data, even though the querier is unfamiliar with the complexities of the underlying schema of the data source itself.
However, with big data, the data itself may change quite frequently. New data sources can be introduced at any time, each data source having its own schema. Furthermore, understanding of the extent of the big data may change over time. Conventionally, the data source schema was pre-defined and understood prior to beginning ingesting the data. However, the principles described with respect to FIGS. 8 and 9 allow for ingestion of data from a data source while the very data source itself continues to undergo change. Such is performed using a version manager.
FIG. 8 illustrates a version management environment 800 in which multiple data consumers 801, 802 and 803 are to access data from a data source system 810. As an example, the data source system 810 may be the data source system 210 of FIG. 2. The data source system 810 is not a static data source, but is subject to change, just a big data might be subject to change as new data sources are added to the big data.
The environment 800 also includes multiple versions 821(1), 821(2), 821(3) through 821(N) of a physical data model (referred to herein as "physical data model 821") that maps to the data source. As an example, the physical data model 821 might be the physical data model 221 of FIG. 2.
The environment 800 also includes multiple versions 831(1), 831(2), 831(3) through 831(N) of a logical data model (referred to herein as "logical data model 831") that maps to a corresponding version 821(1), 821(2), 821(3) through 821(N) of the physical data model 821. For instance, the logical data model version 831(1) is shown mapped (as represented by arrow 841(1)) to the physical data model version 821(1), the logical data model version 831(2) is shown mapped (as represented by arrow 841(2)) to the physical data model version 821(2), the logical data model version 831(3) is shown mapped (as represented by arrow 841(3)) to the physical data model version 821(3), and the logical data model version 831(4) is shown mapped (as represented by arrow 841(4)) to the physical data model version 821(4).
The logical data model 831 might be any logical data model that maps to the physical data model 821. However, the logical data model 831 also might be multiple logical data models that map to the physical data model 821, in which case the logical data model might be referred to as a unified logical data model. An example of a unified logical data model is the entire logical data model layer 230 of FIG. 3, which constitutes multiple logical data models 231 and 232 mapped to the physical data model 221.
At least under some conditions, when the data source 810 changes, a new version of the physical data model 821 and a new version of the logical data model 831 may be created. For instance, in the context of FIG. 8, if the data source 810 were to change and/or the conditions for creating a new version are satisfied, then a new physical data model version 821(N+1) (not shown) would be created, and a corresponding logical data model version 831(N+1) (not shown) would be created and also mapped with mapping 841(N+1) (not shown) to the corresponding physical data model version 821(N+1).
Data consumers are corresponded with a version of the logical data model 831 such that when model queries are received from that data consumer, the corresponding version of the logical data model is used. For instance, data consumer 801 is corresponded to logical data model version 831(1). Thus, any model requests received from data consumer 801 are directed to the logical data model version 831(1). Note that if the model query processing of FIGS. 5 and 6 may then be performed, but with respect to the applicable version of the logical data model 831(1) and physical data model 821(1). Data consumer 802 is corresponded to logical data model version 831(3). Thus, any model requests received from data consumer 802 are directed to the logical data model version 831(3). Data consumer 803 is corresponded to logical data model version 831(N), and thus its model requests are directed towards logical data model version 831(N).
This allows new version of the logical data model to be created and modified, without affecting existing data consumers that correspond to prior versions. For instance, upon creation of a new physical model version 821(N+1) (not shown) and a corresponding new logical model version 831(N+1) (not shown) mapped thereto, those new model versions will not yet be in use by any existing data consumers 801 through 803. Thus, the new versions 821(N+1) and 831(N+1) may be changed to reflect 1) new information learned about the data source 810 and/or 2) new data sources, that may be taken advantage of by future model requests. Meanwhile, the data consumers 801 through 803 may continue to interface with the logical data model version that they are familiar with. Thus, from the perspective of any given data consumer, the data source 810 (or at least its version of the logical data model 831) is static.
The version manager 805 manages the versions of the logical data model 831 and the versions of the physical data model 821. For instance, the version manager 805 keeps track of the versions, creates new versions when appropriate, retires new versions when appropriate (e.g., when the version is no longer used by any data consumers), and corresponds different data consumers with a respective version of the logical data model 831.
FIG. 9 illustrates flowchart of a method 900 for responding to a model query in the context of a logical data model that includes multiple logical data model versions. The method includes acts that may be inserted between acts 602 and 603 of FIG. 6. After act 602 of FIG. 6, the method 900 includes an act of identifying a logical data model version (act 901) that corresponds to the received model query. In the case of the logical data model being a unified logical data model, this would be a unified data model version.
The method 900 then proceeds with act 603 of FIG. 6. Now, the cache being checked in act 603 would be the cache corresponding to this particular version of the logical data model. The logical data models would then be identified (act 604). In the case of the logical data model 831 being a single data model, this would be a trivial exercise. However, in the case of the logical data model 831 being a unified data model having multiple constituent logical data models representing multiple verticals, the model query would be further evaluated to identify which constituent logical data models are to be acted upon in act 610 of FIG. 6.
Accordingly, the principles described herein allow for the ingestion of data from a data source, even while the data source is subject to change, and without affecting the data consumers of the data source.
Previous Patent US 10,198,458 | Next Patent US 10,198,460