Secondary mappings to enable code changes without schema updates

Systems and methods are described for using secondary mappings in an Object Relational Mapping (ORM) system to enable source code changes to be implemented and carried out without requiring updates to the database schema. In order to use secondary mappings, one or more attributes of an object in the ORM system are annotated to indicate that they are secondary mapped attributes. The annotation can indicate to the ORM system that the various operations (e.g. read operations, write operations) referencing attributes of the object may need to be remapped if the column for the attribute being referenced does not actually exist in the database. For example, if no column is directly mapped to the attribute being referenced, the ORM system can remap the operation to point to a secondary data structure (e.g. a map stored in-memory) that will be used to store the value for that attribute.

BACKGROUND

In software development, source code is often modified, recompiled and rebuilt in order to provide updates, eliminate errors and bugs, and to enable additional functionality. When a software application is database-backed, these source code changes may sometimes require updates to the database schema before the modified code can be executed. For example, if a change to the source code references a new property or attribute, then the database schema would need to be modified to reflect the new attribute before the source code can be validly executed against the database. As a result, there is often a tight integration that exists between changes to the source code of a particular application and the schema updates necessary to enable those changes to perform the intended functionality. In many cases, however, it is tedious and impractical for developers to perform schema updates every time source code is changed because schema updates are not generally subject to the same compilation, deployment, rollback and other processes that are inherent in software code development.

Object relational mapping (ORM) systems are used to convert data between entities that may use different data types or formats. For example, an ORM system typically builds a virtual database by creating mappings between objects and attributes residing in-memory and the tables and columns stored in the database. While ORM systems are useful for reducing the amount of code that is written by software developers, they are generally subject to the same limitations and restrictions between source code changes and database schema updates mentioned above.

DETAILED DESCRIPTION

In the following description, various embodiments will be illustrated by way of example and not by way of limitation in the figures of the accompanying drawings. References to various embodiments in this disclosure are not necessarily to the same embodiment, and such references mean at least one. While specific implementations and other details are discussed, it is to be understood that this is done for illustrative purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without departing from the scope and spirit of the claimed subject matter.

Systems and methods in accordance with the embodiments described herein overcome the various deficiencies in existing approaches for providing software updates and modifications. In particular, the various embodiments provide approaches for decoupling source code changes from requiring updates to a database schema.

In accordance with various embodiments, techniques are described for using secondary mappings in an Object Relational Mapping (ORM) system to enable source code changes to be implemented and carried out without requiring updates to the database schema. Conventionally, an ORM system maintains a set of mappings between objects residing in memory and the tables residing in the database. In particular, the ORM system typically maps each attribute of the object in memory to a column in the database.

In order to use secondary mappings, one or more attributes of an object in the ORM system may be annotated as being secondary mappable attributes. This annotation can indicate to the ORM system that the various operations (e.g. read operations, write operations) referencing these attributes of the object may need to be remapped if the column for the attribute being referenced does not actually exist in the database. For example, if no column is directly mapped to the attribute being referenced, the ORM system can remap the operation to point to a secondary data structure (e.g. a map stored in-memory) that will be used to store the value for that attribute.

The secondary mapping attribute can thus dynamically re-map reads and writes to an alternative attribute lookup that could, in one embodiment, be modeled as a JavaScript Object Notation (JSON) hash map stored in a specially-designated column of the table. In accordance with at least one embodiment, the usage of the secondary mapping feature would require all eligible tables to have a single column (e.g. text column) added to the database table that corresponds with the ORM object, specifically for storing the secondary values. Behavior of the read/write operations would then be dependent on the existence of the actual column in the database (i.e. the column directly mapped to the attribute being referenced). In accordance with alternative embodiments, the secondary column can be added to a separate database table (e.g. a shadow table) that is created for storing secondary attributes.

In accordance with an embodiment, if the actual column for the attribute does not exist in the database, all reads and writes would be re-mapped directly to the in-memory map. Before the ORM writes the object to the database, it would serialize the map to the designated column. When reading the object back from the database, the map would be undefined and only re-constituted on-demand (by either read or write request to one of the secondary mapping columns).

In accordance with an embodiment, once the directly mapped column for the attribute is added to the database, reads can still continue to be performed from the in-memory map. If the map has a value, that value is returned. If the value is not present, the value from the actual column is returned instead. Writes, on the other hand, can store directly to the column and also remove any mapped value from the in-memory map. Before storing the object to the database, the map is inspected and any keys which are no longer necessary are removed and their value populated in the corresponding columns, then whatever values remain in the map are serialized as before.

In accordance with various embodiments, the use of the secondary mapping feature can enable the system to compile and execute source code changes without having to first update the database schema. For example, if the source code of an application references a new attribute that has not yet been defined in the database schema, the ORM system can treat that new attribute as a secondary mapped attribute and store any values for that attribute into the JSON hash map as described above. Subsequently, when the database schema is updated to create the column for the new attribute, the ORM system can begin storing the values into the newly created column and clean up any old values from the hash map. In this manner, the developer can implement source code changes without having to first update the schema of the database to reflect all the possible changes to the data objects that would be caused by those source code changes.

FIG. 1illustrates an example of using secondary mappings in an ORM system, in accordance with various embodiments. As illustrated, the ORM system101defines a model object108that has attributes107that map one-to-one with database columns103. For example, conventionally, if a developer wanted the object “widget” to have an attribute “category”, the ORM system would need to annotate the model object with the annotation specifying that the model object contains the attribute “category” and furthermore, the database schema106would need to be updated to create a “category” column in the database100that is directly mapped to the “category” attribute of the object.

In accordance with the illustrated embodiment, to get around this requirement, the attribute of the object can be annotated with a secondary mapping annotation, indicating that the attribute109of object108can be a secondary mapped attribute. In order to enable this, a specially designated column104is added to the table111in the database that corresponds to ORM object108. Alternatively, table III can be a separate database table specifically created and managed for storing secondary attributes. In accordance with an embodiment, column104is specifically designated to hold the values for all secondary mapped attributes (i.e. attributes of the object which have been annotated as being secondary mappable and for which there is no corresponding directly mapped column in the database).

When the ORM system receives an operation (e.g. read/write) from the application source code102that references an attribute of the object108, it can first check whether there exists a corresponding primary mapped column for that attribute in the database. If there is no such column (e.g. the attribute is a new attribute109referenced in the source code), the ORM system re-maps the operation to reference a map110stored in-memory on the ORM system. Consequently, any writes performed by the application would be stored in the map and any reads would be performed on the values in the map. The values in the map110can be serialized and stored in the specially designated column104using a key that is based on the name of the new attribute109. In this manner, column104can hold values for multiple attributes which were never defined to have corresponding columns in the database.

FIG. 2illustrates an example of updating the database schema after using secondary mapped attributes, in accordance with various embodiments. As previously described, the ORM system201can contain mappings of various attributes207of the model object208to the corresponding column203in the database200. As mentioned inFIG. 1, the ORM system has been using the hash map210to enable source code to reference attributes which had not previously been defined in the database.

InFIG. 2, however, after using the secondary mapped attribute, the database schema206may be updated to create the primary column212that is mapped to attribute209. After the actual column for the attribute is added to the database, the ORM can seamlessly transition to use the new column212to store values for the attribute209instead of storing these values in the secondary mapped column204.

In accordance with an embodiment, after the database schema is updated, reads can continue to be performed against the in-memory map210. If the map210contains a value, the value from the map is returned as a result of the read operation. If there is no value present in map210, the value from the actual column212is retrieved and returned as a result of the read operation.

In accordance with an embodiment, after the database schema is updated, write operations store values directly to the new column212and remove any mapped value from the in-memory map210. Before storing the object to the database, the map210is inspected and any keys which are no longer necessary are removed and their value populated in the corresponding columns, then whatever values remaining in the map are serialized and stored in column204.

In this manner, the secondary mapping annotations can be leveraged for initial implementations of source code changes and later be removed after the schema updates have been applied.

FIG. 3illustrates an example process for using secondary mappings in the ORM system, in accordance with various embodiments. Although this figure, as well as any other process illustrations contained in this disclosure may depict functional steps in a particular sequence, the processes are not necessarily limited to the particular order or steps illustrated. One skilled in the art will appreciate that the various steps portrayed in this or other figures can be changed, rearranged, performed in parallel or adapted in various ways. Furthermore, it is to be understood that certain steps or sequences of steps can be added to or omitted from the process, without departing from the scope of the various embodiments.

As shown in step300, the ORM system defines mappings between objects and attributes residing in-memory on the ORM system and tables and columns stored in a database. For example, in many ORM systems, there is a one-to-one mapping between each attribute of the in-memory object and a column in the database.

In step301, an attribute of an object in the ORM system is annotated with a secondary mapping annotation. In alternative embodiments, the entire object may be annotated with the secondary mapping annotation. In accordance with an embodiment, this annotation indicates to the ORM system that read/write operations referencing the annotated attributes of the object may need to be re-mapped if the attribute does not have a corresponding primary mapped column in the database. Thus, for example, if an attribute of an object being referenced by a piece of code does not have a corresponding equivalent column in the database, the operation will need to be remapped to a different location.

In step302, the ORM system receives an operation that references an attribute of the object. For example, the operation can be a method invocation in the source code to read the attribute or to write an updated value to the attribute.

In step303, the ORM determines whether a directly mapped column exists in the database that is associated with the attribute of the object that is being referenced. For example, if the method invocation is referencing an attribute “category” of object “widget”, the ORM system can inspect the database and determine whether there is a corresponding column “category” in the table for “widget”. In accordance with an embodiment, if the actual column exists in the database, the operation is simply passed through to the column and the value is either read or written to the directly mapped column.

In step304, if there is no corresponding column in the database for the attribute being referenced, the ORM system remaps the operation to a secondary data structure (e.g. a map in-memory). The map can be a hash map that is serialized and stored in a special column which has been previously added to the database to contain all secondary mapped attributes for the object. In accordance with one embodiment, the designated column can be part of the database table that corresponds to the ORM object that contains the annotated attribute. For example, all tables that are eligible to use secondary mappings will have a special column added in the database, which will be used specifically for storing the values of all secondary mapped attributes. These values can be keyed and stored using the name of the secondary attribute (e.g. “category”) for which the corresponding column has not been created yet in the database. In an alternative embodiment, the designated column can be part of a separate database that is specifically created for storing the secondary attributes. The use of a separate table could enable the system to store the secondary attributes directly without first encoding it into JSON.

In step305, the ORM system can later detect that the database schema has been updated and that the directly mapped column has now been added to the database. For example, when the ORM is activated, it pulls the metadata for all the relevant tables. A hook can be added to the ORM which, when executed, inspects the model and determines that this model object has some secondary properties (e.g. contains values in the secondary mapped column). If the model object has secondary properties, the ORM can inspect the database to check if any of the actual directly mapped columns exist for any of the values in the secondary mapped column.

In step306, if the primary mapped column has been added to the database, the ORM system can retrieve the values from the secondary mapped column and write those values into the primary mapped column. In accordance with an embodiment, this can be performed upon determining that the column has been added. Alternatively, the values can be maintained in the secondary mapped column and all future write operations can write the value to the primary mapped column. Once the value has been written to the primary mapped column, the values can be removed/cleaned up from the secondary mapped column.

FIG. 4illustrates an example logical implementation flow that the ORM can follow upon receiving an operation, in accordance with various embodiments.

As shown in step400, the ORM system can receive an operation that references an attribute of an object. For example, the operation can be a read or write operation. In step401, a determination is made whether a primary mapped column for the attribute exists in the database. If the primary column does not exist, the ORM system automatically remaps the operation to reference the in-memory hash map (step402). For example, if the operation is a write operation, the ORM system writes the value to the in-memory hash map. In accordance with an embodiment, the hash map is then serialized and stored in a specific column of the database that is designated for storing all secondary mapped attributes (step403).

If the primary column does exist in the database, the ORM system can first determine whether the operation is a read operation or a write operation (step404). As shown in step409, when the primary mapped column exists, all write operations can write the values directly to the primary column. Once the new value has been written to the primary mapped column, any values can be cleaned up from the secondary mapped column, as illustrated in step410.

If the operation is a read operation, the ORM system can first attempt to read the value from the in-memory map (e.g. hash map). If the map contains a value, that value can be returned as a result of the read operation (step407). If the map does not contain a value, the ORM system can instead return the value from the primary mapped column (since it exists) as shown in step408.

In accordance with various embodiments, the flow illustrated inFIG. 4can be modified to account for concurrent access of the database by multiple ORM instances. To account for concurrent access, optimistic locking can be implemented for the tables in the database. For example, the attribute can be versioned to account for multiple ORM instances modifying the attribute.

In accordance with an embodiment, the ORM can cache the database metadata in memory in order to reduce latency. For example, the database schema containing information about the existence of various tables and columns in the database can be cached in each ORM instance at the time that the ORM instance is started. Thereafter, the ORM instances may need to invalidate the cached metadata in order to ensure that the primary column of the table does not exist and that the value should be stored as a secondary attribute. In accordance with an embodiment, this can be done at the time of adding a new key to the hash map. At the point in time when the new key is added, the cached metadata can be invalidated in order to ensure that the primary column does not actually exist and that the value should be stored as a secondary mapped attribute. This can allow multiple ORM instances to utilize the secondary mapped attributes while allowing concurrent access due to optimistic locking.

FIG. 5illustrates a logical arrangement of a set of general components of an example computing device500. In this example, the device includes a processor502for executing instructions that can be stored in a memory device or element504. As would be apparent to one of ordinary skill in the art, the device can include many types of memory, data storage, or non-transitory computer-readable storage media, such as a first data storage for program instructions for execution by the processor502, a separate storage for images or data, a removable memory for sharing information with other devices, etc. The device typically will include some type of display element506, such as a touch screen or liquid crystal display (LCD), although devices such as portable media players might convey information via other means, such as through audio speakers. As discussed, the device in many embodiments will include at least one input element512able to receive conventional input from a user. This conventional input can include, for example, a push button, touch pad, touch screen, wheel, joystick, keyboard, mouse, keypad, or any other such device or element whereby a user can input a command to the device. In some embodiments, however, such a device might not include any buttons at all, and might be controlled only through a combination of visual and audio commands, such that a user can control the device without having to be in contact with the device. In some embodiments, the computing device500ofFIG. 5can include one or more network interface elements508for communicating over various networks, such as a Wi-Fi, Bluetooth, RF, wired, or wireless communication systems. The device in many embodiments can communicate with a network, such as the Internet, and may be able to communicate with other such devices.